2024-04-20T10:45:58.503 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T10:45:58.503 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491 --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491/orig.config.yaml 2024-04-20T10:45:58.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T10:45:58.688 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491 branch: wip-rishabh-testing-20240416.193735 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: null first_in_suite: false job_id: '7664491' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/reef ' name: rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: a8c84f5997950fd7d008204839253291d52e6eda ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda targets: smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKu78gkLLaiWRbDPuhWO5MIBRTaiOWdP6u8pIqGkprNrWsI7pLi5+Ar9l6+i+KOpT++HOFfEgvqUoeM27EqOjtY= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPr/8/+o9OmJU2qLSKgC/92YW26KhRc4916aPpRcO7v6MBEl2glsf18mn/vfW5Si6BXo6XAbf/0pzAEZU5W2HFo= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef roleless: true - print: '**** done end installing reef cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_09:50:15 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-20T10:45:58.688 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T10:45:58.689 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T10:45:58.689 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T10:45:58.691 INFO:teuthology.task.internal:Checking packages... 2024-04-20T10:45:58.718 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T10:45:58.719 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:45:58.719 INFO:teuthology.packaging:ref: None 2024-04-20T10:45:58.719 INFO:teuthology.packaging:tag: None 2024-04-20T10:45:58.719 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T10:45:58.719 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:45:58.719 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T10:45:58.864 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T10:45:58.866 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T10:45:58.873 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T10:45:58.874 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T10:45:58.882 INFO:teuthology.task.internal:Saving configuration 2024-04-20T10:45:58.921 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T10:45:58.928 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:45:58.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:40:49.785588', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKu78gkLLaiWRbDPuhWO5MIBRTaiOWdP6u8pIqGkprNrWsI7pLi5+Ar9l6+i+KOpT++HOFfEgvqUoeM27EqOjtY='} 2024-04-20T10:45:58.975 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:40:49.783977', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPr/8/+o9OmJU2qLSKgC/92YW26KhRc4916aPpRcO7v6MBEl2glsf18mn/vfW5Si6BXo6XAbf/0pzAEZU5W2HFo='} 2024-04-20T10:45:58.975 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T10:45:58.983 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-20T10:45:58.983 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-20T10:45:58.983 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T10:45:59.051 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2783569160>, signals=[15]) 2024-04-20T10:45:59.052 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T10:45:59.059 INFO:teuthology.task.internal:Opening connections... 2024-04-20T10:45:59.060 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T10:45:59.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:45:59.141 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-04-20T10:45:59.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:45:59.227 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T10:45:59.236 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-04-20T10:45:59.254 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-04-20T10:45:59.254 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-04-20T10:45:59.311 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-04-20T10:45:59.311 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-04-20T10:45:59.311 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-04-20T10:45:59.311 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:45:59.311 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-04-20T10:45:59.311 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:45:59.311 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:45:59.312 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-04-20T10:45:59.312 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:45:59.312 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:45:59.312 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:45:59.312 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:45:59.312 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:45:59.312 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:45:59.313 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-04-20T10:45:59.336 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-04-20T10:45:59.355 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-04-20T10:45:59.355 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-04-20T10:45:59.414 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-04-20T10:45:59.414 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-04-20T10:45:59.414 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-04-20T10:45:59.414 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:45:59.414 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-04-20T10:45:59.414 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:45:59.415 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:45:59.415 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-04-20T10:45:59.415 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:45:59.415 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:45:59.415 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:45:59.416 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:45:59.416 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:45:59.416 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:45:59.416 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-04-20T10:45:59.453 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T10:45:59.463 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T10:45:59.471 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T10:45:59.471 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:45:59.474 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:45:59.490 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T10:45:59.497 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T10:45:59.497 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:45:59.531 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:45:59.633 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T10:45:59.677 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T10:45:59.694 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T10:45:59.694 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-20T10:45:59.694 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:45:59.695 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:45:59.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:45:59.711 DEBUG:teuthology.task.kernel:sha1 for host.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:45:59.712 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T10:45:59.712 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-04-20T10:45:59.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:45:59.713 DEBUG:teuthology.task.kernel:sha1 for host.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:45:59.713 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T10:45:59.713 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-04-20T10:45:59.769 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:45:59.769 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:45:59.769 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T10:45:59.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:45:59.774 INFO:teuthology.orchestra.run.smithi173.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:45:59.774 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:45:59.774 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T10:45:59.774 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:45:59.987 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum upgrade -y linux-firmware 2024-04-20T10:46:00.003 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum upgrade -y linux-firmware 2024-04-20T10:46:02.458 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.0 MB 00:00 2024-04-20T10:46:03.442 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.0 MB 00:01 2024-04-20T10:46:06.115 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 7.3 MB/s | 19 MB 00:02 2024-04-20T10:46:07.717 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-04-20T10:46:10.618 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-20T10:46:11.934 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.2 MB 00:00 2024-04-20T10:46:12.366 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-20T10:46:14.109 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-20T10:46:14.535 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-20T10:46:15.503 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-20T10:46:19.117 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-20T10:46:20.519 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-20T10:46:21.106 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:46:21.107 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:46:21.107 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:46:21.236 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on host.b... 2024-04-20T10:46:21.237 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:46:21.457 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T10:46:21.457 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T10:46:21.460 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi173.front.sepia.ceph.com to complete... 2024-04-20T10:46:22.589 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:46:22.589 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:46:22.590 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:46:22.695 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on host.a... 2024-04-20T10:46:22.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:46:22.920 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T10:46:22.920 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T10:46:22.926 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi117.front.sepia.ceph.com to complete... 2024-04-20T10:46:55.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20 10:46:55 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T10:46:55.110 INFO:teuthology.orchestra.run.smithi117.stderr:FINISHED --2024-04-20 10:46:55-- 2024-04-20T10:46:55.110 INFO:teuthology.orchestra.run.smithi117.stderr:Total wall clock time: 32s 2024-04-20T10:46:55.110 INFO:teuthology.orchestra.run.smithi117.stderr:Downloaded: 1 files, 341M in 32s (10.8 MB/s) 2024-04-20T10:46:55.113 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on host.a... 2024-04-20T10:46:55.113 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T10:46:56.853 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20 10:46:56 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T10:46:56.853 INFO:teuthology.orchestra.run.smithi173.stderr:FINISHED --2024-04-20 10:46:56-- 2024-04-20T10:46:56.853 INFO:teuthology.orchestra.run.smithi173.stderr:Total wall clock time: 35s 2024-04-20T10:46:56.854 INFO:teuthology.orchestra.run.smithi173.stderr:Downloaded: 1 files, 341M in 35s (9.75 MB/s) 2024-04-20T10:46:56.890 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on host.b... 2024-04-20T10:46:56.890 DEBUG:teuthology.orchestra.run.smithi173:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T10:46:57.240 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying... ######################################## 2024-04-20T10:46:58.415 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing... ######################################## 2024-04-20T10:46:59.225 INFO:teuthology.orchestra.run.smithi117.stdout:Updating / installing... 2024-04-20T10:46:59.622 INFO:teuthology.orchestra.run.smithi173.stdout:Verifying... ######################################## 2024-04-20T10:46:59.775 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing... ######################################## 2024-04-20T10:46:59.885 INFO:teuthology.orchestra.run.smithi173.stdout:Updating / installing... 2024-04-20T10:47:07.107 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T10:47:07.844 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T10:47:44.495 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T10:47:44.495 DEBUG:teuthology.orchestra.run.smithi117:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T10:47:44.570 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:44.570 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:44.570 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:44.570 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T10:47:44.570 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T10:47:44.570 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T10:47:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T10:47:44.572 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T10:47:44.573 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T10:47:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T10:47:44.575 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T10:47:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T10:47:44.577 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T10:47:44.578 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T10:47:44.579 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T10:47:44.580 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T10:47:44.581 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T10:47:44.582 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T10:47:44.583 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T10:47:44.584 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T10:47:44.585 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T10:47:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T10:47:44.587 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T10:47:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T10:47:44.589 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T10:47:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T10:47:44.591 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T10:47:44.592 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T10:47:44.593 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T10:47:44.594 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T10:47:44.595 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T10:47:44.596 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T10:47:44.597 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T10:47:44.598 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T10:47:44.599 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T10:47:44.600 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T10:47:44.601 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T10:47:44.602 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T10:47:44.603 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T10:47:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T10:47:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T10:47:44.606 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T10:47:44.607 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T10:47:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T10:47:44.609 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T10:47:44.610 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T10:47:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T10:47:44.612 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T10:47:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T10:47:44.614 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T10:47:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T10:47:44.616 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T10:47:44.617 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T10:47:44.618 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T10:47:44.619 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T10:47:44.620 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T10:47:44.621 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T10:47:44.622 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T10:47:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T10:47:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T10:47:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T10:47:44.626 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T10:47:44.627 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T10:47:44.628 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T10:47:44.629 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T10:47:44.630 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T10:47:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T10:47:44.632 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T10:47:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T10:47:44.634 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T10:47:44.635 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T10:47:44.636 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T10:47:44.637 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T10:47:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T10:47:44.639 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T10:47:44.640 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T10:47:44.641 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T10:47:44.642 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T10:47:44.643 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T10:47:44.644 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T10:47:44.645 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T10:47:44.646 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T10:47:44.647 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T10:47:44.648 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T10:47:44.649 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T10:47:44.650 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T10:47:44.651 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T10:47:44.652 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T10:47:44.653 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T10:47:44.654 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T10:47:44.655 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T10:47:44.656 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T10:47:44.657 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T10:47:44.658 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T10:47:44.659 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T10:47:44.660 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T10:47:44.661 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T10:47:44.662 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T10:47:44.663 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T10:47:44.664 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T10:47:44.665 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T10:47:44.666 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T10:47:44.667 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T10:47:44.668 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T10:47:44.669 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T10:47:44.670 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T10:47:44.671 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T10:47:44.672 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T10:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T10:47:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T10:47:44.675 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T10:47:44.676 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T10:47:44.677 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T10:47:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T10:47:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T10:47:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T10:47:44.681 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T10:47:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T10:47:44.683 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T10:47:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T10:47:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T10:47:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T10:47:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T10:47:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T10:47:44.689 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T10:47:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T10:47:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T10:47:44.692 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T10:47:44.693 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T10:47:44.694 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T10:47:44.695 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T10:47:44.696 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T10:47:44.697 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T10:47:44.698 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T10:47:44.699 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T10:47:44.700 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T10:47:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T10:47:44.702 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T10:47:44.703 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T10:47:44.704 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T10:47:44.704 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T10:47:44.704 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T10:47:44.704 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T10:47:44.704 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T10:47:44.704 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T10:47:44.704 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:47:44.708 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:44.708 DEBUG:teuthology.orchestra.run.smithi117:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T10:47:44.730 INFO:teuthology.orchestra.run.smithi117.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T10:47:44.730 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:47:44.730 INFO:teuthology.orchestra.run.smithi117.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:47:44.730 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2024-04-20T10:47:44.730 INFO:teuthology.orchestra.run.smithi117.stdout:for file in vmlinuz System.map config; do 2024-04-20T10:47:44.730 INFO:teuthology.orchestra.run.smithi117.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout: fi 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout:done 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:47:44.731 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:44.732 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2024-04-20T10:47:44.732 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 68.el9 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8264254 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T10:47:44.810 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:47:44.811 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T10:47:44.812 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T10:47:44.812 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:44.812 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T10:47:45.393 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-04-20T10:47:45.576 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi173.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T10:47:45.577 DEBUG:teuthology.orchestra.run.smithi173:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T10:47:45.660 INFO:teuthology.orchestra.run.smithi173.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:45.660 INFO:teuthology.orchestra.run.smithi173.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:45.660 INFO:teuthology.orchestra.run.smithi173.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:45.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T10:47:45.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T10:47:45.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T10:47:45.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T10:47:45.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T10:47:45.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T10:47:45.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T10:47:45.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T10:47:45.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T10:47:45.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T10:47:45.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T10:47:45.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T10:47:45.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T10:47:45.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T10:47:45.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T10:47:45.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T10:47:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T10:47:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T10:47:45.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T10:47:45.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T10:47:45.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T10:47:45.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T10:47:45.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T10:47:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T10:47:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T10:47:45.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T10:47:45.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T10:47:45.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T10:47:45.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T10:47:45.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T10:47:45.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T10:47:45.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T10:47:45.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T10:47:45.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T10:47:45.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T10:47:45.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T10:47:45.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T10:47:45.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T10:47:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T10:47:45.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T10:47:45.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T10:47:45.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T10:47:45.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T10:47:45.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T10:47:45.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T10:47:45.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T10:47:45.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T10:47:45.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T10:47:45.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T10:47:45.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T10:47:45.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T10:47:45.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T10:47:45.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T10:47:45.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T10:47:45.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T10:47:45.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T10:47:45.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T10:47:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T10:47:45.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T10:47:45.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T10:47:45.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T10:47:45.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T10:47:45.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T10:47:45.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T10:47:45.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T10:47:45.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T10:47:45.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T10:47:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T10:47:45.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T10:47:45.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T10:47:45.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T10:47:45.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T10:47:45.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T10:47:45.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T10:47:45.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T10:47:45.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T10:47:45.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T10:47:45.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T10:47:45.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T10:47:45.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T10:47:45.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T10:47:45.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T10:47:45.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T10:47:45.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T10:47:45.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T10:47:45.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T10:47:45.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T10:47:45.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T10:47:45.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T10:47:45.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T10:47:45.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T10:47:45.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T10:47:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T10:47:45.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T10:47:45.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T10:47:45.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T10:47:45.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T10:47:45.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T10:47:45.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T10:47:45.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T10:47:45.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T10:47:45.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T10:47:45.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T10:47:45.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T10:47:45.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T10:47:45.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T10:47:45.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T10:47:45.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T10:47:45.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T10:47:45.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T10:47:45.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T10:47:45.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T10:47:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T10:47:45.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T10:47:45.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T10:47:45.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T10:47:45.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T10:47:45.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T10:47:45.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T10:47:45.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T10:47:45.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T10:47:45.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T10:47:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T10:47:45.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T10:47:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T10:47:45.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T10:47:45.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T10:47:45.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T10:47:45.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T10:47:45.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T10:47:45.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T10:47:45.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T10:47:45.789 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T10:47:45.790 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T10:47:45.791 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T10:47:45.792 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T10:47:45.793 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T10:47:45.794 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T10:47:45.795 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T10:47:45.796 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T10:47:45.797 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T10:47:45.798 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T10:47:45.798 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T10:47:45.798 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T10:47:45.798 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:47:45.801 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:45.801 DEBUG:teuthology.orchestra.run.smithi173:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout:fi 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout:for file in vmlinuz System.map config; do 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout: fi 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout:done 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T10:47:45.824 INFO:teuthology.orchestra.run.smithi173.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout:fi 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout:fi 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:45.825 INFO:teuthology.orchestra.run.smithi173.stdout:fi 2024-04-20T10:47:45.826 DEBUG:teuthology.orchestra.run.smithi173:> sudo rpm -qi grub2-tools 2024-04-20T10:47:45.898 INFO:teuthology.orchestra.run.smithi173.stdout:Name : grub2-tools 2024-04-20T10:47:45.898 INFO:teuthology.orchestra.run.smithi173.stdout:Epoch : 1 2024-04-20T10:47:45.898 INFO:teuthology.orchestra.run.smithi173.stdout:Version : 2.06 2024-04-20T10:47:45.898 INFO:teuthology.orchestra.run.smithi173.stdout:Release : 68.el9 2024-04-20T10:47:45.898 INFO:teuthology.orchestra.run.smithi173.stdout:Architecture: x86_64 2024-04-20T10:47:45.898 INFO:teuthology.orchestra.run.smithi173.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T10:47:45.898 INFO:teuthology.orchestra.run.smithi173.stdout:Group : Unspecified 2024-04-20T10:47:45.898 INFO:teuthology.orchestra.run.smithi173.stdout:Size : 8264254 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:License : GPLv3+ 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:Packager : builder@centos.org 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:Vendor : CentOS 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:Summary : Support tools for GRUB. 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:Description : 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T10:47:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:hardware devices. 2024-04-20T10:47:45.900 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:47:45.900 INFO:teuthology.orchestra.run.smithi173.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T10:47:45.900 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T10:47:45.901 INFO:teuthology.task.kernel:Updating grub on smithi173 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:45.901 DEBUG:teuthology.orchestra.run.smithi173:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T10:47:46.436 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2024-04-20T10:47:46.750 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T10:47:46.783 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-04-20T10:47:46.785 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-04-20T10:47:46.806 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.WDqtzACusM 2024-04-20T10:47:46.806 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WDqtzACusM 2024-04-20T10:47:46.886 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.WDqtzACusM 2024-04-20T10:47:47.032 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.WDqtzACusM is 6KB 2024-04-20T10:47:47.081 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.WDqtzACusM 2024-04-20T10:47:47.095 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T10:47:47.156 INFO:teuthology.orchestra.run.smithi117.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T10:47:47.156 INFO:teuthology.orchestra.run.smithi117.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T10:47:47.156 INFO:teuthology.orchestra.run.smithi117.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T10:47:47.156 INFO:teuthology.orchestra.run.smithi117.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T10:47:47.157 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:47.295 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-04-20T10:47:47.833 INFO:teuthology.orchestra.run.smithi173.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T10:47:47.858 INFO:teuthology.orchestra.run.smithi173.stderr:done 2024-04-20T10:47:47.861 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-04-20T10:47:47.876 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.NtGByhXNt2 2024-04-20T10:47:47.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NtGByhXNt2 2024-04-20T10:47:47.941 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.NtGByhXNt2 2024-04-20T10:47:48.080 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.NtGByhXNt2 is 6KB 2024-04-20T10:47:48.128 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.NtGByhXNt2 2024-04-20T10:47:48.142 DEBUG:teuthology.orchestra.run.smithi173:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T10:47:48.204 INFO:teuthology.orchestra.run.smithi173.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T10:47:48.204 INFO:teuthology.orchestra.run.smithi173.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T10:47:48.204 INFO:teuthology.orchestra.run.smithi173.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T10:47:48.204 INFO:teuthology.orchestra.run.smithi173.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T10:47:48.205 DEBUG:teuthology.orchestra.run.smithi173:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T10:47:48.376 DEBUG:teuthology.orchestra.run.smithi173:> sudo shutdown -r now 2024-04-20T10:48:17.326 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-04-20T10:48:17.326 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-20T10:48:17.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:48:18.380 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-04-20T10:48:18.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-20T10:48:18.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:48:35.871 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-04-20T10:48:44.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-20T10:48:44.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:48:47.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-04-20T10:48:59.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-20T10:48:59.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:49:18.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-04-20T10:49:18.383 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T10:49:27.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-20T10:49:27.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:49:30.920 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-04-20T10:49:31.527 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-20T10:49:31.528 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T10:49:31.528 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-04-20T10:49:31.563 INFO:teuthology.orchestra.run.smithi173.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T10:49:31.563 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T10:49:31.563 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T10:49:31.563 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T10:49:32.564 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-20T10:49:32.565 DEBUG:teuthology.orchestra.run.smithi173:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T10:49:32.691 INFO:teuthology.orchestra.run.smithi173.stdout:ttyS1 2024-04-20T10:49:32.757 DEBUG:teuthology.parallel:result is None 2024-04-20T10:49:33.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-20T10:49:33.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:49:33.749 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-04-20T10:49:34.382 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-20T10:49:34.382 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T10:49:34.382 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-04-20T10:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T10:49:34.423 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T10:49:34.423 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T10:49:34.423 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T10:49:35.424 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-20T10:49:35.424 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T10:49:35.557 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2024-04-20T10:49:35.638 DEBUG:teuthology.parallel:result is None 2024-04-20T10:49:35.639 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T10:49:35.645 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T10:49:35.645 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:49:35.649 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:49:35.699 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T10:49:35.706 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T10:49:35.712 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T10:49:35.712 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:49:35.728 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:49:35.799 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T10:49:35.805 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T10:49:35.805 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:49:35.811 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:49:35.891 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:49:35.921 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:49:35.936 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:49:35.968 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:49:35.973 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T10:49:35.980 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T10:49:35.980 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:49:35.985 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:49:36.071 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T10:49:36.080 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T10:49:36.080 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:49:36.087 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:49:36.130 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:49:36.246 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:49:36.326 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:49:36.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:49:36.485 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:49:36.486 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:49:36.546 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:49:36.613 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:49:36.688 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:49:36.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:49:36.836 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:49:36.836 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:49:36.928 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-04-20T10:49:36.936 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-04-20T10:49:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:49:37.033 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:49:37.346 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T10:49:37.356 INFO:teuthology.task.internal:Starting timer... 2024-04-20T10:49:37.356 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T10:49:37.366 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T10:49:37.374 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-20T10:49:37.375 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-04-20T10:49:37.491 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-04-20T10:49:37.495 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2024-04-20T10:49:37.607 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2024-04-20T10:49:37.611 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T10:49:37.611 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-04-20T10:49:37.670 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-04-20T10:49:37.671 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2024-04-20T10:49:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2024-04-20T10:49:37.722 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive'} 2024-04-20T10:49:37.722 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:49:37.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:49:37.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:49:37.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:49:37.864 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T10:49:37.864 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:49:37.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:49:38.010 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T10:49:38.020 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T10:49:38.039 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T10:49:38.041 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi117.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T10:51:55.136 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-04-20T10:51:55.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-20T10:51:55.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:51:55.242 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-04-20T10:51:55.389 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-20T10:51:55.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-20T10:51:55.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:51:55.497 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-04-20T10:51:55.651 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-20T10:51:55.651 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T10:51:55.661 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T10:51:55.662 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:51:55.662 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:51:55.670 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:51:55.670 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:51:55.758 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:51:55.763 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:51:55.797 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:51:55.802 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:51:55.872 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-04-20T10:51:55.874 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2024-04-20T10:51:55.904 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-04-20T10:51:55.906 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2024-04-20T10:51:55.942 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:51:55.945 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:51:55.985 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:51:55.987 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:51:56.118 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:51:56.127 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:51:56.127 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-04-20T10:51:56.127 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:51:56.127 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:51:56.127 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:51:56.127 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:51:56.132 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:51:56.142 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:51:56.142 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-20T10:51:56.143 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:51:56.143 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:51:56.143 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:51:56.143 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:51:56.144 INFO:teuthology.run_tasks:Running task install... 2024-04-20T10:51:56.152 DEBUG:teuthology.task.install:project ceph 2024-04-20T10:51:56.152 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T10:51:56.152 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T10:51:56.152 INFO:teuthology.task.install:Using flavor: default 2024-04-20T10:51:56.160 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T10:51:56.160 INFO:teuthology.task.install:extra packages: [] 2024-04-20T10:51:56.160 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:51:56.161 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:51:56.161 INFO:teuthology.packaging:ref: None 2024-04-20T10:51:56.161 INFO:teuthology.packaging:tag: None 2024-04-20T10:51:56.161 INFO:teuthology.packaging:branch: reef 2024-04-20T10:51:56.161 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:51:56.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-20T10:51:56.163 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:51:56.163 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:51:56.163 INFO:teuthology.packaging:ref: None 2024-04-20T10:51:56.163 INFO:teuthology.packaging:tag: None 2024-04-20T10:51:56.163 INFO:teuthology.packaging:branch: reef 2024-04-20T10:51:56.163 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:51:56.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-20T10:51:56.384 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/flavors/default/ 2024-04-20T10:51:56.384 INFO:teuthology.task.install.rpm:Package version is 18.2.2-0 2024-04-20T10:51:56.428 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:51:56.428 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:51:56.428 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:51:56.501 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/flavors/default/ 2024-04-20T10:51:56.501 INFO:teuthology.task.install.rpm:Package version is 18.2.2-0 2024-04-20T10:51:56.518 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-20T10:51:56.518 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:51:56.518 INFO:teuthology.packaging:ref: None 2024-04-20T10:51:56.518 INFO:teuthology.packaging:tag: None 2024-04-20T10:51:56.518 INFO:teuthology.packaging:branch: reef 2024-04-20T10:51:56.518 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:51:56.518 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:51:56.530 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:51:56.531 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:51:56.531 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:51:56.619 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-20T10:51:56.619 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:51:56.619 INFO:teuthology.packaging:ref: None 2024-04-20T10:51:56.619 INFO:teuthology.packaging:tag: None 2024-04-20T10:51:56.619 INFO:teuthology.packaging:branch: reef 2024-04-20T10:51:56.619 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:51:56.620 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:51:56.647 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:51:56.745 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:51:56.752 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:51:56.829 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-04-20T10:51:56.834 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-04-20T10:51:56.855 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:51:56.933 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2024-04-20T10:51:56.937 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2024-04-20T10:51:57.153 INFO:teuthology.orchestra.run.smithi117.stdout:48 files removed 2024-04-20T10:51:57.190 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw 2024-04-20T10:51:57.259 INFO:teuthology.orchestra.run.smithi173.stdout:48 files removed 2024-04-20T10:51:57.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw 2024-04-20T10:51:57.839 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 349 kB/s | 76 kB 00:00 2024-04-20T10:51:57.914 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 416 kB/s | 76 kB 00:00 2024-04-20T10:51:58.132 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2024-04-20T10:51:58.163 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 49 kB/s | 9.4 kB 00:00 2024-04-20T10:51:58.406 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 11 kB/s | 2.2 kB 00:00 2024-04-20T10:51:58.407 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 10 kB/s | 2.2 kB 00:00 2024-04-20T10:51:59.098 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-20T10:51:59.648 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-20T10:52:01.357 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-20T10:52:06.000 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.2 MB 00:00 2024-04-20T10:52:07.600 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 16 kB 00:00 2024-04-20T10:52:09.374 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-20T10:52:12.131 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 19 MB 00:11 2024-04-20T10:52:15.359 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-20T10:52:16.447 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-20T10:52:17.829 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:52:17.837 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:17.837 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-20T10:52:17.837 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:18.2.2-0.el9 ceph 7.7 M 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:18.2.2-0.el9 ceph 3.3 M 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:18.2.2-0.el9 ceph 3.0 M 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:18.2.2-0.el9 ceph 5.2 M 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:18.2.2-0.el9 ceph 18 M 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:18.2.2-0.el9 ceph 25 k 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:52:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:18.2.2-0.el9 ceph 658 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:18.2.2-0.el9 ceph 475 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:18.2.2-0.el9 ceph 4.5 M 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:18.2.2-0.el9 ceph 45 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:18.2.2-0.el9 ceph 124 k 2024-04-20T10:52:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:18.2.2-0.el9 ceph 162 k 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:18.2.2-0.el9 ceph 320 k 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:18.2.2-0.el9 ceph 298 k 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:18.2.2-0.el9 ceph 99 k 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout:Install 28 Packages 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 2 Packages 2024-04-20T10:52:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:17.841 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 53 M 2024-04-20T10:52:17.841 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:52:18.265 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-20T10:52:18.536 INFO:teuthology.orchestra.run.smithi173.stdout:(1/30): ceph-base-18.2.2-0.el9.x86_64.rpm 11 MB/s | 5.2 MB 00:00 2024-04-20T10:52:18.569 INFO:teuthology.orchestra.run.smithi173.stdout:(2/30): ceph-selinux-18.2.2-0.el9.x86_64.rpm 768 kB/s | 25 kB 00:00 2024-04-20T10:52:18.687 INFO:teuthology.orchestra.run.smithi173.stdout:(3/30): ceph-radosgw-18.2.2-0.el9.x86_64.rpm 12 MB/s | 7.7 MB 00:00 2024-04-20T10:52:18.737 INFO:teuthology.orchestra.run.smithi173.stdout:(4/30): libcephfs2-18.2.2-0.el9.x86_64.rpm 3.9 MB/s | 658 kB 00:00 2024-04-20T10:52:18.781 INFO:teuthology.orchestra.run.smithi173.stdout:(5/30): libradosstriper1-18.2.2-0.el9.x86_64.rp 5.0 MB/s | 475 kB 00:00 2024-04-20T10:52:18.815 INFO:teuthology.orchestra.run.smithi173.stdout:(6/30): python3-ceph-argparse-18.2.2-0.el9.x86_ 1.4 MB/s | 45 kB 00:00 2024-04-20T10:52:18.890 INFO:teuthology.orchestra.run.smithi173.stdout:(7/30): python3-ceph-common-18.2.2-0.el9.x86_64 1.6 MB/s | 124 kB 00:00 2024-04-20T10:52:18.932 INFO:teuthology.orchestra.run.smithi173.stdout:(8/30): python3-cephfs-18.2.2-0.el9.x86_64.rpm 3.9 MB/s | 162 kB 00:00 2024-04-20T10:52:18.991 INFO:teuthology.orchestra.run.smithi173.stdout:(9/30): python3-rados-18.2.2-0.el9.x86_64.rpm 5.4 MB/s | 320 kB 00:00 2024-04-20T10:52:19.058 INFO:teuthology.orchestra.run.smithi173.stdout:(10/30): python3-rbd-18.2.2-0.el9.x86_64.rpm 4.4 MB/s | 298 kB 00:00 2024-04-20T10:52:19.158 INFO:teuthology.orchestra.run.smithi173.stdout:(11/30): librgw2-18.2.2-0.el9.x86_64.rpm 11 MB/s | 4.5 MB 00:00 2024-04-20T10:52:19.183 INFO:teuthology.orchestra.run.smithi173.stdout:(12/30): python3-rgw-18.2.2-0.el9.x86_64.rpm 799 kB/s | 99 kB 00:00 2024-04-20T10:52:19.242 INFO:teuthology.orchestra.run.smithi173.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 1.3 MB/s | 107 kB 00:00 2024-04-20T10:52:19.275 INFO:teuthology.orchestra.run.smithi173.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-20T10:52:19.326 INFO:teuthology.orchestra.run.smithi173.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 546 kB/s | 45 kB 00:00 2024-04-20T10:52:19.576 INFO:teuthology.orchestra.run.smithi173.stdout:(16/30): ceph-common-18.2.2-0.el9.x86_64.rpm 12 MB/s | 18 MB 00:01 2024-04-20T10:52:19.618 INFO:teuthology.orchestra.run.smithi173.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-04-20T10:52:19.668 INFO:teuthology.orchestra.run.smithi173.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 857 kB/s | 292 kB 00:00 2024-04-20T10:52:19.768 INFO:teuthology.orchestra.run.smithi173.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 221 kB/s | 42 kB 00:00 2024-04-20T10:52:20.044 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-20T10:52:20.127 INFO:teuthology.orchestra.run.smithi173.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 218 kB/s | 111 kB 00:00 2024-04-20T10:52:20.211 INFO:teuthology.orchestra.run.smithi173.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 656 kB/s | 309 kB 00:00 2024-04-20T10:52:20.236 INFO:teuthology.orchestra.run.smithi173.stdout:(22/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-04-20T10:52:20.294 INFO:teuthology.orchestra.run.smithi173.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 587 kB/s | 49 kB 00:00 2024-04-20T10:52:20.328 INFO:teuthology.orchestra.run.smithi173.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 741 kB/s | 67 kB 00:00 2024-04-20T10:52:20.462 INFO:teuthology.orchestra.run.smithi173.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-20T10:52:20.504 INFO:teuthology.orchestra.run.smithi173.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-20T10:52:20.629 INFO:teuthology.orchestra.run.smithi173.stdout:(27/30): libarrow-9.0.0-10.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-04-20T10:52:20.721 INFO:teuthology.orchestra.run.smithi173.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-04-20T10:52:20.829 INFO:teuthology.orchestra.run.smithi173.stdout:(29/30): librados2-18.2.2-0.el9.x86_64.rpm 10 MB/s | 3.3 MB 00:00 2024-04-20T10:52:20.922 INFO:teuthology.orchestra.run.smithi173.stdout:(30/30): librbd1-18.2.2-0.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-04-20T10:52:20.932 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:52:20.932 INFO:teuthology.orchestra.run.smithi173.stdout:Total 17 MB/s | 53 MB 00:03 2024-04-20T10:52:21.157 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:52:21.254 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:52:21.255 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:52:22.177 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:52:22.178 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:52:23.769 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:52:23.885 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-20T10:52:23.930 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-20T10:52:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-20T10:52:24.457 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-20T10:52:24.583 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:18.2.2-0.el9.x86_64 5/32 2024-04-20T10:52:24.718 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:18.2.2-0.el9.x86_64 5/32 2024-04-20T10:52:24.826 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:18.2.2-0.el9.x86_64 6/32 2024-04-20T10:52:24.851 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:18.2.2-0.el9.x86_64 7/32 2024-04-20T10:52:24.935 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:18.2.2-0.el9.x86_64 7/32 2024-04-20T10:52:24.979 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-20T10:52:25.052 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:18.2.2-0.el9.x86_64 9/32 2024-04-20T10:52:25.155 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:18.2.2-0.el9.x86_64 10/32 2024-04-20T10:52:25.187 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:18.2.2-0.el9.x86_64 11/32 2024-04-20T10:52:25.279 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:18.2.2-0.el9.x86_64 11/32 2024-04-20T10:52:25.466 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-20T10:52:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-20T10:52:25.559 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-20T10:52:25.596 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-20T10:52:25.636 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-20T10:52:25.685 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-20T10:52:25.733 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-20T10:52:25.876 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-20T10:52:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-20T10:52:26.192 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:18.2.2-0.el9.x86_64 20/32 2024-04-20T10:52:26.314 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:18.2.2-0.el9.x86_64 20/32 2024-04-20T10:52:26.350 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:18.2.2-0.el9.x86_64 21/32 2024-04-20T10:52:26.410 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-20T10:52:26.886 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-20T10:52:26.919 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:18.2.2-0.el9.x86_64 24/32 2024-04-20T10:52:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:18.2.2-0.el9.x86_64 24/32 2024-04-20T10:52:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:18.2.2-0.el9.x86_64 25/32 2024-04-20T10:52:27.181 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:18.2.2-0.el9.x86_64 26/32 2024-04-20T10:52:28.334 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:52:28.350 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:18.2.2-0.el9 ceph 7.7 M 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:18.2.2-0.el9 ceph 3.3 M 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:18.2.2-0.el9 ceph 3.0 M 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:18.2.2-0.el9 ceph 5.2 M 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:18.2.2-0.el9 ceph 18 M 2024-04-20T10:52:28.351 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:18.2.2-0.el9 ceph 25 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:18.2.2-0.el9 ceph 658 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:18.2.2-0.el9 ceph 475 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:52:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:18.2.2-0.el9 ceph 4.5 M 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:18.2.2-0.el9 ceph 45 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:18.2.2-0.el9 ceph 124 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:18.2.2-0.el9 ceph 162 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:18.2.2-0.el9 ceph 320 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:18.2.2-0.el9 ceph 298 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:18.2.2-0.el9 ceph 99 k 2024-04-20T10:52:28.353 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:52:28.354 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:52:28.354 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:28.354 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:52:28.354 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:52:28.354 INFO:teuthology.orchestra.run.smithi117.stdout:Install 28 Packages 2024-04-20T10:52:28.354 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 2 Packages 2024-04-20T10:52:28.354 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:28.355 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 53 M 2024-04-20T10:52:28.355 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:52:28.936 INFO:teuthology.orchestra.run.smithi117.stdout:(1/30): ceph-base-18.2.2-0.el9.x86_64.rpm 17 MB/s | 5.2 MB 00:00 2024-04-20T10:52:28.961 INFO:teuthology.orchestra.run.smithi117.stdout:(2/30): ceph-selinux-18.2.2-0.el9.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2024-04-20T10:52:29.003 INFO:teuthology.orchestra.run.smithi117.stdout:(3/30): libcephfs2-18.2.2-0.el9.x86_64.rpm 16 MB/s | 658 kB 00:00 2024-04-20T10:52:29.045 INFO:teuthology.orchestra.run.smithi117.stdout:(4/30): libradosstriper1-18.2.2-0.el9.x86_64.rp 11 MB/s | 475 kB 00:00 2024-04-20T10:52:29.204 INFO:teuthology.orchestra.run.smithi117.stdout:(5/30): ceph-radosgw-18.2.2-0.el9.x86_64.rpm 13 MB/s | 7.7 MB 00:00 2024-04-20T10:52:29.237 INFO:teuthology.orchestra.run.smithi117.stdout:(6/30): python3-ceph-argparse-18.2.2-0.el9.x86_ 1.3 MB/s | 45 kB 00:00 2024-04-20T10:52:29.271 INFO:teuthology.orchestra.run.smithi117.stdout:(7/30): python3-ceph-common-18.2.2-0.el9.x86_64 3.7 MB/s | 124 kB 00:00 2024-04-20T10:52:29.305 INFO:teuthology.orchestra.run.smithi117.stdout:(8/30): python3-cephfs-18.2.2-0.el9.x86_64.rpm 4.9 MB/s | 162 kB 00:00 2024-04-20T10:52:29.347 INFO:teuthology.orchestra.run.smithi117.stdout:(9/30): python3-rados-18.2.2-0.el9.x86_64.rpm 7.7 MB/s | 320 kB 00:00 2024-04-20T10:52:29.389 INFO:teuthology.orchestra.run.smithi117.stdout:(10/30): python3-rbd-18.2.2-0.el9.x86_64.rpm 7.2 MB/s | 298 kB 00:00 2024-04-20T10:52:29.422 INFO:teuthology.orchestra.run.smithi117.stdout:(11/30): python3-rgw-18.2.2-0.el9.x86_64.rpm 3.0 MB/s | 99 kB 00:00 2024-04-20T10:52:29.539 INFO:teuthology.orchestra.run.smithi117.stdout:(12/30): librgw2-18.2.2-0.el9.x86_64.rpm 9.0 MB/s | 4.5 MB 00:00 2024-04-20T10:52:29.598 INFO:teuthology.orchestra.run.smithi117.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 614 kB/s | 107 kB 00:00 2024-04-20T10:52:29.648 INFO:teuthology.orchestra.run.smithi117.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 923 kB/s | 45 kB 00:00 2024-04-20T10:52:29.748 INFO:teuthology.orchestra.run.smithi117.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 772 kB/s | 160 kB 00:00 2024-04-20T10:52:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:18.2.2-0.el9.x86_64 27/32 2024-04-20T10:52:29.790 INFO:teuthology.orchestra.run.smithi117.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-04-20T10:52:29.802 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:18.2.2-0.el9.x86_64 27/32 2024-04-20T10:52:30.024 INFO:teuthology.orchestra.run.smithi117.stdout:(17/30): ceph-common-18.2.2-0.el9.x86_64.rpm 13 MB/s | 18 MB 00:01 2024-04-20T10:52:30.057 INFO:teuthology.orchestra.run.smithi117.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 949 kB/s | 292 kB 00:00 2024-04-20T10:52:30.091 INFO:teuthology.orchestra.run.smithi117.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 141 kB/s | 42 kB 00:00 2024-04-20T10:52:30.358 INFO:teuthology.orchestra.run.smithi117.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 333 kB/s | 111 kB 00:00 2024-04-20T10:52:30.391 INFO:teuthology.orchestra.run.smithi117.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 928 kB/s | 309 kB 00:00 2024-04-20T10:52:30.416 INFO:teuthology.orchestra.run.smithi117.stdout:(22/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 448 kB/s | 26 kB 00:00 2024-04-20T10:52:30.467 INFO:teuthology.orchestra.run.smithi117.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 653 kB/s | 49 kB 00:00 2024-04-20T10:52:30.492 INFO:teuthology.orchestra.run.smithi117.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 904 kB/s | 67 kB 00:00 2024-04-20T10:52:30.496 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:18.2.2-0.el9.x86_64 27/32 2024-04-20T10:52:30.547 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:18.2.2-0.el9.x86_64 28/32 2024-04-20T10:52:30.609 INFO:teuthology.orchestra.run.smithi117.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-20T10:52:30.688 INFO:teuthology.orchestra.run.smithi117.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-04-20T10:52:30.746 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:18.2.2-0.el9.x86_64 28/32 2024-04-20T10:52:30.746 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:52:30.747 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:52:30.747 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:30.806 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:18.2.2-0.el9.x86_64 29/32 2024-04-20T10:52:30.818 INFO:teuthology.orchestra.run.smithi117.stdout:(27/30): librados2-18.2.2-0.el9.x86_64.rpm 25 MB/s | 3.3 MB 00:00 2024-04-20T10:52:30.935 INFO:teuthology.orchestra.run.smithi117.stdout:(28/30): librbd1-18.2.2-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-04-20T10:52:31.018 INFO:teuthology.orchestra.run.smithi117.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-04-20T10:52:31.735 INFO:teuthology.orchestra.run.smithi117.stdout:(30/30): libarrow-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-04-20T10:52:31.740 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:52:31.741 INFO:teuthology.orchestra.run.smithi117.stdout:Total 16 MB/s | 53 MB 00:03 2024-04-20T10:52:31.960 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:52:32.067 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:52:32.067 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:52:32.991 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:52:32.992 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:52:34.554 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:52:34.706 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-20T10:52:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-20T10:52:34.860 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-20T10:52:35.270 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-20T10:52:35.364 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:18.2.2-0.el9.x86_64 5/32 2024-04-20T10:52:35.505 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:18.2.2-0.el9.x86_64 5/32 2024-04-20T10:52:35.618 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:18.2.2-0.el9.x86_64 6/32 2024-04-20T10:52:35.648 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:18.2.2-0.el9.x86_64 7/32 2024-04-20T10:52:35.716 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:18.2.2-0.el9.x86_64 7/32 2024-04-20T10:52:35.750 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-20T10:52:35.817 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:18.2.2-0.el9.x86_64 9/32 2024-04-20T10:52:35.915 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:18.2.2-0.el9.x86_64 10/32 2024-04-20T10:52:35.972 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:18.2.2-0.el9.x86_64 11/32 2024-04-20T10:52:36.068 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:18.2.2-0.el9.x86_64 11/32 2024-04-20T10:52:36.256 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-20T10:52:36.294 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-20T10:52:36.349 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-20T10:52:36.385 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-20T10:52:36.424 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-20T10:52:36.482 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-20T10:52:36.531 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-20T10:52:36.933 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-20T10:52:36.973 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:18.2.2-0.el9.x86_64 20/32 2024-04-20T10:52:37.088 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:18.2.2-0.el9.x86_64 20/32 2024-04-20T10:52:37.122 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:18.2.2-0.el9.x86_64 21/32 2024-04-20T10:52:37.179 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-20T10:52:37.669 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-20T10:52:37.700 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:18.2.2-0.el9.x86_64 24/32 2024-04-20T10:52:37.801 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:18.2.2-0.el9.x86_64 24/32 2024-04-20T10:52:37.901 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:18.2.2-0.el9.x86_64 25/32 2024-04-20T10:52:37.927 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:18.2.2-0.el9.x86_64 26/32 2024-04-20T10:52:40.498 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:18.2.2-0.el9.x86_64 27/32 2024-04-20T10:52:40.549 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:18.2.2-0.el9.x86_64 27/32 2024-04-20T10:52:41.229 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:18.2.2-0.el9.x86_64 27/32 2024-04-20T10:52:41.277 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:18.2.2-0.el9.x86_64 28/32 2024-04-20T10:52:41.466 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:18.2.2-0.el9.x86_64 28/32 2024-04-20T10:52:41.467 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:52:41.467 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:52:41.467 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:41.503 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:18.2.2-0.el9.x86_64 29/32 2024-04-20T10:52:41.942 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:18.2.2-0.el9.x86_64 29/32 2024-04-20T10:52:41.943 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2024-04-20T10:52:41.943 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2024-04-20T10:52:41.943 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2024-04-20T10:52:41.943 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2024-04-20T10:52:41.943 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2024-04-20T10:52:41.943 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2024-04-20T10:52:41.943 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2024-04-20T10:52:41.943 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:42.813 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:18.2.2-0.el9.x86_64 30/32 2024-04-20T10:52:42.880 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-0.el9.x86_64 30/32 2024-04-20T10:52:42.880 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:52:42.880 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:52:42.880 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:52:42.880 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:52:42.881 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:42.885 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-20T10:52:43.001 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-20T10:52:43.002 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-20T10:52:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-20T10:52:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:18.2.2-0.el9.x86_64 1/32 2024-04-20T10:52:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:18.2.2-0.el9.x86_64 2/32 2024-04-20T10:52:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:18.2.2-0.el9.x86_64 3/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:18.2.2-0.el9.x86_64 4/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:18.2.2-0.el9.x86_64 5/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:18.2.2-0.el9.x86_64 6/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:18.2.2-0.el9.x86_64 7/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:18.2.2-0.el9.x86_64 8/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:18.2.2-0.el9.x86_64 9/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:18.2.2-0.el9.x86_64 10/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:18.2.2-0.el9.x86_64 11/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:18.2.2-0.el9.x86_64 12/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:18.2.2-0.el9.x86_64 13/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-20T10:52:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-20T10:52:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-20T10:52:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-20T10:52:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:18.2.2-0.el9.x86_64 29/32 2024-04-20T10:52:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-20T10:52:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:18.2.2-0.el9.x86_64 31/32 2024-04-20T10:52:45.323 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:18.2.2-0.el9.x86_64 librbd1-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:52:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:45.326 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:52:45.723 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-test 2024-04-20T10:52:46.471 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:31 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:52:46.608 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:52:46.608 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:46.608 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-20T10:52:46.608 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:46.608 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:18.2.2-0.el9 ceph 40 M 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:18.2.2-0.el9 ceph 165 k 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout:Install 4 Packages 2024-04-20T10:52:46.609 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:46.610 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 40 M 2024-04-20T10:52:46.610 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 166 M 2024-04-20T10:52:46.610 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:52:46.894 INFO:teuthology.orchestra.run.smithi173.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-20T10:52:46.944 INFO:teuthology.orchestra.run.smithi173.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-20T10:52:46.970 INFO:teuthology.orchestra.run.smithi173.stdout:(3/4): libcephsqlite-18.2.2-0.el9.x86_64.rpm 594 kB/s | 165 kB 00:00 2024-04-20T10:52:47.904 INFO:teuthology.orchestra.run.smithi173.stdout:(4/4): ceph-test-18.2.2-0.el9.x86_64.rpm 33 MB/s | 40 MB 00:01 2024-04-20T10:52:47.910 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:52:47.912 INFO:teuthology.orchestra.run.smithi173.stdout:Total 31 MB/s | 40 MB 00:01 2024-04-20T10:52:47.992 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:52:48.051 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:52:48.051 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:52:48.437 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:52:48.437 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:52:49.029 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:52:49.134 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-20T10:52:49.199 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-20T10:52:49.249 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:18.2.2-0.el9.x86_64 3/4 2024-04-20T10:52:52.429 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:18.2.2-0.el9.x86_64 29/32 2024-04-20T10:52:52.430 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-04-20T10:52:52.430 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-04-20T10:52:52.430 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-04-20T10:52:52.430 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-04-20T10:52:52.430 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-04-20T10:52:52.430 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-04-20T10:52:52.430 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-04-20T10:52:52.431 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:53.303 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:18.2.2-0.el9.x86_64 30/32 2024-04-20T10:52:53.391 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-0.el9.x86_64 30/32 2024-04-20T10:52:53.391 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:52:53.391 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:52:53.391 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:52:53.391 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:52:53.391 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:53.396 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-20T10:52:53.495 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:18.2.2-0.el9.x86_64 3/4 2024-04-20T10:52:53.533 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-20T10:52:53.533 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-20T10:52:53.955 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:18.2.2-0.el9.x86_64 4/4 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:18.2.2-0.el9.x86_64 1/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:18.2.2-0.el9.x86_64 2/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:18.2.2-0.el9.x86_64 3/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:18.2.2-0.el9.x86_64 4/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:18.2.2-0.el9.x86_64 5/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:18.2.2-0.el9.x86_64 6/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:18.2.2-0.el9.x86_64 7/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:18.2.2-0.el9.x86_64 8/32 2024-04-20T10:52:55.120 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:18.2.2-0.el9.x86_64 9/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:18.2.2-0.el9.x86_64 10/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:18.2.2-0.el9.x86_64 11/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:18.2.2-0.el9.x86_64 12/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:18.2.2-0.el9.x86_64 13/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-20T10:52:55.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:18.2.2-0.el9.x86_64 29/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-20T10:52:55.122 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:18.2.2-0.el9.x86_64 31/32 2024-04-20T10:52:55.697 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-test-2:18.2.2-0.el9.x86_64 4/4 2024-04-20T10:52:55.697 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:18.2.2-0.el9.x86_64 1/4 2024-04-20T10:52:55.697 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:18.2.2-0.el9.x86_64 2/4 2024-04-20T10:52:55.697 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-20T10:52:55.941 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-20T10:52:55.941 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:55.941 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:52:55.941 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:18.2.2-0.el9.x86_64 libcephsqlite-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.941 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:52:55.941 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:55.941 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:18.2.2-0.el9.x86_64 librbd1-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.962 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:52:55.963 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:18.2.2-0.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:52:56.219 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph 2024-04-20T10:52:56.376 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-test 2024-04-20T10:52:56.992 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:41 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:52:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:32 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:52:57.124 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:52:57.126 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:57.126 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:18.2.2-0.el9 ceph 6.4 k 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:18.2.2-0.el9 ceph 2.1 M 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:18.2.2-0.el9 ceph 1.4 M 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:18.2.2-0.el9 ceph-noarch 242 k 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:18.2.2-0.el9 ceph 4.5 M 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:18.2.2-0.el9 ceph 18 M 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:52:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:52:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:52:57.129 INFO:teuthology.orchestra.run.smithi173.stdout:Install 33 Packages 2024-04-20T10:52:57.130 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:52:57.130 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 30 M 2024-04-20T10:52:57.130 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 110 M 2024-04-20T10:52:57.130 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:52:57.254 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:52:57.254 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:52:57.254 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:18.2.2-0.el9 ceph 40 M 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:18.2.2-0.el9 ceph 165 k 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:52:57.255 INFO:teuthology.orchestra.run.smithi117.stdout:Install 4 Packages 2024-04-20T10:52:57.256 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:52:57.256 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 40 M 2024-04-20T10:52:57.256 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 166 M 2024-04-20T10:52:57.256 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:52:57.540 INFO:teuthology.orchestra.run.smithi117.stdout:(1/4): libcephsqlite-18.2.2-0.el9.x86_64.rpm 1.1 MB/s | 165 kB 00:00 2024-04-20T10:52:57.740 INFO:teuthology.orchestra.run.smithi173.stdout:(1/33): ceph-18.2.2-0.el9.x86_64.rpm 37 kB/s | 6.4 kB 00:00 2024-04-20T10:52:57.791 INFO:teuthology.orchestra.run.smithi117.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 772 kB/s | 305 kB 00:00 2024-04-20T10:52:57.807 INFO:teuthology.orchestra.run.smithi173.stdout:(2/33): ceph-mds-18.2.2-0.el9.x86_64.rpm 8.8 MB/s | 2.1 MB 00:00 2024-04-20T10:52:57.824 INFO:teuthology.orchestra.run.smithi117.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 225 kB/s | 64 kB 00:00 2024-04-20T10:52:58.141 INFO:teuthology.orchestra.run.smithi173.stdout:(3/33): ceph-mgr-18.2.2-0.el9.x86_64.rpm 2.5 MB/s | 1.4 MB 00:00 2024-04-20T10:52:58.241 INFO:teuthology.orchestra.run.smithi173.stdout:(4/33): ceph-mon-18.2.2-0.el9.x86_64.rpm 8.9 MB/s | 4.5 MB 00:00 2024-04-20T10:52:58.275 INFO:teuthology.orchestra.run.smithi173.stdout:(5/33): ceph-mgr-modules-core-18.2.2-0.el9.noar 1.8 MB/s | 242 kB 00:00 2024-04-20T10:52:58.391 INFO:teuthology.orchestra.run.smithi173.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-20T10:52:58.450 INFO:teuthology.orchestra.run.smithi173.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-20T10:52:58.561 INFO:teuthology.orchestra.run.smithi117.stdout:(4/4): ceph-test-18.2.2-0.el9.x86_64.rpm 34 MB/s | 40 MB 00:01 2024-04-20T10:52:58.602 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:52:58.602 INFO:teuthology.orchestra.run.smithi117.stdout:Total 31 MB/s | 40 MB 00:01 2024-04-20T10:52:58.646 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:52:58.659 INFO:teuthology.orchestra.run.smithi173.stdout:(8/33): ceph-osd-18.2.2-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-04-20T10:52:58.692 INFO:teuthology.orchestra.run.smithi173.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 560 kB/s | 135 kB 00:00 2024-04-20T10:52:58.702 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:52:58.702 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:52:58.742 INFO:teuthology.orchestra.run.smithi173.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 2.6 MB/s | 1.2 MB 00:00 2024-04-20T10:52:58.776 INFO:teuthology.orchestra.run.smithi173.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-04-20T10:52:58.810 INFO:teuthology.orchestra.run.smithi173.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 832 kB/s | 125 kB 00:00 2024-04-20T10:52:58.894 INFO:teuthology.orchestra.run.smithi173.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 299 kB/s | 35 kB 00:00 2024-04-20T10:52:58.952 INFO:teuthology.orchestra.run.smithi173.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 720 kB/s | 42 kB 00:00 2024-04-20T10:52:58.978 INFO:teuthology.orchestra.run.smithi173.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 306 kB/s | 51 kB 00:00 2024-04-20T10:52:59.003 INFO:teuthology.orchestra.run.smithi173.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 664 kB/s | 172 kB 00:00 2024-04-20T10:52:59.092 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:52:59.092 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:52:59.545 INFO:teuthology.orchestra.run.smithi173.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 73 kB/s | 43 kB 00:00 2024-04-20T10:52:59.628 INFO:teuthology.orchestra.run.smithi173.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 264 kB/s | 172 kB 00:00 2024-04-20T10:52:59.662 INFO:teuthology.orchestra.run.smithi173.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 92 kB/s | 11 kB 00:00 2024-04-20T10:52:59.683 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:52:59.745 INFO:teuthology.orchestra.run.smithi173.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 152 kB/s | 18 kB 00:00 2024-04-20T10:52:59.776 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-20T10:52:59.796 INFO:teuthology.orchestra.run.smithi173.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 174 kB/s | 23 kB 00:00 2024-04-20T10:52:59.829 INFO:teuthology.orchestra.run.smithi173.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 434 kB/s | 358 kB 00:00 2024-04-20T10:52:59.837 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-20T10:52:59.855 INFO:teuthology.orchestra.run.smithi173.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 180 kB/s | 19 kB 00:00 2024-04-20T10:52:59.896 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:18.2.2-0.el9.x86_64 3/4 2024-04-20T10:52:59.930 INFO:teuthology.orchestra.run.smithi173.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 148 kB/s | 20 kB 00:00 2024-04-20T10:52:59.986 INFO:teuthology.orchestra.run.smithi173.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 298 kB/s | 46 kB 00:00 2024-04-20T10:53:00.014 INFO:teuthology.orchestra.run.smithi173.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 498 kB/s | 79 kB 00:00 2024-04-20T10:53:00.097 INFO:teuthology.orchestra.run.smithi173.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 148 kB/s | 16 kB 00:00 2024-04-20T10:53:00.148 INFO:teuthology.orchestra.run.smithi173.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 676 kB/s | 90 kB 00:00 2024-04-20T10:53:00.214 INFO:teuthology.orchestra.run.smithi173.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 308 kB/s | 36 kB 00:00 2024-04-20T10:53:00.273 INFO:teuthology.orchestra.run.smithi173.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 796 kB/s | 272 kB 00:00 2024-04-20T10:53:00.332 INFO:teuthology.orchestra.run.smithi173.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-20T10:53:00.399 INFO:teuthology.orchestra.run.smithi173.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 159 kB/s | 20 kB 00:00 2024-04-20T10:53:00.432 INFO:teuthology.orchestra.run.smithi173.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.9 MB/s | 427 kB 00:00 2024-04-20T10:53:00.441 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:00.442 INFO:teuthology.orchestra.run.smithi173.stdout:Total 9.1 MB/s | 30 MB 00:03 2024-04-20T10:53:00.782 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:00.870 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:00.870 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:01.375 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:01.376 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:02.721 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:02.786 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-20T10:53:02.822 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-20T10:53:02.865 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-20T10:53:02.998 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-20T10:53:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-20T10:53:03.176 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-20T10:53:03.217 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-20T10:53:03.254 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-20T10:53:03.288 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-20T10:53:03.325 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-20T10:53:03.376 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-20T10:53:03.436 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-20T10:53:03.477 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-20T10:53:03.543 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-20T10:53:03.674 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-20T10:53:03.871 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-20T10:53:03.966 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-20T10:53:04.036 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-20T10:53:04.088 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-20T10:53:04.152 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-20T10:53:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-20T10:53:04.307 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:18.2.2-0.el9.x86_64 3/4 2024-04-20T10:53:04.472 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-20T10:53:04.533 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-20T10:53:04.635 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:18.2.2-0.el9.x86_64 4/4 2024-04-20T10:53:04.652 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-20T10:53:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-20T10:53:04.814 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-20T10:53:04.968 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-20T10:53:05.147 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-0.el9.noarch 28/33 2024-04-20T10:53:05.180 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:18.2.2-0.el9.x86_64 29/33 2024-04-20T10:53:05.257 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:18.2.2-0.el9.x86_64 29/33 2024-04-20T10:53:05.257 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:05.257 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:53:05.257 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:53:05.258 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:53:05.258 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:06.262 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-test-2:18.2.2-0.el9.x86_64 4/4 2024-04-20T10:53:06.262 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:18.2.2-0.el9.x86_64 1/4 2024-04-20T10:53:06.262 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:18.2.2-0.el9.x86_64 2/4 2024-04-20T10:53:06.262 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-20T10:53:06.557 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-20T10:53:06.557 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:06.558 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:53:06.558 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:18.2.2-0.el9.x86_64 libcephsqlite-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:06.558 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:53:06.558 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:06.558 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:06.859 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph 2024-04-20T10:53:06.906 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:18.2.2-0.el9.x86_64 30/33 2024-04-20T10:53:06.983 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:18.2.2-0.el9.x86_64 30/33 2024-04-20T10:53:06.983 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:06.983 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:53:06.984 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:53:06.984 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:53:06.984 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:07.481 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:18.2.2-0.el9.x86_64 31/33 2024-04-20T10:53:07.553 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:18.2.2-0.el9.x86_64 31/33 2024-04-20T10:53:07.554 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:07.554 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:53:07.554 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:53:07.554 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:53:07.554 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:07.615 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:42 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:07.747 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:07.749 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:07.749 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-04-20T10:53:07.750 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:07.750 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:53:07.750 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:18.2.2-0.el9 ceph 6.4 k 2024-04-20T10:53:07.750 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-20T10:53:07.750 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:18.2.2-0.el9 ceph 2.1 M 2024-04-20T10:53:07.750 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:18.2.2-0.el9 ceph 1.4 M 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:18.2.2-0.el9 ceph-noarch 242 k 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:18.2.2-0.el9 ceph 4.5 M 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:18.2.2-0.el9 ceph 18 M 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:53:07.751 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:53:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout:Install 33 Packages 2024-04-20T10:53:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:07.754 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 30 M 2024-04-20T10:53:07.754 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 110 M 2024-04-20T10:53:07.754 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:53:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:18.2.2-0.el9.x86_64 32/33 2024-04-20T10:53:07.849 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:18.2.2-0.el9.x86_64 32/33 2024-04-20T10:53:07.849 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:07.849 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:53:07.849 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:53:07.849 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:53:07.849 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:07.923 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:18.2.2-0.el9.x86_64 33/33 2024-04-20T10:53:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-2:18.2.2-0.el9.x86_64 33/33 2024-04-20T10:53:08.061 INFO:teuthology.orchestra.run.smithi173.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T10:53:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:08.602 INFO:teuthology.orchestra.run.smithi117.stdout:(1/33): ceph-18.2.2-0.el9.x86_64.rpm 33 kB/s | 6.4 kB 00:00 2024-04-20T10:53:08.715 INFO:teuthology.orchestra.run.smithi117.stdout:(2/33): ceph-mgr-18.2.2-0.el9.x86_64.rpm 4.7 MB/s | 1.4 MB 00:00 2024-04-20T10:53:08.777 INFO:teuthology.orchestra.run.smithi117.stdout:(3/33): ceph-mds-18.2.2-0.el9.x86_64.rpm 5.7 MB/s | 2.1 MB 00:00 2024-04-20T10:53:08.811 INFO:teuthology.orchestra.run.smithi117.stdout:(4/33): ceph-mgr-modules-core-18.2.2-0.el9.noar 7.3 MB/s | 242 kB 00:00 2024-04-20T10:53:08.928 INFO:teuthology.orchestra.run.smithi117.stdout:(5/33): ceph-mon-18.2.2-0.el9.x86_64.rpm 14 MB/s | 4.5 MB 00:00 2024-04-20T10:53:09.228 INFO:teuthology.orchestra.run.smithi117.stdout:(6/33): ceph-osd-18.2.2-0.el9.x86_64.rpm 34 MB/s | 18 MB 00:00 2024-04-20T10:53:09.661 INFO:teuthology.orchestra.run.smithi117.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 246 kB/s | 106 kB 00:00 2024-04-20T10:53:09.687 INFO:teuthology.orchestra.run.smithi117.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 289 kB/s | 253 kB 00:00 2024-04-20T10:53:09.838 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:18.2.2-0.el9.x86_64 1/33 2024-04-20T10:53:09.838 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:18.2.2-0.el9.x86_64 2/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:18.2.2-0.el9.x86_64 3/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:18.2.2-0.el9.x86_64 4/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:18.2.2-0.el9.x86_64 5/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-0.el9.noarch 6/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-20T10:53:09.839 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-20T10:53:09.840 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-20T10:53:09.841 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-20T10:53:09.842 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-20T10:53:09.842 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-20T10:53:09.842 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-20T10:53:09.842 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-20T10:53:09.842 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-20T10:53:10.095 INFO:teuthology.orchestra.run.smithi117.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 312 kB/s | 135 kB 00:00 2024-04-20T10:53:10.120 INFO:teuthology.orchestra.run.smithi117.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 288 kB/s | 125 kB 00:00 2024-04-20T10:53:10.296 INFO:teuthology.orchestra.run.smithi117.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 988 kB/s | 172 kB 00:00 2024-04-20T10:53:10.354 INFO:teuthology.orchestra.run.smithi117.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 598 kB/s | 35 kB 00:00 2024-04-20T10:53:10.413 INFO:teuthology.orchestra.run.smithi117.stdout:(13/33): python3-pytz-2021.1-5.el9.noarch.rpm 878 kB/s | 51 kB 00:00 2024-04-20T10:53:10.463 INFO:teuthology.orchestra.run.smithi117.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-04-20T10:53:10.836 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-20T10:53:10.836 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:10.836 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:10.836 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:10.836 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:10.836 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:10.836 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:18.2.2-0.el9.noarch 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:53:10.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:53:10.838 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:10.839 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:10.840 INFO:teuthology.orchestra.run.smithi117.stdout:(15/33): python3-urllib3-1.26.5-5.el9.noarch.rp 289 kB/s | 215 kB 00:00 2024-04-20T10:53:10.872 INFO:teuthology.orchestra.run.smithi117.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 106 kB/s | 43 kB 00:00 2024-04-20T10:53:11.106 INFO:teuthology.orchestra.run.smithi117.stdout:(17/33): python3-cryptography-36.0.1-4.el9.x86_ 577 kB/s | 1.2 MB 00:02 2024-04-20T10:53:11.236 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-base 2024-04-20T10:53:11.281 INFO:teuthology.orchestra.run.smithi117.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 878 kB/s | 358 kB 00:00 2024-04-20T10:53:11.373 INFO:teuthology.orchestra.run.smithi117.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 194 kB/s | 18 kB 00:00 2024-04-20T10:53:11.440 INFO:teuthology.orchestra.run.smithi117.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 32 kB/s | 11 kB 00:00 2024-04-20T10:53:11.457 INFO:teuthology.orchestra.run.smithi117.stdout:(21/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 278 kB/s | 172 kB 00:00 2024-04-20T10:53:11.482 INFO:teuthology.orchestra.run.smithi117.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 214 kB/s | 23 kB 00:00 2024-04-20T10:53:11.557 INFO:teuthology.orchestra.run.smithi117.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 198 kB/s | 20 kB 00:00 2024-04-20T10:53:11.583 INFO:teuthology.orchestra.run.smithi117.stdout:(24/33): python3-jaraco-functools-3.5.0-2.el9.n 137 kB/s | 19 kB 00:00 2024-04-20T10:53:11.608 INFO:teuthology.orchestra.run.smithi117.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 371 kB/s | 46 kB 00:00 2024-04-20T10:53:11.683 INFO:teuthology.orchestra.run.smithi117.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 631 kB/s | 79 kB 00:00 2024-04-20T10:53:11.708 INFO:teuthology.orchestra.run.smithi117.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 164 kB/s | 16 kB 00:00 2024-04-20T10:53:11.809 INFO:teuthology.orchestra.run.smithi117.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 724 kB/s | 90 kB 00:00 2024-04-20T10:53:11.834 INFO:teuthology.orchestra.run.smithi117.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 287 kB/s | 36 kB 00:00 2024-04-20T10:53:11.943 INFO:teuthology.orchestra.run.smithi117.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-20T10:53:11.985 INFO:teuthology.orchestra.run.smithi117.stdout:(31/33): python3-pecan-1.4.2-3.el9.noarch.rpm 679 kB/s | 272 kB 00:00 2024-04-20T10:53:11.987 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:56 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:12.035 INFO:teuthology.orchestra.run.smithi117.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 220 kB/s | 20 kB 00:00 2024-04-20T10:53:12.067 INFO:teuthology.orchestra.run.smithi173.stdout:Package ceph-base-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:53:12.093 INFO:teuthology.orchestra.run.smithi117.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.6 MB/s | 427 kB 00:00 2024-04-20T10:53:12.105 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:12.105 INFO:teuthology.orchestra.run.smithi117.stdout:Total 6.9 MB/s | 30 MB 00:04 2024-04-20T10:53:12.109 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:12.110 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:53:12.110 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:12.198 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install cephadm 2024-04-20T10:53:12.469 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:53:12.555 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:53:12.556 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:53:12.920 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:57 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:13.044 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:18.2.2-0.el9 ceph-noarch 221 k 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:13.045 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 221 k 2024-04-20T10:53:13.046 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 213 k 2024-04-20T10:53:13.046 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:13.076 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:53:13.076 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:53:13.148 INFO:teuthology.orchestra.run.smithi173.stdout:cephadm-18.2.2-0.el9.noarch.rpm 2.1 MB/s | 221 kB 00:00 2024-04-20T10:53:13.148 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:13.149 INFO:teuthology.orchestra.run.smithi173.stdout:Total 2.1 MB/s | 221 kB 00:00 2024-04-20T10:53:13.149 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:13.156 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:13.156 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:13.160 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:13.160 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:13.256 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:13.833 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:18.2.2-0.el9.noarch 1/1 2024-04-20T10:53:13.961 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:18.2.2-0.el9.noarch 1/1 2024-04-20T10:53:14.365 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:53:14.422 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-20T10:53:14.432 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:18.2.2-0.el9.noarch 1/1 2024-04-20T10:53:14.450 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-20T10:53:14.492 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-20T10:53:14.626 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-20T10:53:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:18.2.2-0.el9.noarch 1/1 2024-04-20T10:53:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:14.684 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:18.2.2-0.el9.noarch 2024-04-20T10:53:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:14.684 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:14.722 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-20T10:53:14.778 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-20T10:53:14.820 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-20T10:53:14.857 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-20T10:53:14.894 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-20T10:53:14.936 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-20T10:53:14.951 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-immutable-object-cache 2024-04-20T10:53:14.995 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-20T10:53:15.069 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-20T10:53:15.115 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-20T10:53:15.181 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-20T10:53:15.312 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-20T10:53:15.506 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-20T10:53:15.605 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-20T10:53:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-20T10:53:15.732 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-20T10:53:15.744 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:00 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:15.795 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-20T10:53:15.868 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-20T10:53:15.872 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:15.872 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:15.872 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-04-20T10:53:15.872 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:15.872 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:15.872 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-0.el9 ceph 142 k 2024-04-20T10:53:15.872 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:15.873 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:15.873 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:15.873 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-04-20T10:53:15.873 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:15.873 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 142 k 2024-04-20T10:53:15.873 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 395 k 2024-04-20T10:53:15.873 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:15.970 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-immutable-object-cache-18.2.2-0.el9.x86_64 1.5 MB/s | 142 kB 00:00 2024-04-20T10:53:15.971 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:15.971 INFO:teuthology.orchestra.run.smithi173.stdout:Total 1.4 MB/s | 142 kB 00:00 2024-04-20T10:53:15.971 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:15.982 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:15.982 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:16.055 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:16.056 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:16.147 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-20T10:53:16.211 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-20T10:53:16.227 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:16.279 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:16.302 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-20T10:53:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:16.349 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:16.349 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:53:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:16.409 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-20T10:53:16.458 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-20T10:53:16.610 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-20T10:53:16.783 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-0.el9.noarch 28/33 2024-04-20T10:53:16.832 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:18.2.2-0.el9.x86_64 29/33 2024-04-20T10:53:16.902 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:18.2.2-0.el9.x86_64 29/33 2024-04-20T10:53:16.902 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:16.902 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:53:16.903 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:53:16.903 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:53:16.903 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:17.508 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:17.508 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:17.508 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:17.508 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:17.508 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:17.508 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:17.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr 2024-04-20T10:53:18.541 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:18.2.2-0.el9.x86_64 30/33 2024-04-20T10:53:18.572 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:03 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:18.617 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:18.2.2-0.el9.x86_64 30/33 2024-04-20T10:53:18.617 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:18.617 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:53:18.617 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:53:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:53:18.618 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:18.652 INFO:teuthology.orchestra.run.smithi173.stdout:Package ceph-mgr-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:53:18.695 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:18.696 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:53:18.696 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:18.782 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr-dashboard 2024-04-20T10:53:19.125 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:18.2.2-0.el9.x86_64 31/33 2024-04-20T10:53:19.192 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:18.2.2-0.el9.x86_64 31/33 2024-04-20T10:53:19.192 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:19.192 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:53:19.192 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:53:19.192 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:53:19.192 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:19.425 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:18.2.2-0.el9.x86_64 32/33 2024-04-20T10:53:19.493 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:18.2.2-0.el9.x86_64 32/33 2024-04-20T10:53:19.493 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:19.493 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:53:19.493 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:53:19.493 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:53:19.493 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:19.545 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:04 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:19.584 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:18.2.2-0.el9.x86_64 33/33 2024-04-20T10:53:19.673 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:18.2.2-0.el9 ceph-noarch 1.8 M 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:18.2.2-0.el9 ceph-noarch 23 k 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:18.2.2-0.el9 ceph-noarch 15 k 2024-04-20T10:53:19.674 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout:Installing weak dependencies: 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout:Install 7 Packages 2024-04-20T10:53:19.675 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:19.676 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 2.1 M 2024-04-20T10:53:19.676 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 8.5 M 2024-04-20T10:53:19.676 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:19.706 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-2:18.2.2-0.el9.x86_64 33/33 2024-04-20T10:53:19.706 INFO:teuthology.orchestra.run.smithi117.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T10:53:19.706 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:19.990 INFO:teuthology.orchestra.run.smithi173.stdout:(1/7): ceph-mgr-dashboard-18.2.2-0.el9.noarch.r 7.9 MB/s | 1.8 MB 00:00 2024-04-20T10:53:20.015 INFO:teuthology.orchestra.run.smithi173.stdout:(2/7): ceph-prometheus-alerts-18.2.2-0.el9.noar 59 kB/s | 15 kB 00:00 2024-04-20T10:53:20.041 INFO:teuthology.orchestra.run.smithi173.stdout:(3/7): ceph-grafana-dashboards-18.2.2-0.el9.noa 85 kB/s | 23 kB 00:00 2024-04-20T10:53:20.108 INFO:teuthology.orchestra.run.smithi173.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 78 kB/s | 9.0 kB 00:00 2024-04-20T10:53:20.166 INFO:teuthology.orchestra.run.smithi173.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 272 kB/s | 41 kB 00:00 2024-04-20T10:53:20.191 INFO:teuthology.orchestra.run.smithi173.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 205 kB/s | 31 kB 00:00 2024-04-20T10:53:20.300 INFO:teuthology.orchestra.run.smithi173.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 986 kB/s | 188 kB 00:00 2024-04-20T10:53:20.304 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:20.305 INFO:teuthology.orchestra.run.smithi173.stdout:Total 3.3 MB/s | 2.1 MB 00:00 2024-04-20T10:53:20.306 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:20.334 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:20.334 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:20.487 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:20.487 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:20.882 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:20.951 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-20T10:53:21.004 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-20T10:53:21.090 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-20T10:53:21.133 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-20T10:53:21.188 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-0.el9.noarch 5/7 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:18.2.2-0.el9.x86_64 1/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:18.2.2-0.el9.x86_64 2/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:18.2.2-0.el9.x86_64 3/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:18.2.2-0.el9.x86_64 4/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:18.2.2-0.el9.x86_64 5/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-0.el9.noarch 6/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-20T10:53:21.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-20T10:53:21.513 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-20T10:53:21.514 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-20T10:53:21.514 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-20T10:53:21.514 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-20T10:53:21.514 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-20T10:53:21.514 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-20T10:53:21.514 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-20T10:53:21.546 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-0.el9.noarch 6/7 2024-04-20T10:53:21.595 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-0.el9.noarch 7/7 2024-04-20T10:53:22.202 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-0.el9.noarch 7/7 2024-04-20T10:53:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-0.el9.noarch 1/7 2024-04-20T10:53:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-0.el9.noarch 2/7 2024-04-20T10:53:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-0.el9.noarch 3/7 2024-04-20T10:53:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-20T10:53:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-20T10:53:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-20T10:53:22.458 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:18.2.2-0.el9.noarch 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:53:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:53:22.460 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:22.461 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:18.2.2-0.el9.noarch 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:18.2.2-0.el9.noarch 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:18.2.2-0.el9.noarch 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-20T10:53:22.588 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:53:22.589 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:53:22.589 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:22.589 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:22.870 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-base 2024-04-20T10:53:22.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-20T10:53:23.617 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:58 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:23.669 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:08 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:23.696 INFO:teuthology.orchestra.run.smithi117.stdout:Package ceph-base-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:53:23.738 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:23.739 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:53:23.739 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:23.808 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:23.809 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:23.809 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-0.el9 ceph-noarch 7.4 M 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:53:23.810 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout:Install 12 Packages 2024-04-20T10:53:23.811 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:23.812 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 38 M 2024-04-20T10:53:23.812 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 209 M 2024-04-20T10:53:23.812 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:23.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install cephadm 2024-04-20T10:53:24.465 INFO:teuthology.orchestra.run.smithi173.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 888 kB/s | 195 kB 00:00 2024-04-20T10:53:24.548 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:59 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:24.582 INFO:teuthology.orchestra.run.smithi173.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-04-20T10:53:24.666 INFO:teuthology.orchestra.run.smithi173.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 164 kB/s | 33 kB 00:00 2024-04-20T10:53:24.672 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:24.673 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:24.673 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-04-20T10:53:24.673 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:24.674 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:53:24.674 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:18.2.2-0.el9 ceph-noarch 221 k 2024-04-20T10:53:24.674 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:24.674 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:53:24.674 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:24.674 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-04-20T10:53:24.674 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:24.674 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 221 k 2024-04-20T10:53:24.675 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 213 k 2024-04-20T10:53:24.675 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:53:24.750 INFO:teuthology.orchestra.run.smithi117.stdout:cephadm-18.2.2-0.el9.noarch.rpm 2.8 MB/s | 221 kB 00:00 2024-04-20T10:53:24.751 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:24.751 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.8 MB/s | 221 kB 00:00 2024-04-20T10:53:24.751 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:53:24.756 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:53:24.756 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:53:24.760 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:53:24.761 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:53:24.846 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:53:24.849 INFO:teuthology.orchestra.run.smithi173.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.2-0. 12 MB/s | 7.4 MB 00:00 2024-04-20T10:53:24.874 INFO:teuthology.orchestra.run.smithi173.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 88 kB/s | 18 kB 00:00 2024-04-20T10:53:25.033 INFO:teuthology.orchestra.run.smithi173.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 192 kB/s | 35 kB 00:00 2024-04-20T10:53:25.100 INFO:teuthology.orchestra.run.smithi173.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.8 MB/s | 3.0 MB 00:00 2024-04-20T10:53:25.275 INFO:teuthology.orchestra.run.smithi173.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-04-20T10:53:25.342 INFO:teuthology.orchestra.run.smithi173.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2024-04-20T10:53:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:18.2.2-0.el9.noarch 1/1 2024-04-20T10:53:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:18.2.2-0.el9.noarch 1/1 2024-04-20T10:53:25.567 INFO:teuthology.orchestra.run.smithi173.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-20T10:53:25.592 INFO:teuthology.orchestra.run.smithi173.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 792 kB/s | 250 kB 00:00 2024-04-20T10:53:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:18.2.2-0.el9.noarch 1/1 2024-04-20T10:53:26.042 INFO:teuthology.orchestra.run.smithi173.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-20T10:53:26.045 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:26.046 INFO:teuthology.orchestra.run.smithi173.stdout:Total 17 MB/s | 38 MB 00:02 2024-04-20T10:53:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:18.2.2-0.el9.noarch 1/1 2024-04-20T10:53:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:26.282 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:53:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:18.2.2-0.el9.noarch 2024-04-20T10:53:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:26.282 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:26.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-immutable-object-cache 2024-04-20T10:53:26.677 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:26.754 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:26.754 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:27.317 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:02 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:27.357 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:27.358 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:27.445 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:27.445 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:27.445 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-04-20T10:53:27.445 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:27.445 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:53:27.445 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-0.el9 ceph 142 k 2024-04-20T10:53:27.445 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:27.445 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:53:27.446 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:27.446 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-04-20T10:53:27.446 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:27.446 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 142 k 2024-04-20T10:53:27.446 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 395 k 2024-04-20T10:53:27.447 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:53:27.512 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-immutable-object-cache-18.2.2-0.el9.x86_64 2.1 MB/s | 142 kB 00:00 2024-04-20T10:53:27.513 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:27.514 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.1 MB/s | 142 kB 00:00 2024-04-20T10:53:27.514 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:53:27.523 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:53:27.523 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:53:27.589 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:53:27.589 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:53:27.796 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:53:27.847 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:27.924 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:27.924 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:53:27.924 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:53:27.924 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:28.283 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:28.362 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-20T10:53:28.423 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-20T10:53:28.569 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-20T10:53:28.610 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-20T10:53:28.864 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-20T10:53:28.919 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-20T10:53:29.052 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-20T10:53:29.113 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:29.114 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:29.114 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:53:29.114 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:29.114 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:29.114 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:29.394 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr 2024-04-20T10:53:29.948 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-20T10:53:30.103 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-20T10:53:30.168 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:05 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:30.247 INFO:teuthology.orchestra.run.smithi117.stdout:Package ceph-mgr-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:53:30.289 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:30.290 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:53:30.290 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:30.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-dashboard 2024-04-20T10:53:31.134 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:06 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:31.261 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:31.266 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:31.266 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-04-20T10:53:31.266 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:31.266 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:18.2.2-0.el9 ceph-noarch 1.8 M 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:18.2.2-0.el9 ceph-noarch 23 k 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:18.2.2-0.el9 ceph-noarch 15 k 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:31.267 INFO:teuthology.orchestra.run.smithi117.stdout:Install 7 Packages 2024-04-20T10:53:31.268 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:31.269 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.1 M 2024-04-20T10:53:31.269 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 8.5 M 2024-04-20T10:53:31.269 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:53:31.499 INFO:teuthology.orchestra.run.smithi117.stdout:(1/7): ceph-grafana-dashboards-18.2.2-0.el9.noa 162 kB/s | 23 kB 00:00 2024-04-20T10:53:31.525 INFO:teuthology.orchestra.run.smithi117.stdout:(2/7): ceph-prometheus-alerts-18.2.2-0.el9.noar 87 kB/s | 15 kB 00:00 2024-04-20T10:53:31.583 INFO:teuthology.orchestra.run.smithi117.stdout:(3/7): ceph-mgr-dashboard-18.2.2-0.el9.noarch.r 7.8 MB/s | 1.8 MB 00:00 2024-04-20T10:53:31.669 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-20T10:53:31.775 INFO:teuthology.orchestra.run.smithi117.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 33 kB/s | 9.0 kB 00:00 2024-04-20T10:53:31.842 INFO:teuthology.orchestra.run.smithi117.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 129 kB/s | 41 kB 00:00 2024-04-20T10:53:31.884 INFO:teuthology.orchestra.run.smithi117.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 144 kB/s | 31 kB 00:00 2024-04-20T10:53:31.950 INFO:teuthology.orchestra.run.smithi117.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.1 MB/s | 188 kB 00:00 2024-04-20T10:53:31.961 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:31.962 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.9 MB/s | 2.1 MB 00:00 2024-04-20T10:53:31.962 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:53:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:53:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:53:32.127 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:53:32.127 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:53:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:53:32.610 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-20T10:53:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-20T10:53:32.742 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-20T10:53:32.784 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-20T10:53:32.826 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-0.el9.noarch 5/7 2024-04-20T10:53:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-20T10:53:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-0.el9.noarc 12/12 2024-04-20T10:53:33.139 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-0.el9.noarch 6/7 2024-04-20T10:53:33.171 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-0.el9.noarch 7/7 2024-04-20T10:53:33.717 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-0.el9.noarch 7/7 2024-04-20T10:53:33.717 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-0.el9.noarch 1/7 2024-04-20T10:53:33.717 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-0.el9.noarch 2/7 2024-04-20T10:53:33.717 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-0.el9.noarch 3/7 2024-04-20T10:53:33.718 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-20T10:53:33.718 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-20T10:53:33.718 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-20T10:53:34.072 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-20T10:53:34.072 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:18.2.2-0.el9.noarch 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:18.2.2-0.el9.noarch 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:18.2.2-0.el9.noarch 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:34.073 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:34.327 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-20T10:53:35.084 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:10 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:35.214 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-0.el9 ceph-noarch 7.4 M 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:53:35.215 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout:Install 12 Packages 2024-04-20T10:53:35.216 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:35.217 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 38 M 2024-04-20T10:53:35.217 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 209 M 2024-04-20T10:53:35.217 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-0.el9.noarc 12/12 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-0.el9.noarc 1/12 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-20T10:53:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-20T10:53:35.500 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-20T10:53:35.500 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-20T10:53:35.500 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-20T10:53:35.707 INFO:teuthology.orchestra.run.smithi117.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.2-0. 28 MB/s | 7.4 MB 00:00 2024-04-20T10:53:35.758 INFO:teuthology.orchestra.run.smithi117.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 627 kB/s | 195 kB 00:00 2024-04-20T10:53:35.882 INFO:teuthology.orchestra.run.smithi117.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-20T10:53:35.900 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:18.2.2-0.el9.noarch 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:53:35.901 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:53:35.902 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:53:35.902 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:53:35.902 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:53:35.902 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:35.902 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:36.041 INFO:teuthology.orchestra.run.smithi117.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 115 kB/s | 18 kB 00:00 2024-04-20T10:53:36.066 INFO:teuthology.orchestra.run.smithi117.stdout:(5/12): flexiblas-3.0.4-8.el9.x86_64.rpm 91 kB/s | 33 kB 00:00 2024-04-20T10:53:36.117 INFO:teuthology.orchestra.run.smithi117.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 472 kB/s | 35 kB 00:00 2024-04-20T10:53:36.200 INFO:teuthology.orchestra.run.smithi117.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 3.0 MB/s | 244 kB 00:00 2024-04-20T10:53:36.300 INFO:teuthology.orchestra.run.smithi117.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.5 MB/s | 3.0 MB 00:00 2024-04-20T10:53:36.324 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr-rook 2024-04-20T10:53:36.442 INFO:teuthology.orchestra.run.smithi117.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2024-04-20T10:53:36.468 INFO:teuthology.orchestra.run.smithi117.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.5 MB/s | 250 kB 00:00 2024-04-20T10:53:36.618 INFO:teuthology.orchestra.run.smithi117.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-20T10:53:37.099 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:22 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:37.223 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:37.224 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-20T10:53:37.224 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:37.224 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:18.2.2-0.el9 ceph-noarch 50 k 2024-04-20T10:53:37.224 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:37.225 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:37.226 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:37.226 INFO:teuthology.orchestra.run.smithi173.stdout:Install 13 Packages 2024-04-20T10:53:37.226 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:37.226 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 2.2 M 2024-04-20T10:53:37.226 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 26 M 2024-04-20T10:53:37.226 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:37.351 INFO:teuthology.orchestra.run.smithi117.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-20T10:53:37.353 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:37.353 INFO:teuthology.orchestra.run.smithi117.stdout:Total 18 MB/s | 38 MB 00:02 2024-04-20T10:53:37.650 INFO:teuthology.orchestra.run.smithi173.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 213 kB/s | 26 kB 00:00 2024-04-20T10:53:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 132 kB/s | 19 kB 00:00 2024-04-20T10:53:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:(3/13): ceph-mgr-rook-18.2.2-0.el9.noarch.rpm 292 kB/s | 50 kB 00:00 2024-04-20T10:53:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-20T10:53:37.793 INFO:teuthology.orchestra.run.smithi173.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-04-20T10:53:37.835 INFO:teuthology.orchestra.run.smithi173.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.0 MB/s | 279 kB 00:00 2024-04-20T10:53:37.860 INFO:teuthology.orchestra.run.smithi173.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 588 kB/s | 54 kB 00:00 2024-04-20T10:53:37.988 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:53:38.066 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:53:38.066 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:53:38.294 INFO:teuthology.orchestra.run.smithi173.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 64 kB/s | 32 kB 00:00 2024-04-20T10:53:38.335 INFO:teuthology.orchestra.run.smithi173.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 30 kB/s | 15 kB 00:00 2024-04-20T10:53:38.594 INFO:teuthology.orchestra.run.smithi173.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 229 kB/s | 59 kB 00:00 2024-04-20T10:53:38.677 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:53:38.678 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:53:38.703 INFO:teuthology.orchestra.run.smithi173.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 241 kB/s | 203 kB 00:00 2024-04-20T10:53:38.778 INFO:teuthology.orchestra.run.smithi173.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 491 kB/s | 90 kB 00:00 2024-04-20T10:53:39.581 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:53:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-20T10:53:39.678 INFO:teuthology.orchestra.run.smithi173.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 756 kB/s | 1.0 MB 00:01 2024-04-20T10:53:39.686 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:39.687 INFO:teuthology.orchestra.run.smithi173.stdout:Total 915 kB/s | 2.2 MB 00:02 2024-04-20T10:53:39.701 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-20T10:53:39.837 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-20T10:53:39.885 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-20T10:53:39.900 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:39.947 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:39.947 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:40.088 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:40.090 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:40.138 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-20T10:53:40.175 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-20T10:53:40.303 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-20T10:53:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:41.016 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-20T10:53:41.088 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-20T10:53:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-20T10:53:41.186 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-20T10:53:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-20T10:53:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-20T10:53:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-20T10:53:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-20T10:53:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-20T10:53:41.630 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-20T10:53:42.572 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-20T10:53:42.631 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-20T10:53:42.664 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-20T10:53:42.723 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-20T10:53:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:18.2.2-0.el9.noarch 13/13 2024-04-20T10:53:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-20T10:53:43.663 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-0.el9.noarch 13/13 2024-04-20T10:53:43.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:18.2.2-0.el9.noarch 1/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-20T10:53:43.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-20T10:53:44.042 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-20T10:53:44.072 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-0.el9.noarc 12/12 2024-04-20T10:53:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:18.2.2-0.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:53:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:53:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:53:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:44.114 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:44.455 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr-cephadm 2024-04-20T10:53:45.216 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:30 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:45.342 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:45.343 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:45.343 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-20T10:53:45.343 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:18.2.2-0.el9 ceph-noarch 132 k 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout:Install 6 Packages 2024-04-20T10:53:45.344 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:45.345 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 7.0 M 2024-04-20T10:53:45.345 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 33 M 2024-04-20T10:53:45.345 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:45.714 INFO:teuthology.orchestra.run.smithi173.stdout:(1/6): ceph-mgr-cephadm-18.2.2-0.el9.noarch.rpm 1.0 MB/s | 132 kB 00:00 2024-04-20T10:53:45.847 INFO:teuthology.orchestra.run.smithi173.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 946 kB/s | 248 kB 00:00 2024-04-20T10:53:45.998 INFO:teuthology.orchestra.run.smithi173.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.9 MB/s | 548 kB 00:00 2024-04-20T10:53:46.031 INFO:teuthology.orchestra.run.smithi173.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 316 kB/s | 58 kB 00:00 2024-04-20T10:53:46.056 INFO:teuthology.orchestra.run.smithi173.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 882 kB/s | 51 kB 00:00 2024-04-20T10:53:46.649 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-0.el9.noarc 12/12 2024-04-20T10:53:46.649 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-0.el9.noarc 1/12 2024-04-20T10:53:46.649 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-20T10:53:46.649 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-20T10:53:46.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-20T10:53:46.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-20T10:53:46.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-20T10:53:46.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-20T10:53:46.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-20T10:53:46.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-20T10:53:46.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-20T10:53:46.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-20T10:53:46.673 INFO:teuthology.orchestra.run.smithi173.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-04-20T10:53:46.673 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:46.674 INFO:teuthology.orchestra.run.smithi173.stdout:Total 5.2 MB/s | 7.0 MB 00:01 2024-04-20T10:53:46.806 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:46.845 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:46.846 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:46.998 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:46.999 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:18.2.2-0.el9.noarch 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:53:47.125 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:53:47.126 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:53:47.126 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:53:47.126 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:53:47.126 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:53:47.126 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:47.126 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:47.566 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-rook 2024-04-20T10:53:47.574 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-20T10:53:47.631 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-20T10:53:48.286 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-20T10:53:48.322 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:23 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:48.395 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-20T10:53:48.446 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:18.2.2-0.el9 ceph-noarch 50 k 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:53:48.448 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout:Install 13 Packages 2024-04-20T10:53:48.449 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:48.450 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.2 M 2024-04-20T10:53:48.450 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 26 M 2024-04-20T10:53:48.450 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:53:48.470 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-20T10:53:48.495 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-0.el9.noarch 6/6 2024-04-20T10:53:49.040 INFO:teuthology.orchestra.run.smithi117.stdout:(1/13): ceph-mgr-rook-18.2.2-0.el9.noarch.rpm 503 kB/s | 50 kB 00:00 2024-04-20T10:53:49.132 INFO:teuthology.orchestra.run.smithi117.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 134 kB/s | 26 kB 00:00 2024-04-20T10:53:49.157 INFO:teuthology.orchestra.run.smithi117.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 89 kB/s | 19 kB 00:00 2024-04-20T10:53:49.257 INFO:teuthology.orchestra.run.smithi117.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.0 MB/s | 222 kB 00:00 2024-04-20T10:53:49.282 INFO:teuthology.orchestra.run.smithi117.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-20T10:53:49.312 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-0.el9.noarch 6/6 2024-04-20T10:53:49.312 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-0.el9.noarch 1/6 2024-04-20T10:53:49.312 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-20T10:53:49.312 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-20T10:53:49.313 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-20T10:53:49.313 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-20T10:53:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.7 MB/s | 279 kB 00:00 2024-04-20T10:53:49.342 INFO:teuthology.orchestra.run.smithi117.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 645 kB/s | 54 kB 00:00 2024-04-20T10:53:49.592 INFO:teuthology.orchestra.run.smithi117.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 104 kB/s | 32 kB 00:00 2024-04-20T10:53:49.625 INFO:teuthology.orchestra.run.smithi117.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 48 kB/s | 15 kB 00:00 2024-04-20T10:53:49.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-20T10:53:49.675 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:18.2.2-0.el9.noarch 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:49.676 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:49.842 INFO:teuthology.orchestra.run.smithi117.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 278 kB/s | 59 kB 00:00 2024-04-20T10:53:49.884 INFO:teuthology.orchestra.run.smithi117.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 476 kB/s | 203 kB 00:00 2024-04-20T10:53:49.934 INFO:teuthology.orchestra.run.smithi117.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 3.0 MB/s | 1.0 MB 00:00 2024-04-20T10:53:49.948 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-fuse 2024-04-20T10:53:49.959 INFO:teuthology.orchestra.run.smithi117.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 803 kB/s | 90 kB 00:00 2024-04-20T10:53:49.966 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:49.966 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.5 MB/s | 2.2 MB 00:01 2024-04-20T10:53:50.175 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:53:50.221 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:53:50.221 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:53:50.366 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:53:50.368 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:53:50.707 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:35 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:50.830 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:50.832 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-20T10:53:50.832 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:50.832 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:18.2.2-0.el9 ceph 840 k 2024-04-20T10:53:50.832 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-20T10:53:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:53:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:50.833 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:50.833 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:50.833 INFO:teuthology.orchestra.run.smithi173.stdout:Install 2 Packages 2024-04-20T10:53:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:50.834 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 920 k 2024-04-20T10:53:50.834 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 2.7 M 2024-04-20T10:53:50.834 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:51.088 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:53:51.283 INFO:teuthology.orchestra.run.smithi173.stdout:(1/2): ceph-fuse-18.2.2-0.el9.x86_64.rpm 4.9 MB/s | 840 kB 00:00 2024-04-20T10:53:51.297 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-20T10:53:51.317 INFO:teuthology.orchestra.run.smithi173.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 399 kB/s | 80 kB 00:00 2024-04-20T10:53:51.318 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:51.319 INFO:teuthology.orchestra.run.smithi173.stdout:Total 1.8 MB/s | 920 kB 00:00 2024-04-20T10:53:51.359 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:51.395 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-20T10:53:51.432 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:51.432 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:51.470 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-20T10:53:51.507 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-20T10:53:51.538 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:51.539 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:51.562 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-20T10:53:51.696 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-20T10:53:51.799 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:51.835 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-20T10:53:51.904 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-20T10:53:51.948 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-20T10:53:52.036 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:18.2.2-0.el9.x86_64 2/2 2024-04-20T10:53:52.822 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-20T10:53:52.879 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-20T10:53:52.913 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-20T10:53:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-fuse-2:18.2.2-0.el9.x86_64 2/2 2024-04-20T10:53:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:18.2.2-0.el9.x86_64 1/2 2024-04-20T10:53:52.968 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-20T10:53:52.990 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:18.2.2-0.el9.noarch 13/13 2024-04-20T10:53:53.220 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-20T10:53:53.220 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:53.220 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:53.220 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:18.2.2-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-20T10:53:53.220 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:53.220 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:53.501 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librados-devel 2024-04-20T10:53:53.819 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-0.el9.noarch 13/13 2024-04-20T10:53:53.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:18.2.2-0.el9.noarch 1/13 2024-04-20T10:53:53.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-20T10:53:53.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-20T10:53:53.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-20T10:53:53.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-20T10:53:53.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-20T10:53:53.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-20T10:53:53.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-20T10:53:53.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-20T10:53:53.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-20T10:53:53.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-20T10:53:53.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:18.2.2-0.el9.noarch 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:53:54.229 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:54.230 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:54.280 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:39 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:54.410 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:54.411 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:54.411 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-20T10:53:54.411 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:54.411 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:54.411 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:18.2.2-0.el9 ceph 126 k 2024-04-20T10:53:54.411 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:54.412 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:54.412 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:54.412 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-04-20T10:53:54.412 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:54.412 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 126 k 2024-04-20T10:53:54.412 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 456 k 2024-04-20T10:53:54.412 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:54.496 INFO:teuthology.orchestra.run.smithi173.stdout:librados-devel-18.2.2-0.el9.x86_64.rpm 1.5 MB/s | 126 kB 00:00 2024-04-20T10:53:54.497 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:54.497 INFO:teuthology.orchestra.run.smithi173.stdout:Total 1.4 MB/s | 126 kB 00:00 2024-04-20T10:53:54.497 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:54.504 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:54.504 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:54.546 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-cephadm 2024-04-20T10:53:54.578 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:54.578 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:54.755 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:54.864 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:55.308 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:30 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:53:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:55.436 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:18.2.2-0.el9 ceph-noarch 132 k 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:55.437 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:53:55.438 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:53:55.438 INFO:teuthology.orchestra.run.smithi117.stdout:Install 6 Packages 2024-04-20T10:53:55.438 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:55.438 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 7.0 M 2024-04-20T10:53:55.438 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 33 M 2024-04-20T10:53:55.438 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:53:55.669 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:55.670 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:55.670 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:55.670 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:55.670 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:55.670 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:55.687 INFO:teuthology.orchestra.run.smithi117.stdout:(1/6): ceph-mgr-cephadm-18.2.2-0.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-04-20T10:53:55.779 INFO:teuthology.orchestra.run.smithi117.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-04-20T10:53:55.979 INFO:teuthology.orchestra.run.smithi117.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-20T10:53:55.992 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install libcephfs2 2024-04-20T10:53:56.121 INFO:teuthology.orchestra.run.smithi117.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 169 kB/s | 58 kB 00:00 2024-04-20T10:53:56.296 INFO:teuthology.orchestra.run.smithi117.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 901 kB/s | 548 kB 00:00 2024-04-20T10:53:56.321 INFO:teuthology.orchestra.run.smithi117.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 150 kB/s | 51 kB 00:00 2024-04-20T10:53:56.323 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:56.324 INFO:teuthology.orchestra.run.smithi117.stdout:Total 7.9 MB/s | 7.0 MB 00:00 2024-04-20T10:53:56.451 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:53:56.489 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:53:56.490 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:53:56.643 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:53:56.643 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:53:56.765 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:41 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:56.855 INFO:teuthology.orchestra.run.smithi173.stdout:Package libcephfs2-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:53:56.897 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:56.898 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:53:56.898 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:56.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install libcephfs-devel 2024-04-20T10:53:57.064 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:53:57.265 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-20T10:53:57.318 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-20T10:53:57.710 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:42 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:57.837 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:53:57.838 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:57.838 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-20T10:53:57.838 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:57.838 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:53:57.838 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:18.2.2-0.el9 ceph 31 k 2024-04-20T10:53:57.839 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:57.839 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:53:57.839 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:53:57.839 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-04-20T10:53:57.839 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:57.839 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 31 k 2024-04-20T10:53:57.839 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 139 k 2024-04-20T10:53:57.839 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:53:57.941 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-20T10:53:57.975 INFO:teuthology.orchestra.run.smithi173.stdout:libcephfs-devel-18.2.2-0.el9.x86_64.rpm 229 kB/s | 31 kB 00:00 2024-04-20T10:53:57.975 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:57.976 INFO:teuthology.orchestra.run.smithi173.stdout:Total 226 kB/s | 31 kB 00:00 2024-04-20T10:53:57.976 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:53:57.983 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:53:57.983 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:53:58.002 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:53:58.003 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:53:58.052 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-20T10:53:58.127 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-20T10:53:58.129 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:53:58.160 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-0.el9.noarch 6/6 2024-04-20T10:53:58.266 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:58.674 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:53:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:58.910 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:53:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:18.2.2-0.el9.x86_64 2024-04-20T10:53:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:53:58.910 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:53:59.031 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-0.el9.noarch 6/6 2024-04-20T10:53:59.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-0.el9.noarch 1/6 2024-04-20T10:53:59.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-20T10:53:59.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-20T10:53:59.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-20T10:53:59.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-20T10:53:59.128 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librados2 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:18.2.2-0.el9.noarch 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:53:59.363 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:53:59.680 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-fuse 2024-04-20T10:53:59.918 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:44 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:53:59.999 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:00.041 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:00.042 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:54:00.042 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:00.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librbd1 2024-04-20T10:54:00.440 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:35 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:00.562 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:18.2.2-0.el9 ceph 840 k 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout:Install 2 Packages 2024-04-20T10:54:00.563 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:00.564 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 920 k 2024-04-20T10:54:00.564 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 2.7 M 2024-04-20T10:54:00.564 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:54:00.834 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:45 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:54:00.916 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:00.956 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:00.957 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:54:00.958 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:00.981 INFO:teuthology.orchestra.run.smithi117.stdout:(1/2): ceph-fuse-18.2.2-0.el9.x86_64.rpm 6.2 MB/s | 840 kB 00:00 2024-04-20T10:54:01.023 INFO:teuthology.orchestra.run.smithi117.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 460 kB/s | 80 kB 00:00 2024-04-20T10:54:01.025 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:01.026 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.9 MB/s | 920 kB 00:00 2024-04-20T10:54:01.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rados 2024-04-20T10:54:01.066 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:54:01.139 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:54:01.139 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:54:01.244 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:54:01.244 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:54:01.531 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:54:01.686 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-20T10:54:01.743 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:46 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:54:01.758 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:18.2.2-0.el9.x86_64 2/2 2024-04-20T10:54:01.828 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rados-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:01.868 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:01.870 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:54:01.870 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:01.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rgw 2024-04-20T10:54:02.568 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-fuse-2:18.2.2-0.el9.x86_64 2/2 2024-04-20T10:54:02.568 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:18.2.2-0.el9.x86_64 1/2 2024-04-20T10:54:02.674 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:47 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:54:02.759 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rgw-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:02.799 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:02.801 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:54:02.801 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:02.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-cephfs 2024-04-20T10:54:02.889 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-20T10:54:02.889 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:02.889 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:54:02.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:18.2.2-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-20T10:54:02.889 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:02.889 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:03.226 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librados-devel 2024-04-20T10:54:03.632 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:48 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:54:03.719 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-cephfs-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:03.760 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:03.762 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:54:03.762 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:03.832 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rbd 2024-04-20T10:54:03.995 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:38 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:04.125 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:04.125 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:04.125 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:04.125 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:04.125 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:54:04.125 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:18.2.2-0.el9 ceph 126 k 2024-04-20T10:54:04.126 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:04.126 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:54:04.126 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:04.126 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-04-20T10:54:04.126 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:04.126 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 126 k 2024-04-20T10:54:04.126 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 456 k 2024-04-20T10:54:04.127 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:54:04.193 INFO:teuthology.orchestra.run.smithi117.stdout:librados-devel-18.2.2-0.el9.x86_64.rpm 1.9 MB/s | 126 kB 00:00 2024-04-20T10:54:04.194 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:04.194 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.8 MB/s | 126 kB 00:00 2024-04-20T10:54:04.194 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:54:04.201 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:54:04.201 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:54:04.276 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:54:04.277 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:54:04.439 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:54:04.542 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:04.590 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:49 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:54:04.678 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rbd-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:04.723 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:04.724 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-20T10:54:04.724 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:04.811 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-fuse 2024-04-20T10:54:05.079 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:05.338 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:05.338 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:05.338 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:54:05.338 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:18.2.2-0.el9.x86_64 2024-04-20T10:54:05.338 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:05.338 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:05.539 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:50 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:54:05.661 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:05.662 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:05.662 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:05.662 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:05.662 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:54:05.662 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:18.2.2-0.el9 ceph 86 k 2024-04-20T10:54:05.662 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:05.662 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:54:05.663 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:05.663 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-04-20T10:54:05.663 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:05.663 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 86 k 2024-04-20T10:54:05.663 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 227 k 2024-04-20T10:54:05.663 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:54:05.683 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install libcephfs2 2024-04-20T10:54:05.762 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-fuse-18.2.2-0.el9.x86_64.rpm 876 kB/s | 86 kB 00:00 2024-04-20T10:54:05.763 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:05.763 INFO:teuthology.orchestra.run.smithi173.stdout:Total 861 kB/s | 86 kB 00:00 2024-04-20T10:54:05.763 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:54:05.772 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:54:05.772 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:54:05.850 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:54:05.850 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:54:06.020 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:54:06.123 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:06.440 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:41 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:06.528 INFO:teuthology.orchestra.run.smithi117.stdout:Package libcephfs2-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:06.571 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:06.572 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:54:06.572 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:06.611 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-fuse-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:06.661 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install libcephfs-devel 2024-04-20T10:54:06.854 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:06.854 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:06.855 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:54:06.855 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:18.2.2-0.el9.x86_64 2024-04-20T10:54:06.855 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:06.855 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:07.129 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-mirror 2024-04-20T10:54:07.418 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:42 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:07.545 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:07.546 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:07.546 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:07.546 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:07.546 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:54:07.546 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:18.2.2-0.el9 ceph 31 k 2024-04-20T10:54:07.546 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:54:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-04-20T10:54:07.547 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 31 k 2024-04-20T10:54:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 139 k 2024-04-20T10:54:07.547 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:54:07.630 INFO:teuthology.orchestra.run.smithi117.stdout:libcephfs-devel-18.2.2-0.el9.x86_64.rpm 373 kB/s | 31 kB 00:00 2024-04-20T10:54:07.630 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:07.631 INFO:teuthology.orchestra.run.smithi117.stdout:Total 365 kB/s | 31 kB 00:00 2024-04-20T10:54:07.631 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:54:07.635 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:54:07.635 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:54:07.655 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:54:07.656 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:54:07.784 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:54:07.896 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:52 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:54:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:08.032 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:18.2.2-0.el9 ceph 3.0 M 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-04-20T10:54:08.033 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:08.034 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 3.0 M 2024-04-20T10:54:08.034 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 12 M 2024-04-20T10:54:08.034 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:54:08.222 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-mirror-18.2.2-0.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-04-20T10:54:08.223 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:08.224 INFO:teuthology.orchestra.run.smithi173.stdout:Total 16 MB/s | 3.0 MB 00:00 2024-04-20T10:54:08.224 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:54:08.242 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:54:08.242 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:54:08.337 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:08.354 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:54:08.355 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:54:08.577 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:08.577 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:08.577 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:54:08.578 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:18.2.2-0.el9.x86_64 2024-04-20T10:54:08.578 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:08.578 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:08.802 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librados2 2024-04-20T10:54:08.837 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:54:08.898 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:08.966 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:08.966 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:08.966 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:54:08.967 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:54:08.967 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:54:08.967 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:44 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:09.650 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:09.691 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:09.692 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:54:09.692 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:09.785 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librbd1 2024-04-20T10:54:10.287 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:10.287 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:10.288 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:54:10.288 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:18.2.2-0.el9.x86_64 2024-04-20T10:54:10.288 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:10.288 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:10.534 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:45 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:10.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-nbd 2024-04-20T10:54:10.616 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:10.657 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:10.659 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:54:10.659 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:10.742 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rados 2024-04-20T10:54:11.346 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:56 ago on Sat 20 Apr 2024 10:52:15 AM UTC. 2024-04-20T10:54:11.456 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:46 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:11.470 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:18.2.2-0.el9 ceph 171 k 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-04-20T10:54:11.471 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:11.472 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 171 k 2024-04-20T10:54:11.472 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 494 k 2024-04-20T10:54:11.472 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-20T10:54:11.540 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rados-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:11.568 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-nbd-18.2.2-0.el9.x86_64.rpm 1.8 MB/s | 171 kB 00:00 2024-04-20T10:54:11.569 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:11.569 INFO:teuthology.orchestra.run.smithi173.stdout:Total 1.7 MB/s | 171 kB 00:00 2024-04-20T10:54:11.569 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-20T10:54:11.580 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-20T10:54:11.581 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-20T10:54:11.582 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:11.583 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:54:11.583 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:11.661 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-20T10:54:11.661 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-20T10:54:11.666 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rgw 2024-04-20T10:54:11.842 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-20T10:54:11.946 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:12.387 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:47 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:12.448 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-nbd-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:12.474 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rgw-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:12.517 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:12.518 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:54:12.518 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:12.600 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-cephfs 2024-04-20T10:54:12.706 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:12.707 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:12.707 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-20T10:54:12.707 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:18.2.2-0.el9.x86_64 2024-04-20T10:54:12.707 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:54:12.707 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-20T10:54:13.043 DEBUG:teuthology.parallel:result is None 2024-04-20T10:54:13.328 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:48 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:13.414 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-cephfs-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:13.456 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:13.457 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:54:13.457 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:13.537 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rbd 2024-04-20T10:54:14.252 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:49 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:14.337 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rbd-2:18.2.2-0.el9.x86_64 is already installed. 2024-04-20T10:54:14.379 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:14.381 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-04-20T10:54:14.381 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:14.467 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-fuse 2024-04-20T10:54:15.238 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:50 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:15.367 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:15.368 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:15.368 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:15.368 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:15.368 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:54:15.368 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:18.2.2-0.el9 ceph 86 k 2024-04-20T10:54:15.369 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:15.369 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:54:15.369 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:15.369 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-04-20T10:54:15.369 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:15.369 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 86 k 2024-04-20T10:54:15.369 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 227 k 2024-04-20T10:54:15.369 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:54:15.449 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-fuse-18.2.2-0.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2024-04-20T10:54:15.450 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:15.451 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.0 MB/s | 86 kB 00:00 2024-04-20T10:54:15.451 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:54:15.459 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:54:15.459 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:54:15.537 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:54:15.538 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:54:15.710 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:54:15.824 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:16.327 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-fuse-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:16.631 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:16.631 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:16.632 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:54:16.632 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:18.2.2-0.el9.x86_64 2024-04-20T10:54:16.632 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:16.632 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:16.895 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-mirror 2024-04-20T10:54:17.636 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:52 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:17.768 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:17.769 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:17.769 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:17.769 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:17.769 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:54:17.769 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:18.2.2-0.el9 ceph 3.0 M 2024-04-20T10:54:17.769 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:17.769 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:54:17.769 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:17.770 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-04-20T10:54:17.770 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:17.770 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 3.0 M 2024-04-20T10:54:17.770 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 12 M 2024-04-20T10:54:17.770 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:54:17.894 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-mirror-18.2.2-0.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-04-20T10:54:17.895 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:17.895 INFO:teuthology.orchestra.run.smithi117.stdout:Total 24 MB/s | 3.0 MB 00:00 2024-04-20T10:54:17.895 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:54:17.906 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:54:17.907 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:54:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:54:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:54:18.472 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:54:18.545 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:18.625 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:18.626 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:18.626 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:54:18.626 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:54:18.626 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:54:18.626 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:20.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:20.024 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:20.024 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:54:20.024 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:18.2.2-0.el9.x86_64 2024-04-20T10:54:20.024 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:20.025 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:20.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-nbd 2024-04-20T10:54:21.078 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:01:56 ago on Sat 20 Apr 2024 10:52:25 AM UTC. 2024-04-20T10:54:21.200 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-20T10:54:21.200 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:21.200 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-04-20T10:54:21.200 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:21.200 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-20T10:54:21.200 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:18.2.2-0.el9 ceph 171 k 2024-04-20T10:54:21.200 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:21.201 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-20T10:54:21.201 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-04-20T10:54:21.201 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-04-20T10:54:21.201 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:21.201 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 171 k 2024-04-20T10:54:21.202 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 494 k 2024-04-20T10:54:21.202 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-20T10:54:21.280 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-nbd-18.2.2-0.el9.x86_64.rpm 2.1 MB/s | 171 kB 00:00 2024-04-20T10:54:21.281 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:54:21.281 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.1 MB/s | 171 kB 00:00 2024-04-20T10:54:21.281 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-20T10:54:21.291 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-20T10:54:21.291 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-20T10:54:21.366 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-20T10:54:21.367 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-20T10:54:21.550 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-20T10:54:21.647 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-nbd-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:22.338 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:18.2.2-0.el9.x86_64 1/1 2024-04-20T10:54:22.338 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:22.338 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-20T10:54:22.338 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:18.2.2-0.el9.x86_64 2024-04-20T10:54:22.339 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:54:22.339 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-20T10:54:22.633 DEBUG:teuthology.parallel:result is None 2024-04-20T10:54:22.634 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:54:22.634 INFO:teuthology.packaging:ref: None 2024-04-20T10:54:22.634 INFO:teuthology.packaging:tag: None 2024-04-20T10:54:22.634 INFO:teuthology.packaging:branch: reef 2024-04-20T10:54:22.634 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:54:22.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-20T10:54:22.976 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:54:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:18.2.2-0.el9 2024-04-20T10:54:23.047 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-0.el9 2024-04-20T10:54:23.047 INFO:teuthology.task.install:The correct ceph version 18.2.2-0 is installed. 2024-04-20T10:54:23.049 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:54:23.049 INFO:teuthology.packaging:ref: None 2024-04-20T10:54:23.049 INFO:teuthology.packaging:tag: None 2024-04-20T10:54:23.049 INFO:teuthology.packaging:branch: reef 2024-04-20T10:54:23.049 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:54:23.050 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-20T10:54:23.304 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:54:23.375 INFO:teuthology.orchestra.run.smithi173.stdout:18.2.2-0.el9 2024-04-20T10:54:23.376 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-0.el9 2024-04-20T10:54:23.376 INFO:teuthology.task.install:The correct ceph version 18.2.2-0 is installed. 2024-04-20T10:54:23.378 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T10:54:23.378 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:54:23.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:54:23.468 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:54:23.468 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:54:23.559 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T10:54:23.559 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:54:23.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:54:23.647 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:54:23.731 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:54:23.731 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:54:23.819 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:54:23.944 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T10:54:23.944 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:54:23.945 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:54:24.033 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:54:24.161 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:54:24.161 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:54:24.249 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:54:24.377 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T10:54:24.378 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:54:24.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:54:24.466 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:54:24.546 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:54:24.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:54:24.635 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:54:24.721 INFO:teuthology.run_tasks:Running task print... 2024-04-20T10:54:24.735 INFO:teuthology.task.print:**** done install task... 2024-04-20T10:54:24.735 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-20T10:54:24.871 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T10:54:24.871 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-04-20T10:54:24.872 INFO:tasks.cephadm:Cluster fsid is 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:54:24.872 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-20T10:54:24.872 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-20T10:54:24.872 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi117': '172.21.15.117', 'mon.smithi173': '172.21.15.173'} 2024-04-20T10:54:24.872 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-20T10:54:24.872 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-04-20T10:54:24.961 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2024-04-20T10:54:25.052 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-04-20T10:54:25.053 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:54:25.267 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78637/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3133-ga8c84f59', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3133.ga8c84f59'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 21:41:56.138585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-20T10:54:25.578 INFO:tasks.util.chacra:got chacra host chacra.ceph.com, ref reef, sha1 9d98999acebbad2663cc4938e94c9ffd83485cbd from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-04-20T10:54:25.580 INFO:tasks.cephadm:Discovered cachra url: https://chacra.ceph.com/binaries/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/x86_64/flavors/default/cephadm 2024-04-20T10:54:25.580 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://chacra.ceph.com/binaries/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T10:54:26.047 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 212578 Apr 20 10:54 /home/ubuntu/cephtest/cephadm 2024-04-20T10:54:26.047 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://chacra.ceph.com/binaries/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T10:54:26.387 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r--. 1 ubuntu ubuntu 212578 Apr 20 10:54 /home/ubuntu/cephtest/cephadm 2024-04-20T10:54:26.387 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T10:54:26.445 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T10:54:26.515 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-04-20T10:54:26.515 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-04-20T10:54:26.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-04-20T10:54:28.332 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-20T10:54:28.446 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-20T10:55:07.567 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T10:55:07.567 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)", 2024-04-20T10:55:07.567 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "9900ed1a6b81aaf043a19454fc819f6cb21ac0421c7c720b31d8fae6faebb644", 2024-04-20T10:55:07.567 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-04-20T10:55:07.567 INFO:teuthology.orchestra.run.smithi117.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210" 2024-04-20T10:55:07.567 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-04-20T10:55:07.567 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T10:55:09.463 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-20T10:55:09.464 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)", 2024-04-20T10:55:09.464 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "9900ed1a6b81aaf043a19454fc819f6cb21ac0421c7c720b31d8fae6faebb644", 2024-04-20T10:55:09.464 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2024-04-20T10:55:09.464 INFO:teuthology.orchestra.run.smithi173.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210" 2024-04-20T10:55:09.464 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-04-20T10:55:09.464 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-20T10:55:09.515 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-04-20T10:55:09.608 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2024-04-20T10:55:09.698 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-04-20T10:55:09.785 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2024-04-20T10:55:09.872 INFO:tasks.cephadm:Writing seed config... 2024-04-20T10:55:09.873 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-20T10:55:09.873 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-20T10:55:09.873 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-20T10:55:09.873 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-20T10:55:09.873 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-20T10:55:09.873 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-20T10:55:09.873 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-20T10:55:09.873 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-20T10:55:09.874 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-20T10:55:09.875 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-20T10:55:09.876 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-20T10:55:09.876 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-20T10:55:09.876 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-20T10:55:09.876 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-20T10:55:09.876 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-20T10:55:09.876 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-20T10:55:09.877 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:55:09.877 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-20T10:55:09.934 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 59ecfd22-ff04-11ee-bc92-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-20T10:55:09.934 DEBUG:teuthology.orchestra.run.smithi117:mon.smithi117> sudo journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service 2024-04-20T10:55:09.982 INFO:tasks.cephadm:Bootstrapping... 2024-04-20T10:55:09.982 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.117 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:55:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:55:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', '59ecfd22-ff04-11ee-bc92-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.117', '--skip-admin-label'] 2024-04-20T10:55:10.288 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T10:55:10.289 INFO:teuthology.orchestra.run.smithi117.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-20T10:55:10.289 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying podman|docker is present... 2024-04-20T10:55:10.338 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T10:55:10.338 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying lvm2 is present... 2024-04-20T10:55:10.338 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying time synchronization is in place... 2024-04-20T10:55:10.353 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T10:55:10.354 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T10:55:10.367 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T10:55:10.367 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-04-20T10:55:10.382 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout enabled 2024-04-20T10:55:10.398 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout active 2024-04-20T10:55:10.398 INFO:teuthology.orchestra.run.smithi117.stdout:Unit chronyd.service is enabled and running 2024-04-20T10:55:10.398 INFO:teuthology.orchestra.run.smithi117.stdout:Repeating the final host check... 2024-04-20T10:55:10.454 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T10:55:10.454 INFO:teuthology.orchestra.run.smithi117.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-20T10:55:10.454 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl is present 2024-04-20T10:55:10.454 INFO:teuthology.orchestra.run.smithi117.stdout:lvcreate is present 2024-04-20T10:55:10.468 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T10:55:10.469 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T10:55:10.482 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T10:55:10.482 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-04-20T10:55:10.504 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout enabled 2024-04-20T10:55:10.526 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout active 2024-04-20T10:55:10.527 INFO:teuthology.orchestra.run.smithi117.stdout:Unit chronyd.service is enabled and running 2024-04-20T10:55:10.527 INFO:teuthology.orchestra.run.smithi117.stdout:Host looks OK 2024-04-20T10:55:10.527 INFO:teuthology.orchestra.run.smithi117.stdout:Cluster fsid: 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:10.529 INFO:teuthology.orchestra.run.smithi117.stdout:Acquiring lock 140065450859680 on /run/cephadm/59ecfd22-ff04-11ee-bc92-c7b262605968.lock 2024-04-20T10:55:10.529 INFO:teuthology.orchestra.run.smithi117.stdout:Lock 140065450859680 acquired on /run/cephadm/59ecfd22-ff04-11ee-bc92-c7b262605968.lock 2024-04-20T10:55:10.530 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying IP 172.21.15.117 port 3300 ... 2024-04-20T10:55:10.531 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying IP 172.21.15.117 port 6789 ... 2024-04-20T10:55:10.532 INFO:teuthology.orchestra.run.smithi117.stdout:Base mon IP(s) is [172.21.15.117:3300, 172.21.15.117:6789], mon addrv is [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-04-20T10:55:10.546 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.117 metric 100 2024-04-20T10:55:10.547 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.117 metric 100 2024-04-20T10:55:10.559 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-20T10:55:10.560 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-20T10:55:10.560 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1786sec hoplimit 64 pref medium 2024-04-20T10:55:10.571 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-20T10:55:10.571 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-20T10:55:10.571 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T10:55:10.571 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-20T10:55:10.572 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-04-20T10:55:10.572 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T10:55:10.574 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.0/20` 2024-04-20T10:55:10.574 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.0/20` 2024-04-20T10:55:10.575 INFO:teuthology.orchestra.run.smithi117.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-20T10:55:10.577 INFO:teuthology.orchestra.run.smithi117.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-20T10:55:10.580 INFO:teuthology.orchestra.run.smithi117.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-20T10:55:11.232 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout 9900ed1a6b81aaf043a19454fc819f6cb21ac0421c7c720b31d8fae6faebb644 2024-04-20T10:55:11.232 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-04-20T10:55:11.232 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Getting image source signatures 2024-04-20T10:55:11.232 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying blob sha256:c98db977c11909275f2799ff686e2165262975518f51f9eea9b24a5c8a4ea0d6 2024-04-20T10:55:11.232 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-20T10:55:11.232 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying config sha256:9900ed1a6b81aaf043a19454fc819f6cb21ac0421c7c720b31d8fae6faebb644 2024-04-20T10:55:11.233 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-20T10:55:12.536 INFO:teuthology.orchestra.run.smithi117.stdout:ceph: stdout ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable) 2024-04-20T10:55:12.536 INFO:teuthology.orchestra.run.smithi117.stdout:Ceph version: ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable) 2024-04-20T10:55:12.537 INFO:teuthology.orchestra.run.smithi117.stdout:Extracting ceph user uid/gid from container image... 2024-04-20T10:55:13.791 INFO:teuthology.orchestra.run.smithi117.stdout:stat: stdout 167 167 2024-04-20T10:55:13.791 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial keys... 2024-04-20T10:55:15.047 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQASnyNmQcVdExAAu3uu2rBdsyYVmHraVwq0sw== 2024-04-20T10:55:16.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQATnyNmenp+HxAA6Aew6+sqDeEjFfPB34pBIA== 2024-04-20T10:55:26.966 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQAVnyNmGYnRJxAAf+ivlcDQ8fC39H3zNPgNMw== 2024-04-20T10:55:26.968 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial monmap... 2024-04-20T10:55:28.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T10:55:28.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-20T10:55:28.150 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:28.150 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T10:55:28.150 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool for smithi117 [v2:172.21.15.117:3300,v1:172.21.15.117:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T10:55:28.150 INFO:teuthology.orchestra.run.smithi117.stdout:setting min_mon_release = pacific 2024-04-20T10:55:28.150 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: set fsid to 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:28.150 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T10:55:28.150 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:55:28.151 INFO:teuthology.orchestra.run.smithi117.stdout:Creating mon... 2024-04-20T10:55:29.378 INFO:teuthology.orchestra.run.smithi117.stdout:create mon.smithi117 on 2024-04-20T10:55:29.767 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-20T10:55:30.002 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-20T10:55:30.291 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968.target → /etc/systemd/system/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968.target. 2024-04-20T10:55:30.291 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968.target → /etc/systemd/system/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968.target. 2024-04-20T10:55:30.662 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117 2024-04-20T10:55:30.662 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to reset failed state of unit ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service: Unit ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service not loaded. 2024-04-20T10:55:30.956 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968.target.wants/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service → /etc/systemd/system/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@.service. 2024-04-20T10:55:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:31 smithi117 podman[15265]: 2024-04-20 10:55:31.436797258 +0000 UTC m=+0.209563804 container create d213e63eee303dd617c6630a44dd3ee2775cb43c03f48b17ad0f695159cbc5ba (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=reef-646b8c7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-20T10:55:31.809 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T10:55:31.810 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-04-20T10:55:31.823 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T10:55:31.824 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-04-20T10:55:31.824 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-04-20T10:55:31.824 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T10:55:31.824 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mon to start... 2024-04-20T10:55:31.824 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mon... 2024-04-20T10:55:31.981 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:31 smithi117 podman[15265]: 2024-04-20 10:55:31.692304696 +0000 UTC m=+0.465071274 container init d213e63eee303dd617c6630a44dd3ee2775cb43c03f48b17ad0f695159cbc5ba (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-646b8c7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:55:31.982 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:31 smithi117 podman[15265]: 2024-04-20 10:55:31.702954309 +0000 UTC m=+0.475720890 container start d213e63eee303dd617c6630a44dd3ee2775cb43c03f48b17ad0f695159cbc5ba (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-646b8c7, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:55:31.982 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:31 smithi117 ceph-mon[15284]: mkfs 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:31.982 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:31 smithi117 bash[15265]: d213e63eee303dd617c6630a44dd3ee2775cb43c03f48b17ad0f695159cbc5ba 2024-04-20T10:55:31.982 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:31 smithi117 ceph-mon[15284]: mon.smithi117 is new leader, mons smithi117 in quorum (ranks 0) 2024-04-20T10:55:31.982 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:31 smithi117 systemd[1]: Started Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T10:55:32.882 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:32 smithi117 ceph-mon[15284]: mon.smithi117 is new leader, mons smithi117 in quorum (ranks 0) 2024-04-20T10:55:32.882 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:32 smithi117 ceph-mon[15284]: monmap e1: 1 mons at {smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-20T10:55:32.882 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:32 smithi117 ceph-mon[15284]: fsmap 2024-04-20T10:55:32.882 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:32 smithi117 ceph-mon[15284]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T10:55:32.882 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:32 smithi117 ceph-mon[15284]: mgrmap e1: no daemons active 2024-04-20T10:55:32.883 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:32 smithi117 ceph-mon[15284]: from='client.? 172.21.15.117:0/2981385638' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout cluster: 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout id: 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout services: 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi117 (age 0.766593s) 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout data: 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout pgs: 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.452+0000 7fc847000700 1 Processor -- start 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.452+0000 7fc847000700 1 -- start start 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.452+0000 7fc847000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84010d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.452+0000 7fc847000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84010d940 con 0x7fc84010cf50 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.453+0000 7fc845c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84010d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.453+0000 7fc845c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84010d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55600/0 (socket says 172.21.15.117:55600) 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.453+0000 7fc845c00700 1 -- 172.21.15.117:0/1807647510 learned_addr learned my addr 172.21.15.117:0/1807647510 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:33.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.454+0000 7fc845c00700 1 -- 172.21.15.117:0/1807647510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84010da80 con 0x7fc84010cf50 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.454+0000 7fc845c00700 1 --2- 172.21.15.117:0/1807647510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84010d370 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc8340098a0 tx=0x7fc834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2f6613fcf21818 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.454+0000 7fc83fe00700 1 -- 172.21.15.117:0/1807647510 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc834004180 con 0x7fc84010cf50 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc83fe00700 1 -- 172.21.15.117:0/1807647510 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8340042e0 con 0x7fc84010cf50 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc83fe00700 1 -- 172.21.15.117:0/1807647510 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8340036a0 con 0x7fc84010cf50 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc847000700 1 -- 172.21.15.117:0/1807647510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 msgr2=0x7fc84010d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc847000700 1 --2- 172.21.15.117:0/1807647510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84010d370 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc8340098a0 tx=0x7fc834009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc847000700 1 -- 172.21.15.117:0/1807647510 shutdown_connections 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc847000700 1 --2- 172.21.15.117:0/1807647510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84010d370 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc847000700 1 -- 172.21.15.117:0/1807647510 >> 172.21.15.117:0/1807647510 conn(0x7fc84007b9f0 msgr2=0x7fc84007de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc847000700 1 -- 172.21.15.117:0/1807647510 shutdown_connections 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.455+0000 7fc847000700 1 -- 172.21.15.117:0/1807647510 wait complete. 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.456+0000 7fc847000700 1 Processor -- start 2024-04-20T10:55:33.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.457+0000 7fc847000700 1 -- start start 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.457+0000 7fc847000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.457+0000 7fc847000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401a0350 con 0x7fc84010cf50 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.457+0000 7fc845c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.458+0000 7fc845c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84019fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55602/0 (socket says 172.21.15.117:55602) 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.458+0000 7fc845c00700 1 -- 172.21.15.117:0/2981385638 learned_addr learned my addr 172.21.15.117:0/2981385638 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.458+0000 7fc845c00700 1 -- 172.21.15.117:0/2981385638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834009550 con 0x7fc84010cf50 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.458+0000 7fc845c00700 1 --2- 172.21.15.117:0/2981385638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84019fe10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc834004000 tx=0x7fc834004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.458+0000 7fc83e000700 1 -- 172.21.15.117:0/2981385638 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc834003ce0 con 0x7fc84010cf50 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.459+0000 7fc83e000700 1 -- 172.21.15.117:0/2981385638 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc834020070 con 0x7fc84010cf50 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.459+0000 7fc83e000700 1 -- 172.21.15.117:0/2981385638 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc834011630 con 0x7fc84010cf50 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.459+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8401a0550 con 0x7fc84010cf50 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.459+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8401a09f0 con 0x7fc84010cf50 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.459+0000 7fc83e000700 1 -- 172.21.15.117:0/2981385638 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fc834022030 con 0x7fc84010cf50 2024-04-20T10:55:33.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.459+0000 7fc83e000700 1 -- 172.21.15.117:0/2981385638 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc834025d40 con 0x7fc84010cf50 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.460+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc840199b80 con 0x7fc84010cf50 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.463+0000 7fc83e000700 1 -- 172.21.15.117:0/2981385638 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fc834025480 con 0x7fc84010cf50 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.542+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc84010dfd0 con 0x7fc84010cf50 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.543+0000 7fc83e000700 1 -- 172.21.15.117:0/2981385638 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fc834011c20 con 0x7fc84010cf50 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.544+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 msgr2=0x7fc84019fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.544+0000 7fc847000700 1 --2- 172.21.15.117:0/2981385638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84019fe10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc834004000 tx=0x7fc834004750 comp rx=0 tx=0).stop 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.544+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 shutdown_connections 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.544+0000 7fc847000700 1 --2- 172.21.15.117:0/2981385638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc84010cf50 0x7fc84019fe10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.544+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 >> 172.21.15.117:0/2981385638 conn(0x7fc84007b9f0 msgr2=0x7fc840196cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.545+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 shutdown_connections 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:32.545+0000 7fc847000700 1 -- 172.21.15.117:0/2981385638 wait complete. 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:mon is available 2024-04-20T10:55:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:Assimilating anything we can from ceph.conf... 2024-04-20T10:55:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:33 smithi117 ceph-mon[15284]: from='client.? 172.21.15.117:0/64944817' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T10:55:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:33 smithi117 ceph-mon[15284]: from='client.? 172.21.15.117:0/64944817' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-20T10:55:34.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:34.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T10:55:34.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout fsid = 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:34.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-04-20T10:55:34.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T10:55:34.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T10:55:34.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.834+0000 7ff109a00700 1 Processor -- start 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.834+0000 7ff109a00700 1 -- start start 2024-04-20T10:55:34.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.834+0000 7ff109a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff1041069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.834+0000 7ff109a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104106fb0 con 0x7ff1041065c0 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.835+0000 7ff102a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff1041069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.835+0000 7ff102a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff1041069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52038/0 (socket says 172.21.15.117:52038) 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.835+0000 7ff102a00700 1 -- 172.21.15.117:0/3740293352 learned_addr learned my addr 172.21.15.117:0/3740293352 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.835+0000 7ff102a00700 1 -- 172.21.15.117:0/3740293352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1041077c0 con 0x7ff1041065c0 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.836+0000 7ff102a00700 1 --2- 172.21.15.117:0/3740293352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff1041069e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40098a0 tx=0x7ff0f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7c77170cebe294 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.836+0000 7ff101600700 1 -- 172.21.15.117:0/3740293352 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0f4004180 con 0x7ff1041065c0 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.836+0000 7ff101600700 1 -- 172.21.15.117:0/3740293352 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f40042e0 con 0x7ff1041065c0 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.836+0000 7ff101600700 1 -- 172.21.15.117:0/3740293352 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0f40036a0 con 0x7ff1041065c0 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.837+0000 7ff109a00700 1 -- 172.21.15.117:0/3740293352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 msgr2=0x7ff1041069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.837+0000 7ff109a00700 1 --2- 172.21.15.117:0/3740293352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff1041069e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40098a0 tx=0x7ff0f4009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:34.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.837+0000 7ff109a00700 1 -- 172.21.15.117:0/3740293352 shutdown_connections 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.837+0000 7ff109a00700 1 --2- 172.21.15.117:0/3740293352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff1041069e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.837+0000 7ff109a00700 1 -- 172.21.15.117:0/3740293352 >> 172.21.15.117:0/3740293352 conn(0x7ff104101be0 msgr2=0x7ff104104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.837+0000 7ff109a00700 1 -- 172.21.15.117:0/3740293352 shutdown_connections 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.837+0000 7ff109a00700 1 -- 172.21.15.117:0/3740293352 wait complete. 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.838+0000 7ff109a00700 1 Processor -- start 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.838+0000 7ff109a00700 1 -- start start 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.838+0000 7ff109a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff10419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.838+0000 7ff109a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10419c200 con 0x7ff1041065c0 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.839+0000 7ff102a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff10419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.839+0000 7ff102a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff10419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52042/0 (socket says 172.21.15.117:52042) 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.839+0000 7ff102a00700 1 -- 172.21.15.117:0/64944817 learned_addr learned my addr 172.21.15.117:0/64944817 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.839+0000 7ff102a00700 1 -- 172.21.15.117:0/64944817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4009550 con 0x7ff1041065c0 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.839+0000 7ff102a00700 1 --2- 172.21.15.117:0/64944817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff10419bcc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4004000 tx=0x7ff0f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:34.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.840+0000 7ff0fb400700 1 -- 172.21.15.117:0/64944817 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0f4003ce0 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.840+0000 7ff0fb400700 1 -- 172.21.15.117:0/64944817 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f4020070 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.840+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10419c400 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.840+0000 7ff0fb400700 1 -- 172.21.15.117:0/64944817 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0f4011630 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.840+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10419c8a0 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.840+0000 7ff0fb400700 1 -- 172.21.15.117:0/64944817 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff0f4022030 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.840+0000 7ff0fb400700 1 -- 172.21.15.117:0/64944817 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff0f4025d00 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.841+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff104107b30 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.844+0000 7ff0fb400700 1 -- 172.21.15.117:0/64944817 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff0f4025480 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.920+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff104107cc0 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.929+0000 7ff0fb400700 1 -- 172.21.15.117:0/64944817 <== mon.0 v2:172.21.15.117:3300/0 7 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff0f4011bd0 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.929+0000 7ff0fb400700 1 -- 172.21.15.117:0/64944817 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7ff0f401c500 con 0x7ff1041065c0 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.931+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 msgr2=0x7ff10419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:34.604 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.931+0000 7ff109a00700 1 --2- 172.21.15.117:0/64944817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff10419bcc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4004000 tx=0x7ff0f4004750 comp rx=0 tx=0).stop 2024-04-20T10:55:34.605 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.931+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 shutdown_connections 2024-04-20T10:55:34.605 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.931+0000 7ff109a00700 1 --2- 172.21.15.117:0/64944817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1041065c0 0x7ff10419bcc0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:34.605 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.931+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 >> 172.21.15.117:0/64944817 conn(0x7ff104101be0 msgr2=0x7ff10407b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:34.605 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.931+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 shutdown_connections 2024-04-20T10:55:34.605 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:33.931+0000 7ff109a00700 1 -- 172.21.15.117:0/64944817 wait complete. 2024-04-20T10:55:34.605 INFO:teuthology.orchestra.run.smithi117.stdout:Generating new minimal ceph.conf... 2024-04-20T10:55:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:35 smithi117 ceph-mon[15284]: from='client.? 172.21.15.117:0/2921104380' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:55:36.033 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.227+0000 7febd0600700 1 Processor -- start 2024-04-20T10:55:36.033 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.227+0000 7febd0600700 1 -- start start 2024-04-20T10:55:36.033 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.228+0000 7febd0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc810d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:36.033 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.228+0000 7febd0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc810da70 con 0x7febc810d080 2024-04-20T10:55:36.033 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.228+0000 7febcdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc810d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:36.033 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.228+0000 7febcdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc810d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52050/0 (socket says 172.21.15.117:52050) 2024-04-20T10:55:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.228+0000 7febcdc00700 1 -- 172.21.15.117:0/3887581149 learned_addr learned my addr 172.21.15.117:0/3887581149 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.229+0000 7febcdc00700 1 -- 172.21.15.117:0/3887581149 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc810de00 con 0x7febc810d080 2024-04-20T10:55:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.229+0000 7febcdc00700 1 --2- 172.21.15.117:0/3887581149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc810d4a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098a0 tx=0x7febbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f88e23809566af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.229+0000 7febc7e00700 1 -- 172.21.15.117:0/3887581149 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febbc004180 con 0x7febc810d080 2024-04-20T10:55:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.230+0000 7febc7e00700 1 -- 172.21.15.117:0/3887581149 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7febbc0042e0 con 0x7febc810d080 2024-04-20T10:55:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.230+0000 7febc7e00700 1 -- 172.21.15.117:0/3887581149 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febbc011a70 con 0x7febc810d080 2024-04-20T10:55:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.230+0000 7febd0600700 1 -- 172.21.15.117:0/3887581149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 msgr2=0x7febc810d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:36.034 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.230+0000 7febd0600700 1 --2- 172.21.15.117:0/3887581149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc810d4a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098a0 tx=0x7febbc009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.230+0000 7febd0600700 1 -- 172.21.15.117:0/3887581149 shutdown_connections 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.230+0000 7febd0600700 1 --2- 172.21.15.117:0/3887581149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc810d4a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.230+0000 7febd0600700 1 -- 172.21.15.117:0/3887581149 >> 172.21.15.117:0/3887581149 conn(0x7febc807bbf0 msgr2=0x7febc807e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.231+0000 7febd0600700 1 -- 172.21.15.117:0/3887581149 shutdown_connections 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.231+0000 7febd0600700 1 -- 172.21.15.117:0/3887581149 wait complete. 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.232+0000 7febd0600700 1 Processor -- start 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.232+0000 7febd0600700 1 -- start start 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.232+0000 7febd0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc81a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.232+0000 7febd0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81a0e50 con 0x7febc810d080 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.232+0000 7febcdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc81a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.232+0000 7febcdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc81a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52058/0 (socket says 172.21.15.117:52058) 2024-04-20T10:55:36.038 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.232+0000 7febcdc00700 1 -- 172.21.15.117:0/2921104380 learned_addr learned my addr 172.21.15.117:0/2921104380 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.233+0000 7febcdc00700 1 -- 172.21.15.117:0/2921104380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc009550 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.233+0000 7febcdc00700 1 --2- 172.21.15.117:0/2921104380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc81a0910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7febbc011710 tx=0x7febbc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.233+0000 7febc6000700 1 -- 172.21.15.117:0/2921104380 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febbc003b10 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.233+0000 7febc6000700 1 -- 172.21.15.117:0/2921104380 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7febbc01b8c0 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.233+0000 7febc6000700 1 -- 172.21.15.117:0/2921104380 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febbc019770 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.233+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc81a1050 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.233+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc81a14f0 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.234+0000 7febc6000700 1 -- 172.21.15.117:0/2921104380 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7febbc019cf0 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.234+0000 7febc6000700 1 -- 172.21.15.117:0/2921104380 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7febbc01a710 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.235+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8062510 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.237+0000 7febc6000700 1 -- 172.21.15.117:0/2921104380 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7febbc029060 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.313+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7febc8111030 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.314+0000 7febc6000700 1 -- 172.21.15.117:0/2921104380 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7febbc01f070 con 0x7febc810d080 2024-04-20T10:55:36.039 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.315+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 msgr2=0x7febc81a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:36.040 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.315+0000 7febd0600700 1 --2- 172.21.15.117:0/2921104380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc81a0910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7febbc011710 tx=0x7febbc0047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.040 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.315+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 shutdown_connections 2024-04-20T10:55:36.040 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.315+0000 7febd0600700 1 --2- 172.21.15.117:0/2921104380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc810d080 0x7febc81a0910 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.040 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.315+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 >> 172.21.15.117:0/2921104380 conn(0x7febc807bbf0 msgr2=0x7febc807c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:36.040 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.316+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 shutdown_connections 2024-04-20T10:55:36.040 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:35.316+0000 7febd0600700 1 -- 172.21.15.117:0/2921104380 wait complete. 2024-04-20T10:55:36.040 INFO:teuthology.orchestra.run.smithi117.stdout:Restarting the monitor... 2024-04-20T10:55:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:36 smithi117 systemd[1]: Stopping Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T10:55:37.044 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:36 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[15280]: 2024-04-20T10:55:36.673+0000 7fa5b8c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi117 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:55:37.044 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:36 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[15280]: 2024-04-20T10:55:36.673+0000 7fa5b8c00700 -1 mon.smithi117@0(leader) e1 *** Got Signal Terminated *** 2024-04-20T10:55:37.044 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:36 smithi117 podman[15525]: 2024-04-20 10:55:36.838721751 +0000 UTC m=+0.468942765 container died d213e63eee303dd617c6630a44dd3ee2775cb43c03f48b17ad0f695159cbc5ba (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=reef-646b8c7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:55:37.383 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:37 smithi117 podman[15525]: 2024-04-20 10:55:37.243849416 +0000 UTC m=+0.874070458 container cleanup d213e63eee303dd617c6630a44dd3ee2775cb43c03f48b17ad0f695159cbc5ba (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=reef-646b8c7, org.label-schema.vendor=CentOS) 2024-04-20T10:55:37.384 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:37 smithi117 bash[15525]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117 2024-04-20T10:55:37.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:37 smithi117 podman[15539]: 2024-04-20 10:55:37.436404265 +0000 UTC m=+0.579590131 container remove d213e63eee303dd617c6630a44dd3ee2775cb43c03f48b17ad0f695159cbc5ba (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=reef-646b8c7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-20T10:55:38.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:37 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service: Deactivated successfully. 2024-04-20T10:55:38.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:37 smithi117 systemd[1]: Stopped Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T10:55:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:37 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service: Consumed 1.753s CPU time. 2024-04-20T10:55:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:37 smithi117 systemd[1]: Starting Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T10:55:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 podman[15639]: 2024-04-20 10:55:38.122872875 +0000 UTC m=+0.031835626 image pull 9900ed1a6b81aaf043a19454fc819f6cb21ac0421c7c720b31d8fae6faebb644 quay.ceph.io/ceph-ci/ceph:reef 2024-04-20T10:55:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 podman[15639]: 2024-04-20 10:55:38.262237158 +0000 UTC m=+0.171199937 container create a3ca404fd50f2d067d2fd255bb6cccfc043c4d93c9dc6fcbd68ffd06fab9bd75 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, RELEASE=reef-646b8c7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-20T10:55:38.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 podman[15639]: 2024-04-20 10:55:38.507315326 +0000 UTC m=+0.416278112 container init a3ca404fd50f2d067d2fd255bb6cccfc043c4d93c9dc6fcbd68ffd06fab9bd75 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-646b8c7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-20T10:55:38.606 INFO:teuthology.orchestra.run.smithi117.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-20T10:55:38.774 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 podman[15639]: 2024-04-20 10:55:38.515917475 +0000 UTC m=+0.424880218 container start a3ca404fd50f2d067d2fd255bb6cccfc043c4d93c9dc6fcbd68ffd06fab9bd75 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=reef-646b8c7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-20T10:55:38.774 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T10:55:38.774 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable), process ceph-mon, pid 2 2024-04-20T10:55:38.774 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: pidfile_write: ignore empty --pid-file 2024-04-20T10:55:38.774 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: load: jerasure load: lrc 2024-04-20T10:55:38.774 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: RocksDB version: 7.9.2 2024-04-20T10:55:38.774 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Git sha 0 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Compile date 2024-04-18 11:37:30 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: DB SUMMARY 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: DB Session ID: VQSSEBLY4ULJ76BONTQ5 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: CURRENT file: CURRENT 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi117/store.db dir, Total Num: 1, files: 000008.sst 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi117/store.db: 000009.log size: 98505 ; 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.error_if_exists: 0 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.create_if_missing: 0 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.paranoid_checks: 1 2024-04-20T10:55:38.775 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.env: 0x55f113f49820 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.info_log: 0x55f123f59420 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.statistics: (nil) 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.use_fsync: 0 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_log_file_size: 0 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T10:55:38.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.allow_fallocate: 1 2024-04-20T10:55:38.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T10:55:38.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T10:55:38.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.use_direct_reads: 0 2024-04-20T10:55:38.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.db_log_dir: 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.wal_dir: 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T10:55:38.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.write_buffer_manager: 0x55f1231ce140 2024-04-20T10:55:38.779 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T10:55:38.779 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T10:55:38.779 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T10:55:38.780 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T10:55:38.780 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T10:55:38.780 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T10:55:38.780 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T10:55:38.780 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T10:55:38.780 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.unordered_write: 0 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.row_cache: None 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.wal_filter: None 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.two_write_queues: 0 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.wal_compression: 0 2024-04-20T10:55:38.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.atomic_flush: 0 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.log_readahead_size: 0 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T10:55:38.782 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_background_jobs: 2 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_background_compactions: -1 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_subcompactions: 1 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_open_files: -1 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T10:55:38.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_background_flushes: -1 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Compression algorithms supported: 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: kZSTD supported: 0 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: kXpressCompression supported: 0 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: kZlibCompression supported: 1 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: kSnappyCompression supported: 1 2024-04-20T10:55:38.784 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: kLZ4Compression supported: 1 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: kBZip2Compression supported: 0 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000010 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.merge_operator: 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_filter: None 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f123f59520) 2024-04-20T10:55:38.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_type: 0 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_index_type: 0 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_shortening: 1 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: checksum: 4 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: no_block_cache: 0 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache: 0x55f12326f1f0 2024-04-20T10:55:38.786 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_options: 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: capacity : 536870912 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_shard_bits : 4 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: strict_capacity_limit : 0 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_compressed: (nil) 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: persistent_cache: (nil) 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size: 4096 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size_deviation: 10 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_restart_interval: 16 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_block_restart_interval: 1 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: metadata_block_size: 4096 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: partition_filters: 0 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: use_delta_encoding: 1 2024-04-20T10:55:38.787 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: filter_policy: bloomfilter 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: whole_key_filtering: 1 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: verify_compression: 0 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: format_version: 5 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_index_compression: 1 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_align: 0 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_auto_readahead_size: 262144 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: prepopulate_block_cache: 0 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: initial_auto_readahead_size: 8192 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression: NoCompression 2024-04-20T10:55:38.788 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T10:55:38.789 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T10:55:38.789 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T10:55:38.789 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.num_levels: 7 2024-04-20T10:55:38.789 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T10:55:38.789 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T10:55:38.790 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T10:55:38.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T10:55:38.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.table_properties_collectors: 2024-04-20T10:55:38.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.inplace_update_support: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.bloom_locality: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.max_successive_merges: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.ttl: 2592000 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T10:55:38.794 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.enable_blob_files: false 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.min_blob_size: 0 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5ef1c280-e2bc-4bc7-b956-a528835c0edc 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713610538568814, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-20T10:55:38.795 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713610538571868, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93782, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91816, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84380, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713610538, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5ef1c280-e2bc-4bc7-b956-a528835c0edc", "db_session_id": "VQSSEBLY4ULJ76BONTQ5", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713610538572065, "job": 1, "event": "recovery_finished"} 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi117/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f12330e000 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: DB pointer 0x55f1232fc000 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** DB Stats ** 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T10:55:38.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: L0 2/0 93.46 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Sum 2/0 93.46 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T10:55:38.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 46.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative compaction: 0.00 GB write, 9.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval compaction: 0.00 GB write, 9.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache BinnedLRUCache@0x55f12326f1f0#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-20T10:55:38.798 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-04-20T10:55:38.799 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T10:55:38.799 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T10:55:38.799 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: starting mon.smithi117 rank 0 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi117 fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:38.799 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117@-1(???) e1 preinit fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:38.799 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117@-1(???).mds e1 new map 2024-04-20T10:55:38.799 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117@-1(???).mds e1 print_map 2024-04-20T10:55:38.799 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: e1 2024-04-20T10:55:38.799 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: legacy client fscid: -1 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: No filesystems configured 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 bash[15639]: a3ca404fd50f2d067d2fd255bb6cccfc043c4d93c9dc6fcbd68ffd06fab9bd75 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 systemd[1]: Started Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mon.smithi117 is new leader, mons smithi117 in quorum (ranks 0) 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: monmap e1: 1 mons at {smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: fsmap 2024-04-20T10:55:38.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T10:55:38.801 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:38 smithi117 ceph-mon[15659]: mgrmap e1: no daemons active 2024-04-20T10:55:40.086 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.312+0000 7fd719e00700 1 Processor -- start 2024-04-20T10:55:40.086 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.312+0000 7fd719e00700 1 -- start start 2024-04-20T10:55:40.086 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.313+0000 7fd719e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd7140791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.086 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.313+0000 7fd719e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140797a0 con 0x7fd71407ad70 2024-04-20T10:55:40.086 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.313+0000 7fd718a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd7140791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.086 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.313+0000 7fd718a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd7140791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52070/0 (socket says 172.21.15.117:52070) 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.313+0000 7fd718a00700 1 -- 172.21.15.117:0/4177393469 learned_addr learned my addr 172.21.15.117:0/4177393469 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.314+0000 7fd718a00700 1 -- 172.21.15.117:0/4177393469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7140798e0 con 0x7fd71407ad70 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.314+0000 7fd718a00700 1 --2- 172.21.15.117:0/4177393469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd7140791d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd7040098a0 tx=0x7fd704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37ab77a2d604326 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.314+0000 7fd713e00700 1 -- 172.21.15.117:0/4177393469 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd704004180 con 0x7fd71407ad70 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd713e00700 1 -- 172.21.15.117:0/4177393469 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd7040042e0 con 0x7fd71407ad70 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd713e00700 1 -- 172.21.15.117:0/4177393469 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd704011a70 con 0x7fd71407ad70 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd719e00700 1 -- 172.21.15.117:0/4177393469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 msgr2=0x7fd7140791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd719e00700 1 --2- 172.21.15.117:0/4177393469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd7140791d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd7040098a0 tx=0x7fd704009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd719e00700 1 -- 172.21.15.117:0/4177393469 shutdown_connections 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd719e00700 1 --2- 172.21.15.117:0/4177393469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd7140791d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd719e00700 1 -- 172.21.15.117:0/4177393469 >> 172.21.15.117:0/4177393469 conn(0x7fd714101a30 msgr2=0x7fd714103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd719e00700 1 -- 172.21.15.117:0/4177393469 shutdown_connections 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.315+0000 7fd719e00700 1 -- 172.21.15.117:0/4177393469 wait complete. 2024-04-20T10:55:40.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.316+0000 7fd719e00700 1 Processor -- start 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.317+0000 7fd719e00700 1 -- start start 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.317+0000 7fd719e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd71419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.317+0000 7fd719e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419c820 con 0x7fd71407ad70 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.317+0000 7fd718a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd71419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.317+0000 7fd718a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd71419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52078/0 (socket says 172.21.15.117:52078) 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.317+0000 7fd718a00700 1 -- 172.21.15.117:0/121872414 learned_addr learned my addr 172.21.15.117:0/121872414 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.318+0000 7fd718a00700 1 -- 172.21.15.117:0/121872414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd704009550 con 0x7fd71407ad70 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.318+0000 7fd718a00700 1 --2- 172.21.15.117:0/121872414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd71419c2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd704011710 tx=0x7fd7040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.318+0000 7fd712000700 1 -- 172.21.15.117:0/121872414 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd704003b10 con 0x7fd71407ad70 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.318+0000 7fd712000700 1 -- 172.21.15.117:0/121872414 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd70401b8c0 con 0x7fd71407ad70 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.318+0000 7fd712000700 1 -- 172.21.15.117:0/121872414 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7040197e0 con 0x7fd71407ad70 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.318+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71419ca20 con 0x7fd71407ad70 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.318+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71419cec0 con 0x7fd71407ad70 2024-04-20T10:55:40.088 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.319+0000 7fd712000700 1 -- 172.21.15.117:0/121872414 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd704019cd0 con 0x7fd71407ad70 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.319+0000 7fd712000700 1 -- 172.21.15.117:0/121872414 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd70401a710 con 0x7fd71407ad70 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.319+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71410c750 con 0x7fd71407ad70 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.322+0000 7fd712000700 1 -- 172.21.15.117:0/121872414 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd704029060 con 0x7fd71407ad70 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.397+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fd71419f750 con 0x7fd71407ad70 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.402+0000 7fd712000700 1 -- 172.21.15.117:0/121872414 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd70401f070 con 0x7fd71407ad70 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.403+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 msgr2=0x7fd71419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.403+0000 7fd719e00700 1 --2- 172.21.15.117:0/121872414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd71419c2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd704011710 tx=0x7fd7040047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.403+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 shutdown_connections 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.403+0000 7fd719e00700 1 --2- 172.21.15.117:0/121872414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71407ad70 0x7fd71419c2e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.403+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 >> 172.21.15.117:0/121872414 conn(0x7fd714101a30 msgr2=0x7fd714102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.404+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 shutdown_connections 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:39.404+0000 7fd719e00700 1 -- 172.21.15.117:0/121872414 wait complete. 2024-04-20T10:55:40.089 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-20T10:55:40.103 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:55:40.103 INFO:teuthology.orchestra.run.smithi117.stdout:Creating mgr... 2024-04-20T10:55:40.104 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-20T10:55:40.105 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-20T10:55:40.106 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:8443 ... 2024-04-20T10:55:40.553 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mgr.smithi117.ivnklj 2024-04-20T10:55:40.553 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to reset failed state of unit ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mgr.smithi117.ivnklj.service: Unit ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mgr.smithi117.ivnklj.service not loaded. 2024-04-20T10:55:40.808 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968.target.wants/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mgr.smithi117.ivnklj.service → /etc/systemd/system/ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@.service. 2024-04-20T10:55:40.846 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:40 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/121872414' entity='client.admin' 2024-04-20T10:55:41.641 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T10:55:41.641 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-04-20T10:55:41.657 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T10:55:41.657 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-04-20T10:55:41.657 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-04-20T10:55:41.657 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T10:55:41.676 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T10:55:41.676 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-04-20T10:55:41.692 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T10:55:41.693 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-04-20T10:55:41.693 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-04-20T10:55:41.693 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-04-20T10:55:41.693 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr to start... 2024-04-20T10:55:41.693 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr... 2024-04-20T10:55:42.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:42 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/276597002' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T10:55:43.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:43.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:55:43.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "59ecfd22-ff04-11ee-bc92-c7b262605968", 2024-04-20T10:55:43.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T10:55:43.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "smithi117" 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-20T10:55:43.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T10:55:43.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:43.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T10:55:43.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:43.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T10:55:43.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T10:55:43.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T10:55:43.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T10:55:43.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T10:55:31.778578+0000", 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.292+0000 7f1f25000700 1 Processor -- start 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.292+0000 7f1f25000700 1 -- start start 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.292+0000 7f1f25000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f200791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.292+0000 7f1f25000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200797c0 con 0x7f1f2007ad90 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.293+0000 7f1f1de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f200791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.293+0000 7f1f1de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f200791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52110/0 (socket says 172.21.15.117:52110) 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.293+0000 7f1f1de00700 1 -- 172.21.15.117:0/1550835779 learned_addr learned my addr 172.21.15.117:0/1550835779 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.293+0000 7f1f1de00700 1 -- 172.21.15.117:0/1550835779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20079900 con 0x7f1f2007ad90 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.294+0000 7f1f1de00700 1 --2- 172.21.15.117:0/1550835779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f200791f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1f100098a0 tx=0x7f1f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb5a8350197867c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.294+0000 7f1f1ca00700 1 -- 172.21.15.117:0/1550835779 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f10004180 con 0x7f1f2007ad90 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.294+0000 7f1f1ca00700 1 -- 172.21.15.117:0/1550835779 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1f100042e0 con 0x7f1f2007ad90 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.294+0000 7f1f1ca00700 1 -- 172.21.15.117:0/1550835779 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f10011a70 con 0x7f1f2007ad90 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.295+0000 7f1f25000700 1 -- 172.21.15.117:0/1550835779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 msgr2=0x7f1f200791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.295+0000 7f1f25000700 1 --2- 172.21.15.117:0/1550835779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f200791f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1f100098a0 tx=0x7f1f10009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.295+0000 7f1f25000700 1 -- 172.21.15.117:0/1550835779 shutdown_connections 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.295+0000 7f1f25000700 1 --2- 172.21.15.117:0/1550835779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f200791f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.295+0000 7f1f25000700 1 -- 172.21.15.117:0/1550835779 >> 172.21.15.117:0/1550835779 conn(0x7f1f20101ba0 msgr2=0x7f1f20103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.295+0000 7f1f25000700 1 -- 172.21.15.117:0/1550835779 shutdown_connections 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.295+0000 7f1f25000700 1 -- 172.21.15.117:0/1550835779 wait complete. 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.296+0000 7f1f25000700 1 Processor -- start 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.296+0000 7f1f25000700 1 -- start start 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.297+0000 7f1f25000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f2019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.297+0000 7f1f25000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2019ca90 con 0x7f1f2007ad90 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.297+0000 7f1f1de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f2019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.297+0000 7f1f1de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f2019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52122/0 (socket says 172.21.15.117:52122) 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.297+0000 7f1f1de00700 1 -- 172.21.15.117:0/276597002 learned_addr learned my addr 172.21.15.117:0/276597002 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.297+0000 7f1f1de00700 1 -- 172.21.15.117:0/276597002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10009550 con 0x7f1f2007ad90 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.298+0000 7f1f1de00700 1 --2- 172.21.15.117:0/276597002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f2019c550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f10003df0 tx=0x7f1f100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.298+0000 7f1f16a00700 1 -- 172.21.15.117:0/276597002 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f10003a00 con 0x7f1f2007ad90 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.298+0000 7f1f16a00700 1 -- 172.21.15.117:0/276597002 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1f10003b60 con 0x7f1f2007ad90 2024-04-20T10:55:43.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.298+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2019cc90 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.298+0000 7f1f16a00700 1 -- 172.21.15.117:0/276597002 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f10023640 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.298+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2019d130 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.298+0000 7f1f16a00700 1 -- 172.21.15.117:0/276597002 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1f10023c20 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.299+0000 7f1f16a00700 1 -- 172.21.15.117:0/276597002 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f10019710 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.299+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2010c920 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.302+0000 7f1f16a00700 1 -- 172.21.15.117:0/276597002 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f1f1002ce40 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.381+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1f2010cae0 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.382+0000 7f1f16a00700 1 -- 172.21.15.117:0/276597002 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f1f1001bac0 con 0x7f1f2007ad90 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.383+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 msgr2=0x7f1f2019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.383+0000 7f1f25000700 1 --2- 172.21.15.117:0/276597002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f2019c550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f10003df0 tx=0x7f1f100047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.383+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 shutdown_connections 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.383+0000 7f1f25000700 1 --2- 172.21.15.117:0/276597002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2007ad90 0x7f1f2019c550 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.383+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 >> 172.21.15.117:0/276597002 conn(0x7f1f20101ba0 msgr2=0x7f1f201027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.383+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 shutdown_connections 2024-04-20T10:55:43.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:42.383+0000 7f1f25000700 1 -- 172.21.15.117:0/276597002 wait complete. 2024-04-20T10:55:43.112 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (1/15)... 2024-04-20T10:55:46.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:45 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1822410625' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T10:55:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:55:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "59ecfd22-ff04-11ee-bc92-c7b262605968", 2024-04-20T10:55:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "smithi117" 2024-04-20T10:55:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T10:55:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T10:55:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T10:55:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T10:55:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T10:55:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T10:55:31.778578+0000", 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:55:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T10:55:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:55:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.728+0000 7fe8c2a00700 1 Processor -- start 2024-04-20T10:55:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.728+0000 7fe8c2a00700 1 -- start start 2024-04-20T10:55:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.729+0000 7fe8c2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc079230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.729+0000 7fe8c2a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc079800 con 0x7fe8bc07add0 2024-04-20T10:55:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.729+0000 7fe8bbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc079230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.729+0000 7fe8bbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc079230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45384/0 (socket says 172.21.15.117:45384) 2024-04-20T10:55:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.729+0000 7fe8bbe00700 1 -- 172.21.15.117:0/1517603514 learned_addr learned my addr 172.21.15.117:0/1517603514 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.730+0000 7fe8bbe00700 1 -- 172.21.15.117:0/1517603514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc079940 con 0x7fe8bc07add0 2024-04-20T10:55:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.730+0000 7fe8bbe00700 1 --2- 172.21.15.117:0/1517603514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc079230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0098a0 tx=0x7fe8ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7864703f327ab498 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.730+0000 7fe8baa00700 1 -- 172.21.15.117:0/1517603514 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8ac004180 con 0x7fe8bc07add0 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.730+0000 7fe8baa00700 1 -- 172.21.15.117:0/1517603514 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe8ac0042e0 con 0x7fe8bc07add0 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.730+0000 7fe8baa00700 1 -- 172.21.15.117:0/1517603514 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8ac011a70 con 0x7fe8bc07add0 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.731+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1517603514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 msgr2=0x7fe8bc079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.731+0000 7fe8c2a00700 1 --2- 172.21.15.117:0/1517603514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc079230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0098a0 tx=0x7fe8ac009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.731+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1517603514 shutdown_connections 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.731+0000 7fe8c2a00700 1 --2- 172.21.15.117:0/1517603514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc079230 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.731+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1517603514 >> 172.21.15.117:0/1517603514 conn(0x7fe8bc101ba0 msgr2=0x7fe8bc103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.731+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1517603514 shutdown_connections 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.731+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1517603514 wait complete. 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.732+0000 7fe8c2a00700 1 Processor -- start 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.732+0000 7fe8c2a00700 1 -- start start 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.733+0000 7fe8c2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.733+0000 7fe8c2a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc19cae0 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.733+0000 7fe8bbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.733+0000 7fe8bbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc19c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45394/0 (socket says 172.21.15.117:45394) 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.733+0000 7fe8bbe00700 1 -- 172.21.15.117:0/1822410625 learned_addr learned my addr 172.21.15.117:0/1822410625 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.733+0000 7fe8bbe00700 1 -- 172.21.15.117:0/1822410625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac009550 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.734+0000 7fe8bbe00700 1 --2- 172.21.15.117:0/1822410625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc19c5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac006af0 tx=0x7fe8ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.734+0000 7fe8b8c00700 1 -- 172.21.15.117:0/1822410625 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8ac003a00 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.734+0000 7fe8b8c00700 1 -- 172.21.15.117:0/1822410625 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe8ac003b60 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.734+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc19cce0 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.734+0000 7fe8b8c00700 1 -- 172.21.15.117:0/1822410625 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8ac023750 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.734+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc19d180 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.734+0000 7fe8b8c00700 1 -- 172.21.15.117:0/1822410625 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe8ac023c40 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.735+0000 7fe8b8c00700 1 -- 172.21.15.117:0/1822410625 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe8ac02ce40 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.735+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc10c8e0 con 0x7fe8bc07add0 2024-04-20T10:55:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.738+0000 7fe8b8c00700 1 -- 172.21.15.117:0/1822410625 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fe8ac01bbe0 con 0x7fe8bc07add0 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.816+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe8bc10cad0 con 0x7fe8bc07add0 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.817+0000 7fe8b8c00700 1 -- 172.21.15.117:0/1822410625 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fe8ac01b440 con 0x7fe8bc07add0 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.818+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 msgr2=0x7fe8bc19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.818+0000 7fe8c2a00700 1 --2- 172.21.15.117:0/1822410625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc19c5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac006af0 tx=0x7fe8ac0047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.818+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 shutdown_connections 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.818+0000 7fe8c2a00700 1 --2- 172.21.15.117:0/1822410625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc07add0 0x7fe8bc19c5a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.818+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 >> 172.21.15.117:0/1822410625 conn(0x7fe8bc101ba0 msgr2=0x7fe8bc1027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.818+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 shutdown_connections 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:45.818+0000 7fe8c2a00700 1 -- 172.21.15.117:0/1822410625 wait complete. 2024-04-20T10:55:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (2/15)... 2024-04-20T10:55:49.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:49 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1967736069' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T10:55:49.969 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:49.969 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:55:49.969 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "59ecfd22-ff04-11ee-bc92-c7b262605968", 2024-04-20T10:55:49.969 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T10:55:49.969 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T10:55:49.969 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T10:55:49.969 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "smithi117" 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T10:55:49.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T10:55:49.971 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:49.972 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T10:55:49.973 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T10:55:31.778578+0000", 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.150+0000 7fb8a8400700 1 Processor -- start 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.150+0000 7fb8a8400700 1 -- start start 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.150+0000 7fb8a8400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a0079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.150+0000 7fb8a8400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a00797e0 con 0x7fb8a007adb0 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.151+0000 7fb8a5a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a0079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.151+0000 7fb8a5a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a0079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45404/0 (socket says 172.21.15.117:45404) 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.151+0000 7fb8a5a00700 1 -- 172.21.15.117:0/1765516967 learned_addr learned my addr 172.21.15.117:0/1765516967 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:49.974 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.151+0000 7fb8a5a00700 1 -- 172.21.15.117:0/1765516967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a0079920 con 0x7fb8a007adb0 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.151+0000 7fb8a5a00700 1 --2- 172.21.15.117:0/1765516967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a0079210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb8940098a0 tx=0x7fb894009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693364fcb58a1aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.152+0000 7fb89fe00700 1 -- 172.21.15.117:0/1765516967 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb894004180 con 0x7fb8a007adb0 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.152+0000 7fb89fe00700 1 -- 172.21.15.117:0/1765516967 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb8940042e0 con 0x7fb8a007adb0 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.152+0000 7fb89fe00700 1 -- 172.21.15.117:0/1765516967 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb894011a70 con 0x7fb8a007adb0 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.152+0000 7fb8a8400700 1 -- 172.21.15.117:0/1765516967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 msgr2=0x7fb8a0079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.152+0000 7fb8a8400700 1 --2- 172.21.15.117:0/1765516967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a0079210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb8940098a0 tx=0x7fb894009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.153+0000 7fb8a8400700 1 -- 172.21.15.117:0/1765516967 shutdown_connections 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.153+0000 7fb8a8400700 1 --2- 172.21.15.117:0/1765516967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a0079210 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.153+0000 7fb8a8400700 1 -- 172.21.15.117:0/1765516967 >> 172.21.15.117:0/1765516967 conn(0x7fb8a0101ba0 msgr2=0x7fb8a0103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.153+0000 7fb8a8400700 1 -- 172.21.15.117:0/1765516967 shutdown_connections 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.153+0000 7fb8a8400700 1 -- 172.21.15.117:0/1765516967 wait complete. 2024-04-20T10:55:49.975 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.154+0000 7fb8a8400700 1 Processor -- start 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.154+0000 7fb8a8400700 1 -- start start 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.154+0000 7fb8a8400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.154+0000 7fb8a8400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a019cae0 con 0x7fb8a007adb0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.155+0000 7fb8a5a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.155+0000 7fb8a5a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45410/0 (socket says 172.21.15.117:45410) 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.155+0000 7fb8a5a00700 1 -- 172.21.15.117:0/1967736069 learned_addr learned my addr 172.21.15.117:0/1967736069 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.155+0000 7fb8a5a00700 1 -- 172.21.15.117:0/1967736069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb894009550 con 0x7fb8a007adb0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.155+0000 7fb8a5a00700 1 --2- 172.21.15.117:0/1967736069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a019c5a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb894006af0 tx=0x7fb8940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.156+0000 7fb89e000700 1 -- 172.21.15.117:0/1967736069 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb894003a00 con 0x7fb8a007adb0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.156+0000 7fb89e000700 1 -- 172.21.15.117:0/1967736069 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb894003b60 con 0x7fb8a007adb0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.156+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a019cce0 con 0x7fb8a007adb0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.156+0000 7fb89e000700 1 -- 172.21.15.117:0/1967736069 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb894023750 con 0x7fb8a007adb0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.156+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a019d180 con 0x7fb8a007adb0 2024-04-20T10:55:49.976 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.156+0000 7fb89e000700 1 -- 172.21.15.117:0/1967736069 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb894023c90 con 0x7fb8a007adb0 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.156+0000 7fb89e000700 1 -- 172.21.15.117:0/1967736069 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb894019710 con 0x7fb8a007adb0 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.157+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a010c900 con 0x7fb8a007adb0 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.161+0000 7fb89e000700 1 -- 172.21.15.117:0/1967736069 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fb89401bbe0 con 0x7fb8a007adb0 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.241+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb8a010caf0 con 0x7fb8a007adb0 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.242+0000 7fb89e000700 1 -- 172.21.15.117:0/1967736069 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7fb8940198d0 con 0x7fb8a007adb0 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.244+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 msgr2=0x7fb8a019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.244+0000 7fb8a8400700 1 --2- 172.21.15.117:0/1967736069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a019c5a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb894006af0 tx=0x7fb8940047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.244+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 shutdown_connections 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.244+0000 7fb8a8400700 1 --2- 172.21.15.117:0/1967736069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a007adb0 0x7fb8a019c5a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.244+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 >> 172.21.15.117:0/1967736069 conn(0x7fb8a0101ba0 msgr2=0x7fb8a0102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.244+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 shutdown_connections 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:49.244+0000 7fb8a8400700 1 -- 172.21.15.117:0/1967736069 wait complete. 2024-04-20T10:55:49.977 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (3/15)... 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: Activating manager daemon smithi117.ivnklj 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: mgrmap e2: smithi117.ivnklj(active, starting, since 0.0111158s) 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:55:50.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T10:55:50.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T10:55:50.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: Manager daemon smithi117.ivnklj is now available 2024-04-20T10:55:50.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:55:50.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/trash_purge_schedule"}]: dispatch 2024-04-20T10:55:50.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' 2024-04-20T10:55:50.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:50 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' 2024-04-20T10:55:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:51 smithi117 ceph-mon[15659]: from='mgr.14100 172.21.15.117:0/3855529356' entity='mgr.smithi117.ivnklj' 2024-04-20T10:55:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:51 smithi117 ceph-mon[15659]: mgrmap e3: smithi117.ivnklj(active, since 1.02267s) 2024-04-20T10:55:53.496 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:53 smithi117 ceph-mon[15659]: mgrmap e4: smithi117.ivnklj(active, since 2s) 2024-04-20T10:55:53.497 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:53 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1051301551' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T10:55:53.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:53.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:55:53.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "59ecfd22-ff04-11ee-bc92-c7b262605968", 2024-04-20T10:55:53.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T10:55:53.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T10:55:53.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T10:55:53.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T10:55:53.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:53.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "smithi117" 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:53.516 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T10:55:53.517 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T10:55:53.518 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T10:55:31.778578+0000", 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T10:55:53.519 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:55:53.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.562+0000 7f5364400700 1 Processor -- start 2024-04-20T10:55:53.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.562+0000 7f5364400700 1 -- start start 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.562+0000 7f5364400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.562+0000 7f5364400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0747b0 con 0x7f535c107020 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.563+0000 7f5361a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.563+0000 7f5361a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45478/0 (socket says 172.21.15.117:45478) 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.563+0000 7f5361a00700 1 -- 172.21.15.117:0/1425407366 learned_addr learned my addr 172.21.15.117:0/1425407366 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.563+0000 7f5361a00700 1 -- 172.21.15.117:0/1425407366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c109950 con 0x7f535c107020 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.564+0000 7f5361a00700 1 --2- 172.21.15.117:0/1425407366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c109410 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f53500098a0 tx=0x7f5350009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51acf21c9be3ec1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.564+0000 7f535be00700 1 -- 172.21.15.117:0/1425407366 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5350004180 con 0x7f535c107020 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.564+0000 7f535be00700 1 -- 172.21.15.117:0/1425407366 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f53500042e0 con 0x7f535c107020 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.564+0000 7f535be00700 1 -- 172.21.15.117:0/1425407366 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5350011a70 con 0x7f535c107020 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.565+0000 7f5364400700 1 -- 172.21.15.117:0/1425407366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 msgr2=0x7f535c109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.565+0000 7f5364400700 1 --2- 172.21.15.117:0/1425407366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c109410 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f53500098a0 tx=0x7f5350009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.565+0000 7f5364400700 1 -- 172.21.15.117:0/1425407366 shutdown_connections 2024-04-20T10:55:53.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.565+0000 7f5364400700 1 --2- 172.21.15.117:0/1425407366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c109410 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.565+0000 7f5364400700 1 -- 172.21.15.117:0/1425407366 >> 172.21.15.117:0/1425407366 conn(0x7f535c100a50 msgr2=0x7f535c102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.565+0000 7f5364400700 1 -- 172.21.15.117:0/1425407366 shutdown_connections 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.565+0000 7f5364400700 1 -- 172.21.15.117:0/1425407366 wait complete. 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.566+0000 7f5364400700 1 Processor -- start 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.566+0000 7f5364400700 1 -- start start 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.567+0000 7f5364400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.567+0000 7f5364400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c19ca70 con 0x7f535c107020 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.567+0000 7f5361a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.567+0000 7f5361a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45488/0 (socket says 172.21.15.117:45488) 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.567+0000 7f5361a00700 1 -- 172.21.15.117:0/1051301551 learned_addr learned my addr 172.21.15.117:0/1051301551 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.567+0000 7f5361a00700 1 -- 172.21.15.117:0/1051301551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5350009550 con 0x7f535c107020 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.568+0000 7f5361a00700 1 --2- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c19c530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5350011a20 tx=0x7f53500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.568+0000 7f535a000700 1 -- 172.21.15.117:0/1051301551 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5350003b10 con 0x7f535c107020 2024-04-20T10:55:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.568+0000 7f535a000700 1 -- 172.21.15.117:0/1051301551 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f535001b8c0 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.568+0000 7f535a000700 1 -- 172.21.15.117:0/1051301551 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53500237b0 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.568+0000 7f5364400700 1 -- 172.21.15.117:0/1051301551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c19cc70 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.568+0000 7f5364400700 1 -- 172.21.15.117:0/1051301551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c19d110 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.569+0000 7f535a000700 1 -- 172.21.15.117:0/1051301551 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7f535001b440 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.569+0000 7f534f400700 1 -- 172.21.15.117:0/1051301551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53380052f0 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.570+0000 7f535a000700 1 --2- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f53440386e0 0x7f534403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.570+0000 7f535a000700 1 -- 172.21.15.117:0/1051301551 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f535004caa0 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.570+0000 7f5361000700 1 --2- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f53440386e0 0x7f534403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.571+0000 7f5361000700 1 --2- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f53440386e0 0x7f534403aba0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5348006fd0 tx=0x7f5348006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.576+0000 7f535a000700 1 -- 172.21.15.117:0/1051301551 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f535001b6f0 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.874+0000 7f534f400700 1 -- 172.21.15.117:0/1051301551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5338005160 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.875+0000 7f535a000700 1 -- 172.21.15.117:0/1051301551 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f535002b7c0 con 0x7f535c107020 2024-04-20T10:55:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.879+0000 7f534f400700 1 -- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f53440386e0 msgr2=0x7f534403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.879+0000 7f534f400700 1 --2- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f53440386e0 0x7f534403aba0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5348006fd0 tx=0x7f5348006e40 comp rx=0 tx=0).stop 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.879+0000 7f534f400700 1 -- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 msgr2=0x7f535c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.879+0000 7f534f400700 1 --2- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c19c530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5350011a20 tx=0x7f53500047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.880+0000 7f534f400700 1 -- 172.21.15.117:0/1051301551 shutdown_connections 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.880+0000 7f534f400700 1 --2- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f53440386e0 0x7f534403aba0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.880+0000 7f534f400700 1 --2- 172.21.15.117:0/1051301551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c107020 0x7f535c19c530 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.880+0000 7f534f400700 1 -- 172.21.15.117:0/1051301551 >> 172.21.15.117:0/1051301551 conn(0x7f535c100a50 msgr2=0x7f535c101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.880+0000 7f534f400700 1 -- 172.21.15.117:0/1051301551 shutdown_connections 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:52.880+0000 7f534f400700 1 -- 172.21.15.117:0/1051301551 wait complete. 2024-04-20T10:55:53.528 INFO:teuthology.orchestra.run.smithi117.stdout:mgr is available 2024-04-20T10:55:54.780 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:54 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1495116096' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T10:55:55.048 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:55.048 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T10:55:55.048 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout fsid = 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T10:55:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T10:55:55.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T10:55:55.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.143+0000 7f291f200700 1 Processor -- start 2024-04-20T10:55:55.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.143+0000 7f291f200700 1 -- start start 2024-04-20T10:55:55.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.144+0000 7f291f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f2920079230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:55.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.144+0000 7f291f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920079800 con 0x7f292007add0 2024-04-20T10:55:55.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.144+0000 7f291de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f2920079230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.144+0000 7f291de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f2920079230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38942/0 (socket says 172.21.15.117:38942) 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.144+0000 7f291de00700 1 -- 172.21.15.117:0/2384147034 learned_addr learned my addr 172.21.15.117:0/2384147034 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.144+0000 7f291de00700 1 -- 172.21.15.117:0/2384147034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920079940 con 0x7f292007add0 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.145+0000 7f291de00700 1 --2- 172.21.15.117:0/2384147034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f2920079230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f29100098a0 tx=0x7f2910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8f3c739958496f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.145+0000 7f291ca00700 1 -- 172.21.15.117:0/2384147034 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2910004180 con 0x7f292007add0 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.145+0000 7f291ca00700 1 -- 172.21.15.117:0/2384147034 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f29100042e0 con 0x7f292007add0 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.145+0000 7f291ca00700 1 -- 172.21.15.117:0/2384147034 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2910011a70 con 0x7f292007add0 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.146+0000 7f291f200700 1 -- 172.21.15.117:0/2384147034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 msgr2=0x7f2920079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.146+0000 7f291f200700 1 --2- 172.21.15.117:0/2384147034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f2920079230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f29100098a0 tx=0x7f2910009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.146+0000 7f291f200700 1 -- 172.21.15.117:0/2384147034 shutdown_connections 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.146+0000 7f291f200700 1 --2- 172.21.15.117:0/2384147034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f2920079230 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.146+0000 7f291f200700 1 -- 172.21.15.117:0/2384147034 >> 172.21.15.117:0/2384147034 conn(0x7f2920101a30 msgr2=0x7f2920103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.146+0000 7f291f200700 1 -- 172.21.15.117:0/2384147034 shutdown_connections 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.146+0000 7f291f200700 1 -- 172.21.15.117:0/2384147034 wait complete. 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.147+0000 7f291f200700 1 Processor -- start 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.147+0000 7f291f200700 1 -- start start 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.148+0000 7f291f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f292019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.148+0000 7f291f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292019c860 con 0x7f292007add0 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.148+0000 7f291de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f292019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.148+0000 7f291de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f292019c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38946/0 (socket says 172.21.15.117:38946) 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.148+0000 7f291de00700 1 -- 172.21.15.117:0/1495116096 learned_addr learned my addr 172.21.15.117:0/1495116096 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.148+0000 7f291de00700 1 -- 172.21.15.117:0/1495116096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910009550 con 0x7f292007add0 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.149+0000 7f291de00700 1 --2- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f292019c320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2910011710 tx=0x7f29100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.149+0000 7f2916a00700 1 -- 172.21.15.117:0/1495116096 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2910003b10 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.149+0000 7f2916a00700 1 -- 172.21.15.117:0/1495116096 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f291001b8c0 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.149+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292019ca60 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.149+0000 7f2916a00700 1 -- 172.21.15.117:0/1495116096 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2910019710 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.149+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292019cf00 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.150+0000 7f2916a00700 1 -- 172.21.15.117:0/1495116096 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7f291001b440 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.150+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292010c750 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.151+0000 7f2916a00700 1 --2- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f29040387a0 0x7f290403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.151+0000 7f2916a00700 1 -- 172.21.15.117:0/1495116096 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f291004ca60 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.151+0000 7f291d400700 1 --2- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f29040387a0 0x7f290403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.152+0000 7f291d400700 1 --2- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f29040387a0 0x7f290403ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f290c006fd0 tx=0x7f290c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.156+0000 7f2916a00700 1 -- 172.21.15.117:0/1495116096 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f291001b6f0 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.372+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f292010c9c0 con 0x7f292007add0 2024-04-20T10:55:55.059 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.373+0000 7f2916a00700 1 -- 172.21.15.117:0/1495116096 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f2910018350 con 0x7f292007add0 2024-04-20T10:55:55.060 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f29040387a0 msgr2=0x7f290403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:55.060 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 --2- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f29040387a0 0x7f290403ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f290c006fd0 tx=0x7f290c006e40 comp rx=0 tx=0).stop 2024-04-20T10:55:55.060 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 msgr2=0x7f292019c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:55.060 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 --2- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f292019c320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2910011710 tx=0x7f29100047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:55.060 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 shutdown_connections 2024-04-20T10:55:55.060 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 --2- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f29040387a0 0x7f290403ac60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:55.060 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 --2- 172.21.15.117:0/1495116096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f292007add0 0x7f292019c320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:55.061 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 >> 172.21.15.117:0/1495116096 conn(0x7f2920101a30 msgr2=0x7f2920102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:55.061 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 shutdown_connections 2024-04-20T10:55:55.061 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:54.376+0000 7f291f200700 1 -- 172.21.15.117:0/1495116096 wait complete. 2024-04-20T10:55:55.061 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling cephadm module... 2024-04-20T10:55:56.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:56 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3429224588' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-20T10:55:56.795 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.667+0000 7f362a800700 1 Processor -- start 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.667+0000 7f362a800700 1 -- start start 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.667+0000 7f362a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f362410d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.667+0000 7f362a800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362410db20 con 0x7f362410d130 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.667+0000 7f3623400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f362410d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.668+0000 7f3623400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f362410d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38956/0 (socket says 172.21.15.117:38956) 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.668+0000 7f3623400700 1 -- 172.21.15.117:0/2095662642 learned_addr learned my addr 172.21.15.117:0/2095662642 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.668+0000 7f3623400700 1 -- 172.21.15.117:0/2095662642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362410dda0 con 0x7f362410d130 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.668+0000 7f3623400700 1 --2- 172.21.15.117:0/2095662642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f362410d550 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f36140098a0 tx=0x7f3614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef1f5fdc2d276d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f3622a00700 1 -- 172.21.15.117:0/2095662642 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3614004180 con 0x7f362410d130 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f3622a00700 1 -- 172.21.15.117:0/2095662642 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f36140042e0 con 0x7f362410d130 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f3622a00700 1 -- 172.21.15.117:0/2095662642 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3614011a70 con 0x7f362410d130 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f362a800700 1 -- 172.21.15.117:0/2095662642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 msgr2=0x7f362410d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:56.796 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f362a800700 1 --2- 172.21.15.117:0/2095662642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f362410d550 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f36140098a0 tx=0x7f3614009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f362a800700 1 -- 172.21.15.117:0/2095662642 shutdown_connections 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f362a800700 1 --2- 172.21.15.117:0/2095662642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f362410d550 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f362a800700 1 -- 172.21.15.117:0/2095662642 >> 172.21.15.117:0/2095662642 conn(0x7f362407bbf0 msgr2=0x7f362407e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f362a800700 1 -- 172.21.15.117:0/2095662642 shutdown_connections 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.669+0000 7f362a800700 1 -- 172.21.15.117:0/2095662642 wait complete. 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.670+0000 7f362a800700 1 Processor -- start 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.671+0000 7f362a800700 1 -- start start 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.671+0000 7f362a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f36241a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.671+0000 7f362a800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36241a0e00 con 0x7f362410d130 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.671+0000 7f3623400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f36241a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.671+0000 7f3623400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f36241a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38958/0 (socket says 172.21.15.117:38958) 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.671+0000 7f3623400700 1 -- 172.21.15.117:0/3429224588 learned_addr learned my addr 172.21.15.117:0/3429224588 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.672+0000 7f3623400700 1 -- 172.21.15.117:0/3429224588 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614009550 con 0x7f362410d130 2024-04-20T10:55:56.797 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.672+0000 7f3623400700 1 --2- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f36241a08c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3614011710 tx=0x7f36140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.672+0000 7f3620c00700 1 -- 172.21.15.117:0/3429224588 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3614003b10 con 0x7f362410d130 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.672+0000 7f3620c00700 1 -- 172.21.15.117:0/3429224588 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f361401b8c0 con 0x7f362410d130 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.672+0000 7f3620c00700 1 -- 172.21.15.117:0/3429224588 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3614019770 con 0x7f362410d130 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.672+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36241a1000 con 0x7f362410d130 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.672+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36241a14a0 con 0x7f362410d130 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.673+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3624110fd0 con 0x7f362410d130 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.673+0000 7f3620c00700 1 -- 172.21.15.117:0/3429224588 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7f361401ba30 con 0x7f362410d130 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.674+0000 7f3620c00700 1 --2- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f36080386d0 0x7f360803ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.674+0000 7f3620c00700 1 -- 172.21.15.117:0/3429224588 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f361404c870 con 0x7f362410d130 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.674+0000 7f361aa00700 1 --2- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f36080386d0 0x7f360803ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:56.798 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.675+0000 7f361aa00700 1 --2- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f36080386d0 0x7f360803ab90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f360c006fd0 tx=0x7f360c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.679+0000 7f3620c00700 1 -- 172.21.15.117:0/3429224588 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f361402cce0 con 0x7f362410d130 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:55.958+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f36241a3d90 con 0x7f362410d130 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.016+0000 7f3620c00700 1 -- 172.21.15.117:0/3429224588 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f361404ecd0 con 0x7f362410d130 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.018+0000 7f3620c00700 1 -- 172.21.15.117:0/3429224588 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f361400eb10 con 0x7f362410d130 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f36080386d0 msgr2=0x7f360803ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 --2- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f36080386d0 0x7f360803ab90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f360c006fd0 tx=0x7f360c006e40 comp rx=0 tx=0).stop 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 msgr2=0x7f36241a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 --2- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f36241a08c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3614011710 tx=0x7f36140047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 shutdown_connections 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 --2- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7f36080386d0 0x7f360803ab90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 --2- 172.21.15.117:0/3429224588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362410d130 0x7f36241a08c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 >> 172.21.15.117:0/3429224588 conn(0x7f362407bbf0 msgr2=0x7f362407c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 shutdown_connections 2024-04-20T10:55:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:56.022+0000 7f362a800700 1 -- 172.21.15.117:0/3429224588 wait complete. 2024-04-20T10:55:57.195 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:57 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3429224588' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-20T10:55:57.195 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:57 smithi117 ceph-mon[15659]: mgrmap e5: smithi117.ivnklj(active, since 5s) 2024-04-20T10:55:58.239 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:55:58 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2034618257' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T10:55:58.249 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "active_name": "smithi117.ivnklj", 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.347+0000 7fe46d000700 1 Processor -- start 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.347+0000 7fe46d000700 1 -- start start 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.347+0000 7fe46d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe468079230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:58.251 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.347+0000 7fe46d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe468079800 con 0x7fe46807add0 2024-04-20T10:55:58.252 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.348+0000 7fe465e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe468079230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:58.252 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.348+0000 7fe465e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe468079230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38994/0 (socket says 172.21.15.117:38994) 2024-04-20T10:55:58.252 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.348+0000 7fe465e00700 1 -- 172.21.15.117:0/3227154543 learned_addr learned my addr 172.21.15.117:0/3227154543 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:58.252 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.348+0000 7fe465e00700 1 -- 172.21.15.117:0/3227154543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe468079940 con 0x7fe46807add0 2024-04-20T10:55:58.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.349+0000 7fe465e00700 1 --2- 172.21.15.117:0/3227154543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe468079230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4580098a0 tx=0x7fe458009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d23ec7eae697462 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:58.257 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.349+0000 7fe464a00700 1 -- 172.21.15.117:0/3227154543 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe458004180 con 0x7fe46807add0 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.349+0000 7fe464a00700 1 -- 172.21.15.117:0/3227154543 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe4580042e0 con 0x7fe46807add0 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.349+0000 7fe464a00700 1 -- 172.21.15.117:0/3227154543 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe458011a70 con 0x7fe46807add0 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.350+0000 7fe46d000700 1 -- 172.21.15.117:0/3227154543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 msgr2=0x7fe468079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.350+0000 7fe46d000700 1 --2- 172.21.15.117:0/3227154543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe468079230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4580098a0 tx=0x7fe458009bb0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.350+0000 7fe46d000700 1 -- 172.21.15.117:0/3227154543 shutdown_connections 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.350+0000 7fe46d000700 1 --2- 172.21.15.117:0/3227154543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe468079230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.350+0000 7fe46d000700 1 -- 172.21.15.117:0/3227154543 >> 172.21.15.117:0/3227154543 conn(0x7fe468101ba0 msgr2=0x7fe468103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.350+0000 7fe46d000700 1 -- 172.21.15.117:0/3227154543 shutdown_connections 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.350+0000 7fe46d000700 1 -- 172.21.15.117:0/3227154543 wait complete. 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.351+0000 7fe46d000700 1 Processor -- start 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.351+0000 7fe46d000700 1 -- start start 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.351+0000 7fe46d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe46819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:58.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.351+0000 7fe46d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46819cae0 con 0x7fe46807add0 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.352+0000 7fe465e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe46819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.352+0000 7fe465e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe46819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39000/0 (socket says 172.21.15.117:39000) 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.352+0000 7fe465e00700 1 -- 172.21.15.117:0/2034618257 learned_addr learned my addr 172.21.15.117:0/2034618257 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.352+0000 7fe465e00700 1 -- 172.21.15.117:0/2034618257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458009550 con 0x7fe46807add0 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.352+0000 7fe465e00700 1 --2- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe46819c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe458006af0 tx=0x7fe4580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.353+0000 7fe45ea00700 1 -- 172.21.15.117:0/2034618257 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe458003a00 con 0x7fe46807add0 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.353+0000 7fe45ea00700 1 -- 172.21.15.117:0/2034618257 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe458003b60 con 0x7fe46807add0 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.353+0000 7fe45ea00700 1 -- 172.21.15.117:0/2034618257 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe458023750 con 0x7fe46807add0 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.353+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46819cce0 con 0x7fe46807add0 2024-04-20T10:55:58.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.353+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46819d180 con 0x7fe46807add0 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.354+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46810c8e0 con 0x7fe46807add0 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.354+0000 7fe45ea00700 1 -- 172.21.15.117:0/2034618257 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fe458023940 con 0x7fe46807add0 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.355+0000 7fe45ea00700 1 --2- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fe44c038770 0x7fe44c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.355+0000 7fe45ea00700 1 -- 172.21.15.117:0/2034618257 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe45804c920 con 0x7fe46807add0 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.355+0000 7fe465400700 1 -- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fe44c038770 msgr2=0x7fe44c03ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1432319225 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.355+0000 7fe465400700 1 --2- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fe44c038770 0x7fe44c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.360+0000 7fe45ea00700 1 -- 172.21.15.117:0/2034618257 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe45802b760 con 0x7fe46807add0 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.556+0000 7fe465400700 1 -- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fe44c038770 msgr2=0x7fe44c03ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1432319225 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.556+0000 7fe465400700 1 --2- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fe44c038770 0x7fe44c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.637+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe46819fa70 con 0x7fe46807add0 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.637+0000 7fe45ea00700 1 -- 172.21.15.117:0/2034618257 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fe45800b3e0 con 0x7fe46807add0 2024-04-20T10:55:58.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.640+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fe44c038770 msgr2=0x7fe44c03ac30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.640+0000 7fe46d000700 1 --2- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fe44c038770 0x7fe44c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.640+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 msgr2=0x7fe46819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.640+0000 7fe46d000700 1 --2- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe46819c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe458006af0 tx=0x7fe4580047c0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.640+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 shutdown_connections 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.641+0000 7fe46d000700 1 --2- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fe44c038770 0x7fe44c03ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.641+0000 7fe46d000700 1 --2- 172.21.15.117:0/2034618257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46807add0 0x7fe46819c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.641+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 >> 172.21.15.117:0/2034618257 conn(0x7fe468101ba0 msgr2=0x7fe4681027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.641+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 shutdown_connections 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:57.641+0000 7fe46d000700 1 -- 172.21.15.117:0/2034618257 wait complete. 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for the mgr to restart... 2024-04-20T10:55:58.261 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr epoch 5... 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: Active manager daemon smithi117.ivnklj restarted 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: Activating manager daemon smithi117.ivnklj 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: osdmap e2: 0 total, 0 up, 0 in 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: mgrmap e6: smithi117.ivnklj(active, starting, since 0.0072412s) 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: Manager daemon smithi117.ivnklj is now available 2024-04-20T10:56:05.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:05.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:04 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:05 smithi117 ceph-mon[15659]: Found migration_current of "None". Setting to last migration. 2024-04-20T10:56:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:05 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:56:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:05 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:56:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:05 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:56:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:05 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/trash_purge_schedule"}]: dispatch 2024-04-20T10:56:06.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:05 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:06.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:05 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:06.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:05 smithi117 ceph-mon[15659]: mgrmap e7: smithi117.ivnklj(active, since 1.01836s) 2024-04-20T10:56:06.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:56:06.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.825+0000 7fcf99600700 1 Processor -- start 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.825+0000 7fcf99600700 1 -- start start 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.826+0000 7fcf99600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf941052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.826+0000 7fcf99600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94074730 con 0x7fcf94104e90 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.826+0000 7fcf92a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf941052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.826+0000 7fcf92a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf941052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39008/0 (socket says 172.21.15.117:39008) 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.826+0000 7fcf92a00700 1 -- 172.21.15.117:0/1532691901 learned_addr learned my addr 172.21.15.117:0/1532691901 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.827+0000 7fcf92a00700 1 -- 172.21.15.117:0/1532691901 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf941057f0 con 0x7fcf94104e90 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.827+0000 7fcf92a00700 1 --2- 172.21.15.117:0/1532691901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf941052b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf840098a0 tx=0x7fcf84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25199fbfd627d535 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.827+0000 7fcf91600700 1 -- 172.21.15.117:0/1532691901 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf84004180 con 0x7fcf94104e90 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.827+0000 7fcf91600700 1 -- 172.21.15.117:0/1532691901 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcf840042e0 con 0x7fcf94104e90 2024-04-20T10:56:06.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.827+0000 7fcf91600700 1 -- 172.21.15.117:0/1532691901 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf84011a70 con 0x7fcf94104e90 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.828+0000 7fcf99600700 1 -- 172.21.15.117:0/1532691901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 msgr2=0x7fcf941052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.828+0000 7fcf99600700 1 --2- 172.21.15.117:0/1532691901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf941052b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf840098a0 tx=0x7fcf84009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.828+0000 7fcf99600700 1 -- 172.21.15.117:0/1532691901 shutdown_connections 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.828+0000 7fcf99600700 1 --2- 172.21.15.117:0/1532691901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf941052b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.828+0000 7fcf99600700 1 -- 172.21.15.117:0/1532691901 >> 172.21.15.117:0/1532691901 conn(0x7fcf94100a50 msgr2=0x7fcf94102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.828+0000 7fcf99600700 1 -- 172.21.15.117:0/1532691901 shutdown_connections 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.828+0000 7fcf99600700 1 -- 172.21.15.117:0/1532691901 wait complete. 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.829+0000 7fcf99600700 1 Processor -- start 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.829+0000 7fcf99600700 1 -- start start 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.830+0000 7fcf99600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf9419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.830+0000 7fcf99600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9419ca50 con 0x7fcf94104e90 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.830+0000 7fcf92a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf9419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:06.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.830+0000 7fcf92a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf9419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39010/0 (socket says 172.21.15.117:39010) 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.830+0000 7fcf92a00700 1 -- 172.21.15.117:0/2988407568 learned_addr learned my addr 172.21.15.117:0/2988407568 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.830+0000 7fcf92a00700 1 -- 172.21.15.117:0/2988407568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84009550 con 0x7fcf94104e90 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.830+0000 7fcf92a00700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf9419c510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcf84011710 tx=0x7fcf840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.831+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf84003b10 con 0x7fcf94104e90 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.831+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcf8401b8c0 con 0x7fcf94104e90 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.831+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf840197e0 con 0x7fcf94104e90 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.831+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9419cc50 con 0x7fcf94104e90 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.831+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9419d0f0 con 0x7fcf94104e90 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.832+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fcf8401ba30 con 0x7fcf94104e90 2024-04-20T10:56:06.490 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.833+0000 7fcf8b400700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 0x7fcf7803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.833+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 --> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf7803b340 con 0x7fcf78038770 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.833+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcf8404c8c0 con 0x7fcf94104e90 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.833+0000 7fcf92000700 1 -- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 msgr2=0x7fcf7803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1432319225 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:58.833+0000 7fcf92000700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 0x7fcf7803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:59.034+0000 7fcf92000700 1 -- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 msgr2=0x7fcf7803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1432319225 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:59.034+0000 7fcf92000700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 0x7fcf7803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:59.435+0000 7fcf92000700 1 -- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 msgr2=0x7fcf7803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1432319225 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:55:59.435+0000 7fcf92000700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 0x7fcf7803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:00.236+0000 7fcf92000700 1 -- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 msgr2=0x7fcf7803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1432319225 2024-04-20T10:56:06.491 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:00.236+0000 7fcf92000700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 0x7fcf7803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:01.838+0000 7fcf92000700 1 -- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 msgr2=0x7fcf7803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1432319225 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:01.838+0000 7fcf92000700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 0x7fcf7803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:04.794+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mgrmap(e 6) v1 ==== 45323+0+0 (secure 0 0 0) 0x7fcf8401b440 con 0x7fcf94104e90 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:04.794+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 msgr2=0x7fcf7803ac30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:04.794+0000 7fcf8b400700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 0x7fcf7803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.802+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 7) v1 ==== 45450+0+0 (secure 0 0 0) 0x7fcf8401ad90 con 0x7fcf94104e90 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.802+0000 7fcf8b400700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcf7803bfb0 0x7fcf7803e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.802+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf7803b340 con 0x7fcf7803bfb0 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.804+0000 7fcf92000700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcf7803bfb0 0x7fcf7803e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.807+0000 7fcf92000700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcf7803bfb0 0x7fcf7803e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcf80003a10 tx=0x7fcf800092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.809+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fcf7803b340 con 0x7fcf7803bfb0 2024-04-20T10:56:06.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.816+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcf94106610 con 0x7fcf7803bfb0 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.816+0000 7fcf8b400700 1 -- 172.21.15.117:0/2988407568 <== mgr.14120 v2:172.21.15.117:6800/2597372910 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fcf94106610 con 0x7fcf7803bfb0 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcf7803bfb0 msgr2=0x7fcf7803e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcf7803bfb0 0x7fcf7803e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcf80003a10 tx=0x7fcf800092b0 comp rx=0 tx=0).stop 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 msgr2=0x7fcf9419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf9419c510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcf84011710 tx=0x7fcf840047c0 comp rx=0 tx=0).stop 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 shutdown_connections 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcf7803bfb0 0x7fcf7803e3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104e90 0x7fcf9419c510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 --2- 172.21.15.117:0/2988407568 >> [v2:172.21.15.117:6800/1432319225,v1:172.21.15.117:6801/1432319225] conn(0x7fcf78038770 0x7fcf7803ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 >> 172.21.15.117:0/2988407568 conn(0x7fcf94100a50 msgr2=0x7fcf941933a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 shutdown_connections 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:05.817+0000 7fcf99600700 1 -- 172.21.15.117:0/2988407568 wait complete. 2024-04-20T10:56:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:mgr epoch 5 is available 2024-04-20T10:56:06.494 INFO:teuthology.orchestra.run.smithi117.stdout:Setting orchestrator backend to cephadm... 2024-04-20T10:56:07.095 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:06 smithi117 ceph-mon[15659]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T10:56:07.096 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:06 smithi117 ceph-mon[15659]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T10:56:07.096 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:06 smithi117 ceph-mon[15659]: [20/Apr/2024:10:56:06] ENGINE Bus STARTING 2024-04-20T10:56:07.096 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:06 smithi117 ceph-mon[15659]: [20/Apr/2024:10:56:06] ENGINE Serving on http://172.21.15.117:8765 2024-04-20T10:56:07.096 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:06 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:56:07.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:07 smithi117 ceph-mon[15659]: [20/Apr/2024:10:56:06] ENGINE Serving on https://172.21.15.117:7150 2024-04-20T10:56:07.925 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:07 smithi117 ceph-mon[15659]: [20/Apr/2024:10:56:06] ENGINE Bus STARTED 2024-04-20T10:56:07.926 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:07 smithi117 ceph-mon[15659]: mgrmap e8: smithi117.ivnklj(active, since 2s) 2024-04-20T10:56:07.926 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:07 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:07.926 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:07 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:56:07.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.094+0000 7f9f9f200700 1 Processor -- start 2024-04-20T10:56:07.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.094+0000 7f9f9f200700 1 -- start start 2024-04-20T10:56:07.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.095+0000 7f9f9f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f98079230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.095+0000 7f9f9f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98079800 con 0x7f9f9807add0 2024-04-20T10:56:07.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.095+0000 7f9f97e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f98079230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.095+0000 7f9f97e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f98079230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60260/0 (socket says 172.21.15.117:60260) 2024-04-20T10:56:07.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.095+0000 7f9f97e00700 1 -- 172.21.15.117:0/3641984091 learned_addr learned my addr 172.21.15.117:0/3641984091 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.096+0000 7f9f97e00700 1 -- 172.21.15.117:0/3641984091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98079940 con 0x7f9f9807add0 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.096+0000 7f9f97e00700 1 --2- 172.21.15.117:0/3641984091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f98079230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009e80 tx=0x7f9f8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dad67d270427318 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.096+0000 7f9f97400700 1 -- 172.21.15.117:0/3641984091 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f8800e070 con 0x7f9f9807add0 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.097+0000 7f9f97400700 1 -- 172.21.15.117:0/3641984091 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f9f8800b8c0 con 0x7f9f9807add0 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.097+0000 7f9f97400700 1 -- 172.21.15.117:0/3641984091 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f88004440 con 0x7f9f9807add0 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.097+0000 7f9f9f200700 1 -- 172.21.15.117:0/3641984091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 msgr2=0x7f9f98079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.097+0000 7f9f9f200700 1 --2- 172.21.15.117:0/3641984091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f98079230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009e80 tx=0x7f9f8800b330 comp rx=0 tx=0).stop 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.097+0000 7f9f9f200700 1 -- 172.21.15.117:0/3641984091 shutdown_connections 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.097+0000 7f9f9f200700 1 --2- 172.21.15.117:0/3641984091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f98079230 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.097+0000 7f9f9f200700 1 -- 172.21.15.117:0/3641984091 >> 172.21.15.117:0/3641984091 conn(0x7f9f98101ba0 msgr2=0x7f9f98103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.097+0000 7f9f9f200700 1 -- 172.21.15.117:0/3641984091 shutdown_connections 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.098+0000 7f9f9f200700 1 -- 172.21.15.117:0/3641984091 wait complete. 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.099+0000 7f9f9f200700 1 Processor -- start 2024-04-20T10:56:07.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.099+0000 7f9f9f200700 1 -- start start 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.099+0000 7f9f9f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f9819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.099+0000 7f9f9f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9819ca50 con 0x7f9f9807add0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.099+0000 7f9f97e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f9819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.099+0000 7f9f97e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f9819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60264/0 (socket says 172.21.15.117:60264) 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.099+0000 7f9f97e00700 1 -- 172.21.15.117:0/997286330 learned_addr learned my addr 172.21.15.117:0/997286330 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.100+0000 7f9f97e00700 1 -- 172.21.15.117:0/997286330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88009b30 con 0x7f9f9807add0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.100+0000 7f9f97e00700 1 --2- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f9819c510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f88003c30 tx=0x7f9f88003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.100+0000 7f9f95600700 1 -- 172.21.15.117:0/997286330 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f8800e040 con 0x7f9f9807add0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.100+0000 7f9f95600700 1 -- 172.21.15.117:0/997286330 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f9f8800bc40 con 0x7f9f9807add0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.100+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9819cc50 con 0x7f9f9807add0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.100+0000 7f9f95600700 1 -- 172.21.15.117:0/997286330 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f880125e0 con 0x7f9f9807add0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.100+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9819d150 con 0x7f9f9807add0 2024-04-20T10:56:07.950 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.101+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9810c8e0 con 0x7f9f9807add0 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.101+0000 7f9f95600700 1 -- 172.21.15.117:0/997286330 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f9f88019040 con 0x7f9f9807add0 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.102+0000 7f9f95600700 1 --2- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f9f7c038720 0x7f9f7c03abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.102+0000 7f9f95600700 1 -- 172.21.15.117:0/997286330 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9f8804c070 con 0x7f9f9807add0 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.102+0000 7f9f8fe00700 1 --2- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f9f7c038720 0x7f9f7c03abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.103+0000 7f9f8fe00700 1 --2- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f9f7c038720 0x7f9f7c03abe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.108+0000 7f9f95600700 1 -- 172.21.15.117:0/997286330 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9f8801b9b0 con 0x7f9f9807add0 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.343+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f9819fa40 con 0x7f9f7c038720 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.363+0000 7f9f95600700 1 -- 172.21.15.117:0/997286330 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9f9819fa40 con 0x7f9f7c038720 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.367+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f9f7c038720 msgr2=0x7f9f7c03abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.367+0000 7f9f9f200700 1 --2- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f9f7c038720 0x7f9f7c03abe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.367+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 msgr2=0x7f9f9819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.367+0000 7f9f9f200700 1 --2- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f9819c510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f88003c30 tx=0x7f9f88003d10 comp rx=0 tx=0).stop 2024-04-20T10:56:07.951 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.367+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 shutdown_connections 2024-04-20T10:56:07.952 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.367+0000 7f9f9f200700 1 --2- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f9f7c038720 0x7f9f7c03abe0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.952 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.367+0000 7f9f9f200700 1 --2- 172.21.15.117:0/997286330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9807add0 0x7f9f9819c510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.952 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.367+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 >> 172.21.15.117:0/997286330 conn(0x7f9f98101ba0 msgr2=0x7f9f981027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:07.952 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.368+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 shutdown_connections 2024-04-20T10:56:07.952 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:07.368+0000 7f9f9f200700 1 -- 172.21.15.117:0/997286330 wait complete. 2024-04-20T10:56:09.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:08 smithi117 ceph-mon[15659]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.564+0000 7f68c9a00700 1 Processor -- start 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.564+0000 7f68c9a00700 1 -- start start 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.564+0000 7f68c9a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c41058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.564+0000 7f68c9a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c4105ea0 con 0x7f68c41054b0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.565+0000 7f68c2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c41058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.565+0000 7f68c2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c41058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60276/0 (socket says 172.21.15.117:60276) 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.565+0000 7f68c2a00700 1 -- 172.21.15.117:0/327328103 learned_addr learned my addr 172.21.15.117:0/327328103 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.565+0000 7f68c2a00700 1 -- 172.21.15.117:0/327328103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c41066b0 con 0x7f68c41054b0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.566+0000 7f68c2a00700 1 --2- 172.21.15.117:0/327328103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c41058d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f68b40098a0 tx=0x7f68b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d593efe5f089760d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.566+0000 7f68c2000700 1 -- 172.21.15.117:0/327328103 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b4004180 con 0x7f68c41054b0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.566+0000 7f68c2000700 1 -- 172.21.15.117:0/327328103 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f68b40042e0 con 0x7f68c41054b0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.566+0000 7f68c2000700 1 -- 172.21.15.117:0/327328103 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b4011a70 con 0x7f68c41054b0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.567+0000 7f68c9a00700 1 -- 172.21.15.117:0/327328103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 msgr2=0x7f68c41058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.567+0000 7f68c9a00700 1 --2- 172.21.15.117:0/327328103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c41058d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f68b40098a0 tx=0x7f68b4009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.567+0000 7f68c9a00700 1 -- 172.21.15.117:0/327328103 shutdown_connections 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.567+0000 7f68c9a00700 1 --2- 172.21.15.117:0/327328103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c41058d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.567+0000 7f68c9a00700 1 -- 172.21.15.117:0/327328103 >> 172.21.15.117:0/327328103 conn(0x7f68c4100a50 msgr2=0x7f68c4102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.567+0000 7f68c9a00700 1 -- 172.21.15.117:0/327328103 shutdown_connections 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.567+0000 7f68c9a00700 1 -- 172.21.15.117:0/327328103 wait complete. 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.568+0000 7f68c9a00700 1 Processor -- start 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.569+0000 7f68c9a00700 1 -- start start 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.569+0000 7f68c9a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.569+0000 7f68c9a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c419a960 con 0x7f68c41054b0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.569+0000 7f68c2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.569+0000 7f68c2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60282/0 (socket says 172.21.15.117:60282) 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.569+0000 7f68c2a00700 1 -- 172.21.15.117:0/2356093455 learned_addr learned my addr 172.21.15.117:0/2356093455 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.570+0000 7f68c2a00700 1 -- 172.21.15.117:0/2356093455 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4009550 con 0x7f68c41054b0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.570+0000 7f68c2a00700 1 --2- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c419a420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f68b4011710 tx=0x7f68b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.570+0000 7f68bbe00700 1 -- 172.21.15.117:0/2356093455 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b4003b10 con 0x7f68c41054b0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.570+0000 7f68bbe00700 1 -- 172.21.15.117:0/2356093455 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f68b401b8c0 con 0x7f68c41054b0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.570+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c419ab60 con 0x7f68c41054b0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.570+0000 7f68bbe00700 1 -- 172.21.15.117:0/2356093455 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b4019770 con 0x7f68c41054b0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.570+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c419b000 con 0x7f68c41054b0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.571+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c41944d0 con 0x7f68c41054b0 2024-04-20T10:56:09.508 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.572+0000 7f68bbe00700 1 -- 172.21.15.117:0/2356093455 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f68b401ba30 con 0x7f68c41054b0 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.572+0000 7f68bbe00700 1 --2- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f68a8038770 0x7f68a803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.572+0000 7f68bbe00700 1 -- 172.21.15.117:0/2356093455 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f68b404c990 con 0x7f68c41054b0 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.572+0000 7f68ba000700 1 --2- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f68a8038770 0x7f68a803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.573+0000 7f68ba000700 1 --2- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f68a8038770 0x7f68a803ac30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00ad30 tx=0x7f68ac0093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.578+0000 7f68bbe00700 1 -- 172.21.15.117:0/2356093455 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f68b402c9a0 con 0x7f68c41054b0 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.795+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f68c402cf30 con 0x7f68a8038770 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.796+0000 7f68bbe00700 1 -- 172.21.15.117:0/2356093455 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f68c402cf30 con 0x7f68a8038770 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.798+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f68a8038770 msgr2=0x7f68a803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.798+0000 7f68c9a00700 1 --2- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f68a8038770 0x7f68a803ac30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00ad30 tx=0x7f68ac0093f0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.798+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 msgr2=0x7f68c419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.798+0000 7f68c9a00700 1 --2- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c419a420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f68b4011710 tx=0x7f68b40047c0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.799+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 shutdown_connections 2024-04-20T10:56:09.510 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.799+0000 7f68c9a00700 1 --2- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f68a8038770 0x7f68a803ac30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.510 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.799+0000 7f68c9a00700 1 --2- 172.21.15.117:0/2356093455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68c41054b0 0x7f68c419a420 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.510 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.799+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 >> 172.21.15.117:0/2356093455 conn(0x7f68c4100a50 msgr2=0x7f68c4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:09.510 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.799+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 shutdown_connections 2024-04-20T10:56:09.510 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:08.799+0000 7f68c9a00700 1 -- 172.21.15.117:0/2356093455 wait complete. 2024-04-20T10:56:09.510 INFO:teuthology.orchestra.run.smithi117.stdout:Generating ssh key... 2024-04-20T10:56:10.072 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:09 smithi117 ceph-mon[15659]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.071+0000 7fcae5c00700 1 Processor -- start 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.071+0000 7fcae5c00700 1 -- start start 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.071+0000 7fcae5c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae0105260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.071+0000 7fcae5c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae0079ed0 con 0x7fcae0104e40 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.072+0000 7fcadea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae0105260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.072+0000 7fcadea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae0105260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60284/0 (socket says 172.21.15.117:60284) 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.072+0000 7fcadea00700 1 -- 172.21.15.117:0/3136530382 learned_addr learned my addr 172.21.15.117:0/3136530382 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.072+0000 7fcadea00700 1 -- 172.21.15.117:0/3136530382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae01057a0 con 0x7fcae0104e40 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.073+0000 7fcadea00700 1 --2- 172.21.15.117:0/3136530382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae0105260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcad00098a0 tx=0x7fcad0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228634f2a7e40c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.073+0000 7fcadd600700 1 -- 172.21.15.117:0/3136530382 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcad0004180 con 0x7fcae0104e40 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.073+0000 7fcadd600700 1 -- 172.21.15.117:0/3136530382 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcad00042e0 con 0x7fcae0104e40 2024-04-20T10:56:11.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.073+0000 7fcadd600700 1 -- 172.21.15.117:0/3136530382 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcad0011a70 con 0x7fcae0104e40 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.074+0000 7fcae5c00700 1 -- 172.21.15.117:0/3136530382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 msgr2=0x7fcae0105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.074+0000 7fcae5c00700 1 --2- 172.21.15.117:0/3136530382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae0105260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcad00098a0 tx=0x7fcad0009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.074+0000 7fcae5c00700 1 -- 172.21.15.117:0/3136530382 shutdown_connections 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.074+0000 7fcae5c00700 1 --2- 172.21.15.117:0/3136530382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae0105260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.074+0000 7fcae5c00700 1 -- 172.21.15.117:0/3136530382 >> 172.21.15.117:0/3136530382 conn(0x7fcae0100ae0 msgr2=0x7fcae0102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.074+0000 7fcae5c00700 1 -- 172.21.15.117:0/3136530382 shutdown_connections 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.074+0000 7fcae5c00700 1 -- 172.21.15.117:0/3136530382 wait complete. 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.075+0000 7fcae5c00700 1 Processor -- start 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.075+0000 7fcae5c00700 1 -- start start 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.076+0000 7fcae5c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.076+0000 7fcae5c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae019c9e0 con 0x7fcae0104e40 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.076+0000 7fcadea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.076+0000 7fcadea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60300/0 (socket says 172.21.15.117:60300) 2024-04-20T10:56:11.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.076+0000 7fcadea00700 1 -- 172.21.15.117:0/3910091140 learned_addr learned my addr 172.21.15.117:0/3910091140 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.076+0000 7fcadea00700 1 -- 172.21.15.117:0/3910091140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad0009550 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.076+0000 7fcadea00700 1 --2- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae019c4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcad0011710 tx=0x7fcad00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.077+0000 7fcad7400700 1 -- 172.21.15.117:0/3910091140 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcad0003b10 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.077+0000 7fcad7400700 1 -- 172.21.15.117:0/3910091140 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcad001b8c0 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.077+0000 7fcad7400700 1 -- 172.21.15.117:0/3910091140 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcad00197e0 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.077+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae019cbe0 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.077+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae019d080 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.078+0000 7fcad7400700 1 -- 172.21.15.117:0/3910091140 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fcad001ba30 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.078+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae0196210 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.078+0000 7fcad7400700 1 --2- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcac4038770 0x7fcac403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.079+0000 7fcad7400700 1 -- 172.21.15.117:0/3910091140 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcad004c900 con 0x7fcae0104e40 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.079+0000 7fcade000700 1 --2- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcac4038770 0x7fcac403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.079+0000 7fcade000700 1 --2- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcac4038770 0x7fcac403ac30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcacc006fd0 tx=0x7fcacc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.084+0000 7fcad7400700 1 -- 172.21.15.117:0/3910091140 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcad001bce0 con 0x7fcae0104e40 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.304+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fcae002cfb0 con 0x7fcac4038770 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.614+0000 7fcad7400700 1 -- 172.21.15.117:0/3910091140 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcae002cfb0 con 0x7fcac4038770 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcac4038770 msgr2=0x7fcac403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 --2- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcac4038770 0x7fcac403ac30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcacc006fd0 tx=0x7fcacc006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 msgr2=0x7fcae019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 --2- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae019c4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcad0011710 tx=0x7fcad00047c0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 shutdown_connections 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 --2- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fcac4038770 0x7fcac403ac30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 --2- 172.21.15.117:0/3910091140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcae0104e40 0x7fcae019c4a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 >> 172.21.15.117:0/3910091140 conn(0x7fcae0100ae0 msgr2=0x7fcae0193350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 shutdown_connections 2024-04-20T10:56:11.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:10.617+0000 7fcae5c00700 1 -- 172.21.15.117:0/3910091140 wait complete. 2024-04-20T10:56:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:11 smithi117 ceph-mon[15659]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:11 smithi117 ceph-mon[15659]: Generating ssh key... 2024-04-20T10:56:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:11 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:11 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:13.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZXXVoJa7i0GBvr/U20Z5yWrdwn+CwX/ZfmuRZEyICBWjTtKmAno9b80qBfXp83ugGsGM1LS3sx6k7Dp61Au5Ohz1eT0cj9ZN1Vm10OA3GVSVOE7an4MZBVWNvcmiCfMuCcyc/EzN6Q4+VUK/jFM6lRKHfWbSQSDDNQpsyHIG3PTdfwk3u3XQgb03vh7/HrktnN2OBEiCqvE0/L9n99iGcnqlv1oTaQzKOcntJxv97b3g3BGlIiw/VqY0yLfpTe8J2wl5DK5I8WD62sm87FhupsT9qtKMxaUd857lW8E1FV1w3Ut1QmGhaZydJtfrL8f4DctC798EtJ5cMAWSTZCRniokgB+PWsSieszmholt9otVsxeUYJqP8q4Bbp4TvYhsHWInqiJrBI3opbeFmGYZTnYpK58vDBrFqDz199l+7vqL5nPri7M7CJkgBHGCbl5EzvrQ4aRiaqxQHr4GsoU7KJBKgDQ8xqhL6QeWYioEsIHk6eJKw1qbxFjfJDUDMYL0= ceph-59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:56:13.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.965+0000 7f18eda00700 1 Processor -- start 2024-04-20T10:56:13.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.965+0000 7f18eda00700 1 -- start start 2024-04-20T10:56:13.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.966+0000 7f18eda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e8079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:13.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.966+0000 7f18eda00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e80797e0 con 0x7f18e807adb0 2024-04-20T10:56:13.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.966+0000 7f18e6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e8079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:13.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.966+0000 7f18e6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e8079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60314/0 (socket says 172.21.15.117:60314) 2024-04-20T10:56:13.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.966+0000 7f18e6a00700 1 -- 172.21.15.117:0/1070012784 learned_addr learned my addr 172.21.15.117:0/1070012784 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:13.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.967+0000 7f18e6a00700 1 -- 172.21.15.117:0/1070012784 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8079920 con 0x7f18e807adb0 2024-04-20T10:56:13.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.967+0000 7f18e6a00700 1 --2- 172.21.15.117:0/1070012784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e8079210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009e80 tx=0x7f18d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e86cbf66d56cbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.967+0000 7f18e6000700 1 -- 172.21.15.117:0/1070012784 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18d800e070 con 0x7f18e807adb0 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.967+0000 7f18e6000700 1 -- 172.21.15.117:0/1070012784 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f18d800b8c0 con 0x7f18e807adb0 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.967+0000 7f18e6000700 1 -- 172.21.15.117:0/1070012784 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18d8004440 con 0x7f18e807adb0 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.968+0000 7f18eda00700 1 -- 172.21.15.117:0/1070012784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 msgr2=0x7f18e8079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.968+0000 7f18eda00700 1 --2- 172.21.15.117:0/1070012784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e8079210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009e80 tx=0x7f18d800b330 comp rx=0 tx=0).stop 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.968+0000 7f18eda00700 1 -- 172.21.15.117:0/1070012784 shutdown_connections 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.968+0000 7f18eda00700 1 --2- 172.21.15.117:0/1070012784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e8079210 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.968+0000 7f18eda00700 1 -- 172.21.15.117:0/1070012784 >> 172.21.15.117:0/1070012784 conn(0x7f18e8101ba0 msgr2=0x7f18e8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.968+0000 7f18eda00700 1 -- 172.21.15.117:0/1070012784 shutdown_connections 2024-04-20T10:56:13.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.968+0000 7f18eda00700 1 -- 172.21.15.117:0/1070012784 wait complete. 2024-04-20T10:56:13.162 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.969+0000 7f18eda00700 1 Processor -- start 2024-04-20T10:56:13.162 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.969+0000 7f18eda00700 1 -- start start 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.970+0000 7f18eda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.970+0000 7f18eda00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e819caa0 con 0x7f18e807adb0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.970+0000 7f18e6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.970+0000 7f18e6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60316/0 (socket says 172.21.15.117:60316) 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.970+0000 7f18e6a00700 1 -- 172.21.15.117:0/829326835 learned_addr learned my addr 172.21.15.117:0/829326835 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.970+0000 7f18e6a00700 1 -- 172.21.15.117:0/829326835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d8009b30 con 0x7f18e807adb0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.971+0000 7f18e6a00700 1 --2- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e819c560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18d80099e0 tx=0x7f18d8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.971+0000 7f18dfe00700 1 -- 172.21.15.117:0/829326835 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18d800e040 con 0x7f18e807adb0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.971+0000 7f18dfe00700 1 -- 172.21.15.117:0/829326835 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f18d800bc40 con 0x7f18e807adb0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.971+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e819cca0 con 0x7f18e807adb0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.971+0000 7f18dfe00700 1 -- 172.21.15.117:0/829326835 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18d80125e0 con 0x7f18e807adb0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.971+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e819d1a0 con 0x7f18e807adb0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.972+0000 7f18dfe00700 1 -- 172.21.15.117:0/829326835 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f18d8019040 con 0x7f18e807adb0 2024-04-20T10:56:13.163 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.972+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e810c900 con 0x7f18e807adb0 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.973+0000 7f18dfe00700 1 --2- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f18c8038770 0x7f18c803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.973+0000 7f18dfe00700 1 -- 172.21.15.117:0/829326835 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f18d804c100 con 0x7f18e807adb0 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.973+0000 7f18d7e00700 1 --2- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f18c8038770 0x7f18c803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.974+0000 7f18d7e00700 1 --2- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f18c8038770 0x7f18c803ac30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f18d0006fd0 tx=0x7f18d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:11.978+0000 7f18dfe00700 1 -- 172.21.15.117:0/829326835 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f18d807fc50 con 0x7f18e807adb0 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.198+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f18e819fa90 con 0x7f18c8038770 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.200+0000 7f18dfe00700 1 -- 172.21.15.117:0/829326835 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f18e819fa90 con 0x7f18c8038770 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.202+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f18c8038770 msgr2=0x7f18c803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.202+0000 7f18eda00700 1 --2- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f18c8038770 0x7f18c803ac30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f18d0006fd0 tx=0x7f18d0006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:13.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.202+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 msgr2=0x7f18e819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:13.165 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.202+0000 7f18eda00700 1 --2- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e819c560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18d80099e0 tx=0x7f18d8003d00 comp rx=0 tx=0).stop 2024-04-20T10:56:13.165 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.202+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 shutdown_connections 2024-04-20T10:56:13.165 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.202+0000 7f18eda00700 1 --2- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f18c8038770 0x7f18c803ac30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:13.165 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.202+0000 7f18eda00700 1 --2- 172.21.15.117:0/829326835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e807adb0 0x7f18e819c560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:13.166 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.203+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 >> 172.21.15.117:0/829326835 conn(0x7f18e8101ba0 msgr2=0x7f18e81027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:13.166 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.203+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 shutdown_connections 2024-04-20T10:56:13.166 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:12.203+0000 7f18eda00700 1 -- 172.21.15.117:0/829326835 wait complete. 2024-04-20T10:56:13.166 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-20T10:56:13.166 INFO:teuthology.orchestra.run.smithi117.stdout:Adding key to root@localhost authorized_keys... 2024-04-20T10:56:13.177 INFO:teuthology.orchestra.run.smithi117.stdout:Adding host smithi117... 2024-04-20T10:56:13.521 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:13 smithi117 ceph-mon[15659]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:14 smithi117 ceph-mon[15659]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "addr": "172.21.15.117", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:16 smithi117 ceph-mon[15659]: Deploying cephadm binary to smithi117 2024-04-20T10:56:18.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Added host 'smithi117' with addr '172.21.15.117' 2024-04-20T10:56:18.617 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.838+0000 7f0f29400700 1 Processor -- start 2024-04-20T10:56:18.617 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.839+0000 7f0f29400700 1 -- start start 2024-04-20T10:56:18.617 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.839+0000 7f0f29400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f24109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:18.617 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.839+0000 7f0f29400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f24079e50 con 0x7f0f24107090 2024-04-20T10:56:18.617 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.839+0000 7f0f22800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f24109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.840+0000 7f0f22800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f24109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58634/0 (socket says 172.21.15.117:58634) 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.840+0000 7f0f22800700 1 -- 172.21.15.117:0/2612257422 learned_addr learned my addr 172.21.15.117:0/2612257422 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.840+0000 7f0f22800700 1 -- 172.21.15.117:0/2612257422 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f241099c0 con 0x7f0f24107090 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.840+0000 7f0f22800700 1 --2- 172.21.15.117:0/2612257422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f24109480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0f140098a0 tx=0x7f0f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bf7d1da8640e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.840+0000 7f0f21e00700 1 -- 172.21.15.117:0/2612257422 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f14004180 con 0x7f0f24107090 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.841+0000 7f0f21e00700 1 -- 172.21.15.117:0/2612257422 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0f140042e0 con 0x7f0f24107090 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.841+0000 7f0f21e00700 1 -- 172.21.15.117:0/2612257422 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f14011a70 con 0x7f0f24107090 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.841+0000 7f0f29400700 1 -- 172.21.15.117:0/2612257422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 msgr2=0x7f0f24109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.841+0000 7f0f29400700 1 --2- 172.21.15.117:0/2612257422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f24109480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0f140098a0 tx=0x7f0f14009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.841+0000 7f0f29400700 1 -- 172.21.15.117:0/2612257422 shutdown_connections 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.841+0000 7f0f29400700 1 --2- 172.21.15.117:0/2612257422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f24109480 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.841+0000 7f0f29400700 1 -- 172.21.15.117:0/2612257422 >> 172.21.15.117:0/2612257422 conn(0x7f0f24100aa0 msgr2=0x7f0f24102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.842+0000 7f0f29400700 1 -- 172.21.15.117:0/2612257422 shutdown_connections 2024-04-20T10:56:18.618 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.842+0000 7f0f29400700 1 -- 172.21.15.117:0/2612257422 wait complete. 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.843+0000 7f0f29400700 1 Processor -- start 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.843+0000 7f0f29400700 1 -- start start 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.844+0000 7f0f29400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f2419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.844+0000 7f0f29400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2419a7e0 con 0x7f0f24107090 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.844+0000 7f0f22800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f2419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.844+0000 7f0f22800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f2419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58650/0 (socket says 172.21.15.117:58650) 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.844+0000 7f0f22800700 1 -- 172.21.15.117:0/4148448132 learned_addr learned my addr 172.21.15.117:0/4148448132 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.844+0000 7f0f22800700 1 -- 172.21.15.117:0/4148448132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f14009550 con 0x7f0f24107090 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.845+0000 7f0f22800700 1 --2- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f2419a2a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0f14011710 tx=0x7f0f140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.845+0000 7f0f1be00700 1 -- 172.21.15.117:0/4148448132 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f14003b10 con 0x7f0f24107090 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.845+0000 7f0f1be00700 1 -- 172.21.15.117:0/4148448132 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0f1401b8c0 con 0x7f0f24107090 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.845+0000 7f0f1be00700 1 -- 172.21.15.117:0/4148448132 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f140197e0 con 0x7f0f24107090 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.845+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2419a9e0 con 0x7f0f24107090 2024-04-20T10:56:18.619 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.845+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2419ae80 con 0x7f0f24107090 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.846+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2410a6e0 con 0x7f0f24107090 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.846+0000 7f0f1be00700 1 -- 172.21.15.117:0/4148448132 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f0f1401ba30 con 0x7f0f24107090 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.847+0000 7f0f1be00700 1 --2- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0f08038770 0x7f0f0803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.847+0000 7f0f1be00700 1 -- 172.21.15.117:0/4148448132 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0f1404c900 con 0x7f0f24107090 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.847+0000 7f0f19e00700 1 --2- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0f08038770 0x7f0f0803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.848+0000 7f0f19e00700 1 --2- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0f08038770 0x7f0f0803ac30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c006fd0 tx=0x7f0f0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:13.852+0000 7f0f1be00700 1 -- 172.21.15.117:0/4148448132 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0f1401bce0 con 0x7f0f24107090 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:14.075+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "addr": "172.21.15.117", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f24101d30 con 0x7f0f08038770 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.955+0000 7f0f1be00700 1 -- 172.21.15.117:0/4148448132 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0f24101d30 con 0x7f0f08038770 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0f08038770 msgr2=0x7f0f0803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 --2- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0f08038770 0x7f0f0803ac30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c006fd0 tx=0x7f0f0c006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 msgr2=0x7f0f2419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 --2- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f2419a2a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0f14011710 tx=0x7f0f140047c0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.620 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 shutdown_connections 2024-04-20T10:56:18.621 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 --2- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0f08038770 0x7f0f0803ac30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.621 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 --2- 172.21.15.117:0/4148448132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f24107090 0x7f0f2419a2a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.621 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 >> 172.21.15.117:0/4148448132 conn(0x7f0f24100aa0 msgr2=0x7f0f24101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:18.621 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 shutdown_connections 2024-04-20T10:56:18.621 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:17.960+0000 7f0f29400700 1 -- 172.21.15.117:0/4148448132 wait complete. 2024-04-20T10:56:18.621 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying mon service with default placement... 2024-04-20T10:56:19.280 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:18 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:19.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:18 smithi117 ceph-mon[15659]: Added host smithi117 2024-04-20T10:56:19.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:18 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:56:20.609 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-20T10:56:20.609 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.548+0000 7fdba6e00700 1 Processor -- start 2024-04-20T10:56:20.609 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.548+0000 7fdba6e00700 1 -- start start 2024-04-20T10:56:20.609 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.549+0000 7fdba6e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba01069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.609 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.549+0000 7fdba6e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0106f90 con 0x7fdba01065a0 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.549+0000 7fdb9fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba01069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.549+0000 7fdb9fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba01069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58666/0 (socket says 172.21.15.117:58666) 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.549+0000 7fdb9fe00700 1 -- 172.21.15.117:0/760262258 learned_addr learned my addr 172.21.15.117:0/760262258 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.550+0000 7fdb9fe00700 1 -- 172.21.15.117:0/760262258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba01077a0 con 0x7fdba01065a0 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.550+0000 7fdb9fe00700 1 --2- 172.21.15.117:0/760262258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba01069c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb900098a0 tx=0x7fdb90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543066531a5b69f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.550+0000 7fdb9ea00700 1 -- 172.21.15.117:0/760262258 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb90004180 con 0x7fdba01065a0 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.551+0000 7fdb9ea00700 1 -- 172.21.15.117:0/760262258 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdb900042e0 con 0x7fdba01065a0 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.551+0000 7fdb9ea00700 1 -- 172.21.15.117:0/760262258 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb90011a70 con 0x7fdba01065a0 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.551+0000 7fdba6e00700 1 -- 172.21.15.117:0/760262258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 msgr2=0x7fdba01069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.551+0000 7fdba6e00700 1 --2- 172.21.15.117:0/760262258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba01069c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb900098a0 tx=0x7fdb90009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.551+0000 7fdba6e00700 1 -- 172.21.15.117:0/760262258 shutdown_connections 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.551+0000 7fdba6e00700 1 --2- 172.21.15.117:0/760262258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba01069c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.610 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.551+0000 7fdba6e00700 1 -- 172.21.15.117:0/760262258 >> 172.21.15.117:0/760262258 conn(0x7fdba0101ba0 msgr2=0x7fdba0103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.551+0000 7fdba6e00700 1 -- 172.21.15.117:0/760262258 shutdown_connections 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.552+0000 7fdba6e00700 1 -- 172.21.15.117:0/760262258 wait complete. 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.552+0000 7fdba6e00700 1 Processor -- start 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.553+0000 7fdba6e00700 1 -- start start 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.553+0000 7fdba6e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba007aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.553+0000 7fdba6e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba007af70 con 0x7fdba01065a0 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.553+0000 7fdb9fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba007aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.553+0000 7fdb9fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba007aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58670/0 (socket says 172.21.15.117:58670) 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.553+0000 7fdb9fe00700 1 -- 172.21.15.117:0/3102376277 learned_addr learned my addr 172.21.15.117:0/3102376277 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.554+0000 7fdb9fe00700 1 -- 172.21.15.117:0/3102376277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb90009550 con 0x7fdba01065a0 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.554+0000 7fdb9fe00700 1 --2- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba007aa30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb90006af0 tx=0x7fdb900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.554+0000 7fdb9cc00700 1 -- 172.21.15.117:0/3102376277 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb90003a00 con 0x7fdba01065a0 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.554+0000 7fdb9cc00700 1 -- 172.21.15.117:0/3102376277 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdb90003b60 con 0x7fdba01065a0 2024-04-20T10:56:20.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.554+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba00791b0 con 0x7fdba01065a0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.554+0000 7fdb9cc00700 1 -- 172.21.15.117:0/3102376277 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb90023750 con 0x7fdba01065a0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.554+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba0079650 con 0x7fdba01065a0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.556+0000 7fdb9cc00700 1 -- 172.21.15.117:0/3102376277 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fdb90023990 con 0x7fdba01065a0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.556+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba0062510 con 0x7fdba01065a0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.556+0000 7fdb9cc00700 1 --2- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fdb84038720 0x7fdb8403abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.556+0000 7fdb9cc00700 1 -- 172.21.15.117:0/3102376277 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdb9004c9b0 con 0x7fdba01065a0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.557+0000 7fdb9f400700 1 --2- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fdb84038720 0x7fdb8403abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.558+0000 7fdb9f400700 1 --2- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fdb84038720 0x7fdb8403abe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c006fd0 tx=0x7fdb8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.562+0000 7fdb9cc00700 1 -- 172.21.15.117:0/3102376277 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdb9002b760 con 0x7fdba01065a0 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.812+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fdba0079c90 con 0x7fdb84038720 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.822+0000 7fdb9cc00700 1 -- 172.21.15.117:0/3102376277 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdba0079c90 con 0x7fdb84038720 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.824+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fdb84038720 msgr2=0x7fdb8403abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.824+0000 7fdba6e00700 1 --2- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fdb84038720 0x7fdb8403abe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c006fd0 tx=0x7fdb8c006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:20.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.824+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 msgr2=0x7fdba007aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.613 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.824+0000 7fdba6e00700 1 --2- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba007aa30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb90006af0 tx=0x7fdb900047c0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.613 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.825+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 shutdown_connections 2024-04-20T10:56:20.613 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.825+0000 7fdba6e00700 1 --2- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fdb84038720 0x7fdb8403abe0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.613 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.825+0000 7fdba6e00700 1 --2- 172.21.15.117:0/3102376277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01065a0 0x7fdba007aa30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.613 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.825+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 >> 172.21.15.117:0/3102376277 conn(0x7fdba0101ba0 msgr2=0x7fdba0102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.613 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.825+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 shutdown_connections 2024-04-20T10:56:20.613 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:19.825+0000 7fdba6e00700 1 -- 172.21.15.117:0/3102376277 wait complete. 2024-04-20T10:56:20.613 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying mgr service with default placement... 2024-04-20T10:56:20.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:20 smithi117 ceph-mon[15659]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:20.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:20 smithi117 ceph-mon[15659]: Saving service mon spec with placement count:5 2024-04-20T10:56:20.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:20 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:22.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:22 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:22.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:22 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:22.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:22 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.623+0000 7f7be0400700 1 Processor -- start 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.623+0000 7f7be0400700 1 -- start start 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.623+0000 7f7be0400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd8109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.623+0000 7f7be0400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80747b0 con 0x7f7bd8107020 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.624+0000 7f7bdda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd8109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.624+0000 7f7bdda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd8109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58676/0 (socket says 172.21.15.117:58676) 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.624+0000 7f7bdda00700 1 -- 172.21.15.117:0/1075622649 learned_addr learned my addr 172.21.15.117:0/1075622649 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.624+0000 7f7bdda00700 1 -- 172.21.15.117:0/1075622649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd8109950 con 0x7f7bd8107020 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.624+0000 7f7bdda00700 1 --2- 172.21.15.117:0/1075622649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd8109410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0098a0 tx=0x7f7bcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6805dab4909d489f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.625+0000 7f7bd7e00700 1 -- 172.21.15.117:0/1075622649 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7bcc004180 con 0x7f7bd8107020 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.625+0000 7f7bd7e00700 1 -- 172.21.15.117:0/1075622649 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bcc0042e0 con 0x7f7bd8107020 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.625+0000 7f7bd7e00700 1 -- 172.21.15.117:0/1075622649 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7bcc011b10 con 0x7f7bd8107020 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.626+0000 7f7be0400700 1 -- 172.21.15.117:0/1075622649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 msgr2=0x7f7bd8109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.626+0000 7f7be0400700 1 --2- 172.21.15.117:0/1075622649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd8109410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0098a0 tx=0x7f7bcc009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.626+0000 7f7be0400700 1 -- 172.21.15.117:0/1075622649 shutdown_connections 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.626+0000 7f7be0400700 1 --2- 172.21.15.117:0/1075622649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd8109410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.626+0000 7f7be0400700 1 -- 172.21.15.117:0/1075622649 >> 172.21.15.117:0/1075622649 conn(0x7f7bd8100a50 msgr2=0x7f7bd8102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.626+0000 7f7be0400700 1 -- 172.21.15.117:0/1075622649 shutdown_connections 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.626+0000 7f7be0400700 1 -- 172.21.15.117:0/1075622649 wait complete. 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.627+0000 7f7be0400700 1 Processor -- start 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.627+0000 7f7be0400700 1 -- start start 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.628+0000 7f7be0400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.628+0000 7f7be0400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd819cac0 con 0x7f7bd8107020 2024-04-20T10:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.628+0000 7f7bdda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.628+0000 7f7bdda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58684/0 (socket says 172.21.15.117:58684) 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.628+0000 7f7bdda00700 1 -- 172.21.15.117:0/3606582019 learned_addr learned my addr 172.21.15.117:0/3606582019 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.628+0000 7f7bdda00700 1 -- 172.21.15.117:0/3606582019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc009550 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.629+0000 7f7bdda00700 1 --2- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd819c580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0117b0 tx=0x7f7bcc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.629+0000 7f7bd6000700 1 -- 172.21.15.117:0/3606582019 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7bcc0038b0 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.629+0000 7f7bd6000700 1 -- 172.21.15.117:0/3606582019 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bcc003a10 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.629+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd819ccc0 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.629+0000 7f7bd6000700 1 -- 172.21.15.117:0/3606582019 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7bcc024780 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.629+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd819d160 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.631+0000 7f7bd6000700 1 -- 172.21.15.117:0/3606582019 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f7bcc01ac10 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.631+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd810a740 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.631+0000 7f7bd6000700 1 --2- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f7bc0038770 0x7f7bc003ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.632+0000 7f7bd6000700 1 -- 172.21.15.117:0/3606582019 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7bcc04c760 con 0x7f7bd8107020 2024-04-20T10:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.632+0000 7f7bdd000700 1 --2- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f7bc0038770 0x7f7bc003ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.633+0000 7f7bdd000700 1 --2- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f7bc0038770 0x7f7bc003ac30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4006fd0 tx=0x7f7bc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.636+0000 7f7bd6000700 1 -- 172.21.15.117:0/3606582019 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7bcc02cb40 con 0x7f7bd8107020 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.876+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bd8101c30 con 0x7f7bc0038770 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.882+0000 7f7bd6000700 1 -- 172.21.15.117:0/3606582019 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7bd8101c30 con 0x7f7bc0038770 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f7bc0038770 msgr2=0x7f7bc003ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 --2- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f7bc0038770 0x7f7bc003ac30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4006fd0 tx=0x7f7bc4006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 msgr2=0x7f7bd819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 --2- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd819c580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0117b0 tx=0x7f7bcc011890 comp rx=0 tx=0).stop 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 shutdown_connections 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 --2- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f7bc0038770 0x7f7bc003ac30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 --2- 172.21.15.117:0/3606582019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8107020 0x7f7bd819c580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 >> 172.21.15.117:0/3606582019 conn(0x7f7bd8100a50 msgr2=0x7f7bd8101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 shutdown_connections 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:21.885+0000 7f7be0400700 1 -- 172.21.15.117:0/3606582019 wait complete. 2024-04-20T10:56:22.682 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying crash service with default placement... 2024-04-20T10:56:23.245 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:23 smithi117 ceph-mon[15659]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:23.245 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:23 smithi117 ceph-mon[15659]: Saving service mgr spec with placement count:2 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.296+0000 7fa6d5400700 1 Processor -- start 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.296+0000 7fa6d5400700 1 -- start start 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.296+0000 7fa6d5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d01052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.296+0000 7fa6d5400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0074710 con 0x7fa6d0104eb0 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.297+0000 7fa6ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d01052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.297+0000 7fa6ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d01052d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58700/0 (socket says 172.21.15.117:58700) 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.297+0000 7fa6ce800700 1 -- 172.21.15.117:0/2047229930 learned_addr learned my addr 172.21.15.117:0/2047229930 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:24.648 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.297+0000 7fa6ce800700 1 -- 172.21.15.117:0/2047229930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0074850 con 0x7fa6d0104eb0 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.297+0000 7fa6ce800700 1 --2- 172.21.15.117:0/2047229930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d01052d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00098a0 tx=0x7fa6c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bf688d85d70793 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.298+0000 7fa6cd400700 1 -- 172.21.15.117:0/2047229930 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6c0004180 con 0x7fa6d0104eb0 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.298+0000 7fa6cd400700 1 -- 172.21.15.117:0/2047229930 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6c00042e0 con 0x7fa6d0104eb0 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.298+0000 7fa6cd400700 1 -- 172.21.15.117:0/2047229930 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6c0011b10 con 0x7fa6d0104eb0 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.298+0000 7fa6d5400700 1 -- 172.21.15.117:0/2047229930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 msgr2=0x7fa6d01052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.298+0000 7fa6d5400700 1 --2- 172.21.15.117:0/2047229930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d01052d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00098a0 tx=0x7fa6c0009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.298+0000 7fa6d5400700 1 -- 172.21.15.117:0/2047229930 shutdown_connections 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.298+0000 7fa6d5400700 1 --2- 172.21.15.117:0/2047229930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d01052d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.298+0000 7fa6d5400700 1 -- 172.21.15.117:0/2047229930 >> 172.21.15.117:0/2047229930 conn(0x7fa6d0100a50 msgr2=0x7fa6d0102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.299+0000 7fa6d5400700 1 -- 172.21.15.117:0/2047229930 shutdown_connections 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.299+0000 7fa6d5400700 1 -- 172.21.15.117:0/2047229930 wait complete. 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.300+0000 7fa6d5400700 1 Processor -- start 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.300+0000 7fa6d5400700 1 -- start start 2024-04-20T10:56:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.300+0000 7fa6d5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.300+0000 7fa6d5400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d019ca80 con 0x7fa6d0104eb0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.300+0000 7fa6ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.300+0000 7fa6ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58716/0 (socket says 172.21.15.117:58716) 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.300+0000 7fa6ce800700 1 -- 172.21.15.117:0/1909570671 learned_addr learned my addr 172.21.15.117:0/1909570671 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.301+0000 7fa6ce800700 1 -- 172.21.15.117:0/1909570671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0009550 con 0x7fa6d0104eb0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.301+0000 7fa6ce800700 1 --2- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d019c540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00117b0 tx=0x7fa6c0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.301+0000 7fa6c7400700 1 -- 172.21.15.117:0/1909570671 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6c00038b0 con 0x7fa6d0104eb0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.301+0000 7fa6c7400700 1 -- 172.21.15.117:0/1909570671 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6c0003a10 con 0x7fa6d0104eb0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.301+0000 7fa6c7400700 1 -- 172.21.15.117:0/1909570671 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6c0024780 con 0x7fa6d0104eb0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.301+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d019cc80 con 0x7fa6d0104eb0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.301+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d019d120 con 0x7fa6d0104eb0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.303+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d0196230 con 0x7fa6d0104eb0 2024-04-20T10:56:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.303+0000 7fa6c7400700 1 -- 172.21.15.117:0/1909570671 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fa6c001ac10 con 0x7fa6d0104eb0 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.304+0000 7fa6c7400700 1 --2- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fa6b4038770 0x7fa6b403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.304+0000 7fa6c7400700 1 -- 172.21.15.117:0/1909570671 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa6c004c760 con 0x7fa6d0104eb0 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.304+0000 7fa6cde00700 1 --2- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fa6b4038770 0x7fa6b403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.305+0000 7fa6cde00700 1 --2- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fa6b4038770 0x7fa6b403ac30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fd0 tx=0x7fa6bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.309+0000 7fa6c7400700 1 -- 172.21.15.117:0/1909570671 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa6c00248e0 con 0x7fa6d0104eb0 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.559+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6d002cfb0 con 0x7fa6b4038770 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.569+0000 7fa6c7400700 1 -- 172.21.15.117:0/1909570671 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fa6d002cfb0 con 0x7fa6b4038770 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fa6b4038770 msgr2=0x7fa6b403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 --2- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fa6b4038770 0x7fa6b403ac30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fd0 tx=0x7fa6bc006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 msgr2=0x7fa6d019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 --2- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d019c540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00117b0 tx=0x7fa6c0011890 comp rx=0 tx=0).stop 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 shutdown_connections 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 --2- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fa6b4038770 0x7fa6b403ac30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.651 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 --2- 172.21.15.117:0/1909570671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6d0104eb0 0x7fa6d019c540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 >> 172.21.15.117:0/1909570671 conn(0x7fa6d0100a50 msgr2=0x7fa6d0193370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:24.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.572+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 shutdown_connections 2024-04-20T10:56:24.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:23.573+0000 7fa6d5400700 1 -- 172.21.15.117:0/1909570671 wait complete. 2024-04-20T10:56:24.652 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying ceph-exporter service with default placement... 2024-04-20T10:56:24.835 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:24 smithi117 ceph-mon[15659]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:24.835 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:24 smithi117 ceph-mon[15659]: Saving service crash spec with placement * 2024-04-20T10:56:24.835 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:24 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:25.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:25 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:25 smithi117 ceph-mon[15659]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:56:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:25 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:26.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-04-20T10:56:26.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.343+0000 7fd582600700 1 Processor -- start 2024-04-20T10:56:26.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.343+0000 7fd582600700 1 -- start start 2024-04-20T10:56:26.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.343+0000 7fd582600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:26.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.343+0000 7fd582600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c106fd0 con 0x7fd57c1065e0 2024-04-20T10:56:26.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.344+0000 7fd57b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:26.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.344+0000 7fd57b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40608/0 (socket says 172.21.15.117:40608) 2024-04-20T10:56:26.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.344+0000 7fd57b400700 1 -- 172.21.15.117:0/1325878301 learned_addr learned my addr 172.21.15.117:0/1325878301 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:26.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.345+0000 7fd57b400700 1 -- 172.21.15.117:0/1325878301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c1077e0 con 0x7fd57c1065e0 2024-04-20T10:56:26.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.345+0000 7fd57b400700 1 --2- 172.21.15.117:0/1325878301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c106a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0098a0 tx=0x7fd56c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6357afa49672820 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:26.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.345+0000 7fd57a000700 1 -- 172.21.15.117:0/1325878301 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd56c004180 con 0x7fd57c1065e0 2024-04-20T10:56:26.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.345+0000 7fd57a000700 1 -- 172.21.15.117:0/1325878301 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56c0042e0 con 0x7fd57c1065e0 2024-04-20T10:56:26.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.345+0000 7fd57a000700 1 -- 172.21.15.117:0/1325878301 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd56c011b10 con 0x7fd57c1065e0 2024-04-20T10:56:26.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.346+0000 7fd582600700 1 -- 172.21.15.117:0/1325878301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 msgr2=0x7fd57c106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:26.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.346+0000 7fd582600700 1 --2- 172.21.15.117:0/1325878301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c106a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0098a0 tx=0x7fd56c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:26.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.346+0000 7fd582600700 1 -- 172.21.15.117:0/1325878301 shutdown_connections 2024-04-20T10:56:26.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.346+0000 7fd582600700 1 --2- 172.21.15.117:0/1325878301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c106a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:26.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.346+0000 7fd582600700 1 -- 172.21.15.117:0/1325878301 >> 172.21.15.117:0/1325878301 conn(0x7fd57c101c00 msgr2=0x7fd57c104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:26.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.346+0000 7fd582600700 1 -- 172.21.15.117:0/1325878301 shutdown_connections 2024-04-20T10:56:26.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.346+0000 7fd582600700 1 -- 172.21.15.117:0/1325878301 wait complete. 2024-04-20T10:56:26.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.347+0000 7fd582600700 1 Processor -- start 2024-04-20T10:56:26.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.347+0000 7fd582600700 1 -- start start 2024-04-20T10:56:26.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.348+0000 7fd582600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c07aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:26.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.348+0000 7fd582600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c07af70 con 0x7fd57c1065e0 2024-04-20T10:56:26.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.348+0000 7fd57b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c07aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:26.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.348+0000 7fd57b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c07aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40614/0 (socket says 172.21.15.117:40614) 2024-04-20T10:56:26.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.348+0000 7fd57b400700 1 -- 172.21.15.117:0/2759250877 learned_addr learned my addr 172.21.15.117:0/2759250877 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:26.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.348+0000 7fd57b400700 1 -- 172.21.15.117:0/2759250877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c009550 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.349+0000 7fd57b400700 1 --2- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c07aa30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0117b0 tx=0x7fd56c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.349+0000 7fd573e00700 1 -- 172.21.15.117:0/2759250877 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd56c0038b0 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.349+0000 7fd573e00700 1 -- 172.21.15.117:0/2759250877 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56c003a10 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.349+0000 7fd573e00700 1 -- 172.21.15.117:0/2759250877 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd56c0246e0 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.349+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c0791b0 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.349+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c079650 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.350+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c062510 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.350+0000 7fd573e00700 1 -- 172.21.15.117:0/2759250877 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fd56c01ab80 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.351+0000 7fd573e00700 1 --2- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fd5600387c0 0x7fd56003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.351+0000 7fd573e00700 1 -- 172.21.15.117:0/2759250877 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd56c04c9b0 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.351+0000 7fd57aa00700 1 --2- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fd5600387c0 0x7fd56003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.352+0000 7fd57aa00700 1 --2- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fd5600387c0 0x7fd56003ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd568006fd0 tx=0x7fd568006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.357+0000 7fd573e00700 1 -- 172.21.15.117:0/2759250877 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd56c010380 con 0x7fd57c1065e0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.591+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fd57c079db0 con 0x7fd5600387c0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.600+0000 7fd573e00700 1 -- 172.21.15.117:0/2759250877 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fd57c079db0 con 0x7fd5600387c0 2024-04-20T10:56:26.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fd5600387c0 msgr2=0x7fd56003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 --2- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fd5600387c0 0x7fd56003ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd568006fd0 tx=0x7fd568006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 msgr2=0x7fd57c07aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 --2- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c07aa30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0117b0 tx=0x7fd56c011890 comp rx=0 tx=0).stop 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 shutdown_connections 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 --2- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fd5600387c0 0x7fd56003ac80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 --2- 172.21.15.117:0/2759250877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd57c1065e0 0x7fd57c07aa30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 >> 172.21.15.117:0/2759250877 conn(0x7fd57c101c00 msgr2=0x7fd57c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.603+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 shutdown_connections 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:25.604+0000 7fd582600700 1 -- 172.21.15.117:0/2759250877 wait complete. 2024-04-20T10:56:26.310 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying prometheus service with default placement... 2024-04-20T10:56:26.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:26 smithi117 ceph-mon[15659]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:26.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:26 smithi117 ceph-mon[15659]: Saving service ceph-exporter spec with placement * 2024-04-20T10:56:26.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:26 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:27.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:27 smithi117 ceph-mon[15659]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:56:27.842 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:27 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:28.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-20T10:56:28.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.902+0000 7f8826a00700 1 Processor -- start 2024-04-20T10:56:28.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.902+0000 7f8826a00700 1 -- start start 2024-04-20T10:56:28.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.903+0000 7f8826a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f882007a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:28.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.903+0000 7f8826a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882007a930 con 0x7f882007beb0 2024-04-20T10:56:28.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.903+0000 7f881fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f882007a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:28.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.903+0000 7f881fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f882007a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40628/0 (socket says 172.21.15.117:40628) 2024-04-20T10:56:28.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.903+0000 7f881fe00700 1 -- 172.21.15.117:0/3634099317 learned_addr learned my addr 172.21.15.117:0/3634099317 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:28.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.904+0000 7f881fe00700 1 -- 172.21.15.117:0/3634099317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882007aa70 con 0x7f882007beb0 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.904+0000 7f881fe00700 1 --2- 172.21.15.117:0/3634099317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f882007a360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88100098a0 tx=0x7f8810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34090b972b8c5752 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.904+0000 7f881ea00700 1 -- 172.21.15.117:0/3634099317 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8810004180 con 0x7f882007beb0 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.904+0000 7f881ea00700 1 -- 172.21.15.117:0/3634099317 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88100042e0 con 0x7f882007beb0 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.904+0000 7f881ea00700 1 -- 172.21.15.117:0/3634099317 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8810011b10 con 0x7f882007beb0 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.905+0000 7f8826a00700 1 -- 172.21.15.117:0/3634099317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 msgr2=0x7f882007a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.905+0000 7f8826a00700 1 --2- 172.21.15.117:0/3634099317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f882007a360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88100098a0 tx=0x7f8810009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.905+0000 7f8826a00700 1 -- 172.21.15.117:0/3634099317 shutdown_connections 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.905+0000 7f8826a00700 1 --2- 172.21.15.117:0/3634099317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f882007a360 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.905+0000 7f8826a00700 1 -- 172.21.15.117:0/3634099317 >> 172.21.15.117:0/3634099317 conn(0x7f8820101bb0 msgr2=0x7f8820103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.906+0000 7f8826a00700 1 -- 172.21.15.117:0/3634099317 shutdown_connections 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.906+0000 7f8826a00700 1 -- 172.21.15.117:0/3634099317 wait complete. 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.907+0000 7f8826a00700 1 Processor -- start 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.907+0000 7f8826a00700 1 -- start start 2024-04-20T10:56:28.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.907+0000 7f8826a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f88201a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.907+0000 7f8826a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88201a0e60 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.908+0000 7f881fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f88201a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.908+0000 7f881fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f88201a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40642/0 (socket says 172.21.15.117:40642) 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.908+0000 7f881fe00700 1 -- 172.21.15.117:0/2241369974 learned_addr learned my addr 172.21.15.117:0/2241369974 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.908+0000 7f881fe00700 1 -- 172.21.15.117:0/2241369974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8810009550 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.908+0000 7f881fe00700 1 --2- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f88201a0920 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88100117b0 tx=0x7f8810003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.909+0000 7f881cc00700 1 -- 172.21.15.117:0/2241369974 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88100036a0 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.909+0000 7f881cc00700 1 -- 172.21.15.117:0/2241369974 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8810003800 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.909+0000 7f881cc00700 1 -- 172.21.15.117:0/2241369974 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88100245c0 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.909+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88201a1060 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.909+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88201a1500 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.910+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882019a090 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.911+0000 7f881cc00700 1 -- 172.21.15.117:0/2241369974 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f881001a430 con 0x7f882007beb0 2024-04-20T10:56:28.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.911+0000 7f881cc00700 1 --2- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f8804038770 0x7f880403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.911+0000 7f881cc00700 1 -- 172.21.15.117:0/2241369974 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f881004c6c0 con 0x7f882007beb0 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.912+0000 7f881f400700 1 --2- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f8804038770 0x7f880403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.912+0000 7f881f400700 1 --2- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f8804038770 0x7f880403ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f880c006fd0 tx=0x7f880c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:26.917+0000 7f881cc00700 1 -- 172.21.15.117:0/2241369974 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f881002cce0 con 0x7f882007beb0 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.177+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f882002cfc0 con 0x7f8804038770 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.186+0000 7f881cc00700 1 -- 172.21.15.117:0/2241369974 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f882002cfc0 con 0x7f8804038770 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.189+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f8804038770 msgr2=0x7f880403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.189+0000 7f8826a00700 1 --2- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f8804038770 0x7f880403ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f880c006fd0 tx=0x7f880c006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.190+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 msgr2=0x7f88201a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.190+0000 7f8826a00700 1 --2- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f88201a0920 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88100117b0 tx=0x7f8810003e50 comp rx=0 tx=0).stop 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.190+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 shutdown_connections 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.190+0000 7f8826a00700 1 --2- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f8804038770 0x7f880403ac30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:28.488 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.190+0000 7f8826a00700 1 --2- 172.21.15.117:0/2241369974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f882007beb0 0x7f88201a0920 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:28.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.190+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 >> 172.21.15.117:0/2241369974 conn(0x7f8820101bb0 msgr2=0x7f8820103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:28.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.190+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 shutdown_connections 2024-04-20T10:56:28.489 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:27.190+0000 7f8826a00700 1 -- 172.21.15.117:0/2241369974 wait complete. 2024-04-20T10:56:28.489 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying grafana service with default placement... 2024-04-20T10:56:29.071 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:28 smithi117 ceph-mon[15659]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:29.071 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:28 smithi117 ceph-mon[15659]: Saving service prometheus spec with placement count:1 2024-04-20T10:56:30.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:29 smithi117 ceph-mon[15659]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.520+0000 7febe6400700 1 Processor -- start 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.520+0000 7febe6400700 1 -- start start 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.520+0000 7febe6400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe00791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.520+0000 7febe6400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00797c0 con 0x7febe007ad90 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.521+0000 7febdf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe00791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.521+0000 7febdf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe00791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40652/0 (socket says 172.21.15.117:40652) 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.521+0000 7febdf400700 1 -- 172.21.15.117:0/1040035787 learned_addr learned my addr 172.21.15.117:0/1040035787 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:30.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.521+0000 7febdf400700 1 -- 172.21.15.117:0/1040035787 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0079900 con 0x7febe007ad90 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.522+0000 7febdf400700 1 --2- 172.21.15.117:0/1040035787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe00791f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7febd00098a0 tx=0x7febd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff47aac8f7201117 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.522+0000 7febde000700 1 -- 172.21.15.117:0/1040035787 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd0004180 con 0x7febe007ad90 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.522+0000 7febde000700 1 -- 172.21.15.117:0/1040035787 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febd00042e0 con 0x7febe007ad90 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.522+0000 7febde000700 1 -- 172.21.15.117:0/1040035787 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd0011b10 con 0x7febe007ad90 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.523+0000 7febe6400700 1 -- 172.21.15.117:0/1040035787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 msgr2=0x7febe00791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.523+0000 7febe6400700 1 --2- 172.21.15.117:0/1040035787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe00791f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7febd00098a0 tx=0x7febd0009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.523+0000 7febe6400700 1 -- 172.21.15.117:0/1040035787 shutdown_connections 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.523+0000 7febe6400700 1 --2- 172.21.15.117:0/1040035787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe00791f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.523+0000 7febe6400700 1 -- 172.21.15.117:0/1040035787 >> 172.21.15.117:0/1040035787 conn(0x7febe0101b70 msgr2=0x7febe0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.523+0000 7febe6400700 1 -- 172.21.15.117:0/1040035787 shutdown_connections 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.523+0000 7febe6400700 1 -- 172.21.15.117:0/1040035787 wait complete. 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.524+0000 7febe6400700 1 Processor -- start 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.524+0000 7febe6400700 1 -- start start 2024-04-20T10:56:30.443 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.524+0000 7febe6400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.524+0000 7febe6400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe019ca50 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.525+0000 7febdf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.525+0000 7febdf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40662/0 (socket says 172.21.15.117:40662) 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.525+0000 7febdf400700 1 -- 172.21.15.117:0/327434007 learned_addr learned my addr 172.21.15.117:0/327434007 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.525+0000 7febdf400700 1 -- 172.21.15.117:0/327434007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd0009550 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.525+0000 7febdf400700 1 --2- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe019c510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7febd00117b0 tx=0x7febd0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.525+0000 7febd7e00700 1 -- 172.21.15.117:0/327434007 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd00038b0 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.526+0000 7febd7e00700 1 -- 172.21.15.117:0/327434007 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febd0003a10 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.526+0000 7febd7e00700 1 -- 172.21.15.117:0/327434007 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd0024780 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.526+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe019cc50 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.526+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe019d0f0 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.527+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe010c970 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.527+0000 7febd7e00700 1 -- 172.21.15.117:0/327434007 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7febd001ac10 con 0x7febe007ad90 2024-04-20T10:56:30.444 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.527+0000 7febd7e00700 1 --2- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7febc4038770 0x7febc403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.527+0000 7febd7e00700 1 -- 172.21.15.117:0/327434007 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7febd004c800 con 0x7febe007ad90 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.528+0000 7febdea00700 1 --2- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7febc4038770 0x7febc403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.528+0000 7febdea00700 1 --2- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7febc4038770 0x7febc403ac30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7febcc006fd0 tx=0x7febcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.533+0000 7febd7e00700 1 -- 172.21.15.117:0/327434007 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7febd0018760 con 0x7febe007ad90 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.767+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7febe0102e10 con 0x7febc4038770 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.780+0000 7febd7e00700 1 -- 172.21.15.117:0/327434007 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7febe0102e10 con 0x7febc4038770 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7febc4038770 msgr2=0x7febc403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 --2- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7febc4038770 0x7febc403ac30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7febcc006fd0 tx=0x7febcc006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 msgr2=0x7febe019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 --2- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe019c510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7febd00117b0 tx=0x7febd0011890 comp rx=0 tx=0).stop 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 shutdown_connections 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 --2- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7febc4038770 0x7febc403ac30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 --2- 172.21.15.117:0/327434007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe007ad90 0x7febe019c510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:30.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 >> 172.21.15.117:0/327434007 conn(0x7febe0101b70 msgr2=0x7febe0102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:30.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 shutdown_connections 2024-04-20T10:56:30.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:29.783+0000 7febe6400700 1 -- 172.21.15.117:0/327434007 wait complete. 2024-04-20T10:56:30.446 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying node-exporter service with default placement... 2024-04-20T10:56:31.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:30 smithi117 ceph-mon[15659]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:31.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:30 smithi117 ceph-mon[15659]: Saving service grafana spec with placement count:1 2024-04-20T10:56:31.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:30 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:31.965 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:31 smithi117 ceph-mon[15659]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:56:31.965 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:31 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:31.983 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-20T10:56:31.983 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.073+0000 7f3612a00700 1 Processor -- start 2024-04-20T10:56:31.983 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.074+0000 7f3612a00700 1 -- start start 2024-04-20T10:56:31.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.074+0000 7f3612a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c07a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:31.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.074+0000 7f3612a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c07a930 con 0x7f360c07bf00 2024-04-20T10:56:31.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.074+0000 7f360be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c07a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:31.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.075+0000 7f360be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c07a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40664/0 (socket says 172.21.15.117:40664) 2024-04-20T10:56:31.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.075+0000 7f360be00700 1 -- 172.21.15.117:0/868373495 learned_addr learned my addr 172.21.15.117:0/868373495 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:31.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.075+0000 7f360be00700 1 -- 172.21.15.117:0/868373495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c07aa70 con 0x7f360c07bf00 2024-04-20T10:56:31.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.075+0000 7f360be00700 1 --2- 172.21.15.117:0/868373495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c07a360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0098a0 tx=0x7f35fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff049e7914a3933 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.076+0000 7f360aa00700 1 -- 172.21.15.117:0/868373495 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35fc004180 con 0x7f360c07bf00 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.076+0000 7f360aa00700 1 -- 172.21.15.117:0/868373495 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35fc0042e0 con 0x7f360c07bf00 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.076+0000 7f360aa00700 1 -- 172.21.15.117:0/868373495 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35fc011b10 con 0x7f360c07bf00 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.076+0000 7f3612a00700 1 -- 172.21.15.117:0/868373495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 msgr2=0x7f360c07a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.076+0000 7f3612a00700 1 --2- 172.21.15.117:0/868373495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c07a360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0098a0 tx=0x7f35fc009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.077+0000 7f3612a00700 1 -- 172.21.15.117:0/868373495 shutdown_connections 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.077+0000 7f3612a00700 1 --2- 172.21.15.117:0/868373495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c07a360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.077+0000 7f3612a00700 1 -- 172.21.15.117:0/868373495 >> 172.21.15.117:0/868373495 conn(0x7f360c101be0 msgr2=0x7f360c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.077+0000 7f3612a00700 1 -- 172.21.15.117:0/868373495 shutdown_connections 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.077+0000 7f3612a00700 1 -- 172.21.15.117:0/868373495 wait complete. 2024-04-20T10:56:31.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.078+0000 7f3612a00700 1 Processor -- start 2024-04-20T10:56:31.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.078+0000 7f3612a00700 1 -- start start 2024-04-20T10:56:31.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.078+0000 7f3612a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:31.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.078+0000 7f3612a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c1a0e60 con 0x7f360c07bf00 2024-04-20T10:56:31.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.078+0000 7f360be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:31.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.078+0000 7f360be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c1a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40672/0 (socket says 172.21.15.117:40672) 2024-04-20T10:56:31.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.078+0000 7f360be00700 1 -- 172.21.15.117:0/1448808822 learned_addr learned my addr 172.21.15.117:0/1448808822 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:31.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.079+0000 7f360be00700 1 -- 172.21.15.117:0/1448808822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc009550 con 0x7f360c07bf00 2024-04-20T10:56:31.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.079+0000 7f360be00700 1 --2- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c1a0920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0117b0 tx=0x7f35fc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:31.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.079+0000 7f3608c00700 1 -- 172.21.15.117:0/1448808822 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35fc0038b0 con 0x7f360c07bf00 2024-04-20T10:56:31.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.079+0000 7f3608c00700 1 -- 172.21.15.117:0/1448808822 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35fc003a10 con 0x7f360c07bf00 2024-04-20T10:56:31.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.079+0000 7f3608c00700 1 -- 172.21.15.117:0/1448808822 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35fc0246e0 con 0x7f360c07bf00 2024-04-20T10:56:31.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.079+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c1a1060 con 0x7f360c07bf00 2024-04-20T10:56:31.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.080+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c1a1500 con 0x7f360c07bf00 2024-04-20T10:56:31.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.081+0000 7f3608c00700 1 -- 172.21.15.117:0/1448808822 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f35fc01a430 con 0x7f360c07bf00 2024-04-20T10:56:31.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.081+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c108580 con 0x7f360c07bf00 2024-04-20T10:56:31.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.081+0000 7f3608c00700 1 --2- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f35f0038720 0x7f35f003abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:31.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.081+0000 7f3608c00700 1 -- 172.21.15.117:0/1448808822 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f35fc04c900 con 0x7f360c07bf00 2024-04-20T10:56:31.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.081+0000 7f360b400700 1 --2- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f35f0038720 0x7f35f003abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:31.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.082+0000 7f360b400700 1 --2- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f35f0038720 0x7f35f003abe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f35f8006fd0 tx=0x7f35f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:31.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.086+0000 7f3608c00700 1 -- 172.21.15.117:0/1448808822 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f35fc018b40 con 0x7f360c07bf00 2024-04-20T10:56:31.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.320+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f360c061320 con 0x7f35f0038720 2024-04-20T10:56:31.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.329+0000 7f3608c00700 1 -- 172.21.15.117:0/1448808822 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f360c061320 con 0x7f35f0038720 2024-04-20T10:56:31.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.331+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f35f0038720 msgr2=0x7f35f003abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:31.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.331+0000 7f3612a00700 1 --2- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f35f0038720 0x7f35f003abe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f35f8006fd0 tx=0x7f35f8006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:31.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.332+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 msgr2=0x7f360c1a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:31.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.332+0000 7f3612a00700 1 --2- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c1a0920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0117b0 tx=0x7f35fc011890 comp rx=0 tx=0).stop 2024-04-20T10:56:31.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.332+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 shutdown_connections 2024-04-20T10:56:31.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.332+0000 7f3612a00700 1 --2- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f35f0038720 0x7f35f003abe0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:31.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.332+0000 7f3612a00700 1 --2- 172.21.15.117:0/1448808822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f360c07bf00 0x7f360c1a0920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:31.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.332+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 >> 172.21.15.117:0/1448808822 conn(0x7f360c101be0 msgr2=0x7f360c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:31.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.332+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 shutdown_connections 2024-04-20T10:56:31.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:31.332+0000 7f3612a00700 1 -- 172.21.15.117:0/1448808822 wait complete. 2024-04-20T10:56:31.996 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying alertmanager service with default placement... 2024-04-20T10:56:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:32 smithi117 ceph-mon[15659]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:32 smithi117 ceph-mon[15659]: Saving service node-exporter spec with placement * 2024-04-20T10:56:33.567 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.638+0000 7f449a400700 1 Processor -- start 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.638+0000 7f449a400700 1 -- start start 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.638+0000 7f449a400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f44941069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.638+0000 7f449a400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4494106fb0 con 0x7f44941065c0 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.639+0000 7f4493400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f44941069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.639+0000 7f4493400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f44941069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40688/0 (socket says 172.21.15.117:40688) 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.639+0000 7f4493400700 1 -- 172.21.15.117:0/3229391347 learned_addr learned my addr 172.21.15.117:0/3229391347 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.639+0000 7f4493400700 1 -- 172.21.15.117:0/3229391347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44941077c0 con 0x7f44941065c0 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.639+0000 7f4493400700 1 --2- 172.21.15.117:0/3229391347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f44941069e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f44840098a0 tx=0x7f4484009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2ae148a9addab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.640+0000 7f4492a00700 1 -- 172.21.15.117:0/3229391347 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4484004180 con 0x7f44941065c0 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.640+0000 7f4492a00700 1 -- 172.21.15.117:0/3229391347 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44840042e0 con 0x7f44941065c0 2024-04-20T10:56:33.568 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.640+0000 7f4492a00700 1 -- 172.21.15.117:0/3229391347 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4484011b10 con 0x7f44941065c0 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.640+0000 7f449a400700 1 -- 172.21.15.117:0/3229391347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 msgr2=0x7f44941069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.641+0000 7f449a400700 1 --2- 172.21.15.117:0/3229391347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f44941069e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f44840098a0 tx=0x7f4484009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.641+0000 7f449a400700 1 -- 172.21.15.117:0/3229391347 shutdown_connections 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.641+0000 7f449a400700 1 --2- 172.21.15.117:0/3229391347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f44941069e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.641+0000 7f449a400700 1 -- 172.21.15.117:0/3229391347 >> 172.21.15.117:0/3229391347 conn(0x7f4494101be0 msgr2=0x7f4494104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.641+0000 7f449a400700 1 -- 172.21.15.117:0/3229391347 shutdown_connections 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.641+0000 7f449a400700 1 -- 172.21.15.117:0/3229391347 wait complete. 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.642+0000 7f449a400700 1 Processor -- start 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.642+0000 7f449a400700 1 -- start start 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.642+0000 7f449a400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f449407baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.642+0000 7f449a400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449407c030 con 0x7f44941065c0 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.643+0000 7f4493400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f449407baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.643+0000 7f4493400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f449407baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40694/0 (socket says 172.21.15.117:40694) 2024-04-20T10:56:33.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.643+0000 7f4493400700 1 -- 172.21.15.117:0/2440146117 learned_addr learned my addr 172.21.15.117:0/2440146117 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.643+0000 7f4493400700 1 -- 172.21.15.117:0/2440146117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484009550 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.643+0000 7f4493400700 1 --2- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f449407baf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44840117b0 tx=0x7f4484011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.644+0000 7f4490c00700 1 -- 172.21.15.117:0/2440146117 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44840038b0 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.644+0000 7f4490c00700 1 -- 172.21.15.117:0/2440146117 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4484003a10 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.644+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449407a2c0 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.644+0000 7f4490c00700 1 -- 172.21.15.117:0/2440146117 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44840246e0 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.644+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449407a760 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.645+0000 7f4490c00700 1 -- 172.21.15.117:0/2440146117 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f448401ab80 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.645+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449407c170 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.645+0000 7f4490c00700 1 --2- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f4478038720 0x7f447803abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.645+0000 7f4490c00700 1 -- 172.21.15.117:0/2440146117 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f448404ca10 con 0x7f44941065c0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.646+0000 7f448aa00700 1 --2- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f4478038720 0x7f447803abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:33.570 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.646+0000 7f448aa00700 1 --2- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f4478038720 0x7f447803abe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f447c006fd0 tx=0x7f447c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.650+0000 7f4490c00700 1 -- 172.21.15.117:0/2440146117 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f449407c170 con 0x7f44941065c0 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.882+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f449407c170 con 0x7f4478038720 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.890+0000 7f4490c00700 1 -- 172.21.15.117:0/2440146117 <== mgr.14120 v2:172.21.15.117:6800/2597372910 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f449407c170 con 0x7f4478038720 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.893+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f4478038720 msgr2=0x7f447803abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.893+0000 7f449a400700 1 --2- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f4478038720 0x7f447803abe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f447c006fd0 tx=0x7f447c006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.893+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 msgr2=0x7f449407baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.893+0000 7f449a400700 1 --2- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f449407baf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44840117b0 tx=0x7f4484011890 comp rx=0 tx=0).stop 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.893+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 shutdown_connections 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.893+0000 7f449a400700 1 --2- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f4478038720 0x7f447803abe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.893+0000 7f449a400700 1 --2- 172.21.15.117:0/2440146117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44941065c0 0x7f449407baf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.893+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 >> 172.21.15.117:0/2440146117 conn(0x7f4494101be0 msgr2=0x7f44941026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.894+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 shutdown_connections 2024-04-20T10:56:33.571 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:32.894+0000 7f449a400700 1 -- 172.21.15.117:0/2440146117 wait complete. 2024-04-20T10:56:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:33 smithi117 ceph-mon[15659]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:56:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:33 smithi117 ceph-mon[15659]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:33 smithi117 ceph-mon[15659]: Saving service alertmanager spec with placement count:1 2024-04-20T10:56:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:33 smithi117 ceph-mon[15659]: from='mgr.14120 172.21.15.117:0/3286527751' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:35.190 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.231+0000 7f5133200700 1 Processor -- start 2024-04-20T10:56:35.190 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.232+0000 7f5133200700 1 -- start start 2024-04-20T10:56:35.190 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.232+0000 7f5133200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c10d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.232+0000 7f5133200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c10d940 con 0x7f512c10cf50 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.232+0000 7f5131e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c10d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.233+0000 7f5131e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c10d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49398/0 (socket says 172.21.15.117:49398) 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.233+0000 7f5131e00700 1 -- 172.21.15.117:0/2746485124 learned_addr learned my addr 172.21.15.117:0/2746485124 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.233+0000 7f5131e00700 1 -- 172.21.15.117:0/2746485124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c10da80 con 0x7f512c10cf50 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.233+0000 7f5131e00700 1 --2- 172.21.15.117:0/2746485124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c10d370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f51200098a0 tx=0x7f5120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64e2c71735a599f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.233+0000 7f5130a00700 1 -- 172.21.15.117:0/2746485124 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5120004180 con 0x7f512c10cf50 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5130a00700 1 -- 172.21.15.117:0/2746485124 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51200042e0 con 0x7f512c10cf50 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5130a00700 1 -- 172.21.15.117:0/2746485124 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5120011b10 con 0x7f512c10cf50 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5133200700 1 -- 172.21.15.117:0/2746485124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 msgr2=0x7f512c10d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5133200700 1 --2- 172.21.15.117:0/2746485124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c10d370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f51200098a0 tx=0x7f5120009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5133200700 1 -- 172.21.15.117:0/2746485124 shutdown_connections 2024-04-20T10:56:35.192 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5133200700 1 --2- 172.21.15.117:0/2746485124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c10d370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:35.196 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5133200700 1 -- 172.21.15.117:0/2746485124 >> 172.21.15.117:0/2746485124 conn(0x7f512c07b9f0 msgr2=0x7f512c07de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:35.196 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5133200700 1 -- 172.21.15.117:0/2746485124 shutdown_connections 2024-04-20T10:56:35.196 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.234+0000 7f5133200700 1 -- 172.21.15.117:0/2746485124 wait complete. 2024-04-20T10:56:35.196 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.235+0000 7f5133200700 1 Processor -- start 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.236+0000 7f5133200700 1 -- start start 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.236+0000 7f5133200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.236+0000 7f5133200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c1a0bf0 con 0x7f512c10cf50 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.236+0000 7f5131e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.236+0000 7f5131e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49410/0 (socket says 172.21.15.117:49410) 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.236+0000 7f5131e00700 1 -- 172.21.15.117:0/1481347009 learned_addr learned my addr 172.21.15.117:0/1481347009 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.237+0000 7f5131e00700 1 -- 172.21.15.117:0/1481347009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5120009550 con 0x7f512c10cf50 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.237+0000 7f5131e00700 1 --2- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c1a06b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f51200117b0 tx=0x7f5120011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.237+0000 7f5126a00700 1 -- 172.21.15.117:0/1481347009 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51200038b0 con 0x7f512c10cf50 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.237+0000 7f5126a00700 1 -- 172.21.15.117:0/1481347009 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5120003a10 con 0x7f512c10cf50 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.237+0000 7f5126a00700 1 -- 172.21.15.117:0/1481347009 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51200246e0 con 0x7f512c10cf50 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.237+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c1a0df0 con 0x7f512c10cf50 2024-04-20T10:56:35.197 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.237+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c1a1290 con 0x7f512c10cf50 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.238+0000 7f5126a00700 1 -- 172.21.15.117:0/1481347009 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f512001ab80 con 0x7f512c10cf50 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.239+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c10e930 con 0x7f512c10cf50 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.239+0000 7f5126a00700 1 --2- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f51140387e0 0x7f511403aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.239+0000 7f5126a00700 1 -- 172.21.15.117:0/1481347009 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f512004c930 con 0x7f512c10cf50 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.239+0000 7f5131400700 1 --2- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f51140387e0 0x7f511403aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.240+0000 7f5131400700 1 --2- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f51140387e0 0x7f511403aca0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f511c006fd0 tx=0x7f511c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.245+0000 7f5126a00700 1 -- 172.21.15.117:0/1481347009 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5120018ba0 con 0x7f512c10cf50 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.464+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f512c1a3b20 con 0x7f512c10cf50 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.472+0000 7f5126a00700 1 -- 172.21.15.117:0/1481347009 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f51200187c0 con 0x7f512c10cf50 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f51140387e0 msgr2=0x7f511403aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 --2- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f51140387e0 0x7f511403aca0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f511c006fd0 tx=0x7f511c006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 msgr2=0x7f512c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:35.198 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 --2- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c1a06b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f51200117b0 tx=0x7f5120011890 comp rx=0 tx=0).stop 2024-04-20T10:56:35.199 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 shutdown_connections 2024-04-20T10:56:35.199 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 --2- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f51140387e0 0x7f511403aca0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:35.199 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 --2- 172.21.15.117:0/1481347009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f512c10cf50 0x7f512c1a06b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:35.199 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 >> 172.21.15.117:0/1481347009 conn(0x7f512c07b9f0 msgr2=0x7f512c07c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:35.199 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.475+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 shutdown_connections 2024-04-20T10:56:35.199 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:34.476+0000 7f5133200700 1 -- 172.21.15.117:0/1481347009 wait complete. 2024-04-20T10:56:35.737 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:35 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1481347009' entity='client.admin' 2024-04-20T10:56:35.737 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:35 smithi117 ceph-mon[15659]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.812+0000 7f0c82400700 1 Processor -- start 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.812+0000 7f0c82400700 1 -- start start 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.812+0000 7f0c82400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c106820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.812+0000 7f0c82400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c106df0 con 0x7f0c7c106400 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.813+0000 7f0c81000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.813+0000 7f0c81000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c106820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49418/0 (socket says 172.21.15.117:49418) 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.813+0000 7f0c81000700 1 -- 172.21.15.117:0/2679475705 learned_addr learned my addr 172.21.15.117:0/2679475705 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.813+0000 7f0c81000700 1 -- 172.21.15.117:0/2679475705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c107600 con 0x7f0c7c106400 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.814+0000 7f0c81000700 1 --2- 172.21.15.117:0/2679475705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c106820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009e80 tx=0x7f0c7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600ce0addab3656b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.814+0000 7f0c7b400700 1 -- 172.21.15.117:0/2679475705 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c7000e070 con 0x7f0c7c106400 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.814+0000 7f0c7b400700 1 -- 172.21.15.117:0/2679475705 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c7000b8c0 con 0x7f0c7c106400 2024-04-20T10:56:36.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.814+0000 7f0c7b400700 1 -- 172.21.15.117:0/2679475705 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c700044d0 con 0x7f0c7c106400 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.815+0000 7f0c82400700 1 -- 172.21.15.117:0/2679475705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 msgr2=0x7f0c7c106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.815+0000 7f0c82400700 1 --2- 172.21.15.117:0/2679475705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c106820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009e80 tx=0x7f0c7000b330 comp rx=0 tx=0).stop 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.815+0000 7f0c82400700 1 -- 172.21.15.117:0/2679475705 shutdown_connections 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.815+0000 7f0c82400700 1 --2- 172.21.15.117:0/2679475705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c106820 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.815+0000 7f0c82400700 1 -- 172.21.15.117:0/2679475705 >> 172.21.15.117:0/2679475705 conn(0x7f0c7c101a40 msgr2=0x7f0c7c103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.815+0000 7f0c82400700 1 -- 172.21.15.117:0/2679475705 shutdown_connections 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.815+0000 7f0c82400700 1 -- 172.21.15.117:0/2679475705 wait complete. 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.816+0000 7f0c82400700 1 Processor -- start 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.816+0000 7f0c82400700 1 -- start start 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.817+0000 7f0c82400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.817+0000 7f0c82400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c19c830 con 0x7f0c7c106400 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.817+0000 7f0c81000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:36.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.817+0000 7f0c81000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49432/0 (socket says 172.21.15.117:49432) 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.817+0000 7f0c81000700 1 -- 172.21.15.117:0/120707849 learned_addr learned my addr 172.21.15.117:0/120707849 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.817+0000 7f0c81000700 1 -- 172.21.15.117:0/120707849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70009b30 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.818+0000 7f0c81000700 1 --2- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c19c2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c70000f80 tx=0x7f0c7000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.818+0000 7f0c79600700 1 -- 172.21.15.117:0/120707849 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c7000e040 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.818+0000 7f0c79600700 1 -- 172.21.15.117:0/120707849 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c70003c30 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.818+0000 7f0c79600700 1 -- 172.21.15.117:0/120707849 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c700125e0 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.818+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c19ca30 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.818+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c19cf30 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.819+0000 7f0c79600700 1 -- 172.21.15.117:0/120707849 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f0c70019040 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.819+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7c10a830 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.820+0000 7f0c79600700 1 --2- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0c64038830 0x7f0c6403acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.820+0000 7f0c79600700 1 -- 172.21.15.117:0/120707849 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0c7004c0d0 con 0x7f0c7c106400 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.820+0000 7f0c7be00700 1 --2- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0c64038830 0x7f0c6403acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:36.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.821+0000 7f0c7be00700 1 --2- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0c64038830 0x7f0c6403acf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006fd0 tx=0x7f0c68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:35.825+0000 7f0c79600700 1 -- 172.21.15.117:0/120707849 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0c7007fc50 con 0x7f0c7c106400 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.044+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f0c7c19f760 con 0x7f0c7c106400 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.051+0000 7f0c79600700 1 -- 172.21.15.117:0/120707849 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f0c7001f0c0 con 0x7f0c7c106400 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.054+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0c64038830 msgr2=0x7f0c6403acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.054+0000 7f0c82400700 1 --2- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0c64038830 0x7f0c6403acf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006fd0 tx=0x7f0c68006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.054+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 msgr2=0x7f0c7c19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.054+0000 7f0c82400700 1 --2- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c19c2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c70000f80 tx=0x7f0c7000bcf0 comp rx=0 tx=0).stop 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.055+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 shutdown_connections 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.055+0000 7f0c82400700 1 --2- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0c64038830 0x7f0c6403acf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.055+0000 7f0c82400700 1 --2- 172.21.15.117:0/120707849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c7c106400 0x7f0c7c19c2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.055+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 >> 172.21.15.117:0/120707849 conn(0x7f0c7c101a40 msgr2=0x7f0c7c1024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.055+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 shutdown_connections 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:36.055+0000 7f0c82400700 1 -- 172.21.15.117:0/120707849 wait complete. 2024-04-20T10:56:36.721 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling the dashboard module... 2024-04-20T10:56:37.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:37 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/120707849' entity='client.admin' 2024-04-20T10:56:38.370 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:38 smithi117 ceph-mon[15659]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:56:38.370 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:38 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3728644138' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-20T10:56:38.838 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.333+0000 7f0980200700 1 Processor -- start 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.333+0000 7f0980200700 1 -- start start 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.334+0000 7f0980200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f09780791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.334+0000 7f0980200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09780797c0 con 0x7f097807ad90 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.334+0000 7f097d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f09780791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.334+0000 7f097d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f09780791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49434/0 (socket says 172.21.15.117:49434) 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.334+0000 7f097d800700 1 -- 172.21.15.117:0/767978606 learned_addr learned my addr 172.21.15.117:0/767978606 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.335+0000 7f097d800700 1 -- 172.21.15.117:0/767978606 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0978079900 con 0x7f097807ad90 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.335+0000 7f097d800700 1 --2- 172.21.15.117:0/767978606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f09780791f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f096c0098a0 tx=0x7f096c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ad3d4c1271f7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:38.841 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.335+0000 7f0977e00700 1 -- 172.21.15.117:0/767978606 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f096c004180 con 0x7f097807ad90 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.335+0000 7f0977e00700 1 -- 172.21.15.117:0/767978606 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f096c0042e0 con 0x7f097807ad90 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.335+0000 7f0977e00700 1 -- 172.21.15.117:0/767978606 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f096c011b10 con 0x7f097807ad90 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.336+0000 7f0980200700 1 -- 172.21.15.117:0/767978606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 msgr2=0x7f09780791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.336+0000 7f0980200700 1 --2- 172.21.15.117:0/767978606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f09780791f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f096c0098a0 tx=0x7f096c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.336+0000 7f0980200700 1 -- 172.21.15.117:0/767978606 shutdown_connections 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.336+0000 7f0980200700 1 --2- 172.21.15.117:0/767978606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f09780791f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.336+0000 7f0980200700 1 -- 172.21.15.117:0/767978606 >> 172.21.15.117:0/767978606 conn(0x7f0978101bf0 msgr2=0x7f0978104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.336+0000 7f0980200700 1 -- 172.21.15.117:0/767978606 shutdown_connections 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.336+0000 7f0980200700 1 -- 172.21.15.117:0/767978606 wait complete. 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.337+0000 7f0980200700 1 Processor -- start 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.337+0000 7f0980200700 1 -- start start 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.338+0000 7f0980200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f097819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:38.842 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.338+0000 7f0980200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097819ca70 con 0x7f097807ad90 2024-04-20T10:56:38.843 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.338+0000 7f097d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f097819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:38.843 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.338+0000 7f097d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f097819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49442/0 (socket says 172.21.15.117:49442) 2024-04-20T10:56:38.845 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.338+0000 7f097d800700 1 -- 172.21.15.117:0/3728644138 learned_addr learned my addr 172.21.15.117:0/3728644138 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.338+0000 7f097d800700 1 -- 172.21.15.117:0/3728644138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c009550 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.339+0000 7f097d800700 1 --2- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f097819c530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f096c0117b0 tx=0x7f096c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.339+0000 7f0976000700 1 -- 172.21.15.117:0/3728644138 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f096c0038b0 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.339+0000 7f0976000700 1 -- 172.21.15.117:0/3728644138 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f096c003a10 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.339+0000 7f0976000700 1 -- 172.21.15.117:0/3728644138 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f096c024780 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.339+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097819cc70 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.339+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097819d110 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.340+0000 7f0976000700 1 -- 172.21.15.117:0/3728644138 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f096c01ac10 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.340+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097810c9b0 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.341+0000 7f0976000700 1 --2- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0960038770 0x7f096003ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.341+0000 7f0976000700 1 -- 172.21.15.117:0/3728644138 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f096c04c840 con 0x7f097807ad90 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.341+0000 7f097ce00700 1 --2- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0960038770 0x7f096003ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:38.846 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.342+0000 7f097ce00700 1 --2- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0960038770 0x7f096003ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0964006fd0 tx=0x7f0964006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.346+0000 7f0976000700 1 -- 172.21.15.117:0/3728644138 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f096c018760 con 0x7f097807ad90 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:37.624+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f097810cbf0 con 0x7f097807ad90 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.057+0000 7f0976000700 1 -- 172.21.15.117:0/3728644138 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f096c04ec40 con 0x7f097807ad90 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.061+0000 7f0976000700 1 -- 172.21.15.117:0/3728644138 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f096c018760 con 0x7f097807ad90 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.064+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0960038770 msgr2=0x7f096003ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.064+0000 7f0980200700 1 --2- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0960038770 0x7f096003ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0964006fd0 tx=0x7f0964006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.064+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 msgr2=0x7f097819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.064+0000 7f0980200700 1 --2- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f097819c530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f096c0117b0 tx=0x7f096c011890 comp rx=0 tx=0).stop 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.064+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 shutdown_connections 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.064+0000 7f0980200700 1 --2- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f0960038770 0x7f096003ac30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.064+0000 7f0980200700 1 --2- 172.21.15.117:0/3728644138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097807ad90 0x7f097819c530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.064+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 >> 172.21.15.117:0/3728644138 conn(0x7f0978101bf0 msgr2=0x7f0978102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:38.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.065+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 shutdown_connections 2024-04-20T10:56:38.848 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:38.065+0000 7f0980200700 1 -- 172.21.15.117:0/3728644138 wait complete. 2024-04-20T10:56:39.290 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:39 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3728644138' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-20T10:56:39.290 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:39 smithi117 ceph-mon[15659]: mgrmap e9: smithi117.ivnklj(active, since 33s) 2024-04-20T10:56:40.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:40 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3920813835' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T10:56:40.437 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "active_name": "smithi117.ivnklj", 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.465+0000 7f1616200700 1 Processor -- start 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.466+0000 7f1616200700 1 -- start start 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.466+0000 7f1616200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f1610109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.466+0000 7f1616200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100747b0 con 0x7f1610107020 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.466+0000 7f160f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f1610109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:40.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.467+0000 7f160f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f1610109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49482/0 (socket says 172.21.15.117:49482) 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.467+0000 7f160f400700 1 -- 172.21.15.117:0/3623309050 learned_addr learned my addr 172.21.15.117:0/3623309050 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.467+0000 7f160f400700 1 -- 172.21.15.117:0/3623309050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610109950 con 0x7f1610107020 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.467+0000 7f160f400700 1 --2- 172.21.15.117:0/3623309050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f1610109410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16000098a0 tx=0x7f1600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4f79b5b047630 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.468+0000 7f160e000700 1 -- 172.21.15.117:0/3623309050 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1600004180 con 0x7f1610107020 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.468+0000 7f160e000700 1 -- 172.21.15.117:0/3623309050 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16000042e0 con 0x7f1610107020 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.468+0000 7f160e000700 1 -- 172.21.15.117:0/3623309050 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1600011b10 con 0x7f1610107020 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.468+0000 7f1616200700 1 -- 172.21.15.117:0/3623309050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 msgr2=0x7f1610109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.468+0000 7f1616200700 1 --2- 172.21.15.117:0/3623309050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f1610109410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16000098a0 tx=0x7f1600009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.468+0000 7f1616200700 1 -- 172.21.15.117:0/3623309050 shutdown_connections 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.468+0000 7f1616200700 1 --2- 172.21.15.117:0/3623309050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f1610109410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.469+0000 7f1616200700 1 -- 172.21.15.117:0/3623309050 >> 172.21.15.117:0/3623309050 conn(0x7f1610100a50 msgr2=0x7f1610102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.469+0000 7f1616200700 1 -- 172.21.15.117:0/3623309050 shutdown_connections 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.469+0000 7f1616200700 1 -- 172.21.15.117:0/3623309050 wait complete. 2024-04-20T10:56:40.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.470+0000 7f1616200700 1 Processor -- start 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.470+0000 7f1616200700 1 -- start start 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.470+0000 7f1616200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f161019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.470+0000 7f1616200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161019a840 con 0x7f1610107020 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.470+0000 7f160f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f161019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.470+0000 7f160f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f161019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49490/0 (socket says 172.21.15.117:49490) 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.470+0000 7f160f400700 1 -- 172.21.15.117:0/3920813835 learned_addr learned my addr 172.21.15.117:0/3920813835 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.471+0000 7f160f400700 1 -- 172.21.15.117:0/3920813835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600009550 con 0x7f1610107020 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.471+0000 7f160f400700 1 --2- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f161019a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16000117b0 tx=0x7f1600011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.471+0000 7f1607e00700 1 -- 172.21.15.117:0/3920813835 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16000038b0 con 0x7f1610107020 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.471+0000 7f1607e00700 1 -- 172.21.15.117:0/3920813835 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1600003a10 con 0x7f1610107020 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.471+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161019aa40 con 0x7f1610107020 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.471+0000 7f1607e00700 1 -- 172.21.15.117:0/3920813835 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1600024780 con 0x7f1610107020 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.472+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161019aee0 con 0x7f1610107020 2024-04-20T10:56:40.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.473+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161010a740 con 0x7f1610107020 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.473+0000 7f1607e00700 1 -- 172.21.15.117:0/3920813835 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f160001a430 con 0x7f1610107020 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.473+0000 7f1607e00700 1 --2- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f15f4038770 0x7f15f403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.473+0000 7f1607e00700 1 -- 172.21.15.117:0/3920813835 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f160004c8a0 con 0x7f1610107020 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.474+0000 7f160ea00700 1 -- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f15f4038770 msgr2=0x7f15f403ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2597372910 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.474+0000 7f160ea00700 1 --2- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f15f4038770 0x7f15f403ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.483+0000 7f1607e00700 1 -- 172.21.15.117:0/3920813835 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1600018350 con 0x7f1610107020 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.675+0000 7f160ea00700 1 -- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f15f4038770 msgr2=0x7f15f403ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2597372910 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.675+0000 7f160ea00700 1 --2- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f15f4038770 0x7f15f403ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.766+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f161019d770 con 0x7f1610107020 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.767+0000 7f1607e00700 1 -- 172.21.15.117:0/3920813835 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f1600018350 con 0x7f1610107020 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.769+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f15f4038770 msgr2=0x7f15f403ac30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.769+0000 7f1616200700 1 --2- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f15f4038770 0x7f15f403ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:40.441 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.769+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 msgr2=0x7f161019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.769+0000 7f1616200700 1 --2- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f161019a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16000117b0 tx=0x7f1600011890 comp rx=0 tx=0).stop 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.769+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 shutdown_connections 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.769+0000 7f1616200700 1 --2- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7f15f4038770 0x7f15f403ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.769+0000 7f1616200700 1 --2- 172.21.15.117:0/3920813835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610107020 0x7f161019a300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.769+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 >> 172.21.15.117:0/3920813835 conn(0x7f1610100a50 msgr2=0x7f16101015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.770+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 shutdown_connections 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:39.770+0000 7f1616200700 1 -- 172.21.15.117:0/3920813835 wait complete. 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for the mgr to restart... 2024-04-20T10:56:40.442 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr epoch 9... 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: Active manager daemon smithi117.ivnklj restarted 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: Activating manager daemon smithi117.ivnklj 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: osdmap e3: 0 total, 0 up, 0 in 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: mgrmap e10: smithi117.ivnklj(active, starting, since 0.00527562s) 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:56:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:56:47.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:46 smithi117 ceph-mon[15659]: Manager daemon smithi117.ivnklj is now available 2024-04-20T10:56:48.233 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:47 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:56:48.234 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:47 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:56:48.234 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:47 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/trash_purge_schedule"}]: dispatch 2024-04-20T10:56:48.234 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:47 smithi117 ceph-mon[15659]: mgrmap e11: smithi117.ivnklj(active, since 1.01329s) 2024-04-20T10:56:49.119 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:48 smithi117 ceph-mon[15659]: [20/Apr/2024:10:56:47] ENGINE Bus STARTING 2024-04-20T10:56:49.119 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:48 smithi117 ceph-mon[15659]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T10:56:49.119 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:48 smithi117 ceph-mon[15659]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T10:56:49.119 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:48 smithi117 ceph-mon[15659]: [20/Apr/2024:10:56:47] ENGINE Serving on https://172.21.15.117:7150 2024-04-20T10:56:49.119 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:48 smithi117 ceph-mon[15659]: [20/Apr/2024:10:56:48] ENGINE Serving on http://172.21.15.117:8765 2024-04-20T10:56:49.119 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:48 smithi117 ceph-mon[15659]: [20/Apr/2024:10:56:48] ENGINE Bus STARTED 2024-04-20T10:56:49.126 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:56:49.126 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-20T10:56:49.126 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.045+0000 7fc144800700 1 Processor -- start 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.045+0000 7fc144800700 1 -- start start 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.046+0000 7fc144800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c0791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.046+0000 7fc144800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c0797c0 con 0x7fc13c07ad90 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.046+0000 7fc141e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c0791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.046+0000 7fc141e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c0791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49504/0 (socket says 172.21.15.117:49504) 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.046+0000 7fc141e00700 1 -- 172.21.15.117:0/2683426956 learned_addr learned my addr 172.21.15.117:0/2683426956 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.047+0000 7fc141e00700 1 -- 172.21.15.117:0/2683426956 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c079900 con 0x7fc13c07ad90 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.047+0000 7fc141e00700 1 --2- 172.21.15.117:0/2683426956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c0791f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc1300098a0 tx=0x7fc130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2437fac71da9f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.047+0000 7fc140a00700 1 -- 172.21.15.117:0/2683426956 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc130004180 con 0x7fc13c07ad90 2024-04-20T10:56:49.127 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.047+0000 7fc140a00700 1 -- 172.21.15.117:0/2683426956 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1300042e0 con 0x7fc13c07ad90 2024-04-20T10:56:49.133 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.047+0000 7fc140a00700 1 -- 172.21.15.117:0/2683426956 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc130011b10 con 0x7fc13c07ad90 2024-04-20T10:56:49.133 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.048+0000 7fc144800700 1 -- 172.21.15.117:0/2683426956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 msgr2=0x7fc13c0791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.048+0000 7fc144800700 1 --2- 172.21.15.117:0/2683426956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c0791f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc1300098a0 tx=0x7fc130009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.048+0000 7fc144800700 1 -- 172.21.15.117:0/2683426956 shutdown_connections 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.048+0000 7fc144800700 1 --2- 172.21.15.117:0/2683426956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c0791f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.048+0000 7fc144800700 1 -- 172.21.15.117:0/2683426956 >> 172.21.15.117:0/2683426956 conn(0x7fc13c101ba0 msgr2=0x7fc13c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.048+0000 7fc144800700 1 -- 172.21.15.117:0/2683426956 shutdown_connections 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.048+0000 7fc144800700 1 -- 172.21.15.117:0/2683426956 wait complete. 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.049+0000 7fc144800700 1 Processor -- start 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.050+0000 7fc144800700 1 -- start start 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.050+0000 7fc144800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.050+0000 7fc144800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c19cad0 con 0x7fc13c07ad90 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.050+0000 7fc141e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.050+0000 7fc141e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49506/0 (socket says 172.21.15.117:49506) 2024-04-20T10:56:49.134 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.050+0000 7fc141e00700 1 -- 172.21.15.117:0/2403386461 learned_addr learned my addr 172.21.15.117:0/2403386461 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.051+0000 7fc141e00700 1 -- 172.21.15.117:0/2403386461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc130009550 con 0x7fc13c07ad90 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.051+0000 7fc141e00700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c19c590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc130011670 tx=0x7fc130011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.051+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1300038b0 con 0x7fc13c07ad90 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.051+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc130003a10 con 0x7fc13c07ad90 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.051+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c19ccd0 con 0x7fc13c07ad90 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.051+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc13001a7c0 con 0x7fc13c07ad90 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.051+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c19d170 con 0x7fc13c07ad90 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.053+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fc130024430 con 0x7fc13c07ad90 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.053+0000 7fc136a00700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 0x7fc12403ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.053+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 --> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc12403b350 con 0x7fc124038780 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.053+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc13004ca00 con 0x7fc13c07ad90 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.053+0000 7fc141400700 1 -- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 msgr2=0x7fc12403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2597372910 2024-04-20T10:56:49.135 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.053+0000 7fc141400700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 0x7fc12403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.254+0000 7fc141400700 1 -- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 msgr2=0x7fc12403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2597372910 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.254+0000 7fc141400700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 0x7fc12403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.655+0000 7fc141400700 1 -- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 msgr2=0x7fc12403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2597372910 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:41.655+0000 7fc141400700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 0x7fc12403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:42.456+0000 7fc141400700 1 -- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 msgr2=0x7fc12403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2597372910 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:42.456+0000 7fc141400700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 0x7fc12403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:44.058+0000 7fc141400700 1 -- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 msgr2=0x7fc12403ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2597372910 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:44.058+0000 7fc141400700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 0x7fc12403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:46.891+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mgrmap(e 10) v1 ==== 45336+0+0 (secure 0 0 0) 0x7fc130024c10 con 0x7fc13c07ad90 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:46.891+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 msgr2=0x7fc12403ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:46.891+0000 7fc136a00700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 0x7fc12403ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.895+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 11) v1 ==== 45463+0+0 (secure 0 0 0) 0x7fc13004e170 con 0x7fc13c07ad90 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.895+0000 7fc136a00700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fc12403bfd0 0x7fc12403e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:49.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.895+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 --> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc12403b350 con 0x7fc12403bfd0 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.897+0000 7fc141400700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fc12403bfd0 0x7fc12403e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.898+0000 7fc141400700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fc12403bfd0 0x7fc12403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc12c003a10 tx=0x7fc12c009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.899+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mgr.14164 v2:172.21.15.117:6800/1996106837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fc12403b350 con 0x7fc12403bfd0 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.906+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 --> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc13c10c980 con 0x7fc12403bfd0 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.907+0000 7fc136a00700 1 -- 172.21.15.117:0/2403386461 <== mgr.14164 v2:172.21.15.117:6800/1996106837 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc13c10c980 con 0x7fc12403bfd0 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.907+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fc12403bfd0 msgr2=0x7fc12403e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.907+0000 7fc144800700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fc12403bfd0 0x7fc12403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc12c003a10 tx=0x7fc12c009270 comp rx=0 tx=0).stop 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.907+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 msgr2=0x7fc13c19c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.907+0000 7fc144800700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c19c590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc130011670 tx=0x7fc130011890 comp rx=0 tx=0).stop 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.908+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 shutdown_connections 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.908+0000 7fc144800700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fc12403bfd0 0x7fc12403e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.908+0000 7fc144800700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc13c07ad90 0x7fc13c19c590 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.908+0000 7fc144800700 1 --2- 172.21.15.117:0/2403386461 >> [v2:172.21.15.117:6800/2597372910,v1:172.21.15.117:6801/2597372910] conn(0x7fc124038780 0x7fc12403ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:49.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.908+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 >> 172.21.15.117:0/2403386461 conn(0x7fc13c101ba0 msgr2=0x7fc13c1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:49.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.908+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 shutdown_connections 2024-04-20T10:56:49.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:47.908+0000 7fc144800700 1 -- 172.21.15.117:0/2403386461 wait complete. 2024-04-20T10:56:49.138 INFO:teuthology.orchestra.run.smithi117.stdout:mgr epoch 9 is available 2024-04-20T10:56:49.138 INFO:teuthology.orchestra.run.smithi117.stdout:Generating a dashboard self-signed certificate... 2024-04-20T10:56:50.007 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:49 smithi117 ceph-mon[15659]: mgrmap e12: smithi117.ivnklj(active, since 2s) 2024-04-20T10:56:50.008 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:49 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:50.921 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-20T10:56:50.921 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.763+0000 7fcb83c00700 1 Processor -- start 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.763+0000 7fcb83c00700 1 -- start start 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.763+0000 7fcb83c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.763+0000 7fcb83c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c0747b0 con 0x7fcb7c107020 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.764+0000 7fcb81200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.764+0000 7fcb81200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56368/0 (socket says 172.21.15.117:56368) 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.764+0000 7fcb81200700 1 -- 172.21.15.117:0/2755169791 learned_addr learned my addr 172.21.15.117:0/2755169791 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.764+0000 7fcb81200700 1 -- 172.21.15.117:0/2755169791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c109950 con 0x7fcb7c107020 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.765+0000 7fcb81200700 1 --2- 172.21.15.117:0/2755169791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c109410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb700098a0 tx=0x7fcb70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b2f7ae787f3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.765+0000 7fcb7b400700 1 -- 172.21.15.117:0/2755169791 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb70004180 con 0x7fcb7c107020 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.765+0000 7fcb7b400700 1 -- 172.21.15.117:0/2755169791 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb700042e0 con 0x7fcb7c107020 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.765+0000 7fcb7b400700 1 -- 172.21.15.117:0/2755169791 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb70011b10 con 0x7fcb7c107020 2024-04-20T10:56:50.922 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.766+0000 7fcb83c00700 1 -- 172.21.15.117:0/2755169791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 msgr2=0x7fcb7c109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.766+0000 7fcb83c00700 1 --2- 172.21.15.117:0/2755169791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c109410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb700098a0 tx=0x7fcb70009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.766+0000 7fcb83c00700 1 -- 172.21.15.117:0/2755169791 shutdown_connections 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.766+0000 7fcb83c00700 1 --2- 172.21.15.117:0/2755169791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c109410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.766+0000 7fcb83c00700 1 -- 172.21.15.117:0/2755169791 >> 172.21.15.117:0/2755169791 conn(0x7fcb7c100a50 msgr2=0x7fcb7c102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.766+0000 7fcb83c00700 1 -- 172.21.15.117:0/2755169791 shutdown_connections 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.766+0000 7fcb83c00700 1 -- 172.21.15.117:0/2755169791 wait complete. 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.767+0000 7fcb83c00700 1 Processor -- start 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.767+0000 7fcb83c00700 1 -- start start 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.767+0000 7fcb83c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.767+0000 7fcb83c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c19ca60 con 0x7fcb7c107020 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.768+0000 7fcb81200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.768+0000 7fcb81200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56370/0 (socket says 172.21.15.117:56370) 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.768+0000 7fcb81200700 1 -- 172.21.15.117:0/256338076 learned_addr learned my addr 172.21.15.117:0/256338076 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:50.923 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.768+0000 7fcb81200700 1 -- 172.21.15.117:0/256338076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb70009550 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.768+0000 7fcb81200700 1 --2- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c19c520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb700117b0 tx=0x7fcb70011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.769+0000 7fcb79600700 1 -- 172.21.15.117:0/256338076 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb700038b0 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.769+0000 7fcb79600700 1 -- 172.21.15.117:0/256338076 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb70003a10 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.769+0000 7fcb79600700 1 -- 172.21.15.117:0/256338076 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb70024780 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.769+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c19cc60 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.769+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c19d100 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.770+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c10a740 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.771+0000 7fcb79600700 1 -- 172.21.15.117:0/256338076 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fcb7001ac10 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.771+0000 7fcb79600700 1 --2- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fcb640387c0 0x7fcb6403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.771+0000 7fcb79600700 1 -- 172.21.15.117:0/256338076 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcb7004c980 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.772+0000 7fcb7be00700 1 --2- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fcb640387c0 0x7fcb6403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.772+0000 7fcb7be00700 1 --2- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fcb640387c0 0x7fcb6403ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb68006fd0 tx=0x7fcb68006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:49.777+0000 7fcb79600700 1 -- 172.21.15.117:0/256338076 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcb70018760 con 0x7fcb7c107020 2024-04-20T10:56:50.924 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.006+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 --> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb7c101ce0 con 0x7fcb640387c0 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.207+0000 7fcb79600700 1 -- 172.21.15.117:0/256338076 <== mgr.14164 v2:172.21.15.117:6800/1996106837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fcb7c101ce0 con 0x7fcb640387c0 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.210+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fcb640387c0 msgr2=0x7fcb6403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.210+0000 7fcb83c00700 1 --2- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fcb640387c0 0x7fcb6403ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb68006fd0 tx=0x7fcb68006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.210+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 msgr2=0x7fcb7c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.210+0000 7fcb83c00700 1 --2- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c19c520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb700117b0 tx=0x7fcb70011890 comp rx=0 tx=0).stop 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.210+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 shutdown_connections 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.210+0000 7fcb83c00700 1 --2- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fcb640387c0 0x7fcb6403ac80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.210+0000 7fcb83c00700 1 --2- 172.21.15.117:0/256338076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb7c107020 0x7fcb7c19c520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.210+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 >> 172.21.15.117:0/256338076 conn(0x7fcb7c100a50 msgr2=0x7fcb7c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.211+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 shutdown_connections 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:50.211+0000 7fcb83c00700 1 -- 172.21.15.117:0/256338076 wait complete. 2024-04-20T10:56:50.925 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial admin user... 2024-04-20T10:56:51.454 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:51 smithi117 ceph-mon[15659]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:51.454 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:51.454 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:51.454 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:52.959 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fQ2ol61B8XGCpVUM4a6DbeSScpgwlASxtbqK4KoJa/FFGht4nt9i6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713610612, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-20T10:56:52.959 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.535+0000 7fb706e00700 1 Processor -- start 2024-04-20T10:56:52.959 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.535+0000 7fb706e00700 1 -- start start 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.535+0000 7fb706e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb700106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.535+0000 7fb706e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb700106fd0 con 0x7fb7001065e0 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.536+0000 7fb6ffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb700106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.536+0000 7fb6ffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb700106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56384/0 (socket says 172.21.15.117:56384) 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.536+0000 7fb6ffe00700 1 -- 172.21.15.117:0/2038632106 learned_addr learned my addr 172.21.15.117:0/2038632106 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.536+0000 7fb6ffe00700 1 -- 172.21.15.117:0/2038632106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7001077e0 con 0x7fb7001065e0 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.537+0000 7fb6ffe00700 1 --2- 172.21.15.117:0/2038632106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb700106a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00098a0 tx=0x7fb6f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c47c49e865da8a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.537+0000 7fb6fea00700 1 -- 172.21.15.117:0/2038632106 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f0004180 con 0x7fb7001065e0 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.537+0000 7fb6fea00700 1 -- 172.21.15.117:0/2038632106 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6f00042e0 con 0x7fb7001065e0 2024-04-20T10:56:52.960 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.537+0000 7fb6fea00700 1 -- 172.21.15.117:0/2038632106 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f0011b10 con 0x7fb7001065e0 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.538+0000 7fb706e00700 1 -- 172.21.15.117:0/2038632106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 msgr2=0x7fb700106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.538+0000 7fb706e00700 1 --2- 172.21.15.117:0/2038632106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb700106a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00098a0 tx=0x7fb6f0009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.538+0000 7fb706e00700 1 -- 172.21.15.117:0/2038632106 shutdown_connections 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.538+0000 7fb706e00700 1 --2- 172.21.15.117:0/2038632106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb700106a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.538+0000 7fb706e00700 1 -- 172.21.15.117:0/2038632106 >> 172.21.15.117:0/2038632106 conn(0x7fb700101c00 msgr2=0x7fb700104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.538+0000 7fb706e00700 1 -- 172.21.15.117:0/2038632106 shutdown_connections 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.538+0000 7fb706e00700 1 -- 172.21.15.117:0/2038632106 wait complete. 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.539+0000 7fb706e00700 1 Processor -- start 2024-04-20T10:56:52.961 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.540+0000 7fb706e00700 1 -- start start 2024-04-20T10:56:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.540+0000 7fb706e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb70019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.540+0000 7fb706e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70019ca90 con 0x7fb7001065e0 2024-04-20T10:56:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.540+0000 7fb6ffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb70019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.540+0000 7fb6ffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb70019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56394/0 (socket says 172.21.15.117:56394) 2024-04-20T10:56:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.540+0000 7fb6ffe00700 1 -- 172.21.15.117:0/3678198452 learned_addr learned my addr 172.21.15.117:0/3678198452 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.541+0000 7fb6ffe00700 1 -- 172.21.15.117:0/3678198452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f0009550 con 0x7fb7001065e0 2024-04-20T10:56:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.541+0000 7fb6ffe00700 1 --2- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb70019c550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00117b0 tx=0x7fb6f0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.541+0000 7fb6fcc00700 1 -- 172.21.15.117:0/3678198452 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f00038b0 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.541+0000 7fb6fcc00700 1 -- 172.21.15.117:0/3678198452 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6f0003a10 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.541+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70019cc90 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.541+0000 7fb6fcc00700 1 -- 172.21.15.117:0/3678198452 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f00246e0 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.542+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70019d130 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.543+0000 7fb6fcc00700 1 -- 172.21.15.117:0/3678198452 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fb6f002d3f0 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.543+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70010aa10 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.544+0000 7fb6fcc00700 1 --2- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fb6e4038770 0x7fb6e403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.544+0000 7fb6fcc00700 1 -- 172.21.15.117:0/3678198452 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb6f004cb00 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.544+0000 7fb6ff400700 1 --2- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fb6e4038770 0x7fb6e403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.545+0000 7fb6ff400700 1 --2- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fb6e4038770 0x7fb6e403ac30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec006fd0 tx=0x7fb6ec006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.549+0000 7fb6fcc00700 1 -- 172.21.15.117:0/3678198452 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb6f0010380 con 0x7fb7001065e0 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:51.808+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 --> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb700062330 con 0x7fb6e4038770 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.076+0000 7fb6fcc00700 1 -- 172.21.15.117:0/3678198452 <== mgr.14164 v2:172.21.15.117:6800/1996106837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fb700062330 con 0x7fb6e4038770 2024-04-20T10:56:52.963 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.079+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fb6e4038770 msgr2=0x7fb6e403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.079+0000 7fb706e00700 1 --2- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fb6e4038770 0x7fb6e403ac30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec006fd0 tx=0x7fb6ec006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.079+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 msgr2=0x7fb70019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.079+0000 7fb706e00700 1 --2- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb70019c550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00117b0 tx=0x7fb6f0011890 comp rx=0 tx=0).stop 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.080+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 shutdown_connections 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.080+0000 7fb706e00700 1 --2- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fb6e4038770 0x7fb6e403ac30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.080+0000 7fb706e00700 1 --2- 172.21.15.117:0/3678198452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7001065e0 0x7fb70019c550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.080+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 >> 172.21.15.117:0/3678198452 conn(0x7fb700101c00 msgr2=0x7fb7001026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.080+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 shutdown_connections 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:52.080+0000 7fb706e00700 1 -- 172.21.15.117:0/3678198452 wait complete. 2024-04-20T10:56:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:Fetching dashboard port number... 2024-04-20T10:56:53.249 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:53 smithi117 ceph-mon[15659]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:56:53.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:53 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:56:54.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:54 smithi117 ceph-mon[15659]: mgrmap e13: smithi117.ivnklj(active, since 6s) 2024-04-20T10:56:54.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:54 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2141140587' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-20T10:56:54.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 8443 2024-04-20T10:56:54.536 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.617+0000 7fa83d600700 1 Processor -- start 2024-04-20T10:56:54.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.618+0000 7fa83d600700 1 -- start start 2024-04-20T10:56:54.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.618+0000 7fa83d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa8381069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:54.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.618+0000 7fa83d600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa838106f90 con 0x7fa8381065a0 2024-04-20T10:56:54.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.618+0000 7fa836a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa8381069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:54.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.618+0000 7fa836a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa8381069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56400/0 (socket says 172.21.15.117:56400) 2024-04-20T10:56:54.537 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.618+0000 7fa836a00700 1 -- 172.21.15.117:0/918569775 learned_addr learned my addr 172.21.15.117:0/918569775 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:54.542 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.619+0000 7fa836a00700 1 -- 172.21.15.117:0/918569775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8381077a0 con 0x7fa8381065a0 2024-04-20T10:56:54.542 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.619+0000 7fa836a00700 1 --2- 172.21.15.117:0/918569775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa8381069c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa8280098a0 tx=0x7fa828009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a28c9468ecaf704 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:54.542 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.619+0000 7fa836000700 1 -- 172.21.15.117:0/918569775 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa828004180 con 0x7fa8381065a0 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.619+0000 7fa836000700 1 -- 172.21.15.117:0/918569775 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8280042e0 con 0x7fa8381065a0 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.619+0000 7fa836000700 1 -- 172.21.15.117:0/918569775 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa828011b10 con 0x7fa8381065a0 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.620+0000 7fa83d600700 1 -- 172.21.15.117:0/918569775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 msgr2=0x7fa8381069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.620+0000 7fa83d600700 1 --2- 172.21.15.117:0/918569775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa8381069c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa8280098a0 tx=0x7fa828009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.620+0000 7fa83d600700 1 -- 172.21.15.117:0/918569775 shutdown_connections 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.620+0000 7fa83d600700 1 --2- 172.21.15.117:0/918569775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa8381069c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.620+0000 7fa83d600700 1 -- 172.21.15.117:0/918569775 >> 172.21.15.117:0/918569775 conn(0x7fa838101b80 msgr2=0x7fa838103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.620+0000 7fa83d600700 1 -- 172.21.15.117:0/918569775 shutdown_connections 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.620+0000 7fa83d600700 1 -- 172.21.15.117:0/918569775 wait complete. 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.621+0000 7fa83d600700 1 Processor -- start 2024-04-20T10:56:54.543 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.622+0000 7fa83d600700 1 -- start start 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.622+0000 7fa83d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa83819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.622+0000 7fa83d600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83819a870 con 0x7fa8381065a0 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.622+0000 7fa836a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa83819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.622+0000 7fa836a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa83819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56416/0 (socket says 172.21.15.117:56416) 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.622+0000 7fa836a00700 1 -- 172.21.15.117:0/2141140587 learned_addr learned my addr 172.21.15.117:0/2141140587 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.623+0000 7fa836a00700 1 -- 172.21.15.117:0/2141140587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828009550 con 0x7fa8381065a0 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.623+0000 7fa836a00700 1 --2- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa83819a330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8280117b0 tx=0x7fa828011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.623+0000 7fa82fe00700 1 -- 172.21.15.117:0/2141140587 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8280038b0 con 0x7fa8381065a0 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.623+0000 7fa82fe00700 1 -- 172.21.15.117:0/2141140587 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa828003a10 con 0x7fa8381065a0 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.623+0000 7fa82fe00700 1 -- 172.21.15.117:0/2141140587 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8280246e0 con 0x7fa8381065a0 2024-04-20T10:56:54.544 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.623+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83819aa70 con 0x7fa8381065a0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.623+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83819af10 con 0x7fa8381065a0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.624+0000 7fa82fe00700 1 -- 172.21.15.117:0/2141140587 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7fa82801ab80 con 0x7fa8381065a0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.624+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83810a9d0 con 0x7fa8381065a0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.625+0000 7fa82fe00700 1 --2- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fa81c038860 0x7fa81c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.625+0000 7fa82fe00700 1 -- 172.21.15.117:0/2141140587 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa82804ca10 con 0x7fa8381065a0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.625+0000 7fa82e000700 1 --2- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fa81c038860 0x7fa81c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.626+0000 7fa82e000700 1 --2- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fa81c038860 0x7fa81c03ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa82000ad30 tx=0x7fa8200093f0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.631+0000 7fa82fe00700 1 -- 172.21.15.117:0/2141140587 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa82801a430 con 0x7fa8381065a0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.850+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fa83819d7a0 con 0x7fa8381065a0 2024-04-20T10:56:54.545 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.851+0000 7fa82fe00700 1 -- 172.21.15.117:0/2141140587 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fa828018b40 con 0x7fa8381065a0 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.854+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fa81c038860 msgr2=0x7fa81c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.854+0000 7fa83d600700 1 --2- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fa81c038860 0x7fa81c03ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa82000ad30 tx=0x7fa8200093f0 comp rx=0 tx=0).stop 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.854+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 msgr2=0x7fa83819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.854+0000 7fa83d600700 1 --2- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa83819a330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8280117b0 tx=0x7fa828011890 comp rx=0 tx=0).stop 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.855+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 shutdown_connections 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.855+0000 7fa83d600700 1 --2- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fa81c038860 0x7fa81c03ad20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.855+0000 7fa83d600700 1 --2- 172.21.15.117:0/2141140587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8381065a0 0x7fa83819a330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.855+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 >> 172.21.15.117:0/2141140587 conn(0x7fa838101b80 msgr2=0x7fa8381026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.855+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 shutdown_connections 2024-04-20T10:56:54.546 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:53.855+0000 7fa83d600700 1 -- 172.21.15.117:0/2141140587 wait complete. 2024-04-20T10:56:54.575 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T10:56:54.575 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-04-20T10:56:54.601 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T10:56:54.602 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-04-20T10:56:54.602 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-04-20T10:56:54.602 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-20T10:56:54.610 INFO:teuthology.orchestra.run.smithi117.stdout:Ceph Dashboard is now available at: 2024-04-20T10:56:54.610 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:54.610 INFO:teuthology.orchestra.run.smithi117.stdout: URL: https://smithi117.front.sepia.ceph.com:8443/ 2024-04-20T10:56:54.610 INFO:teuthology.orchestra.run.smithi117.stdout: User: admin 2024-04-20T10:56:54.610 INFO:teuthology.orchestra.run.smithi117.stdout: Password: jma1lx4ol4 2024-04-20T10:56:54.610 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:54.611 INFO:teuthology.orchestra.run.smithi117.stdout:Saving cluster configuration to /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config directory 2024-04-20T10:56:54.613 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling autotune for osd_memory_target 2024-04-20T10:56:56.172 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.238+0000 7f042a600700 1 Processor -- start 2024-04-20T10:56:56.172 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.238+0000 7f042a600700 1 -- start start 2024-04-20T10:56:56.172 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.239+0000 7f042a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:56.172 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.239+0000 7f042a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0424074710 con 0x7f04241070b0 2024-04-20T10:56:56.172 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.239+0000 7f0423400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.239+0000 7f0423400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35030/0 (socket says 172.21.15.117:35030) 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.239+0000 7f0423400700 1 -- 172.21.15.117:0/2026243920 learned_addr learned my addr 172.21.15.117:0/2026243920 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.240+0000 7f0423400700 1 -- 172.21.15.117:0/2026243920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424074850 con 0x7f04241070b0 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.240+0000 7f0423400700 1 --2- 172.21.15.117:0/2026243920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241094a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f04140098a0 tx=0x7f0414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8946b39f5c5ce2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.240+0000 7f0422000700 1 -- 172.21.15.117:0/2026243920 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0414004180 con 0x7f04241070b0 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.240+0000 7f0422000700 1 -- 172.21.15.117:0/2026243920 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04140042e0 con 0x7f04241070b0 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.240+0000 7f0422000700 1 -- 172.21.15.117:0/2026243920 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0414011b10 con 0x7f04241070b0 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.241+0000 7f042a600700 1 -- 172.21.15.117:0/2026243920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 msgr2=0x7f04241094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.241+0000 7f042a600700 1 --2- 172.21.15.117:0/2026243920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241094a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f04140098a0 tx=0x7f0414009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.241+0000 7f042a600700 1 -- 172.21.15.117:0/2026243920 shutdown_connections 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.241+0000 7f042a600700 1 --2- 172.21.15.117:0/2026243920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241094a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:56.173 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.241+0000 7f042a600700 1 -- 172.21.15.117:0/2026243920 >> 172.21.15.117:0/2026243920 conn(0x7f0424100a60 msgr2=0x7f0424102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.241+0000 7f042a600700 1 -- 172.21.15.117:0/2026243920 shutdown_connections 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.242+0000 7f042a600700 1 -- 172.21.15.117:0/2026243920 wait complete. 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.243+0000 7f042a600700 1 Processor -- start 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.243+0000 7f042a600700 1 -- start start 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.243+0000 7f042a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241068e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.243+0000 7f042a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0424106e20 con 0x7f04241070b0 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.243+0000 7f0423400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.243+0000 7f0423400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35036/0 (socket says 172.21.15.117:35036) 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.243+0000 7f0423400700 1 -- 172.21.15.117:0/1515411751 learned_addr learned my addr 172.21.15.117:0/1515411751 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.244+0000 7f0423400700 1 -- 172.21.15.117:0/1515411751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0414009550 con 0x7f04241070b0 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.244+0000 7f0423400700 1 --2- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241068e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f04140117b0 tx=0x7f0414011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.244+0000 7f041be00700 1 -- 172.21.15.117:0/1515411751 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f04140038b0 con 0x7f04241070b0 2024-04-20T10:56:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.244+0000 7f041be00700 1 -- 172.21.15.117:0/1515411751 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0414003a10 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.244+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0424105060 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.244+0000 7f041be00700 1 -- 172.21.15.117:0/1515411751 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0414024780 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.245+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0424105500 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.246+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042410a700 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.246+0000 7f041be00700 1 -- 172.21.15.117:0/1515411751 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f041401ac10 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.246+0000 7f041be00700 1 --2- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f0408038810 0x7f040803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.246+0000 7f041be00700 1 -- 172.21.15.117:0/1515411751 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f041404ca70 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.246+0000 7f0422a00700 1 --2- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f0408038810 0x7f040803acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.247+0000 7f0422a00700 1 --2- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f0408038810 0x7f040803acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0410006fd0 tx=0x7f0410006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.251+0000 7f041be00700 1 -- 172.21.15.117:0/1515411751 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0414018760 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.475+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f04241059f0 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.476+0000 7f041be00700 1 -- 172.21.15.117:0/1515411751 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f041402c350 con 0x7f04241070b0 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f0408038810 msgr2=0x7f040803acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 --2- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f0408038810 0x7f040803acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0410006fd0 tx=0x7f0410006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:56.176 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 msgr2=0x7f04241068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:56.176 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 --2- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241068e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f04140117b0 tx=0x7f0414011890 comp rx=0 tx=0).stop 2024-04-20T10:56:56.176 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 shutdown_connections 2024-04-20T10:56:56.176 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 --2- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f0408038810 0x7f040803acd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:56.176 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 --2- 172.21.15.117:0/1515411751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04241070b0 0x7f04241068e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:56.176 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 >> 172.21.15.117:0/1515411751 conn(0x7f0424100a60 msgr2=0x7f04241015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:56.176 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 shutdown_connections 2024-04-20T10:56:56.176 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:55.479+0000 7f042a600700 1 -- 172.21.15.117:0/1515411751 wait complete. 2024-04-20T10:56:57.810 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.784+0000 7f6e42c00700 1 Processor -- start 2024-04-20T10:56:57.810 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.784+0000 7f6e42c00700 1 -- start start 2024-04-20T10:56:57.810 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.785+0000 7f6e42c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c1058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:57.810 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.785+0000 7f6e42c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c105e90 con 0x7f6e3c1054a0 2024-04-20T10:56:57.815 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.785+0000 7f6e3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c1058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:57.815 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.785+0000 7f6e3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c1058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35050/0 (socket says 172.21.15.117:35050) 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.785+0000 7f6e3be00700 1 -- 172.21.15.117:0/834311904 learned_addr learned my addr 172.21.15.117:0/834311904 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.786+0000 7f6e3be00700 1 -- 172.21.15.117:0/834311904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c1066a0 con 0x7f6e3c1054a0 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.786+0000 7f6e3be00700 1 --2- 172.21.15.117:0/834311904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c1058c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0098a0 tx=0x7f6e2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8061484ee424754 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.786+0000 7f6e3b400700 1 -- 172.21.15.117:0/834311904 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2c004180 con 0x7f6e3c1054a0 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.786+0000 7f6e3b400700 1 -- 172.21.15.117:0/834311904 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e2c0042e0 con 0x7f6e3c1054a0 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.786+0000 7f6e3b400700 1 -- 172.21.15.117:0/834311904 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2c011b10 con 0x7f6e3c1054a0 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.787+0000 7f6e42c00700 1 -- 172.21.15.117:0/834311904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 msgr2=0x7f6e3c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.787+0000 7f6e42c00700 1 --2- 172.21.15.117:0/834311904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c1058c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0098a0 tx=0x7f6e2c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.787+0000 7f6e42c00700 1 -- 172.21.15.117:0/834311904 shutdown_connections 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.787+0000 7f6e42c00700 1 --2- 172.21.15.117:0/834311904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c1058c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.787+0000 7f6e42c00700 1 -- 172.21.15.117:0/834311904 >> 172.21.15.117:0/834311904 conn(0x7f6e3c100a60 msgr2=0x7f6e3c102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.787+0000 7f6e42c00700 1 -- 172.21.15.117:0/834311904 shutdown_connections 2024-04-20T10:56:57.816 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.787+0000 7f6e42c00700 1 -- 172.21.15.117:0/834311904 wait complete. 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.789+0000 7f6e42c00700 1 Processor -- start 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.789+0000 7f6e42c00700 1 -- start start 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.789+0000 7f6e42c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.789+0000 7f6e42c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c19a8e0 con 0x7f6e3c1054a0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.789+0000 7f6e3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.790+0000 7f6e3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35062/0 (socket says 172.21.15.117:35062) 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.790+0000 7f6e3be00700 1 -- 172.21.15.117:0/3220022247 learned_addr learned my addr 172.21.15.117:0/3220022247 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.790+0000 7f6e3be00700 1 -- 172.21.15.117:0/3220022247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c009550 con 0x7f6e3c1054a0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.790+0000 7f6e3be00700 1 --2- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c19a3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0117b0 tx=0x7f6e2c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.790+0000 7f6e39600700 1 -- 172.21.15.117:0/3220022247 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2c0038b0 con 0x7f6e3c1054a0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.791+0000 7f6e39600700 1 -- 172.21.15.117:0/3220022247 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e2c003a10 con 0x7f6e3c1054a0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.791+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c19aae0 con 0x7f6e3c1054a0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.791+0000 7f6e39600700 1 -- 172.21.15.117:0/3220022247 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2c0246e0 con 0x7f6e3c1054a0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.791+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3c19af80 con 0x7f6e3c1054a0 2024-04-20T10:56:57.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.792+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3c193fa0 con 0x7f6e3c1054a0 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.792+0000 7f6e39600700 1 -- 172.21.15.117:0/3220022247 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f6e2c01ab80 con 0x7f6e3c1054a0 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.793+0000 7f6e39600700 1 --2- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f6e20038810 0x7f6e2003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.793+0000 7f6e39600700 1 -- 172.21.15.117:0/3220022247 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e2c04cb10 con 0x7f6e3c1054a0 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.793+0000 7f6e33e00700 1 --2- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f6e20038810 0x7f6e2003acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.794+0000 7f6e33e00700 1 --2- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f6e20038810 0x7f6e2003acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006fd0 tx=0x7f6e24006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:56.798+0000 7f6e39600700 1 -- 172.21.15.117:0/3220022247 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6e2c01a430 con 0x7f6e3c1054a0 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.127+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f6e3c0738d0 con 0x7f6e3c1054a0 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.133+0000 7f6e39600700 1 -- 172.21.15.117:0/3220022247 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6e2c018b40 con 0x7f6e3c1054a0 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.135+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f6e20038810 msgr2=0x7f6e2003acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.135+0000 7f6e42c00700 1 --2- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f6e20038810 0x7f6e2003acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006fd0 tx=0x7f6e24006e40 comp rx=0 tx=0).stop 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.135+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 msgr2=0x7f6e3c19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.135+0000 7f6e42c00700 1 --2- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c19a3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0117b0 tx=0x7f6e2c011890 comp rx=0 tx=0).stop 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.136+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 shutdown_connections 2024-04-20T10:56:57.818 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.136+0000 7f6e42c00700 1 --2- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f6e20038810 0x7f6e2003acd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.136+0000 7f6e42c00700 1 --2- 172.21.15.117:0/3220022247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e3c1054a0 0x7f6e3c19a3a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.136+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 >> 172.21.15.117:0/3220022247 conn(0x7f6e3c100a60 msgr2=0x7f6e3c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.136+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 shutdown_connections 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-04-20T10:56:57.136+0000 7f6e42c00700 1 -- 172.21.15.117:0/3220022247 wait complete. 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout:Or, if you are only running a single cluster on this host: 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:57.819 INFO:teuthology.orchestra.run.smithi117.stdout: ceph telemetry on 2024-04-20T10:56:57.820 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:57.820 INFO:teuthology.orchestra.run.smithi117.stdout:For more information see: 2024-04-20T10:56:57.820 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:57.820 INFO:teuthology.orchestra.run.smithi117.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-20T10:56:57.820 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:56:57.820 INFO:teuthology.orchestra.run.smithi117.stdout:Bootstrap complete. 2024-04-20T10:56:57.912 INFO:tasks.cephadm:Fetching config... 2024-04-20T10:56:57.912 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:56:57.912 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-20T10:56:57.961 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-20T10:56:57.961 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:56:57.961 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-20T10:56:58.043 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-20T10:56:58.043 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:56:58.043 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/keyring of=/dev/stdout 2024-04-20T10:56:58.164 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-20T10:56:58.164 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:56:58.164 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-20T10:56:58.214 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-20T10:56:58.214 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZXXVoJa7i0GBvr/U20Z5yWrdwn+CwX/ZfmuRZEyICBWjTtKmAno9b80qBfXp83ugGsGM1LS3sx6k7Dp61Au5Ohz1eT0cj9ZN1Vm10OA3GVSVOE7an4MZBVWNvcmiCfMuCcyc/EzN6Q4+VUK/jFM6lRKHfWbSQSDDNQpsyHIG3PTdfwk3u3XQgb03vh7/HrktnN2OBEiCqvE0/L9n99iGcnqlv1oTaQzKOcntJxv97b3g3BGlIiw/VqY0yLfpTe8J2wl5DK5I8WD62sm87FhupsT9qtKMxaUd857lW8E1FV1w3Ut1QmGhaZydJtfrL8f4DctC798EtJ5cMAWSTZCRniokgB+PWsSieszmholt9otVsxeUYJqP8q4Bbp4TvYhsHWInqiJrBI3opbeFmGYZTnYpK58vDBrFqDz199l+7vqL5nPri7M7CJkgBHGCbl5EzvrQ4aRiaqxQHr4GsoU7KJBKgDQ8xqhL6QeWYioEsIHk6eJKw1qbxFjfJDUDMYL0= ceph-59ecfd22-ff04-11ee-bc92-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T10:56:58.310 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZXXVoJa7i0GBvr/U20Z5yWrdwn+CwX/ZfmuRZEyICBWjTtKmAno9b80qBfXp83ugGsGM1LS3sx6k7Dp61Au5Ohz1eT0cj9ZN1Vm10OA3GVSVOE7an4MZBVWNvcmiCfMuCcyc/EzN6Q4+VUK/jFM6lRKHfWbSQSDDNQpsyHIG3PTdfwk3u3XQgb03vh7/HrktnN2OBEiCqvE0/L9n99iGcnqlv1oTaQzKOcntJxv97b3g3BGlIiw/VqY0yLfpTe8J2wl5DK5I8WD62sm87FhupsT9qtKMxaUd857lW8E1FV1w3Ut1QmGhaZydJtfrL8f4DctC798EtJ5cMAWSTZCRniokgB+PWsSieszmholt9otVsxeUYJqP8q4Bbp4TvYhsHWInqiJrBI3opbeFmGYZTnYpK58vDBrFqDz199l+7vqL5nPri7M7CJkgBHGCbl5EzvrQ4aRiaqxQHr4GsoU7KJBKgDQ8xqhL6QeWYioEsIHk6eJKw1qbxFjfJDUDMYL0= ceph-59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:56:58.343 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:56:58 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3220022247' entity='client.admin' 2024-04-20T10:56:58.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZXXVoJa7i0GBvr/U20Z5yWrdwn+CwX/ZfmuRZEyICBWjTtKmAno9b80qBfXp83ugGsGM1LS3sx6k7Dp61Au5Ohz1eT0cj9ZN1Vm10OA3GVSVOE7an4MZBVWNvcmiCfMuCcyc/EzN6Q4+VUK/jFM6lRKHfWbSQSDDNQpsyHIG3PTdfwk3u3XQgb03vh7/HrktnN2OBEiCqvE0/L9n99iGcnqlv1oTaQzKOcntJxv97b3g3BGlIiw/VqY0yLfpTe8J2wl5DK5I8WD62sm87FhupsT9qtKMxaUd857lW8E1FV1w3Ut1QmGhaZydJtfrL8f4DctC798EtJ5cMAWSTZCRniokgB+PWsSieszmholt9otVsxeUYJqP8q4Bbp4TvYhsHWInqiJrBI3opbeFmGYZTnYpK58vDBrFqDz199l+7vqL5nPri7M7CJkgBHGCbl5EzvrQ4aRiaqxQHr4GsoU7KJBKgDQ8xqhL6QeWYioEsIHk6eJKw1qbxFjfJDUDMYL0= ceph-59ecfd22-ff04-11ee-bc92-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T10:56:58.473 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZXXVoJa7i0GBvr/U20Z5yWrdwn+CwX/ZfmuRZEyICBWjTtKmAno9b80qBfXp83ugGsGM1LS3sx6k7Dp61Au5Ohz1eT0cj9ZN1Vm10OA3GVSVOE7an4MZBVWNvcmiCfMuCcyc/EzN6Q4+VUK/jFM6lRKHfWbSQSDDNQpsyHIG3PTdfwk3u3XQgb03vh7/HrktnN2OBEiCqvE0/L9n99iGcnqlv1oTaQzKOcntJxv97b3g3BGlIiw/VqY0yLfpTe8J2wl5DK5I8WD62sm87FhupsT9qtKMxaUd857lW8E1FV1w3Ut1QmGhaZydJtfrL8f4DctC798EtJ5cMAWSTZCRniokgB+PWsSieszmholt9otVsxeUYJqP8q4Bbp4TvYhsHWInqiJrBI3opbeFmGYZTnYpK58vDBrFqDz199l+7vqL5nPri7M7CJkgBHGCbl5EzvrQ4aRiaqxQHr4GsoU7KJBKgDQ8xqhL6QeWYioEsIHk6eJKw1qbxFjfJDUDMYL0= ceph-59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:56:58.527 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-20T10:56:58.898 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:57:01.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.134+0000 7f935b200700 1 -- 172.21.15.117:0/1970645631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 msgr2=0x7f9354107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:01.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.134+0000 7f935b200700 1 --2- 172.21.15.117:0/1970645631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 0x7f9354107510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9344009910 tx=0x7f9344009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:01.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.135+0000 7f935b200700 1 -- 172.21.15.117:0/1970645631 shutdown_connections 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.135+0000 7f935b200700 1 --2- 172.21.15.117:0/1970645631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 0x7f9354107510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.135+0000 7f935b200700 1 -- 172.21.15.117:0/1970645631 >> 172.21.15.117:0/1970645631 conn(0x7f9354075fd0 msgr2=0x7f93540783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.135+0000 7f935b200700 1 -- 172.21.15.117:0/1970645631 shutdown_connections 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.135+0000 7f935b200700 1 -- 172.21.15.117:0/1970645631 wait complete. 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.136+0000 7f935b200700 1 Processor -- start 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.136+0000 7f935b200700 1 -- start start 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.137+0000 7f935b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 0x7f935419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.137+0000 7f935b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935419c7f0 con 0x7f93541070f0 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.137+0000 7f9353e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 0x7f935419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.137+0000 7f9353e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 0x7f935419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35082/0 (socket says 172.21.15.117:35082) 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.137+0000 7f9353e00700 1 -- 172.21.15.117:0/1454219230 learned_addr learned my addr 172.21.15.117:0/1454219230 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:57:01.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.137+0000 7f9353e00700 1 -- 172.21.15.117:0/1454219230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440095f0 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.137+0000 7f9353e00700 1 --2- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 0x7f935419c2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9344006010 tx=0x7f93440050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.138+0000 7f9350c00700 1 -- 172.21.15.117:0/1454219230 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f934401d070 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.138+0000 7f9350c00700 1 -- 172.21.15.117:0/1454219230 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f934400be20 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.138+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935419c9f0 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.138+0000 7f9350c00700 1 -- 172.21.15.117:0/1454219230 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f934400f460 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.138+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935419ce90 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.139+0000 7f9350c00700 1 -- 172.21.15.117:0/1454219230 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f934400b960 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.139+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9354195c30 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.139+0000 7f9350c00700 1 --2- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f93380387d0 0x7f933803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.140+0000 7f9350c00700 1 -- 172.21.15.117:0/1454219230 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f934404cb70 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.140+0000 7f9353400700 1 --2- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f93380387d0 0x7f933803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.140+0000 7f9353400700 1 --2- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f93380387d0 0x7f933803ac90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9340006fd0 tx=0x7f9340006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.145+0000 7f9350c00700 1 -- 172.21.15.117:0/1454219230 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9344005c00 con 0x7f93541070f0 2024-04-20T10:57:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.374+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f9354062510 con 0x7f93541070f0 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.383+0000 7f9350c00700 1 -- 172.21.15.117:0/1454219230 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9344027070 con 0x7f93541070f0 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.386+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f93380387d0 msgr2=0x7f933803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.386+0000 7f935b200700 1 --2- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f93380387d0 0x7f933803ac90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9340006fd0 tx=0x7f9340006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.386+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 msgr2=0x7f935419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.386+0000 7f935b200700 1 --2- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 0x7f935419c2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9344006010 tx=0x7f93440050b0 comp rx=0 tx=0).stop 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.386+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 shutdown_connections 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.386+0000 7f935b200700 1 --2- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f93380387d0 0x7f933803ac90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.386+0000 7f935b200700 1 --2- 172.21.15.117:0/1454219230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93541070f0 0x7f935419c2b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.386+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 >> 172.21.15.117:0/1454219230 conn(0x7f9354075fd0 msgr2=0x7f9354076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.387+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 shutdown_connections 2024-04-20T10:57:01.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:01.387+0000 7f935b200700 1 -- 172.21.15.117:0/1454219230 wait complete. 2024-04-20T10:57:02.366 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-20T10:57:02.366 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: Deploying daemon ceph-exporter.smithi117 on smithi117 2024-04-20T10:57:02.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:02 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1454219230' entity='client.admin' 2024-04-20T10:57:02.752 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:57:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.890+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2963716112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 msgr2=0x7f4bc0107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.890+0000 7f4bc7a00700 1 --2- 172.21.15.117:0/2963716112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 0x7f4bc0107510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009910 tx=0x7f4bb4009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.890+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2963716112 shutdown_connections 2024-04-20T10:57:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.890+0000 7f4bc7a00700 1 --2- 172.21.15.117:0/2963716112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 0x7f4bc0107510 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.890+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2963716112 >> 172.21.15.117:0/2963716112 conn(0x7f4bc0075f60 msgr2=0x7f4bc0078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.890+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2963716112 shutdown_connections 2024-04-20T10:57:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.890+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2963716112 wait complete. 2024-04-20T10:57:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.891+0000 7f4bc7a00700 1 Processor -- start 2024-04-20T10:57:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.891+0000 7f4bc7a00700 1 -- start start 2024-04-20T10:57:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.892+0000 7f4bc7a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 0x7f4bc019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.892+0000 7f4bc7a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc019c8a0 con 0x7f4bc01070f0 2024-04-20T10:57:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.892+0000 7f4bc5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 0x7f4bc019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.892+0000 7f4bc5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 0x7f4bc019c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58100/0 (socket says 172.21.15.117:58100) 2024-04-20T10:57:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.892+0000 7f4bc5000700 1 -- 172.21.15.117:0/2509848434 learned_addr learned my addr 172.21.15.117:0/2509848434 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:57:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.892+0000 7f4bc5000700 1 -- 172.21.15.117:0/2509848434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb40095f0 con 0x7f4bc01070f0 2024-04-20T10:57:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.893+0000 7f4bc5000700 1 --2- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 0x7f4bc019c360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4006010 tx=0x7f4bb40050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.893+0000 7f4bbd600700 1 -- 172.21.15.117:0/2509848434 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bb401d070 con 0x7f4bc01070f0 2024-04-20T10:57:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.893+0000 7f4bbd600700 1 -- 172.21.15.117:0/2509848434 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bb400be20 con 0x7f4bc01070f0 2024-04-20T10:57:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.893+0000 7f4bbd600700 1 -- 172.21.15.117:0/2509848434 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bb400f460 con 0x7f4bc01070f0 2024-04-20T10:57:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.893+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc019caa0 con 0x7f4bc01070f0 2024-04-20T10:57:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.893+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc019cf40 con 0x7f4bc01070f0 2024-04-20T10:57:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.894+0000 7f4bbd600700 1 -- 172.21.15.117:0/2509848434 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f4bb4022470 con 0x7f4bc01070f0 2024-04-20T10:57:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.894+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0195c20 con 0x7f4bc01070f0 2024-04-20T10:57:04.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.895+0000 7f4bbd600700 1 --2- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f4ba8038820 0x7f4ba803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:04.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.895+0000 7f4bbd600700 1 -- 172.21.15.117:0/2509848434 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4bb404cdb0 con 0x7f4bc01070f0 2024-04-20T10:57:04.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.895+0000 7f4bbfe00700 1 --2- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f4ba8038820 0x7f4ba803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:04.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.896+0000 7f4bbfe00700 1 --2- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f4ba8038820 0x7f4ba803ace0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4bac006fd0 tx=0x7f4bac006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:04.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:04.901+0000 7f4bbd600700 1 -- 172.21.15.117:0/2509848434 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4bb4005c00 con 0x7f4bc01070f0 2024-04-20T10:57:05.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.125+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 --> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bc0061320 con 0x7f4ba8038820 2024-04-20T10:57:05.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.133+0000 7f4bbd600700 1 -- 172.21.15.117:0/2509848434 <== mgr.14164 v2:172.21.15.117:6800/1996106837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4bc0061320 con 0x7f4ba8038820 2024-04-20T10:57:05.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.135+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f4ba8038820 msgr2=0x7f4ba803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:05.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.135+0000 7f4bc7a00700 1 --2- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f4ba8038820 0x7f4ba803ace0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4bac006fd0 tx=0x7f4bac006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.135+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 msgr2=0x7f4bc019c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.135+0000 7f4bc7a00700 1 --2- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 0x7f4bc019c360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4006010 tx=0x7f4bb40050b0 comp rx=0 tx=0).stop 2024-04-20T10:57:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.135+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 shutdown_connections 2024-04-20T10:57:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.135+0000 7f4bc7a00700 1 --2- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f4ba8038820 0x7f4ba803ace0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.135+0000 7f4bc7a00700 1 --2- 172.21.15.117:0/2509848434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc01070f0 0x7f4bc019c360 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.135+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 >> 172.21.15.117:0/2509848434 conn(0x7f4bc0075f60 msgr2=0x7f4bc00769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.136+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 shutdown_connections 2024-04-20T10:57:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:05.136+0000 7f4bc7a00700 1 -- 172.21.15.117:0/2509848434 wait complete. 2024-04-20T10:57:05.864 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2024-04-20T10:57:05.865 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:57:05.865 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2024-04-20T10:57:05.926 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:57:05.926 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:57:06.021 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2024-04-20T10:57:06.021 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch host add smithi173 2024-04-20T10:57:06.374 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:06 smithi117 ceph-mon[15659]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:57:06.374 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:06 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:07.147 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:57:08.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:08 smithi117 ceph-mon[15659]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:09.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.659+0000 7fe1c1c00700 1 -- 172.21.15.117:0/2268226490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 msgr2=0x7fe1bc073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:09.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.659+0000 7fe1c1c00700 1 --2- 172.21.15.117:0/2268226490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 0x7fe1bc073370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009910 tx=0x7fe1ac009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:09.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.660+0000 7fe1c1c00700 1 -- 172.21.15.117:0/2268226490 shutdown_connections 2024-04-20T10:57:09.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.660+0000 7fe1c1c00700 1 --2- 172.21.15.117:0/2268226490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 0x7fe1bc073370 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:09.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.660+0000 7fe1c1c00700 1 -- 172.21.15.117:0/2268226490 >> 172.21.15.117:0/2268226490 conn(0x7fe1bc0fbe30 msgr2=0x7fe1bc0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:09.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.660+0000 7fe1c1c00700 1 -- 172.21.15.117:0/2268226490 shutdown_connections 2024-04-20T10:57:09.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.660+0000 7fe1c1c00700 1 -- 172.21.15.117:0/2268226490 wait complete. 2024-04-20T10:57:09.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.661+0000 7fe1c1c00700 1 Processor -- start 2024-04-20T10:57:09.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.661+0000 7fe1c1c00700 1 -- start start 2024-04-20T10:57:09.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.661+0000 7fe1c1c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 0x7fe1bc19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:09.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.661+0000 7fe1c1c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc19ca00 con 0x7fe1bc074f10 2024-04-20T10:57:09.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.662+0000 7fe1baa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 0x7fe1bc19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:09.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.662+0000 7fe1baa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 0x7fe1bc19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58116/0 (socket says 172.21.15.117:58116) 2024-04-20T10:57:09.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.662+0000 7fe1baa00700 1 -- 172.21.15.117:0/3121752921 learned_addr learned my addr 172.21.15.117:0/3121752921 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:57:09.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.662+0000 7fe1baa00700 1 -- 172.21.15.117:0/3121752921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac0095f0 con 0x7fe1bc074f10 2024-04-20T10:57:09.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.663+0000 7fe1baa00700 1 --2- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 0x7fe1bc19c4c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac006010 tx=0x7fe1ac0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:09.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.663+0000 7fe1b3400700 1 -- 172.21.15.117:0/3121752921 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe1ac01d070 con 0x7fe1bc074f10 2024-04-20T10:57:09.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.663+0000 7fe1b3400700 1 -- 172.21.15.117:0/3121752921 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1ac00be20 con 0x7fe1bc074f10 2024-04-20T10:57:09.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.663+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc104490 con 0x7fe1bc074f10 2024-04-20T10:57:09.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.663+0000 7fe1b3400700 1 -- 172.21.15.117:0/3121752921 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe1ac022de0 con 0x7fe1bc074f10 2024-04-20T10:57:09.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.663+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc19cf10 con 0x7fe1bc074f10 2024-04-20T10:57:09.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.664+0000 7fe1b3400700 1 -- 172.21.15.117:0/3121752921 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7fe1ac00f460 con 0x7fe1bc074f10 2024-04-20T10:57:09.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.665+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc195e20 con 0x7fe1bc074f10 2024-04-20T10:57:09.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.665+0000 7fe1b3400700 1 --2- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe1a0038820 0x7fe1a003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:09.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.665+0000 7fe1b3400700 1 -- 172.21.15.117:0/3121752921 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe1ac04ca30 con 0x7fe1bc074f10 2024-04-20T10:57:09.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.665+0000 7fe1ba000700 1 --2- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe1a0038820 0x7fe1a003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:09.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.666+0000 7fe1ba000700 1 --2- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe1a0038820 0x7fe1a003ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8006fd0 tx=0x7fe1a8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:09.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.675+0000 7fe1b3400700 1 -- 172.21.15.117:0/3121752921 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe1ac017440 con 0x7fe1bc074f10 2024-04-20T10:57:09.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:09.913+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 --> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1bc061320 con 0x7fe1a0038820 2024-04-20T10:57:10.355 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:10 smithi117 ceph-mon[15659]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:10.355 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:10 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:10.355 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:10 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:10.355 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:10 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:10.356 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:10 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:10.356 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:10 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T10:57:10.356 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:10 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-20T10:57:10.356 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:10 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:57:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:11 smithi117 ceph-mon[15659]: Deploying daemon crash.smithi117 on smithi117 2024-04-20T10:57:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:11 smithi117 ceph-mon[15659]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:57:12.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:12 smithi117 ceph-mon[15659]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:12.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:12 smithi117 ceph-mon[15659]: Deploying cephadm binary to smithi173 2024-04-20T10:57:12.301 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:12 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:14.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.105+0000 7fe1b3400700 1 -- 172.21.15.117:0/3121752921 <== mgr.14164 v2:172.21.15.117:6800/1996106837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe1bc061320 con 0x7fe1a0038820 2024-04-20T10:57:14.108 INFO:teuthology.orchestra.run.smithi117.stdout:Added host 'smithi173' with addr '172.21.15.173' 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.108+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe1a0038820 msgr2=0x7fe1a003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.108+0000 7fe1c1c00700 1 --2- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe1a0038820 0x7fe1a003ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8006fd0 tx=0x7fe1a8006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.109+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 msgr2=0x7fe1bc19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.109+0000 7fe1c1c00700 1 --2- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 0x7fe1bc19c4c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac006010 tx=0x7fe1ac0050b0 comp rx=0 tx=0).stop 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.109+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 shutdown_connections 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.109+0000 7fe1c1c00700 1 --2- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe1a0038820 0x7fe1a003ace0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.109+0000 7fe1c1c00700 1 --2- 172.21.15.117:0/3121752921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1bc074f10 0x7fe1bc19c4c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.109+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 >> 172.21.15.117:0/3121752921 conn(0x7fe1bc0fbe30 msgr2=0x7fe1bc0fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:14.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.109+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 shutdown_connections 2024-04-20T10:57:14.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:14.109+0000 7fe1c1c00700 1 -- 172.21.15.117:0/3121752921 wait complete. 2024-04-20T10:57:14.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:14 smithi117 ceph-mon[15659]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:14.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:14 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:14.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:14 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:14.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:14 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:14.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:14 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:14.221 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:14 smithi117 ceph-mon[15659]: Deploying daemon node-exporter.smithi117 on smithi117 2024-04-20T10:57:14.221 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:14 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:14.957 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T10:57:15.210 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:15 smithi117 ceph-mon[15659]: Added host smithi173 2024-04-20T10:57:15.329 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:57:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:16 smithi117 ceph-mon[15659]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:18.205 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:17 smithi117 ceph-mon[15659]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:18.205 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:17 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:18.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.519+0000 7f1c86400700 1 -- 172.21.15.117:0/3788920972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 msgr2=0x7f1c80102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.519+0000 7f1c86400700 1 --2- 172.21.15.117:0/3788920972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 0x7f1c80102e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009910 tx=0x7f1c70009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:18.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.519+0000 7f1c86400700 1 -- 172.21.15.117:0/3788920972 shutdown_connections 2024-04-20T10:57:18.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.519+0000 7f1c86400700 1 --2- 172.21.15.117:0/3788920972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 0x7f1c80102e90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.519+0000 7f1c86400700 1 -- 172.21.15.117:0/3788920972 >> 172.21.15.117:0/3788920972 conn(0x7f1c800fe010 msgr2=0x7f1c80100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.519+0000 7f1c86400700 1 -- 172.21.15.117:0/3788920972 shutdown_connections 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.519+0000 7f1c86400700 1 -- 172.21.15.117:0/3788920972 wait complete. 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.521+0000 7f1c86400700 1 Processor -- start 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.521+0000 7f1c86400700 1 -- start start 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.521+0000 7f1c86400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 0x7f1c80197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.521+0000 7f1c86400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c80198400 con 0x7f1c80102a70 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.522+0000 7f1c7f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 0x7f1c80197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.522+0000 7f1c7f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 0x7f1c80197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56378/0 (socket says 172.21.15.117:56378) 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.522+0000 7f1c7f400700 1 -- 172.21.15.117:0/3714917617 learned_addr learned my addr 172.21.15.117:0/3714917617 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.522+0000 7f1c7f400700 1 -- 172.21.15.117:0/3714917617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c700095f0 con 0x7f1c80102a70 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.522+0000 7f1c7f400700 1 --2- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 0x7f1c80197ec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1c70006010 tx=0x7f1c700050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.523+0000 7f1c77e00700 1 -- 172.21.15.117:0/3714917617 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c7001d070 con 0x7f1c80102a70 2024-04-20T10:57:18.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.523+0000 7f1c77e00700 1 -- 172.21.15.117:0/3714917617 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c7000be20 con 0x7f1c80102a70 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.523+0000 7f1c77e00700 1 -- 172.21.15.117:0/3714917617 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c70022de0 con 0x7f1c80102a70 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.523+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c80198600 con 0x7f1c80102a70 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.523+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c80198aa0 con 0x7f1c80102a70 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.524+0000 7f1c77e00700 1 -- 172.21.15.117:0/3714917617 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f1c700226c0 con 0x7f1c80102a70 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.524+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c801918a0 con 0x7f1c80102a70 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.524+0000 7f1c77e00700 1 --2- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1c64038820 0x7f1c6403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.524+0000 7f1c77e00700 1 -- 172.21.15.117:0/3714917617 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1c7004c8f0 con 0x7f1c80102a70 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.525+0000 7f1c7ea00700 1 --2- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1c64038820 0x7f1c6403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.525+0000 7f1c7ea00700 1 --2- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1c64038820 0x7f1c6403ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c006fd0 tx=0x7f1c6c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:18.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.530+0000 7f1c77e00700 1 -- 172.21.15.117:0/3714917617 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1c7000b960 con 0x7f1c80102a70 2024-04-20T10:57:18.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.762+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 --> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c8002cdb0 con 0x7f1c64038820 2024-04-20T10:57:18.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.765+0000 7f1c77e00700 1 -- 172.21.15.117:0/3714917617 <== mgr.14164 v2:172.21.15.117:6800/1996106837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f1c8002cdb0 con 0x7f1c64038820 2024-04-20T10:57:18.765 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T10:57:18.765 INFO:teuthology.orchestra.run.smithi117.stdout:[{"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.767+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1c64038820 msgr2=0x7f1c6403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 --2- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1c64038820 0x7f1c6403ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c006fd0 tx=0x7f1c6c006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 msgr2=0x7f1c80197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 --2- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 0x7f1c80197ec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1c70006010 tx=0x7f1c700050b0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 shutdown_connections 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 --2- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1c64038820 0x7f1c6403ace0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 --2- 172.21.15.117:0/3714917617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80102a70 0x7f1c80197ec0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 >> 172.21.15.117:0/3714917617 conn(0x7f1c800fe010 msgr2=0x7f1c800fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 shutdown_connections 2024-04-20T10:57:18.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:18.768+0000 7f1c86400700 1 -- 172.21.15.117:0/3714917617 wait complete. 2024-04-20T10:57:19.753 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-20T10:57:19.753 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd crush tunables default 2024-04-20T10:57:20.133 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:57:20.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:19 smithi117 ceph-mon[15659]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:57:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:19 smithi117 ceph-mon[15659]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:19 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:19 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:19 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:19 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:20 smithi117 ceph-mon[15659]: Deploying daemon alertmanager.smithi117 on smithi117 2024-04-20T10:57:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:21 smithi117 ceph-mon[15659]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:21 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:22.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.939+0000 7f8cfa200700 1 -- 172.21.15.117:0/1898992302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 msgr2=0x7f8cf4107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:22.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.939+0000 7f8cfa200700 1 --2- 172.21.15.117:0/1898992302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 0x7f8cf4107510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009910 tx=0x7f8ce8009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:22.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.940+0000 7f8cfa200700 1 -- 172.21.15.117:0/1898992302 shutdown_connections 2024-04-20T10:57:22.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.940+0000 7f8cfa200700 1 --2- 172.21.15.117:0/1898992302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 0x7f8cf4107510 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:22.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.940+0000 7f8cfa200700 1 -- 172.21.15.117:0/1898992302 >> 172.21.15.117:0/1898992302 conn(0x7f8cf4075ce0 msgr2=0x7f8cf4078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:22.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.940+0000 7f8cfa200700 1 -- 172.21.15.117:0/1898992302 shutdown_connections 2024-04-20T10:57:22.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.940+0000 7f8cfa200700 1 -- 172.21.15.117:0/1898992302 wait complete. 2024-04-20T10:57:22.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.941+0000 7f8cfa200700 1 Processor -- start 2024-04-20T10:57:22.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.941+0000 7f8cfa200700 1 -- start start 2024-04-20T10:57:22.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.941+0000 7f8cfa200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 0x7f8cf419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:22.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.941+0000 7f8cfa200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf419c670 con 0x7f8cf41070f0 2024-04-20T10:57:22.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.942+0000 7f8cf8e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 0x7f8cf419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:22.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.942+0000 7f8cf8e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 0x7f8cf419c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56402/0 (socket says 172.21.15.117:56402) 2024-04-20T10:57:22.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.942+0000 7f8cf8e00700 1 -- 172.21.15.117:0/3623636418 learned_addr learned my addr 172.21.15.117:0/3623636418 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:57:22.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.942+0000 7f8cf8e00700 1 -- 172.21.15.117:0/3623636418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80095f0 con 0x7f8cf41070f0 2024-04-20T10:57:22.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.942+0000 7f8cf8e00700 1 --2- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 0x7f8cf419c130 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8006010 tx=0x7f8ce8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:22.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.942+0000 7f8cf1600700 1 -- 172.21.15.117:0/3623636418 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce801d070 con 0x7f8cf41070f0 2024-04-20T10:57:22.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.943+0000 7f8cf1600700 1 -- 172.21.15.117:0/3623636418 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ce8005dc0 con 0x7f8cf41070f0 2024-04-20T10:57:22.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.943+0000 7f8cf1600700 1 -- 172.21.15.117:0/3623636418 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce8022e90 con 0x7f8cf41070f0 2024-04-20T10:57:22.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.943+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf419c870 con 0x7f8cf41070f0 2024-04-20T10:57:22.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.943+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf419cd10 con 0x7f8cf41070f0 2024-04-20T10:57:22.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.944+0000 7f8cf1600700 1 -- 172.21.15.117:0/3623636418 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f8ce8022470 con 0x7f8cf41070f0 2024-04-20T10:57:22.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.944+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf4195ae0 con 0x7f8cf41070f0 2024-04-20T10:57:22.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.944+0000 7f8cf1600700 1 --2- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f8cdc038760 0x7f8cdc03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:22.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.944+0000 7f8cf1600700 1 -- 172.21.15.117:0/3623636418 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ce804c8b0 con 0x7f8cf41070f0 2024-04-20T10:57:22.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.945+0000 7f8cf3e00700 1 --2- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f8cdc038760 0x7f8cdc03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:22.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.945+0000 7f8cf3e00700 1 --2- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f8cdc038760 0x7f8cdc03ac20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4006fd0 tx=0x7f8ce4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:22.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:22.950+0000 7f8cf1600700 1 -- 172.21.15.117:0/3623636418 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8ce800f4f0 con 0x7f8cf41070f0 2024-04-20T10:57:23.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.176+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8cf4062510 con 0x7f8cf41070f0 2024-04-20T10:57:23.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.985+0000 7f8cf1600700 1 -- 172.21.15.117:0/3623636418 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8ce8027020 con 0x7f8cf41070f0 2024-04-20T10:57:23.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.988+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f8cdc038760 msgr2=0x7f8cdc03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:23.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.988+0000 7f8cfa200700 1 --2- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f8cdc038760 0x7f8cdc03ac20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4006fd0 tx=0x7f8ce4006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:23.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.988+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 msgr2=0x7f8cf419c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:23.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.988+0000 7f8cfa200700 1 --2- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 0x7f8cf419c130 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8006010 tx=0x7f8ce8009f80 comp rx=0 tx=0).stop 2024-04-20T10:57:23.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.988+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 shutdown_connections 2024-04-20T10:57:23.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.988+0000 7f8cfa200700 1 --2- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f8cdc038760 0x7f8cdc03ac20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.988+0000 7f8cfa200700 1 --2- 172.21.15.117:0/3623636418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf41070f0 0x7f8cf419c130 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.988+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 >> 172.21.15.117:0/3623636418 conn(0x7f8cf4075ce0 msgr2=0x7f8cf4076760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:23.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.989+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 shutdown_connections 2024-04-20T10:57:23.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:57:23.989+0000 7f8cfa200700 1 -- 172.21.15.117:0/3623636418 wait complete. 2024-04-20T10:57:23.990 INFO:teuthology.orchestra.run.smithi117.stderr:adjusted tunables profile to default 2024-04-20T10:57:24.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:23 smithi117 ceph-mon[15659]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:24.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:23 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3623636418' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-20T10:57:25.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:24 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3623636418' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-20T10:57:25.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:24 smithi117 ceph-mon[15659]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T10:57:25.347 INFO:tasks.cephadm:Adding mon.smithi117 on smithi117 2024-04-20T10:57:25.347 INFO:tasks.cephadm:Adding mon.smithi173 on smithi173 2024-04-20T10:57:25.347 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch apply mon '2;smithi117:172.21.15.117=smithi117;smithi173:172.21.15.173=smithi173' 2024-04-20T10:57:25.756 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:25.857 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:26.123 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:25 smithi117 ceph-mon[15659]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:27.953 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.952+0000 7fe57a600700 1 -- 172.21.15.173:0/2515139968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 msgr2=0x7fe574100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:27.954 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.952+0000 7fe57a600700 1 --2- 172.21.15.173:0/2515139968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 0x7fe574100ad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe568009910 tx=0x7fe568009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:27.954 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.952+0000 7fe57a600700 1 -- 172.21.15.173:0/2515139968 shutdown_connections 2024-04-20T10:57:27.954 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.952+0000 7fe57a600700 1 --2- 172.21.15.173:0/2515139968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 0x7fe574100ad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:27.954 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.952+0000 7fe57a600700 1 -- 172.21.15.173:0/2515139968 >> 172.21.15.173:0/2515139968 conn(0x7fe5740fbc50 msgr2=0x7fe5740fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:27.954 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.952+0000 7fe57a600700 1 -- 172.21.15.173:0/2515139968 shutdown_connections 2024-04-20T10:57:27.954 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.952+0000 7fe57a600700 1 -- 172.21.15.173:0/2515139968 wait complete. 2024-04-20T10:57:27.955 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.953+0000 7fe57a600700 1 Processor -- start 2024-04-20T10:57:27.955 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.954+0000 7fe57a600700 1 -- start start 2024-04-20T10:57:27.955 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.954+0000 7fe57a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 0x7fe574197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:27.955 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.954+0000 7fe57a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5741982b0 con 0x7fe5741006b0 2024-04-20T10:57:27.955 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.955+0000 7fe579200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 0x7fe574197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:27.955 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.955+0000 7fe579200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 0x7fe574197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:55992/0 (socket says 172.21.15.173:55992) 2024-04-20T10:57:27.955 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.955+0000 7fe579200700 1 -- 172.21.15.173:0/2856566302 learned_addr learned my addr 172.21.15.173:0/2856566302 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:57:27.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.955+0000 7fe579200700 1 -- 172.21.15.173:0/2856566302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680095f0 con 0x7fe5741006b0 2024-04-20T10:57:27.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.956+0000 7fe579200700 1 --2- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 0x7fe574197d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe568006010 tx=0x7fe568009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:27.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.956+0000 7fe571600700 1 -- 172.21.15.173:0/2856566302 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe56801d070 con 0x7fe5741006b0 2024-04-20T10:57:27.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.956+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5741984b0 con 0x7fe5741006b0 2024-04-20T10:57:27.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.956+0000 7fe571600700 1 -- 172.21.15.173:0/2856566302 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe568005dc0 con 0x7fe5741006b0 2024-04-20T10:57:27.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.956+0000 7fe571600700 1 -- 172.21.15.173:0/2856566302 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe568022e90 con 0x7fe5741006b0 2024-04-20T10:57:27.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.957+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe574198950 con 0x7fe5741006b0 2024-04-20T10:57:27.958 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.958+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5741918e0 con 0x7fe5741006b0 2024-04-20T10:57:27.959 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.958+0000 7fe571600700 1 -- 172.21.15.173:0/2856566302 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7fe568022470 con 0x7fe5741006b0 2024-04-20T10:57:27.959 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.959+0000 7fe571600700 1 --2- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe55c038760 0x7fe55c03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:27.959 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.959+0000 7fe571600700 1 -- 172.21.15.173:0/2856566302 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe56804c9e0 con 0x7fe5741006b0 2024-04-20T10:57:27.960 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.959+0000 7fe573e00700 1 --2- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe55c038760 0x7fe55c03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:27.961 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.960+0000 7fe573e00700 1 --2- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe55c038760 0x7fe55c03ac20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe560006fd0 tx=0x7fe560006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:27.975 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:27.974+0000 7fe571600700 1 -- 172.21.15.173:0/2856566302 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe56800b960 con 0x7fe5741006b0 2024-04-20T10:57:28.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:27 smithi117 ceph-mon[15659]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:28.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.207+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 --> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi117:172.21.15.117=smithi117;smithi173:172.21.15.173=smithi173", "target": ["mon-mgr", ""]}) v1 -- 0x7fe57402cdb0 con 0x7fe55c038760 2024-04-20T10:57:28.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.223+0000 7fe571600700 1 -- 172.21.15.173:0/2856566302 <== mgr.14164 v2:172.21.15.117:6800/1996106837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe57402cdb0 con 0x7fe55c038760 2024-04-20T10:57:28.224 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mon update... 2024-04-20T10:57:28.227 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.226+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe55c038760 msgr2=0x7fe55c03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.226+0000 7fe57a600700 1 --2- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe55c038760 0x7fe55c03ac20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe560006fd0 tx=0x7fe560006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.226+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 msgr2=0x7fe574197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.226+0000 7fe57a600700 1 --2- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 0x7fe574197d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe568006010 tx=0x7fe568009f80 comp rx=0 tx=0).stop 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.227+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 shutdown_connections 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.227+0000 7fe57a600700 1 --2- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe55c038760 0x7fe55c03ac20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.227+0000 7fe57a600700 1 --2- 172.21.15.173:0/2856566302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5741006b0 0x7fe574197d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.227+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 >> 172.21.15.173:0/2856566302 conn(0x7fe5740fbc50 msgr2=0x7fe5740fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.227+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 shutdown_connections 2024-04-20T10:57:28.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:28.227+0000 7fe57a600700 1 -- 172.21.15.173:0/2856566302 wait complete. 2024-04-20T10:57:28.946 DEBUG:teuthology.orchestra.run.smithi173:mon.smithi173> sudo journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi173.service 2024-04-20T10:57:28.952 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:57:28.952 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:57:29.238 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:28 smithi117 ceph-mon[15659]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi117:172.21.15.117=smithi117;smithi173:172.21.15.173=smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:57:29.238 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:28 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:29.333 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:29.437 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:29 smithi117 ceph-mon[15659]: Saving service mon spec with placement smithi117:172.21.15.117=smithi117;smithi173:172.21.15.173=smithi173;count:2 2024-04-20T10:57:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:29 smithi117 ceph-mon[15659]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:31.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: Deploying daemon grafana.smithi117 on smithi117 2024-04-20T10:57:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:31 smithi117 ceph-mon[15659]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:32.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.075+0000 7fe560600700 1 -- 172.21.15.173:0/2244745233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 msgr2=0x7fe558100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:32.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.075+0000 7fe560600700 1 --2- 172.21.15.173:0/2244745233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 0x7fe558100c70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00b0a0 tx=0x7fe54c00b3b0 comp rx=0 tx=0).stop 2024-04-20T10:57:32.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.075+0000 7fe560600700 1 -- 172.21.15.173:0/2244745233 shutdown_connections 2024-04-20T10:57:32.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.075+0000 7fe560600700 1 --2- 172.21.15.173:0/2244745233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 0x7fe558100c70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:32.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.075+0000 7fe560600700 1 -- 172.21.15.173:0/2244745233 >> 172.21.15.173:0/2244745233 conn(0x7fe5580fbe30 msgr2=0x7fe5580fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:32.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.075+0000 7fe560600700 1 -- 172.21.15.173:0/2244745233 shutdown_connections 2024-04-20T10:57:32.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.075+0000 7fe560600700 1 -- 172.21.15.173:0/2244745233 wait complete. 2024-04-20T10:57:32.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.076+0000 7fe560600700 1 Processor -- start 2024-04-20T10:57:32.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.077+0000 7fe560600700 1 -- start start 2024-04-20T10:57:32.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.077+0000 7fe560600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 0x7fe558197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:32.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.077+0000 7fe560600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe558198460 con 0x7fe558100850 2024-04-20T10:57:32.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.077+0000 7fe55dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 0x7fe558197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:32.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.078+0000 7fe55dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 0x7fe558197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:37792/0 (socket says 172.21.15.173:37792) 2024-04-20T10:57:32.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.078+0000 7fe55dc00700 1 -- 172.21.15.173:0/1487454774 learned_addr learned my addr 172.21.15.173:0/1487454774 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:57:32.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.078+0000 7fe55dc00700 1 -- 172.21.15.173:0/1487454774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c009d00 con 0x7fe558100850 2024-04-20T10:57:32.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.079+0000 7fe55dc00700 1 --2- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 0x7fe558197f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00bc90 tx=0x7fe54c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:32.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.079+0000 7fe556000700 1 -- 172.21.15.173:0/1487454774 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe54c00e040 con 0x7fe558100850 2024-04-20T10:57:32.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.079+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe558198660 con 0x7fe558100850 2024-04-20T10:57:32.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.079+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe558198b00 con 0x7fe558100850 2024-04-20T10:57:32.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.079+0000 7fe556000700 1 -- 172.21.15.173:0/1487454774 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe54c003d50 con 0x7fe558100850 2024-04-20T10:57:32.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.080+0000 7fe556000700 1 -- 172.21.15.173:0/1487454774 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe54c026900 con 0x7fe558100850 2024-04-20T10:57:32.080 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.080+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe558191a80 con 0x7fe558100850 2024-04-20T10:57:32.081 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.081+0000 7fe556000700 1 -- 172.21.15.173:0/1487454774 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7fe54c012640 con 0x7fe558100850 2024-04-20T10:57:32.082 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.082+0000 7fe556000700 1 --2- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe5400388b0 0x7fe54003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:32.082 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.082+0000 7fe556000700 1 -- 172.21.15.173:0/1487454774 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe54c04c9d0 con 0x7fe558100850 2024-04-20T10:57:32.082 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.083+0000 7fe55d200700 1 --2- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe5400388b0 0x7fe54003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:32.083 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.084+0000 7fe55d200700 1 --2- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe5400388b0 0x7fe54003ad70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe544006fd0 tx=0x7fe544006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:32.097 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.097+0000 7fe556000700 1 -- 172.21.15.173:0/1487454774 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe54c026460 con 0x7fe558100850 2024-04-20T10:57:32.396 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.396+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe558062510 con 0x7fe558100850 2024-04-20T10:57:32.397 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.397+0000 7fe556000700 1 -- 172.21.15.173:0/1487454774 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe558062510 con 0x7fe558100850 2024-04-20T10:57:32.397 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:57:32.397 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:57:32.400 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.400+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe5400388b0 msgr2=0x7fe54003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:32.400 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.400+0000 7fe560600700 1 --2- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe5400388b0 0x7fe54003ad70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe544006fd0 tx=0x7fe544006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:32.400 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.400+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 msgr2=0x7fe558197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:32.400 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.400+0000 7fe560600700 1 --2- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 0x7fe558197f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00bc90 tx=0x7fe54c00bd70 comp rx=0 tx=0).stop 2024-04-20T10:57:32.400 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.400+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 shutdown_connections 2024-04-20T10:57:32.400 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.400+0000 7fe560600700 1 --2- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fe5400388b0 0x7fe54003ad70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:32.400 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.400+0000 7fe560600700 1 --2- 172.21.15.173:0/1487454774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe558100850 0x7fe558197f20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:32.400 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.400+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 >> 172.21.15.173:0/1487454774 conn(0x7fe5580fbe30 msgr2=0x7fe5580fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:32.401 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.401+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 shutdown_connections 2024-04-20T10:57:32.401 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:32.401+0000 7fe560600700 1 -- 172.21.15.173:0/1487454774 wait complete. 2024-04-20T10:57:32.402 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:57:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:32 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:32 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/1487454774' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:57:34.082 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:57:34.082 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:57:34.293 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:33 smithi117 ceph-mon[15659]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:34.452 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:34.551 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:35 smithi117 ceph-mon[15659]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:36.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.473+0000 7ff1c7000700 1 -- 172.21.15.173:0/2561956459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 msgr2=0x7ff1c0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:36.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.473+0000 7ff1c7000700 1 --2- 172.21.15.173:0/2561956459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 0x7ff1c0100cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009910 tx=0x7ff1b0009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:36.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.474+0000 7ff1c7000700 1 -- 172.21.15.173:0/2561956459 shutdown_connections 2024-04-20T10:57:36.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.474+0000 7ff1c7000700 1 --2- 172.21.15.173:0/2561956459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 0x7ff1c0100cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:36.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.474+0000 7ff1c7000700 1 -- 172.21.15.173:0/2561956459 >> 172.21.15.173:0/2561956459 conn(0x7ff1c00fbe60 msgr2=0x7ff1c00fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:36.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.474+0000 7ff1c7000700 1 -- 172.21.15.173:0/2561956459 shutdown_connections 2024-04-20T10:57:36.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.474+0000 7ff1c7000700 1 -- 172.21.15.173:0/2561956459 wait complete. 2024-04-20T10:57:36.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.475+0000 7ff1c7000700 1 Processor -- start 2024-04-20T10:57:36.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.475+0000 7ff1c7000700 1 -- start start 2024-04-20T10:57:36.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.475+0000 7ff1c7000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 0x7ff1c0074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:36.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.475+0000 7ff1c7000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0075040 con 0x7ff1c01008a0 2024-04-20T10:57:36.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.476+0000 7ff1bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 0x7ff1c0074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:36.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.476+0000 7ff1bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 0x7ff1c0074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:37810/0 (socket says 172.21.15.173:37810) 2024-04-20T10:57:36.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.476+0000 7ff1bfe00700 1 -- 172.21.15.173:0/2879094593 learned_addr learned my addr 172.21.15.173:0/2879094593 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:57:36.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.476+0000 7ff1bfe00700 1 -- 172.21.15.173:0/2879094593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b00095f0 con 0x7ff1c01008a0 2024-04-20T10:57:36.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.477+0000 7ff1bfe00700 1 --2- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 0x7ff1c0074b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006010 tx=0x7ff1b00050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:36.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.477+0000 7ff1bcc00700 1 -- 172.21.15.173:0/2879094593 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1b001d070 con 0x7ff1c01008a0 2024-04-20T10:57:36.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.477+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c00732d0 con 0x7ff1c01008a0 2024-04-20T10:57:36.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.477+0000 7ff1bcc00700 1 -- 172.21.15.173:0/2879094593 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1b000be20 con 0x7ff1c01008a0 2024-04-20T10:57:36.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.477+0000 7ff1bcc00700 1 -- 172.21.15.173:0/2879094593 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1b0022de0 con 0x7ff1c01008a0 2024-04-20T10:57:36.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.477+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c0073770 con 0x7ff1c01008a0 2024-04-20T10:57:36.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.478+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c0074180 con 0x7ff1c01008a0 2024-04-20T10:57:36.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.478+0000 7ff1bcc00700 1 -- 172.21.15.173:0/2879094593 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7ff1b00226c0 con 0x7ff1c01008a0 2024-04-20T10:57:36.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.479+0000 7ff1bcc00700 1 --2- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7ff1a40387b0 0x7ff1a403ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:36.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.479+0000 7ff1bcc00700 1 -- 172.21.15.173:0/2879094593 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff1b004ca60 con 0x7ff1c01008a0 2024-04-20T10:57:36.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.480+0000 7ff1bf400700 1 --2- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7ff1a40387b0 0x7ff1a403ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:36.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.481+0000 7ff1bf400700 1 --2- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7ff1a40387b0 0x7ff1a403ac70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac006fd0 tx=0x7ff1ac006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:36.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.494+0000 7ff1bcc00700 1 -- 172.21.15.173:0/2879094593 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff1b000b960 con 0x7ff1c01008a0 2024-04-20T10:57:36.798 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.798+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff1c0062560 con 0x7ff1c01008a0 2024-04-20T10:57:36.799 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.800+0000 7ff1bcc00700 1 -- 172.21.15.173:0/2879094593 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7ff1b0027030 con 0x7ff1c01008a0 2024-04-20T10:57:36.800 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:57:36.800 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.802+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7ff1a40387b0 msgr2=0x7ff1a403ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.802+0000 7ff1c7000700 1 --2- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7ff1a40387b0 0x7ff1a403ac70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac006fd0 tx=0x7ff1ac006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.802+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 msgr2=0x7ff1c0074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.802+0000 7ff1c7000700 1 --2- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 0x7ff1c0074b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006010 tx=0x7ff1b00050b0 comp rx=0 tx=0).stop 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.803+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 shutdown_connections 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.803+0000 7ff1c7000700 1 --2- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7ff1a40387b0 0x7ff1a403ac70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.803+0000 7ff1c7000700 1 --2- 172.21.15.173:0/2879094593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1c01008a0 0x7ff1c0074b00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.803+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 >> 172.21.15.173:0/2879094593 conn(0x7ff1c00fbe60 msgr2=0x7ff1c00fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.803+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 shutdown_connections 2024-04-20T10:57:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:36.803+0000 7ff1c7000700 1 -- 172.21.15.173:0/2879094593 wait complete. 2024-04-20T10:57:36.804 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:57:37.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:36 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/2879094593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:57:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:37 smithi117 ceph-mon[15659]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:38.516 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:57:38.516 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:57:38.888 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:38.984 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:40.382 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:39 smithi117 ceph-mon[15659]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:40.865 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.865+0000 7f9bf0a00700 1 -- 172.21.15.173:0/2944288271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 msgr2=0x7f9bec0ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:40.865 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.865+0000 7f9bf0a00700 1 --2- 172.21.15.173:0/2944288271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 0x7f9bec0ffb80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009910 tx=0x7f9bdc009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:40.865 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.865+0000 7f9bf0a00700 1 -- 172.21.15.173:0/2944288271 shutdown_connections 2024-04-20T10:57:40.866 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.865+0000 7f9bf0a00700 1 --2- 172.21.15.173:0/2944288271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 0x7f9bec0ffb80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:40.866 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.865+0000 7f9bf0a00700 1 -- 172.21.15.173:0/2944288271 >> 172.21.15.173:0/2944288271 conn(0x7f9bec0fad00 msgr2=0x7f9bec0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:40.866 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.865+0000 7f9bf0a00700 1 -- 172.21.15.173:0/2944288271 shutdown_connections 2024-04-20T10:57:40.866 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.865+0000 7f9bf0a00700 1 -- 172.21.15.173:0/2944288271 wait complete. 2024-04-20T10:57:40.867 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.866+0000 7f9bf0a00700 1 Processor -- start 2024-04-20T10:57:40.867 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.866+0000 7f9bf0a00700 1 -- start start 2024-04-20T10:57:40.867 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.867+0000 7f9bf0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 0x7f9bec195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:40.867 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.867+0000 7f9bf0a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec196290 con 0x7f9bec0ff760 2024-04-20T10:57:40.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.867+0000 7f9be9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 0x7f9bec195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.867+0000 7f9be9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 0x7f9bec195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:37830/0 (socket says 172.21.15.173:37830) 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.867+0000 7f9be9e00700 1 -- 172.21.15.173:0/665574930 learned_addr learned my addr 172.21.15.173:0/665574930 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.868+0000 7f9be9e00700 1 -- 172.21.15.173:0/665574930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc0095f0 con 0x7f9bec0ff760 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.868+0000 7f9be9e00700 1 --2- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 0x7f9bec195d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc006010 tx=0x7f9bdc0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.869+0000 7f9be2a00700 1 -- 172.21.15.173:0/665574930 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bdc01d070 con 0x7f9bec0ff760 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.869+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec196490 con 0x7f9bec0ff760 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.869+0000 7f9be2a00700 1 -- 172.21.15.173:0/665574930 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bdc00be20 con 0x7f9bec0ff760 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.869+0000 7f9be2a00700 1 -- 172.21.15.173:0/665574930 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bdc022de0 con 0x7f9bec0ff760 2024-04-20T10:57:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.869+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec196930 con 0x7f9bec0ff760 2024-04-20T10:57:40.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.870+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec18f850 con 0x7f9bec0ff760 2024-04-20T10:57:40.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.870+0000 7f9be2a00700 1 -- 172.21.15.173:0/665574930 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f9bdc0226c0 con 0x7f9bec0ff760 2024-04-20T10:57:40.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.871+0000 7f9be2a00700 1 --2- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f9bd00387b0 0x7f9bd003ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:40.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.871+0000 7f9be2a00700 1 -- 172.21.15.173:0/665574930 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9bdc04ca60 con 0x7f9bec0ff760 2024-04-20T10:57:40.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.871+0000 7f9be9400700 1 --2- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f9bd00387b0 0x7f9bd003ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:40.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.872+0000 7f9be9400700 1 --2- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f9bd00387b0 0x7f9bd003ac70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8006fd0 tx=0x7f9bd8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:40.889 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:40.889+0000 7f9be2a00700 1 -- 172.21.15.173:0/665574930 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9bdc00b960 con 0x7f9bec0ff760 2024-04-20T10:57:41.192 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.191+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9bec062510 con 0x7f9bec0ff760 2024-04-20T10:57:41.192 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.192+0000 7f9be2a00700 1 -- 172.21.15.173:0/665574930 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f9bec062510 con 0x7f9bec0ff760 2024-04-20T10:57:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:57:41.193 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:57:41.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f9bd00387b0 msgr2=0x7f9bd003ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:41.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 --2- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f9bd00387b0 0x7f9bd003ac70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8006fd0 tx=0x7f9bd8006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:41.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 msgr2=0x7f9bec195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:41.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 --2- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 0x7f9bec195d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc006010 tx=0x7f9bdc0050b0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 shutdown_connections 2024-04-20T10:57:41.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 --2- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f9bd00387b0 0x7f9bd003ac70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 --2- 172.21.15.173:0/665574930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bec0ff760 0x7f9bec195d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 >> 172.21.15.173:0/665574930 conn(0x7f9bec0fad00 msgr2=0x7f9bec069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:41.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 shutdown_connections 2024-04-20T10:57:41.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:41.195+0000 7f9bf0a00700 1 -- 172.21.15.173:0/665574930 wait complete. 2024-04-20T10:57:41.196 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:57:42.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:41 smithi117 ceph-mon[15659]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:42.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:41 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/665574930' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:57:42.859 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:57:42.860 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:57:43.227 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:43.332 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:44.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:43 smithi117 ceph-mon[15659]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:45.190 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.190+0000 7f928da00700 1 -- 172.21.15.173:0/2360493123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 msgr2=0x7f9288107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:45.190 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.190+0000 7f928da00700 1 --2- 172.21.15.173:0/2360493123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 0x7f9288107510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9278009910 tx=0x7f9278009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:45.190 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.190+0000 7f928da00700 1 -- 172.21.15.173:0/2360493123 shutdown_connections 2024-04-20T10:57:45.190 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.190+0000 7f928da00700 1 --2- 172.21.15.173:0/2360493123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 0x7f9288107510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:45.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.190+0000 7f928da00700 1 -- 172.21.15.173:0/2360493123 >> 172.21.15.173:0/2360493123 conn(0x7f9288075cb0 msgr2=0x7f92880780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:45.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.190+0000 7f928da00700 1 -- 172.21.15.173:0/2360493123 shutdown_connections 2024-04-20T10:57:45.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.190+0000 7f928da00700 1 -- 172.21.15.173:0/2360493123 wait complete. 2024-04-20T10:57:45.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.191+0000 7f928da00700 1 Processor -- start 2024-04-20T10:57:45.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.191+0000 7f928da00700 1 -- start start 2024-04-20T10:57:45.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.192+0000 7f928da00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 0x7f928819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:45.192 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.192+0000 7f928da00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928819c640 con 0x7f92881070f0 2024-04-20T10:57:45.192 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.192+0000 7f9287e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 0x7f928819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:45.192 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.192+0000 7f9287e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 0x7f928819c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:42760/0 (socket says 172.21.15.173:42760) 2024-04-20T10:57:45.192 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.192+0000 7f9287e00700 1 -- 172.21.15.173:0/2598333385 learned_addr learned my addr 172.21.15.173:0/2598333385 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:57:45.193 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.193+0000 7f9287e00700 1 -- 172.21.15.173:0/2598333385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92780095f0 con 0x7f92881070f0 2024-04-20T10:57:45.193 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.193+0000 7f9287e00700 1 --2- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 0x7f928819c100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9278009ed0 tx=0x7f9278004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.194+0000 7f9284c00700 1 -- 172.21.15.173:0/2598333385 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927801d070 con 0x7f92881070f0 2024-04-20T10:57:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.194+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928819c840 con 0x7f92881070f0 2024-04-20T10:57:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.194+0000 7f9284c00700 1 -- 172.21.15.173:0/2598333385 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f927800be20 con 0x7f92881070f0 2024-04-20T10:57:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.194+0000 7f9284c00700 1 -- 172.21.15.173:0/2598333385 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927800f460 con 0x7f92881070f0 2024-04-20T10:57:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.194+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928819cce0 con 0x7f92881070f0 2024-04-20T10:57:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.195+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9288195ae0 con 0x7f92881070f0 2024-04-20T10:57:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.196+0000 7f9284c00700 1 -- 172.21.15.173:0/2598333385 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f9278022800 con 0x7f92881070f0 2024-04-20T10:57:45.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.196+0000 7f9284c00700 1 --2- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f926c0387d0 0x7f926c03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:45.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.196+0000 7f9284c00700 1 -- 172.21.15.173:0/2598333385 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f927804ccc0 con 0x7f92881070f0 2024-04-20T10:57:45.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.197+0000 7f9287400700 1 --2- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f926c0387d0 0x7f926c03ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:45.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.198+0000 7f9287400700 1 --2- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f926c0387d0 0x7f926c03ac90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9274006fd0 tx=0x7f9274006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:45.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.212+0000 7f9284c00700 1 -- 172.21.15.173:0/2598333385 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9278053080 con 0x7f92881070f0 2024-04-20T10:57:45.514 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.514+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9288062510 con 0x7f92881070f0 2024-04-20T10:57:45.515 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.516+0000 7f9284c00700 1 -- 172.21.15.173:0/2598333385 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f9288062510 con 0x7f92881070f0 2024-04-20T10:57:45.516 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:57:45.516 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f926c0387d0 msgr2=0x7f926c03ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 --2- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f926c0387d0 0x7f926c03ac90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9274006fd0 tx=0x7f9274006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 msgr2=0x7f928819c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 --2- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 0x7f928819c100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9278009ed0 tx=0x7f9278004690 comp rx=0 tx=0).stop 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 shutdown_connections 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 --2- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f926c0387d0 0x7f926c03ac90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 --2- 172.21.15.173:0/2598333385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92881070f0 0x7f928819c100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 >> 172.21.15.173:0/2598333385 conn(0x7f9288075cb0 msgr2=0x7f9288076730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 shutdown_connections 2024-04-20T10:57:45.519 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:45.519+0000 7f928da00700 1 -- 172.21.15.173:0/2598333385 wait complete. 2024-04-20T10:57:45.520 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:57:46.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:45 smithi117 ceph-mon[15659]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:46.331 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:45 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/2598333385' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:57:47.232 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:57:47.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:57:47.606 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:47.709 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:48.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:47 smithi117 ceph-mon[15659]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:49.489 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.489+0000 7f5741200700 1 -- 172.21.15.173:0/4096145133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 msgr2=0x7f573c073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:49.489 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.489+0000 7f5741200700 1 --2- 172.21.15.173:0/4096145133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 0x7f573c073370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f572c009910 tx=0x7f572c009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:49.489 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.489+0000 7f5741200700 1 -- 172.21.15.173:0/4096145133 shutdown_connections 2024-04-20T10:57:49.489 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.489+0000 7f5741200700 1 --2- 172.21.15.173:0/4096145133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 0x7f573c073370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.489 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.489+0000 7f5741200700 1 -- 172.21.15.173:0/4096145133 >> 172.21.15.173:0/4096145133 conn(0x7f573c0fbe30 msgr2=0x7f573c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:49.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.489+0000 7f5741200700 1 -- 172.21.15.173:0/4096145133 shutdown_connections 2024-04-20T10:57:49.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.489+0000 7f5741200700 1 -- 172.21.15.173:0/4096145133 wait complete. 2024-04-20T10:57:49.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.490+0000 7f5741200700 1 Processor -- start 2024-04-20T10:57:49.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.490+0000 7f5741200700 1 -- start start 2024-04-20T10:57:49.491 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.491+0000 7f5741200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 0x7f573c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.491 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.491+0000 7f5741200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c198480 con 0x7f573c074f10 2024-04-20T10:57:49.491 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.491+0000 7f5739e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 0x7f573c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.491 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.491+0000 7f5739e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 0x7f573c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:42776/0 (socket says 172.21.15.173:42776) 2024-04-20T10:57:49.491 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.491+0000 7f5739e00700 1 -- 172.21.15.173:0/1382984618 learned_addr learned my addr 172.21.15.173:0/1382984618 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:57:49.492 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.492+0000 7f5739e00700 1 -- 172.21.15.173:0/1382984618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c0095f0 con 0x7f573c074f10 2024-04-20T10:57:49.492 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.493+0000 7f5739e00700 1 --2- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 0x7f573c197f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f572c006010 tx=0x7f572c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:49.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.493+0000 7f5733400700 1 -- 172.21.15.173:0/1382984618 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f572c01d070 con 0x7f573c074f10 2024-04-20T10:57:49.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.493+0000 7f5733400700 1 -- 172.21.15.173:0/1382984618 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f572c00be20 con 0x7f573c074f10 2024-04-20T10:57:49.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.493+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c198680 con 0x7f573c074f10 2024-04-20T10:57:49.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.493+0000 7f5733400700 1 -- 172.21.15.173:0/1382984618 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f572c022de0 con 0x7f573c074f10 2024-04-20T10:57:49.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.494+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c198b20 con 0x7f573c074f10 2024-04-20T10:57:49.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.495+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573c191890 con 0x7f573c074f10 2024-04-20T10:57:49.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.495+0000 7f5733400700 1 -- 172.21.15.173:0/1382984618 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f572c0226c0 con 0x7f573c074f10 2024-04-20T10:57:49.495 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.495+0000 7f5733400700 1 --2- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f5720038820 0x7f572003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.495 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.495+0000 7f5733400700 1 -- 172.21.15.173:0/1382984618 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f572c04cac0 con 0x7f573c074f10 2024-04-20T10:57:49.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.496+0000 7f5731400700 1 --2- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f5720038820 0x7f572003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.497+0000 7f5731400700 1 --2- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f5720038820 0x7f572003ace0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5724006fd0 tx=0x7f5724006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:49.511 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.511+0000 7f5733400700 1 -- 172.21.15.173:0/1382984618 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f572c02ba00 con 0x7f573c074f10 2024-04-20T10:57:49.808 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.808+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f573c02cd40 con 0x7f573c074f10 2024-04-20T10:57:49.809 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.810+0000 7f5733400700 1 -- 172.21.15.173:0/1382984618 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f572c027030 con 0x7f573c074f10 2024-04-20T10:57:49.810 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:57:49.810 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:57:49.812 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f5720038820 msgr2=0x7f572003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:49.812 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 --2- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f5720038820 0x7f572003ace0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5724006fd0 tx=0x7f5724006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:49.812 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 msgr2=0x7f573c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:49.812 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 --2- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 0x7f573c197f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f572c006010 tx=0x7f572c0050b0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.812 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 shutdown_connections 2024-04-20T10:57:49.813 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 --2- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f5720038820 0x7f572003ace0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.813 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 --2- 172.21.15.173:0/1382984618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f573c074f10 0x7f573c197f40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.813 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 >> 172.21.15.173:0/1382984618 conn(0x7f573c0fbe30 msgr2=0x7f573c0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:49.813 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 shutdown_connections 2024-04-20T10:57:49.813 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:49.812+0000 7f5741200700 1 -- 172.21.15.173:0/1382984618 wait complete. 2024-04-20T10:57:49.814 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:57:50.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:50 smithi117 ceph-mon[15659]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:50.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:50 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/1382984618' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:57:51.539 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:57:51.539 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: Deploying daemon prometheus.smithi117 on smithi117 2024-04-20T10:57:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:51 smithi117 ceph-mon[15659]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:51.913 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:52.011 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:53.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:52 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:57:53.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.989+0000 7f7c00400700 1 -- 172.21.15.173:0/463936064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 msgr2=0x7f7bf8073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:53.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.989+0000 7f7c00400700 1 --2- 172.21.15.173:0/463936064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 0x7f7bf8073370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009910 tx=0x7f7bec009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:53.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.989+0000 7f7c00400700 1 -- 172.21.15.173:0/463936064 shutdown_connections 2024-04-20T10:57:53.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.989+0000 7f7c00400700 1 --2- 172.21.15.173:0/463936064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 0x7f7bf8073370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:53.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.989+0000 7f7c00400700 1 -- 172.21.15.173:0/463936064 >> 172.21.15.173:0/463936064 conn(0x7f7bf80fbe30 msgr2=0x7f7bf80fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:53.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.990+0000 7f7c00400700 1 -- 172.21.15.173:0/463936064 shutdown_connections 2024-04-20T10:57:53.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.990+0000 7f7c00400700 1 -- 172.21.15.173:0/463936064 wait complete. 2024-04-20T10:57:53.992 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.991+0000 7f7c00400700 1 Processor -- start 2024-04-20T10:57:53.992 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.991+0000 7f7c00400700 1 -- start start 2024-04-20T10:57:53.992 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.992+0000 7f7c00400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 0x7f7bf819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:53.992 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.992+0000 7f7c00400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf819ca00 con 0x7f7bf8074f10 2024-04-20T10:57:53.993 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.992+0000 7f7bfda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 0x7f7bf819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:53.993 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.992+0000 7f7bfda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 0x7f7bf819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:36212/0 (socket says 172.21.15.173:36212) 2024-04-20T10:57:53.993 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.992+0000 7f7bfda00700 1 -- 172.21.15.173:0/2922747234 learned_addr learned my addr 172.21.15.173:0/2922747234 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:57:53.994 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.993+0000 7f7bfda00700 1 -- 172.21.15.173:0/2922747234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0095f0 con 0x7f7bf8074f10 2024-04-20T10:57:53.994 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.993+0000 7f7bfda00700 1 --2- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 0x7f7bf819c4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006010 tx=0x7f7bec0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:53.995 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.994+0000 7f7bf6000700 1 -- 172.21.15.173:0/2922747234 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7bec01d070 con 0x7f7bf8074f10 2024-04-20T10:57:53.995 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.994+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf8104490 con 0x7f7bf8074f10 2024-04-20T10:57:53.995 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.994+0000 7f7bf6000700 1 -- 172.21.15.173:0/2922747234 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bec00be20 con 0x7f7bf8074f10 2024-04-20T10:57:53.995 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.994+0000 7f7bf6000700 1 -- 172.21.15.173:0/2922747234 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7bec022de0 con 0x7f7bf8074f10 2024-04-20T10:57:53.995 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.994+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf819cf10 con 0x7f7bf8074f10 2024-04-20T10:57:53.996 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.995+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8195e20 con 0x7f7bf8074f10 2024-04-20T10:57:53.996 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.996+0000 7f7bf6000700 1 -- 172.21.15.173:0/2922747234 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f7bec0226c0 con 0x7f7bf8074f10 2024-04-20T10:57:53.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.996+0000 7f7bf6000700 1 --2- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f7be00387b0 0x7f7be003ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:53.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.996+0000 7f7bf6000700 1 -- 172.21.15.173:0/2922747234 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7bec04c9a0 con 0x7f7bf8074f10 2024-04-20T10:57:53.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.997+0000 7f7bfd000700 1 --2- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f7be00387b0 0x7f7be003ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:53.999 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:53.998+0000 7f7bfd000700 1 --2- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f7be00387b0 0x7f7be003ac70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7be4006fd0 tx=0x7f7be4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:54.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.015+0000 7f7bf6000700 1 -- 172.21.15.173:0/2922747234 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7bec00b960 con 0x7f7bf8074f10 2024-04-20T10:57:54.318 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.317+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7bf810d0f0 con 0x7f7bf8074f10 2024-04-20T10:57:54.320 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.319+0000 7f7bf6000700 1 -- 172.21.15.173:0/2922747234 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f7bec027030 con 0x7f7bf8074f10 2024-04-20T10:57:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:57:54.320 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f7be00387b0 msgr2=0x7f7be003ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 --2- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f7be00387b0 0x7f7be003ac70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7be4006fd0 tx=0x7f7be4006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 msgr2=0x7f7bf819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 --2- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 0x7f7bf819c4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006010 tx=0x7f7bec0050b0 comp rx=0 tx=0).stop 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 shutdown_connections 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 --2- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f7be00387b0 0x7f7be003ac70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 --2- 172.21.15.173:0/2922747234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bf8074f10 0x7f7bf819c4c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 >> 172.21.15.173:0/2922747234 conn(0x7f7bf80fbe30 msgr2=0x7f7bf80fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:54.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 shutdown_connections 2024-04-20T10:57:54.324 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:54.322+0000 7f7c00400700 1 -- 172.21.15.173:0/2922747234 wait complete. 2024-04-20T10:57:54.324 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:57:54.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:53 smithi117 ceph-mon[15659]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:55.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:54 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/2922747234' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:57:56.043 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:57:56.043 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:57:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:55 smithi117 ceph-mon[15659]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:56.411 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:56.517 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:57:58.340 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.339+0000 7fdc12000700 1 -- 172.21.15.173:0/1729317050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 msgr2=0x7fdc0c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:58.341 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.339+0000 7fdc12000700 1 --2- 172.21.15.173:0/1729317050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 0x7fdc0c100cc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009910 tx=0x7fdbfc009c20 comp rx=0 tx=0).stop 2024-04-20T10:57:58.341 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.339+0000 7fdc12000700 1 -- 172.21.15.173:0/1729317050 shutdown_connections 2024-04-20T10:57:58.341 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.339+0000 7fdc12000700 1 --2- 172.21.15.173:0/1729317050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 0x7fdc0c100cc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:58.341 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.339+0000 7fdc12000700 1 -- 172.21.15.173:0/1729317050 >> 172.21.15.173:0/1729317050 conn(0x7fdc0c0fbe60 msgr2=0x7fdc0c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:58.341 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.339+0000 7fdc12000700 1 -- 172.21.15.173:0/1729317050 shutdown_connections 2024-04-20T10:57:58.341 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.340+0000 7fdc12000700 1 -- 172.21.15.173:0/1729317050 wait complete. 2024-04-20T10:57:58.342 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.341+0000 7fdc12000700 1 Processor -- start 2024-04-20T10:57:58.342 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.341+0000 7fdc12000700 1 -- start start 2024-04-20T10:57:58.342 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.341+0000 7fdc12000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 0x7fdc0c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:58.342 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.341+0000 7fdc12000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c1984a0 con 0x7fdc0c1008a0 2024-04-20T10:57:58.343 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.342+0000 7fdc0b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 0x7fdc0c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:58.343 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.342+0000 7fdc0b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 0x7fdc0c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:36232/0 (socket says 172.21.15.173:36232) 2024-04-20T10:57:58.343 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.342+0000 7fdc0b400700 1 -- 172.21.15.173:0/3169252981 learned_addr learned my addr 172.21.15.173:0/3169252981 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:57:58.343 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.342+0000 7fdc0b400700 1 -- 172.21.15.173:0/3169252981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0095f0 con 0x7fdc0c1008a0 2024-04-20T10:57:58.344 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.343+0000 7fdc0b400700 1 --2- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 0x7fdc0c197f60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006010 tx=0x7fdbfc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:58.344 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.343+0000 7fdc03e00700 1 -- 172.21.15.173:0/3169252981 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbfc01d070 con 0x7fdc0c1008a0 2024-04-20T10:57:58.344 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.343+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0c1986a0 con 0x7fdc0c1008a0 2024-04-20T10:57:58.344 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.343+0000 7fdc03e00700 1 -- 172.21.15.173:0/3169252981 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbfc005dc0 con 0x7fdc0c1008a0 2024-04-20T10:57:58.344 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.343+0000 7fdc03e00700 1 -- 172.21.15.173:0/3169252981 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbfc022e90 con 0x7fdc0c1008a0 2024-04-20T10:57:58.344 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.343+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c198b40 con 0x7fdc0c1008a0 2024-04-20T10:57:58.345 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.344+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0c191ad0 con 0x7fdc0c1008a0 2024-04-20T10:57:58.345 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.345+0000 7fdc03e00700 1 -- 172.21.15.173:0/3169252981 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7fdbfc022470 con 0x7fdc0c1008a0 2024-04-20T10:57:58.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.345+0000 7fdc03e00700 1 --2- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fdbf0038760 0x7fdbf003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:58.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.345+0000 7fdc03e00700 1 -- 172.21.15.173:0/3169252981 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdbfc04c8b0 con 0x7fdc0c1008a0 2024-04-20T10:57:58.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.346+0000 7fdc0aa00700 1 --2- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fdbf0038760 0x7fdbf003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:58.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.347+0000 7fdc0aa00700 1 --2- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fdbf0038760 0x7fdbf003ac20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006fd0 tx=0x7fdbf8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:58.361 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.361+0000 7fdc03e00700 1 -- 172.21.15.173:0/3169252981 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdbfc00f4f0 con 0x7fdc0c1008a0 2024-04-20T10:57:58.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:57 smithi117 ceph-mon[15659]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:57:58.668 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.667+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdc0c062510 con 0x7fdc0c1008a0 2024-04-20T10:57:58.668 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.668+0000 7fdc03e00700 1 -- 172.21.15.173:0/3169252981 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fdbfc027070 con 0x7fdc0c1008a0 2024-04-20T10:57:58.669 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:57:58.669 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:57:58.671 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fdbf0038760 msgr2=0x7fdbf003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:58.672 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 --2- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fdbf0038760 0x7fdbf003ac20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006fd0 tx=0x7fdbf8006e40 comp rx=0 tx=0).stop 2024-04-20T10:57:58.672 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 msgr2=0x7fdc0c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:58.672 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 --2- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 0x7fdc0c197f60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006010 tx=0x7fdbfc009f80 comp rx=0 tx=0).stop 2024-04-20T10:57:58.672 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 shutdown_connections 2024-04-20T10:57:58.672 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 --2- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7fdbf0038760 0x7fdbf003ac20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:58.672 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 --2- 172.21.15.173:0/3169252981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc0c1008a0 0x7fdc0c197f60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:58.673 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 >> 172.21.15.173:0/3169252981 conn(0x7fdc0c0fbe60 msgr2=0x7fdc0c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:58.673 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 shutdown_connections 2024-04-20T10:57:58.673 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:57:58.671+0000 7fdc12000700 1 -- 172.21.15.173:0/3169252981 wait complete. 2024-04-20T10:57:58.673 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:57:59.117 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:57:59 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/3169252981' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:00.190 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:00 smithi117 ceph-mon[15659]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:00.397 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:00.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:00.775 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:00.875 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:02.014 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:02 smithi117 ceph-mon[15659]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:03.045 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.044+0000 7f1120000700 1 -- 172.21.15.173:0/4233281822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 msgr2=0x7f1118102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:03.045 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.044+0000 7f1120000700 1 --2- 172.21.15.173:0/4233281822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 0x7f1118102eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f110c009910 tx=0x7f110c009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:03.045 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.044+0000 7f1120000700 1 -- 172.21.15.173:0/4233281822 shutdown_connections 2024-04-20T10:58:03.045 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.044+0000 7f1120000700 1 --2- 172.21.15.173:0/4233281822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 0x7f1118102eb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:03.046 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.044+0000 7f1120000700 1 -- 172.21.15.173:0/4233281822 >> 172.21.15.173:0/4233281822 conn(0x7f11180fe050 msgr2=0x7f1118100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:03.046 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.044+0000 7f1120000700 1 -- 172.21.15.173:0/4233281822 shutdown_connections 2024-04-20T10:58:03.046 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.044+0000 7f1120000700 1 -- 172.21.15.173:0/4233281822 wait complete. 2024-04-20T10:58:03.046 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.046+0000 7f1120000700 1 Processor -- start 2024-04-20T10:58:03.047 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.046+0000 7f1120000700 1 -- start start 2024-04-20T10:58:03.047 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.047+0000 7f1120000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 0x7f1118197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:03.048 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.047+0000 7f1120000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1118198480 con 0x7f1118102a90 2024-04-20T10:58:03.048 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.047+0000 7f111d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 0x7f1118197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:03.048 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.047+0000 7f111d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 0x7f1118197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:35642/0 (socket says 172.21.15.173:35642) 2024-04-20T10:58:03.048 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.047+0000 7f111d600700 1 -- 172.21.15.173:0/1691940809 learned_addr learned my addr 172.21.15.173:0/1691940809 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:03.048 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.048+0000 7f111d600700 1 -- 172.21.15.173:0/1691940809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c0095f0 con 0x7f1118102a90 2024-04-20T10:58:03.049 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.048+0000 7f111d600700 1 --2- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 0x7f1118197f40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f110c006010 tx=0x7f110c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:03.050 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.049+0000 7f1116000700 1 -- 172.21.15.173:0/1691940809 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f110c01d070 con 0x7f1118102a90 2024-04-20T10:58:03.050 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.049+0000 7f1116000700 1 -- 172.21.15.173:0/1691940809 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f110c00be20 con 0x7f1118102a90 2024-04-20T10:58:03.050 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.049+0000 7f1116000700 1 -- 172.21.15.173:0/1691940809 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f110c022de0 con 0x7f1118102a90 2024-04-20T10:58:03.050 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.049+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1118198680 con 0x7f1118102a90 2024-04-20T10:58:03.050 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.049+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1118198b20 con 0x7f1118102a90 2024-04-20T10:58:03.051 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.050+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11181918c0 con 0x7f1118102a90 2024-04-20T10:58:03.051 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.051+0000 7f1116000700 1 -- 172.21.15.173:0/1691940809 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45613+0+0 (secure 0 0 0) 0x7f110c0226c0 con 0x7f1118102a90 2024-04-20T10:58:03.052 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.051+0000 7f1116000700 1 --2- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1100038820 0x7f110003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:03.052 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.051+0000 7f1116000700 1 -- 172.21.15.173:0/1691940809 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f110c04cc00 con 0x7f1118102a90 2024-04-20T10:58:03.052 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.052+0000 7f111cc00700 1 --2- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1100038820 0x7f110003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:03.053 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.053+0000 7f111cc00700 1 --2- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1100038820 0x7f110003ace0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1104006fd0 tx=0x7f1104006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:03.068 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.067+0000 7f1116000700 1 -- 172.21.15.173:0/1691940809 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f110c02ba00 con 0x7f1118102a90 2024-04-20T10:58:03.373 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.372+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f111802cd40 con 0x7f1118102a90 2024-04-20T10:58:03.374 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.373+0000 7f1116000700 1 -- 172.21.15.173:0/1691940809 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f110c04ce60 con 0x7f1118102a90 2024-04-20T10:58:03.374 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:03.374 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1100038820 msgr2=0x7f110003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 --2- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1100038820 0x7f110003ace0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1104006fd0 tx=0x7f1104006e40 comp rx=0 tx=0).stop 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 msgr2=0x7f1118197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 --2- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 0x7f1118197f40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f110c006010 tx=0x7f110c0050b0 comp rx=0 tx=0).stop 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 shutdown_connections 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 --2- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f1100038820 0x7f110003ace0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 --2- 172.21.15.173:0/1691940809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1118102a90 0x7f1118197f40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 >> 172.21.15.173:0/1691940809 conn(0x7f11180fe050 msgr2=0x7f11180feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:03.377 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 shutdown_connections 2024-04-20T10:58:03.378 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:03.376+0000 7f1120000700 1 -- 172.21.15.173:0/1691940809 wait complete. 2024-04-20T10:58:03.378 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:04.384 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:04 smithi117 ceph-mon[15659]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:04.384 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:04 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/1691940809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:04.384 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:04 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:04.384 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:04 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:04.384 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:04 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:04.384 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:04 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-20T10:58:04.986 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:04.986 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:05.360 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:05.464 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:05.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:05 smithi117 ceph-mon[15659]: from='mgr.14164 172.21.15.117:0/506133696' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-20T10:58:05.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:05 smithi117 ceph-mon[15659]: mgrmap e14: smithi117.ivnklj(active, since 77s) 2024-04-20T10:58:07.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.427+0000 7f07d3c00700 1 -- 172.21.15.173:0/2432794519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 msgr2=0x7f07cc102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:07.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.427+0000 7f07d3c00700 1 --2- 172.21.15.173:0/2432794519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 0x7f07cc102eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009910 tx=0x7f07c0009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:07.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.428+0000 7f07d3c00700 1 -- 172.21.15.173:0/2432794519 shutdown_connections 2024-04-20T10:58:07.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.428+0000 7f07d3c00700 1 --2- 172.21.15.173:0/2432794519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 0x7f07cc102eb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.428+0000 7f07d3c00700 1 -- 172.21.15.173:0/2432794519 >> 172.21.15.173:0/2432794519 conn(0x7f07cc0fe050 msgr2=0x7f07cc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:07.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.428+0000 7f07d3c00700 1 -- 172.21.15.173:0/2432794519 shutdown_connections 2024-04-20T10:58:07.430 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.428+0000 7f07d3c00700 1 -- 172.21.15.173:0/2432794519 wait complete. 2024-04-20T10:58:07.430 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.429+0000 7f07d3c00700 1 Processor -- start 2024-04-20T10:58:07.430 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.429+0000 7f07d3c00700 1 -- start start 2024-04-20T10:58:07.430 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.429+0000 7f07d3c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 0x7f07cc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:07.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.429+0000 7f07d3c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc198480 con 0x7f07cc102a90 2024-04-20T10:58:07.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.430+0000 7f07d1200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 0x7f07cc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:07.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.430+0000 7f07d1200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 0x7f07cc197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:35656/0 (socket says 172.21.15.173:35656) 2024-04-20T10:58:07.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.430+0000 7f07d1200700 1 -- 172.21.15.173:0/159061170 learned_addr learned my addr 172.21.15.173:0/159061170 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:07.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.430+0000 7f07d1200700 1 -- 172.21.15.173:0/159061170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c00095f0 con 0x7f07cc102a90 2024-04-20T10:58:07.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.431+0000 7f07d1200700 1 --2- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 0x7f07cc197f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07c0006010 tx=0x7f07c00050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.431+0000 7f07c9600700 1 -- 172.21.15.173:0/159061170 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07c001d070 con 0x7f07cc102a90 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.431+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc198680 con 0x7f07cc102a90 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.431+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc198b20 con 0x7f07cc102a90 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.431+0000 7f07c9600700 1 -- 172.21.15.173:0/159061170 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07c000be20 con 0x7f07cc102a90 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.431+0000 7f07c9600700 1 -- 172.21.15.173:0/159061170 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07c0022dc0 con 0x7f07cc102a90 2024-04-20T10:58:07.433 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.432+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc1918c0 con 0x7f07cc102a90 2024-04-20T10:58:07.433 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.433+0000 7f07c9600700 1 -- 172.21.15.173:0/159061170 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 14) v1 ==== 45627+0+0 (secure 0 0 0) 0x7f07c000b960 con 0x7f07cc102a90 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.433+0000 7f07c9600700 1 --2- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f07b4038870 0x7f07b403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.433+0000 7f07c9600700 1 -- 172.21.15.173:0/159061170 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f07c004cac0 con 0x7f07cc102a90 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.433+0000 7f07cbe00700 1 -- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f07b4038870 msgr2=0x7f07b403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1996106837 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.433+0000 7f07cbe00700 1 --2- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f07b4038870 0x7f07b403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:58:07.449 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.448+0000 7f07c9600700 1 -- 172.21.15.173:0/159061170 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f07c002a7d0 con 0x7f07cc102a90 2024-04-20T10:58:07.635 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.634+0000 7f07cbe00700 1 -- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f07b4038870 msgr2=0x7f07b403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1996106837 2024-04-20T10:58:07.635 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.634+0000 7f07cbe00700 1 --2- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f07b4038870 0x7f07b403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:58:07.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.742+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f07cc062510 con 0x7f07cc102a90 2024-04-20T10:58:07.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.744+0000 7f07c9600700 1 -- 172.21.15.173:0/159061170 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f07cc062510 con 0x7f07cc102a90 2024-04-20T10:58:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:07.745 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:07.747 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.746+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f07b4038870 msgr2=0x7f07b403ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:58:07.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.746+0000 7f07d3c00700 1 --2- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f07b4038870 0x7f07b403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.746+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 msgr2=0x7f07cc197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:07.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.746+0000 7f07d3c00700 1 --2- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 0x7f07cc197f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07c0006010 tx=0x7f07c00050b0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.746+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 shutdown_connections 2024-04-20T10:58:07.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.746+0000 7f07d3c00700 1 --2- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f07b4038870 0x7f07b403ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.746+0000 7f07d3c00700 1 --2- 172.21.15.173:0/159061170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07cc102a90 0x7f07cc197f40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.746+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 >> 172.21.15.173:0/159061170 conn(0x7f07cc0fe050 msgr2=0x7f07cc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:07.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.747+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 shutdown_connections 2024-04-20T10:58:07.749 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:07.747+0000 7f07d3c00700 1 -- 172.21.15.173:0/159061170 wait complete. 2024-04-20T10:58:07.749 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:08.046 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:07 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/159061170' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:09.426 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:09.426 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:09.805 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:09.906 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:11.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.903+0000 7f3d0c800700 1 -- 172.21.15.173:0/1990016119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 msgr2=0x7f3d041074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:11.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.903+0000 7f3d0c800700 1 --2- 172.21.15.173:0/1990016119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 0x7f3d041074c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009910 tx=0x7f3cf8009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:11.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.903+0000 7f3d0c800700 1 -- 172.21.15.173:0/1990016119 shutdown_connections 2024-04-20T10:58:11.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.903+0000 7f3d0c800700 1 --2- 172.21.15.173:0/1990016119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 0x7f3d041074c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:11.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.903+0000 7f3d0c800700 1 -- 172.21.15.173:0/1990016119 >> 172.21.15.173:0/1990016119 conn(0x7f3d04075f60 msgr2=0x7f3d040783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:11.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.903+0000 7f3d0c800700 1 -- 172.21.15.173:0/1990016119 shutdown_connections 2024-04-20T10:58:11.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.903+0000 7f3d0c800700 1 -- 172.21.15.173:0/1990016119 wait complete. 2024-04-20T10:58:11.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.904+0000 7f3d0c800700 1 Processor -- start 2024-04-20T10:58:11.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.904+0000 7f3d0c800700 1 -- start start 2024-04-20T10:58:11.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.904+0000 7f3d0c800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 0x7f3d0419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:11.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.905+0000 7f3d0c800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0419c860 con 0x7f3d041070a0 2024-04-20T10:58:11.906 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.905+0000 7f3d09e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 0x7f3d0419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:11.906 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.905+0000 7f3d09e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 0x7f3d0419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:34512/0 (socket says 172.21.15.173:34512) 2024-04-20T10:58:11.906 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.905+0000 7f3d09e00700 1 -- 172.21.15.173:0/460101302 learned_addr learned my addr 172.21.15.173:0/460101302 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:11.907 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.906+0000 7f3d09e00700 1 -- 172.21.15.173:0/460101302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80095f0 con 0x7f3d041070a0 2024-04-20T10:58:11.907 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.906+0000 7f3d09e00700 1 --2- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 0x7f3d0419c320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009ed0 tx=0x7f3cf8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:11.908 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.907+0000 7f3cfea00700 1 -- 172.21.15.173:0/460101302 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cf801d070 con 0x7f3d041070a0 2024-04-20T10:58:11.908 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.907+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0419ca60 con 0x7f3d041070a0 2024-04-20T10:58:11.908 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.907+0000 7f3cfea00700 1 -- 172.21.15.173:0/460101302 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cf800be20 con 0x7f3d041070a0 2024-04-20T10:58:11.908 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.907+0000 7f3cfea00700 1 -- 172.21.15.173:0/460101302 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cf800f460 con 0x7f3d041070a0 2024-04-20T10:58:11.908 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.907+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0419cf00 con 0x7f3d041070a0 2024-04-20T10:58:11.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.908+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d04195c40 con 0x7f3d041070a0 2024-04-20T10:58:11.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.908+0000 7f3cfea00700 1 -- 172.21.15.173:0/460101302 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 14) v1 ==== 45627+0+0 (secure 0 0 0) 0x7f3cf8022800 con 0x7f3d041070a0 2024-04-20T10:58:11.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.909+0000 7f3cfea00700 1 --2- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f3cec038820 0x7f3cec03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:11.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.909+0000 7f3cfea00700 1 -- 172.21.15.173:0/460101302 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3cf804cdd0 con 0x7f3d041070a0 2024-04-20T10:58:11.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.909+0000 7f3d09400700 1 -- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f3cec038820 msgr2=0x7f3cec03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1996106837 2024-04-20T10:58:11.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.909+0000 7f3d09400700 1 --2- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f3cec038820 0x7f3cec03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:58:11.924 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:11.924+0000 7f3cfea00700 1 -- 172.21.15.173:0/460101302 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3cf802bba0 con 0x7f3d041070a0 2024-04-20T10:58:12.111 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.110+0000 7f3d09400700 1 -- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f3cec038820 msgr2=0x7f3cec03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1996106837 2024-04-20T10:58:12.111 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.110+0000 7f3d09400700 1 --2- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f3cec038820 0x7f3cec03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:58:12.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.233+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3d04062510 con 0x7f3d041070a0 2024-04-20T10:58:12.235 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.234+0000 7f3cfea00700 1 -- 172.21.15.173:0/460101302 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3d04062510 con 0x7f3d041070a0 2024-04-20T10:58:12.235 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:12.236 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:12.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.237+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f3cec038820 msgr2=0x7f3cec03ace0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:58:12.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.237+0000 7f3d0c800700 1 --2- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f3cec038820 0x7f3cec03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:12.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.237+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 msgr2=0x7f3d0419c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:12.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.237+0000 7f3d0c800700 1 --2- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 0x7f3d0419c320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009ed0 tx=0x7f3cf8004690 comp rx=0 tx=0).stop 2024-04-20T10:58:12.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.237+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 shutdown_connections 2024-04-20T10:58:12.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.237+0000 7f3d0c800700 1 --2- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:6800/1996106837,v1:172.21.15.117:6801/1996106837] conn(0x7f3cec038820 0x7f3cec03ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:12.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.237+0000 7f3d0c800700 1 --2- 172.21.15.173:0/460101302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d041070a0 0x7f3d0419c320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:12.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.237+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 >> 172.21.15.173:0/460101302 conn(0x7f3d04075f60 msgr2=0x7f3d040769f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:12.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.238+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 shutdown_connections 2024-04-20T10:58:12.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:12.238+0000 7f3d0c800700 1 -- 172.21.15.173:0/460101302 wait complete. 2024-04-20T10:58:12.241 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:12.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:12 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/460101302' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: Active manager daemon smithi117.ivnklj restarted 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: Activating manager daemon smithi117.ivnklj 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: osdmap e5: 0 total, 0 up, 0 in 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: mgrmap e15: smithi117.ivnklj(active, starting, since 0.00790875s) 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:58:13.763 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:58:13.764 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:13 smithi117 ceph-mon[15659]: Manager daemon smithi117.ivnklj is now available 2024-04-20T10:58:13.944 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:13.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:14.315 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:14.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:58:14.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:14.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:58:14.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:58:14.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/trash_purge_schedule"}]: dispatch 2024-04-20T10:58:14.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:14 smithi117 ceph-mon[15659]: mgrmap e16: smithi117.ivnklj(active, since 1.01028s) 2024-04-20T10:58:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:15 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:16.337 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.336+0000 7fcc4a200700 1 -- 172.21.15.173:0/991713256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 msgr2=0x7fcc44100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.337 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.336+0000 7fcc4a200700 1 --2- 172.21.15.173:0/991713256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 0x7fcc44100c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009910 tx=0x7fcc34009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:16.337 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.336+0000 7fcc4a200700 1 -- 172.21.15.173:0/991713256 shutdown_connections 2024-04-20T10:58:16.337 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.336+0000 7fcc4a200700 1 --2- 172.21.15.173:0/991713256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 0x7fcc44100c70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.337 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.336+0000 7fcc4a200700 1 -- 172.21.15.173:0/991713256 >> 172.21.15.173:0/991713256 conn(0x7fcc440fbe30 msgr2=0x7fcc440fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:16.338 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.336+0000 7fcc4a200700 1 -- 172.21.15.173:0/991713256 shutdown_connections 2024-04-20T10:58:16.338 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.336+0000 7fcc4a200700 1 -- 172.21.15.173:0/991713256 wait complete. 2024-04-20T10:58:16.338 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.337+0000 7fcc4a200700 1 Processor -- start 2024-04-20T10:58:16.338 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.337+0000 7fcc4a200700 1 -- start start 2024-04-20T10:58:16.339 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.338+0000 7fcc4a200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 0x7fcc44195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:16.339 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.338+0000 7fcc4a200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44196230 con 0x7fcc44100850 2024-04-20T10:58:16.339 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.338+0000 7fcc43400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 0x7fcc44195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:16.339 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.338+0000 7fcc43400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 0x7fcc44195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:34524/0 (socket says 172.21.15.173:34524) 2024-04-20T10:58:16.339 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.338+0000 7fcc43400700 1 -- 172.21.15.173:0/3846859477 learned_addr learned my addr 172.21.15.173:0/3846859477 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:16.339 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.339+0000 7fcc43400700 1 -- 172.21.15.173:0/3846859477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340095f0 con 0x7fcc44100850 2024-04-20T10:58:16.340 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.339+0000 7fcc43400700 1 --2- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 0x7fcc44195cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc34006010 tx=0x7fcc34009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:16.340 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.340+0000 7fcc3be00700 1 -- 172.21.15.173:0/3846859477 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc3401d070 con 0x7fcc44100850 2024-04-20T10:58:16.340 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.340+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc44196430 con 0x7fcc44100850 2024-04-20T10:58:16.340 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.340+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc441968d0 con 0x7fcc44100850 2024-04-20T10:58:16.340 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.340+0000 7fcc3be00700 1 -- 172.21.15.173:0/3846859477 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc34005dc0 con 0x7fcc44100850 2024-04-20T10:58:16.340 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.340+0000 7fcc3be00700 1 -- 172.21.15.173:0/3846859477 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc34022e70 con 0x7fcc44100850 2024-04-20T10:58:16.341 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.341+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4418f680 con 0x7fcc44100850 2024-04-20T10:58:16.341 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.342+0000 7fcc3be00700 1 -- 172.21.15.173:0/3846859477 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 45671+0+0 (secure 0 0 0) 0x7fcc34022730 con 0x7fcc44100850 2024-04-20T10:58:16.342 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.342+0000 7fcc3be00700 1 --2- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fcc280388b0 0x7fcc2803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:16.342 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.342+0000 7fcc3be00700 1 -- 172.21.15.173:0/3846859477 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fcc3404ca80 con 0x7fcc44100850 2024-04-20T10:58:16.342 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.343+0000 7fcc42a00700 1 --2- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fcc280388b0 0x7fcc2803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:16.343 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.344+0000 7fcc42a00700 1 --2- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fcc280388b0 0x7fcc2803ad70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:16.356 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.357+0000 7fcc3be00700 1 -- 172.21.15.173:0/3846859477 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcc3400b960 con 0x7fcc44100850 2024-04-20T10:58:16.665 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.665+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcc44062510 con 0x7fcc44100850 2024-04-20T10:58:16.665 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.665+0000 7fcc3be00700 1 -- 172.21.15.173:0/3846859477 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fcc34027020 con 0x7fcc44100850 2024-04-20T10:58:16.666 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:16.666 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.669+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fcc280388b0 msgr2=0x7fcc2803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.669+0000 7fcc4a200700 1 --2- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fcc280388b0 0x7fcc2803ad70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30006e40 comp rx=0 tx=0).stop 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.669+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 msgr2=0x7fcc44195cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.669+0000 7fcc4a200700 1 --2- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 0x7fcc44195cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc34006010 tx=0x7fcc34009f80 comp rx=0 tx=0).stop 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.669+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 shutdown_connections 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.669+0000 7fcc4a200700 1 --2- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fcc280388b0 0x7fcc2803ad70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.669+0000 7fcc4a200700 1 --2- 172.21.15.173:0/3846859477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44100850 0x7fcc44195cf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.669+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 >> 172.21.15.173:0/3846859477 conn(0x7fcc440fbe30 msgr2=0x7fcc440fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:16.670 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.670+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 shutdown_connections 2024-04-20T10:58:16.671 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:16.670+0000 7fcc4a200700 1 -- 172.21.15.173:0/3846859477 wait complete. 2024-04-20T10:58:16.671 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:16.769 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:16 smithi117 ceph-mon[15659]: [20/Apr/2024:10:58:15] ENGINE Bus STARTING 2024-04-20T10:58:16.769 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:16 smithi117 ceph-mon[15659]: [20/Apr/2024:10:58:15] ENGINE Serving on https://172.21.15.117:7150 2024-04-20T10:58:16.769 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:16 smithi117 ceph-mon[15659]: [20/Apr/2024:10:58:15] ENGINE Serving on http://172.21.15.117:8765 2024-04-20T10:58:16.769 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:16 smithi117 ceph-mon[15659]: [20/Apr/2024:10:58:15] ENGINE Bus STARTED 2024-04-20T10:58:16.769 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:16 smithi117 ceph-mon[15659]: mgrmap e17: smithi117.ivnklj(active, since 2s) 2024-04-20T10:58:16.770 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:16 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:17.851 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:17 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/3846859477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:17.851 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:17 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:18.486 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:18.486 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:19.037 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:19.136 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-20T10:58:21.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.642+0000 7fc5d1400700 1 -- 172.21.15.173:0/487807772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 msgr2=0x7fc5cc100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.642+0000 7fc5d1400700 1 --2- 172.21.15.173:0/487807772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 0x7fc5cc100c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009910 tx=0x7fc5bc009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.642+0000 7fc5d1400700 1 -- 172.21.15.173:0/487807772 shutdown_connections 2024-04-20T10:58:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.642+0000 7fc5d1400700 1 --2- 172.21.15.173:0/487807772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 0x7fc5cc100c70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.642+0000 7fc5d1400700 1 -- 172.21.15.173:0/487807772 >> 172.21.15.173:0/487807772 conn(0x7fc5cc0fbe30 msgr2=0x7fc5cc0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.642+0000 7fc5d1400700 1 -- 172.21.15.173:0/487807772 shutdown_connections 2024-04-20T10:58:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.643+0000 7fc5d1400700 1 -- 172.21.15.173:0/487807772 wait complete. 2024-04-20T10:58:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.643+0000 7fc5d1400700 1 Processor -- start 2024-04-20T10:58:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.644+0000 7fc5d1400700 1 -- start start 2024-04-20T10:58:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.644+0000 7fc5d1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 0x7fc5cc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.644+0000 7fc5d1400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc198460 con 0x7fc5cc100850 2024-04-20T10:58:21.645 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.644+0000 7fc5ca800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 0x7fc5cc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:21.645 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.644+0000 7fc5ca800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 0x7fc5cc197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:34544/0 (socket says 172.21.15.173:34544) 2024-04-20T10:58:21.645 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.644+0000 7fc5ca800700 1 -- 172.21.15.173:0/1950843135 learned_addr learned my addr 172.21.15.173:0/1950843135 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:21.645 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.645+0000 7fc5ca800700 1 -- 172.21.15.173:0/1950843135 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0095f0 con 0x7fc5cc100850 2024-04-20T10:58:21.645 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.645+0000 7fc5ca800700 1 --2- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 0x7fc5cc197f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc006010 tx=0x7fc5bc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:21.645 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.645+0000 7fc5c3400700 1 -- 172.21.15.173:0/1950843135 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5bc01d070 con 0x7fc5cc100850 2024-04-20T10:58:21.646 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.645+0000 7fc5c3400700 1 -- 172.21.15.173:0/1950843135 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5bc005dc0 con 0x7fc5cc100850 2024-04-20T10:58:21.646 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.645+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc198660 con 0x7fc5cc100850 2024-04-20T10:58:21.646 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.646+0000 7fc5c3400700 1 -- 172.21.15.173:0/1950843135 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5bc022e90 con 0x7fc5cc100850 2024-04-20T10:58:21.646 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.646+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc198b00 con 0x7fc5cc100850 2024-04-20T10:58:21.646 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.647+0000 7fc5c3400700 1 -- 172.21.15.173:0/1950843135 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 45671+0+0 (secure 0 0 0) 0x7fc5bc00b960 con 0x7fc5cc100850 2024-04-20T10:58:21.647 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.647+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc191a80 con 0x7fc5cc100850 2024-04-20T10:58:21.647 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.647+0000 7fc5c3400700 1 --2- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc5b0038850 0x7fc5b003ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:21.647 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.647+0000 7fc5c3400700 1 -- 172.21.15.173:0/1950843135 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc5bc04cac0 con 0x7fc5cc100850 2024-04-20T10:58:21.647 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.648+0000 7fc5c9e00700 1 --2- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc5b0038850 0x7fc5b003ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:21.648 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.648+0000 7fc5c9e00700 1 --2- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc5b0038850 0x7fc5b003ad10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b8006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:21.655 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.656+0000 7fc5c3400700 1 -- 172.21.15.173:0/1950843135 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc5bc02a540 con 0x7fc5cc100850 2024-04-20T10:58:21.978 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.978+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc5cc062510 con 0x7fc5cc100850 2024-04-20T10:58:21.980 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.979+0000 7fc5c3400700 1 -- 172.21.15.173:0/1950843135 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fc5cc062510 con 0x7fc5cc100850 2024-04-20T10:58:21.980 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:21.980 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:21.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc5b0038850 msgr2=0x7fc5b003ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 --2- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc5b0038850 0x7fc5b003ad10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b8006e40 comp rx=0 tx=0).stop 2024-04-20T10:58:21.984 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 msgr2=0x7fc5cc197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.984 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 --2- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 0x7fc5cc197f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc006010 tx=0x7fc5bc009f80 comp rx=0 tx=0).stop 2024-04-20T10:58:21.984 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 shutdown_connections 2024-04-20T10:58:21.984 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 --2- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc5b0038850 0x7fc5b003ad10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.984 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 --2- 172.21.15.173:0/1950843135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc100850 0x7fc5cc197f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.985 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 >> 172.21.15.173:0/1950843135 conn(0x7fc5cc0fbe30 msgr2=0x7fc5cc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:21.985 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 shutdown_connections 2024-04-20T10:58:21.985 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:21.983+0000 7fc5d1400700 1 -- 172.21.15.173:0/1950843135 wait complete. 2024-04-20T10:58:21.985 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/1950843135' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:58:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:23 smithi117 ceph-mon[15659]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T10:58:23.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:23 smithi117 ceph-mon[15659]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T10:58:23.813 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:23.814 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:24.164 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:24 smithi117 ceph-mon[15659]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:24 smithi117 ceph-mon[15659]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:24 smithi117 ceph-mon[15659]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:58:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:58:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-20T10:58:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:25 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:26.133 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.131+0000 7f0d6d400700 1 -- 172.21.15.173:0/3758023441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 msgr2=0x7f0d68108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.133 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.131+0000 7f0d6d400700 1 --2- 172.21.15.173:0/3758023441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 0x7f0d68108de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009910 tx=0x7f0d58009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:26.133 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.131+0000 7f0d6d400700 1 -- 172.21.15.173:0/3758023441 shutdown_connections 2024-04-20T10:58:26.133 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.131+0000 7f0d6d400700 1 --2- 172.21.15.173:0/3758023441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 0x7f0d68108de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.133 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.131+0000 7f0d6d400700 1 -- 172.21.15.173:0/3758023441 >> 172.21.15.173:0/3758023441 conn(0x7f0d680fe4f0 msgr2=0x7f0d68100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:26.133 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.131+0000 7f0d6d400700 1 -- 172.21.15.173:0/3758023441 shutdown_connections 2024-04-20T10:58:26.133 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.131+0000 7f0d6d400700 1 -- 172.21.15.173:0/3758023441 wait complete. 2024-04-20T10:58:26.133 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.132+0000 7f0d6d400700 1 Processor -- start 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.132+0000 7f0d6d400700 1 -- start start 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.132+0000 7f0d6d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 0x7f0d681982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.132+0000 7f0d6d400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68198830 con 0x7f0d68108a00 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.133+0000 7f0d66800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 0x7f0d681982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.133+0000 7f0d66800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 0x7f0d681982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:39026/0 (socket says 172.21.15.173:39026) 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.133+0000 7f0d66800700 1 -- 172.21.15.173:0/133945969 learned_addr learned my addr 172.21.15.173:0/133945969 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.134+0000 7f0d66800700 1 -- 172.21.15.173:0/133945969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d580095f0 con 0x7f0d68108a00 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.134+0000 7f0d66800700 1 --2- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 0x7f0d681982f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0d58005490 tx=0x7f0d58005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.135+0000 7f0d5fe00700 1 -- 172.21.15.173:0/133945969 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d5801d070 con 0x7f0d68108a00 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.135+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d68198a90 con 0x7f0d68108a00 2024-04-20T10:58:26.135 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.135+0000 7f0d5fe00700 1 -- 172.21.15.173:0/133945969 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d58005700 con 0x7f0d68108a00 2024-04-20T10:58:26.136 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.135+0000 7f0d5fe00700 1 -- 172.21.15.173:0/133945969 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d580038e0 con 0x7f0d68108a00 2024-04-20T10:58:26.136 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.135+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d68198f10 con 0x7f0d68108a00 2024-04-20T10:58:26.136 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.136+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d68104140 con 0x7f0d68108a00 2024-04-20T10:58:26.136 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.136+0000 7f0d5fe00700 1 -- 172.21.15.173:0/133945969 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 45671+0+0 (secure 0 0 0) 0x7f0d58003b50 con 0x7f0d68108a00 2024-04-20T10:58:26.137 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.137+0000 7f0d5fe00700 1 --2- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f0d48038910 0x7f0d4803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:26.137 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.137+0000 7f0d5fe00700 1 -- 172.21.15.173:0/133945969 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f0d5804c630 con 0x7f0d68108a00 2024-04-20T10:58:26.137 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.138+0000 7f0d5de00700 1 --2- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f0d48038910 0x7f0d4803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:26.138 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.139+0000 7f0d5de00700 1 --2- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f0d48038910 0x7f0d4803add0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006fd0 tx=0x7f0d50006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:26.153 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.153+0000 7f0d5fe00700 1 -- 172.21.15.173:0/133945969 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0d5802a590 con 0x7f0d68108a00 2024-04-20T10:58:26.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.468+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0d6810cdf0 con 0x7f0d68108a00 2024-04-20T10:58:26.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.469+0000 7f0d5fe00700 1 -- 172.21.15.173:0/133945969 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f0d58026070 con 0x7f0d68108a00 2024-04-20T10:58:26.470 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:26.470 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.472+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f0d48038910 msgr2=0x7f0d4803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.472+0000 7f0d6d400700 1 --2- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f0d48038910 0x7f0d4803add0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006fd0 tx=0x7f0d50006e40 comp rx=0 tx=0).stop 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.472+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 msgr2=0x7f0d681982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.472+0000 7f0d6d400700 1 --2- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 0x7f0d681982f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0d58005490 tx=0x7f0d58005570 comp rx=0 tx=0).stop 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.472+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 shutdown_connections 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.472+0000 7f0d6d400700 1 --2- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f0d48038910 0x7f0d4803add0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.472+0000 7f0d6d400700 1 --2- 172.21.15.173:0/133945969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d68108a00 0x7f0d681982f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.472+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 >> 172.21.15.173:0/133945969 conn(0x7f0d680fe4f0 msgr2=0x7f0d6810a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.473+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 shutdown_connections 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:26.473+0000 7f0d6d400700 1 -- 172.21.15.173:0/133945969 wait complete. 2024-04-20T10:58:26.474 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:26.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:26 smithi117 ceph-mon[15659]: Deploying daemon ceph-exporter.smithi173 on smithi173 2024-04-20T10:58:27.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:27 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/133945969' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:28.716 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:28.716 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:29.066 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:29.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:58:31.506 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.505+0000 7fb002400700 1 -- 172.21.15.173:0/3457668402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 msgr2=0x7faffc106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.506 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.505+0000 7fb002400700 1 --2- 172.21.15.173:0/3457668402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 0x7faffc106bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafec009910 tx=0x7fafec009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:31.506 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.505+0000 7fb002400700 1 -- 172.21.15.173:0/3457668402 shutdown_connections 2024-04-20T10:58:31.506 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.505+0000 7fb002400700 1 --2- 172.21.15.173:0/3457668402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 0x7faffc106bd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.506 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.505+0000 7fb002400700 1 -- 172.21.15.173:0/3457668402 >> 172.21.15.173:0/3457668402 conn(0x7faffc0fc300 msgr2=0x7faffc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:31.506 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.505+0000 7fb002400700 1 -- 172.21.15.173:0/3457668402 shutdown_connections 2024-04-20T10:58:31.506 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.505+0000 7fb002400700 1 -- 172.21.15.173:0/3457668402 wait complete. 2024-04-20T10:58:31.507 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.507+0000 7fb002400700 1 Processor -- start 2024-04-20T10:58:31.507 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.508+0000 7fb002400700 1 -- start start 2024-04-20T10:58:31.508 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.508+0000 7fb002400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 0x7faffc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:31.508 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.508+0000 7fb002400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc1987f0 con 0x7faffc1067f0 2024-04-20T10:58:31.508 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.509+0000 7faffb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 0x7faffc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:31.508 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.509+0000 7faffb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 0x7faffc1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:39062/0 (socket says 172.21.15.173:39062) 2024-04-20T10:58:31.509 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.509+0000 7faffb400700 1 -- 172.21.15.173:0/3913978053 learned_addr learned my addr 172.21.15.173:0/3913978053 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:31.509 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.510+0000 7faffb400700 1 -- 172.21.15.173:0/3913978053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec0095f0 con 0x7faffc1067f0 2024-04-20T10:58:31.509 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.510+0000 7faffb400700 1 --2- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 0x7faffc1982b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fafec005490 tx=0x7fafec005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:31.510 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.510+0000 7faff3e00700 1 -- 172.21.15.173:0/3913978053 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fafec01d070 con 0x7faffc1067f0 2024-04-20T10:58:31.510 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.510+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc198a50 con 0x7faffc1067f0 2024-04-20T10:58:31.510 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.511+0000 7faff3e00700 1 -- 172.21.15.173:0/3913978053 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafec005700 con 0x7faffc1067f0 2024-04-20T10:58:31.510 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.511+0000 7faff3e00700 1 -- 172.21.15.173:0/3913978053 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fafec0038e0 con 0x7faffc1067f0 2024-04-20T10:58:31.511 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.511+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc198ed0 con 0x7faffc1067f0 2024-04-20T10:58:31.512 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.512+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc101f30 con 0x7faffc1067f0 2024-04-20T10:58:31.512 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.512+0000 7faff3e00700 1 -- 172.21.15.173:0/3913978053 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 45671+0+0 (secure 0 0 0) 0x7fafec003ba0 con 0x7faffc1067f0 2024-04-20T10:58:31.512 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.513+0000 7faff3e00700 1 --2- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fafe00388c0 0x7fafe003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:31.512 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.513+0000 7faff3e00700 1 -- 172.21.15.173:0/3913978053 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fafec04c740 con 0x7faffc1067f0 2024-04-20T10:58:31.513 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.514+0000 7faffaa00700 1 --2- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fafe00388c0 0x7fafe003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:31.514 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.515+0000 7faffaa00700 1 --2- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fafe00388c0 0x7fafe003ad80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fafe8006fd0 tx=0x7fafe8006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:31.530 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.531+0000 7faff3e00700 1 -- 172.21.15.173:0/3913978053 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fafec02aa00 con 0x7faffc1067f0 2024-04-20T10:58:31.867 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.866+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faffc10abe0 con 0x7faffc1067f0 2024-04-20T10:58:31.867 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.868+0000 7faff3e00700 1 -- 172.21.15.173:0/3913978053 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fafec026020 con 0x7faffc1067f0 2024-04-20T10:58:31.867 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:31.868 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:31.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.870+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fafe00388c0 msgr2=0x7fafe003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.870+0000 7fb002400700 1 --2- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fafe00388c0 0x7fafe003ad80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fafe8006fd0 tx=0x7fafe8006e40 comp rx=0 tx=0).stop 2024-04-20T10:58:31.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.870+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 msgr2=0x7faffc1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.870+0000 7fb002400700 1 --2- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 0x7faffc1982b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fafec005490 tx=0x7fafec005570 comp rx=0 tx=0).stop 2024-04-20T10:58:31.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.870+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 shutdown_connections 2024-04-20T10:58:31.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.870+0000 7fb002400700 1 --2- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fafe00388c0 0x7fafe003ad80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.870+0000 7fb002400700 1 --2- 172.21.15.173:0/3913978053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc1067f0 0x7faffc1982b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.870+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 >> 172.21.15.173:0/3913978053 conn(0x7faffc0fc300 msgr2=0x7faffc074bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:31.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.871+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 shutdown_connections 2024-04-20T10:58:31.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:31.871+0000 7fb002400700 1 -- 172.21.15.173:0/3913978053 wait complete. 2024-04-20T10:58:31.871 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:32.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:32.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T10:58:32.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-20T10:58:32.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:32.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: Deploying daemon crash.smithi173 on smithi173 2024-04-20T10:58:32.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:32 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/3913978053' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:33.754 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:33.754 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:34.274 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:34.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:34 smithi117 ceph-mon[15659]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:34.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:36.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.382+0000 7fd39be00700 1 -- 172.21.15.173:0/306428175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 msgr2=0x7fd394106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:36.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.382+0000 7fd39be00700 1 --2- 172.21.15.173:0/306428175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 0x7fd394106bd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd388009910 tx=0x7fd388009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:36.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.382+0000 7fd39be00700 1 -- 172.21.15.173:0/306428175 shutdown_connections 2024-04-20T10:58:36.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.382+0000 7fd39be00700 1 --2- 172.21.15.173:0/306428175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 0x7fd394106bd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:36.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.382+0000 7fd39be00700 1 -- 172.21.15.173:0/306428175 >> 172.21.15.173:0/306428175 conn(0x7fd3940fc300 msgr2=0x7fd3940fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:36.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.382+0000 7fd39be00700 1 -- 172.21.15.173:0/306428175 shutdown_connections 2024-04-20T10:58:36.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.382+0000 7fd39be00700 1 -- 172.21.15.173:0/306428175 wait complete. 2024-04-20T10:58:36.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.383+0000 7fd39be00700 1 Processor -- start 2024-04-20T10:58:36.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.384+0000 7fd39be00700 1 -- start start 2024-04-20T10:58:36.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.384+0000 7fd39be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 0x7fd3940731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:36.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.384+0000 7fd39be00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940750a0 con 0x7fd3941067f0 2024-04-20T10:58:36.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.385+0000 7fd399400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 0x7fd3940731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:36.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.385+0000 7fd399400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 0x7fd3940731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:43178/0 (socket says 172.21.15.173:43178) 2024-04-20T10:58:36.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.385+0000 7fd399400700 1 -- 172.21.15.173:0/2887201163 learned_addr learned my addr 172.21.15.173:0/2887201163 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:36.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.385+0000 7fd399400700 1 -- 172.21.15.173:0/2887201163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880095f0 con 0x7fd3941067f0 2024-04-20T10:58:36.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.386+0000 7fd399400700 1 --2- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 0x7fd3940731e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd388005490 tx=0x7fd388005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:36.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.386+0000 7fd392000700 1 -- 172.21.15.173:0/2887201163 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd38801d070 con 0x7fd3941067f0 2024-04-20T10:58:36.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.386+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd394073720 con 0x7fd3941067f0 2024-04-20T10:58:36.387 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.386+0000 7fd392000700 1 -- 172.21.15.173:0/2887201163 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd388005700 con 0x7fd3941067f0 2024-04-20T10:58:36.387 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.386+0000 7fd392000700 1 -- 172.21.15.173:0/2887201163 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3880038e0 con 0x7fd3941067f0 2024-04-20T10:58:36.387 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.386+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd394073ba0 con 0x7fd3941067f0 2024-04-20T10:58:36.387 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.388+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394101f30 con 0x7fd3941067f0 2024-04-20T10:58:36.388 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.388+0000 7fd392000700 1 -- 172.21.15.173:0/2887201163 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 45671+0+0 (secure 0 0 0) 0x7fd388003ba0 con 0x7fd3941067f0 2024-04-20T10:58:36.388 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.389+0000 7fd392000700 1 --2- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd37c0388c0 0x7fd37c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:36.388 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.389+0000 7fd392000700 1 -- 172.21.15.173:0/2887201163 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd38804c6d0 con 0x7fd3941067f0 2024-04-20T10:58:36.389 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.389+0000 7fd398a00700 1 --2- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd37c0388c0 0x7fd37c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:36.390 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.390+0000 7fd398a00700 1 --2- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd37c0388c0 0x7fd37c03ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd380006fd0 tx=0x7fd380006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:36.403 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.404+0000 7fd392000700 1 -- 172.21.15.173:0/2887201163 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd38802a9a0 con 0x7fd3941067f0 2024-04-20T10:58:36.728 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.728+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd394073de0 con 0x7fd3941067f0 2024-04-20T10:58:36.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.730+0000 7fd392000700 1 -- 172.21.15.173:0/2887201163 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd388026070 con 0x7fd3941067f0 2024-04-20T10:58:36.729 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:36.730 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:36.732 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.732+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd37c0388c0 msgr2=0x7fd37c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.732+0000 7fd39be00700 1 --2- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd37c0388c0 0x7fd37c03ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd380006fd0 tx=0x7fd380006e40 comp rx=0 tx=0).stop 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.732+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 msgr2=0x7fd3940731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.732+0000 7fd39be00700 1 --2- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 0x7fd3940731e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd388005490 tx=0x7fd388005570 comp rx=0 tx=0).stop 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.733+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 shutdown_connections 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.733+0000 7fd39be00700 1 --2- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd37c0388c0 0x7fd37c03ad80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.733+0000 7fd39be00700 1 --2- 172.21.15.173:0/2887201163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3941067f0 0x7fd3940731e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.733+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 >> 172.21.15.173:0/2887201163 conn(0x7fd3940fc300 msgr2=0x7fd3940fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.733+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 shutdown_connections 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:36.733+0000 7fd39be00700 1 -- 172.21.15.173:0/2887201163 wait complete. 2024-04-20T10:58:36.733 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:36 smithi117 ceph-mon[15659]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:37 smithi117 ceph-mon[15659]: Deploying daemon node-exporter.smithi173 on smithi173 2024-04-20T10:58:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:37 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/2887201163' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:38.669 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:38.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:39.092 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:39.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:38 smithi117 ceph-mon[15659]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:39.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:41.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:40 smithi117 ceph-mon[15659]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:42.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.755+0000 7fb568200700 1 -- 172.21.15.173:0/632467298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 msgr2=0x7fb560101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:42.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.755+0000 7fb568200700 1 --2- 172.21.15.173:0/632467298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 0x7fb560101da0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb554009910 tx=0x7fb554009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:42.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.755+0000 7fb568200700 1 -- 172.21.15.173:0/632467298 shutdown_connections 2024-04-20T10:58:42.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.755+0000 7fb568200700 1 --2- 172.21.15.173:0/632467298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 0x7fb560101da0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:42.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.755+0000 7fb568200700 1 -- 172.21.15.173:0/632467298 >> 172.21.15.173:0/632467298 conn(0x7fb560076390 msgr2=0x7fb5600fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:42.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.756+0000 7fb568200700 1 -- 172.21.15.173:0/632467298 shutdown_connections 2024-04-20T10:58:42.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.756+0000 7fb568200700 1 -- 172.21.15.173:0/632467298 wait complete. 2024-04-20T10:58:42.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.757+0000 7fb568200700 1 Processor -- start 2024-04-20T10:58:42.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.758+0000 7fb568200700 1 -- start start 2024-04-20T10:58:42.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.758+0000 7fb568200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 0x7fb560196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:42.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.758+0000 7fb568200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560196750 con 0x7fb5601019c0 2024-04-20T10:58:42.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.759+0000 7fb565800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 0x7fb560196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:42.760 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.759+0000 7fb565800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 0x7fb560196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:58354/0 (socket says 172.21.15.173:58354) 2024-04-20T10:58:42.760 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.759+0000 7fb565800700 1 -- 172.21.15.173:0/4090178076 learned_addr learned my addr 172.21.15.173:0/4090178076 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:42.760 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.759+0000 7fb565800700 1 -- 172.21.15.173:0/4090178076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540095f0 con 0x7fb5601019c0 2024-04-20T10:58:42.760 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.760+0000 7fb565800700 1 --2- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 0x7fb560196210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5540098e0 tx=0x7fb554005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:42.761 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.760+0000 7fb55e000700 1 -- 172.21.15.173:0/4090178076 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55401d070 con 0x7fb5601019c0 2024-04-20T10:58:42.761 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.760+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb560196950 con 0x7fb5601019c0 2024-04-20T10:58:42.761 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.761+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb560196d70 con 0x7fb5601019c0 2024-04-20T10:58:42.762 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.761+0000 7fb55e000700 1 -- 172.21.15.173:0/4090178076 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5540056b0 con 0x7fb5601019c0 2024-04-20T10:58:42.762 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.761+0000 7fb55e000700 1 -- 172.21.15.173:0/4090178076 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55400fb00 con 0x7fb5601019c0 2024-04-20T10:58:42.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.762+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5400052f0 con 0x7fb5601019c0 2024-04-20T10:58:42.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.763+0000 7fb55e000700 1 -- 172.21.15.173:0/4090178076 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 45671+0+0 (secure 0 0 0) 0x7fb55400f460 con 0x7fb5601019c0 2024-04-20T10:58:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.763+0000 7fb55e000700 1 --2- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb5480388b0 0x7fb54803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.763+0000 7fb55e000700 1 -- 172.21.15.173:0/4090178076 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb55404c960 con 0x7fb5601019c0 2024-04-20T10:58:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.764+0000 7fb564e00700 1 --2- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb5480388b0 0x7fb54803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:42.765 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.765+0000 7fb564e00700 1 --2- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb5480388b0 0x7fb54803ad70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006fd0 tx=0x7fb54c006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:42.780 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:42.779+0000 7fb55e000700 1 -- 172.21.15.173:0/4090178076 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb55400be80 con 0x7fb5601019c0 2024-04-20T10:58:43.095 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.094+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb540005160 con 0x7fb5601019c0 2024-04-20T10:58:43.096 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.095+0000 7fb55e000700 1 -- 172.21.15.173:0/4090178076 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fb554027070 con 0x7fb5601019c0 2024-04-20T10:58:43.096 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:43.096 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:43.099 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb5480388b0 msgr2=0x7fb54803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:43.099 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 --2- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb5480388b0 0x7fb54803ad70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006fd0 tx=0x7fb54c006e40 comp rx=0 tx=0).stop 2024-04-20T10:58:43.099 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 msgr2=0x7fb560196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:43.099 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 --2- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 0x7fb560196210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5540098e0 tx=0x7fb554005520 comp rx=0 tx=0).stop 2024-04-20T10:58:43.099 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 shutdown_connections 2024-04-20T10:58:43.100 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 --2- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb5480388b0 0x7fb54803ad70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.100 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 --2- 172.21.15.173:0/4090178076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5601019c0 0x7fb560196210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.100 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 >> 172.21.15.173:0/4090178076 conn(0x7fb560076390 msgr2=0x7fb560102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:43.100 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 shutdown_connections 2024-04-20T10:58:43.100 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:43.098+0000 7fb568200700 1 -- 172.21.15.173:0/4090178076 wait complete. 2024-04-20T10:58:43.100 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:58:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T10:58:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:58:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:43 smithi117 ceph-mon[15659]: Deploying daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T10:58:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:43 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/4090178076' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:58:44.886 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:44.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:44 smithi117 ceph-mon[15659]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:45.386 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:46 smithi117 ceph-mon[15659]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:47.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.429+0000 7f7b45c00700 1 -- 172.21.15.173:0/3117679201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 msgr2=0x7f7b40101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:47.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.429+0000 7f7b45c00700 1 --2- 172.21.15.173:0/3117679201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 0x7f7b40101e40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009910 tx=0x7f7b30009c20 comp rx=0 tx=0).stop 2024-04-20T10:58:47.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.429+0000 7f7b45c00700 1 -- 172.21.15.173:0/3117679201 shutdown_connections 2024-04-20T10:58:47.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.429+0000 7f7b45c00700 1 --2- 172.21.15.173:0/3117679201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 0x7f7b40101e40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:47.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.429+0000 7f7b45c00700 1 -- 172.21.15.173:0/3117679201 >> 172.21.15.173:0/3117679201 conn(0x7f7b40076390 msgr2=0x7f7b400fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:47.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.429+0000 7f7b45c00700 1 -- 172.21.15.173:0/3117679201 shutdown_connections 2024-04-20T10:58:47.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.429+0000 7f7b45c00700 1 -- 172.21.15.173:0/3117679201 wait complete. 2024-04-20T10:58:47.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.431+0000 7f7b45c00700 1 Processor -- start 2024-04-20T10:58:47.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.431+0000 7f7b45c00700 1 -- start start 2024-04-20T10:58:47.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.431+0000 7f7b45c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 0x7f7b401982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:47.433 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.431+0000 7f7b45c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b40198800 con 0x7f7b40101a60 2024-04-20T10:58:47.433 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.432+0000 7f7b3ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 0x7f7b401982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:47.433 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.432+0000 7f7b3ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 0x7f7b401982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.173:58396/0 (socket says 172.21.15.173:58396) 2024-04-20T10:58:47.433 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.432+0000 7f7b3ea00700 1 -- 172.21.15.173:0/3989048848 learned_addr learned my addr 172.21.15.173:0/3989048848 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:47.433 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.432+0000 7f7b3ea00700 1 -- 172.21.15.173:0/3989048848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300095f0 con 0x7f7b40101a60 2024-04-20T10:58:47.433 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.433+0000 7f7b3ea00700 1 --2- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 0x7f7b401982c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b30006010 tx=0x7f7b3000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:47.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.433+0000 7f7b37400700 1 -- 172.21.15.173:0/3989048848 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b30005700 con 0x7f7b40101a60 2024-04-20T10:58:47.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.433+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b40198a00 con 0x7f7b40101a60 2024-04-20T10:58:47.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.433+0000 7f7b37400700 1 -- 172.21.15.173:0/3989048848 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b30005dc0 con 0x7f7b40101a60 2024-04-20T10:58:47.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.433+0000 7f7b37400700 1 -- 172.21.15.173:0/3989048848 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b3000fc70 con 0x7f7b40101a60 2024-04-20T10:58:47.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.433+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b40198e20 con 0x7f7b40101a60 2024-04-20T10:58:47.435 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.435+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b40107c30 con 0x7f7b40101a60 2024-04-20T10:58:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.435+0000 7f7b37400700 1 -- 172.21.15.173:0/3989048848 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 45671+0+0 (secure 0 0 0) 0x7f7b3000f4f0 con 0x7f7b40101a60 2024-04-20T10:58:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.435+0000 7f7b37400700 1 --2- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b24038910 0x7f7b2403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.436+0000 7f7b37400700 1 -- 172.21.15.173:0/3989048848 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7b3004c8d0 con 0x7f7b40101a60 2024-04-20T10:58:47.437 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.436+0000 7f7b37e00700 1 --2- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b24038910 0x7f7b2403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:47.438 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.437+0000 7f7b37e00700 1 --2- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b24038910 0x7f7b2403add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7b28006fd0 tx=0x7f7b28006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:47.453 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.452+0000 7f7b37400700 1 -- 172.21.15.173:0/3989048848 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7b3002a360 con 0x7f7b40101a60 2024-04-20T10:58:47.762 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.761+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7b40068820 con 0x7f7b40101a60 2024-04-20T10:58:47.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.762+0000 7f7b37400700 1 -- 172.21.15.173:0/3989048848 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f7b30027070 con 0x7f7b40101a60 2024-04-20T10:58:47.763 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:58:47.763 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:55:27.473649Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:58:47.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b24038910 msgr2=0x7f7b2403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:47.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 --2- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b24038910 0x7f7b2403add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7b28006fd0 tx=0x7f7b28006e40 comp rx=0 tx=0).stop 2024-04-20T10:58:47.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 msgr2=0x7f7b401982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:47.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 --2- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 0x7f7b401982c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b30006010 tx=0x7f7b3000bda0 comp rx=0 tx=0).stop 2024-04-20T10:58:47.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 shutdown_connections 2024-04-20T10:58:47.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 --2- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b24038910 0x7f7b2403add0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:47.767 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 --2- 172.21.15.173:0/3989048848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b40101a60 0x7f7b401982c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:47.767 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 >> 172.21.15.173:0/3989048848 conn(0x7f7b40076390 msgr2=0x7f7b40106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:47.767 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 shutdown_connections 2024-04-20T10:58:47.767 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:47.765+0000 7f7b45c00700 1 -- 172.21.15.173:0/3989048848 wait complete. 2024-04-20T10:58:47.767 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: Deploying daemon mon.smithi173 on smithi173 2024-04-20T10:58:48.837 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:48 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/3989048848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:49.789 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-20T10:58:49.789 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:58:49.801 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:49 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:51.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:50 smithi117 ceph-mon[15659]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:51.588 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T10:58:52.393 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:52 smithi173 systemd[1]: Starting Ceph mon.smithi173 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T10:58:53.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:52 smithi117 ceph-mon[15659]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.517 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 podman[23761]: 2024-04-20 10:58:53.090952184 +0000 UTC m=+0.040358724 image pull 9900ed1a6b81aaf043a19454fc819f6cb21ac0421c7c720b31d8fae6faebb644 quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 2024-04-20T10:58:53.517 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 podman[23761]: 2024-04-20 10:58:53.343881369 +0000 UTC m=+0.293287915 container create 243eed744f9205671aa5b19bc4745a06cf34b478718b166c32e09fecb3e588b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, RELEASE=reef-646b8c7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:58:53.886 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 podman[23761]: 2024-04-20 10:58:53.675479696 +0000 UTC m=+0.624886245 container init 243eed744f9205671aa5b19bc4745a06cf34b478718b166c32e09fecb3e588b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=reef-646b8c7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:58:53.886 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 podman[23761]: 2024-04-20 10:58:53.684931877 +0000 UTC m=+0.634338426 container start 243eed744f9205671aa5b19bc4745a06cf34b478718b166c32e09fecb3e588b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=reef-646b8c7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-20T10:58:53.886 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T10:58:53.886 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable), process ceph-mon, pid 2 2024-04-20T10:58:53.886 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pidfile_write: ignore empty --pid-file 2024-04-20T10:58:53.886 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: load: jerasure load: lrc 2024-04-20T10:58:53.886 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: RocksDB version: 7.9.2 2024-04-20T10:58:53.886 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Git sha 0 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Compile date 2024-04-18 11:37:30 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: DB SUMMARY 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: DB Session ID: Q0LKF7HX55XLTBKAHPA0 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: CURRENT file: CURRENT 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi173/store.db dir, Total Num: 0, files: 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi173/store.db: 000004.log size: 511 ; 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.error_if_exists: 0 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.create_if_missing: 0 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.paranoid_checks: 1 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T10:58:53.887 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T10:58:53.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.env: 0x56153c55d820 2024-04-20T10:58:53.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T10:58:53.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.info_log: 0x5615463d3420 2024-04-20T10:58:53.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T10:58:53.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.statistics: (nil) 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.use_fsync: 0 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_log_file_size: 0 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.allow_fallocate: 1 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T10:58:53.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.use_direct_reads: 0 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.db_log_dir: 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.wal_dir: 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T10:58:53.890 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.write_buffer_manager: 0x561545648140 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.unordered_write: 0 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T10:58:53.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.row_cache: None 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.wal_filter: None 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.two_write_queues: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.wal_compression: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.atomic_flush: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.log_readahead_size: 0 2024-04-20T10:58:53.892 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_background_jobs: 2 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_background_compactions: -1 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_subcompactions: 1 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T10:58:53.893 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T10:58:53.894 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T10:58:53.894 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T10:58:53.894 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T10:58:53.894 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_open_files: -1 2024-04-20T10:58:53.894 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T10:58:53.894 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T10:58:53.894 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_background_flushes: -1 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Compression algorithms supported: 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: kZSTD supported: 0 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: kXpressCompression supported: 0 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: kZlibCompression supported: 1 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: kSnappyCompression supported: 1 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: kLZ4Compression supported: 1 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: kBZip2Compression supported: 0 2024-04-20T10:58:53.895 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi173/store.db/MANIFEST-000005 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.merge_operator: 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_filter: None 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5615463d3520) 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: cache_index_and_filter_blocks: 1 2024-04-20T10:58:53.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: pin_top_level_index_and_filter: 1 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: index_type: 0 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: data_block_index_type: 0 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: index_shortening: 1 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: checksum: 4 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: no_block_cache: 0 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_cache: 0x5615456e91f0 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_cache_name: BinnedLRUCache 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_cache_options: 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: capacity : 536870912 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: num_shard_bits : 4 2024-04-20T10:58:53.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: strict_capacity_limit : 0 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: high_pri_pool_ratio: 0.000 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_cache_compressed: (nil) 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: persistent_cache: (nil) 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_size: 4096 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_size_deviation: 10 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_restart_interval: 16 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: index_block_restart_interval: 1 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: metadata_block_size: 4096 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: partition_filters: 0 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: use_delta_encoding: 1 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: filter_policy: bloomfilter 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: whole_key_filtering: 1 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: verify_compression: 0 2024-04-20T10:58:53.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: read_amp_bytes_per_bit: 0 2024-04-20T10:58:53.899 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: format_version: 5 2024-04-20T10:58:53.899 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: enable_index_compression: 1 2024-04-20T10:58:53.899 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_align: 0 2024-04-20T10:58:53.899 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: max_auto_readahead_size: 262144 2024-04-20T10:58:53.899 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: prepopulate_block_cache: 0 2024-04-20T10:58:53.899 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: initial_auto_readahead_size: 8192 2024-04-20T10:58:53.899 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T10:58:53.900 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T10:58:53.900 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression: NoCompression 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.num_levels: 7 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T10:58:53.901 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T10:58:53.902 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T10:58:53.903 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T10:58:53.904 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.table_properties_collectors: 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.inplace_update_support: 0 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.bloom_locality: 0 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.max_successive_merges: 0 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T10:58:53.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.ttl: 2592000 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.enable_blob_files: false 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.min_blob_size: 0 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T10:58:53.906 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi173/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8051f2e2-8a7a-4439-ac38-628c50624f53 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713610733739602, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713610733740397, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713610733, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8051f2e2-8a7a-4439-ac38-628c50624f53", "db_session_id": "Q0LKF7HX55XLTBKAHPA0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713610733740635, "job": 1, "event": "recovery_finished"} 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi173/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561545788000 2024-04-20T10:58:53.907 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: DB pointer 0x561545776000 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ** DB Stats ** 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ** Compaction Stats [default] ** 2024-04-20T10:58:53.908 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ** Compaction Stats [default] ** 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T10:58:53.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Cumulative compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Interval compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Block cache BinnedLRUCache@0x5615456e91f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173 does not exist in monmap, will attempt to join an existing cluster 2024-04-20T10:58:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: using public_addr v2:172.21.15.173:0/0 -> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: starting mon.smithi173 rank -1 at public addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] at bind addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi173 fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(???) e0 preinit fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).mds e1 new map 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).mds e1 print_map 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: e1 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: legacy client fscid: -1 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: No filesystems configured 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-20T10:58:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/3846859477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/1950843135' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:58:53.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Deploying daemon ceph-exporter.smithi173 on smithi173 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/133945969' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:58:53.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Deploying daemon crash.smithi173 on smithi173 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/3913978053' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Deploying daemon node-exporter.smithi173 on smithi173 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/2887201163' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:58:53.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Deploying daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/4090178076' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: Deploying daemon mon.smithi173 on smithi173 2024-04-20T10:58:53.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/3989048848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:58:53.918 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:53.918 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.918 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:53.918 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:53 smithi173 ceph-mon[23800]: mon.smithi173@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-04-20T10:58:54.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:54 smithi173 bash[23761]: 243eed744f9205671aa5b19bc4745a06cf34b478718b166c32e09fecb3e588b1 2024-04-20T10:58:54.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:54 smithi173 systemd[1]: Started Ceph mon.smithi173 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T10:58:59.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T10:58:59.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: mon.smithi117 calling monitor election 2024-04-20T10:58:59.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:59.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: mon.smithi173 calling monitor election 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/2551709202' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: mon.smithi117 is new leader, mons smithi117,smithi173 in quorum (ranks 0,1) 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: monmap e2: 2 mons at {smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],smithi173=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: fsmap 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: osdmap e5: 0 total, 0 up, 0 in 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: mgrmap e17: smithi117.ivnklj(active, since 45s) 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: overall HEALTH_OK 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: Standby manager daemon smithi173.gnrjof started 2024-04-20T10:58:59.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/2551709202' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:58:59.144 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:59.144 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/2551709202' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T10:58:59.144 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/2551709202' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:58:59.144 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:59.144 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:59.144 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: mon.smithi117 calling monitor election 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: mon.smithi173 calling monitor election 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/2551709202' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: mon.smithi117 is new leader, mons smithi117,smithi173 in quorum (ranks 0,1) 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: monmap e2: 2 mons at {smithi117=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],smithi173=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: fsmap 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: osdmap e5: 0 total, 0 up, 0 in 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: mgrmap e17: smithi117.ivnklj(active, since 45s) 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: overall HEALTH_OK 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: Standby manager daemon smithi173.gnrjof started 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/2551709202' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:59.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/2551709202' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T10:58:59.164 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/2551709202' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:58:59.164 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:59.164 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:59.164 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:58:59.936 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.934+0000 7f1721e00700 1 -- 172.21.15.173:0/886150169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000035c0 msgr2=0x7f1700005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:59.936 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.934+0000 7f1721e00700 1 --2- 172.21.15.173:0/886150169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000035c0 0x7f1700005a50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f171c100850 tx=0x7f170c00ff50 comp rx=0 tx=0).stop 2024-04-20T10:58:59.937 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.935+0000 7f1721e00700 1 -- 172.21.15.173:0/886150169 shutdown_connections 2024-04-20T10:58:59.937 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.935+0000 7f1721e00700 1 --2- 172.21.15.173:0/886150169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17000035c0 0x7f1700005a50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:59.937 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.935+0000 7f1721e00700 1 -- 172.21.15.173:0/886150169 >> 172.21.15.173:0/886150169 conn(0x7f171c0fc430 msgr2=0x7f171c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:59.937 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.935+0000 7f1721e00700 1 -- 172.21.15.173:0/886150169 shutdown_connections 2024-04-20T10:58:59.937 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.935+0000 7f1721e00700 1 -- 172.21.15.173:0/886150169 wait complete. 2024-04-20T10:58:59.940 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.936+0000 7f1721e00700 1 Processor -- start 2024-04-20T10:58:59.940 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.936+0000 7f1721e00700 1 -- start start 2024-04-20T10:58:59.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.936+0000 7f1721e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 0x7f171c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:59.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.937+0000 7f1721e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f171c196910 0x7f171c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:59.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.937+0000 7f1721e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c196f30 con 0x7f171c196910 2024-04-20T10:58:59.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.937+0000 7f1721e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1970a0 con 0x7f17000035c0 2024-04-20T10:58:59.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.937+0000 7f171aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 0x7f171c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:59.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.937+0000 7f171aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 0x7f171c1963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:50272/0 (socket says 172.21.15.173:50272) 2024-04-20T10:58:59.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.937+0000 7f171aa00700 1 -- 172.21.15.173:0/868388536 learned_addr learned my addr 172.21.15.173:0/868388536 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T10:58:59.942 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.937+0000 7f171a000700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f171c196910 0x7f171c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:59.942 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.937+0000 7f171aa00700 1 -- 172.21.15.173:0/868388536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 msgr2=0x7f171c1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-20T10:58:59.942 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.938+0000 7f171aa00700 1 -- 172.21.15.173:0/868388536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 msgr2=0x7f171c1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T10:58:59.942 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.938+0000 7f171aa00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 0x7f171c1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:58:59.942 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.938+0000 7f171aa00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 0x7f171c1963d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:58:59.943 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.938+0000 7f171a000700 1 -- 172.21.15.173:0/868388536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 msgr2=0x7f171c1963d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:58:59.943 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.938+0000 7f171a000700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 0x7f171c1963d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:59.943 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.938+0000 7f171a000700 1 -- 172.21.15.173:0/868388536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c00f9a0 con 0x7f171c196910 2024-04-20T10:58:59.943 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.939+0000 7f171a000700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f171c196910 0x7f171c19ac70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f170800b4d0 tx=0x7f170800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:59.943 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.939+0000 7f1720a00700 1 -- 172.21.15.173:0/868388536 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1708011800 con 0x7f171c196910 2024-04-20T10:58:59.943 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.939+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c19b270 con 0x7f171c196910 2024-04-20T10:58:59.943 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.939+0000 7f1720a00700 1 -- 172.21.15.173:0/868388536 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1708011e20 con 0x7f171c196910 2024-04-20T10:58:59.944 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.939+0000 7f1720a00700 1 -- 172.21.15.173:0/868388536 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f170800f2e0 con 0x7f171c196910 2024-04-20T10:58:59.944 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.940+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c19b7c0 con 0x7f171c196910 2024-04-20T10:58:59.944 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.941+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c105370 con 0x7f171c196910 2024-04-20T10:58:59.944 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.943+0000 7f1720a00700 1 -- 172.21.15.173:0/868388536 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f1708010380 con 0x7f171c196910 2024-04-20T10:58:59.945 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.944+0000 7f1720a00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f16fc06cdb0 0x7f16fc06f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:59.945 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.945+0000 7f1720a00700 1 -- 172.21.15.173:0/868388536 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f170808ab80 con 0x7f171c196910 2024-04-20T10:58:59.946 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.945+0000 7f171aa00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f16fc06cdb0 0x7f16fc06f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:59.947 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.946+0000 7f171aa00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f16fc06cdb0 0x7f16fc06f270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f170c00f570 tx=0x7f170c00f4a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:59.958 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:58:59.957+0000 7f1720a00700 1 -- 172.21.15.173:0/868388536 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1708056150 con 0x7f171c196910 2024-04-20T10:59:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:59 smithi173 ceph-mon[23800]: mgrmap e18: smithi117.ivnklj(active, since 45s), standbys: smithi173.gnrjof 2024-04-20T10:59:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi173.gnrjof", "id": "smithi173.gnrjof"}]: dispatch 2024-04-20T10:59:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:59:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:59 smithi173 ceph-mon[23800]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:58:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:59:00.287 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.285+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f171c19ba90 con 0x7f171c196910 2024-04-20T10:59:00.287 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.287+0000 7f1720a00700 1 -- 172.21.15.173:0/868388536 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7f1708015420 con 0x7f171c196910 2024-04-20T10:59:00.288 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-20T10:59:00.288 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","modified":"2024-04-20T10:58:53.800023Z","created":"2024-04-20T10:55:27.473649Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi117","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi173","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.289+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f16fc06cdb0 msgr2=0x7f16fc06f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.289+0000 7f1721e00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f16fc06cdb0 0x7f16fc06f270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f170c00f570 tx=0x7f170c00f4a0 comp rx=0 tx=0).stop 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.289+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f171c196910 msgr2=0x7f171c19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.289+0000 7f1721e00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f171c196910 0x7f171c19ac70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f170800b4d0 tx=0x7f170800b890 comp rx=0 tx=0).stop 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.289+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 shutdown_connections 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.290+0000 7f1721e00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f16fc06cdb0 0x7f16fc06f270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.290+0000 7f1721e00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17000035c0 0x7f171c1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.290+0000 7f1721e00700 1 --2- 172.21.15.173:0/868388536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f171c196910 0x7f171c19ac70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.290+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 >> 172.21.15.173:0/868388536 conn(0x7f171c0fc430 msgr2=0x7f171c0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.290+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 shutdown_connections 2024-04-20T10:59:00.291 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T10:59:00.290+0000 7f1721e00700 1 -- 172.21.15.173:0/868388536 wait complete. 2024-04-20T10:59:00.292 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-04-20T10:59:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:59 smithi117 ceph-mon[15659]: mgrmap e18: smithi117.ivnklj(active, since 45s), standbys: smithi173.gnrjof 2024-04-20T10:59:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi173.gnrjof", "id": "smithi173.gnrjof"}]: dispatch 2024-04-20T10:59:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:59:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:59 smithi117 ceph-mon[15659]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:58:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T10:59:01.031 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:00 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/868388536' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:59:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:00 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/868388536' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:59:01.374 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-20T10:59:01.374 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph config generate-minimal-conf 2024-04-20T10:59:01.774 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:59:02.216 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:01 smithi117 ceph-mon[15659]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:01 smithi173 ceph-mon[23800]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:03.377 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:03 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:03.377 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:03 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:03.377 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:03 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:03 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:59:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:03 smithi117 ceph-mon[15659]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T10:59:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:03 smithi117 ceph-mon[15659]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T10:59:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:03 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:03 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:03 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:03.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:03 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:59:03.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:03 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T10:59:03.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:03 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T10:59:03.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.862+0000 7fdfd0800700 1 -- 172.21.15.117:0/2935760725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 msgr2=0x7fdfc8106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:03.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.862+0000 7fdfd0800700 1 --2- 172.21.15.117:0/2935760725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 0x7fdfc8106bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009910 tx=0x7fdfbc009c20 comp rx=0 tx=0).stop 2024-04-20T10:59:03.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.862+0000 7fdfd0800700 1 -- 172.21.15.117:0/2935760725 shutdown_connections 2024-04-20T10:59:03.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.862+0000 7fdfd0800700 1 --2- 172.21.15.117:0/2935760725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 0x7fdfc8106bd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:03.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.862+0000 7fdfd0800700 1 -- 172.21.15.117:0/2935760725 >> 172.21.15.117:0/2935760725 conn(0x7fdfc80fc300 msgr2=0x7fdfc80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:03.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.863+0000 7fdfd0800700 1 -- 172.21.15.117:0/2935760725 shutdown_connections 2024-04-20T10:59:03.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.863+0000 7fdfd0800700 1 -- 172.21.15.117:0/2935760725 wait complete. 2024-04-20T10:59:03.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.864+0000 7fdfd0800700 1 Processor -- start 2024-04-20T10:59:03.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.864+0000 7fdfd0800700 1 -- start start 2024-04-20T10:59:03.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.865+0000 7fdfd0800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 0x7fdfc80731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:03.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.865+0000 7fdfd0800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc8073720 0x7fdfc8073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:03.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.865+0000 7fdfd0800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8075130 con 0x7fdfc81067f0 2024-04-20T10:59:03.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.865+0000 7fdfd0800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc81a2450 con 0x7fdfc8073720 2024-04-20T10:59:03.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.865+0000 7fdfcde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 0x7fdfc80731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:03.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.865+0000 7fdfcde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 0x7fdfc80731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57132/0 (socket says 172.21.15.117:57132) 2024-04-20T10:59:03.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.865+0000 7fdfcde00700 1 -- 172.21.15.117:0/2512316370 learned_addr learned my addr 172.21.15.117:0/2512316370 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:59:03.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.865+0000 7fdfcd400700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc8073720 0x7fdfc8073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:03.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfcde00700 1 -- 172.21.15.117:0/2512316370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc8073720 msgr2=0x7fdfc8073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:03.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfcde00700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc8073720 0x7fdfc8073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:03.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfcde00700 1 -- 172.21.15.117:0/2512316370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc0095f0 con 0x7fdfc81067f0 2024-04-20T10:59:03.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfcde00700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 0x7fdfc80731e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc006010 tx=0x7fdfbc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:03.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2512316370 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfbc01d070 con 0x7fdfc81067f0 2024-04-20T10:59:03.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2512316370 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfbc003d50 con 0x7fdfc81067f0 2024-04-20T10:59:03.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2512316370 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfbc017440 con 0x7fdfc81067f0 2024-04-20T10:59:03.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc81a2650 con 0x7fdfc81067f0 2024-04-20T10:59:03.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.866+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc81a2b70 con 0x7fdfc81067f0 2024-04-20T10:59:03.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.867+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc8101f30 con 0x7fdfc81067f0 2024-04-20T10:59:03.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.868+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2512316370 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7fdfbc00f460 con 0x7fdfc81067f0 2024-04-20T10:59:03.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.869+0000 7fdfc2a00700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdfb006cdb0 0x7fdfb006f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:03.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.869+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2512316370 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fdfbc08c3b0 con 0x7fdfc81067f0 2024-04-20T10:59:03.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.870+0000 7fdfcd400700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdfb006cdb0 0x7fdfb006f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:03.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.870+0000 7fdfcd400700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdfb006cdb0 0x7fdfb006f270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009850 tx=0x7fdfb8008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:03.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:03.874+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2512316370 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdfbc0577c0 con 0x7fdfc81067f0 2024-04-20T10:59:04.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.101+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fdfc810abe0 con 0x7fdfc81067f0 2024-04-20T10:59:04.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.102+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2512316370 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7fdfbc05ade0 con 0x7fdfc81067f0 2024-04-20T10:59:04.102 INFO:teuthology.orchestra.run.smithi117.stdout:# minimal ceph.conf for 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:59:04.102 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-04-20T10:59:04.103 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T10:59:04.103 INFO:teuthology.orchestra.run.smithi117.stdout: mon_host = [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-04-20T10:59:04.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.105+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdfb006cdb0 msgr2=0x7fdfb006f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:04.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.105+0000 7fdfd0800700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdfb006cdb0 0x7fdfb006f270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009850 tx=0x7fdfb8008040 comp rx=0 tx=0).stop 2024-04-20T10:59:04.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.105+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 msgr2=0x7fdfc80731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:04.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.105+0000 7fdfd0800700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 0x7fdfc80731e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc006010 tx=0x7fdfbc00bda0 comp rx=0 tx=0).stop 2024-04-20T10:59:04.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.106+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 shutdown_connections 2024-04-20T10:59:04.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.106+0000 7fdfd0800700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdfb006cdb0 0x7fdfb006f270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:04.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.106+0000 7fdfd0800700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc81067f0 0x7fdfc80731e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:04.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.106+0000 7fdfd0800700 1 --2- 172.21.15.117:0/2512316370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc8073720 0x7fdfc8073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:04.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.106+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 >> 172.21.15.117:0/2512316370 conn(0x7fdfc80fc300 msgr2=0x7fdfc81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:04.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.106+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 shutdown_connections 2024-04-20T10:59:04.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:04.106+0000 7fdfd0800700 1 -- 172.21.15.117:0/2512316370 wait complete. 2024-04-20T10:59:04.812 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2512316370' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:59:04.813 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:04.958 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-20T10:59:04.958 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:59:04.958 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T10:59:05.024 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:59:05.024 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:59:05.135 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:59:05.135 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T10:59:05.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2512316370' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:59:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:59:05.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:05.228 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:59:05.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:59:05.359 INFO:tasks.cephadm:Deploying OSDs... 2024-04-20T10:59:05.359 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T10:59:05.359 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:59:05.399 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:59:05.400 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-04-20T10:59:05.483 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:59:05.483 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:59:05.483 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-20T10:59:05.483 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:59:05.483 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:59:05.484 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-20 10:58:02.457376905 +0000 2024-04-20T10:59:05.484 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-20 10:51:27.404955255 +0000 2024-04-20T10:59:05.484 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-20 10:51:27.404955255 +0000 2024-04-20T10:59:05.484 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-20 10:51:27.404955255 +0000 2024-04-20T10:59:05.484 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:59:05.595 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-20T10:59:05.595 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-20T10:59:05.596 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000375254 s, 1.4 MB/s 2024-04-20T10:59:05.599 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:59:05.647 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-04-20T10:59:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:59:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:59:05.691 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-20T10:59:05.691 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:59:05.691 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:59:05.691 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-20 10:58:02.457376905 +0000 2024-04-20T10:59:05.691 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-20 10:51:28.011943465 +0000 2024-04-20T10:59:05.691 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-20 10:51:28.011943465 +0000 2024-04-20T10:59:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-20 10:51:28.011943465 +0000 2024-04-20T10:59:05.692 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:59:05.788 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-20T10:59:05.789 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-20T10:59:05.789 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000271978 s, 1.9 MB/s 2024-04-20T10:59:05.791 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:59:05.835 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-04-20T10:59:05.850 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:05 smithi117 ceph-mon[15659]: Reconfiguring mon.smithi117 (unknown last config time)... 2024-04-20T10:59:05.851 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:05 smithi117 ceph-mon[15659]: Reconfiguring daemon mon.smithi117 on smithi117 2024-04-20T10:59:05.889 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:59:05.889 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:59:05.889 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-20T10:59:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:59:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:59:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-20 10:58:02.458376886 +0000 2024-04-20T10:59:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-20 10:51:28.608931869 +0000 2024-04-20T10:59:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-20 10:51:28.608931869 +0000 2024-04-20T10:59:05.890 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-20 10:51:28.608931869 +0000 2024-04-20T10:59:05.890 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:59:05.985 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:05 smithi173 ceph-mon[23800]: Reconfiguring mon.smithi117 (unknown last config time)... 2024-04-20T10:59:05.986 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:05 smithi173 ceph-mon[23800]: Reconfiguring daemon mon.smithi117 on smithi117 2024-04-20T10:59:06.008 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-20T10:59:06.008 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-20T10:59:06.008 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000344872 s, 1.5 MB/s 2024-04-20T10:59:06.012 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:59:06.065 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-04-20T10:59:06.109 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:59:06.110 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:59:06.110 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-20T10:59:06.110 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:59:06.110 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:59:06.110 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-20 10:58:02.459376866 +0000 2024-04-20T10:59:06.110 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-20 10:51:29.203920311 +0000 2024-04-20T10:59:06.110 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-20 10:51:29.203920311 +0000 2024-04-20T10:59:06.110 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-20 10:51:29.203920311 +0000 2024-04-20T10:59:06.110 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:59:06.230 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-20T10:59:06.230 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-20T10:59:06.230 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000399748 s, 1.3 MB/s 2024-04-20T10:59:06.234 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:59:06.281 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T10:59:06.281 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:59:06.338 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:59:06.338 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-20 10:58:52.205693569 +0000 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-20 10:51:27.406229893 +0000 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-20 10:51:27.406229893 +0000 2024-04-20T10:59:06.388 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-20 10:51:27.406229893 +0000 2024-04-20T10:59:06.389 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:59:06.466 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-20T10:59:06.466 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-20T10:59:06.467 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000388379 s, 1.3 MB/s 2024-04-20T10:59:06.469 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:59:06.524 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-04-20T10:59:06.574 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:59:06.575 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:59:06.575 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-20T10:59:06.575 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:59:06.575 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:59:06.575 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-20 10:58:52.205693569 +0000 2024-04-20T10:59:06.575 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-20 10:51:28.019216857 +0000 2024-04-20T10:59:06.575 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-20 10:51:28.019216857 +0000 2024-04-20T10:59:06.575 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-20 10:51:28.019216857 +0000 2024-04-20T10:59:06.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:59:06.651 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-20T10:59:06.651 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-20T10:59:06.651 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000392592 s, 1.3 MB/s 2024-04-20T10:59:06.654 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:59:06.707 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-04-20T10:59:06.754 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:59:06.754 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:59:06.754 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-20T10:59:06.754 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:59:06.754 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:59:06.754 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-20 10:58:52.206693547 +0000 2024-04-20T10:59:06.755 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-20 10:51:28.613204226 +0000 2024-04-20T10:59:06.755 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-20 10:51:28.613204226 +0000 2024-04-20T10:59:06.755 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-20 10:51:28.613204226 +0000 2024-04-20T10:59:06.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:59:06.814 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:06 smithi173 ceph-mon[23800]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:06.814 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:06 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:06.814 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:06 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:06.814 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:06 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.ivnklj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:59:06.814 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:06 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:59:06.814 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:06 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:06.821 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-20T10:59:06.821 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-20T10:59:06.821 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000353625 s, 1.4 MB/s 2024-04-20T10:59:06.824 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:59:06.879 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-04-20T10:59:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:59:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:59:06.927 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-20T10:59:06.927 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:59:06.927 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:59:06.927 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-20 10:58:52.207693525 +0000 2024-04-20T10:59:06.928 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-20 10:51:29.210191530 +0000 2024-04-20T10:59:06.928 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-20 10:51:29.210191530 +0000 2024-04-20T10:59:06.928 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-20 10:51:29.210191530 +0000 2024-04-20T10:59:06.928 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:59:07.002 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-20T10:59:07.002 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-20T10:59:07.002 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000355457 s, 1.4 MB/s 2024-04-20T10:59:07.005 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:59:07.059 INFO:tasks.cephadm:Deploying osd.0 on smithi117 with /dev/vg_nvme/lv_4... 2024-04-20T10:59:07.059 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T10:59:07.096 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:06 smithi117 ceph-mon[15659]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:07.096 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:06 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:07.097 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:06 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:07.097 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:06 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.ivnklj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:59:07.097 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:06 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:59:07.097 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:06 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:08.121 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:07 smithi117 ceph-mon[15659]: Reconfiguring mgr.smithi117.ivnklj (unknown last config time)... 2024-04-20T10:59:08.121 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:07 smithi117 ceph-mon[15659]: Reconfiguring daemon mgr.smithi117.ivnklj on smithi117 2024-04-20T10:59:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:07 smithi173 ceph-mon[23800]: Reconfiguring mgr.smithi117.ivnklj (unknown last config time)... 2024-04-20T10:59:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:07 smithi173 ceph-mon[23800]: Reconfiguring daemon mgr.smithi117.ivnklj on smithi117 2024-04-20T10:59:08.394 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:59:08.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:08 smithi117 ceph-mon[15659]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:08.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:08 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:08.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:08 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:08.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:08 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T10:59:08.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:08 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:08 smithi173 ceph-mon[23800]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:08 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:08 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:09.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:08 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T10:59:09.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:08 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:10.070 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:09 smithi117 ceph-mon[15659]: Reconfiguring ceph-exporter.smithi117 (monmap changed)... 2024-04-20T10:59:10.070 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:09 smithi117 ceph-mon[15659]: Reconfiguring daemon ceph-exporter.smithi117 on smithi117 2024-04-20T10:59:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:09 smithi173 ceph-mon[23800]: Reconfiguring ceph-exporter.smithi117 (monmap changed)... 2024-04-20T10:59:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:09 smithi173 ceph-mon[23800]: Reconfiguring daemon ceph-exporter.smithi117 on smithi117 2024-04-20T10:59:11.005 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:10 smithi173 ceph-mon[23800]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:11.123 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:10 smithi117 ceph-mon[15659]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:11.715 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T10:59:11.715 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T10:59:11.715 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-20T10:59:11.715 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-20T10:59:11.715 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0349083 s, 300 MB/s 2024-04-20T10:59:11.715 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-20T10:59:12.598 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-04-20T10:59:12.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:12 smithi117 ceph-mon[15659]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:12.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:12.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:12.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:12 smithi117 ceph-mon[15659]: Reconfiguring crash.smithi117 (monmap changed)... 2024-04-20T10:59:12.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T10:59:12.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:12.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:12 smithi117 ceph-mon[15659]: Reconfiguring daemon crash.smithi117 on smithi117 2024-04-20T10:59:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:12 smithi173 ceph-mon[23800]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:12.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:12.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:12 smithi173 ceph-mon[23800]: Reconfiguring crash.smithi117 (monmap changed)... 2024-04-20T10:59:12.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T10:59:12.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:12.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:12 smithi173 ceph-mon[23800]: Reconfiguring daemon crash.smithi117 on smithi117 2024-04-20T10:59:13.986 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:59:14.880 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:14 smithi173 ceph-mon[23800]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:14.880 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:59:14.880 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:14.880 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:14 smithi117 ceph-mon[15659]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:59:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:15.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:15 smithi117 ceph-mon[15659]: Reconfiguring alertmanager.smithi117 (dependencies changed)... 2024-04-20T10:59:15.778 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:15 smithi117 ceph-mon[15659]: Reconfiguring daemon alertmanager.smithi117 on smithi117 2024-04-20T10:59:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:15 smithi173 ceph-mon[23800]: Reconfiguring alertmanager.smithi117 (dependencies changed)... 2024-04-20T10:59:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:15 smithi173 ceph-mon[23800]: Reconfiguring daemon alertmanager.smithi117 on smithi117 2024-04-20T10:59:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.837+0000 7fa7d1000700 1 -- 172.21.15.117:0/2696055876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 msgr2=0x7fa7cc102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.837+0000 7fa7d1000700 1 --2- 172.21.15.117:0/2696055876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 0x7fa7cc102270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009910 tx=0x7fa7b8009c20 comp rx=0 tx=0).stop 2024-04-20T10:59:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.837+0000 7fa7d1000700 1 -- 172.21.15.117:0/2696055876 shutdown_connections 2024-04-20T10:59:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.837+0000 7fa7d1000700 1 --2- 172.21.15.117:0/2696055876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 0x7fa7cc102270 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.837+0000 7fa7d1000700 1 --2- 172.21.15.117:0/2696055876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7cc100c10 0x7fa7cc101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.837+0000 7fa7d1000700 1 -- 172.21.15.117:0/2696055876 >> 172.21.15.117:0/2696055876 conn(0x7fa7cc0fc1d0 msgr2=0x7fa7cc0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.838+0000 7fa7d1000700 1 -- 172.21.15.117:0/2696055876 shutdown_connections 2024-04-20T10:59:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.838+0000 7fa7d1000700 1 -- 172.21.15.117:0/2696055876 wait complete. 2024-04-20T10:59:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.839+0000 7fa7d1000700 1 Processor -- start 2024-04-20T10:59:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.839+0000 7fa7d1000700 1 -- start start 2024-04-20T10:59:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.839+0000 7fa7d1000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7cc100c10 0x7fa7cc198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.839+0000 7fa7d1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 0x7fa7cc198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.839+0000 7fa7d1000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc199300 con 0x7fa7cc101e10 2024-04-20T10:59:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.839+0000 7fa7d1000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc199440 con 0x7fa7cc100c10 2024-04-20T10:59:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.840+0000 7fa7c9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 0x7fa7cc198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.840+0000 7fa7c9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 0x7fa7cc198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35844/0 (socket says 172.21.15.117:35844) 2024-04-20T10:59:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.840+0000 7fa7c9400700 1 -- 172.21.15.117:0/3904922563 learned_addr learned my addr 172.21.15.117:0/3904922563 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T10:59:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.840+0000 7fa7c9e00700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7cc100c10 0x7fa7cc198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.840+0000 7fa7c9400700 1 -- 172.21.15.117:0/3904922563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7cc100c10 msgr2=0x7fa7cc198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.840+0000 7fa7c9400700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7cc100c10 0x7fa7cc198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.840+0000 7fa7c9400700 1 -- 172.21.15.117:0/3904922563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b80095f0 con 0x7fa7cc101e10 2024-04-20T10:59:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.840+0000 7fa7c9400700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 0x7fa7cc198d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80092e0 tx=0x7fa7b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.841+0000 7fa7c2a00700 1 -- 172.21.15.117:0/3904922563 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7b801d070 con 0x7fa7cc101e10 2024-04-20T10:59:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.841+0000 7fa7c2a00700 1 -- 172.21.15.117:0/3904922563 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7b800f460 con 0x7fa7cc101e10 2024-04-20T10:59:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.841+0000 7fa7c2a00700 1 -- 172.21.15.117:0/3904922563 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7b8021640 con 0x7fa7cc101e10 2024-04-20T10:59:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.841+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc19de50 con 0x7fa7cc101e10 2024-04-20T10:59:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.841+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc19e390 con 0x7fa7cc101e10 2024-04-20T10:59:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.842+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc062510 con 0x7fa7cc101e10 2024-04-20T10:59:16.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.843+0000 7fa7c2a00700 1 -- 172.21.15.117:0/3904922563 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7fa7b800f5d0 con 0x7fa7cc101e10 2024-04-20T10:59:16.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.845+0000 7fa7c2a00700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa7b006cc60 0x7fa7b006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:16.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.845+0000 7fa7c2a00700 1 -- 172.21.15.117:0/3904922563 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa7b808c4e0 con 0x7fa7cc101e10 2024-04-20T10:59:16.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.845+0000 7fa7c9e00700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa7b006cc60 0x7fa7b006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:16.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.846+0000 7fa7c9e00700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa7b006cc60 0x7fa7b006f120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc005fd0 tx=0x7fa7bc005e50 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:16.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:16.850+0000 7fa7c2a00700 1 -- 172.21.15.117:0/3904922563 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa7b805b400 con 0x7fa7cc101e10 2024-04-20T10:59:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:16 smithi117 ceph-mon[15659]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:16 smithi173 ceph-mon[23800]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:17.084+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7cc074b50 con 0x7fa7b006cc60 2024-04-20T10:59:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:17 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:59:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:17 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:59:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:17 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:17 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:59:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:17 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:59:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:17 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:18.687 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:18 smithi117 ceph-mon[15659]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:59:18.687 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:18 smithi117 ceph-mon[15659]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:18 smithi173 ceph-mon[23800]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:59:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:18 smithi173 ceph-mon[23800]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:20.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:20 smithi117 ceph-mon[15659]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:20 smithi173 ceph-mon[23800]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:22.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:22 smithi173 ceph-mon[23800]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:23.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:22 smithi117 ceph-mon[15659]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:23 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:23 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:23 smithi117 ceph-mon[15659]: Reconfiguring grafana.smithi117 (dependencies changed)... 2024-04-20T10:59:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:23 smithi117 ceph-mon[15659]: Reconfiguring daemon grafana.smithi117 on smithi117 2024-04-20T10:59:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:23 smithi117 ceph-mon[15659]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:24.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:23 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:24.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:23 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:24.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:23 smithi173 ceph-mon[23800]: Reconfiguring grafana.smithi117 (dependencies changed)... 2024-04-20T10:59:24.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:23 smithi173 ceph-mon[23800]: Reconfiguring daemon grafana.smithi117 on smithi117 2024-04-20T10:59:24.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:23 smithi173 ceph-mon[23800]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:25.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:24 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3127010926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b"}]: dispatch 2024-04-20T10:59:25.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:24 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3127010926' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b"}]': finished 2024-04-20T10:59:25.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:24 smithi117 ceph-mon[15659]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T10:59:25.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:24 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:59:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:24 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3127010926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b"}]: dispatch 2024-04-20T10:59:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:24 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3127010926' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b"}]': finished 2024-04-20T10:59:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:24 smithi173 ceph-mon[23800]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T10:59:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:24 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:59:26.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:25 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3500587674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:59:26.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:25 smithi117 ceph-mon[15659]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:25 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3500587674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:59:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:25 smithi173 ceph-mon[23800]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:28.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:28 smithi117 ceph-mon[15659]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:28 smithi173 ceph-mon[23800]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:29.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:59:29.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:59:30.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:30 smithi117 ceph-mon[15659]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:30.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:30 smithi173 ceph-mon[23800]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:32.849 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:32 smithi117 ceph-mon[15659]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:32 smithi173 ceph-mon[23800]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:34.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:34 smithi117 ceph-mon[15659]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:34 smithi173 ceph-mon[23800]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:36.573 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:36 smithi117 ceph-mon[15659]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:36.573 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:36.573 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:36.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:36 smithi173 ceph-mon[23800]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:36.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:36 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:36.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:36 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:37.614 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:37 smithi117 ceph-mon[15659]: Reconfiguring prometheus.smithi117 (dependencies changed)... 2024-04-20T10:59:37.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:37 smithi173 ceph-mon[23800]: Reconfiguring prometheus.smithi117 (dependencies changed)... 2024-04-20T10:59:38.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:38 smithi117 ceph-mon[15659]: Reconfiguring daemon prometheus.smithi117 on smithi117 2024-04-20T10:59:38.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:38 smithi117 ceph-mon[15659]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:38.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T10:59:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:38 smithi173 ceph-mon[23800]: Reconfiguring daemon prometheus.smithi117 on smithi117 2024-04-20T10:59:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:38 smithi173 ceph-mon[23800]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T10:59:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:39.581 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:39 smithi117 ceph-mon[15659]: Deploying daemon osd.0 on smithi117 2024-04-20T10:59:39.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:39 smithi173 ceph-mon[23800]: Deploying daemon osd.0 on smithi117 2024-04-20T10:59:40.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:40 smithi117 ceph-mon[15659]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:40.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:40 smithi173 ceph-mon[23800]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:42.800 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:42 smithi117 ceph-mon[15659]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:42 smithi173 ceph-mon[23800]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:44.724 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:44 smithi117 ceph-mon[15659]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:44.724 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:59:44.724 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:44.724 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:44.724 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T10:59:44.724 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:44 smithi173 ceph-mon[23800]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:44.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:59:44.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:44.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:44.916 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T10:59:44.917 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:45.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:45 smithi117 ceph-mon[15659]: Reconfiguring ceph-exporter.smithi173 (monmap changed)... 2024-04-20T10:59:45.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:45 smithi117 ceph-mon[15659]: Reconfiguring daemon ceph-exporter.smithi173 on smithi173 2024-04-20T10:59:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:45 smithi173 ceph-mon[23800]: Reconfiguring ceph-exporter.smithi173 (monmap changed)... 2024-04-20T10:59:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:45 smithi173 ceph-mon[23800]: Reconfiguring daemon ceph-exporter.smithi173 on smithi173 2024-04-20T10:59:46.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:46 smithi117 ceph-mon[15659]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:46.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:46.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:46.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T10:59:46.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:46 smithi173 ceph-mon[23800]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T10:59:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:47.845 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:47 smithi117 ceph-mon[15659]: Reconfiguring crash.smithi173 (monmap changed)... 2024-04-20T10:59:47.845 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:47 smithi117 ceph-mon[15659]: Reconfiguring daemon crash.smithi173 on smithi173 2024-04-20T10:59:47.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:47 smithi173 ceph-mon[23800]: Reconfiguring crash.smithi173 (monmap changed)... 2024-04-20T10:59:47.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:47 smithi173 ceph-mon[23800]: Reconfiguring daemon crash.smithi173 on smithi173 2024-04-20T10:59:48.879 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:48 smithi117 ceph-mon[15659]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:48.879 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:48.879 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:48.879 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:59:48.879 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:59:48.880 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:48 smithi173 ceph-mon[23800]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:48.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:59:48.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:59:48.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:49 smithi117 ceph-mon[15659]: Reconfiguring mgr.smithi173.gnrjof (monmap changed)... 2024-04-20T10:59:49.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:49 smithi117 ceph-mon[15659]: Reconfiguring daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T10:59:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:49 smithi173 ceph-mon[23800]: Reconfiguring mgr.smithi173.gnrjof (monmap changed)... 2024-04-20T10:59:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:49 smithi173 ceph-mon[23800]: Reconfiguring daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T10:59:50.829 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:50 smithi117 ceph-mon[15659]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:50 smithi173 ceph-mon[23800]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:51.743 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:51.743 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:51.744 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:51 smithi173 ceph-mon[23800]: Reconfiguring mon.smithi173 (monmap changed)... 2024-04-20T10:59:51.744 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:59:51.744 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:59:51.744 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:51.744 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:51 smithi173 ceph-mon[23800]: Reconfiguring daemon mon.smithi173 on smithi173 2024-04-20T10:59:51.825 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:51.826 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:51.826 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:51 smithi117 ceph-mon[15659]: Reconfiguring mon.smithi173 (monmap changed)... 2024-04-20T10:59:51.826 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:59:51.826 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:59:51.826 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:59:51.826 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:51 smithi117 ceph-mon[15659]: Reconfiguring daemon mon.smithi173 on smithi173 2024-04-20T10:59:52.690 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:52 smithi117 ceph-mon[15659]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:52.690 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:52.690 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:52.690 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:52.690 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:52 smithi173 ceph-mon[23800]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-20T10:59:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-20T10:59:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-04-20T10:59:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-04-20T10:59:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-20T10:59:54.004 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:54.005 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:53 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:59:54.018 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-20T10:59:54.018 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-20T10:59:54.018 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-04-20T10:59:54.018 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-04-20T10:59:54.018 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:54.018 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:54.019 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:53 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:59:54.978 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:54 smithi173 ceph-mon[23800]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:55.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:54 smithi117 ceph-mon[15659]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:56.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.356+0000 7fa7c2a00700 1 -- 172.21.15.117:0/3904922563 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa7cc074b50 con 0x7fa7b006cc60 2024-04-20T10:59:56.356 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 0 on host 'smithi117' 2024-04-20T10:59:56.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.359+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa7b006cc60 msgr2=0x7fa7b006f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:56.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.359+0000 7fa7d1000700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa7b006cc60 0x7fa7b006f120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc005fd0 tx=0x7fa7bc005e50 comp rx=0 tx=0).stop 2024-04-20T10:59:56.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.359+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 msgr2=0x7fa7cc198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:56.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.359+0000 7fa7d1000700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 0x7fa7cc198d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80092e0 tx=0x7fa7b800be20 comp rx=0 tx=0).stop 2024-04-20T10:59:56.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.360+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 shutdown_connections 2024-04-20T10:59:56.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.360+0000 7fa7d1000700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa7b006cc60 0x7fa7b006f120 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.360+0000 7fa7d1000700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7cc100c10 0x7fa7cc198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.360+0000 7fa7d1000700 1 --2- 172.21.15.117:0/3904922563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101e10 0x7fa7cc198d70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.360+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 >> 172.21.15.117:0/3904922563 conn(0x7fa7cc0fc1d0 msgr2=0x7fa7cc073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:56.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.360+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 shutdown_connections 2024-04-20T10:59:56.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T10:59:56.360+0000 7fa7d1000700 1 -- 172.21.15.117:0/3904922563 wait complete. 2024-04-20T10:59:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:56 smithi117 ceph-mon[15659]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:56.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:56 smithi117 ceph-mon[15659]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T10:59:56.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:56.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:56.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:56 smithi173 ceph-mon[23800]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:56.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:56.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:56.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:56 smithi173 ceph-mon[23800]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T10:59:56.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:56.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T10:59:57.123 DEBUG:teuthology.orchestra.run.smithi117:osd.0> sudo journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.0.service 2024-04-20T10:59:57.130 INFO:tasks.cephadm:Deploying osd.1 on smithi117 with /dev/vg_nvme/lv_3... 2024-04-20T10:59:57.131 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T10:59:57.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:57 smithi117 ceph-mon[15659]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T10:59:57.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:57 smithi117 ceph-mon[15659]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T10:59:57.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:57 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:59:57.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:57 smithi117 ceph-mon[15659]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T10:59:57.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:57 smithi117 ceph-mon[15659]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:58.042 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T10:59:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:57 smithi173 ceph-mon[23800]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T10:59:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:57 smithi173 ceph-mon[23800]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T10:59:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:57 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:59:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:57 smithi173 ceph-mon[23800]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T10:59:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:57 smithi173 ceph-mon[23800]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:59:58.847 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:58 smithi117 ceph-mon[15659]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-20T10:59:58.847 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:58 smithi117 ceph-mon[15659]: osdmap e8: 1 total, 0 up, 1 in 2024-04-20T10:59:58.847 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:59:58.847 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:59:58.848 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T10:59:58.848 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 10:59:58 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[34342]: 2024-04-20T10:59:58.797+0000 7f32b5800700 -1 osd.0 0 waiting for initial osdmap 2024-04-20T10:59:58.848 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 10:59:58 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[34342]: 2024-04-20T10:59:58.818+0000 7f32ac000700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:59:59.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:58 smithi173 ceph-mon[23800]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-20T10:59:59.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:58 smithi173 ceph-mon[23800]: osdmap e8: 1 total, 0 up, 1 in 2024-04-20T10:59:59.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:59:59.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:59:59.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:00:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:59 smithi117 ceph-mon[15659]: purged_snaps scrub starts 2024-04-20T11:00:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:59 smithi117 ceph-mon[15659]: purged_snaps scrub ok 2024-04-20T11:00:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:00:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:59 smithi117 ceph-mon[15659]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' 2024-04-20T11:00:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 10:59:59 smithi117 ceph-mon[15659]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:00:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:59 smithi173 ceph-mon[23800]: purged_snaps scrub starts 2024-04-20T11:00:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:59 smithi173 ceph-mon[23800]: purged_snaps scrub ok 2024-04-20T11:00:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:00:00.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:59 smithi173 ceph-mon[23800]: from='osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302]' entity='osd.0' 2024-04-20T11:00:00.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 10:59:59 smithi173 ceph-mon[23800]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:00:00.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:00 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:00:00.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:00 smithi173 ceph-mon[23800]: osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] boot 2024-04-20T11:00:00.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:00 smithi173 ceph-mon[23800]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T11:00:00.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:00 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:00:00.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:00 smithi173 ceph-mon[23800]: overall HEALTH_OK 2024-04-20T11:00:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:00 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:00:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:00 smithi117 ceph-mon[15659]: osd.0 [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] boot 2024-04-20T11:00:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:00 smithi117 ceph-mon[15659]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T11:00:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:00 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:00:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:00 smithi117 ceph-mon[15659]: overall HEALTH_OK 2024-04-20T11:00:01.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:01 smithi117 ceph-mon[15659]: osdmap e10: 1 total, 1 up, 1 in 2024-04-20T11:00:01.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:01 smithi117 ceph-mon[15659]: pgmap v52: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T11:00:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T11:00:02.020 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-20T11:00:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-20T11:00:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0364859 s, 287 MB/s 2024-04-20T11:00:02.020 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-20T11:00:02.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:01 smithi173 ceph-mon[23800]: osdmap e10: 1 total, 1 up, 1 in 2024-04-20T11:00:02.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:01 smithi173 ceph-mon[23800]: pgmap v52: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:02.802 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-04-20T11:00:03.203 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:00:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:03 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:03 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:03.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:03 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:03.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:03 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:04 smithi117 ceph-mon[15659]: pgmap v53: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:04 smithi173 ceph-mon[23800]: pgmap v53: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:05.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.452+0000 7f469ca00700 1 -- 172.21.15.117:0/1780564119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698073940 msgr2=0x7f4698073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:05.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.452+0000 7f469ca00700 1 --2- 172.21.15.117:0/1780564119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698073940 0x7f4698073dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4684009910 tx=0x7f4684009c20 comp rx=0 tx=0).stop 2024-04-20T11:00:05.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.453+0000 7f469ca00700 1 -- 172.21.15.117:0/1780564119 shutdown_connections 2024-04-20T11:00:05.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.453+0000 7f469ca00700 1 --2- 172.21.15.117:0/1780564119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698073940 0x7f4698073dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:05.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.453+0000 7f469ca00700 1 --2- 172.21.15.117:0/1780564119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4698074f10 0x7f4698073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:05.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.453+0000 7f469ca00700 1 -- 172.21.15.117:0/1780564119 >> 172.21.15.117:0/1780564119 conn(0x7f46980fc1c0 msgr2=0x7f46980fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:05.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.453+0000 7f469ca00700 1 -- 172.21.15.117:0/1780564119 shutdown_connections 2024-04-20T11:00:05.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.453+0000 7f469ca00700 1 -- 172.21.15.117:0/1780564119 wait complete. 2024-04-20T11:00:05.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.455+0000 7f469ca00700 1 Processor -- start 2024-04-20T11:00:05.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.455+0000 7f469ca00700 1 -- start start 2024-04-20T11:00:05.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.455+0000 7f469ca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4698073940 0x7f4698198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:05.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f469ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698074f10 0x7f4698198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:05.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f469ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981993a0 con 0x7f4698074f10 2024-04-20T11:00:05.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f469ca00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981994e0 con 0x7f4698073940 2024-04-20T11:00:05.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f4695400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698074f10 0x7f4698198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:05.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f4695400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698074f10 0x7f4698198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60474/0 (socket says 172.21.15.117:60474) 2024-04-20T11:00:05.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f4695400700 1 -- 172.21.15.117:0/2154014782 learned_addr learned my addr 172.21.15.117:0/2154014782 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:00:05.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f4695e00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4698073940 0x7f4698198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:05.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f4695400700 1 -- 172.21.15.117:0/2154014782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4698073940 msgr2=0x7f4698198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:05.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f4695400700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4698073940 0x7f4698198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:05.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.456+0000 7f4695400700 1 -- 172.21.15.117:0/2154014782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840095f0 con 0x7f4698074f10 2024-04-20T11:00:05.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.457+0000 7f4695400700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698074f10 0x7f4698198d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f46840059d0 tx=0x7f468400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:05.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.457+0000 7f468ea00700 1 -- 172.21.15.117:0/2154014782 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f468401d070 con 0x7f4698074f10 2024-04-20T11:00:05.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.457+0000 7f468ea00700 1 -- 172.21.15.117:0/2154014782 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4684005dc0 con 0x7f4698074f10 2024-04-20T11:00:05.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.457+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469819dee0 con 0x7f4698074f10 2024-04-20T11:00:05.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.457+0000 7f468ea00700 1 -- 172.21.15.117:0/2154014782 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4684021410 con 0x7f4698074f10 2024-04-20T11:00:05.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.457+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469819e3d0 con 0x7f4698074f10 2024-04-20T11:00:05.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.458+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4678005320 con 0x7f4698074f10 2024-04-20T11:00:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.460+0000 7f468ea00700 1 -- 172.21.15.117:0/2154014782 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f4684003a60 con 0x7f4698074f10 2024-04-20T11:00:05.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.461+0000 7f468ea00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f467c06cc60 0x7f467c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:05.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.461+0000 7f468ea00700 1 -- 172.21.15.117:0/2154014782 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f468408c890 con 0x7f4698074f10 2024-04-20T11:00:05.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.461+0000 7f4695e00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f467c06cc60 0x7f467c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:05.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.462+0000 7f4695e00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f467c06cc60 0x7f467c06f120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f46980752f0 tx=0x7f468800b3c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:05.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.465+0000 7f468ea00700 1 -- 172.21.15.117:0/2154014782 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f468405b6f0 con 0x7f4698074f10 2024-04-20T11:00:05.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:05.712+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4678000bf0 con 0x7f467c06cc60 2024-04-20T11:00:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:06 smithi117 ceph-mon[15659]: pgmap v54: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:06 smithi117 ceph-mon[15659]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:00:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:06 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:00:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:06 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:00:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:06 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:06 smithi173 ceph-mon[23800]: pgmap v54: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:06 smithi173 ceph-mon[23800]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:00:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:06 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:00:06.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:06 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:00:06.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:06 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:08.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:08 smithi117 ceph-mon[15659]: pgmap v55: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:08 smithi173 ceph-mon[23800]: pgmap v55: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:10.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: pgmap v56: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:10.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:10.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:10.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:00:10.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:10.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:10.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:00:10.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:10.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:00:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: pgmap v56: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:00:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:00:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:10.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:00:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:11 smithi117 ceph-mon[15659]: Detected new or changed devices on smithi117 2024-04-20T11:00:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:11 smithi117 ceph-mon[15659]: Adjusting osd_memory_target on smithi117 to 11841M 2024-04-20T11:00:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:11 smithi173 ceph-mon[23800]: Detected new or changed devices on smithi117 2024-04-20T11:00:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:11 smithi173 ceph-mon[23800]: Adjusting osd_memory_target on smithi117 to 11841M 2024-04-20T11:00:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:12 smithi173 ceph-mon[23800]: pgmap v57: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:12 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/129759551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75ea37ae-266a-459b-a32b-e1d91927cbfb"}]: dispatch 2024-04-20T11:00:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:12 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/129759551' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75ea37ae-266a-459b-a32b-e1d91927cbfb"}]': finished 2024-04-20T11:00:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:12 smithi173 ceph-mon[23800]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T11:00:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:12 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2209433145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:00:12.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:12 smithi117 ceph-mon[15659]: pgmap v57: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:12.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:12 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/129759551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75ea37ae-266a-459b-a32b-e1d91927cbfb"}]: dispatch 2024-04-20T11:00:12.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:12 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/129759551' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75ea37ae-266a-459b-a32b-e1d91927cbfb"}]': finished 2024-04-20T11:00:12.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:12 smithi117 ceph-mon[15659]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T11:00:12.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:12.762 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:12 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2209433145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:00:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:14.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:14 smithi117 ceph-mon[15659]: pgmap v59: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:14.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:00:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:14 smithi173 ceph-mon[23800]: pgmap v59: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:00:16.874 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:16 smithi117 ceph-mon[15659]: pgmap v60: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:16.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:16 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:16.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:16 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:16.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:16 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:16.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:16 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:00:16.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:16 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:16 smithi173 ceph-mon[23800]: pgmap v60: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:16.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:16 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:16.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:16 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:16.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:16 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:16.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:16 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:00:16.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:16 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:18.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:18 smithi117 ceph-mon[15659]: pgmap v61: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:18 smithi173 ceph-mon[23800]: pgmap v61: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:20.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:20 smithi117 ceph-mon[15659]: pgmap v62: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:20 smithi173 ceph-mon[23800]: pgmap v62: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:21.853 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:00:21.854 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:21.854 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:21 smithi117 ceph-mon[15659]: Deploying daemon osd.1 on smithi117 2024-04-20T11:00:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:00:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:21 smithi173 ceph-mon[23800]: Deploying daemon osd.1 on smithi117 2024-04-20T11:00:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:22 smithi117 ceph-mon[15659]: pgmap v63: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:22 smithi173 ceph-mon[23800]: pgmap v63: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:24.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:24 smithi117 ceph-mon[15659]: pgmap v64: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:24 smithi173 ceph-mon[23800]: pgmap v64: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:26.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:26 smithi117 ceph-mon[15659]: pgmap v65: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:26 smithi173 ceph-mon[23800]: pgmap v65: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:28.742 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:28 smithi117 ceph-mon[15659]: pgmap v66: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:28 smithi173 ceph-mon[23800]: pgmap v66: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:00:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:00:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:00:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:00:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:30.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:30 smithi117 ceph-mon[15659]: pgmap v67: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:30.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:30 smithi173 ceph-mon[23800]: pgmap v67: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:32 smithi117 ceph-mon[15659]: pgmap v68: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:32 smithi173 ceph-mon[23800]: pgmap v68: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:32 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:32 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:33.853 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:33 smithi117 ceph-mon[15659]: from='osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:00:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:33 smithi173 ceph-mon[23800]: from='osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:00:33.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.946+0000 7f468ea00700 1 -- 172.21.15.117:0/2154014782 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4678000bf0 con 0x7f467c06cc60 2024-04-20T11:00:33.946 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 1 on host 'smithi117' 2024-04-20T11:00:33.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f467c06cc60 msgr2=0x7f467c06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:33.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f467c06cc60 0x7f467c06f120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f46980752f0 tx=0x7f468800b3c0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698074f10 msgr2=0x7f4698198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:33.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698074f10 0x7f4698198d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f46840059d0 tx=0x7f468400bd70 comp rx=0 tx=0).stop 2024-04-20T11:00:33.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 shutdown_connections 2024-04-20T11:00:33.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f467c06cc60 0x7f467c06f120 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4698073940 0x7f4698198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 --2- 172.21.15.117:0/2154014782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4698074f10 0x7f4698198d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 >> 172.21.15.117:0/2154014782 conn(0x7f46980fc1c0 msgr2=0x7f4698106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:33.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 shutdown_connections 2024-04-20T11:00:33.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:33.949+0000 7f469ca00700 1 -- 172.21.15.117:0/2154014782 wait complete. 2024-04-20T11:00:34.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:34 smithi117 ceph-mon[15659]: pgmap v69: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:34.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:34 smithi117 ceph-mon[15659]: from='osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T11:00:34.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:34 smithi117 ceph-mon[15659]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T11:00:34.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:34.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:34 smithi117 ceph-mon[15659]: from='osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:00:34.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:34.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:34 smithi173 ceph-mon[23800]: pgmap v69: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:34 smithi173 ceph-mon[23800]: from='osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T11:00:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:34 smithi173 ceph-mon[23800]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T11:00:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:34.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:34 smithi173 ceph-mon[23800]: from='osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:00:34.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:34.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:35.397 DEBUG:teuthology.orchestra.run.smithi117:osd.1> sudo journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.1.service 2024-04-20T11:00:35.401 INFO:tasks.cephadm:Deploying osd.2 on smithi117 with /dev/vg_nvme/lv_2... 2024-04-20T11:00:35.401 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T11:00:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:35 smithi117 ceph-mon[15659]: from='osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-20T11:00:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:35 smithi117 ceph-mon[15659]: osdmap e13: 2 total, 1 up, 2 in 2024-04-20T11:00:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:35 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:35 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:35 smithi173 ceph-mon[23800]: from='osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-20T11:00:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:35 smithi173 ceph-mon[23800]: osdmap e13: 2 total, 1 up, 2 in 2024-04-20T11:00:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:35 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:35 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:36.241 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:00:36.708 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:36 smithi117 ceph-mon[15659]: purged_snaps scrub starts 2024-04-20T11:00:36.708 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:36 smithi117 ceph-mon[15659]: purged_snaps scrub ok 2024-04-20T11:00:36.708 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:36 smithi117 ceph-mon[15659]: pgmap v72: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:36.709 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:36.709 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:36 smithi117 ceph-mon[15659]: osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] boot 2024-04-20T11:00:36.709 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:36 smithi117 ceph-mon[15659]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T11:00:36.709 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:36.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:36 smithi173 ceph-mon[23800]: purged_snaps scrub starts 2024-04-20T11:00:36.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:36 smithi173 ceph-mon[23800]: purged_snaps scrub ok 2024-04-20T11:00:36.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:36 smithi173 ceph-mon[23800]: pgmap v72: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:00:36.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:36 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:36.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:36 smithi173 ceph-mon[23800]: osd.1 [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] boot 2024-04-20T11:00:36.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:36 smithi173 ceph-mon[23800]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T11:00:36.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:36 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:00:37.726 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:37 smithi117 ceph-mon[15659]: osdmap e15: 2 total, 2 up, 2 in 2024-04-20T11:00:37.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:37 smithi173 ceph-mon[23800]: osdmap e15: 2 total, 2 up, 2 in 2024-04-20T11:00:38.833 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:38 smithi117 ceph-mon[15659]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:38 smithi173 ceph-mon[23800]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T11:00:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T11:00:39.691 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-20T11:00:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-20T11:00:39.692 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0319599 s, 328 MB/s 2024-04-20T11:00:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-20T11:00:40.344 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:40 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:40.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:40 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:40.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:40 smithi117 ceph-mon[15659]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:40.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:40 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:40.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:40 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:40.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:40 smithi173 ceph-mon[23800]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:40.768 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-04-20T11:00:41.645 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:00:42.828 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:42 smithi117 ceph-mon[15659]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:42 smithi173 ceph-mon[23800]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.442+0000 7f9cc1000700 1 -- 172.21.15.117:0/569319030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc101e00 msgr2=0x7f9cbc102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.442+0000 7f9cc1000700 1 --2- 172.21.15.117:0/569319030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc101e00 0x7f9cbc102260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009910 tx=0x7f9ca8009c20 comp rx=0 tx=0).stop 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.442+0000 7f9cc1000700 1 -- 172.21.15.117:0/569319030 shutdown_connections 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.442+0000 7f9cc1000700 1 --2- 172.21.15.117:0/569319030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc101e00 0x7f9cbc102260 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.442+0000 7f9cc1000700 1 --2- 172.21.15.117:0/569319030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cbc100c00 0x7f9cbc101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.442+0000 7f9cc1000700 1 -- 172.21.15.117:0/569319030 >> 172.21.15.117:0/569319030 conn(0x7f9cbc0fc1c0 msgr2=0x7f9cbc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.442+0000 7f9cc1000700 1 -- 172.21.15.117:0/569319030 shutdown_connections 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.442+0000 7f9cc1000700 1 -- 172.21.15.117:0/569319030 wait complete. 2024-04-20T11:00:44.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.443+0000 7f9cc1000700 1 Processor -- start 2024-04-20T11:00:44.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.443+0000 7f9cc1000700 1 -- start start 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cc1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc100c00 0x7f9cbc074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cc1000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cbc101e00 0x7f9cbc0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cc1000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc19a240 con 0x7f9cbc100c00 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cc1000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc19a380 con 0x7f9cbc101e00 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cb9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc100c00 0x7f9cbc074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cb9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc100c00 0x7f9cbc074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55646/0 (socket says 172.21.15.117:55646) 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cb9e00700 1 -- 172.21.15.117:0/1253492965 learned_addr learned my addr 172.21.15.117:0/1253492965 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cb9400700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cbc101e00 0x7f9cbc0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:44.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cb9e00700 1 -- 172.21.15.117:0/1253492965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cbc101e00 msgr2=0x7f9cbc0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cb9e00700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cbc101e00 0x7f9cbc0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.444+0000 7f9cb9e00700 1 -- 172.21.15.117:0/1253492965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca80095f0 con 0x7f9cbc100c00 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.445+0000 7f9cb9e00700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc100c00 0x7f9cbc074c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00d900 tx=0x7f9cac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.445+0000 7f9cb2a00700 1 -- 172.21.15.117:0/1253492965 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cac0041f0 con 0x7f9cbc100c00 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.445+0000 7f9cb2a00700 1 -- 172.21.15.117:0/1253492965 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cac004350 con 0x7f9cbc100c00 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.445+0000 7f9cb2a00700 1 -- 172.21.15.117:0/1253492965 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cac003dc0 con 0x7f9cbc100c00 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.445+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc073810 con 0x7f9cbc100c00 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.445+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc073db0 con 0x7f9cbc100c00 2024-04-20T11:00:44.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.446+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc190780 con 0x7f9cbc100c00 2024-04-20T11:00:44.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.447+0000 7f9cb2a00700 1 -- 172.21.15.117:0/1253492965 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f9cac009730 con 0x7f9cbc100c00 2024-04-20T11:00:44.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.448+0000 7f9cb2a00700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9ca006cca0 0x7f9ca006f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:44.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.448+0000 7f9cb2a00700 1 -- 172.21.15.117:0/1253492965 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f9cac08b600 con 0x7f9cbc100c00 2024-04-20T11:00:44.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.449+0000 7f9cb9400700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9ca006cca0 0x7f9ca006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:44.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.449+0000 7f9cb9400700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9ca006cca0 0x7f9ca006f160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800b5c0 tx=0x7f9ca8005160 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:44.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.454+0000 7f9cb2a00700 1 -- 172.21.15.117:0/1253492965 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9cac05a250 con 0x7f9cbc100c00 2024-04-20T11:00:44.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:00:44.702+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cbc109a00 con 0x7f9ca006cca0 2024-04-20T11:00:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:44 smithi117 ceph-mon[15659]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:00:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:44 smithi173 ceph-mon[23800]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:00:45.814 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:45 smithi117 ceph-mon[15659]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:00:45.814 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:45 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:00:45.814 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:45 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:00:45.814 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:45 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:45 smithi173 ceph-mon[23800]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:00:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:45 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:00:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:45 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:00:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:45 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:46.861 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: Detected new or changed devices on smithi117 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: Adjusting osd_memory_target on smithi117 to 5920M 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:46.862 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:00:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: Detected new or changed devices on smithi117 2024-04-20T11:00:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:00:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:00:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: Adjusting osd_memory_target on smithi117 to 5920M 2024-04-20T11:00:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:46.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:46.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:00:46.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:46.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:00:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:48 smithi173 ceph-mon[23800]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:49.063 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:48 smithi117 ceph-mon[15659]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:49.063 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:49.063 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:49 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2954160839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2323b6ba-7d04-4744-b61f-4936e4f419bd"}]: dispatch 2024-04-20T11:00:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:49 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2954160839' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2323b6ba-7d04-4744-b61f-4936e4f419bd"}]': finished 2024-04-20T11:00:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:49 smithi173 ceph-mon[23800]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T11:00:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:49 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:00:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:49 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/905567248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:00:49.993 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:49 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2954160839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2323b6ba-7d04-4744-b61f-4936e4f419bd"}]: dispatch 2024-04-20T11:00:49.993 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:49 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2954160839' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2323b6ba-7d04-4744-b61f-4936e4f419bd"}]': finished 2024-04-20T11:00:49.993 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:49 smithi117 ceph-mon[15659]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T11:00:49.994 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:49 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:00:49.994 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:49 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/905567248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:00:50.979 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:50 smithi117 ceph-mon[15659]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:51.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:50 smithi173 ceph-mon[23800]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:53.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:52 smithi117 ceph-mon[15659]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:53.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:53.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:53.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:53.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:00:53.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:52 smithi173 ceph-mon[23800]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:53.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:00:53.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:00:54.961 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:54 smithi117 ceph-mon[15659]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:55.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:54 smithi173 ceph-mon[23800]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:56.847 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:56 smithi117 ceph-mon[15659]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:56 smithi173 ceph-mon[23800]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:59.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:58 smithi117 ceph-mon[15659]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:59.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:00:59.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:59.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:00:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:58 smithi173 ceph-mon[23800]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:00:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:00:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:00:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:01:00.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:00:59 smithi173 ceph-mon[23800]: Deploying daemon osd.2 on smithi117 2024-04-20T11:01:00.098 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:00:59 smithi117 ceph-mon[15659]: Deploying daemon osd.2 on smithi117 2024-04-20T11:01:00.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:00 smithi117 ceph-mon[15659]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:01.116 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:00 smithi173 ceph-mon[23800]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:02.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:02 smithi117 ceph-mon[15659]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:03.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:02 smithi173 ceph-mon[23800]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:03.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:03 smithi117 ceph-mon[15659]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:03 smithi173 ceph-mon[23800]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:06 smithi117 ceph-mon[15659]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:06 smithi173 ceph-mon[23800]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:07.673 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:07 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:07.673 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:07 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:07.673 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:07 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:01:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:07 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:07 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:07 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:01:08.820 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:08 smithi117 ceph-mon[15659]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:08 smithi173 ceph-mon[23800]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:10.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:10 smithi117 ceph-mon[15659]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:10 smithi173 ceph-mon[23800]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:11.693 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 2 on host 'smithi117' 2024-04-20T11:01:11.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.691+0000 7f9cb2a00700 1 -- 172.21.15.117:0/1253492965 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9cbc109a00 con 0x7f9ca006cca0 2024-04-20T11:01:11.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.694+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9ca006cca0 msgr2=0x7f9ca006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:11.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.694+0000 7f9cc1000700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9ca006cca0 0x7f9ca006f160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800b5c0 tx=0x7f9ca8005160 comp rx=0 tx=0).stop 2024-04-20T11:01:11.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc100c00 msgr2=0x7f9cbc074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:11.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc100c00 0x7f9cbc074c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00d900 tx=0x7f9cac00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 shutdown_connections 2024-04-20T11:01:11.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9ca006cca0 0x7f9ca006f160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cbc100c00 0x7f9cbc074c30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 --2- 172.21.15.117:0/1253492965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cbc101e00 0x7f9cbc0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 >> 172.21.15.117:0/1253492965 conn(0x7f9cbc0fc1c0 msgr2=0x7f9cbc105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:11.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 shutdown_connections 2024-04-20T11:01:11.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:01:11.695+0000 7f9cc1000700 1 -- 172.21.15.117:0/1253492965 wait complete. 2024-04-20T11:01:11.735 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:11 smithi173 ceph-mon[23800]: from='osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:01:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:11 smithi117 ceph-mon[15659]: from='osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:01:12.762 DEBUG:teuthology.orchestra.run.smithi117:osd.2> sudo journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.2.service 2024-04-20T11:01:12.769 INFO:tasks.cephadm:Deploying osd.3 on smithi173 with /dev/vg_nvme/lv_4... 2024-04-20T11:01:12.769 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T11:01:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:12 smithi117 ceph-mon[15659]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:12 smithi117 ceph-mon[15659]: from='osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T11:01:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:12 smithi117 ceph-mon[15659]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T11:01:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:01:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:12 smithi117 ceph-mon[15659]: from='osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:01:12.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:12.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:12 smithi173 ceph-mon[23800]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:12 smithi173 ceph-mon[23800]: from='osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T11:01:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:12 smithi173 ceph-mon[23800]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T11:01:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:01:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:12 smithi173 ceph-mon[23800]: from='osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:01:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:01:13.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:13 smithi117 ceph-mon[15659]: purged_snaps scrub starts 2024-04-20T11:01:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:13 smithi117 ceph-mon[15659]: purged_snaps scrub ok 2024-04-20T11:01:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:13 smithi117 ceph-mon[15659]: from='osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-20T11:01:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:13 smithi117 ceph-mon[15659]: osdmap e18: 3 total, 2 up, 3 in 2024-04-20T11:01:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:01:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:01:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:13 smithi173 ceph-mon[23800]: purged_snaps scrub starts 2024-04-20T11:01:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:13 smithi173 ceph-mon[23800]: purged_snaps scrub ok 2024-04-20T11:01:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:13 smithi173 ceph-mon[23800]: from='osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-20T11:01:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:13 smithi173 ceph-mon[23800]: osdmap e18: 3 total, 2 up, 3 in 2024-04-20T11:01:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:13 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:01:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:13 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:01:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:14 smithi117 ceph-mon[15659]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:14 smithi117 ceph-mon[15659]: osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] boot 2024-04-20T11:01:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:14 smithi117 ceph-mon[15659]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T11:01:14.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:01:14.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:01:14.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:01:14.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:14 smithi173 ceph-mon[23800]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:01:14.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:14 smithi173 ceph-mon[23800]: osd.2 [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] boot 2024-04-20T11:01:14.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:14 smithi173 ceph-mon[23800]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T11:01:14.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:01:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:01:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:01:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:15 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T11:01:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:15 smithi173 ceph-mon[23800]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T11:01:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:15 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:01:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:15 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:15.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:15 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:15.972 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T11:01:15.972 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T11:01:15.972 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-20T11:01:15.972 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-20T11:01:15.972 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0384271 s, 273 MB/s 2024-04-20T11:01:15.972 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-20T11:01:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:15 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T11:01:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:15 smithi117 ceph-mon[15659]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T11:01:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:15 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:01:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:15 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:15 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:16.653 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:01:16 smithi117 sudo[51290]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:01:16.653 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:01:16 smithi117 sudo[51290]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:16.654 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:01:16 smithi117 sudo[51290]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:16.654 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:01:16 smithi117 sudo[51290]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:16.803 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2024-04-20T11:01:16.870 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:16 smithi173 ceph-mon[23800]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:16.870 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:16 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T11:01:16.870 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:16 smithi173 ceph-mon[23800]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T11:01:16.870 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:16 smithi173 ceph-mon[23800]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T11:01:16.977 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:16 smithi117 ceph-mon[15659]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:16.977 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:16 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T11:01:16.977 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:16 smithi117 ceph-mon[15659]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T11:01:16.977 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:16 smithi117 ceph-mon[15659]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T11:01:17.213 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:01:17.319 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:01:16 smithi117 sudo[51359]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:01:17.319 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:01:16 smithi117 sudo[51359]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:17.319 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:01:16 smithi117 sudo[51359]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:17.319 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:01:16 smithi117 sudo[51359]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:17 smithi117 ceph-mon[15659]: osdmap e22: 3 total, 3 up, 3 in 2024-04-20T11:01:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:17 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:17.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:01:17 smithi117 sudo[51421]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:01:17.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:01:17 smithi117 sudo[51421]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:17.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:01:17 smithi117 sudo[51421]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:17.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:01:17 smithi117 sudo[51421]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:18.137 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:17 smithi173 ceph-mon[23800]: osdmap e22: 3 total, 3 up, 3 in 2024-04-20T11:01:18.137 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:17 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:18.640 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:01:18 smithi117 sudo[51441]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:01:18.640 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:01:18 smithi117 sudo[51441]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:18.640 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:01:18 smithi117 sudo[51441]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:18.640 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:01:18 smithi117 sudo[51441]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:18.816 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:18 smithi173 ceph-mon[23800]: pgmap v102: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:18.817 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:18 smithi173 ceph-mon[23800]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T11:01:18.817 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:18 smithi173 ceph-mon[23800]: Cluster is now healthy 2024-04-20T11:01:18.817 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:18 smithi173 ceph-mon[23800]: mgrmap e19: smithi117.ivnklj(active, since 3m), standbys: smithi173.gnrjof 2024-04-20T11:01:18.959 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:01:18 smithi117 sudo[51446]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:01:18.959 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:01:18 smithi117 sudo[51446]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:18.959 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:01:18 smithi117 sudo[51446]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:18.959 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:01:18 smithi117 sudo[51446]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:18.960 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:18 smithi117 ceph-mon[15659]: pgmap v102: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:18.960 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:18 smithi117 ceph-mon[15659]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T11:01:18.960 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:18 smithi117 ceph-mon[15659]: Cluster is now healthy 2024-04-20T11:01:18.960 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:18 smithi117 ceph-mon[15659]: mgrmap e19: smithi117.ivnklj(active, since 3m), standbys: smithi173.gnrjof 2024-04-20T11:01:19.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.271+0000 7fad0c000700 1 -- 172.21.15.173:0/2641679967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 msgr2=0x7fad04103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:19.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.271+0000 7fad0c000700 1 --2- 172.21.15.173:0/2641679967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 0x7fad04103240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7facf800b0a0 tx=0x7facf800b3b0 comp rx=0 tx=0).stop 2024-04-20T11:01:19.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.271+0000 7fad0c000700 1 -- 172.21.15.173:0/2641679967 shutdown_connections 2024-04-20T11:01:19.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.271+0000 7fad0c000700 1 --2- 172.21.15.173:0/2641679967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad04103f80 0x7fad04104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:19.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.271+0000 7fad0c000700 1 --2- 172.21.15.173:0/2641679967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 0x7fad04103240 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:19.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.271+0000 7fad0c000700 1 -- 172.21.15.173:0/2641679967 >> 172.21.15.173:0/2641679967 conn(0x7fad040fe3a0 msgr2=0x7fad04100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:19.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.272+0000 7fad0c000700 1 -- 172.21.15.173:0/2641679967 shutdown_connections 2024-04-20T11:01:19.273 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.272+0000 7fad0c000700 1 -- 172.21.15.173:0/2641679967 wait complete. 2024-04-20T11:01:19.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad0c000700 1 Processor -- start 2024-04-20T11:01:19.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad0c000700 1 -- start start 2024-04-20T11:01:19.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad0c000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 0x7fad04198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:19.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad0c000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad04103f80 0x7fad04198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:19.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad0c000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041993c0 con 0x7fad04103f80 2024-04-20T11:01:19.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad0c000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad04199500 con 0x7fad04102e20 2024-04-20T11:01:19.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad09600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 0x7fad04198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:19.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad09600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 0x7fad04198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:55054/0 (socket says 172.21.15.173:55054) 2024-04-20T11:01:19.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.273+0000 7fad09600700 1 -- 172.21.15.173:0/2244435069 learned_addr learned my addr 172.21.15.173:0/2244435069 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T11:01:19.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.274+0000 7fad08c00700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad04103f80 0x7fad04198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:19.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.274+0000 7fad09600700 1 -- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad04103f80 msgr2=0x7fad04198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:19.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.274+0000 7fad09600700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad04103f80 0x7fad04198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:19.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.274+0000 7fad09600700 1 -- 172.21.15.173:0/2244435069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8009d00 con 0x7fad04102e20 2024-04-20T11:01:19.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.274+0000 7fad08c00700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad04103f80 0x7fad04198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:01:19.278 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.274+0000 7fad09600700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 0x7fad04198860 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7facf8007950 tx=0x7facf8003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:19.278 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.274+0000 7fad02000700 1 -- 172.21.15.173:0/2244435069 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facf8021070 con 0x7fad04102e20 2024-04-20T11:01:19.278 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.275+0000 7fad02000700 1 -- 172.21.15.173:0/2244435069 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facf800ce60 con 0x7fad04102e20 2024-04-20T11:01:19.278 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.275+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0419deb0 con 0x7fad04102e20 2024-04-20T11:01:19.278 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.275+0000 7fad02000700 1 -- 172.21.15.173:0/2244435069 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facf800e040 con 0x7fad04102e20 2024-04-20T11:01:19.278 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.275+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0419e3a0 con 0x7fad04102e20 2024-04-20T11:01:19.278 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.276+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad04192920 con 0x7fad04102e20 2024-04-20T11:01:19.279 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.277+0000 7fad02000700 1 -- 172.21.15.173:0/2244435069 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7facf80074e0 con 0x7fad04102e20 2024-04-20T11:01:19.279 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.278+0000 7fad02000700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7facec06cce0 0x7facec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:19.279 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.278+0000 7fad02000700 1 -- 172.21.15.173:0/2244435069 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3232+0+0 (secure 0 0 0) 0x7facf808cb40 con 0x7fad04102e20 2024-04-20T11:01:19.279 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.278+0000 7fad08c00700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7facec06cce0 0x7facec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:19.279 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.279+0000 7fad08c00700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7facec06cce0 0x7facec06f1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7facf0005fd0 tx=0x7facf0005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:19.283 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.283+0000 7fad02000700 1 -- 172.21.15.173:0/2244435069 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7facf805b1e0 con 0x7fad04102e20 2024-04-20T11:01:19.515 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:19.515+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fad0402cf30 con 0x7facec06cce0 2024-04-20T11:01:19.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:01:19 smithi117 sudo[51475]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:01:19.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:01:19 smithi117 sudo[51475]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:19.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:01:19 smithi117 sudo[51475]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:19.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:01:19 smithi117 sudo[51475]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:19 smithi117 sudo[51491]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:01:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:19 smithi117 sudo[51491]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:19 smithi117 sudo[51491]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:19 smithi117 sudo[51491]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:20.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:20 smithi117 sudo[52260]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:01:20.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:20 smithi117 sudo[52260]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:20.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:20 smithi117 sudo[52260]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:20.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:20 smithi117 sudo[52260]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:21.300 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 sudo[27514]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:01:21.300 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 sudo[27514]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:21.300 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 sudo[27514]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:21.300 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 sudo[27514]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:01:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: pgmap v103: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:01:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:01:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:01:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:01:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:01:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:01:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:01:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:01:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 sudo[27562]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:01:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 sudo[27562]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 sudo[27562]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 sudo[27562]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: pgmap v103: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:01:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: pgmap v104: 1 pgs: 1 active+clean; 449 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: Detected new or changed devices on smithi117 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:22.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:01:22.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.889 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:01:22.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: pgmap v104: 1 pgs: 1 active+clean; 449 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: Detected new or changed devices on smithi117 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:01:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:22.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:01:23.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:23 smithi117 ceph-mon[15659]: Adjusting osd_memory_target on smithi117 to 3947M 2024-04-20T11:01:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:23 smithi173 ceph-mon[23800]: Adjusting osd_memory_target on smithi117 to 3947M 2024-04-20T11:01:24.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:24 smithi117 ceph-mon[15659]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 115 KiB/s wr, 8 op/s 2024-04-20T11:01:24.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:24 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/1154590468' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d3df6-0d27-4e2d-9e40-55895e0a891d"}]: dispatch 2024-04-20T11:01:24.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:24 smithi117 ceph-mon[15659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d3df6-0d27-4e2d-9e40-55895e0a891d"}]: dispatch 2024-04-20T11:01:24.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:24 smithi117 ceph-mon[15659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b87d3df6-0d27-4e2d-9e40-55895e0a891d"}]': finished 2024-04-20T11:01:24.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:24 smithi117 ceph-mon[15659]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T11:01:24.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:24 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:24 smithi173 ceph-mon[23800]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 115 KiB/s wr, 8 op/s 2024-04-20T11:01:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:24 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/1154590468' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d3df6-0d27-4e2d-9e40-55895e0a891d"}]: dispatch 2024-04-20T11:01:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:24 smithi173 ceph-mon[23800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b87d3df6-0d27-4e2d-9e40-55895e0a891d"}]: dispatch 2024-04-20T11:01:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:24 smithi173 ceph-mon[23800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b87d3df6-0d27-4e2d-9e40-55895e0a891d"}]': finished 2024-04-20T11:01:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:24 smithi173 ceph-mon[23800]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T11:01:24.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:24 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:25 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/3688946471' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:01:25.953 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:25 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/3688946471' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:01:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:26 smithi173 ceph-mon[23800]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 89 KiB/s wr, 6 op/s 2024-04-20T11:01:27.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:26 smithi117 ceph-mon[15659]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 89 KiB/s wr, 6 op/s 2024-04-20T11:01:28.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:28 smithi117 ceph-mon[15659]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-20T11:01:28.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:28 smithi173 ceph-mon[23800]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-20T11:01:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:28.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:01:28.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:01:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:01:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:01:31.015 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:30 smithi117 ceph-mon[15659]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-20T11:01:31.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:30 smithi173 ceph-mon[23800]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-20T11:01:32.979 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:32 smithi173 ceph-mon[23800]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-20T11:01:33.147 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:32 smithi117 ceph-mon[15659]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-20T11:01:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:01:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:01:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:34.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:34 smithi173 ceph-mon[23800]: Deploying daemon osd.3 on smithi173 2024-04-20T11:01:34.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:34 smithi173 ceph-mon[23800]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 52 KiB/s wr, 3 op/s 2024-04-20T11:01:35.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:34 smithi117 ceph-mon[15659]: Deploying daemon osd.3 on smithi173 2024-04-20T11:01:35.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:34 smithi117 ceph-mon[15659]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 52 KiB/s wr, 3 op/s 2024-04-20T11:01:36.872 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:36 smithi173 ceph-mon[23800]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 983 B/s rd, 45 KiB/s wr, 2 op/s 2024-04-20T11:01:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:36 smithi117 ceph-mon[15659]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 983 B/s rd, 45 KiB/s wr, 2 op/s 2024-04-20T11:01:38.850 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:38 smithi173 ceph-mon[23800]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 938 B/s rd, 43 KiB/s wr, 2 op/s 2024-04-20T11:01:39.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:38 smithi117 ceph-mon[15659]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 938 B/s rd, 43 KiB/s wr, 2 op/s 2024-04-20T11:01:40.798 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:40 smithi173 ceph-mon[23800]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:41.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:40 smithi117 ceph-mon[15659]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:42.343 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:42.343 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:42.343 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:01:42.343 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:42 smithi173 ceph-mon[23800]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:01:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:42 smithi117 ceph-mon[15659]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:44.823 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:44 smithi173 ceph-mon[23800]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:44.823 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:01:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:44 smithi117 ceph-mon[15659]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:01:45.877 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.876+0000 7fad02000700 1 -- 172.21.15.173:0/2244435069 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fad0402cf30 con 0x7facec06cce0 2024-04-20T11:01:45.877 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 3 on host 'smithi173' 2024-04-20T11:01:45.877 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:45 smithi173 ceph-mon[23800]: from='osd.3 [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:01:45.877 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:45 smithi173 ceph-mon[23800]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:01:45.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7facec06cce0 msgr2=0x7facec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:45.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7facec06cce0 0x7facec06f1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7facf0005fd0 tx=0x7facf0005ee0 comp rx=0 tx=0).stop 2024-04-20T11:01:45.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 msgr2=0x7fad04198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:45.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 0x7fad04198860 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7facf8007950 tx=0x7facf8003ce0 comp rx=0 tx=0).stop 2024-04-20T11:01:45.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 shutdown_connections 2024-04-20T11:01:45.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7facec06cce0 0x7facec06f1a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:45.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04102e20 0x7fad04198860 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:45.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 --2- 172.21.15.173:0/2244435069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad04103f80 0x7fad04198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:45.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 >> 172.21.15.173:0/2244435069 conn(0x7fad040fe3a0 msgr2=0x7fad04107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:45.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.880+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 shutdown_connections 2024-04-20T11:01:45.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:45.881+0000 7fad0c000700 1 -- 172.21.15.173:0/2244435069 wait complete. 2024-04-20T11:01:45.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:45 smithi117 ceph-mon[15659]: from='osd.3 [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:01:45.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:45 smithi117 ceph-mon[15659]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:01:46.633 DEBUG:teuthology.orchestra.run.smithi173:osd.3> sudo journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.3.service 2024-04-20T11:01:46.637 INFO:tasks.cephadm:Deploying osd.4 on smithi173 with /dev/vg_nvme/lv_3... 2024-04-20T11:01:46.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T11:01:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T11:01:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T11:01:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: from='osd.3 [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:01:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:01:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:46 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T11:01:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T11:01:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: from='osd.3 [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:01:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:01:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:46.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:46 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:47.606 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:01:47.863 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:47 smithi173 ceph-mon[23800]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-20T11:01:47.863 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:47 smithi173 ceph-mon[23800]: osdmap e25: 4 total, 3 up, 4 in 2024-04-20T11:01:47.863 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:47 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:47.863 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:47 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:47.863 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:01:47 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[29765]: 2024-04-20T11:01:47.667+0000 7f7fd0200700 -1 osd.3 0 waiting for initial osdmap 2024-04-20T11:01:47.863 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:01:47 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[29765]: 2024-04-20T11:01:47.686+0000 7f7fc6a00700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:01:47.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:47 smithi117 ceph-mon[15659]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-20T11:01:47.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:47 smithi117 ceph-mon[15659]: osdmap e25: 4 total, 3 up, 4 in 2024-04-20T11:01:47.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:47 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:47.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:47 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:48.744 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:48 smithi173 ceph-mon[23800]: purged_snaps scrub starts 2024-04-20T11:01:48.744 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:48 smithi173 ceph-mon[23800]: purged_snaps scrub ok 2024-04-20T11:01:48.744 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:48 smithi173 ceph-mon[23800]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:48.745 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:48 smithi173 ceph-mon[23800]: osdmap e26: 4 total, 3 up, 4 in 2024-04-20T11:01:48.745 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:48.745 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:48 smithi173 ceph-mon[23800]: from='osd.3 ' entity='osd.3' 2024-04-20T11:01:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:48 smithi117 ceph-mon[15659]: purged_snaps scrub starts 2024-04-20T11:01:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:48 smithi117 ceph-mon[15659]: purged_snaps scrub ok 2024-04-20T11:01:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:48 smithi117 ceph-mon[15659]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:48 smithi117 ceph-mon[15659]: osdmap e26: 4 total, 3 up, 4 in 2024-04-20T11:01:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:48 smithi117 ceph-mon[15659]: from='osd.3 ' entity='osd.3' 2024-04-20T11:01:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:49 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:49 smithi117 ceph-mon[15659]: osd.3 [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] boot 2024-04-20T11:01:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:49 smithi117 ceph-mon[15659]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T11:01:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:49 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:49.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:49 smithi117 ceph-mon[15659]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T11:01:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:49 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:49 smithi173 ceph-mon[23800]: osd.3 [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] boot 2024-04-20T11:01:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:49 smithi173 ceph-mon[23800]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T11:01:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:49 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:01:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:49 smithi173 ceph-mon[23800]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T11:01:51.057 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:50 smithi173 ceph-mon[23800]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:51.057 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:50 smithi173 ceph-mon[23800]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T11:01:51.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:50 smithi117 ceph-mon[15659]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:01:51.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:50 smithi117 ceph-mon[15659]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T11:01:51.213 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T11:01:51.213 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T11:01:51.213 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-20T11:01:51.213 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-20T11:01:51.213 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036897 s, 284 MB/s 2024-04-20T11:01:51.213 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: osdmap e30: 4 total, 4 up, 4 in 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:01:51.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:51.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:01:51.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:51 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: osdmap e30: 4 total, 4 up, 4 in 2024-04-20T11:01:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:01:51.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:51 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:01:51.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2024-04-20T11:01:52.375 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:01:52.839 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:52 smithi173 ceph-mon[23800]: Detected new or changed devices on smithi173 2024-04-20T11:01:52.839 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:52 smithi173 ceph-mon[23800]: Adjusting osd_memory_target on smithi173 to 14913M 2024-04-20T11:01:52.839 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:52 smithi173 ceph-mon[23800]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:01:52.839 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:52 smithi173 ceph-mon[23800]: osdmap e31: 4 total, 4 up, 4 in 2024-04-20T11:01:53.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:52 smithi117 ceph-mon[15659]: Detected new or changed devices on smithi173 2024-04-20T11:01:53.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:52 smithi117 ceph-mon[15659]: Adjusting osd_memory_target on smithi173 to 14913M 2024-04-20T11:01:53.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:52 smithi117 ceph-mon[15659]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:01:53.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:52 smithi117 ceph-mon[15659]: osdmap e31: 4 total, 4 up, 4 in 2024-04-20T11:01:54.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.474+0000 7f91c2000700 1 -- 172.21.15.173:0/905224397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 msgr2=0x7f91bc1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:54.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.474+0000 7f91c2000700 1 --2- 172.21.15.173:0/905224397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 0x7f91bc1088b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009910 tx=0x7f91a8009c20 comp rx=0 tx=0).stop 2024-04-20T11:01:54.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.474+0000 7f91c2000700 1 -- 172.21.15.173:0/905224397 shutdown_connections 2024-04-20T11:01:54.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.474+0000 7f91c2000700 1 --2- 172.21.15.173:0/905224397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 0x7f91bc1088b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:54.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.474+0000 7f91c2000700 1 --2- 172.21.15.173:0/905224397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc107260 0x7f91bc107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:54.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.474+0000 7f91c2000700 1 -- 172.21.15.173:0/905224397 >> 172.21.15.173:0/905224397 conn(0x7f91bc076400 msgr2=0x7f91bc078860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:54.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.474+0000 7f91c2000700 1 -- 172.21.15.173:0/905224397 shutdown_connections 2024-04-20T11:01:54.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.474+0000 7f91c2000700 1 -- 172.21.15.173:0/905224397 wait complete. 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.475+0000 7f91c2000700 1 Processor -- start 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.475+0000 7f91c2000700 1 -- start start 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.475+0000 7f91c2000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc107260 0x7f91bc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.475+0000 7f91c2000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 0x7f91bc19d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.475+0000 7f91c2000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc19d710 con 0x7f91bc107260 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.475+0000 7f91c2000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc19d850 con 0x7f91bc108430 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.476+0000 7f91baa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 0x7f91bc19d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.476+0000 7f91baa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 0x7f91bc19d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:37590/0 (socket says 172.21.15.173:37590) 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.476+0000 7f91baa00700 1 -- 172.21.15.173:0/2584970699 learned_addr learned my addr 172.21.15.173:0/2584970699 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T11:01:54.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.476+0000 7f91bb400700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc107260 0x7f91bc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:54.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.476+0000 7f91baa00700 1 -- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc107260 msgr2=0x7f91bc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:54.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.476+0000 7f91baa00700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc107260 0x7f91bc19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:54.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.476+0000 7f91baa00700 1 -- 172.21.15.173:0/2584970699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a80095f0 con 0x7f91bc108430 2024-04-20T11:01:54.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.476+0000 7f91baa00700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 0x7f91bc19d0f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91a800bd70 tx=0x7f91a800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:54.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.477+0000 7f91b3e00700 1 -- 172.21.15.173:0/2584970699 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91a801d070 con 0x7f91bc108430 2024-04-20T11:01:54.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.477+0000 7f91b3e00700 1 -- 172.21.15.173:0/2584970699 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91a800f460 con 0x7f91bc108430 2024-04-20T11:01:54.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.477+0000 7f91b3e00700 1 -- 172.21.15.173:0/2584970699 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91a8021620 con 0x7f91bc108430 2024-04-20T11:01:54.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.477+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc1a2250 con 0x7f91bc108430 2024-04-20T11:01:54.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.477+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc1a2770 con 0x7f91bc108430 2024-04-20T11:01:54.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.478+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc196dd0 con 0x7f91bc108430 2024-04-20T11:01:54.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.479+0000 7f91b3e00700 1 -- 172.21.15.173:0/2584970699 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f91a800fa70 con 0x7f91bc108430 2024-04-20T11:01:54.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.480+0000 7f91b3e00700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f91a006ccf0 0x7f91a006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:54.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.480+0000 7f91b3e00700 1 -- 172.21.15.173:0/2584970699 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f91a808cf30 con 0x7f91bc108430 2024-04-20T11:01:54.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.481+0000 7f91bb400700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f91a006ccf0 0x7f91a006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:54.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.482+0000 7f91bb400700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f91a006ccf0 0x7f91a006f1b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f91ac005fd0 tx=0x7f91ac00a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:54.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.486+0000 7f91b3e00700 1 -- 172.21.15.173:0/2584970699 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f91a805b410 con 0x7f91bc108430 2024-04-20T11:01:54.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:01:54.716+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f91bc02ceb0 con 0x7f91a006ccf0 2024-04-20T11:01:55.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:54 smithi117 ceph-mon[15659]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-20T11:01:55.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:54 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:01:55.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:54 smithi173 ceph-mon[23800]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-20T11:01:55.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:54 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:01:55.858 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:55 smithi173 ceph-mon[23800]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:01:55.858 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:55 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:01:55.858 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:55 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:56.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:55 smithi117 ceph-mon[15659]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:01:56.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:55 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:01:56.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:55 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:01:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:56 smithi117 ceph-mon[15659]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T11:01:57.015 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:56 smithi173 ceph-mon[23800]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T11:01:59.029 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:58 smithi173 ceph-mon[23800]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-20T11:01:59.029 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:58 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/32471878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ceed42-a923-47e4-919d-98caab4e06ae"}]: dispatch 2024-04-20T11:01:59.029 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:58 smithi173 ceph-mon[23800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ceed42-a923-47e4-919d-98caab4e06ae"}]: dispatch 2024-04-20T11:01:59.029 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:58 smithi173 ceph-mon[23800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5ceed42-a923-47e4-919d-98caab4e06ae"}]': finished 2024-04-20T11:01:59.029 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:58 smithi173 ceph-mon[23800]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T11:01:59.030 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:01:59.030 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:01:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:58 smithi117 ceph-mon[15659]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-20T11:01:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:58 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/32471878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ceed42-a923-47e4-919d-98caab4e06ae"}]: dispatch 2024-04-20T11:01:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:58 smithi117 ceph-mon[15659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ceed42-a923-47e4-919d-98caab4e06ae"}]: dispatch 2024-04-20T11:01:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:58 smithi117 ceph-mon[15659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5ceed42-a923-47e4-919d-98caab4e06ae"}]': finished 2024-04-20T11:01:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:58 smithi117 ceph-mon[15659]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T11:01:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:01:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:00.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:01:59 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/927306617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:02:00.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:01:59 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/927306617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:02:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:00 smithi117 ceph-mon[15659]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:02:01.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:00 smithi173 ceph-mon[23800]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:02:02.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:01 smithi117 ceph-mon[15659]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:02.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:01 smithi173 ceph-mon[23800]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:04.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:04 smithi117 ceph-mon[15659]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:04 smithi173 ceph-mon[23800]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:06.849 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:06 smithi173 ceph-mon[23800]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:06 smithi117 ceph-mon[15659]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:07.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:07 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:02:07.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:07 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:07 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:02:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:07 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:08.753 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:08 smithi173 ceph-mon[23800]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:08.753 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:08 smithi173 ceph-mon[23800]: Deploying daemon osd.4 on smithi173 2024-04-20T11:02:08.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:08 smithi117 ceph-mon[15659]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:08.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:08 smithi117 ceph-mon[15659]: Deploying daemon osd.4 on smithi173 2024-04-20T11:02:10.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:10 smithi117 ceph-mon[15659]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:10.891 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:10 smithi173 ceph-mon[23800]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:12.856 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:12 smithi173 ceph-mon[23800]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:12.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:12 smithi117 ceph-mon[15659]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:14 smithi117 ceph-mon[15659]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:14 smithi173 ceph-mon[23800]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:15 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:02:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:15 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:15 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:15 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:02:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:15 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:15 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:16 smithi117 ceph-mon[15659]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:16.905 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:16 smithi173 ceph-mon[23800]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:18.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:18 smithi117 ceph-mon[15659]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:18 smithi173 ceph-mon[23800]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:19.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:19 smithi117 ceph-mon[15659]: from='osd.4 [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:02:19.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:19 smithi117 ceph-mon[15659]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:02:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:19 smithi173 ceph-mon[23800]: from='osd.4 [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:02:19.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:19 smithi173 ceph-mon[23800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:02:20.471 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 4 on host 'smithi173' 2024-04-20T11:02:20.471 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.470+0000 7f91b3e00700 1 -- 172.21.15.173:0/2584970699 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f91bc02ceb0 con 0x7f91a006ccf0 2024-04-20T11:02:20.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f91a006ccf0 msgr2=0x7f91a006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:20.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f91a006ccf0 0x7f91a006f1b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f91ac005fd0 tx=0x7f91ac00a560 comp rx=0 tx=0).stop 2024-04-20T11:02:20.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 msgr2=0x7f91bc19d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:20.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 0x7f91bc19d0f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91a800bd70 tx=0x7f91a800be50 comp rx=0 tx=0).stop 2024-04-20T11:02:20.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 shutdown_connections 2024-04-20T11:02:20.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f91a006ccf0 0x7f91a006f1b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:20.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc107260 0x7f91bc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:20.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 --2- 172.21.15.173:0/2584970699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91bc108430 0x7f91bc19d0f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:20.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.474+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 >> 172.21.15.173:0/2584970699 conn(0x7f91bc076400 msgr2=0x7f91bc10ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:20.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.475+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 shutdown_connections 2024-04-20T11:02:20.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:20.475+0000 7f91c2000700 1 -- 172.21.15.173:0/2584970699 wait complete. 2024-04-20T11:02:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:20 smithi173 ceph-mon[23800]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:20 smithi173 ceph-mon[23800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T11:02:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:20 smithi173 ceph-mon[23800]: osdmap e33: 5 total, 4 up, 5 in 2024-04-20T11:02:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:20 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:20 smithi173 ceph-mon[23800]: from='osd.4 [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:02:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:20 smithi173 ceph-mon[23800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:02:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:20 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:20 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:20 smithi117 ceph-mon[15659]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:20 smithi117 ceph-mon[15659]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T11:02:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:20 smithi117 ceph-mon[15659]: osdmap e33: 5 total, 4 up, 5 in 2024-04-20T11:02:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:20 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:20 smithi117 ceph-mon[15659]: from='osd.4 [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:02:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:20 smithi117 ceph-mon[15659]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:02:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:20 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:21.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:20 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:21.314 DEBUG:teuthology.orchestra.run.smithi173:osd.4> sudo journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.4.service 2024-04-20T11:02:21.319 INFO:tasks.cephadm:Deploying osd.5 on smithi173 with /dev/vg_nvme/lv_2... 2024-04-20T11:02:21.319 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T11:02:21.648 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-mon[23800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-20T11:02:21.648 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-mon[23800]: osdmap e34: 5 total, 4 up, 5 in 2024-04-20T11:02:21.648 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:21.648 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:21.648 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:21.648 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:21.648 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-mon[23800]: osdmap e35: 5 total, 4 up, 5 in 2024-04-20T11:02:21.648 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:21.832 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:02:21.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[36531]: 2024-04-20T11:02:21.759+0000 7f3cc6000700 -1 osd.4 0 waiting for initial osdmap 2024-04-20T11:02:21.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:02:21 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[36531]: 2024-04-20T11:02:21.785+0000 7f3cbaa00700 -1 osd.4 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:02:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:21 smithi117 ceph-mon[15659]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-20T11:02:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:21 smithi117 ceph-mon[15659]: osdmap e34: 5 total, 4 up, 5 in 2024-04-20T11:02:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:21 smithi117 ceph-mon[15659]: osdmap e35: 5 total, 4 up, 5 in 2024-04-20T11:02:22.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:21 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:22 smithi173 ceph-mon[23800]: purged_snaps scrub starts 2024-04-20T11:02:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:22 smithi173 ceph-mon[23800]: purged_snaps scrub ok 2024-04-20T11:02:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:22 smithi173 ceph-mon[23800]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:22 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:22 smithi173 ceph-mon[23800]: from='osd.4 ' entity='osd.4' 2024-04-20T11:02:23.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:22 smithi117 ceph-mon[15659]: purged_snaps scrub starts 2024-04-20T11:02:23.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:22 smithi117 ceph-mon[15659]: purged_snaps scrub ok 2024-04-20T11:02:23.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:22 smithi117 ceph-mon[15659]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:23.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:22 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:23.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:22 smithi117 ceph-mon[15659]: from='osd.4 ' entity='osd.4' 2024-04-20T11:02:23.835 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:23 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:23.836 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:23 smithi173 ceph-mon[23800]: osd.4 [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] boot 2024-04-20T11:02:23.836 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:23 smithi173 ceph-mon[23800]: osdmap e36: 5 total, 5 up, 5 in 2024-04-20T11:02:23.836 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:23 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:23 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:23 smithi117 ceph-mon[15659]: osd.4 [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] boot 2024-04-20T11:02:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:23 smithi117 ceph-mon[15659]: osdmap e36: 5 total, 5 up, 5 in 2024-04-20T11:02:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:23 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:02:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:24 smithi173 ceph-mon[23800]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:24 smithi173 ceph-mon[23800]: osdmap e37: 5 total, 5 up, 5 in 2024-04-20T11:02:25.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:24 smithi117 ceph-mon[15659]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:02:25.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:24 smithi117 ceph-mon[15659]: osdmap e37: 5 total, 5 up, 5 in 2024-04-20T11:02:25.164 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T11:02:25.164 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T11:02:25.164 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-20T11:02:25.164 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-20T11:02:25.164 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0314839 s, 333 MB/s 2024-04-20T11:02:25.165 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-20T11:02:26.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2024-04-20T11:02:26.530 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:02:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:26 smithi173 ceph-mon[23800]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:27.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:26 smithi117 ceph-mon[15659]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:28.971 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:02:29.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:29.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:02:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:02:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:02:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:29.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:02:29.463 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.462+0000 7ffab1800700 1 -- 172.21.15.173:0/163646017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 msgr2=0x7ffaac103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.463 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.462+0000 7ffab1800700 1 --2- 172.21.15.173:0/163646017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 0x7ffaac103ab0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009910 tx=0x7ffa9c009c20 comp rx=0 tx=0).stop 2024-04-20T11:02:29.463 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.462+0000 7ffab1800700 1 -- 172.21.15.173:0/163646017 shutdown_connections 2024-04-20T11:02:29.464 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.462+0000 7ffab1800700 1 --2- 172.21.15.173:0/163646017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac103ff0 0x7ffaac1063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.464 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.462+0000 7ffab1800700 1 --2- 172.21.15.173:0/163646017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 0x7ffaac103ab0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.464 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.462+0000 7ffab1800700 1 -- 172.21.15.173:0/163646017 >> 172.21.15.173:0/163646017 conn(0x7ffaac0fb0d0 msgr2=0x7ffaac0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:29.464 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.462+0000 7ffab1800700 1 -- 172.21.15.173:0/163646017 shutdown_connections 2024-04-20T11:02:29.464 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.462+0000 7ffab1800700 1 -- 172.21.15.173:0/163646017 wait complete. 2024-04-20T11:02:29.466 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.463+0000 7ffab1800700 1 Processor -- start 2024-04-20T11:02:29.466 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.463+0000 7ffab1800700 1 -- start start 2024-04-20T11:02:29.466 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffab1800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 0x7ffaac1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.466 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffab1800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac103ff0 0x7ffaac198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.466 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffab1800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac199310 con 0x7ffaac103ff0 2024-04-20T11:02:29.466 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffab1800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac199450 con 0x7ffaac1016c0 2024-04-20T11:02:29.466 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffaaaa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 0x7ffaac1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.466 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffaaaa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 0x7ffaac1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:37986/0 (socket says 172.21.15.173:37986) 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffaaaa00700 1 -- 172.21.15.173:0/2282608714 learned_addr learned my addr 172.21.15.173:0/2282608714 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffaa3e00700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac103ff0 0x7ffaac198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffaaaa00700 1 -- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac103ff0 msgr2=0x7ffaac198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffaaaa00700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac103ff0 0x7ffaac198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.464+0000 7ffaaaa00700 1 -- 172.21.15.173:0/2282608714 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0095f0 con 0x7ffaac1016c0 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.465+0000 7ffaaaa00700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 0x7ffaac1987b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00bd70 tx=0x7ffa9c00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.465+0000 7ffaa3400700 1 -- 172.21.15.173:0/2282608714 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa9c01d070 con 0x7ffaac1016c0 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.465+0000 7ffaa3400700 1 -- 172.21.15.173:0/2282608714 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa9c005dc0 con 0x7ffaac1016c0 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.465+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac19de50 con 0x7ffaac1016c0 2024-04-20T11:02:29.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.465+0000 7ffaa3400700 1 -- 172.21.15.173:0/2282608714 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa9c021410 con 0x7ffaac1016c0 2024-04-20T11:02:29.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.465+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac19e340 con 0x7ffaac1016c0 2024-04-20T11:02:29.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.466+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac0fcca0 con 0x7ffaac1016c0 2024-04-20T11:02:29.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.467+0000 7ffaa3400700 1 -- 172.21.15.173:0/2282608714 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ffa9c00f820 con 0x7ffaac1016c0 2024-04-20T11:02:29.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.468+0000 7ffaa3400700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ffa9006cce0 0x7ffa9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.468+0000 7ffaa3400700 1 -- 172.21.15.173:0/2282608714 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4206+0+0 (secure 0 0 0) 0x7ffa9c08cfa0 con 0x7ffaac1016c0 2024-04-20T11:02:29.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.469+0000 7ffaa3e00700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ffa9006cce0 0x7ffa9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.470 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.470+0000 7ffaa3e00700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ffa9006cce0 0x7ffa9006f1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa94005950 tx=0x7ffa940058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.473+0000 7ffaa3400700 1 -- 172.21.15.173:0/2282608714 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ffa9c05b2d0 con 0x7ffaac1016c0 2024-04-20T11:02:29.709 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:29.708+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaac02cef0 con 0x7ffa9006cce0 2024-04-20T11:02:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:29 smithi173 ceph-mon[23800]: Detected new or changed devices on smithi173 2024-04-20T11:02:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:29 smithi173 ceph-mon[23800]: Adjusting osd_memory_target on smithi173 to 7456M 2024-04-20T11:02:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:02:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:29 smithi117 ceph-mon[15659]: Detected new or changed devices on smithi173 2024-04-20T11:02:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:29 smithi117 ceph-mon[15659]: Adjusting osd_memory_target on smithi173 to 7456M 2024-04-20T11:02:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:02:30.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:30.829 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:30 smithi173 ceph-mon[23800]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:30.829 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:30 smithi173 ceph-mon[23800]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:02:30.829 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:30 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:02:30.829 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:30 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:02:30.829 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:30 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:30 smithi117 ceph-mon[15659]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:30 smithi117 ceph-mon[15659]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:02:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:30 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:02:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:30 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:02:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:30 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:33.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:32 smithi117 ceph-mon[15659]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:33.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:32 smithi173 ceph-mon[23800]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:34.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:33 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/1743057512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b99706-a27f-4f30-9e6d-619ee6a35ec5"}]: dispatch 2024-04-20T11:02:34.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:33 smithi117 ceph-mon[15659]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b99706-a27f-4f30-9e6d-619ee6a35ec5"}]: dispatch 2024-04-20T11:02:34.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:33 smithi117 ceph-mon[15659]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47b99706-a27f-4f30-9e6d-619ee6a35ec5"}]': finished 2024-04-20T11:02:34.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:33 smithi117 ceph-mon[15659]: osdmap e38: 6 total, 5 up, 6 in 2024-04-20T11:02:34.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:34.142 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:33 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/1743057512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b99706-a27f-4f30-9e6d-619ee6a35ec5"}]: dispatch 2024-04-20T11:02:34.143 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:33 smithi173 ceph-mon[23800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b99706-a27f-4f30-9e6d-619ee6a35ec5"}]: dispatch 2024-04-20T11:02:34.143 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:33 smithi173 ceph-mon[23800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47b99706-a27f-4f30-9e6d-619ee6a35ec5"}]': finished 2024-04-20T11:02:34.143 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:33 smithi173 ceph-mon[23800]: osdmap e38: 6 total, 5 up, 6 in 2024-04-20T11:02:34.143 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:35.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:34 smithi117 ceph-mon[15659]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:35.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:34 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/2888147801' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:02:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:34 smithi173 ceph-mon[23800]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:34 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/2888147801' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:02:37.135 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:36 smithi173 ceph-mon[23800]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:36 smithi117 ceph-mon[15659]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:39.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:38 smithi117 ceph-mon[15659]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:39.123 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:38 smithi173 ceph-mon[23800]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:41.077 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:40 smithi173 ceph-mon[23800]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:41.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:40 smithi117 ceph-mon[15659]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:42.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:42 smithi117 ceph-mon[15659]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:42 smithi173 ceph-mon[23800]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:02:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:02:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:02:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:44 smithi173 ceph-mon[23800]: Deploying daemon osd.5 on smithi173 2024-04-20T11:02:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:44 smithi173 ceph-mon[23800]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:45.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:44 smithi117 ceph-mon[15659]: Deploying daemon osd.5 on smithi173 2024-04-20T11:02:45.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:44 smithi117 ceph-mon[15659]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:46.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:45 smithi117 ceph-mon[15659]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:46.145 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:45 smithi173 ceph-mon[23800]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:48.864 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:48 smithi173 ceph-mon[23800]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:48.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:48 smithi117 ceph-mon[15659]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:50.748 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:50 smithi173 ceph-mon[23800]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:50.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:50 smithi117 ceph-mon[15659]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:52.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:02:52.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:52.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:52 smithi173 ceph-mon[23800]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:02:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:52 smithi117 ceph-mon[15659]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:54.865 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:54 smithi173 ceph-mon[23800]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:54.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:54 smithi117 ceph-mon[15659]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:55.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:55 smithi117 ceph-mon[15659]: from='osd.5 [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:02:55.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:55 smithi117 ceph-mon[15659]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:02:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:55 smithi173 ceph-mon[23800]: from='osd.5 [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:02:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:55 smithi173 ceph-mon[23800]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:02:56.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.065+0000 7ffaa3400700 1 -- 172.21.15.173:0/2282608714 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffaac02cef0 con 0x7ffa9006cce0 2024-04-20T11:02:56.066 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 5 on host 'smithi173' 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ffa9006cce0 msgr2=0x7ffa9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ffa9006cce0 0x7ffa9006f1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa94005950 tx=0x7ffa940058e0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 msgr2=0x7ffaac1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 0x7ffaac1987b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00bd70 tx=0x7ffa9c00be50 comp rx=0 tx=0).stop 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 shutdown_connections 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ffa9006cce0 0x7ffa9006f1a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaac1016c0 0x7ffaac1987b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 --2- 172.21.15.173:0/2282608714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac103ff0 0x7ffaac198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 >> 172.21.15.173:0/2282608714 conn(0x7ffaac0fb0d0 msgr2=0x7ffaac100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:56.069 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 shutdown_connections 2024-04-20T11:02:56.070 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:02:56.069+0000 7ffab1800700 1 -- 172.21.15.173:0/2282608714 wait complete. 2024-04-20T11:02:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T11:02:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: osdmap e39: 6 total, 5 up, 6 in 2024-04-20T11:02:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: from='osd.5 [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:02:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:02:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:56.810 DEBUG:teuthology.orchestra.run.smithi173:osd.5> sudo journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.5.service 2024-04-20T11:02:56.815 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-20T11:02:56.815 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd stat -f json 2024-04-20T11:02:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T11:02:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: osdmap e39: 6 total, 5 up, 6 in 2024-04-20T11:02:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: from='osd.5 [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:02:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:02:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:56.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:56.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:56.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:02:57.227 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:02:57.634 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:57 smithi117 ceph-mon[15659]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-20T11:02:57.635 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:57 smithi117 ceph-mon[15659]: osdmap e40: 6 total, 5 up, 6 in 2024-04-20T11:02:57.635 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:57 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:57.635 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:57 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:57.651 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:57 smithi173 ceph-mon[23800]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-20T11:02:57.651 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:57 smithi173 ceph-mon[23800]: osdmap e40: 6 total, 5 up, 6 in 2024-04-20T11:02:57.651 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:57 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:57.651 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:57 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:57.651 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:02:57 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[43171]: 2024-04-20T11:02:57.636+0000 7fc055e00700 -1 osd.5 0 waiting for initial osdmap 2024-04-20T11:02:57.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:02:57 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[43171]: 2024-04-20T11:02:57.663+0000 7fc04c600700 -1 osd.5 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:02:58.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:58 smithi117 ceph-mon[15659]: purged_snaps scrub starts 2024-04-20T11:02:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:58 smithi117 ceph-mon[15659]: purged_snaps scrub ok 2024-04-20T11:02:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:58 smithi117 ceph-mon[15659]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:58 smithi117 ceph-mon[15659]: from='osd.5 ' entity='osd.5' 2024-04-20T11:02:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:58.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:58 smithi173 ceph-mon[23800]: purged_snaps scrub starts 2024-04-20T11:02:58.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:58 smithi173 ceph-mon[23800]: purged_snaps scrub ok 2024-04-20T11:02:58.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:58 smithi173 ceph-mon[23800]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:02:58.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:58.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:58 smithi173 ceph-mon[23800]: from='osd.5 ' entity='osd.5' 2024-04-20T11:02:58.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.715+0000 7fc806400700 1 -- 172.21.15.117:0/3729314862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 msgr2=0x7fc800107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.715+0000 7fc806400700 1 --2- 172.21.15.117:0/3729314862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 0x7fc800107680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009910 tx=0x7fc7f0009c20 comp rx=0 tx=0).stop 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.715+0000 7fc806400700 1 -- 172.21.15.117:0/3729314862 shutdown_connections 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.715+0000 7fc806400700 1 --2- 172.21.15.117:0/3729314862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc800108430 0x7fc8001088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.715+0000 7fc806400700 1 --2- 172.21.15.117:0/3729314862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 0x7fc800107680 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.715+0000 7fc806400700 1 -- 172.21.15.117:0/3729314862 >> 172.21.15.117:0/3729314862 conn(0x7fc800076400 msgr2=0x7fc800078860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.715+0000 7fc806400700 1 -- 172.21.15.117:0/3729314862 shutdown_connections 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.715+0000 7fc806400700 1 -- 172.21.15.117:0/3729314862 wait complete. 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.716+0000 7fc806400700 1 Processor -- start 2024-04-20T11:02:59.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.716+0000 7fc806400700 1 -- start start 2024-04-20T11:02:59.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.716+0000 7fc806400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 0x7fc80019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:59.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.716+0000 7fc806400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc800108430 0x7fc80019d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:59.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc806400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80019d6e0 con 0x7fc800107260 2024-04-20T11:02:59.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc806400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80019d820 con 0x7fc800108430 2024-04-20T11:02:59.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc7ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 0x7fc80019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:59.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc7ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 0x7fc80019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60504/0 (socket says 172.21.15.117:60504) 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc7ff400700 1 -- 172.21.15.117:0/4173869186 learned_addr learned my addr 172.21.15.117:0/4173869186 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc7fea00700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc800108430 0x7fc80019d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc7ff400700 1 -- 172.21.15.117:0/4173869186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc800108430 msgr2=0x7fc80019d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc7ff400700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc800108430 0x7fc80019d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.717+0000 7fc7ff400700 1 -- 172.21.15.117:0/4173869186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00095f0 con 0x7fc800107260 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.718+0000 7fc7ff400700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 0x7fc80019cc10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006010 tx=0x7fc7f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.718+0000 7fc7f7e00700 1 -- 172.21.15.117:0/4173869186 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7f001d070 con 0x7fc800107260 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.718+0000 7fc7f7e00700 1 -- 172.21.15.117:0/4173869186 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7f000f460 con 0x7fc800107260 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.718+0000 7fc7f7e00700 1 -- 172.21.15.117:0/4173869186 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7f0021640 con 0x7fc800107260 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.718+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8001a2230 con 0x7fc800107260 2024-04-20T11:02:59.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.718+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8001a2770 con 0x7fc800107260 2024-04-20T11:02:59.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.719+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc800196dd0 con 0x7fc800107260 2024-04-20T11:02:59.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.720+0000 7fc7f7e00700 1 -- 172.21.15.117:0/4173869186 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fc7f000fab0 con 0x7fc800107260 2024-04-20T11:02:59.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.721+0000 7fc7f7e00700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc7e406cce0 0x7fc7e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:59.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.721+0000 7fc7f7e00700 1 -- 172.21.15.117:0/4173869186 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc7f008d2a0 con 0x7fc800107260 2024-04-20T11:02:59.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.722+0000 7fc7fea00700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc7e406cce0 0x7fc7e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:59.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.722+0000 7fc7fea00700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc7e406cce0 0x7fc7e406f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005ea0 tx=0x7fc7ec005e30 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:59.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.727+0000 7fc7f7e00700 1 -- 172.21.15.117:0/4173869186 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc7f005b420 con 0x7fc800107260 2024-04-20T11:02:59.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:59 smithi173 ceph-mon[23800]: osd.5 [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] boot 2024-04-20T11:02:59.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:59 smithi173 ceph-mon[23800]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T11:02:59.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:59.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:02:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:59.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.951+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc80004ebb0 con 0x7fc800107260 2024-04-20T11:02:59.953 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:59 smithi117 ceph-mon[15659]: osd.5 [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] boot 2024-04-20T11:02:59.953 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:59 smithi117 ceph-mon[15659]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T11:02:59.953 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:02:59.953 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:02:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:02:59.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.952+0000 7fc7f7e00700 1 -- 172.21.15.117:0/4173869186 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v42) v1 ==== 74+0+130 (secure 0 0 0) 0x7fc7f005afb0 con 0x7fc800107260 2024-04-20T11:02:59.954 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:02:59.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc7e406cce0 msgr2=0x7fc7e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:59.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc7e406cce0 0x7fc7e406f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005ea0 tx=0x7fc7ec005e30 comp rx=0 tx=0).stop 2024-04-20T11:02:59.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 msgr2=0x7fc80019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:59.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 0x7fc80019cc10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006010 tx=0x7fc7f000be20 comp rx=0 tx=0).stop 2024-04-20T11:02:59.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 shutdown_connections 2024-04-20T11:02:59.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc7e406cce0 0x7fc7e406f1a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:59.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc800107260 0x7fc80019cc10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:59.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 --2- 172.21.15.117:0/4173869186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc800108430 0x7fc80019d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:59.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.957+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 >> 172.21.15.117:0/4173869186 conn(0x7fc800076400 msgr2=0x7fc80010ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:59.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.958+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 shutdown_connections 2024-04-20T11:02:59.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:02:59.958+0000 7fc806400700 1 -- 172.21.15.117:0/4173869186 wait complete. 2024-04-20T11:03:00.828 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":42,"num_osds":6,"num_up_osds":6,"osd_up_since":1713610978,"num_in_osds":6,"osd_in_since":1713610953,"num_remapped_pgs":0} 2024-04-20T11:03:00.829 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T11:03:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:00 smithi117 ceph-mon[15659]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:03:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:00 smithi117 ceph-mon[15659]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T11:03:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:00 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/4173869186' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T11:03:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:00 smithi173 ceph-mon[23800]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:03:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:00 smithi173 ceph-mon[23800]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T11:03:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:00 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/4173869186' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T11:03:01.241 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:02.884 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:03:02.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:03:02.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:03:03.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.394+0000 7f5b49c00700 1 -- 172.21.15.117:0/3785955776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 msgr2=0x7f5b44105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:03.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.394+0000 7f5b49c00700 1 --2- 172.21.15.117:0/3785955776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 0x7f5b44105820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009910 tx=0x7f5b30009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:03.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.394+0000 7f5b49c00700 1 -- 172.21.15.117:0/3785955776 shutdown_connections 2024-04-20T11:03:03.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.394+0000 7f5b49c00700 1 --2- 172.21.15.117:0/3785955776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 0x7f5b44105820 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:03.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.394+0000 7f5b49c00700 1 --2- 172.21.15.117:0/3785955776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b44100e20 0x7f5b44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:03.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.394+0000 7f5b49c00700 1 -- 172.21.15.117:0/3785955776 >> 172.21.15.117:0/3785955776 conn(0x7f5b440fc6f0 msgr2=0x7f5b440feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:03.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.394+0000 7f5b49c00700 1 -- 172.21.15.117:0/3785955776 shutdown_connections 2024-04-20T11:03:03.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.395+0000 7f5b49c00700 1 -- 172.21.15.117:0/3785955776 wait complete. 2024-04-20T11:03:03.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.395+0000 7f5b49c00700 1 Processor -- start 2024-04-20T11:03:03.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.396+0000 7f5b49c00700 1 -- start start 2024-04-20T11:03:03.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.396+0000 7f5b49c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b44100e20 0x7f5b44198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:03.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.396+0000 7f5b49c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 0x7f5b44199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:03.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.396+0000 7f5b49c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44199a20 con 0x7f5b441017d0 2024-04-20T11:03:03.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.396+0000 7f5b49c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4419d6f0 con 0x7f5b44100e20 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.396+0000 7f5b42000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 0x7f5b44199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.396+0000 7f5b42000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 0x7f5b44199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60510/0 (socket says 172.21.15.117:60510) 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.396+0000 7f5b42000700 1 -- 172.21.15.117:0/4071166154 learned_addr learned my addr 172.21.15.117:0/4071166154 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.397+0000 7f5b42a00700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b44100e20 0x7f5b44198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.397+0000 7f5b42000700 1 -- 172.21.15.117:0/4071166154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b44100e20 msgr2=0x7f5b44198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.397+0000 7f5b42000700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b44100e20 0x7f5b44198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.397+0000 7f5b42000700 1 -- 172.21.15.117:0/4071166154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b300095f0 con 0x7f5b441017d0 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.397+0000 7f5b42000700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 0x7f5b44199340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000bd70 tx=0x7f5b3000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.397+0000 7f5b3b400700 1 -- 172.21.15.117:0/4071166154 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3001d070 con 0x7f5b441017d0 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.397+0000 7f5b3b400700 1 -- 172.21.15.117:0/4071166154 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b3000f460 con 0x7f5b441017d0 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.397+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4419d970 con 0x7f5b441017d0 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.398+0000 7f5b3b400700 1 -- 172.21.15.117:0/4071166154 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b30017630 con 0x7f5b441017d0 2024-04-20T11:03:03.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.398+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4419de60 con 0x7f5b441017d0 2024-04-20T11:03:03.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.399+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b44102300 con 0x7f5b441017d0 2024-04-20T11:03:03.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.400+0000 7f5b3b400700 1 -- 172.21.15.117:0/4071166154 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f5b3000f5d0 con 0x7f5b441017d0 2024-04-20T11:03:03.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.401+0000 7f5b3b400700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5b2806cce0 0x7f5b2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:03.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.401+0000 7f5b3b400700 1 -- 172.21.15.117:0/4071166154 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5b3008d660 con 0x7f5b441017d0 2024-04-20T11:03:03.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.401+0000 7f5b42a00700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5b2806cce0 0x7f5b2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:03.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.402+0000 7f5b42a00700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5b2806cce0 0x7f5b2806f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b34006fd0 tx=0x7f5b34008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:03.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.405+0000 7f5b3b400700 1 -- 172.21.15.117:0/4071166154 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5b30057e70 con 0x7f5b441017d0 2024-04-20T11:03:03.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.625+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b44062510 con 0x7f5b441017d0 2024-04-20T11:03:03.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.628+0000 7f5b3b400700 1 -- 172.21.15.117:0/4071166154 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+11402 (secure 0 0 0) 0x7f5b30005740 con 0x7f5b441017d0 2024-04-20T11:03:03.629 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:03:03.629 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":42,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","created":"2024-04-20T10:55:31.778050+0000","modified":"2024-04-20T11:02:59.647082+0000","last_up_change":"2024-04-20T11:02:58.642045+0000","last_in_change":"2024-04-20T11:02:33.545131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:01:14.283466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6803","nonce":2292675302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6805","nonce":2292675302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6809","nonce":2292675302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6807","nonce":2292675302}]},"public_addr":"172.21.15.117:6803/2292675302","cluster_addr":"172.21.15.117:6805/2292675302","heartbeat_back_addr":"172.21.15.117:6809/2292675302","heartbeat_front_addr":"172.21.15.117:6807/2292675302","state":["exists","up"]},{"osd":1,"uuid":"75ea37ae-266a-459b-a32b-e1d91927cbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6811","nonce":2871230534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6813","nonce":2871230534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6817","nonce":2871230534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6815","nonce":2871230534}]},"public_addr":"172.21.15.117:6811/2871230534","cluster_addr":"172.21.15.117:6813/2871230534","heartbeat_back_addr":"172.21.15.117:6817/2871230534","heartbeat_front_addr":"172.21.15.117:6815/2871230534","state":["exists","up"]},{"osd":2,"uuid":"2323b6ba-7d04-4744-b61f-4936e4f419bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6819","nonce":3091267150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6821","nonce":3091267150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6825","nonce":3091267150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6823","nonce":3091267150}]},"public_addr":"172.21.15.117:6819/3091267150","cluster_addr":"172.21.15.117:6821/3091267150","heartbeat_back_addr":"172.21.15.117:6825/3091267150","heartbeat_front_addr":"172.21.15.117:6823/3091267150","state":["exists","up"]},{"osd":3,"uuid":"b87d3df6-0d27-4e2d-9e40-55895e0a891d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6801","nonce":2587001742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6803","nonce":2587001742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6807","nonce":2587001742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6805","nonce":2587001742}]},"public_addr":"172.21.15.173:6801/2587001742","cluster_addr":"172.21.15.173:6803/2587001742","heartbeat_back_addr":"172.21.15.173:6807/2587001742","heartbeat_front_addr":"172.21.15.173:6805/2587001742","state":["exists","up"]},{"osd":4,"uuid":"c5ceed42-a923-47e4-919d-98caab4e06ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6809","nonce":1468348921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6811","nonce":1468348921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6815","nonce":1468348921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6813","nonce":1468348921}]},"public_addr":"172.21.15.173:6809/1468348921","cluster_addr":"172.21.15.173:6811/1468348921","heartbeat_back_addr":"172.21.15.173:6815/1468348921","heartbeat_front_addr":"172.21.15.173:6813/1468348921","state":["exists","up"]},{"osd":5,"uuid":"47b99706-a27f-4f30-9e6d-619ee6a35ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6817","nonce":1909838821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6819","nonce":1909838821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6823","nonce":1909838821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6821","nonce":1909838821}]},"public_addr":"172.21.15.173:6817/1909838821","cluster_addr":"172.21.15.173:6819/1909838821","heartbeat_back_addr":"172.21.15.173:6823/1909838821","heartbeat_front_addr":"172.21.15.173:6821/1909838821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:59:57.050973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:00:33.939553+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:11.618080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:46.287328+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:20.465163+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:56.550872+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/3713452282":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6800/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1089633358":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6801/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6801/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1807203325":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6800/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1384022397":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2096654470":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/4088938050":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/1375611439":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2304890710":"2024-04-21T10:56:04.789705+0000","172.21.15.117:6800/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2230602257":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6801/1432319225":"2024-04-21T10:56:04.789705+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:03:03.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5b2806cce0 msgr2=0x7f5b2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:03.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5b2806cce0 0x7f5b2806f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b34006fd0 tx=0x7f5b34008040 comp rx=0 tx=0).stop 2024-04-20T11:03:03.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 msgr2=0x7f5b44199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:03.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 0x7f5b44199340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000bd70 tx=0x7f5b3000be50 comp rx=0 tx=0).stop 2024-04-20T11:03:03.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 shutdown_connections 2024-04-20T11:03:03.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5b2806cce0 0x7f5b2806f1a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:03.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b44100e20 0x7f5b44198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:03.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 --2- 172.21.15.117:0/4071166154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b441017d0 0x7f5b44199340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:03.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 >> 172.21.15.117:0/4071166154 conn(0x7f5b440fc6f0 msgr2=0x7f5b440fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:03.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 shutdown_connections 2024-04-20T11:03:03.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:03.631+0000 7f5b49c00700 1 -- 172.21.15.117:0/4071166154 wait complete. 2024-04-20T11:03:03.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:03 smithi117 ceph-mon[15659]: Detected new or changed devices on smithi173 2024-04-20T11:03:03.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:03 smithi117 ceph-mon[15659]: Adjusting osd_memory_target on smithi173 to 4971M 2024-04-20T11:03:03.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:03 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/4071166154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:03:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:03 smithi173 ceph-mon[23800]: Detected new or changed devices on smithi173 2024-04-20T11:03:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:03 smithi173 ceph-mon[23800]: Adjusting osd_memory_target on smithi173 to 4971M 2024-04-20T11:03:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:03 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/4071166154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:03:04.339 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:01:14.283466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:03:04.339 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-20T11:03:04.743 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:05.061 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:04 smithi117 ceph-mon[15659]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:05.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:04 smithi173 ceph-mon[23800]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:06.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.944+0000 7f1877c00700 1 -- 172.21.15.117:0/3331956146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870104a00 msgr2=0x7f1870104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:06.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.944+0000 7f1877c00700 1 --2- 172.21.15.117:0/3331956146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870104a00 0x7f1870104e80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f185c009910 tx=0x7f185c009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:06.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.945+0000 7f1877c00700 1 -- 172.21.15.117:0/3331956146 shutdown_connections 2024-04-20T11:03:06.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.945+0000 7f1877c00700 1 --2- 172.21.15.117:0/3331956146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870104a00 0x7f1870104e80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:06.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.945+0000 7f1877c00700 1 --2- 172.21.15.117:0/3331956146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1870103110 0x7f18701034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:06.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.945+0000 7f1877c00700 1 -- 172.21.15.117:0/3331956146 >> 172.21.15.117:0/3331956146 conn(0x7f18700fe880 msgr2=0x7f1870100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:06.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.945+0000 7f1877c00700 1 -- 172.21.15.117:0/3331956146 shutdown_connections 2024-04-20T11:03:06.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.945+0000 7f1877c00700 1 -- 172.21.15.117:0/3331956146 wait complete. 2024-04-20T11:03:06.947 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:06 smithi117 ceph-mon[15659]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.946+0000 7f1877c00700 1 Processor -- start 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.946+0000 7f1877c00700 1 -- start start 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.947+0000 7f1877c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870103110 0x7f187019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.947+0000 7f1877c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1870104a00 0x7f187019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.947+0000 7f1877c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187019f5b0 con 0x7f1870103110 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.947+0000 7f1877c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1870198b90 con 0x7f1870104a00 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.947+0000 7f1875200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870103110 0x7f187019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.947+0000 7f1875200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870103110 0x7f187019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45818/0 (socket says 172.21.15.117:45818) 2024-04-20T11:03:06.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.947+0000 7f1875200700 1 -- 172.21.15.117:0/4050830514 learned_addr learned my addr 172.21.15.117:0/4050830514 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:06.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.947+0000 7f186fe00700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1870104a00 0x7f187019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:06.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f1875200700 1 -- 172.21.15.117:0/4050830514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1870104a00 msgr2=0x7f187019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:06.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f1875200700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1870104a00 0x7f187019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:06.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f1875200700 1 -- 172.21.15.117:0/4050830514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c0095f0 con 0x7f1870103110 2024-04-20T11:03:06.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f1875200700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870103110 0x7f187019ea50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f186400c8a0 tx=0x7f186400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:06.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f186d600700 1 -- 172.21.15.117:0/4050830514 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f186400cea0 con 0x7f1870103110 2024-04-20T11:03:06.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1870198e70 con 0x7f1870103110 2024-04-20T11:03:06.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f186d600700 1 -- 172.21.15.117:0/4050830514 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1864004830 con 0x7f1870103110 2024-04-20T11:03:06.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f186d600700 1 -- 172.21.15.117:0/4050830514 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18640055b0 con 0x7f1870103110 2024-04-20T11:03:06.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.948+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1870199440 con 0x7f1870103110 2024-04-20T11:03:06.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.950+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187010b480 con 0x7f1870103110 2024-04-20T11:03:06.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.952+0000 7f186d600700 1 -- 172.21.15.117:0/4050830514 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f18640049a0 con 0x7f1870103110 2024-04-20T11:03:06.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.953+0000 7f186d600700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f185806ccf0 0x7f185806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:06.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.953+0000 7f186d600700 1 -- 172.21.15.117:0/4050830514 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f186408bfa0 con 0x7f1870103110 2024-04-20T11:03:06.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.953+0000 7f186fe00700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f185806ccf0 0x7f185806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:06.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.954+0000 7f186fe00700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f185806ccf0 0x7f185806f1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f185c006010 tx=0x7f185c00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:06.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:06.957+0000 7f186d600700 1 -- 172.21.15.117:0/4050830514 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f186405a260 con 0x7f1870103110 2024-04-20T11:03:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:06 smithi173 ceph-mon[23800]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:07.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.177+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1870199fe0 con 0x7f1870103110 2024-04-20T11:03:07.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.178+0000 7f186d600700 1 -- 172.21.15.117:0/4050830514 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f186401d0a0 con 0x7f1870103110 2024-04-20T11:03:07.179 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-04-20T11:03:07.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.180+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f185806ccf0 msgr2=0x7f185806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:07.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.180+0000 7f1877c00700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f185806ccf0 0x7f185806f1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f185c006010 tx=0x7f185c00b540 comp rx=0 tx=0).stop 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.180+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870103110 msgr2=0x7f187019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.180+0000 7f1877c00700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870103110 0x7f187019ea50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f186400c8a0 tx=0x7f186400cc60 comp rx=0 tx=0).stop 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.181+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 shutdown_connections 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.181+0000 7f1877c00700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f185806ccf0 0x7f185806f1b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.181+0000 7f1877c00700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1870103110 0x7f187019ea50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.181+0000 7f1877c00700 1 --2- 172.21.15.117:0/4050830514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1870104a00 0x7f187019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.181+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 >> 172.21.15.117:0/4050830514 conn(0x7f18700fe880 msgr2=0x7f18700ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.181+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 shutdown_connections 2024-04-20T11:03:07.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:07.181+0000 7f1877c00700 1 -- 172.21.15.117:0/4050830514 wait complete. 2024-04-20T11:03:07.850 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:07 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/4050830514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:03:07.908 INFO:tasks.cephadm:Setting up client nodes... 2024-04-20T11:03:07.908 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-20T11:03:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:07 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/4050830514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:03:08.312 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:08.829 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:08 smithi117 ceph-mon[15659]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:08 smithi173 ceph-mon[23800]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:10.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.479+0000 7f9c82200700 1 -- 172.21.15.117:0/1461134786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c075890 msgr2=0x7f9c7c075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:10.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.479+0000 7f9c82200700 1 --2- 172.21.15.117:0/1461134786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c075890 0x7f9c7c075cb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009910 tx=0x7f9c70009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:10.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.479+0000 7f9c82200700 1 -- 172.21.15.117:0/1461134786 shutdown_connections 2024-04-20T11:03:10.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.479+0000 7f9c82200700 1 --2- 172.21.15.117:0/1461134786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c7c076ae0 0x7f9c7c076f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:10.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.479+0000 7f9c82200700 1 --2- 172.21.15.117:0/1461134786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c075890 0x7f9c7c075cb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:10.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.479+0000 7f9c82200700 1 -- 172.21.15.117:0/1461134786 >> 172.21.15.117:0/1461134786 conn(0x7f9c7c0fe1d0 msgr2=0x7f9c7c100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:10.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.479+0000 7f9c82200700 1 -- 172.21.15.117:0/1461134786 shutdown_connections 2024-04-20T11:03:10.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.479+0000 7f9c82200700 1 -- 172.21.15.117:0/1461134786 wait complete. 2024-04-20T11:03:10.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.480+0000 7f9c82200700 1 Processor -- start 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.480+0000 7f9c82200700 1 -- start start 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.480+0000 7f9c82200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c7c075890 0x7f9c7c19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.480+0000 7f9c82200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c076ae0 0x7f9c7c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c82200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c19d560 con 0x7f9c7c076ae0 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c82200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c19d6a0 con 0x7f9c7c075890 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c076ae0 0x7f9c7c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c076ae0 0x7f9c7c19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45838/0 (socket says 172.21.15.117:45838) 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c7be00700 1 -- 172.21.15.117:0/1069590703 learned_addr learned my addr 172.21.15.117:0/1069590703 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c80e00700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c7c075890 0x7f9c7c19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c7be00700 1 -- 172.21.15.117:0/1069590703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c7c075890 msgr2=0x7f9c7c19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c7be00700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c7c075890 0x7f9c7c19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:10.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.481+0000 7f9c7be00700 1 -- 172.21.15.117:0/1069590703 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c700095f0 con 0x7f9c7c076ae0 2024-04-20T11:03:10.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.482+0000 7f9c7be00700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c076ae0 0x7f9c7c19cf40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00d8d0 tx=0x7f9c6c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:10.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.482+0000 7f9c79600700 1 -- 172.21.15.117:0/1069590703 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6c009940 con 0x7f9c7c076ae0 2024-04-20T11:03:10.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.482+0000 7f9c79600700 1 -- 172.21.15.117:0/1069590703 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c6c010460 con 0x7f9c7c076ae0 2024-04-20T11:03:10.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.482+0000 7f9c79600700 1 -- 172.21.15.117:0/1069590703 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6c00f5f0 con 0x7f9c7c076ae0 2024-04-20T11:03:10.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.482+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7c1a2100 con 0x7f9c7c076ae0 2024-04-20T11:03:10.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.482+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7c1a26d0 con 0x7f9c7c076ae0 2024-04-20T11:03:10.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.483+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7c196ba0 con 0x7f9c7c076ae0 2024-04-20T11:03:10.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.484+0000 7f9c79600700 1 -- 172.21.15.117:0/1069590703 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f9c6c0105d0 con 0x7f9c7c076ae0 2024-04-20T11:03:10.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.485+0000 7f9c79600700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9c6406cce0 0x7f9c6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:10.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.485+0000 7f9c79600700 1 -- 172.21.15.117:0/1069590703 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9c6c08bef0 con 0x7f9c7c076ae0 2024-04-20T11:03:10.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.485+0000 7f9c80e00700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9c6406cce0 0x7f9c6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:10.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.486+0000 7f9c80e00700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9c6406cce0 0x7f9c6406f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c700051c0 tx=0x7f9c70005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:10.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.490+0000 7f9c79600700 1 -- 172.21.15.117:0/1069590703 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9c6c05a070 con 0x7f9c7c076ae0 2024-04-20T11:03:10.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.800+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f9c7c04f400 con 0x7f9c7c076ae0 2024-04-20T11:03:10.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.809+0000 7f9c79600700 1 -- 172.21.15.117:0/1069590703 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f9c6c0160f0 con 0x7f9c7c076ae0 2024-04-20T11:03:10.809 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-04-20T11:03:10.810 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDuoCNmYvbELxAADXdra62qyjj7fFI2DltcEQ== 2024-04-20T11:03:10.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9c6406cce0 msgr2=0x7f9c6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:10.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9c6406cce0 0x7f9c6406f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c700051c0 tx=0x7f9c70005a90 comp rx=0 tx=0).stop 2024-04-20T11:03:10.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c076ae0 msgr2=0x7f9c7c19cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:10.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c076ae0 0x7f9c7c19cf40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00d8d0 tx=0x7f9c6c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:03:10.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 shutdown_connections 2024-04-20T11:03:10.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9c6406cce0 0x7f9c6406f1a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:10.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c7c075890 0x7f9c7c19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:10.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 --2- 172.21.15.117:0/1069590703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c7c076ae0 0x7f9c7c19cf40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:10.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 >> 172.21.15.117:0/1069590703 conn(0x7f9c7c0fe1d0 msgr2=0x7f9c7c10d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:10.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.812+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 shutdown_connections 2024-04-20T11:03:10.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:10.813+0000 7f9c82200700 1 -- 172.21.15.117:0/1069590703 wait complete. 2024-04-20T11:03:11.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:10 smithi117 ceph-mon[15659]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:10 smithi173 ceph-mon[23800]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:11.585 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T11:03:11.586 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-20T11:03:11.586 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T11:03:11.707 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-20T11:03:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:11 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1069590703' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:03:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:11 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1069590703' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T11:03:12.115 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:11 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1069590703' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:03:12.115 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:11 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1069590703' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T11:03:12.122 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:03:13.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:12 smithi117 ceph-mon[15659]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:12 smithi173 ceph-mon[23800]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:13 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:03:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:03:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.422+0000 7f600d600700 1 -- 172.21.15.173:0/2160790544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 msgr2=0x7f6008073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.422+0000 7f600d600700 1 --2- 172.21.15.173:0/2160790544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 0x7f6008073dc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009910 tx=0x7f5ff0009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.423+0000 7f600d600700 1 -- 172.21.15.173:0/2160790544 shutdown_connections 2024-04-20T11:03:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.423+0000 7f600d600700 1 --2- 172.21.15.173:0/2160790544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 0x7f6008073dc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.423+0000 7f600d600700 1 --2- 172.21.15.173:0/2160790544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008074f10 0x7f6008073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.423+0000 7f600d600700 1 -- 172.21.15.173:0/2160790544 >> 172.21.15.173:0/2160790544 conn(0x7f60080fc1c0 msgr2=0x7f60080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.423+0000 7f600d600700 1 -- 172.21.15.173:0/2160790544 shutdown_connections 2024-04-20T11:03:14.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.423+0000 7f600d600700 1 -- 172.21.15.173:0/2160790544 wait complete. 2024-04-20T11:03:14.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.424+0000 7f600d600700 1 Processor -- start 2024-04-20T11:03:14.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.424+0000 7f600d600700 1 -- start start 2024-04-20T11:03:14.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.424+0000 7f600d600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 0x7f6008198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.424+0000 7f600d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008074f10 0x7f6008198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.424+0000 7f600d600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008199300 con 0x7f6008074f10 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.424+0000 7f600d600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008199440 con 0x7f6008073940 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.424+0000 7f6006a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 0x7f6008198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.425+0000 7f6006a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 0x7f6008198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:35776/0 (socket says 172.21.15.173:35776) 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.425+0000 7f6006a00700 1 -- 172.21.15.173:0/3471679216 learned_addr learned my addr 172.21.15.173:0/3471679216 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.425+0000 7f5ffe000700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008074f10 0x7f6008198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.425+0000 7f6006a00700 1 -- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008074f10 msgr2=0x7f6008198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.425+0000 7f6006a00700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008074f10 0x7f6008198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:14.426 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.425+0000 7f6006a00700 1 -- 172.21.15.173:0/3471679216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00095f0 con 0x7f6008073940 2024-04-20T11:03:14.427 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.425+0000 7f6006a00700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 0x7f6008198830 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800c960 tx=0x7f5ff800cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:14.427 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.426+0000 7f5fffe00700 1 -- 172.21.15.173:0/3471679216 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ff8007a70 con 0x7f6008073940 2024-04-20T11:03:14.427 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.426+0000 7f5fffe00700 1 -- 172.21.15.173:0/3471679216 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ff8007bd0 con 0x7f6008073940 2024-04-20T11:03:14.427 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.426+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600819deb0 con 0x7f6008073940 2024-04-20T11:03:14.427 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.426+0000 7f5fffe00700 1 -- 172.21.15.173:0/3471679216 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ff80186c0 con 0x7f6008073940 2024-04-20T11:03:14.427 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.426+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600819e3f0 con 0x7f6008073940 2024-04-20T11:03:14.428 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.427+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60081929d0 con 0x7f6008073940 2024-04-20T11:03:14.428 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.428+0000 7f5fffe00700 1 -- 172.21.15.173:0/3471679216 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f5ff800faa0 con 0x7f6008073940 2024-04-20T11:03:14.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.429+0000 7f5fffe00700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5fe806cce0 0x7f5fe806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:14.430 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.429+0000 7f5fffe00700 1 -- 172.21.15.173:0/3471679216 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5ff808bf30 con 0x7f6008073940 2024-04-20T11:03:14.430 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.430+0000 7f5ffe000700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5fe806cce0 0x7f5fe806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:14.431 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.431+0000 7f5ffe000700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5fe806cce0 0x7f5fe806f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0006010 tx=0x7f5ff000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:14.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.434+0000 7f5fffe00700 1 -- 172.21.15.173:0/3471679216 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5ff8059e20 con 0x7f6008073940 2024-04-20T11:03:14.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.743+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f600804ebb0 con 0x7f6008073940 2024-04-20T11:03:14.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.754+0000 7f5fffe00700 1 -- 172.21.15.173:0/3471679216 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5ff8059c40 con 0x7f6008073940 2024-04-20T11:03:14.755 INFO:teuthology.orchestra.run.smithi173.stdout:[client.1] 2024-04-20T11:03:14.755 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQDyoCNmmo2HLBAA5OmGktASjEBLrgilEFZsxg== 2024-04-20T11:03:14.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5fe806cce0 msgr2=0x7f5fe806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:14.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5fe806cce0 0x7f5fe806f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0006010 tx=0x7f5ff000b540 comp rx=0 tx=0).stop 2024-04-20T11:03:14.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 msgr2=0x7f6008198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:14.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 0x7f6008198830 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800c960 tx=0x7f5ff800cd20 comp rx=0 tx=0).stop 2024-04-20T11:03:14.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 shutdown_connections 2024-04-20T11:03:14.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5fe806cce0 0x7f5fe806f1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:14.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6008073940 0x7f6008198830 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:14.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 --2- 172.21.15.173:0/3471679216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008074f10 0x7f6008198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:14.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.757+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 >> 172.21.15.173:0/3471679216 conn(0x7f60080fc1c0 msgr2=0x7f6008106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:14.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.758+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 shutdown_connections 2024-04-20T11:03:14.759 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:03:14.758+0000 7f600d600700 1 -- 172.21.15.173:0/3471679216 wait complete. 2024-04-20T11:03:15.103 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:14 smithi173 ceph-mon[23800]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:15.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:14 smithi117 ceph-mon[15659]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:15.587 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T11:03:15.587 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-20T11:03:15.587 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-20T11:03:15.716 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T11:03:15.716 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-20T11:03:15.716 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph mgr dump --format=json 2024-04-20T11:03:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:15 smithi173 ceph-mon[23800]: from='client.? 172.21.15.173:0/3471679216' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:03:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:15 smithi173 ceph-mon[23800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:03:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:15 smithi173 ceph-mon[23800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T11:03:16.104 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:16.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:15 smithi117 ceph-mon[15659]: from='client.? 172.21.15.173:0/3471679216' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:03:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:15 smithi117 ceph-mon[15659]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:03:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:15 smithi117 ceph-mon[15659]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T11:03:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:16 smithi117 ceph-mon[15659]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:17.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:16 smithi173 ceph-mon[23800]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.167+0000 7ff719400700 1 -- 172.21.15.117:0/3325477498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 msgr2=0x7ff7141034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.167+0000 7ff719400700 1 --2- 172.21.15.117:0/3325477498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 0x7ff7141034f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff704009910 tx=0x7ff704009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.167+0000 7ff719400700 1 -- 172.21.15.117:0/3325477498 shutdown_connections 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.167+0000 7ff719400700 1 --2- 172.21.15.117:0/3325477498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff714104a00 0x7ff714104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.167+0000 7ff719400700 1 --2- 172.21.15.117:0/3325477498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 0x7ff7141034f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.167+0000 7ff719400700 1 -- 172.21.15.117:0/3325477498 >> 172.21.15.117:0/3325477498 conn(0x7ff7140fe880 msgr2=0x7ff714100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.167+0000 7ff719400700 1 -- 172.21.15.117:0/3325477498 shutdown_connections 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.167+0000 7ff719400700 1 -- 172.21.15.117:0/3325477498 wait complete. 2024-04-20T11:03:18.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.168+0000 7ff719400700 1 Processor -- start 2024-04-20T11:03:18.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.168+0000 7ff719400700 1 -- start start 2024-04-20T11:03:18.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.169+0000 7ff719400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 0x7ff714198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:18.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.169+0000 7ff719400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff714104a00 0x7ff714199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:18.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.169+0000 7ff719400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7141999e0 con 0x7ff714103110 2024-04-20T11:03:18.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.169+0000 7ff719400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71419d660 con 0x7ff714104a00 2024-04-20T11:03:18.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.169+0000 7ff712800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 0x7ff714198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:18.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.169+0000 7ff712800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 0x7ff714198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50148/0 (socket says 172.21.15.117:50148) 2024-04-20T11:03:18.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.169+0000 7ff712800700 1 -- 172.21.15.117:0/3479477609 learned_addr learned my addr 172.21.15.117:0/3479477609 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff711e00700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff714104a00 0x7ff714199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff712800700 1 -- 172.21.15.117:0/3479477609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff714104a00 msgr2=0x7ff714199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff712800700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff714104a00 0x7ff714199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff712800700 1 -- 172.21.15.117:0/3479477609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7040095f0 con 0x7ff714103110 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff712800700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 0x7ff714198dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff704000c00 tx=0x7ff70400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff70b400700 1 -- 172.21.15.117:0/3479477609 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff70401d070 con 0x7ff714103110 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff70b400700 1 -- 172.21.15.117:0/3479477609 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7040056e0 con 0x7ff714103110 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff70b400700 1 -- 172.21.15.117:0/3479477609 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff70400fe80 con 0x7ff714103110 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71419d8e0 con 0x7ff714103110 2024-04-20T11:03:18.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.170+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71419de50 con 0x7ff714103110 2024-04-20T11:03:18.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.171+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71410b480 con 0x7ff714103110 2024-04-20T11:03:18.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.172+0000 7ff70b400700 1 -- 172.21.15.117:0/3479477609 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ff70400f6f0 con 0x7ff714103110 2024-04-20T11:03:18.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.173+0000 7ff70b400700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff6f806ccf0 0x7ff6f806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:18.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.173+0000 7ff70b400700 1 -- 172.21.15.117:0/3479477609 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff70408d2f0 con 0x7ff714103110 2024-04-20T11:03:18.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.174+0000 7ff711e00700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff6f806ccf0 0x7ff6f806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:18.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.174+0000 7ff711e00700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff6f806ccf0 0x7ff6f806f1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff700005950 tx=0x7ff7000058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:18.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.178+0000 7ff70b400700 1 -- 172.21.15.117:0/3479477609 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff7040579e0 con 0x7ff714103110 2024-04-20T11:03:18.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.464+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff71419a080 con 0x7ff714103110 2024-04-20T11:03:18.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.474+0000 7ff70b400700 1 -- 172.21.15.117:0/3479477609 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+174020 (secure 0 0 0) 0x7ff70405b000 con 0x7ff714103110 2024-04-20T11:03:18.477 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff6f806ccf0 msgr2=0x7ff6f806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff6f806ccf0 0x7ff6f806f1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff700005950 tx=0x7ff7000058e0 comp rx=0 tx=0).stop 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 msgr2=0x7ff714198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 0x7ff714198dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff704000c00 tx=0x7ff70400be20 comp rx=0 tx=0).stop 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 shutdown_connections 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff6f806ccf0 0x7ff6f806f1b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714103110 0x7ff714198dc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 --2- 172.21.15.117:0/3479477609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff714104a00 0x7ff714199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:18.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 >> 172.21.15.117:0/3479477609 conn(0x7ff7140fe880 msgr2=0x7ff7140ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:18.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 shutdown_connections 2024-04-20T11:03:18.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:18.479+0000 7ff719400700 1 -- 172.21.15.117:0/3479477609 wait complete. 2024-04-20T11:03:18.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:18 smithi117 ceph-mon[15659]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:18.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:18 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3479477609' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T11:03:19.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:18 smithi173 ceph-mon[23800]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:19.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:18 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3479477609' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T11:03:19.275 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":19,"flags":0,"active_gid":14215,"active_name":"smithi117.ivnklj","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2612478075},{"type":"v1","addr":"172.21.15.117:6801","nonce":2612478075}]},"active_addr":"172.21.15.117:6801/2612478075","active_change":"2024-04-20T10:58:13.447367+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi173.gnrjof","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.117:8443/","prometheus":"http://172.21.15.117:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":4182681741}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":519807094}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":2961758989}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":571056469}]}]} 2024-04-20T11:03:19.279 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-20T11:03:19.279 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-20T11:03:19.279 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T11:03:19.663 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:21.033 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:20 smithi117 ceph-mon[15659]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:21.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:20 smithi173 ceph-mon[23800]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:21.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.799+0000 7fbbd3200700 1 -- 172.21.15.117:0/1429534700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 msgr2=0x7fbbcc10ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:21.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.799+0000 7fbbd3200700 1 --2- 172.21.15.117:0/1429534700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 0x7fbbcc10ceb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009910 tx=0x7fbbb4009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:21.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.799+0000 7fbbd3200700 1 -- 172.21.15.117:0/1429534700 shutdown_connections 2024-04-20T11:03:21.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.799+0000 7fbbd3200700 1 --2- 172.21.15.117:0/1429534700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 0x7fbbcc10ceb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:21.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.799+0000 7fbbd3200700 1 --2- 172.21.15.117:0/1429534700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbcc073310 0x7fbbcc0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:21.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.799+0000 7fbbd3200700 1 -- 172.21.15.117:0/1429534700 >> 172.21.15.117:0/1429534700 conn(0x7fbbcc0fc6c0 msgr2=0x7fbbcc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:21.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.799+0000 7fbbd3200700 1 -- 172.21.15.117:0/1429534700 shutdown_connections 2024-04-20T11:03:21.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.799+0000 7fbbd3200700 1 -- 172.21.15.117:0/1429534700 wait complete. 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbd3200700 1 Processor -- start 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbd3200700 1 -- start start 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbd3200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbcc073310 0x7fbbcc198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbd3200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 0x7fbbcc199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbd3200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc199a00 con 0x7fbbcc073c30 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbd3200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc19d6d0 con 0x7fbbcc073310 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbc3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 0x7fbbcc199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbc3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 0x7fbbcc199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50158/0 (socket says 172.21.15.117:50158) 2024-04-20T11:03:21.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.801+0000 7fbbc3400700 1 -- 172.21.15.117:0/629734370 learned_addr learned my addr 172.21.15.117:0/629734370 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:21.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.802+0000 7fbbcbe00700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbcc073310 0x7fbbcc198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:21.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.802+0000 7fbbc3400700 1 -- 172.21.15.117:0/629734370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbcc073310 msgr2=0x7fbbcc198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:21.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.802+0000 7fbbc3400700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbcc073310 0x7fbbcc198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:21.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.802+0000 7fbbc3400700 1 -- 172.21.15.117:0/629734370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb40095f0 con 0x7fbbcc073c30 2024-04-20T11:03:21.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.802+0000 7fbbc3400700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 0x7fbbcc199320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40092e0 tx=0x7fbbb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:21.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.802+0000 7fbbc9600700 1 -- 172.21.15.117:0/629734370 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbb401d070 con 0x7fbbcc073c30 2024-04-20T11:03:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.802+0000 7fbbc9600700 1 -- 172.21.15.117:0/629734370 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbb400f460 con 0x7fbbcc073c30 2024-04-20T11:03:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.802+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc19d950 con 0x7fbbcc073c30 2024-04-20T11:03:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.803+0000 7fbbc9600700 1 -- 172.21.15.117:0/629734370 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbb4017630 con 0x7fbbcc073c30 2024-04-20T11:03:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.803+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc19de40 con 0x7fbbcc073c30 2024-04-20T11:03:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.804+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbcc0749c0 con 0x7fbbcc073c30 2024-04-20T11:03:21.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.805+0000 7fbbc9600700 1 -- 172.21.15.117:0/629734370 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fbbb400f5d0 con 0x7fbbcc073c30 2024-04-20T11:03:21.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.806+0000 7fbbc9600700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbbb006cce0 0x7fbbb006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:21.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.806+0000 7fbbc9600700 1 -- 172.21.15.117:0/629734370 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbbb408d410 con 0x7fbbcc073c30 2024-04-20T11:03:21.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.806+0000 7fbbcbe00700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbbb006cce0 0x7fbbb006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:21.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.807+0000 7fbbcbe00700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbbb006cce0 0x7fbbb006f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006fd0 tx=0x7fbbbc008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:21.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:21.811+0000 7fbbc9600700 1 -- 172.21.15.117:0/629734370 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbbb4057b00 con 0x7fbbcc073c30 2024-04-20T11:03:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:21 smithi117 ceph-mon[15659]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:22.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.032+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbcc107750 con 0x7fbbcc073c30 2024-04-20T11:03:22.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.034+0000 7fbbc9600700 1 -- 172.21.15.117:0/629734370 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+11402 (secure 0 0 0) 0x7fbbb405b120 con 0x7fbbcc073c30 2024-04-20T11:03:22.035 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:03:22.035 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":42,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","created":"2024-04-20T10:55:31.778050+0000","modified":"2024-04-20T11:02:59.647082+0000","last_up_change":"2024-04-20T11:02:58.642045+0000","last_in_change":"2024-04-20T11:02:33.545131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:01:14.283466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6803","nonce":2292675302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6805","nonce":2292675302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6809","nonce":2292675302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6807","nonce":2292675302}]},"public_addr":"172.21.15.117:6803/2292675302","cluster_addr":"172.21.15.117:6805/2292675302","heartbeat_back_addr":"172.21.15.117:6809/2292675302","heartbeat_front_addr":"172.21.15.117:6807/2292675302","state":["exists","up"]},{"osd":1,"uuid":"75ea37ae-266a-459b-a32b-e1d91927cbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6811","nonce":2871230534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6813","nonce":2871230534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6817","nonce":2871230534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6815","nonce":2871230534}]},"public_addr":"172.21.15.117:6811/2871230534","cluster_addr":"172.21.15.117:6813/2871230534","heartbeat_back_addr":"172.21.15.117:6817/2871230534","heartbeat_front_addr":"172.21.15.117:6815/2871230534","state":["exists","up"]},{"osd":2,"uuid":"2323b6ba-7d04-4744-b61f-4936e4f419bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6819","nonce":3091267150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6821","nonce":3091267150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6825","nonce":3091267150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6823","nonce":3091267150}]},"public_addr":"172.21.15.117:6819/3091267150","cluster_addr":"172.21.15.117:6821/3091267150","heartbeat_back_addr":"172.21.15.117:6825/3091267150","heartbeat_front_addr":"172.21.15.117:6823/3091267150","state":["exists","up"]},{"osd":3,"uuid":"b87d3df6-0d27-4e2d-9e40-55895e0a891d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6801","nonce":2587001742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6803","nonce":2587001742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6807","nonce":2587001742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6805","nonce":2587001742}]},"public_addr":"172.21.15.173:6801/2587001742","cluster_addr":"172.21.15.173:6803/2587001742","heartbeat_back_addr":"172.21.15.173:6807/2587001742","heartbeat_front_addr":"172.21.15.173:6805/2587001742","state":["exists","up"]},{"osd":4,"uuid":"c5ceed42-a923-47e4-919d-98caab4e06ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6809","nonce":1468348921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6811","nonce":1468348921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6815","nonce":1468348921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6813","nonce":1468348921}]},"public_addr":"172.21.15.173:6809/1468348921","cluster_addr":"172.21.15.173:6811/1468348921","heartbeat_back_addr":"172.21.15.173:6815/1468348921","heartbeat_front_addr":"172.21.15.173:6813/1468348921","state":["exists","up"]},{"osd":5,"uuid":"47b99706-a27f-4f30-9e6d-619ee6a35ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6817","nonce":1909838821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6819","nonce":1909838821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6823","nonce":1909838821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6821","nonce":1909838821}]},"public_addr":"172.21.15.173:6817/1909838821","cluster_addr":"172.21.15.173:6819/1909838821","heartbeat_back_addr":"172.21.15.173:6823/1909838821","heartbeat_front_addr":"172.21.15.173:6821/1909838821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:59:57.050973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:00:33.939553+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:11.618080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:46.287328+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:20.465163+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:56.550872+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/3713452282":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6800/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1089633358":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6801/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6801/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1807203325":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6800/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1384022397":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2096654470":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/4088938050":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/1375611439":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2304890710":"2024-04-21T10:56:04.789705+0000","172.21.15.117:6800/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2230602257":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6801/1432319225":"2024-04-21T10:56:04.789705+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbbb006cce0 msgr2=0x7fbbb006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbbb006cce0 0x7fbbb006f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006fd0 tx=0x7fbbbc008040 comp rx=0 tx=0).stop 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 msgr2=0x7fbbcc199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 0x7fbbcc199320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40092e0 tx=0x7fbbb400be20 comp rx=0 tx=0).stop 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 shutdown_connections 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbbb006cce0 0x7fbbb006f1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbcc073310 0x7fbbcc198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 --2- 172.21.15.117:0/629734370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbcc073c30 0x7fbbcc199320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 >> 172.21.15.117:0/629734370 conn(0x7fbbcc0fc6c0 msgr2=0x7fbbcc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 shutdown_connections 2024-04-20T11:03:22.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:22.037+0000 7fbbd3200700 1 -- 172.21.15.117:0/629734370 wait complete. 2024-04-20T11:03:22.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:21 smithi173 ceph-mon[23800]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:22.721 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-20T11:03:22.721 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T11:03:23.120 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:23.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:22 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/629734370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:03:23.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:22 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/629734370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:03:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:23 smithi117 ceph-mon[15659]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:24.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:23 smithi173 ceph-mon[23800]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:25.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.242+0000 7f09e1400700 1 -- 172.21.15.117:0/1852947045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 msgr2=0x7f09dc10cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.242+0000 7f09e1400700 1 --2- 172.21.15.117:0/1852947045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 0x7f09dc10cf10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009910 tx=0x7f09c8009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.242+0000 7f09e1400700 1 -- 172.21.15.117:0/1852947045 shutdown_connections 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.242+0000 7f09e1400700 1 --2- 172.21.15.117:0/1852947045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 0x7f09dc10cf10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.242+0000 7f09e1400700 1 --2- 172.21.15.117:0/1852947045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09dc0732b0 0x7f09dc073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.242+0000 7f09e1400700 1 -- 172.21.15.117:0/1852947045 >> 172.21.15.117:0/1852947045 conn(0x7f09dc0fc6a0 msgr2=0x7f09dc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.242+0000 7f09e1400700 1 -- 172.21.15.117:0/1852947045 shutdown_connections 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.242+0000 7f09e1400700 1 -- 172.21.15.117:0/1852947045 wait complete. 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.243+0000 7f09e1400700 1 Processor -- start 2024-04-20T11:03:25.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.243+0000 7f09e1400700 1 -- start start 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.243+0000 7f09e1400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09dc0732b0 0x7f09dc198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09e1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 0x7f09dc199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09e1400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc199840 con 0x7f09dc073bd0 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09e1400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc1999b0 con 0x7f09dc0732b0 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09d9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 0x7f09dc199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09d9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 0x7f09dc199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33736/0 (socket says 172.21.15.117:33736) 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09d9e00700 1 -- 172.21.15.117:0/1949958168 learned_addr learned my addr 172.21.15.117:0/1949958168 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09da800700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09dc0732b0 0x7f09dc198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09d9e00700 1 -- 172.21.15.117:0/1949958168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09dc0732b0 msgr2=0x7f09dc198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09d9e00700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09dc0732b0 0x7f09dc198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.244+0000 7f09d9e00700 1 -- 172.21.15.117:0/1949958168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80095f0 con 0x7f09dc073bd0 2024-04-20T11:03:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.245+0000 7f09d9e00700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 0x7f09dc199240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f09c80092e0 tx=0x7f09c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:25.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.245+0000 7f09d3400700 1 -- 172.21.15.117:0/1949958168 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c801d070 con 0x7f09dc073bd0 2024-04-20T11:03:25.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.245+0000 7f09d3400700 1 -- 172.21.15.117:0/1949958168 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09c800f460 con 0x7f09dc073bd0 2024-04-20T11:03:25.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.245+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc19c630 con 0x7f09dc073bd0 2024-04-20T11:03:25.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.245+0000 7f09d3400700 1 -- 172.21.15.117:0/1949958168 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c8017630 con 0x7f09dc073bd0 2024-04-20T11:03:25.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.245+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc19cb20 con 0x7f09dc073bd0 2024-04-20T11:03:25.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.246+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc074960 con 0x7f09dc073bd0 2024-04-20T11:03:25.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.247+0000 7f09d3400700 1 -- 172.21.15.117:0/1949958168 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f09c800f5d0 con 0x7f09dc073bd0 2024-04-20T11:03:25.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.248+0000 7f09d3400700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f09c006ccf0 0x7f09c006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:25.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.248+0000 7f09d3400700 1 -- 172.21.15.117:0/1949958168 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f09c808d4c0 con 0x7f09dc073bd0 2024-04-20T11:03:25.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.249+0000 7f09da800700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f09c006ccf0 0x7f09c006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:25.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.249+0000 7f09da800700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f09c006ccf0 0x7f09c006f1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f09cc005ea0 tx=0x7f09cc005e30 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:25.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.253+0000 7f09d3400700 1 -- 172.21.15.117:0/1949958168 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f09c8057b30 con 0x7f09dc073bd0 2024-04-20T11:03:25.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.478+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09dc04ebb0 con 0x7f09dc073bd0 2024-04-20T11:03:25.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.480+0000 7f09d3400700 1 -- 172.21.15.117:0/1949958168 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+11402 (secure 0 0 0) 0x7f09c805b150 con 0x7f09dc073bd0 2024-04-20T11:03:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:03:25.480 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":42,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","created":"2024-04-20T10:55:31.778050+0000","modified":"2024-04-20T11:02:59.647082+0000","last_up_change":"2024-04-20T11:02:58.642045+0000","last_in_change":"2024-04-20T11:02:33.545131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:01:14.283466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6803","nonce":2292675302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6805","nonce":2292675302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6809","nonce":2292675302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6807","nonce":2292675302}]},"public_addr":"172.21.15.117:6803/2292675302","cluster_addr":"172.21.15.117:6805/2292675302","heartbeat_back_addr":"172.21.15.117:6809/2292675302","heartbeat_front_addr":"172.21.15.117:6807/2292675302","state":["exists","up"]},{"osd":1,"uuid":"75ea37ae-266a-459b-a32b-e1d91927cbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6811","nonce":2871230534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6813","nonce":2871230534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6817","nonce":2871230534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6815","nonce":2871230534}]},"public_addr":"172.21.15.117:6811/2871230534","cluster_addr":"172.21.15.117:6813/2871230534","heartbeat_back_addr":"172.21.15.117:6817/2871230534","heartbeat_front_addr":"172.21.15.117:6815/2871230534","state":["exists","up"]},{"osd":2,"uuid":"2323b6ba-7d04-4744-b61f-4936e4f419bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6819","nonce":3091267150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6821","nonce":3091267150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6825","nonce":3091267150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6823","nonce":3091267150}]},"public_addr":"172.21.15.117:6819/3091267150","cluster_addr":"172.21.15.117:6821/3091267150","heartbeat_back_addr":"172.21.15.117:6825/3091267150","heartbeat_front_addr":"172.21.15.117:6823/3091267150","state":["exists","up"]},{"osd":3,"uuid":"b87d3df6-0d27-4e2d-9e40-55895e0a891d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6801","nonce":2587001742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6803","nonce":2587001742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6807","nonce":2587001742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6805","nonce":2587001742}]},"public_addr":"172.21.15.173:6801/2587001742","cluster_addr":"172.21.15.173:6803/2587001742","heartbeat_back_addr":"172.21.15.173:6807/2587001742","heartbeat_front_addr":"172.21.15.173:6805/2587001742","state":["exists","up"]},{"osd":4,"uuid":"c5ceed42-a923-47e4-919d-98caab4e06ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6809","nonce":1468348921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6811","nonce":1468348921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6815","nonce":1468348921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6813","nonce":1468348921}]},"public_addr":"172.21.15.173:6809/1468348921","cluster_addr":"172.21.15.173:6811/1468348921","heartbeat_back_addr":"172.21.15.173:6815/1468348921","heartbeat_front_addr":"172.21.15.173:6813/1468348921","state":["exists","up"]},{"osd":5,"uuid":"47b99706-a27f-4f30-9e6d-619ee6a35ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6817","nonce":1909838821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6819","nonce":1909838821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6823","nonce":1909838821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6821","nonce":1909838821}]},"public_addr":"172.21.15.173:6817/1909838821","cluster_addr":"172.21.15.173:6819/1909838821","heartbeat_back_addr":"172.21.15.173:6823/1909838821","heartbeat_front_addr":"172.21.15.173:6821/1909838821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:59:57.050973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:00:33.939553+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:11.618080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:46.287328+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:20.465163+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:56.550872+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/3713452282":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6800/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1089633358":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6801/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6801/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1807203325":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6800/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1384022397":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2096654470":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/4088938050":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/1375611439":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2304890710":"2024-04-21T10:56:04.789705+0000","172.21.15.117:6800/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2230602257":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6801/1432319225":"2024-04-21T10:56:04.789705+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:03:25.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f09c006ccf0 msgr2=0x7f09c006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:25.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f09c006ccf0 0x7f09c006f1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f09cc005ea0 tx=0x7f09cc005e30 comp rx=0 tx=0).stop 2024-04-20T11:03:25.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 msgr2=0x7f09dc199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:25.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 0x7f09dc199240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f09c80092e0 tx=0x7f09c800be20 comp rx=0 tx=0).stop 2024-04-20T11:03:25.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 shutdown_connections 2024-04-20T11:03:25.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f09c006ccf0 0x7f09c006f1b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:25.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09dc0732b0 0x7f09dc198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:25.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 --2- 172.21.15.117:0/1949958168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc073bd0 0x7f09dc199240 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:25.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 >> 172.21.15.117:0/1949958168 conn(0x7f09dc0fc6a0 msgr2=0x7f09dc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:25.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.483+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 shutdown_connections 2024-04-20T11:03:25.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:25.484+0000 7f09e1400700 1 -- 172.21.15.117:0/1949958168 wait complete. 2024-04-20T11:03:25.865 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:25 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1949958168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:03:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:25 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1949958168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:03:26.296 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-20T11:03:26.297 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-20T11:03:26.297 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-20T11:03:26.298 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-20T11:03:26.298 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-20T11:03:26.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-20T11:03:26.544 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:26 smithi117 ceph-mon[15659]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:26.818 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:26 smithi173 ceph-mon[23800]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:27.141 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:27.153 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:28.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:28 smithi117 ceph-mon[15659]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:28 smithi173 ceph-mon[23800]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:03:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:03:30.812 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:30 smithi173 ceph-mon[23800]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:30.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:30 smithi117 ceph-mon[15659]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:32.805 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:32 smithi117 ceph-mon[15659]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:32 smithi173 ceph-mon[23800]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:33.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.977+0000 7fb196a00700 1 -- 172.21.15.117:0/3513648090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901009d0 msgr2=0x7fb190100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:33.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.977+0000 7fb196a00700 1 --2- 172.21.15.117:0/3513648090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901009d0 0x7fb190100e50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009910 tx=0x7fb17c009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:33.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.977+0000 7fb196a00700 1 -- 172.21.15.117:0/3513648090 shutdown_connections 2024-04-20T11:03:33.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.977+0000 7fb196a00700 1 --2- 172.21.15.117:0/3513648090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901009d0 0x7fb190100e50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:33.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.977+0000 7fb196a00700 1 --2- 172.21.15.117:0/3513648090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1901069d0 0x7fb190106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:33.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.977+0000 7fb196a00700 1 -- 172.21.15.117:0/3513648090 >> 172.21.15.117:0/3513648090 conn(0x7fb1900fc500 msgr2=0x7fb1900fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.978+0000 7fb196a00700 1 -- 172.21.15.117:0/3513648090 shutdown_connections 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.978+0000 7fb196a00700 1 -- 172.21.15.117:0/3513648090 wait complete. 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb196a00700 1 Processor -- start 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb196a00700 1 -- start start 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb196a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1901009d0 0x7fb19019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb196a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901069d0 0x7fb19019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb196a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb190198980 con 0x7fb1901069d0 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb196a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb190198af0 con 0x7fb1901009d0 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb194c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901069d0 0x7fb19019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb194c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901069d0 0x7fb19019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33754/0 (socket says 172.21.15.117:33754) 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.979+0000 7fb194c00700 1 -- 172.21.15.117:0/1841675844 learned_addr learned my addr 172.21.15.117:0/1841675844 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:33.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.980+0000 7fb195600700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1901009d0 0x7fb19019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.980+0000 7fb194c00700 1 -- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1901009d0 msgr2=0x7fb19019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.980+0000 7fb194c00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1901009d0 0x7fb19019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.980+0000 7fb194c00700 1 -- 172.21.15.117:0/1841675844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0095f0 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.980+0000 7fb194c00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901069d0 0x7fb19019ee10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0092e0 tx=0x7fb17c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.980+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17c01d070 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.981+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb17c00f460 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.981+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb190198d70 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.981+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17c017630 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.981+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb190199260 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.982+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb190102110 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.983+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fb17c00f5d0 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.984+0000 7fb18e000700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb17806cce0 0x7fb17806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.984+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb17c08d820 con 0x7fb1901069d0 2024-04-20T11:03:33.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.984+0000 7fb18e000700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] conn(0x7fb1780729c0 0x7fb178074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.984+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 --> [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb178075490 con 0x7fb1780729c0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.984+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb17c08dbe0 con 0x7fb1901069d0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.984+0000 7fb195600700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb17806cce0 0x7fb17806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.985+0000 7fb195600700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb17806cce0 0x7fb17806f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb184005fd0 tx=0x7fb18400a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.985+0000 7fb196000700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] conn(0x7fb1780729c0 0x7fb178074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.986+0000 7fb196000700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] conn(0x7fb1780729c0 0x7fb178074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:32.988+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 <== osd.5 v2:172.21.15.173:6816/1909838821 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb178075490 con 0x7fb1780729c0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.005+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 --> [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb19010de10 con 0x7fb1780729c0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.006+0000 7fb18e000700 1 -- 172.21.15.117:0/1841675844 <== osd.5 v2:172.21.15.173:6816/1909838821 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb19010de10 con 0x7fb1780729c0 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] conn(0x7fb1780729c0 msgr2=0x7fb178074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] conn(0x7fb1780729c0 0x7fb178074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb17806cce0 msgr2=0x7fb17806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:33.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb17806cce0 0x7fb17806f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb184005fd0 tx=0x7fb18400a560 comp rx=0 tx=0).stop 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901069d0 msgr2=0x7fb19019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901069d0 0x7fb19019ee10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0092e0 tx=0x7fb17c00be20 comp rx=0 tx=0).stop 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 shutdown_connections 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:6816/1909838821,v1:172.21.15.173:6817/1909838821] conn(0x7fb1780729c0 0x7fb178074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb17806cce0 0x7fb17806f1a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1901009d0 0x7fb19019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 --2- 172.21.15.117:0/1841675844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1901069d0 0x7fb19019ee10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 >> 172.21.15.117:0/1841675844 conn(0x7fb1900fc500 msgr2=0x7fb190073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 shutdown_connections 2024-04-20T11:03:33.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:33.007+0000 7fb196a00700 1 -- 172.21.15.117:0/1841675844 wait complete. 2024-04-20T11:03:34.070 INFO:teuthology.orchestra.run.smithi117.stdout:176093659145 2024-04-20T11:03:34.070 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-20T11:03:34.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.311+0000 7f49f3e00700 1 -- 172.21.15.117:0/123471136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 msgr2=0x7f49ec105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:34.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.311+0000 7f49f3e00700 1 --2- 172.21.15.117:0/123471136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 0x7f49ec105810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009910 tx=0x7f49d8009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:34.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.311+0000 7f49f3e00700 1 -- 172.21.15.117:0/123471136 shutdown_connections 2024-04-20T11:03:34.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.311+0000 7f49f3e00700 1 --2- 172.21.15.117:0/123471136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 0x7f49ec105810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:34.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.311+0000 7f49f3e00700 1 --2- 172.21.15.117:0/123471136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec100e10 0x7f49ec1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:34.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.311+0000 7f49f3e00700 1 -- 172.21.15.117:0/123471136 >> 172.21.15.117:0/123471136 conn(0x7f49ec0fc6c0 msgr2=0x7f49ec0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:34.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.311+0000 7f49f3e00700 1 -- 172.21.15.117:0/123471136 shutdown_connections 2024-04-20T11:03:34.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.312+0000 7f49f3e00700 1 -- 172.21.15.117:0/123471136 wait complete. 2024-04-20T11:03:34.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.313+0000 7f49f3e00700 1 Processor -- start 2024-04-20T11:03:34.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.313+0000 7f49f3e00700 1 -- start start 2024-04-20T11:03:34.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.313+0000 7f49f3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec100e10 0x7f49ec198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:34.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.313+0000 7f49f3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 0x7f49ec199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:34.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.313+0000 7f49f3e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec199a10 con 0x7f49ec1017c0 2024-04-20T11:03:34.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.313+0000 7f49f3e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec19d6e0 con 0x7f49ec100e10 2024-04-20T11:03:34.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.314+0000 7f49f0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 0x7f49ec199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:34.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.314+0000 7f49f1400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec100e10 0x7f49ec198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:34.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.314+0000 7f49f0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 0x7f49ec199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57448/0 (socket says 172.21.15.117:57448) 2024-04-20T11:03:34.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.314+0000 7f49f0a00700 1 -- 172.21.15.117:0/4220739179 learned_addr learned my addr 172.21.15.117:0/4220739179 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:34.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.314+0000 7f49f0a00700 1 -- 172.21.15.117:0/4220739179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec100e10 msgr2=0x7f49ec198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:34.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.314+0000 7f49f0a00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec100e10 0x7f49ec198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:34.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.314+0000 7f49f0a00700 1 -- 172.21.15.117:0/4220739179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d80095f0 con 0x7f49ec1017c0 2024-04-20T11:03:34.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.315+0000 7f49f0a00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 0x7f49ec199330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f49d80092e0 tx=0x7f49d800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:34.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.315+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d801d070 con 0x7f49ec1017c0 2024-04-20T11:03:34.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.315+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49d800f460 con 0x7f49ec1017c0 2024-04-20T11:03:34.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.315+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d8017610 con 0x7f49ec1017c0 2024-04-20T11:03:34.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.315+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec19d960 con 0x7f49ec1017c0 2024-04-20T11:03:34.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.316+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec19de50 con 0x7f49ec1017c0 2024-04-20T11:03:34.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.317+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49ec10e130 con 0x7f49ec1017c0 2024-04-20T11:03:34.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.319+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f49d800f5d0 con 0x7f49ec1017c0 2024-04-20T11:03:34.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.320+0000 7f49ea000700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f49d406cce0 0x7f49d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:34.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.320+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f49d808dd70 con 0x7f49ec1017c0 2024-04-20T11:03:34.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.321+0000 7f49f1400700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f49d406cce0 0x7f49d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:34.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.321+0000 7f49ea000700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] conn(0x7f49d40729c0 0x7f49d4074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:34.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.321+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 --> [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49d4075490 con 0x7f49d40729c0 2024-04-20T11:03:34.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.321+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d808e130 con 0x7f49ec1017c0 2024-04-20T11:03:34.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.321+0000 7f49f1e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] conn(0x7f49d40729c0 0x7f49d4074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:34.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.321+0000 7f49f1400700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f49d406cce0 0x7f49d406f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009a10 tx=0x7f49e0009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:34.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.321+0000 7f49f1e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] conn(0x7f49d40729c0 0x7f49d4074de0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:34.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.322+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 <== osd.1 v2:172.21.15.117:6810/2871230534 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f49d4075490 con 0x7f49d40729c0 2024-04-20T11:03:34.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.339+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 --> [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49ec19e400 con 0x7f49d40729c0 2024-04-20T11:03:34.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49ea000700 1 -- 172.21.15.117:0/4220739179 <== osd.1 v2:172.21.15.117:6810/2871230534 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f49ec19e400 con 0x7f49d40729c0 2024-04-20T11:03:34.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] conn(0x7f49d40729c0 msgr2=0x7f49d4074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:34.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] conn(0x7f49d40729c0 0x7f49d4074de0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:34.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f49d406cce0 msgr2=0x7f49d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:34.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f49d406cce0 0x7f49d406f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009a10 tx=0x7f49e0009190 comp rx=0 tx=0).stop 2024-04-20T11:03:34.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 msgr2=0x7f49ec199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:34.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 0x7f49ec199330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f49d80092e0 tx=0x7f49d800bd70 comp rx=0 tx=0).stop 2024-04-20T11:03:34.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 shutdown_connections 2024-04-20T11:03:34.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f49d406cce0 0x7f49d406f1a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:34.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:6810/2871230534,v1:172.21.15.117:6811/2871230534] conn(0x7f49d40729c0 0x7f49d4074de0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:34.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec100e10 0x7f49ec198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:34.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 --2- 172.21.15.117:0/4220739179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ec1017c0 0x7f49ec199330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:34.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 >> 172.21.15.117:0/4220739179 conn(0x7f49ec0fc6c0 msgr2=0x7f49ec0744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:34.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 shutdown_connections 2024-04-20T11:03:34.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:34.340+0000 7f49f3e00700 1 -- 172.21.15.117:0/4220739179 wait complete. 2024-04-20T11:03:35.064 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:34 smithi117 ceph-mon[15659]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:34 smithi173 ceph-mon[23800]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:36.448 INFO:teuthology.orchestra.run.smithi117.stdout:60129542183 2024-04-20T11:03:36.449 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-20T11:03:36.871 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:36.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.896+0000 7fe5efe00700 1 -- 172.21.15.117:0/1975027464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01069d0 msgr2=0x7fe5f0106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:36.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.896+0000 7fe5efe00700 1 --2- 172.21.15.117:0/1975027464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01069d0 0x7fe5f0106db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009910 tx=0x7fe5e0009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:36.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.896+0000 7fe5efe00700 1 -- 172.21.15.117:0/1975027464 shutdown_connections 2024-04-20T11:03:36.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.896+0000 7fe5efe00700 1 --2- 172.21.15.117:0/1975027464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5f01009d0 0x7fe5f0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.896+0000 7fe5efe00700 1 --2- 172.21.15.117:0/1975027464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01069d0 0x7fe5f0106db0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.896+0000 7fe5efe00700 1 -- 172.21.15.117:0/1975027464 >> 172.21.15.117:0/1975027464 conn(0x7fe5f00fc500 msgr2=0x7fe5f00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:36.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.896+0000 7fe5efe00700 1 -- 172.21.15.117:0/1975027464 shutdown_connections 2024-04-20T11:03:36.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.896+0000 7fe5efe00700 1 -- 172.21.15.117:0/1975027464 wait complete. 2024-04-20T11:03:36.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.897+0000 7fe5efe00700 1 Processor -- start 2024-04-20T11:03:36.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.897+0000 7fe5efe00700 1 -- start start 2024-04-20T11:03:36.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5efe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01009d0 0x7fe5f019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:36.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5efe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5f01069d0 0x7fe5f019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:36.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5efe00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0198980 con 0x7fe5f01009d0 2024-04-20T11:03:36.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5efe00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0198af0 con 0x7fe5f01069d0 2024-04-20T11:03:36.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5eea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01009d0 0x7fe5f019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:36.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5eea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01009d0 0x7fe5f019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57468/0 (socket says 172.21.15.117:57468) 2024-04-20T11:03:36.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5eea00700 1 -- 172.21.15.117:0/2030831387 learned_addr learned my addr 172.21.15.117:0/2030831387 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5ee000700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5f01069d0 0x7fe5f019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5eea00700 1 -- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5f01069d0 msgr2=0x7fe5f019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5eea00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5f01069d0 0x7fe5f019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5eea00700 1 -- 172.21.15.117:0/2030831387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00095f0 con 0x7fe5f01009d0 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.898+0000 7fe5ee000700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5f01069d0 0x7fe5f019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.899+0000 7fe5eea00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01009d0 0x7fe5f019e8d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00092e0 tx=0x7fe5e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.899+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e001d070 con 0x7fe5f01009d0 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.899+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5e000f460 con 0x7fe5f01009d0 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.899+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e0017630 con 0x7fe5f01009d0 2024-04-20T11:03:36.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.899+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f0198d70 con 0x7fe5f01009d0 2024-04-20T11:03:36.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.899+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f0199260 con 0x7fe5f01009d0 2024-04-20T11:03:36.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.900+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5f010ded0 con 0x7fe5f01009d0 2024-04-20T11:03:36.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.902+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fe5e000f5d0 con 0x7fe5f01009d0 2024-04-20T11:03:36.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.903+0000 7fe5e7400700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5d406cce0 0x7fe5d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:36.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.903+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe5e008d410 con 0x7fe5f01009d0 2024-04-20T11:03:36.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.903+0000 7fe5e7400700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] conn(0x7fe5d40729c0 0x7fe5d4074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:36.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.903+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 --> [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5d4075490 con 0x7fe5d40729c0 2024-04-20T11:03:36.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.903+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5e008fa40 con 0x7fe5f01009d0 2024-04-20T11:03:36.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.903+0000 7fe5ee000700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5d406cce0 0x7fe5d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:36.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.904+0000 7fe5ef400700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] conn(0x7fe5d40729c0 0x7fe5d4074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:36.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.904+0000 7fe5ee000700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5d406cce0 0x7fe5d406f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc005fd0 tx=0x7fe5dc005dc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:36.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.905+0000 7fe5ef400700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] conn(0x7fe5d40729c0 0x7fe5d4074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:36.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.907+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 <== osd.4 v2:172.21.15.173:6808/1468348921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe5d4075490 con 0x7fe5d40729c0 2024-04-20T11:03:36.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.928+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 --> [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe5f0199710 con 0x7fe5d40729c0 2024-04-20T11:03:36.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.928+0000 7fe5e7400700 1 -- 172.21.15.117:0/2030831387 <== osd.4 v2:172.21.15.173:6808/1468348921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe5f0199710 con 0x7fe5d40729c0 2024-04-20T11:03:36.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] conn(0x7fe5d40729c0 msgr2=0x7fe5d4074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:36.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] conn(0x7fe5d40729c0 0x7fe5d4074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5d406cce0 msgr2=0x7fe5d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:36.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5d406cce0 0x7fe5d406f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc005fd0 tx=0x7fe5dc005dc0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01009d0 msgr2=0x7fe5f019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01009d0 0x7fe5f019e8d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00092e0 tx=0x7fe5e000be20 comp rx=0 tx=0).stop 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 shutdown_connections 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:6808/1468348921,v1:172.21.15.173:6809/1468348921] conn(0x7fe5d40729c0 0x7fe5d4074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5d406cce0 0x7fe5d406f1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f01009d0 0x7fe5f019e8d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 --2- 172.21.15.117:0/2030831387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5f01069d0 0x7fe5f019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.929+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 >> 172.21.15.117:0/2030831387 conn(0x7fe5f00fc500 msgr2=0x7fe5f0073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.930+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 shutdown_connections 2024-04-20T11:03:36.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:36.930+0000 7fe5efe00700 1 -- 172.21.15.117:0/2030831387 wait complete. 2024-04-20T11:03:36.987 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:36 smithi117 ceph-mon[15659]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:36 smithi173 ceph-mon[23800]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.592+0000 7f8ea1600700 1 -- 172.21.15.117:0/289368512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c1017c0 msgr2=0x7f8e9c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.592+0000 7f8ea1600700 1 --2- 172.21.15.117:0/289368512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c1017c0 0x7f8e9c105810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009910 tx=0x7f8e88009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.592+0000 7f8ea1600700 1 -- 172.21.15.117:0/289368512 shutdown_connections 2024-04-20T11:03:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.592+0000 7f8ea1600700 1 --2- 172.21.15.117:0/289368512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c1017c0 0x7f8e9c105810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.592+0000 7f8ea1600700 1 --2- 172.21.15.117:0/289368512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e9c100e10 0x7f8e9c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.592+0000 7f8ea1600700 1 -- 172.21.15.117:0/289368512 >> 172.21.15.117:0/289368512 conn(0x7f8e9c0fc6c0 msgr2=0x7f8e9c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.592+0000 7f8ea1600700 1 -- 172.21.15.117:0/289368512 shutdown_connections 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.592+0000 7f8ea1600700 1 -- 172.21.15.117:0/289368512 wait complete. 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.593+0000 7f8ea1600700 1 Processor -- start 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.593+0000 7f8ea1600700 1 -- start start 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8ea1600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c100e10 0x7f8e9c0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8ea1600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e9c1017c0 0x7f8e9c073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8ea1600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c073c60 con 0x7f8e9c100e10 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8ea1600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c073da0 con 0x7f8e9c1017c0 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8e9aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c100e10 0x7f8e9c0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8e9aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c100e10 0x7f8e9c0731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57480/0 (socket says 172.21.15.117:57480) 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8e9aa00700 1 -- 172.21.15.117:0/4267591215 learned_addr learned my addr 172.21.15.117:0/4267591215 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8e9a000700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e9c1017c0 0x7f8e9c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8e9aa00700 1 -- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e9c1017c0 msgr2=0x7f8e9c073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8e9aa00700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e9c1017c0 0x7f8e9c073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.594+0000 7f8e9aa00700 1 -- 172.21.15.117:0/4267591215 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e880095f0 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.595+0000 7f8e9aa00700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c100e10 0x7f8e9c0731e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00d8d0 tx=0x7f8e8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.595+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e8c009940 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.595+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e8c010460 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.595+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c1a2cf0 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.595+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e8c00f5f0 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.595+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c1a3050 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.596+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e9c1022f0 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.598+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f8e8c0105d0 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.599+0000 7f8e93400700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8e8006cce0 0x7f8e8006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.599+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8e8c08bff0 con 0x7f8e9c100e10 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.599+0000 7f8e93400700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] conn(0x7f8e800729c0 0x7f8e80074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.600+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 --> [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e80075490 con 0x7f8e800729c0 2024-04-20T11:03:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.600+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e8c08e5d0 con 0x7f8e9c100e10 2024-04-20T11:03:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.600+0000 7f8e9a000700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8e8006cce0 0x7f8e8006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.600+0000 7f8e9b400700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] conn(0x7f8e800729c0 0x7f8e80074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.600+0000 7f8e9a000700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8e8006cce0 0x7f8e8006f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0748c0 tx=0x7f8e8801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.601+0000 7f8e9b400700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] conn(0x7f8e800729c0 0x7f8e80074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.603+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 <== osd.3 v2:172.21.15.173:6800/2587001742 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f8e80075490 con 0x7f8e800729c0 2024-04-20T11:03:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.620+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 --> [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e9c10dff0 con 0x7f8e800729c0 2024-04-20T11:03:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.621+0000 7f8e93400700 1 -- 172.21.15.117:0/4267591215 <== osd.3 v2:172.21.15.173:6800/2587001742 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8e9c10dff0 con 0x7f8e800729c0 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] conn(0x7f8e800729c0 msgr2=0x7f8e80074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] conn(0x7f8e800729c0 0x7f8e80074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8e8006cce0 msgr2=0x7f8e8006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8e8006cce0 0x7f8e8006f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0748c0 tx=0x7f8e8801a040 comp rx=0 tx=0).stop 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c100e10 msgr2=0x7f8e9c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c100e10 0x7f8e9c0731e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00d8d0 tx=0x7f8e8c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 shutdown_connections 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:6800/2587001742,v1:172.21.15.173:6801/2587001742] conn(0x7f8e800729c0 0x7f8e80074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8e8006cce0 0x7f8e8006f1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c100e10 0x7f8e9c0731e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 --2- 172.21.15.117:0/4267591215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e9c1017c0 0x7f8e9c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 >> 172.21.15.117:0/4267591215 conn(0x7f8e9c0fc6c0 msgr2=0x7f8e9c0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.622+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 shutdown_connections 2024-04-20T11:03:37.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.623+0000 7f8ea1600700 1 -- 172.21.15.117:0/4267591215 wait complete. 2024-04-20T11:03:38.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.696+0000 7fd48e600700 1 -- 172.21.15.117:0/3496478101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 msgr2=0x7fd488103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:38.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.696+0000 7fd48e600700 1 --2- 172.21.15.117:0/3496478101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 0x7fd488103420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd478009910 tx=0x7fd478009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:38.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.696+0000 7fd48e600700 1 -- 172.21.15.117:0/3496478101 shutdown_connections 2024-04-20T11:03:38.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.696+0000 7fd48e600700 1 --2- 172.21.15.117:0/3496478101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4881039f0 0x7fd488107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.696+0000 7fd48e600700 1 --2- 172.21.15.117:0/3496478101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 0x7fd488103420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.696+0000 7fd48e600700 1 -- 172.21.15.117:0/3496478101 >> 172.21.15.117:0/3496478101 conn(0x7fd4880fe8d0 msgr2=0x7fd488100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.696+0000 7fd48e600700 1 -- 172.21.15.117:0/3496478101 shutdown_connections 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.696+0000 7fd48e600700 1 -- 172.21.15.117:0/3496478101 wait complete. 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.697+0000 7fd48e600700 1 Processor -- start 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.697+0000 7fd48e600700 1 -- start start 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd48e600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 0x7fd488198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd48e600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4881039f0 0x7fd4881992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd48e600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4881999a0 con 0x7fd488103040 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd48e600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48819d670 con 0x7fd4881039f0 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd487400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 0x7fd488198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd487400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 0x7fd488198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57502/0 (socket says 172.21.15.117:57502) 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd487400700 1 -- 172.21.15.117:0/2167112406 learned_addr learned my addr 172.21.15.117:0/2167112406 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd486a00700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4881039f0 0x7fd4881992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:38.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd487400700 1 -- 172.21.15.117:0/2167112406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4881039f0 msgr2=0x7fd4881992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd487400700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4881039f0 0x7fd4881992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd487400700 1 -- 172.21.15.117:0/2167112406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780095f0 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.698+0000 7fd486a00700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4881039f0 0x7fd4881992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.699+0000 7fd487400700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 0x7fd488198d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd4780092e0 tx=0x7fd47800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.699+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47801d070 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.699+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd47800f460 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.699+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48819d8f0 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.699+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd478017630 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.699+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48819dde0 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.700+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd4881102d0 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.701+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fd47800f5d0 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.702+0000 7fd47fe00700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd46c06cce0 0x7fd46c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.702+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd47808d420 con 0x7fd488103040 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.702+0000 7fd486a00700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd46c06cce0 0x7fd46c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:38.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.703+0000 7fd47fe00700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] conn(0x7fd46c0729c0 0x7fd46c074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:38.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.703+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 --> [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd46c075490 con 0x7fd46c0729c0 2024-04-20T11:03:38.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.703+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd47808faf0 con 0x7fd488103040 2024-04-20T11:03:38.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.703+0000 7fd487e00700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] conn(0x7fd46c0729c0 0x7fd46c074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:38.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.703+0000 7fd486a00700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd46c06cce0 0x7fd46c06f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd474005fd0 tx=0x7fd474005dc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:38.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.703+0000 7fd487e00700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] conn(0x7fd46c0729c0 0x7fd46c074de0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:38.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.704+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 <== osd.0 v2:172.21.15.117:6802/2292675302 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fd46c075490 con 0x7fd46c0729c0 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.721+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 --> [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd488104520 con 0x7fd46c0729c0 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd47fe00700 1 -- 172.21.15.117:0/2167112406 <== osd.0 v2:172.21.15.117:6802/2292675302 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd488104520 con 0x7fd46c0729c0 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] conn(0x7fd46c0729c0 msgr2=0x7fd46c074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] conn(0x7fd46c0729c0 0x7fd46c074de0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd46c06cce0 msgr2=0x7fd46c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd46c06cce0 0x7fd46c06f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd474005fd0 tx=0x7fd474005dc0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 msgr2=0x7fd488198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 0x7fd488198d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd4780092e0 tx=0x7fd47800be20 comp rx=0 tx=0).stop 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 shutdown_connections 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6802/2292675302,v1:172.21.15.117:6803/2292675302] conn(0x7fd46c0729c0 0x7fd46c074de0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd46c06cce0 0x7fd46c06f1a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd488103040 0x7fd488198d80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 --2- 172.21.15.117:0/2167112406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4881039f0 0x7fd4881992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:38.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.722+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 >> 172.21.15.117:0/2167112406 conn(0x7fd4880fe8d0 msgr2=0x7fd4880fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:38.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.723+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 shutdown_connections 2024-04-20T11:03:38.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:37.723+0000 7fd48e600700 1 -- 172.21.15.117:0/2167112406 wait complete. 2024-04-20T11:03:38.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.998+0000 7f7d72200700 1 -- 172.21.15.117:0/3424587106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107c00 msgr2=0x7f7d6c10bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:38.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.998+0000 7f7d72200700 1 --2- 172.21.15.117:0/3424587106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107c00 0x7f7d6c10bc50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009910 tx=0x7f7d58009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:39.000 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:38 smithi117 ceph-mon[15659]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.998+0000 7f7d72200700 1 -- 172.21.15.117:0/3424587106 shutdown_connections 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.998+0000 7f7d72200700 1 --2- 172.21.15.117:0/3424587106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107c00 0x7f7d6c10bc50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.998+0000 7f7d72200700 1 --2- 172.21.15.117:0/3424587106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d6c107250 0x7f7d6c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.998+0000 7f7d72200700 1 -- 172.21.15.117:0/3424587106 >> 172.21.15.117:0/3424587106 conn(0x7f7d6c0767e0 msgr2=0x7f7d6c078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.998+0000 7f7d72200700 1 -- 172.21.15.117:0/3424587106 shutdown_connections 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.998+0000 7f7d72200700 1 -- 172.21.15.117:0/3424587106 wait complete. 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:38.999+0000 7f7d72200700 1 Processor -- start 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d72200700 1 -- start start 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d72200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107250 0x7f7d6c07cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d72200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d6c107c00 0x7f7d6c07d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d72200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c081e00 con 0x7f7d6c107250 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d72200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c07d7f0 con 0x7f7d6c107c00 2024-04-20T11:03:39.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d70e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107250 0x7f7d6c07cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:39.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d70e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107250 0x7f7d6c07cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57526/0 (socket says 172.21.15.117:57526) 2024-04-20T11:03:39.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d70e00700 1 -- 172.21.15.117:0/2893241251 learned_addr learned my addr 172.21.15.117:0/2893241251 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.000+0000 7f7d6be00700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d6c107c00 0x7f7d6c07d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d70e00700 1 -- 172.21.15.117:0/2893241251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d6c107c00 msgr2=0x7f7d6c07d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d70e00700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d6c107c00 0x7f7d6c07d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d70e00700 1 -- 172.21.15.117:0/2893241251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580095f0 con 0x7f7d6c107250 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d70e00700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107250 0x7f7d6c07cd70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000b4b0 tx=0x7f7d6000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d60011800 con 0x7f7d6c107250 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d60011e20 con 0x7f7d6c107250 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c07da80 con 0x7f7d6c107250 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6000f2e0 con 0x7f7d6c107250 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.001+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c1afba0 con 0x7f7d6c107250 2024-04-20T11:03:39.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.002+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d6c1144f0 con 0x7f7d6c107250 2024-04-20T11:03:39.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.003+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f7d60010380 con 0x7f7d6c107250 2024-04-20T11:03:39.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.004+0000 7f7d69600700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7d5406cce0 0x7f7d5406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:39.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.004+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7d6008bb40 con 0x7f7d6c107250 2024-04-20T11:03:39.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.005+0000 7f7d6be00700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7d5406cce0 0x7f7d5406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:39.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.005+0000 7f7d69600700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] conn(0x7f7d540729c0 0x7f7d54074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:39.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.005+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 --> [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d54075490 con 0x7f7d540729c0 2024-04-20T11:03:39.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.005+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d600902a0 con 0x7f7d6c107250 2024-04-20T11:03:39.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.005+0000 7f7d71800700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] conn(0x7f7d540729c0 0x7f7d54074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:39.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.005+0000 7f7d6be00700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7d5406cce0 0x7f7d5406f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006010 tx=0x7f7d5800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:39.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.006+0000 7f7d71800700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] conn(0x7f7d540729c0 0x7f7d54074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:39.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.006+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 <== osd.2 v2:172.21.15.117:6818/3091267150 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7d54075490 con 0x7f7d540729c0 2024-04-20T11:03:39.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.023+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 --> [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d6c108730 con 0x7f7d540729c0 2024-04-20T11:03:39.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.024+0000 7f7d69600700 1 -- 172.21.15.117:0/2893241251 <== osd.2 v2:172.21.15.117:6818/3091267150 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d6c108730 con 0x7f7d540729c0 2024-04-20T11:03:39.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.024+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] conn(0x7f7d540729c0 msgr2=0x7f7d54074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:39.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.024+0000 7f7d72200700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] conn(0x7f7d540729c0 0x7f7d54074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:39.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.024+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7d5406cce0 msgr2=0x7f7d5406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:39.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.024+0000 7f7d72200700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7d5406cce0 0x7f7d5406f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006010 tx=0x7f7d5800b540 comp rx=0 tx=0).stop 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.024+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107250 msgr2=0x7f7d6c07cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.024+0000 7f7d72200700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107250 0x7f7d6c07cd70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000b4b0 tx=0x7f7d6000b870 comp rx=0 tx=0).stop 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.025+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 shutdown_connections 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.025+0000 7f7d72200700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6818/3091267150,v1:172.21.15.117:6819/3091267150] conn(0x7f7d540729c0 0x7f7d54074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.025+0000 7f7d72200700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7d5406cce0 0x7f7d5406f1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.025+0000 7f7d72200700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c107250 0x7f7d6c07cd70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.025+0000 7f7d72200700 1 --2- 172.21.15.117:0/2893241251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d6c107c00 0x7f7d6c07d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.025+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 >> 172.21.15.117:0/2893241251 conn(0x7f7d6c0767e0 msgr2=0x7f7d6c077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.025+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 shutdown_connections 2024-04-20T11:03:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:39.025+0000 7f7d72200700 1 -- 172.21.15.117:0/2893241251 wait complete. 2024-04-20T11:03:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:38 smithi173 ceph-mon[23800]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:40.187 INFO:teuthology.orchestra.run.smithi117.stdout:38654705712 2024-04-20T11:03:40.187 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-20T11:03:40.908 INFO:teuthology.orchestra.run.smithi117.stdout:81604378655 2024-04-20T11:03:40.908 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-20T11:03:41.035 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:40 smithi117 ceph-mon[15659]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:41.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:40 smithi173 ceph-mon[23800]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:41.742 INFO:teuthology.orchestra.run.smithi117.stdout:115964117017 2024-04-20T11:03:41.742 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-20T11:03:42.311 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:42.333 INFO:teuthology.orchestra.run.smithi117.stdout:154618822673 2024-04-20T11:03:42.334 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-20T11:03:42.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:42 smithi117 ceph-mon[15659]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:43.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.140+0000 7fe36e600700 1 -- 172.21.15.117:0/588669253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 msgr2=0x7fe368103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:43.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.140+0000 7fe36e600700 1 --2- 172.21.15.117:0/588669253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 0x7fe368103200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b0a0 tx=0x7fe35c00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:03:43.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.140+0000 7fe36e600700 1 -- 172.21.15.117:0/588669253 shutdown_connections 2024-04-20T11:03:43.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.140+0000 7fe36e600700 1 --2- 172.21.15.117:0/588669253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3681037d0 0x7fe368107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:43.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.140+0000 7fe36e600700 1 --2- 172.21.15.117:0/588669253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 0x7fe368103200 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:43.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.140+0000 7fe36e600700 1 -- 172.21.15.117:0/588669253 >> 172.21.15.117:0/588669253 conn(0x7fe3680fe6d0 msgr2=0x7fe368100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:43.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.140+0000 7fe36e600700 1 -- 172.21.15.117:0/588669253 shutdown_connections 2024-04-20T11:03:43.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.140+0000 7fe36e600700 1 -- 172.21.15.117:0/588669253 wait complete. 2024-04-20T11:03:43.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.141+0000 7fe36e600700 1 Processor -- start 2024-04-20T11:03:43.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.141+0000 7fe36e600700 1 -- start start 2024-04-20T11:03:43.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.142+0000 7fe36e600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 0x7fe36819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:43.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.142+0000 7fe36e600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3681037d0 0x7fe36819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.142+0000 7fe36e600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36819f470 con 0x7fe368102e20 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.142+0000 7fe36e600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368198a50 con 0x7fe3681037d0 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.142+0000 7fe36d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 0x7fe36819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.142+0000 7fe36d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 0x7fe36819e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57546/0 (socket says 172.21.15.117:57546) 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.142+0000 7fe36d200700 1 -- 172.21.15.117:0/98231492 learned_addr learned my addr 172.21.15.117:0/98231492 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe36d200700 1 -- 172.21.15.117:0/98231492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3681037d0 msgr2=0x7fe36819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe36d200700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3681037d0 0x7fe36819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe36d200700 1 -- 172.21.15.117:0/98231492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009d00 con 0x7fe368102e20 2024-04-20T11:03:43.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe36d200700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 0x7fe36819e910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b070 tx=0x7fe35c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:43.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe365600700 1 -- 172.21.15.117:0/98231492 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35c00e040 con 0x7fe368102e20 2024-04-20T11:03:43.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe365600700 1 -- 172.21.15.117:0/98231492 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe35c003de0 con 0x7fe368102e20 2024-04-20T11:03:43.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe365600700 1 -- 172.21.15.117:0/98231492 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35c01d6b0 con 0x7fe368102e20 2024-04-20T11:03:43.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe368198cd0 con 0x7fe368102e20 2024-04-20T11:03:43.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.143+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3681991c0 con 0x7fe368102e20 2024-04-20T11:03:43.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.145+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36810b1c0 con 0x7fe368102e20 2024-04-20T11:03:43.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.145+0000 7fe365600700 1 -- 172.21.15.117:0/98231492 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fe35c004040 con 0x7fe368102e20 2024-04-20T11:03:43.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.146+0000 7fe365600700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe35006cce0 0x7fe35006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:43.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.146+0000 7fe365600700 1 -- 172.21.15.117:0/98231492 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe35c08d7b0 con 0x7fe368102e20 2024-04-20T11:03:43.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.147+0000 7fe367e00700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe35006cce0 0x7fe35006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:43.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.147+0000 7fe367e00700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe35006cce0 0x7fe35006f1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe368199c10 tx=0x7fe354005c80 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:43.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.151+0000 7fe365600700 1 -- 172.21.15.117:0/98231492 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe35c05b930 con 0x7fe368102e20 2024-04-20T11:03:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:42 smithi173 ceph-mon[23800]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:43.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.379+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe368199f40 con 0x7fe368102e20 2024-04-20T11:03:43.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.379+0000 7fe365600700 1 -- 172.21.15.117:0/98231492 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe35c05b4c0 con 0x7fe368102e20 2024-04-20T11:03:43.381 INFO:teuthology.orchestra.run.smithi117.stdout:176093659147 2024-04-20T11:03:43.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe35006cce0 msgr2=0x7fe35006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:43.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe35006cce0 0x7fe35006f1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe368199c10 tx=0x7fe354005c80 comp rx=0 tx=0).stop 2024-04-20T11:03:43.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 msgr2=0x7fe36819e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:43.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 0x7fe36819e910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b070 tx=0x7fe35c007bd0 comp rx=0 tx=0).stop 2024-04-20T11:03:43.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 shutdown_connections 2024-04-20T11:03:43.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe35006cce0 0x7fe35006f1a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:43.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe368102e20 0x7fe36819e910 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:43.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 --2- 172.21.15.117:0/98231492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3681037d0 0x7fe36819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:43.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 >> 172.21.15.117:0/98231492 conn(0x7fe3680fe6d0 msgr2=0x7fe3680ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:43.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 shutdown_connections 2024-04-20T11:03:43.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:43.382+0000 7fe36e600700 1 -- 172.21.15.117:0/98231492 wait complete. 2024-04-20T11:03:43.798 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:44.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:43 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/98231492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T11:03:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:03:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:43 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/98231492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T11:03:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:03:44.261 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:44.262 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:45.053 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:44 smithi117 ceph-mon[15659]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:44 smithi173 ceph-mon[23800]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:46.428 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659145 got 176093659147 for osd.5 2024-04-20T11:03:46.429 DEBUG:teuthology.parallel:result is None 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.624+0000 7f9be6400700 1 -- 172.21.15.117:0/3874747039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107df0 msgr2=0x7f9be010be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.624+0000 7f9be6400700 1 --2- 172.21.15.117:0/3874747039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107df0 0x7f9be010be40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009910 tx=0x7f9bc8009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.624+0000 7f9be6400700 1 -- 172.21.15.117:0/3874747039 shutdown_connections 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.624+0000 7f9be6400700 1 --2- 172.21.15.117:0/3874747039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107df0 0x7f9be010be40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.624+0000 7f9be6400700 1 --2- 172.21.15.117:0/3874747039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be0107440 0x7f9be0107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.624+0000 7f9be6400700 1 -- 172.21.15.117:0/3874747039 >> 172.21.15.117:0/3874747039 conn(0x7f9be0076a10 msgr2=0x7f9be0078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.624+0000 7f9be6400700 1 -- 172.21.15.117:0/3874747039 shutdown_connections 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.624+0000 7f9be6400700 1 -- 172.21.15.117:0/3874747039 wait complete. 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.626+0000 7f9be6400700 1 Processor -- start 2024-04-20T11:03:46.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.626+0000 7f9be6400700 1 -- start start 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.626+0000 7f9be6400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107440 0x7f9be019d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.626+0000 7f9be6400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be0107df0 0x7f9be019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.626+0000 7f9be6400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be019dd10 con 0x7f9be0107440 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.626+0000 7f9be6400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be01a19e0 con 0x7f9be0107df0 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bdf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107440 0x7f9be019d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bdf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107440 0x7f9be019d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54098/0 (socket says 172.21.15.117:54098) 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bdf400700 1 -- 172.21.15.117:0/3304137081 learned_addr learned my addr 172.21.15.117:0/3304137081 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bd7e00700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be0107df0 0x7f9be019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bdf400700 1 -- 172.21.15.117:0/3304137081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be0107df0 msgr2=0x7f9be019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bdf400700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be0107df0 0x7f9be019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bdf400700 1 -- 172.21.15.117:0/3304137081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc80095f0 con 0x7f9be0107440 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bd7e00700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be0107df0 0x7f9be019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.627+0000 7f9bdf400700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107440 0x7f9be019d0f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c8a0 tx=0x7f9bd000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:46.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.628+0000 7f9bdcc00700 1 -- 172.21.15.117:0/3304137081 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bd000cea0 con 0x7f9be0107440 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.628+0000 7f9bdcc00700 1 -- 172.21.15.117:0/3304137081 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bd0004830 con 0x7f9be0107440 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.628+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be01a1cc0 con 0x7f9be0107440 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.628+0000 7f9bdcc00700 1 -- 172.21.15.117:0/3304137081 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bd00055b0 con 0x7f9be0107440 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.628+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be01a2210 con 0x7f9be0107440 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.629+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be010f840 con 0x7f9be0107440 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.630+0000 7f9bdcc00700 1 -- 172.21.15.117:0/3304137081 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f9bd00049a0 con 0x7f9be0107440 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.631+0000 7f9bdcc00700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9bc406cce0 0x7f9bc406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.631+0000 7f9bdcc00700 1 -- 172.21.15.117:0/3304137081 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9bd008bb90 con 0x7f9be0107440 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.631+0000 7f9bd7e00700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9bc406cce0 0x7f9bc406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.632+0000 7f9bd7e00700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9bc406cce0 0x7f9bc406f1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005f50 tx=0x7f9bc8005dc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:46.481 INFO:teuthology.orchestra.run.smithi117.stdout:60129542185 2024-04-20T11:03:46.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.636+0000 7f9bdcc00700 1 -- 172.21.15.117:0/3304137081 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9bd0056280 con 0x7f9be0107440 2024-04-20T11:03:46.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.862+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9be019e4a0 con 0x7f9be0107440 2024-04-20T11:03:46.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.863+0000 7f9bdcc00700 1 -- 172.21.15.117:0/3304137081 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9bd001d090 con 0x7f9be0107440 2024-04-20T11:03:46.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.866+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9bc406cce0 msgr2=0x7f9bc406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:46.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.866+0000 7f9be6400700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9bc406cce0 0x7f9bc406f1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005f50 tx=0x7f9bc8005dc0 comp rx=0 tx=0).stop 2024-04-20T11:03:46.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.866+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107440 msgr2=0x7f9be019d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:46.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.866+0000 7f9be6400700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107440 0x7f9be019d0f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c8a0 tx=0x7f9bd000cc60 comp rx=0 tx=0).stop 2024-04-20T11:03:46.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.866+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 shutdown_connections 2024-04-20T11:03:46.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.866+0000 7f9be6400700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9bc406cce0 0x7f9bc406f1a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:46.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.866+0000 7f9be6400700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be0107440 0x7f9be019d0f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:46.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.866+0000 7f9be6400700 1 --2- 172.21.15.117:0/3304137081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be0107df0 0x7f9be019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:46.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.867+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 >> 172.21.15.117:0/3304137081 conn(0x7f9be0076a10 msgr2=0x7f9be0076df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:46.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.867+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 shutdown_connections 2024-04-20T11:03:46.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:45.867+0000 7f9be6400700 1 -- 172.21.15.117:0/3304137081 wait complete. 2024-04-20T11:03:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:46 smithi117 ceph-mon[15659]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:46 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3304137081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T11:03:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:46 smithi173 ceph-mon[23800]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:46 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3304137081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T11:03:47.700 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:48.140 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542183 got 60129542185 for osd.1 2024-04-20T11:03:48.140 DEBUG:teuthology.parallel:result is None 2024-04-20T11:03:48.881 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:48 smithi117 ceph-mon[15659]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:49.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:48 smithi173 ceph-mon[23800]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.662+0000 7f5989400700 1 -- 172.21.15.117:0/836969244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 msgr2=0x7f5984105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.662+0000 7f5989400700 1 --2- 172.21.15.117:0/836969244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 0x7f5984105820 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f596c009910 tx=0x7f596c009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.662+0000 7f5989400700 1 -- 172.21.15.117:0/836969244 shutdown_connections 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.662+0000 7f5989400700 1 --2- 172.21.15.117:0/836969244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 0x7f5984105820 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.662+0000 7f5989400700 1 --2- 172.21.15.117:0/836969244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5984100e20 0x7f5984101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.662+0000 7f5989400700 1 -- 172.21.15.117:0/836969244 >> 172.21.15.117:0/836969244 conn(0x7f59840fc6d0 msgr2=0x7f59840feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.662+0000 7f5989400700 1 -- 172.21.15.117:0/836969244 shutdown_connections 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.662+0000 7f5989400700 1 -- 172.21.15.117:0/836969244 wait complete. 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.663+0000 7f5989400700 1 Processor -- start 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.663+0000 7f5989400700 1 -- start start 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.663+0000 7f5989400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5984100e20 0x7f59840731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f5989400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 0x7f5984073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f5989400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5984073c60 con 0x7f59841017d0 2024-04-20T11:03:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f5989400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5984073da0 con 0x7f5984100e20 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f597be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 0x7f5984073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f597be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 0x7f5984073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54122/0 (socket says 172.21.15.117:54122) 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f597be00700 1 -- 172.21.15.117:0/3702047415 learned_addr learned my addr 172.21.15.117:0/3702047415 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f597be00700 1 -- 172.21.15.117:0/3702047415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5984100e20 msgr2=0x7f59840731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f5982800700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5984100e20 0x7f59840731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f597be00700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5984100e20 0x7f59840731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f597be00700 1 -- 172.21.15.117:0/3702047415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0095f0 con 0x7f59841017d0 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.664+0000 7f597be00700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 0x7f5984073720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f596c005b40 tx=0x7f596c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.665+0000 7f597b400700 1 -- 172.21.15.117:0/3702047415 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f596c01d070 con 0x7f59841017d0 2024-04-20T11:03:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.665+0000 7f597b400700 1 -- 172.21.15.117:0/3702047415 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f596c00f460 con 0x7f59841017d0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.665+0000 7f597b400700 1 -- 172.21.15.117:0/3702047415 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f596c017630 con 0x7f59841017d0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.665+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5984074020 con 0x7f59841017d0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.665+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59841a2f60 con 0x7f59841017d0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.666+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5984109220 con 0x7f59841017d0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.667+0000 7f597b400700 1 -- 172.21.15.117:0/3702047415 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f596c003a60 con 0x7f59841017d0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.668+0000 7f597b400700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f596806cce0 0x7f596806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.668+0000 7f597b400700 1 -- 172.21.15.117:0/3702047415 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f596c08d370 con 0x7f59841017d0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.668+0000 7f5982800700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f596806cce0 0x7f596806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.669+0000 7f5982800700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f596806cce0 0x7f596806f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5974005950 tx=0x7f5974005cf0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.673+0000 7f597b400700 1 -- 172.21.15.117:0/3702047415 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f596c057a60 con 0x7f59841017d0 2024-04-20T11:03:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.895+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5984102300 con 0x7f59841017d0 2024-04-20T11:03:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.896+0000 7f597b400700 1 -- 172.21.15.117:0/3702047415 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f596c05b080 con 0x7f59841017d0 2024-04-20T11:03:50.897 INFO:teuthology.orchestra.run.smithi117.stdout:38654705714 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f596806cce0 msgr2=0x7f596806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f596806cce0 0x7f596806f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5974005950 tx=0x7f5974005cf0 comp rx=0 tx=0).stop 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 msgr2=0x7f5984073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 0x7f5984073720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f596c005b40 tx=0x7f596c00be20 comp rx=0 tx=0).stop 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 shutdown_connections 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f596806cce0 0x7f596806f1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5984100e20 0x7f59840731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 --2- 172.21.15.117:0/3702047415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59841017d0 0x7f5984073720 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 >> 172.21.15.117:0/3702047415 conn(0x7f59840fc6d0 msgr2=0x7f59840fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 shutdown_connections 2024-04-20T11:03:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:50.899+0000 7f5989400700 1 -- 172.21.15.117:0/3702047415 wait complete. 2024-04-20T11:03:50.918 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:50 smithi117 ceph-mon[15659]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:51.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.031+0000 7f8d36600700 1 -- 172.21.15.117:0/3580969891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 msgr2=0x7f8d30104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:51.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.031+0000 7f8d36600700 1 --2- 172.21.15.117:0/3580969891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 0x7f8d30104e80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009910 tx=0x7f8d1c009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:51.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.031+0000 7f8d36600700 1 -- 172.21.15.117:0/3580969891 shutdown_connections 2024-04-20T11:03:51.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.031+0000 7f8d36600700 1 --2- 172.21.15.117:0/3580969891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 0x7f8d30104e80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:51.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.031+0000 7f8d36600700 1 --2- 172.21.15.117:0/3580969891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d30103110 0x7f8d301034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:51.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.031+0000 7f8d36600700 1 -- 172.21.15.117:0/3580969891 >> 172.21.15.117:0/3580969891 conn(0x7f8d300fe880 msgr2=0x7f8d30100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:51.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.032+0000 7f8d36600700 1 -- 172.21.15.117:0/3580969891 shutdown_connections 2024-04-20T11:03:51.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.032+0000 7f8d36600700 1 -- 172.21.15.117:0/3580969891 wait complete. 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.033+0000 7f8d36600700 1 Processor -- start 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.033+0000 7f8d36600700 1 -- start start 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.033+0000 7f8d36600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d30103110 0x7f8d3019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.033+0000 7f8d36600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 0x7f8d3019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.033+0000 7f8d36600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3019f5b0 con 0x7f8d30104a00 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.033+0000 7f8d36600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30198b90 con 0x7f8d30103110 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.034+0000 7f8d2ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 0x7f8d3019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.034+0000 7f8d2ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 0x7f8d3019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54128/0 (socket says 172.21.15.117:54128) 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.034+0000 7f8d2ea00700 1 -- 172.21.15.117:0/413399516 learned_addr learned my addr 172.21.15.117:0/413399516 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:51.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.034+0000 7f8d2f400700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d30103110 0x7f8d3019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:51.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.034+0000 7f8d2ea00700 1 -- 172.21.15.117:0/413399516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d30103110 msgr2=0x7f8d3019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:51.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.034+0000 7f8d2ea00700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d30103110 0x7f8d3019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:51.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.034+0000 7f8d2ea00700 1 -- 172.21.15.117:0/413399516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0095f0 con 0x7f8d30104a00 2024-04-20T11:03:51.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.034+0000 7f8d2ea00700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 0x7f8d3019ef90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0092e0 tx=0x7f8d1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:51.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.035+0000 7f8d27e00700 1 -- 172.21.15.117:0/413399516 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d1c01d070 con 0x7f8d30104a00 2024-04-20T11:03:51.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.035+0000 7f8d27e00700 1 -- 172.21.15.117:0/413399516 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d1c00f460 con 0x7f8d30104a00 2024-04-20T11:03:51.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.035+0000 7f8d27e00700 1 -- 172.21.15.117:0/413399516 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d1c017630 con 0x7f8d30104a00 2024-04-20T11:03:51.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.035+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d30198e10 con 0x7f8d30104a00 2024-04-20T11:03:51.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.035+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d30199380 con 0x7f8d30104a00 2024-04-20T11:03:51.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.036+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3010b480 con 0x7f8d30104a00 2024-04-20T11:03:51.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.037+0000 7f8d27e00700 1 -- 172.21.15.117:0/413399516 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f8d1c00f5d0 con 0x7f8d30104a00 2024-04-20T11:03:51.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.038+0000 7f8d27e00700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8d1406ccf0 0x7f8d1406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:51.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.038+0000 7f8d27e00700 1 -- 172.21.15.117:0/413399516 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8d1c08d410 con 0x7f8d30104a00 2024-04-20T11:03:51.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.038+0000 7f8d2f400700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8d1406ccf0 0x7f8d1406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:51.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.039+0000 7f8d2f400700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8d1406ccf0 0x7f8d1406f1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d20005fd0 tx=0x7f8d2000a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:51.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.043+0000 7f8d27e00700 1 -- 172.21.15.117:0/413399516 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8d1c05b590 con 0x7f8d30104a00 2024-04-20T11:03:51.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:50 smithi173 ceph-mon[23800]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:51.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.266+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8d30104510 con 0x7f8d30104a00 2024-04-20T11:03:51.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.267+0000 7f8d27e00700 1 -- 172.21.15.117:0/413399516 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d1c05b120 con 0x7f8d30104a00 2024-04-20T11:03:51.268 INFO:teuthology.orchestra.run.smithi117.stdout:154618822676 2024-04-20T11:03:51.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8d1406ccf0 msgr2=0x7f8d1406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:51.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8d1406ccf0 0x7f8d1406f1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d20005fd0 tx=0x7f8d2000a560 comp rx=0 tx=0).stop 2024-04-20T11:03:51.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 msgr2=0x7f8d3019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:51.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 0x7f8d3019ef90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0092e0 tx=0x7f8d1c00be20 comp rx=0 tx=0).stop 2024-04-20T11:03:51.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 shutdown_connections 2024-04-20T11:03:51.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8d1406ccf0 0x7f8d1406f1b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:51.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d30103110 0x7f8d3019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:51.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 --2- 172.21.15.117:0/413399516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d30104a00 0x7f8d3019ef90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:51.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 >> 172.21.15.117:0/413399516 conn(0x7f8d300fe880 msgr2=0x7f8d300ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:51.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 shutdown_connections 2024-04-20T11:03:51.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:51.271+0000 7f8d36600700 1 -- 172.21.15.117:0/413399516 wait complete. 2024-04-20T11:03:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:51 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3702047415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T11:03:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:51 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/413399516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T11:03:52.069 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822673 got 154618822676 for osd.4 2024-04-20T11:03:52.069 DEBUG:teuthology.parallel:result is None 2024-04-20T11:03:52.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:51 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3702047415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T11:03:52.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:51 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/413399516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T11:03:53.036 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:52 smithi117 ceph-mon[15659]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:52 smithi173 ceph-mon[23800]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.189+0000 7f7118400700 1 -- 172.21.15.117:0/3449772146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073ba0 msgr2=0x7f7110111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.189+0000 7f7118400700 1 --2- 172.21.15.117:0/3449772146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073ba0 0x7f7110111680 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009910 tx=0x7f70fc009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.189+0000 7f7118400700 1 -- 172.21.15.117:0/3449772146 shutdown_connections 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.189+0000 7f7118400700 1 --2- 172.21.15.117:0/3449772146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073ba0 0x7f7110111680 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.189+0000 7f7118400700 1 --2- 172.21.15.117:0/3449772146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7110073280 0x7f7110073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.189+0000 7f7118400700 1 -- 172.21.15.117:0/3449772146 >> 172.21.15.117:0/3449772146 conn(0x7f71100fc6c0 msgr2=0x7f71100feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.189+0000 7f7118400700 1 -- 172.21.15.117:0/3449772146 shutdown_connections 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.189+0000 7f7118400700 1 -- 172.21.15.117:0/3449772146 wait complete. 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.190+0000 7f7118400700 1 Processor -- start 2024-04-20T11:03:53.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.191+0000 7f7118400700 1 -- start start 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.191+0000 7f7118400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073280 0x7f711019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.191+0000 7f7118400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7110073ba0 0x7f711019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.191+0000 7f7118400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711019dcc0 con 0x7f7110073280 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.191+0000 7f7118400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101a1990 con 0x7f7110073ba0 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.191+0000 7f7115a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073280 0x7f711019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f7115a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073280 0x7f711019d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54156/0 (socket says 172.21.15.117:54156) 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f7115a00700 1 -- 172.21.15.117:0/3315073470 learned_addr learned my addr 172.21.15.117:0/3315073470 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f7115000700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7110073ba0 0x7f711019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f7115a00700 1 -- 172.21.15.117:0/3315073470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7110073ba0 msgr2=0x7f711019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f7115a00700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7110073ba0 0x7f711019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:53.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f7115a00700 1 -- 172.21.15.117:0/3315073470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0095f0 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f7115a00700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073280 0x7f711019d0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f710400ada0 tx=0x7f710400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f710e000700 1 -- 172.21.15.117:0/3315073470 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f710400f070 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f710e000700 1 -- 172.21.15.117:0/3315073470 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7104007bf0 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71101a1c70 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.192+0000 7f710e000700 1 -- 172.21.15.117:0/3315073470 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7104005690 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.193+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71101a21c0 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.194+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71100749c0 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.195+0000 7f710e000700 1 -- 172.21.15.117:0/3315073470 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f7104005900 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.196+0000 7f710e000700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f70f806cce0 0x7f70f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.196+0000 7f710e000700 1 -- 172.21.15.117:0/3315073470 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f710408b920 con 0x7f7110073280 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.196+0000 7f7115000700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f70f806cce0 0x7f70f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.197+0000 7f7115000700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f70f806cce0 0x7f70f806f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005100 tx=0x7f70fc01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.200+0000 7f710e000700 1 -- 172.21.15.117:0/3315073470 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7104056030 con 0x7f7110073280 2024-04-20T11:03:53.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.421+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7110114440 con 0x7f7110073280 2024-04-20T11:03:53.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.422+0000 7f710e000700 1 -- 172.21.15.117:0/3315073470 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7104059650 con 0x7f7110073280 2024-04-20T11:03:53.423 INFO:teuthology.orchestra.run.smithi117.stdout:115964117020 2024-04-20T11:03:53.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f70f806cce0 msgr2=0x7f70f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:53.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f70f806cce0 0x7f70f806f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005100 tx=0x7f70fc01a040 comp rx=0 tx=0).stop 2024-04-20T11:03:53.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073280 msgr2=0x7f711019d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:53.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073280 0x7f711019d0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f710400ada0 tx=0x7f710400c330 comp rx=0 tx=0).stop 2024-04-20T11:03:53.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 shutdown_connections 2024-04-20T11:03:53.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f70f806cce0 0x7f70f806f1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:53.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7110073280 0x7f711019d0a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:53.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 --2- 172.21.15.117:0/3315073470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7110073ba0 0x7f711019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:53.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 >> 172.21.15.117:0/3315073470 conn(0x7f71100fc6c0 msgr2=0x7f71101008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:53.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 shutdown_connections 2024-04-20T11:03:53.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:53.425+0000 7f7118400700 1 -- 172.21.15.117:0/3315073470 wait complete. 2024-04-20T11:03:53.487 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705712 got 38654705714 for osd.0 2024-04-20T11:03:53.487 DEBUG:teuthology.parallel:result is None 2024-04-20T11:03:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:53 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3315073470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T11:03:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:53 smithi117 ceph-mon[15659]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:54.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3315073470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T11:03:54.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:53 smithi173 ceph-mon[23800]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:54.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.161+0000 7fd2e2a00700 1 -- 172.21.15.117:0/3451085245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1069d0 msgr2=0x7fd2dc106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:54.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.161+0000 7fd2e2a00700 1 --2- 172.21.15.117:0/3451085245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1069d0 0x7fd2dc106db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009910 tx=0x7fd2d0009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:54.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.161+0000 7fd2e2a00700 1 -- 172.21.15.117:0/3451085245 shutdown_connections 2024-04-20T11:03:54.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.161+0000 7fd2e2a00700 1 --2- 172.21.15.117:0/3451085245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2dc1009d0 0x7fd2dc100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:54.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.161+0000 7fd2e2a00700 1 --2- 172.21.15.117:0/3451085245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1069d0 0x7fd2dc106db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:54.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.161+0000 7fd2e2a00700 1 -- 172.21.15.117:0/3451085245 >> 172.21.15.117:0/3451085245 conn(0x7fd2dc0fc500 msgr2=0x7fd2dc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:54.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.161+0000 7fd2e2a00700 1 -- 172.21.15.117:0/3451085245 shutdown_connections 2024-04-20T11:03:54.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.161+0000 7fd2e2a00700 1 -- 172.21.15.117:0/3451085245 wait complete. 2024-04-20T11:03:54.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.162+0000 7fd2e2a00700 1 Processor -- start 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.163+0000 7fd2e2a00700 1 -- start start 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.163+0000 7fd2e2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1009d0 0x7fd2dc19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.163+0000 7fd2e2a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2dc1069d0 0x7fd2dc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.163+0000 7fd2e2a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc19d240 con 0x7fd2dc1009d0 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.163+0000 7fd2e2a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1967b0 con 0x7fd2dc1069d0 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.163+0000 7fd2e1600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1009d0 0x7fd2dc19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.163+0000 7fd2e1600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1009d0 0x7fd2dc19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53162/0 (socket says 172.21.15.117:53162) 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.163+0000 7fd2e1600700 1 -- 172.21.15.117:0/1697395304 learned_addr learned my addr 172.21.15.117:0/1697395304 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.164+0000 7fd2e0c00700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2dc1069d0 0x7fd2dc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.164+0000 7fd2e1600700 1 -- 172.21.15.117:0/1697395304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2dc1069d0 msgr2=0x7fd2dc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.164+0000 7fd2e1600700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2dc1069d0 0x7fd2dc19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:54.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.164+0000 7fd2e1600700 1 -- 172.21.15.117:0/1697395304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d00095f0 con 0x7fd2dc1009d0 2024-04-20T11:03:54.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.164+0000 7fd2e1600700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1009d0 0x7fd2dc19c670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00092e0 tx=0x7fd2d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:54.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.164+0000 7fd2da000700 1 -- 172.21.15.117:0/1697395304 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2d001d070 con 0x7fd2dc1009d0 2024-04-20T11:03:54.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.164+0000 7fd2da000700 1 -- 172.21.15.117:0/1697395304 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2d000f460 con 0x7fd2dc1009d0 2024-04-20T11:03:54.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.164+0000 7fd2da000700 1 -- 172.21.15.117:0/1697395304 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2d0017630 con 0x7fd2dc1009d0 2024-04-20T11:03:54.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.165+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc196a30 con 0x7fd2dc1009d0 2024-04-20T11:03:54.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.165+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc196f20 con 0x7fd2dc1009d0 2024-04-20T11:03:54.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.166+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc108fd0 con 0x7fd2dc1009d0 2024-04-20T11:03:54.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.167+0000 7fd2da000700 1 -- 172.21.15.117:0/1697395304 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fd2d000f5d0 con 0x7fd2dc1009d0 2024-04-20T11:03:54.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.168+0000 7fd2da000700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd2c406cce0 0x7fd2c406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:54.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.168+0000 7fd2da000700 1 -- 172.21.15.117:0/1697395304 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd2d008d430 con 0x7fd2dc1009d0 2024-04-20T11:03:54.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.168+0000 7fd2e0c00700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd2c406cce0 0x7fd2c406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:54.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.169+0000 7fd2e0c00700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd2c406cce0 0x7fd2c406f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8005fd0 tx=0x7fd2c800a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:54.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.173+0000 7fd2da000700 1 -- 172.21.15.117:0/1697395304 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd2d005b5b0 con 0x7fd2dc1009d0 2024-04-20T11:03:54.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.400+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd2dc102110 con 0x7fd2dc1009d0 2024-04-20T11:03:54.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.401+0000 7fd2da000700 1 -- 172.21.15.117:0/1697395304 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd2d005b140 con 0x7fd2dc1009d0 2024-04-20T11:03:54.402 INFO:teuthology.orchestra.run.smithi117.stdout:81604378658 2024-04-20T11:03:54.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd2c406cce0 msgr2=0x7fd2c406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:54.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd2c406cce0 0x7fd2c406f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8005fd0 tx=0x7fd2c800a560 comp rx=0 tx=0).stop 2024-04-20T11:03:54.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1009d0 msgr2=0x7fd2dc19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:54.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1009d0 0x7fd2dc19c670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00092e0 tx=0x7fd2d000be20 comp rx=0 tx=0).stop 2024-04-20T11:03:54.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 shutdown_connections 2024-04-20T11:03:54.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd2c406cce0 0x7fd2c406f1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:54.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2dc1009d0 0x7fd2dc19c670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:54.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 --2- 172.21.15.117:0/1697395304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2dc1069d0 0x7fd2dc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:54.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 >> 172.21.15.117:0/1697395304 conn(0x7fd2dc0fc500 msgr2=0x7fd2dc105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:54.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 shutdown_connections 2024-04-20T11:03:54.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:54.404+0000 7fd2e2a00700 1 -- 172.21.15.117:0/1697395304 wait complete. 2024-04-20T11:03:54.637 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117017 got 115964117020 for osd.3 2024-04-20T11:03:54.637 DEBUG:teuthology.parallel:result is None 2024-04-20T11:03:54.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:54 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1697395304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T11:03:55.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:54 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1697395304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T11:03:55.271 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378655 got 81604378658 for osd.2 2024-04-20T11:03:55.271 DEBUG:teuthology.parallel:result is None 2024-04-20T11:03:55.271 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-20T11:03:55.271 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph pg dump --format=json 2024-04-20T11:03:55.657 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:56.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:55 smithi117 ceph-mon[15659]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:56.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:55 smithi173 ceph-mon[23800]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:57.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.693+0000 7f6282e00700 1 -- 172.21.15.117:0/57056347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 msgr2=0x7f627c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:57.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.693+0000 7f6282e00700 1 --2- 172.21.15.117:0/57056347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 0x7f627c1034f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f626c009910 tx=0x7f626c009c20 comp rx=0 tx=0).stop 2024-04-20T11:03:57.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.693+0000 7f6282e00700 1 -- 172.21.15.117:0/57056347 shutdown_connections 2024-04-20T11:03:57.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.693+0000 7f6282e00700 1 --2- 172.21.15.117:0/57056347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f627c104a00 0x7f627c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:57.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.693+0000 7f6282e00700 1 --2- 172.21.15.117:0/57056347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 0x7f627c1034f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:57.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.693+0000 7f6282e00700 1 -- 172.21.15.117:0/57056347 >> 172.21.15.117:0/57056347 conn(0x7f627c0fe880 msgr2=0x7f627c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:57.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.693+0000 7f6282e00700 1 -- 172.21.15.117:0/57056347 shutdown_connections 2024-04-20T11:03:57.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.693+0000 7f6282e00700 1 -- 172.21.15.117:0/57056347 wait complete. 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.694+0000 7f6282e00700 1 Processor -- start 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.694+0000 7f6282e00700 1 -- start start 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f6282e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 0x7f627c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f6282e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f627c104a00 0x7f627c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f6282e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1999e0 con 0x7f627c103110 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f6282e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c19d660 con 0x7f627c104a00 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f627be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 0x7f627c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f627be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 0x7f627c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53174/0 (socket says 172.21.15.117:53174) 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f627be00700 1 -- 172.21.15.117:0/1889210324 learned_addr learned my addr 172.21.15.117:0/1889210324 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:03:57.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f627be00700 1 -- 172.21.15.117:0/1889210324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f627c104a00 msgr2=0x7f627c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:57.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f627be00700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f627c104a00 0x7f627c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:57.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.695+0000 7f627be00700 1 -- 172.21.15.117:0/1889210324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c0095f0 con 0x7f627c103110 2024-04-20T11:03:57.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.696+0000 7f627be00700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 0x7f627c198dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:57.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.696+0000 7f6278c00700 1 -- 172.21.15.117:0/1889210324 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626c01d070 con 0x7f627c103110 2024-04-20T11:03:57.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.696+0000 7f6278c00700 1 -- 172.21.15.117:0/1889210324 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f626c0056e0 con 0x7f627c103110 2024-04-20T11:03:57.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.696+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c19d8e0 con 0x7f627c103110 2024-04-20T11:03:57.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.696+0000 7f6278c00700 1 -- 172.21.15.117:0/1889210324 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626c00fe80 con 0x7f627c103110 2024-04-20T11:03:57.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.696+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c19de50 con 0x7f627c103110 2024-04-20T11:03:57.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.697+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c10b480 con 0x7f627c103110 2024-04-20T11:03:57.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.698+0000 7f6278c00700 1 -- 172.21.15.117:0/1889210324 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f626c00f6f0 con 0x7f627c103110 2024-04-20T11:03:57.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.699+0000 7f6278c00700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f626006ccf0 0x7f626006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:03:57.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.699+0000 7f6278c00700 1 -- 172.21.15.117:0/1889210324 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f626c08d2f0 con 0x7f627c103110 2024-04-20T11:03:57.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.699+0000 7f627b400700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f626006ccf0 0x7f626006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:03:57.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.700+0000 7f627b400700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f626006ccf0 0x7f626006f1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6268005950 tx=0x7f62680058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:03:57.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.704+0000 7f6278c00700 1 -- 172.21.15.117:0/1889210324 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f626c0579e0 con 0x7f627c103110 2024-04-20T11:03:57.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.919+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f627c19a080 con 0x7f626006ccf0 2024-04-20T11:03:57.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.922+0000 7f6278c00700 1 -- 172.21.15.117:0/1889210324 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27085 (secure 0 0 0) 0x7f627c19a080 con 0x7f626006ccf0 2024-04-20T11:03:57.927 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:03:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.924+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f626006ccf0 msgr2=0x7f626006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.924+0000 7f6282e00700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f626006ccf0 0x7f626006f1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6268005950 tx=0x7f62680058e0 comp rx=0 tx=0).stop 2024-04-20T11:03:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.924+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 msgr2=0x7f627c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:03:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.924+0000 7f6282e00700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 0x7f627c198dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c00be20 comp rx=0 tx=0).stop 2024-04-20T11:03:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.925+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 shutdown_connections 2024-04-20T11:03:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.925+0000 7f6282e00700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f626006ccf0 0x7f626006f1b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.925+0000 7f6282e00700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627c103110 0x7f627c198dc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:57.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.925+0000 7f6282e00700 1 --2- 172.21.15.117:0/1889210324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f627c104a00 0x7f627c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:03:57.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.925+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 >> 172.21.15.117:0/1889210324 conn(0x7f627c0fe880 msgr2=0x7f627c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:03:57.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.925+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 shutdown_connections 2024-04-20T11:03:57.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:03:57.925+0000 7f6282e00700 1 -- 172.21.15.117:0/1889210324 wait complete. 2024-04-20T11:03:57.929 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-04-20T11:03:58.686 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-04-20T11:03:57.542971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":190676,"kb_used_data":3968,"kb_used_omap":9,"kb_used_meta":186614,"kb_avail":562280236,"statfs":{"total":575970213888,"available":575774961664,"internally_reserved":0,"allocated":4063232,"data_stored":2655663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":191093414},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003287"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T11:02:56.601404+0000","last_change":"2024-04-20T11:02:21.645283+0000","last_active":"2024-04-20T11:02:56.601404+0000","last_peered":"2024-04-20T11:02:56.601404+0000","last_clean":"2024-04-20T11:02:56.601404+0000","last_became_active":"2024-04-20T11:02:21.644737+0000","last_became_peered":"2024-04-20T11:02:21.644737+0000","last_unstale":"2024-04-20T11:02:56.601404+0000","last_undegraded":"2024-04-20T11:02:56.601404+0000","last_fullsized":"2024-04-20T11:02:56.601404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:01:14.631869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:01:14.631869+0000","last_clean_scrub_stamp":"2024-04-20T11:01:14.631869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:56.504206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":41,"seq":176093659150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31488,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713664,"statfs":{"total":95995035648,"available":95962791936,"internally_reserved":0,"allocated":380928,"data_stored":147538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]}]},{"osd":4,"up_from":36,"seq":154618822677,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31496,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713656,"statfs":{"total":95995035648,"available":95962783744,"internally_reserved":0,"allocated":380928,"data_stored":147538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 11:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.95199999999999996}]},{"osd":1,"last update":"Sat Apr 20 11:03:26 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.389}]},{"osd":2,"last update":"Sat Apr 20 11:03:26 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.0469999999999999}]},{"osd":3,"last update":"Sat Apr 20 11:03:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.1499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]}]},{"osd":3,"up_from":27,"seq":115964117021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":974848,"data_stored":737906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 11:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.92600000000000005,"15min":0.91600000000000004},"min":{"1min":0.58999999999999997,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.93500000000000005,"15min":0.91900000000000004},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.66000000000000003}]},{"osd":1,"last update":"Sat Apr 20 11:03:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.96499999999999997,"15min":0.93999999999999995},"min":{"1min":0.66500000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.621},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.998,"15min":0.99199999999999999},"min":{"1min":0.81799999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.876}]},{"osd":2,"last update":"Sat Apr 20 11:03:52 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.0649999999999999,"15min":1.056},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.351,"5min":1.385,"15min":1.385},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.123,"15min":1.113},"min":{"1min":0.70599999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.4099999999999999,"5min":1.819,"15min":1.819},"last":1.2669999999999999}]},{"osd":4,"last update":"Sat Apr 20 11:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.4099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717764,"statfs":{"total":95995035648,"available":95966990336,"internally_reserved":0,"allocated":376832,"data_stored":146869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 11:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.82399999999999995,"15min":0.81599999999999995},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.91000000000000003,"15min":0.89900000000000002},"min":{"1min":0.68700000000000006,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0389999999999999}]},{"osd":1,"last update":"Sat Apr 20 11:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91600000000000004,"15min":0.91400000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.98299999999999998,"15min":0.97499999999999998},"min":{"1min":0.84599999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.123}]},{"osd":3,"last update":"Sat Apr 20 11:02:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0900000000000001}]},{"osd":4,"last update":"Sat Apr 20 11:03:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.294}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]}]},{"osd":0,"up_from":9,"seq":38654705716,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":974848,"data_stored":737906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 11:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68000000000000005,"15min":0.63200000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0880000000000001,"5min":1.127,"15min":1.127},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.73999999999999999,"15min":0.67800000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.159,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Apr 20 11:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.85799999999999998,"15min":0.83999999999999997},"min":{"1min":0.46500000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.92700000000000005,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.48399999999999999}]},{"osd":3,"last update":"Sat Apr 20 11:03:53 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.0920000000000001,"15min":1.0820000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.0800000000000001,"15min":1.087},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.323,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Sat Apr 20 11:03:28 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.0289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":737906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 11:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.03},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.76900000000000002,"15min":0.71499999999999997},"min":{"1min":0.56100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":1.216,"5min":1.359,"15min":1.359},"last":0.86499999999999999}]},{"osd":2,"last update":"Sat Apr 20 11:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90100000000000002,"15min":0.89600000000000002},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4650000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.034,"5min":0.99299999999999999,"15min":0.98599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3380000000000001,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.99299999999999999}]},{"osd":3,"last update":"Sat Apr 20 11:03:50 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":0.998,"15min":0.98599999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.268},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.101,"15min":1.0369999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"last":1.3560000000000001}]},{"osd":4,"last update":"Sat Apr 20 11:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.083},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.5109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:03:58.687 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph pg dump --format=json 2024-04-20T11:03:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:58 smithi117 ceph-mon[15659]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:58 smithi173 ceph-mon[23800]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:03:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:03:59.702 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:59 smithi117 ceph-mon[15659]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:03:59.702 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:03:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:03:59.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:59 smithi173 ceph-mon[23800]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:03:59.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:03:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:04:00.755 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:00 smithi117 ceph-mon[15659]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:00 smithi173 ceph-mon[23800]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.277+0000 7fdbe2e00700 1 -- 172.21.15.117:0/1051217267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 msgr2=0x7fdbdc103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.277+0000 7fdbe2e00700 1 --2- 172.21.15.117:0/1051217267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 0x7fdbdc103400 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009910 tx=0x7fdbcc009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.277+0000 7fdbe2e00700 1 -- 172.21.15.117:0/1051217267 shutdown_connections 2024-04-20T11:04:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.277+0000 7fdbe2e00700 1 --2- 172.21.15.117:0/1051217267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbdc1039d0 0x7fdbdc107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.277+0000 7fdbe2e00700 1 --2- 172.21.15.117:0/1051217267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 0x7fdbdc103400 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.277+0000 7fdbe2e00700 1 -- 172.21.15.117:0/1051217267 >> 172.21.15.117:0/1051217267 conn(0x7fdbdc0fe8b0 msgr2=0x7fdbdc100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:01.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.277+0000 7fdbe2e00700 1 -- 172.21.15.117:0/1051217267 shutdown_connections 2024-04-20T11:04:01.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.277+0000 7fdbe2e00700 1 -- 172.21.15.117:0/1051217267 wait complete. 2024-04-20T11:04:01.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.278+0000 7fdbe2e00700 1 Processor -- start 2024-04-20T11:04:01.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.278+0000 7fdbe2e00700 1 -- start start 2024-04-20T11:04:01.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.279+0000 7fdbe2e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 0x7fdbdc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.279+0000 7fdbe2e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbdc1039d0 0x7fdbdc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.279+0000 7fdbe2e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc19f530 con 0x7fdbdc103020 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.279+0000 7fdbe2e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc198b10 con 0x7fdbdc1039d0 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.279+0000 7fdbdbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 0x7fdbdc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.279+0000 7fdbdbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 0x7fdbdc19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53188/0 (socket says 172.21.15.117:53188) 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.279+0000 7fdbdbe00700 1 -- 172.21.15.117:0/2614146082 learned_addr learned my addr 172.21.15.117:0/2614146082 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbdb400700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbdc1039d0 0x7fdbdc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbdbe00700 1 -- 172.21.15.117:0/2614146082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbdc1039d0 msgr2=0x7fdbdc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbdbe00700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbdc1039d0 0x7fdbdc19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbdbe00700 1 -- 172.21.15.117:0/2614146082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc0095f0 con 0x7fdbdc103020 2024-04-20T11:04:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbdbe00700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 0x7fdbdc19e9d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000c00 tx=0x7fdbcc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbd8c00700 1 -- 172.21.15.117:0/2614146082 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbcc01d070 con 0x7fdbdc103020 2024-04-20T11:04:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc198d90 con 0x7fdbdc103020 2024-04-20T11:04:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbd8c00700 1 -- 172.21.15.117:0/2614146082 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbcc0056e0 con 0x7fdbdc103020 2024-04-20T11:04:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.280+0000 7fdbd8c00700 1 -- 172.21.15.117:0/2614146082 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbcc00fe80 con 0x7fdbdc103020 2024-04-20T11:04:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.281+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc199280 con 0x7fdbdc103020 2024-04-20T11:04:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.282+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc10b3c0 con 0x7fdbdc103020 2024-04-20T11:04:01.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.283+0000 7fdbd8c00700 1 -- 172.21.15.117:0/2614146082 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fdbcc00f6f0 con 0x7fdbdc103020 2024-04-20T11:04:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.284+0000 7fdbd8c00700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdbc006cce0 0x7fdbc006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.284+0000 7fdbd8c00700 1 -- 172.21.15.117:0/2614146082 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdbcc08d5c0 con 0x7fdbdc103020 2024-04-20T11:04:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.284+0000 7fdbdb400700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdbc006cce0 0x7fdbc006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.285+0000 7fdbdb400700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdbc006cce0 0x7fdbc006f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005950 tx=0x7fdbc80058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:01.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.290+0000 7fdbd8c00700 1 -- 172.21.15.117:0/2614146082 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdbcc05b740 con 0x7fdbdc103020 2024-04-20T11:04:01.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.508+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbdc10fff0 con 0x7fdbc006cce0 2024-04-20T11:04:01.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.511+0000 7fdbd8c00700 1 -- 172.21.15.117:0/2614146082 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27071 (secure 0 0 0) 0x7fdbdc10fff0 con 0x7fdbc006cce0 2024-04-20T11:04:01.513 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.514+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdbc006cce0 msgr2=0x7fdbc006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.514+0000 7fdbe2e00700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdbc006cce0 0x7fdbc006f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005950 tx=0x7fdbc80058e0 comp rx=0 tx=0).stop 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.514+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 msgr2=0x7fdbdc19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.514+0000 7fdbe2e00700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 0x7fdbdc19e9d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000c00 tx=0x7fdbcc00be20 comp rx=0 tx=0).stop 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.515+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 shutdown_connections 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.515+0000 7fdbe2e00700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdbc006cce0 0x7fdbc006f1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.515+0000 7fdbe2e00700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc103020 0x7fdbdc19e9d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.515+0000 7fdbe2e00700 1 --2- 172.21.15.117:0/2614146082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbdc1039d0 0x7fdbdc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.515+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 >> 172.21.15.117:0/2614146082 conn(0x7fdbdc0fe8b0 msgr2=0x7fdbdc0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.515+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 shutdown_connections 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:01.515+0000 7fdbe2e00700 1 -- 172.21.15.117:0/2614146082 wait complete. 2024-04-20T11:04:01.516 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-04-20T11:04:02.396 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-04-20T11:03:59.543446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":190676,"kb_used_data":3968,"kb_used_omap":9,"kb_used_meta":186614,"kb_avail":562280236,"statfs":{"total":575970213888,"available":575774961664,"internally_reserved":0,"allocated":4063232,"data_stored":2655663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":191093414},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003162"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T11:02:56.601404+0000","last_change":"2024-04-20T11:02:21.645283+0000","last_active":"2024-04-20T11:02:56.601404+0000","last_peered":"2024-04-20T11:02:56.601404+0000","last_clean":"2024-04-20T11:02:56.601404+0000","last_became_active":"2024-04-20T11:02:21.644737+0000","last_became_peered":"2024-04-20T11:02:21.644737+0000","last_unstale":"2024-04-20T11:02:56.601404+0000","last_undegraded":"2024-04-20T11:02:56.601404+0000","last_fullsized":"2024-04-20T11:02:56.601404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:01:14.631869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:01:14.631869+0000","last_clean_scrub_stamp":"2024-04-20T11:01:14.631869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:56.504206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":41,"seq":176093659150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31488,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713664,"statfs":{"total":95995035648,"available":95962791936,"internally_reserved":0,"allocated":380928,"data_stored":147538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]}]},{"osd":4,"up_from":36,"seq":154618822678,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31496,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713656,"statfs":{"total":95995035648,"available":95962783744,"internally_reserved":0,"allocated":380928,"data_stored":147538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 11:03:26 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.755},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.016}]},{"osd":1,"last update":"Sat Apr 20 11:03:26 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.129}]},{"osd":2,"last update":"Sat Apr 20 11:03:26 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.1659999999999999}]},{"osd":3,"last update":"Sat Apr 20 11:03:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]}]},{"osd":3,"up_from":27,"seq":115964117021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":974848,"data_stored":737906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 11:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.92600000000000005,"15min":0.91600000000000004},"min":{"1min":0.58999999999999997,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.93500000000000005,"15min":0.91900000000000004},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.66000000000000003}]},{"osd":1,"last update":"Sat Apr 20 11:03:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.96499999999999997,"15min":0.93999999999999995},"min":{"1min":0.66500000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.621},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.998,"15min":0.99199999999999999},"min":{"1min":0.81799999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.876}]},{"osd":2,"last update":"Sat Apr 20 11:03:52 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.0649999999999999,"15min":1.056},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.351,"5min":1.385,"15min":1.385},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.123,"15min":1.113},"min":{"1min":0.70599999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.4099999999999999,"5min":1.819,"15min":1.819},"last":1.2669999999999999}]},{"osd":4,"last update":"Sat Apr 20 11:03:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.4099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717764,"statfs":{"total":95995035648,"available":95966990336,"internally_reserved":0,"allocated":376832,"data_stored":146869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 11:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.82399999999999995,"15min":0.81599999999999995},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.91000000000000003,"15min":0.89900000000000002},"min":{"1min":0.68700000000000006,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0389999999999999}]},{"osd":1,"last update":"Sat Apr 20 11:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91600000000000004,"15min":0.91400000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.98299999999999998,"15min":0.97499999999999998},"min":{"1min":0.84599999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.123}]},{"osd":3,"last update":"Sat Apr 20 11:02:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0900000000000001}]},{"osd":4,"last update":"Sat Apr 20 11:03:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.294}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]}]},{"osd":0,"up_from":9,"seq":38654705716,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":974848,"data_stored":737906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 11:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68000000000000005,"15min":0.63200000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0880000000000001,"5min":1.127,"15min":1.127},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.73999999999999999,"15min":0.67800000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.159,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Apr 20 11:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.85799999999999998,"15min":0.83999999999999997},"min":{"1min":0.46500000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.92700000000000005,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.48399999999999999}]},{"osd":3,"last update":"Sat Apr 20 11:03:53 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.0920000000000001,"15min":1.0820000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.0800000000000001,"15min":1.087},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.323,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Sat Apr 20 11:03:28 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.0289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":737906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 11:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.03},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.76900000000000002,"15min":0.71499999999999997},"min":{"1min":0.56100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":1.216,"5min":1.359,"15min":1.359},"last":0.86499999999999999}]},{"osd":2,"last update":"Sat Apr 20 11:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.90100000000000002,"15min":0.89600000000000002},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4650000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.034,"5min":0.99299999999999999,"15min":0.98599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3380000000000001,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.99299999999999999}]},{"osd":3,"last update":"Sat Apr 20 11:03:50 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":0.998,"15min":0.98599999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.268},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.101,"15min":1.0369999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"last":1.3560000000000001}]},{"osd":4,"last update":"Sat Apr 20 11:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.083},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.5109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:04:02.397 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-20T11:04:02.397 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T11:04:02.397 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-20T11:04:02.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph health --format=json 2024-04-20T11:04:02.796 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:02.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:02 smithi117 ceph-mon[15659]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:04:02.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:02 smithi117 ceph-mon[15659]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:02.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:02 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:04:02.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:02 smithi173 ceph-mon[23800]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:04:02.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:02 smithi173 ceph-mon[23800]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:02.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:02 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:04:04.744 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:04 smithi117 ceph-mon[15659]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:04.744 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:04.745 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:04:04.745 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:04 smithi173 ceph-mon[23800]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:04:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.042+0000 7fb0b0600700 1 -- 172.21.15.117:0/2921093171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 msgr2=0x7fb0a8109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.042+0000 7fb0b0600700 1 --2- 172.21.15.117:0/2921093171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 0x7fb0a8109290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb094009910 tx=0x7fb094009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.042+0000 7fb0b0600700 1 -- 172.21.15.117:0/2921093171 shutdown_connections 2024-04-20T11:04:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.042+0000 7fb0b0600700 1 --2- 172.21.15.117:0/2921093171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 0x7fb0a8109290 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.042+0000 7fb0b0600700 1 --2- 172.21.15.117:0/2921093171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8107520 0x7fb0a8107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.042+0000 7fb0b0600700 1 -- 172.21.15.117:0/2921093171 >> 172.21.15.117:0/2921093171 conn(0x7fb0a80768f0 msgr2=0x7fb0a8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.043+0000 7fb0b0600700 1 -- 172.21.15.117:0/2921093171 shutdown_connections 2024-04-20T11:04:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.043+0000 7fb0b0600700 1 -- 172.21.15.117:0/2921093171 wait complete. 2024-04-20T11:04:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.044+0000 7fb0b0600700 1 Processor -- start 2024-04-20T11:04:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.044+0000 7fb0b0600700 1 -- start start 2024-04-20T11:04:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0b0600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8107520 0x7fb0a819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0b0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 0x7fb0a819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0b0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a819ddf0 con 0x7fb0a8108e10 2024-04-20T11:04:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0b0600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a81a1a70 con 0x7fb0a8107520 2024-04-20T11:04:05.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0ad200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 0x7fb0a819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0ad200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 0x7fb0a819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51110/0 (socket says 172.21.15.117:51110) 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0ad200700 1 -- 172.21.15.117:0/646578245 learned_addr learned my addr 172.21.15.117:0/646578245 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0adc00700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8107520 0x7fb0a819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0ad200700 1 -- 172.21.15.117:0/646578245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8107520 msgr2=0x7fb0a819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0ad200700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8107520 0x7fb0a819d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.045+0000 7fb0ad200700 1 -- 172.21.15.117:0/646578245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0940095f0 con 0x7fb0a8108e10 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.046+0000 7fb0ad200700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 0x7fb0a819d710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb0940092e0 tx=0x7fb09400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.046+0000 7fb0a6000700 1 -- 172.21.15.117:0/646578245 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb09401d070 con 0x7fb0a8108e10 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.046+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a81a1cf0 con 0x7fb0a8108e10 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.046+0000 7fb0a6000700 1 -- 172.21.15.117:0/646578245 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0940056e0 con 0x7fb0a8108e10 2024-04-20T11:04:05.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.046+0000 7fb0a6000700 1 -- 172.21.15.117:0/646578245 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb09400fe80 con 0x7fb0a8108e10 2024-04-20T11:04:05.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.047+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a81a2260 con 0x7fb0a8108e10 2024-04-20T11:04:05.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.048+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a810f870 con 0x7fb0a8108e10 2024-04-20T11:04:05.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.048+0000 7fb0a6000700 1 -- 172.21.15.117:0/646578245 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fb09400f6f0 con 0x7fb0a8108e10 2024-04-20T11:04:05.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.049+0000 7fb0a6000700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb09006ccf0 0x7fb09006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:05.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.049+0000 7fb0a6000700 1 -- 172.21.15.117:0/646578245 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb09408d440 con 0x7fb0a8108e10 2024-04-20T11:04:05.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.049+0000 7fb0adc00700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb09006ccf0 0x7fb09006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:05.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.050+0000 7fb0adc00700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb09006ccf0 0x7fb09006f1b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb09c005950 tx=0x7fb09c0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:05.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.054+0000 7fb0a6000700 1 -- 172.21.15.117:0/646578245 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb094057ab0 con 0x7fb0a8108e10 2024-04-20T11:04:05.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.316+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb0a819e490 con 0x7fb0a8108e10 2024-04-20T11:04:05.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.316+0000 7fb0a6000700 1 -- 172.21.15.117:0/646578245 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb09405b0d0 con 0x7fb0a8108e10 2024-04-20T11:04:05.316 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:05.317 INFO:teuthology.orchestra.run.smithi117.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T11:04:05.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb09006ccf0 msgr2=0x7fb09006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:05.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb09006ccf0 0x7fb09006f1b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb09c005950 tx=0x7fb09c0058e0 comp rx=0 tx=0).stop 2024-04-20T11:04:05.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 msgr2=0x7fb0a819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:05.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 0x7fb0a819d710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb0940092e0 tx=0x7fb09400be20 comp rx=0 tx=0).stop 2024-04-20T11:04:05.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 shutdown_connections 2024-04-20T11:04:05.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fb09006ccf0 0x7fb09006f1b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:05.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8107520 0x7fb0a819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:05.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 --2- 172.21.15.117:0/646578245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a8108e10 0x7fb0a819d710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:05.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 >> 172.21.15.117:0/646578245 conn(0x7fb0a80768f0 msgr2=0x7fb0a8077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:05.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 shutdown_connections 2024-04-20T11:04:05.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:05.320+0000 7fb0b0600700 1 -- 172.21.15.117:0/646578245 wait complete. 2024-04-20T11:04:05.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:05 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/646578245' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T11:04:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:05 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/646578245' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T11:04:06.245 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-20T11:04:06.245 INFO:tasks.cephadm:Setup complete, yielding 2024-04-20T11:04:06.246 INFO:teuthology.run_tasks:Running task print... 2024-04-20T11:04:06.255 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-04-20T11:04:06.256 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:04:06.309 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:04:06.309 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-20T11:04:06.708 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:06.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:06 smithi117 ceph-mon[15659]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:06 smithi173 ceph-mon[23800]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:08.847 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:08 smithi117 ceph-mon[15659]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:08 smithi173 ceph-mon[23800]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.017+0000 7f258a000700 1 -- 172.21.15.117:0/1705237213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584103fb0 msgr2=0x7f2584104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.017+0000 7f258a000700 1 --2- 172.21.15.117:0/1705237213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584103fb0 0x7f2584104430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2570009910 tx=0x7f2570009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.017+0000 7f258a000700 1 -- 172.21.15.117:0/1705237213 shutdown_connections 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.017+0000 7f258a000700 1 --2- 172.21.15.117:0/1705237213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584103fb0 0x7f2584104430 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.017+0000 7f258a000700 1 --2- 172.21.15.117:0/1705237213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2584102e50 0x7f2584103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.017+0000 7f258a000700 1 -- 172.21.15.117:0/1705237213 >> 172.21.15.117:0/1705237213 conn(0x7f25840fe3d0 msgr2=0x7f2584100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.017+0000 7f258a000700 1 -- 172.21.15.117:0/1705237213 shutdown_connections 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.017+0000 7f258a000700 1 -- 172.21.15.117:0/1705237213 wait complete. 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.018+0000 7f258a000700 1 Processor -- start 2024-04-20T11:04:09.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.018+0000 7f258a000700 1 -- start start 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.018+0000 7f258a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584102e50 0x7f2584198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f258a000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2584103fb0 0x7f2584198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f258a000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2584199380 con 0x7f2584102e50 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f258a000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841994c0 con 0x7f2584103fb0 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f2583400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584102e50 0x7f2584198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f2583400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584102e50 0x7f2584198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51126/0 (socket says 172.21.15.117:51126) 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f2583400700 1 -- 172.21.15.117:0/2559311498 learned_addr learned my addr 172.21.15.117:0/2559311498 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f2582a00700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2584103fb0 0x7f2584198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:09.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f2583400700 1 -- 172.21.15.117:0/2559311498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2584103fb0 msgr2=0x7f2584198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f2583400700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2584103fb0 0x7f2584198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.019+0000 7f2583400700 1 -- 172.21.15.117:0/2559311498 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25700095f0 con 0x7f2584102e50 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.020+0000 7f2583400700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584102e50 0x7f2584198820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f257400c8f0 tx=0x7f257400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.020+0000 7f257be00700 1 -- 172.21.15.117:0/2559311498 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2574007a70 con 0x7f2584102e50 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.020+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258419ded0 con 0x7f2584102e50 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.020+0000 7f257be00700 1 -- 172.21.15.117:0/2559311498 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2574004cf0 con 0x7f2584102e50 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.020+0000 7f257be00700 1 -- 172.21.15.117:0/2559311498 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2574005610 con 0x7f2584102e50 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.020+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258419e410 con 0x7f2584102e50 2024-04-20T11:04:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.021+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584192970 con 0x7f2584102e50 2024-04-20T11:04:09.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.023+0000 7f257be00700 1 -- 172.21.15.117:0/2559311498 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f2574004750 con 0x7f2584102e50 2024-04-20T11:04:09.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.024+0000 7f257be00700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f256806ccf0 0x7f256806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:09.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.024+0000 7f257be00700 1 -- 172.21.15.117:0/2559311498 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2574013070 con 0x7f2584102e50 2024-04-20T11:04:09.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.024+0000 7f2582a00700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f256806ccf0 0x7f256806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:09.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.025+0000 7f2582a00700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f256806ccf0 0x7f256806f1b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25700051c0 tx=0x7f257000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:09.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.027+0000 7f257be00700 1 -- 172.21.15.117:0/2559311498 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f257405a550 con 0x7f2584102e50 2024-04-20T11:04:09.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.252+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f258404ebb0 con 0x7f2584102e50 2024-04-20T11:04:09.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.258+0000 7f257be00700 1 -- 172.21.15.117:0/2559311498 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v24)=0 v24) v1 ==== 143+0+0 (secure 0 0 0) 0x7f257405a0e0 con 0x7f2584102e50 2024-04-20T11:04:09.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f256806ccf0 msgr2=0x7f256806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:09.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f256806ccf0 0x7f256806f1b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25700051c0 tx=0x7f257000b540 comp rx=0 tx=0).stop 2024-04-20T11:04:09.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584102e50 msgr2=0x7f2584198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:09.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584102e50 0x7f2584198820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f257400c8f0 tx=0x7f257400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:04:09.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 shutdown_connections 2024-04-20T11:04:09.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f256806ccf0 0x7f256806f1b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:09.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2584102e50 0x7f2584198820 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:09.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 --2- 172.21.15.117:0/2559311498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2584103fb0 0x7f2584198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:09.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.261+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 >> 172.21.15.117:0/2559311498 conn(0x7f25840fe3d0 msgr2=0x7f2584107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:09.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.262+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 shutdown_connections 2024-04-20T11:04:09.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:09.262+0000 7f258a000700 1 -- 172.21.15.117:0/2559311498 wait complete. 2024-04-20T11:04:10.142 INFO:teuthology.run_tasks:Running task print... 2024-04-20T11:04:10.152 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-20T11:04:10.152 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:04:10.189 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:04:10.189 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph orch status' 2024-04-20T11:04:10.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:10 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2559311498' entity='client.admin' 2024-04-20T11:04:10.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:04:10.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:10.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:04:10.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:10 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:10.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:10 smithi117 ceph-mon[15659]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:10.584 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:10 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2559311498' entity='client.admin' 2024-04-20T11:04:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:04:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:04:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:10 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:10 smithi173 ceph-mon[23800]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:12.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.761+0000 7f3e83000700 1 -- 172.21.15.117:0/846969041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ffe30 msgr2=0x7f3e7c1002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:12.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.761+0000 7f3e83000700 1 --2- 172.21.15.117:0/846969041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ffe30 0x7f3e7c1002b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009910 tx=0x7f3e68009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:12.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.761+0000 7f3e83000700 1 -- 172.21.15.117:0/846969041 shutdown_connections 2024-04-20T11:04:12.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.761+0000 7f3e83000700 1 --2- 172.21.15.117:0/846969041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ffe30 0x7f3e7c1002b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:12.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.761+0000 7f3e83000700 1 --2- 172.21.15.117:0/846969041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e7c0ff4d0 0x7f3e7c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:12.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.761+0000 7f3e83000700 1 -- 172.21.15.117:0/846969041 >> 172.21.15.117:0/846969041 conn(0x7f3e7c0fb0d0 msgr2=0x7f3e7c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:12.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.761+0000 7f3e83000700 1 -- 172.21.15.117:0/846969041 shutdown_connections 2024-04-20T11:04:12.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.761+0000 7f3e83000700 1 -- 172.21.15.117:0/846969041 wait complete. 2024-04-20T11:04:12.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.762+0000 7f3e83000700 1 Processor -- start 2024-04-20T11:04:12.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.762+0000 7f3e83000700 1 -- start start 2024-04-20T11:04:12.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.762+0000 7f3e83000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ff4d0 0x7f3e7c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:12.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e83000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e7c0ffe30 0x7f3e7c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:12.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e83000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c199360 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e83000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1994a0 con 0x7f3e7c0ffe30 2024-04-20T11:04:12.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e7be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ff4d0 0x7f3e7c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:12.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e7be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ff4d0 0x7f3e7c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51140/0 (socket says 172.21.15.117:51140) 2024-04-20T11:04:12.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e7be00700 1 -- 172.21.15.117:0/3345715293 learned_addr learned my addr 172.21.15.117:0/3345715293 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:12.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e7b400700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e7c0ffe30 0x7f3e7c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:12.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e7be00700 1 -- 172.21.15.117:0/3345715293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e7c0ffe30 msgr2=0x7f3e7c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e7be00700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e7c0ffe30 0x7f3e7c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.763+0000 7f3e7be00700 1 -- 172.21.15.117:0/3345715293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e680095f0 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.764+0000 7f3e7be00700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ff4d0 0x7f3e7c198800 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00d8d0 tx=0x7f3e6c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.764+0000 7f3e78c00700 1 -- 172.21.15.117:0/3345715293 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6c009940 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.764+0000 7f3e78c00700 1 -- 172.21.15.117:0/3345715293 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e6c010460 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.764+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c19deb0 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.764+0000 7f3e78c00700 1 -- 172.21.15.117:0/3345715293 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6c00f5f0 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.764+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c19e3f0 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.765+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c192970 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.766+0000 7f3e78c00700 1 -- 172.21.15.117:0/3345715293 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f3e6c0105d0 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.767+0000 7f3e78c00700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f3e6006cce0 0x7f3e6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:12.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.767+0000 7f3e78c00700 1 -- 172.21.15.117:0/3345715293 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3e6c08be10 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.767+0000 7f3e7b400700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f3e6006cce0 0x7f3e6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:12.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.768+0000 7f3e7b400700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f3e6006cce0 0x7f3e6006f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e680051c0 tx=0x7f3e6800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:12.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:12.772+0000 7f3e78c00700 1 -- 172.21.15.117:0/3345715293 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3e6c05a030 con 0x7f3e7c0ff4d0 2024-04-20T11:04:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:12 smithi117 ceph-mon[15659]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:12 smithi173 ceph-mon[23800]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.007+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e7c02ceb0 con 0x7f3e6006cce0 2024-04-20T11:04:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.009+0000 7f3e78c00700 1 -- 172.21.15.117:0/3345715293 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3e7c02ceb0 con 0x7f3e6006cce0 2024-04-20T11:04:13.009 INFO:teuthology.orchestra.run.smithi117.stdout:Backend: cephadm 2024-04-20T11:04:13.009 INFO:teuthology.orchestra.run.smithi117.stdout:Available: Yes 2024-04-20T11:04:13.009 INFO:teuthology.orchestra.run.smithi117.stdout:Paused: No 2024-04-20T11:04:13.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f3e6006cce0 msgr2=0x7f3e6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:13.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f3e6006cce0 0x7f3e6006f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e680051c0 tx=0x7f3e6800b540 comp rx=0 tx=0).stop 2024-04-20T11:04:13.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ff4d0 msgr2=0x7f3e7c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:13.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ff4d0 0x7f3e7c198800 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00d8d0 tx=0x7f3e6c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:04:13.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 shutdown_connections 2024-04-20T11:04:13.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f3e6006cce0 0x7f3e6006f1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:13.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0ff4d0 0x7f3e7c198800 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:13.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 --2- 172.21.15.117:0/3345715293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e7c0ffe30 0x7f3e7c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:13.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.012+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 >> 172.21.15.117:0/3345715293 conn(0x7f3e7c0fb0d0 msgr2=0x7f3e7c107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:13.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.013+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 shutdown_connections 2024-04-20T11:04:13.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:13.013+0000 7f3e83000700 1 -- 172.21.15.117:0/3345715293 wait complete. 2024-04-20T11:04:13.894 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-20T11:04:14.296 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:14.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:14 smithi117 ceph-mon[15659]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:14 smithi117 ceph-mon[15659]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:04:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:14 smithi173 ceph-mon[23800]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:14 smithi173 ceph-mon[23800]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:04:16.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.516+0000 7f9d53600700 1 -- 172.21.15.117:0/2192093379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 msgr2=0x7f9d4c1063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:16.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.516+0000 7f9d53600700 1 --2- 172.21.15.117:0/2192093379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 0x7f9d4c1063e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009910 tx=0x7f9d38009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:16.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.516+0000 7f9d53600700 1 -- 172.21.15.117:0/2192093379 shutdown_connections 2024-04-20T11:04:16.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.516+0000 7f9d53600700 1 --2- 172.21.15.117:0/2192093379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 0x7f9d4c1063e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:16.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.516+0000 7f9d53600700 1 --2- 172.21.15.117:0/2192093379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d4c1016c0 0x7f9d4c103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:16.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.516+0000 7f9d53600700 1 -- 172.21.15.117:0/2192093379 >> 172.21.15.117:0/2192093379 conn(0x7f9d4c0fb0d0 msgr2=0x7f9d4c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:16.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.516+0000 7f9d53600700 1 -- 172.21.15.117:0/2192093379 shutdown_connections 2024-04-20T11:04:16.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.516+0000 7f9d53600700 1 -- 172.21.15.117:0/2192093379 wait complete. 2024-04-20T11:04:16.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.517+0000 7f9d53600700 1 Processor -- start 2024-04-20T11:04:16.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.518+0000 7f9d53600700 1 -- start start 2024-04-20T11:04:16.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.518+0000 7f9d53600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d4c1016c0 0x7f9d4c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:16.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.518+0000 7f9d53600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 0x7f9d4c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:16.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.518+0000 7f9d53600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c197260 con 0x7f9d4c103ff0 2024-04-20T11:04:16.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.518+0000 7f9d53600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c1973a0 con 0x7f9d4c1016c0 2024-04-20T11:04:16.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.518+0000 7f9d4be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 0x7f9d4c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:16.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.518+0000 7f9d4be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 0x7f9d4c196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33744/0 (socket says 172.21.15.117:33744) 2024-04-20T11:04:16.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.518+0000 7f9d4be00700 1 -- 172.21.15.117:0/3503956360 learned_addr learned my addr 172.21.15.117:0/3503956360 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:16.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.519+0000 7f9d50c00700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d4c1016c0 0x7f9d4c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:16.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.519+0000 7f9d4be00700 1 -- 172.21.15.117:0/3503956360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d4c1016c0 msgr2=0x7f9d4c196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:16.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.519+0000 7f9d4be00700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d4c1016c0 0x7f9d4c196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:16.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.519+0000 7f9d4be00700 1 -- 172.21.15.117:0/3503956360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380095f0 con 0x7f9d4c103ff0 2024-04-20T11:04:16.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.519+0000 7f9d4be00700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 0x7f9d4c196c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006010 tx=0x7f9d3800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.520+0000 7f9d49600700 1 -- 172.21.15.117:0/3503956360 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d3801d070 con 0x7f9d4c103ff0 2024-04-20T11:04:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.520+0000 7f9d49600700 1 -- 172.21.15.117:0/3503956360 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d3800f460 con 0x7f9d4c103ff0 2024-04-20T11:04:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.520+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c19bda0 con 0x7f9d4c103ff0 2024-04-20T11:04:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.520+0000 7f9d49600700 1 -- 172.21.15.117:0/3503956360 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d38021620 con 0x7f9d4c103ff0 2024-04-20T11:04:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.520+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c19c310 con 0x7f9d4c103ff0 2024-04-20T11:04:16.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.521+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c062510 con 0x7f9d4c103ff0 2024-04-20T11:04:16.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.522+0000 7f9d49600700 1 -- 172.21.15.117:0/3503956360 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f9d3800f5d0 con 0x7f9d4c103ff0 2024-04-20T11:04:16.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.523+0000 7f9d49600700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9d3406cce0 0x7f9d3406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:16.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.523+0000 7f9d49600700 1 -- 172.21.15.117:0/3503956360 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9d3808d2c0 con 0x7f9d4c103ff0 2024-04-20T11:04:16.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.523+0000 7f9d50c00700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9d3406cce0 0x7f9d3406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:16.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.524+0000 7f9d50c00700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9d3406cce0 0x7f9d3406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009a10 tx=0x7f9d40009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:16.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.528+0000 7f9d49600700 1 -- 172.21.15.117:0/3503956360 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9d3805b3c0 con 0x7f9d4c103ff0 2024-04-20T11:04:16.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.759+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d4c104a30 con 0x7f9d3406cce0 2024-04-20T11:04:16.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.786+0000 7f9d49600700 1 -- 172.21.15.117:0/3503956360 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3100 (secure 0 0 0) 0x7f9d4c104a30 con 0x7f9d3406cce0 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (4m) 2m ago 6m 14.0M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (7m) 2m ago 7m 7121k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (5m) 80s ago 5m 7033k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (7m) 2m ago 7m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (5m) 80s ago 5m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (4m) 2m ago 6m 75.0M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:9283,8765,8443 running (8m) 2m ago 8m 492M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 2cf5112ba77d 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (5m) 80s ago 5m 448M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 88bcd8bcb785 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (8m) 2m ago 8m 43.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (5m) 80s ago 5m 40.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 2m ago 6m 7314k - 1.5.0 0da6a335fe13 6b69ffbf445a 2024-04-20T11:04:16.787 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (5m) 80s ago 5m 7730k - 1.5.0 0da6a335fe13 81909a8618f8 2024-04-20T11:04:16.788 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (4m) 2m ago 4m 47.7M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:04:16.788 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (3m) 2m ago 3m 38.9M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:04:16.788 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (3m) 2m ago 3m 38.8M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:04:16.788 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (2m) 80s ago 2m 47.4M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:04:16.788 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (2m) 80s ago 2m 45.5M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:04:16.788 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (86s) 80s ago 85s 11.0M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:04:16.788 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (4m) 2m ago 6m 36.8M - 2.43.0 a07b618ecd1d 532b98b31363 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9d3406cce0 msgr2=0x7f9d3406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9d3406cce0 0x7f9d3406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009a10 tx=0x7f9d40009190 comp rx=0 tx=0).stop 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 msgr2=0x7f9d4c196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 0x7f9d4c196c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006010 tx=0x7f9d3800bd70 comp rx=0 tx=0).stop 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 shutdown_connections 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f9d3406cce0 0x7f9d3406f1a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d4c1016c0 0x7f9d4c196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 --2- 172.21.15.117:0/3503956360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d4c103ff0 0x7f9d4c196c40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 >> 172.21.15.117:0/3503956360 conn(0x7f9d4c0fb0d0 msgr2=0x7f9d4c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:16.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 shutdown_connections 2024-04-20T11:04:16.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:16.790+0000 7f9d53600700 1 -- 172.21.15.117:0/3503956360 wait complete. 2024-04-20T11:04:16.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:16 smithi117 ceph-mon[15659]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:16 smithi173 ceph-mon[23800]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:17.681 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-20T11:04:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:17 smithi117 ceph-mon[15659]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:17 smithi173 ceph-mon[23800]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:18.081 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:18.780 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:18 smithi117 ceph-mon[15659]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:18 smithi173 ceph-mon[23800]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.344+0000 7fe3f9800700 1 -- 172.21.15.117:0/2067136277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f4100d30 msgr2=0x7fe3f4101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.344+0000 7fe3f9800700 1 --2- 172.21.15.117:0/2067136277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f4100d30 0x7fe3f4101190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009910 tx=0x7fe3e0009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.345+0000 7fe3f9800700 1 -- 172.21.15.117:0/2067136277 shutdown_connections 2024-04-20T11:04:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.345+0000 7fe3f9800700 1 --2- 172.21.15.117:0/2067136277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f4100d30 0x7fe3f4101190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.345+0000 7fe3f9800700 1 --2- 172.21.15.117:0/2067136277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3f40ffb30 0x7fe3f40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.345+0000 7fe3f9800700 1 -- 172.21.15.117:0/2067136277 >> 172.21.15.117:0/2067136277 conn(0x7fe3f40fb0d0 msgr2=0x7fe3f40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.345+0000 7fe3f9800700 1 -- 172.21.15.117:0/2067136277 shutdown_connections 2024-04-20T11:04:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.345+0000 7fe3f9800700 1 -- 172.21.15.117:0/2067136277 wait complete. 2024-04-20T11:04:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.346+0000 7fe3f9800700 1 Processor -- start 2024-04-20T11:04:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.346+0000 7fe3f9800700 1 -- start start 2024-04-20T11:04:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.346+0000 7fe3f9800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f40ffb30 0x7fe3f41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.346+0000 7fe3f9800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3f4100d30 0x7fe3f4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.346+0000 7fe3f9800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41992b0 con 0x7fe3f40ffb30 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.346+0000 7fe3f9800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41993f0 con 0x7fe3f4100d30 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.346+0000 7fe3f2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f40ffb30 0x7fe3f41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.347+0000 7fe3f2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f40ffb30 0x7fe3f41987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33764/0 (socket says 172.21.15.117:33764) 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.347+0000 7fe3f2a00700 1 -- 172.21.15.117:0/578846969 learned_addr learned my addr 172.21.15.117:0/578846969 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.347+0000 7fe3f2000700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3f4100d30 0x7fe3f4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.347+0000 7fe3f2a00700 1 -- 172.21.15.117:0/578846969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3f4100d30 msgr2=0x7fe3f4198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.347+0000 7fe3f2a00700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3f4100d30 0x7fe3f4198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.347+0000 7fe3f2a00700 1 -- 172.21.15.117:0/578846969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00095f0 con 0x7fe3f40ffb30 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.347+0000 7fe3f2a00700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f40ffb30 0x7fe3f41987e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c910 tx=0x7fe3e400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.348+0000 7fe3eb400700 1 -- 172.21.15.117:0/578846969 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3e40041f0 con 0x7fe3f40ffb30 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.348+0000 7fe3eb400700 1 -- 172.21.15.117:0/578846969 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3e4007dc0 con 0x7fe3f40ffb30 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.348+0000 7fe3eb400700 1 -- 172.21.15.117:0/578846969 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3e4003e70 con 0x7fe3f40ffb30 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.348+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f419de60 con 0x7fe3f40ffb30 2024-04-20T11:04:20.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.348+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f419e3a0 con 0x7fe3f40ffb30 2024-04-20T11:04:20.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.349+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f4062510 con 0x7fe3f40ffb30 2024-04-20T11:04:20.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.350+0000 7fe3eb400700 1 -- 172.21.15.117:0/578846969 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fe3e4004350 con 0x7fe3f40ffb30 2024-04-20T11:04:20.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.351+0000 7fe3eb400700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe3d806cc80 0x7fe3d806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:20.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.351+0000 7fe3eb400700 1 -- 172.21.15.117:0/578846969 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe3e408bd20 con 0x7fe3f40ffb30 2024-04-20T11:04:20.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.351+0000 7fe3f2000700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe3d806cc80 0x7fe3d806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:20.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.352+0000 7fe3f2000700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe3d806cc80 0x7fe3d806f140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00051c0 tx=0x7fe3e000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:20.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.355+0000 7fe3eb400700 1 -- 172.21.15.117:0/578846969 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe3e4059e20 con 0x7fe3f40ffb30 2024-04-20T11:04:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.586+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3f4069510 con 0x7fe3d806cc80 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.594+0000 7fe3eb400700 1 -- 172.21.15.117:0/578846969 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7fe3f4069510 con 0x7fe3d806cc80 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter 2/2 2m ago 7m * 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:crash 2/2 2m ago 7m * 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:mgr 2/2 2m ago 7m count:2 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:mon 2/2 2m ago 6m smithi117:172.21.15.117=smithi117;smithi173:172.21.15.173=smithi173;count:2 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-04-20T11:04:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:osd 6 2m ago - 2024-04-20T11:04:20.595 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-04-20T11:04:20.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe3d806cc80 msgr2=0x7fe3d806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:20.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe3d806cc80 0x7fe3d806f140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00051c0 tx=0x7fe3e000b540 comp rx=0 tx=0).stop 2024-04-20T11:04:20.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f40ffb30 msgr2=0x7fe3f41987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:20.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f40ffb30 0x7fe3f41987e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c910 tx=0x7fe3e400ccd0 comp rx=0 tx=0).stop 2024-04-20T11:04:20.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 shutdown_connections 2024-04-20T11:04:20.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe3d806cc80 0x7fe3d806f140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:20.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f40ffb30 0x7fe3f41987e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:20.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 --2- 172.21.15.117:0/578846969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3f4100d30 0x7fe3f4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:20.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 >> 172.21.15.117:0/578846969 conn(0x7fe3f40fb0d0 msgr2=0x7fe3f4068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:20.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 shutdown_connections 2024-04-20T11:04:20.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:20.597+0000 7fe3f9800700 1 -- 172.21.15.117:0/578846969 wait complete. 2024-04-20T11:04:20.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:20 smithi117 ceph-mon[15659]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:20 smithi173 ceph-mon[23800]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:21.426 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-20T11:04:21.835 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:21 smithi173 ceph-mon[23800]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:21.980 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:21 smithi117 ceph-mon[15659]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:22 smithi173 ceph-mon[23800]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:23.009 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:22 smithi117 ceph-mon[15659]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:24.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.021+0000 7fd98ca00700 1 -- 172.21.15.117:0/3522273101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988102e50 msgr2=0x7fd988103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:24.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.021+0000 7fd98ca00700 1 --2- 172.21.15.117:0/3522273101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988102e50 0x7fd988103270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd978009910 tx=0x7fd978009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:24.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.021+0000 7fd98ca00700 1 -- 172.21.15.117:0/3522273101 shutdown_connections 2024-04-20T11:04:24.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.021+0000 7fd98ca00700 1 --2- 172.21.15.117:0/3522273101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd988103fb0 0x7fd988104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:24.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.021+0000 7fd98ca00700 1 --2- 172.21.15.117:0/3522273101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988102e50 0x7fd988103270 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:24.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.021+0000 7fd98ca00700 1 -- 172.21.15.117:0/3522273101 >> 172.21.15.117:0/3522273101 conn(0x7fd9880fe3d0 msgr2=0x7fd988100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:24.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.021+0000 7fd98ca00700 1 -- 172.21.15.117:0/3522273101 shutdown_connections 2024-04-20T11:04:24.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.021+0000 7fd98ca00700 1 -- 172.21.15.117:0/3522273101 wait complete. 2024-04-20T11:04:24.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.023+0000 7fd98ca00700 1 Processor -- start 2024-04-20T11:04:24.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.023+0000 7fd98ca00700 1 -- start start 2024-04-20T11:04:24.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.023+0000 7fd98ca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd988102e50 0x7fd988078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:24.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.023+0000 7fd98ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988103fb0 0x7fd988079180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:24.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.023+0000 7fd98ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd988075780 con 0x7fd988103fb0 2024-04-20T11:04:24.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.023+0000 7fd98ca00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880758c0 con 0x7fd988102e50 2024-04-20T11:04:24.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd985400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988103fb0 0x7fd988079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:24.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd985400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988103fb0 0x7fd988079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55138/0 (socket says 172.21.15.117:55138) 2024-04-20T11:04:24.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd985400700 1 -- 172.21.15.117:0/3831004723 learned_addr learned my addr 172.21.15.117:0/3831004723 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:24.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd985e00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd988102e50 0x7fd988078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:24.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd985400700 1 -- 172.21.15.117:0/3831004723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd988102e50 msgr2=0x7fd988078c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:24.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd985400700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd988102e50 0x7fd988078c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:24.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd985400700 1 -- 172.21.15.117:0/3831004723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780095f0 con 0x7fd988103fb0 2024-04-20T11:04:24.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd985400700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988103fb0 0x7fd988079180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd97400d8d0 tx=0x7fd97400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:24.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.024+0000 7fd97ea00700 1 -- 172.21.15.117:0/3831004723 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd974009940 con 0x7fd988103fb0 2024-04-20T11:04:24.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.025+0000 7fd97ea00700 1 -- 172.21.15.117:0/3831004723 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd974010460 con 0x7fd988103fb0 2024-04-20T11:04:24.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.025+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd988075ba0 con 0x7fd988103fb0 2024-04-20T11:04:24.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.025+0000 7fd97ea00700 1 -- 172.21.15.117:0/3831004723 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd97400f5f0 con 0x7fd988103fb0 2024-04-20T11:04:24.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.025+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd988076170 con 0x7fd988103fb0 2024-04-20T11:04:24.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.026+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd988192950 con 0x7fd988103fb0 2024-04-20T11:04:24.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.027+0000 7fd97ea00700 1 -- 172.21.15.117:0/3831004723 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fd9740105d0 con 0x7fd988103fb0 2024-04-20T11:04:24.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.028+0000 7fd97ea00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96c06cce0 0x7fd96c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:24.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.029+0000 7fd97ea00700 1 -- 172.21.15.117:0/3831004723 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd97408bef0 con 0x7fd988103fb0 2024-04-20T11:04:24.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.029+0000 7fd985e00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96c06cce0 0x7fd96c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.029+0000 7fd985e00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96c06cce0 0x7fd96c06f1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd9780051c0 tx=0x7fd978005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:24.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.034+0000 7fd97ea00700 1 -- 172.21.15.117:0/3831004723 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd97405a070 con 0x7fd988103fb0 2024-04-20T11:04:24.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.262+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd988061320 con 0x7fd96c06cce0 2024-04-20T11:04:24.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.265+0000 7fd97ea00700 1 -- 172.21.15.117:0/3831004723 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fd988061320 con 0x7fd96c06cce0 2024-04-20T11:04:24.265 INFO:teuthology.orchestra.run.smithi117.stdout:HOST ADDR LABELS STATUS 2024-04-20T11:04:24.265 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117 172.21.15.117 2024-04-20T11:04:24.265 INFO:teuthology.orchestra.run.smithi117.stdout:smithi173 172.21.15.173 2024-04-20T11:04:24.265 INFO:teuthology.orchestra.run.smithi117.stdout:2 hosts in cluster 2024-04-20T11:04:24.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96c06cce0 msgr2=0x7fd96c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:24.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96c06cce0 0x7fd96c06f1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd9780051c0 tx=0x7fd978005a90 comp rx=0 tx=0).stop 2024-04-20T11:04:24.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988103fb0 msgr2=0x7fd988079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:24.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988103fb0 0x7fd988079180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd97400d8d0 tx=0x7fd97400dc90 comp rx=0 tx=0).stop 2024-04-20T11:04:24.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 shutdown_connections 2024-04-20T11:04:24.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96c06cce0 0x7fd96c06f1a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:24.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd988102e50 0x7fd988078c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:24.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 --2- 172.21.15.117:0/3831004723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd988103fb0 0x7fd988079180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:24.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 >> 172.21.15.117:0/3831004723 conn(0x7fd9880fe3d0 msgr2=0x7fd988107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:24.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 shutdown_connections 2024-04-20T11:04:24.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:24.268+0000 7fd98ca00700 1 -- 172.21.15.117:0/3831004723 wait complete. 2024-04-20T11:04:24.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:24 smithi117 ceph-mon[15659]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:24 smithi173 ceph-mon[23800]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:25.103 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-20T11:04:25.500 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:25.980 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:25 smithi117 ceph-mon[15659]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:26.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:25 smithi173 ceph-mon[23800]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:26.850 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:26 smithi117 ceph-mon[15659]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:27.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:26 smithi173 ceph-mon[23800]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.688+0000 7fe60ac00700 1 -- 172.21.15.117:0/2094034912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604102e50 msgr2=0x7fe604103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.688+0000 7fe60ac00700 1 --2- 172.21.15.117:0/2094034912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604102e50 0x7fe604103270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009910 tx=0x7fe5f4009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.688+0000 7fe60ac00700 1 -- 172.21.15.117:0/2094034912 shutdown_connections 2024-04-20T11:04:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.688+0000 7fe60ac00700 1 --2- 172.21.15.117:0/2094034912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604103fb0 0x7fe604104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.688+0000 7fe60ac00700 1 --2- 172.21.15.117:0/2094034912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604102e50 0x7fe604103270 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.688+0000 7fe60ac00700 1 -- 172.21.15.117:0/2094034912 >> 172.21.15.117:0/2094034912 conn(0x7fe6040fe3d0 msgr2=0x7fe604100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.688+0000 7fe60ac00700 1 -- 172.21.15.117:0/2094034912 shutdown_connections 2024-04-20T11:04:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.688+0000 7fe60ac00700 1 -- 172.21.15.117:0/2094034912 wait complete. 2024-04-20T11:04:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.689+0000 7fe60ac00700 1 Processor -- start 2024-04-20T11:04:27.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.690+0000 7fe60ac00700 1 -- start start 2024-04-20T11:04:27.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.690+0000 7fe60ac00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604102e50 0x7fe604198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:27.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.690+0000 7fe60ac00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604103fb0 0x7fe604198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:27.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.690+0000 7fe60ac00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041994c0 con 0x7fe604103fb0 2024-04-20T11:04:27.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.690+0000 7fe60ac00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe604199600 con 0x7fe604102e50 2024-04-20T11:04:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.690+0000 7fe603400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604103fb0 0x7fe604198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.690+0000 7fe603400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604103fb0 0x7fe604198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55162/0 (socket says 172.21.15.117:55162) 2024-04-20T11:04:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.690+0000 7fe603400700 1 -- 172.21.15.117:0/1698516857 learned_addr learned my addr 172.21.15.117:0/1698516857 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.691+0000 7fe603e00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604102e50 0x7fe604198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.691+0000 7fe603400700 1 -- 172.21.15.117:0/1698516857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604102e50 msgr2=0x7fe604198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.691+0000 7fe603400700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604102e50 0x7fe604198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.691+0000 7fe603400700 1 -- 172.21.15.117:0/1698516857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40095f0 con 0x7fe604103fb0 2024-04-20T11:04:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.691+0000 7fe603400700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604103fb0 0x7fe604198ea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000b4b0 tx=0x7fe5f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:27.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.691+0000 7fe600c00700 1 -- 172.21.15.117:0/1698516857 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5f0011800 con 0x7fe604103fb0 2024-04-20T11:04:27.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.692+0000 7fe600c00700 1 -- 172.21.15.117:0/1698516857 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5f0011e20 con 0x7fe604103fb0 2024-04-20T11:04:27.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.692+0000 7fe600c00700 1 -- 172.21.15.117:0/1698516857 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5f000f2e0 con 0x7fe604103fb0 2024-04-20T11:04:27.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.692+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60419dfa0 con 0x7fe604103fb0 2024-04-20T11:04:27.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.692+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60419e540 con 0x7fe604103fb0 2024-04-20T11:04:27.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.693+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6041929d0 con 0x7fe604103fb0 2024-04-20T11:04:27.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.694+0000 7fe600c00700 1 -- 172.21.15.117:0/1698516857 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fe5f0010380 con 0x7fe604103fb0 2024-04-20T11:04:27.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.695+0000 7fe600c00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5e806ccf0 0x7fe5e806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:27.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.695+0000 7fe600c00700 1 -- 172.21.15.117:0/1698516857 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe5f008ba70 con 0x7fe604103fb0 2024-04-20T11:04:27.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.696+0000 7fe603e00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5e806ccf0 0x7fe5e806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:27.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.696+0000 7fe603e00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5e806ccf0 0x7fe5e806f1b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40051c0 tx=0x7fe5f400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:27.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.700+0000 7fe600c00700 1 -- 172.21.15.117:0/1698516857 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe5f0059bf0 con 0x7fe604103fb0 2024-04-20T11:04:27.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.930+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe60402cdb0 con 0x7fe5e806ccf0 2024-04-20T11:04:27.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.934+0000 7fe600c00700 1 -- 172.21.15.117:0/1698516857 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fe60402cdb0 con 0x7fe5e806ccf0 2024-04-20T11:04:27.935 INFO:teuthology.orchestra.run.smithi117.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-20T11:04:27.935 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-20T11:04:27.935 INFO:teuthology.orchestra.run.smithi117.stdout:smithi173 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204019S400BGN 372G No 86s ago Has a FileSystem, LVM detected 2024-04-20T11:04:27.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.936+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5e806ccf0 msgr2=0x7fe5e806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:27.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.936+0000 7fe60ac00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5e806ccf0 0x7fe5e806f1b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40051c0 tx=0x7fe5f400b540 comp rx=0 tx=0).stop 2024-04-20T11:04:27.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.936+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604103fb0 msgr2=0x7fe604198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:27.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.936+0000 7fe60ac00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604103fb0 0x7fe604198ea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000b4b0 tx=0x7fe5f000b870 comp rx=0 tx=0).stop 2024-04-20T11:04:27.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.937+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 shutdown_connections 2024-04-20T11:04:27.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.937+0000 7fe60ac00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fe5e806ccf0 0x7fe5e806f1b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:27.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.937+0000 7fe60ac00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604102e50 0x7fe604198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:27.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.937+0000 7fe60ac00700 1 --2- 172.21.15.117:0/1698516857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe604103fb0 0x7fe604198ea0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:27.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.937+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 >> 172.21.15.117:0/1698516857 conn(0x7fe6040fe3d0 msgr2=0x7fe604107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:27.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.937+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 shutdown_connections 2024-04-20T11:04:27.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:27.937+0000 7fe60ac00700 1 -- 172.21.15.117:0/1698516857 wait complete. 2024-04-20T11:04:28.762 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:04:28.773 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:04:28.773 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-20T11:04:29.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:28 smithi117 ceph-mon[15659]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:29.156 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:29.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:28 smithi173 ceph-mon[23800]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:29.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:29 smithi117 ceph-mon[15659]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:29.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:04:30.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:29 smithi173 ceph-mon[23800]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:30.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:04:30.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:30 smithi117 ceph-mon[15659]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:31.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:30 smithi173 ceph-mon[23800]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:31.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.474+0000 7f5ed2800700 1 -- 172.21.15.117:0/1637255517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc1014c0 msgr2=0x7f5ecc1038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:31.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.474+0000 7f5ed2800700 1 --2- 172.21.15.117:0/1637255517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc1014c0 0x7f5ecc1038b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009910 tx=0x7f5ec0009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:31.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.474+0000 7f5ed2800700 1 -- 172.21.15.117:0/1637255517 shutdown_connections 2024-04-20T11:04:31.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.474+0000 7f5ed2800700 1 --2- 172.21.15.117:0/1637255517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ecc103df0 0x7f5ecc1061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:31.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.474+0000 7f5ed2800700 1 --2- 172.21.15.117:0/1637255517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc1014c0 0x7f5ecc1038b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:31.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.474+0000 7f5ed2800700 1 -- 172.21.15.117:0/1637255517 >> 172.21.15.117:0/1637255517 conn(0x7f5ecc0faef0 msgr2=0x7f5ecc0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:31.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.474+0000 7f5ed2800700 1 -- 172.21.15.117:0/1637255517 shutdown_connections 2024-04-20T11:04:31.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.474+0000 7f5ed2800700 1 -- 172.21.15.117:0/1637255517 wait complete. 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.476+0000 7f5ed2800700 1 Processor -- start 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.476+0000 7f5ed2800700 1 -- start start 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.476+0000 7f5ed2800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ecc1014c0 0x7f5ecc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.476+0000 7f5ed2800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc103df0 0x7f5ecc198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.476+0000 7f5ed2800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc1991e0 con 0x7f5ecc103df0 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.476+0000 7f5ed2800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc199320 con 0x7f5ecc1014c0 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.477+0000 7f5ed0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc103df0 0x7f5ecc198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.477+0000 7f5ed0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc103df0 0x7f5ecc198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55178/0 (socket says 172.21.15.117:55178) 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.477+0000 7f5ed0a00700 1 -- 172.21.15.117:0/1418099625 learned_addr learned my addr 172.21.15.117:0/1418099625 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.477+0000 7f5ed1400700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ecc1014c0 0x7f5ecc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.477+0000 7f5ed0a00700 1 -- 172.21.15.117:0/1418099625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ecc1014c0 msgr2=0x7f5ecc198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.477+0000 7f5ed0a00700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ecc1014c0 0x7f5ecc198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.477+0000 7f5ed0a00700 1 -- 172.21.15.117:0/1418099625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00095f0 con 0x7f5ecc103df0 2024-04-20T11:04:31.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.478+0000 7f5ed0a00700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc103df0 0x7f5ecc198bc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800d8d0 tx=0x7f5eb800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:31.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.478+0000 7f5eca000700 1 -- 172.21.15.117:0/1418099625 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5eb80098a0 con 0x7f5ecc103df0 2024-04-20T11:04:31.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.478+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc19dce0 con 0x7f5ecc103df0 2024-04-20T11:04:31.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.478+0000 7f5eca000700 1 -- 172.21.15.117:0/1418099625 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5eb8010460 con 0x7f5ecc103df0 2024-04-20T11:04:31.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.478+0000 7f5eca000700 1 -- 172.21.15.117:0/1418099625 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5eb800f5f0 con 0x7f5ecc103df0 2024-04-20T11:04:31.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.478+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc19e2b0 con 0x7f5ecc103df0 2024-04-20T11:04:31.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.479+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc062510 con 0x7f5ecc103df0 2024-04-20T11:04:31.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.480+0000 7f5eca000700 1 -- 172.21.15.117:0/1418099625 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f5eb8009a00 con 0x7f5ecc103df0 2024-04-20T11:04:31.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.482+0000 7f5eca000700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5eb406cce0 0x7f5eb406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:31.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.482+0000 7f5eca000700 1 -- 172.21.15.117:0/1418099625 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5eb808bdb0 con 0x7f5ecc103df0 2024-04-20T11:04:31.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.482+0000 7f5ed1400700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5eb406cce0 0x7f5eb406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:31.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.483+0000 7f5ed1400700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5eb406cce0 0x7f5eb406f1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0006010 tx=0x7f5ec0005180 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:31.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.487+0000 7f5eca000700 1 -- 172.21.15.117:0/1418099625 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5eb8059f50 con 0x7f5ecc103df0 2024-04-20T11:04:31.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:31.738+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ecc0ffce0 con 0x7f5eb406cce0 2024-04-20T11:04:33.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:32 smithi117 ceph-mon[15659]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:33.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:32 smithi117 ceph-mon[15659]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:33.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-20T11:04:33.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:32 smithi173 ceph-mon[23800]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:33.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:32 smithi173 ceph-mon[23800]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:04:33.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:32 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-20T11:04:33.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.771+0000 7f5eca000700 1 -- 172.21.15.117:0/1418099625 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5ecc0ffce0 con 0x7f5eb406cce0 2024-04-20T11:04:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5eb406cce0 msgr2=0x7f5eb406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5eb406cce0 0x7f5eb406f1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0006010 tx=0x7f5ec0005180 comp rx=0 tx=0).stop 2024-04-20T11:04:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc103df0 msgr2=0x7f5ecc198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc103df0 0x7f5ecc198bc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800d8d0 tx=0x7f5eb800dbe0 comp rx=0 tx=0).stop 2024-04-20T11:04:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 shutdown_connections 2024-04-20T11:04:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5eb406cce0 0x7f5eb406f1a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ecc1014c0 0x7f5ecc198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 --2- 172.21.15.117:0/1418099625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc103df0 0x7f5ecc198bc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:33.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 >> 172.21.15.117:0/1418099625 conn(0x7f5ecc0faef0 msgr2=0x7f5ecc0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:33.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.774+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 shutdown_connections 2024-04-20T11:04:33.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:33.775+0000 7f5ed2800700 1 -- 172.21.15.117:0/1418099625 wait complete. 2024-04-20T11:04:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-20T11:04:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:33 smithi117 ceph-mon[15659]: osdmap e43: 6 total, 6 up, 6 in 2024-04-20T11:04:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-20T11:04:34.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:33 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[15655]: 2024-04-20T11:04:33.733+0000 7fcd02400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:04:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-20T11:04:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:33 smithi173 ceph-mon[23800]: osdmap e43: 6 total, 6 up, 6 in 2024-04-20T11:04:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-20T11:04:34.644 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-20T11:04:34.854 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: pgmap v221: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: osdmap e44: 6 total, 6 up, 6 in 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: osdmap e45: 6 total, 6 up, 6 in 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: fsmap cephfs:0 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: Saving service mds.cephfs spec with placement count:4 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:34.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.umapup", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:04:34.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.umapup", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:04:34.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: pgmap v221: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: osdmap e44: 6 total, 6 up, 6 in 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: osdmap e45: 6 total, 6 up, 6 in 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: fsmap cephfs:0 2024-04-20T11:04:35.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: Saving service mds.cephfs spec with placement count:4 2024-04-20T11:04:35.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:35.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:04:35.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:35.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:04:35.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:35.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.umapup", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:04:35.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.umapup", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:04:35.163 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:35 smithi117 ceph-mon[15659]: Deploying daemon mds.cephfs.smithi117.umapup on smithi117 2024-04-20T11:04:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:35 smithi117 ceph-mon[15659]: osdmap e46: 6 total, 6 up, 6 in 2024-04-20T11:04:35.978 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:35 smithi173 ceph-mon[23800]: Deploying daemon mds.cephfs.smithi117.umapup on smithi117 2024-04-20T11:04:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:35 smithi173 ceph-mon[23800]: osdmap e46: 6 total, 6 up, 6 in 2024-04-20T11:04:37.075 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:36 smithi117 ceph-mon[15659]: pgmap v225: 65 pgs: 26 active+clean, 15 creating+peering, 24 unknown; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:37.075 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:36 smithi117 ceph-mon[15659]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T11:04:37.076 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:36 smithi117 ceph-mon[15659]: osdmap e47: 6 total, 6 up, 6 in 2024-04-20T11:04:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:36 smithi173 ceph-mon[23800]: pgmap v225: 65 pgs: 26 active+clean, 15 creating+peering, 24 unknown; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:36 smithi173 ceph-mon[23800]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T11:04:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:36 smithi173 ceph-mon[23800]: osdmap e47: 6 total, 6 up, 6 in 2024-04-20T11:04:38.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.630+0000 7efbf9200700 1 -- 172.21.15.117:0/2203144178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 msgr2=0x7efbf4104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:38.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.630+0000 7efbf9200700 1 --2- 172.21.15.117:0/2203144178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 0x7efbf4104430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efbe0009910 tx=0x7efbe0009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.630+0000 7efbf9200700 1 -- 172.21.15.117:0/2203144178 shutdown_connections 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.630+0000 7efbf9200700 1 --2- 172.21.15.117:0/2203144178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 0x7efbf4104430 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.630+0000 7efbf9200700 1 --2- 172.21.15.117:0/2203144178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efbf4102e50 0x7efbf4103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.630+0000 7efbf9200700 1 -- 172.21.15.117:0/2203144178 >> 172.21.15.117:0/2203144178 conn(0x7efbf40fe3d0 msgr2=0x7efbf4100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.630+0000 7efbf9200700 1 -- 172.21.15.117:0/2203144178 shutdown_connections 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.630+0000 7efbf9200700 1 -- 172.21.15.117:0/2203144178 wait complete. 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.632+0000 7efbf9200700 1 Processor -- start 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.632+0000 7efbf9200700 1 -- start start 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.632+0000 7efbf9200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efbf4102e50 0x7efbf41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.632+0000 7efbf9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 0x7efbf4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.632+0000 7efbf9200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4199340 con 0x7efbf4103fb0 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.632+0000 7efbf9200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4199480 con 0x7efbf4102e50 2024-04-20T11:04:38.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.633+0000 7efbf1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 0x7efbf4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.633+0000 7efbf1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 0x7efbf4198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53058/0 (socket says 172.21.15.117:53058) 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.633+0000 7efbf1400700 1 -- 172.21.15.117:0/2417469164 learned_addr learned my addr 172.21.15.117:0/2417469164 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.633+0000 7efbf1e00700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efbf4102e50 0x7efbf41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.633+0000 7efbf1400700 1 -- 172.21.15.117:0/2417469164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efbf4102e50 msgr2=0x7efbf41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.633+0000 7efbf1400700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efbf4102e50 0x7efbf41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.633+0000 7efbf1400700 1 -- 172.21.15.117:0/2417469164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe00095f0 con 0x7efbf4103fb0 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.634+0000 7efbf1400700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 0x7efbf4198d20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efbe0006010 tx=0x7efbe000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.634+0000 7efbeaa00700 1 -- 172.21.15.117:0/2417469164 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe001d070 con 0x7efbf4103fb0 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.634+0000 7efbeaa00700 1 -- 172.21.15.117:0/2417469164 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbe0005dc0 con 0x7efbf4103fb0 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.634+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf419de80 con 0x7efbf4103fb0 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.634+0000 7efbeaa00700 1 -- 172.21.15.117:0/2417469164 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe0021410 con 0x7efbf4103fb0 2024-04-20T11:04:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.634+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf419e3c0 con 0x7efbf4103fb0 2024-04-20T11:04:38.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.635+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf4192950 con 0x7efbf4103fb0 2024-04-20T11:04:38.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.636+0000 7efbeaa00700 1 -- 172.21.15.117:0/2417469164 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7efbe000f460 con 0x7efbf4103fb0 2024-04-20T11:04:38.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.637+0000 7efbeaa00700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efbd806ccf0 0x7efbd806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:38.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.637+0000 7efbeaa00700 1 -- 172.21.15.117:0/2417469164 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7efbe008d4b0 con 0x7efbf4103fb0 2024-04-20T11:04:38.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.637+0000 7efbf1e00700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efbd806ccf0 0x7efbd806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:38.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.638+0000 7efbf1e00700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efbd806ccf0 0x7efbd806f1b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efbe4005950 tx=0x7efbe400b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:38.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.643+0000 7efbeaa00700 1 -- 172.21.15.117:0/2417469164 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7efbe005b310 con 0x7efbf4103fb0 2024-04-20T11:04:38.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: pgmap v227: 65 pgs: 47 active+clean, 15 creating+peering, 3 unknown; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pybzht", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pybzht", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: daemon mds.cephfs.smithi117.umapup assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: Cluster is now healthy 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] up:boot 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: fsmap cephfs:1 {0=cephfs.smithi117.umapup=up:creating} 2024-04-20T11:04:38.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:04:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.927+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efbf410b930 con 0x7efbf4103fb0 2024-04-20T11:04:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.927+0000 7efbeaa00700 1 -- 172.21.15.117:0/2417469164 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 3 v3) v1 ==== 75+0+1285 (secure 0 0 0) 0x7efbe005aea0 con 0x7efbf4103fb0 2024-04-20T11:04:38.928 INFO:teuthology.orchestra.run.smithi117.stdout:e3 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 3 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:38.744822+0000 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:04:38.929 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 1 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:in 0 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460} 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:04:38.930 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:04:38.931 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:04:38.931 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data disabled 2024-04-20T11:04:38.931 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:04:38.931 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:04:38.931 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 0 2024-04-20T11:04:38.931 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:creating seq 1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:04:38.931 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:38.931 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efbd806ccf0 msgr2=0x7efbd806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efbd806ccf0 0x7efbd806f1b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efbe4005950 tx=0x7efbe400b410 comp rx=0 tx=0).stop 2024-04-20T11:04:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 msgr2=0x7efbf4198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 0x7efbf4198d20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efbe0006010 tx=0x7efbe000bd70 comp rx=0 tx=0).stop 2024-04-20T11:04:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 shutdown_connections 2024-04-20T11:04:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efbd806ccf0 0x7efbd806f1b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efbf4102e50 0x7efbf41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 --2- 172.21.15.117:0/2417469164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf4103fb0 0x7efbf4198d20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:38.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.930+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 >> 172.21.15.117:0/2417469164 conn(0x7efbf40fe3d0 msgr2=0x7efbf4107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:38.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.931+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 shutdown_connections 2024-04-20T11:04:38.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:38.931+0000 7efbf9200700 1 -- 172.21.15.117:0/2417469164 wait complete. 2024-04-20T11:04:38.933 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 3 2024-04-20T11:04:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: pgmap v227: 65 pgs: 47 active+clean, 15 creating+peering, 3 unknown; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pybzht", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pybzht", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: daemon mds.cephfs.smithi117.umapup assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: Cluster is now healthy 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] up:boot 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: fsmap cephfs:1 {0=cephfs.smithi117.umapup=up:creating} 2024-04-20T11:04:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:04:39.807 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:04:39.817 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:04:39.817 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-04-20T11:04:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:39 smithi117 ceph-mon[15659]: Deploying daemon mds.cephfs.smithi173.pybzht on smithi173 2024-04-20T11:04:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:39 smithi117 ceph-mon[15659]: daemon mds.cephfs.smithi117.umapup is now active in filesystem cephfs as rank 0 2024-04-20T11:04:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:39 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2417469164' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:04:40.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:39 smithi173 ceph-mon[23800]: Deploying daemon mds.cephfs.smithi173.pybzht on smithi173 2024-04-20T11:04:40.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:39 smithi173 ceph-mon[23800]: daemon mds.cephfs.smithi117.umapup is now active in filesystem cephfs as rank 0 2024-04-20T11:04:40.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:39 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2417469164' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:04:40.198 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:41.014 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:40 smithi173 ceph-mon[23800]: pgmap v228: 65 pgs: 53 active+clean, 12 creating+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:41.014 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:40 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] up:active 2024-04-20T11:04:41.015 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:40 smithi173 ceph-mon[23800]: fsmap cephfs:1 {0=cephfs.smithi117.umapup=up:active} 2024-04-20T11:04:41.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:40 smithi117 ceph-mon[15659]: pgmap v228: 65 pgs: 53 active+clean, 12 creating+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:04:41.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:40 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] up:active 2024-04-20T11:04:41.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:40 smithi117 ceph-mon[15659]: fsmap cephfs:1 {0=cephfs.smithi117.umapup=up:active} 2024-04-20T11:04:41.973 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:41 smithi117 ceph-mon[15659]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-20T11:04:42.067 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:41 smithi173 ceph-mon[23800]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-20T11:04:42.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.480+0000 7fd980200700 1 -- 172.21.15.117:0/1972623041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978100c00 msgr2=0x7fd978101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:42.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.480+0000 7fd980200700 1 --2- 172.21.15.117:0/1972623041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978100c00 0x7fd978101020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009910 tx=0x7fd96c009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:42.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.481+0000 7fd980200700 1 -- 172.21.15.117:0/1972623041 shutdown_connections 2024-04-20T11:04:42.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.481+0000 7fd980200700 1 --2- 172.21.15.117:0/1972623041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd978101e00 0x7fd978102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:42.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.481+0000 7fd980200700 1 --2- 172.21.15.117:0/1972623041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978100c00 0x7fd978101020 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:42.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.481+0000 7fd980200700 1 -- 172.21.15.117:0/1972623041 >> 172.21.15.117:0/1972623041 conn(0x7fd9780fc1c0 msgr2=0x7fd9780fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:42.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.481+0000 7fd980200700 1 -- 172.21.15.117:0/1972623041 shutdown_connections 2024-04-20T11:04:42.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.481+0000 7fd980200700 1 -- 172.21.15.117:0/1972623041 wait complete. 2024-04-20T11:04:42.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.482+0000 7fd980200700 1 Processor -- start 2024-04-20T11:04:42.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.483+0000 7fd980200700 1 -- start start 2024-04-20T11:04:42.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.483+0000 7fd980200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd978100c00 0x7fd978196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:42.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.483+0000 7fd980200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978101e00 0x7fd978196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:42.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.483+0000 7fd980200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9781971c0 con 0x7fd978101e00 2024-04-20T11:04:42.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.483+0000 7fd980200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd978197300 con 0x7fd978100c00 2024-04-20T11:04:42.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.483+0000 7fd97ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978101e00 0x7fd978196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:42.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.484+0000 7fd97ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978101e00 0x7fd978196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53076/0 (socket says 172.21.15.117:53076) 2024-04-20T11:04:42.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.484+0000 7fd97ce00700 1 -- 172.21.15.117:0/2046424360 learned_addr learned my addr 172.21.15.117:0/2046424360 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:42.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.484+0000 7fd97d800700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd978100c00 0x7fd978196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:42.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.484+0000 7fd97ce00700 1 -- 172.21.15.117:0/2046424360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd978100c00 msgr2=0x7fd978196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:42.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.484+0000 7fd97ce00700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd978100c00 0x7fd978196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:42.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.484+0000 7fd97ce00700 1 -- 172.21.15.117:0/2046424360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0095f0 con 0x7fd978101e00 2024-04-20T11:04:42.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.484+0000 7fd97d800700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd978100c00 0x7fd978196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:04:42.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.485+0000 7fd97ce00700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978101e00 0x7fd978196ba0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd96400d8d0 tx=0x7fd96400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:42.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.485+0000 7fd976000700 1 -- 172.21.15.117:0/2046424360 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd964009940 con 0x7fd978101e00 2024-04-20T11:04:42.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.485+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97819bd10 con 0x7fd978101e00 2024-04-20T11:04:42.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.485+0000 7fd976000700 1 -- 172.21.15.117:0/2046424360 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd964010460 con 0x7fd978101e00 2024-04-20T11:04:42.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.485+0000 7fd976000700 1 -- 172.21.15.117:0/2046424360 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96400f5f0 con 0x7fd978101e00 2024-04-20T11:04:42.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.485+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97819c2e0 con 0x7fd978101e00 2024-04-20T11:04:42.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.486+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd978190780 con 0x7fd978101e00 2024-04-20T11:04:42.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.487+0000 7fd976000700 1 -- 172.21.15.117:0/2046424360 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fd9640105d0 con 0x7fd978101e00 2024-04-20T11:04:42.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.488+0000 7fd976000700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96006cce0 0x7fd96006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:42.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.488+0000 7fd976000700 1 -- 172.21.15.117:0/2046424360 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd96408c320 con 0x7fd978101e00 2024-04-20T11:04:42.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.489+0000 7fd97d800700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96006cce0 0x7fd96006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:42.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.489+0000 7fd97d800700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96006cce0 0x7fd96006f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006010 tx=0x7fd96c00b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:42.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.494+0000 7fd976000700 1 -- 172.21.15.117:0/2046424360 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd96405a0e0 con 0x7fd978101e00 2024-04-20T11:04:42.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.770+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fd97802cea0 con 0x7fd978101e00 2024-04-20T11:04:42.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.844+0000 7fd976000700 1 -- 172.21.15.117:0/2046424360 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7fd964059c70 con 0x7fd978101e00 2024-04-20T11:04:42.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.847+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96006cce0 msgr2=0x7fd96006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:42.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.847+0000 7fd980200700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96006cce0 0x7fd96006f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006010 tx=0x7fd96c00b560 comp rx=0 tx=0).stop 2024-04-20T11:04:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.847+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978101e00 msgr2=0x7fd978196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.847+0000 7fd980200700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978101e00 0x7fd978196ba0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd96400d8d0 tx=0x7fd96400dc90 comp rx=0 tx=0).stop 2024-04-20T11:04:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.848+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 shutdown_connections 2024-04-20T11:04:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.848+0000 7fd980200700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fd96006cce0 0x7fd96006f1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.848+0000 7fd980200700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd978100c00 0x7fd978196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.848+0000 7fd980200700 1 --2- 172.21.15.117:0/2046424360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd978101e00 0x7fd978196ba0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.848+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 >> 172.21.15.117:0/2046424360 conn(0x7fd9780fc1c0 msgr2=0x7fd978105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.848+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 shutdown_connections 2024-04-20T11:04:42.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:42.848+0000 7fd980200700 1 -- 172.21.15.117:0/2046424360 wait complete. 2024-04-20T11:04:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.miaplu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:04:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.miaplu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:04:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: Deploying daemon mds.cephfs.smithi117.miaplu on smithi117 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2046424360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] up:boot 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2046424360' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: daemon mds.cephfs.smithi173.pybzht assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: fsmap cephfs:1 {0=cephfs.smithi117.umapup=up:active} 1 up:standby 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:creating} 2024-04-20T11:04:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:43 smithi117 ceph-mon[15659]: daemon mds.cephfs.smithi173.pybzht is now active in filesystem cephfs as rank 1 2024-04-20T11:04:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.miaplu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.miaplu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: Deploying daemon mds.cephfs.smithi117.miaplu on smithi117 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2046424360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] up:boot 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2046424360' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: daemon mds.cephfs.smithi173.pybzht assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-20T11:04:43.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T11:04:43.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: fsmap cephfs:1 {0=cephfs.smithi117.umapup=up:active} 1 up:standby 2024-04-20T11:04:43.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:04:43.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:creating} 2024-04-20T11:04:43.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:43 smithi173 ceph-mon[23800]: daemon mds.cephfs.smithi173.pybzht is now active in filesystem cephfs as rank 1 2024-04-20T11:04:43.775 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:04:43.785 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:04:43.786 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-20T11:04:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:44 smithi173 ceph-mon[23800]: pgmap v230: 65 pgs: 65 active+clean; 579 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-20T11:04:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:04:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:44 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] up:active 2024-04-20T11:04:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:44 smithi173 ceph-mon[23800]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2024-04-20T11:04:45.028 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:44 smithi117 ceph-mon[15659]: pgmap v230: 65 pgs: 65 active+clean; 579 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-20T11:04:45.028 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:45.029 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:04:45.029 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:44 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] up:active 2024-04-20T11:04:45.029 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:44 smithi117 ceph-mon[15659]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2024-04-20T11:04:45.133 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:46.844 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:46 smithi117 ceph-mon[15659]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-20T11:04:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:46 smithi173 ceph-mon[23800]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-20T11:04:47.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.755+0000 7fbe4c000700 1 -- 172.21.15.117:0/3723642862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44101680 msgr2=0x7fbe44103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.755+0000 7fbe4c000700 1 --2- 172.21.15.117:0/3723642862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44101680 0x7fbe44103a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009910 tx=0x7fbe38009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.755+0000 7fbe4c000700 1 -- 172.21.15.117:0/3723642862 shutdown_connections 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.755+0000 7fbe4c000700 1 --2- 172.21.15.117:0/3723642862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe44103fb0 0x7fbe441063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.755+0000 7fbe4c000700 1 --2- 172.21.15.117:0/3723642862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44101680 0x7fbe44103a70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.755+0000 7fbe4c000700 1 -- 172.21.15.117:0/3723642862 >> 172.21.15.117:0/3723642862 conn(0x7fbe440fb0d0 msgr2=0x7fbe440fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.755+0000 7fbe4c000700 1 -- 172.21.15.117:0/3723642862 shutdown_connections 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.755+0000 7fbe4c000700 1 -- 172.21.15.117:0/3723642862 wait complete. 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.756+0000 7fbe4c000700 1 Processor -- start 2024-04-20T11:04:47.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.756+0000 7fbe4c000700 1 -- start start 2024-04-20T11:04:47.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.756+0000 7fbe4c000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe44101680 0x7fbe44198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:47.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.756+0000 7fbe4c000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44103fb0 0x7fbe44198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:47.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.756+0000 7fbe4c000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441993a0 con 0x7fbe44103fb0 2024-04-20T11:04:47.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.756+0000 7fbe4c000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441994e0 con 0x7fbe44101680 2024-04-20T11:04:47.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.757+0000 7fbe48c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44103fb0 0x7fbe44198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:47.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.757+0000 7fbe48c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44103fb0 0x7fbe44198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47298/0 (socket says 172.21.15.117:47298) 2024-04-20T11:04:47.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.757+0000 7fbe48c00700 1 -- 172.21.15.117:0/2462446098 learned_addr learned my addr 172.21.15.117:0/2462446098 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:47.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.757+0000 7fbe49600700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe44101680 0x7fbe44198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:47.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.757+0000 7fbe48c00700 1 -- 172.21.15.117:0/2462446098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe44101680 msgr2=0x7fbe44198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:47.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.757+0000 7fbe48c00700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe44101680 0x7fbe44198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:47.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.757+0000 7fbe48c00700 1 -- 172.21.15.117:0/2462446098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380095f0 con 0x7fbe44103fb0 2024-04-20T11:04:47.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.758+0000 7fbe48c00700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44103fb0 0x7fbe44198d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e920 tx=0x7fbe3000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:47.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.758+0000 7fbe42000700 1 -- 172.21.15.117:0/2462446098 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe3000c9d0 con 0x7fbe44103fb0 2024-04-20T11:04:47.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.758+0000 7fbe42000700 1 -- 172.21.15.117:0/2462446098 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe30010430 con 0x7fbe44103fb0 2024-04-20T11:04:47.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.758+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4419dea0 con 0x7fbe44103fb0 2024-04-20T11:04:47.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.758+0000 7fbe42000700 1 -- 172.21.15.117:0/2462446098 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe30018630 con 0x7fbe44103fb0 2024-04-20T11:04:47.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.759+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4419e440 con 0x7fbe44103fb0 2024-04-20T11:04:47.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.760+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe44062510 con 0x7fbe44103fb0 2024-04-20T11:04:47.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.761+0000 7fbe42000700 1 -- 172.21.15.117:0/2462446098 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fbe300105a0 con 0x7fbe44103fb0 2024-04-20T11:04:47.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.762+0000 7fbe42000700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbe2c06cce0 0x7fbe2c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:47.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.762+0000 7fbe42000700 1 -- 172.21.15.117:0/2462446098 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbe30014070 con 0x7fbe44103fb0 2024-04-20T11:04:47.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.762+0000 7fbe49600700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbe2c06cce0 0x7fbe2c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:47.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.763+0000 7fbe49600700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbe2c06cce0 0x7fbe2c06f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800b6f0 tx=0x7fbe38005fb0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:47.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:47.771+0000 7fbe42000700 1 -- 172.21.15.117:0/2462446098 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbe3005a8c0 con 0x7fbe44103fb0 2024-04-20T11:04:48.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.038+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbe441099a0 con 0x7fbe44103fb0 2024-04-20T11:04:48.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.695+0000 7fbe42000700 1 -- 172.21.15.117:0/2462446098 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbe30026020 con 0x7fbe44103fb0 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbe2c06cce0 msgr2=0x7fbe2c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbe2c06cce0 0x7fbe2c06f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800b6f0 tx=0x7fbe38005fb0 comp rx=0 tx=0).stop 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44103fb0 msgr2=0x7fbe44198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44103fb0 0x7fbe44198d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e920 tx=0x7fbe3000ec30 comp rx=0 tx=0).stop 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 shutdown_connections 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fbe2c06cce0 0x7fbe2c06f1a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe44101680 0x7fbe44198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 --2- 172.21.15.117:0/2462446098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe44103fb0 0x7fbe44198d80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 >> 172.21.15.117:0/2462446098 conn(0x7fbe440fb0d0 msgr2=0x7fbe440ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:48.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 shutdown_connections 2024-04-20T11:04:48.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:48.698+0000 7fbe4c000700 1 -- 172.21.15.117:0/2462446098 wait complete. 2024-04-20T11:04:48.737 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-20T11:04:48.737 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:48.737 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:48.737 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:48.737 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pfrgat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:04:48.737 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pfrgat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:04:48.738 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:48.738 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: Deploying daemon mds.cephfs.smithi173.pfrgat on smithi173 2024-04-20T11:04:48.738 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-20T11:04:48.738 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: Cluster is now healthy 2024-04-20T11:04:48.738 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] up:boot 2024-04-20T11:04:48.738 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 1 up:standby 2024-04-20T11:04:48.738 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:04:48.738 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:48 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2462446098' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-20T11:04:48.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pfrgat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pfrgat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: Deploying daemon mds.cephfs.smithi173.pfrgat on smithi173 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: Cluster is now healthy 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] up:boot 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 1 up:standby 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:04:48.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:48 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2462446098' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-20T11:04:49.583 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:04:49.594 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:04:49.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-04-20T11:04:49.816 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:49 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2462446098' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-20T11:04:49.816 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:49 smithi173 ceph-mon[23800]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 1 up:standby 2024-04-20T11:04:49.817 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:49 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:49.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:49 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2462446098' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-20T11:04:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:49 smithi117 ceph-mon[15659]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 1 up:standby 2024-04-20T11:04:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:49 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:50.010 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:50 smithi173 ceph-mon[23800]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-20T11:04:51.016 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:50 smithi117 ceph-mon[15659]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.273+0000 7f3789400700 1 -- 172.21.15.117:0/2620376535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 msgr2=0x7f3784102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.273+0000 7f3789400700 1 --2- 172.21.15.117:0/2620376535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 0x7f3784102280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3770009910 tx=0x7f3770009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.273+0000 7f3789400700 1 -- 172.21.15.117:0/2620376535 shutdown_connections 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.273+0000 7f3789400700 1 --2- 172.21.15.117:0/2620376535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 0x7f3784102280 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.273+0000 7f3789400700 1 --2- 172.21.15.117:0/2620376535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100c20 0x7f3784101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.273+0000 7f3789400700 1 -- 172.21.15.117:0/2620376535 >> 172.21.15.117:0/2620376535 conn(0x7f37840fc1c0 msgr2=0x7f37840fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.273+0000 7f3789400700 1 -- 172.21.15.117:0/2620376535 shutdown_connections 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.273+0000 7f3789400700 1 -- 172.21.15.117:0/2620376535 wait complete. 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.274+0000 7f3789400700 1 Processor -- start 2024-04-20T11:04:52.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.274+0000 7f3789400700 1 -- start start 2024-04-20T11:04:52.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.274+0000 7f3789400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100c20 0x7f3784196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:52.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.274+0000 7f3789400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 0x7f3784196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:52.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.274+0000 7f3789400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784197190 con 0x7f3784101e20 2024-04-20T11:04:52.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.274+0000 7f3789400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37841972d0 con 0x7f3784100c20 2024-04-20T11:04:52.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.275+0000 7f3781e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 0x7f3784196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:52.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.275+0000 7f3781e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 0x7f3784196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47326/0 (socket says 172.21.15.117:47326) 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.275+0000 7f3781e00700 1 -- 172.21.15.117:0/1576604755 learned_addr learned my addr 172.21.15.117:0/1576604755 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.275+0000 7f3782800700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100c20 0x7f3784196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.275+0000 7f3781e00700 1 -- 172.21.15.117:0/1576604755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100c20 msgr2=0x7f3784196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.275+0000 7f3781e00700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100c20 0x7f3784196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.275+0000 7f3781e00700 1 -- 172.21.15.117:0/1576604755 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37700095f0 con 0x7f3784101e20 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.275+0000 7f3782800700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100c20 0x7f3784196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.276+0000 7f3781e00700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 0x7f3784196b70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f377000bd70 tx=0x7f377000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.276+0000 7f377b400700 1 -- 172.21.15.117:0/1576604755 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f377001d070 con 0x7f3784101e20 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.276+0000 7f377b400700 1 -- 172.21.15.117:0/1576604755 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3770005dc0 con 0x7f3784101e20 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.276+0000 7f377b400700 1 -- 172.21.15.117:0/1576604755 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3770021410 con 0x7f3784101e20 2024-04-20T11:04:52.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.276+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378419bc80 con 0x7f3784101e20 2024-04-20T11:04:52.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.276+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378419c1f0 con 0x7f3784101e20 2024-04-20T11:04:52.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.278+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37841907a0 con 0x7f3784101e20 2024-04-20T11:04:52.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.279+0000 7f377b400700 1 -- 172.21.15.117:0/1576604755 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f377000f820 con 0x7f3784101e20 2024-04-20T11:04:52.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.280+0000 7f377b400700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f376806cce0 0x7f376806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:52.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.280+0000 7f377b400700 1 -- 172.21.15.117:0/1576604755 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f377008d400 con 0x7f3784101e20 2024-04-20T11:04:52.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.280+0000 7f3782800700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f376806cce0 0x7f376806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:52.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.281+0000 7f3782800700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f376806cce0 0x7f376806f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3774005950 tx=0x7f37740058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:52.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.285+0000 7f377b400700 1 -- 172.21.15.117:0/1576604755 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f377005b2e0 con 0x7f3784101e20 2024-04-20T11:04:52.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:52.626+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3784062510 con 0x7f3784101e20 2024-04-20T11:04:52.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-20T11:04:52.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] up:boot 2024-04-20T11:04:52.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:04:52.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:04:52.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:52 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:04:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:52.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-20T11:04:52.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] up:boot 2024-04-20T11:04:52.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:04:52.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:04:52.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:52 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:04:53.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.548+0000 7f377b400700 1 -- 172.21.15.117:0/1576604755 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) v1 ==== 168+0+0 (secure 0 0 0) 0x7f377005ae70 con 0x7f3784101e20 2024-04-20T11:04:53.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f376806cce0 msgr2=0x7f376806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:53.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f376806cce0 0x7f376806f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3774005950 tx=0x7f37740058e0 comp rx=0 tx=0).stop 2024-04-20T11:04:53.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 msgr2=0x7f3784196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:53.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 0x7f3784196b70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f377000bd70 tx=0x7f377000be50 comp rx=0 tx=0).stop 2024-04-20T11:04:53.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 shutdown_connections 2024-04-20T11:04:53.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f376806cce0 0x7f376806f1a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:53.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100c20 0x7f3784196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:53.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 --2- 172.21.15.117:0/1576604755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101e20 0x7f3784196b70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:53.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 >> 172.21.15.117:0/1576604755 conn(0x7f37840fc1c0 msgr2=0x7f3784105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:53.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 shutdown_connections 2024-04-20T11:04:53.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:53.551+0000 7f3789400700 1 -- 172.21.15.117:0/1576604755 wait complete. 2024-04-20T11:04:53.553 INFO:teuthology.orchestra.run.smithi117.stderr:inline data enabled 2024-04-20T11:04:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:53 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1576604755' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:04:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:53 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1576604755' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:04:54.367 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:04:54.379 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:04:54.379 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-20T11:04:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:54 smithi117 ceph-mon[15659]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-20T11:04:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:54 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1576604755' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T11:04:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:54 smithi117 ceph-mon[15659]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:04:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:54 smithi117 ceph-mon[15659]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-04-20T11:04:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:54 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:54.858 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:54 smithi173 ceph-mon[23800]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-20T11:04:54.858 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:54 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1576604755' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T11:04:54.859 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:54 smithi173 ceph-mon[23800]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:04:54.859 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:54 smithi173 ceph-mon[23800]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-04-20T11:04:54.859 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:54 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:55.419 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:04:56.315 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:56.315 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:56 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:56.315 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:56 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] up:active 2024-04-20T11:04:56.315 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:56 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] up:active 2024-04-20T11:04:56.315 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:56 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] up:standby 2024-04-20T11:04:56.315 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:56 smithi117 ceph-mon[15659]: mds.? [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] up:standby 2024-04-20T11:04:56.315 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:56 smithi117 ceph-mon[15659]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:04:56.315 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:56 smithi117 ceph-mon[15659]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-20T11:04:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:56 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:56 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] up:active 2024-04-20T11:04:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:56 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] up:active 2024-04-20T11:04:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:56 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] up:standby 2024-04-20T11:04:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:56 smithi173 ceph-mon[23800]: mds.? [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] up:standby 2024-04-20T11:04:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:56 smithi173 ceph-mon[23800]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:04:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:56 smithi173 ceph-mon[23800]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-20T11:04:58.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.711+0000 7f1937c00700 1 -- 172.21.15.117:0/1561240860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1930100c00 msgr2=0x7f1930101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:58.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.711+0000 7f1937c00700 1 --2- 172.21.15.117:0/1561240860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1930100c00 0x7f1930101020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1924009910 tx=0x7f1924009c20 comp rx=0 tx=0).stop 2024-04-20T11:04:58.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.711+0000 7f1937c00700 1 -- 172.21.15.117:0/1561240860 shutdown_connections 2024-04-20T11:04:58.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.711+0000 7f1937c00700 1 --2- 172.21.15.117:0/1561240860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1930101e00 0x7f1930102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:58.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.711+0000 7f1937c00700 1 --2- 172.21.15.117:0/1561240860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1930100c00 0x7f1930101020 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.711+0000 7f1937c00700 1 -- 172.21.15.117:0/1561240860 >> 172.21.15.117:0/1561240860 conn(0x7f19300fc1c0 msgr2=0x7f19300fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.711+0000 7f1937c00700 1 -- 172.21.15.117:0/1561240860 shutdown_connections 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.711+0000 7f1937c00700 1 -- 172.21.15.117:0/1561240860 wait complete. 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.712+0000 7f1937c00700 1 Processor -- start 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.712+0000 7f1937c00700 1 -- start start 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1937c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1930100c00 0x7f1930196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1937c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1930101e00 0x7f1930196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1937c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1930197180 con 0x7f1930100c00 2024-04-20T11:04:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1937c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19301972c0 con 0x7f1930101e00 2024-04-20T11:04:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1935200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1930100c00 0x7f1930196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1935200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1930100c00 0x7f1930196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51970/0 (socket says 172.21.15.117:51970) 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1935200700 1 -- 172.21.15.117:0/544093033 learned_addr learned my addr 172.21.15.117:0/544093033 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f192fe00700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1930101e00 0x7f1930196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1935200700 1 -- 172.21.15.117:0/544093033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1930101e00 msgr2=0x7f1930196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1935200700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1930101e00 0x7f1930196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.713+0000 7f1935200700 1 -- 172.21.15.117:0/544093033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19240095f0 con 0x7f1930100c00 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.714+0000 7f1935200700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1930100c00 0x7f1930196620 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f192400b800 tx=0x7f192400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.714+0000 7f192d600700 1 -- 172.21.15.117:0/544093033 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f192401d070 con 0x7f1930100c00 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.714+0000 7f192d600700 1 -- 172.21.15.117:0/544093033 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f192400fe70 con 0x7f1930100c00 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.714+0000 7f1937c00700 1 -- 172.21.15.117:0/544093033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193019bcc0 con 0x7f1930100c00 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.714+0000 7f192d600700 1 -- 172.21.15.117:0/544093033 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1924017400 con 0x7f1930100c00 2024-04-20T11:04:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.714+0000 7f1937c00700 1 -- 172.21.15.117:0/544093033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193019c1b0 con 0x7f1930100c00 2024-04-20T11:04:58.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.716+0000 7f1922a00700 1 -- 172.21.15.117:0/544093033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c0052f0 con 0x7f1930100c00 2024-04-20T11:04:58.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.717+0000 7f192d600700 1 -- 172.21.15.117:0/544093033 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f192400f9b0 con 0x7f1930100c00 2024-04-20T11:04:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.718+0000 7f192d600700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f191806cd30 0x7f191806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:04:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.718+0000 7f192d600700 1 -- 172.21.15.117:0/544093033 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f192408d640 con 0x7f1930100c00 2024-04-20T11:04:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.718+0000 7f192fe00700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f191806cd30 0x7f191806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:04:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.719+0000 7f192fe00700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f191806cd30 0x7f191806f1f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f191c009bf0 tx=0x7f191c009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:04:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:58.723+0000 7f192d600700 1 -- 172.21.15.117:0/544093033 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f192405b500 con 0x7f1930100c00 2024-04-20T11:04:58.866 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:58 smithi117 ceph-mon[15659]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:04:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:58 smithi173 ceph-mon[23800]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:04:59.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.031+0000 7f1922a00700 1 -- 172.21.15.117:0/544093033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f190c005db0 con 0x7f1930100c00 2024-04-20T11:04:59.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.032+0000 7f192d600700 1 -- 172.21.15.117:0/544093033 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1854 (secure 0 0 0) 0x7f192405b090 con 0x7f1930100c00 2024-04-20T11:04:59.032 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:04:59.032 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:04:59.032 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:04:59.032 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:04:59.032 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:59.032 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:04:59.032 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:04:59.033 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:04:59.034 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:04:59.035 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:04:59.036 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:04:59.036 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:04:59.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 -- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f191806cd30 msgr2=0x7f191806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f191806cd30 0x7f191806f1f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f191c009bf0 tx=0x7f191c009260 comp rx=0 tx=0).stop 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 -- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1930100c00 msgr2=0x7f1930196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1930100c00 0x7f1930196620 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f192400b800 tx=0x7f192400f510 comp rx=0 tx=0).stop 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 -- 172.21.15.117:0/544093033 shutdown_connections 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f191806cd30 0x7f191806f1f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1930100c00 0x7f1930196620 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 --2- 172.21.15.117:0/544093033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1930101e00 0x7f1930196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.034+0000 7f1922a00700 1 -- 172.21.15.117:0/544093033 >> 172.21.15.117:0/544093033 conn(0x7f19300fc1c0 msgr2=0x7f1930105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.035+0000 7f1922a00700 1 -- 172.21.15.117:0/544093033 shutdown_connections 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:04:59.035+0000 7f1922a00700 1 -- 172.21.15.117:0/544093033 wait complete. 2024-04-20T11:04:59.037 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:04:59.934 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:04:59.934 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:59 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:04:59.934 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:04:59 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/544093033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:05:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:59 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:05:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:04:59 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/544093033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:05:00.249 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-20T11:05:00.645 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:05:00.947 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:00 smithi117 ceph-mon[15659]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-20T11:05:00.947 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:00 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:00.947 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:00 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:00.947 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:00 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:05:00.947 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:00 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:05:00.948 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:00 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:01.048 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:00 smithi173 ceph-mon[23800]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-20T11:05:01.048 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:00 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:01.048 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:00 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:01.048 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:00 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:05:01.048 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:00 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:05:01.049 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:00 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:01.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:01 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:05:02.020 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:01 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:05:02.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.683+0000 7efd45a00700 1 -- 172.21.15.117:0/251244180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40101e00 msgr2=0x7efd40102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:02.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.683+0000 7efd45a00700 1 --2- 172.21.15.117:0/251244180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40101e00 0x7efd40102260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009910 tx=0x7efd2c009c20 comp rx=0 tx=0).stop 2024-04-20T11:05:02.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.684+0000 7efd45a00700 1 -- 172.21.15.117:0/251244180 shutdown_connections 2024-04-20T11:05:02.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.684+0000 7efd45a00700 1 --2- 172.21.15.117:0/251244180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40101e00 0x7efd40102260 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:02.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.684+0000 7efd45a00700 1 --2- 172.21.15.117:0/251244180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd40100c00 0x7efd40101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:02.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.684+0000 7efd45a00700 1 -- 172.21.15.117:0/251244180 >> 172.21.15.117:0/251244180 conn(0x7efd400fc1c0 msgr2=0x7efd400fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:02.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.684+0000 7efd45a00700 1 -- 172.21.15.117:0/251244180 shutdown_connections 2024-04-20T11:05:02.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.684+0000 7efd45a00700 1 -- 172.21.15.117:0/251244180 wait complete. 2024-04-20T11:05:02.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.685+0000 7efd45a00700 1 Processor -- start 2024-04-20T11:05:02.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.685+0000 7efd45a00700 1 -- start start 2024-04-20T11:05:02.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.685+0000 7efd45a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40100c00 0x7efd40074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:02.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.685+0000 7efd45a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd40101e00 0x7efd400732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:02.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.685+0000 7efd45a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd400751c0 con 0x7efd40100c00 2024-04-20T11:05:02.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.685+0000 7efd45a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd40073810 con 0x7efd40101e00 2024-04-20T11:05:02.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.686+0000 7efd3ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40100c00 0x7efd40074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:02.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.686+0000 7efd3ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40100c00 0x7efd40074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51982/0 (socket says 172.21.15.117:51982) 2024-04-20T11:05:02.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.686+0000 7efd3ea00700 1 -- 172.21.15.117:0/2243894918 learned_addr learned my addr 172.21.15.117:0/2243894918 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:02.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.686+0000 7efd3e000700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd40101e00 0x7efd400732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:02.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.686+0000 7efd3ea00700 1 -- 172.21.15.117:0/2243894918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd40101e00 msgr2=0x7efd400732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:02.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.686+0000 7efd3ea00700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd40101e00 0x7efd400732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:02.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.686+0000 7efd3ea00700 1 -- 172.21.15.117:0/2243894918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c0095f0 con 0x7efd40100c00 2024-04-20T11:05:02.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.686+0000 7efd3ea00700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40100c00 0x7efd40074c30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c910 tx=0x7efd3000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:02.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.687+0000 7efd37400700 1 -- 172.21.15.117:0/2243894918 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd300041f0 con 0x7efd40100c00 2024-04-20T11:05:02.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.687+0000 7efd37400700 1 -- 172.21.15.117:0/2243894918 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd30007dc0 con 0x7efd40100c00 2024-04-20T11:05:02.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.687+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd40073af0 con 0x7efd40100c00 2024-04-20T11:05:02.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.687+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd400740c0 con 0x7efd40100c00 2024-04-20T11:05:02.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.687+0000 7efd37400700 1 -- 172.21.15.117:0/2243894918 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd30003e70 con 0x7efd40100c00 2024-04-20T11:05:02.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.688+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd40190780 con 0x7efd40100c00 2024-04-20T11:05:02.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.689+0000 7efd37400700 1 -- 172.21.15.117:0/2243894918 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7efd30004350 con 0x7efd40100c00 2024-04-20T11:05:02.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.690+0000 7efd37400700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efd2406cc80 0x7efd2406f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:02.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.690+0000 7efd37400700 1 -- 172.21.15.117:0/2243894918 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7efd3008bfd0 con 0x7efd40100c00 2024-04-20T11:05:02.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.690+0000 7efd3e000700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efd2406cc80 0x7efd2406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:02.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.691+0000 7efd3e000700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efd2406cc80 0x7efd2406f140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0051c0 tx=0x7efd2c00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:02.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.695+0000 7efd37400700 1 -- 172.21.15.117:0/2243894918 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7efd30059e30 con 0x7efd40100c00 2024-04-20T11:05:02.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:02 smithi117 ceph-mon[15659]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-20T11:05:02.955 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:02 smithi173 ceph-mon[23800]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-20T11:05:02.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.965+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd40109a10 con 0x7efd40100c00 2024-04-20T11:05:02.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.966+0000 7efd37400700 1 -- 172.21.15.117:0/2243894918 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4841 (secure 0 0 0) 0x7efd300599c0 con 0x7efd40100c00 2024-04-20T11:05:02.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.969+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efd2406cc80 msgr2=0x7efd2406f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:02.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.969+0000 7efd45a00700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efd2406cc80 0x7efd2406f140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0051c0 tx=0x7efd2c00b540 comp rx=0 tx=0).stop 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.969+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40100c00 msgr2=0x7efd40074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.969+0000 7efd45a00700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40100c00 0x7efd40074c30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c910 tx=0x7efd3000ccd0 comp rx=0 tx=0).stop 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.969+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 shutdown_connections 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.970+0000 7efd45a00700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7efd2406cc80 0x7efd2406f140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.970+0000 7efd45a00700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd40100c00 0x7efd40074c30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.970+0000 7efd45a00700 1 --2- 172.21.15.117:0/2243894918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd40101e00 0x7efd400732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.970+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 >> 172.21.15.117:0/2243894918 conn(0x7efd400fc1c0 msgr2=0x7efd40105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.970+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 shutdown_connections 2024-04-20T11:05:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:02.970+0000 7efd45a00700 1 -- 172.21.15.117:0/2243894918 wait complete. 2024-04-20T11:05:02.971 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:05:02.984 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:05:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:03 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2243894918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:05:04.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:03 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2243894918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:05:04.470 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-20T11:05:04.710 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:04 smithi117 ceph-mon[15659]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-20T11:05:04.710 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:04.710 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:04 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:05.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:04 smithi173 ceph-mon[23800]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-20T11:05:05.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:05.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:04 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:05.114 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:05:07.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:06 smithi117 ceph-mon[15659]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:06 smithi173 ceph-mon[23800]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:08.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.733+0000 7f7ba0800700 1 -- 172.21.15.117:0/3129454167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 msgr2=0x7f7b98103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:08.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.733+0000 7f7ba0800700 1 --2- 172.21.15.117:0/3129454167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 0x7f7b98103a70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009910 tx=0x7f7b8c009c20 comp rx=0 tx=0).stop 2024-04-20T11:05:08.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.733+0000 7f7ba0800700 1 -- 172.21.15.117:0/3129454167 shutdown_connections 2024-04-20T11:05:08.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.733+0000 7f7ba0800700 1 --2- 172.21.15.117:0/3129454167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b98103fb0 0x7f7b981063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:08.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.733+0000 7f7ba0800700 1 --2- 172.21.15.117:0/3129454167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 0x7f7b98103a70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:08.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.733+0000 7f7ba0800700 1 -- 172.21.15.117:0/3129454167 >> 172.21.15.117:0/3129454167 conn(0x7f7b980fb0d0 msgr2=0x7f7b980fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:08.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.734+0000 7f7ba0800700 1 -- 172.21.15.117:0/3129454167 shutdown_connections 2024-04-20T11:05:08.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.734+0000 7f7ba0800700 1 -- 172.21.15.117:0/3129454167 wait complete. 2024-04-20T11:05:08.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.735+0000 7f7ba0800700 1 Processor -- start 2024-04-20T11:05:08.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.735+0000 7f7ba0800700 1 -- start start 2024-04-20T11:05:08.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.736+0000 7f7ba0800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 0x7f7b98196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:08.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.736+0000 7f7ba0800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b98103fb0 0x7f7b98196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:08.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.736+0000 7f7ba0800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981970f0 con 0x7f7b98101680 2024-04-20T11:05:08.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.736+0000 7f7ba0800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b98197230 con 0x7f7b98103fb0 2024-04-20T11:05:08.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.736+0000 7f7b9de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 0x7f7b98196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:08.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.736+0000 7f7b9de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 0x7f7b98196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33460/0 (socket says 172.21.15.117:33460) 2024-04-20T11:05:08.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.736+0000 7f7b9de00700 1 -- 172.21.15.117:0/644570768 learned_addr learned my addr 172.21.15.117:0/644570768 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:08.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.736+0000 7f7b9d400700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b98103fb0 0x7f7b98196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:08.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.737+0000 7f7b9de00700 1 -- 172.21.15.117:0/644570768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b98103fb0 msgr2=0x7f7b98196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:08.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.737+0000 7f7b9de00700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b98103fb0 0x7f7b98196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:08.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.737+0000 7f7b9de00700 1 -- 172.21.15.117:0/644570768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0095f0 con 0x7f7b98101680 2024-04-20T11:05:08.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.737+0000 7f7b9de00700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 0x7f7b98196590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c006010 tx=0x7f7b8c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:08.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.738+0000 7f7b92a00700 1 -- 172.21.15.117:0/644570768 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b8c01d070 con 0x7f7b98101680 2024-04-20T11:05:08.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.738+0000 7f7b92a00700 1 -- 172.21.15.117:0/644570768 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b8c005dc0 con 0x7f7b98101680 2024-04-20T11:05:08.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.738+0000 7f7b92a00700 1 -- 172.21.15.117:0/644570768 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b8c021410 con 0x7f7b98101680 2024-04-20T11:05:08.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.738+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980ff5b0 con 0x7f7b98101680 2024-04-20T11:05:08.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.738+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b980ffb20 con 0x7f7b98101680 2024-04-20T11:05:08.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.740+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b98062510 con 0x7f7b98101680 2024-04-20T11:05:08.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.740+0000 7f7b92a00700 1 -- 172.21.15.117:0/644570768 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f7b8c00f460 con 0x7f7b98101680 2024-04-20T11:05:08.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.741+0000 7f7b92a00700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b8006cce0 0x7f7b8006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:08.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.741+0000 7f7b92a00700 1 -- 172.21.15.117:0/644570768 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7b8c08d4b0 con 0x7f7b98101680 2024-04-20T11:05:08.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.742+0000 7f7b9d400700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b8006cce0 0x7f7b8006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:08.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.742+0000 7f7b9d400700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b8006cce0 0x7f7b8006f1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005950 tx=0x7f7b8800b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:08.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:08.747+0000 7f7b92a00700 1 -- 172.21.15.117:0/644570768 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7b8c05b310 con 0x7f7b98101680 2024-04-20T11:05:08.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:08 smithi117 ceph-mon[15659]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:09.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.073+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f7b98109750 con 0x7f7b98101680 2024-04-20T11:05:09.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.074+0000 7f7b92a00700 1 -- 172.21.15.117:0/644570768 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v12) v1 ==== 78+0+98 (secure 0 0 0) 0x7f7b8c05aea0 con 0x7f7b98101680 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b8006cce0 msgr2=0x7f7b8006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b8006cce0 0x7f7b8006f1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005950 tx=0x7f7b8800b410 comp rx=0 tx=0).stop 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 msgr2=0x7f7b98196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 0x7f7b98196590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c006010 tx=0x7f7b8c00bd70 comp rx=0 tx=0).stop 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 shutdown_connections 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f7b8006cce0 0x7f7b8006f1a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b98101680 0x7f7b98196590 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 --2- 172.21.15.117:0/644570768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b98103fb0 0x7f7b98196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 >> 172.21.15.117:0/644570768 conn(0x7f7b980fb0d0 msgr2=0x7f7b980fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 shutdown_connections 2024-04-20T11:05:09.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:09.077+0000 7f7ba0800700 1 -- 172.21.15.117:0/644570768 wait complete. 2024-04-20T11:05:09.094 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:05:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:08 smithi173 ceph-mon[23800]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:09.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:09 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/644570768' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-20T11:05:10.024 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-20T11:05:10.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:05:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:09 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/644570768' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-20T11:05:10.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.292+0000 7fdd9f200640 1 -- 172.21.15.117:0/1981824930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0107720 msgr2=0x7fdda0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:10.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.292+0000 7fdd9f200640 1 --2- 172.21.15.117:0/1981824930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0107720 0x7fdda0107b80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009880 tx=0x7fdd8802bcf0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.293+0000 7fdd9f200640 1 -- 172.21.15.117:0/1981824930 shutdown_connections 2024-04-20T11:05:10.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.293+0000 7fdd9f200640 1 --2- 172.21.15.117:0/1981824930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0107720 0x7fdda0107b80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.293+0000 7fdd9f200640 1 --2- 172.21.15.117:0/1981824930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0106510 0x7fdda0106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.293+0000 7fdd9f200640 1 -- 172.21.15.117:0/1981824930 >> 172.21.15.117:0/1981824930 conn(0x7fdda0075530 msgr2=0x7fdda0077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:10.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.293+0000 7fdd9f200640 1 -- 172.21.15.117:0/1981824930 shutdown_connections 2024-04-20T11:05:10.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.293+0000 7fdd9f200640 1 -- 172.21.15.117:0/1981824930 wait complete. 2024-04-20T11:05:10.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9f200640 1 Processor -- start 2024-04-20T11:05:10.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9f200640 1 -- start start 2024-04-20T11:05:10.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0106510 0x7fdda019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:10.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0107720 0x7fdda019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:10.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019c8d0 con 0x7fdda0106510 2024-04-20T11:05:10.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019ca40 con 0x7fdda0107720 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0106510 0x7fdda019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0106510 0x7fdda019bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33486/0 (socket says 172.21.15.117:33486) 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.294+0000 7fdd9de00640 1 -- 172.21.15.117:0/2648206489 learned_addr learned my addr 172.21.15.117:0/2648206489 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd95400640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0107720 0x7fdda019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd9de00640 1 -- 172.21.15.117:0/2648206489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0107720 msgr2=0x7fdda019c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd9de00640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0107720 0x7fdda019c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd9de00640 1 -- 172.21.15.117:0/2648206489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd88009560 con 0x7fdda0106510 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd95400640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0107720 0x7fdda019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd9de00640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0106510 0x7fdda019bdc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000cc60 tx=0x7fdd9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd97400640 1 -- 172.21.15.117:0/2648206489 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd90004b20 con 0x7fdda0106510 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd97400640 1 -- 172.21.15.117:0/2648206489 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd90004c80 con 0x7fdda0106510 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd97400640 1 -- 172.21.15.117:0/2648206489 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd9000f640 con 0x7fdda0106510 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.295+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda01a14c0 con 0x7fdda0106510 2024-04-20T11:05:10.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.296+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda01a1aa0 con 0x7fdda0106510 2024-04-20T11:05:10.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.297+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda010f290 con 0x7fdda0106510 2024-04-20T11:05:10.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.298+0000 7fdd97400640 1 -- 172.21.15.117:0/2648206489 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fdd900055e0 con 0x7fdda0106510 2024-04-20T11:05:10.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.298+0000 7fdd97400640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdd8406cd10 0x7fdd8406f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:10.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.299+0000 7fdd97400640 1 -- 172.21.15.117:0/2648206489 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fdd9008c340 con 0x7fdda0106510 2024-04-20T11:05:10.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.299+0000 7fdd95400640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdd8406cd10 0x7fdd8406f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:10.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.299+0000 7fdd95400640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdd8406cd10 0x7fdd8406f1d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005f70 tx=0x7fdd88000bc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:10.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.303+0000 7fdd97400640 1 -- 172.21.15.117:0/2648206489 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdd90056790 con 0x7fdda0106510 2024-04-20T11:05:10.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.534+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdda01a1d70 con 0x7fdda0106510 2024-04-20T11:05:10.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.535+0000 7fdd97400640 1 -- 172.21.15.117:0/2648206489 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15075 (secure 0 0 0) 0x7fdd90059db0 con 0x7fdda0106510 2024-04-20T11:05:10.535 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:05:10.535 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":47,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","created":"2024-04-20T10:55:31.778050+0000","modified":"2024-04-20T11:04:35.751100+0000","last_up_change":"2024-04-20T11:02:58.642045+0000","last_in_change":"2024-04-20T11:02:33.545131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:01:14.283466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-20T11:04:32.294012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-20T11:04:33.229067+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6803","nonce":2292675302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6805","nonce":2292675302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6809","nonce":2292675302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2292675302},{"type":"v1","addr":"172.21.15.117:6807","nonce":2292675302}]},"public_addr":"172.21.15.117:6803/2292675302","cluster_addr":"172.21.15.117:6805/2292675302","heartbeat_back_addr":"172.21.15.117:6809/2292675302","heartbeat_front_addr":"172.21.15.117:6807/2292675302","state":["exists","up"]},{"osd":1,"uuid":"75ea37ae-266a-459b-a32b-e1d91927cbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6811","nonce":2871230534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6813","nonce":2871230534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6817","nonce":2871230534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2871230534},{"type":"v1","addr":"172.21.15.117:6815","nonce":2871230534}]},"public_addr":"172.21.15.117:6811/2871230534","cluster_addr":"172.21.15.117:6813/2871230534","heartbeat_back_addr":"172.21.15.117:6817/2871230534","heartbeat_front_addr":"172.21.15.117:6815/2871230534","state":["exists","up"]},{"osd":2,"uuid":"2323b6ba-7d04-4744-b61f-4936e4f419bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6819","nonce":3091267150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6821","nonce":3091267150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6825","nonce":3091267150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3091267150},{"type":"v1","addr":"172.21.15.117:6823","nonce":3091267150}]},"public_addr":"172.21.15.117:6819/3091267150","cluster_addr":"172.21.15.117:6821/3091267150","heartbeat_back_addr":"172.21.15.117:6825/3091267150","heartbeat_front_addr":"172.21.15.117:6823/3091267150","state":["exists","up"]},{"osd":3,"uuid":"b87d3df6-0d27-4e2d-9e40-55895e0a891d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6801","nonce":2587001742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6803","nonce":2587001742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6807","nonce":2587001742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2587001742},{"type":"v1","addr":"172.21.15.173:6805","nonce":2587001742}]},"public_addr":"172.21.15.173:6801/2587001742","cluster_addr":"172.21.15.173:6803/2587001742","heartbeat_back_addr":"172.21.15.173:6807/2587001742","heartbeat_front_addr":"172.21.15.173:6805/2587001742","state":["exists","up"]},{"osd":4,"uuid":"c5ceed42-a923-47e4-919d-98caab4e06ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6809","nonce":1468348921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6811","nonce":1468348921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6815","nonce":1468348921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1468348921},{"type":"v1","addr":"172.21.15.173:6813","nonce":1468348921}]},"public_addr":"172.21.15.173:6809/1468348921","cluster_addr":"172.21.15.173:6811/1468348921","heartbeat_back_addr":"172.21.15.173:6815/1468348921","heartbeat_front_addr":"172.21.15.173:6813/1468348921","state":["exists","up"]},{"osd":5,"uuid":"47b99706-a27f-4f30-9e6d-619ee6a35ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6817","nonce":1909838821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6819","nonce":1909838821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6823","nonce":1909838821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1909838821},{"type":"v1","addr":"172.21.15.173:6821","nonce":1909838821}]},"public_addr":"172.21.15.173:6817/1909838821","cluster_addr":"172.21.15.173:6819/1909838821","heartbeat_back_addr":"172.21.15.173:6823/1909838821","heartbeat_front_addr":"172.21.15.173:6821/1909838821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:59:57.050973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:00:33.939553+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:11.618080+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:46.287328+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:20.465163+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:56.550872+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/3713452282":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6800/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1089633358":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6801/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6801/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1807203325":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6800/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1384022397":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2096654470":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/4088938050":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/1375611439":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2304890710":"2024-04-21T10:56:04.789705+0000","172.21.15.117:6800/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2230602257":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6801/1432319225":"2024-04-21T10:56:04.789705+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdd8406cd10 msgr2=0x7fdd8406f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdd8406cd10 0x7fdd8406f1d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005f70 tx=0x7fdd88000bc0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0106510 msgr2=0x7fdda019bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0106510 0x7fdda019bdc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000cc60 tx=0x7fdd9000a620 comp rx=0 tx=0).stop 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 shutdown_connections 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fdd8406cd10 0x7fdd8406f1d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0107720 0x7fdda019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 --2- 172.21.15.117:0/2648206489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdda0106510 0x7fdda019bdc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 >> 172.21.15.117:0/2648206489 conn(0x7fdda0075530 msgr2=0x7fdda010a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 shutdown_connections 2024-04-20T11:05:10.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.538+0000 7fdd9f200640 1 -- 172.21.15.117:0/2648206489 wait complete. 2024-04-20T11:05:10.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:01:14.283466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-20T11:04:32.294012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-20T11:04:33.229067+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:05:10.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:05:10.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.738+0000 7fc434c00640 1 -- 172.21.15.117:0/821096800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc430073010 msgr2=0x7fc430107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:10.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.738+0000 7fc434c00640 1 --2- 172.21.15.117:0/821096800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc430073010 0x7fc430107d60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009880 tx=0x7fc41c02bce0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.738+0000 7fc434c00640 1 -- 172.21.15.117:0/821096800 shutdown_connections 2024-04-20T11:05:10.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.738+0000 7fc434c00640 1 --2- 172.21.15.117:0/821096800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc430073010 0x7fc430107d60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.738+0000 7fc434c00640 1 --2- 172.21.15.117:0/821096800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4300726b0 0x7fc430072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.738+0000 7fc434c00640 1 -- 172.21.15.117:0/821096800 >> 172.21.15.117:0/821096800 conn(0x7fc4300fb470 msgr2=0x7fc4300fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.738+0000 7fc434c00640 1 -- 172.21.15.117:0/821096800 shutdown_connections 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.738+0000 7fc434c00640 1 -- 172.21.15.117:0/821096800 wait complete. 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.739+0000 7fc434c00640 1 Processor -- start 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.739+0000 7fc434c00640 1 -- start start 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.740+0000 7fc434c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4300726b0 0x7fc430197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.740+0000 7fc434c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc430073010 0x7fc430197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.740+0000 7fc434c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4301985c0 con 0x7fc4300726b0 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.740+0000 7fc434c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc430198730 con 0x7fc430073010 2024-04-20T11:05:10.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.740+0000 7fc42f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4300726b0 0x7fc430197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.740+0000 7fc42f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4300726b0 0x7fc430197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33510/0 (socket says 172.21.15.117:33510) 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.740+0000 7fc42f400640 1 -- 172.21.15.117:0/3127380558 learned_addr learned my addr 172.21.15.117:0/3127380558 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.740+0000 7fc42ea00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc430073010 0x7fc430197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.741+0000 7fc42f400640 1 -- 172.21.15.117:0/3127380558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc430073010 msgr2=0x7fc430197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.741+0000 7fc42f400640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc430073010 0x7fc430197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.741+0000 7fc42f400640 1 -- 172.21.15.117:0/3127380558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c009560 con 0x7fc4300726b0 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.741+0000 7fc42f400640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4300726b0 0x7fc430197ab0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc42000c8f0 tx=0x7fc42000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.741+0000 7fc427e00640 1 -- 172.21.15.117:0/3127380558 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc420007b80 con 0x7fc4300726b0 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.742+0000 7fc427e00640 1 -- 172.21.15.117:0/3127380558 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4200054e0 con 0x7fc4300726b0 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.742+0000 7fc427e00640 1 -- 172.21.15.117:0/3127380558 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc42000f450 con 0x7fc4300726b0 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.742+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43019d1c0 con 0x7fc4300726b0 2024-04-20T11:05:10.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.742+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43019d7a0 con 0x7fc4300726b0 2024-04-20T11:05:10.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.743+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc430191b60 con 0x7fc4300726b0 2024-04-20T11:05:10.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.744+0000 7fc427e00640 1 -- 172.21.15.117:0/3127380558 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fc420004f60 con 0x7fc4300726b0 2024-04-20T11:05:10.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.745+0000 7fc427e00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc41406cc50 0x7fc41406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:10.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.745+0000 7fc427e00640 1 -- 172.21.15.117:0/3127380558 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc42008bfc0 con 0x7fc4300726b0 2024-04-20T11:05:10.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.745+0000 7fc42ea00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc41406cc50 0x7fc41406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:10.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.746+0000 7fc42ea00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc41406cc50 0x7fc41406f110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc430198fc0 tx=0x7fc41c000bc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:10.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.750+0000 7fc427e00640 1 -- 172.21.15.117:0/3127380558 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc420056410 con 0x7fc4300726b0 2024-04-20T11:05:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.973+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc43002c500 con 0x7fc4300726b0 2024-04-20T11:05:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.973+0000 7fc427e00640 1 -- 172.21.15.117:0/3127380558 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc42001d020 con 0x7fc4300726b0 2024-04-20T11:05:10.973 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.976+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc41406cc50 msgr2=0x7fc41406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.976+0000 7fc434c00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc41406cc50 0x7fc41406f110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc430198fc0 tx=0x7fc41c000bc0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.976+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4300726b0 msgr2=0x7fc430197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.976+0000 7fc434c00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4300726b0 0x7fc430197ab0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc42000c8f0 tx=0x7fc42000cdb0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.977+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 shutdown_connections 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.977+0000 7fc434c00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fc41406cc50 0x7fc41406f110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.977+0000 7fc434c00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc430073010 0x7fc430197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.977+0000 7fc434c00640 1 --2- 172.21.15.117:0/3127380558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4300726b0 0x7fc430197ab0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.977+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 >> 172.21.15.117:0/3127380558 conn(0x7fc4300fb470 msgr2=0x7fc4301065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.977+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 shutdown_connections 2024-04-20T11:05:10.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:10.977+0000 7fc434c00640 1 -- 172.21.15.117:0/3127380558 wait complete. 2024-04-20T11:05:10.992 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:10 smithi117 ceph-mon[15659]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:10.992 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:10 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2648206489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:05:10.995 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-20T11:05:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:10 smithi173 ceph-mon[23800]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:10 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2648206489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:05:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.172+0000 7f5f92c00640 1 -- 172.21.15.117:0/1365842745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 msgr2=0x7f5f8c1026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.172+0000 7f5f92c00640 1 --2- 172.21.15.117:0/1365842745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 0x7f5f8c1026f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009880 tx=0x7f5f7c02bce0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.172+0000 7f5f92c00640 1 -- 172.21.15.117:0/1365842745 shutdown_connections 2024-04-20T11:05:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.172+0000 7f5f92c00640 1 --2- 172.21.15.117:0/1365842745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f8c1034c0 0x7f5f8c103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.172+0000 7f5f92c00640 1 --2- 172.21.15.117:0/1365842745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 0x7f5f8c1026f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.172+0000 7f5f92c00640 1 -- 172.21.15.117:0/1365842745 >> 172.21.15.117:0/1365842745 conn(0x7f5f8c0fd840 msgr2=0x7f5f8c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.172+0000 7f5f92c00640 1 -- 172.21.15.117:0/1365842745 shutdown_connections 2024-04-20T11:05:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.172+0000 7f5f92c00640 1 -- 172.21.15.117:0/1365842745 wait complete. 2024-04-20T11:05:11.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.173+0000 7f5f92c00640 1 Processor -- start 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.173+0000 7f5f92c00640 1 -- start start 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f92c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 0x7f5f8c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f92c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f8c1034c0 0x7f5f8c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f92c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1986a0 con 0x7f5f8c1022d0 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f92c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c198810 con 0x7f5f8c1034c0 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f8be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 0x7f5f8c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f8be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 0x7f5f8c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33528/0 (socket says 172.21.15.117:33528) 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f8be00640 1 -- 172.21.15.117:0/2344654998 learned_addr learned my addr 172.21.15.117:0/2344654998 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f8be00640 1 -- 172.21.15.117:0/2344654998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f8c1034c0 msgr2=0x7f5f8c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f8be00640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f8c1034c0 0x7f5f8c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.174+0000 7f5f8be00640 1 -- 172.21.15.117:0/2344654998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c009560 con 0x7f5f8c1022d0 2024-04-20T11:05:11.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.175+0000 7f5f8be00640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 0x7f5f8c197b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c02c220 tx=0x7f5f7c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:11.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.175+0000 7f5f88c00640 1 -- 172.21.15.117:0/2344654998 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f7c0035f0 con 0x7f5f8c1022d0 2024-04-20T11:05:11.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.175+0000 7f5f88c00640 1 -- 172.21.15.117:0/2344654998 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f7c003c10 con 0x7f5f8c1022d0 2024-04-20T11:05:11.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.175+0000 7f5f88c00640 1 -- 172.21.15.117:0/2344654998 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f7c034440 con 0x7f5f8c1022d0 2024-04-20T11:05:11.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.175+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c19d240 con 0x7f5f8c1022d0 2024-04-20T11:05:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.175+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c19d7c0 con 0x7f5f8c1022d0 2024-04-20T11:05:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.176+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c10b020 con 0x7f5f8c1022d0 2024-04-20T11:05:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.177+0000 7f5f88c00640 1 -- 172.21.15.117:0/2344654998 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f5f7c005060 con 0x7f5f8c1022d0 2024-04-20T11:05:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.178+0000 7f5f88c00640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5f7006cc50 0x7f5f7006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.178+0000 7f5f88c00640 1 -- 172.21.15.117:0/2344654998 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5f7c0ae600 con 0x7f5f8c1022d0 2024-04-20T11:05:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.179+0000 7f5f8b400640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5f7006cc50 0x7f5f7006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.179+0000 7f5f8b400640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5f7006cc50 0x7f5f7006f110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f78005fd0 tx=0x7f5f78005850 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:11.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.183+0000 7f5f88c00640 1 -- 172.21.15.117:0/2344654998 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5f7c078a50 con 0x7f5f8c1022d0 2024-04-20T11:05:11.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.372+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f5f8c10b1b0 con 0x7f5f8c1022d0 2024-04-20T11:05:11.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.373+0000 7f5f88c00640 1 -- 172.21.15.117:0/2344654998 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5f7c041020 con 0x7f5f8c1022d0 2024-04-20T11:05:11.373 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5f7006cc50 msgr2=0x7f5f7006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5f7006cc50 0x7f5f7006f110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f78005fd0 tx=0x7f5f78005850 comp rx=0 tx=0).stop 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 msgr2=0x7f5f8c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 0x7f5f8c197b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c02c220 tx=0x7f5f7c004040 comp rx=0 tx=0).stop 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 shutdown_connections 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f5f7006cc50 0x7f5f7006f110 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f8c1034c0 0x7f5f8c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 --2- 172.21.15.117:0/2344654998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f8c1022d0 0x7f5f8c197b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 >> 172.21.15.117:0/2344654998 conn(0x7f5f8c0fd840 msgr2=0x7f5f8c1066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:11.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 shutdown_connections 2024-04-20T11:05:11.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.376+0000 7f5f92c00640 1 -- 172.21.15.117:0/2344654998 wait complete. 2024-04-20T11:05:11.395 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-20T11:05:11.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.571+0000 7fa407200640 1 -- 172.21.15.117:0/1124412638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4081032b0 msgr2=0x7fa4081056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.571+0000 7fa407200640 1 --2- 172.21.15.117:0/1124412638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4081032b0 0x7fa4081056a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00098b0 tx=0x7fa3f002bcd0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.571+0000 7fa407200640 1 -- 172.21.15.117:0/1124412638 shutdown_connections 2024-04-20T11:05:11.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.571+0000 7fa407200640 1 --2- 172.21.15.117:0/1124412638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4081032b0 0x7fa4081056a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.571+0000 7fa407200640 1 --2- 172.21.15.117:0/1124412638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408100980 0x7fa408102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.571+0000 7fa407200640 1 -- 172.21.15.117:0/1124412638 >> 172.21.15.117:0/1124412638 conn(0x7fa4080fa330 msgr2=0x7fa4080fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:11.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.571+0000 7fa407200640 1 -- 172.21.15.117:0/1124412638 shutdown_connections 2024-04-20T11:05:11.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.571+0000 7fa407200640 1 -- 172.21.15.117:0/1124412638 wait complete. 2024-04-20T11:05:11.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.572+0000 7fa407200640 1 Processor -- start 2024-04-20T11:05:11.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.572+0000 7fa407200640 1 -- start start 2024-04-20T11:05:11.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.572+0000 7fa407200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408100980 0x7fa4081958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.572+0000 7fa407200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4081032b0 0x7fa408195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.572+0000 7fa407200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4081963d0 con 0x7fa408100980 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.572+0000 7fa407200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa408196540 con 0x7fa4081032b0 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa405e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408100980 0x7fa4081958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa405e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408100980 0x7fa4081958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33546/0 (socket says 172.21.15.117:33546) 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa405e00640 1 -- 172.21.15.117:0/1848021232 learned_addr learned my addr 172.21.15.117:0/1848021232 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa3ffe00640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4081032b0 0x7fa408195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa405e00640 1 -- 172.21.15.117:0/1848021232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4081032b0 msgr2=0x7fa408195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa405e00640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4081032b0 0x7fa408195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa405e00640 1 -- 172.21.15.117:0/1848021232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0009560 con 0x7fa408100980 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa3ffe00640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4081032b0 0x7fa408195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.573+0000 7fa405e00640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408100980 0x7fa4081958c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b420 tx=0x7fa3f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:11.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.574+0000 7fa3fea00640 1 -- 172.21.15.117:0/1848021232 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3f8011800 con 0x7fa408100980 2024-04-20T11:05:11.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.574+0000 7fa3fea00640 1 -- 172.21.15.117:0/1848021232 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3f8011e20 con 0x7fa408100980 2024-04-20T11:05:11.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.574+0000 7fa3fea00640 1 -- 172.21.15.117:0/1848021232 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3f800f2e0 con 0x7fa408100980 2024-04-20T11:05:11.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.574+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40819afd0 con 0x7fa408100980 2024-04-20T11:05:11.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.574+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40819b550 con 0x7fa408100980 2024-04-20T11:05:11.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.575+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa408108cf0 con 0x7fa408100980 2024-04-20T11:05:11.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.576+0000 7fa3fea00640 1 -- 172.21.15.117:0/1848021232 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fa3f8010380 con 0x7fa408100980 2024-04-20T11:05:11.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.577+0000 7fa3fea00640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa3ec06cc50 0x7fa3ec06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.577+0000 7fa3fea00640 1 -- 172.21.15.117:0/1848021232 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa3f808bbc0 con 0x7fa408100980 2024-04-20T11:05:11.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.577+0000 7fa3ffe00640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa3ec06cc50 0x7fa3ec06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:11.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.578+0000 7fa3ffe00640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa3ec06cc50 0x7fa3ec06f110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0036040 tx=0x7fa3f0005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:11.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.586+0000 7fa3fea00640 1 -- 172.21.15.117:0/1848021232 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa3f8055f70 con 0x7fa408100980 2024-04-20T11:05:11.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.775+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa40819b820 con 0x7fa408100980 2024-04-20T11:05:11.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.776+0000 7fa3fea00640 1 -- 172.21.15.117:0/1848021232 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa3f8059590 con 0x7fa408100980 2024-04-20T11:05:11.776 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa3ec06cc50 msgr2=0x7fa3ec06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa3ec06cc50 0x7fa3ec06f110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0036040 tx=0x7fa3f0005fd0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408100980 msgr2=0x7fa4081958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408100980 0x7fa4081958c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b420 tx=0x7fa3f800b8e0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 shutdown_connections 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa3ec06cc50 0x7fa3ec06f110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4081032b0 0x7fa408195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 --2- 172.21.15.117:0/1848021232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408100980 0x7fa4081958c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 >> 172.21.15.117:0/1848021232 conn(0x7fa4080fa330 msgr2=0x7fa4080fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 shutdown_connections 2024-04-20T11:05:11.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.779+0000 7fa407200640 1 -- 172.21.15.117:0/1848021232 wait complete. 2024-04-20T11:05:11.797 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:05:11.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.986+0000 7f87fd000640 1 -- 172.21.15.117:0/922436910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 msgr2=0x7f87f80fe5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.986+0000 7f87fd000640 1 --2- 172.21.15.117:0/922436910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 0x7f87f80fe5a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009880 tx=0x7f87e002bce0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.986+0000 7f87fd000640 1 -- 172.21.15.117:0/922436910 shutdown_connections 2024-04-20T11:05:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.986+0000 7f87fd000640 1 --2- 172.21.15.117:0/922436910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 0x7f87f80fe5a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.986+0000 7f87fd000640 1 --2- 172.21.15.117:0/922436910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87f8100630 0x7f87f8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.986+0000 7f87fd000640 1 -- 172.21.15.117:0/922436910 >> 172.21.15.117:0/922436910 conn(0x7f87f80f9c60 msgr2=0x7f87f80fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.986+0000 7f87fd000640 1 -- 172.21.15.117:0/922436910 shutdown_connections 2024-04-20T11:05:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.987+0000 7f87fd000640 1 -- 172.21.15.117:0/922436910 wait complete. 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.987+0000 7f87fd000640 1 Processor -- start 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87fd000640 1 -- start start 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87fd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 0x7f87f8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87fd000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87f8100630 0x7f87f8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87fd000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f81985e0 con 0x7f87f80fe120 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87fd000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8198750 con 0x7f87f8100630 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87f5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 0x7f87f8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87f5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 0x7f87f8197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33554/0 (socket says 172.21.15.117:33554) 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87f5e00640 1 -- 172.21.15.117:0/2258098253 learned_addr learned my addr 172.21.15.117:0/2258098253 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87efe00640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87f8100630 0x7f87f8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87f5e00640 1 -- 172.21.15.117:0/2258098253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87f8100630 msgr2=0x7f87f8198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.988+0000 7f87f5e00640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87f8100630 0x7f87f8198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.989+0000 7f87f5e00640 1 -- 172.21.15.117:0/2258098253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e0009560 con 0x7f87f80fe120 2024-04-20T11:05:11.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.989+0000 7f87efe00640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87f8100630 0x7f87f8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:05:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.989+0000 7f87f5e00640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 0x7f87f8197ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d6f0 tx=0x7f87e800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.989+0000 7f87eea00640 1 -- 172.21.15.117:0/2258098253 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e8009a30 con 0x7f87f80fe120 2024-04-20T11:05:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.989+0000 7f87eea00640 1 -- 172.21.15.117:0/2258098253 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87e80054e0 con 0x7f87f80fe120 2024-04-20T11:05:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.989+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f819d1e0 con 0x7f87f80fe120 2024-04-20T11:05:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.989+0000 7f87eea00640 1 -- 172.21.15.117:0/2258098253 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e8010460 con 0x7f87f80fe120 2024-04-20T11:05:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.989+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f819d7c0 con 0x7f87f80fe120 2024-04-20T11:05:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.990+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f810b0d0 con 0x7f87f80fe120 2024-04-20T11:05:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.991+0000 7f87eea00640 1 -- 172.21.15.117:0/2258098253 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f87e8009b90 con 0x7f87f80fe120 2024-04-20T11:05:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.992+0000 7f87eea00640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f87dc06cd10 0x7f87dc06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.992+0000 7f87eea00640 1 -- 172.21.15.117:0/2258098253 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f87e808bdb0 con 0x7f87f80fe120 2024-04-20T11:05:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.993+0000 7f87efe00640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f87dc06cd10 0x7f87dc06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.993+0000 7f87efe00640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f87dc06cd10 0x7f87dc06f1d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87e0036040 tx=0x7f87e002cf90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:11.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:11.996+0000 7f87eea00640 1 -- 172.21.15.117:0/2258098253 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f87e8056160 con 0x7f87f80fe120 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3127380558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2344654998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1848021232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:05:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:11 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3127380558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:05:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2344654998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-20T11:05:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1848021232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-20T11:05:12.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:05:12.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:05:12.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:11 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:05:12.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.245+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87f819da30 con 0x7f87f80fe120 2024-04-20T11:05:12.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.247+0000 7f87eea00640 1 -- 172.21.15.117:0/2258098253 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4841 (secure 0 0 0) 0x7f87e8020090 con 0x7f87f80fe120 2024-04-20T11:05:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:05:12.247 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6829/1089389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1089389303},{"type":"v1","addr":"172.21.15.117:6829","nonce":1089389303}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:04:55.549816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14460,"mds_1":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14460":{"gid":14460,"name":"cephfs.smithi117.umapup","rank":0,"incarnation":3,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6827/361271135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":361271135},{"type":"v1","addr":"172.21.15.117:6827","nonce":361271135}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24253":{"gid":24253,"name":"cephfs.smithi173.pybzht","rank":1,"incarnation":6,"state":"up:active","state_seq":5,"addr":"172.21.15.173:6825/3563933435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3563933435},{"type":"v1","addr":"172.21.15.173:6825","nonce":3563933435}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-20T11:05:12.247 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.249+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f87dc06cd10 msgr2=0x7f87dc06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.249+0000 7f87fd000640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f87dc06cd10 0x7f87dc06f1d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87e0036040 tx=0x7f87e002cf90 comp rx=0 tx=0).stop 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.249+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 msgr2=0x7f87f8197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.249+0000 7f87fd000640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 0x7f87f8197ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d6f0 tx=0x7f87e800dbb0 comp rx=0 tx=0).stop 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.250+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 shutdown_connections 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.250+0000 7f87fd000640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f87dc06cd10 0x7f87dc06f1d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.250+0000 7f87fd000640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87f8100630 0x7f87f8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.250+0000 7f87fd000640 1 --2- 172.21.15.117:0/2258098253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87f80fe120 0x7f87f8197ad0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.250+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 >> 172.21.15.117:0/2258098253 conn(0x7f87f80f9c60 msgr2=0x7f87f8107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.250+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 shutdown_connections 2024-04-20T11:05:12.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:12.250+0000 7f87fd000640 1 -- 172.21.15.117:0/2258098253 wait complete. 2024-04-20T11:05:12.268 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 12, 'max_mds': 2, 'flags': 18} 2024-04-20T11:05:12.268 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-20T11:05:12.294 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-20T11:05:12.294 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-20T11:05:12.294 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-20T11:05:12.294 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T11:05:12.294 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:12.294 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-20T11:05:12.294 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T11:05:12.294 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:12.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:12.295 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-04-20T11:05:12.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:12.348 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-04-20T11:05:12.432 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-04-20T11:05:12.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:05:12.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:12.437 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2024-04-20T11:05:12.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:12.496 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2024-04-20T11:05:12.586 INFO:teuthology.orchestra.run.smithi173.stderr:Cannot find device "ceph-brx" 2024-04-20T11:05:12.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:05:12.591 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-20T11:05:12.591 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph fs ls 2024-04-20T11:05:13.014 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:05:13.200 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:12 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2258098253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:05:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:12 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2258098253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:05:14.211 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:13 smithi117 ceph-mon[15659]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:14.211 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:05:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:13 smithi173 ceph-mon[23800]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:13 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:05:15.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.192+0000 7f879aa00700 1 -- 172.21.15.117:0/2151166321 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87880054b0 con 0x7f879c107df0 2024-04-20T11:05:15.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.192+0000 7f87a2e00700 1 -- 172.21.15.117:0/2151166321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 msgr2=0x7f879c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:15.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.192+0000 7f87a2e00700 1 --2- 172.21.15.117:0/2151166321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 0x7f879c10be40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8788009910 tx=0x7f8788009c20 comp rx=0 tx=0).stop 2024-04-20T11:05:15.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.192+0000 7f87a2e00700 1 -- 172.21.15.117:0/2151166321 shutdown_connections 2024-04-20T11:05:15.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.192+0000 7f87a2e00700 1 --2- 172.21.15.117:0/2151166321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 0x7f879c10be40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:15.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.192+0000 7f87a2e00700 1 --2- 172.21.15.117:0/2151166321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f879c107440 0x7f879c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:15.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.192+0000 7f87a2e00700 1 -- 172.21.15.117:0/2151166321 >> 172.21.15.117:0/2151166321 conn(0x7f879c076a10 msgr2=0x7f879c078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:15.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.193+0000 7f87a2e00700 1 -- 172.21.15.117:0/2151166321 shutdown_connections 2024-04-20T11:05:15.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.193+0000 7f87a2e00700 1 -- 172.21.15.117:0/2151166321 wait complete. 2024-04-20T11:05:15.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.194+0000 7f87a2e00700 1 Processor -- start 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.194+0000 7f87a2e00700 1 -- start start 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.194+0000 7f87a2e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f879c107440 0x7f879c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.194+0000 7f87a2e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 0x7f879c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.194+0000 7f87a2e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c19ddf0 con 0x7f879c107df0 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.194+0000 7f87a2e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c1a1ac0 con 0x7f879c107440 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.195+0000 7f879b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 0x7f879c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.195+0000 7f879b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 0x7f879c19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52118/0 (socket says 172.21.15.117:52118) 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.195+0000 7f879b400700 1 -- 172.21.15.117:0/3308579402 learned_addr learned my addr 172.21.15.117:0/3308579402 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.195+0000 7f879be00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f879c107440 0x7f879c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.195+0000 7f879b400700 1 -- 172.21.15.117:0/3308579402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f879c107440 msgr2=0x7f879c19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.195+0000 7f879b400700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f879c107440 0x7f879c19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:15.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.195+0000 7f879b400700 1 -- 172.21.15.117:0/3308579402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87880095f0 con 0x7f879c107df0 2024-04-20T11:05:15.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.195+0000 7f879b400700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 0x7f879c19d710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f878800bd70 tx=0x7f878800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:15.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.196+0000 7f8798c00700 1 -- 172.21.15.117:0/3308579402 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878801d070 con 0x7f879c107df0 2024-04-20T11:05:15.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.196+0000 7f8798c00700 1 -- 172.21.15.117:0/3308579402 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f878800f460 con 0x7f879c107df0 2024-04-20T11:05:15.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.196+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c1a1d40 con 0x7f879c107df0 2024-04-20T11:05:15.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.196+0000 7f8798c00700 1 -- 172.21.15.117:0/3308579402 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8788017630 con 0x7f879c107df0 2024-04-20T11:05:15.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.196+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c1a2230 con 0x7f879c107df0 2024-04-20T11:05:15.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.198+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c10f790 con 0x7f879c107df0 2024-04-20T11:05:15.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.199+0000 7f8798c00700 1 -- 172.21.15.117:0/3308579402 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f878800f5d0 con 0x7f879c107df0 2024-04-20T11:05:15.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.200+0000 7f8798c00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878006cce0 0x7f878006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:15.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.200+0000 7f8798c00700 1 -- 172.21.15.117:0/3308579402 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f878808d740 con 0x7f879c107df0 2024-04-20T11:05:15.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.200+0000 7f879be00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878006cce0 0x7f878006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:15.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.201+0000 7f879be00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878006cce0 0x7f878006f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f878c006fd0 tx=0x7f878c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:15.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.205+0000 7f8798c00700 1 -- 172.21.15.117:0/3308579402 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8788057a70 con 0x7f879c107df0 2024-04-20T11:05:15.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.470+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f879c19e4e0 con 0x7f879c107df0 2024-04-20T11:05:15.471 INFO:teuthology.orchestra.run.smithi117.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-20T11:05:15.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.471+0000 7f8798c00700 1 -- 172.21.15.117:0/3308579402 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8788005740 con 0x7f879c107df0 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878006cce0 msgr2=0x7f878006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878006cce0 0x7f878006f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f878c006fd0 tx=0x7f878c008040 comp rx=0 tx=0).stop 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 msgr2=0x7f879c19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 0x7f879c19d710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f878800bd70 tx=0x7f878800be50 comp rx=0 tx=0).stop 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 shutdown_connections 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878006cce0 0x7f878006f1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f879c107440 0x7f879c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 --2- 172.21.15.117:0/3308579402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879c107df0 0x7f879c19d710 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 >> 172.21.15.117:0/3308579402 conn(0x7f879c076a10 msgr2=0x7f879c077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 shutdown_connections 2024-04-20T11:05:15.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:15.474+0000 7f87a2e00700 1 -- 172.21.15.117:0/3308579402 wait complete. 2024-04-20T11:05:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:15 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3308579402' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T11:05:15.860 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:15 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3308579402' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T11:05:16.216 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:05:16.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:05:16.217 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-04-20T11:05:16.217 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:05:16.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:16.217 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:05:16.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:05:16.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:05:16.217 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:05:16.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:16.217 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-04-20T11:05:16.274 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42256sec preferred_lft 42256sec 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-04-20T11:05:16.275 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:05:16.276 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T11:05:16.276 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:16.276 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-20T11:05:16.276 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-04-20T11:05:16.276 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-04-20T11:05:16.277 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-04-20T11:05:16.277 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T11:05:16.277 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-20T11:05:16.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:16.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:16.785 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:16 smithi117 ceph-mon[15659]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:16.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:16.787 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:05:16.855 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-04-20T11:05:16.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:16.858 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-04-20T11:05:16.910 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-04-20T11:05:16.910 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:05:16.910 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:05:16.910 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:05:16.910 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:05:16.911 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:16.912 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-20T11:05:16.912 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:05:16.912 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:05:16.912 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:05:16.912 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-20T11:05:16.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:16 smithi173 ceph-mon[23800]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:17.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:17.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:17.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:17.645 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-04-20T11:05:17.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:17.688 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-04-20T11:05:17.775 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:17.775 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-20T11:05:17.776 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:05:17.776 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T11:05:17.776 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-20T11:05:17.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:17.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:17.982 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T11:05:17.983 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:17.983 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-20T11:05:17.983 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T11:05:17.983 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T11:05:17.983 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T11:05:17.983 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T11:05:17.983 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T11:05:17.983 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-20T11:05:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:18.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:18.265 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:18.266 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-20T11:05:18.266 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-04-20T11:05:18.266 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T11:05:18.266 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-20T11:05:18.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:18.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:18.457 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T11:05:18.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:05:18.457 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:18.500 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:05:18.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:05:18.501 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:18.591 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-04-20T11:05:18.666 DEBUG:teuthology.orchestra.run.smithi117:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:05:18.712 INFO:teuthology.orchestra.run.smithi117.stdout:/proc 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/sys 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/dev 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/security 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/shm 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/pts 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/run 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/cgroup 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/pstore 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/bpf 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/ 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/selinux 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/hugepages 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/mqueue 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/debug 2024-04-20T11:05:18.713 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/tracing 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/fuse/connections 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/config 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/1000 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/ceph 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/02c914796dfeb7b6a287c5af591515dc05a666fba10ea2a94a390b5e5d752bb2/merged 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/ba58abd7816dbc28262b916f7cb36fe0c864d2b2856e5add89971d617c0038e4/merged 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/0 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:18.714 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2830e983bb04968ecf3ab82ee5972516fbe1a8bd92f6d761e646efb87658b0f6/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/e4a040b7fe57746ba640ad9a528ede553e9714590059d90ef1f08eb2d825a499/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/460ce59a8cd85c566400930168f5ba4a8e5bbf204fd23a1ed79c3fc3d782511e/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2aed274db816cfdfe562f9b134fdd98ed75a5cb01493251862ca80017af3c347/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/30d949f5954ad3b9185359115c3b3eb188c37693daf0640b7318a2de6d80d4c7/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/bf2708e9bd3a4a9d31beb9e6a84ce46ea603c52edf9ff89d68402238199aa0a7/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/60e2013524658711690b29599f7cede1c300caaea09fc4742737a3af38a8796d/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/da9be9e6de96e43501ecf047bf302eff2552afc1817b73f4e9451ddb128351a2/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/c34606d665d4d017b9af3a579181aa727c56fa5d50a74989e040340f6c9e05bd/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/62de653a36fb533a3986c8d0954c0d0a81cc8ae7a420def9c9db7ceb540fd326/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/0936a12f6182b6ad29578973d59733f4f25e744cf079f0a3f061806778fab5eb/merged 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:05:18.715 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:05:18.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:18.716 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-04-20T11:05:18.763 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:05:18.764 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T11:05:18.768 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-04-20T11:05:18.831 DEBUG:teuthology.orchestra.run.smithi117:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/proc 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/sys 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/dev 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/security 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/shm 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/pts 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/run 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/cgroup 2024-04-20T11:05:18.873 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/pstore 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/bpf 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/ 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/selinux 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/hugepages 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/mqueue 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/debug 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/tracing 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/fuse/connections 2024-04-20T11:05:18.874 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/config 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/1000 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/ceph 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/02c914796dfeb7b6a287c5af591515dc05a666fba10ea2a94a390b5e5d752bb2/merged 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/ba58abd7816dbc28262b916f7cb36fe0c864d2b2856e5add89971d617c0038e4/merged 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/0 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2830e983bb04968ecf3ab82ee5972516fbe1a8bd92f6d761e646efb87658b0f6/merged 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/e4a040b7fe57746ba640ad9a528ede553e9714590059d90ef1f08eb2d825a499/merged 2024-04-20T11:05:18.875 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/460ce59a8cd85c566400930168f5ba4a8e5bbf204fd23a1ed79c3fc3d782511e/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2aed274db816cfdfe562f9b134fdd98ed75a5cb01493251862ca80017af3c347/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/30d949f5954ad3b9185359115c3b3eb188c37693daf0640b7318a2de6d80d4c7/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/bf2708e9bd3a4a9d31beb9e6a84ce46ea603c52edf9ff89d68402238199aa0a7/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/60e2013524658711690b29599f7cede1c300caaea09fc4742737a3af38a8796d/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/da9be9e6de96e43501ecf047bf302eff2552afc1817b73f4e9451ddb128351a2/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/c34606d665d4d017b9af3a579181aa727c56fa5d50a74989e040340f6c9e05bd/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/62de653a36fb533a3986c8d0954c0d0a81cc8ae7a420def9c9db7ceb540fd326/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/0936a12f6182b6ad29578973d59733f4f25e744cf079f0a3f061806778fab5eb/merged 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:05:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:05:18.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:18.878 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-04-20T11:05:18.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:18 smithi117 ceph-mon[15659]: pgmap v247: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:18 smithi173 ceph-mon[23800]: pgmap v247: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:18.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:2024-04-20T11:05:18.945+0000 7f7b20f96480 -1 init, newargv = 0x558b3fe2fda0 newargc=15 2024-04-20T11:05:18.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[63413]: starting ceph client 2024-04-20T11:05:18.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[63413]: starting fuse 2024-04-20T11:05:20.028 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-04-20T11:05:20.119 DEBUG:teuthology.orchestra.run.smithi117:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:05:20.177 INFO:teuthology.orchestra.run.smithi117.stdout:/proc 2024-04-20T11:05:20.177 INFO:teuthology.orchestra.run.smithi117.stdout:/sys 2024-04-20T11:05:20.177 INFO:teuthology.orchestra.run.smithi117.stdout:/dev 2024-04-20T11:05:20.177 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/security 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/shm 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/pts 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/run 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/cgroup 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/pstore 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/bpf 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/ 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/selinux 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/hugepages 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/mqueue 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/debug 2024-04-20T11:05:20.178 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/tracing 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/fuse/connections 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/config 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/1000 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/ceph 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/02c914796dfeb7b6a287c5af591515dc05a666fba10ea2a94a390b5e5d752bb2/merged 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/ba58abd7816dbc28262b916f7cb36fe0c864d2b2856e5add89971d617c0038e4/merged 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/0 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2830e983bb04968ecf3ab82ee5972516fbe1a8bd92f6d761e646efb87658b0f6/merged 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/e4a040b7fe57746ba640ad9a528ede553e9714590059d90ef1f08eb2d825a499/merged 2024-04-20T11:05:20.179 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/460ce59a8cd85c566400930168f5ba4a8e5bbf204fd23a1ed79c3fc3d782511e/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2aed274db816cfdfe562f9b134fdd98ed75a5cb01493251862ca80017af3c347/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/30d949f5954ad3b9185359115c3b3eb188c37693daf0640b7318a2de6d80d4c7/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/bf2708e9bd3a4a9d31beb9e6a84ce46ea603c52edf9ff89d68402238199aa0a7/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/60e2013524658711690b29599f7cede1c300caaea09fc4742737a3af38a8796d/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/da9be9e6de96e43501ecf047bf302eff2552afc1817b73f4e9451ddb128351a2/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/c34606d665d4d017b9af3a579181aa727c56fa5d50a74989e040340f6c9e05bd/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/62de653a36fb533a3986c8d0954c0d0a81cc8ae7a420def9c9db7ceb540fd326/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/0936a12f6182b6ad29578973d59733f4f25e744cf079f0a3f061806778fab5eb/merged 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:05:20.180 INFO:teuthology.orchestra.run.smithi117.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:20.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:20.181 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-04-20T11:05:20.234 INFO:teuthology.orchestra.run.smithi117.stdout:155 2024-04-20T11:05:20.234 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [155] 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> import glob 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> import re 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> import os 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> import subprocess 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> def _find_admin_socket(client_name): 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2024-04-20T11:05:20.235 DEBUG:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> return files[0] 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> for f in files: 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> contents = proc_f.read() 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> if mountpoint in contents: 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> return f 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> print(_find_admin_socket("client.0")) 2024-04-20T11:05:20.236 DEBUG:teuthology.orchestra.run.smithi117:> ' 2024-04-20T11:05:20.400 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.0.63413.asok 2024-04-20T11:05:20.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:20.419 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.63413.asok 2024-04-20T11:05:20.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:20.419 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.63413.asok status 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "9d98999acebbad2663cc4938e94c9ffd83485cbd", 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 18.2.2 (9d98999acebbad2663cc4938e94c9ffd83485cbd) reef (stable)", 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "0", 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "63413", 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 14524, 2024-04-20T11:05:20.649 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 14524 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 846389270 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:05:20.650 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 846389270 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.14524 192.168.0.1:0/846389270", 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "192.168.0.1:0/846389270", 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 12, 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 47, 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: "blocklisted": false, 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs" 2024-04-20T11:05:20.651 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:05:20.664 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- ceph fs ls 2024-04-20T11:05:20.731 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:20 smithi117 ceph-mon[15659]: pgmap v248: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:20 smithi173 ceph-mon[23800]: pgmap v248: 65 pgs: 65 active+clean; 580 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:05:21.061 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:05:22.819 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:22 smithi117 ceph-mon[15659]: pgmap v249: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-20T11:05:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:22 smithi173 ceph-mon[23800]: pgmap v249: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-20T11:05:23.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.300+0000 7fa5c6400700 1 -- 172.21.15.117:0/609936172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 msgr2=0x7fa5c0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:23.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.300+0000 7fa5c6400700 1 --2- 172.21.15.117:0/609936172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 0x7fa5c0104e80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009910 tx=0x7fa5ac009c20 comp rx=0 tx=0).stop 2024-04-20T11:05:23.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.300+0000 7fa5c6400700 1 -- 172.21.15.117:0/609936172 shutdown_connections 2024-04-20T11:05:23.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.300+0000 7fa5c6400700 1 --2- 172.21.15.117:0/609936172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 0x7fa5c0104e80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:23.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.300+0000 7fa5c6400700 1 --2- 172.21.15.117:0/609936172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c0103110 0x7fa5c01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:23.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.300+0000 7fa5c6400700 1 -- 172.21.15.117:0/609936172 >> 172.21.15.117:0/609936172 conn(0x7fa5c00fe880 msgr2=0x7fa5c0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:23.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.301+0000 7fa5c6400700 1 -- 172.21.15.117:0/609936172 shutdown_connections 2024-04-20T11:05:23.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.301+0000 7fa5c6400700 1 -- 172.21.15.117:0/609936172 wait complete. 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5c6400700 1 Processor -- start 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5c6400700 1 -- start start 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5c6400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c0103110 0x7fa5c019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5c6400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 0x7fa5c019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5c6400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c019f520 con 0x7fa5c0104a00 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5c6400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0198b90 con 0x7fa5c0103110 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 0x7fa5c019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 0x7fa5c019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52134/0 (socket says 172.21.15.117:52134) 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.302+0000 7fa5bea00700 1 -- 172.21.15.117:0/968099945 learned_addr learned my addr 172.21.15.117:0/968099945 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.303+0000 7fa5bf400700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c0103110 0x7fa5c019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.303+0000 7fa5bea00700 1 -- 172.21.15.117:0/968099945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c0103110 msgr2=0x7fa5c019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:23.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.303+0000 7fa5bea00700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c0103110 0x7fa5c019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:23.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.303+0000 7fa5bea00700 1 -- 172.21.15.117:0/968099945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac0095f0 con 0x7fa5c0104a00 2024-04-20T11:05:23.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.303+0000 7fa5bea00700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 0x7fa5c019ef90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00bd70 tx=0x7fa5ac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:23.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.303+0000 7fa5b7e00700 1 -- 172.21.15.117:0/968099945 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5ac01d070 con 0x7fa5c0104a00 2024-04-20T11:05:23.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.304+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c0198e10 con 0x7fa5c0104a00 2024-04-20T11:05:23.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.304+0000 7fa5b7e00700 1 -- 172.21.15.117:0/968099945 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5ac00f460 con 0x7fa5c0104a00 2024-04-20T11:05:23.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.304+0000 7fa5b7e00700 1 -- 172.21.15.117:0/968099945 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5ac017630 con 0x7fa5c0104a00 2024-04-20T11:05:23.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.304+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c0199380 con 0x7fa5c0104a00 2024-04-20T11:05:23.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.305+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c010b480 con 0x7fa5c0104a00 2024-04-20T11:05:23.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.306+0000 7fa5b7e00700 1 -- 172.21.15.117:0/968099945 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fa5ac00f5d0 con 0x7fa5c0104a00 2024-04-20T11:05:23.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.307+0000 7fa5b7e00700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa5a406cce0 0x7fa5a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:23.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.307+0000 7fa5b7e00700 1 -- 172.21.15.117:0/968099945 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa5ac08d770 con 0x7fa5c0104a00 2024-04-20T11:05:23.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.307+0000 7fa5bf400700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa5a406cce0 0x7fa5a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:23.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.308+0000 7fa5bf400700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa5a406cce0 0x7fa5a406f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0005ea0 tx=0x7fa5b0005e30 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:23.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.312+0000 7fa5b7e00700 1 -- 172.21.15.117:0/968099945 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa5ac05b5b0 con 0x7fa5c0104a00 2024-04-20T11:05:23.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.581+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5c0104510 con 0x7fa5c0104a00 2024-04-20T11:05:23.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.582+0000 7fa5b7e00700 1 -- 172.21.15.117:0/968099945 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+83 (secure 0 0 0) 0x7fa5ac05b140 con 0x7fa5c0104a00 2024-04-20T11:05:23.583 INFO:teuthology.orchestra.run.smithi117.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa5a406cce0 msgr2=0x7fa5a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa5a406cce0 0x7fa5a406f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0005ea0 tx=0x7fa5b0005e30 comp rx=0 tx=0).stop 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 msgr2=0x7fa5c019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 0x7fa5c019ef90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00bd70 tx=0x7fa5ac00be50 comp rx=0 tx=0).stop 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 shutdown_connections 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa5a406cce0 0x7fa5a406f1a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c0103110 0x7fa5c019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 --2- 172.21.15.117:0/968099945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5c0104a00 0x7fa5c019ef90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:23.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 >> 172.21.15.117:0/968099945 conn(0x7fa5c00fe880 msgr2=0x7fa5c00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:23.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.585+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 shutdown_connections 2024-04-20T11:05:23.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:23.586+0000 7fa5c6400700 1 -- 172.21.15.117:0/968099945 wait complete. 2024-04-20T11:05:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:23 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/968099945' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T11:05:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:23 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/968099945' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:05:24.481 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:05:24.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:24.482 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2024-04-20T11:05:24.539 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:05:24.539 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:05:24.539 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T11:05:24.540 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42245sec preferred_lft 42245sec 2024-04-20T11:05:24.541 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2024-04-20T11:05:24.541 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:05:24.541 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T11:05:24.542 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:24.542 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-04-20T11:05:24.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link add name ceph-brx type bridge 2024-04-20T11:05:24.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip addr flush dev ceph-brx 2024-04-20T11:05:24.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set ceph-brx up 2024-04-20T11:05:24.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T11:05:24.542 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-04-20T11:05:24.643 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:24.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:24 smithi117 ceph-mon[15659]: pgmap v250: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:05:24.898 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:24 smithi173 ceph-mon[23800]: pgmap v250: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:05:24.996 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:25.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:25.005 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:05:25.069 INFO:teuthology.orchestra.run.smithi173.stdout:1 2024-04-20T11:05:25.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:25.073 DEBUG:teuthology.orchestra.run.smithi173:> route 2024-04-20T11:05:25.123 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2024-04-20T11:05:25.123 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:05:25.123 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T11:05:25.123 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T11:05:25.123 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:05:25.124 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:25.124 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-04-20T11:05:25.124 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T11:05:25.124 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T11:05:25.125 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T11:05:25.125 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-04-20T11:05:25.219 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:25.832 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:25.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:25.842 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2024-04-20T11:05:25.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:25.889 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list-id 2024-04-20T11:05:25.972 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:25.972 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-04-20T11:05:25.972 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:05:25.972 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-20T11:05:25.972 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-04-20T11:05:26.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:26.172 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:26.181 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-20T11:05:26.181 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:26.181 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-04-20T11:05:26.181 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-20T11:05:26.181 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T11:05:26.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-20T11:05:26.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-20T11:05:26.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-20T11:05:26.182 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-04-20T11:05:26.273 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:26.457 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:26.465 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:05:26.465 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-04-20T11:05:26.465 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2024-04-20T11:05:26.465 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T11:05:26.466 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-04-20T11:05:26.552 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:05:26.646 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:26.655 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T11:05:26.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:05:26.655 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:26.696 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T11:05:26.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:05:26.696 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:26.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2024-04-20T11:05:26.863 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:05:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:26 smithi173 ceph-mon[23800]: pgmap v251: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:05:26.912 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2024-04-20T11:05:26.912 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2024-04-20T11:05:26.912 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2024-04-20T11:05:26.913 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2024-04-20T11:05:26.913 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2024-04-20T11:05:26.913 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2024-04-20T11:05:26.913 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2024-04-20T11:05:26.913 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2024-04-20T11:05:26.913 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2024-04-20T11:05:26.913 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/bpf 2024-04-20T11:05:26.913 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:05:26.914 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/0 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/97949c1ab9079f2c3adb75ccd0e1ef27b75783113bcc57f32a71513cfdaa9385/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/976e253a991892d6e2bc54191c5a05276d0e96119af28e92ff9f74f4441acc7a/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/c52402defc7fcec51101b76acf201cce35e71bc53fa45875760179ef299ffde4/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/c7c79f24f27c07b1dad08ec8a7f32557021e2d86d603f2ceccb85536c83fc260/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/2e80b60e3c6eba0f8f971dac6673cfa374cf3c8be81bf395b8553146f23ee381/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/335013f88bd803e9cf7b59801c9dd5782acb8b572dbc61dcac25799dfc26d624/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/b56ca6a94a33c99c7639585c87c47847474032ab67843b8b4b0183b99db2a642/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/ed758308b0993a78eb814c2aa8da5949ae6e78a2bf39bd4a7085147b7e0b0f41/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/03dc7b8dd45cb6c56318330e8c8013bb9aac60e1842583c9cf6ddb9116607db5/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/817b7007efd9efd828cbae65d34b8ba7c051abd0c6eb837ef85a17747a877c10/merged 2024-04-20T11:05:26.915 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2024-04-20T11:05:26.916 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:05:26.916 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:05:26.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:26.917 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2024-04-20T11:05:26.963 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T11:05:26.963 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T11:05:26.970 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2024-04-20T11:05:27.040 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/bpf 2024-04-20T11:05:27.079 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2024-04-20T11:05:27.080 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/0 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/97949c1ab9079f2c3adb75ccd0e1ef27b75783113bcc57f32a71513cfdaa9385/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/976e253a991892d6e2bc54191c5a05276d0e96119af28e92ff9f74f4441acc7a/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/c52402defc7fcec51101b76acf201cce35e71bc53fa45875760179ef299ffde4/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/c7c79f24f27c07b1dad08ec8a7f32557021e2d86d603f2ceccb85536c83fc260/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/2e80b60e3c6eba0f8f971dac6673cfa374cf3c8be81bf395b8553146f23ee381/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/335013f88bd803e9cf7b59801c9dd5782acb8b572dbc61dcac25799dfc26d624/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/b56ca6a94a33c99c7639585c87c47847474032ab67843b8b4b0183b99db2a642/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/ed758308b0993a78eb814c2aa8da5949ae6e78a2bf39bd4a7085147b7e0b0f41/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/03dc7b8dd45cb6c56318330e8c8013bb9aac60e1842583c9cf6ddb9116607db5/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/817b7007efd9efd828cbae65d34b8ba7c051abd0c6eb837ef85a17747a877c10/merged 2024-04-20T11:05:27.081 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2024-04-20T11:05:27.082 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:05:27.082 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:05:27.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:27.083 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2024-04-20T11:05:27.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:26 smithi117 ceph-mon[15659]: pgmap v251: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:05:27.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:2024-04-20T11:05:27.154+0000 7fe827b60480 -1 init, newargv = 0x563942d19580 newargc=15 2024-04-20T11:05:27.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[49898]: starting ceph client 2024-04-20T11:05:27.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[49898]: starting fuse 2024-04-20T11:05:28.271 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2024-04-20T11:05:28.361 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/bpf 2024-04-20T11:05:28.416 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2024-04-20T11:05:28.417 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/0 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/97949c1ab9079f2c3adb75ccd0e1ef27b75783113bcc57f32a71513cfdaa9385/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/976e253a991892d6e2bc54191c5a05276d0e96119af28e92ff9f74f4441acc7a/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/c52402defc7fcec51101b76acf201cce35e71bc53fa45875760179ef299ffde4/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/c7c79f24f27c07b1dad08ec8a7f32557021e2d86d603f2ceccb85536c83fc260/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/2e80b60e3c6eba0f8f971dac6673cfa374cf3c8be81bf395b8553146f23ee381/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/335013f88bd803e9cf7b59801c9dd5782acb8b572dbc61dcac25799dfc26d624/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/b56ca6a94a33c99c7639585c87c47847474032ab67843b8b4b0183b99db2a642/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/ed758308b0993a78eb814c2aa8da5949ae6e78a2bf39bd4a7085147b7e0b0f41/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/03dc7b8dd45cb6c56318330e8c8013bb9aac60e1842583c9cf6ddb9116607db5/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/containers/storage/overlay/817b7007efd9efd828cbae65d34b8ba7c051abd0c6eb837ef85a17747a877c10/merged 2024-04-20T11:05:28.418 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2024-04-20T11:05:28.419 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:05:28.419 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T11:05:28.419 INFO:teuthology.orchestra.run.smithi173.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:28.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:28.420 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2024-04-20T11:05:28.472 INFO:teuthology.orchestra.run.smithi173.stdout:122 2024-04-20T11:05:28.473 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [122] 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> sudo stdin-killer -- python3 -c ' 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> import glob 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> import re 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> import os 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> import subprocess 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> def _find_admin_socket(client_name): 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> files = glob.glob(asok_path) 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> 2024-04-20T11:05:28.473 DEBUG:teuthology.orchestra.run.smithi173:> # Given a non-glob path, it better be there 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> if "*" not in asok_path: 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> assert(len(files) == 1) 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> return files[0] 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> for f in files: 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> contents = proc_f.read() 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> if mountpoint in contents: 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> return f 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> print(_find_admin_socket("client.1")) 2024-04-20T11:05:28.474 DEBUG:teuthology.orchestra.run.smithi173:> ' 2024-04-20T11:05:28.631 INFO:teuthology.orchestra.run.smithi173.stdout:/var/run/ceph/ceph-client.1.49898.asok 2024-04-20T11:05:28.635 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-20T11:05:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:05:28.648 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.49898.asok 2024-04-20T11:05:28.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:28.649 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.49898.asok status 2024-04-20T11:05:28.878 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-20T11:05:28.878 INFO:teuthology.orchestra.run.smithi173.stdout: "metadata": { 2024-04-20T11:05:28.878 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_sha1": "9d98999acebbad2663cc4938e94c9ffd83485cbd", 2024-04-20T11:05:28.878 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.2.2 (9d98999acebbad2663cc4938e94c9ffd83485cbd) reef (stable)", 2024-04-20T11:05:28.878 INFO:teuthology.orchestra.run.smithi173.stdout: "entity_id": "1", 2024-04-20T11:05:28.878 INFO:teuthology.orchestra.run.smithi173.stdout: "hostname": "smithi173", 2024-04-20T11:05:28.878 INFO:teuthology.orchestra.run.smithi173.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T11:05:28.878 INFO:teuthology.orchestra.run.smithi173.stdout: "pid": "49898", 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "root": "/" 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_count": 0, 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_pinned_count": 0, 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "id": 24295, 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "inst": { 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "name": { 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "client", 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "num": 24295 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 272380713 2024-04-20T11:05:28.879 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 272380713 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "inst_str": "client.24295 192.168.0.1:0/272380713", 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "addr_str": "192.168.0.1:0/272380713", 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "inode_count": 1, 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_epoch": 12, 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch": 47, 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch_barrier": 0, 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "blocklisted": false, 2024-04-20T11:05:28.880 INFO:teuthology.orchestra.run.smithi173.stdout: "fs_name": "cephfs" 2024-04-20T11:05:28.881 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-20T11:05:28.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:28.893 DEBUG:teuthology.orchestra.run.smithi117:> stat --file-system '--printf=%T 2024-04-20T11:05:28.893 DEBUG:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:28 smithi173 ceph-mon[23800]: pgmap v252: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:05:28.920 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:28 smithi117 ceph-mon[15659]: pgmap v252: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:05:28.946 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2024-04-20T11:05:28.946 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:28.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:28.947 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:29.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:29.087 DEBUG:teuthology.orchestra.run.smithi173:> stat --file-system '--printf=%T 2024-04-20T11:05:29.087 DEBUG:teuthology.orchestra.run.smithi173:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:29.147 INFO:teuthology.orchestra.run.smithi173.stdout:fuseblk 2024-04-20T11:05:29.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:29.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:05:29.148 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:05:30.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:05:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:30 smithi117 ceph-mon[15659]: pgmap v253: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:05:31.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:30 smithi173 ceph-mon[23800]: pgmap v253: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-20T11:05:33.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:32 smithi117 ceph-mon[15659]: pgmap v254: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-20T11:05:33.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:32 smithi173 ceph-mon[23800]: pgmap v254: 65 pgs: 65 active+clean; 581 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-20T11:05:33.233 INFO:teuthology.run_tasks:Running task print... 2024-04-20T11:05:33.243 INFO:teuthology.task.print:**** done client 2024-04-20T11:05:33.243 INFO:teuthology.run_tasks:Running task parallel... 2024-04-20T11:05:33.290 INFO:teuthology.task.parallel:starting parallel... 2024-04-20T11:05:33.291 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-20T11:05:33.291 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-20T11:05:33.291 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:05:33.292 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-20T11:05:33.292 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-20T11:05:33.292 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-20T11:05:33.296 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:05:33.296 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-20T11:05:33.296 INFO:tasks.workunit:timeout=3h 2024-04-20T11:05:33.296 INFO:tasks.workunit:cleanup=True 2024-04-20T11:05:33.296 DEBUG:teuthology.orchestra.run.smithi117:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 9bh/155d Inode: 1 Links: 2 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-20 11:04:38.752531240 +0000 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-20 11:05:29.078997300 +0000 2024-04-20T11:05:33.351 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-04-20T11:05:33.352 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-20T11:05:33.352 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-20T11:05:33.481 DEBUG:teuthology.orchestra.run.smithi173:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:33.543 INFO:teuthology.orchestra.run.smithi173.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:33.544 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-20T11:05:33.544 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 7ah/122d Inode: 1 Links: 3 2024-04-20T11:05:33.544 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:05:33.544 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-20T11:05:33.544 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-20T11:05:33.544 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-20 11:05:33.467956994 +0000 2024-04-20T11:05:33.544 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-20 11:05:33.467956994 +0000 2024-04-20T11:05:33.544 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-04-20T11:05:33.544 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-20T11:05:33.544 DEBUG:teuthology.orchestra.run.smithi173:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-20T11:05:33.650 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:05:33.650 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:05:33.687 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:05:33.695 INFO:tasks.workunit.client.0.smithi117.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-20T11:05:33.707 INFO:tasks.workunit.client.1.smithi173.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-20T11:05:34.939 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:34 smithi117 ceph-mon[15659]: pgmap v255: 65 pgs: 65 active+clean; 582 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-20T11:05:35.074 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:34 smithi173 ceph-mon[23800]: pgmap v255: 65 pgs: 65 active+clean; 582 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-20T11:05:36.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.216+0000 7faa0a000700 1 -- 172.21.15.117:0/641351892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04074f10 msgr2=0x7faa04073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:36.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.216+0000 7faa0a000700 1 --2- 172.21.15.117:0/641351892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04074f10 0x7faa04073370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009910 tx=0x7fa9f4009c20 comp rx=0 tx=0).stop 2024-04-20T11:05:36.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.217+0000 7faa0a000700 1 -- 172.21.15.117:0/641351892 shutdown_connections 2024-04-20T11:05:36.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.217+0000 7faa0a000700 1 --2- 172.21.15.117:0/641351892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa04073940 0x7faa04073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:36.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.217+0000 7faa0a000700 1 --2- 172.21.15.117:0/641351892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04074f10 0x7faa04073370 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:36.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.217+0000 7faa0a000700 1 -- 172.21.15.117:0/641351892 >> 172.21.15.117:0/641351892 conn(0x7faa040fc1c0 msgr2=0x7faa040fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:36.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.217+0000 7faa0a000700 1 -- 172.21.15.117:0/641351892 shutdown_connections 2024-04-20T11:05:36.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.217+0000 7faa0a000700 1 -- 172.21.15.117:0/641351892 wait complete. 2024-04-20T11:05:36.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.218+0000 7faa0a000700 1 Processor -- start 2024-04-20T11:05:36.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.218+0000 7faa0a000700 1 -- start start 2024-04-20T11:05:36.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.218+0000 7faa0a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04073940 0x7faa04198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:36.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.218+0000 7faa0a000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa04074f10 0x7faa04198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:36.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.218+0000 7faa0a000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041993e0 con 0x7faa04073940 2024-04-20T11:05:36.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.218+0000 7faa0a000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa04199520 con 0x7faa04074f10 2024-04-20T11:05:36.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.219+0000 7faa03400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04073940 0x7faa04198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:36.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.219+0000 7faa03400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04073940 0x7faa04198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44690/0 (socket says 172.21.15.117:44690) 2024-04-20T11:05:36.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.219+0000 7faa03400700 1 -- 172.21.15.117:0/466063111 learned_addr learned my addr 172.21.15.117:0/466063111 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:36.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.219+0000 7faa02a00700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa04074f10 0x7faa04198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:36.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.219+0000 7faa03400700 1 -- 172.21.15.117:0/466063111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa04074f10 msgr2=0x7faa04198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.219+0000 7faa03400700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa04074f10 0x7faa04198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.219+0000 7faa03400700 1 -- 172.21.15.117:0/466063111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40095f0 con 0x7faa04073940 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.219+0000 7faa03400700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04073940 0x7faa04198880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40059d0 tx=0x7fa9f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.220+0000 7fa9fbe00700 1 -- 172.21.15.117:0/466063111 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9f401d070 con 0x7faa04073940 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.220+0000 7fa9fbe00700 1 -- 172.21.15.117:0/466063111 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9f4005dc0 con 0x7faa04073940 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.220+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0419ded0 con 0x7faa04073940 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.220+0000 7fa9fbe00700 1 -- 172.21.15.117:0/466063111 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9f4021410 con 0x7faa04073940 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.220+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0419e3c0 con 0x7faa04073940 2024-04-20T11:05:36.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.221+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa041929b0 con 0x7faa04073940 2024-04-20T11:05:36.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.222+0000 7fa9fbe00700 1 -- 172.21.15.117:0/466063111 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fa9f400f840 con 0x7faa04073940 2024-04-20T11:05:36.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.223+0000 7fa9fbe00700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa9e806cce0 0x7fa9e806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:36.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.223+0000 7fa9fbe00700 1 -- 172.21.15.117:0/466063111 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa9f408d530 con 0x7faa04073940 2024-04-20T11:05:36.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.223+0000 7faa02a00700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa9e806cce0 0x7fa9e806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:36.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.224+0000 7faa02a00700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa9e806cce0 0x7fa9e806f1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0005950 tx=0x7fa9f00058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:36.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.228+0000 7fa9fbe00700 1 -- 172.21.15.117:0/466063111 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa9f405b2f0 con 0x7faa04073940 2024-04-20T11:05:36.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.447+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7faa04062510 con 0x7faa04073940 2024-04-20T11:05:36.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.448+0000 7fa9fbe00700 1 -- 172.21.15.117:0/466063111 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v25)=0 v25) v1 ==== 155+0+0 (secure 0 0 0) 0x7faa04062510 con 0x7faa04073940 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa9e806cce0 msgr2=0x7fa9e806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa9e806cce0 0x7fa9e806f1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0005950 tx=0x7fa9f00058e0 comp rx=0 tx=0).stop 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04073940 msgr2=0x7faa04198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04073940 0x7faa04198880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40059d0 tx=0x7fa9f400be20 comp rx=0 tx=0).stop 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 shutdown_connections 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fa9e806cce0 0x7fa9e806f1a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04073940 0x7faa04198880 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 --2- 172.21.15.117:0/466063111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa04074f10 0x7faa04198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 >> 172.21.15.117:0/466063111 conn(0x7faa040fc1c0 msgr2=0x7faa04106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 shutdown_connections 2024-04-20T11:05:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:36.451+0000 7faa0a000700 1 -- 172.21.15.117:0/466063111 wait complete. 2024-04-20T11:05:36.852 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:36 smithi117 ceph-mon[15659]: pgmap v256: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-20T11:05:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:36 smithi173 ceph-mon[23800]: pgmap v256: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-20T11:05:37.313 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-20T11:05:37.752 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:05:38.962 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:38 smithi117 ceph-mon[15659]: pgmap v257: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:38 smithi173 ceph-mon[23800]: pgmap v257: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:39.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.941+0000 7f13bdc00700 1 -- 172.21.15.117:0/2852389705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 msgr2=0x7f13b8103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:39.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.941+0000 7f13bdc00700 1 --2- 172.21.15.117:0/2852389705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 0x7f13b8103270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009910 tx=0x7f13a8009c20 comp rx=0 tx=0).stop 2024-04-20T11:05:39.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.941+0000 7f13bdc00700 1 -- 172.21.15.117:0/2852389705 shutdown_connections 2024-04-20T11:05:39.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.941+0000 7f13bdc00700 1 --2- 172.21.15.117:0/2852389705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13b8103fb0 0x7f13b8104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:39.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.941+0000 7f13bdc00700 1 --2- 172.21.15.117:0/2852389705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 0x7f13b8103270 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:39.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.941+0000 7f13bdc00700 1 -- 172.21.15.117:0/2852389705 >> 172.21.15.117:0/2852389705 conn(0x7f13b80fe3d0 msgr2=0x7f13b8100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:39.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.941+0000 7f13bdc00700 1 -- 172.21.15.117:0/2852389705 shutdown_connections 2024-04-20T11:05:39.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.941+0000 7f13bdc00700 1 -- 172.21.15.117:0/2852389705 wait complete. 2024-04-20T11:05:39.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.943+0000 7f13bdc00700 1 Processor -- start 2024-04-20T11:05:39.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.943+0000 7f13bdc00700 1 -- start start 2024-04-20T11:05:39.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.943+0000 7f13bdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 0x7f13b8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:39.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.943+0000 7f13bdc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13b8103fb0 0x7f13b8198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:39.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.943+0000 7f13bdc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8199370 con 0x7f13b8102e50 2024-04-20T11:05:39.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.943+0000 7f13bdc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b81994b0 con 0x7f13b8103fb0 2024-04-20T11:05:39.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.944+0000 7f13b6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 0x7f13b8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:39.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.944+0000 7f13b6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 0x7f13b8198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44706/0 (socket says 172.21.15.117:44706) 2024-04-20T11:05:39.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.944+0000 7f13b6a00700 1 -- 172.21.15.117:0/4154669736 learned_addr learned my addr 172.21.15.117:0/4154669736 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:39.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.944+0000 7f13b6000700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13b8103fb0 0x7f13b8198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:39.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.944+0000 7f13b6a00700 1 -- 172.21.15.117:0/4154669736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13b8103fb0 msgr2=0x7f13b8198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:39.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.944+0000 7f13b6a00700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13b8103fb0 0x7f13b8198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:39.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.944+0000 7f13b6a00700 1 -- 172.21.15.117:0/4154669736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a80095f0 con 0x7f13b8102e50 2024-04-20T11:05:39.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.945+0000 7f13b6a00700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 0x7f13b8198810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13a8006010 tx=0x7f13a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:39.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.945+0000 7f13af400700 1 -- 172.21.15.117:0/4154669736 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a801d070 con 0x7f13b8102e50 2024-04-20T11:05:39.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.945+0000 7f13af400700 1 -- 172.21.15.117:0/4154669736 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13a8005dc0 con 0x7f13b8102e50 2024-04-20T11:05:39.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.945+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b819deb0 con 0x7f13b8102e50 2024-04-20T11:05:39.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.945+0000 7f13af400700 1 -- 172.21.15.117:0/4154669736 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a8021640 con 0x7f13b8102e50 2024-04-20T11:05:39.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.945+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b819e3a0 con 0x7f13b8102e50 2024-04-20T11:05:39.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.946+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b8192950 con 0x7f13b8102e50 2024-04-20T11:05:39.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.947+0000 7f13af400700 1 -- 172.21.15.117:0/4154669736 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f13a800f880 con 0x7f13b8102e50 2024-04-20T11:05:39.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.948+0000 7f13af400700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f139c06ccf0 0x7f139c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:39.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.948+0000 7f13af400700 1 -- 172.21.15.117:0/4154669736 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f13a808d4c0 con 0x7f13b8102e50 2024-04-20T11:05:39.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.948+0000 7f13b6000700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f139c06ccf0 0x7f139c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:39.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.949+0000 7f13b6000700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f139c06ccf0 0x7f139c06f1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13a4005950 tx=0x7f13a40058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:39.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:39.953+0000 7f13af400700 1 -- 172.21.15.117:0/4154669736 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f13a805b300 con 0x7f13b8102e50 2024-04-20T11:05:40.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.177+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f13b810b930 con 0x7f13b8102e50 2024-04-20T11:05:40.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.178+0000 7f13af400700 1 -- 172.21.15.117:0/4154669736 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v25)=0 v25) v1 ==== 163+0+0 (secure 0 0 0) 0x7f13a808be60 con 0x7f13b8102e50 2024-04-20T11:05:40.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f139c06ccf0 msgr2=0x7f139c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:40.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f139c06ccf0 0x7f139c06f1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13a4005950 tx=0x7f13a40058e0 comp rx=0 tx=0).stop 2024-04-20T11:05:40.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 msgr2=0x7f13b8198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:40.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 0x7f13b8198810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13a8006010 tx=0x7f13a800be20 comp rx=0 tx=0).stop 2024-04-20T11:05:40.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 shutdown_connections 2024-04-20T11:05:40.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f139c06ccf0 0x7f139c06f1b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:40.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102e50 0x7f13b8198810 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:40.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 --2- 172.21.15.117:0/4154669736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13b8103fb0 0x7f13b8198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:40.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.181+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 >> 172.21.15.117:0/4154669736 conn(0x7f13b80fe3d0 msgr2=0x7f13b81075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:40.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.182+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 shutdown_connections 2024-04-20T11:05:40.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:40.182+0000 7f13bdc00700 1 -- 172.21.15.117:0/4154669736 wait complete. 2024-04-20T11:05:40.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:40 smithi117 ceph-mon[15659]: pgmap v258: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:41.052 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:40 smithi173 ceph-mon[23800]: pgmap v258: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:41.062 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-20T11:05:41.487 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:05:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:42 smithi117 ceph-mon[15659]: pgmap v259: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:42 smithi173 ceph-mon[23800]: pgmap v259: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:43.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.655+0000 7f1bf5200700 1 -- 172.21.15.117:0/1350758717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf0108290 msgr2=0x7f1bf01086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.655+0000 7f1bf5200700 1 --2- 172.21.15.117:0/1350758717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf0108290 0x7f1bf01086f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009910 tx=0x7f1bd8009c20 comp rx=0 tx=0).stop 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.655+0000 7f1bf5200700 1 -- 172.21.15.117:0/1350758717 shutdown_connections 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.655+0000 7f1bf5200700 1 --2- 172.21.15.117:0/1350758717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf0108290 0x7f1bf01086f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.655+0000 7f1bf5200700 1 --2- 172.21.15.117:0/1350758717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bf01070f0 0x7f1bf0107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.655+0000 7f1bf5200700 1 -- 172.21.15.117:0/1350758717 >> 172.21.15.117:0/1350758717 conn(0x7f1bf00761d0 msgr2=0x7f1bf0078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.655+0000 7f1bf5200700 1 -- 172.21.15.117:0/1350758717 shutdown_connections 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.655+0000 7f1bf5200700 1 -- 172.21.15.117:0/1350758717 wait complete. 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.656+0000 7f1bf5200700 1 Processor -- start 2024-04-20T11:05:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.656+0000 7f1bf5200700 1 -- start start 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.656+0000 7f1bf5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf01070f0 0x7f1bf019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.656+0000 7f1bf5200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bf0108290 0x7f1bf019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.656+0000 7f1bf5200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf019d5d0 con 0x7f1bf01070f0 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.656+0000 7f1bf5200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf019d710 con 0x7f1bf0108290 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.657+0000 7f1bef400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf01070f0 0x7f1bf019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.657+0000 7f1bef400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf01070f0 0x7f1bf019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44718/0 (socket says 172.21.15.117:44718) 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.657+0000 7f1bef400700 1 -- 172.21.15.117:0/2266291175 learned_addr learned my addr 172.21.15.117:0/2266291175 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.657+0000 7f1be7e00700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bf0108290 0x7f1bf019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.657+0000 7f1bef400700 1 -- 172.21.15.117:0/2266291175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bf0108290 msgr2=0x7f1bf019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.657+0000 7f1bef400700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bf0108290 0x7f1bf019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.657+0000 7f1bef400700 1 -- 172.21.15.117:0/2266291175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd80095f0 con 0x7f1bf01070f0 2024-04-20T11:05:43.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.658+0000 7f1bef400700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf01070f0 0x7f1bf019ca70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1be000c8f0 tx=0x7f1be000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:43.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.658+0000 7f1becc00700 1 -- 172.21.15.117:0/2266291175 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1be0007a70 con 0x7f1bf01070f0 2024-04-20T11:05:43.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.658+0000 7f1becc00700 1 -- 172.21.15.117:0/2266291175 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1be0007bd0 con 0x7f1bf01070f0 2024-04-20T11:05:43.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.658+0000 7f1becc00700 1 -- 172.21.15.117:0/2266291175 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1be0018630 con 0x7f1bf01070f0 2024-04-20T11:05:43.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.658+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf01a2120 con 0x7f1bf01070f0 2024-04-20T11:05:43.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.658+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf01a26a0 con 0x7f1bf01070f0 2024-04-20T11:05:43.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.659+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf0196c10 con 0x7f1bf01070f0 2024-04-20T11:05:43.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.660+0000 7f1becc00700 1 -- 172.21.15.117:0/2266291175 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f1be000f450 con 0x7f1bf01070f0 2024-04-20T11:05:43.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.661+0000 7f1becc00700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f1bd406ccf0 0x7f1bd406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:43.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.661+0000 7f1becc00700 1 -- 172.21.15.117:0/2266291175 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f1be008c340 con 0x7f1bf01070f0 2024-04-20T11:05:43.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.662+0000 7f1be7e00700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f1bd406ccf0 0x7f1bd406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:43.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.662+0000 7f1be7e00700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f1bd406ccf0 0x7f1bd406f1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80051c0 tx=0x7f1bd800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:43.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.666+0000 7f1becc00700 1 -- 172.21.15.117:0/2266291175 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1be005a230 con 0x7f1bf01070f0 2024-04-20T11:05:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.895+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f1bf010fc40 con 0x7f1bf01070f0 2024-04-20T11:05:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.896+0000 7f1becc00700 1 -- 172.21.15.117:0/2266291175 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v25)=0 v25) v1 ==== 135+0+0 (secure 0 0 0) 0x7f1be0059dc0 con 0x7f1bf01070f0 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f1bd406ccf0 msgr2=0x7f1bd406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f1bd406ccf0 0x7f1bd406f1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80051c0 tx=0x7f1bd800b540 comp rx=0 tx=0).stop 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf01070f0 msgr2=0x7f1bf019ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf01070f0 0x7f1bf019ca70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1be000c8f0 tx=0x7f1be000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 shutdown_connections 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f1bd406ccf0 0x7f1bd406f1b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bf01070f0 0x7f1bf019ca70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 --2- 172.21.15.117:0/2266291175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bf0108290 0x7f1bf019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 >> 172.21.15.117:0/2266291175 conn(0x7f1bf00761d0 msgr2=0x7f1bf010b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 shutdown_connections 2024-04-20T11:05:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:43.899+0000 7f1bf5200700 1 -- 172.21.15.117:0/2266291175 wait complete. 2024-04-20T11:05:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:43 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:05:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:43 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:05:44.735 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-20T11:05:45.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:44 smithi117 ceph-mon[15659]: pgmap v260: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:45.150 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:05:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:44 smithi173 ceph-mon[23800]: pgmap v260: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:46.793 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:46 smithi117 ceph-mon[15659]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 1 op/s 2024-04-20T11:05:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:46 smithi173 ceph-mon[23800]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 1 op/s 2024-04-20T11:05:47.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.324+0000 7f989f200700 1 -- 172.21.15.117:0/4150356937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898100c30 msgr2=0x7f9898101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:47.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.324+0000 7f989f200700 1 --2- 172.21.15.117:0/4150356937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898100c30 0x7f9898101050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9888009910 tx=0x7f9888009c20 comp rx=0 tx=0).stop 2024-04-20T11:05:47.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.324+0000 7f989f200700 1 -- 172.21.15.117:0/4150356937 shutdown_connections 2024-04-20T11:05:47.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.324+0000 7f989f200700 1 --2- 172.21.15.117:0/4150356937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9898101e30 0x7f9898102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:47.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.324+0000 7f989f200700 1 --2- 172.21.15.117:0/4150356937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898100c30 0x7f9898101050 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:47.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.324+0000 7f989f200700 1 -- 172.21.15.117:0/4150356937 >> 172.21.15.117:0/4150356937 conn(0x7f98980fc1f0 msgr2=0x7f98980fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:05:47.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.324+0000 7f989f200700 1 -- 172.21.15.117:0/4150356937 shutdown_connections 2024-04-20T11:05:47.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.324+0000 7f989f200700 1 -- 172.21.15.117:0/4150356937 wait complete. 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.325+0000 7f989f200700 1 Processor -- start 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.325+0000 7f989f200700 1 -- start start 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.325+0000 7f989f200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9898100c30 0x7f9898198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.325+0000 7f989f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898101e30 0x7f9898198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.325+0000 7f989f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9898199390 con 0x7f9898101e30 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.325+0000 7f989f200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98981994d0 con 0x7f9898100c30 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898101e30 0x7f9898198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898101e30 0x7f9898198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39240/0 (socket says 172.21.15.117:39240) 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897400700 1 -- 172.21.15.117:0/3256854529 learned_addr learned my addr 172.21.15.117:0/3256854529 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:05:47.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897e00700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9898100c30 0x7f9898198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:47.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897400700 1 -- 172.21.15.117:0/3256854529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9898100c30 msgr2=0x7f9898198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:05:47.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897400700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9898100c30 0x7f9898198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:05:47.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897400700 1 -- 172.21.15.117:0/3256854529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880095f0 con 0x7f9898101e30 2024-04-20T11:05:47.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897e00700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9898100c30 0x7f9898198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:05:47.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.326+0000 7f9897400700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898101e30 0x7f9898198d70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f988400c8a0 tx=0x7f988400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:47.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.327+0000 7f9894c00700 1 -- 172.21.15.117:0/3256854529 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f988400cea0 con 0x7f9898101e30 2024-04-20T11:05:47.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.327+0000 7f9894c00700 1 -- 172.21.15.117:0/3256854529 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9884004830 con 0x7f9898101e30 2024-04-20T11:05:47.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.327+0000 7f9894c00700 1 -- 172.21.15.117:0/3256854529 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98840055b0 con 0x7f9898101e30 2024-04-20T11:05:47.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.327+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989819df30 con 0x7f9898101e30 2024-04-20T11:05:47.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.327+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989819e4d0 con 0x7f9898101e30 2024-04-20T11:05:47.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.328+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9898106800 con 0x7f9898101e30 2024-04-20T11:05:47.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.329+0000 7f9894c00700 1 -- 172.21.15.117:0/3256854529 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f98840049a0 con 0x7f9898101e30 2024-04-20T11:05:47.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.330+0000 7f9894c00700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f987c06ccf0 0x7f987c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:05:47.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.330+0000 7f9894c00700 1 -- 172.21.15.117:0/3256854529 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f988408bea0 con 0x7f9898101e30 2024-04-20T11:05:47.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.330+0000 7f9897e00700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f987c06ccf0 0x7f987c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:05:47.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.331+0000 7f9897e00700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f987c06ccf0 0x7f987c06f1b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f988800b5c0 tx=0x7f9888005160 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:05:47.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.335+0000 7f9894c00700 1 -- 172.21.15.117:0/3256854529 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9884059ce0 con 0x7f9898101e30 2024-04-20T11:05:47.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:05:47.577+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f989802ce80 con 0x7f987c06ccf0 2024-04-20T11:05:49.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:48 smithi117 ceph-mon[15659]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:05:49.113 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:48 smithi117 ceph-mon[15659]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:49.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:48 smithi173 ceph-mon[23800]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:05:49.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:48 smithi173 ceph-mon[23800]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-20T11:05:51.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:50 smithi117 ceph-mon[15659]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-20T11:05:51.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:50 smithi173 ceph-mon[23800]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-20T11:05:53.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:52 smithi117 ceph-mon[15659]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-20T11:05:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:52 smithi173 ceph-mon[23800]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-20T11:05:55.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:54 smithi117 ceph-mon[15659]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-20T11:05:55.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:54 smithi173 ceph-mon[23800]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-20T11:05:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:56 smithi117 ceph-mon[15659]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:05:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:56 smithi173 ceph-mon[23800]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:05:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:58 smithi117 ceph-mon[15659]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-20T11:05:59.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:05:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:05:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:58 smithi173 ceph-mon[23800]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-20T11:05:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:05:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:06:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:00 smithi117 ceph-mon[15659]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:01.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:00 smithi173 ceph-mon[23800]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:01.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:01 smithi173 ceph-mon[23800]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:02.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:01 smithi117 ceph-mon[15659]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:04 smithi173 ceph-mon[23800]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:05.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:04 smithi117 ceph-mon[15659]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:06 smithi173 ceph-mon[23800]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:06.952 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:06 smithi117 ceph-mon[15659]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:08 smithi173 ceph-mon[23800]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:09.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:08 smithi117 ceph-mon[15659]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:11.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:10 smithi117 ceph-mon[15659]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:10 smithi173 ceph-mon[23800]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:12 smithi117 ceph-mon[15659]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:06:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:12 smithi173 ceph-mon[23800]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:06:14.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:06:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:06:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:14.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:13 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:06:14.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:13 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:06:14.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:13 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:15.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:14 smithi117 ceph-mon[15659]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:15.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:14 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:06:15.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:14 smithi173 ceph-mon[23800]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:15.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:14 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:06:17.082 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:16 smithi173 ceph-mon[23800]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:17.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:16 smithi117 ceph-mon[15659]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:19.112 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:18 smithi117 ceph-mon[15659]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:19.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:18 smithi173 ceph-mon[23800]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:21.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:20 smithi117 ceph-mon[15659]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:21.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:20 smithi173 ceph-mon[23800]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:22.694 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:22 smithi117 ceph-mon[15659]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:23.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:22 smithi173 ceph-mon[23800]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:25.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:24 smithi117 ceph-mon[15659]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:24 smithi173 ceph-mon[23800]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:27.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:26 smithi117 ceph-mon[15659]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:27.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:26 smithi173 ceph-mon[23800]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:29.114 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:28 smithi117 ceph-mon[15659]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:29.114 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:06:29.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:28 smithi173 ceph-mon[23800]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:29.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:06:30.963 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:30 smithi173 ceph-mon[23800]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:31.098 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:30 smithi117 ceph-mon[15659]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:32.710 INFO:teuthology.orchestra.run.smithi117.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:06:32.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.709+0000 7f9894c00700 1 -- 172.21.15.117:0/3256854529 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f9888028070 con 0x7f987c06ccf0 2024-04-20T11:06:32.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f987c06ccf0 msgr2=0x7f987c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:32.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f987c06ccf0 0x7f987c06f1b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f988800b5c0 tx=0x7f9888005160 comp rx=0 tx=0).stop 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898101e30 msgr2=0x7f9898198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898101e30 0x7f9898198d70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f988400c8a0 tx=0x7f988400cc60 comp rx=0 tx=0).stop 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 shutdown_connections 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f987c06ccf0 0x7f987c06f1b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9898100c30 0x7f9898198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 --2- 172.21.15.117:0/3256854529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9898101e30 0x7f9898198d70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 >> 172.21.15.117:0/3256854529 conn(0x7f98980fc1f0 msgr2=0x7f9898073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 shutdown_connections 2024-04-20T11:06:32.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:32.712+0000 7f989f200700 1 -- 172.21.15.117:0/3256854529 wait complete. 2024-04-20T11:06:32.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:32 smithi117 ceph-mon[15659]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:32.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:32 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:32 smithi173 ceph-mon[23800]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:32 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:33.505 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-20T11:06:33.722 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:33 smithi117 ceph-mon[15659]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:06:33.723 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:06:33.723 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:33.723 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:33.723 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:06:33.723 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:06:33.723 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:33 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:34.029 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:06:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:33 smithi173 ceph-mon[23800]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:06:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:06:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:06:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:06:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:33 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:34 smithi117 ceph-mon[15659]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:35.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:34 smithi173 ceph-mon[23800]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:35.965 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:35 smithi117 ceph-mon[15659]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:06:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:35 smithi173 ceph-mon[23800]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:06:36.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.589+0000 7fca3a200700 1 -- 172.21.15.117:0/2599186262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ff2b0 msgr2=0x7fca340ff6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:36.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.589+0000 7fca3a200700 1 --2- 172.21.15.117:0/2599186262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ff2b0 0x7fca340ff6d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca28009910 tx=0x7fca28009c20 comp rx=0 tx=0).stop 2024-04-20T11:06:36.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.590+0000 7fca3a200700 1 -- 172.21.15.117:0/2599186262 shutdown_connections 2024-04-20T11:06:36.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.590+0000 7fca3a200700 1 --2- 172.21.15.117:0/2599186262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca340ffc10 0x7fca34100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:36.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.590+0000 7fca3a200700 1 --2- 172.21.15.117:0/2599186262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ff2b0 0x7fca340ff6d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:36.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.590+0000 7fca3a200700 1 -- 172.21.15.117:0/2599186262 >> 172.21.15.117:0/2599186262 conn(0x7fca340faef0 msgr2=0x7fca340fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:36.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.590+0000 7fca3a200700 1 -- 172.21.15.117:0/2599186262 shutdown_connections 2024-04-20T11:06:36.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.590+0000 7fca3a200700 1 -- 172.21.15.117:0/2599186262 wait complete. 2024-04-20T11:06:36.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.592+0000 7fca3a200700 1 Processor -- start 2024-04-20T11:06:36.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.592+0000 7fca3a200700 1 -- start start 2024-04-20T11:06:36.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.592+0000 7fca3a200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca340ff2b0 0x7fca3419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:36.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.592+0000 7fca3a200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ffc10 0x7fca3419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:36.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.592+0000 7fca3a200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3419d5b0 con 0x7fca340ffc10 2024-04-20T11:06:36.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.592+0000 7fca3a200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3419d6f0 con 0x7fca340ff2b0 2024-04-20T11:06:36.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.593+0000 7fca38e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca340ff2b0 0x7fca3419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:36.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.593+0000 7fca38e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca340ff2b0 0x7fca3419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:41448/0 (socket says 172.21.15.117:41448) 2024-04-20T11:06:36.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.593+0000 7fca38e00700 1 -- 172.21.15.117:0/1977816897 learned_addr learned my addr 172.21.15.117:0/1977816897 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:06:36.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.593+0000 7fca33e00700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ffc10 0x7fca3419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:36.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.593+0000 7fca38e00700 1 -- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ffc10 msgr2=0x7fca3419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:36.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.594+0000 7fca38e00700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ffc10 0x7fca3419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:36.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.594+0000 7fca38e00700 1 -- 172.21.15.117:0/1977816897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280095f0 con 0x7fca340ff2b0 2024-04-20T11:06:36.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.594+0000 7fca33e00700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ffc10 0x7fca3419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:06:36.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.594+0000 7fca38e00700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca340ff2b0 0x7fca3419ca50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca2800b650 tx=0x7fca2800f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:36.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.595+0000 7fca31600700 1 -- 172.21.15.117:0/1977816897 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca2801d070 con 0x7fca340ff2b0 2024-04-20T11:06:36.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.595+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca341a20a0 con 0x7fca340ff2b0 2024-04-20T11:06:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.595+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca341a2610 con 0x7fca340ff2b0 2024-04-20T11:06:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.595+0000 7fca31600700 1 -- 172.21.15.117:0/1977816897 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca2800fe70 con 0x7fca340ff2b0 2024-04-20T11:06:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.595+0000 7fca31600700 1 -- 172.21.15.117:0/1977816897 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca28017400 con 0x7fca340ff2b0 2024-04-20T11:06:36.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.596+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca14005320 con 0x7fca340ff2b0 2024-04-20T11:06:36.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.599+0000 7fca31600700 1 -- 172.21.15.117:0/1977816897 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fca28003680 con 0x7fca340ff2b0 2024-04-20T11:06:36.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.600+0000 7fca31600700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fca1c06cce0 0x7fca1c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:36.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.600+0000 7fca31600700 1 -- 172.21.15.117:0/1977816897 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fca2808d850 con 0x7fca340ff2b0 2024-04-20T11:06:36.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.601+0000 7fca33e00700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fca1c06cce0 0x7fca1c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:36.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.601+0000 7fca33e00700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fca1c06cce0 0x7fca1c06f1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca20009cb0 tx=0x7fca20009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:36.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:36.613+0000 7fca31600700 1 -- 172.21.15.117:0/1977816897 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fca2805b710 con 0x7fca340ff2b0 2024-04-20T11:06:36.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:36 smithi117 ceph-mon[15659]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.050+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca14000bf0 con 0x7fca1c06cce0 2024-04-20T11:06:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.052+0000 7fca31600700 1 -- 172.21.15.117:0/1977816897 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fca14000bf0 con 0x7fca1c06cce0 2024-04-20T11:06:37.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.054+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fca1c06cce0 msgr2=0x7fca1c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:37.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.054+0000 7fca3a200700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fca1c06cce0 0x7fca1c06f1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca20009cb0 tx=0x7fca20009260 comp rx=0 tx=0).stop 2024-04-20T11:06:37.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.054+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca340ff2b0 msgr2=0x7fca3419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:37.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.055+0000 7fca3a200700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca340ff2b0 0x7fca3419ca50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca2800b650 tx=0x7fca2800f510 comp rx=0 tx=0).stop 2024-04-20T11:06:37.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.058+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 shutdown_connections 2024-04-20T11:06:37.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.058+0000 7fca3a200700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7fca1c06cce0 0x7fca1c06f1a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.058+0000 7fca3a200700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca340ff2b0 0x7fca3419ca50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.058+0000 7fca3a200700 1 --2- 172.21.15.117:0/1977816897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca340ffc10 0x7fca3419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.058+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 >> 172.21.15.117:0/1977816897 conn(0x7fca340faef0 msgr2=0x7fca34105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:37.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.058+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 shutdown_connections 2024-04-20T11:06:37.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.058+0000 7fca3a200700 1 -- 172.21.15.117:0/1977816897 wait complete. 2024-04-20T11:06:37.075 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:06:37.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:36 smithi173 ceph-mon[23800]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:37.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.211+0000 7f6b09e00700 1 -- 172.21.15.117:0/675198477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101c70 msgr2=0x7f6b041020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:37.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.211+0000 7f6b09e00700 1 --2- 172.21.15.117:0/675198477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101c70 0x7f6b041020d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009910 tx=0x7f6af0009c20 comp rx=0 tx=0).stop 2024-04-20T11:06:37.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.211+0000 7f6b09e00700 1 -- 172.21.15.117:0/675198477 shutdown_connections 2024-04-20T11:06:37.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.211+0000 7f6b09e00700 1 --2- 172.21.15.117:0/675198477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101c70 0x7f6b041020d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.211+0000 7f6b09e00700 1 --2- 172.21.15.117:0/675198477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b04100a70 0x7f6b04100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.211+0000 7f6b09e00700 1 -- 172.21.15.117:0/675198477 >> 172.21.15.117:0/675198477 conn(0x7f6b040fc050 msgr2=0x7f6b040fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:37.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.211+0000 7f6b09e00700 1 -- 172.21.15.117:0/675198477 shutdown_connections 2024-04-20T11:06:37.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.211+0000 7f6b09e00700 1 -- 172.21.15.117:0/675198477 wait complete. 2024-04-20T11:06:37.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.213+0000 7f6b09e00700 1 Processor -- start 2024-04-20T11:06:37.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.213+0000 7f6b09e00700 1 -- start start 2024-04-20T11:06:37.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.213+0000 7f6b09e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04100a70 0x7f6b04194260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:37.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.213+0000 7f6b09e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b04101c70 0x7f6b041947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:37.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.213+0000 7f6b09e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04194dc0 con 0x7f6b04100a70 2024-04-20T11:06:37.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.213+0000 7f6b09e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04194f00 con 0x7f6b04101c70 2024-04-20T11:06:37.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.213+0000 7f6b08a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04100a70 0x7f6b04194260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:37.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.214+0000 7f6b03e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b04101c70 0x7f6b041947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:37.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.214+0000 7f6b03e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b04101c70 0x7f6b041947a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:41468/0 (socket says 172.21.15.117:41468) 2024-04-20T11:06:37.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.214+0000 7f6b03e00700 1 -- 172.21.15.117:0/7678140 learned_addr learned my addr 172.21.15.117:0/7678140 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:06:37.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.214+0000 7f6b03e00700 1 -- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04100a70 msgr2=0x7f6b04194260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:37.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.214+0000 7f6b03e00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04100a70 0x7f6b04194260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.214+0000 7f6b03e00700 1 -- 172.21.15.117:0/7678140 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00095f0 con 0x7f6b04101c70 2024-04-20T11:06:37.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.215+0000 7f6b08a00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04100a70 0x7f6b04194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:06:37.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.215+0000 7f6b03e00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b04101c70 0x7f6b041947a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6af0005b40 tx=0x7f6af000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:37.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.215+0000 7f6b01600700 1 -- 172.21.15.117:0/7678140 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af001d070 con 0x7f6b04101c70 2024-04-20T11:06:37.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.215+0000 7f6b01600700 1 -- 172.21.15.117:0/7678140 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6af00050d0 con 0x7f6b04101c70 2024-04-20T11:06:37.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.215+0000 7f6b01600700 1 -- 172.21.15.117:0/7678140 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af0022470 con 0x7f6b04101c70 2024-04-20T11:06:37.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.215+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b04199900 con 0x7f6b04101c70 2024-04-20T11:06:37.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.215+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b04199e20 con 0x7f6b04101c70 2024-04-20T11:06:37.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.216+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0404f400 con 0x7f6b04101c70 2024-04-20T11:06:37.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.218+0000 7f6b01600700 1 -- 172.21.15.117:0/7678140 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f6af000fac0 con 0x7f6b04101c70 2024-04-20T11:06:37.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.219+0000 7f6b01600700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6aec06ccf0 0x7f6aec06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:37.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.220+0000 7f6b01600700 1 -- 172.21.15.117:0/7678140 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6af008d990 con 0x7f6b04101c70 2024-04-20T11:06:37.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.220+0000 7f6b08a00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6aec06ccf0 0x7f6aec06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:37.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.221+0000 7f6b08a00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6aec06ccf0 0x7f6aec06f1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6b04101ad0 tx=0x7f6af8009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:37.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.226+0000 7f6b01600700 1 -- 172.21.15.117:0/7678140 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6af005b7d0 con 0x7f6b04101c70 2024-04-20T11:06:37.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.544+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b0402ce50 con 0x7f6aec06ccf0 2024-04-20T11:06:37.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.546+0000 7f6b01600700 1 -- 172.21.15.117:0/7678140 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f6b0402ce50 con 0x7f6aec06ccf0 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6aec06ccf0 msgr2=0x7f6aec06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6aec06ccf0 0x7f6aec06f1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6b04101ad0 tx=0x7f6af8009260 comp rx=0 tx=0).stop 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b04101c70 msgr2=0x7f6b041947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b04101c70 0x7f6b041947a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6af0005b40 tx=0x7f6af000f690 comp rx=0 tx=0).stop 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 shutdown_connections 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6aec06ccf0 0x7f6aec06f1b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04100a70 0x7f6b04194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 --2- 172.21.15.117:0/7678140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b04101c70 0x7f6b041947a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.548+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 >> 172.21.15.117:0/7678140 conn(0x7f6b040fc050 msgr2=0x7f6b04073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:37.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.549+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 shutdown_connections 2024-04-20T11:06:37.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.549+0000 7f6b09e00700 1 -- 172.21.15.117:0/7678140 wait complete. 2024-04-20T11:06:37.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.698+0000 7ff811400700 1 -- 172.21.15.117:0/3899516893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c101e00 msgr2=0x7ff80c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:37.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.698+0000 7ff811400700 1 --2- 172.21.15.117:0/3899516893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c101e00 0x7ff80c102260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009910 tx=0x7ff7f8009c20 comp rx=0 tx=0).stop 2024-04-20T11:06:37.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.699+0000 7ff811400700 1 -- 172.21.15.117:0/3899516893 shutdown_connections 2024-04-20T11:06:37.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.699+0000 7ff811400700 1 --2- 172.21.15.117:0/3899516893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c101e00 0x7ff80c102260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.699+0000 7ff811400700 1 --2- 172.21.15.117:0/3899516893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c100c00 0x7ff80c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.699+0000 7ff811400700 1 -- 172.21.15.117:0/3899516893 >> 172.21.15.117:0/3899516893 conn(0x7ff80c0fc1c0 msgr2=0x7ff80c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:37.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.699+0000 7ff811400700 1 -- 172.21.15.117:0/3899516893 shutdown_connections 2024-04-20T11:06:37.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.699+0000 7ff811400700 1 -- 172.21.15.117:0/3899516893 wait complete. 2024-04-20T11:06:37.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.701+0000 7ff811400700 1 Processor -- start 2024-04-20T11:06:37.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.701+0000 7ff811400700 1 -- start start 2024-04-20T11:06:37.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.701+0000 7ff811400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c100c00 0x7ff80c1943f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:37.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff811400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c101e00 0x7ff80c194930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:37.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff811400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c194f50 con 0x7ff80c100c00 2024-04-20T11:06:37.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff811400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c195090 con 0x7ff80c101e00 2024-04-20T11:06:37.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff80a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c100c00 0x7ff80c1943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:37.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff80a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c100c00 0x7ff80c1943f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34098/0 (socket says 172.21.15.117:34098) 2024-04-20T11:06:37.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff80a800700 1 -- 172.21.15.117:0/2865144604 learned_addr learned my addr 172.21.15.117:0/2865144604 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:06:37.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff80a800700 1 -- 172.21.15.117:0/2865144604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c101e00 msgr2=0x7ff80c194930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:37.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff80a800700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c101e00 0x7ff80c194930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:37.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.702+0000 7ff80a800700 1 -- 172.21.15.117:0/2865144604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80095f0 con 0x7ff80c100c00 2024-04-20T11:06:37.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.703+0000 7ff80a800700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c100c00 0x7ff80c1943f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e8f0 tx=0x7ff7fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:37.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.703+0000 7ff803400700 1 -- 172.21.15.117:0/2865144604 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7fc009790 con 0x7ff80c100c00 2024-04-20T11:06:37.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.704+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c199af0 con 0x7ff80c100c00 2024-04-20T11:06:37.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.704+0000 7ff803400700 1 -- 172.21.15.117:0/2865144604 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7fc0044e0 con 0x7ff80c100c00 2024-04-20T11:06:37.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.704+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c19a0c0 con 0x7ff80c100c00 2024-04-20T11:06:37.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.704+0000 7ff803400700 1 -- 172.21.15.117:0/2865144604 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7fc003e00 con 0x7ff80c100c00 2024-04-20T11:06:37.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.706+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c18e580 con 0x7ff80c100c00 2024-04-20T11:06:37.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.708+0000 7ff803400700 1 -- 172.21.15.117:0/2865144604 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ff7fc00cba0 con 0x7ff80c100c00 2024-04-20T11:06:37.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.709+0000 7ff803400700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff7f0071090 0x7ff7f0073550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:37.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.709+0000 7ff803400700 1 -- 172.21.15.117:0/2865144604 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff7fc014070 con 0x7ff80c100c00 2024-04-20T11:06:37.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.710+0000 7ff809e00700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff7f0071090 0x7ff7f0073550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:37.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.712+0000 7ff809e00700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff7f0071090 0x7ff7f0073550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8006010 tx=0x7ff7f800b7c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:37.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:37.719+0000 7ff803400700 1 -- 172.21.15.117:0/2865144604 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff7fc05a480 con 0x7ff80c100c00 2024-04-20T11:06:38.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.029+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff80c02ceb0 con 0x7ff7f0071090 2024-04-20T11:06:38.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.046+0000 7ff803400700 1 -- 172.21.15.117:0/2865144604 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3864 (secure 0 0 0) 0x7ff80c02ceb0 con 0x7ff7f0071090 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (7m) 87s ago 9m 13.1M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (9m) 87s ago 9m 7532k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (8m) 93s ago 8m 7448k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (9m) 87s ago 9m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (8m) 93s ago 8m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (7m) 87s ago 8m 74.7M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (111s) 87s ago 110s 11.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (2m) 87s ago 119s 14.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (107s) 93s ago 106s 11.6M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (116s) 93s ago 115s 14.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:9283,8765,8443 running (10m) 87s ago 10m 500M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 2cf5112ba77d 2024-04-20T11:06:38.048 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (7m) 93s ago 7m 449M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 88bcd8bcb785 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (10m) 87s ago 11m 45.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (7m) 93s ago 7m 34.8M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (9m) 87s ago 9m 7596k - 1.5.0 0da6a335fe13 6b69ffbf445a 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (7m) 93s ago 7m 7834k - 1.5.0 0da6a335fe13 81909a8618f8 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (6m) 87s ago 6m 54.1M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (6m) 87s ago 6m 47.5M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (5m) 87s ago 5m 45.1M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (4m) 93s ago 4m 51.7M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (4m) 93s ago 4m 50.0M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (3m) 93s ago 3m 51.0M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:06:38.049 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (6m) 87s ago 8m 49.8M - 2.43.0 a07b618ecd1d 532b98b31363 2024-04-20T11:06:38.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.049+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff7f0071090 msgr2=0x7ff7f0073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.049+0000 7ff811400700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff7f0071090 0x7ff7f0073550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8006010 tx=0x7ff7f800b7c0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.049+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c100c00 msgr2=0x7ff80c1943f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.049+0000 7ff811400700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c100c00 0x7ff80c1943f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e8f0 tx=0x7ff7fc00ecb0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.050+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 shutdown_connections 2024-04-20T11:06:38.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.050+0000 7ff811400700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff7f0071090 0x7ff7f0073550 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.050+0000 7ff811400700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c100c00 0x7ff80c1943f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.050+0000 7ff811400700 1 --2- 172.21.15.117:0/2865144604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c101e00 0x7ff80c194930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.050+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 >> 172.21.15.117:0/2865144604 conn(0x7ff80c0fc1c0 msgr2=0x7ff80c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:38.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.050+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 shutdown_connections 2024-04-20T11:06:38.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.050+0000 7ff811400700 1 -- 172.21.15.117:0/2865144604 wait complete. 2024-04-20T11:06:38.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.197+0000 7ff935200700 1 -- 172.21.15.117:0/3019748374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ff470 msgr2=0x7ff9300ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.197+0000 7ff935200700 1 --2- 172.21.15.117:0/3019748374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ff470 0x7ff9300ff890 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff920009910 tx=0x7ff920009c20 comp rx=0 tx=0).stop 2024-04-20T11:06:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.198+0000 7ff935200700 1 -- 172.21.15.117:0/3019748374 shutdown_connections 2024-04-20T11:06:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.198+0000 7ff935200700 1 --2- 172.21.15.117:0/3019748374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9300ffdd0 0x7ff930100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.198+0000 7ff935200700 1 --2- 172.21.15.117:0/3019748374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ff470 0x7ff9300ff890 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.198+0000 7ff935200700 1 -- 172.21.15.117:0/3019748374 >> 172.21.15.117:0/3019748374 conn(0x7ff9300fb0d0 msgr2=0x7ff9300fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.198+0000 7ff935200700 1 -- 172.21.15.117:0/3019748374 shutdown_connections 2024-04-20T11:06:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.198+0000 7ff935200700 1 -- 172.21.15.117:0/3019748374 wait complete. 2024-04-20T11:06:38.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.199+0000 7ff935200700 1 Processor -- start 2024-04-20T11:06:38.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.199+0000 7ff935200700 1 -- start start 2024-04-20T11:06:38.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.200+0000 7ff935200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9300ff470 0x7ff930198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:38.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.200+0000 7ff935200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ffdd0 0x7ff930198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:38.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.200+0000 7ff935200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9301993a0 con 0x7ff9300ffdd0 2024-04-20T11:06:38.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.200+0000 7ff935200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9301994e0 con 0x7ff9300ff470 2024-04-20T11:06:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.200+0000 7ff92de00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9300ff470 0x7ff930198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.200+0000 7ff92d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ffdd0 0x7ff930198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.200+0000 7ff92d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ffdd0 0x7ff930198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34114/0 (socket says 172.21.15.117:34114) 2024-04-20T11:06:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.200+0000 7ff92d400700 1 -- 172.21.15.117:0/1230480332 learned_addr learned my addr 172.21.15.117:0/1230480332 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:06:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.201+0000 7ff92d400700 1 -- 172.21.15.117:0/1230480332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9300ff470 msgr2=0x7ff930198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.201+0000 7ff92d400700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9300ff470 0x7ff930198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.201+0000 7ff92d400700 1 -- 172.21.15.117:0/1230480332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9200095f0 con 0x7ff9300ffdd0 2024-04-20T11:06:38.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.201+0000 7ff92d400700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ffdd0 0x7ff930198d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00e940 tx=0x7ff91c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:38.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.202+0000 7ff926a00700 1 -- 172.21.15.117:0/1230480332 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff91c00cba0 con 0x7ff9300ffdd0 2024-04-20T11:06:38.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.202+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93019def0 con 0x7ff9300ffdd0 2024-04-20T11:06:38.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.202+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93019e490 con 0x7ff9300ffdd0 2024-04-20T11:06:38.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.202+0000 7ff926a00700 1 -- 172.21.15.117:0/1230480332 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff91c00ee90 con 0x7ff9300ffdd0 2024-04-20T11:06:38.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.202+0000 7ff926a00700 1 -- 172.21.15.117:0/1230480332 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff91c005580 con 0x7ff9300ffdd0 2024-04-20T11:06:38.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.203+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff930192980 con 0x7ff9300ffdd0 2024-04-20T11:06:38.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.205+0000 7ff926a00700 1 -- 172.21.15.117:0/1230480332 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ff91c004750 con 0x7ff9300ffdd0 2024-04-20T11:06:38.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.206+0000 7ff926a00700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff91406cd30 0x7ff91406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:38.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.206+0000 7ff926a00700 1 -- 172.21.15.117:0/1230480332 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff91c014070 con 0x7ff9300ffdd0 2024-04-20T11:06:38.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.207+0000 7ff92de00700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff91406cd30 0x7ff91406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:38.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.208+0000 7ff92de00700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff91406cd30 0x7ff91406f1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b6f0 tx=0x7ff920005fb0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:38.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.214+0000 7ff926a00700 1 -- 172.21.15.117:0/1230480332 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff91c05a580 con 0x7ff9300ffdd0 2024-04-20T11:06:38.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: Upgrade: Target is version 19.0.0-3133-ga8c84f59 (unknown) 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, digests ['quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1'] 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: Upgrade: Need to upgrade myself (mgr.smithi117.ivnklj) 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:38.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:38 smithi117 ceph-mon[15659]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 on smithi173 2024-04-20T11:06:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:06:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: Upgrade: Target is version 19.0.0-3133-ga8c84f59 (unknown) 2024-04-20T11:06:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, digests ['quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1'] 2024-04-20T11:06:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:06:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:06:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: Upgrade: Need to upgrade myself (mgr.smithi117.ivnklj) 2024-04-20T11:06:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:06:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:38 smithi173 ceph-mon[23800]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 on smithi173 2024-04-20T11:06:38.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.512+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff930062510 con 0x7ff9300ffdd0 2024-04-20T11:06:38.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.513+0000 7ff926a00700 1 -- 172.21.15.117:0/1230480332 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ff91c05a110 con 0x7ff9300ffdd0 2024-04-20T11:06:38.513 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:06:38.513 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:06:38.513 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:06:38.513 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:06:38.514 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:06:38.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff91406cd30 msgr2=0x7ff91406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff91406cd30 0x7ff91406f1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b6f0 tx=0x7ff920005fb0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ffdd0 msgr2=0x7ff930198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ffdd0 0x7ff930198d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00e940 tx=0x7ff91c00ec50 comp rx=0 tx=0).stop 2024-04-20T11:06:38.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 shutdown_connections 2024-04-20T11:06:38.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff91406cd30 0x7ff91406f1f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9300ff470 0x7ff930198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 --2- 172.21.15.117:0/1230480332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9300ffdd0 0x7ff930198d80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.516+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 >> 172.21.15.117:0/1230480332 conn(0x7ff9300fb0d0 msgr2=0x7ff930107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:38.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.517+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 shutdown_connections 2024-04-20T11:06:38.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.517+0000 7ff935200700 1 -- 172.21.15.117:0/1230480332 wait complete. 2024-04-20T11:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.674+0000 7f6bea800700 1 -- 172.21.15.117:0/3959003031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be4103df0 msgr2=0x7f6be41061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.674+0000 7f6bea800700 1 --2- 172.21.15.117:0/3959003031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be4103df0 0x7f6be41061e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009910 tx=0x7f6bd0009c20 comp rx=0 tx=0).stop 2024-04-20T11:06:38.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.674+0000 7f6bea800700 1 -- 172.21.15.117:0/3959003031 shutdown_connections 2024-04-20T11:06:38.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.674+0000 7f6bea800700 1 --2- 172.21.15.117:0/3959003031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be4103df0 0x7f6be41061e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.674+0000 7f6bea800700 1 --2- 172.21.15.117:0/3959003031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6be41014c0 0x7f6be41038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.674+0000 7f6bea800700 1 -- 172.21.15.117:0/3959003031 >> 172.21.15.117:0/3959003031 conn(0x7f6be40faef0 msgr2=0x7f6be40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:38.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.674+0000 7f6bea800700 1 -- 172.21.15.117:0/3959003031 shutdown_connections 2024-04-20T11:06:38.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.674+0000 7f6bea800700 1 -- 172.21.15.117:0/3959003031 wait complete. 2024-04-20T11:06:38.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.675+0000 7f6bea800700 1 Processor -- start 2024-04-20T11:06:38.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.675+0000 7f6bea800700 1 -- start start 2024-04-20T11:06:38.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.676+0000 7f6bea800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be41014c0 0x7f6be419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:38.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.676+0000 7f6bea800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6be4103df0 0x7f6be419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:38.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.676+0000 7f6bea800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be419d560 con 0x7f6be41014c0 2024-04-20T11:06:38.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.676+0000 7f6bea800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be419d6a0 con 0x7f6be4103df0 2024-04-20T11:06:38.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.676+0000 7f6be9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be41014c0 0x7f6be419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:38.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.676+0000 7f6be8a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6be4103df0 0x7f6be419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:38.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.676+0000 7f6be8a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6be4103df0 0x7f6be419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:41522/0 (socket says 172.21.15.117:41522) 2024-04-20T11:06:38.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.676+0000 7f6be8a00700 1 -- 172.21.15.117:0/1872251256 learned_addr learned my addr 172.21.15.117:0/1872251256 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:06:38.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.677+0000 7f6be8a00700 1 -- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be41014c0 msgr2=0x7f6be419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.677+0000 7f6be8a00700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be41014c0 0x7f6be419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.677+0000 7f6be8a00700 1 -- 172.21.15.117:0/1872251256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd00095f0 con 0x7f6be4103df0 2024-04-20T11:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.677+0000 7f6be8a00700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6be4103df0 0x7f6be419cf40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000bd70 tx=0x7f6bd000bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.678+0000 7f6be2000700 1 -- 172.21.15.117:0/1872251256 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bd001d070 con 0x7f6be4103df0 2024-04-20T11:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.678+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be41a20a0 con 0x7f6be4103df0 2024-04-20T11:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.678+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be41a2610 con 0x7f6be4103df0 2024-04-20T11:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.678+0000 7f6be2000700 1 -- 172.21.15.117:0/1872251256 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bd000f460 con 0x7f6be4103df0 2024-04-20T11:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.678+0000 7f6be2000700 1 -- 172.21.15.117:0/1872251256 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bd0021640 con 0x7f6be4103df0 2024-04-20T11:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.679+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc4005320 con 0x7f6be4103df0 2024-04-20T11:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.680+0000 7f6be2000700 1 -- 172.21.15.117:0/1872251256 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f6bd0003a60 con 0x7f6be4103df0 2024-04-20T11:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.681+0000 7f6be2000700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6bcc06cce0 0x7f6bcc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.681+0000 7f6be2000700 1 -- 172.21.15.117:0/1872251256 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6bd008d320 con 0x7f6be4103df0 2024-04-20T11:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.682+0000 7f6be9400700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6bcc06cce0 0x7f6bcc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:06:38.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.682+0000 7f6be9400700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6bcc06cce0 0x7f6bcc06f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8005950 tx=0x7f6bd800b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:06:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.690+0000 7f6be2000700 1 -- 172.21.15.117:0/1872251256 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6bd005b210 con 0x7f6be4103df0 2024-04-20T11:06:38.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.927+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bc4000bf0 con 0x7f6bcc06cce0 2024-04-20T11:06:38.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.928+0000 7f6be2000700 1 -- 172.21.15.117:0/1872251256 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6bc4000bf0 con 0x7f6bcc06cce0 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [], 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "0/2 daemons upgraded", 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 image on host smithi173", 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:06:38.928 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:06:38.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6bcc06cce0 msgr2=0x7f6bcc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6bcc06cce0 0x7f6bcc06f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8005950 tx=0x7f6bd800b410 comp rx=0 tx=0).stop 2024-04-20T11:06:38.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6be4103df0 msgr2=0x7f6be419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:06:38.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6be4103df0 0x7f6be419cf40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000bd70 tx=0x7f6bd000bda0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 shutdown_connections 2024-04-20T11:06:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f6bcc06cce0 0x7f6bcc06f1a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6be41014c0 0x7f6be419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 --2- 172.21.15.117:0/1872251256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6be4103df0 0x7f6be419cf40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:06:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 >> 172.21.15.117:0/1872251256 conn(0x7f6be40faef0 msgr2=0x7f6be4100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:06:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 shutdown_connections 2024-04-20T11:06:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:06:38.931+0000 7f6bea800700 1 -- 172.21.15.117:0/1872251256 wait complete. 2024-04-20T11:06:39.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:39 smithi117 ceph-mon[15659]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:06:39.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:39 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1230480332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:06:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:39 smithi173 ceph-mon[23800]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:06:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:39 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1230480332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:06:40.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:40 smithi173 ceph-mon[23800]: from='client.24321 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:06:40.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:40 smithi173 ceph-mon[23800]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:40.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:40 smithi117 ceph-mon[15659]: from='client.24321 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:06:40.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:40 smithi117 ceph-mon[15659]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:42.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:42 smithi173 ceph-mon[23800]: pgmap v289: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:43.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:42 smithi117 ceph-mon[15659]: pgmap v289: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:44 smithi173 ceph-mon[23800]: pgmap v290: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:44 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:06:45.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:44 smithi117 ceph-mon[15659]: pgmap v290: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:45.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:44 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:06:46.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:46 smithi173 ceph-mon[23800]: pgmap v291: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:46 smithi117 ceph-mon[15659]: pgmap v291: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:49.111 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:48 smithi117 ceph-mon[15659]: pgmap v292: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:49.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:48 smithi173 ceph-mon[23800]: pgmap v292: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:51.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:50 smithi117 ceph-mon[15659]: pgmap v293: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:51.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:50 smithi173 ceph-mon[23800]: pgmap v293: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:52.939 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:52 smithi117 ceph-mon[15659]: pgmap v294: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:52 smithi173 ceph-mon[23800]: pgmap v294: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:55.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:54 smithi117 ceph-mon[15659]: pgmap v295: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:55.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:54 smithi173 ceph-mon[23800]: pgmap v295: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:57.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:56 smithi117 ceph-mon[15659]: pgmap v296: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:57.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:56 smithi173 ceph-mon[23800]: pgmap v296: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:59.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:58 smithi117 ceph-mon[15659]: pgmap v297: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:59.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:06:58 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:06:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:58 smithi173 ceph-mon[23800]: pgmap v297: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:06:59.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:06:58 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:00.963 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:00 smithi173 ceph-mon[23800]: pgmap v298: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:01.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:00 smithi117 ceph-mon[15659]: pgmap v298: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:03.038 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:02 smithi173 ceph-mon[23800]: pgmap v299: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:03.125 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:02 smithi117 ceph-mon[15659]: pgmap v299: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:04.521 INFO:tasks.workunit.client.0.smithi117.stderr:Updating files: 61% (7522/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-20T11:07:05.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:04 smithi117 ceph-mon[15659]: pgmap v300: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:05.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:04 smithi173 ceph-mon[23800]: pgmap v300: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr: git switch -c 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-04-20T11:07:05.331 INFO:tasks.workunit.client.0.smithi117.stderr:Or undo this operation with: 2024-04-20T11:07:05.332 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-04-20T11:07:05.332 INFO:tasks.workunit.client.0.smithi117.stderr: git switch - 2024-04-20T11:07:05.332 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-04-20T11:07:05.332 INFO:tasks.workunit.client.0.smithi117.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T11:07:05.332 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-04-20T11:07:05.332 INFO:tasks.workunit.client.0.smithi117.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-20T11:07:05.345 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-20T11:07:05.407 INFO:tasks.workunit.client.0.smithi117.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T11:07:05.414 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T11:07:05.415 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T11:07:06.266 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T11:07:06.387 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T11:07:06.472 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T11:07:06.477 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T11:07:06.477 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T11:07:06.555 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T11:07:06.566 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T11:07:06.566 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-20T11:07:06.615 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-20T11:07:06.617 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-20T11:07:06.618 DEBUG:teuthology.orchestra.run.smithi117:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-20T11:07:06.749 INFO:tasks.workunit.client.0.smithi117.stderr:+ mkdir -p fsstress 2024-04-20T11:07:06.757 INFO:tasks.workunit.client.0.smithi117.stderr:+ pushd fsstress 2024-04-20T11:07:06.762 INFO:tasks.workunit.client.0.smithi117.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-20T11:07:06.762 INFO:tasks.workunit.client.0.smithi117.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-20T11:07:07.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:06 smithi117 ceph-mon[15659]: pgmap v301: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:07.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:06 smithi173 ceph-mon[23800]: pgmap v301: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:08.636 INFO:tasks.workunit.client.0.smithi117.stderr:+ tar xzf ltp-full.tgz 2024-04-20T11:07:08.763 INFO:tasks.workunit.client.1.smithi173.stderr:Updating files: 53% (6560/12271) Updating files: 54% (6627/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 84% (10340/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-20T11:07:09.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:08 smithi117 ceph-mon[15659]: pgmap v302: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:09.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.119+0000 7f8942800700 1 -- 172.21.15.117:0/3315733512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 msgr2=0x7f893c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:09.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.119+0000 7f8942800700 1 --2- 172.21.15.117:0/3315733512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 0x7f893c1020d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8928009910 tx=0x7f8928009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:09.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.119+0000 7f8942800700 1 -- 172.21.15.117:0/3315733512 shutdown_connections 2024-04-20T11:07:09.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.119+0000 7f8942800700 1 --2- 172.21.15.117:0/3315733512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 0x7f893c1020d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.119+0000 7f8942800700 1 --2- 172.21.15.117:0/3315733512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f893c100a70 0x7f893c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.119+0000 7f8942800700 1 -- 172.21.15.117:0/3315733512 >> 172.21.15.117:0/3315733512 conn(0x7f893c0fc050 msgr2=0x7f893c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:09.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.119+0000 7f8942800700 1 -- 172.21.15.117:0/3315733512 shutdown_connections 2024-04-20T11:07:09.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.119+0000 7f8942800700 1 -- 172.21.15.117:0/3315733512 wait complete. 2024-04-20T11:07:09.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.120+0000 7f8942800700 1 Processor -- start 2024-04-20T11:07:09.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.120+0000 7f8942800700 1 -- start start 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.121+0000 7f8942800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f893c100a70 0x7f893c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.121+0000 7f8942800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 0x7f893c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.121+0000 7f8942800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1991c0 con 0x7f893c101c70 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.121+0000 7f8942800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c199300 con 0x7f893c100a70 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.121+0000 7f8940a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 0x7f893c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.121+0000 7f8940a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 0x7f893c198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59230/0 (socket says 172.21.15.117:59230) 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.121+0000 7f8940a00700 1 -- 172.21.15.117:0/2820825512 learned_addr learned my addr 172.21.15.117:0/2820825512 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.121+0000 7f8941400700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f893c100a70 0x7f893c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.122+0000 7f8940a00700 1 -- 172.21.15.117:0/2820825512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f893c100a70 msgr2=0x7f893c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.122+0000 7f8940a00700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f893c100a70 0x7f893c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.122+0000 7f8940a00700 1 -- 172.21.15.117:0/2820825512 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89280095f0 con 0x7f893c101c70 2024-04-20T11:07:09.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.122+0000 7f8940a00700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 0x7f893c198ba0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f892800bd70 tx=0x7f892800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:09.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.122+0000 7f893a000700 1 -- 172.21.15.117:0/2820825512 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f892801d070 con 0x7f893c101c70 2024-04-20T11:07:09.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.122+0000 7f893a000700 1 -- 172.21.15.117:0/2820825512 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8928005dc0 con 0x7f893c101c70 2024-04-20T11:07:09.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.122+0000 7f893a000700 1 -- 172.21.15.117:0/2820825512 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8928021410 con 0x7f893c101c70 2024-04-20T11:07:09.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.122+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c19dd00 con 0x7f893c101c70 2024-04-20T11:07:09.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.123+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c19e270 con 0x7f893c101c70 2024-04-20T11:07:09.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.124+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c106640 con 0x7f893c101c70 2024-04-20T11:07:09.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.125+0000 7f893a000700 1 -- 172.21.15.117:0/2820825512 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f892800f820 con 0x7f893c101c70 2024-04-20T11:07:09.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.126+0000 7f893a000700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f892406ccf0 0x7f892406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:09.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.126+0000 7f893a000700 1 -- 172.21.15.117:0/2820825512 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f892808d4a0 con 0x7f893c101c70 2024-04-20T11:07:09.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.127+0000 7f8941400700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f892406ccf0 0x7f892406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:09.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.127+0000 7f8941400700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f892406ccf0 0x7f892406f1b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f893c101ad0 tx=0x7f8930009040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:09.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.132+0000 7f893a000700 1 -- 172.21.15.117:0/2820825512 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f892805b2e0 con 0x7f893c101c70 2024-04-20T11:07:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:08 smithi173 ceph-mon[23800]: pgmap v302: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:09.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.389+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f893c02cea0 con 0x7f892406ccf0 2024-04-20T11:07:09.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.390+0000 7f893a000700 1 -- 172.21.15.117:0/2820825512 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f893c02cea0 con 0x7f892406ccf0 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f892406ccf0 msgr2=0x7f892406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f892406ccf0 0x7f892406f1b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f893c101ad0 tx=0x7f8930009040 comp rx=0 tx=0).stop 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 msgr2=0x7f893c198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 0x7f893c198ba0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f892800bd70 tx=0x7f892800be50 comp rx=0 tx=0).stop 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 shutdown_connections 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f892406ccf0 0x7f892406f1b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f893c100a70 0x7f893c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 --2- 172.21.15.117:0/2820825512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893c101c70 0x7f893c198ba0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 >> 172.21.15.117:0/2820825512 conn(0x7f893c0fc050 msgr2=0x7f893c073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:09.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 shutdown_connections 2024-04-20T11:07:09.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.394+0000 7f8942800700 1 -- 172.21.15.117:0/2820825512 wait complete. 2024-04-20T11:07:09.416 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:07:09.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.580+0000 7ff980200700 1 -- 172.21.15.117:0/577380557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 msgr2=0x7ff978103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:09.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.580+0000 7ff980200700 1 --2- 172.21.15.117:0/577380557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 0x7ff978103a70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009910 tx=0x7ff96c009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:09.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.581+0000 7ff980200700 1 -- 172.21.15.117:0/577380557 shutdown_connections 2024-04-20T11:07:09.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.581+0000 7ff980200700 1 --2- 172.21.15.117:0/577380557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff978103fb0 0x7ff9781063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.581+0000 7ff980200700 1 --2- 172.21.15.117:0/577380557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 0x7ff978103a70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.581+0000 7ff980200700 1 -- 172.21.15.117:0/577380557 >> 172.21.15.117:0/577380557 conn(0x7ff9780fb0d0 msgr2=0x7ff9780fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:09.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.581+0000 7ff980200700 1 -- 172.21.15.117:0/577380557 shutdown_connections 2024-04-20T11:07:09.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.581+0000 7ff980200700 1 -- 172.21.15.117:0/577380557 wait complete. 2024-04-20T11:07:09.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.582+0000 7ff980200700 1 Processor -- start 2024-04-20T11:07:09.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.582+0000 7ff980200700 1 -- start start 2024-04-20T11:07:09.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.583+0000 7ff980200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 0x7ff978198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:09.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.583+0000 7ff980200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff978103fb0 0x7ff978198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:09.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.583+0000 7ff980200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9781992e0 con 0x7ff978101680 2024-04-20T11:07:09.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.583+0000 7ff980200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff978199420 con 0x7ff978103fb0 2024-04-20T11:07:09.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.583+0000 7ff97d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 0x7ff978198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:09.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.583+0000 7ff97d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 0x7ff978198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59246/0 (socket says 172.21.15.117:59246) 2024-04-20T11:07:09.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.583+0000 7ff97d800700 1 -- 172.21.15.117:0/2721937277 learned_addr learned my addr 172.21.15.117:0/2721937277 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:09.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.583+0000 7ff97ce00700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff978103fb0 0x7ff978198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:09.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.584+0000 7ff97d800700 1 -- 172.21.15.117:0/2721937277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff978103fb0 msgr2=0x7ff978198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:09.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.584+0000 7ff97d800700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff978103fb0 0x7ff978198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.584+0000 7ff97d800700 1 -- 172.21.15.117:0/2721937277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0095f0 con 0x7ff978101680 2024-04-20T11:07:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.584+0000 7ff97ce00700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff978103fb0 0x7ff978198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:07:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.584+0000 7ff97d800700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 0x7ff978198780 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b650 tx=0x7ff96c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.584+0000 7ff976000700 1 -- 172.21.15.117:0/2721937277 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96c01d070 con 0x7ff978101680 2024-04-20T11:07:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.584+0000 7ff976000700 1 -- 172.21.15.117:0/2721937277 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff96c0052a0 con 0x7ff978101680 2024-04-20T11:07:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.584+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97819ddb0 con 0x7ff978101680 2024-04-20T11:07:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.585+0000 7ff976000700 1 -- 172.21.15.117:0/2721937277 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96c017630 con 0x7ff978101680 2024-04-20T11:07:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.585+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97819e2c0 con 0x7ff978101680 2024-04-20T11:07:09.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.586+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff978062580 con 0x7ff978101680 2024-04-20T11:07:09.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.587+0000 7ff976000700 1 -- 172.21.15.117:0/2721937277 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ff96c003860 con 0x7ff978101680 2024-04-20T11:07:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.588+0000 7ff976000700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff96006cce0 0x7ff96006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.589+0000 7ff976000700 1 -- 172.21.15.117:0/2721937277 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff96c08d680 con 0x7ff978101680 2024-04-20T11:07:09.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.589+0000 7ff97ce00700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff96006cce0 0x7ff96006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:09.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.590+0000 7ff97ce00700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff96006cce0 0x7ff96006f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff964009cb0 tx=0x7ff964009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:09.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.594+0000 7ff976000700 1 -- 172.21.15.117:0/2721937277 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff978062580 con 0x7ff978101680 2024-04-20T11:07:09.738 INFO:tasks.workunit.client.1.smithi173.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-20T11:07:09.738 INFO:tasks.workunit.client.1.smithi173.stderr: 2024-04-20T11:07:09.738 INFO:tasks.workunit.client.1.smithi173.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T11:07:09.738 INFO:tasks.workunit.client.1.smithi173.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T11:07:09.738 INFO:tasks.workunit.client.1.smithi173.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T11:07:09.738 INFO:tasks.workunit.client.1.smithi173.stderr: 2024-04-20T11:07:09.738 INFO:tasks.workunit.client.1.smithi173.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr: 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr: git switch -c 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr: 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr:Or undo this operation with: 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr: 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr: git switch - 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr: 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr: 2024-04-20T11:07:09.739 INFO:tasks.workunit.client.1.smithi173.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-20T11:07:09.752 DEBUG:teuthology.orchestra.run.smithi173:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-20T11:07:09.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.894+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff978062510 con 0x7ff96006cce0 2024-04-20T11:07:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.896+0000 7ff976000700 1 -- 172.21.15.117:0/2721937277 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7ff978062510 con 0x7ff96006cce0 2024-04-20T11:07:09.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff96006cce0 msgr2=0x7ff96006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:09.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff96006cce0 0x7ff96006f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff964009cb0 tx=0x7ff964009260 comp rx=0 tx=0).stop 2024-04-20T11:07:09.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 msgr2=0x7ff978198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:09.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 0x7ff978198780 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b650 tx=0x7ff96c00f510 comp rx=0 tx=0).stop 2024-04-20T11:07:09.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 shutdown_connections 2024-04-20T11:07:09.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7ff96006cce0 0x7ff96006f1a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff978101680 0x7ff978198780 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 --2- 172.21.15.117:0/2721937277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff978103fb0 0x7ff978198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:09.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.899+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 >> 172.21.15.117:0/2721937277 conn(0x7ff9780fb0d0 msgr2=0x7ff978100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:09.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.900+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 shutdown_connections 2024-04-20T11:07:09.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:09.900+0000 7ff980200700 1 -- 172.21.15.117:0/2721937277 wait complete. 2024-04-20T11:07:09.905 INFO:tasks.workunit.client.1.smithi173.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T11:07:09.914 INFO:tasks.workunit.client.1.smithi173.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-20T11:07:09.915 INFO:tasks.workunit.client.1.smithi173.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T11:07:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.063+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3645985646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 msgr2=0x7f8a701038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.063+0000 7f8a6fe00700 1 --2- 172.21.15.117:0/3645985646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 0x7f8a701038b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009910 tx=0x7f8a60009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.063+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3645985646 shutdown_connections 2024-04-20T11:07:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.063+0000 7f8a6fe00700 1 --2- 172.21.15.117:0/3645985646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a70103df0 0x7f8a701061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.063+0000 7f8a6fe00700 1 --2- 172.21.15.117:0/3645985646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 0x7f8a701038b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.063+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3645985646 >> 172.21.15.117:0/3645985646 conn(0x7f8a700faef0 msgr2=0x7f8a700fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.063+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3645985646 shutdown_connections 2024-04-20T11:07:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.063+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3645985646 wait complete. 2024-04-20T11:07:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.064+0000 7f8a6fe00700 1 Processor -- start 2024-04-20T11:07:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.065+0000 7f8a6fe00700 1 -- start start 2024-04-20T11:07:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.065+0000 7f8a6fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 0x7f8a701986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.065+0000 7f8a6fe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a70103df0 0x7f8a70198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.065+0000 7f8a6fe00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70199210 con 0x7f8a701014c0 2024-04-20T11:07:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.065+0000 7f8a6fe00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70199350 con 0x7f8a70103df0 2024-04-20T11:07:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.065+0000 7f8a6ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 0x7f8a701986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.065+0000 7f8a6ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 0x7f8a701986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59268/0 (socket says 172.21.15.117:59268) 2024-04-20T11:07:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.065+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3644349541 learned_addr learned my addr 172.21.15.117:0/3644349541 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.066+0000 7f8a6e000700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a70103df0 0x7f8a70198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.066+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3644349541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a70103df0 msgr2=0x7f8a70198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.066+0000 7f8a6ea00700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a70103df0 0x7f8a70198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.066+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3644349541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600095f0 con 0x7f8a701014c0 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.066+0000 7f8a6ea00700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 0x7f8a701986b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000bd70 tx=0x7f8a6000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.067+0000 7f8a67400700 1 -- 172.21.15.117:0/3644349541 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a6001d070 con 0x7f8a701014c0 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.067+0000 7f8a67400700 1 -- 172.21.15.117:0/3644349541 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a6000f460 con 0x7f8a701014c0 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.067+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7019dcb0 con 0x7f8a701014c0 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.067+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7019e1a0 con 0x7f8a701014c0 2024-04-20T11:07:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.067+0000 7f8a67400700 1 -- 172.21.15.117:0/3644349541 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a60021620 con 0x7f8a701014c0 2024-04-20T11:07:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.068+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70062510 con 0x7f8a701014c0 2024-04-20T11:07:10.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.069+0000 7f8a67400700 1 -- 172.21.15.117:0/3644349541 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f8a6000fa70 con 0x7f8a701014c0 2024-04-20T11:07:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.070+0000 7f8a67400700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8a5406cce0 0x7f8a5406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.071+0000 7f8a67400700 1 -- 172.21.15.117:0/3644349541 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f8a6008d610 con 0x7f8a701014c0 2024-04-20T11:07:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.071+0000 7f8a6e000700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8a5406cce0 0x7f8a5406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:10.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.072+0000 7f8a6e000700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8a5406cce0 0x7f8a5406f1a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c005fd0 tx=0x7f8a5c00a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:10.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.075+0000 7f8a67400700 1 -- 172.21.15.117:0/3644349541 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8a6005b470 con 0x7f8a701014c0 2024-04-20T11:07:10.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.350+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a700ffc80 con 0x7f8a5406cce0 2024-04-20T11:07:10.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.386+0000 7f8a67400700 1 -- 172.21.15.117:0/3644349541 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8a700ffc80 con 0x7f8a5406cce0 2024-04-20T11:07:10.387 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:07:10.387 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (7m) 119s ago 9m 13.1M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:07:10.387 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (10m) 119s ago 10m 7532k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:07:10.387 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (8m) 2m ago 8m 7448k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:07:10.387 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (9m) 119s ago 9m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:07:10.387 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (8m) 2m ago 8m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (7m) 119s ago 9m 74.7M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (2m) 119s ago 2m 11.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (2m) 119s ago 2m 14.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (2m) 2m ago 2m 11.6M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (2m) 2m ago 2m 14.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:9283,8765,8443 running (11m) 119s ago 11m 500M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 2cf5112ba77d 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (8m) 2m ago 8m 449M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 88bcd8bcb785 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (11m) 119s ago 11m 45.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (8m) 2m ago 8m 34.8M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (9m) 119s ago 9m 7596k - 1.5.0 0da6a335fe13 6b69ffbf445a 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (8m) 2m ago 8m 7834k - 1.5.0 0da6a335fe13 81909a8618f8 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (7m) 119s ago 7m 54.1M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (6m) 119s ago 6m 47.5M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (6m) 119s ago 6m 45.1M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (5m) 2m ago 5m 51.7M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:07:10.388 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (4m) 2m ago 4m 50.0M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:07:10.389 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (4m) 2m ago 4m 51.0M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:07:10.389 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (7m) 119s ago 9m 49.8M - 2.43.0 a07b618ecd1d 532b98b31363 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8a5406cce0 msgr2=0x7f8a5406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8a5406cce0 0x7f8a5406f1a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c005fd0 tx=0x7f8a5c00a560 comp rx=0 tx=0).stop 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 msgr2=0x7f8a701986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 0x7f8a701986b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000bd70 tx=0x7f8a6000be50 comp rx=0 tx=0).stop 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 shutdown_connections 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f8a5406cce0 0x7f8a5406f1a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a701014c0 0x7f8a701986b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 --2- 172.21.15.117:0/3644349541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a70103df0 0x7f8a70198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 >> 172.21.15.117:0/3644349541 conn(0x7f8a700faef0 msgr2=0x7f8a700ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:10.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.390+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 shutdown_connections 2024-04-20T11:07:10.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.391+0000 7f8a6fe00700 1 -- 172.21.15.117:0/3644349541 wait complete. 2024-04-20T11:07:10.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.567+0000 7f275dc00700 1 -- 172.21.15.117:0/2643251949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758073940 msgr2=0x7f2758073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:10.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.567+0000 7f275dc00700 1 --2- 172.21.15.117:0/2643251949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758073940 0x7f2758073dc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2744009910 tx=0x7f2744009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:10.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.567+0000 7f275dc00700 1 -- 172.21.15.117:0/2643251949 shutdown_connections 2024-04-20T11:07:10.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.567+0000 7f275dc00700 1 --2- 172.21.15.117:0/2643251949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758073940 0x7f2758073dc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.567+0000 7f275dc00700 1 --2- 172.21.15.117:0/2643251949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2758074f10 0x7f2758073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.567+0000 7f275dc00700 1 -- 172.21.15.117:0/2643251949 >> 172.21.15.117:0/2643251949 conn(0x7f27580fc1c0 msgr2=0x7f27580fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:10.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.567+0000 7f275dc00700 1 -- 172.21.15.117:0/2643251949 shutdown_connections 2024-04-20T11:07:10.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.567+0000 7f275dc00700 1 -- 172.21.15.117:0/2643251949 wait complete. 2024-04-20T11:07:10.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.568+0000 7f275dc00700 1 Processor -- start 2024-04-20T11:07:10.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.569+0000 7f275dc00700 1 -- start start 2024-04-20T11:07:10.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.569+0000 7f275dc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2758073940 0x7f275819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:10.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.569+0000 7f275dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758074f10 0x7f275819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:10.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.569+0000 7f275dc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275819d780 con 0x7f2758074f10 2024-04-20T11:07:10.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.569+0000 7f275dc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275819d8c0 con 0x7f2758073940 2024-04-20T11:07:10.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.569+0000 7f2756000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758074f10 0x7f275819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:10.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.569+0000 7f2756000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758074f10 0x7f275819d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59288/0 (socket says 172.21.15.117:59288) 2024-04-20T11:07:10.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.569+0000 7f2756000700 1 -- 172.21.15.117:0/119365511 learned_addr learned my addr 172.21.15.117:0/119365511 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:10.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.570+0000 7f2756000700 1 -- 172.21.15.117:0/119365511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2758073940 msgr2=0x7f275819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:10.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.570+0000 7f2756a00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2758073940 0x7f275819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.570+0000 7f2756000700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2758073940 0x7f275819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.570+0000 7f2756000700 1 -- 172.21.15.117:0/119365511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27440095f0 con 0x7f2758074f10 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.570+0000 7f2756a00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2758073940 0x7f275819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.570+0000 7f2756000700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758074f10 0x7f275819d160 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f274400bd70 tx=0x7f274400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.571+0000 7f274f400700 1 -- 172.21.15.117:0/119365511 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f274401d070 con 0x7f2758074f10 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.571+0000 7f274f400700 1 -- 172.21.15.117:0/119365511 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f274400f460 con 0x7f2758074f10 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.571+0000 7f274f400700 1 -- 172.21.15.117:0/119365511 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2744021620 con 0x7f2758074f10 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.571+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27581a22c0 con 0x7f2758074f10 2024-04-20T11:07:10.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.571+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27581a27e0 con 0x7f2758074f10 2024-04-20T11:07:10.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.572+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2758062510 con 0x7f2758074f10 2024-04-20T11:07:10.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.573+0000 7f274f400700 1 -- 172.21.15.117:0/119365511 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f274400fa70 con 0x7f2758074f10 2024-04-20T11:07:10.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.574+0000 7f274f400700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f273c06cce0 0x7f273c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:10.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.575+0000 7f274f400700 1 -- 172.21.15.117:0/119365511 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f274408d610 con 0x7f2758074f10 2024-04-20T11:07:10.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.575+0000 7f2756a00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f273c06cce0 0x7f273c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:10.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.576+0000 7f2756a00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f273c06cce0 0x7f273c06f1a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2748005fd0 tx=0x7f2748005dc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:10.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.579+0000 7f274f400700 1 -- 172.21.15.117:0/119365511 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f274405b3d0 con 0x7f2758074f10 2024-04-20T11:07:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:10 smithi173 ceph-mon[23800]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:10 smithi173 ceph-mon[23800]: pgmap v303: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:10.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.920+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f27581a2c50 con 0x7f2758074f10 2024-04-20T11:07:10.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.921+0000 7f274f400700 1 -- 172.21.15.117:0/119365511 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f274405af60 con 0x7f2758074f10 2024-04-20T11:07:10.922 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:07:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:07:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:07:10.923 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:07:10.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f273c06cce0 msgr2=0x7f273c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:10.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f273c06cce0 0x7f273c06f1a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2748005fd0 tx=0x7f2748005dc0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758074f10 msgr2=0x7f275819d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:10.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758074f10 0x7f275819d160 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f274400bd70 tx=0x7f274400be50 comp rx=0 tx=0).stop 2024-04-20T11:07:10.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 shutdown_connections 2024-04-20T11:07:10.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f273c06cce0 0x7f273c06f1a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2758073940 0x7f275819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 --2- 172.21.15.117:0/119365511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758074f10 0x7f275819d160 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:10.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.924+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 >> 172.21.15.117:0/119365511 conn(0x7f27580fc1c0 msgr2=0x7f2758102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:10.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.925+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 shutdown_connections 2024-04-20T11:07:10.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:10.925+0000 7f275dc00700 1 -- 172.21.15.117:0/119365511 wait complete. 2024-04-20T11:07:11.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.116+0000 7f641ac00700 1 -- 172.21.15.117:0/1068411968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 msgr2=0x7f64141061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:11.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.116+0000 7f641ac00700 1 --2- 172.21.15.117:0/1068411968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 0x7f64141061e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6400009910 tx=0x7f6400009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:11.117 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:10 smithi117 ceph-mon[15659]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:11.117 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:10 smithi117 ceph-mon[15659]: pgmap v303: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:11.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.116+0000 7f641ac00700 1 -- 172.21.15.117:0/1068411968 shutdown_connections 2024-04-20T11:07:11.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.116+0000 7f641ac00700 1 --2- 172.21.15.117:0/1068411968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 0x7f64141061e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:11.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.116+0000 7f641ac00700 1 --2- 172.21.15.117:0/1068411968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64141014c0 0x7f64141038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:11.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.116+0000 7f641ac00700 1 -- 172.21.15.117:0/1068411968 >> 172.21.15.117:0/1068411968 conn(0x7f64140faef0 msgr2=0x7f64140fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:11.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.116+0000 7f641ac00700 1 -- 172.21.15.117:0/1068411968 shutdown_connections 2024-04-20T11:07:11.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.116+0000 7f641ac00700 1 -- 172.21.15.117:0/1068411968 wait complete. 2024-04-20T11:07:11.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.118+0000 7f641ac00700 1 Processor -- start 2024-04-20T11:07:11.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.118+0000 7f641ac00700 1 -- start start 2024-04-20T11:07:11.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.118+0000 7f641ac00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64141014c0 0x7f6414196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:11.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.118+0000 7f641ac00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 0x7f6414196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:11.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.118+0000 7f641ac00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6414196f60 con 0x7f6414103df0 2024-04-20T11:07:11.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.118+0000 7f641ac00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141970a0 con 0x7f64141014c0 2024-04-20T11:07:11.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.119+0000 7f6418e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 0x7f6414196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:11.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.119+0000 7f6418e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 0x7f6414196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59318/0 (socket says 172.21.15.117:59318) 2024-04-20T11:07:11.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.119+0000 7f6418e00700 1 -- 172.21.15.117:0/1683707920 learned_addr learned my addr 172.21.15.117:0/1683707920 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:11.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.119+0000 7f6419800700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64141014c0 0x7f6414196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:11.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.119+0000 7f6418e00700 1 -- 172.21.15.117:0/1683707920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64141014c0 msgr2=0x7f6414196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:11.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.119+0000 7f6418e00700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64141014c0 0x7f6414196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:11.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.119+0000 7f6418e00700 1 -- 172.21.15.117:0/1683707920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64000095f0 con 0x7f6414103df0 2024-04-20T11:07:11.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.120+0000 7f6419800700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64141014c0 0x7f6414196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:07:11.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.120+0000 7f6418e00700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 0x7f6414196940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6400006010 tx=0x7f640000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:11.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.120+0000 7f6412000700 1 -- 172.21.15.117:0/1683707920 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f640001d070 con 0x7f6414103df0 2024-04-20T11:07:11.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.120+0000 7f6412000700 1 -- 172.21.15.117:0/1683707920 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f640000f460 con 0x7f6414103df0 2024-04-20T11:07:11.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.120+0000 7f6412000700 1 -- 172.21.15.117:0/1683707920 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6400021640 con 0x7f6414103df0 2024-04-20T11:07:11.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.120+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641419baa0 con 0x7f6414103df0 2024-04-20T11:07:11.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.121+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641419c010 con 0x7f6414103df0 2024-04-20T11:07:11.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.122+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6414062510 con 0x7f6414103df0 2024-04-20T11:07:11.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.124+0000 7f6412000700 1 -- 172.21.15.117:0/1683707920 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f640000fad0 con 0x7f6414103df0 2024-04-20T11:07:11.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.125+0000 7f6412000700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f63fc06cce0 0x7f63fc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:11.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.125+0000 7f6412000700 1 -- 172.21.15.117:0/1683707920 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f640008d5c0 con 0x7f6414103df0 2024-04-20T11:07:11.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.125+0000 7f6419800700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f63fc06cce0 0x7f63fc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:11.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.126+0000 7f6419800700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f63fc06cce0 0x7f63fc06f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6408005950 tx=0x7f6408005f50 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:11.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.131+0000 7f6412000700 1 -- 172.21.15.117:0/1683707920 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f640005b400 con 0x7f6414103df0 2024-04-20T11:07:11.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.419+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 --> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6414105d00 con 0x7f63fc06cce0 2024-04-20T11:07:11.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.421+0000 7f6412000700 1 -- 172.21.15.117:0/1683707920 <== mgr.14215 v2:172.21.15.117:6800/2612478075 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6414105d00 con 0x7f63fc06cce0 2024-04-20T11:07:11.422 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:07:11.422 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:07:11.422 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:07:11.422 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:07:11.423 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [], 2024-04-20T11:07:11.423 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "0/2 daemons upgraded", 2024-04-20T11:07:11.423 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 image on host smithi173", 2024-04-20T11:07:11.423 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:07:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:07:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.424+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f63fc06cce0 msgr2=0x7f63fc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.424+0000 7f641ac00700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f63fc06cce0 0x7f63fc06f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6408005950 tx=0x7f6408005f50 comp rx=0 tx=0).stop 2024-04-20T11:07:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.424+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 msgr2=0x7f6414196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.424+0000 7f641ac00700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 0x7f6414196940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6400006010 tx=0x7f640000be20 comp rx=0 tx=0).stop 2024-04-20T11:07:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.425+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 shutdown_connections 2024-04-20T11:07:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.425+0000 7f641ac00700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f63fc06cce0 0x7f63fc06f1a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.425+0000 7f641ac00700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64141014c0 0x7f6414196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.425+0000 7f641ac00700 1 --2- 172.21.15.117:0/1683707920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6414103df0 0x7f6414196940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.425+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 >> 172.21.15.117:0/1683707920 conn(0x7f64140faef0 msgr2=0x7f64140fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.425+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 shutdown_connections 2024-04-20T11:07:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:11.425+0000 7f641ac00700 1 -- 172.21.15.117:0/1683707920 wait complete. 2024-04-20T11:07:11.971 INFO:tasks.workunit.client.1.smithi173.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T11:07:12.055 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:11 smithi173 ceph-mon[23800]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:12.055 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:11 smithi173 ceph-mon[23800]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:12.055 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:11 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/119365511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:07:12.088 INFO:tasks.workunit.client.1.smithi173.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T11:07:12.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:11 smithi117 ceph-mon[15659]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:11 smithi117 ceph-mon[15659]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:12.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:11 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/119365511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:07:12.160 INFO:tasks.workunit.client.1.smithi173.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-20T11:07:12.164 INFO:tasks.workunit.client.1.smithi173.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-20T11:07:12.164 INFO:tasks.workunit.client.1.smithi173.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T11:07:12.239 INFO:tasks.workunit.client.1.smithi173.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-20T11:07:12.246 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-20T11:07:12.246 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-20T11:07:12.285 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-20T11:07:12.286 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-20T11:07:12.286 DEBUG:teuthology.orchestra.run.smithi173:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-20T11:07:12.403 INFO:tasks.workunit.client.1.smithi173.stderr:+ mkdir -p fsstress 2024-04-20T11:07:12.410 INFO:tasks.workunit.client.1.smithi173.stderr:+ pushd fsstress 2024-04-20T11:07:12.412 INFO:tasks.workunit.client.1.smithi173.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-20T11:07:12.413 INFO:tasks.workunit.client.1.smithi173.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-20T11:07:13.008 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:12 smithi173 ceph-mon[23800]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:13.008 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:12 smithi173 ceph-mon[23800]: pgmap v304: 65 pgs: 65 active+clean; 11 MiB data, 214 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 919 KiB/s wr, 5 op/s 2024-04-20T11:07:13.008 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:13.008 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:07:13.008 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:07:13.008 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:12 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:07:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:12 smithi117 ceph-mon[15659]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:12 smithi117 ceph-mon[15659]: pgmap v304: 65 pgs: 65 active+clean; 11 MiB data, 214 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 919 KiB/s wr, 5 op/s 2024-04-20T11:07:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:07:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:07:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:12 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:07:13.442 INFO:tasks.workunit.client.1.smithi173.stderr:+ tar xzf ltp-full.tgz 2024-04-20T11:07:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:13 smithi173 ceph-mon[23800]: Upgrade: Updating mgr.smithi173.gnrjof 2024-04-20T11:07:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:13 smithi173 ceph-mon[23800]: Deploying daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T11:07:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:13 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:13 smithi117 ceph-mon[15659]: Upgrade: Updating mgr.smithi173.gnrjof 2024-04-20T11:07:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:13 smithi117 ceph-mon[15659]: Deploying daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T11:07:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:13 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:14 smithi173 ceph-mon[23800]: pgmap v305: 65 pgs: 65 active+clean; 12 MiB data, 223 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 948 KiB/s wr, 8 op/s 2024-04-20T11:07:15.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:14 smithi117 ceph-mon[15659]: pgmap v305: 65 pgs: 65 active+clean; 12 MiB data, 223 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 948 KiB/s wr, 8 op/s 2024-04-20T11:07:17.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:16 smithi117 ceph-mon[15659]: pgmap v306: 65 pgs: 65 active+clean; 18 MiB data, 246 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 23 op/s 2024-04-20T11:07:17.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:16 smithi173 ceph-mon[23800]: pgmap v306: 65 pgs: 65 active+clean; 18 MiB data, 246 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 23 op/s 2024-04-20T11:07:18.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:18 smithi117 ceph-mon[15659]: pgmap v307: 65 pgs: 65 active+clean; 25 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 31 op/s 2024-04-20T11:07:19.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:18 smithi173 ceph-mon[23800]: pgmap v307: 65 pgs: 65 active+clean; 25 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 31 op/s 2024-04-20T11:07:20.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:20 smithi117 ceph-mon[15659]: pgmap v308: 65 pgs: 65 active+clean; 25 MiB data, 268 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 33 op/s 2024-04-20T11:07:21.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:20 smithi173 ceph-mon[23800]: pgmap v308: 65 pgs: 65 active+clean; 25 MiB data, 268 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 33 op/s 2024-04-20T11:07:23.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:22 smithi117 ceph-mon[15659]: pgmap v309: 65 pgs: 65 active+clean; 29 MiB data, 280 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 56 op/s 2024-04-20T11:07:23.140 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:22 smithi173 ceph-mon[23800]: pgmap v309: 65 pgs: 65 active+clean; 29 MiB data, 280 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 56 op/s 2024-04-20T11:07:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:24 smithi173 ceph-mon[23800]: pgmap v310: 65 pgs: 65 active+clean; 30 MiB data, 282 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 59 op/s 2024-04-20T11:07:25.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:24 smithi117 ceph-mon[15659]: pgmap v310: 65 pgs: 65 active+clean; 30 MiB data, 282 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 59 op/s 2024-04-20T11:07:27.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:26 smithi117 ceph-mon[15659]: pgmap v311: 65 pgs: 65 active+clean; 32 MiB data, 292 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 72 op/s 2024-04-20T11:07:27.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:26 smithi173 ceph-mon[23800]: pgmap v311: 65 pgs: 65 active+clean; 32 MiB data, 292 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 72 op/s 2024-04-20T11:07:28.979 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:28 smithi173 ceph-mon[23800]: pgmap v312: 65 pgs: 65 active+clean; 35 MiB data, 304 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.5 MiB/s wr, 68 op/s 2024-04-20T11:07:28.980 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:28 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:29.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:28 smithi117 ceph-mon[15659]: pgmap v312: 65 pgs: 65 active+clean; 35 MiB data, 304 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.5 MiB/s wr, 68 op/s 2024-04-20T11:07:29.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:28 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:30.338 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:30.339 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:30.339 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:29 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:07:30.339 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:29 smithi173 ceph-mon[23800]: pgmap v313: 65 pgs: 65 active+clean; 35 MiB data, 309 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 960 KiB/s wr, 62 op/s 2024-04-20T11:07:30.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:30.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:30.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:29 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:07:30.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:29 smithi117 ceph-mon[15659]: pgmap v313: 65 pgs: 65 active+clean; 35 MiB data, 309 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 960 KiB/s wr, 62 op/s 2024-04-20T11:07:33.083 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:32 smithi173 ceph-mon[23800]: pgmap v314: 65 pgs: 65 active+clean; 43 MiB data, 324 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.5 MiB/s wr, 78 op/s 2024-04-20T11:07:33.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:32 smithi117 ceph-mon[15659]: pgmap v314: 65 pgs: 65 active+clean; 43 MiB data, 324 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.5 MiB/s wr, 78 op/s 2024-04-20T11:07:35.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:34 smithi117 ceph-mon[15659]: pgmap v315: 65 pgs: 65 active+clean; 43 MiB data, 324 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 62 op/s 2024-04-20T11:07:35.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:35.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:34 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:34 smithi173 ceph-mon[23800]: pgmap v315: 65 pgs: 65 active+clean; 43 MiB data, 324 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 62 op/s 2024-04-20T11:07:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:34 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:37.001 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:36 smithi173 ceph-mon[23800]: pgmap v316: 65 pgs: 65 active+clean; 49 MiB data, 351 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 76 op/s 2024-04-20T11:07:37.002 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:36 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:37.002 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:36 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:37.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:36 smithi117 ceph-mon[15659]: pgmap v316: 65 pgs: 65 active+clean; 49 MiB data, 351 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 76 op/s 2024-04-20T11:07:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:36 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:39.059 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:38 smithi173 ceph-mon[23800]: pgmap v317: 65 pgs: 65 active+clean; 50 MiB data, 355 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 70 op/s 2024-04-20T11:07:39.110 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:38 smithi117 ceph-mon[15659]: pgmap v317: 65 pgs: 65 active+clean; 50 MiB data, 355 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 70 op/s 2024-04-20T11:07:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:39 smithi117 ceph-mon[15659]: Standby manager daemon smithi173.gnrjof restarted 2024-04-20T11:07:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:39 smithi117 ceph-mon[15659]: Standby manager daemon smithi173.gnrjof started 2024-04-20T11:07:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:39 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T11:07:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:39 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:07:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:39 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T11:07:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:39 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:07:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:39 smithi173 ceph-mon[23800]: Standby manager daemon smithi173.gnrjof restarted 2024-04-20T11:07:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:39 smithi173 ceph-mon[23800]: Standby manager daemon smithi173.gnrjof started 2024-04-20T11:07:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:39 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T11:07:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:39 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:07:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:39 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T11:07:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:39 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:07:41.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:40 smithi117 ceph-mon[15659]: pgmap v318: 65 pgs: 65 active+clean; 50 MiB data, 365 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 62 op/s 2024-04-20T11:07:41.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:40 smithi117 ceph-mon[15659]: mgrmap e20: smithi117.ivnklj(active, since 9m), standbys: smithi173.gnrjof 2024-04-20T11:07:41.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:40 smithi173 ceph-mon[23800]: pgmap v318: 65 pgs: 65 active+clean; 50 MiB data, 365 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 62 op/s 2024-04-20T11:07:41.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:40 smithi173 ceph-mon[23800]: mgrmap e20: smithi117.ivnklj(active, since 9m), standbys: smithi173.gnrjof 2024-04-20T11:07:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.656+0000 7f87ad200700 1 -- 172.21.15.117:0/1043516232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8101e00 msgr2=0x7f87a8102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.656+0000 7f87ad200700 1 --2- 172.21.15.117:0/1043516232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8101e00 0x7f87a8102260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8798009910 tx=0x7f8798009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.657+0000 7f87ad200700 1 -- 172.21.15.117:0/1043516232 shutdown_connections 2024-04-20T11:07:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.657+0000 7f87ad200700 1 --2- 172.21.15.117:0/1043516232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8101e00 0x7f87a8102260 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:41.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.657+0000 7f87ad200700 1 --2- 172.21.15.117:0/1043516232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87a8100c00 0x7f87a8101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:41.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.657+0000 7f87ad200700 1 -- 172.21.15.117:0/1043516232 >> 172.21.15.117:0/1043516232 conn(0x7f87a80fc1c0 msgr2=0x7f87a80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:41.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.657+0000 7f87ad200700 1 -- 172.21.15.117:0/1043516232 shutdown_connections 2024-04-20T11:07:41.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.657+0000 7f87ad200700 1 -- 172.21.15.117:0/1043516232 wait complete. 2024-04-20T11:07:41.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.658+0000 7f87ad200700 1 Processor -- start 2024-04-20T11:07:41.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.659+0000 7f87ad200700 1 -- start start 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.659+0000 7f87ad200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8100c00 0x7f87a8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.659+0000 7f87ad200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87a8101e00 0x7f87a8198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.659+0000 7f87ad200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a8199380 con 0x7f87a8100c00 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.659+0000 7f87ad200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81994c0 con 0x7f87a8101e00 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.659+0000 7f87a5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8100c00 0x7f87a8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.659+0000 7f87a5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8100c00 0x7f87a8198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58502/0 (socket says 172.21.15.117:58502) 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.659+0000 7f87a5e00700 1 -- 172.21.15.117:0/1414450710 learned_addr learned my addr 172.21.15.117:0/1414450710 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.660+0000 7f87a5400700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87a8101e00 0x7f87a8198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.660+0000 7f87a5e00700 1 -- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87a8101e00 msgr2=0x7f87a8198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.660+0000 7f87a5e00700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87a8101e00 0x7f87a8198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.660+0000 7f87a5e00700 1 -- 172.21.15.117:0/1414450710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980095f0 con 0x7f87a8100c00 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.660+0000 7f87a5e00700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8100c00 0x7f87a8198820 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f879000e8f0 tx=0x7f879000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.661+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8790009790 con 0x7f87a8100c00 2024-04-20T11:07:41.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.661+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87900044e0 con 0x7f87a8100c00 2024-04-20T11:07:41.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.661+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8790003e00 con 0x7f87a8100c00 2024-04-20T11:07:41.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.661+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a819df20 con 0x7f87a8100c00 2024-04-20T11:07:41.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.661+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a819e460 con 0x7f87a8100c00 2024-04-20T11:07:41.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.663+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a81067d0 con 0x7f87a8100c00 2024-04-20T11:07:41.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.664+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) v1 ==== 91730+0+0 (secure 0 0 0) 0x7f879000cba0 con 0x7f87a8100c00 2024-04-20T11:07:41.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.665+0000 7f879ea00700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878c06db10 0x7f878c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:41.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.665+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f8790014070 con 0x7f87a8100c00 2024-04-20T11:07:41.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.666+0000 7f87a5400700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878c06db10 0x7f878c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:41.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.667+0000 7f87a5400700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878c06db10 0x7f878c06ffd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f879800b800 tx=0x7f8798009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:41.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.671+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f879005b620 con 0x7f87a8100c00 2024-04-20T11:07:41.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.760+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 21) v1 ==== 46228+0+0 (secure 0 0 0) 0x7f8790057f60 con 0x7f87a8100c00 2024-04-20T11:07:41.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.760+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878c06db10 msgr2=0x7f878c06ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:41.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:41.760+0000 7f879ea00700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878c06db10 0x7f878c06ffd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f879800b800 tx=0x7f8798009f70 comp rx=0 tx=0).stop 2024-04-20T11:07:42.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.764+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mgrmap(e 22) v1 ==== 46355+0+0 (secure 0 0 0) 0x7f879007ccd0 con 0x7f87a8100c00 2024-04-20T11:07:42.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.764+0000 7f879ea00700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f878c005d60 0x7f878c034de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:42.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.764+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 --> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f878c036de0 con 0x7f878c005d60 2024-04-20T11:07:42.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:42.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:42.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:07:42.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:07:42.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:42.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:07:42.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr fail", "who": "smithi117.ivnklj"}]: dispatch 2024-04-20T11:07:42.766 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: Activating manager daemon smithi173.gnrjof 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: osdmap e48: 6 total, 6 up, 6 in 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "mgr fail", "who": "smithi117.ivnklj"}]': finished 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: mgrmap e21: smithi173.gnrjof(active, starting, since 0.0157242s) 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr metadata", "who": "smithi173.gnrjof", "id": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:07:42.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: Manager daemon smithi173.gnrjof is now available 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi173.gnrjof/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi173.gnrjof/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi173.gnrjof/trash_purge_schedule"}]: dispatch 2024-04-20T11:07:42.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:42 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi173.gnrjof/trash_purge_schedule"}]: dispatch 2024-04-20T11:07:42.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.766+0000 7f87a5400700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f878c005d60 0x7f878c034de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:42.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.785+0000 7f87a5400700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f878c005d60 0x7f878c034de0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8798005850 tx=0x7f8798022000 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:42.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.789+0000 7f879ea00700 1 -- 172.21.15.117:0/1414450710 <== mgr.24337 v2:172.21.15.173:6828/2992581883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f878c036de0 con 0x7f878c005d60 2024-04-20T11:07:42.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f878c005d60 msgr2=0x7f878c034de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:42.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f878c005d60 0x7f878c034de0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8798005850 tx=0x7f8798022000 comp rx=0 tx=0).stop 2024-04-20T11:07:42.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8100c00 msgr2=0x7f87a8198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:42.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8100c00 0x7f87a8198820 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f879000e8f0 tx=0x7f879000ecb0 comp rx=0 tx=0).stop 2024-04-20T11:07:42.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 shutdown_connections 2024-04-20T11:07:42.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f878c005d60 0x7f878c034de0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:42.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:6800/2612478075,v1:172.21.15.117:6801/2612478075] conn(0x7f878c06db10 0x7f878c06ffd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:42.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a8100c00 0x7f87a8198820 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:42.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 --2- 172.21.15.117:0/1414450710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87a8101e00 0x7f87a8198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:42.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.793+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 >> 172.21.15.117:0/1414450710 conn(0x7f87a80fc1c0 msgr2=0x7f87a8073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:42.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.794+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 shutdown_connections 2024-04-20T11:07:42.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.794+0000 7f87ad200700 1 -- 172.21.15.117:0/1414450710 wait complete. 2024-04-20T11:07:42.813 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:07:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:07:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:07:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' 2024-04-20T11:07:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:07:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr fail", "who": "smithi117.ivnklj"}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: Activating manager daemon smithi173.gnrjof 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: osdmap e48: 6 total, 6 up, 6 in 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.14215 172.21.15.117:0/2254131621' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "mgr fail", "who": "smithi117.ivnklj"}]': finished 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: mgrmap e21: smithi173.gnrjof(active, starting, since 0.0157242s) 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr metadata", "who": "smithi173.gnrjof", "id": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:07:42.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: Manager daemon smithi173.gnrjof is now available 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi173.gnrjof/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi173.gnrjof/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi173.gnrjof/trash_purge_schedule"}]: dispatch 2024-04-20T11:07:42.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:42 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi173.gnrjof/trash_purge_schedule"}]: dispatch 2024-04-20T11:07:42.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.995+0000 7f4b43a00700 1 -- 172.21.15.117:0/2354252160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c100c60 msgr2=0x7f4b3c101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:42.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.995+0000 7f4b43a00700 1 --2- 172.21.15.117:0/2354252160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c100c60 0x7f4b3c101080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009910 tx=0x7f4b30009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:42.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.995+0000 7f4b43a00700 1 -- 172.21.15.117:0/2354252160 shutdown_connections 2024-04-20T11:07:42.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.995+0000 7f4b43a00700 1 --2- 172.21.15.117:0/2354252160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c101e60 0x7f4b3c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:42.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.995+0000 7f4b43a00700 1 --2- 172.21.15.117:0/2354252160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c100c60 0x7f4b3c101080 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:42.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.995+0000 7f4b43a00700 1 -- 172.21.15.117:0/2354252160 >> 172.21.15.117:0/2354252160 conn(0x7f4b3c0fc1e0 msgr2=0x7f4b3c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.996+0000 7f4b43a00700 1 -- 172.21.15.117:0/2354252160 shutdown_connections 2024-04-20T11:07:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.996+0000 7f4b43a00700 1 -- 172.21.15.117:0/2354252160 wait complete. 2024-04-20T11:07:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.997+0000 7f4b43a00700 1 Processor -- start 2024-04-20T11:07:42.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.997+0000 7f4b43a00700 1 -- start start 2024-04-20T11:07:42.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.997+0000 7f4b43a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c100c60 0x7f4b3c074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:42.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.997+0000 7f4b43a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c101e60 0x7f4b3c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:42.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.997+0000 7f4b43a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c19a270 con 0x7f4b3c101e60 2024-04-20T11:07:42.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.997+0000 7f4b43a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c19a3b0 con 0x7f4b3c100c60 2024-04-20T11:07:42.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.998+0000 7f4b3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c101e60 0x7f4b3c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:42.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.998+0000 7f4b3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c101e60 0x7f4b3c0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58552/0 (socket says 172.21.15.117:58552) 2024-04-20T11:07:42.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.998+0000 7f4b3be00700 1 -- 172.21.15.117:0/1231569793 learned_addr learned my addr 172.21.15.117:0/1231569793 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.998+0000 7f4b41000700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c100c60 0x7f4b3c074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.998+0000 7f4b3be00700 1 -- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c100c60 msgr2=0x7f4b3c074c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.998+0000 7f4b3be00700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c100c60 0x7f4b3c074c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.998+0000 7f4b3be00700 1 -- 172.21.15.117:0/1231569793 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300095f0 con 0x7f4b3c101e60 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.999+0000 7f4b3be00700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c101e60 0x7f4b3c0732d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e920 tx=0x7f4b2800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.999+0000 7f4b39600700 1 -- 172.21.15.117:0/1231569793 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2800cba0 con 0x7f4b3c101e60 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.999+0000 7f4b39600700 1 -- 172.21.15.117:0/1231569793 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b2800cd00 con 0x7f4b3c101e60 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.999+0000 7f4b39600700 1 -- 172.21.15.117:0/1231569793 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b28018770 con 0x7f4b3c101e60 2024-04-20T11:07:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:42.999+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c19a690 con 0x7f4b3c101e60 2024-04-20T11:07:43.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.000+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c19ac60 con 0x7f4b3c101e60 2024-04-20T11:07:43.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.001+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c04ebb0 con 0x7f4b3c101e60 2024-04-20T11:07:43.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.001+0000 7f4b39600700 1 -- 172.21.15.117:0/1231569793 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) v1 ==== 46355+0+0 (secure 0 0 0) 0x7f4b28010430 con 0x7f4b3c101e60 2024-04-20T11:07:43.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.002+0000 7f4b39600700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f4b24039540 0x7f4b2403ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:43.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.002+0000 7f4b39600700 1 -- 172.21.15.117:0/1231569793 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f4b28014070 con 0x7f4b3c101e60 2024-04-20T11:07:43.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.003+0000 7f4b41000700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f4b24039540 0x7f4b2403ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:43.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.004+0000 7f4b41000700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f4b24039540 0x7f4b2403ba00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b800 tx=0x7f4b30009f70 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:43.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.010+0000 7f4b39600700 1 -- 172.21.15.117:0/1231569793 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f4b28016740 con 0x7f4b3c101e60 2024-04-20T11:07:43.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.347+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 --> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b3c073810 con 0x7f4b24039540 2024-04-20T11:07:43.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.351+0000 7f4b39600700 1 -- 172.21.15.117:0/1231569793 <== mgr.24337 v2:172.21.15.173:6828/2992581883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f4b3c073810 con 0x7f4b24039540 2024-04-20T11:07:43.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.354+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f4b24039540 msgr2=0x7f4b2403ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:43.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.354+0000 7f4b43a00700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f4b24039540 0x7f4b2403ba00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b800 tx=0x7f4b30009f70 comp rx=0 tx=0).stop 2024-04-20T11:07:43.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.354+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c101e60 msgr2=0x7f4b3c0732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:43.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.354+0000 7f4b43a00700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c101e60 0x7f4b3c0732d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e920 tx=0x7f4b2800ece0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.355+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 shutdown_connections 2024-04-20T11:07:43.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.355+0000 7f4b43a00700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f4b24039540 0x7f4b2403ba00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.355+0000 7f4b43a00700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c100c60 0x7f4b3c074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.355+0000 7f4b43a00700 1 --2- 172.21.15.117:0/1231569793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c101e60 0x7f4b3c0732d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.355+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 >> 172.21.15.117:0/1231569793 conn(0x7f4b3c0fc1e0 msgr2=0x7f4b3c105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:43.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.355+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 shutdown_connections 2024-04-20T11:07:43.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.355+0000 7f4b43a00700 1 -- 172.21.15.117:0/1231569793 wait complete. 2024-04-20T11:07:43.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.550+0000 7f41aa000700 1 -- 172.21.15.117:0/1699350906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4073940 msgr2=0x7f41a4073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:43.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.550+0000 7f41aa000700 1 --2- 172.21.15.117:0/1699350906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4073940 0x7f41a4073dc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4190009910 tx=0x7f4190009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:43.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.550+0000 7f41aa000700 1 -- 172.21.15.117:0/1699350906 shutdown_connections 2024-04-20T11:07:43.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.550+0000 7f41aa000700 1 --2- 172.21.15.117:0/1699350906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4073940 0x7f41a4073dc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.550+0000 7f41aa000700 1 --2- 172.21.15.117:0/1699350906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a4074f10 0x7f41a4073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.550+0000 7f41aa000700 1 -- 172.21.15.117:0/1699350906 >> 172.21.15.117:0/1699350906 conn(0x7f41a40fc000 msgr2=0x7f41a40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:43.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.550+0000 7f41aa000700 1 -- 172.21.15.117:0/1699350906 shutdown_connections 2024-04-20T11:07:43.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.550+0000 7f41aa000700 1 -- 172.21.15.117:0/1699350906 wait complete. 2024-04-20T11:07:43.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.551+0000 7f41aa000700 1 Processor -- start 2024-04-20T11:07:43.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.552+0000 7f41aa000700 1 -- start start 2024-04-20T11:07:43.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.552+0000 7f41aa000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a4073940 0x7f41a419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:43.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.552+0000 7f41aa000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4074f10 0x7f41a419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:43.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.552+0000 7f41aa000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a419d5f0 con 0x7f41a4074f10 2024-04-20T11:07:43.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.552+0000 7f41aa000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a419d730 con 0x7f41a4073940 2024-04-20T11:07:43.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.553+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4074f10 0x7f41a419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:43.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.553+0000 7f41a8c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a4073940 0x7f41a419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:43.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.553+0000 7f41a3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4074f10 0x7f41a419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58574/0 (socket says 172.21.15.117:58574) 2024-04-20T11:07:43.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.553+0000 7f41a3e00700 1 -- 172.21.15.117:0/680579857 learned_addr learned my addr 172.21.15.117:0/680579857 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:43.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.553+0000 7f41a3e00700 1 -- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a4073940 msgr2=0x7f41a419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:43.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.553+0000 7f41a3e00700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a4073940 0x7f41a419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.553+0000 7f41a3e00700 1 -- 172.21.15.117:0/680579857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41900095f0 con 0x7f41a4074f10 2024-04-20T11:07:43.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.554+0000 7f41a3e00700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4074f10 0x7f41a419cfd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f41900059d0 tx=0x7f419000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:43.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.554+0000 7f41a1600700 1 -- 172.21.15.117:0/680579857 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f419001d070 con 0x7f41a4074f10 2024-04-20T11:07:43.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.554+0000 7f41a1600700 1 -- 172.21.15.117:0/680579857 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4190005dc0 con 0x7f41a4074f10 2024-04-20T11:07:43.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.554+0000 7f41a1600700 1 -- 172.21.15.117:0/680579857 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4190021410 con 0x7f41a4074f10 2024-04-20T11:07:43.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.554+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a41a2130 con 0x7f41a4074f10 2024-04-20T11:07:43.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.555+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a41a26a0 con 0x7f41a4074f10 2024-04-20T11:07:43.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.556+0000 7f41a1600700 1 -- 172.21.15.117:0/680579857 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) v1 ==== 46355+0+0 (secure 0 0 0) 0x7f4190003a60 con 0x7f41a4074f10 2024-04-20T11:07:43.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.556+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a4196bf0 con 0x7f41a4074f10 2024-04-20T11:07:43.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.557+0000 7f41a1600700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f418c0394b0 0x7f418c03b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:43.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.557+0000 7f41a1600700 1 -- 172.21.15.117:0/680579857 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f419004e400 con 0x7f41a4074f10 2024-04-20T11:07:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.557+0000 7f41a8c00700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f418c0394b0 0x7f418c03b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:43.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.558+0000 7f41a8c00700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f418c0394b0 0x7f418c03b970 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4198005950 tx=0x7f419800b410 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:43.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.563+0000 7f41a1600700 1 -- 172.21.15.117:0/680579857 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f4190016590 con 0x7f41a4074f10 2024-04-20T11:07:43.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:43 smithi117 ceph-mon[15659]: [20/Apr/2024:11:07:42] ENGINE Bus STARTING 2024-04-20T11:07:43.767 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:43 smithi117 ceph-mon[15659]: [20/Apr/2024:11:07:42] ENGINE Serving on https://172.21.15.173:7150 2024-04-20T11:07:43.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:43 smithi117 ceph-mon[15659]: [20/Apr/2024:11:07:42] ENGINE Serving on http://172.21.15.173:8765 2024-04-20T11:07:43.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:43 smithi117 ceph-mon[15659]: [20/Apr/2024:11:07:42] ENGINE Bus STARTED 2024-04-20T11:07:43.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:43 smithi117 ceph-mon[15659]: Deploying cephadm binary to smithi173 2024-04-20T11:07:43.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:43 smithi117 ceph-mon[15659]: Deploying cephadm binary to smithi117 2024-04-20T11:07:43.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:43 smithi117 ceph-mon[15659]: mgrmap e22: smithi173.gnrjof(active, since 1.01707s) 2024-04-20T11:07:43.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.773+0000 7f41a1600700 1 -- 172.21.15.117:0/680579857 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 23) v1 ==== 46651+0+0 (secure 0 0 0) 0x7f419002b440 con 0x7f41a4074f10 2024-04-20T11:07:43.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.930+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 --> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f41a402cdb0 con 0x7f418c0394b0 2024-04-20T11:07:43.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.982+0000 7f41a1600700 1 -- 172.21.15.117:0/680579857 <== mgr.24337 v2:172.21.15.173:6828/2992581883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f41a402cdb0 con 0x7f418c0394b0 2024-04-20T11:07:43.982 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:07:43.982 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (8m) 2m ago 10m 13.1M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:07:43.982 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (10m) 2m ago 10m 7532k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:07:43.982 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (9m) 9s ago 9m 7650k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:07:43.982 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (10m) 2m ago 10m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:07:43.982 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (9m) 9s ago 9m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (8m) 2m ago 9m 74.7M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (2m) 2m ago 2m 11.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (3m) 2m ago 3m 14.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (2m) 9s ago 2m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (3m) 9s ago 3m 19.5M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:9283,8765,8443 running (12m) 2m ago 12m 500M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 2cf5112ba77d 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (17s) 9s ago 8m 52.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 7199977e9e8d 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (12m) 2m ago 12m 45.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (8m) 9s ago 8m 34.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (10m) 2m ago 10m 7596k - 1.5.0 0da6a335fe13 6b69ffbf445a 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (9m) 9s ago 9m 8116k - 1.5.0 0da6a335fe13 81909a8618f8 2024-04-20T11:07:43.983 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (7m) 2m ago 7m 54.1M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:07:43.984 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (7m) 2m ago 7m 47.5M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:07:43.984 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (6m) 2m ago 6m 45.1M 3947M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:07:43.984 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (6m) 9s ago 6m 63.9M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:07:43.984 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (5m) 9s ago 5m 62.1M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:07:43.984 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (4m) 9s ago 4m 61.8M 4971M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:07:43.984 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (7m) 2m ago 9m 49.8M - 2.43.0 a07b618ecd1d 532b98b31363 2024-04-20T11:07:43.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f418c0394b0 msgr2=0x7f418c03b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:43.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f418c0394b0 0x7f418c03b970 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4198005950 tx=0x7f419800b410 comp rx=0 tx=0).stop 2024-04-20T11:07:43.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4074f10 msgr2=0x7f41a419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:43.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4074f10 0x7f41a419cfd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f41900059d0 tx=0x7f419000bd70 comp rx=0 tx=0).stop 2024-04-20T11:07:43.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 shutdown_connections 2024-04-20T11:07:43.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f418c0394b0 0x7f418c03b970 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a4073940 0x7f41a419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 --2- 172.21.15.117:0/680579857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a4074f10 0x7f41a419cfd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:43.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 >> 172.21.15.117:0/680579857 conn(0x7f41a40fc000 msgr2=0x7f41a41027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:43.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 shutdown_connections 2024-04-20T11:07:43.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:43.985+0000 7f41aa000700 1 -- 172.21.15.117:0/680579857 wait complete. 2024-04-20T11:07:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:43 smithi173 ceph-mon[23800]: [20/Apr/2024:11:07:42] ENGINE Bus STARTING 2024-04-20T11:07:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:43 smithi173 ceph-mon[23800]: [20/Apr/2024:11:07:42] ENGINE Serving on https://172.21.15.173:7150 2024-04-20T11:07:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:43 smithi173 ceph-mon[23800]: [20/Apr/2024:11:07:42] ENGINE Serving on http://172.21.15.173:8765 2024-04-20T11:07:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:43 smithi173 ceph-mon[23800]: [20/Apr/2024:11:07:42] ENGINE Bus STARTED 2024-04-20T11:07:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:43 smithi173 ceph-mon[23800]: Deploying cephadm binary to smithi173 2024-04-20T11:07:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:43 smithi173 ceph-mon[23800]: Deploying cephadm binary to smithi117 2024-04-20T11:07:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:43 smithi173 ceph-mon[23800]: mgrmap e22: smithi173.gnrjof(active, since 1.01707s) 2024-04-20T11:07:44.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.207+0000 7f2f79e00700 1 -- 172.21.15.117:0/4258266980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f740ffaf0 msgr2=0x7f2f740fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:44.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.207+0000 7f2f79e00700 1 --2- 172.21.15.117:0/4258266980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f740ffaf0 0x7f2f740fff10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009870 tx=0x7f2f64009b80 comp rx=0 tx=0).stop 2024-04-20T11:07:44.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.208+0000 7f2f79e00700 1 -- 172.21.15.117:0/4258266980 shutdown_connections 2024-04-20T11:07:44.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.208+0000 7f2f79e00700 1 --2- 172.21.15.117:0/4258266980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f74100cf0 0x7f2f74101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:44.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.208+0000 7f2f79e00700 1 --2- 172.21.15.117:0/4258266980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f740ffaf0 0x7f2f740fff10 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009870 tx=0x7f2f64009b80 comp rx=0 tx=0).stop 2024-04-20T11:07:44.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.208+0000 7f2f79e00700 1 -- 172.21.15.117:0/4258266980 >> 172.21.15.117:0/4258266980 conn(0x7f2f740fb0d0 msgr2=0x7f2f740fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:44.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.208+0000 7f2f79e00700 1 -- 172.21.15.117:0/4258266980 shutdown_connections 2024-04-20T11:07:44.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.209+0000 7f2f79e00700 1 -- 172.21.15.117:0/4258266980 wait complete. 2024-04-20T11:07:44.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.210+0000 7f2f79e00700 1 Processor -- start 2024-04-20T11:07:44.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.210+0000 7f2f79e00700 1 -- start start 2024-04-20T11:07:44.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f79e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f74100cf0 0x7f2f741928f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:44.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f79e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f74192e30 0x7f2f7418f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:44.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f79e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7418f8e0 con 0x7f2f74100cf0 2024-04-20T11:07:44.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f79e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7418fa50 con 0x7f2f74192e30 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f72a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f74100cf0 0x7f2f741928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f72a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f74100cf0 0x7f2f741928f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35700/0 (socket says 172.21.15.117:35700) 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f72a00700 1 -- 172.21.15.117:0/795703968 learned_addr learned my addr 172.21.15.117:0/795703968 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f72000700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f74192e30 0x7f2f7418f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f72a00700 1 -- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f74192e30 msgr2=0x7f2f7418f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f72a00700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f74192e30 0x7f2f7418f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.211+0000 7f2f72a00700 1 -- 172.21.15.117:0/795703968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64009520 con 0x7f2f74100cf0 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.212+0000 7f2f72a00700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f74100cf0 0x7f2f741928f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2f64000c00 tx=0x7f2f6400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:44.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.212+0000 7f2f78a00700 1 -- 172.21.15.117:0/795703968 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6401c070 con 0x7f2f74100cf0 2024-04-20T11:07:44.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.212+0000 7f2f78a00700 1 -- 172.21.15.117:0/795703968 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f6400be40 con 0x7f2f74100cf0 2024-04-20T11:07:44.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.212+0000 7f2f78a00700 1 -- 172.21.15.117:0/795703968 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f64022c80 con 0x7f2f74100cf0 2024-04-20T11:07:44.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.212+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7418fc10 con 0x7f2f74100cf0 2024-04-20T11:07:44.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.212+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f74198700 con 0x7f2f74100cf0 2024-04-20T11:07:44.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.214+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f54005320 con 0x7f2f74100cf0 2024-04-20T11:07:44.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.214+0000 7f2f78a00700 1 -- 172.21.15.117:0/795703968 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) v1 ==== 46651+0+0 (secure 0 0 0) 0x7f2f64020070 con 0x7f2f74100cf0 2024-04-20T11:07:44.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.214+0000 7f2f78a00700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f2f58039840 0x7f2f5803bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:44.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.215+0000 7f2f78a00700 1 -- 172.21.15.117:0/795703968 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f2f6404ed90 con 0x7f2f74100cf0 2024-04-20T11:07:44.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.215+0000 7f2f72000700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f2f58039840 0x7f2f5803bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:44.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.216+0000 7f2f72000700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f2f58039840 0x7f2f5803bd00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2f60005fd0 tx=0x7f2f60009310 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:44.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.222+0000 7f2f78a00700 1 -- 172.21.15.117:0/795703968 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f2f6402a920 con 0x7f2f74100cf0 2024-04-20T11:07:44.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.679+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2f540058a0 con 0x7f2f74100cf0 2024-04-20T11:07:44.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.679+0000 7f2f78a00700 1 -- 172.21.15.117:0/795703968 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f2f64027020 con 0x7f2f74100cf0 2024-04-20T11:07:44.679 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:07:44.679 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:07:44.679 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:07:44.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:07:44.680 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:07:44.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.682+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f2f58039840 msgr2=0x7f2f5803bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:44.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.682+0000 7f2f79e00700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f2f58039840 0x7f2f5803bd00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2f60005fd0 tx=0x7f2f60009310 comp rx=0 tx=0).stop 2024-04-20T11:07:44.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.682+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f74100cf0 msgr2=0x7f2f741928f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:44.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.682+0000 7f2f79e00700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f74100cf0 0x7f2f741928f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2f64000c00 tx=0x7f2f6400f690 comp rx=0 tx=0).stop 2024-04-20T11:07:44.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.683+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 shutdown_connections 2024-04-20T11:07:44.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.683+0000 7f2f79e00700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f2f58039840 0x7f2f5803bd00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:44.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.683+0000 7f2f79e00700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f74100cf0 0x7f2f741928f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:44.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.683+0000 7f2f79e00700 1 --2- 172.21.15.117:0/795703968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f74192e30 0x7f2f7418f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:44.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.683+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 >> 172.21.15.117:0/795703968 conn(0x7f2f740fb0d0 msgr2=0x7f2f741043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:44.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.684+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 shutdown_connections 2024-04-20T11:07:44.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.685+0000 7f2f79e00700 1 -- 172.21.15.117:0/795703968 wait complete. 2024-04-20T11:07:44.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:44 smithi117 ceph-mon[15659]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:44.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:44 smithi117 ceph-mon[15659]: pgmap v3: 65 pgs: 65 active+clean; 65 MiB data, 414 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:44.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:44 smithi117 ceph-mon[15659]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:44.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:44 smithi117 ceph-mon[15659]: mgrmap e23: smithi173.gnrjof(active, since 2s) 2024-04-20T11:07:44.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:44 smithi117 ceph-mon[15659]: pgmap v4: 65 pgs: 65 active+clean; 65 MiB data, 414 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:44.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:44 smithi117 ceph-mon[15659]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:44.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:44 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/795703968' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:07:44.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.854+0000 7f1938800700 1 -- 172.21.15.117:0/2002174508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ff470 msgr2=0x7f19300ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:44.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.854+0000 7f1938800700 1 --2- 172.21.15.117:0/2002174508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ff470 0x7f19300ff890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1924009910 tx=0x7f1924009c20 comp rx=0 tx=0).stop 2024-04-20T11:07:44.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.855+0000 7f1938800700 1 -- 172.21.15.117:0/2002174508 shutdown_connections 2024-04-20T11:07:44.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.855+0000 7f1938800700 1 --2- 172.21.15.117:0/2002174508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19300ffdd0 0x7f1930100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:44.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.855+0000 7f1938800700 1 --2- 172.21.15.117:0/2002174508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ff470 0x7f19300ff890 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:44.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.855+0000 7f1938800700 1 -- 172.21.15.117:0/2002174508 >> 172.21.15.117:0/2002174508 conn(0x7f19300fb0d0 msgr2=0x7f19300fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:44.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.855+0000 7f1938800700 1 -- 172.21.15.117:0/2002174508 shutdown_connections 2024-04-20T11:07:44.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.855+0000 7f1938800700 1 -- 172.21.15.117:0/2002174508 wait complete. 2024-04-20T11:07:44.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.856+0000 7f1938800700 1 Processor -- start 2024-04-20T11:07:44.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.856+0000 7f1938800700 1 -- start start 2024-04-20T11:07:44.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.857+0000 7f1938800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19300ff470 0x7f1930198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:44.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.857+0000 7f1938800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ffdd0 0x7f1930198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:44.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.857+0000 7f1938800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19301992f0 con 0x7f19300ffdd0 2024-04-20T11:07:44.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.857+0000 7f1938800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1930199430 con 0x7f19300ff470 2024-04-20T11:07:44.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.857+0000 7f1935400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ffdd0 0x7f1930198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:44.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.857+0000 7f1935400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ffdd0 0x7f1930198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35718/0 (socket says 172.21.15.117:35718) 2024-04-20T11:07:44.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.857+0000 7f1935400700 1 -- 172.21.15.117:0/827109660 learned_addr learned my addr 172.21.15.117:0/827109660 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:07:44.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.857+0000 7f1935e00700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19300ff470 0x7f1930198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:44.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.858+0000 7f1935400700 1 -- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19300ff470 msgr2=0x7f1930198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:44.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.858+0000 7f1935400700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19300ff470 0x7f1930198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:44.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.858+0000 7f1935400700 1 -- 172.21.15.117:0/827109660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19240095f0 con 0x7f19300ffdd0 2024-04-20T11:07:44.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.858+0000 7f1935400700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ffdd0 0x7f1930198d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f192000c930 tx=0x7f192000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:44.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.859+0000 7f192aa00700 1 -- 172.21.15.117:0/827109660 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1920007ad0 con 0x7f19300ffdd0 2024-04-20T11:07:44.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.859+0000 7f192aa00700 1 -- 172.21.15.117:0/827109660 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1920007c30 con 0x7f19300ffdd0 2024-04-20T11:07:44.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.859+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193019de50 con 0x7f19300ffdd0 2024-04-20T11:07:44.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.859+0000 7f192aa00700 1 -- 172.21.15.117:0/827109660 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1920018630 con 0x7f19300ffdd0 2024-04-20T11:07:44.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.859+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193019e3f0 con 0x7f19300ffdd0 2024-04-20T11:07:44.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.860+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1930192960 con 0x7f19300ffdd0 2024-04-20T11:07:44.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.861+0000 7f192aa00700 1 -- 172.21.15.117:0/827109660 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) v1 ==== 46651+0+0 (secure 0 0 0) 0x7f192000f450 con 0x7f19300ffdd0 2024-04-20T11:07:44.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.861+0000 7f192aa00700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f1918039840 0x7f191803bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:07:44.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.861+0000 7f192aa00700 1 -- 172.21.15.117:0/827109660 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f192004dc70 con 0x7f19300ffdd0 2024-04-20T11:07:44.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.862+0000 7f1935e00700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f1918039840 0x7f191803bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:07:44.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.864+0000 7f1935e00700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f1918039840 0x7f191803bd00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f192400b5c0 tx=0x7f1924005160 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:07:44.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:44.868+0000 7f192aa00700 1 -- 172.21.15.117:0/827109660 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f1920016300 con 0x7f19300ffdd0 2024-04-20T11:07:45.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.158+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 --> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f193002cea0 con 0x7f1918039840 2024-04-20T11:07:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:44 smithi173 ceph-mon[23800]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:44 smithi173 ceph-mon[23800]: pgmap v3: 65 pgs: 65 active+clean; 65 MiB data, 414 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:44 smithi173 ceph-mon[23800]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:44 smithi173 ceph-mon[23800]: mgrmap e23: smithi173.gnrjof(active, since 2s) 2024-04-20T11:07:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:44 smithi173 ceph-mon[23800]: pgmap v4: 65 pgs: 65 active+clean; 65 MiB data, 414 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:44 smithi173 ceph-mon[23800]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:44 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/795703968' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:07:45.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.163+0000 7f192aa00700 1 -- 172.21.15.117:0/827109660 <== mgr.24337 v2:172.21.15.173:6828/2992581883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f193002cea0 con 0x7f1918039840 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [], 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "1/2 daemons upgraded", 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "", 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:07:45.164 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:07:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f1918039840 msgr2=0x7f191803bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f1918039840 0x7f191803bd00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f192400b5c0 tx=0x7f1924005160 comp rx=0 tx=0).stop 2024-04-20T11:07:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ffdd0 msgr2=0x7f1930198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:07:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ffdd0 0x7f1930198d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f192000c930 tx=0x7f192000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:07:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 shutdown_connections 2024-04-20T11:07:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f1918039840 0x7f191803bd00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19300ff470 0x7f1930198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 --2- 172.21.15.117:0/827109660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19300ffdd0 0x7f1930198d60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:07:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.167+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 >> 172.21.15.117:0/827109660 conn(0x7f19300fb0d0 msgr2=0x7f1930107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:07:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.168+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 shutdown_connections 2024-04-20T11:07:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:07:45.168+0000 7f1938800700 1 -- 172.21.15.117:0/827109660 wait complete. 2024-04-20T11:07:45.993 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:45 smithi117 ceph-mon[15659]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:46.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:45 smithi173 ceph-mon[23800]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:07:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:46 smithi117 ceph-mon[15659]: pgmap v5: 65 pgs: 65 active+clean; 65 MiB data, 414 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:46 smithi117 ceph-mon[15659]: mgrmap e24: smithi173.gnrjof(active, since 4s) 2024-04-20T11:07:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:46 smithi173 ceph-mon[23800]: pgmap v5: 65 pgs: 65 active+clean; 65 MiB data, 414 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:07:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:46 smithi173 ceph-mon[23800]: mgrmap e24: smithi173.gnrjof(active, since 4s) 2024-04-20T11:07:49.069 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:48 smithi117 ceph-mon[15659]: pgmap v6: 65 pgs: 65 active+clean; 69 MiB data, 432 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 761 KiB/s wr, 38 op/s 2024-04-20T11:07:49.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:48 smithi173 ceph-mon[23800]: pgmap v6: 65 pgs: 65 active+clean; 69 MiB data, 432 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 761 KiB/s wr, 38 op/s 2024-04-20T11:07:51.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:50 smithi117 ceph-mon[15659]: pgmap v7: 65 pgs: 65 active+clean; 82 MiB data, 466 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.4 MiB/s wr, 112 op/s 2024-04-20T11:07:51.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:50 smithi173 ceph-mon[23800]: pgmap v7: 65 pgs: 65 active+clean; 82 MiB data, 466 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.4 MiB/s wr, 112 op/s 2024-04-20T11:07:53.101 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:52 smithi117 ceph-mon[15659]: pgmap v8: 65 pgs: 65 active+clean; 82 MiB data, 466 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-04-20T11:07:53.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:52 smithi173 ceph-mon[23800]: pgmap v8: 65 pgs: 65 active+clean; 82 MiB data, 466 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-04-20T11:07:54.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:54 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:54.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:54 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:54.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:54 smithi117 ceph-mon[15659]: Standby manager daemon smithi117.ivnklj started 2024-04-20T11:07:54.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:54 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.117:0/2329581845' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.ivnklj/crt"}]: dispatch 2024-04-20T11:07:54.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:54 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.117:0/2329581845' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:07:54.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:54 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.117:0/2329581845' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.ivnklj/key"}]: dispatch 2024-04-20T11:07:54.802 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:54 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.117:0/2329581845' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:07:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:54 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:54 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:54 smithi173 ceph-mon[23800]: Standby manager daemon smithi117.ivnklj started 2024-04-20T11:07:54.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:54 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.117:0/2329581845' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.ivnklj/crt"}]: dispatch 2024-04-20T11:07:54.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:54 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.117:0/2329581845' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:07:54.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:54 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.117:0/2329581845' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.ivnklj/key"}]: dispatch 2024-04-20T11:07:54.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:54 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.117:0/2329581845' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:07:55.849 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:55 smithi117 ceph-mon[15659]: pgmap v9: 65 pgs: 65 active+clean; 92 MiB data, 501 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 2.4 MiB/s wr, 114 op/s 2024-04-20T11:07:55.849 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:55 smithi117 ceph-mon[15659]: mgrmap e25: smithi173.gnrjof(active, since 12s), standbys: smithi117.ivnklj 2024-04-20T11:07:55.849 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:55 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T11:07:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:55 smithi173 ceph-mon[23800]: pgmap v9: 65 pgs: 65 active+clean; 92 MiB data, 501 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 2.4 MiB/s wr, 114 op/s 2024-04-20T11:07:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:55 smithi173 ceph-mon[23800]: mgrmap e25: smithi173.gnrjof(active, since 12s), standbys: smithi117.ivnklj 2024-04-20T11:07:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:55 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T11:07:57.057 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:56 smithi173 ceph-mon[23800]: pgmap v10: 65 pgs: 65 active+clean; 92 MiB data, 501 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.2 MiB/s wr, 104 op/s 2024-04-20T11:07:57.057 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:56 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:57.057 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:56 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:57.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:56 smithi117 ceph-mon[15659]: pgmap v10: 65 pgs: 65 active+clean; 92 MiB data, 501 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.2 MiB/s wr, 104 op/s 2024-04-20T11:07:57.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:56 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:57.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:56 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:58.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:57 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:58.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:57 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:58.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:57 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:57 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:07:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:57 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:57 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:07:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:07:59 smithi173 ceph-mon[23800]: pgmap v11: 65 pgs: 65 active+clean; 92 MiB data, 524 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.3 MiB/s wr, 118 op/s 2024-04-20T11:07:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:07:59 smithi117 ceph-mon[15659]: pgmap v11: 65 pgs: 65 active+clean; 92 MiB data, 524 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.3 MiB/s wr, 118 op/s 2024-04-20T11:08:00.562 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:00.562 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:00.562 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.562 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.562 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: Adjusting osd_memory_target on smithi117 to 1216M 2024-04-20T11:08:00.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:00 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:00.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: Adjusting osd_memory_target on smithi117 to 1216M 2024-04-20T11:08:00.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:00 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:01.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:01 smithi117 ceph-mon[15659]: pgmap v12: 65 pgs: 65 active+clean; 103 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.8 MiB/s wr, 131 op/s 2024-04-20T11:08:01.589 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:01 smithi173 ceph-mon[23800]: pgmap v12: 65 pgs: 65 active+clean; 103 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.8 MiB/s wr, 131 op/s 2024-04-20T11:08:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: pgmap v13: 65 pgs: 65 active+clean; 103 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 81 op/s 2024-04-20T11:08:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: Adjusting osd_memory_target on smithi173 to 2240M 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T11:08:03.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:03 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: pgmap v13: 65 pgs: 65 active+clean; 103 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 81 op/s 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:08:03.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: Adjusting osd_memory_target on smithi173 to 2240M 2024-04-20T11:08:03.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:03.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:08:03.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:08:03.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T11:08:03.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:03 smithi117 ceph-mon[15659]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T11:08:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:04 smithi117 ceph-mon[15659]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:08:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:04 smithi117 ceph-mon[15659]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:08:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:04 smithi117 ceph-mon[15659]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:08:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:04 smithi117 ceph-mon[15659]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:08:04.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:04 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:08:04.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:04 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:08:04.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:04 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:08:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:04 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:08:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:05 smithi117 ceph-mon[15659]: pgmap v14: 65 pgs: 65 active+clean; 107 MiB data, 559 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 109 op/s 2024-04-20T11:08:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:05 smithi117 ceph-mon[15659]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:08:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:05 smithi117 ceph-mon[15659]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:08:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:05 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:05 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:05 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:05 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:05 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:05 smithi173 ceph-mon[23800]: pgmap v14: 65 pgs: 65 active+clean; 107 MiB data, 559 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 109 op/s 2024-04-20T11:08:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:05 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:08:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:05 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:08:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:05 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:05 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:05 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:05 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:05.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:05 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:06.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:06 smithi117 ceph-mon[15659]: Reconfiguring prometheus.smithi117 (dependencies changed)... 2024-04-20T11:08:06.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:06 smithi117 ceph-mon[15659]: Reconfiguring daemon prometheus.smithi117 on smithi117 2024-04-20T11:08:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:06 smithi173 ceph-mon[23800]: Reconfiguring prometheus.smithi117 (dependencies changed)... 2024-04-20T11:08:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:06 smithi173 ceph-mon[23800]: Reconfiguring daemon prometheus.smithi117 on smithi117 2024-04-20T11:08:07.348 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:07 smithi117 ceph-mon[15659]: pgmap v15: 65 pgs: 65 active+clean; 107 MiB data, 559 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 70 op/s 2024-04-20T11:08:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:07 smithi173 ceph-mon[23800]: pgmap v15: 65 pgs: 65 active+clean; 107 MiB data, 559 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 70 op/s 2024-04-20T11:08:09.249 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:09 smithi117 ceph-mon[15659]: pgmap v16: 65 pgs: 65 active+clean; 108 MiB data, 568 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 78 op/s 2024-04-20T11:08:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:09 smithi173 ceph-mon[23800]: pgmap v16: 65 pgs: 65 active+clean; 108 MiB data, 568 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 78 op/s 2024-04-20T11:08:11.563 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:11 smithi117 ceph-mon[15659]: pgmap v17: 65 pgs: 65 active+clean; 116 MiB data, 583 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 90 op/s 2024-04-20T11:08:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:11 smithi173 ceph-mon[23800]: pgmap v17: 65 pgs: 65 active+clean; 116 MiB data, 583 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 90 op/s 2024-04-20T11:08:12.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:12 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:08:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:12 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:08:13.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:13 smithi117 ceph-mon[15659]: pgmap v18: 65 pgs: 65 active+clean; 116 MiB data, 583 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 61 op/s 2024-04-20T11:08:13.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:13 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:13 smithi173 ceph-mon[23800]: pgmap v18: 65 pgs: 65 active+clean; 116 MiB data, 583 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 61 op/s 2024-04-20T11:08:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:13 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:14.518 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:14 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:14.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:14 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:08:14.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:14 smithi117 ceph-mon[15659]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:08:14.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:14 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:08:14.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:14 smithi117 ceph-mon[15659]: Upgrade: Need to upgrade myself (mgr.smithi173.gnrjof) 2024-04-20T11:08:14.519 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:14 smithi117 ceph-mon[15659]: Upgrade: Need to upgrade myself (mgr.smithi173.gnrjof) 2024-04-20T11:08:14.547 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:14 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:14.548 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:14 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:08:14.548 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:14 smithi173 ceph-mon[23800]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:08:14.548 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:14 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:08:14.548 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:14 smithi173 ceph-mon[23800]: Upgrade: Need to upgrade myself (mgr.smithi173.gnrjof) 2024-04-20T11:08:14.548 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:14 smithi173 ceph-mon[23800]: Upgrade: Need to upgrade myself (mgr.smithi173.gnrjof) 2024-04-20T11:08:15.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.378+0000 7f7944400700 1 -- 172.21.15.117:0/1608959034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 msgr2=0x7f793c1063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:15.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.378+0000 7f7944400700 1 --2- 172.21.15.117:0/1608959034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 0x7f793c1063a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7928009910 tx=0x7f7928009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:15.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.378+0000 7f7944400700 1 -- 172.21.15.117:0/1608959034 shutdown_connections 2024-04-20T11:08:15.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.378+0000 7f7944400700 1 --2- 172.21.15.117:0/1608959034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 0x7f793c1063a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.378+0000 7f7944400700 1 --2- 172.21.15.117:0/1608959034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793c101680 0x7f793c103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.378+0000 7f7944400700 1 -- 172.21.15.117:0/1608959034 >> 172.21.15.117:0/1608959034 conn(0x7f793c0fb0d0 msgr2=0x7f793c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:15.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.378+0000 7f7944400700 1 -- 172.21.15.117:0/1608959034 shutdown_connections 2024-04-20T11:08:15.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.378+0000 7f7944400700 1 -- 172.21.15.117:0/1608959034 wait complete. 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.379+0000 7f7944400700 1 Processor -- start 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.380+0000 7f7944400700 1 -- start start 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.380+0000 7f7944400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793c101680 0x7f793c1965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.380+0000 7f7944400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 0x7f793c196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.380+0000 7f7944400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c197110 con 0x7f793c103fb0 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.380+0000 7f7944400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c197250 con 0x7f793c101680 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.380+0000 7f7941000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 0x7f793c196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.380+0000 7f7941000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 0x7f793c196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38370/0 (socket says 172.21.15.117:38370) 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.380+0000 7f7941000700 1 -- 172.21.15.117:0/2184771395 learned_addr learned my addr 172.21.15.117:0/2184771395 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:15.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.381+0000 7f7941a00700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793c101680 0x7f793c1965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:15.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.381+0000 7f7941000700 1 -- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793c101680 msgr2=0x7f793c1965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:15.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.381+0000 7f7941000700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793c101680 0x7f793c1965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.381+0000 7f7941000700 1 -- 172.21.15.117:0/2184771395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280095f0 con 0x7f793c103fb0 2024-04-20T11:08:15.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.381+0000 7f7941000700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 0x7f793c196af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f792800bd70 tx=0x7f792800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:15.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.382+0000 7f793a000700 1 -- 172.21.15.117:0/2184771395 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f792801d070 con 0x7f793c103fb0 2024-04-20T11:08:15.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.382+0000 7f793a000700 1 -- 172.21.15.117:0/2184771395 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f792800f460 con 0x7f793c103fb0 2024-04-20T11:08:15.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.382+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c19bc50 con 0x7f793c103fb0 2024-04-20T11:08:15.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.382+0000 7f793a000700 1 -- 172.21.15.117:0/2184771395 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7928021620 con 0x7f793c103fb0 2024-04-20T11:08:15.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.382+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c19c1c0 con 0x7f793c103fb0 2024-04-20T11:08:15.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.383+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c062510 con 0x7f793c103fb0 2024-04-20T11:08:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.385+0000 7f793a000700 1 -- 172.21.15.117:0/2184771395 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 91832+0+0 (secure 0 0 0) 0x7f792800fa70 con 0x7f793c103fb0 2024-04-20T11:08:15.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.385+0000 7f793a000700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f792406dc40 0x7f7924070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:15.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.386+0000 7f793a000700 1 -- 172.21.15.117:0/2184771395 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f792808eb00 con 0x7f793c103fb0 2024-04-20T11:08:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.387+0000 7f7941a00700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f792406dc40 0x7f7924070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.388+0000 7f7941a00700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f792406dc40 0x7f7924070100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7930005fd0 tx=0x7f793000a560 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:15.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.392+0000 7f793a000700 1 -- 172.21.15.117:0/2184771395 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f792805b870 con 0x7f793c103fb0 2024-04-20T11:08:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:15 smithi117 ceph-mon[15659]: pgmap v19: 65 pgs: 65 active+clean; 127 MiB data, 574 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 108 op/s 2024-04-20T11:08:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:15 smithi173 ceph-mon[23800]: pgmap v19: 65 pgs: 65 active+clean; 127 MiB data, 574 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 108 op/s 2024-04-20T11:08:15.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.708+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 --> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f793c105eb0 con 0x7f792406dc40 2024-04-20T11:08:15.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.711+0000 7f793a000700 1 -- 172.21.15.117:0/2184771395 <== mgr.24337 v2:172.21.15.173:6828/2992581883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f793c105eb0 con 0x7f792406dc40 2024-04-20T11:08:15.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.714+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f792406dc40 msgr2=0x7f7924070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:15.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.714+0000 7f7944400700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f792406dc40 0x7f7924070100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7930005fd0 tx=0x7f793000a560 comp rx=0 tx=0).stop 2024-04-20T11:08:15.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.714+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 msgr2=0x7f793c196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:15.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.714+0000 7f7944400700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 0x7f793c196af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f792800bd70 tx=0x7f792800be50 comp rx=0 tx=0).stop 2024-04-20T11:08:15.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.714+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 shutdown_connections 2024-04-20T11:08:15.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.714+0000 7f7944400700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f792406dc40 0x7f7924070100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.714+0000 7f7944400700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793c101680 0x7f793c1965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.715+0000 7f7944400700 1 --2- 172.21.15.117:0/2184771395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f793c103fb0 0x7f793c196af0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.715+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 >> 172.21.15.117:0/2184771395 conn(0x7f793c0fb0d0 msgr2=0x7f793c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:15.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.716+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 shutdown_connections 2024-04-20T11:08:15.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.716+0000 7f7944400700 1 -- 172.21.15.117:0/2184771395 wait complete. 2024-04-20T11:08:15.734 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:08:15.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.902+0000 7f3047600700 1 -- 172.21.15.117:0/2208571317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3040101680 msgr2=0x7f3040103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:15.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.902+0000 7f3047600700 1 --2- 172.21.15.117:0/2208571317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3040101680 0x7f3040103a70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3034009910 tx=0x7f3034009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:15.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.902+0000 7f3047600700 1 -- 172.21.15.117:0/2208571317 shutdown_connections 2024-04-20T11:08:15.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.902+0000 7f3047600700 1 --2- 172.21.15.117:0/2208571317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3040103fb0 0x7f30401063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.902+0000 7f3047600700 1 --2- 172.21.15.117:0/2208571317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3040101680 0x7f3040103a70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.902+0000 7f3047600700 1 -- 172.21.15.117:0/2208571317 >> 172.21.15.117:0/2208571317 conn(0x7f30400fb0d0 msgr2=0x7f30400fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:15.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.903+0000 7f3047600700 1 -- 172.21.15.117:0/2208571317 shutdown_connections 2024-04-20T11:08:15.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.903+0000 7f3047600700 1 -- 172.21.15.117:0/2208571317 wait complete. 2024-04-20T11:08:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.904+0000 7f3047600700 1 Processor -- start 2024-04-20T11:08:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.905+0000 7f3047600700 1 -- start start 2024-04-20T11:08:15.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.905+0000 7f3047600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3040103fb0 0x7f30401a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:15.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.905+0000 7f3047600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30401a5c50 0x7f30401aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:15.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.905+0000 7f3047600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a6160 con 0x7f30401a5c50 2024-04-20T11:08:15.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.905+0000 7f3047600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a62d0 con 0x7f3040103fb0 2024-04-20T11:08:15.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.906+0000 7f303fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30401a5c50 0x7f30401aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:15.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.906+0000 7f303fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30401a5c50 0x7f30401aac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38390/0 (socket says 172.21.15.117:38390) 2024-04-20T11:08:15.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.906+0000 7f303fe00700 1 -- 172.21.15.117:0/83455370 learned_addr learned my addr 172.21.15.117:0/83455370 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:15.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.906+0000 7f3044c00700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3040103fb0 0x7f30401a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:15.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.907+0000 7f303fe00700 1 -- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3040103fb0 msgr2=0x7f30401a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:15.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.907+0000 7f303fe00700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3040103fb0 0x7f30401a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:15.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.907+0000 7f303fe00700 1 -- 172.21.15.117:0/83455370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340095f0 con 0x7f30401a5c50 2024-04-20T11:08:15.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.907+0000 7f303fe00700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30401a5c50 0x7f30401aac70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f302c00b4b0 tx=0x7f302c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:15.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.907+0000 7f303d600700 1 -- 172.21.15.117:0/83455370 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302c011800 con 0x7f30401a5c50 2024-04-20T11:08:15.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.907+0000 7f303d600700 1 -- 172.21.15.117:0/83455370 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f302c011e20 con 0x7f30401a5c50 2024-04-20T11:08:15.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.907+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30401ab210 con 0x7f30401a5c50 2024-04-20T11:08:15.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.908+0000 7f303d600700 1 -- 172.21.15.117:0/83455370 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302c00f2e0 con 0x7f30401a5c50 2024-04-20T11:08:15.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.908+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30401ab7e0 con 0x7f30401a5c50 2024-04-20T11:08:15.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.909+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30401090b0 con 0x7f30401a5c50 2024-04-20T11:08:15.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.913+0000 7f303d600700 1 -- 172.21.15.117:0/83455370 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 91832+0+0 (secure 0 0 0) 0x7f302c011960 con 0x7f30401a5c50 2024-04-20T11:08:15.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.914+0000 7f303d600700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f302806dd50 0x7f3028070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:15.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.914+0000 7f303d600700 1 -- 172.21.15.117:0/83455370 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f302c08d1d0 con 0x7f30401a5c50 2024-04-20T11:08:15.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.915+0000 7f3044c00700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f302806dd50 0x7f3028070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:15.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.918+0000 7f3044c00700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f302806dd50 0x7f3028070210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3034009fd0 tx=0x7f3034005fd0 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:15.920+0000 7f303d600700 1 -- 172.21.15.117:0/83455370 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f302c05a090 con 0x7f30401a5c50 2024-04-20T11:08:16.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.324+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 --> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3040102200 con 0x7f302806dd50 2024-04-20T11:08:16.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.328+0000 7f303d600700 1 -- 172.21.15.117:0/83455370 <== mgr.24337 v2:172.21.15.173:6828/2992581883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f3040102200 con 0x7f302806dd50 2024-04-20T11:08:16.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.333+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f302806dd50 msgr2=0x7f3028070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:16.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.333+0000 7f3047600700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f302806dd50 0x7f3028070210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3034009fd0 tx=0x7f3034005fd0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.333+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30401a5c50 msgr2=0x7f30401aac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.333+0000 7f3047600700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30401a5c50 0x7f30401aac70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f302c00b4b0 tx=0x7f302c00b870 comp rx=0 tx=0).stop 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.334+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 shutdown_connections 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.334+0000 7f3047600700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f302806dd50 0x7f3028070210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.334+0000 7f3047600700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3040103fb0 0x7f30401a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.334+0000 7f3047600700 1 --2- 172.21.15.117:0/83455370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30401a5c50 0x7f30401aac70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.334+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 >> 172.21.15.117:0/83455370 conn(0x7f30400fb0d0 msgr2=0x7f30400fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.334+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 shutdown_connections 2024-04-20T11:08:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.334+0000 7f3047600700 1 -- 172.21.15.117:0/83455370 wait complete. 2024-04-20T11:08:16.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.564+0000 7f2718c00700 1 -- 172.21.15.117:0/4290536591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714108420 msgr2=0x7f2714108880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:16.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.564+0000 7f2718c00700 1 --2- 172.21.15.117:0/4290536591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714108420 0x7f2714108880 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009910 tx=0x7f26fc009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:16.566 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:16 smithi117 ceph-mon[15659]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:16.566 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:16 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:16.566 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:16 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.ivnklj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:08:16.566 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:16 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.ivnklj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:08:16.567 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:16 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:08:16.567 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:16 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:08:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.564+0000 7f2718c00700 1 -- 172.21.15.117:0/4290536591 shutdown_connections 2024-04-20T11:08:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.564+0000 7f2718c00700 1 --2- 172.21.15.117:0/4290536591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714108420 0x7f2714108880 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.564+0000 7f2718c00700 1 --2- 172.21.15.117:0/4290536591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714107220 0x7f2714107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.564+0000 7f2718c00700 1 -- 172.21.15.117:0/4290536591 >> 172.21.15.117:0/4290536591 conn(0x7f27140761d0 msgr2=0x7f2714078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.566+0000 7f2718c00700 1 -- 172.21.15.117:0/4290536591 shutdown_connections 2024-04-20T11:08:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.566+0000 7f2718c00700 1 -- 172.21.15.117:0/4290536591 wait complete. 2024-04-20T11:08:16.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.567+0000 7f2718c00700 1 Processor -- start 2024-04-20T11:08:16.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.568+0000 7f2718c00700 1 -- start start 2024-04-20T11:08:16.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.569+0000 7f2718c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714107220 0x7f271419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:16.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.569+0000 7f2718c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714108420 0x7f271419d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:16.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.569+0000 7f2718c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271419d760 con 0x7f2714107220 2024-04-20T11:08:16.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.569+0000 7f2718c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271419d8a0 con 0x7f2714108420 2024-04-20T11:08:16.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.569+0000 7f2711e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714107220 0x7f271419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:16.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.569+0000 7f2711e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714107220 0x7f271419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38416/0 (socket says 172.21.15.117:38416) 2024-04-20T11:08:16.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.569+0000 7f2711e00700 1 -- 172.21.15.117:0/2339954368 learned_addr learned my addr 172.21.15.117:0/2339954368 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:16.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.570+0000 7f2711e00700 1 -- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714108420 msgr2=0x7f271419d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:16.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.570+0000 7f2711e00700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714108420 0x7f271419d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.570+0000 7f2711e00700 1 -- 172.21.15.117:0/2339954368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0095f0 con 0x7f2714107220 2024-04-20T11:08:16.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.571+0000 7f2711e00700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714107220 0x7f271419cc00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f270400ada0 tx=0x7f270400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:16.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.571+0000 7f270b400700 1 -- 172.21.15.117:0/2339954368 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f270400f070 con 0x7f2714107220 2024-04-20T11:08:16.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.571+0000 7f270b400700 1 -- 172.21.15.117:0/2339954368 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2704007bc0 con 0x7f2714107220 2024-04-20T11:08:16.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.571+0000 7f270b400700 1 -- 172.21.15.117:0/2339954368 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2704004250 con 0x7f2714107220 2024-04-20T11:08:16.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.571+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27141a2300 con 0x7f2714107220 2024-04-20T11:08:16.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.572+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27141a28d0 con 0x7f2714107220 2024-04-20T11:08:16.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.575+0000 7f270b400700 1 -- 172.21.15.117:0/2339954368 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 91832+0+0 (secure 0 0 0) 0x7f270400c900 con 0x7f2714107220 2024-04-20T11:08:16.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.574+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2714196dc0 con 0x7f2714107220 2024-04-20T11:08:16.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.576+0000 7f270b400700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f26f806dcd0 0x7f26f8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:16.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.576+0000 7f270b400700 1 -- 172.21.15.117:0/2339954368 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f270408cb80 con 0x7f2714107220 2024-04-20T11:08:16.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.577+0000 7f2709400700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f26f806dcd0 0x7f26f8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:16.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.578+0000 7f2709400700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f26f806dcd0 0x7f26f8070190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0098e0 tx=0x7f26fc0055c0 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:16.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.585+0000 7f270b400700 1 -- 172.21.15.117:0/2339954368 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f2704059930 con 0x7f2714107220 2024-04-20T11:08:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:16 smithi173 ceph-mon[23800]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:16 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:16 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.ivnklj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:08:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:16 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.ivnklj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:08:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:16 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:08:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:16 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:08:16.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.888+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 --> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f271402ce30 con 0x7f26f806dcd0 2024-04-20T11:08:16.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.926+0000 7f270b400700 1 -- 172.21.15.117:0/2339954368 <== mgr.24337 v2:172.21.15.173:6828/2992581883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f271402ce30 con 0x7f26f806dcd0 2024-04-20T11:08:16.927 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (8m) 19s ago 10m 13.6M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (11m) 19s ago 11m 7872k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (9m) 23s ago 9m 7680k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (11m) 19s ago 11m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (9m) 23s ago 9m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (8m) 19s ago 10m 75.8M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (3m) 19s ago 3m 13.5M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (3m) 19s ago 3m 95.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (3m) 23s ago 3m 13.5M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (3m) 23s ago 3m 21.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:9283,8765,8443 running (12m) 19s ago 12m 322M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 2cf5112ba77d 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (50s) 23s ago 9m 493M - 19.0.0-3133-ga8c84f59 76bc6b91b726 7199977e9e8d 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (12m) 19s ago 12m 51.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (9m) 23s ago 9m 38.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:08:16.928 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (10m) 19s ago 10m 7784k - 1.5.0 0da6a335fe13 6b69ffbf445a 2024-04-20T11:08:16.929 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (9m) 23s ago 9m 8024k - 1.5.0 0da6a335fe13 81909a8618f8 2024-04-20T11:08:16.929 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (8m) 19s ago 8m 73.9M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:08:16.929 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (7m) 19s ago 7m 72.6M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:08:16.929 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (7m) 19s ago 7m 62.0M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:08:16.929 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (6m) 23s ago 6m 77.2M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:08:16.929 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (6m) 23s ago 6m 72.7M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:08:16.929 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (5m) 23s ago 5m 73.0M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:08:16.929 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 starting - - - - 2024-04-20T11:08:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.929+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f26f806dcd0 msgr2=0x7f26f8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.929+0000 7f2718c00700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f26f806dcd0 0x7f26f8070190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0098e0 tx=0x7f26fc0055c0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.929+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714107220 msgr2=0x7f271419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.929+0000 7f2718c00700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714107220 0x7f271419cc00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f270400ada0 tx=0x7f270400c330 comp rx=0 tx=0).stop 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.930+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 shutdown_connections 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.930+0000 7f2718c00700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f26f806dcd0 0x7f26f8070190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.930+0000 7f2718c00700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2714107220 0x7f271419cc00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.930+0000 7f2718c00700 1 --2- 172.21.15.117:0/2339954368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714108420 0x7f271419d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.930+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 >> 172.21.15.117:0/2339954368 conn(0x7f27140761d0 msgr2=0x7f271410ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.930+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 shutdown_connections 2024-04-20T11:08:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:16.930+0000 7f2718c00700 1 -- 172.21.15.117:0/2339954368 wait complete. 2024-04-20T11:08:17.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.101+0000 7f8987200700 1 -- 172.21.15.117:0/1353725311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 msgr2=0x7f8988103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:17.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.101+0000 7f8987200700 1 --2- 172.21.15.117:0/1353725311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 0x7f8988103050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8978009910 tx=0x7f8978009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:17.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.101+0000 7f8987200700 1 -- 172.21.15.117:0/1353725311 shutdown_connections 2024-04-20T11:08:17.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.101+0000 7f8987200700 1 --2- 172.21.15.117:0/1353725311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8988103e30 0x7f8988104290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.101+0000 7f8987200700 1 --2- 172.21.15.117:0/1353725311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 0x7f8988103050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.101+0000 7f8987200700 1 -- 172.21.15.117:0/1353725311 >> 172.21.15.117:0/1353725311 conn(0x7f89880fe210 msgr2=0x7f8988100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:17.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.101+0000 7f8987200700 1 -- 172.21.15.117:0/1353725311 shutdown_connections 2024-04-20T11:08:17.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.101+0000 7f8987200700 1 -- 172.21.15.117:0/1353725311 wait complete. 2024-04-20T11:08:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.102+0000 7f8987200700 1 Processor -- start 2024-04-20T11:08:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.102+0000 7f8987200700 1 -- start start 2024-04-20T11:08:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.103+0000 7f8987200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 0x7f8988198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.103+0000 7f8987200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8988103e30 0x7f8988198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.103+0000 7f8987200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881991c0 con 0x7f8988102c30 2024-04-20T11:08:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.103+0000 7f8987200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8988199300 con 0x7f8988103e30 2024-04-20T11:08:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.103+0000 7f8985e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 0x7f8988198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.103+0000 7f8985e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 0x7f8988198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38426/0 (socket says 172.21.15.117:38426) 2024-04-20T11:08:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.103+0000 7f8985e00700 1 -- 172.21.15.117:0/532779160 learned_addr learned my addr 172.21.15.117:0/532779160 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.103+0000 7f8985400700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8988103e30 0x7f8988198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.104+0000 7f8985e00700 1 -- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8988103e30 msgr2=0x7f8988198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.104+0000 7f8985e00700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8988103e30 0x7f8988198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.104+0000 7f8985e00700 1 -- 172.21.15.117:0/532779160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780095f0 con 0x7f8988102c30 2024-04-20T11:08:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.104+0000 7f8985e00700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 0x7f8988198660 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f89780059d0 tx=0x7f897800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.104+0000 7f897ea00700 1 -- 172.21.15.117:0/532779160 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897801d070 con 0x7f8988102c30 2024-04-20T11:08:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.104+0000 7f897ea00700 1 -- 172.21.15.117:0/532779160 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f897800f460 con 0x7f8988102c30 2024-04-20T11:08:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.104+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898819dd00 con 0x7f8988102c30 2024-04-20T11:08:17.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.104+0000 7f897ea00700 1 -- 172.21.15.117:0/532779160 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8978021640 con 0x7f8988102c30 2024-04-20T11:08:17.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.105+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898819e220 con 0x7f8988102c30 2024-04-20T11:08:17.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.106+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89881927b0 con 0x7f8988102c30 2024-04-20T11:08:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.107+0000 7f897ea00700 1 -- 172.21.15.117:0/532779160 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 91832+0+0 (secure 0 0 0) 0x7f897800f5d0 con 0x7f8988102c30 2024-04-20T11:08:17.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.108+0000 7f897ea00700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f896c06dc50 0x7f896c070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:17.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.108+0000 7f897ea00700 1 -- 172.21.15.117:0/532779160 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f897808e9f0 con 0x7f8988102c30 2024-04-20T11:08:17.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.109+0000 7f8985400700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f896c06dc50 0x7f896c070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.111+0000 7f8985400700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f896c06dc50 0x7f896c070110 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8974005950 tx=0x7f89740058e0 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:17.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.113+0000 7f897ea00700 1 -- 172.21.15.117:0/532779160 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f8978090e60 con 0x7f8988102c30 2024-04-20T11:08:17.423 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:17 smithi117 ceph-mon[15659]: pgmap v20: 65 pgs: 65 active+clean; 127 MiB data, 574 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 80 op/s 2024-04-20T11:08:17.423 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:17 smithi117 ceph-mon[15659]: Upgrade: Updating mgr.smithi117.ivnklj 2024-04-20T11:08:17.423 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:17 smithi117 ceph-mon[15659]: Deploying daemon mgr.smithi117.ivnklj on smithi117 2024-04-20T11:08:17.423 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:17 smithi117 ceph-mon[15659]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:17.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.500+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f898810b790 con 0x7f8988102c30 2024-04-20T11:08:17.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.501+0000 7f897ea00700 1 -- 172.21.15.117:0/532779160 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+841 (secure 0 0 0) 0x7f897805b7c0 con 0x7f8988102c30 2024-04-20T11:08:17.502 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:08:17.502 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:08:17.502 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:08:17.502 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:08:17.502 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:08:17.502 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:08:17.502 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13, 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:08:17.503 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:08:17.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.505+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f896c06dc50 msgr2=0x7f896c070110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:17.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.505+0000 7f8987200700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f896c06dc50 0x7f896c070110 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8974005950 tx=0x7f89740058e0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.505+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 msgr2=0x7f8988198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:17.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.505+0000 7f8987200700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 0x7f8988198660 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f89780059d0 tx=0x7f897800be20 comp rx=0 tx=0).stop 2024-04-20T11:08:17.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.506+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 shutdown_connections 2024-04-20T11:08:17.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.506+0000 7f8987200700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f896c06dc50 0x7f896c070110 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.506+0000 7f8987200700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8988102c30 0x7f8988198660 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.506+0000 7f8987200700 1 --2- 172.21.15.117:0/532779160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8988103e30 0x7f8988198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.506+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 >> 172.21.15.117:0/532779160 conn(0x7f89880fe210 msgr2=0x7f8988107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:17.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.506+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 shutdown_connections 2024-04-20T11:08:17.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.506+0000 7f8987200700 1 -- 172.21.15.117:0/532779160 wait complete. 2024-04-20T11:08:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:17 smithi173 ceph-mon[23800]: pgmap v20: 65 pgs: 65 active+clean; 127 MiB data, 574 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 80 op/s 2024-04-20T11:08:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:17 smithi173 ceph-mon[23800]: Upgrade: Updating mgr.smithi117.ivnklj 2024-04-20T11:08:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:17 smithi173 ceph-mon[23800]: Deploying daemon mgr.smithi117.ivnklj on smithi117 2024-04-20T11:08:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:17 smithi173 ceph-mon[23800]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.685+0000 7f03d8400700 1 -- 172.21.15.117:0/4248365767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0100c30 msgr2=0x7f03d0101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.685+0000 7f03d8400700 1 --2- 172.21.15.117:0/4248365767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0100c30 0x7f03d0101050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009910 tx=0x7f03c4009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.687+0000 7f03d8400700 1 -- 172.21.15.117:0/4248365767 shutdown_connections 2024-04-20T11:08:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.687+0000 7f03d8400700 1 --2- 172.21.15.117:0/4248365767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03d0101e30 0x7f03d0102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.687+0000 7f03d8400700 1 --2- 172.21.15.117:0/4248365767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0100c30 0x7f03d0101050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.687+0000 7f03d8400700 1 -- 172.21.15.117:0/4248365767 >> 172.21.15.117:0/4248365767 conn(0x7f03d00fc1f0 msgr2=0x7f03d00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.687+0000 7f03d8400700 1 -- 172.21.15.117:0/4248365767 shutdown_connections 2024-04-20T11:08:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.687+0000 7f03d8400700 1 -- 172.21.15.117:0/4248365767 wait complete. 2024-04-20T11:08:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.688+0000 7f03d8400700 1 Processor -- start 2024-04-20T11:08:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.688+0000 7f03d8400700 1 -- start start 2024-04-20T11:08:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.689+0000 7f03d8400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03d0100c30 0x7f03d0196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.689+0000 7f03d8400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0101e30 0x7f03d0196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.689+0000 7f03d8400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d0197270 con 0x7f03d0101e30 2024-04-20T11:08:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.689+0000 7f03d8400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01973b0 con 0x7f03d0100c30 2024-04-20T11:08:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.690+0000 7f03d5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0101e30 0x7f03d0196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.690+0000 7f03d5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0101e30 0x7f03d0196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38446/0 (socket says 172.21.15.117:38446) 2024-04-20T11:08:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.690+0000 7f03d5000700 1 -- 172.21.15.117:0/2478117050 learned_addr learned my addr 172.21.15.117:0/2478117050 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.690+0000 7f03d5a00700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03d0100c30 0x7f03d0196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.690+0000 7f03d5000700 1 -- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03d0100c30 msgr2=0x7f03d0196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.690+0000 7f03d5000700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03d0100c30 0x7f03d0196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.690+0000 7f03d5000700 1 -- 172.21.15.117:0/2478117050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c40095f0 con 0x7f03d0101e30 2024-04-20T11:08:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.690+0000 7f03d5000700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0101e30 0x7f03d0196c50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00b4b0 tx=0x7f03bc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.691+0000 7f03ce000700 1 -- 172.21.15.117:0/2478117050 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03bc011800 con 0x7f03d0101e30 2024-04-20T11:08:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.691+0000 7f03ce000700 1 -- 172.21.15.117:0/2478117050 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03bc011e20 con 0x7f03d0101e30 2024-04-20T11:08:17.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.691+0000 7f03ce000700 1 -- 172.21.15.117:0/2478117050 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03bc00f2e0 con 0x7f03d0101e30 2024-04-20T11:08:17.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.692+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d019be10 con 0x7f03d0101e30 2024-04-20T11:08:17.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.692+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d019c360 con 0x7f03d0101e30 2024-04-20T11:08:17.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.693+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d0190830 con 0x7f03d0101e30 2024-04-20T11:08:17.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.694+0000 7f03ce000700 1 -- 172.21.15.117:0/2478117050 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) v1 ==== 91832+0+0 (secure 0 0 0) 0x7f03bc010380 con 0x7f03d0101e30 2024-04-20T11:08:17.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.696+0000 7f03ce000700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f03b806dc50 0x7f03b8070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:17.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.696+0000 7f03ce000700 1 -- 172.21.15.117:0/2478117050 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f03bc08d080 con 0x7f03d0101e30 2024-04-20T11:08:17.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.697+0000 7f03d5a00700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f03b806dc50 0x7f03b8070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:17.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.699+0000 7f03d5a00700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f03b806dc50 0x7f03b8070110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f03c400b5c0 tx=0x7f03c4005160 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:17.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:17.702+0000 7f03ce000700 1 -- 172.21.15.117:0/2478117050 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f03bc059e40 con 0x7f03d0101e30 2024-04-20T11:08:18.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.009+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 --> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03d002cea0 con 0x7f03b806dc50 2024-04-20T11:08:18.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.012+0000 7f03ce000700 1 -- 172.21.15.117:0/2478117050 <== mgr.24337 v2:172.21.15.173:6828/2992581883 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f03d002cea0 con 0x7f03b806dc50 2024-04-20T11:08:18.013 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:08:18.013 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:08:18.013 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:08:18.013 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:08:18.014 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [], 2024-04-20T11:08:18.014 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "1/2 daemons upgraded", 2024-04-20T11:08:18.014 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mgr daemons", 2024-04-20T11:08:18.014 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:08:18.014 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:08:18.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f03b806dc50 msgr2=0x7f03b8070110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:18.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f03b806dc50 0x7f03b8070110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f03c400b5c0 tx=0x7f03c4005160 comp rx=0 tx=0).stop 2024-04-20T11:08:18.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0101e30 msgr2=0x7f03d0196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:18.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0101e30 0x7f03d0196c50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00b4b0 tx=0x7f03bc00b870 comp rx=0 tx=0).stop 2024-04-20T11:08:18.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 shutdown_connections 2024-04-20T11:08:18.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:6828/2992581883,v1:172.21.15.173:6829/2992581883] conn(0x7f03b806dc50 0x7f03b8070110 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:18.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03d0100c30 0x7f03d0196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:18.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 --2- 172.21.15.117:0/2478117050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d0101e30 0x7f03d0196c50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:18.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.015+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 >> 172.21.15.117:0/2478117050 conn(0x7f03d00fc1f0 msgr2=0x7f03d0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:18.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.016+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 shutdown_connections 2024-04-20T11:08:18.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:18.016+0000 7f03d8400700 1 -- 172.21.15.117:0/2478117050 wait complete. 2024-04-20T11:08:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:18 smithi117 ceph-mon[15659]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:18.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:18 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/532779160' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:08:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:18 smithi173 ceph-mon[23800]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:18 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/532779160' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:08:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:19 smithi117 ceph-mon[15659]: pgmap v21: 65 pgs: 65 active+clean; 128 MiB data, 589 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 95 op/s 2024-04-20T11:08:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:19 smithi117 ceph-mon[15659]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:19 smithi173 ceph-mon[23800]: pgmap v21: 65 pgs: 65 active+clean; 128 MiB data, 589 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 95 op/s 2024-04-20T11:08:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:19 smithi173 ceph-mon[23800]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:21.615 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:21 smithi117 ceph-mon[15659]: pgmap v22: 65 pgs: 65 active+clean; 136 MiB data, 616 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 116 op/s 2024-04-20T11:08:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:21 smithi173 ceph-mon[23800]: pgmap v22: 65 pgs: 65 active+clean; 136 MiB data, 616 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 116 op/s 2024-04-20T11:08:23.589 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:23 smithi117 ceph-mon[15659]: pgmap v23: 65 pgs: 65 active+clean; 136 MiB data, 616 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 91 op/s 2024-04-20T11:08:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:23 smithi173 ceph-mon[23800]: pgmap v23: 65 pgs: 65 active+clean; 136 MiB data, 616 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 91 op/s 2024-04-20T11:08:25.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:25 smithi117 ceph-mon[15659]: pgmap v24: 65 pgs: 65 active+clean; 142 MiB data, 646 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 135 op/s 2024-04-20T11:08:25.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:25 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:25.104 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:25 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:25.105 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:25 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:08:25.105 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:25 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:25.105 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:25 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:25 smithi173 ceph-mon[23800]: pgmap v24: 65 pgs: 65 active+clean; 142 MiB data, 646 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 135 op/s 2024-04-20T11:08:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:25 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:25 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:25 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:08:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:25 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:25 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:27.371 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:27 smithi117 ceph-mon[15659]: pgmap v25: 65 pgs: 65 active+clean; 142 MiB data, 646 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 88 op/s 2024-04-20T11:08:27.371 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:27 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:08:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:27 smithi173 ceph-mon[23800]: pgmap v25: 65 pgs: 65 active+clean; 142 MiB data, 646 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 88 op/s 2024-04-20T11:08:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:27 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:08:29.261 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:29 smithi117 ceph-mon[15659]: pgmap v26: 65 pgs: 65 active+clean; 143 MiB data, 648 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 102 op/s 2024-04-20T11:08:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:29 smithi173 ceph-mon[23800]: pgmap v26: 65 pgs: 65 active+clean; 143 MiB data, 648 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 102 op/s 2024-04-20T11:08:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:31 smithi117 ceph-mon[15659]: pgmap v27: 65 pgs: 65 active+clean; 148 MiB data, 653 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 118 op/s 2024-04-20T11:08:31.343 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:31 smithi173 ceph-mon[23800]: pgmap v27: 65 pgs: 65 active+clean; 148 MiB data, 653 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.7 MiB/s wr, 118 op/s 2024-04-20T11:08:33.056 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:33 smithi117 ceph-mon[15659]: pgmap v28: 65 pgs: 65 active+clean; 148 MiB data, 653 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 90 op/s 2024-04-20T11:08:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:33 smithi173 ceph-mon[23800]: pgmap v28: 65 pgs: 65 active+clean; 148 MiB data, 653 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 90 op/s 2024-04-20T11:08:35.287 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:35 smithi117 ceph-mon[15659]: pgmap v29: 65 pgs: 65 active+clean; 161 MiB data, 697 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 170 op/s 2024-04-20T11:08:35.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:35 smithi173 ceph-mon[23800]: pgmap v29: 65 pgs: 65 active+clean; 161 MiB data, 697 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 170 op/s 2024-04-20T11:08:36.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:36 smithi117 ceph-mon[15659]: Standby manager daemon smithi117.ivnklj restarted 2024-04-20T11:08:36.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:36 smithi117 ceph-mon[15659]: Standby manager daemon smithi117.ivnklj started 2024-04-20T11:08:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:36 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.ivnklj/crt"}]: dispatch 2024-04-20T11:08:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:36 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:08:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:36 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.ivnklj/key"}]: dispatch 2024-04-20T11:08:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:36 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:08:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:36 smithi173 ceph-mon[23800]: Standby manager daemon smithi117.ivnklj restarted 2024-04-20T11:08:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:36 smithi173 ceph-mon[23800]: Standby manager daemon smithi117.ivnklj started 2024-04-20T11:08:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:36 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.ivnklj/crt"}]: dispatch 2024-04-20T11:08:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:36 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:08:36.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:36 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi117.ivnklj/key"}]: dispatch 2024-04-20T11:08:36.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:36 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:08:37.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:37 smithi117 ceph-mon[15659]: pgmap v30: 65 pgs: 65 active+clean; 161 MiB data, 697 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 127 op/s 2024-04-20T11:08:37.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:37 smithi117 ceph-mon[15659]: mgrmap e26: smithi173.gnrjof(active, since 54s), standbys: smithi117.ivnklj 2024-04-20T11:08:37.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:37 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:37.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:37 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:37 smithi173 ceph-mon[23800]: pgmap v30: 65 pgs: 65 active+clean; 161 MiB data, 697 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 127 op/s 2024-04-20T11:08:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:37 smithi173 ceph-mon[23800]: mgrmap e26: smithi173.gnrjof(active, since 54s), standbys: smithi117.ivnklj 2024-04-20T11:08:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:37 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:37 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:39.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:39 smithi117 ceph-mon[15659]: pgmap v31: 65 pgs: 65 active+clean; 169 MiB data, 708 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.3 MiB/s wr, 180 op/s 2024-04-20T11:08:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:39 smithi173 ceph-mon[23800]: pgmap v31: 65 pgs: 65 active+clean; 169 MiB data, 708 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.3 MiB/s wr, 180 op/s 2024-04-20T11:08:40.360 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:40 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:40.360 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:40 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:40 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:40 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:41.374 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:41 smithi117 ceph-mon[15659]: pgmap v32: 65 pgs: 65 active+clean; 171 MiB data, 729 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 197 op/s 2024-04-20T11:08:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:41 smithi173 ceph-mon[23800]: pgmap v32: 65 pgs: 65 active+clean; 171 MiB data, 729 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 197 op/s 2024-04-20T11:08:42.434 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:42 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:08:42.515 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:42 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:08:43.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:43 smithi117 ceph-mon[15659]: pgmap v33: 65 pgs: 65 active+clean; 171 MiB data, 729 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 165 op/s 2024-04-20T11:08:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:43 smithi173 ceph-mon[23800]: pgmap v33: 65 pgs: 65 active+clean; 171 MiB data, 729 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.0 MiB/s wr, 165 op/s 2024-04-20T11:08:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:45 smithi173 ceph-mon[23800]: pgmap v34: 65 pgs: 65 active+clean; 184 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 230 op/s 2024-04-20T11:08:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:45 smithi173 ceph-mon[23800]: mgrmap e27: smithi173.gnrjof(active, since 62s), standbys: smithi117.ivnklj 2024-04-20T11:08:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:45 smithi117 ceph-mon[15659]: pgmap v34: 65 pgs: 65 active+clean; 184 MiB data, 772 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 230 op/s 2024-04-20T11:08:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:45 smithi117 ceph-mon[15659]: mgrmap e27: smithi173.gnrjof(active, since 62s), standbys: smithi117.ivnklj 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr fail", "who": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr fail", "who": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:08:46.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: Activating manager daemon smithi117.ivnklj 2024-04-20T11:08:46.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: osdmap e49: 6 total, 6 up, 6 in 2024-04-20T11:08:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:08:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:08:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' 2024-04-20T11:08:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:08:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 172.21.15.173:0/3624951989' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr fail", "who": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "mgr fail", "who": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: Activating manager daemon smithi117.ivnklj 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: osdmap e49: 6 total, 6 up, 6 in 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd='[{"prefix": "mgr fail", "who": "smithi173.gnrjof"}]': finished 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: mgrmap e28: smithi117.ivnklj(active, starting, since 0.0208687s) 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:08:46.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: Manager daemon smithi117.ivnklj is now available 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:08:46.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/trash_purge_schedule"}]: dispatch 2024-04-20T11:08:46.735 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.24337 ' entity='mgr.smithi173.gnrjof' cmd='[{"prefix": "mgr fail", "who": "smithi173.gnrjof"}]': finished 2024-04-20T11:08:46.735 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: mgrmap e28: smithi117.ivnklj(active, starting, since 0.0208687s) 2024-04-20T11:08:46.735 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:08:46.735 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:08:46.735 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:08:46.735 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: Manager daemon smithi117.ivnklj is now available 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:08:46.736 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:08:46.737 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:08:46.737 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/trash_purge_schedule"}]: dispatch 2024-04-20T11:08:47.145 INFO:tasks.workunit.client.0.smithi117.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-20T11:08:47.167 INFO:tasks.workunit.client.0.smithi117.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-20T11:08:47.168 INFO:tasks.workunit.client.0.smithi117.stderr:+ make 2024-04-20T11:08:47.441 INFO:tasks.workunit.client.0.smithi117.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-20T11:08:47.869 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:47 smithi173 ceph-mon[23800]: [20/Apr/2024:11:08:46] ENGINE Bus STARTING 2024-04-20T11:08:47.869 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:47 smithi173 ceph-mon[23800]: [20/Apr/2024:11:08:46] ENGINE Serving on https://172.21.15.117:7150 2024-04-20T11:08:47.869 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:47 smithi173 ceph-mon[23800]: mgrmap e29: smithi117.ivnklj(active, since 1.0208s) 2024-04-20T11:08:47.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:47 smithi117 ceph-mon[15659]: [20/Apr/2024:11:08:46] ENGINE Bus STARTING 2024-04-20T11:08:47.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:47 smithi117 ceph-mon[15659]: [20/Apr/2024:11:08:46] ENGINE Serving on https://172.21.15.117:7150 2024-04-20T11:08:47.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:47 smithi117 ceph-mon[15659]: mgrmap e29: smithi117.ivnklj(active, since 1.0208s) 2024-04-20T11:08:48.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.203+0000 7f6de7000700 1 -- 172.21.15.117:0/46900882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 msgr2=0x7f6de00732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:48.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.203+0000 7f6de7000700 1 --2- 172.21.15.117:0/46900882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 0x7f6de00732f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009910 tx=0x7f6dd0009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:48.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.203+0000 7f6de7000700 1 -- 172.21.15.117:0/46900882 shutdown_connections 2024-04-20T11:08:48.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.203+0000 7f6de7000700 1 --2- 172.21.15.117:0/46900882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6de00738c0 0x7f6de0073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.203+0000 7f6de7000700 1 --2- 172.21.15.117:0/46900882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 0x7f6de00732f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.203+0000 7f6de7000700 1 -- 172.21.15.117:0/46900882 >> 172.21.15.117:0/46900882 conn(0x7f6de00fc1f0 msgr2=0x7f6de00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:48.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.203+0000 7f6de7000700 1 -- 172.21.15.117:0/46900882 shutdown_connections 2024-04-20T11:08:48.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.203+0000 7f6de7000700 1 -- 172.21.15.117:0/46900882 wait complete. 2024-04-20T11:08:48.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.205+0000 7f6de7000700 1 Processor -- start 2024-04-20T11:08:48.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.205+0000 7f6de7000700 1 -- start start 2024-04-20T11:08:48.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.206+0000 7f6de7000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6de00738c0 0x7f6de0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:48.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.206+0000 7f6de7000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 0x7f6de0198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:48.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.206+0000 7f6de7000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01992d0 con 0x7f6de0074e90 2024-04-20T11:08:48.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.206+0000 7f6de7000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0199410 con 0x7f6de00738c0 2024-04-20T11:08:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.206+0000 7f6ddf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 0x7f6de0198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.206+0000 7f6ddf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 0x7f6de0198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57570/0 (socket says 172.21.15.117:57570) 2024-04-20T11:08:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.206+0000 7f6ddf400700 1 -- 172.21.15.117:0/16453505 learned_addr learned my addr 172.21.15.117:0/16453505 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.206+0000 7f6ddfe00700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6de00738c0 0x7f6de0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:48.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.207+0000 7f6ddf400700 1 -- 172.21.15.117:0/16453505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6de00738c0 msgr2=0x7f6de0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:48.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.207+0000 7f6ddf400700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6de00738c0 0x7f6de0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.207+0000 7f6ddf400700 1 -- 172.21.15.117:0/16453505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00095f0 con 0x7f6de0074e90 2024-04-20T11:08:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.207+0000 7f6ddf400700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 0x7f6de0198d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c930 tx=0x7f6dcc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.207+0000 7f6ddcc00700 1 -- 172.21.15.117:0/16453505 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dcc007ad0 con 0x7f6de0074e90 2024-04-20T11:08:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.207+0000 7f6ddcc00700 1 -- 172.21.15.117:0/16453505 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dcc00ce80 con 0x7f6de0074e90 2024-04-20T11:08:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.208+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de019de80 con 0x7f6de0074e90 2024-04-20T11:08:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.208+0000 7f6ddcc00700 1 -- 172.21.15.117:0/16453505 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dcc018630 con 0x7f6de0074e90 2024-04-20T11:08:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.208+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de019e420 con 0x7f6de0074e90 2024-04-20T11:08:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.209+0000 7f6ddcc00700 1 -- 172.21.15.117:0/16453505 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 30) v1 ==== 46651+0+0 (secure 0 0 0) 0x7f6dcc018790 con 0x7f6de0074e90 2024-04-20T11:08:48.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.209+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de01929a0 con 0x7f6de0074e90 2024-04-20T11:08:48.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.210+0000 7f6ddcc00700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6dc4039840 0x7f6dc403bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:48.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.210+0000 7f6ddcc00700 1 -- 172.21.15.117:0/16453505 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f6dcc04db70 con 0x7f6de0074e90 2024-04-20T11:08:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.210+0000 7f6ddfe00700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6dc4039840 0x7f6dc403bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.211+0000 7f6ddfe00700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6dc4039840 0x7f6dc403bd00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0005090 tx=0x7f6dd0005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:48.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.216+0000 7f6ddcc00700 1 -- 172.21.15.117:0/16453505 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f6dcc01fe00 con 0x7f6de0074e90 2024-04-20T11:08:48.503 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:48 smithi117 ceph-mon[15659]: [20/Apr/2024:11:08:46] ENGINE Serving on http://172.21.15.117:8765 2024-04-20T11:08:48.503 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:48 smithi117 ceph-mon[15659]: [20/Apr/2024:11:08:46] ENGINE Bus STARTED 2024-04-20T11:08:48.503 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:48 smithi117 ceph-mon[15659]: pgmap v3: 65 pgs: 65 active+clean; 193 MiB data, 807 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:08:48.504 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:48 smithi117 ceph-mon[15659]: mgrmap e30: smithi117.ivnklj(active, since 2s) 2024-04-20T11:08:48.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.572+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6de002cef0 con 0x7f6dc4039840 2024-04-20T11:08:48.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.575+0000 7f6ddcc00700 1 -- 172.21.15.117:0/16453505 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f6de002cef0 con 0x7f6dc4039840 2024-04-20T11:08:48.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.579+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6dc4039840 msgr2=0x7f6dc403bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:48.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.579+0000 7f6de7000700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6dc4039840 0x7f6dc403bd00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0005090 tx=0x7f6dd0005960 comp rx=0 tx=0).stop 2024-04-20T11:08:48.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.579+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 msgr2=0x7f6de0198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:48.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.579+0000 7f6de7000700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 0x7f6de0198d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c930 tx=0x7f6dcc00cc40 comp rx=0 tx=0).stop 2024-04-20T11:08:48.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.580+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 shutdown_connections 2024-04-20T11:08:48.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.580+0000 7f6de7000700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6dc4039840 0x7f6dc403bd00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.580+0000 7f6de7000700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6de00738c0 0x7f6de0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.580+0000 7f6de7000700 1 --2- 172.21.15.117:0/16453505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de0074e90 0x7f6de0198d40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.580+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 >> 172.21.15.117:0/16453505 conn(0x7f6de00fc1f0 msgr2=0x7f6de0106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:48.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.581+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 shutdown_connections 2024-04-20T11:08:48.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.581+0000 7f6de7000700 1 -- 172.21.15.117:0/16453505 wait complete. 2024-04-20T11:08:48.606 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:08:48.616 INFO:tasks.workunit.client.0.smithi117.stderr:++ readlink -f fsstress 2024-04-20T11:08:48.622 INFO:tasks.workunit.client.0.smithi117.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-20T11:08:48.622 INFO:tasks.workunit.client.0.smithi117.stderr:+ popd 2024-04-20T11:08:48.626 INFO:tasks.workunit.client.0.smithi117.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-20T11:08:48.626 INFO:tasks.workunit.client.0.smithi117.stderr:+ popd 2024-04-20T11:08:48.628 INFO:tasks.workunit.client.0.smithi117.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-20T11:08:48.629 INFO:tasks.workunit.client.0.smithi117.stderr:++ mktemp -d -p . 2024-04-20T11:08:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.807+0000 7f6ccf400700 1 -- 172.21.15.117:0/2687258442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc80ffde0 msgr2=0x7f6cc8100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.807+0000 7f6ccf400700 1 --2- 172.21.15.117:0/2687258442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc80ffde0 0x7f6cc8100260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009910 tx=0x7f6cb4009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.807+0000 7f6ccf400700 1 -- 172.21.15.117:0/2687258442 shutdown_connections 2024-04-20T11:08:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.807+0000 7f6ccf400700 1 --2- 172.21.15.117:0/2687258442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc80ffde0 0x7f6cc8100260 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.807+0000 7f6ccf400700 1 --2- 172.21.15.117:0/2687258442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc80ff480 0x7f6cc80ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.807+0000 7f6ccf400700 1 -- 172.21.15.117:0/2687258442 >> 172.21.15.117:0/2687258442 conn(0x7f6cc80fb060 msgr2=0x7f6cc80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.808+0000 7f6ccf400700 1 -- 172.21.15.117:0/2687258442 shutdown_connections 2024-04-20T11:08:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.808+0000 7f6ccf400700 1 -- 172.21.15.117:0/2687258442 wait complete. 2024-04-20T11:08:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.809+0000 7f6ccf400700 1 Processor -- start 2024-04-20T11:08:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.809+0000 7f6ccf400700 1 -- start start 2024-04-20T11:08:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.809+0000 7f6ccf400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc80ff480 0x7f6cc8104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.809+0000 7f6ccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc80ffde0 0x7f6cc81054f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.809+0000 7f6ccf400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8101af0 con 0x7f6cc80ffde0 2024-04-20T11:08:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.809+0000 7f6ccf400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8101c30 con 0x7f6cc80ff480 2024-04-20T11:08:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.810+0000 7f6ccca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc80ff480 0x7f6cc8104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:48.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.810+0000 7f6ccca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc80ff480 0x7f6cc8104fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:38542/0 (socket says 172.21.15.117:38542) 2024-04-20T11:08:48.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.810+0000 7f6ccca00700 1 -- 172.21.15.117:0/2910057410 learned_addr learned my addr 172.21.15.117:0/2910057410 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:48.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.810+0000 7f6cc7e00700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc80ffde0 0x7f6cc81054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:48.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.810+0000 7f6ccca00700 1 -- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc80ffde0 msgr2=0x7f6cc81054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:48.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.810+0000 7f6ccca00700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc80ffde0 0x7f6cc81054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:48.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.810+0000 7f6ccca00700 1 -- 172.21.15.117:0/2910057410 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40095f0 con 0x7f6cc80ff480 2024-04-20T11:08:48.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.811+0000 7f6ccca00700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc80ff480 0x7f6cc8104fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e920 tx=0x7f6cbc00ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:48.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.812+0000 7f6cc5600700 1 -- 172.21.15.117:0/2910057410 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cbc00c9d0 con 0x7f6cc80ff480 2024-04-20T11:08:48.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.812+0000 7f6cc5600700 1 -- 172.21.15.117:0/2910057410 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cbc010430 con 0x7f6cc80ff480 2024-04-20T11:08:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.812+0000 7f6cc5600700 1 -- 172.21.15.117:0/2910057410 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cbc018630 con 0x7f6cc80ff480 2024-04-20T11:08:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.812+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc8101f10 con 0x7f6cc80ff480 2024-04-20T11:08:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.812+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc81024e0 con 0x7f6cc80ff480 2024-04-20T11:08:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.813+0000 7f6cc5600700 1 -- 172.21.15.117:0/2910057410 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 30) v1 ==== 46651+0+0 (secure 0 0 0) 0x7f6cbc018790 con 0x7f6cc80ff480 2024-04-20T11:08:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.814+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc8192900 con 0x7f6cc80ff480 2024-04-20T11:08:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.814+0000 7f6cc5600700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6cb0039870 0x7f6cb003bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.815+0000 7f6cc5600700 1 -- 172.21.15.117:0/2910057410 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f6cbc014070 con 0x7f6cc80ff480 2024-04-20T11:08:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.815+0000 7f6cc7e00700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6cb0039870 0x7f6cb003bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.816+0000 7f6cc7e00700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6cb0039870 0x7f6cb003bd30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400b6f0 tx=0x7f6cb4005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:48.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:48.828+0000 7f6cc5600700 1 -- 172.21.15.117:0/2910057410 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f6cbc0152d0 con 0x7f6cc80ff480 2024-04-20T11:08:48.870 INFO:tasks.workunit.client.0.smithi117.stderr:+ T=./tmp.akUEe7qXEm 2024-04-20T11:08:48.870 INFO:tasks.workunit.client.0.smithi117.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.akUEe7qXEm -l 1 -n 1000 -p 10 -v 2024-04-20T11:08:48.877 INFO:tasks.workunit.client.0.smithi117.stdout:seed = 1714341353 2024-04-20T11:08:48.883 INFO:tasks.workunit.client.0.smithi117.stdout:0/0: rmdir - no directory 2024-04-20T11:08:48.883 INFO:tasks.workunit.client.0.smithi117.stdout:0/1: read - no filename 2024-04-20T11:08:48.885 INFO:tasks.workunit.client.0.smithi117.stdout:0/2: symlink l0 0 2024-04-20T11:08:48.886 INFO:tasks.workunit.client.0.smithi117.stdout:0/3: chown l0 790 1 2024-04-20T11:08:48.886 INFO:tasks.workunit.client.0.smithi117.stdout:0/4: write - no filename 2024-04-20T11:08:48.887 INFO:tasks.workunit.client.0.smithi117.stdout:0/5: write - no filename 2024-04-20T11:08:48.889 INFO:tasks.workunit.client.0.smithi117.stdout:9/0: creat f0 x:0 0 0 2024-04-20T11:08:48.890 INFO:tasks.workunit.client.0.smithi117.stdout:9/1: chown f0 0 1 2024-04-20T11:08:48.893 INFO:tasks.workunit.client.0.smithi117.stdout:1/0: creat f0 x:0 0 0 2024-04-20T11:08:48.898 INFO:tasks.workunit.client.0.smithi117.stdout:9/2: creat f1 x:0 0 0 2024-04-20T11:08:48.899 INFO:tasks.workunit.client.0.smithi117.stdout:9/3: stat f1 0 2024-04-20T11:08:48.900 INFO:tasks.workunit.client.0.smithi117.stdout:3/0: fdatasync - no filename 2024-04-20T11:08:48.900 INFO:tasks.workunit.client.0.smithi117.stdout:3/1: unlink - no file 2024-04-20T11:08:48.900 INFO:tasks.workunit.client.0.smithi117.stdout:3/2: dwrite - no filename 2024-04-20T11:08:48.901 INFO:tasks.workunit.client.0.smithi117.stdout:2/0: getdents . 0 2024-04-20T11:08:48.905 INFO:tasks.workunit.client.0.smithi117.stdout:4/0: dwrite - no filename 2024-04-20T11:08:48.906 INFO:tasks.workunit.client.0.smithi117.stdout:4/1: chown . 147326 1 2024-04-20T11:08:48.906 INFO:tasks.workunit.client.0.smithi117.stdout:4/2: chown . 0 1 2024-04-20T11:08:48.906 INFO:tasks.workunit.client.0.smithi117.stdout:4/3: chown . 5 1 2024-04-20T11:08:48.906 INFO:tasks.workunit.client.0.smithi117.stdout:4/4: stat - no entries 2024-04-20T11:08:48.906 INFO:tasks.workunit.client.0.smithi117.stdout:4/5: dwrite - no filename 2024-04-20T11:08:48.906 INFO:tasks.workunit.client.0.smithi117.stdout:4/6: stat - no entries 2024-04-20T11:08:48.906 INFO:tasks.workunit.client.0.smithi117.stdout:4/7: dwrite - no filename 2024-04-20T11:08:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:48 smithi173 ceph-mon[23800]: [20/Apr/2024:11:08:46] ENGINE Serving on http://172.21.15.117:8765 2024-04-20T11:08:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:48 smithi173 ceph-mon[23800]: [20/Apr/2024:11:08:46] ENGINE Bus STARTED 2024-04-20T11:08:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:48 smithi173 ceph-mon[23800]: pgmap v3: 65 pgs: 65 active+clean; 193 MiB data, 807 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:08:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:48 smithi173 ceph-mon[23800]: mgrmap e30: smithi117.ivnklj(active, since 2s) 2024-04-20T11:08:48.965 INFO:tasks.workunit.client.0.smithi117.stdout:3/3: mkdir d0 0 2024-04-20T11:08:48.966 INFO:tasks.workunit.client.0.smithi117.stdout:2/1: creat f0 x:0 0 0 2024-04-20T11:08:48.967 INFO:tasks.workunit.client.0.smithi117.stdout:3/4: chown d0 2 1 2024-04-20T11:08:48.968 INFO:tasks.workunit.client.0.smithi117.stdout:9/4: dwrite f1 [0,4194304] 0 2024-04-20T11:08:48.972 INFO:tasks.workunit.client.0.smithi117.stdout:3/5: chown d0 2538086 1 2024-04-20T11:08:48.972 INFO:tasks.workunit.client.0.smithi117.stdout:3/6: dread - no filename 2024-04-20T11:08:48.972 INFO:tasks.workunit.client.0.smithi117.stdout:3/7: fdatasync - no filename 2024-04-20T11:08:48.974 INFO:tasks.workunit.client.0.smithi117.stdout:2/2: mkdir d1 0 2024-04-20T11:08:48.975 INFO:tasks.workunit.client.0.smithi117.stdout:9/5: creat f2 x:0 0 0 2024-04-20T11:08:48.982 INFO:tasks.workunit.client.0.smithi117.stdout:4/8: creat f0 x:0 0 0 2024-04-20T11:08:48.983 INFO:tasks.workunit.client.0.smithi117.stdout:4/9: dread - f0 zero size 2024-04-20T11:08:48.991 INFO:tasks.workunit.client.0.smithi117.stdout:6/0: fsync - no filename 2024-04-20T11:08:48.991 INFO:tasks.workunit.client.0.smithi117.stdout:9/6: dread f1 [0,4194304] 0 2024-04-20T11:08:49.001 INFO:tasks.workunit.client.0.smithi117.stdout:1/1: getdents . 0 2024-04-20T11:08:49.003 INFO:tasks.workunit.client.0.smithi117.stdout:5/0: creat f0 x:0 0 0 2024-04-20T11:08:49.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/0: read - no filename 2024-04-20T11:08:49.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/1: dread - no filename 2024-04-20T11:08:49.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/2: dread - no filename 2024-04-20T11:08:49.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/3: dwrite - no filename 2024-04-20T11:08:49.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/4: rename - no filename 2024-04-20T11:08:49.011 INFO:tasks.workunit.client.0.smithi117.stdout:3/8: mknod d0/c1 0 2024-04-20T11:08:49.011 INFO:tasks.workunit.client.0.smithi117.stdout:2/3: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.013 INFO:tasks.workunit.client.0.smithi117.stdout:2/4: truncate f0 4239364 0 2024-04-20T11:08:49.016 INFO:tasks.workunit.client.0.smithi117.stdout:9/7: mknod c3 0 2024-04-20T11:08:49.018 INFO:tasks.workunit.client.0.smithi117.stdout:1/2: mknod c1 0 2024-04-20T11:08:49.019 INFO:tasks.workunit.client.0.smithi117.stdout:2/5: write f0 [1447419,118558] 0 2024-04-20T11:08:49.019 INFO:tasks.workunit.client.0.smithi117.stdout:6/1: symlink l0 0 2024-04-20T11:08:49.023 INFO:tasks.workunit.client.0.smithi117.stdout:3/9: creat d0/f2 x:0 0 0 2024-04-20T11:08:49.025 INFO:tasks.workunit.client.0.smithi117.stdout:1/3: write f0 [289032,121230] 0 2024-04-20T11:08:49.026 INFO:tasks.workunit.client.0.smithi117.stdout:4/10: fsync f0 0 2024-04-20T11:08:49.026 INFO:tasks.workunit.client.0.smithi117.stdout:9/8: mkdir d4 0 2024-04-20T11:08:49.027 INFO:tasks.workunit.client.0.smithi117.stdout:7/5: mkdir d0 0 2024-04-20T11:08:49.034 INFO:tasks.workunit.client.0.smithi117.stdout:7/6: chown d0 0 1 2024-04-20T11:08:49.035 INFO:tasks.workunit.client.0.smithi117.stdout:4/11: dread - f0 zero size 2024-04-20T11:08:49.040 INFO:tasks.workunit.client.0.smithi117.stdout:6/2: link l0 l1 0 2024-04-20T11:08:49.040 INFO:tasks.workunit.client.0.smithi117.stdout:1/4: write f0 [1135520,63367] 0 2024-04-20T11:08:49.045 INFO:tasks.workunit.client.0.smithi117.stdout:8/0: mkdir d0 0 2024-04-20T11:08:49.045 INFO:tasks.workunit.client.0.smithi117.stdout:8/1: dread - no filename 2024-04-20T11:08:49.056 INFO:tasks.workunit.client.0.smithi117.stdout:3/10: link d0/f2 d0/f3 0 2024-04-20T11:08:49.058 INFO:tasks.workunit.client.0.smithi117.stdout:2/6: truncate f0 2974504 0 2024-04-20T11:08:49.059 INFO:tasks.workunit.client.0.smithi117.stdout:5/1: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.062 INFO:tasks.workunit.client.0.smithi117.stdout:7/7: getdents d0 0 2024-04-20T11:08:49.067 INFO:tasks.workunit.client.0.smithi117.stdout:1/5: link f0 f2 0 2024-04-20T11:08:49.067 INFO:tasks.workunit.client.0.smithi117.stdout:6/3: chown l0 18808 1 2024-04-20T11:08:49.069 INFO:tasks.workunit.client.0.smithi117.stdout:9/9: dread f1 [0,4194304] 0 2024-04-20T11:08:49.070 INFO:tasks.workunit.client.0.smithi117.stdout:8/2: creat d0/f1 x:0 0 0 2024-04-20T11:08:49.072 INFO:tasks.workunit.client.0.smithi117.stdout:3/11: dread - d0/f2 zero size 2024-04-20T11:08:49.075 INFO:tasks.workunit.client.0.smithi117.stdout:6/4: chown l0 1970676 1 2024-04-20T11:08:49.085 INFO:tasks.workunit.client.0.smithi117.stdout:4/12: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.090 INFO:tasks.workunit.client.0.smithi117.stdout:6/5: rename l1 to l2 0 2024-04-20T11:08:49.090 INFO:tasks.workunit.client.0.smithi117.stdout:6/6: rmdir - no directory 2024-04-20T11:08:49.095 INFO:tasks.workunit.client.0.smithi117.stdout:9/10: mkdir d4/d5 0 2024-04-20T11:08:49.095 INFO:tasks.workunit.client.0.smithi117.stdout:8/3: symlink d0/l2 0 2024-04-20T11:08:49.095 INFO:tasks.workunit.client.0.smithi117.stdout:9/11: fsync f2 0 2024-04-20T11:08:49.095 INFO:tasks.workunit.client.0.smithi117.stdout:8/4: chown d0/f1 711 1 2024-04-20T11:08:49.095 INFO:tasks.workunit.client.0.smithi117.stdout:7/8: rmdir d0 0 2024-04-20T11:08:49.095 INFO:tasks.workunit.client.0.smithi117.stdout:8/5: fsync d0/f1 0 2024-04-20T11:08:49.096 INFO:tasks.workunit.client.0.smithi117.stdout:1/6: write f0 [81666,123812] 0 2024-04-20T11:08:49.098 INFO:tasks.workunit.client.0.smithi117.stdout:4/13: mkdir d1 0 2024-04-20T11:08:49.098 INFO:tasks.workunit.client.0.smithi117.stdout:3/12: write d0/f3 [63184,4283] 0 2024-04-20T11:08:49.098 INFO:tasks.workunit.client.0.smithi117.stdout:5/2: truncate f0 2600388 0 2024-04-20T11:08:49.100 INFO:tasks.workunit.client.0.smithi117.stdout:6/7: mkdir d3 0 2024-04-20T11:08:49.100 INFO:tasks.workunit.client.0.smithi117.stdout:6/8: dread - no filename 2024-04-20T11:08:49.101 INFO:tasks.workunit.client.0.smithi117.stdout:7/9: mkdir d1 0 2024-04-20T11:08:49.104 INFO:tasks.workunit.client.0.smithi117.stdout:9/12: rename f1 to d4/f6 0 2024-04-20T11:08:49.106 INFO:tasks.workunit.client.0.smithi117.stdout:3/13: unlink d0/f2 0 2024-04-20T11:08:49.107 INFO:tasks.workunit.client.0.smithi117.stdout:2/7: dread f0 [0,4194304] 0 2024-04-20T11:08:49.109 INFO:tasks.workunit.client.0.smithi117.stdout:8/6: link d0/f1 d0/f3 0 2024-04-20T11:08:49.109 INFO:tasks.workunit.client.0.smithi117.stdout:7/10: rename d1 to d1/d2 22 2024-04-20T11:08:49.109 INFO:tasks.workunit.client.0.smithi117.stdout:7/11: link - no file 2024-04-20T11:08:49.109 INFO:tasks.workunit.client.0.smithi117.stdout:7/12: dread - no filename 2024-04-20T11:08:49.111 INFO:tasks.workunit.client.0.smithi117.stdout:4/14: creat d1/f2 x:0 0 0 2024-04-20T11:08:49.111 INFO:tasks.workunit.client.0.smithi117.stdout:9/13: write f0 [185164,63140] 0 2024-04-20T11:08:49.113 INFO:tasks.workunit.client.0.smithi117.stdout:6/9: mkdir d3/d4 0 2024-04-20T11:08:49.116 INFO:tasks.workunit.client.0.smithi117.stdout:2/8: write f0 [3579046,106772] 0 2024-04-20T11:08:49.116 INFO:tasks.workunit.client.0.smithi117.stdout:6/10: rename d3/d4 to d3/d4/d5 22 2024-04-20T11:08:49.118 INFO:tasks.workunit.client.0.smithi117.stdout:8/7: creat d0/f4 x:0 0 0 2024-04-20T11:08:49.118 INFO:tasks.workunit.client.0.smithi117.stdout:4/15: symlink d1/l3 0 2024-04-20T11:08:49.121 INFO:tasks.workunit.client.0.smithi117.stdout:8/8: read - d0/f1 zero size 2024-04-20T11:08:49.122 INFO:tasks.workunit.client.0.smithi117.stdout:7/13: creat d1/f3 x:0 0 0 2024-04-20T11:08:49.124 INFO:tasks.workunit.client.0.smithi117.stdout:7/14: chown d1 11 1 2024-04-20T11:08:49.126 INFO:tasks.workunit.client.0.smithi117.stdout:6/11: readlink l0 0 2024-04-20T11:08:49.127 INFO:tasks.workunit.client.0.smithi117.stdout:4/16: mknod d1/c4 0 2024-04-20T11:08:49.129 INFO:tasks.workunit.client.0.smithi117.stdout:3/14: link d0/c1 d0/c4 0 2024-04-20T11:08:49.130 INFO:tasks.workunit.client.0.smithi117.stdout:4/17: chown d1 12827559 1 2024-04-20T11:08:49.138 INFO:tasks.workunit.client.0.smithi117.stdout:9/14: creat d4/d5/f7 x:0 0 0 2024-04-20T11:08:49.138 INFO:tasks.workunit.client.0.smithi117.stdout:4/18: chown d1/c4 106505 1 2024-04-20T11:08:49.138 INFO:tasks.workunit.client.0.smithi117.stdout:2/9: link f0 d1/f2 0 2024-04-20T11:08:49.140 INFO:tasks.workunit.client.0.smithi117.stdout:5/3: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.141 INFO:tasks.workunit.client.0.smithi117.stdout:6/12: creat d3/f6 x:0 0 0 2024-04-20T11:08:49.141 INFO:tasks.workunit.client.0.smithi117.stdout:3/15: mkdir d0/d5 0 2024-04-20T11:08:49.145 INFO:tasks.workunit.client.0.smithi117.stdout:5/4: read f0 [1581154,32411] 0 2024-04-20T11:08:49.145 INFO:tasks.workunit.client.0.smithi117.stdout:9/15: link f0 d4/f8 0 2024-04-20T11:08:49.146 INFO:tasks.workunit.client.0.smithi117.stdout:4/19: write d1/f2 [330291,23816] 0 2024-04-20T11:08:49.149 INFO:tasks.workunit.client.0.smithi117.stdout:6/13: dread - d3/f6 zero size 2024-04-20T11:08:49.153 INFO:tasks.workunit.client.0.smithi117.stdout:9/16: rename d4 to d4/d5/d9 22 2024-04-20T11:08:49.154 INFO:tasks.workunit.client.0.smithi117.stdout:2/10: link d1/f2 d1/f3 0 2024-04-20T11:08:49.163 INFO:tasks.workunit.client.0.smithi117.stdout:9/17: write f2 [571819,116416] 0 2024-04-20T11:08:49.169 INFO:tasks.workunit.client.0.smithi117.stdout:3/16: link d0/c4 d0/c6 0 2024-04-20T11:08:49.170 INFO:tasks.workunit.client.0.smithi117.stdout:6/14: truncate d3/f6 9246 0 2024-04-20T11:08:49.175 INFO:tasks.workunit.client.0.smithi117.stdout:7/15: dwrite d1/f3 [0,4194304] 0 2024-04-20T11:08:49.178 INFO:tasks.workunit.client.0.smithi117.stdout:6/15: dread d3/f6 [0,4194304] 0 2024-04-20T11:08:49.182 INFO:tasks.workunit.client.0.smithi117.stdout:7/16: mkdir d1/d4 0 2024-04-20T11:08:49.186 INFO:tasks.workunit.client.0.smithi117.stdout:6/16: read d3/f6 [1739,77124] 0 2024-04-20T11:08:49.191 INFO:tasks.workunit.client.0.smithi117.stdout:5/5: dread f0 [0,4194304] 0 2024-04-20T11:08:49.196 INFO:tasks.workunit.client.0.smithi117.stdout:6/17: rename d3/d4 to d3/d4/d7 22 2024-04-20T11:08:49.196 INFO:tasks.workunit.client.0.smithi117.stdout:4/20: dread f0 [0,4194304] 0 2024-04-20T11:08:49.198 INFO:tasks.workunit.client.0.smithi117.stdout:8/9: dwrite d0/f3 [0,4194304] 0 2024-04-20T11:08:49.206 INFO:tasks.workunit.client.0.smithi117.stdout:6/18: creat d3/d4/f8 x:0 0 0 2024-04-20T11:08:49.206 INFO:tasks.workunit.client.0.smithi117.stdout:2/11: dread f0 [0,4194304] 0 2024-04-20T11:08:49.206 INFO:tasks.workunit.client.0.smithi117.stdout:6/19: chown l0 1946164846 1 2024-04-20T11:08:49.206 INFO:tasks.workunit.client.0.smithi117.stdout:9/18: dread d4/f6 [0,4194304] 0 2024-04-20T11:08:49.213 INFO:tasks.workunit.client.0.smithi117.stdout:8/10: unlink d0/f4 0 2024-04-20T11:08:49.213 INFO:tasks.workunit.client.0.smithi117.stdout:7/17: dread d1/f3 [0,4194304] 0 2024-04-20T11:08:49.221 INFO:tasks.workunit.client.0.smithi117.stdout:8/11: creat d0/f5 x:0 0 0 2024-04-20T11:08:49.224 INFO:tasks.workunit.client.0.smithi117.stdout:8/12: symlink d0/l6 0 2024-04-20T11:08:49.229 INFO:tasks.workunit.client.0.smithi117.stdout:3/17: mkdir d0/d5/d7 0 2024-04-20T11:08:49.234 INFO:tasks.workunit.client.0.smithi117.stdout:4/21: dread f0 [0,4194304] 0 2024-04-20T11:08:49.235 INFO:tasks.workunit.client.0.smithi117.stdout:8/13: creat d0/f7 x:0 0 0 2024-04-20T11:08:49.240 INFO:tasks.workunit.client.0.smithi117.stdout:2/12: write f0 [3911246,40047] 0 2024-04-20T11:08:49.241 INFO:tasks.workunit.client.0.smithi117.stdout:9/19: write f0 [1199084,82866] 0 2024-04-20T11:08:49.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.242+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cc802cdb0 con 0x7f6cb0039870 2024-04-20T11:08:49.245 INFO:tasks.workunit.client.0.smithi117.stdout:3/18: symlink d0/d5/d7/l8 0 2024-04-20T11:08:49.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.245+0000 7f6cc5600700 1 -- 172.21.15.117:0/2910057410 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f6cc802cdb0 con 0x7f6cb0039870 2024-04-20T11:08:49.246 INFO:tasks.workunit.client.0.smithi117.stdout:8/14: mkdir d0/d8 0 2024-04-20T11:08:49.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6cb0039870 msgr2=0x7f6cb003bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:49.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6cb0039870 0x7f6cb003bd30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400b6f0 tx=0x7f6cb4005fb0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc80ff480 msgr2=0x7f6cc8104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:49.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc80ff480 0x7f6cc8104fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e920 tx=0x7f6cbc00ec30 comp rx=0 tx=0).stop 2024-04-20T11:08:49.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 shutdown_connections 2024-04-20T11:08:49.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6cb0039870 0x7f6cb003bd30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc80ff480 0x7f6cc8104fb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 --2- 172.21.15.117:0/2910057410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc80ffde0 0x7f6cc81054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.249+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 >> 172.21.15.117:0/2910057410 conn(0x7f6cc80fb060 msgr2=0x7f6cc8107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:49.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.250+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 shutdown_connections 2024-04-20T11:08:49.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.250+0000 7f6ccf400700 1 -- 172.21.15.117:0/2910057410 wait complete. 2024-04-20T11:08:49.256 INFO:tasks.workunit.client.0.smithi117.stdout:8/15: fdatasync d0/f1 0 2024-04-20T11:08:49.257 INFO:tasks.workunit.client.0.smithi117.stdout:9/20: write d4/d5/f7 [375683,56359] 0 2024-04-20T11:08:49.260 INFO:tasks.workunit.client.0.smithi117.stdout:6/20: dwrite d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:49.261 INFO:tasks.workunit.client.0.smithi117.stdout:5/6: dwrite f0 [4194304,4194304] 0 2024-04-20T11:08:49.266 INFO:tasks.workunit.client.0.smithi117.stdout:3/19: unlink d0/c4 0 2024-04-20T11:08:49.271 INFO:tasks.workunit.client.0.smithi117.stdout:7/18: dwrite d1/f3 [0,4194304] 0 2024-04-20T11:08:49.272 INFO:tasks.workunit.client.0.smithi117.stdout:9/21: mkdir d4/d5/da 0 2024-04-20T11:08:49.273 INFO:tasks.workunit.client.0.smithi117.stdout:4/22: dread f0 [0,4194304] 0 2024-04-20T11:08:49.274 INFO:tasks.workunit.client.0.smithi117.stdout:7/19: write d1/f3 [998556,41176] 0 2024-04-20T11:08:49.277 INFO:tasks.workunit.client.0.smithi117.stdout:4/23: write d1/f2 [79261,15245] 0 2024-04-20T11:08:49.283 INFO:tasks.workunit.client.0.smithi117.stdout:3/20: mkdir d0/d9 0 2024-04-20T11:08:49.283 INFO:tasks.workunit.client.0.smithi117.stdout:4/24: creat d1/f5 x:0 0 0 2024-04-20T11:08:49.287 INFO:tasks.workunit.client.0.smithi117.stdout:7/20: mknod d1/d4/c5 0 2024-04-20T11:08:49.288 INFO:tasks.workunit.client.0.smithi117.stdout:2/13: dread d1/f3 [0,4194304] 0 2024-04-20T11:08:49.293 INFO:tasks.workunit.client.0.smithi117.stdout:3/21: truncate d0/f3 112356 0 2024-04-20T11:08:49.295 INFO:tasks.workunit.client.0.smithi117.stdout:4/25: read d1/f2 [118882,15139] 0 2024-04-20T11:08:49.301 INFO:tasks.workunit.client.0.smithi117.stdout:8/16: write d0/f3 [5208361,53564] 0 2024-04-20T11:08:49.301 INFO:tasks.workunit.client.0.smithi117.stdout:3/22: dread d0/f3 [0,4194304] 0 2024-04-20T11:08:49.304 INFO:tasks.workunit.client.0.smithi117.stdout:2/14: mkdir d1/d4 0 2024-04-20T11:08:49.305 INFO:tasks.workunit.client.0.smithi117.stdout:4/26: rename f0 to d1/f6 0 2024-04-20T11:08:49.307 INFO:tasks.workunit.client.0.smithi117.stdout:8/17: chown d0/l6 5 1 2024-04-20T11:08:49.308 INFO:tasks.workunit.client.0.smithi117.stdout:6/21: fsync d3/d4/f8 0 2024-04-20T11:08:49.310 INFO:tasks.workunit.client.0.smithi117.stdout:9/22: dwrite f2 [0,4194304] 0 2024-04-20T11:08:49.312 INFO:tasks.workunit.client.0.smithi117.stdout:5/7: truncate f0 14922 0 2024-04-20T11:08:49.312 INFO:tasks.workunit.client.0.smithi117.stdout:3/23: chown d0/c1 1031220062 1 2024-04-20T11:08:49.317 INFO:tasks.workunit.client.0.smithi117.stdout:5/8: chown f0 4945 1 2024-04-20T11:08:49.317 INFO:tasks.workunit.client.0.smithi117.stdout:5/9: readlink - no filename 2024-04-20T11:08:49.317 INFO:tasks.workunit.client.0.smithi117.stdout:7/21: truncate d1/f3 1411439 0 2024-04-20T11:08:49.324 INFO:tasks.workunit.client.0.smithi117.stdout:9/23: read d4/f8 [1063684,39489] 0 2024-04-20T11:08:49.327 INFO:tasks.workunit.client.0.smithi117.stdout:7/22: mkdir d1/d4/d6 0 2024-04-20T11:08:49.331 INFO:tasks.workunit.client.0.smithi117.stdout:5/10: dread f0 [0,4194304] 0 2024-04-20T11:08:49.337 INFO:tasks.workunit.client.0.smithi117.stdout:9/24: fsync f0 0 2024-04-20T11:08:49.337 INFO:tasks.workunit.client.0.smithi117.stdout:3/24: creat d0/d5/d7/fa x:0 0 0 2024-04-20T11:08:49.342 INFO:tasks.workunit.client.0.smithi117.stdout:4/27: getdents d1 0 2024-04-20T11:08:49.347 INFO:tasks.workunit.client.0.smithi117.stdout:8/18: dwrite d0/f3 [4194304,4194304] 0 2024-04-20T11:08:49.349 INFO:tasks.workunit.client.0.smithi117.stdout:3/25: write d0/d5/d7/fa [1016669,30849] 0 2024-04-20T11:08:49.351 INFO:tasks.workunit.client.0.smithi117.stdout:4/28: creat d1/f7 x:0 0 0 2024-04-20T11:08:49.354 INFO:tasks.workunit.client.0.smithi117.stdout:2/15: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.355 INFO:tasks.workunit.client.0.smithi117.stdout:9/25: creat d4/d5/da/fb x:0 0 0 2024-04-20T11:08:49.355 INFO:tasks.workunit.client.0.smithi117.stdout:6/22: dwrite d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:49.356 INFO:tasks.workunit.client.0.smithi117.stdout:4/29: mkdir d1/d8 0 2024-04-20T11:08:49.359 INFO:tasks.workunit.client.0.smithi117.stdout:3/26: write d0/d5/d7/fa [1937301,33461] 0 2024-04-20T11:08:49.365 INFO:tasks.workunit.client.0.smithi117.stdout:6/23: rename l0 to d3/l9 0 2024-04-20T11:08:49.366 INFO:tasks.workunit.client.0.smithi117.stdout:3/27: readlink d0/d5/d7/l8 0 2024-04-20T11:08:49.368 INFO:tasks.workunit.client.0.smithi117.stdout:6/24: rename d3 to d3/da 22 2024-04-20T11:08:49.370 INFO:tasks.workunit.client.0.smithi117.stdout:8/19: getdents d0 0 2024-04-20T11:08:49.372 INFO:tasks.workunit.client.0.smithi117.stdout:4/30: rename d1/d8 to d1/d9 0 2024-04-20T11:08:49.373 INFO:tasks.workunit.client.0.smithi117.stdout:2/16: link d1/f3 d1/d4/f5 0 2024-04-20T11:08:49.374 INFO:tasks.workunit.client.0.smithi117.stdout:3/28: dread d0/f3 [0,4194304] 0 2024-04-20T11:08:49.376 INFO:tasks.workunit.client.0.smithi117.stdout:8/20: mkdir d0/d9 0 2024-04-20T11:08:49.378 INFO:tasks.workunit.client.0.smithi117.stdout:6/25: rename d3/f6 to d3/d4/fb 0 2024-04-20T11:08:49.381 INFO:tasks.workunit.client.0.smithi117.stdout:7/23: dwrite d1/f3 [0,4194304] 0 2024-04-20T11:08:49.381 INFO:tasks.workunit.client.0.smithi117.stdout:8/21: write d0/f3 [7986978,61272] 0 2024-04-20T11:08:49.383 INFO:tasks.workunit.client.0.smithi117.stdout:9/26: read f0 [534586,100651] 0 2024-04-20T11:08:49.457 INFO:tasks.workunit.client.0.smithi117.stdout:2/17: rename f0 to d1/f6 0 2024-04-20T11:08:49.463 INFO:tasks.workunit.client.0.smithi117.stdout:8/22: creat d0/fa x:0 0 0 2024-04-20T11:08:49.463 INFO:tasks.workunit.client.0.smithi117.stdout:9/27: truncate f0 1690881 0 2024-04-20T11:08:49.464 INFO:tasks.workunit.client.0.smithi117.stdout:5/11: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.465 INFO:tasks.workunit.client.0.smithi117.stdout:7/24: link d1/f3 d1/f7 0 2024-04-20T11:08:49.467 INFO:tasks.workunit.client.0.smithi117.stdout:2/18: creat d1/d4/f7 x:0 0 0 2024-04-20T11:08:49.473 INFO:tasks.workunit.client.0.smithi117.stdout:5/12: write f0 [1685287,76357] 0 2024-04-20T11:08:49.474 INFO:tasks.workunit.client.0.smithi117.stdout:9/28: dread d4/d5/f7 [0,4194304] 0 2024-04-20T11:08:49.476 INFO:tasks.workunit.client.0.smithi117.stdout:6/26: dwrite d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:49.478 INFO:tasks.workunit.client.0.smithi117.stdout:2/19: write d1/f6 [5177064,1553] 0 2024-04-20T11:08:49.478 INFO:tasks.workunit.client.0.smithi117.stdout:9/29: chown d4/d5/da 0 1 2024-04-20T11:08:49.482 INFO:tasks.workunit.client.0.smithi117.stdout:6/27: creat d3/fc x:0 0 0 2024-04-20T11:08:49.483 INFO:tasks.workunit.client.0.smithi117.stdout:5/13: mknod c1 0 2024-04-20T11:08:49.486 INFO:tasks.workunit.client.0.smithi117.stdout:7/25: link d1/d4/c5 d1/d4/d6/c8 0 2024-04-20T11:08:49.490 INFO:tasks.workunit.client.0.smithi117.stdout:5/14: mkdir d2 0 2024-04-20T11:08:49.495 INFO:tasks.workunit.client.0.smithi117.stdout:7/26: symlink d1/d4/l9 0 2024-04-20T11:08:49.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.496+0000 7fd72f600700 1 -- 172.21.15.117:0/756854417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 msgr2=0x7fd728106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:49.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.496+0000 7fd72f600700 1 --2- 172.21.15.117:0/756854417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 0x7fd728106440 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd714009910 tx=0x7fd714009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:49.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.496+0000 7fd72f600700 1 -- 172.21.15.117:0/756854417 shutdown_connections 2024-04-20T11:08:49.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.496+0000 7fd72f600700 1 --2- 172.21.15.117:0/756854417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 0x7fd728106440 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.496+0000 7fd72f600700 1 --2- 172.21.15.117:0/756854417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd728101720 0x7fd728103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.497+0000 7fd72f600700 1 -- 172.21.15.117:0/756854417 >> 172.21.15.117:0/756854417 conn(0x7fd7280fb0d0 msgr2=0x7fd7280fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:49.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.497+0000 7fd72f600700 1 -- 172.21.15.117:0/756854417 shutdown_connections 2024-04-20T11:08:49.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.497+0000 7fd72f600700 1 -- 172.21.15.117:0/756854417 wait complete. 2024-04-20T11:08:49.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.499+0000 7fd72f600700 1 Processor -- start 2024-04-20T11:08:49.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.499+0000 7fd72f600700 1 -- start start 2024-04-20T11:08:49.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.500+0000 7fd72f600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd728101720 0x7fd728071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:49.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.500+0000 7fd72f600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 0x7fd728072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:49.501 INFO:tasks.workunit.client.0.smithi117.stdout:5/15: fsync f0 0 2024-04-20T11:08:49.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.500+0000 7fd72f600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728072a20 con 0x7fd728104050 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.500+0000 7fd72f600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7281aa8d0 con 0x7fd728101720 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.500+0000 7fd72cc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd728101720 0x7fd728071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.500+0000 7fd72cc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd728101720 0x7fd728071ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:38554/0 (socket says 172.21.15.117:38554) 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.500+0000 7fd72cc00700 1 -- 172.21.15.117:0/596869897 learned_addr learned my addr 172.21.15.117:0/596869897 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.500+0000 7fd727e00700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 0x7fd728072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.501+0000 7fd72cc00700 1 -- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 msgr2=0x7fd728072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.501+0000 7fd72cc00700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 0x7fd728072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.501+0000 7fd72cc00700 1 -- 172.21.15.117:0/596869897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7140095f0 con 0x7fd728101720 2024-04-20T11:08:49.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.501+0000 7fd727e00700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 0x7fd728072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:08:49.503 INFO:tasks.workunit.client.0.smithi117.stdout:9/30: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.502+0000 7fd72cc00700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd728101720 0x7fd728071ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009a00 tx=0x7fd71c009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:49.503 INFO:tasks.workunit.client.0.smithi117.stdout:5/16: write f0 [2655145,20958] 0 2024-04-20T11:08:49.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.502+0000 7fd725600700 1 -- 172.21.15.117:0/596869897 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c00c970 con 0x7fd728101720 2024-04-20T11:08:49.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.502+0000 7fd725600700 1 -- 172.21.15.117:0/596869897 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd71c017400 con 0x7fd728101720 2024-04-20T11:08:49.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.502+0000 7fd725600700 1 -- 172.21.15.117:0/596869897 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c0165d0 con 0x7fd728101720 2024-04-20T11:08:49.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.503+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7281aaad0 con 0x7fd728101720 2024-04-20T11:08:49.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.503+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7281ab020 con 0x7fd728101720 2024-04-20T11:08:49.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.505+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd728062510 con 0x7fd728101720 2024-04-20T11:08:49.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.515+0000 7fd725600700 1 -- 172.21.15.117:0/596869897 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 30) v1 ==== 46651+0+0 (secure 0 0 0) 0x7fd71c017570 con 0x7fd728101720 2024-04-20T11:08:49.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.516+0000 7fd725600700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd710039840 0x7fd71003bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:49.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.516+0000 7fd725600700 1 -- 172.21.15.117:0/596869897 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fd71c04e500 con 0x7fd728101720 2024-04-20T11:08:49.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.516+0000 7fd727e00700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd710039840 0x7fd71003bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:49.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.517+0000 7fd727e00700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd710039840 0x7fd71003bd00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd71400b6f0 tx=0x7fd714005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:49.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.524+0000 7fd725600700 1 -- 172.21.15.117:0/596869897 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fd71c00cad0 con 0x7fd728101720 2024-04-20T11:08:49.526 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:49 smithi173 ceph-mon[23800]: pgmap v4: 65 pgs: 65 active+clean; 193 MiB data, 807 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:08:49.526 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:49 smithi173 ceph-mon[23800]: from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:49.646 INFO:tasks.workunit.client.0.smithi117.stdout:7/27: rename d1/f3 to d1/d4/d6/fa 0 2024-04-20T11:08:49.650 INFO:tasks.workunit.client.0.smithi117.stdout:5/17: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.694 INFO:tasks.workunit.client.0.smithi117.stdout:8/23: rmdir d0 39 2024-04-20T11:08:49.706 INFO:tasks.workunit.client.0.smithi117.stdout:6/28: write d3/d4/fb [583438,74704] 0 2024-04-20T11:08:49.788 INFO:tasks.workunit.client.0.smithi117.stdout:9/31: link d4/f8 d4/d5/da/fc 0 2024-04-20T11:08:49.790 INFO:tasks.workunit.client.0.smithi117.stdout:7/28: creat d1/fb x:0 0 0 2024-04-20T11:08:49.791 INFO:tasks.workunit.client.0.smithi117.stdout:7/29: dread - d1/fb zero size 2024-04-20T11:08:49.792 INFO:tasks.workunit.client.0.smithi117.stdout:6/29: symlink d3/ld 0 2024-04-20T11:08:49.793 INFO:tasks.workunit.client.0.smithi117.stdout:8/24: readlink d0/l6 0 2024-04-20T11:08:49.793 INFO:tasks.workunit.client.0.smithi117.stdout:7/30: dread - d1/fb zero size 2024-04-20T11:08:49.796 INFO:tasks.workunit.client.0.smithi117.stdout:5/18: rmdir d2 0 2024-04-20T11:08:49.798 INFO:tasks.workunit.client.0.smithi117.stdout:6/30: creat d3/d4/fe x:0 0 0 2024-04-20T11:08:49.800 INFO:tasks.workunit.client.0.smithi117.stdout:6/31: write d3/fc [437667,99297] 0 2024-04-20T11:08:49.804 INFO:tasks.workunit.client.0.smithi117.stdout:6/32: truncate d3/d4/fe 686637 0 2024-04-20T11:08:49.818 INFO:tasks.workunit.client.0.smithi117.stdout:9/32: dwrite d4/d5/f7 [0,4194304] 0 2024-04-20T11:08:49.825 INFO:tasks.workunit.client.0.smithi117.stdout:5/19: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.826 INFO:tasks.workunit.client.0.smithi117.stdout:6/33: dread d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:49.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.830+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd728105f50 con 0x7fd710039840 2024-04-20T11:08:49.831 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:49 smithi117 ceph-mon[15659]: pgmap v4: 65 pgs: 65 active+clean; 193 MiB data, 807 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:08:49.831 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:49 smithi117 ceph-mon[15659]: from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:49.835 INFO:tasks.workunit.client.0.smithi117.stdout:6/34: dread d3/d4/fe [0,4194304] 0 2024-04-20T11:08:49.838 INFO:tasks.workunit.client.0.smithi117.stdout:9/33: dread d4/f6 [0,4194304] 0 2024-04-20T11:08:49.841 INFO:tasks.workunit.client.0.smithi117.stdout:6/35: write d3/d4/f8 [949237,6118] 0 2024-04-20T11:08:49.845 INFO:tasks.workunit.client.0.smithi117.stdout:9/34: creat d4/d5/fd x:0 0 0 2024-04-20T11:08:49.848 INFO:tasks.workunit.client.0.smithi117.stdout:6/36: mkdir d3/df 0 2024-04-20T11:08:49.849 INFO:tasks.workunit.client.0.smithi117.stdout:9/35: mkdir d4/de 0 2024-04-20T11:08:49.852 INFO:tasks.workunit.client.0.smithi117.stdout:5/20: dwrite f0 [0,4194304] 0 2024-04-20T11:08:49.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.851+0000 7fd725600700 1 -- 172.21.15.117:0/596869897 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd728105f50 con 0x7fd710039840 2024-04-20T11:08:49.852 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:08:49.852 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (9m) 13s ago 11m 13.7M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:08:49.852 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (11m) 13s ago 11m 7918k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (10m) 56s ago 10m 7680k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (11m) 13s ago 11m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (10m) 56s ago 10m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (9m) 13s ago 10m 75.8M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (4m) 13s ago 4m 13.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (4m) 13s ago 4m 146M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (3m) 56s ago 3m 13.5M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (4m) 56s ago 4m 21.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (26s) 13s ago 13m 64.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (83s) 56s ago 10m 493M - 19.0.0-3133-ga8c84f59 76bc6b91b726 7199977e9e8d 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (13m) 13s ago 13m 51.3M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (9m) 56s ago 9m 38.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (11m) 13s ago 11m 8102k - 1.5.0 0da6a335fe13 6b69ffbf445a 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (10m) 56s ago 10m 8024k - 1.5.0 0da6a335fe13 81909a8618f8 2024-04-20T11:08:49.853 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (8m) 13s ago 8m 83.0M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:08:49.854 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (8m) 13s ago 8m 80.7M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:08:49.854 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (7m) 13s ago 7m 72.1M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:08:49.854 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (7m) 56s ago 7m 77.2M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:08:49.854 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (6m) 56s ago 6m 72.7M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:08:49.854 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (5m) 56s ago 5m 73.0M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:08:49.854 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (37s) 13s ago 10m 40.1M - 2.43.0 a07b618ecd1d ca77d508764b 2024-04-20T11:08:49.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd710039840 msgr2=0x7fd71003bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:49.857 INFO:tasks.workunit.client.0.smithi117.stdout:9/36: mknod d4/d5/da/cf 0 2024-04-20T11:08:49.857 INFO:tasks.workunit.client.0.smithi117.stdout:5/21: creat f3 x:0 0 0 2024-04-20T11:08:49.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd710039840 0x7fd71003bd00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd71400b6f0 tx=0x7fd714005fb0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd728101720 msgr2=0x7fd728071ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:49.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd728101720 0x7fd728071ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009a00 tx=0x7fd71c009d10 comp rx=0 tx=0).stop 2024-04-20T11:08:49.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 shutdown_connections 2024-04-20T11:08:49.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd710039840 0x7fd71003bd00 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd71400b6f0 tx=0x7fd714005fb0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd728101720 0x7fd728071ec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 --2- 172.21.15.117:0/596869897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd728104050 0x7fd728072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:49.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.856+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 >> 172.21.15.117:0/596869897 conn(0x7fd7280fb0d0 msgr2=0x7fd7280fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:49.859 INFO:tasks.workunit.client.0.smithi117.stdout:5/22: symlink l4 0 2024-04-20T11:08:49.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.860+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 shutdown_connections 2024-04-20T11:08:49.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:49.860+0000 7fd72f600700 1 -- 172.21.15.117:0/596869897 wait complete. 2024-04-20T11:08:49.861 INFO:tasks.workunit.client.0.smithi117.stdout:5/23: write f3 [797779,62107] 0 2024-04-20T11:08:49.863 INFO:tasks.workunit.client.0.smithi117.stdout:5/24: chown f3 3765827 1 2024-04-20T11:08:49.872 INFO:tasks.workunit.client.0.smithi117.stdout:6/37: dread d3/fc [0,4194304] 0 2024-04-20T11:08:49.888 INFO:tasks.workunit.client.0.smithi117.stdout:6/38: dwrite d3/fc [0,4194304] 0 2024-04-20T11:08:49.897 INFO:tasks.workunit.client.0.smithi117.stdout:6/39: creat d3/df/f10 x:0 0 0 2024-04-20T11:08:49.900 INFO:tasks.workunit.client.0.smithi117.stdout:6/40: mknod d3/df/c11 0 2024-04-20T11:08:49.918 INFO:tasks.workunit.client.0.smithi117.stdout:7/31: write d1/d4/d6/fa [3502507,9857] 0 2024-04-20T11:08:49.923 INFO:tasks.workunit.client.0.smithi117.stdout:9/37: getdents d4 0 2024-04-20T11:08:49.925 INFO:tasks.workunit.client.0.smithi117.stdout:7/32: rmdir d1 39 2024-04-20T11:08:49.927 INFO:tasks.workunit.client.0.smithi117.stdout:9/38: write f2 [2301766,13791] 0 2024-04-20T11:08:49.929 INFO:tasks.workunit.client.0.smithi117.stdout:6/41: dwrite d3/df/f10 [0,4194304] 0 2024-04-20T11:08:49.934 INFO:tasks.workunit.client.0.smithi117.stdout:5/25: getdents . 0 2024-04-20T11:08:49.937 INFO:tasks.workunit.client.0.smithi117.stdout:6/42: unlink d3/d4/fb 0 2024-04-20T11:08:49.939 INFO:tasks.workunit.client.0.smithi117.stdout:9/39: mkdir d4/de/d10 0 2024-04-20T11:08:49.946 INFO:tasks.workunit.client.0.smithi117.stdout:9/40: truncate d4/d5/da/fc 5033491 0 2024-04-20T11:08:49.955 INFO:tasks.workunit.client.0.smithi117.stdout:7/33: dread d1/d4/d6/fa [0,4194304] 0 2024-04-20T11:08:49.957 INFO:tasks.workunit.client.0.smithi117.stdout:5/26: truncate f3 615902 0 2024-04-20T11:08:49.960 INFO:tasks.workunit.client.0.smithi117.stdout:7/34: write d1/f7 [1698805,31951] 0 2024-04-20T11:08:49.961 INFO:tasks.workunit.client.0.smithi117.stdout:5/27: creat f5 x:0 0 0 2024-04-20T11:08:49.962 INFO:tasks.workunit.client.0.smithi117.stdout:6/43: dwrite d3/df/f10 [0,4194304] 0 2024-04-20T11:08:49.969 INFO:tasks.workunit.client.0.smithi117.stdout:7/35: unlink d1/d4/l9 0 2024-04-20T11:08:49.971 INFO:tasks.workunit.client.0.smithi117.stdout:7/36: write d1/fb [381474,11288] 0 2024-04-20T11:08:49.972 INFO:tasks.workunit.client.0.smithi117.stdout:9/41: dread d4/f8 [0,4194304] 0 2024-04-20T11:08:49.975 INFO:tasks.workunit.client.0.smithi117.stdout:9/42: dread - d4/d5/da/fb zero size 2024-04-20T11:08:49.976 INFO:tasks.workunit.client.0.smithi117.stdout:7/37: creat d1/fc x:0 0 0 2024-04-20T11:08:49.977 INFO:tasks.workunit.client.0.smithi117.stdout:9/43: chown d4/d5/da/fb 6712485 1 2024-04-20T11:08:49.978 INFO:tasks.workunit.client.0.smithi117.stdout:6/44: getdents d3/df 0 2024-04-20T11:08:49.979 INFO:tasks.workunit.client.0.smithi117.stdout:7/38: write d1/d4/d6/fa [3355097,122083] 0 2024-04-20T11:08:49.989 INFO:tasks.workunit.client.0.smithi117.stdout:6/45: rename d3/df/f10 to d3/d4/f12 0 2024-04-20T11:08:49.992 INFO:tasks.workunit.client.0.smithi117.stdout:7/39: rename d1/d4/d6/fa to d1/d4/d6/fd 0 2024-04-20T11:08:49.993 INFO:tasks.workunit.client.0.smithi117.stdout:9/44: dread d4/d5/f7 [0,4194304] 0 2024-04-20T11:08:50.000 INFO:tasks.workunit.client.0.smithi117.stdout:5/28: write f3 [1205719,92553] 0 2024-04-20T11:08:50.002 INFO:tasks.workunit.client.0.smithi117.stdout:5/29: fsync f0 0 2024-04-20T11:08:50.003 INFO:tasks.workunit.client.0.smithi117.stdout:5/30: write f3 [1496093,109223] 0 2024-04-20T11:08:50.005 INFO:tasks.workunit.client.0.smithi117.stdout:7/40: rmdir d1/d4 39 2024-04-20T11:08:50.005 INFO:tasks.workunit.client.0.smithi117.stdout:5/31: write f0 [4085494,121345] 0 2024-04-20T11:08:50.009 INFO:tasks.workunit.client.0.smithi117.stdout:7/41: truncate d1/fc 544853 0 2024-04-20T11:08:50.009 INFO:tasks.workunit.client.0.smithi117.stdout:9/45: symlink d4/de/d10/l11 0 2024-04-20T11:08:50.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/42: chown d1/fb 1099 1 2024-04-20T11:08:50.011 INFO:tasks.workunit.client.0.smithi117.stdout:9/46: dread - d4/d5/da/fb zero size 2024-04-20T11:08:50.012 INFO:tasks.workunit.client.0.smithi117.stdout:9/47: chown d4/d5 4 1 2024-04-20T11:08:50.013 INFO:tasks.workunit.client.0.smithi117.stdout:6/46: write d3/d4/fe [419343,58859] 0 2024-04-20T11:08:50.016 INFO:tasks.workunit.client.0.smithi117.stdout:9/48: readlink d4/de/d10/l11 0 2024-04-20T11:08:50.018 INFO:tasks.workunit.client.0.smithi117.stdout:9/49: truncate d4/d5/f7 4649608 0 2024-04-20T11:08:50.018 INFO:tasks.workunit.client.0.smithi117.stdout:6/47: symlink d3/d4/l13 0 2024-04-20T11:08:50.019 INFO:tasks.workunit.client.0.smithi117.stdout:7/43: creat d1/fe x:0 0 0 2024-04-20T11:08:50.021 INFO:tasks.workunit.client.0.smithi117.stdout:5/32: dread f0 [0,4194304] 0 2024-04-20T11:08:50.021 INFO:tasks.workunit.client.0.smithi117.stdout:6/48: fdatasync d3/d4/f8 0 2024-04-20T11:08:50.022 INFO:tasks.workunit.client.0.smithi117.stdout:9/50: symlink d4/d5/da/l12 0 2024-04-20T11:08:50.025 INFO:tasks.workunit.client.0.smithi117.stdout:5/33: symlink l6 0 2024-04-20T11:08:50.026 INFO:tasks.workunit.client.0.smithi117.stdout:9/51: creat d4/f13 x:0 0 0 2024-04-20T11:08:50.027 INFO:tasks.workunit.client.0.smithi117.stdout:5/34: read f0 [1277415,52044] 0 2024-04-20T11:08:50.028 INFO:tasks.workunit.client.0.smithi117.stdout:5/35: chown l4 5984 1 2024-04-20T11:08:50.028 INFO:tasks.workunit.client.0.smithi117.stdout:9/52: write f2 [4124973,125670] 0 2024-04-20T11:08:50.033 INFO:tasks.workunit.client.0.smithi117.stdout:5/36: mknod c7 0 2024-04-20T11:08:50.034 INFO:tasks.workunit.client.0.smithi117.stdout:9/53: write d4/d5/da/fb [956889,57591] 0 2024-04-20T11:08:50.043 INFO:tasks.workunit.client.0.smithi117.stdout:6/49: dwrite d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:50.050 INFO:tasks.workunit.client.0.smithi117.stdout:7/44: dwrite d1/d4/d6/fd [0,4194304] 0 2024-04-20T11:08:50.050 INFO:tasks.workunit.client.0.smithi117.stdout:6/50: truncate d3/fc 4525768 0 2024-04-20T11:08:50.054 INFO:tasks.workunit.client.0.smithi117.stdout:5/37: dread f0 [0,4194304] 0 2024-04-20T11:08:50.057 INFO:tasks.workunit.client.0.smithi117.stdout:5/38: chown f3 6 1 2024-04-20T11:08:50.063 INFO:tasks.workunit.client.0.smithi117.stdout:6/51: link d3/d4/fe d3/df/f14 0 2024-04-20T11:08:50.064 INFO:tasks.workunit.client.0.smithi117.stdout:9/54: dwrite d4/f6 [0,4194304] 0 2024-04-20T11:08:50.064 INFO:tasks.workunit.client.0.smithi117.stdout:5/39: mknod c8 0 2024-04-20T11:08:50.067 INFO:tasks.workunit.client.0.smithi117.stdout:5/40: stat c8 0 2024-04-20T11:08:50.071 INFO:tasks.workunit.client.0.smithi117.stdout:6/52: mknod d3/d4/c15 0 2024-04-20T11:08:50.072 INFO:tasks.workunit.client.0.smithi117.stdout:9/55: mknod d4/d5/da/c14 0 2024-04-20T11:08:50.076 INFO:tasks.workunit.client.0.smithi117.stdout:9/56: chown d4/d5/f7 393 1 2024-04-20T11:08:50.078 INFO:tasks.workunit.client.0.smithi117.stdout:6/53: read d3/d4/fe [5680,72546] 0 2024-04-20T11:08:50.086 INFO:tasks.workunit.client.0.smithi117.stdout:9/57: symlink d4/d5/da/l15 0 2024-04-20T11:08:50.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.087+0000 7f286ce00700 1 -- 172.21.15.117:0/4136999053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 msgr2=0x7f2868073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:50.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.087+0000 7f286ce00700 1 --2- 172.21.15.117:0/4136999053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 0x7f2868073370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2858009910 tx=0x7f2858009c20 comp rx=0 tx=0).stop 2024-04-20T11:08:50.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.088+0000 7f286ce00700 1 -- 172.21.15.117:0/4136999053 shutdown_connections 2024-04-20T11:08:50.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.088+0000 7f286ce00700 1 --2- 172.21.15.117:0/4136999053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868073940 0x7f2868073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.088+0000 7f286ce00700 1 --2- 172.21.15.117:0/4136999053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 0x7f2868073370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.088+0000 7f286ce00700 1 -- 172.21.15.117:0/4136999053 >> 172.21.15.117:0/4136999053 conn(0x7f28680fc1c0 msgr2=0x7f28680fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:50.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.088+0000 7f286ce00700 1 -- 172.21.15.117:0/4136999053 shutdown_connections 2024-04-20T11:08:50.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.088+0000 7f286ce00700 1 -- 172.21.15.117:0/4136999053 wait complete. 2024-04-20T11:08:50.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.089+0000 7f286ce00700 1 Processor -- start 2024-04-20T11:08:50.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.090+0000 7f286ce00700 1 -- start start 2024-04-20T11:08:50.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.091+0000 7f286ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868073940 0x7f2868071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:50.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.091+0000 7f286ce00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 0x7f2868072500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:50.091 INFO:tasks.workunit.client.0.smithi117.stdout:6/54: dread d3/df/f14 [0,4194304] 0 2024-04-20T11:08:50.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.091+0000 7f286ce00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2868072b20 con 0x7f2868073940 2024-04-20T11:08:50.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.091+0000 7f286ce00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2868072c60 con 0x7f2868074f10 2024-04-20T11:08:50.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.091+0000 7f2865400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 0x7f2868072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:50.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.091+0000 7f2865e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868073940 0x7f2868071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:50.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.091+0000 7f2865400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 0x7f2868072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:38574/0 (socket says 172.21.15.117:38574) 2024-04-20T11:08:50.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.091+0000 7f2865400700 1 -- 172.21.15.117:0/2208756751 learned_addr learned my addr 172.21.15.117:0/2208756751 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:50.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.092+0000 7f2865400700 1 -- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868073940 msgr2=0x7f2868071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:50.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.092+0000 7f2865400700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868073940 0x7f2868071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.092+0000 7f2865400700 1 -- 172.21.15.117:0/2208756751 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28580095f0 con 0x7f2868074f10 2024-04-20T11:08:50.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.093+0000 7f2865400700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 0x7f2868072500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f285400b480 tx=0x7f285400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:50.094 INFO:tasks.workunit.client.0.smithi117.stdout:6/55: symlink d3/d4/l16 0 2024-04-20T11:08:50.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.094+0000 7f285ea00700 1 -- 172.21.15.117:0/2208756751 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2854011800 con 0x7f2868074f10 2024-04-20T11:08:50.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.094+0000 7f285ea00700 1 -- 172.21.15.117:0/2208756751 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2854011e20 con 0x7f2868074f10 2024-04-20T11:08:50.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.094+0000 7f285ea00700 1 -- 172.21.15.117:0/2208756751 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2854010ea0 con 0x7f2868074f10 2024-04-20T11:08:50.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.094+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28681aac00 con 0x7f2868074f10 2024-04-20T11:08:50.096 INFO:tasks.workunit.client.0.smithi117.stdout:7/45: truncate d1/f7 447806 0 2024-04-20T11:08:50.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.095+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28681ab180 con 0x7f2868074f10 2024-04-20T11:08:50.099 INFO:tasks.workunit.client.0.smithi117.stdout:6/56: mknod d3/c17 0 2024-04-20T11:08:50.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.099+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286806c0a0 con 0x7f2868074f10 2024-04-20T11:08:50.099 INFO:tasks.workunit.client.0.smithi117.stdout:5/41: getdents . 0 2024-04-20T11:08:50.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.099+0000 7f285ea00700 1 -- 172.21.15.117:0/2208756751 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 31) v1 ==== 46707+0+0 (secure 0 0 0) 0x7f285401a480 con 0x7f2868074f10 2024-04-20T11:08:50.101 INFO:tasks.workunit.client.0.smithi117.stdout:6/57: readlink d3/d4/l13 0 2024-04-20T11:08:50.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.101+0000 7f285ea00700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f284c039910 0x7f284c03bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:50.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.101+0000 7f285ea00700 1 -- 172.21.15.117:0/2208756751 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f285404ca20 con 0x7f2868074f10 2024-04-20T11:08:50.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.101+0000 7f2865e00700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f284c039910 0x7f284c03bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:50.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.103+0000 7f2865e00700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f284c039910 0x7f284c03bdd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2858005090 tx=0x7f2858005180 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:50.103 INFO:tasks.workunit.client.0.smithi117.stdout:9/58: dread d4/d5/da/fb [0,4194304] 0 2024-04-20T11:08:50.103 INFO:tasks.workunit.client.0.smithi117.stdout:5/42: truncate f3 2343397 0 2024-04-20T11:08:50.105 INFO:tasks.workunit.client.0.smithi117.stdout:5/43: fsync f5 0 2024-04-20T11:08:50.105 INFO:tasks.workunit.client.0.smithi117.stdout:6/58: truncate d3/fc 5232354 0 2024-04-20T11:08:50.107 INFO:tasks.workunit.client.0.smithi117.stdout:6/59: read d3/fc [2037098,24884] 0 2024-04-20T11:08:50.108 INFO:tasks.workunit.client.0.smithi117.stdout:6/60: chown d3/d4/l16 0 1 2024-04-20T11:08:50.108 INFO:tasks.workunit.client.0.smithi117.stdout:9/59: symlink d4/de/l16 0 2024-04-20T11:08:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.110+0000 7f285ea00700 1 -- 172.21.15.117:0/2208756751 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f285400e850 con 0x7f2868074f10 2024-04-20T11:08:50.111 INFO:tasks.workunit.client.0.smithi117.stdout:5/44: rename l6 to l9 0 2024-04-20T11:08:50.116 INFO:tasks.workunit.client.0.smithi117.stdout:6/61: chown d3/l9 27760 1 2024-04-20T11:08:50.119 INFO:tasks.workunit.client.0.smithi117.stdout:5/45: symlink la 0 2024-04-20T11:08:50.121 INFO:tasks.workunit.client.0.smithi117.stdout:9/60: link d4/d5/fd d4/de/d10/f17 0 2024-04-20T11:08:50.123 INFO:tasks.workunit.client.0.smithi117.stdout:9/61: dread - d4/f13 zero size 2024-04-20T11:08:50.124 INFO:tasks.workunit.client.0.smithi117.stdout:5/46: creat fb x:0 0 0 2024-04-20T11:08:50.126 INFO:tasks.workunit.client.0.smithi117.stdout:9/62: fsync d4/f6 0 2024-04-20T11:08:50.128 INFO:tasks.workunit.client.0.smithi117.stdout:5/47: symlink lc 0 2024-04-20T11:08:50.129 INFO:tasks.workunit.client.0.smithi117.stdout:7/46: dwrite d1/d4/d6/fd [0,4194304] 0 2024-04-20T11:08:50.132 INFO:tasks.workunit.client.0.smithi117.stdout:9/63: mknod d4/de/c18 0 2024-04-20T11:08:50.132 INFO:tasks.workunit.client.0.smithi117.stdout:6/62: getdents d3 0 2024-04-20T11:08:50.133 INFO:tasks.workunit.client.0.smithi117.stdout:6/63: readlink l2 0 2024-04-20T11:08:50.134 INFO:tasks.workunit.client.0.smithi117.stdout:5/48: mknod cd 0 2024-04-20T11:08:50.135 INFO:tasks.workunit.client.0.smithi117.stdout:5/49: fdatasync f3 0 2024-04-20T11:08:50.137 INFO:tasks.workunit.client.0.smithi117.stdout:5/50: dread - f5 zero size 2024-04-20T11:08:50.139 INFO:tasks.workunit.client.0.smithi117.stdout:6/64: dread d3/d4/fe [0,4194304] 0 2024-04-20T11:08:50.145 INFO:tasks.workunit.client.0.smithi117.stdout:7/47: unlink d1/d4/d6/c8 0 2024-04-20T11:08:50.146 INFO:tasks.workunit.client.0.smithi117.stdout:9/64: creat d4/d5/da/f19 x:0 0 0 2024-04-20T11:08:50.148 INFO:tasks.workunit.client.0.smithi117.stdout:9/65: write f2 [4063655,121355] 0 2024-04-20T11:08:50.149 INFO:tasks.workunit.client.0.smithi117.stdout:7/48: write d1/f7 [2033508,59075] 0 2024-04-20T11:08:50.149 INFO:tasks.workunit.client.0.smithi117.stdout:6/65: write d3/d4/f12 [1159086,22297] 0 2024-04-20T11:08:50.159 INFO:tasks.workunit.client.0.smithi117.stdout:7/49: fsync d1/d4/d6/fd 0 2024-04-20T11:08:50.160 INFO:tasks.workunit.client.0.smithi117.stdout:6/66: dread d3/d4/fe [0,4194304] 0 2024-04-20T11:08:50.160 INFO:tasks.workunit.client.0.smithi117.stdout:9/66: truncate f0 2505886 0 2024-04-20T11:08:50.165 INFO:tasks.workunit.client.0.smithi117.stdout:5/51: dwrite fb [0,4194304] 0 2024-04-20T11:08:50.165 INFO:tasks.workunit.client.0.smithi117.stdout:7/50: symlink d1/d4/d6/lf 0 2024-04-20T11:08:50.167 INFO:tasks.workunit.client.0.smithi117.stdout:5/52: write f3 [543557,59623] 0 2024-04-20T11:08:50.170 INFO:tasks.workunit.client.0.smithi117.stdout:9/67: creat d4/de/d10/f1a x:0 0 0 2024-04-20T11:08:50.172 INFO:tasks.workunit.client.0.smithi117.stdout:5/53: mkdir de 0 2024-04-20T11:08:50.178 INFO:tasks.workunit.client.0.smithi117.stdout:9/68: creat d4/de/f1b x:0 0 0 2024-04-20T11:08:50.181 INFO:tasks.workunit.client.0.smithi117.stdout:7/51: creat d1/f10 x:0 0 0 2024-04-20T11:08:50.181 INFO:tasks.workunit.client.0.smithi117.stdout:5/54: mknod de/cf 0 2024-04-20T11:08:50.186 INFO:tasks.workunit.client.0.smithi117.stdout:9/69: mknod d4/d5/c1c 0 2024-04-20T11:08:50.186 INFO:tasks.workunit.client.0.smithi117.stdout:5/55: unlink la 0 2024-04-20T11:08:50.186 INFO:tasks.workunit.client.0.smithi117.stdout:7/52: dread d1/fc [0,4194304] 0 2024-04-20T11:08:50.188 INFO:tasks.workunit.client.0.smithi117.stdout:5/56: chown f0 110163420 1 2024-04-20T11:08:50.190 INFO:tasks.workunit.client.0.smithi117.stdout:7/53: truncate d1/d4/d6/fd 4908022 0 2024-04-20T11:08:50.192 INFO:tasks.workunit.client.0.smithi117.stdout:9/70: symlink d4/de/d10/l1d 0 2024-04-20T11:08:50.194 INFO:tasks.workunit.client.0.smithi117.stdout:6/67: truncate d3/d4/f12 705530 0 2024-04-20T11:08:50.195 INFO:tasks.workunit.client.0.smithi117.stdout:5/57: mknod de/c10 0 2024-04-20T11:08:50.197 INFO:tasks.workunit.client.0.smithi117.stdout:5/58: stat de/c10 0 2024-04-20T11:08:50.198 INFO:tasks.workunit.client.0.smithi117.stdout:5/59: readlink lc 0 2024-04-20T11:08:50.198 INFO:tasks.workunit.client.0.smithi117.stdout:6/68: write d3/d4/f8 [3376868,89983] 0 2024-04-20T11:08:50.201 INFO:tasks.workunit.client.0.smithi117.stdout:6/69: rmdir d3/df 39 2024-04-20T11:08:50.202 INFO:tasks.workunit.client.0.smithi117.stdout:5/60: write f0 [1595636,12127] 0 2024-04-20T11:08:50.205 INFO:tasks.workunit.client.0.smithi117.stdout:5/61: write f5 [163648,101579] 0 2024-04-20T11:08:50.208 INFO:tasks.workunit.client.0.smithi117.stdout:6/70: creat d3/f18 x:0 0 0 2024-04-20T11:08:50.208 INFO:tasks.workunit.client.0.smithi117.stdout:7/54: dread d1/d4/d6/fd [0,4194304] 0 2024-04-20T11:08:50.211 INFO:tasks.workunit.client.0.smithi117.stdout:5/62: creat de/f11 x:0 0 0 2024-04-20T11:08:50.213 INFO:tasks.workunit.client.0.smithi117.stdout:6/71: fdatasync d3/d4/fe 0 2024-04-20T11:08:50.214 INFO:tasks.workunit.client.0.smithi117.stdout:7/55: symlink d1/d4/l11 0 2024-04-20T11:08:50.218 INFO:tasks.workunit.client.0.smithi117.stdout:6/72: creat d3/d4/f19 x:0 0 0 2024-04-20T11:08:50.226 INFO:tasks.workunit.client.0.smithi117.stdout:7/56: rename d1/fe to d1/f12 0 2024-04-20T11:08:50.226 INFO:tasks.workunit.client.0.smithi117.stdout:9/71: truncate d4/d5/da/fb 623062 0 2024-04-20T11:08:50.228 INFO:tasks.workunit.client.0.smithi117.stdout:9/72: truncate d4/d5/fd 706002 0 2024-04-20T11:08:50.228 INFO:tasks.workunit.client.0.smithi117.stdout:5/63: fsync de/f11 0 2024-04-20T11:08:50.231 INFO:tasks.workunit.client.0.smithi117.stdout:7/57: mkdir d1/d4/d6/d13 0 2024-04-20T11:08:50.234 INFO:tasks.workunit.client.0.smithi117.stdout:5/64: symlink de/l12 0 2024-04-20T11:08:50.235 INFO:tasks.workunit.client.0.smithi117.stdout:5/65: chown l4 0 1 2024-04-20T11:08:50.235 INFO:tasks.workunit.client.0.smithi117.stdout:6/73: getdents d3 0 2024-04-20T11:08:50.237 INFO:tasks.workunit.client.0.smithi117.stdout:9/73: fdatasync d4/d5/da/fc 0 2024-04-20T11:08:50.240 INFO:tasks.workunit.client.0.smithi117.stdout:7/58: mknod d1/d4/c14 0 2024-04-20T11:08:50.254 INFO:tasks.workunit.client.0.smithi117.stdout:5/66: getdents de 0 2024-04-20T11:08:50.257 INFO:tasks.workunit.client.0.smithi117.stdout:5/67: symlink de/l13 0 2024-04-20T11:08:50.258 INFO:tasks.workunit.client.0.smithi117.stdout:5/68: dread - de/f11 zero size 2024-04-20T11:08:50.260 INFO:tasks.workunit.client.0.smithi117.stdout:6/74: dwrite d3/fc [0,4194304] 0 2024-04-20T11:08:50.365 INFO:tasks.workunit.client.0.smithi117.stdout:9/74: dwrite d4/d5/da/fc [0,4194304] 0 2024-04-20T11:08:50.373 INFO:tasks.workunit.client.0.smithi117.stdout:6/75: write d3/d4/f12 [288860,56] 0 2024-04-20T11:08:50.378 INFO:tasks.workunit.client.0.smithi117.stdout:6/76: link d3/d4/l16 d3/df/l1a 0 2024-04-20T11:08:50.380 INFO:tasks.workunit.client.0.smithi117.stdout:6/77: fdatasync d3/d4/f19 0 2024-04-20T11:08:50.381 INFO:tasks.workunit.client.0.smithi117.stdout:6/78: write d3/f18 [656642,56424] 0 2024-04-20T11:08:50.382 INFO:tasks.workunit.client.0.smithi117.stdout:6/79: chown d3/f18 184251 1 2024-04-20T11:08:50.388 INFO:tasks.workunit.client.0.smithi117.stdout:6/80: link d3/d4/fe d3/d4/f1b 0 2024-04-20T11:08:50.391 INFO:tasks.workunit.client.0.smithi117.stdout:9/75: dwrite d4/d5/da/f19 [0,4194304] 0 2024-04-20T11:08:50.411 INFO:tasks.workunit.client.0.smithi117.stdout:9/76: link f2 d4/d5/f1e 0 2024-04-20T11:08:50.412 INFO:tasks.workunit.client.0.smithi117.stdout:9/77: chown d4/de 150 1 2024-04-20T11:08:50.413 INFO:tasks.workunit.client.0.smithi117.stdout:9/78: read d4/f6 [2104112,101436] 0 2024-04-20T11:08:50.418 INFO:tasks.workunit.client.0.smithi117.stdout:9/79: write f2 [4870553,24578] 0 2024-04-20T11:08:50.423 INFO:tasks.workunit.client.0.smithi117.stdout:9/80: write d4/de/f1b [647260,30065] 0 2024-04-20T11:08:50.430 INFO:tasks.workunit.client.0.smithi117.stdout:6/81: dwrite d3/d4/fe [0,4194304] 0 2024-04-20T11:08:50.439 INFO:tasks.workunit.client.0.smithi117.stdout:6/82: symlink d3/df/l1c 0 2024-04-20T11:08:50.446 INFO:tasks.workunit.client.0.smithi117.stdout:6/83: rename l2 to d3/l1d 0 2024-04-20T11:08:50.447 INFO:tasks.workunit.client.0.smithi117.stdout:9/81: dwrite d4/f8 [0,4194304] 0 2024-04-20T11:08:50.454 INFO:tasks.workunit.client.0.smithi117.stdout:9/82: unlink d4/d5/fd 0 2024-04-20T11:08:50.469 INFO:tasks.workunit.client.0.smithi117.stdout:6/84: symlink d3/l1e 0 2024-04-20T11:08:50.474 INFO:tasks.workunit.client.0.smithi117.stdout:6/85: dread d3/d4/f12 [0,4194304] 0 2024-04-20T11:08:50.477 INFO:tasks.workunit.client.0.smithi117.stdout:9/83: getdents d4/d5 0 2024-04-20T11:08:50.481 INFO:tasks.workunit.client.0.smithi117.stdout:9/84: symlink d4/de/l1f 0 2024-04-20T11:08:50.483 INFO:tasks.workunit.client.0.smithi117.stdout:6/86: symlink d3/df/l1f 0 2024-04-20T11:08:50.489 INFO:tasks.workunit.client.0.smithi117.stdout:9/85: symlink d4/l20 0 2024-04-20T11:08:50.494 INFO:tasks.workunit.client.0.smithi117.stdout:6/87: rename d3/df/l1c to d3/d4/l20 0 2024-04-20T11:08:50.494 INFO:tasks.workunit.client.0.smithi117.stdout:6/88: write d3/df/f14 [3209225,57654] 0 2024-04-20T11:08:50.494 INFO:tasks.workunit.client.0.smithi117.stdout:9/86: creat d4/d5/f21 x:0 0 0 2024-04-20T11:08:50.495 INFO:tasks.workunit.client.0.smithi117.stdout:6/89: write d3/d4/f8 [979358,77755] 0 2024-04-20T11:08:50.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.541+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2868062590 con 0x7f2868074f10 2024-04-20T11:08:50.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.542+0000 7f285ea00700 1 -- 172.21.15.117:0/2208756751 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f2854015030 con 0x7f2868074f10 2024-04-20T11:08:50.543 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:08:50.543 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:08:50.543 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:08:50.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:08:50.543 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:08:50.543 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:08:50.544 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f284c039910 msgr2=0x7f284c03bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f284c039910 0x7f284c03bdd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2858005090 tx=0x7f2858005180 comp rx=0 tx=0).stop 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 msgr2=0x7f2868072500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 0x7f2868072500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f285400b480 tx=0x7f285400b790 comp rx=0 tx=0).stop 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 shutdown_connections 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f284c039910 0x7f284c03bdd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868073940 0x7f2868071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 --2- 172.21.15.117:0/2208756751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2868074f10 0x7f2868072500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 >> 172.21.15.117:0/2208756751 conn(0x7f28680fc1c0 msgr2=0x7f2868102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 shutdown_connections 2024-04-20T11:08:50.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.545+0000 7f286ce00700 1 -- 172.21.15.117:0/2208756751 wait complete. 2024-04-20T11:08:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:50 smithi173 ceph-mon[23800]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:50 smithi173 ceph-mon[23800]: mgrmap e31: smithi117.ivnklj(active, since 4s) 2024-04-20T11:08:50.683 INFO:tasks.workunit.client.0.smithi117.stdout:6/90: truncate d3/d4/f8 1088409 0 2024-04-20T11:08:50.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.702+0000 7fdc66200700 1 -- 172.21.15.117:0/3425924747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 msgr2=0x7fdc601020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:50.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.702+0000 7fdc66200700 1 --2- 172.21.15.117:0/3425924747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 0x7fdc601020a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800b0a0 tx=0x7fdc4800b3b0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.702+0000 7fdc66200700 1 -- 172.21.15.117:0/3425924747 shutdown_connections 2024-04-20T11:08:50.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.702+0000 7fdc66200700 1 --2- 172.21.15.117:0/3425924747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 0x7fdc601020a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.702+0000 7fdc66200700 1 --2- 172.21.15.117:0/3425924747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc60100a40 0x7fdc60100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.702+0000 7fdc66200700 1 -- 172.21.15.117:0/3425924747 >> 172.21.15.117:0/3425924747 conn(0x7fdc600fc000 msgr2=0x7fdc600fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:50.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.702+0000 7fdc66200700 1 -- 172.21.15.117:0/3425924747 shutdown_connections 2024-04-20T11:08:50.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.702+0000 7fdc66200700 1 -- 172.21.15.117:0/3425924747 wait complete. 2024-04-20T11:08:50.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.704+0000 7fdc66200700 1 Processor -- start 2024-04-20T11:08:50.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.704+0000 7fdc66200700 1 -- start start 2024-04-20T11:08:50.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc66200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc60100a40 0x7fdc60196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:50.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc66200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 0x7fdc60196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:50.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc66200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60196f90 con 0x7fdc60101c40 2024-04-20T11:08:50.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc66200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601970d0 con 0x7fdc60100a40 2024-04-20T11:08:50.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc5fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 0x7fdc60196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:50.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc5fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 0x7fdc60196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57642/0 (socket says 172.21.15.117:57642) 2024-04-20T11:08:50.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc5fe00700 1 -- 172.21.15.117:0/2254141037 learned_addr learned my addr 172.21.15.117:0/2254141037 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:08:50.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc64e00700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc60100a40 0x7fdc60196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc5fe00700 1 -- 172.21.15.117:0/2254141037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc60100a40 msgr2=0x7fdc60196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc5fe00700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc60100a40 0x7fdc60196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.705+0000 7fdc5fe00700 1 -- 172.21.15.117:0/2254141037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc48009d00 con 0x7fdc60101c40 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.706+0000 7fdc5fe00700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 0x7fdc60196970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdc48015040 tx=0x7fdc4800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.706+0000 7fdc5d600700 1 -- 172.21.15.117:0/2254141037 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4800e040 con 0x7fdc60101c40 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.706+0000 7fdc5d600700 1 -- 172.21.15.117:0/2254141037 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc48003de0 con 0x7fdc60101c40 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.706+0000 7fdc5d600700 1 -- 172.21.15.117:0/2254141037 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4801d730 con 0x7fdc60101c40 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.706+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6019bad0 con 0x7fdc60101c40 2024-04-20T11:08:50.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.706+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6019c040 con 0x7fdc60101c40 2024-04-20T11:08:50.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.707+0000 7fdc5d600700 1 -- 172.21.15.117:0/2254141037 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 31) v1 ==== 46707+0+0 (secure 0 0 0) 0x7fdc48019070 con 0x7fdc60101c40 2024-04-20T11:08:50.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.708+0000 7fdc5d600700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc440398c0 0x7fdc4403bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:08:50.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.708+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc601905c0 con 0x7fdc60101c40 2024-04-20T11:08:50.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.708+0000 7fdc5d600700 1 -- 172.21.15.117:0/2254141037 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fdc4804f350 con 0x7fdc60101c40 2024-04-20T11:08:50.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.708+0000 7fdc64e00700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc440398c0 0x7fdc4403bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:08:50.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.709+0000 7fdc64e00700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc440398c0 0x7fdc4403bd80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdc60101aa0 tx=0x7fdc58009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:08:50.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:50.714+0000 7fdc5d600700 1 -- 172.21.15.117:0/2254141037 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fdc48017070 con 0x7fdc60101c40 2024-04-20T11:08:50.747 INFO:tasks.workunit.client.0.smithi117.stdout:6/91: creat d3/df/f21 x:0 0 0 2024-04-20T11:08:50.748 INFO:tasks.workunit.client.0.smithi117.stdout:9/87: unlink d4/de/d10/f17 0 2024-04-20T11:08:50.750 INFO:tasks.workunit.client.0.smithi117.stdout:6/92: write d3/fc [3774500,7954] 0 2024-04-20T11:08:50.750 INFO:tasks.workunit.client.0.smithi117.stdout:9/88: write f0 [4042860,54373] 0 2024-04-20T11:08:50.753 INFO:tasks.workunit.client.0.smithi117.stdout:6/93: mknod d3/d4/c22 0 2024-04-20T11:08:50.819 INFO:tasks.workunit.client.0.smithi117.stdout:6/94: truncate d3/d4/f1b 2006514 0 2024-04-20T11:08:50.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:50 smithi117 ceph-mon[15659]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:50.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:50 smithi117 ceph-mon[15659]: mgrmap e31: smithi117.ivnklj(active, since 4s) 2024-04-20T11:08:51.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.004+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc6002cdb0 con 0x7fdc440398c0 2024-04-20T11:08:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.008+0000 7fdc5d600700 1 -- 172.21.15.117:0/2254141037 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fdc6002cdb0 con 0x7fdc440398c0 2024-04-20T11:08:51.009 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:08:51.009 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:08:51.009 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:08:51.009 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:08:51.010 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:08:51.010 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:08:51.010 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:08:51.010 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "2/2 daemons upgraded", 2024-04-20T11:08:51.010 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "", 2024-04-20T11:08:51.010 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:08:51.010 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:08:51.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc440398c0 msgr2=0x7fdc4403bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc440398c0 0x7fdc4403bd80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdc60101aa0 tx=0x7fdc58009260 comp rx=0 tx=0).stop 2024-04-20T11:08:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 msgr2=0x7fdc60196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:08:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 0x7fdc60196970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdc48015040 tx=0x7fdc4800bd90 comp rx=0 tx=0).stop 2024-04-20T11:08:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 shutdown_connections 2024-04-20T11:08:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc440398c0 0x7fdc4403bd80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc60100a40 0x7fdc60196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 --2- 172.21.15.117:0/2254141037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc60101c40 0x7fdc60196970 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:08:51.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.011+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 >> 172.21.15.117:0/2254141037 conn(0x7fdc600fc000 msgr2=0x7fdc60105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:08:51.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.012+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 shutdown_connections 2024-04-20T11:08:51.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:08:51.012+0000 7fdc66200700 1 -- 172.21.15.117:0/2254141037 wait complete. 2024-04-20T11:08:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:51 smithi117 ceph-mon[15659]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:51 smithi117 ceph-mon[15659]: pgmap v5: 65 pgs: 65 active+clean; 193 MiB data, 807 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:08:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:51 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2208756751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:08:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:51 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:08:51.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:51 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:08:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:51 smithi173 ceph-mon[23800]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:51 smithi173 ceph-mon[23800]: pgmap v5: 65 pgs: 65 active+clean; 193 MiB data, 807 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:08:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:51 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2208756751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:08:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:51 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:08:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:51 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:08:52.721 INFO:tasks.workunit.client.0.smithi117.stdout:0/6: sync 2024-04-20T11:08:52.721 INFO:tasks.workunit.client.0.smithi117.stdout:1/7: sync 2024-04-20T11:08:52.722 INFO:tasks.workunit.client.0.smithi117.stdout:1/8: fsync f0 0 2024-04-20T11:08:52.725 INFO:tasks.workunit.client.0.smithi117.stdout:1/9: write f2 [857312,44596] 0 2024-04-20T11:08:52.728 INFO:tasks.workunit.client.0.smithi117.stdout:1/10: chown f2 44512580 1 2024-04-20T11:08:52.761 INFO:tasks.workunit.client.0.smithi117.stdout:1/11: dwrite f0 [0,4194304] 0 2024-04-20T11:08:52.815 INFO:tasks.workunit.client.0.smithi117.stdout:1/12: fdatasync f2 0 2024-04-20T11:08:52.838 INFO:tasks.workunit.client.0.smithi117.stdout:4/31: sync 2024-04-20T11:08:52.838 INFO:tasks.workunit.client.0.smithi117.stdout:3/29: sync 2024-04-20T11:08:52.839 INFO:tasks.workunit.client.0.smithi117.stdout:4/32: chown d1/f5 1 1 2024-04-20T11:08:52.858 INFO:tasks.workunit.client.0.smithi117.stdout:4/33: dwrite d1/f5 [0,4194304] 0 2024-04-20T11:08:52.881 INFO:tasks.workunit.client.0.smithi117.stdout:0/7: symlink l1 0 2024-04-20T11:08:52.886 INFO:tasks.workunit.client.0.smithi117.stdout:1/13: unlink f0 0 2024-04-20T11:08:52.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:52 smithi117 ceph-mon[15659]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:52.891 INFO:tasks.workunit.client.0.smithi117.stdout:4/34: creat d1/fa x:0 0 0 2024-04-20T11:08:52.893 INFO:tasks.workunit.client.0.smithi117.stdout:4/35: chown d1/c4 5789805 1 2024-04-20T11:08:52.895 INFO:tasks.workunit.client.0.smithi117.stdout:1/14: creat f3 x:0 0 0 2024-04-20T11:08:52.898 INFO:tasks.workunit.client.0.smithi117.stdout:3/30: creat d0/fb x:0 0 0 2024-04-20T11:08:52.900 INFO:tasks.workunit.client.0.smithi117.stdout:3/31: fdatasync d0/d5/d7/fa 0 2024-04-20T11:08:52.900 INFO:tasks.workunit.client.0.smithi117.stdout:1/15: rename f3 to f4 0 2024-04-20T11:08:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:52 smithi173 ceph-mon[23800]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:08:52.922 INFO:tasks.workunit.client.0.smithi117.stdout:3/32: dwrite d0/fb [0,4194304] 0 2024-04-20T11:08:52.941 INFO:tasks.workunit.client.0.smithi117.stdout:3/33: dread d0/d5/d7/fa [0,4194304] 0 2024-04-20T11:08:52.944 INFO:tasks.workunit.client.0.smithi117.stdout:3/34: creat d0/d5/fc x:0 0 0 2024-04-20T11:08:52.951 INFO:tasks.workunit.client.0.smithi117.stdout:3/35: rename d0/d9 to d0/d5/d7/dd 0 2024-04-20T11:08:52.952 INFO:tasks.workunit.client.0.smithi117.stdout:3/36: dread d0/f3 [0,4194304] 0 2024-04-20T11:08:52.969 INFO:tasks.workunit.client.0.smithi117.stdout:3/37: dwrite d0/fb [0,4194304] 0 2024-04-20T11:08:52.972 INFO:tasks.workunit.client.0.smithi117.stdout:3/38: mkdir d0/d5/de 0 2024-04-20T11:08:52.986 INFO:tasks.workunit.client.0.smithi117.stdout:3/39: dread d0/fb [0,4194304] 0 2024-04-20T11:08:52.989 INFO:tasks.workunit.client.0.smithi117.stdout:3/40: write d0/f3 [95064,40953] 0 2024-04-20T11:08:52.994 INFO:tasks.workunit.client.0.smithi117.stdout:3/41: rename d0/d5/d7/fa to d0/d5/d7/dd/ff 0 2024-04-20T11:08:52.997 INFO:tasks.workunit.client.0.smithi117.stdout:3/42: write d0/d5/d7/dd/ff [1028011,52633] 0 2024-04-20T11:08:53.002 INFO:tasks.workunit.client.0.smithi117.stdout:3/43: creat d0/d5/d7/dd/f10 x:0 0 0 2024-04-20T11:08:53.005 INFO:tasks.workunit.client.0.smithi117.stdout:3/44: write d0/fb [3783104,104008] 0 2024-04-20T11:08:53.007 INFO:tasks.workunit.client.0.smithi117.stdout:3/45: mkdir d0/d5/d11 0 2024-04-20T11:08:53.009 INFO:tasks.workunit.client.0.smithi117.stdout:3/46: read - d0/d5/d7/dd/f10 zero size 2024-04-20T11:08:53.011 INFO:tasks.workunit.client.0.smithi117.stdout:3/47: truncate d0/f3 859565 0 2024-04-20T11:08:53.013 INFO:tasks.workunit.client.0.smithi117.stdout:7/59: sync 2024-04-20T11:08:53.013 INFO:tasks.workunit.client.0.smithi117.stdout:8/25: sync 2024-04-20T11:08:53.013 INFO:tasks.workunit.client.0.smithi117.stdout:2/20: sync 2024-04-20T11:08:53.015 INFO:tasks.workunit.client.0.smithi117.stdout:8/26: chown d0/l2 167240142 1 2024-04-20T11:08:53.017 INFO:tasks.workunit.client.0.smithi117.stdout:8/27: chown d0/f1 404 1 2024-04-20T11:08:53.018 INFO:tasks.workunit.client.0.smithi117.stdout:8/28: dread - d0/fa zero size 2024-04-20T11:08:53.027 INFO:tasks.workunit.client.0.smithi117.stdout:8/29: link d0/f5 d0/fb 0 2024-04-20T11:08:53.034 INFO:tasks.workunit.client.0.smithi117.stdout:8/30: mkdir d0/dc 0 2024-04-20T11:08:53.040 INFO:tasks.workunit.client.0.smithi117.stdout:8/31: creat d0/fd x:0 0 0 2024-04-20T11:08:53.041 INFO:tasks.workunit.client.0.smithi117.stdout:3/48: dwrite d0/d5/d7/dd/ff [0,4194304] 0 2024-04-20T11:08:53.042 INFO:tasks.workunit.client.0.smithi117.stdout:7/60: dread d1/d4/d6/fd [0,4194304] 0 2024-04-20T11:08:53.042 INFO:tasks.workunit.client.0.smithi117.stdout:2/21: dwrite d1/f2 [0,4194304] 0 2024-04-20T11:08:53.047 INFO:tasks.workunit.client.0.smithi117.stdout:2/22: read d1/f6 [1649354,32633] 0 2024-04-20T11:08:53.048 INFO:tasks.workunit.client.0.smithi117.stdout:2/23: chown d1/f3 1857 1 2024-04-20T11:08:53.048 INFO:tasks.workunit.client.0.smithi117.stdout:2/24: readlink - no filename 2024-04-20T11:08:53.052 INFO:tasks.workunit.client.0.smithi117.stdout:8/32: creat d0/d9/fe x:0 0 0 2024-04-20T11:08:53.055 INFO:tasks.workunit.client.0.smithi117.stdout:2/25: creat d1/d4/f8 x:0 0 0 2024-04-20T11:08:53.058 INFO:tasks.workunit.client.0.smithi117.stdout:3/49: symlink d0/d5/d11/l12 0 2024-04-20T11:08:53.063 INFO:tasks.workunit.client.0.smithi117.stdout:2/26: creat d1/d4/f9 x:0 0 0 2024-04-20T11:08:53.063 INFO:tasks.workunit.client.0.smithi117.stdout:8/33: symlink d0/d8/lf 0 2024-04-20T11:08:53.066 INFO:tasks.workunit.client.0.smithi117.stdout:9/89: sync 2024-04-20T11:08:53.066 INFO:tasks.workunit.client.0.smithi117.stdout:5/69: sync 2024-04-20T11:08:53.066 INFO:tasks.workunit.client.0.smithi117.stdout:0/8: sync 2024-04-20T11:08:53.066 INFO:tasks.workunit.client.0.smithi117.stdout:6/95: sync 2024-04-20T11:08:53.067 INFO:tasks.workunit.client.0.smithi117.stdout:9/90: chown d4 12003014 1 2024-04-20T11:08:53.068 INFO:tasks.workunit.client.0.smithi117.stdout:0/9: readlink l0 0 2024-04-20T11:08:53.071 INFO:tasks.workunit.client.0.smithi117.stdout:2/27: mkdir d1/d4/da 0 2024-04-20T11:08:53.072 INFO:tasks.workunit.client.0.smithi117.stdout:8/34: mknod d0/c10 0 2024-04-20T11:08:53.073 INFO:tasks.workunit.client.0.smithi117.stdout:5/70: creat de/f14 x:0 0 0 2024-04-20T11:08:53.075 INFO:tasks.workunit.client.0.smithi117.stdout:7/61: dwrite d1/f12 [0,4194304] 0 2024-04-20T11:08:53.075 INFO:tasks.workunit.client.0.smithi117.stdout:6/96: mknod d3/d4/c23 0 2024-04-20T11:08:53.076 INFO:tasks.workunit.client.0.smithi117.stdout:8/35: dread - d0/f5 zero size 2024-04-20T11:08:53.076 INFO:tasks.workunit.client.0.smithi117.stdout:5/71: readlink de/l13 0 2024-04-20T11:08:53.078 INFO:tasks.workunit.client.0.smithi117.stdout:5/72: readlink de/l12 0 2024-04-20T11:08:53.080 INFO:tasks.workunit.client.0.smithi117.stdout:8/36: write d0/f7 [992726,19095] 0 2024-04-20T11:08:53.083 INFO:tasks.workunit.client.0.smithi117.stdout:8/37: chown d0/d8 0 1 2024-04-20T11:08:53.084 INFO:tasks.workunit.client.0.smithi117.stdout:2/28: rename d1/d4/f9 to d1/d4/fb 0 2024-04-20T11:08:53.086 INFO:tasks.workunit.client.0.smithi117.stdout:2/29: stat d1/f6 0 2024-04-20T11:08:53.088 INFO:tasks.workunit.client.0.smithi117.stdout:8/38: write d0/d9/fe [540529,74928] 0 2024-04-20T11:08:53.091 INFO:tasks.workunit.client.0.smithi117.stdout:8/39: dread - d0/fb zero size 2024-04-20T11:08:53.095 INFO:tasks.workunit.client.0.smithi117.stdout:0/10: link l0 l2 0 2024-04-20T11:08:53.095 INFO:tasks.workunit.client.0.smithi117.stdout:0/11: dwrite - no filename 2024-04-20T11:08:53.095 INFO:tasks.workunit.client.0.smithi117.stdout:0/12: dread - no filename 2024-04-20T11:08:53.095 INFO:tasks.workunit.client.0.smithi117.stdout:0/13: write - no filename 2024-04-20T11:08:53.097 INFO:tasks.workunit.client.0.smithi117.stdout:2/30: write d1/f2 [1604046,54949] 0 2024-04-20T11:08:53.098 INFO:tasks.workunit.client.0.smithi117.stdout:8/40: mkdir d0/d8/d11 0 2024-04-20T11:08:53.102 INFO:tasks.workunit.client.0.smithi117.stdout:0/14: mkdir d3 0 2024-04-20T11:08:53.105 INFO:tasks.workunit.client.0.smithi117.stdout:7/62: dread d1/f12 [0,4194304] 0 2024-04-20T11:08:53.106 INFO:tasks.workunit.client.0.smithi117.stdout:8/41: creat d0/d9/f12 x:0 0 0 2024-04-20T11:08:53.109 INFO:tasks.workunit.client.0.smithi117.stdout:2/31: mkdir d1/d4/da/dc 0 2024-04-20T11:08:53.109 INFO:tasks.workunit.client.0.smithi117.stdout:9/91: dwrite d4/d5/f7 [0,4194304] 0 2024-04-20T11:08:53.111 INFO:tasks.workunit.client.0.smithi117.stdout:0/15: mknod d3/c4 0 2024-04-20T11:08:53.111 INFO:tasks.workunit.client.0.smithi117.stdout:0/16: write - no filename 2024-04-20T11:08:53.111 INFO:tasks.workunit.client.0.smithi117.stdout:0/17: dwrite - no filename 2024-04-20T11:08:53.112 INFO:tasks.workunit.client.0.smithi117.stdout:5/73: dwrite fb [0,4194304] 0 2024-04-20T11:08:53.113 INFO:tasks.workunit.client.0.smithi117.stdout:9/92: chown d4/d5/da/fc 15 1 2024-04-20T11:08:53.113 INFO:tasks.workunit.client.0.smithi117.stdout:2/32: chown d1/d4/f7 226824817 1 2024-04-20T11:08:53.115 INFO:tasks.workunit.client.0.smithi117.stdout:5/74: dread - de/f11 zero size 2024-04-20T11:08:53.117 INFO:tasks.workunit.client.0.smithi117.stdout:8/42: creat d0/d8/f13 x:0 0 0 2024-04-20T11:08:53.120 INFO:tasks.workunit.client.0.smithi117.stdout:0/18: unlink d3/c4 0 2024-04-20T11:08:53.122 INFO:tasks.workunit.client.0.smithi117.stdout:9/93: rename d4/d5/da/l12 to d4/de/l22 0 2024-04-20T11:08:53.126 INFO:tasks.workunit.client.0.smithi117.stdout:5/75: mknod de/c15 0 2024-04-20T11:08:53.128 INFO:tasks.workunit.client.0.smithi117.stdout:8/43: write d0/fb [878742,85003] 0 2024-04-20T11:08:53.132 INFO:tasks.workunit.client.0.smithi117.stdout:2/33: rename d1/d4/da/dc to d1/d4/dd 0 2024-04-20T11:08:53.138 INFO:tasks.workunit.client.0.smithi117.stdout:5/76: mkdir de/d16 0 2024-04-20T11:08:53.139 INFO:tasks.workunit.client.0.smithi117.stdout:0/19: chown l2 48078 1 2024-04-20T11:08:53.146 INFO:tasks.workunit.client.0.smithi117.stdout:9/94: dread d4/d5/da/fc [0,4194304] 0 2024-04-20T11:08:53.153 INFO:tasks.workunit.client.0.smithi117.stdout:7/63: dwrite d1/f10 [0,4194304] 0 2024-04-20T11:08:53.154 INFO:tasks.workunit.client.0.smithi117.stdout:0/20: chown l2 2 1 2024-04-20T11:08:53.154 INFO:tasks.workunit.client.0.smithi117.stdout:0/21: dread - no filename 2024-04-20T11:08:53.155 INFO:tasks.workunit.client.0.smithi117.stdout:9/95: rename d4 to d4/d5/da/d23 22 2024-04-20T11:08:53.158 INFO:tasks.workunit.client.0.smithi117.stdout:9/96: read d4/d5/f1e [1547066,36504] 0 2024-04-20T11:08:53.165 INFO:tasks.workunit.client.0.smithi117.stdout:5/77: dread f3 [0,4194304] 0 2024-04-20T11:08:53.165 INFO:tasks.workunit.client.0.smithi117.stdout:0/22: mkdir d3/d5 0 2024-04-20T11:08:53.165 INFO:tasks.workunit.client.0.smithi117.stdout:0/23: dwrite - no filename 2024-04-20T11:08:53.165 INFO:tasks.workunit.client.0.smithi117.stdout:0/24: write - no filename 2024-04-20T11:08:53.168 INFO:tasks.workunit.client.0.smithi117.stdout:5/78: chown de/c15 1037443 1 2024-04-20T11:08:53.173 INFO:tasks.workunit.client.0.smithi117.stdout:0/25: creat d3/f6 x:0 0 0 2024-04-20T11:08:53.174 INFO:tasks.workunit.client.0.smithi117.stdout:9/97: rename d4/f6 to d4/de/d10/f24 0 2024-04-20T11:08:53.181 INFO:tasks.workunit.client.0.smithi117.stdout:5/79: rename de/l12 to de/l17 0 2024-04-20T11:08:53.181 INFO:tasks.workunit.client.0.smithi117.stdout:0/26: mkdir d3/d7 0 2024-04-20T11:08:53.184 INFO:tasks.workunit.client.0.smithi117.stdout:0/27: write d3/f6 [667345,25493] 0 2024-04-20T11:08:53.185 INFO:tasks.workunit.client.0.smithi117.stdout:7/64: dwrite d1/f10 [0,4194304] 0 2024-04-20T11:08:53.185 INFO:tasks.workunit.client.0.smithi117.stdout:5/80: getdents de/d16 0 2024-04-20T11:08:53.187 INFO:tasks.workunit.client.0.smithi117.stdout:9/98: fdatasync d4/d5/da/fb 0 2024-04-20T11:08:53.195 INFO:tasks.workunit.client.0.smithi117.stdout:0/28: mkdir d3/d7/d8 0 2024-04-20T11:08:53.197 INFO:tasks.workunit.client.0.smithi117.stdout:9/99: creat d4/f25 x:0 0 0 2024-04-20T11:08:53.198 INFO:tasks.workunit.client.0.smithi117.stdout:0/29: read d3/f6 [227041,22577] 0 2024-04-20T11:08:53.200 INFO:tasks.workunit.client.0.smithi117.stdout:7/65: creat d1/d4/d6/d13/f15 x:0 0 0 2024-04-20T11:08:53.202 INFO:tasks.workunit.client.0.smithi117.stdout:9/100: mknod d4/d5/da/c26 0 2024-04-20T11:08:53.209 INFO:tasks.workunit.client.0.smithi117.stdout:5/81: dwrite f3 [0,4194304] 0 2024-04-20T11:08:53.213 INFO:tasks.workunit.client.0.smithi117.stdout:5/82: chown de 40 1 2024-04-20T11:08:53.213 INFO:tasks.workunit.client.0.smithi117.stdout:0/30: dread d3/f6 [0,4194304] 0 2024-04-20T11:08:53.215 INFO:tasks.workunit.client.0.smithi117.stdout:5/83: creat de/f18 x:0 0 0 2024-04-20T11:08:53.217 INFO:tasks.workunit.client.0.smithi117.stdout:0/31: rename d3/f6 to d3/d7/f9 0 2024-04-20T11:08:53.218 INFO:tasks.workunit.client.0.smithi117.stdout:5/84: creat de/f19 x:0 0 0 2024-04-20T11:08:53.221 INFO:tasks.workunit.client.0.smithi117.stdout:5/85: truncate fb 4613818 0 2024-04-20T11:08:53.224 INFO:tasks.workunit.client.0.smithi117.stdout:0/32: mknod d3/ca 0 2024-04-20T11:08:53.224 INFO:tasks.workunit.client.0.smithi117.stdout:5/86: write fb [774208,39353] 0 2024-04-20T11:08:53.229 INFO:tasks.workunit.client.0.smithi117.stdout:0/33: mkdir d3/d7/d8/db 0 2024-04-20T11:08:53.230 INFO:tasks.workunit.client.0.smithi117.stdout:0/34: chown l1 2724 1 2024-04-20T11:08:53.232 INFO:tasks.workunit.client.0.smithi117.stdout:9/101: dwrite f0 [0,4194304] 0 2024-04-20T11:08:53.238 INFO:tasks.workunit.client.0.smithi117.stdout:9/102: symlink d4/d5/da/l27 0 2024-04-20T11:08:53.243 INFO:tasks.workunit.client.0.smithi117.stdout:9/103: creat d4/d5/f28 x:0 0 0 2024-04-20T11:08:53.249 INFO:tasks.workunit.client.0.smithi117.stdout:9/104: creat d4/d5/da/f29 x:0 0 0 2024-04-20T11:08:53.249 INFO:tasks.workunit.client.0.smithi117.stdout:5/87: dwrite fb [0,4194304] 0 2024-04-20T11:08:53.253 INFO:tasks.workunit.client.0.smithi117.stdout:5/88: chown de/l13 8392 1 2024-04-20T11:08:53.258 INFO:tasks.workunit.client.0.smithi117.stdout:9/105: rename d4/f8 to d4/d5/f2a 0 2024-04-20T11:08:53.262 INFO:tasks.workunit.client.0.smithi117.stdout:0/35: dwrite d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:53.264 INFO:tasks.workunit.client.0.smithi117.stdout:9/106: creat d4/d5/f2b x:0 0 0 2024-04-20T11:08:53.267 INFO:tasks.workunit.client.0.smithi117.stdout:4/36: truncate d1/f2 44473 0 2024-04-20T11:08:53.269 INFO:tasks.workunit.client.0.smithi117.stdout:5/89: rename l9 to de/l1a 0 2024-04-20T11:08:53.270 INFO:tasks.workunit.client.0.smithi117.stdout:5/90: chown de 59597981 1 2024-04-20T11:08:53.273 INFO:tasks.workunit.client.0.smithi117.stdout:9/107: creat d4/de/f2c x:0 0 0 2024-04-20T11:08:53.275 INFO:tasks.workunit.client.0.smithi117.stdout:0/36: mknod d3/d5/cc 0 2024-04-20T11:08:53.278 INFO:tasks.workunit.client.0.smithi117.stdout:1/16: dread f2 [0,4194304] 0 2024-04-20T11:08:53.283 INFO:tasks.workunit.client.0.smithi117.stdout:9/108: creat d4/de/f2d x:0 0 0 2024-04-20T11:08:53.285 INFO:tasks.workunit.client.0.smithi117.stdout:9/109: write d4/de/f2c [109550,50641] 0 2024-04-20T11:08:53.291 INFO:tasks.workunit.client.0.smithi117.stdout:0/37: dread d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:53.292 INFO:tasks.workunit.client.0.smithi117.stdout:5/91: creat de/d16/f1b x:0 0 0 2024-04-20T11:08:53.294 INFO:tasks.workunit.client.0.smithi117.stdout:5/92: dread - de/f18 zero size 2024-04-20T11:08:53.294 INFO:tasks.workunit.client.0.smithi117.stdout:6/97: sync 2024-04-20T11:08:53.295 INFO:tasks.workunit.client.0.smithi117.stdout:3/50: fsync d0/d5/d7/dd/f10 0 2024-04-20T11:08:53.296 INFO:tasks.workunit.client.0.smithi117.stdout:1/17: write f4 [178972,126347] 0 2024-04-20T11:08:53.297 INFO:tasks.workunit.client.0.smithi117.stdout:9/110: link d4/de/f1b d4/f2e 0 2024-04-20T11:08:53.299 INFO:tasks.workunit.client.0.smithi117.stdout:9/111: chown d4/d5/f7 1 1 2024-04-20T11:08:53.300 INFO:tasks.workunit.client.0.smithi117.stdout:0/38: creat d3/d5/fd x:0 0 0 2024-04-20T11:08:53.302 INFO:tasks.workunit.client.0.smithi117.stdout:5/93: mknod de/d16/c1c 0 2024-04-20T11:08:53.304 INFO:tasks.workunit.client.0.smithi117.stdout:3/51: creat d0/d5/f13 x:0 0 0 2024-04-20T11:08:53.306 INFO:tasks.workunit.client.0.smithi117.stdout:9/112: symlink d4/d5/da/l2f 0 2024-04-20T11:08:53.307 INFO:tasks.workunit.client.0.smithi117.stdout:3/52: write d0/d5/d7/dd/ff [3568247,87276] 0 2024-04-20T11:08:53.315 INFO:tasks.workunit.client.0.smithi117.stdout:9/113: mkdir d4/d30 0 2024-04-20T11:08:53.317 INFO:tasks.workunit.client.0.smithi117.stdout:5/94: dread f0 [0,4194304] 0 2024-04-20T11:08:53.320 INFO:tasks.workunit.client.0.smithi117.stdout:6/98: rename d3/c17 to d3/c24 0 2024-04-20T11:08:53.321 INFO:tasks.workunit.client.0.smithi117.stdout:0/39: link l1 d3/d7/le 0 2024-04-20T11:08:53.323 INFO:tasks.workunit.client.0.smithi117.stdout:6/99: fdatasync d3/d4/f12 0 2024-04-20T11:08:53.326 INFO:tasks.workunit.client.0.smithi117.stdout:0/40: mkdir d3/d7/d8/df 0 2024-04-20T11:08:53.328 INFO:tasks.workunit.client.0.smithi117.stdout:6/100: symlink d3/d4/l25 0 2024-04-20T11:08:53.329 INFO:tasks.workunit.client.0.smithi117.stdout:5/95: dread f3 [0,4194304] 0 2024-04-20T11:08:53.329 INFO:tasks.workunit.client.0.smithi117.stdout:6/101: read d3/d4/f12 [115172,54032] 0 2024-04-20T11:08:53.331 INFO:tasks.workunit.client.0.smithi117.stdout:5/96: dread - de/f19 zero size 2024-04-20T11:08:53.332 INFO:tasks.workunit.client.0.smithi117.stdout:5/97: stat l4 0 2024-04-20T11:08:53.333 INFO:tasks.workunit.client.0.smithi117.stdout:6/102: symlink d3/df/l26 0 2024-04-20T11:08:53.338 INFO:tasks.workunit.client.0.smithi117.stdout:6/103: link d3/d4/l13 d3/df/l27 0 2024-04-20T11:08:53.339 INFO:tasks.workunit.client.0.smithi117.stdout:6/104: stat d3/ld 0 2024-04-20T11:08:53.347 INFO:tasks.workunit.client.0.smithi117.stdout:2/34: getdents d1/d4 0 2024-04-20T11:08:53.350 INFO:tasks.workunit.client.0.smithi117.stdout:2/35: truncate d1/f2 5731232 0 2024-04-20T11:08:53.351 INFO:tasks.workunit.client.0.smithi117.stdout:2/36: chown d1/d4/dd 0 1 2024-04-20T11:08:53.353 INFO:tasks.workunit.client.0.smithi117.stdout:5/98: dwrite f3 [0,4194304] 0 2024-04-20T11:08:53.381 INFO:tasks.workunit.client.0.smithi117.stdout:8/44: truncate d0/f5 940602 0 2024-04-20T11:08:53.382 INFO:tasks.workunit.client.0.smithi117.stdout:8/45: read - d0/fa zero size 2024-04-20T11:08:53.383 INFO:tasks.workunit.client.0.smithi117.stdout:2/37: rename d1/d4/da to d1/d4/dd/de 0 2024-04-20T11:08:53.385 INFO:tasks.workunit.client.0.smithi117.stdout:8/46: truncate d0/fd 337744 0 2024-04-20T11:08:53.386 INFO:tasks.workunit.client.0.smithi117.stdout:8/47: write d0/f3 [5331358,129941] 0 2024-04-20T11:08:53.387 INFO:tasks.workunit.client.0.smithi117.stdout:5/99: rename f3 to de/f1d 0 2024-04-20T11:08:53.389 INFO:tasks.workunit.client.0.smithi117.stdout:2/38: creat d1/d4/ff x:0 0 0 2024-04-20T11:08:53.390 INFO:tasks.workunit.client.0.smithi117.stdout:2/39: rename d1 to d1/d4/dd/d10 22 2024-04-20T11:08:53.393 INFO:tasks.workunit.client.0.smithi117.stdout:8/48: unlink d0/d8/f13 0 2024-04-20T11:08:53.394 INFO:tasks.workunit.client.0.smithi117.stdout:8/49: chown d0/fd 2323972 1 2024-04-20T11:08:53.401 INFO:tasks.workunit.client.0.smithi117.stdout:8/50: chown d0/fb 6860384 1 2024-04-20T11:08:53.401 INFO:tasks.workunit.client.0.smithi117.stdout:7/66: write d1/fc [616832,114110] 0 2024-04-20T11:08:53.411 INFO:tasks.workunit.client.0.smithi117.stdout:2/40: dwrite d1/f6 [4194304,4194304] 0 2024-04-20T11:08:53.414 INFO:tasks.workunit.client.0.smithi117.stdout:2/41: chown d1/f6 261286 1 2024-04-20T11:08:53.416 INFO:tasks.workunit.client.0.smithi117.stdout:8/51: dread d0/f7 [0,4194304] 0 2024-04-20T11:08:53.419 INFO:tasks.workunit.client.0.smithi117.stdout:1/18: mknod c5 0 2024-04-20T11:08:53.422 INFO:tasks.workunit.client.0.smithi117.stdout:1/19: fsync f4 0 2024-04-20T11:08:53.423 INFO:tasks.workunit.client.0.smithi117.stdout:2/42: dread - d1/d4/f8 zero size 2024-04-20T11:08:53.427 INFO:tasks.workunit.client.0.smithi117.stdout:1/20: mkdir d6 0 2024-04-20T11:08:53.427 INFO:tasks.workunit.client.0.smithi117.stdout:2/43: chown d1/d4/fb 44 1 2024-04-20T11:08:53.430 INFO:tasks.workunit.client.0.smithi117.stdout:1/21: chown f4 3648758 1 2024-04-20T11:08:53.432 INFO:tasks.workunit.client.0.smithi117.stdout:7/67: dwrite d1/f12 [4194304,4194304] 0 2024-04-20T11:08:53.434 INFO:tasks.workunit.client.0.smithi117.stdout:1/22: chown f4 745763910 1 2024-04-20T11:08:53.434 INFO:tasks.workunit.client.0.smithi117.stdout:0/41: rmdir d3 39 2024-04-20T11:08:53.441 INFO:tasks.workunit.client.0.smithi117.stdout:6/105: write d3/d4/f8 [979216,41113] 0 2024-04-20T11:08:53.447 INFO:tasks.workunit.client.0.smithi117.stdout:2/44: link d1/f3 d1/f11 0 2024-04-20T11:08:53.454 INFO:tasks.workunit.client.0.smithi117.stdout:0/42: rmdir d3 39 2024-04-20T11:08:53.462 INFO:tasks.workunit.client.0.smithi117.stdout:1/23: creat d6/f7 x:0 0 0 2024-04-20T11:08:53.463 INFO:tasks.workunit.client.0.smithi117.stdout:1/24: write f4 [358946,106732] 0 2024-04-20T11:08:53.463 INFO:tasks.workunit.client.0.smithi117.stdout:1/25: readlink - no filename 2024-04-20T11:08:53.465 INFO:tasks.workunit.client.0.smithi117.stdout:1/26: chown c5 15127 1 2024-04-20T11:08:53.466 INFO:tasks.workunit.client.0.smithi117.stdout:7/68: creat d1/f16 x:0 0 0 2024-04-20T11:08:53.468 INFO:tasks.workunit.client.0.smithi117.stdout:0/43: stat d3/d5/cc 0 2024-04-20T11:08:53.469 INFO:tasks.workunit.client.0.smithi117.stdout:2/45: dread d1/f6 [0,4194304] 0 2024-04-20T11:08:53.476 INFO:tasks.workunit.client.0.smithi117.stdout:7/69: mkdir d1/d4/d6/d17 0 2024-04-20T11:08:53.480 INFO:tasks.workunit.client.0.smithi117.stdout:0/44: dread d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:53.480 INFO:tasks.workunit.client.0.smithi117.stdout:6/106: creat d3/f28 x:0 0 0 2024-04-20T11:08:53.486 INFO:tasks.workunit.client.0.smithi117.stdout:5/100: unlink de/f1d 0 2024-04-20T11:08:53.489 INFO:tasks.workunit.client.0.smithi117.stdout:2/46: rmdir d1/d4 39 2024-04-20T11:08:53.496 INFO:tasks.workunit.client.0.smithi117.stdout:1/27: link c1 d6/c8 0 2024-04-20T11:08:53.497 INFO:tasks.workunit.client.0.smithi117.stdout:6/107: dwrite d3/d4/f12 [0,4194304] 0 2024-04-20T11:08:53.499 INFO:tasks.workunit.client.0.smithi117.stdout:6/108: chown d3/ld 171851 1 2024-04-20T11:08:53.500 INFO:tasks.workunit.client.0.smithi117.stdout:2/47: chown d1/d4/f5 0 1 2024-04-20T11:08:53.502 INFO:tasks.workunit.client.0.smithi117.stdout:7/70: creat d1/f18 x:0 0 0 2024-04-20T11:08:53.506 INFO:tasks.workunit.client.0.smithi117.stdout:8/52: truncate d0/f7 160315 0 2024-04-20T11:08:53.516 INFO:tasks.workunit.client.0.smithi117.stdout:4/37: dwrite d1/f2 [0,4194304] 0 2024-04-20T11:08:53.517 INFO:tasks.workunit.client.0.smithi117.stdout:4/38: stat d1/f7 0 2024-04-20T11:08:53.519 INFO:tasks.workunit.client.0.smithi117.stdout:7/71: symlink d1/d4/d6/d13/l19 0 2024-04-20T11:08:53.532 INFO:tasks.workunit.client.0.smithi117.stdout:4/39: unlink d1/f5 0 2024-04-20T11:08:53.540 INFO:tasks.workunit.client.0.smithi117.stdout:2/48: creat d1/f12 x:0 0 0 2024-04-20T11:08:53.544 INFO:tasks.workunit.client.0.smithi117.stdout:4/40: symlink d1/lb 0 2024-04-20T11:08:53.546 INFO:tasks.workunit.client.0.smithi117.stdout:2/49: rmdir d1/d4/dd 39 2024-04-20T11:08:53.547 INFO:tasks.workunit.client.0.smithi117.stdout:2/50: stat d1/f11 0 2024-04-20T11:08:53.549 INFO:tasks.workunit.client.0.smithi117.stdout:6/109: link d3/df/l1a d3/l29 0 2024-04-20T11:08:53.554 INFO:tasks.workunit.client.0.smithi117.stdout:6/110: mknod d3/d4/c2a 0 2024-04-20T11:08:53.556 INFO:tasks.workunit.client.0.smithi117.stdout:6/111: write d3/d4/f19 [590209,53323] 0 2024-04-20T11:08:53.557 INFO:tasks.workunit.client.0.smithi117.stdout:2/51: symlink d1/d4/dd/l13 0 2024-04-20T11:08:53.557 INFO:tasks.workunit.client.0.smithi117.stdout:5/101: rmdir de 39 2024-04-20T11:08:53.557 INFO:tasks.workunit.client.0.smithi117.stdout:6/112: chown d3/df 1783349676 1 2024-04-20T11:08:53.558 INFO:tasks.workunit.client.0.smithi117.stdout:2/52: dread - d1/f12 zero size 2024-04-20T11:08:53.562 INFO:tasks.workunit.client.0.smithi117.stdout:6/113: truncate d3/f28 403331 0 2024-04-20T11:08:53.563 INFO:tasks.workunit.client.0.smithi117.stdout:4/41: link d1/c4 d1/d9/cc 0 2024-04-20T11:08:53.563 INFO:tasks.workunit.client.0.smithi117.stdout:4/42: truncate d1/f7 959796 0 2024-04-20T11:08:53.565 INFO:tasks.workunit.client.0.smithi117.stdout:6/114: mknod d3/d4/c2b 0 2024-04-20T11:08:53.568 INFO:tasks.workunit.client.0.smithi117.stdout:6/115: write d3/df/f21 [618303,128001] 0 2024-04-20T11:08:53.568 INFO:tasks.workunit.client.0.smithi117.stdout:5/102: creat de/d16/f1e x:0 0 0 2024-04-20T11:08:53.569 INFO:tasks.workunit.client.0.smithi117.stdout:6/116: fsync d3/f28 0 2024-04-20T11:08:53.580 INFO:tasks.workunit.client.0.smithi117.stdout:4/43: dwrite d1/fa [0,4194304] 0 2024-04-20T11:08:53.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:53 smithi117 ceph-mon[15659]: pgmap v6: 65 pgs: 65 active+clean; 209 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-20T11:08:53.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:53 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:08:53.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:53 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:08:53.646 INFO:tasks.workunit.client.0.smithi117.stdout:5/103: creat de/f1f x:0 0 0 2024-04-20T11:08:53.654 INFO:tasks.workunit.client.0.smithi117.stdout:4/44: mkdir d1/dd 0 2024-04-20T11:08:53.656 INFO:tasks.workunit.client.0.smithi117.stdout:4/45: chown d1/f7 57342 1 2024-04-20T11:08:53.656 INFO:tasks.workunit.client.0.smithi117.stdout:5/104: mknod de/c20 0 2024-04-20T11:08:53.658 INFO:tasks.workunit.client.0.smithi117.stdout:5/105: stat fb 0 2024-04-20T11:08:53.660 INFO:tasks.workunit.client.0.smithi117.stdout:1/28: write f2 [1581485,48569] 0 2024-04-20T11:08:53.660 INFO:tasks.workunit.client.0.smithi117.stdout:5/106: dread - de/f11 zero size 2024-04-20T11:08:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:53 smithi173 ceph-mon[23800]: pgmap v6: 65 pgs: 65 active+clean; 209 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-20T11:08:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:53 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:08:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:53 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:08:53.662 INFO:tasks.workunit.client.0.smithi117.stdout:5/107: fdatasync de/f11 0 2024-04-20T11:08:53.666 INFO:tasks.workunit.client.0.smithi117.stdout:1/29: fsync f2 0 2024-04-20T11:08:53.671 INFO:tasks.workunit.client.0.smithi117.stdout:8/53: truncate d0/f1 1682324 0 2024-04-20T11:08:53.676 INFO:tasks.workunit.client.0.smithi117.stdout:4/46: creat d1/dd/fe x:0 0 0 2024-04-20T11:08:53.679 INFO:tasks.workunit.client.0.smithi117.stdout:1/30: creat d6/f9 x:0 0 0 2024-04-20T11:08:53.712 INFO:tasks.workunit.client.0.smithi117.stdout:4/47: dwrite d1/dd/fe [0,4194304] 0 2024-04-20T11:08:53.717 INFO:tasks.workunit.client.0.smithi117.stdout:4/48: symlink d1/dd/lf 0 2024-04-20T11:08:53.734 INFO:tasks.workunit.client.0.smithi117.stdout:4/49: dread d1/f2 [0,4194304] 0 2024-04-20T11:08:53.737 INFO:tasks.workunit.client.0.smithi117.stdout:4/50: read d1/f7 [405009,103680] 0 2024-04-20T11:08:53.868 INFO:tasks.workunit.client.0.smithi117.stdout:6/117: truncate d3/f18 174342 0 2024-04-20T11:08:53.872 INFO:tasks.workunit.client.0.smithi117.stdout:6/118: write d3/fc [2001096,108295] 0 2024-04-20T11:08:53.875 INFO:tasks.workunit.client.0.smithi117.stdout:6/119: stat d3/f28 0 2024-04-20T11:08:53.886 INFO:tasks.workunit.client.0.smithi117.stdout:6/120: chown d3/l29 12868856 1 2024-04-20T11:08:53.889 INFO:tasks.workunit.client.0.smithi117.stdout:6/121: chown d3/d4/c23 204867545 1 2024-04-20T11:08:53.894 INFO:tasks.workunit.client.0.smithi117.stdout:6/122: truncate d3/d4/f19 1327791 0 2024-04-20T11:08:53.901 INFO:tasks.workunit.client.0.smithi117.stdout:3/53: sync 2024-04-20T11:08:53.902 INFO:tasks.workunit.client.0.smithi117.stdout:0/45: sync 2024-04-20T11:08:53.902 INFO:tasks.workunit.client.0.smithi117.stdout:9/114: sync 2024-04-20T11:08:53.906 INFO:tasks.workunit.client.0.smithi117.stdout:3/54: dread - d0/d5/d7/dd/f10 zero size 2024-04-20T11:08:53.906 INFO:tasks.workunit.client.0.smithi117.stdout:6/123: dread d3/fc [4194304,4194304] 0 2024-04-20T11:08:53.911 INFO:tasks.workunit.client.0.smithi117.stdout:6/124: truncate d3/f28 1438642 0 2024-04-20T11:08:53.917 INFO:tasks.workunit.client.0.smithi117.stdout:0/46: rmdir d3/d7/d8 39 2024-04-20T11:08:53.920 INFO:tasks.workunit.client.0.smithi117.stdout:4/51: write d1/dd/fe [4241593,25007] 0 2024-04-20T11:08:53.923 INFO:tasks.workunit.client.0.smithi117.stdout:3/55: mknod d0/d5/d11/c14 0 2024-04-20T11:08:53.928 INFO:tasks.workunit.client.0.smithi117.stdout:3/56: write d0/d5/f13 [784549,35646] 0 2024-04-20T11:08:53.938 INFO:tasks.workunit.client.0.smithi117.stdout:6/125: dread d3/fc [0,4194304] 0 2024-04-20T11:08:53.938 INFO:tasks.workunit.client.0.smithi117.stdout:9/115: dread f2 [0,4194304] 0 2024-04-20T11:08:53.940 INFO:tasks.workunit.client.0.smithi117.stdout:3/57: symlink d0/d5/de/l15 0 2024-04-20T11:08:53.947 INFO:tasks.workunit.client.0.smithi117.stdout:3/58: chown d0/f3 1691732633 1 2024-04-20T11:08:53.949 INFO:tasks.workunit.client.0.smithi117.stdout:0/47: dwrite d3/d5/fd [0,4194304] 0 2024-04-20T11:08:53.953 INFO:tasks.workunit.client.0.smithi117.stdout:6/126: creat d3/f2c x:0 0 0 2024-04-20T11:08:53.958 INFO:tasks.workunit.client.0.smithi117.stdout:6/127: chown d3/f28 92698 1 2024-04-20T11:08:53.958 INFO:tasks.workunit.client.0.smithi117.stdout:3/59: mkdir d0/d5/d7/d16 0 2024-04-20T11:08:53.958 INFO:tasks.workunit.client.0.smithi117.stdout:0/48: write d3/d5/fd [2328144,9838] 0 2024-04-20T11:08:53.959 INFO:tasks.workunit.client.0.smithi117.stdout:5/108: sync 2024-04-20T11:08:53.959 INFO:tasks.workunit.client.0.smithi117.stdout:1/31: sync 2024-04-20T11:08:53.959 INFO:tasks.workunit.client.0.smithi117.stdout:2/53: sync 2024-04-20T11:08:53.959 INFO:tasks.workunit.client.0.smithi117.stdout:7/72: sync 2024-04-20T11:08:53.960 INFO:tasks.workunit.client.0.smithi117.stdout:0/49: chown d3 4111792 1 2024-04-20T11:08:53.961 INFO:tasks.workunit.client.0.smithi117.stdout:8/54: dwrite d0/f5 [0,4194304] 0 2024-04-20T11:08:53.965 INFO:tasks.workunit.client.0.smithi117.stdout:1/32: stat d6/f9 0 2024-04-20T11:08:53.965 INFO:tasks.workunit.client.0.smithi117.stdout:7/73: readlink d1/d4/d6/d13/l19 0 2024-04-20T11:08:53.970 INFO:tasks.workunit.client.0.smithi117.stdout:6/128: write d3/d4/f19 [1174463,124344] 0 2024-04-20T11:08:53.970 INFO:tasks.workunit.client.0.smithi117.stdout:7/74: chown d1/f12 147 1 2024-04-20T11:08:53.970 INFO:tasks.workunit.client.0.smithi117.stdout:5/109: mkdir de/d21 0 2024-04-20T11:08:53.970 INFO:tasks.workunit.client.0.smithi117.stdout:2/54: chown d1/d4/ff 926 1 2024-04-20T11:08:53.970 INFO:tasks.workunit.client.0.smithi117.stdout:3/60: rename d0/d5/d7 to d0/d5/d7/dd/d17 22 2024-04-20T11:08:53.972 INFO:tasks.workunit.client.0.smithi117.stdout:0/50: chown d3/d7/d8/db 15739 1 2024-04-20T11:08:53.972 INFO:tasks.workunit.client.0.smithi117.stdout:9/116: dwrite d4/d5/da/f19 [0,4194304] 0 2024-04-20T11:08:53.974 INFO:tasks.workunit.client.0.smithi117.stdout:2/55: readlink d1/d4/dd/l13 0 2024-04-20T11:08:53.975 INFO:tasks.workunit.client.0.smithi117.stdout:5/110: rmdir de/d16 39 2024-04-20T11:08:53.976 INFO:tasks.workunit.client.0.smithi117.stdout:7/75: truncate d1/fc 804819 0 2024-04-20T11:08:53.978 INFO:tasks.workunit.client.0.smithi117.stdout:0/51: symlink d3/d7/d8/l10 0 2024-04-20T11:08:53.978 INFO:tasks.workunit.client.0.smithi117.stdout:7/76: write d1/f16 [135431,268] 0 2024-04-20T11:08:53.981 INFO:tasks.workunit.client.0.smithi117.stdout:2/56: unlink d1/f12 0 2024-04-20T11:08:53.982 INFO:tasks.workunit.client.0.smithi117.stdout:2/57: dread - d1/d4/fb zero size 2024-04-20T11:08:53.983 INFO:tasks.workunit.client.0.smithi117.stdout:8/55: truncate d0/f7 137416 0 2024-04-20T11:08:53.990 INFO:tasks.workunit.client.0.smithi117.stdout:8/56: chown d0/fa 1385 1 2024-04-20T11:08:53.993 INFO:tasks.workunit.client.0.smithi117.stdout:6/129: dread d3/df/f14 [0,4194304] 0 2024-04-20T11:08:53.994 INFO:tasks.workunit.client.0.smithi117.stdout:4/52: dwrite d1/f2 [0,4194304] 0 2024-04-20T11:08:53.994 INFO:tasks.workunit.client.0.smithi117.stdout:9/117: mkdir d4/d30/d31 0 2024-04-20T11:08:53.996 INFO:tasks.workunit.client.0.smithi117.stdout:2/58: unlink d1/d4/ff 0 2024-04-20T11:08:54.000 INFO:tasks.workunit.client.0.smithi117.stdout:2/59: readlink d1/d4/dd/l13 0 2024-04-20T11:08:54.002 INFO:tasks.workunit.client.0.smithi117.stdout:8/57: rmdir d0/d8 39 2024-04-20T11:08:54.013 INFO:tasks.workunit.client.0.smithi117.stdout:9/118: creat d4/d5/da/f32 x:0 0 0 2024-04-20T11:08:54.014 INFO:tasks.workunit.client.0.smithi117.stdout:0/52: dwrite d3/d5/fd [0,4194304] 0 2024-04-20T11:08:54.016 INFO:tasks.workunit.client.0.smithi117.stdout:3/61: getdents d0/d5/de 0 2024-04-20T11:08:54.023 INFO:tasks.workunit.client.0.smithi117.stdout:2/60: creat d1/d4/dd/f14 x:0 0 0 2024-04-20T11:08:54.023 INFO:tasks.workunit.client.0.smithi117.stdout:5/111: dwrite de/d16/f1e [0,4194304] 0 2024-04-20T11:08:54.035 INFO:tasks.workunit.client.0.smithi117.stdout:2/61: mkdir d1/d15 0 2024-04-20T11:08:54.037 INFO:tasks.workunit.client.0.smithi117.stdout:5/112: mknod de/d21/c22 0 2024-04-20T11:08:54.044 INFO:tasks.workunit.client.0.smithi117.stdout:4/53: truncate d1/f2 3225462 0 2024-04-20T11:08:54.048 INFO:tasks.workunit.client.0.smithi117.stdout:2/62: write d1/d4/f8 [661795,53727] 0 2024-04-20T11:08:54.048 INFO:tasks.workunit.client.0.smithi117.stdout:0/53: dread d3/d5/fd [0,4194304] 0 2024-04-20T11:08:54.052 INFO:tasks.workunit.client.0.smithi117.stdout:0/54: rename d3/d5 to d3/d5/d11 22 2024-04-20T11:08:54.052 INFO:tasks.workunit.client.0.smithi117.stdout:3/62: truncate d0/fb 934325 0 2024-04-20T11:08:54.057 INFO:tasks.workunit.client.0.smithi117.stdout:2/63: fdatasync d1/d4/f8 0 2024-04-20T11:08:54.057 INFO:tasks.workunit.client.0.smithi117.stdout:5/113: rename de/d21/c22 to de/d21/c23 0 2024-04-20T11:08:54.064 INFO:tasks.workunit.client.0.smithi117.stdout:0/55: write d3/d7/f9 [2431696,26278] 0 2024-04-20T11:08:54.064 INFO:tasks.workunit.client.0.smithi117.stdout:9/119: dwrite d4/d5/da/f32 [0,4194304] 0 2024-04-20T11:08:54.072 INFO:tasks.workunit.client.0.smithi117.stdout:2/64: mkdir d1/d4/dd/d16 0 2024-04-20T11:08:54.074 INFO:tasks.workunit.client.0.smithi117.stdout:9/120: chown d4/de/l1f 107038 1 2024-04-20T11:08:54.075 INFO:tasks.workunit.client.0.smithi117.stdout:6/130: dwrite d3/d4/f1b [0,4194304] 0 2024-04-20T11:08:54.078 INFO:tasks.workunit.client.0.smithi117.stdout:0/56: creat d3/d7/d8/df/f12 x:0 0 0 2024-04-20T11:08:54.082 INFO:tasks.workunit.client.0.smithi117.stdout:2/65: dread - d1/d4/f7 zero size 2024-04-20T11:08:54.084 INFO:tasks.workunit.client.0.smithi117.stdout:4/54: dwrite d1/dd/fe [0,4194304] 0 2024-04-20T11:08:54.094 INFO:tasks.workunit.client.0.smithi117.stdout:2/66: mkdir d1/d4/dd/d17 0 2024-04-20T11:08:54.098 INFO:tasks.workunit.client.0.smithi117.stdout:4/55: rename d1/fa to d1/f10 0 2024-04-20T11:08:54.099 INFO:tasks.workunit.client.0.smithi117.stdout:3/63: dread d0/d5/d7/dd/ff [0,4194304] 0 2024-04-20T11:08:54.102 INFO:tasks.workunit.client.0.smithi117.stdout:5/114: dwrite de/f14 [0,4194304] 0 2024-04-20T11:08:54.106 INFO:tasks.workunit.client.0.smithi117.stdout:5/115: stat de/c20 0 2024-04-20T11:08:54.118 INFO:tasks.workunit.client.0.smithi117.stdout:3/64: dread d0/d5/f13 [0,4194304] 0 2024-04-20T11:08:54.120 INFO:tasks.workunit.client.0.smithi117.stdout:9/121: dwrite d4/d5/da/f29 [0,4194304] 0 2024-04-20T11:08:54.120 INFO:tasks.workunit.client.0.smithi117.stdout:6/131: dread d3/d4/f12 [0,4194304] 0 2024-04-20T11:08:54.122 INFO:tasks.workunit.client.0.smithi117.stdout:3/65: readlink d0/d5/de/l15 0 2024-04-20T11:08:54.123 INFO:tasks.workunit.client.0.smithi117.stdout:9/122: chown d4/de 441085 1 2024-04-20T11:08:54.124 INFO:tasks.workunit.client.0.smithi117.stdout:4/56: link d1/c4 d1/d9/c11 0 2024-04-20T11:08:54.127 INFO:tasks.workunit.client.0.smithi117.stdout:9/123: fdatasync d4/d5/da/f19 0 2024-04-20T11:08:54.129 INFO:tasks.workunit.client.0.smithi117.stdout:9/124: read d4/d5/da/fc [1122528,12827] 0 2024-04-20T11:08:54.130 INFO:tasks.workunit.client.0.smithi117.stdout:3/66: creat d0/d5/d7/f18 x:0 0 0 2024-04-20T11:08:54.130 INFO:tasks.workunit.client.0.smithi117.stdout:2/67: write d1/f2 [8983212,88662] 0 2024-04-20T11:08:54.132 INFO:tasks.workunit.client.0.smithi117.stdout:5/116: dread f0 [0,4194304] 0 2024-04-20T11:08:54.132 INFO:tasks.workunit.client.0.smithi117.stdout:2/68: fdatasync d1/d4/f7 0 2024-04-20T11:08:54.134 INFO:tasks.workunit.client.0.smithi117.stdout:3/67: dread - d0/d5/fc zero size 2024-04-20T11:08:54.135 INFO:tasks.workunit.client.0.smithi117.stdout:4/57: creat d1/d9/f12 x:0 0 0 2024-04-20T11:08:54.139 INFO:tasks.workunit.client.0.smithi117.stdout:2/69: truncate d1/d4/fb 490234 0 2024-04-20T11:08:54.139 INFO:tasks.workunit.client.0.smithi117.stdout:6/132: rename d3/d4/c15 to d3/c2d 0 2024-04-20T11:08:54.140 INFO:tasks.workunit.client.0.smithi117.stdout:9/125: rmdir d4/de/d10 39 2024-04-20T11:08:54.145 INFO:tasks.workunit.client.0.smithi117.stdout:5/117: mkdir de/d16/d24 0 2024-04-20T11:08:54.145 INFO:tasks.workunit.client.0.smithi117.stdout:4/58: creat d1/dd/f13 x:0 0 0 2024-04-20T11:08:54.145 INFO:tasks.workunit.client.0.smithi117.stdout:3/68: creat d0/d5/f19 x:0 0 0 2024-04-20T11:08:54.145 INFO:tasks.workunit.client.0.smithi117.stdout:5/118: dread - de/f11 zero size 2024-04-20T11:08:54.146 INFO:tasks.workunit.client.0.smithi117.stdout:5/119: read - de/f18 zero size 2024-04-20T11:08:54.146 INFO:tasks.workunit.client.0.smithi117.stdout:9/126: creat d4/d30/d31/f33 x:0 0 0 2024-04-20T11:08:54.147 INFO:tasks.workunit.client.0.smithi117.stdout:5/120: fdatasync de/f18 0 2024-04-20T11:08:54.148 INFO:tasks.workunit.client.0.smithi117.stdout:5/121: stat de 0 2024-04-20T11:08:54.149 INFO:tasks.workunit.client.0.smithi117.stdout:3/69: chown d0/fb 195584 1 2024-04-20T11:08:54.152 INFO:tasks.workunit.client.0.smithi117.stdout:2/70: link d1/d4/fb d1/d4/dd/de/f18 0 2024-04-20T11:08:54.161 INFO:tasks.workunit.client.0.smithi117.stdout:3/70: dread d0/f3 [0,4194304] 0 2024-04-20T11:08:54.164 INFO:tasks.workunit.client.0.smithi117.stdout:9/127: getdents d4/d5/da 0 2024-04-20T11:08:54.178 INFO:tasks.workunit.client.0.smithi117.stdout:3/71: readlink d0/d5/d11/l12 0 2024-04-20T11:08:54.180 INFO:tasks.workunit.client.0.smithi117.stdout:6/133: dwrite d3/f2c [0,4194304] 0 2024-04-20T11:08:54.186 INFO:tasks.workunit.client.0.smithi117.stdout:4/59: dwrite d1/dd/f13 [0,4194304] 0 2024-04-20T11:08:54.186 INFO:tasks.workunit.client.0.smithi117.stdout:3/72: dread - d0/d5/d7/dd/f10 zero size 2024-04-20T11:08:54.187 INFO:tasks.workunit.client.0.smithi117.stdout:9/128: creat d4/de/d10/f34 x:0 0 0 2024-04-20T11:08:54.192 INFO:tasks.workunit.client.0.smithi117.stdout:6/134: mkdir d3/d4/d2e 0 2024-04-20T11:08:54.197 INFO:tasks.workunit.client.0.smithi117.stdout:9/129: mkdir d4/d35 0 2024-04-20T11:08:54.197 INFO:tasks.workunit.client.0.smithi117.stdout:5/122: dwrite de/d16/f1b [0,4194304] 0 2024-04-20T11:08:54.202 INFO:tasks.workunit.client.0.smithi117.stdout:9/130: readlink d4/de/l1f 0 2024-04-20T11:08:54.203 INFO:tasks.workunit.client.0.smithi117.stdout:6/135: creat d3/d4/f2f x:0 0 0 2024-04-20T11:08:54.204 INFO:tasks.workunit.client.0.smithi117.stdout:5/123: write de/f1f [621988,55612] 0 2024-04-20T11:08:54.206 INFO:tasks.workunit.client.0.smithi117.stdout:3/73: link d0/d5/d7/dd/f10 d0/f1a 0 2024-04-20T11:08:54.206 INFO:tasks.workunit.client.0.smithi117.stdout:2/71: dwrite d1/f3 [0,4194304] 0 2024-04-20T11:08:54.208 INFO:tasks.workunit.client.0.smithi117.stdout:9/131: chown d4/d5/f2a 79573931 1 2024-04-20T11:08:54.212 INFO:tasks.workunit.client.0.smithi117.stdout:3/74: rename d0/d5 to d0/d5/d7/d1b 22 2024-04-20T11:08:54.214 INFO:tasks.workunit.client.0.smithi117.stdout:3/75: fdatasync d0/d5/d7/f18 0 2024-04-20T11:08:54.215 INFO:tasks.workunit.client.0.smithi117.stdout:9/132: creat d4/d30/f36 x:0 0 0 2024-04-20T11:08:54.218 INFO:tasks.workunit.client.0.smithi117.stdout:9/133: chown d4/de/f1b 1 1 2024-04-20T11:08:54.221 INFO:tasks.workunit.client.0.smithi117.stdout:6/136: link d3/df/f14 d3/f30 0 2024-04-20T11:08:54.224 INFO:tasks.workunit.client.0.smithi117.stdout:4/60: truncate d1/dd/fe 1286982 0 2024-04-20T11:08:54.227 INFO:tasks.workunit.client.0.smithi117.stdout:6/137: write d3/d4/f8 [830379,11865] 0 2024-04-20T11:08:54.227 INFO:tasks.workunit.client.0.smithi117.stdout:4/61: mknod d1/d9/c14 0 2024-04-20T11:08:54.230 INFO:tasks.workunit.client.0.smithi117.stdout:6/138: readlink d3/l29 0 2024-04-20T11:08:54.232 INFO:tasks.workunit.client.0.smithi117.stdout:4/62: chown d1/f2 3 1 2024-04-20T11:08:54.233 INFO:tasks.workunit.client.0.smithi117.stdout:5/124: dwrite de/f11 [0,4194304] 0 2024-04-20T11:08:54.237 INFO:tasks.workunit.client.0.smithi117.stdout:2/72: fdatasync d1/f11 0 2024-04-20T11:08:54.241 INFO:tasks.workunit.client.0.smithi117.stdout:4/63: write d1/f2 [2533083,125833] 0 2024-04-20T11:08:54.245 INFO:tasks.workunit.client.0.smithi117.stdout:5/125: dread f5 [0,4194304] 0 2024-04-20T11:08:54.245 INFO:tasks.workunit.client.0.smithi117.stdout:9/134: truncate d4/de/d10/f24 1923845 0 2024-04-20T11:08:54.248 INFO:tasks.workunit.client.0.smithi117.stdout:2/73: creat d1/d4/dd/f19 x:0 0 0 2024-04-20T11:08:54.249 INFO:tasks.workunit.client.0.smithi117.stdout:3/76: dwrite d0/d5/fc [0,4194304] 0 2024-04-20T11:08:54.252 INFO:tasks.workunit.client.0.smithi117.stdout:5/126: write de/f11 [4657044,48457] 0 2024-04-20T11:08:54.253 INFO:tasks.workunit.client.0.smithi117.stdout:4/64: fdatasync d1/f2 0 2024-04-20T11:08:54.253 INFO:tasks.workunit.client.0.smithi117.stdout:9/135: mknod d4/d30/d31/c37 0 2024-04-20T11:08:54.255 INFO:tasks.workunit.client.0.smithi117.stdout:9/136: stat d4/de/d10/f34 0 2024-04-20T11:08:54.258 INFO:tasks.workunit.client.0.smithi117.stdout:3/77: dread d0/f3 [0,4194304] 0 2024-04-20T11:08:54.261 INFO:tasks.workunit.client.0.smithi117.stdout:4/65: creat d1/f15 x:0 0 0 2024-04-20T11:08:54.261 INFO:tasks.workunit.client.0.smithi117.stdout:3/78: chown d0/f3 26789 1 2024-04-20T11:08:54.264 INFO:tasks.workunit.client.0.smithi117.stdout:2/74: mknod d1/c1a 0 2024-04-20T11:08:54.264 INFO:tasks.workunit.client.0.smithi117.stdout:3/79: truncate d0/f1a 519021 0 2024-04-20T11:08:54.266 INFO:tasks.workunit.client.0.smithi117.stdout:2/75: readlink d1/d4/dd/l13 0 2024-04-20T11:08:54.267 INFO:tasks.workunit.client.0.smithi117.stdout:2/76: fsync d1/d4/f7 0 2024-04-20T11:08:54.269 INFO:tasks.workunit.client.0.smithi117.stdout:5/127: symlink de/l25 0 2024-04-20T11:08:54.270 INFO:tasks.workunit.client.0.smithi117.stdout:5/128: chown de/d16 1287666847 1 2024-04-20T11:08:54.271 INFO:tasks.workunit.client.0.smithi117.stdout:5/129: rename de to de/d21/d26 22 2024-04-20T11:08:54.271 INFO:tasks.workunit.client.0.smithi117.stdout:4/66: creat d1/d9/f16 x:0 0 0 2024-04-20T11:08:54.272 INFO:tasks.workunit.client.0.smithi117.stdout:3/80: truncate d0/fb 72280 0 2024-04-20T11:08:54.274 INFO:tasks.workunit.client.0.smithi117.stdout:3/81: read - d0/d5/f19 zero size 2024-04-20T11:08:54.275 INFO:tasks.workunit.client.0.smithi117.stdout:3/82: fdatasync d0/d5/fc 0 2024-04-20T11:08:54.276 INFO:tasks.workunit.client.0.smithi117.stdout:9/137: creat d4/de/d10/f38 x:0 0 0 2024-04-20T11:08:54.277 INFO:tasks.workunit.client.0.smithi117.stdout:2/77: symlink d1/d15/l1b 0 2024-04-20T11:08:54.278 INFO:tasks.workunit.client.0.smithi117.stdout:9/138: dread - d4/de/d10/f34 zero size 2024-04-20T11:08:54.279 INFO:tasks.workunit.client.0.smithi117.stdout:2/78: fdatasync d1/d4/dd/f14 0 2024-04-20T11:08:54.283 INFO:tasks.workunit.client.0.smithi117.stdout:2/79: truncate d1/d4/dd/f19 239194 0 2024-04-20T11:08:54.283 INFO:tasks.workunit.client.0.smithi117.stdout:5/130: link de/l13 de/d21/l27 0 2024-04-20T11:08:54.286 INFO:tasks.workunit.client.0.smithi117.stdout:4/67: getdents d1/dd 0 2024-04-20T11:08:54.293 INFO:tasks.workunit.client.0.smithi117.stdout:9/139: dread d4/d5/da/f19 [0,4194304] 0 2024-04-20T11:08:54.295 INFO:tasks.workunit.client.0.smithi117.stdout:4/68: link d1/f15 d1/d9/f17 0 2024-04-20T11:08:54.296 INFO:tasks.workunit.client.0.smithi117.stdout:2/80: link d1/c1a d1/d15/c1c 0 2024-04-20T11:08:54.298 INFO:tasks.workunit.client.0.smithi117.stdout:2/81: write d1/f3 [2092426,2526] 0 2024-04-20T11:08:54.300 INFO:tasks.workunit.client.0.smithi117.stdout:2/82: readlink d1/d4/dd/l13 0 2024-04-20T11:08:54.302 INFO:tasks.workunit.client.0.smithi117.stdout:4/69: mknod d1/c18 0 2024-04-20T11:08:54.305 INFO:tasks.workunit.client.0.smithi117.stdout:2/83: mkdir d1/d15/d1d 0 2024-04-20T11:08:54.306 INFO:tasks.workunit.client.0.smithi117.stdout:2/84: readlink d1/d15/l1b 0 2024-04-20T11:08:54.307 INFO:tasks.workunit.client.0.smithi117.stdout:4/70: rename d1/dd/fe to d1/f19 0 2024-04-20T11:08:54.311 INFO:tasks.workunit.client.0.smithi117.stdout:4/71: mkdir d1/dd/d1a 0 2024-04-20T11:08:54.418 INFO:tasks.workunit.client.0.smithi117.stdout:0/57: sync 2024-04-20T11:08:54.418 INFO:tasks.workunit.client.0.smithi117.stdout:1/33: sync 2024-04-20T11:08:54.418 INFO:tasks.workunit.client.0.smithi117.stdout:8/58: sync 2024-04-20T11:08:54.418 INFO:tasks.workunit.client.0.smithi117.stdout:7/77: sync 2024-04-20T11:08:54.437 INFO:tasks.workunit.client.0.smithi117.stdout:7/78: rmdir d1 39 2024-04-20T11:08:54.441 INFO:tasks.workunit.client.0.smithi117.stdout:3/83: dwrite d0/fb [0,4194304] 0 2024-04-20T11:08:54.443 INFO:tasks.workunit.client.0.smithi117.stdout:1/34: write d6/f7 [966710,86599] 0 2024-04-20T11:08:54.446 INFO:tasks.workunit.client.0.smithi117.stdout:3/84: chown d0/d5/d7/d16 0 1 2024-04-20T11:08:54.446 INFO:tasks.workunit.client.0.smithi117.stdout:9/140: dwrite f0 [0,4194304] 0 2024-04-20T11:08:54.447 INFO:tasks.workunit.client.0.smithi117.stdout:1/35: chown f2 701419263 1 2024-04-20T11:08:54.451 INFO:tasks.workunit.client.0.smithi117.stdout:7/79: chown d1/d4/d6/d13 1126 1 2024-04-20T11:08:54.451 INFO:tasks.workunit.client.0.smithi117.stdout:1/36: write f4 [1279094,44575] 0 2024-04-20T11:08:54.453 INFO:tasks.workunit.client.0.smithi117.stdout:1/37: chown c5 18756621 1 2024-04-20T11:08:54.453 INFO:tasks.workunit.client.0.smithi117.stdout:1/38: readlink - no filename 2024-04-20T11:08:54.458 INFO:tasks.workunit.client.0.smithi117.stdout:3/85: rmdir d0/d5 39 2024-04-20T11:08:54.461 INFO:tasks.workunit.client.0.smithi117.stdout:9/141: creat d4/f39 x:0 0 0 2024-04-20T11:08:54.465 INFO:tasks.workunit.client.0.smithi117.stdout:2/85: truncate d1/d4/f5 4618143 0 2024-04-20T11:08:54.471 INFO:tasks.workunit.client.0.smithi117.stdout:7/80: unlink d1/f16 0 2024-04-20T11:08:54.471 INFO:tasks.workunit.client.0.smithi117.stdout:7/81: fdatasync d1/fc 0 2024-04-20T11:08:54.473 INFO:tasks.workunit.client.0.smithi117.stdout:1/39: unlink d6/f7 0 2024-04-20T11:08:54.474 INFO:tasks.workunit.client.0.smithi117.stdout:8/59: creat d0/d8/d11/f14 x:0 0 0 2024-04-20T11:08:54.476 INFO:tasks.workunit.client.0.smithi117.stdout:8/60: chown d0/f5 5 1 2024-04-20T11:08:54.485 INFO:tasks.workunit.client.0.smithi117.stdout:7/82: creat d1/d4/f1a x:0 0 0 2024-04-20T11:08:54.490 INFO:tasks.workunit.client.0.smithi117.stdout:3/86: rename d0/d5/d7/d16 to d0/d5/de/d1c 0 2024-04-20T11:08:54.491 INFO:tasks.workunit.client.0.smithi117.stdout:7/83: dread d1/fb [0,4194304] 0 2024-04-20T11:08:54.493 INFO:tasks.workunit.client.0.smithi117.stdout:3/87: write d0/fb [4568560,63831] 0 2024-04-20T11:08:54.496 INFO:tasks.workunit.client.0.smithi117.stdout:4/72: truncate d1/dd/f13 2190884 0 2024-04-20T11:08:54.497 INFO:tasks.workunit.client.0.smithi117.stdout:4/73: fdatasync d1/d9/f12 0 2024-04-20T11:08:54.498 INFO:tasks.workunit.client.0.smithi117.stdout:2/86: creat d1/d4/dd/d17/f1e x:0 0 0 2024-04-20T11:08:54.500 INFO:tasks.workunit.client.0.smithi117.stdout:0/58: truncate d3/d7/f9 2523994 0 2024-04-20T11:08:54.504 INFO:tasks.workunit.client.0.smithi117.stdout:8/61: mknod d0/c15 0 2024-04-20T11:08:54.508 INFO:tasks.workunit.client.0.smithi117.stdout:7/84: fdatasync d1/f7 0 2024-04-20T11:08:54.510 INFO:tasks.workunit.client.0.smithi117.stdout:7/85: chown d1/d4/d6/d17 8117 1 2024-04-20T11:08:54.512 INFO:tasks.workunit.client.0.smithi117.stdout:9/142: creat d4/de/d10/f3a x:0 0 0 2024-04-20T11:08:54.513 INFO:tasks.workunit.client.0.smithi117.stdout:7/86: write d1/d4/d6/d13/f15 [71318,99635] 0 2024-04-20T11:08:54.517 INFO:tasks.workunit.client.0.smithi117.stdout:2/87: creat d1/d4/dd/d17/f1f x:0 0 0 2024-04-20T11:08:54.532 INFO:tasks.workunit.client.0.smithi117.stdout:3/88: unlink d0/c6 0 2024-04-20T11:08:54.532 INFO:tasks.workunit.client.0.smithi117.stdout:9/143: dwrite d4/de/f2c [0,4194304] 0 2024-04-20T11:08:54.534 INFO:tasks.workunit.client.0.smithi117.stdout:9/144: readlink d4/d5/da/l2f 0 2024-04-20T11:08:54.535 INFO:tasks.workunit.client.0.smithi117.stdout:4/74: creat d1/dd/d1a/f1b x:0 0 0 2024-04-20T11:08:54.536 INFO:tasks.workunit.client.0.smithi117.stdout:4/75: stat d1/d9/f12 0 2024-04-20T11:08:54.541 INFO:tasks.workunit.client.0.smithi117.stdout:1/40: getdents d6 0 2024-04-20T11:08:54.542 INFO:tasks.workunit.client.0.smithi117.stdout:1/41: fsync d6/f9 0 2024-04-20T11:08:54.545 INFO:tasks.workunit.client.0.smithi117.stdout:0/59: link d3/d5/fd d3/d7/d8/f13 0 2024-04-20T11:08:54.545 INFO:tasks.workunit.client.0.smithi117.stdout:0/60: chown d3/d5 2 1 2024-04-20T11:08:54.554 INFO:tasks.workunit.client.0.smithi117.stdout:7/87: mknod d1/d4/d6/d17/c1b 0 2024-04-20T11:08:54.557 INFO:tasks.workunit.client.0.smithi117.stdout:4/76: mkdir d1/dd/d1c 0 2024-04-20T11:08:54.558 INFO:tasks.workunit.client.0.smithi117.stdout:4/77: chown d1/f6 133800313 1 2024-04-20T11:08:54.559 INFO:tasks.workunit.client.0.smithi117.stdout:2/88: mkdir d1/d20 0 2024-04-20T11:08:54.567 INFO:tasks.workunit.client.0.smithi117.stdout:7/88: mkdir d1/d4/d1c 0 2024-04-20T11:08:54.576 INFO:tasks.workunit.client.0.smithi117.stdout:1/42: rename d6/c8 to d6/ca 0 2024-04-20T11:08:54.576 INFO:tasks.workunit.client.0.smithi117.stdout:9/145: write d4/de/f2c [4139601,89004] 0 2024-04-20T11:08:54.582 INFO:tasks.workunit.client.0.smithi117.stdout:2/89: creat d1/d20/f21 x:0 0 0 2024-04-20T11:08:54.587 INFO:tasks.workunit.client.0.smithi117.stdout:8/62: dwrite d0/f7 [0,4194304] 0 2024-04-20T11:08:54.589 INFO:tasks.workunit.client.0.smithi117.stdout:4/78: link d1/f7 d1/dd/d1a/f1d 0 2024-04-20T11:08:54.590 INFO:tasks.workunit.client.0.smithi117.stdout:1/43: dread f4 [0,4194304] 0 2024-04-20T11:08:54.601 INFO:tasks.workunit.client.0.smithi117.stdout:7/89: rmdir d1/d4 39 2024-04-20T11:08:54.603 INFO:tasks.workunit.client.0.smithi117.stdout:1/44: symlink d6/lb 0 2024-04-20T11:08:54.605 INFO:tasks.workunit.client.0.smithi117.stdout:0/61: chown d3/d7/f9 1819942 1 2024-04-20T11:08:54.606 INFO:tasks.workunit.client.0.smithi117.stdout:2/90: fdatasync d1/d4/f5 0 2024-04-20T11:08:54.608 INFO:tasks.workunit.client.0.smithi117.stdout:2/91: dread - d1/d4/dd/f14 zero size 2024-04-20T11:08:54.617 INFO:tasks.workunit.client.0.smithi117.stdout:1/45: symlink d6/lc 0 2024-04-20T11:08:54.618 INFO:tasks.workunit.client.0.smithi117.stdout:1/46: stat d6/lb 0 2024-04-20T11:08:54.620 INFO:tasks.workunit.client.0.smithi117.stdout:0/62: chown l2 15486412 1 2024-04-20T11:08:54.623 INFO:tasks.workunit.client.0.smithi117.stdout:9/146: link d4/de/d10/l11 d4/d35/l3b 0 2024-04-20T11:08:54.624 INFO:tasks.workunit.client.0.smithi117.stdout:9/147: fdatasync d4/de/f2c 0 2024-04-20T11:08:54.625 INFO:tasks.workunit.client.0.smithi117.stdout:9/148: read - d4/de/d10/f38 zero size 2024-04-20T11:08:54.628 INFO:tasks.workunit.client.0.smithi117.stdout:4/79: write d1/d9/f17 [266258,98966] 0 2024-04-20T11:08:54.629 INFO:tasks.workunit.client.0.smithi117.stdout:7/90: mkdir d1/d1d 0 2024-04-20T11:08:54.632 INFO:tasks.workunit.client.0.smithi117.stdout:1/47: dread f2 [0,4194304] 0 2024-04-20T11:08:54.635 INFO:tasks.workunit.client.0.smithi117.stdout:8/63: getdents d0/d8 0 2024-04-20T11:08:54.638 INFO:tasks.workunit.client.0.smithi117.stdout:2/92: link d1/d4/dd/l13 d1/d15/d1d/l22 0 2024-04-20T11:08:54.638 INFO:tasks.workunit.client.0.smithi117.stdout:4/80: fdatasync d1/dd/f13 0 2024-04-20T11:08:54.640 INFO:tasks.workunit.client.0.smithi117.stdout:1/48: dread f4 [0,4194304] 0 2024-04-20T11:08:54.642 INFO:tasks.workunit.client.0.smithi117.stdout:2/93: dread - d1/d4/dd/f14 zero size 2024-04-20T11:08:54.642 INFO:tasks.workunit.client.0.smithi117.stdout:0/63: creat d3/d7/f14 x:0 0 0 2024-04-20T11:08:54.643 INFO:tasks.workunit.client.0.smithi117.stdout:7/91: mkdir d1/d4/d6/d17/d1e 0 2024-04-20T11:08:54.648 INFO:tasks.workunit.client.0.smithi117.stdout:6/139: sync 2024-04-20T11:08:54.649 INFO:tasks.workunit.client.0.smithi117.stdout:5/131: sync 2024-04-20T11:08:54.650 INFO:tasks.workunit.client.0.smithi117.stdout:1/49: write f4 [293455,35756] 0 2024-04-20T11:08:54.652 INFO:tasks.workunit.client.0.smithi117.stdout:9/149: dwrite d4/d5/f21 [0,4194304] 0 2024-04-20T11:08:54.654 INFO:tasks.workunit.client.0.smithi117.stdout:7/92: chown d1/d4/d6/d13/f15 29522807 1 2024-04-20T11:08:54.654 INFO:tasks.workunit.client.0.smithi117.stdout:5/132: fsync de/f19 0 2024-04-20T11:08:54.658 INFO:tasks.workunit.client.0.smithi117.stdout:6/140: creat d3/df/f31 x:0 0 0 2024-04-20T11:08:54.663 INFO:tasks.workunit.client.0.smithi117.stdout:9/150: mkdir d4/d5/d3c 0 2024-04-20T11:08:54.670 INFO:tasks.workunit.client.0.smithi117.stdout:8/64: dwrite d0/d9/f12 [0,4194304] 0 2024-04-20T11:08:54.671 INFO:tasks.workunit.client.0.smithi117.stdout:7/93: chown d1/d4/c5 100978 1 2024-04-20T11:08:54.673 INFO:tasks.workunit.client.0.smithi117.stdout:4/81: write d1/f19 [2273121,65857] 0 2024-04-20T11:08:54.676 INFO:tasks.workunit.client.0.smithi117.stdout:7/94: dread - d1/f18 zero size 2024-04-20T11:08:54.684 INFO:tasks.workunit.client.0.smithi117.stdout:0/64: dread d3/d7/d8/f13 [0,4194304] 0 2024-04-20T11:08:54.685 INFO:tasks.workunit.client.0.smithi117.stdout:4/82: fsync d1/f19 0 2024-04-20T11:08:54.685 INFO:tasks.workunit.client.0.smithi117.stdout:1/50: getdents d6 0 2024-04-20T11:08:54.692 INFO:tasks.workunit.client.0.smithi117.stdout:0/65: mknod d3/d5/c15 0 2024-04-20T11:08:54.692 INFO:tasks.workunit.client.0.smithi117.stdout:1/51: dread - d6/f9 zero size 2024-04-20T11:08:54.693 INFO:tasks.workunit.client.0.smithi117.stdout:8/65: truncate d0/f3 2569043 0 2024-04-20T11:08:54.696 INFO:tasks.workunit.client.0.smithi117.stdout:2/94: dwrite d1/d4/dd/f14 [0,4194304] 0 2024-04-20T11:08:54.712 INFO:tasks.workunit.client.0.smithi117.stdout:0/66: rename d3/d7/d8/db to d3/d7/d16 0 2024-04-20T11:08:54.713 INFO:tasks.workunit.client.0.smithi117.stdout:2/95: truncate d1/d4/f5 2261166 0 2024-04-20T11:08:54.714 INFO:tasks.workunit.client.0.smithi117.stdout:6/141: dwrite d3/d4/f12 [4194304,4194304] 0 2024-04-20T11:08:54.717 INFO:tasks.workunit.client.0.smithi117.stdout:9/151: dwrite d4/de/d10/f24 [0,4194304] 0 2024-04-20T11:08:54.719 INFO:tasks.workunit.client.0.smithi117.stdout:0/67: mkdir d3/d17 0 2024-04-20T11:08:54.723 INFO:tasks.workunit.client.0.smithi117.stdout:2/96: fdatasync d1/f3 0 2024-04-20T11:08:54.724 INFO:tasks.workunit.client.0.smithi117.stdout:4/83: dwrite d1/f19 [0,4194304] 0 2024-04-20T11:08:54.726 INFO:tasks.workunit.client.0.smithi117.stdout:3/89: sync 2024-04-20T11:08:54.730 INFO:tasks.workunit.client.0.smithi117.stdout:4/84: rmdir d1 39 2024-04-20T11:08:54.732 INFO:tasks.workunit.client.0.smithi117.stdout:3/90: truncate d0/d5/f19 684223 0 2024-04-20T11:08:54.740 INFO:tasks.workunit.client.0.smithi117.stdout:6/142: dread d3/d4/fe [0,4194304] 0 2024-04-20T11:08:54.741 INFO:tasks.workunit.client.0.smithi117.stdout:6/143: readlink d3/d4/l25 0 2024-04-20T11:08:54.743 INFO:tasks.workunit.client.0.smithi117.stdout:7/95: truncate d1/f10 92444 0 2024-04-20T11:08:54.747 INFO:tasks.workunit.client.0.smithi117.stdout:9/152: getdents d4/de/d10 0 2024-04-20T11:08:54.749 INFO:tasks.workunit.client.0.smithi117.stdout:1/52: truncate f4 638356 0 2024-04-20T11:08:54.749 INFO:tasks.workunit.client.0.smithi117.stdout:3/91: rename d0/d5/de/d1c to d0/d5/de/d1d 0 2024-04-20T11:08:54.756 INFO:tasks.workunit.client.0.smithi117.stdout:0/68: link d3/d7/d8/f13 d3/d7/f18 0 2024-04-20T11:08:54.761 INFO:tasks.workunit.client.0.smithi117.stdout:6/144: chown d3/d4/l20 214 1 2024-04-20T11:08:54.763 INFO:tasks.workunit.client.0.smithi117.stdout:4/85: creat d1/dd/d1a/f1e x:0 0 0 2024-04-20T11:08:54.764 INFO:tasks.workunit.client.0.smithi117.stdout:3/92: creat d0/d5/d7/f1e x:0 0 0 2024-04-20T11:08:54.768 INFO:tasks.workunit.client.0.smithi117.stdout:7/96: mkdir d1/d4/d1c/d1f 0 2024-04-20T11:08:54.770 INFO:tasks.workunit.client.0.smithi117.stdout:9/153: creat d4/d35/f3d x:0 0 0 2024-04-20T11:08:54.772 INFO:tasks.workunit.client.0.smithi117.stdout:4/86: truncate d1/f19 5101524 0 2024-04-20T11:08:54.776 INFO:tasks.workunit.client.0.smithi117.stdout:1/53: dwrite d6/f9 [0,4194304] 0 2024-04-20T11:08:54.778 INFO:tasks.workunit.client.0.smithi117.stdout:6/145: symlink d3/l32 0 2024-04-20T11:08:54.781 INFO:tasks.workunit.client.0.smithi117.stdout:2/97: write d1/f3 [3127012,99279] 0 2024-04-20T11:08:54.782 INFO:tasks.workunit.client.0.smithi117.stdout:0/69: creat d3/d7/f19 x:0 0 0 2024-04-20T11:08:54.789 INFO:tasks.workunit.client.0.smithi117.stdout:8/66: dwrite d0/f3 [0,4194304] 0 2024-04-20T11:08:54.794 INFO:tasks.workunit.client.0.smithi117.stdout:7/97: symlink d1/l20 0 2024-04-20T11:08:54.795 INFO:tasks.workunit.client.0.smithi117.stdout:2/98: dread d1/d4/fb [0,4194304] 0 2024-04-20T11:08:54.798 INFO:tasks.workunit.client.0.smithi117.stdout:4/87: getdents d1/dd/d1c 0 2024-04-20T11:08:54.800 INFO:tasks.workunit.client.0.smithi117.stdout:2/99: write d1/d4/f8 [240570,103210] 0 2024-04-20T11:08:54.802 INFO:tasks.workunit.client.0.smithi117.stdout:9/154: creat d4/de/f3e x:0 0 0 2024-04-20T11:08:54.805 INFO:tasks.workunit.client.0.smithi117.stdout:3/93: write d0/f1a [507801,103783] 0 2024-04-20T11:08:54.806 INFO:tasks.workunit.client.0.smithi117.stdout:2/100: write d1/f11 [2895656,59324] 0 2024-04-20T11:08:54.811 INFO:tasks.workunit.client.0.smithi117.stdout:2/101: dread d1/d4/fb [0,4194304] 0 2024-04-20T11:08:54.812 INFO:tasks.workunit.client.0.smithi117.stdout:3/94: dread d0/f3 [0,4194304] 0 2024-04-20T11:08:54.812 INFO:tasks.workunit.client.0.smithi117.stdout:0/70: write d3/d7/f9 [1351900,73573] 0 2024-04-20T11:08:54.814 INFO:tasks.workunit.client.0.smithi117.stdout:6/146: link d3/df/l1a d3/df/l33 0 2024-04-20T11:08:54.815 INFO:tasks.workunit.client.0.smithi117.stdout:9/155: symlink d4/d5/da/l3f 0 2024-04-20T11:08:54.820 INFO:tasks.workunit.client.0.smithi117.stdout:7/98: dread d1/f7 [0,4194304] 0 2024-04-20T11:08:54.820 INFO:tasks.workunit.client.0.smithi117.stdout:6/147: write d3/f28 [206823,16846] 0 2024-04-20T11:08:54.821 INFO:tasks.workunit.client.0.smithi117.stdout:7/99: readlink d1/l20 0 2024-04-20T11:08:54.824 INFO:tasks.workunit.client.0.smithi117.stdout:7/100: dread - d1/d4/f1a zero size 2024-04-20T11:08:54.834 INFO:tasks.workunit.client.0.smithi117.stdout:4/88: write d1/f7 [1180083,23789] 0 2024-04-20T11:08:54.835 INFO:tasks.workunit.client.0.smithi117.stdout:1/54: link f4 d6/fd 0 2024-04-20T11:08:54.837 INFO:tasks.workunit.client.0.smithi117.stdout:1/55: stat d6/f9 0 2024-04-20T11:08:54.840 INFO:tasks.workunit.client.0.smithi117.stdout:6/148: rename d3/df/l1a to d3/d4/l34 0 2024-04-20T11:08:54.840 INFO:tasks.workunit.client.0.smithi117.stdout:0/71: link d3/d7/le d3/d17/l1a 0 2024-04-20T11:08:54.844 INFO:tasks.workunit.client.0.smithi117.stdout:2/102: truncate d1/f6 36953 0 2024-04-20T11:08:54.844 INFO:tasks.workunit.client.0.smithi117.stdout:1/56: chown f2 940 1 2024-04-20T11:08:54.847 INFO:tasks.workunit.client.0.smithi117.stdout:3/95: dwrite d0/d5/f19 [0,4194304] 0 2024-04-20T11:08:54.855 INFO:tasks.workunit.client.0.smithi117.stdout:9/156: dwrite d4/d30/f36 [0,4194304] 0 2024-04-20T11:08:54.860 INFO:tasks.workunit.client.0.smithi117.stdout:4/89: truncate d1/f2 333765 0 2024-04-20T11:08:54.860 INFO:tasks.workunit.client.0.smithi117.stdout:2/103: chown d1/d4/dd/d17/f1f 1064639781 1 2024-04-20T11:08:54.868 INFO:tasks.workunit.client.0.smithi117.stdout:7/101: dwrite d1/f18 [0,4194304] 0 2024-04-20T11:08:54.872 INFO:tasks.workunit.client.0.smithi117.stdout:0/72: dread d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:54.874 INFO:tasks.workunit.client.0.smithi117.stdout:7/102: rmdir d1/d4/d1c 39 2024-04-20T11:08:54.877 INFO:tasks.workunit.client.0.smithi117.stdout:6/149: truncate d3/fc 2012029 0 2024-04-20T11:08:54.880 INFO:tasks.workunit.client.0.smithi117.stdout:1/57: dread f2 [0,4194304] 0 2024-04-20T11:08:54.880 INFO:tasks.workunit.client.0.smithi117.stdout:4/90: write d1/f6 [2699336,111380] 0 2024-04-20T11:08:54.882 INFO:tasks.workunit.client.0.smithi117.stdout:2/104: getdents d1/d15 0 2024-04-20T11:08:54.884 INFO:tasks.workunit.client.0.smithi117.stdout:3/96: dwrite d0/d5/fc [0,4194304] 0 2024-04-20T11:08:54.885 INFO:tasks.workunit.client.0.smithi117.stdout:7/103: creat d1/d1d/f21 x:0 0 0 2024-04-20T11:08:54.889 INFO:tasks.workunit.client.0.smithi117.stdout:6/150: mkdir d3/d4/d2e/d35 0 2024-04-20T11:08:54.889 INFO:tasks.workunit.client.0.smithi117.stdout:4/91: mkdir d1/d9/d1f 0 2024-04-20T11:08:54.891 INFO:tasks.workunit.client.0.smithi117.stdout:2/105: creat d1/d4/dd/de/f23 x:0 0 0 2024-04-20T11:08:54.892 INFO:tasks.workunit.client.0.smithi117.stdout:4/92: dread - d1/d9/f12 zero size 2024-04-20T11:08:54.898 INFO:tasks.workunit.client.0.smithi117.stdout:1/58: truncate d6/fd 1647773 0 2024-04-20T11:08:54.898 INFO:tasks.workunit.client.0.smithi117.stdout:0/73: getdents d3/d7 0 2024-04-20T11:08:54.898 INFO:tasks.workunit.client.0.smithi117.stdout:0/74: dread - d3/d7/d8/df/f12 zero size 2024-04-20T11:08:54.899 INFO:tasks.workunit.client.0.smithi117.stdout:4/93: creat d1/f20 x:0 0 0 2024-04-20T11:08:54.900 INFO:tasks.workunit.client.0.smithi117.stdout:6/151: fdatasync d3/fc 0 2024-04-20T11:08:54.904 INFO:tasks.workunit.client.0.smithi117.stdout:6/152: write d3/df/f31 [924528,113938] 0 2024-04-20T11:08:54.904 INFO:tasks.workunit.client.0.smithi117.stdout:0/75: creat d3/d7/d8/df/f1b x:0 0 0 2024-04-20T11:08:54.912 INFO:tasks.workunit.client.0.smithi117.stdout:0/76: mkdir d3/d17/d1c 0 2024-04-20T11:08:54.914 INFO:tasks.workunit.client.0.smithi117.stdout:4/94: dread d1/dd/d1a/f1d [0,4194304] 0 2024-04-20T11:08:54.915 INFO:tasks.workunit.client.0.smithi117.stdout:4/95: chown d1/d9/f16 1471447846 1 2024-04-20T11:08:54.917 INFO:tasks.workunit.client.0.smithi117.stdout:0/77: symlink d3/l1d 0 2024-04-20T11:08:54.921 INFO:tasks.workunit.client.0.smithi117.stdout:0/78: dread - d3/d7/f14 zero size 2024-04-20T11:08:54.922 INFO:tasks.workunit.client.0.smithi117.stdout:4/96: dread d1/f7 [0,4194304] 0 2024-04-20T11:08:54.923 INFO:tasks.workunit.client.0.smithi117.stdout:6/153: write d3/df/f14 [4317869,68382] 0 2024-04-20T11:08:54.923 INFO:tasks.workunit.client.0.smithi117.stdout:2/106: write d1/f2 [829986,81509] 0 2024-04-20T11:08:54.925 INFO:tasks.workunit.client.0.smithi117.stdout:1/59: write d6/fd [1884191,107320] 0 2024-04-20T11:08:54.930 INFO:tasks.workunit.client.0.smithi117.stdout:2/107: unlink d1/d15/c1c 0 2024-04-20T11:08:54.933 INFO:tasks.workunit.client.0.smithi117.stdout:4/97: mknod d1/d9/c21 0 2024-04-20T11:08:54.935 INFO:tasks.workunit.client.0.smithi117.stdout:7/104: dwrite d1/f10 [0,4194304] 0 2024-04-20T11:08:54.936 INFO:tasks.workunit.client.0.smithi117.stdout:1/60: creat d6/fe x:0 0 0 2024-04-20T11:08:54.939 INFO:tasks.workunit.client.0.smithi117.stdout:4/98: symlink d1/d9/l22 0 2024-04-20T11:08:54.946 INFO:tasks.workunit.client.0.smithi117.stdout:6/154: dread d3/f2c [0,4194304] 0 2024-04-20T11:08:54.950 INFO:tasks.workunit.client.0.smithi117.stdout:6/155: write d3/df/f14 [4717792,3196] 0 2024-04-20T11:08:54.951 INFO:tasks.workunit.client.0.smithi117.stdout:0/79: link d3/d5/c15 d3/d17/c1e 0 2024-04-20T11:08:54.954 INFO:tasks.workunit.client.0.smithi117.stdout:1/61: rename c5 to d6/cf 0 2024-04-20T11:08:54.954 INFO:tasks.workunit.client.0.smithi117.stdout:6/156: write d3/df/f21 [687791,33067] 0 2024-04-20T11:08:54.957 INFO:tasks.workunit.client.0.smithi117.stdout:1/62: fsync f4 0 2024-04-20T11:08:54.958 INFO:tasks.workunit.client.0.smithi117.stdout:2/108: rmdir d1/d4/dd/d16 0 2024-04-20T11:08:54.959 INFO:tasks.workunit.client.0.smithi117.stdout:6/157: write d3/fc [412186,96656] 0 2024-04-20T11:08:54.964 INFO:tasks.workunit.client.0.smithi117.stdout:4/99: dwrite d1/f20 [0,4194304] 0 2024-04-20T11:08:54.968 INFO:tasks.workunit.client.0.smithi117.stdout:2/109: mkdir d1/d4/dd/d17/d24 0 2024-04-20T11:08:54.972 INFO:tasks.workunit.client.0.smithi117.stdout:6/158: symlink d3/d4/l36 0 2024-04-20T11:08:54.979 INFO:tasks.workunit.client.0.smithi117.stdout:6/159: write d3/d4/f2f [92607,56122] 0 2024-04-20T11:08:54.981 INFO:tasks.workunit.client.0.smithi117.stdout:2/110: getdents d1/d15/d1d 0 2024-04-20T11:08:54.985 INFO:tasks.workunit.client.0.smithi117.stdout:1/63: dread d6/fd [0,4194304] 0 2024-04-20T11:08:54.986 INFO:tasks.workunit.client.0.smithi117.stdout:4/100: dread d1/f6 [0,4194304] 0 2024-04-20T11:08:54.990 INFO:tasks.workunit.client.0.smithi117.stdout:6/160: stat d3/d4/l34 0 2024-04-20T11:08:54.990 INFO:tasks.workunit.client.0.smithi117.stdout:4/101: chown d1/d9 1400188 1 2024-04-20T11:08:54.993 INFO:tasks.workunit.client.0.smithi117.stdout:7/105: dwrite d1/d4/d6/fd [0,4194304] 0 2024-04-20T11:08:54.993 INFO:tasks.workunit.client.0.smithi117.stdout:4/102: chown d1/dd/d1c 260194 1 2024-04-20T11:08:54.993 INFO:tasks.workunit.client.0.smithi117.stdout:2/111: truncate d1/d20/f21 993329 0 2024-04-20T11:08:54.994 INFO:tasks.workunit.client.0.smithi117.stdout:4/103: rename d1 to d1/d23 22 2024-04-20T11:08:54.996 INFO:tasks.workunit.client.0.smithi117.stdout:0/80: dwrite d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:54.999 INFO:tasks.workunit.client.0.smithi117.stdout:2/112: mkdir d1/d4/dd/d17/d25 0 2024-04-20T11:08:55.000 INFO:tasks.workunit.client.0.smithi117.stdout:4/104: mknod d1/d9/d1f/c24 0 2024-04-20T11:08:55.004 INFO:tasks.workunit.client.0.smithi117.stdout:2/113: fsync d1/f11 0 2024-04-20T11:08:55.006 INFO:tasks.workunit.client.0.smithi117.stdout:0/81: mknod d3/d5/c1f 0 2024-04-20T11:08:55.009 INFO:tasks.workunit.client.0.smithi117.stdout:4/105: creat d1/f25 x:0 0 0 2024-04-20T11:08:55.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/106: rename d1/fc to d1/f22 0 2024-04-20T11:08:55.011 INFO:tasks.workunit.client.0.smithi117.stdout:0/82: mkdir d3/d7/d20 0 2024-04-20T11:08:55.014 INFO:tasks.workunit.client.0.smithi117.stdout:7/107: readlink d1/d4/d6/lf 0 2024-04-20T11:08:55.020 INFO:tasks.workunit.client.0.smithi117.stdout:4/106: write d1/dd/d1a/f1e [796057,61883] 0 2024-04-20T11:08:55.024 INFO:tasks.workunit.client.0.smithi117.stdout:7/108: mknod d1/d4/d6/c23 0 2024-04-20T11:08:55.025 INFO:tasks.workunit.client.0.smithi117.stdout:6/161: dwrite d3/d4/f12 [0,4194304] 0 2024-04-20T11:08:55.028 INFO:tasks.workunit.client.0.smithi117.stdout:1/64: dwrite d6/fd [0,4194304] 0 2024-04-20T11:08:55.035 INFO:tasks.workunit.client.0.smithi117.stdout:6/162: mkdir d3/d4/d2e/d37 0 2024-04-20T11:08:55.037 INFO:tasks.workunit.client.0.smithi117.stdout:1/65: rmdir d6 39 2024-04-20T11:08:55.043 INFO:tasks.workunit.client.0.smithi117.stdout:6/163: mknod d3/c38 0 2024-04-20T11:08:55.045 INFO:tasks.workunit.client.0.smithi117.stdout:7/109: getdents d1/d4/d6/d17 0 2024-04-20T11:08:55.047 INFO:tasks.workunit.client.0.smithi117.stdout:4/107: truncate d1/f6 1607061 0 2024-04-20T11:08:55.050 INFO:tasks.workunit.client.0.smithi117.stdout:4/108: truncate d1/f15 805497 0 2024-04-20T11:08:55.051 INFO:tasks.workunit.client.0.smithi117.stdout:6/164: mkdir d3/df/d39 0 2024-04-20T11:08:55.052 INFO:tasks.workunit.client.0.smithi117.stdout:7/110: unlink d1/f18 0 2024-04-20T11:08:55.052 INFO:tasks.workunit.client.0.smithi117.stdout:4/109: chown d1/f19 7 1 2024-04-20T11:08:55.055 INFO:tasks.workunit.client.0.smithi117.stdout:5/133: sync 2024-04-20T11:08:55.055 INFO:tasks.workunit.client.0.smithi117.stdout:7/111: chown d1/f7 157 1 2024-04-20T11:08:55.055 INFO:tasks.workunit.client.0.smithi117.stdout:4/110: read - d1/f25 zero size 2024-04-20T11:08:55.055 INFO:tasks.workunit.client.0.smithi117.stdout:6/165: rmdir d3/d4 39 2024-04-20T11:08:55.057 INFO:tasks.workunit.client.0.smithi117.stdout:1/66: getdents d6 0 2024-04-20T11:08:55.058 INFO:tasks.workunit.client.0.smithi117.stdout:6/166: chown d3/f30 3795 1 2024-04-20T11:08:55.059 INFO:tasks.workunit.client.0.smithi117.stdout:0/83: dwrite d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:55.061 INFO:tasks.workunit.client.0.smithi117.stdout:7/112: write d1/f22 [1568313,103221] 0 2024-04-20T11:08:55.064 INFO:tasks.workunit.client.0.smithi117.stdout:1/67: unlink d6/fe 0 2024-04-20T11:08:55.065 INFO:tasks.workunit.client.0.smithi117.stdout:4/111: write d1/f7 [788412,126514] 0 2024-04-20T11:08:55.072 INFO:tasks.workunit.client.0.smithi117.stdout:0/84: creat d3/f21 x:0 0 0 2024-04-20T11:08:55.076 INFO:tasks.workunit.client.0.smithi117.stdout:5/134: creat de/f28 x:0 0 0 2024-04-20T11:08:55.078 INFO:tasks.workunit.client.0.smithi117.stdout:5/135: write de/d16/f1b [3149205,77009] 0 2024-04-20T11:08:55.079 INFO:tasks.workunit.client.0.smithi117.stdout:4/112: mknod d1/c26 0 2024-04-20T11:08:55.080 INFO:tasks.workunit.client.0.smithi117.stdout:1/68: dread d6/f9 [0,4194304] 0 2024-04-20T11:08:55.080 INFO:tasks.workunit.client.0.smithi117.stdout:5/136: fsync de/f18 0 2024-04-20T11:08:55.080 INFO:tasks.workunit.client.0.smithi117.stdout:6/167: fdatasync d3/f18 0 2024-04-20T11:08:55.083 INFO:tasks.workunit.client.0.smithi117.stdout:0/85: dread - d3/d7/d8/df/f1b zero size 2024-04-20T11:08:55.088 INFO:tasks.workunit.client.0.smithi117.stdout:7/113: link d1/l20 d1/l24 0 2024-04-20T11:08:55.094 INFO:tasks.workunit.client.0.smithi117.stdout:6/168: mkdir d3/d3a 0 2024-04-20T11:08:55.097 INFO:tasks.workunit.client.0.smithi117.stdout:4/113: rmdir d1/dd/d1c 0 2024-04-20T11:08:55.099 INFO:tasks.workunit.client.0.smithi117.stdout:4/114: stat d1 0 2024-04-20T11:08:55.101 INFO:tasks.workunit.client.0.smithi117.stdout:6/169: symlink d3/df/l3b 0 2024-04-20T11:08:55.105 INFO:tasks.workunit.client.0.smithi117.stdout:1/69: dwrite f2 [0,4194304] 0 2024-04-20T11:08:55.105 INFO:tasks.workunit.client.0.smithi117.stdout:4/115: rename d1/c26 to d1/c27 0 2024-04-20T11:08:55.106 INFO:tasks.workunit.client.0.smithi117.stdout:0/86: dread d3/d7/f18 [0,4194304] 0 2024-04-20T11:08:55.109 INFO:tasks.workunit.client.0.smithi117.stdout:1/70: rmdir d6 39 2024-04-20T11:08:55.112 INFO:tasks.workunit.client.0.smithi117.stdout:4/116: mknod d1/dd/d1a/c28 0 2024-04-20T11:08:55.113 INFO:tasks.workunit.client.0.smithi117.stdout:4/117: dread - d1/f25 zero size 2024-04-20T11:08:55.114 INFO:tasks.workunit.client.0.smithi117.stdout:0/87: rename d3/d17 to d3/d7/d8/df/d22 0 2024-04-20T11:08:55.119 INFO:tasks.workunit.client.0.smithi117.stdout:7/114: getdents d1/d4/d1c 0 2024-04-20T11:08:55.120 INFO:tasks.workunit.client.0.smithi117.stdout:6/170: getdents d3/d4/d2e/d35 0 2024-04-20T11:08:55.121 INFO:tasks.workunit.client.0.smithi117.stdout:7/115: chown d1/d4/d6 2106125107 1 2024-04-20T11:08:55.121 INFO:tasks.workunit.client.0.smithi117.stdout:6/171: chown d3/d4/d2e 5 1 2024-04-20T11:08:55.122 INFO:tasks.workunit.client.0.smithi117.stdout:1/71: dread f4 [0,4194304] 0 2024-04-20T11:08:55.122 INFO:tasks.workunit.client.0.smithi117.stdout:0/88: unlink l0 0 2024-04-20T11:08:55.124 INFO:tasks.workunit.client.0.smithi117.stdout:4/118: mknod d1/c29 0 2024-04-20T11:08:55.124 INFO:tasks.workunit.client.0.smithi117.stdout:0/89: chown d3/d5/c1f 122892518 1 2024-04-20T11:08:55.129 INFO:tasks.workunit.client.0.smithi117.stdout:4/119: creat d1/d9/f2a x:0 0 0 2024-04-20T11:08:55.132 INFO:tasks.workunit.client.0.smithi117.stdout:6/172: symlink d3/d4/d2e/d35/l3c 0 2024-04-20T11:08:55.133 INFO:tasks.workunit.client.0.smithi117.stdout:1/72: creat d6/f10 x:0 0 0 2024-04-20T11:08:55.133 INFO:tasks.workunit.client.0.smithi117.stdout:0/90: getdents d3/d7/d20 0 2024-04-20T11:08:55.136 INFO:tasks.workunit.client.0.smithi117.stdout:1/73: symlink d6/l11 0 2024-04-20T11:08:55.138 INFO:tasks.workunit.client.0.smithi117.stdout:0/91: stat d3/d7/d16 0 2024-04-20T11:08:55.139 INFO:tasks.workunit.client.0.smithi117.stdout:1/74: dread - d6/f10 zero size 2024-04-20T11:08:55.142 INFO:tasks.workunit.client.0.smithi117.stdout:6/173: unlink d3/l9 0 2024-04-20T11:08:55.144 INFO:tasks.workunit.client.0.smithi117.stdout:7/116: dwrite d1/d4/d6/fd [0,4194304] 0 2024-04-20T11:08:55.150 INFO:tasks.workunit.client.0.smithi117.stdout:7/117: truncate d1/f7 5068871 0 2024-04-20T11:08:55.151 INFO:tasks.workunit.client.0.smithi117.stdout:4/120: dwrite d1/f19 [0,4194304] 0 2024-04-20T11:08:55.153 INFO:tasks.workunit.client.0.smithi117.stdout:0/92: mknod d3/d7/c23 0 2024-04-20T11:08:55.159 INFO:tasks.workunit.client.0.smithi117.stdout:6/174: dread d3/df/f31 [0,4194304] 0 2024-04-20T11:08:55.160 INFO:tasks.workunit.client.0.smithi117.stdout:7/118: mknod d1/d4/d6/d13/c25 0 2024-04-20T11:08:55.163 INFO:tasks.workunit.client.0.smithi117.stdout:1/75: dread f4 [0,4194304] 0 2024-04-20T11:08:55.163 INFO:tasks.workunit.client.0.smithi117.stdout:0/93: rmdir d3/d5 39 2024-04-20T11:08:55.165 INFO:tasks.workunit.client.0.smithi117.stdout:1/76: read - d6/f10 zero size 2024-04-20T11:08:55.165 INFO:tasks.workunit.client.0.smithi117.stdout:4/121: dread d1/dd/f13 [0,4194304] 0 2024-04-20T11:08:55.167 INFO:tasks.workunit.client.0.smithi117.stdout:7/119: creat d1/d4/d6/f26 x:0 0 0 2024-04-20T11:08:55.169 INFO:tasks.workunit.client.0.smithi117.stdout:0/94: chown d3/d7/d8/df/f12 0 1 2024-04-20T11:08:55.170 INFO:tasks.workunit.client.0.smithi117.stdout:1/77: rename f4 to d6/f12 0 2024-04-20T11:08:55.171 INFO:tasks.workunit.client.0.smithi117.stdout:1/78: read f2 [3733162,86370] 0 2024-04-20T11:08:55.171 INFO:tasks.workunit.client.0.smithi117.stdout:4/122: mkdir d1/dd/d1a/d2b 0 2024-04-20T11:08:55.172 INFO:tasks.workunit.client.0.smithi117.stdout:7/120: mknod d1/d4/d1c/c27 0 2024-04-20T11:08:55.173 INFO:tasks.workunit.client.0.smithi117.stdout:4/123: dread - d1/d9/f2a zero size 2024-04-20T11:08:55.175 INFO:tasks.workunit.client.0.smithi117.stdout:1/79: write d6/f12 [3236524,24150] 0 2024-04-20T11:08:55.179 INFO:tasks.workunit.client.0.smithi117.stdout:1/80: read d6/f12 [1418964,101080] 0 2024-04-20T11:08:55.182 INFO:tasks.workunit.client.0.smithi117.stdout:4/124: rename d1/d9/d1f/c24 to d1/d9/c2c 0 2024-04-20T11:08:55.183 INFO:tasks.workunit.client.0.smithi117.stdout:6/175: chown d3/d4/d2e/d35 145721356 1 2024-04-20T11:08:55.185 INFO:tasks.workunit.client.0.smithi117.stdout:4/125: symlink d1/l2d 0 2024-04-20T11:08:55.186 INFO:tasks.workunit.client.0.smithi117.stdout:6/176: chown d3/ld 197746 1 2024-04-20T11:08:55.189 INFO:tasks.workunit.client.0.smithi117.stdout:4/126: creat d1/dd/d1a/f2e x:0 0 0 2024-04-20T11:08:55.190 INFO:tasks.workunit.client.0.smithi117.stdout:6/177: readlink d3/l29 0 2024-04-20T11:08:55.195 INFO:tasks.workunit.client.0.smithi117.stdout:0/95: dread d3/d7/f18 [0,4194304] 0 2024-04-20T11:08:55.196 INFO:tasks.workunit.client.0.smithi117.stdout:4/127: read d1/dd/d1a/f1d [1200056,19577] 0 2024-04-20T11:08:55.196 INFO:tasks.workunit.client.0.smithi117.stdout:4/128: stat d1/d9/f12 0 2024-04-20T11:08:55.197 INFO:tasks.workunit.client.0.smithi117.stdout:1/81: dread d6/fd [0,4194304] 0 2024-04-20T11:08:55.200 INFO:tasks.workunit.client.0.smithi117.stdout:1/82: read d6/f12 [2069918,55061] 0 2024-04-20T11:08:55.201 INFO:tasks.workunit.client.0.smithi117.stdout:7/121: dwrite d1/d1d/f21 [0,4194304] 0 2024-04-20T11:08:55.204 INFO:tasks.workunit.client.0.smithi117.stdout:1/83: dread - d6/f10 zero size 2024-04-20T11:08:55.205 INFO:tasks.workunit.client.0.smithi117.stdout:4/129: rename d1/dd/lf to d1/dd/d1a/l2f 0 2024-04-20T11:08:55.206 INFO:tasks.workunit.client.0.smithi117.stdout:1/84: creat d6/f13 x:0 0 0 2024-04-20T11:08:55.209 INFO:tasks.workunit.client.0.smithi117.stdout:7/122: creat d1/d4/d6/d13/f28 x:0 0 0 2024-04-20T11:08:55.210 INFO:tasks.workunit.client.0.smithi117.stdout:4/130: stat d1/d9/c11 0 2024-04-20T11:08:55.212 INFO:tasks.workunit.client.0.smithi117.stdout:0/96: dread d3/d7/f18 [0,4194304] 0 2024-04-20T11:08:55.212 INFO:tasks.workunit.client.0.smithi117.stdout:1/85: fdatasync f2 0 2024-04-20T11:08:55.215 INFO:tasks.workunit.client.0.smithi117.stdout:4/131: creat d1/f30 x:0 0 0 2024-04-20T11:08:55.215 INFO:tasks.workunit.client.0.smithi117.stdout:6/178: write d3/f18 [244743,54797] 0 2024-04-20T11:08:55.218 INFO:tasks.workunit.client.0.smithi117.stdout:4/132: stat d1/d9/c2c 0 2024-04-20T11:08:55.221 INFO:tasks.workunit.client.0.smithi117.stdout:7/123: rmdir d1/d4/d6/d17/d1e 0 2024-04-20T11:08:55.224 INFO:tasks.workunit.client.0.smithi117.stdout:6/179: creat d3/d3a/f3d x:0 0 0 2024-04-20T11:08:55.226 INFO:tasks.workunit.client.0.smithi117.stdout:1/86: dread d6/f12 [0,4194304] 0 2024-04-20T11:08:55.227 INFO:tasks.workunit.client.0.smithi117.stdout:4/133: truncate d1/f7 977571 0 2024-04-20T11:08:55.229 INFO:tasks.workunit.client.0.smithi117.stdout:1/87: chown d6/f13 32110532 1 2024-04-20T11:08:55.229 INFO:tasks.workunit.client.0.smithi117.stdout:4/134: chown d1/d9/l22 49546227 1 2024-04-20T11:08:55.233 INFO:tasks.workunit.client.0.smithi117.stdout:0/97: dread d3/d5/fd [0,4194304] 0 2024-04-20T11:08:55.236 INFO:tasks.workunit.client.0.smithi117.stdout:1/88: link d6/l11 d6/l14 0 2024-04-20T11:08:55.237 INFO:tasks.workunit.client.0.smithi117.stdout:6/180: fsync d3/d3a/f3d 0 2024-04-20T11:08:55.237 INFO:tasks.workunit.client.0.smithi117.stdout:1/89: readlink d6/lc 0 2024-04-20T11:08:55.241 INFO:tasks.workunit.client.0.smithi117.stdout:7/124: truncate d1/f12 2516764 0 2024-04-20T11:08:55.241 INFO:tasks.workunit.client.0.smithi117.stdout:1/90: symlink d6/l15 0 2024-04-20T11:08:55.242 INFO:tasks.workunit.client.0.smithi117.stdout:1/91: chown f2 83383074 1 2024-04-20T11:08:55.243 INFO:tasks.workunit.client.0.smithi117.stdout:1/92: chown d6/lb 1 1 2024-04-20T11:08:55.243 INFO:tasks.workunit.client.0.smithi117.stdout:4/135: write d1/dd/f13 [3064392,114722] 0 2024-04-20T11:08:55.244 INFO:tasks.workunit.client.0.smithi117.stdout:4/136: readlink d1/l3 0 2024-04-20T11:08:55.246 INFO:tasks.workunit.client.0.smithi117.stdout:7/125: readlink d1/l20 0 2024-04-20T11:08:55.246 INFO:tasks.workunit.client.0.smithi117.stdout:0/98: link d3/d7/c23 d3/d7/d8/df/c24 0 2024-04-20T11:08:55.248 INFO:tasks.workunit.client.0.smithi117.stdout:6/181: creat d3/df/d39/f3e x:0 0 0 2024-04-20T11:08:55.248 INFO:tasks.workunit.client.0.smithi117.stdout:1/93: mkdir d6/d16 0 2024-04-20T11:08:55.250 INFO:tasks.workunit.client.0.smithi117.stdout:4/137: read d1/f6 [104184,111103] 0 2024-04-20T11:08:55.250 INFO:tasks.workunit.client.0.smithi117.stdout:0/99: truncate d3/d7/d8/df/f12 331988 0 2024-04-20T11:08:55.250 INFO:tasks.workunit.client.0.smithi117.stdout:7/126: fsync d1/fb 0 2024-04-20T11:08:55.252 INFO:tasks.workunit.client.0.smithi117.stdout:7/127: stat d1/d1d 0 2024-04-20T11:08:55.252 INFO:tasks.workunit.client.0.smithi117.stdout:6/182: write d3/d4/fe [9969,6745] 0 2024-04-20T11:08:55.253 INFO:tasks.workunit.client.0.smithi117.stdout:1/94: mknod d6/d16/c17 0 2024-04-20T11:08:55.254 INFO:tasks.workunit.client.0.smithi117.stdout:1/95: dread - d6/f10 zero size 2024-04-20T11:08:55.257 INFO:tasks.workunit.client.0.smithi117.stdout:4/138: creat d1/dd/f31 x:0 0 0 2024-04-20T11:08:55.258 INFO:tasks.workunit.client.0.smithi117.stdout:7/128: mknod d1/d1d/c29 0 2024-04-20T11:08:55.259 INFO:tasks.workunit.client.0.smithi117.stdout:6/183: mkdir d3/d4/d2e/d3f 0 2024-04-20T11:08:55.262 INFO:tasks.workunit.client.0.smithi117.stdout:0/100: creat d3/d7/d20/f25 x:0 0 0 2024-04-20T11:08:55.268 INFO:tasks.workunit.client.0.smithi117.stdout:4/139: dread d1/f2 [0,4194304] 0 2024-04-20T11:08:55.275 INFO:tasks.workunit.client.0.smithi117.stdout:4/140: unlink d1/dd/d1a/f1b 0 2024-04-20T11:08:55.280 INFO:tasks.workunit.client.0.smithi117.stdout:6/184: dwrite d3/df/f14 [4194304,4194304] 0 2024-04-20T11:08:55.281 INFO:tasks.workunit.client.0.smithi117.stdout:0/101: dread d3/d7/d8/f13 [0,4194304] 0 2024-04-20T11:08:55.284 INFO:tasks.workunit.client.0.smithi117.stdout:6/185: write d3/df/f21 [374007,18860] 0 2024-04-20T11:08:55.286 INFO:tasks.workunit.client.0.smithi117.stdout:0/102: read - d3/d7/d20/f25 zero size 2024-04-20T11:08:55.289 INFO:tasks.workunit.client.0.smithi117.stdout:1/96: dwrite f2 [0,4194304] 0 2024-04-20T11:08:55.290 INFO:tasks.workunit.client.0.smithi117.stdout:4/141: write d1/dd/d1a/f1d [1497020,20277] 0 2024-04-20T11:08:55.294 INFO:tasks.workunit.client.0.smithi117.stdout:7/129: dread d1/f12 [0,4194304] 0 2024-04-20T11:08:55.294 INFO:tasks.workunit.client.0.smithi117.stdout:6/186: creat d3/d4/d2e/f40 x:0 0 0 2024-04-20T11:08:55.295 INFO:tasks.workunit.client.0.smithi117.stdout:1/97: creat d6/f18 x:0 0 0 2024-04-20T11:08:55.296 INFO:tasks.workunit.client.0.smithi117.stdout:3/97: sync 2024-04-20T11:08:55.296 INFO:tasks.workunit.client.0.smithi117.stdout:8/67: sync 2024-04-20T11:08:55.296 INFO:tasks.workunit.client.0.smithi117.stdout:9/157: sync 2024-04-20T11:08:55.300 INFO:tasks.workunit.client.0.smithi117.stdout:9/158: fdatasync d4/de/f2c 0 2024-04-20T11:08:55.302 INFO:tasks.workunit.client.0.smithi117.stdout:8/68: read d0/f7 [3741942,87281] 0 2024-04-20T11:08:55.302 INFO:tasks.workunit.client.0.smithi117.stdout:1/98: write d6/f13 [295743,22321] 0 2024-04-20T11:08:55.303 INFO:tasks.workunit.client.0.smithi117.stdout:7/130: creat d1/d1d/f2a x:0 0 0 2024-04-20T11:08:55.308 INFO:tasks.workunit.client.0.smithi117.stdout:1/99: symlink d6/l19 0 2024-04-20T11:08:55.309 INFO:tasks.workunit.client.0.smithi117.stdout:3/98: dread d0/f3 [0,4194304] 0 2024-04-20T11:08:55.315 INFO:tasks.workunit.client.0.smithi117.stdout:9/159: rmdir d4/d35 39 2024-04-20T11:08:55.318 INFO:tasks.workunit.client.0.smithi117.stdout:1/100: fdatasync f2 0 2024-04-20T11:08:55.318 INFO:tasks.workunit.client.0.smithi117.stdout:4/142: dread d1/dd/f13 [0,4194304] 0 2024-04-20T11:08:55.319 INFO:tasks.workunit.client.0.smithi117.stdout:0/103: dwrite d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:55.319 INFO:tasks.workunit.client.0.smithi117.stdout:7/131: unlink d1/d4/d6/fd 0 2024-04-20T11:08:55.320 INFO:tasks.workunit.client.0.smithi117.stdout:6/187: dread d3/d4/f12 [0,4194304] 0 2024-04-20T11:08:55.324 INFO:tasks.workunit.client.0.smithi117.stdout:6/188: write d3/f18 [993001,28283] 0 2024-04-20T11:08:55.324 INFO:tasks.workunit.client.0.smithi117.stdout:1/101: creat d6/f1a x:0 0 0 2024-04-20T11:08:55.327 INFO:tasks.workunit.client.0.smithi117.stdout:1/102: chown d6/d16/c17 1984 1 2024-04-20T11:08:55.328 INFO:tasks.workunit.client.0.smithi117.stdout:0/104: creat d3/d5/f26 x:0 0 0 2024-04-20T11:08:55.328 INFO:tasks.workunit.client.0.smithi117.stdout:7/132: mkdir d1/d4/d6/d2b 0 2024-04-20T11:08:55.331 INFO:tasks.workunit.client.0.smithi117.stdout:0/105: chown d3/d7/d8/df/d22 0 1 2024-04-20T11:08:55.333 INFO:tasks.workunit.client.0.smithi117.stdout:7/133: write d1/d1d/f21 [2229378,125483] 0 2024-04-20T11:08:55.338 INFO:tasks.workunit.client.0.smithi117.stdout:1/103: rename d6/cf to d6/d16/c1b 0 2024-04-20T11:08:55.341 INFO:tasks.workunit.client.0.smithi117.stdout:8/69: dwrite d0/f7 [0,4194304] 0 2024-04-20T11:08:55.346 INFO:tasks.workunit.client.0.smithi117.stdout:4/143: getdents d1/d9 0 2024-04-20T11:08:55.347 INFO:tasks.workunit.client.0.smithi117.stdout:2/114: sync 2024-04-20T11:08:55.347 INFO:tasks.workunit.client.0.smithi117.stdout:5/137: sync 2024-04-20T11:08:55.350 INFO:tasks.workunit.client.0.smithi117.stdout:7/134: unlink d1/d4/d6/lf 0 2024-04-20T11:08:55.352 INFO:tasks.workunit.client.0.smithi117.stdout:1/104: symlink d6/l1c 0 2024-04-20T11:08:55.353 INFO:tasks.workunit.client.0.smithi117.stdout:8/70: creat d0/d8/f16 x:0 0 0 2024-04-20T11:08:55.355 INFO:tasks.workunit.client.0.smithi117.stdout:8/71: write d0/fa [980054,4615] 0 2024-04-20T11:08:55.356 INFO:tasks.workunit.client.0.smithi117.stdout:4/144: mknod d1/dd/d1a/c32 0 2024-04-20T11:08:55.357 INFO:tasks.workunit.client.0.smithi117.stdout:5/138: write f5 [730269,77608] 0 2024-04-20T11:08:55.362 INFO:tasks.workunit.client.0.smithi117.stdout:4/145: read d1/f10 [1523927,80465] 0 2024-04-20T11:08:55.362 INFO:tasks.workunit.client.0.smithi117.stdout:0/106: rename d3/d7/d8/df/c24 to d3/d7/d8/df/c27 0 2024-04-20T11:08:55.362 INFO:tasks.workunit.client.0.smithi117.stdout:6/189: link d3/d4/c23 d3/d4/d2e/d3f/c41 0 2024-04-20T11:08:55.362 INFO:tasks.workunit.client.0.smithi117.stdout:1/105: rename d6/fd to d6/d16/f1d 0 2024-04-20T11:08:55.364 INFO:tasks.workunit.client.0.smithi117.stdout:8/72: rmdir d0/d8/d11 39 2024-04-20T11:08:55.370 INFO:tasks.workunit.client.0.smithi117.stdout:2/115: truncate d1/f2 814953 0 2024-04-20T11:08:55.373 INFO:tasks.workunit.client.0.smithi117.stdout:4/146: stat d1/c27 0 2024-04-20T11:08:55.376 INFO:tasks.workunit.client.0.smithi117.stdout:1/106: symlink d6/d16/l1e 0 2024-04-20T11:08:55.383 INFO:tasks.workunit.client.0.smithi117.stdout:4/147: mknod d1/d9/d1f/c33 0 2024-04-20T11:08:55.384 INFO:tasks.workunit.client.0.smithi117.stdout:6/190: getdents d3/d4/d2e/d37 0 2024-04-20T11:08:55.385 INFO:tasks.workunit.client.0.smithi117.stdout:0/107: mknod d3/d7/d16/c28 0 2024-04-20T11:08:55.385 INFO:tasks.workunit.client.0.smithi117.stdout:5/139: creat de/f29 x:0 0 0 2024-04-20T11:08:55.385 INFO:tasks.workunit.client.0.smithi117.stdout:4/148: write d1/f30 [954541,31984] 0 2024-04-20T11:08:55.388 INFO:tasks.workunit.client.0.smithi117.stdout:8/73: link d0/fd d0/d9/f17 0 2024-04-20T11:08:55.388 INFO:tasks.workunit.client.0.smithi117.stdout:7/135: mknod d1/d4/d1c/d1f/c2c 0 2024-04-20T11:08:55.393 INFO:tasks.workunit.client.0.smithi117.stdout:6/191: write d3/df/f14 [6364834,115451] 0 2024-04-20T11:08:55.397 INFO:tasks.workunit.client.0.smithi117.stdout:4/149: mknod d1/d9/d1f/c34 0 2024-04-20T11:08:55.398 INFO:tasks.workunit.client.0.smithi117.stdout:1/107: dread d6/f12 [0,4194304] 0 2024-04-20T11:08:55.399 INFO:tasks.workunit.client.0.smithi117.stdout:5/140: symlink de/l2a 0 2024-04-20T11:08:55.402 INFO:tasks.workunit.client.0.smithi117.stdout:2/116: dread d1/d4/f5 [0,4194304] 0 2024-04-20T11:08:55.404 INFO:tasks.workunit.client.0.smithi117.stdout:6/192: creat d3/d4/d2e/d3f/f42 x:0 0 0 2024-04-20T11:08:55.404 INFO:tasks.workunit.client.0.smithi117.stdout:0/108: dread d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:55.405 INFO:tasks.workunit.client.0.smithi117.stdout:6/193: chown d3/d4/c22 3100 1 2024-04-20T11:08:55.407 INFO:tasks.workunit.client.0.smithi117.stdout:1/108: mkdir d6/d1f 0 2024-04-20T11:08:55.410 INFO:tasks.workunit.client.0.smithi117.stdout:5/141: mkdir de/d16/d2b 0 2024-04-20T11:08:55.412 INFO:tasks.workunit.client.0.smithi117.stdout:2/117: mkdir d1/d20/d26 0 2024-04-20T11:08:55.412 INFO:tasks.workunit.client.0.smithi117.stdout:0/109: mknod d3/d7/d8/c29 0 2024-04-20T11:08:55.413 INFO:tasks.workunit.client.0.smithi117.stdout:6/194: mknod d3/d4/d2e/d3f/c43 0 2024-04-20T11:08:55.414 INFO:tasks.workunit.client.0.smithi117.stdout:1/109: mknod d6/c20 0 2024-04-20T11:08:55.414 INFO:tasks.workunit.client.0.smithi117.stdout:5/142: rename de/d16 to de/d16/d24/d2c 22 2024-04-20T11:08:55.416 INFO:tasks.workunit.client.0.smithi117.stdout:7/136: getdents d1/d4 0 2024-04-20T11:08:55.417 INFO:tasks.workunit.client.0.smithi117.stdout:1/110: mkdir d6/d21 0 2024-04-20T11:08:55.420 INFO:tasks.workunit.client.0.smithi117.stdout:5/143: write f0 [1760047,33098] 0 2024-04-20T11:08:55.421 INFO:tasks.workunit.client.0.smithi117.stdout:4/150: dwrite d1/d9/f2a [0,4194304] 0 2024-04-20T11:08:55.425 INFO:tasks.workunit.client.0.smithi117.stdout:1/111: fsync d6/f9 0 2024-04-20T11:08:55.425 INFO:tasks.workunit.client.0.smithi117.stdout:6/195: write d3/f2c [84038,11508] 0 2024-04-20T11:08:55.426 INFO:tasks.workunit.client.0.smithi117.stdout:7/137: write d1/f10 [671024,6051] 0 2024-04-20T11:08:55.437 INFO:tasks.workunit.client.0.smithi117.stdout:4/151: rmdir d1/dd/d1a 39 2024-04-20T11:08:55.440 INFO:tasks.workunit.client.0.smithi117.stdout:6/196: stat d3/c38 0 2024-04-20T11:08:55.442 INFO:tasks.workunit.client.0.smithi117.stdout:0/110: rmdir d3/d7/d8/df/d22/d1c 0 2024-04-20T11:08:55.445 INFO:tasks.workunit.client.0.smithi117.stdout:7/138: mkdir d1/d4/d1c/d1f/d2d 0 2024-04-20T11:08:55.450 INFO:tasks.workunit.client.0.smithi117.stdout:4/152: creat d1/f35 x:0 0 0 2024-04-20T11:08:55.450 INFO:tasks.workunit.client.0.smithi117.stdout:1/112: mknod d6/d21/c22 0 2024-04-20T11:08:55.452 INFO:tasks.workunit.client.0.smithi117.stdout:4/153: dread - d1/d9/f12 zero size 2024-04-20T11:08:55.453 INFO:tasks.workunit.client.0.smithi117.stdout:6/197: unlink d3/d4/fe 0 2024-04-20T11:08:55.459 INFO:tasks.workunit.client.0.smithi117.stdout:5/144: rmdir de/d16/d2b 0 2024-04-20T11:08:55.460 INFO:tasks.workunit.client.0.smithi117.stdout:5/145: fdatasync de/f14 0 2024-04-20T11:08:55.461 INFO:tasks.workunit.client.0.smithi117.stdout:5/146: readlink de/l2a 0 2024-04-20T11:08:55.463 INFO:tasks.workunit.client.0.smithi117.stdout:1/113: creat d6/d21/f23 x:0 0 0 2024-04-20T11:08:55.465 INFO:tasks.workunit.client.0.smithi117.stdout:5/147: write de/d16/f1b [1217338,87882] 0 2024-04-20T11:08:55.467 INFO:tasks.workunit.client.0.smithi117.stdout:6/198: unlink d3/d4/f12 0 2024-04-20T11:08:55.470 INFO:tasks.workunit.client.0.smithi117.stdout:4/154: rename d1/dd/f31 to d1/d9/d1f/f36 0 2024-04-20T11:08:55.474 INFO:tasks.workunit.client.0.smithi117.stdout:1/114: unlink d6/l19 0 2024-04-20T11:08:55.479 INFO:tasks.workunit.client.0.smithi117.stdout:5/148: creat de/d16/f2d x:0 0 0 2024-04-20T11:08:55.484 INFO:tasks.workunit.client.0.smithi117.stdout:6/199: mknod d3/d4/d2e/d35/c44 0 2024-04-20T11:08:55.487 INFO:tasks.workunit.client.0.smithi117.stdout:1/115: rename d6/d16/f1d to d6/d1f/f24 0 2024-04-20T11:08:55.492 INFO:tasks.workunit.client.0.smithi117.stdout:0/111: getdents d3/d7/d16 0 2024-04-20T11:08:55.494 INFO:tasks.workunit.client.0.smithi117.stdout:5/149: creat de/d21/f2e x:0 0 0 2024-04-20T11:08:55.496 INFO:tasks.workunit.client.0.smithi117.stdout:5/150: fsync de/f14 0 2024-04-20T11:08:55.497 INFO:tasks.workunit.client.0.smithi117.stdout:4/155: creat d1/dd/d1a/f37 x:0 0 0 2024-04-20T11:08:55.497 INFO:tasks.workunit.client.0.smithi117.stdout:5/151: read - de/f29 zero size 2024-04-20T11:08:55.502 INFO:tasks.workunit.client.0.smithi117.stdout:7/139: write d1/fb [424757,12449] 0 2024-04-20T11:08:55.505 INFO:tasks.workunit.client.0.smithi117.stdout:4/156: read d1/dd/d1a/f1d [1439163,82342] 0 2024-04-20T11:08:55.506 INFO:tasks.workunit.client.0.smithi117.stdout:1/116: creat d6/f25 x:0 0 0 2024-04-20T11:08:55.508 INFO:tasks.workunit.client.0.smithi117.stdout:1/117: truncate d6/d21/f23 397219 0 2024-04-20T11:08:55.509 INFO:tasks.workunit.client.0.smithi117.stdout:0/112: mkdir d3/d7/d8/d2a 0 2024-04-20T11:08:55.513 INFO:tasks.workunit.client.0.smithi117.stdout:5/152: symlink de/d21/l2f 0 2024-04-20T11:08:55.514 INFO:tasks.workunit.client.0.smithi117.stdout:5/153: chown f0 1103 1 2024-04-20T11:08:55.515 INFO:tasks.workunit.client.0.smithi117.stdout:6/200: fdatasync d3/d4/f1b 0 2024-04-20T11:08:55.520 INFO:tasks.workunit.client.0.smithi117.stdout:6/201: dread - d3/df/d39/f3e zero size 2024-04-20T11:08:55.521 INFO:tasks.workunit.client.0.smithi117.stdout:7/140: unlink d1/d4/c14 0 2024-04-20T11:08:55.526 INFO:tasks.workunit.client.0.smithi117.stdout:0/113: dread - d3/d7/f14 zero size 2024-04-20T11:08:55.528 INFO:tasks.workunit.client.0.smithi117.stdout:1/118: dwrite d6/f10 [0,4194304] 0 2024-04-20T11:08:55.531 INFO:tasks.workunit.client.0.smithi117.stdout:1/119: write d6/f13 [338676,34108] 0 2024-04-20T11:08:55.547 INFO:tasks.workunit.client.0.smithi117.stdout:5/154: unlink de/l13 0 2024-04-20T11:08:55.551 INFO:tasks.workunit.client.0.smithi117.stdout:1/120: symlink d6/d1f/l26 0 2024-04-20T11:08:55.552 INFO:tasks.workunit.client.0.smithi117.stdout:1/121: dread - d6/f18 zero size 2024-04-20T11:08:55.554 INFO:tasks.workunit.client.0.smithi117.stdout:4/157: creat d1/dd/f38 x:0 0 0 2024-04-20T11:08:55.558 INFO:tasks.workunit.client.0.smithi117.stdout:1/122: mknod d6/d21/c27 0 2024-04-20T11:08:55.589 INFO:tasks.workunit.client.0.smithi117.stdout:4/158: creat d1/d9/d1f/f39 x:0 0 0 2024-04-20T11:08:55.589 INFO:tasks.workunit.client.0.smithi117.stdout:7/141: getdents d1 0 2024-04-20T11:08:55.590 INFO:tasks.workunit.client.0.smithi117.stdout:1/123: mknod d6/c28 0 2024-04-20T11:08:55.590 INFO:tasks.workunit.client.0.smithi117.stdout:6/202: truncate d3/df/f31 319952 0 2024-04-20T11:08:55.591 INFO:tasks.workunit.client.0.smithi117.stdout:4/159: chown d1/d9/c21 9791 1 2024-04-20T11:08:55.592 INFO:tasks.workunit.client.0.smithi117.stdout:5/155: rmdir de/d16/d24 0 2024-04-20T11:08:55.594 INFO:tasks.workunit.client.0.smithi117.stdout:4/160: write d1/f30 [1322720,104938] 0 2024-04-20T11:08:55.601 INFO:tasks.workunit.client.0.smithi117.stdout:1/124: symlink d6/d21/l29 0 2024-04-20T11:08:55.608 INFO:tasks.workunit.client.0.smithi117.stdout:5/156: dread de/d16/f1b [0,4194304] 0 2024-04-20T11:08:55.612 INFO:tasks.workunit.client.0.smithi117.stdout:1/125: creat d6/f2a x:0 0 0 2024-04-20T11:08:55.612 INFO:tasks.workunit.client.0.smithi117.stdout:1/126: stat d6/f25 0 2024-04-20T11:08:55.617 INFO:tasks.workunit.client.0.smithi117.stdout:6/203: dread d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:55.617 INFO:tasks.workunit.client.0.smithi117.stdout:7/142: symlink d1/d4/l2e 0 2024-04-20T11:08:55.618 INFO:tasks.workunit.client.0.smithi117.stdout:6/204: chown d3/ld 41777 1 2024-04-20T11:08:55.619 INFO:tasks.workunit.client.0.smithi117.stdout:7/143: write d1/fb [480431,113951] 0 2024-04-20T11:08:55.621 INFO:tasks.workunit.client.0.smithi117.stdout:5/157: unlink de/l25 0 2024-04-20T11:08:55.627 INFO:tasks.workunit.client.0.smithi117.stdout:7/144: mknod d1/d1d/c2f 0 2024-04-20T11:08:55.628 INFO:tasks.workunit.client.0.smithi117.stdout:4/161: link d1/d9/cc d1/c3a 0 2024-04-20T11:08:55.630 INFO:tasks.workunit.client.0.smithi117.stdout:4/162: truncate d1/f25 964784 0 2024-04-20T11:08:55.631 INFO:tasks.workunit.client.0.smithi117.stdout:4/163: chown d1/d9 88034 1 2024-04-20T11:08:55.633 INFO:tasks.workunit.client.0.smithi117.stdout:6/205: symlink d3/d4/d2e/l45 0 2024-04-20T11:08:55.634 INFO:tasks.workunit.client.0.smithi117.stdout:7/145: dread d1/f12 [0,4194304] 0 2024-04-20T11:08:55.636 INFO:tasks.workunit.client.0.smithi117.stdout:6/206: write d3/df/d39/f3e [204019,58241] 0 2024-04-20T11:08:55.637 INFO:tasks.workunit.client.0.smithi117.stdout:4/164: creat d1/d9/f3b x:0 0 0 2024-04-20T11:08:55.638 INFO:tasks.workunit.client.0.smithi117.stdout:4/165: chown d1/dd/d1a/d2b 11828 1 2024-04-20T11:08:55.639 INFO:tasks.workunit.client.0.smithi117.stdout:3/99: sync 2024-04-20T11:08:55.639 INFO:tasks.workunit.client.0.smithi117.stdout:9/160: sync 2024-04-20T11:08:55.639 INFO:tasks.workunit.client.0.smithi117.stdout:8/74: sync 2024-04-20T11:08:55.639 INFO:tasks.workunit.client.0.smithi117.stdout:2/118: sync 2024-04-20T11:08:55.642 INFO:tasks.workunit.client.0.smithi117.stdout:7/146: unlink d1/d1d/f2a 0 2024-04-20T11:08:55.643 INFO:tasks.workunit.client.0.smithi117.stdout:1/127: getdents d6/d21 0 2024-04-20T11:08:55.646 INFO:tasks.workunit.client.0.smithi117.stdout:2/119: readlink d1/d15/d1d/l22 0 2024-04-20T11:08:55.647 INFO:tasks.workunit.client.0.smithi117.stdout:2/120: stat d1/d4/dd/f19 0 2024-04-20T11:08:55.648 INFO:tasks.workunit.client.0.smithi117.stdout:6/207: dread d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:55.650 INFO:tasks.workunit.client.0.smithi117.stdout:3/100: creat d0/d5/de/d1d/f1f x:0 0 0 2024-04-20T11:08:55.653 INFO:tasks.workunit.client.0.smithi117.stdout:9/161: creat d4/de/d10/f40 x:0 0 0 2024-04-20T11:08:55.660 INFO:tasks.workunit.client.0.smithi117.stdout:5/158: getdents de 0 2024-04-20T11:08:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:55 smithi173 ceph-mon[23800]: pgmap v7: 65 pgs: 65 active+clean; 268 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 484 KiB/s rd, 22 MiB/s wr, 142 op/s 2024-04-20T11:08:55.664 INFO:tasks.workunit.client.0.smithi117.stdout:5/159: write de/d16/f1e [2925933,31152] 0 2024-04-20T11:08:55.670 INFO:tasks.workunit.client.0.smithi117.stdout:2/121: mknod d1/d4/dd/d17/d25/c27 0 2024-04-20T11:08:55.673 INFO:tasks.workunit.client.0.smithi117.stdout:8/75: creat d0/d8/d11/f18 x:0 0 0 2024-04-20T11:08:55.673 INFO:tasks.workunit.client.0.smithi117.stdout:7/147: creat d1/d4/f30 x:0 0 0 2024-04-20T11:08:55.673 INFO:tasks.workunit.client.0.smithi117.stdout:6/208: dread d3/fc [0,4194304] 0 2024-04-20T11:08:55.675 INFO:tasks.workunit.client.0.smithi117.stdout:3/101: dwrite d0/f1a [0,4194304] 0 2024-04-20T11:08:55.675 INFO:tasks.workunit.client.0.smithi117.stdout:2/122: readlink d1/d15/d1d/l22 0 2024-04-20T11:08:55.676 INFO:tasks.workunit.client.0.smithi117.stdout:4/166: getdents d1/d9 0 2024-04-20T11:08:55.680 INFO:tasks.workunit.client.0.smithi117.stdout:2/123: chown d1/d4/dd/d17/d24 1 1 2024-04-20T11:08:55.680 INFO:tasks.workunit.client.0.smithi117.stdout:7/148: unlink d1/f12 0 2024-04-20T11:08:55.683 INFO:tasks.workunit.client.0.smithi117.stdout:1/128: truncate d6/f12 4012988 0 2024-04-20T11:08:55.683 INFO:tasks.workunit.client.0.smithi117.stdout:4/167: mknod d1/d9/d1f/c3c 0 2024-04-20T11:08:55.685 INFO:tasks.workunit.client.0.smithi117.stdout:6/209: rmdir d3/d4 39 2024-04-20T11:08:55.686 INFO:tasks.workunit.client.0.smithi117.stdout:9/162: dread - d4/de/d10/f3a zero size 2024-04-20T11:08:55.690 INFO:tasks.workunit.client.0.smithi117.stdout:3/102: fdatasync d0/d5/d7/dd/ff 0 2024-04-20T11:08:55.691 INFO:tasks.workunit.client.0.smithi117.stdout:7/149: dread - d1/d4/f1a zero size 2024-04-20T11:08:55.695 INFO:tasks.workunit.client.0.smithi117.stdout:8/76: getdents d0/dc 0 2024-04-20T11:08:55.695 INFO:tasks.workunit.client.0.smithi117.stdout:1/129: stat d6/l14 0 2024-04-20T11:08:55.695 INFO:tasks.workunit.client.0.smithi117.stdout:5/160: creat de/d21/f30 x:0 0 0 2024-04-20T11:08:55.696 INFO:tasks.workunit.client.0.smithi117.stdout:9/163: truncate d4/d5/da/fb 370749 0 2024-04-20T11:08:55.698 INFO:tasks.workunit.client.0.smithi117.stdout:7/150: chown d1/d1d/c29 1571 1 2024-04-20T11:08:55.702 INFO:tasks.workunit.client.0.smithi117.stdout:2/124: getdents d1/d4/dd/d17/d24 0 2024-04-20T11:08:55.703 INFO:tasks.workunit.client.0.smithi117.stdout:9/164: symlink d4/d5/l41 0 2024-04-20T11:08:55.712 INFO:tasks.workunit.client.0.smithi117.stdout:8/77: creat d0/d8/d11/f19 x:0 0 0 2024-04-20T11:08:55.712 INFO:tasks.workunit.client.0.smithi117.stdout:4/168: dwrite d1/f19 [0,4194304] 0 2024-04-20T11:08:55.712 INFO:tasks.workunit.client.0.smithi117.stdout:7/151: dread d1/f22 [0,4194304] 0 2024-04-20T11:08:55.713 INFO:tasks.workunit.client.0.smithi117.stdout:1/130: creat d6/d1f/f2b x:0 0 0 2024-04-20T11:08:55.719 INFO:tasks.workunit.client.0.smithi117.stdout:3/103: dwrite d0/d5/f19 [0,4194304] 0 2024-04-20T11:08:55.727 INFO:tasks.workunit.client.0.smithi117.stdout:9/165: unlink d4/f25 0 2024-04-20T11:08:55.730 INFO:tasks.workunit.client.0.smithi117.stdout:1/131: mknod d6/d1f/c2c 0 2024-04-20T11:08:55.731 INFO:tasks.workunit.client.0.smithi117.stdout:1/132: readlink d6/l1c 0 2024-04-20T11:08:55.737 INFO:tasks.workunit.client.0.smithi117.stdout:3/104: creat d0/d5/d7/f20 x:0 0 0 2024-04-20T11:08:55.740 INFO:tasks.workunit.client.0.smithi117.stdout:5/161: creat de/d21/f31 x:0 0 0 2024-04-20T11:08:55.745 INFO:tasks.workunit.client.0.smithi117.stdout:3/105: mkdir d0/d5/d21 0 2024-04-20T11:08:55.750 INFO:tasks.workunit.client.0.smithi117.stdout:7/152: dwrite d1/fb [0,4194304] 0 2024-04-20T11:08:55.750 INFO:tasks.workunit.client.0.smithi117.stdout:4/169: rename d1/l2d to d1/dd/l3d 0 2024-04-20T11:08:55.754 INFO:tasks.workunit.client.0.smithi117.stdout:2/125: dwrite d1/f2 [0,4194304] 0 2024-04-20T11:08:55.756 INFO:tasks.workunit.client.0.smithi117.stdout:9/166: getdents d4/d30/d31 0 2024-04-20T11:08:55.758 INFO:tasks.workunit.client.0.smithi117.stdout:7/153: fdatasync d1/d4/d6/f26 0 2024-04-20T11:08:55.760 INFO:tasks.workunit.client.0.smithi117.stdout:3/106: write d0/d5/d7/dd/f10 [3201667,61724] 0 2024-04-20T11:08:55.767 INFO:tasks.workunit.client.0.smithi117.stdout:5/162: mknod de/d21/c32 0 2024-04-20T11:08:55.767 INFO:tasks.workunit.client.0.smithi117.stdout:2/126: chown d1/d4/f7 118 1 2024-04-20T11:08:55.767 INFO:tasks.workunit.client.0.smithi117.stdout:4/170: write d1/f10 [233881,73638] 0 2024-04-20T11:08:55.772 INFO:tasks.workunit.client.0.smithi117.stdout:4/171: write d1/f15 [1308214,53268] 0 2024-04-20T11:08:55.773 INFO:tasks.workunit.client.0.smithi117.stdout:6/210: rename d3/d3a/f3d to d3/d4/f46 0 2024-04-20T11:08:55.775 INFO:tasks.workunit.client.0.smithi117.stdout:1/133: dread d6/f10 [0,4194304] 0 2024-04-20T11:08:55.776 INFO:tasks.workunit.client.0.smithi117.stdout:2/127: mknod d1/d4/dd/c28 0 2024-04-20T11:08:55.781 INFO:tasks.workunit.client.0.smithi117.stdout:2/128: write d1/d4/f7 [442279,115455] 0 2024-04-20T11:08:55.785 INFO:tasks.workunit.client.0.smithi117.stdout:4/172: chown d1/dd/l3d 1007969289 1 2024-04-20T11:08:55.786 INFO:tasks.workunit.client.0.smithi117.stdout:2/129: read - d1/d4/dd/de/f23 zero size 2024-04-20T11:08:55.786 INFO:tasks.workunit.client.0.smithi117.stdout:3/107: dread d0/f1a [0,4194304] 0 2024-04-20T11:08:55.786 INFO:tasks.workunit.client.0.smithi117.stdout:1/134: creat d6/d21/f2d x:0 0 0 2024-04-20T11:08:55.789 INFO:tasks.workunit.client.0.smithi117.stdout:1/135: chown d6/lb 451 1 2024-04-20T11:08:55.790 INFO:tasks.workunit.client.0.smithi117.stdout:7/154: link d1/l24 d1/l31 0 2024-04-20T11:08:55.791 INFO:tasks.workunit.client.0.smithi117.stdout:2/130: truncate d1/d20/f21 1207539 0 2024-04-20T11:08:55.793 INFO:tasks.workunit.client.0.smithi117.stdout:9/167: truncate d4/d5/da/f19 127061 0 2024-04-20T11:08:55.795 INFO:tasks.workunit.client.0.smithi117.stdout:6/211: creat d3/d4/d2e/d37/f47 x:0 0 0 2024-04-20T11:08:55.795 INFO:tasks.workunit.client.0.smithi117.stdout:4/173: creat d1/dd/d1a/f3e x:0 0 0 2024-04-20T11:08:55.797 INFO:tasks.workunit.client.0.smithi117.stdout:3/108: symlink d0/d5/d11/l22 0 2024-04-20T11:08:55.803 INFO:tasks.workunit.client.0.smithi117.stdout:9/168: rmdir d4/de/d10 39 2024-04-20T11:08:55.813 INFO:tasks.workunit.client.0.smithi117.stdout:2/131: dread d1/f3 [0,4194304] 0 2024-04-20T11:08:55.813 INFO:tasks.workunit.client.0.smithi117.stdout:5/163: truncate de/f11 174374 0 2024-04-20T11:08:55.814 INFO:tasks.workunit.client.0.smithi117.stdout:3/109: mknod d0/d5/de/c23 0 2024-04-20T11:08:55.817 INFO:tasks.workunit.client.0.smithi117.stdout:7/155: mknod d1/d4/d6/d2b/c32 0 2024-04-20T11:08:55.817 INFO:tasks.workunit.client.0.smithi117.stdout:2/132: write d1/d4/f8 [665534,80086] 0 2024-04-20T11:08:55.823 INFO:tasks.workunit.client.0.smithi117.stdout:4/174: symlink d1/dd/d1a/d2b/l3f 0 2024-04-20T11:08:55.829 INFO:tasks.workunit.client.0.smithi117.stdout:1/136: write d6/d1f/f24 [807130,10548] 0 2024-04-20T11:08:55.834 INFO:tasks.workunit.client.0.smithi117.stdout:3/110: creat d0/d5/de/f24 x:0 0 0 2024-04-20T11:08:55.836 INFO:tasks.workunit.client.0.smithi117.stdout:4/175: write d1/d9/f3b [130272,9790] 0 2024-04-20T11:08:55.839 INFO:tasks.workunit.client.0.smithi117.stdout:9/169: link d4/de/c18 d4/d30/d31/c42 0 2024-04-20T11:08:55.842 INFO:tasks.workunit.client.0.smithi117.stdout:4/176: chown d1/dd 30025971 1 2024-04-20T11:08:55.842 INFO:tasks.workunit.client.0.smithi117.stdout:2/133: mkdir d1/d4/dd/de/d29 0 2024-04-20T11:08:55.845 INFO:tasks.workunit.client.0.smithi117.stdout:0/114: sync 2024-04-20T11:08:55.845 INFO:tasks.workunit.client.0.smithi117.stdout:9/170: fdatasync d4/d5/f21 0 2024-04-20T11:08:55.846 INFO:tasks.workunit.client.0.smithi117.stdout:1/137: rmdir d6/d21 39 2024-04-20T11:08:55.849 INFO:tasks.workunit.client.0.smithi117.stdout:6/212: dwrite d3/d4/f19 [0,4194304] 0 2024-04-20T11:08:55.856 INFO:tasks.workunit.client.0.smithi117.stdout:7/156: dwrite d1/d4/d6/f26 [0,4194304] 0 2024-04-20T11:08:55.857 INFO:tasks.workunit.client.0.smithi117.stdout:3/111: rename d0/d5/de/f24 to d0/d5/f25 0 2024-04-20T11:08:55.860 INFO:tasks.workunit.client.0.smithi117.stdout:7/157: chown d1/d1d/f21 91887707 1 2024-04-20T11:08:55.862 INFO:tasks.workunit.client.0.smithi117.stdout:5/164: dwrite de/d16/f2d [0,4194304] 0 2024-04-20T11:08:55.862 INFO:tasks.workunit.client.0.smithi117.stdout:0/115: creat d3/d7/d16/f2b x:0 0 0 2024-04-20T11:08:55.863 INFO:tasks.workunit.client.0.smithi117.stdout:7/158: readlink d1/d4/d6/d13/l19 0 2024-04-20T11:08:55.867 INFO:tasks.workunit.client.0.smithi117.stdout:9/171: creat d4/d5/da/f43 x:0 0 0 2024-04-20T11:08:55.872 INFO:tasks.workunit.client.0.smithi117.stdout:1/138: unlink d6/d1f/l26 0 2024-04-20T11:08:55.876 INFO:tasks.workunit.client.0.smithi117.stdout:2/134: rmdir d1/d4/dd/d17/d24 0 2024-04-20T11:08:55.880 INFO:tasks.workunit.client.0.smithi117.stdout:2/135: chown d1/f2 1988236675 1 2024-04-20T11:08:55.880 INFO:tasks.workunit.client.0.smithi117.stdout:5/165: symlink de/d21/l33 0 2024-04-20T11:08:55.882 INFO:tasks.workunit.client.0.smithi117.stdout:0/116: mkdir d3/d7/d2c 0 2024-04-20T11:08:55.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:55 smithi117 ceph-mon[15659]: pgmap v7: 65 pgs: 65 active+clean; 268 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 484 KiB/s rd, 22 MiB/s wr, 142 op/s 2024-04-20T11:08:55.892 INFO:tasks.workunit.client.0.smithi117.stdout:2/136: mkdir d1/d15/d1d/d2a 0 2024-04-20T11:08:55.893 INFO:tasks.workunit.client.0.smithi117.stdout:0/117: chown d3/d7/d8/df 1058 1 2024-04-20T11:08:55.895 INFO:tasks.workunit.client.0.smithi117.stdout:1/139: creat d6/f2e x:0 0 0 2024-04-20T11:08:55.899 INFO:tasks.workunit.client.0.smithi117.stdout:5/166: truncate de/f14 3127210 0 2024-04-20T11:08:55.900 INFO:tasks.workunit.client.0.smithi117.stdout:0/118: dread - d3/d7/d8/df/f1b zero size 2024-04-20T11:08:55.902 INFO:tasks.workunit.client.0.smithi117.stdout:7/159: dwrite d1/fb [0,4194304] 0 2024-04-20T11:08:55.905 INFO:tasks.workunit.client.0.smithi117.stdout:2/137: chown d1/d4/dd/d17/f1f 33520037 1 2024-04-20T11:08:55.908 INFO:tasks.workunit.client.0.smithi117.stdout:7/160: write d1/d4/d6/d13/f15 [879593,45488] 0 2024-04-20T11:08:55.908 INFO:tasks.workunit.client.0.smithi117.stdout:4/177: dwrite d1/f7 [0,4194304] 0 2024-04-20T11:08:55.910 INFO:tasks.workunit.client.0.smithi117.stdout:5/167: creat de/f34 x:0 0 0 2024-04-20T11:08:55.913 INFO:tasks.workunit.client.0.smithi117.stdout:0/119: mknod d3/d5/c2d 0 2024-04-20T11:08:55.922 INFO:tasks.workunit.client.0.smithi117.stdout:4/178: rename d1/c29 to d1/d9/d1f/c40 0 2024-04-20T11:08:55.924 INFO:tasks.workunit.client.0.smithi117.stdout:5/168: mkdir de/d21/d35 0 2024-04-20T11:08:55.927 INFO:tasks.workunit.client.0.smithi117.stdout:0/120: rename d3/d5/fd to d3/d7/d16/f2e 0 2024-04-20T11:08:55.931 INFO:tasks.workunit.client.0.smithi117.stdout:4/179: mknod d1/dd/d1a/d2b/c41 0 2024-04-20T11:08:55.931 INFO:tasks.workunit.client.0.smithi117.stdout:5/169: readlink de/l1a 0 2024-04-20T11:08:55.936 INFO:tasks.workunit.client.0.smithi117.stdout:7/161: dwrite d1/d4/f1a [0,4194304] 0 2024-04-20T11:08:55.943 INFO:tasks.workunit.client.0.smithi117.stdout:2/138: write d1/d4/fb [166225,53312] 0 2024-04-20T11:08:55.943 INFO:tasks.workunit.client.0.smithi117.stdout:0/121: symlink d3/d7/d20/l2f 0 2024-04-20T11:08:55.944 INFO:tasks.workunit.client.0.smithi117.stdout:0/122: fdatasync d3/d7/f14 0 2024-04-20T11:08:55.944 INFO:tasks.workunit.client.0.smithi117.stdout:2/139: chown d1/d4 31 1 2024-04-20T11:08:55.945 INFO:tasks.workunit.client.0.smithi117.stdout:7/162: write d1/d4/d6/d13/f15 [15239,113411] 0 2024-04-20T11:08:55.949 INFO:tasks.workunit.client.0.smithi117.stdout:4/180: write d1/dd/f13 [143753,7381] 0 2024-04-20T11:08:55.951 INFO:tasks.workunit.client.0.smithi117.stdout:0/123: dread - d3/d5/f26 zero size 2024-04-20T11:08:55.953 INFO:tasks.workunit.client.0.smithi117.stdout:2/140: chown d1/d4/dd/l13 3358 1 2024-04-20T11:08:55.955 INFO:tasks.workunit.client.0.smithi117.stdout:7/163: rename d1/fb to d1/d4/d6/d2b/f33 0 2024-04-20T11:08:55.956 INFO:tasks.workunit.client.0.smithi117.stdout:4/181: mknod d1/c42 0 2024-04-20T11:08:55.957 INFO:tasks.workunit.client.0.smithi117.stdout:5/170: link de/f11 de/d21/d35/f36 0 2024-04-20T11:08:55.957 INFO:tasks.workunit.client.0.smithi117.stdout:2/141: write d1/d4/dd/d17/f1e [566998,100850] 0 2024-04-20T11:08:55.957 INFO:tasks.workunit.client.0.smithi117.stdout:0/124: symlink d3/d7/d20/l30 0 2024-04-20T11:08:55.961 INFO:tasks.workunit.client.0.smithi117.stdout:4/182: write d1/f15 [247009,107402] 0 2024-04-20T11:08:55.961 INFO:tasks.workunit.client.0.smithi117.stdout:5/171: mknod de/d21/d35/c37 0 2024-04-20T11:08:55.964 INFO:tasks.workunit.client.0.smithi117.stdout:7/164: creat d1/d4/d1c/f34 x:0 0 0 2024-04-20T11:08:55.965 INFO:tasks.workunit.client.0.smithi117.stdout:5/172: symlink de/d16/l38 0 2024-04-20T11:08:55.966 INFO:tasks.workunit.client.0.smithi117.stdout:2/142: creat d1/d4/dd/f2b x:0 0 0 2024-04-20T11:08:55.969 INFO:tasks.workunit.client.0.smithi117.stdout:8/78: sync 2024-04-20T11:08:55.970 INFO:tasks.workunit.client.0.smithi117.stdout:2/143: creat d1/d4/dd/f2c x:0 0 0 2024-04-20T11:08:55.972 INFO:tasks.workunit.client.0.smithi117.stdout:5/173: mkdir de/d16/d39 0 2024-04-20T11:08:55.972 INFO:tasks.workunit.client.0.smithi117.stdout:8/79: write d0/d9/fe [498252,69524] 0 2024-04-20T11:08:55.972 INFO:tasks.workunit.client.0.smithi117.stdout:7/165: link d1/d4/d6/d13/c25 d1/d1d/c35 0 2024-04-20T11:08:55.976 INFO:tasks.workunit.client.0.smithi117.stdout:8/80: readlink d0/d8/lf 0 2024-04-20T11:08:55.980 INFO:tasks.workunit.client.0.smithi117.stdout:5/174: creat de/d21/d35/f3a x:0 0 0 2024-04-20T11:08:55.982 INFO:tasks.workunit.client.0.smithi117.stdout:2/144: dread d1/d4/dd/de/f18 [0,4194304] 0 2024-04-20T11:08:55.983 INFO:tasks.workunit.client.0.smithi117.stdout:5/175: stat de/l1a 0 2024-04-20T11:08:55.985 INFO:tasks.workunit.client.0.smithi117.stdout:8/81: creat d0/dc/f1a x:0 0 0 2024-04-20T11:08:55.990 INFO:tasks.workunit.client.0.smithi117.stdout:0/125: dwrite d3/d7/f18 [0,4194304] 0 2024-04-20T11:08:55.990 INFO:tasks.workunit.client.0.smithi117.stdout:8/82: write d0/fb [1949711,90819] 0 2024-04-20T11:08:55.990 INFO:tasks.workunit.client.0.smithi117.stdout:4/183: dwrite d1/f35 [0,4194304] 0 2024-04-20T11:08:55.991 INFO:tasks.workunit.client.0.smithi117.stdout:4/184: chown d1/dd 3419 1 2024-04-20T11:08:55.996 INFO:tasks.workunit.client.0.smithi117.stdout:4/185: truncate d1/d9/f3b 1022377 0 2024-04-20T11:08:56.010 INFO:tasks.workunit.client.0.smithi117.stdout:7/166: getdents d1/d4/d6/d2b 0 2024-04-20T11:08:56.010 INFO:tasks.workunit.client.0.smithi117.stdout:0/126: creat d3/d7/d8/f31 x:0 0 0 2024-04-20T11:08:56.011 INFO:tasks.workunit.client.0.smithi117.stdout:5/176: dwrite de/d21/d35/f3a [0,4194304] 0 2024-04-20T11:08:56.018 INFO:tasks.workunit.client.0.smithi117.stdout:0/127: stat d3/d7/le 0 2024-04-20T11:08:56.023 INFO:tasks.workunit.client.0.smithi117.stdout:7/167: fsync d1/d4/d6/d2b/f33 0 2024-04-20T11:08:56.027 INFO:tasks.workunit.client.0.smithi117.stdout:8/83: dread d0/f5 [0,4194304] 0 2024-04-20T11:08:56.032 INFO:tasks.workunit.client.0.smithi117.stdout:7/168: fsync d1/f22 0 2024-04-20T11:08:56.034 INFO:tasks.workunit.client.0.smithi117.stdout:0/128: mkdir d3/d7/d8/d32 0 2024-04-20T11:08:56.036 INFO:tasks.workunit.client.0.smithi117.stdout:8/84: unlink d0/fb 0 2024-04-20T11:08:56.038 INFO:tasks.workunit.client.0.smithi117.stdout:4/186: dwrite d1/dd/d1a/f1d [0,4194304] 0 2024-04-20T11:08:56.039 INFO:tasks.workunit.client.0.smithi117.stdout:2/145: dwrite d1/d4/dd/de/f18 [0,4194304] 0 2024-04-20T11:08:56.040 INFO:tasks.workunit.client.0.smithi117.stdout:7/169: chown d1/d4/d1c/d1f 367 1 2024-04-20T11:08:56.042 INFO:tasks.workunit.client.0.smithi117.stdout:0/129: read - d3/d5/f26 zero size 2024-04-20T11:08:56.044 INFO:tasks.workunit.client.0.smithi117.stdout:2/146: chown d1/d4/dd 417 1 2024-04-20T11:08:56.045 INFO:tasks.workunit.client.0.smithi117.stdout:5/177: dwrite de/f34 [0,4194304] 0 2024-04-20T11:08:56.050 INFO:tasks.workunit.client.0.smithi117.stdout:2/147: write d1/d4/dd/d17/f1f [966592,122880] 0 2024-04-20T11:08:56.055 INFO:tasks.workunit.client.0.smithi117.stdout:5/178: getdents de/d16/d39 0 2024-04-20T11:08:56.056 INFO:tasks.workunit.client.0.smithi117.stdout:2/148: rmdir d1/d4/dd/d17 39 2024-04-20T11:08:56.065 INFO:tasks.workunit.client.0.smithi117.stdout:5/179: creat de/d21/d35/f3b x:0 0 0 2024-04-20T11:08:56.065 INFO:tasks.workunit.client.0.smithi117.stdout:2/149: symlink d1/d15/l2d 0 2024-04-20T11:08:56.068 INFO:tasks.workunit.client.0.smithi117.stdout:8/85: dread d0/f5 [0,4194304] 0 2024-04-20T11:08:56.071 INFO:tasks.workunit.client.0.smithi117.stdout:2/150: read d1/d4/f5 [2306017,58953] 0 2024-04-20T11:08:56.072 INFO:tasks.workunit.client.0.smithi117.stdout:7/170: truncate d1/d1d/f21 2603361 0 2024-04-20T11:08:56.074 INFO:tasks.workunit.client.0.smithi117.stdout:8/86: chown d0/f1 0 1 2024-04-20T11:08:56.077 INFO:tasks.workunit.client.0.smithi117.stdout:4/187: dwrite d1/d9/f3b [0,4194304] 0 2024-04-20T11:08:56.077 INFO:tasks.workunit.client.0.smithi117.stdout:2/151: chown d1/d4/dd/c28 262 1 2024-04-20T11:08:56.078 INFO:tasks.workunit.client.0.smithi117.stdout:8/87: fdatasync d0/d9/f12 0 2024-04-20T11:08:56.080 INFO:tasks.workunit.client.0.smithi117.stdout:0/130: dwrite d3/d7/f19 [0,4194304] 0 2024-04-20T11:08:56.080 INFO:tasks.workunit.client.0.smithi117.stdout:2/152: write d1/d4/f8 [1174635,24334] 0 2024-04-20T11:08:56.081 INFO:tasks.workunit.client.0.smithi117.stdout:5/180: link de/f19 de/d21/f3c 0 2024-04-20T11:08:56.084 INFO:tasks.workunit.client.0.smithi117.stdout:5/181: write f0 [4006235,89309] 0 2024-04-20T11:08:56.087 INFO:tasks.workunit.client.0.smithi117.stdout:7/171: mknod d1/d4/c36 0 2024-04-20T11:08:56.092 INFO:tasks.workunit.client.0.smithi117.stdout:4/188: mkdir d1/dd/d1a/d43 0 2024-04-20T11:08:56.093 INFO:tasks.workunit.client.0.smithi117.stdout:8/88: mknod d0/d8/d11/c1b 0 2024-04-20T11:08:56.094 INFO:tasks.workunit.client.0.smithi117.stdout:0/131: symlink d3/d7/d8/l33 0 2024-04-20T11:08:56.096 INFO:tasks.workunit.client.0.smithi117.stdout:2/153: rename d1/d15/d1d/d2a to d1/d4/dd/de/d29/d2e 0 2024-04-20T11:08:56.096 INFO:tasks.workunit.client.0.smithi117.stdout:4/189: fsync d1/dd/d1a/f37 0 2024-04-20T11:08:56.101 INFO:tasks.workunit.client.0.smithi117.stdout:5/182: creat de/d21/d35/f3d x:0 0 0 2024-04-20T11:08:56.105 INFO:tasks.workunit.client.0.smithi117.stdout:2/154: fsync d1/d4/dd/de/f23 0 2024-04-20T11:08:56.105 INFO:tasks.workunit.client.0.smithi117.stdout:5/183: chown de/d21/l33 76385543 1 2024-04-20T11:08:56.107 INFO:tasks.workunit.client.0.smithi117.stdout:2/155: mknod d1/d15/d1d/c2f 0 2024-04-20T11:08:56.107 INFO:tasks.workunit.client.0.smithi117.stdout:7/172: dread d1/d4/d6/f26 [0,4194304] 0 2024-04-20T11:08:56.109 INFO:tasks.workunit.client.0.smithi117.stdout:5/184: rmdir de/d16 39 2024-04-20T11:08:56.110 INFO:tasks.workunit.client.0.smithi117.stdout:7/173: write d1/f22 [1022170,89904] 0 2024-04-20T11:08:56.113 INFO:tasks.workunit.client.0.smithi117.stdout:2/156: write d1/d4/f5 [530715,90707] 0 2024-04-20T11:08:56.114 INFO:tasks.workunit.client.0.smithi117.stdout:5/185: dread - de/d21/f2e zero size 2024-04-20T11:08:56.118 INFO:tasks.workunit.client.0.smithi117.stdout:0/132: dread d3/d7/d8/f13 [0,4194304] 0 2024-04-20T11:08:56.118 INFO:tasks.workunit.client.0.smithi117.stdout:5/186: write de/d21/f30 [129008,88511] 0 2024-04-20T11:08:56.121 INFO:tasks.workunit.client.0.smithi117.stdout:0/133: write d3/d7/f19 [1719824,65643] 0 2024-04-20T11:08:56.121 INFO:tasks.workunit.client.0.smithi117.stdout:5/187: unlink c1 0 2024-04-20T11:08:56.124 INFO:tasks.workunit.client.0.smithi117.stdout:8/89: truncate d0/d9/f12 2458868 0 2024-04-20T11:08:56.126 INFO:tasks.workunit.client.0.smithi117.stdout:8/90: dread - d0/dc/f1a zero size 2024-04-20T11:08:56.133 INFO:tasks.workunit.client.0.smithi117.stdout:7/174: dwrite d1/d4/d6/d2b/f33 [0,4194304] 0 2024-04-20T11:08:56.134 INFO:tasks.workunit.client.0.smithi117.stdout:0/134: link d3/l1d d3/d7/d20/l34 0 2024-04-20T11:08:56.139 INFO:tasks.workunit.client.0.smithi117.stdout:8/91: rename d0/d8/d11/f18 to d0/d8/d11/f1c 0 2024-04-20T11:08:56.142 INFO:tasks.workunit.client.0.smithi117.stdout:7/175: stat d1/d4/d1c/d1f/d2d 0 2024-04-20T11:08:56.143 INFO:tasks.workunit.client.0.smithi117.stdout:0/135: write d3/d7/d8/df/f12 [1085595,125031] 0 2024-04-20T11:08:56.146 INFO:tasks.workunit.client.0.smithi117.stdout:2/157: dwrite d1/d4/dd/d17/f1f [0,4194304] 0 2024-04-20T11:08:56.148 INFO:tasks.workunit.client.0.smithi117.stdout:0/136: stat d3/d7/d8/c29 0 2024-04-20T11:08:56.151 INFO:tasks.workunit.client.0.smithi117.stdout:5/188: dwrite de/d21/f31 [0,4194304] 0 2024-04-20T11:08:56.156 INFO:tasks.workunit.client.0.smithi117.stdout:0/137: readlink d3/l1d 0 2024-04-20T11:08:56.157 INFO:tasks.workunit.client.0.smithi117.stdout:8/92: rename d0/c10 to d0/d8/c1d 0 2024-04-20T11:08:56.157 INFO:tasks.workunit.client.0.smithi117.stdout:0/138: chown d3/d7/f9 19850 1 2024-04-20T11:08:56.158 INFO:tasks.workunit.client.0.smithi117.stdout:7/176: write d1/d1d/f21 [381550,9028] 0 2024-04-20T11:08:56.160 INFO:tasks.workunit.client.0.smithi117.stdout:8/93: read d0/f1 [1440715,117263] 0 2024-04-20T11:08:56.162 INFO:tasks.workunit.client.0.smithi117.stdout:5/189: fdatasync de/f14 0 2024-04-20T11:08:56.166 INFO:tasks.workunit.client.0.smithi117.stdout:8/94: stat d0/d9/f12 0 2024-04-20T11:08:56.172 INFO:tasks.workunit.client.0.smithi117.stdout:0/139: rename d3/d5/c2d to d3/d7/d20/c35 0 2024-04-20T11:08:56.175 INFO:tasks.workunit.client.0.smithi117.stdout:5/190: chown de/f29 39542394 1 2024-04-20T11:08:56.176 INFO:tasks.workunit.client.0.smithi117.stdout:8/95: unlink d0/d9/f12 0 2024-04-20T11:08:56.178 INFO:tasks.workunit.client.0.smithi117.stdout:7/177: dwrite d1/d4/d6/d2b/f33 [4194304,4194304] 0 2024-04-20T11:08:56.178 INFO:tasks.workunit.client.0.smithi117.stdout:0/140: chown l2 3840263 1 2024-04-20T11:08:56.181 INFO:tasks.workunit.client.0.smithi117.stdout:0/141: fdatasync d3/d7/d8/df/f1b 0 2024-04-20T11:08:56.183 INFO:tasks.workunit.client.0.smithi117.stdout:8/96: link d0/dc/f1a d0/dc/f1e 0 2024-04-20T11:08:56.184 INFO:tasks.workunit.client.0.smithi117.stdout:7/178: truncate d1/d4/d1c/f34 941118 0 2024-04-20T11:08:56.193 INFO:tasks.workunit.client.0.smithi117.stdout:7/179: rename d1/d4/l11 to d1/d4/d6/d13/l37 0 2024-04-20T11:08:56.196 INFO:tasks.workunit.client.0.smithi117.stdout:7/180: truncate d1/d4/f30 704564 0 2024-04-20T11:08:56.198 INFO:tasks.workunit.client.0.smithi117.stdout:5/191: dwrite de/d21/f31 [0,4194304] 0 2024-04-20T11:08:56.198 INFO:tasks.workunit.client.0.smithi117.stdout:7/181: write d1/d4/d6/d13/f28 [963738,47089] 0 2024-04-20T11:08:56.199 INFO:tasks.workunit.client.0.smithi117.stdout:0/142: truncate d3/d7/f9 1975206 0 2024-04-20T11:08:56.200 INFO:tasks.workunit.client.0.smithi117.stdout:8/97: truncate d0/f7 1098519 0 2024-04-20T11:08:56.202 INFO:tasks.workunit.client.0.smithi117.stdout:0/143: stat d3/d7/d8/df/f1b 0 2024-04-20T11:08:56.206 INFO:tasks.workunit.client.0.smithi117.stdout:8/98: creat d0/d8/d11/f1f x:0 0 0 2024-04-20T11:08:56.206 INFO:tasks.workunit.client.0.smithi117.stdout:0/144: write d3/d7/d8/f13 [3242695,125651] 0 2024-04-20T11:08:56.209 INFO:tasks.workunit.client.0.smithi117.stdout:5/192: rmdir de/d16/d39 0 2024-04-20T11:08:56.211 INFO:tasks.workunit.client.0.smithi117.stdout:8/99: creat d0/f20 x:0 0 0 2024-04-20T11:08:56.213 INFO:tasks.workunit.client.0.smithi117.stdout:0/145: chown d3/d7/d20/c35 0 1 2024-04-20T11:08:56.213 INFO:tasks.workunit.client.0.smithi117.stdout:7/182: dread d1/d4/d6/f26 [0,4194304] 0 2024-04-20T11:08:56.214 INFO:tasks.workunit.client.0.smithi117.stdout:5/193: truncate de/d21/d35/f3b 518588 0 2024-04-20T11:08:56.218 INFO:tasks.workunit.client.0.smithi117.stdout:5/194: chown de/f29 1630 1 2024-04-20T11:08:56.219 INFO:tasks.workunit.client.0.smithi117.stdout:7/183: mknod d1/c38 0 2024-04-20T11:08:56.219 INFO:tasks.workunit.client.0.smithi117.stdout:0/146: mknod d3/d7/d16/c36 0 2024-04-20T11:08:56.219 INFO:tasks.workunit.client.0.smithi117.stdout:5/195: rename f0 to de/d21/d35/f3e 0 2024-04-20T11:08:56.222 INFO:tasks.workunit.client.0.smithi117.stdout:7/184: rename d1/d4/d1c to d1/d4/d39 0 2024-04-20T11:08:56.224 INFO:tasks.workunit.client.0.smithi117.stdout:7/185: write d1/f22 [1199775,75533] 0 2024-04-20T11:08:56.284 INFO:tasks.workunit.client.0.smithi117.stdout:3/112: sync 2024-04-20T11:08:56.284 INFO:tasks.workunit.client.0.smithi117.stdout:9/172: sync 2024-04-20T11:08:56.284 INFO:tasks.workunit.client.0.smithi117.stdout:1/140: sync 2024-04-20T11:08:56.284 INFO:tasks.workunit.client.0.smithi117.stdout:6/213: sync 2024-04-20T11:08:56.286 INFO:tasks.workunit.client.0.smithi117.stdout:1/141: write d6/f25 [802899,115463] 0 2024-04-20T11:08:56.316 INFO:tasks.workunit.client.0.smithi117.stdout:5/196: mknod de/c3f 0 2024-04-20T11:08:56.319 INFO:tasks.workunit.client.0.smithi117.stdout:5/197: chown de/d16/f2d 548 1 2024-04-20T11:08:56.324 INFO:tasks.workunit.client.0.smithi117.stdout:7/186: creat d1/d4/d6/d17/f3a x:0 0 0 2024-04-20T11:08:56.326 INFO:tasks.workunit.client.0.smithi117.stdout:7/187: dread - d1/d4/d6/d17/f3a zero size 2024-04-20T11:08:56.336 INFO:tasks.workunit.client.0.smithi117.stdout:9/173: mknod d4/d35/c44 0 2024-04-20T11:08:56.340 INFO:tasks.workunit.client.0.smithi117.stdout:6/214: rename d3/d4/d2e/d35/l3c to d3/l48 0 2024-04-20T11:08:56.347 INFO:tasks.workunit.client.0.smithi117.stdout:5/198: link de/c3f de/d21/d35/c40 0 2024-04-20T11:08:56.350 INFO:tasks.workunit.client.0.smithi117.stdout:3/113: rename d0/d5/d11/l22 to d0/d5/d7/l26 0 2024-04-20T11:08:56.351 INFO:tasks.workunit.client.0.smithi117.stdout:1/142: dwrite d6/d21/f2d [0,4194304] 0 2024-04-20T11:08:56.358 INFO:tasks.workunit.client.0.smithi117.stdout:3/114: creat d0/d5/d7/f27 x:0 0 0 2024-04-20T11:08:56.360 INFO:tasks.workunit.client.0.smithi117.stdout:1/143: symlink d6/d21/l2f 0 2024-04-20T11:08:56.361 INFO:tasks.workunit.client.0.smithi117.stdout:6/215: dwrite d3/d4/d2e/d3f/f42 [0,4194304] 0 2024-04-20T11:08:56.363 INFO:tasks.workunit.client.0.smithi117.stdout:9/174: creat d4/f45 x:0 0 0 2024-04-20T11:08:56.364 INFO:tasks.workunit.client.0.smithi117.stdout:7/188: truncate d1/f10 4040495 0 2024-04-20T11:08:56.364 INFO:tasks.workunit.client.0.smithi117.stdout:9/175: chown d4/de/f1b 376 1 2024-04-20T11:08:56.366 INFO:tasks.workunit.client.0.smithi117.stdout:7/189: chown d1/d4/d6/d2b/f33 1892 1 2024-04-20T11:08:56.367 INFO:tasks.workunit.client.0.smithi117.stdout:5/199: mkdir de/d41 0 2024-04-20T11:08:56.369 INFO:tasks.workunit.client.0.smithi117.stdout:1/144: mkdir d6/d1f/d30 0 2024-04-20T11:08:56.372 INFO:tasks.workunit.client.0.smithi117.stdout:3/115: creat d0/d5/d21/f28 x:0 0 0 2024-04-20T11:08:56.376 INFO:tasks.workunit.client.0.smithi117.stdout:1/145: creat d6/d21/f31 x:0 0 0 2024-04-20T11:08:56.378 INFO:tasks.workunit.client.0.smithi117.stdout:5/200: rename de/c20 to de/d16/c42 0 2024-04-20T11:08:56.379 INFO:tasks.workunit.client.0.smithi117.stdout:1/146: stat d6/d1f/d30 0 2024-04-20T11:08:56.380 INFO:tasks.workunit.client.0.smithi117.stdout:2/158: sync 2024-04-20T11:08:56.380 INFO:tasks.workunit.client.0.smithi117.stdout:4/190: sync 2024-04-20T11:08:56.387 INFO:tasks.workunit.client.0.smithi117.stdout:9/176: rename d4/de/d10/f34 to d4/d5/d3c/f46 0 2024-04-20T11:08:56.387 INFO:tasks.workunit.client.0.smithi117.stdout:6/216: dread d3/f2c [0,4194304] 0 2024-04-20T11:08:56.388 INFO:tasks.workunit.client.0.smithi117.stdout:1/147: mkdir d6/d1f/d30/d32 0 2024-04-20T11:08:56.392 INFO:tasks.workunit.client.0.smithi117.stdout:7/190: dread d1/f7 [0,4194304] 0 2024-04-20T11:08:56.393 INFO:tasks.workunit.client.0.smithi117.stdout:9/177: write d4/d5/f28 [533362,13636] 0 2024-04-20T11:08:56.395 INFO:tasks.workunit.client.0.smithi117.stdout:2/159: rename d1/f2 to d1/d4/dd/d17/f30 0 2024-04-20T11:08:56.399 INFO:tasks.workunit.client.0.smithi117.stdout:1/148: creat d6/d21/f33 x:0 0 0 2024-04-20T11:08:56.399 INFO:tasks.workunit.client.0.smithi117.stdout:5/201: link de/d21/d35/f3b de/d21/d35/f43 0 2024-04-20T11:08:56.402 INFO:tasks.workunit.client.0.smithi117.stdout:9/178: chown d4/de/f2d 3326919 1 2024-04-20T11:08:56.402 INFO:tasks.workunit.client.0.smithi117.stdout:2/160: fsync d1/d4/dd/de/f23 0 2024-04-20T11:08:56.405 INFO:tasks.workunit.client.0.smithi117.stdout:7/191: mknod d1/d4/d39/d1f/c3b 0 2024-04-20T11:08:56.407 INFO:tasks.workunit.client.0.smithi117.stdout:3/116: dwrite d0/d5/d7/dd/f10 [0,4194304] 0 2024-04-20T11:08:56.407 INFO:tasks.workunit.client.0.smithi117.stdout:2/161: stat d1/d4/dd/f2c 0 2024-04-20T11:08:56.408 INFO:tasks.workunit.client.0.smithi117.stdout:1/149: mknod d6/d1f/d30/c34 0 2024-04-20T11:08:56.408 INFO:tasks.workunit.client.0.smithi117.stdout:9/179: truncate d4/de/f3e 467127 0 2024-04-20T11:08:56.410 INFO:tasks.workunit.client.0.smithi117.stdout:7/192: chown d1/d4/d6/d17 10959017 1 2024-04-20T11:08:56.411 INFO:tasks.workunit.client.0.smithi117.stdout:4/191: dwrite d1/f20 [0,4194304] 0 2024-04-20T11:08:56.413 INFO:tasks.workunit.client.0.smithi117.stdout:5/202: mknod de/d21/c44 0 2024-04-20T11:08:56.418 INFO:tasks.workunit.client.0.smithi117.stdout:2/162: unlink d1/d4/dd/f2c 0 2024-04-20T11:08:56.422 INFO:tasks.workunit.client.0.smithi117.stdout:3/117: truncate d0/f3 158693 0 2024-04-20T11:08:56.422 INFO:tasks.workunit.client.0.smithi117.stdout:6/217: dwrite d3/d4/d2e/d3f/f42 [0,4194304] 0 2024-04-20T11:08:56.423 INFO:tasks.workunit.client.0.smithi117.stdout:0/147: sync 2024-04-20T11:08:56.423 INFO:tasks.workunit.client.0.smithi117.stdout:8/100: sync 2024-04-20T11:08:56.427 INFO:tasks.workunit.client.0.smithi117.stdout:4/192: rename d1 to d1/dd/d44 22 2024-04-20T11:08:56.428 INFO:tasks.workunit.client.0.smithi117.stdout:9/180: write d4/de/d10/f24 [254456,99844] 0 2024-04-20T11:08:56.434 INFO:tasks.workunit.client.0.smithi117.stdout:3/118: write d0/d5/d21/f28 [888802,3519] 0 2024-04-20T11:08:56.438 INFO:tasks.workunit.client.0.smithi117.stdout:7/193: rename d1/d4/d6/c23 to d1/d4/d39/d1f/d2d/c3c 0 2024-04-20T11:08:56.440 INFO:tasks.workunit.client.0.smithi117.stdout:4/193: mkdir d1/d45 0 2024-04-20T11:08:56.444 INFO:tasks.workunit.client.0.smithi117.stdout:6/218: dread d3/d4/d2e/d3f/f42 [0,4194304] 0 2024-04-20T11:08:56.446 INFO:tasks.workunit.client.0.smithi117.stdout:9/181: creat d4/d5/d3c/f47 x:0 0 0 2024-04-20T11:08:56.446 INFO:tasks.workunit.client.0.smithi117.stdout:1/150: getdents d6/d16 0 2024-04-20T11:08:56.450 INFO:tasks.workunit.client.0.smithi117.stdout:2/163: dwrite d1/d4/dd/d17/f1f [0,4194304] 0 2024-04-20T11:08:56.451 INFO:tasks.workunit.client.0.smithi117.stdout:0/148: rename d3/d7/d8/df/f12 to d3/d5/f37 0 2024-04-20T11:08:56.454 INFO:tasks.workunit.client.0.smithi117.stdout:4/194: readlink d1/dd/d1a/l2f 0 2024-04-20T11:08:56.460 INFO:tasks.workunit.client.0.smithi117.stdout:2/164: stat d1/d4/f5 0 2024-04-20T11:08:56.463 INFO:tasks.workunit.client.0.smithi117.stdout:0/149: chown d3/d5/c1f 868534 1 2024-04-20T11:08:56.471 INFO:tasks.workunit.client.0.smithi117.stdout:1/151: mkdir d6/d35 0 2024-04-20T11:08:56.473 INFO:tasks.workunit.client.0.smithi117.stdout:2/165: write d1/d20/f21 [1220062,3265] 0 2024-04-20T11:08:56.479 INFO:tasks.workunit.client.0.smithi117.stdout:4/195: rename d1/f15 to d1/dd/d1a/d2b/f46 0 2024-04-20T11:08:56.480 INFO:tasks.workunit.client.0.smithi117.stdout:3/119: dwrite d0/d5/fc [4194304,4194304] 0 2024-04-20T11:08:56.491 INFO:tasks.workunit.client.0.smithi117.stdout:7/194: dwrite d1/d4/d6/d2b/f33 [0,4194304] 0 2024-04-20T11:08:56.493 INFO:tasks.workunit.client.0.smithi117.stdout:2/166: creat d1/d15/d1d/f31 x:0 0 0 2024-04-20T11:08:56.502 INFO:tasks.workunit.client.0.smithi117.stdout:0/150: creat d3/f38 x:0 0 0 2024-04-20T11:08:56.505 INFO:tasks.workunit.client.0.smithi117.stdout:5/203: dread fb [0,4194304] 0 2024-04-20T11:08:56.508 INFO:tasks.workunit.client.0.smithi117.stdout:9/182: dwrite d4/d5/d3c/f47 [0,4194304] 0 2024-04-20T11:08:56.514 INFO:tasks.workunit.client.0.smithi117.stdout:0/151: dread - d3/f21 zero size 2024-04-20T11:08:56.516 INFO:tasks.workunit.client.0.smithi117.stdout:4/196: symlink d1/dd/d1a/d43/l47 0 2024-04-20T11:08:56.524 INFO:tasks.workunit.client.0.smithi117.stdout:6/219: dwrite d3/f30 [4194304,4194304] 0 2024-04-20T11:08:56.527 INFO:tasks.workunit.client.0.smithi117.stdout:8/101: dwrite d0/f1 [0,4194304] 0 2024-04-20T11:08:56.535 INFO:tasks.workunit.client.0.smithi117.stdout:8/102: chown d0/d8/d11/f1c 309 1 2024-04-20T11:08:56.536 INFO:tasks.workunit.client.0.smithi117.stdout:8/103: stat d0/d8/d11/f14 0 2024-04-20T11:08:56.538 INFO:tasks.workunit.client.0.smithi117.stdout:8/104: truncate d0/f3 4775273 0 2024-04-20T11:08:56.538 INFO:tasks.workunit.client.0.smithi117.stdout:7/195: mkdir d1/d4/d3d 0 2024-04-20T11:08:56.539 INFO:tasks.workunit.client.0.smithi117.stdout:4/197: rmdir d1/dd/d1a/d2b 39 2024-04-20T11:08:56.539 INFO:tasks.workunit.client.0.smithi117.stdout:1/152: dwrite d6/f1a [0,4194304] 0 2024-04-20T11:08:56.540 INFO:tasks.workunit.client.0.smithi117.stdout:2/167: symlink d1/d4/dd/l32 0 2024-04-20T11:08:56.543 INFO:tasks.workunit.client.0.smithi117.stdout:1/153: readlink d6/lc 0 2024-04-20T11:08:56.544 INFO:tasks.workunit.client.0.smithi117.stdout:6/220: readlink d3/l1d 0 2024-04-20T11:08:56.544 INFO:tasks.workunit.client.0.smithi117.stdout:3/120: dwrite d0/d5/f13 [0,4194304] 0 2024-04-20T11:08:56.544 INFO:tasks.workunit.client.0.smithi117.stdout:5/204: mkdir de/d21/d45 0 2024-04-20T11:08:56.545 INFO:tasks.workunit.client.0.smithi117.stdout:4/198: chown d1/dd/d1a/f37 721410 1 2024-04-20T11:08:56.550 INFO:tasks.workunit.client.0.smithi117.stdout:1/154: write d6/d21/f23 [351466,964] 0 2024-04-20T11:08:56.550 INFO:tasks.workunit.client.0.smithi117.stdout:9/183: creat d4/de/d10/f48 x:0 0 0 2024-04-20T11:08:56.552 INFO:tasks.workunit.client.0.smithi117.stdout:3/121: write d0/d5/f19 [1960135,119753] 0 2024-04-20T11:08:56.558 INFO:tasks.workunit.client.0.smithi117.stdout:2/168: rmdir d1/d20 39 2024-04-20T11:08:56.560 INFO:tasks.workunit.client.0.smithi117.stdout:7/196: chown d1/d4/d6/d13/l37 223394 1 2024-04-20T11:08:56.562 INFO:tasks.workunit.client.0.smithi117.stdout:0/152: rmdir d3/d7/d8/d32 0 2024-04-20T11:08:56.570 INFO:tasks.workunit.client.0.smithi117.stdout:0/153: truncate d3/d7/d16/f2b 275499 0 2024-04-20T11:08:56.572 INFO:tasks.workunit.client.0.smithi117.stdout:6/221: dwrite d3/f30 [0,4194304] 0 2024-04-20T11:08:56.574 INFO:tasks.workunit.client.0.smithi117.stdout:1/155: unlink d6/l14 0 2024-04-20T11:08:56.575 INFO:tasks.workunit.client.0.smithi117.stdout:1/156: fdatasync d6/f2e 0 2024-04-20T11:08:56.581 INFO:tasks.workunit.client.0.smithi117.stdout:5/205: dwrite de/d21/d35/f3a [0,4194304] 0 2024-04-20T11:08:56.586 INFO:tasks.workunit.client.0.smithi117.stdout:9/184: creat d4/d5/da/f49 x:0 0 0 2024-04-20T11:08:56.587 INFO:tasks.workunit.client.0.smithi117.stdout:3/122: mkdir d0/d5/de/d1d/d29 0 2024-04-20T11:08:56.589 INFO:tasks.workunit.client.0.smithi117.stdout:7/197: dwrite d1/d1d/f21 [0,4194304] 0 2024-04-20T11:08:56.599 INFO:tasks.workunit.client.0.smithi117.stdout:2/169: rename d1/d15/d1d/l22 to d1/d4/dd/de/l33 0 2024-04-20T11:08:56.601 INFO:tasks.workunit.client.0.smithi117.stdout:2/170: chown d1/d15/d1d/c2f 334 1 2024-04-20T11:08:56.618 INFO:tasks.workunit.client.0.smithi117.stdout:4/199: creat d1/dd/d1a/f48 x:0 0 0 2024-04-20T11:08:56.619 INFO:tasks.workunit.client.0.smithi117.stdout:2/171: fsync d1/d20/f21 0 2024-04-20T11:08:56.621 INFO:tasks.workunit.client.0.smithi117.stdout:1/157: creat d6/d35/f36 x:0 0 0 2024-04-20T11:08:56.622 INFO:tasks.workunit.client.0.smithi117.stdout:5/206: getdents de/d41 0 2024-04-20T11:08:56.623 INFO:tasks.workunit.client.0.smithi117.stdout:9/185: mkdir d4/d4a 0 2024-04-20T11:08:56.625 INFO:tasks.workunit.client.0.smithi117.stdout:3/123: mkdir d0/d2a 0 2024-04-20T11:08:56.625 INFO:tasks.workunit.client.0.smithi117.stdout:2/172: stat d1/d4/dd/de/d29 0 2024-04-20T11:08:56.626 INFO:tasks.workunit.client.0.smithi117.stdout:8/105: dwrite d0/fd [0,4194304] 0 2024-04-20T11:08:56.626 INFO:tasks.workunit.client.0.smithi117.stdout:1/158: chown d6/f1a 518860729 1 2024-04-20T11:08:56.629 INFO:tasks.workunit.client.0.smithi117.stdout:3/124: read d0/d5/fc [1579215,78683] 0 2024-04-20T11:08:56.633 INFO:tasks.workunit.client.0.smithi117.stdout:6/222: link d3/l32 d3/d4/d2e/d37/l49 0 2024-04-20T11:08:56.633 INFO:tasks.workunit.client.0.smithi117.stdout:2/173: write d1/d4/dd/f2b [96184,102035] 0 2024-04-20T11:08:56.638 INFO:tasks.workunit.client.0.smithi117.stdout:1/159: write d6/f10 [849266,88727] 0 2024-04-20T11:08:56.641 INFO:tasks.workunit.client.0.smithi117.stdout:3/125: creat d0/d5/d7/dd/f2b x:0 0 0 2024-04-20T11:08:56.643 INFO:tasks.workunit.client.0.smithi117.stdout:1/160: read d6/f1a [3515759,32954] 0 2024-04-20T11:08:56.645 INFO:tasks.workunit.client.0.smithi117.stdout:1/161: fsync d6/d1f/f24 0 2024-04-20T11:08:56.646 INFO:tasks.workunit.client.0.smithi117.stdout:9/186: mkdir d4/de/d4b 0 2024-04-20T11:08:56.650 INFO:tasks.workunit.client.0.smithi117.stdout:4/200: dwrite d1/d9/f3b [0,4194304] 0 2024-04-20T11:08:56.650 INFO:tasks.workunit.client.0.smithi117.stdout:9/187: chown d4/d5/da/f29 88819836 1 2024-04-20T11:08:56.651 INFO:tasks.workunit.client.0.smithi117.stdout:9/188: chown d4/d5/c1c 50 1 2024-04-20T11:08:56.660 INFO:tasks.workunit.client.0.smithi117.stdout:2/174: creat d1/d4/dd/de/f34 x:0 0 0 2024-04-20T11:08:56.662 INFO:tasks.workunit.client.0.smithi117.stdout:8/106: symlink d0/l21 0 2024-04-20T11:08:56.664 INFO:tasks.workunit.client.0.smithi117.stdout:5/207: creat de/f46 x:0 0 0 2024-04-20T11:08:56.666 INFO:tasks.workunit.client.0.smithi117.stdout:4/201: mknod d1/d9/d1f/c49 0 2024-04-20T11:08:56.669 INFO:tasks.workunit.client.0.smithi117.stdout:9/189: mknod d4/d5/da/c4c 0 2024-04-20T11:08:56.671 INFO:tasks.workunit.client.0.smithi117.stdout:2/175: rmdir d1/d20 39 2024-04-20T11:08:56.671 INFO:tasks.workunit.client.0.smithi117.stdout:1/162: dwrite d6/f12 [0,4194304] 0 2024-04-20T11:08:56.672 INFO:tasks.workunit.client.0.smithi117.stdout:2/176: stat d1/f11 0 2024-04-20T11:08:56.673 INFO:tasks.workunit.client.0.smithi117.stdout:1/163: chown d6/d1f/c2c 55021704 1 2024-04-20T11:08:56.674 INFO:tasks.workunit.client.0.smithi117.stdout:2/177: readlink d1/d4/dd/l32 0 2024-04-20T11:08:56.688 INFO:tasks.workunit.client.0.smithi117.stdout:1/164: dwrite f2 [4194304,4194304] 0 2024-04-20T11:08:56.814 INFO:tasks.workunit.client.0.smithi117.stdout:9/190: rename d4/d5/da/c4c to d4/de/d4b/c4d 0 2024-04-20T11:08:56.815 INFO:tasks.workunit.client.0.smithi117.stdout:9/191: stat d4/de/l16 0 2024-04-20T11:08:56.823 INFO:tasks.workunit.client.0.smithi117.stdout:1/165: unlink d6/d21/f23 0 2024-04-20T11:08:56.827 INFO:tasks.workunit.client.0.smithi117.stdout:5/208: mkdir de/d41/d47 0 2024-04-20T11:08:56.831 INFO:tasks.workunit.client.0.smithi117.stdout:4/202: mkdir d1/d45/d4a 0 2024-04-20T11:08:56.836 INFO:tasks.workunit.client.0.smithi117.stdout:8/107: creat d0/f22 x:0 0 0 2024-04-20T11:08:56.847 INFO:tasks.workunit.client.0.smithi117.stdout:5/209: write de/d21/d35/f3e [4686395,65014] 0 2024-04-20T11:08:56.851 INFO:tasks.workunit.client.0.smithi117.stdout:2/178: rename d1/c1a to d1/d15/c35 0 2024-04-20T11:08:56.855 INFO:tasks.workunit.client.0.smithi117.stdout:2/179: truncate d1/d4/dd/f19 1031152 0 2024-04-20T11:08:56.869 INFO:tasks.workunit.client.0.smithi117.stdout:5/210: mkdir de/d21/d45/d48 0 2024-04-20T11:08:56.870 INFO:tasks.workunit.client.0.smithi117.stdout:5/211: fdatasync de/f29 0 2024-04-20T11:08:56.872 INFO:tasks.workunit.client.0.smithi117.stdout:4/203: link d1/d9/f2a d1/dd/f4b 0 2024-04-20T11:08:56.874 INFO:tasks.workunit.client.0.smithi117.stdout:4/204: stat d1/dd/d1a/c32 0 2024-04-20T11:08:56.874 INFO:tasks.workunit.client.0.smithi117.stdout:9/192: getdents d4/d30/d31 0 2024-04-20T11:08:56.876 INFO:tasks.workunit.client.0.smithi117.stdout:5/212: dread de/d21/d35/f3b [0,4194304] 0 2024-04-20T11:08:56.876 INFO:tasks.workunit.client.0.smithi117.stdout:8/108: creat d0/f23 x:0 0 0 2024-04-20T11:08:56.878 INFO:tasks.workunit.client.0.smithi117.stdout:2/180: dread d1/d4/f5 [0,4194304] 0 2024-04-20T11:08:56.879 INFO:tasks.workunit.client.0.smithi117.stdout:1/166: getdents d6 0 2024-04-20T11:08:56.880 INFO:tasks.workunit.client.0.smithi117.stdout:8/109: dread - d0/d8/d11/f14 zero size 2024-04-20T11:08:56.883 INFO:tasks.workunit.client.0.smithi117.stdout:2/181: read d1/d4/dd/d17/f30 [2523048,14110] 0 2024-04-20T11:08:56.883 INFO:tasks.workunit.client.0.smithi117.stdout:8/110: fdatasync d0/d8/f16 0 2024-04-20T11:08:56.885 INFO:tasks.workunit.client.0.smithi117.stdout:9/193: rename d4/de/d10/f48 to d4/d30/f4e 0 2024-04-20T11:08:56.886 INFO:tasks.workunit.client.0.smithi117.stdout:1/167: symlink d6/d16/l37 0 2024-04-20T11:08:56.887 INFO:tasks.workunit.client.0.smithi117.stdout:8/111: truncate d0/d9/f17 4353657 0 2024-04-20T11:08:56.890 INFO:tasks.workunit.client.0.smithi117.stdout:1/168: read d6/d1f/f24 [145674,39397] 0 2024-04-20T11:08:56.890 INFO:tasks.workunit.client.0.smithi117.stdout:8/112: write d0/fd [1046530,107759] 0 2024-04-20T11:08:56.894 INFO:tasks.workunit.client.0.smithi117.stdout:5/213: rename de/d21/f30 to de/d21/f49 0 2024-04-20T11:08:56.897 INFO:tasks.workunit.client.0.smithi117.stdout:4/205: dwrite d1/f25 [0,4194304] 0 2024-04-20T11:08:56.906 INFO:tasks.workunit.client.0.smithi117.stdout:1/169: mknod d6/d1f/d30/c38 0 2024-04-20T11:08:56.908 INFO:tasks.workunit.client.0.smithi117.stdout:2/182: creat d1/d20/f36 x:0 0 0 2024-04-20T11:08:56.908 INFO:tasks.workunit.client.0.smithi117.stdout:6/223: dwrite d3/df/f31 [0,4194304] 0 2024-04-20T11:08:56.908 INFO:tasks.workunit.client.0.smithi117.stdout:8/113: mkdir d0/d8/d11/d24 0 2024-04-20T11:08:56.909 INFO:tasks.workunit.client.0.smithi117.stdout:2/183: chown d1/d20/d26 165 1 2024-04-20T11:08:56.910 INFO:tasks.workunit.client.0.smithi117.stdout:8/114: chown d0/c15 0 1 2024-04-20T11:08:56.911 INFO:tasks.workunit.client.0.smithi117.stdout:6/224: write d3/df/f21 [1553764,113603] 0 2024-04-20T11:08:56.913 INFO:tasks.workunit.client.0.smithi117.stdout:8/115: chown d0/dc/f1e 15832314 1 2024-04-20T11:08:56.916 INFO:tasks.workunit.client.0.smithi117.stdout:2/184: dread d1/d4/dd/f19 [0,4194304] 0 2024-04-20T11:08:56.925 INFO:tasks.workunit.client.0.smithi117.stdout:4/206: rmdir d1 39 2024-04-20T11:08:56.928 INFO:tasks.workunit.client.0.smithi117.stdout:5/214: creat de/d21/d45/d48/f4a x:0 0 0 2024-04-20T11:08:56.930 INFO:tasks.workunit.client.0.smithi117.stdout:3/126: sync 2024-04-20T11:08:56.930 INFO:tasks.workunit.client.0.smithi117.stdout:0/154: sync 2024-04-20T11:08:56.930 INFO:tasks.workunit.client.0.smithi117.stdout:1/170: rename d6/f2a to d6/d21/f39 0 2024-04-20T11:08:56.930 INFO:tasks.workunit.client.0.smithi117.stdout:7/198: sync 2024-04-20T11:08:56.932 INFO:tasks.workunit.client.0.smithi117.stdout:3/127: chown d0/d5/d11/c14 11464659 1 2024-04-20T11:08:56.933 INFO:tasks.workunit.client.0.smithi117.stdout:0/155: chown d3/d5/f26 886720447 1 2024-04-20T11:08:56.933 INFO:tasks.workunit.client.0.smithi117.stdout:6/225: chown d3/d4/c22 101 1 2024-04-20T11:08:56.935 INFO:tasks.workunit.client.0.smithi117.stdout:3/128: write d0/d5/d7/f18 [71781,121609] 0 2024-04-20T11:08:56.936 INFO:tasks.workunit.client.0.smithi117.stdout:8/116: rmdir d0/dc 39 2024-04-20T11:08:56.938 INFO:tasks.workunit.client.0.smithi117.stdout:8/117: readlink d0/d8/lf 0 2024-04-20T11:08:56.940 INFO:tasks.workunit.client.0.smithi117.stdout:3/129: write d0/d5/de/d1d/f1f [215374,22357] 0 2024-04-20T11:08:56.941 INFO:tasks.workunit.client.0.smithi117.stdout:5/215: mkdir de/d41/d4b 0 2024-04-20T11:08:56.942 INFO:tasks.workunit.client.0.smithi117.stdout:3/130: write d0/d5/d7/f20 [525497,116645] 0 2024-04-20T11:08:56.942 INFO:tasks.workunit.client.0.smithi117.stdout:4/207: truncate d1/dd/d1a/f2e 906662 0 2024-04-20T11:08:56.943 INFO:tasks.workunit.client.0.smithi117.stdout:0/156: mkdir d3/d7/d20/d39 0 2024-04-20T11:08:56.945 INFO:tasks.workunit.client.0.smithi117.stdout:2/185: link d1/d4/dd/f19 d1/d20/d26/f37 0 2024-04-20T11:08:56.946 INFO:tasks.workunit.client.0.smithi117.stdout:3/131: fsync d0/d5/d7/dd/f10 0 2024-04-20T11:08:56.955 INFO:tasks.workunit.client.0.smithi117.stdout:2/186: creat d1/d4/dd/de/f38 x:0 0 0 2024-04-20T11:08:56.956 INFO:tasks.workunit.client.0.smithi117.stdout:9/194: truncate d4/d5/f28 346178 0 2024-04-20T11:08:56.958 INFO:tasks.workunit.client.0.smithi117.stdout:8/118: creat d0/dc/f25 x:0 0 0 2024-04-20T11:08:56.959 INFO:tasks.workunit.client.0.smithi117.stdout:5/216: mkdir de/d41/d47/d4c 0 2024-04-20T11:08:56.961 INFO:tasks.workunit.client.0.smithi117.stdout:0/157: mkdir d3/d7/d20/d39/d3a 0 2024-04-20T11:08:56.962 INFO:tasks.workunit.client.0.smithi117.stdout:6/226: dwrite d3/f28 [0,4194304] 0 2024-04-20T11:08:56.964 INFO:tasks.workunit.client.0.smithi117.stdout:4/208: creat d1/dd/d1a/d2b/f4c x:0 0 0 2024-04-20T11:08:56.966 INFO:tasks.workunit.client.0.smithi117.stdout:9/195: mkdir d4/d35/d4f 0 2024-04-20T11:08:56.969 INFO:tasks.workunit.client.0.smithi117.stdout:1/171: write d6/f9 [3324579,24634] 0 2024-04-20T11:08:56.969 INFO:tasks.workunit.client.0.smithi117.stdout:8/119: mknod d0/d8/d11/c26 0 2024-04-20T11:08:56.975 INFO:tasks.workunit.client.0.smithi117.stdout:8/120: write d0/d8/d11/f14 [264506,106882] 0 2024-04-20T11:08:56.976 INFO:tasks.workunit.client.0.smithi117.stdout:7/199: truncate d1/d4/d6/d2b/f33 1969289 0 2024-04-20T11:08:56.979 INFO:tasks.workunit.client.0.smithi117.stdout:2/187: dwrite d1/d4/fb [0,4194304] 0 2024-04-20T11:08:56.980 INFO:tasks.workunit.client.0.smithi117.stdout:6/227: rename d3/d4/d2e/d35 to d3/d4/d2e/d37/d4a 0 2024-04-20T11:08:56.980 INFO:tasks.workunit.client.0.smithi117.stdout:8/121: dread - d0/d8/d11/f1c zero size 2024-04-20T11:08:56.986 INFO:tasks.workunit.client.0.smithi117.stdout:3/132: getdents d0/d5/d21 0 2024-04-20T11:08:56.987 INFO:tasks.workunit.client.0.smithi117.stdout:9/196: mknod d4/c50 0 2024-04-20T11:08:56.990 INFO:tasks.workunit.client.0.smithi117.stdout:0/158: creat d3/d7/f3b x:0 0 0 2024-04-20T11:08:56.991 INFO:tasks.workunit.client.0.smithi117.stdout:4/209: dwrite d1/dd/f13 [0,4194304] 0 2024-04-20T11:08:56.991 INFO:tasks.workunit.client.0.smithi117.stdout:2/188: fsync d1/d4/dd/d17/f30 0 2024-04-20T11:08:56.995 INFO:tasks.workunit.client.0.smithi117.stdout:6/228: write d3/f18 [2048443,113574] 0 2024-04-20T11:08:56.999 INFO:tasks.workunit.client.0.smithi117.stdout:1/172: getdents d6 0 2024-04-20T11:08:56.999 INFO:tasks.workunit.client.0.smithi117.stdout:4/210: truncate d1/f25 4990855 0 2024-04-20T11:08:57.001 INFO:tasks.workunit.client.0.smithi117.stdout:5/217: write de/d21/f3c [584549,48007] 0 2024-04-20T11:08:57.004 INFO:tasks.workunit.client.0.smithi117.stdout:8/122: mkdir d0/d8/d11/d24/d27 0 2024-04-20T11:08:57.005 INFO:tasks.workunit.client.0.smithi117.stdout:0/159: mkdir d3/d7/d8/d3c 0 2024-04-20T11:08:57.006 INFO:tasks.workunit.client.0.smithi117.stdout:7/200: getdents d1/d4/d39 0 2024-04-20T11:08:57.007 INFO:tasks.workunit.client.0.smithi117.stdout:6/229: creat d3/df/d39/f4b x:0 0 0 2024-04-20T11:08:57.007 INFO:tasks.workunit.client.0.smithi117.stdout:0/160: read - d3/d7/f3b zero size 2024-04-20T11:08:57.013 INFO:tasks.workunit.client.0.smithi117.stdout:2/189: write d1/d20/d26/f37 [930536,83486] 0 2024-04-20T11:08:57.014 INFO:tasks.workunit.client.0.smithi117.stdout:5/218: readlink de/l2a 0 2024-04-20T11:08:57.017 INFO:tasks.workunit.client.0.smithi117.stdout:9/197: rmdir d4/d35/d4f 0 2024-04-20T11:08:57.020 INFO:tasks.workunit.client.0.smithi117.stdout:4/211: mknod d1/dd/d1a/c4d 0 2024-04-20T11:08:57.021 INFO:tasks.workunit.client.0.smithi117.stdout:7/201: chown d1/d4/d39/d1f/d2d/c3c 438928754 1 2024-04-20T11:08:57.024 INFO:tasks.workunit.client.0.smithi117.stdout:4/212: chown d1/d45/d4a 215 1 2024-04-20T11:08:57.025 INFO:tasks.workunit.client.0.smithi117.stdout:5/219: rmdir de/d16 39 2024-04-20T11:08:57.025 INFO:tasks.workunit.client.0.smithi117.stdout:2/190: dread d1/d4/dd/d17/f1e [0,4194304] 0 2024-04-20T11:08:57.027 INFO:tasks.workunit.client.0.smithi117.stdout:9/198: mkdir d4/d5/da/d51 0 2024-04-20T11:08:57.028 INFO:tasks.workunit.client.0.smithi117.stdout:5/220: chown c7 443872285 1 2024-04-20T11:08:57.029 INFO:tasks.workunit.client.0.smithi117.stdout:0/161: symlink d3/d7/d20/d39/d3a/l3d 0 2024-04-20T11:08:57.030 INFO:tasks.workunit.client.0.smithi117.stdout:7/202: symlink d1/d4/d6/l3e 0 2024-04-20T11:08:57.031 INFO:tasks.workunit.client.0.smithi117.stdout:3/133: link d0/f3 d0/f2c 0 2024-04-20T11:08:57.032 INFO:tasks.workunit.client.0.smithi117.stdout:8/123: getdents d0/d8 0 2024-04-20T11:08:57.034 INFO:tasks.workunit.client.0.smithi117.stdout:6/230: creat d3/f4c x:0 0 0 2024-04-20T11:08:57.034 INFO:tasks.workunit.client.0.smithi117.stdout:4/213: write d1/dd/f13 [2868626,61793] 0 2024-04-20T11:08:57.037 INFO:tasks.workunit.client.0.smithi117.stdout:2/191: write d1/d4/f5 [4173881,90637] 0 2024-04-20T11:08:57.040 INFO:tasks.workunit.client.0.smithi117.stdout:2/192: chown d1/d15 1 1 2024-04-20T11:08:57.048 INFO:tasks.workunit.client.0.smithi117.stdout:6/231: write d3/d4/f1b [8269260,63075] 0 2024-04-20T11:08:57.050 INFO:tasks.workunit.client.0.smithi117.stdout:9/199: fdatasync d4/de/d10/f1a 0 2024-04-20T11:08:57.051 INFO:tasks.workunit.client.0.smithi117.stdout:1/173: dread d6/f9 [0,4194304] 0 2024-04-20T11:08:57.052 INFO:tasks.workunit.client.0.smithi117.stdout:8/124: mknod d0/d8/d11/d24/d27/c28 0 2024-04-20T11:08:57.054 INFO:tasks.workunit.client.0.smithi117.stdout:9/200: fsync d4/d5/f2b 0 2024-04-20T11:08:57.054 INFO:tasks.workunit.client.0.smithi117.stdout:4/214: write d1/f6 [29003,84119] 0 2024-04-20T11:08:57.055 INFO:tasks.workunit.client.0.smithi117.stdout:2/193: unlink d1/d4/dd/f2b 0 2024-04-20T11:08:57.058 INFO:tasks.workunit.client.0.smithi117.stdout:6/232: creat d3/d3a/f4d x:0 0 0 2024-04-20T11:08:57.060 INFO:tasks.workunit.client.0.smithi117.stdout:3/134: dwrite d0/d5/d7/f18 [0,4194304] 0 2024-04-20T11:08:57.061 INFO:tasks.workunit.client.0.smithi117.stdout:1/174: mkdir d6/d16/d3a 0 2024-04-20T11:08:57.063 INFO:tasks.workunit.client.0.smithi117.stdout:3/135: fdatasync d0/d5/d7/f20 0 2024-04-20T11:08:57.064 INFO:tasks.workunit.client.0.smithi117.stdout:8/125: mkdir d0/d9/d29 0 2024-04-20T11:08:57.065 INFO:tasks.workunit.client.0.smithi117.stdout:5/221: dwrite de/d16/f2d [0,4194304] 0 2024-04-20T11:08:57.068 INFO:tasks.workunit.client.0.smithi117.stdout:0/162: getdents d3 0 2024-04-20T11:08:57.072 INFO:tasks.workunit.client.0.smithi117.stdout:6/233: mkdir d3/d4/d2e/d4e 0 2024-04-20T11:08:57.074 INFO:tasks.workunit.client.0.smithi117.stdout:2/194: symlink d1/l39 0 2024-04-20T11:08:57.074 INFO:tasks.workunit.client.0.smithi117.stdout:8/126: creat d0/d9/f2a x:0 0 0 2024-04-20T11:08:57.076 INFO:tasks.workunit.client.0.smithi117.stdout:1/175: mkdir d6/d35/d3b 0 2024-04-20T11:08:57.076 INFO:tasks.workunit.client.0.smithi117.stdout:4/215: dread d1/dd/f4b [0,4194304] 0 2024-04-20T11:08:57.082 INFO:tasks.workunit.client.0.smithi117.stdout:4/216: dread - d1/d9/f16 zero size 2024-04-20T11:08:57.088 INFO:tasks.workunit.client.0.smithi117.stdout:2/195: creat d1/d4/dd/de/f3a x:0 0 0 2024-04-20T11:08:57.094 INFO:tasks.workunit.client.0.smithi117.stdout:0/163: dwrite d3/d7/d20/f25 [0,4194304] 0 2024-04-20T11:08:57.094 INFO:tasks.workunit.client.0.smithi117.stdout:5/222: dwrite de/d21/f2e [0,4194304] 0 2024-04-20T11:08:57.095 INFO:tasks.workunit.client.0.smithi117.stdout:8/127: mkdir d0/d8/d11/d24/d2b 0 2024-04-20T11:08:57.096 INFO:tasks.workunit.client.0.smithi117.stdout:4/217: mkdir d1/dd/d1a/d43/d4e 0 2024-04-20T11:08:57.097 INFO:tasks.workunit.client.0.smithi117.stdout:9/201: rename d4/de/d10/l11 to d4/l52 0 2024-04-20T11:08:57.101 INFO:tasks.workunit.client.0.smithi117.stdout:2/196: mkdir d1/d4/dd/d17/d3b 0 2024-04-20T11:08:57.104 INFO:tasks.workunit.client.0.smithi117.stdout:9/202: dread - d4/d5/da/f43 zero size 2024-04-20T11:08:57.104 INFO:tasks.workunit.client.0.smithi117.stdout:0/164: creat d3/f3e x:0 0 0 2024-04-20T11:08:57.105 INFO:tasks.workunit.client.0.smithi117.stdout:5/223: fsync de/f28 0 2024-04-20T11:08:57.106 INFO:tasks.workunit.client.0.smithi117.stdout:3/136: getdents d0/d5 0 2024-04-20T11:08:57.109 INFO:tasks.workunit.client.0.smithi117.stdout:5/224: chown c8 1731098377 1 2024-04-20T11:08:57.111 INFO:tasks.workunit.client.0.smithi117.stdout:8/128: creat d0/d8/d11/d24/d2b/f2c x:0 0 0 2024-04-20T11:08:57.111 INFO:tasks.workunit.client.0.smithi117.stdout:4/218: write d1/d9/d1f/f36 [1029186,45169] 0 2024-04-20T11:08:57.114 INFO:tasks.workunit.client.0.smithi117.stdout:3/137: mkdir d0/d5/d7/d2d 0 2024-04-20T11:08:57.116 INFO:tasks.workunit.client.0.smithi117.stdout:2/197: write d1/d4/dd/d17/f1f [4224013,75883] 0 2024-04-20T11:08:57.119 INFO:tasks.workunit.client.0.smithi117.stdout:4/219: write d1/f20 [1090312,118276] 0 2024-04-20T11:08:57.123 INFO:tasks.workunit.client.0.smithi117.stdout:5/225: truncate de/d21/d45/d48/f4a 539655 0 2024-04-20T11:08:57.127 INFO:tasks.workunit.client.0.smithi117.stdout:3/138: symlink d0/d5/d7/l2e 0 2024-04-20T11:08:57.130 INFO:tasks.workunit.client.0.smithi117.stdout:9/203: fsync d4/d5/da/f19 0 2024-04-20T11:08:57.133 INFO:tasks.workunit.client.0.smithi117.stdout:6/234: dwrite d3/fc [0,4194304] 0 2024-04-20T11:08:57.133 INFO:tasks.workunit.client.0.smithi117.stdout:2/198: symlink d1/d4/dd/de/d29/d2e/l3c 0 2024-04-20T11:08:57.137 INFO:tasks.workunit.client.0.smithi117.stdout:1/176: dwrite d6/f12 [0,4194304] 0 2024-04-20T11:08:57.138 INFO:tasks.workunit.client.0.smithi117.stdout:8/129: fdatasync d0/f7 0 2024-04-20T11:08:57.140 INFO:tasks.workunit.client.0.smithi117.stdout:1/177: chown d6/d1f 2571 1 2024-04-20T11:08:57.141 INFO:tasks.workunit.client.0.smithi117.stdout:6/235: write d3/d4/f19 [1693565,124914] 0 2024-04-20T11:08:57.143 INFO:tasks.workunit.client.0.smithi117.stdout:8/130: write d0/fd [3206213,83258] 0 2024-04-20T11:08:57.145 INFO:tasks.workunit.client.0.smithi117.stdout:1/178: write d6/f2e [781887,101654] 0 2024-04-20T11:08:57.145 INFO:tasks.workunit.client.0.smithi117.stdout:2/199: chown d1/d4/dd/de/f18 961 1 2024-04-20T11:08:57.149 INFO:tasks.workunit.client.0.smithi117.stdout:6/236: truncate d3/f4c 770253 0 2024-04-20T11:08:57.153 INFO:tasks.workunit.client.0.smithi117.stdout:3/139: symlink d0/d5/de/d1d/d29/l2f 0 2024-04-20T11:08:57.156 INFO:tasks.workunit.client.0.smithi117.stdout:4/220: dread d1/f35 [0,4194304] 0 2024-04-20T11:08:57.157 INFO:tasks.workunit.client.0.smithi117.stdout:6/237: creat d3/df/f4f x:0 0 0 2024-04-20T11:08:57.162 INFO:tasks.workunit.client.0.smithi117.stdout:2/200: unlink d1/d20/f36 0 2024-04-20T11:08:57.164 INFO:tasks.workunit.client.0.smithi117.stdout:1/179: link d6/d1f/f24 d6/f3c 0 2024-04-20T11:08:57.164 INFO:tasks.workunit.client.0.smithi117.stdout:6/238: dread d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:57.165 INFO:tasks.workunit.client.0.smithi117.stdout:9/204: dwrite d4/d5/f2a [0,4194304] 0 2024-04-20T11:08:57.170 INFO:tasks.workunit.client.0.smithi117.stdout:2/201: chown d1/d4/dd/de/f3a 4 1 2024-04-20T11:08:57.171 INFO:tasks.workunit.client.0.smithi117.stdout:8/131: getdents d0/d8 0 2024-04-20T11:08:57.173 INFO:tasks.workunit.client.0.smithi117.stdout:2/202: write d1/d4/f5 [70289,53649] 0 2024-04-20T11:08:57.176 INFO:tasks.workunit.client.0.smithi117.stdout:5/226: truncate de/d21/d35/f3e 1874194 0 2024-04-20T11:08:57.179 INFO:tasks.workunit.client.0.smithi117.stdout:0/165: creat d3/d7/f3f x:0 0 0 2024-04-20T11:08:57.182 INFO:tasks.workunit.client.0.smithi117.stdout:5/227: read de/d16/f1e [560282,100505] 0 2024-04-20T11:08:57.184 INFO:tasks.workunit.client.0.smithi117.stdout:0/166: read d3/d7/d8/f13 [2839224,43494] 0 2024-04-20T11:08:57.185 INFO:tasks.workunit.client.0.smithi117.stdout:3/140: getdents d0/d5/de 0 2024-04-20T11:08:57.187 INFO:tasks.workunit.client.0.smithi117.stdout:5/228: write de/f19 [92105,82167] 0 2024-04-20T11:08:57.187 INFO:tasks.workunit.client.0.smithi117.stdout:1/180: creat d6/d1f/f3d x:0 0 0 2024-04-20T11:08:57.187 INFO:tasks.workunit.client.0.smithi117.stdout:8/132: link d0/l6 d0/d9/l2d 0 2024-04-20T11:08:57.188 INFO:tasks.workunit.client.0.smithi117.stdout:6/239: truncate d3/d4/f8 84693 0 2024-04-20T11:08:57.192 INFO:tasks.workunit.client.0.smithi117.stdout:4/221: dwrite d1/dd/d1a/d2b/f46 [0,4194304] 0 2024-04-20T11:08:57.192 INFO:tasks.workunit.client.0.smithi117.stdout:0/167: dread - d3/d7/d8/f31 zero size 2024-04-20T11:08:57.199 INFO:tasks.workunit.client.0.smithi117.stdout:1/181: fsync d6/d21/f31 0 2024-04-20T11:08:57.204 INFO:tasks.workunit.client.0.smithi117.stdout:4/222: creat d1/d45/f4f x:0 0 0 2024-04-20T11:08:57.207 INFO:tasks.workunit.client.0.smithi117.stdout:3/141: rename d0/d5/d7/f27 to d0/d5/de/d1d/d29/f30 0 2024-04-20T11:08:57.212 INFO:tasks.workunit.client.0.smithi117.stdout:1/182: creat d6/d1f/f3e x:0 0 0 2024-04-20T11:08:57.213 INFO:tasks.workunit.client.0.smithi117.stdout:0/168: mknod d3/d7/d8/d2a/c40 0 2024-04-20T11:08:57.214 INFO:tasks.workunit.client.0.smithi117.stdout:2/203: dwrite d1/f3 [0,4194304] 0 2024-04-20T11:08:57.218 INFO:tasks.workunit.client.0.smithi117.stdout:1/183: dread - d6/d1f/f2b zero size 2024-04-20T11:08:57.218 INFO:tasks.workunit.client.0.smithi117.stdout:4/223: creat d1/d45/d4a/f50 x:0 0 0 2024-04-20T11:08:57.223 INFO:tasks.workunit.client.0.smithi117.stdout:0/169: symlink d3/d7/d20/d39/l41 0 2024-04-20T11:08:57.225 INFO:tasks.workunit.client.0.smithi117.stdout:8/133: dread d0/f5 [0,4194304] 0 2024-04-20T11:08:57.227 INFO:tasks.workunit.client.0.smithi117.stdout:8/134: chown d0/d9 6875507 1 2024-04-20T11:08:57.229 INFO:tasks.workunit.client.0.smithi117.stdout:1/184: unlink d6/d21/c27 0 2024-04-20T11:08:57.232 INFO:tasks.workunit.client.0.smithi117.stdout:5/229: dwrite de/d21/f49 [0,4194304] 0 2024-04-20T11:08:57.234 INFO:tasks.workunit.client.0.smithi117.stdout:9/205: dwrite d4/de/d10/f1a [0,4194304] 0 2024-04-20T11:08:57.238 INFO:tasks.workunit.client.0.smithi117.stdout:4/224: write d1/dd/d1a/f1d [75712,122886] 0 2024-04-20T11:08:57.239 INFO:tasks.workunit.client.0.smithi117.stdout:8/135: mkdir d0/dc/d2e 0 2024-04-20T11:08:57.239 INFO:tasks.workunit.client.0.smithi117.stdout:6/240: dwrite d3/d4/d2e/d3f/f42 [0,4194304] 0 2024-04-20T11:08:57.240 INFO:tasks.workunit.client.0.smithi117.stdout:4/225: write d1/d9/d1f/f36 [773004,118356] 0 2024-04-20T11:08:57.243 INFO:tasks.workunit.client.0.smithi117.stdout:4/226: chown d1/dd/d1a/c32 1064 1 2024-04-20T11:08:57.247 INFO:tasks.workunit.client.0.smithi117.stdout:5/230: creat de/d41/d47/f4d x:0 0 0 2024-04-20T11:08:57.251 INFO:tasks.workunit.client.0.smithi117.stdout:4/227: write d1/dd/d1a/d2b/f46 [4339131,87365] 0 2024-04-20T11:08:57.252 INFO:tasks.workunit.client.0.smithi117.stdout:1/185: rename d6/l11 to d6/d21/l3f 0 2024-04-20T11:08:57.255 INFO:tasks.workunit.client.0.smithi117.stdout:8/136: dread d0/f1 [4194304,4194304] 0 2024-04-20T11:08:57.256 INFO:tasks.workunit.client.0.smithi117.stdout:8/137: chown d0/d8/lf 496 1 2024-04-20T11:08:57.257 INFO:tasks.workunit.client.0.smithi117.stdout:5/231: mkdir de/d21/d45/d48/d4e 0 2024-04-20T11:08:57.259 INFO:tasks.workunit.client.0.smithi117.stdout:3/142: truncate d0/d5/d21/f28 361699 0 2024-04-20T11:08:57.260 INFO:tasks.workunit.client.0.smithi117.stdout:6/241: mkdir d3/d50 0 2024-04-20T11:08:57.265 INFO:tasks.workunit.client.0.smithi117.stdout:0/170: write d3/d7/d16/f2e [2983800,16442] 0 2024-04-20T11:08:57.267 INFO:tasks.workunit.client.0.smithi117.stdout:2/204: truncate d1/f11 2540410 0 2024-04-20T11:08:57.267 INFO:tasks.workunit.client.0.smithi117.stdout:4/228: chown d1/d9/cc 3020237 1 2024-04-20T11:08:57.268 INFO:tasks.workunit.client.0.smithi117.stdout:7/203: sync 2024-04-20T11:08:57.269 INFO:tasks.workunit.client.0.smithi117.stdout:0/171: dread - d3/d7/f14 zero size 2024-04-20T11:08:57.271 INFO:tasks.workunit.client.0.smithi117.stdout:2/205: write d1/d4/dd/d17/f1f [4701765,70497] 0 2024-04-20T11:08:57.275 INFO:tasks.workunit.client.0.smithi117.stdout:9/206: rename d4/de/c18 to d4/d5/da/c53 0 2024-04-20T11:08:57.279 INFO:tasks.workunit.client.0.smithi117.stdout:4/229: mknod d1/d9/c51 0 2024-04-20T11:08:57.281 INFO:tasks.workunit.client.0.smithi117.stdout:0/172: mknod d3/d7/d8/c42 0 2024-04-20T11:08:57.282 INFO:tasks.workunit.client.0.smithi117.stdout:7/204: write d1/d4/d6/f26 [452202,486] 0 2024-04-20T11:08:57.285 INFO:tasks.workunit.client.0.smithi117.stdout:3/143: dwrite d0/d5/f13 [0,4194304] 0 2024-04-20T11:08:57.288 INFO:tasks.workunit.client.0.smithi117.stdout:7/205: write d1/d4/f1a [1307320,76809] 0 2024-04-20T11:08:57.288 INFO:tasks.workunit.client.0.smithi117.stdout:6/242: dwrite d3/d4/f1b [4194304,4194304] 0 2024-04-20T11:08:57.290 INFO:tasks.workunit.client.0.smithi117.stdout:3/144: dread - d0/d5/d7/f1e zero size 2024-04-20T11:08:57.291 INFO:tasks.workunit.client.0.smithi117.stdout:1/186: readlink d6/l1c 0 2024-04-20T11:08:57.292 INFO:tasks.workunit.client.0.smithi117.stdout:9/207: mkdir d4/d35/d54 0 2024-04-20T11:08:57.293 INFO:tasks.workunit.client.0.smithi117.stdout:3/145: write d0/d5/f19 [3156842,16766] 0 2024-04-20T11:08:57.298 INFO:tasks.workunit.client.0.smithi117.stdout:2/206: symlink d1/l3d 0 2024-04-20T11:08:57.305 INFO:tasks.workunit.client.0.smithi117.stdout:8/138: rename d0/d9/l2d to d0/d8/l2f 0 2024-04-20T11:08:57.311 INFO:tasks.workunit.client.0.smithi117.stdout:8/139: fsync d0/d9/fe 0 2024-04-20T11:08:57.312 INFO:tasks.workunit.client.0.smithi117.stdout:6/243: dread d3/d4/f1b [4194304,4194304] 0 2024-04-20T11:08:57.312 INFO:tasks.workunit.client.0.smithi117.stdout:4/230: mkdir d1/dd/d52 0 2024-04-20T11:08:57.315 INFO:tasks.workunit.client.0.smithi117.stdout:2/207: write d1/f3 [3137931,15385] 0 2024-04-20T11:08:57.316 INFO:tasks.workunit.client.0.smithi117.stdout:0/173: mknod d3/d7/d2c/c43 0 2024-04-20T11:08:57.316 INFO:tasks.workunit.client.0.smithi117.stdout:9/208: dwrite d4/de/f3e [0,4194304] 0 2024-04-20T11:08:57.319 INFO:tasks.workunit.client.0.smithi117.stdout:2/208: chown d1/d4/dd/de/f18 567172 1 2024-04-20T11:08:57.321 INFO:tasks.workunit.client.0.smithi117.stdout:5/232: rename de/d16/f1b to de/d21/d45/d48/d4e/f4f 0 2024-04-20T11:08:57.321 INFO:tasks.workunit.client.0.smithi117.stdout:9/209: write d4/de/f2d [796999,111927] 0 2024-04-20T11:08:57.327 INFO:tasks.workunit.client.0.smithi117.stdout:3/146: link d0/d5/d7/dd/ff d0/d5/de/d1d/d29/f31 0 2024-04-20T11:08:57.329 INFO:tasks.workunit.client.0.smithi117.stdout:8/140: mknod d0/d8/c30 0 2024-04-20T11:08:57.332 INFO:tasks.workunit.client.0.smithi117.stdout:2/209: read d1/d4/dd/f14 [1048461,78073] 0 2024-04-20T11:08:57.332 INFO:tasks.workunit.client.0.smithi117.stdout:8/141: dread - d0/dc/f25 zero size 2024-04-20T11:08:57.337 INFO:tasks.workunit.client.0.smithi117.stdout:7/206: write d1/d4/d6/d2b/f33 [1412238,16023] 0 2024-04-20T11:08:57.337 INFO:tasks.workunit.client.0.smithi117.stdout:5/233: unlink de/d21/f3c 0 2024-04-20T11:08:57.339 INFO:tasks.workunit.client.0.smithi117.stdout:8/142: write d0/d8/d11/f19 [66524,104964] 0 2024-04-20T11:08:57.342 INFO:tasks.workunit.client.0.smithi117.stdout:3/147: mknod d0/d5/d7/c32 0 2024-04-20T11:08:57.342 INFO:tasks.workunit.client.0.smithi117.stdout:0/174: dwrite d3/d7/d8/df/f1b [0,4194304] 0 2024-04-20T11:08:57.348 INFO:tasks.workunit.client.0.smithi117.stdout:8/143: fdatasync d0/f1 0 2024-04-20T11:08:57.351 INFO:tasks.workunit.client.0.smithi117.stdout:9/210: dwrite d4/de/d10/f1a [0,4194304] 0 2024-04-20T11:08:57.351 INFO:tasks.workunit.client.0.smithi117.stdout:5/234: creat de/d21/d45/d48/d4e/f50 x:0 0 0 2024-04-20T11:08:57.352 INFO:tasks.workunit.client.0.smithi117.stdout:4/231: creat d1/dd/f53 x:0 0 0 2024-04-20T11:08:57.355 INFO:tasks.workunit.client.0.smithi117.stdout:9/211: write d4/d5/da/f43 [476590,125634] 0 2024-04-20T11:08:57.355 INFO:tasks.workunit.client.0.smithi117.stdout:9/212: read - d4/f39 zero size 2024-04-20T11:08:57.358 INFO:tasks.workunit.client.0.smithi117.stdout:9/213: write d4/d30/f36 [2072609,40442] 0 2024-04-20T11:08:57.360 INFO:tasks.workunit.client.0.smithi117.stdout:1/187: getdents d6/d1f 0 2024-04-20T11:08:57.360 INFO:tasks.workunit.client.0.smithi117.stdout:9/214: readlink d4/d5/da/l2f 0 2024-04-20T11:08:57.365 INFO:tasks.workunit.client.0.smithi117.stdout:0/175: creat d3/d7/d8/f44 x:0 0 0 2024-04-20T11:08:57.366 INFO:tasks.workunit.client.0.smithi117.stdout:2/210: mkdir d1/d4/dd/d3e 0 2024-04-20T11:08:57.369 INFO:tasks.workunit.client.0.smithi117.stdout:8/144: rmdir d0/d8/d11/d24/d2b 39 2024-04-20T11:08:57.370 INFO:tasks.workunit.client.0.smithi117.stdout:4/232: dwrite d1/f7 [0,4194304] 0 2024-04-20T11:08:57.372 INFO:tasks.workunit.client.0.smithi117.stdout:3/148: dread d0/f2c [0,4194304] 0 2024-04-20T11:08:57.374 INFO:tasks.workunit.client.0.smithi117.stdout:1/188: creat d6/d1f/f40 x:0 0 0 2024-04-20T11:08:57.376 INFO:tasks.workunit.client.0.smithi117.stdout:6/244: rename d3/d4/l13 to d3/d4/l51 0 2024-04-20T11:08:57.380 INFO:tasks.workunit.client.0.smithi117.stdout:9/215: fdatasync d4/d5/f28 0 2024-04-20T11:08:57.386 INFO:tasks.workunit.client.0.smithi117.stdout:4/233: creat d1/dd/d52/f54 x:0 0 0 2024-04-20T11:08:57.386 INFO:tasks.workunit.client.0.smithi117.stdout:3/149: creat d0/d5/d7/d2d/f33 x:0 0 0 2024-04-20T11:08:57.389 INFO:tasks.workunit.client.0.smithi117.stdout:3/150: write d0/d5/f13 [4111344,42168] 0 2024-04-20T11:08:57.390 INFO:tasks.workunit.client.0.smithi117.stdout:1/189: symlink d6/d1f/d30/d32/l41 0 2024-04-20T11:08:57.391 INFO:tasks.workunit.client.0.smithi117.stdout:4/234: write d1/d9/f3b [4165355,122957] 0 2024-04-20T11:08:57.392 INFO:tasks.workunit.client.0.smithi117.stdout:5/235: getdents de 0 2024-04-20T11:08:57.394 INFO:tasks.workunit.client.0.smithi117.stdout:0/176: getdents d3/d7/d16 0 2024-04-20T11:08:57.398 INFO:tasks.workunit.client.0.smithi117.stdout:6/245: link d3/df/l3b d3/df/d39/l52 0 2024-04-20T11:08:57.399 INFO:tasks.workunit.client.0.smithi117.stdout:4/235: fsync d1/d45/d4a/f50 0 2024-04-20T11:08:57.399 INFO:tasks.workunit.client.0.smithi117.stdout:3/151: chown d0/d5/d7/d2d/f33 13037 1 2024-04-20T11:08:57.400 INFO:tasks.workunit.client.0.smithi117.stdout:8/145: dwrite d0/d8/d11/f1c [0,4194304] 0 2024-04-20T11:08:57.401 INFO:tasks.workunit.client.0.smithi117.stdout:1/190: fsync d6/f12 0 2024-04-20T11:08:57.403 INFO:tasks.workunit.client.0.smithi117.stdout:3/152: fdatasync d0/d5/f13 0 2024-04-20T11:08:57.404 INFO:tasks.workunit.client.0.smithi117.stdout:5/236: mkdir de/d21/d51 0 2024-04-20T11:08:57.404 INFO:tasks.workunit.client.0.smithi117.stdout:0/177: rmdir d3/d5 39 2024-04-20T11:08:57.405 INFO:tasks.workunit.client.0.smithi117.stdout:6/246: mknod d3/df/c53 0 2024-04-20T11:08:57.406 INFO:tasks.workunit.client.0.smithi117.stdout:3/153: readlink d0/d5/d7/l8 0 2024-04-20T11:08:57.407 INFO:tasks.workunit.client.0.smithi117.stdout:9/216: dwrite d4/d30/d31/f33 [0,4194304] 0 2024-04-20T11:08:57.408 INFO:tasks.workunit.client.0.smithi117.stdout:1/191: chown d6/d16/c17 0 1 2024-04-20T11:08:57.408 INFO:tasks.workunit.client.0.smithi117.stdout:5/237: creat de/d41/f52 x:0 0 0 2024-04-20T11:08:57.411 INFO:tasks.workunit.client.0.smithi117.stdout:3/154: symlink d0/d5/de/l34 0 2024-04-20T11:08:57.413 INFO:tasks.workunit.client.0.smithi117.stdout:6/247: symlink d3/df/d39/l54 0 2024-04-20T11:08:57.417 INFO:tasks.workunit.client.0.smithi117.stdout:6/248: chown d3/d4/l36 924623641 1 2024-04-20T11:08:57.419 INFO:tasks.workunit.client.0.smithi117.stdout:5/238: rename de/d21/l2f to de/d41/d47/l53 0 2024-04-20T11:08:57.419 INFO:tasks.workunit.client.0.smithi117.stdout:1/192: link d6/d16/c17 d6/d16/c42 0 2024-04-20T11:08:57.420 INFO:tasks.workunit.client.0.smithi117.stdout:5/239: readlink lc 0 2024-04-20T11:08:57.421 INFO:tasks.workunit.client.0.smithi117.stdout:3/155: creat d0/f35 x:0 0 0 2024-04-20T11:08:57.421 INFO:tasks.workunit.client.0.smithi117.stdout:6/249: fdatasync d3/d4/f46 0 2024-04-20T11:08:57.425 INFO:tasks.workunit.client.0.smithi117.stdout:6/250: truncate d3/f18 2853480 0 2024-04-20T11:08:57.426 INFO:tasks.workunit.client.0.smithi117.stdout:6/251: stat d3 0 2024-04-20T11:08:57.428 INFO:tasks.workunit.client.0.smithi117.stdout:9/217: dwrite d4/d30/f4e [0,4194304] 0 2024-04-20T11:08:57.435 INFO:tasks.workunit.client.0.smithi117.stdout:9/218: link d4/de/l1f d4/d5/l55 0 2024-04-20T11:08:57.439 INFO:tasks.workunit.client.0.smithi117.stdout:1/193: dwrite d6/d35/f36 [0,4194304] 0 2024-04-20T11:08:57.440 INFO:tasks.workunit.client.0.smithi117.stdout:9/219: creat d4/d35/f56 x:0 0 0 2024-04-20T11:08:57.442 INFO:tasks.workunit.client.0.smithi117.stdout:1/194: write d6/f12 [5130181,70403] 0 2024-04-20T11:08:57.443 INFO:tasks.workunit.client.0.smithi117.stdout:9/220: write d4/d5/f7 [689238,78077] 0 2024-04-20T11:08:57.447 INFO:tasks.workunit.client.0.smithi117.stdout:9/221: write d4/d5/da/f32 [1176557,90981] 0 2024-04-20T11:08:57.454 INFO:tasks.workunit.client.0.smithi117.stdout:9/222: mkdir d4/d5/d57 0 2024-04-20T11:08:57.456 INFO:tasks.workunit.client.0.smithi117.stdout:1/195: rmdir d6/d35/d3b 0 2024-04-20T11:08:57.459 INFO:tasks.workunit.client.0.smithi117.stdout:9/223: rmdir d4/de/d10 39 2024-04-20T11:08:57.463 INFO:tasks.workunit.client.0.smithi117.stdout:9/224: fsync d4/de/d10/f1a 0 2024-04-20T11:08:57.474 INFO:tasks.workunit.client.0.smithi117.stdout:2/211: truncate d1/d4/dd/d17/f1e 615885 0 2024-04-20T11:08:57.476 INFO:tasks.workunit.client.0.smithi117.stdout:9/225: dread d4/d30/f4e [0,4194304] 0 2024-04-20T11:08:57.477 INFO:tasks.workunit.client.0.smithi117.stdout:9/226: chown d4/d30/f4e 2092170 1 2024-04-20T11:08:57.481 INFO:tasks.workunit.client.0.smithi117.stdout:3/156: dread d0/d5/d7/dd/ff [0,4194304] 0 2024-04-20T11:08:57.484 INFO:tasks.workunit.client.0.smithi117.stdout:1/196: dwrite d6/f9 [0,4194304] 0 2024-04-20T11:08:57.493 INFO:tasks.workunit.client.0.smithi117.stdout:2/212: dread d1/d4/f5 [0,4194304] 0 2024-04-20T11:08:57.494 INFO:tasks.workunit.client.0.smithi117.stdout:3/157: fsync d0/d5/f25 0 2024-04-20T11:08:57.499 INFO:tasks.workunit.client.0.smithi117.stdout:3/158: dread d0/f3 [0,4194304] 0 2024-04-20T11:08:57.501 INFO:tasks.workunit.client.0.smithi117.stdout:9/227: link f0 d4/de/d4b/f58 0 2024-04-20T11:08:57.505 INFO:tasks.workunit.client.0.smithi117.stdout:4/236: truncate d1/d9/f17 4108966 0 2024-04-20T11:08:57.508 INFO:tasks.workunit.client.0.smithi117.stdout:1/197: dwrite d6/d1f/f2b [0,4194304] 0 2024-04-20T11:08:57.512 INFO:tasks.workunit.client.0.smithi117.stdout:5/240: truncate de/f1f 570473 0 2024-04-20T11:08:57.513 INFO:tasks.workunit.client.0.smithi117.stdout:2/213: rename d1/d4/dd/d17/d25 to d1/d4/dd/de/d29/d3f 0 2024-04-20T11:08:57.521 INFO:tasks.workunit.client.0.smithi117.stdout:3/159: unlink d0/d5/d7/f18 0 2024-04-20T11:08:57.525 INFO:tasks.workunit.client.0.smithi117.stdout:4/237: creat d1/d9/d1f/f55 x:0 0 0 2024-04-20T11:08:57.526 INFO:tasks.workunit.client.0.smithi117.stdout:1/198: rmdir d6/d1f/d30 39 2024-04-20T11:08:57.527 INFO:tasks.workunit.client.0.smithi117.stdout:4/238: dread - d1/dd/d1a/d2b/f4c zero size 2024-04-20T11:08:57.528 INFO:tasks.workunit.client.0.smithi117.stdout:4/239: chown d1/d9/d1f/c33 337 1 2024-04-20T11:08:57.532 INFO:tasks.workunit.client.0.smithi117.stdout:9/228: mkdir d4/d5/da/d51/d59 0 2024-04-20T11:08:57.534 INFO:tasks.workunit.client.0.smithi117.stdout:9/229: dread - d4/de/d10/f3a zero size 2024-04-20T11:08:57.534 INFO:tasks.workunit.client.0.smithi117.stdout:1/199: chown d6/d21/f33 1004 1 2024-04-20T11:08:57.536 INFO:tasks.workunit.client.0.smithi117.stdout:1/200: write d6/f12 [3524596,49264] 0 2024-04-20T11:08:57.537 INFO:tasks.workunit.client.0.smithi117.stdout:5/241: rename de/d21/l27 to de/d16/l54 0 2024-04-20T11:08:57.540 INFO:tasks.workunit.client.0.smithi117.stdout:5/242: write de/f29 [752467,115624] 0 2024-04-20T11:08:57.542 INFO:tasks.workunit.client.0.smithi117.stdout:9/230: dread d4/f2e [0,4194304] 0 2024-04-20T11:08:57.544 INFO:tasks.workunit.client.0.smithi117.stdout:9/231: write d4/de/f2c [1656522,69815] 0 2024-04-20T11:08:57.545 INFO:tasks.workunit.client.0.smithi117.stdout:9/232: dread - d4/d5/f2b zero size 2024-04-20T11:08:57.547 INFO:tasks.workunit.client.0.smithi117.stdout:1/201: dread - d6/f18 zero size 2024-04-20T11:08:57.549 INFO:tasks.workunit.client.0.smithi117.stdout:1/202: write d6/d21/f33 [559929,45516] 0 2024-04-20T11:08:57.550 INFO:tasks.workunit.client.0.smithi117.stdout:3/160: rename d0/d5/d7/d2d to d0/d5/de/d1d/d36 0 2024-04-20T11:08:57.559 INFO:tasks.workunit.client.0.smithi117.stdout:5/243: mkdir de/d41/d47/d55 0 2024-04-20T11:08:57.571 INFO:tasks.workunit.client.0.smithi117.stdout:2/214: getdents d1/d4/dd/de 0 2024-04-20T11:08:57.571 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:57 smithi117 ceph-mon[15659]: pgmap v8: 65 pgs: 65 active+clean; 268 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 376 KiB/s rd, 17 MiB/s wr, 111 op/s 2024-04-20T11:08:57.571 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:57 smithi117 ceph-mon[15659]: Standby manager daemon smithi173.gnrjof started 2024-04-20T11:08:57.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:57 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/712987045' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T11:08:57.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:57 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/712987045' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:08:57.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:57 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/712987045' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T11:08:57.572 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:57 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/712987045' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:08:57.572 INFO:tasks.workunit.client.0.smithi117.stdout:2/215: fdatasync d1/d4/dd/de/f34 0 2024-04-20T11:08:57.574 INFO:tasks.workunit.client.0.smithi117.stdout:3/161: creat d0/d5/de/f37 x:0 0 0 2024-04-20T11:08:57.580 INFO:tasks.workunit.client.0.smithi117.stdout:1/203: creat d6/d1f/d30/f43 x:0 0 0 2024-04-20T11:08:57.585 INFO:tasks.workunit.client.0.smithi117.stdout:5/244: mkdir de/d41/d47/d55/d56 0 2024-04-20T11:08:57.590 INFO:tasks.workunit.client.0.smithi117.stdout:4/240: truncate d1/f20 112424 0 2024-04-20T11:08:57.590 INFO:tasks.workunit.client.0.smithi117.stdout:1/204: rmdir d6/d1f/d30/d32 39 2024-04-20T11:08:57.594 INFO:tasks.workunit.client.0.smithi117.stdout:2/216: link d1/d4/dd/de/f38 d1/d4/dd/de/d29/d2e/f40 0 2024-04-20T11:08:57.596 INFO:tasks.workunit.client.0.smithi117.stdout:5/245: mkdir de/d21/d35/d57 0 2024-04-20T11:08:57.598 INFO:tasks.workunit.client.0.smithi117.stdout:9/233: link d4/d35/l3b d4/d5/da/l5a 0 2024-04-20T11:08:57.601 INFO:tasks.workunit.client.0.smithi117.stdout:4/241: symlink d1/dd/d1a/d2b/l56 0 2024-04-20T11:08:57.601 INFO:tasks.workunit.client.0.smithi117.stdout:4/242: chown d1/dd 6840 1 2024-04-20T11:08:57.603 INFO:tasks.workunit.client.0.smithi117.stdout:4/243: chown d1/dd/f13 2052934 1 2024-04-20T11:08:57.605 INFO:tasks.workunit.client.0.smithi117.stdout:4/244: chown d1/d9/d1f/f36 258 1 2024-04-20T11:08:57.606 INFO:tasks.workunit.client.0.smithi117.stdout:5/246: creat de/d21/d45/d48/f58 x:0 0 0 2024-04-20T11:08:57.607 INFO:tasks.workunit.client.0.smithi117.stdout:4/245: read - d1/dd/f38 zero size 2024-04-20T11:08:57.610 INFO:tasks.workunit.client.0.smithi117.stdout:2/217: dwrite d1/d4/dd/de/f23 [0,4194304] 0 2024-04-20T11:08:57.610 INFO:tasks.workunit.client.0.smithi117.stdout:9/234: stat d4/d30/d31/c42 0 2024-04-20T11:08:57.611 INFO:tasks.workunit.client.0.smithi117.stdout:3/162: write d0/d5/f25 [810539,101919] 0 2024-04-20T11:08:57.613 INFO:tasks.workunit.client.0.smithi117.stdout:3/163: chown d0/d5/d11 6824842 1 2024-04-20T11:08:57.614 INFO:tasks.workunit.client.0.smithi117.stdout:3/164: chown d0/d5/de/d1d 0 1 2024-04-20T11:08:57.617 INFO:tasks.workunit.client.0.smithi117.stdout:4/246: creat d1/f57 x:0 0 0 2024-04-20T11:08:57.618 INFO:tasks.workunit.client.0.smithi117.stdout:2/218: dread d1/d4/dd/f19 [0,4194304] 0 2024-04-20T11:08:57.618 INFO:tasks.workunit.client.0.smithi117.stdout:4/247: fsync d1/d9/d1f/f36 0 2024-04-20T11:08:57.619 INFO:tasks.workunit.client.0.smithi117.stdout:9/235: rmdir d4/d35 39 2024-04-20T11:08:57.620 INFO:tasks.workunit.client.0.smithi117.stdout:2/219: write d1/d4/dd/f19 [1291923,96514] 0 2024-04-20T11:08:57.622 INFO:tasks.workunit.client.0.smithi117.stdout:5/247: symlink de/d21/d51/l59 0 2024-04-20T11:08:57.638 INFO:tasks.workunit.client.0.smithi117.stdout:5/248: dwrite de/d21/d45/d48/f4a [0,4194304] 0 2024-04-20T11:08:57.646 INFO:tasks.workunit.client.0.smithi117.stdout:2/220: mknod d1/d4/c41 0 2024-04-20T11:08:57.651 INFO:tasks.workunit.client.0.smithi117.stdout:5/249: rename de/d41/d47/f4d to de/d41/d4b/f5a 0 2024-04-20T11:08:57.654 INFO:tasks.workunit.client.0.smithi117.stdout:4/248: link d1/c27 d1/d45/c58 0 2024-04-20T11:08:57.655 INFO:tasks.workunit.client.0.smithi117.stdout:3/165: getdents d0/d5/de/d1d/d29 0 2024-04-20T11:08:57.655 INFO:tasks.workunit.client.0.smithi117.stdout:1/205: truncate d6/d21/f2d 3903693 0 2024-04-20T11:08:57.657 INFO:tasks.workunit.client.0.smithi117.stdout:2/221: dread d1/d4/f8 [0,4194304] 0 2024-04-20T11:08:57.657 INFO:tasks.workunit.client.0.smithi117.stdout:4/249: write d1/f19 [2665198,26095] 0 2024-04-20T11:08:57.659 INFO:tasks.workunit.client.0.smithi117.stdout:3/166: read d0/d5/d7/dd/ff [1801621,23614] 0 2024-04-20T11:08:57.663 INFO:tasks.workunit.client.0.smithi117.stdout:5/250: mkdir de/d5b 0 2024-04-20T11:08:57.667 INFO:tasks.workunit.client.0.smithi117.stdout:1/206: write d6/f13 [1386290,47806] 0 2024-04-20T11:08:57.670 INFO:tasks.workunit.client.0.smithi117.stdout:4/250: mkdir d1/d45/d4a/d59 0 2024-04-20T11:08:57.674 INFO:tasks.workunit.client.0.smithi117.stdout:3/167: mknod d0/d5/c38 0 2024-04-20T11:08:57.677 INFO:tasks.workunit.client.0.smithi117.stdout:9/236: dwrite f0 [0,4194304] 0 2024-04-20T11:08:57.679 INFO:tasks.workunit.client.0.smithi117.stdout:9/237: chown d4/de/f1b 14164 1 2024-04-20T11:08:57.680 INFO:tasks.workunit.client.0.smithi117.stdout:1/207: creat d6/d35/f44 x:0 0 0 2024-04-20T11:08:57.682 INFO:tasks.workunit.client.0.smithi117.stdout:2/222: symlink d1/d4/dd/d3e/l42 0 2024-04-20T11:08:57.685 INFO:tasks.workunit.client.0.smithi117.stdout:2/223: write d1/d4/dd/de/f34 [127778,54350] 0 2024-04-20T11:08:57.686 INFO:tasks.workunit.client.0.smithi117.stdout:3/168: symlink d0/d5/d21/l39 0 2024-04-20T11:08:57.686 INFO:tasks.workunit.client.0.smithi117.stdout:2/224: chown d1/l39 82946126 1 2024-04-20T11:08:57.691 INFO:tasks.workunit.client.0.smithi117.stdout:9/238: rmdir d4/d5 39 2024-04-20T11:08:57.695 INFO:tasks.workunit.client.0.smithi117.stdout:1/208: dwrite d6/d1f/d30/f43 [0,4194304] 0 2024-04-20T11:08:57.697 INFO:tasks.workunit.client.0.smithi117.stdout:3/169: rmdir d0/d5/d7/dd 39 2024-04-20T11:08:57.698 INFO:tasks.workunit.client.0.smithi117.stdout:3/170: chown d0/d5/d7 121 1 2024-04-20T11:08:57.704 INFO:tasks.workunit.client.0.smithi117.stdout:4/251: link d1/c42 d1/d45/c5a 0 2024-04-20T11:08:57.711 INFO:tasks.workunit.client.0.smithi117.stdout:3/171: creat d0/d5/d7/dd/f3a x:0 0 0 2024-04-20T11:08:57.718 INFO:tasks.workunit.client.0.smithi117.stdout:1/209: creat d6/d1f/f45 x:0 0 0 2024-04-20T11:08:57.718 INFO:tasks.workunit.client.0.smithi117.stdout:2/225: dwrite d1/d4/dd/d17/f30 [0,4194304] 0 2024-04-20T11:08:57.722 INFO:tasks.workunit.client.0.smithi117.stdout:2/226: write d1/d4/dd/d17/f30 [3823368,28391] 0 2024-04-20T11:08:57.722 INFO:tasks.workunit.client.0.smithi117.stdout:3/172: creat d0/d2a/f3b x:0 0 0 2024-04-20T11:08:57.723 INFO:tasks.workunit.client.0.smithi117.stdout:3/173: readlink d0/d5/d7/l8 0 2024-04-20T11:08:57.725 INFO:tasks.workunit.client.0.smithi117.stdout:9/239: getdents d4/d5 0 2024-04-20T11:08:57.726 INFO:tasks.workunit.client.0.smithi117.stdout:4/252: link d1/d9/c14 d1/d45/c5b 0 2024-04-20T11:08:57.732 INFO:tasks.workunit.client.0.smithi117.stdout:1/210: creat d6/d1f/d30/d32/f46 x:0 0 0 2024-04-20T11:08:57.735 INFO:tasks.workunit.client.0.smithi117.stdout:9/240: dread - d4/d35/f56 zero size 2024-04-20T11:08:57.735 INFO:tasks.workunit.client.0.smithi117.stdout:1/211: chown d6/d1f/d30/d32/l41 3 1 2024-04-20T11:08:57.737 INFO:tasks.workunit.client.0.smithi117.stdout:1/212: write d6/d1f/d30/f43 [3896765,48313] 0 2024-04-20T11:08:57.740 INFO:tasks.workunit.client.0.smithi117.stdout:3/174: truncate d0/d5/d7/f20 42810 0 2024-04-20T11:08:57.743 INFO:tasks.workunit.client.0.smithi117.stdout:1/213: symlink d6/d1f/l47 0 2024-04-20T11:08:57.745 INFO:tasks.workunit.client.0.smithi117.stdout:3/175: unlink d0/d5/d7/c32 0 2024-04-20T11:08:57.745 INFO:tasks.workunit.client.0.smithi117.stdout:4/253: dwrite d1/f10 [0,4194304] 0 2024-04-20T11:08:57.753 INFO:tasks.workunit.client.0.smithi117.stdout:4/254: link d1/dd/d1a/f2e d1/d9/d1f/f5c 0 2024-04-20T11:08:57.758 INFO:tasks.workunit.client.0.smithi117.stdout:4/255: mknod d1/d45/d4a/c5d 0 2024-04-20T11:08:57.762 INFO:tasks.workunit.client.0.smithi117.stdout:2/227: write d1/d4/dd/f14 [591976,8395] 0 2024-04-20T11:08:57.763 INFO:tasks.workunit.client.0.smithi117.stdout:1/214: getdents d6/d21 0 2024-04-20T11:08:57.763 INFO:tasks.workunit.client.0.smithi117.stdout:5/251: dwrite de/f11 [0,4194304] 0 2024-04-20T11:08:57.764 INFO:tasks.workunit.client.0.smithi117.stdout:4/256: mknod d1/d9/d1f/c5e 0 2024-04-20T11:08:57.767 INFO:tasks.workunit.client.0.smithi117.stdout:3/176: dread d0/d5/d7/dd/ff [0,4194304] 0 2024-04-20T11:08:57.770 INFO:tasks.workunit.client.0.smithi117.stdout:5/252: chown de/d41/d47/l53 352 1 2024-04-20T11:08:57.776 INFO:tasks.workunit.client.0.smithi117.stdout:2/228: mknod d1/d4/dd/de/d29/d3f/c43 0 2024-04-20T11:08:57.778 INFO:tasks.workunit.client.0.smithi117.stdout:2/229: dread - d1/d4/dd/de/d29/d2e/f40 zero size 2024-04-20T11:08:57.780 INFO:tasks.workunit.client.0.smithi117.stdout:3/177: mkdir d0/d5/de/d1d/d3c 0 2024-04-20T11:08:57.786 INFO:tasks.workunit.client.0.smithi117.stdout:4/257: rename d1/dd/d1a/f2e to d1/d45/f5f 0 2024-04-20T11:08:57.788 INFO:tasks.workunit.client.0.smithi117.stdout:1/215: truncate d6/f12 1784949 0 2024-04-20T11:08:57.790 INFO:tasks.workunit.client.0.smithi117.stdout:1/216: write d6/d35/f36 [4495668,60918] 0 2024-04-20T11:08:57.790 INFO:tasks.workunit.client.0.smithi117.stdout:5/253: fsync de/d16/f1e 0 2024-04-20T11:08:57.792 INFO:tasks.workunit.client.0.smithi117.stdout:5/254: write f5 [1784973,74640] 0 2024-04-20T11:08:57.793 INFO:tasks.workunit.client.0.smithi117.stdout:4/258: creat d1/dd/d1a/d43/f60 x:0 0 0 2024-04-20T11:08:57.795 INFO:tasks.workunit.client.0.smithi117.stdout:4/259: write d1/f19 [3899003,87197] 0 2024-04-20T11:08:57.795 INFO:tasks.workunit.client.0.smithi117.stdout:2/230: mknod d1/d20/d26/c44 0 2024-04-20T11:08:57.798 INFO:tasks.workunit.client.0.smithi117.stdout:3/178: mknod d0/d5/de/d1d/d3c/c3d 0 2024-04-20T11:08:57.798 INFO:tasks.workunit.client.0.smithi117.stdout:4/260: write d1/dd/f38 [815688,18521] 0 2024-04-20T11:08:57.799 INFO:tasks.workunit.client.0.smithi117.stdout:1/217: mkdir d6/d35/d48 0 2024-04-20T11:08:57.808 INFO:tasks.workunit.client.0.smithi117.stdout:1/218: chown d6/d21/l3f 14755964 1 2024-04-20T11:08:57.809 INFO:tasks.workunit.client.0.smithi117.stdout:3/179: mknod d0/d5/de/d1d/d3c/c3e 0 2024-04-20T11:08:57.811 INFO:tasks.workunit.client.0.smithi117.stdout:3/180: dread - d0/d5/d7/f1e zero size 2024-04-20T11:08:57.812 INFO:tasks.workunit.client.0.smithi117.stdout:2/231: dwrite d1/f6 [0,4194304] 0 2024-04-20T11:08:57.814 INFO:tasks.workunit.client.0.smithi117.stdout:4/261: rename d1/d9/d1f/c33 to d1/dd/d1a/d2b/c61 0 2024-04-20T11:08:57.817 INFO:tasks.workunit.client.0.smithi117.stdout:3/181: symlink d0/d5/d11/l3f 0 2024-04-20T11:08:57.818 INFO:tasks.workunit.client.0.smithi117.stdout:2/232: fdatasync d1/d4/dd/de/d29/d2e/f40 0 2024-04-20T11:08:57.822 INFO:tasks.workunit.client.0.smithi117.stdout:1/219: getdents d6/d16/d3a 0 2024-04-20T11:08:57.832 INFO:tasks.workunit.client.0.smithi117.stdout:4/262: dwrite d1/d9/f3b [0,4194304] 0 2024-04-20T11:08:57.833 INFO:tasks.workunit.client.0.smithi117.stdout:2/233: creat d1/d4/f45 x:0 0 0 2024-04-20T11:08:57.835 INFO:tasks.workunit.client.0.smithi117.stdout:5/255: dwrite de/d21/d45/d48/d4e/f4f [0,4194304] 0 2024-04-20T11:08:57.835 INFO:tasks.workunit.client.0.smithi117.stdout:3/182: write d0/d5/d7/f20 [263898,6245] 0 2024-04-20T11:08:57.837 INFO:tasks.workunit.client.0.smithi117.stdout:3/183: dread - d0/f35 zero size 2024-04-20T11:08:57.840 INFO:tasks.workunit.client.0.smithi117.stdout:5/256: chown l4 118136 1 2024-04-20T11:08:57.840 INFO:tasks.workunit.client.0.smithi117.stdout:4/263: unlink d1/dd/d1a/d2b/f4c 0 2024-04-20T11:08:57.845 INFO:tasks.workunit.client.0.smithi117.stdout:4/264: stat d1/dd/d1a/d2b/c61 0 2024-04-20T11:08:57.847 INFO:tasks.workunit.client.0.smithi117.stdout:3/184: link d0/d5/de/f37 d0/d5/f40 0 2024-04-20T11:08:57.850 INFO:tasks.workunit.client.0.smithi117.stdout:3/185: creat d0/d5/d21/f41 x:0 0 0 2024-04-20T11:08:57.857 INFO:tasks.workunit.client.0.smithi117.stdout:2/234: dwrite d1/d4/dd/d17/f30 [0,4194304] 0 2024-04-20T11:08:57.858 INFO:tasks.workunit.client.0.smithi117.stdout:3/186: rename d0/d5/de/f37 to d0/d5/d21/f42 0 2024-04-20T11:08:57.863 INFO:tasks.workunit.client.0.smithi117.stdout:3/187: mknod d0/d5/d7/dd/c43 0 2024-04-20T11:08:57.864 INFO:tasks.workunit.client.0.smithi117.stdout:5/257: dwrite de/f34 [0,4194304] 0 2024-04-20T11:08:57.866 INFO:tasks.workunit.client.0.smithi117.stdout:3/188: dread - d0/d5/de/d1d/d36/f33 zero size 2024-04-20T11:08:57.869 INFO:tasks.workunit.client.0.smithi117.stdout:3/189: fsync d0/d5/d7/f20 0 2024-04-20T11:08:57.876 INFO:tasks.workunit.client.0.smithi117.stdout:3/190: creat d0/d5/de/d1d/f44 x:0 0 0 2024-04-20T11:08:57.880 INFO:tasks.workunit.client.0.smithi117.stdout:2/235: dwrite d1/d20/f21 [0,4194304] 0 2024-04-20T11:08:57.884 INFO:tasks.workunit.client.0.smithi117.stdout:5/258: dread de/d21/d45/d48/f4a [0,4194304] 0 2024-04-20T11:08:57.887 INFO:tasks.workunit.client.0.smithi117.stdout:5/259: chown de/c3f 320 1 2024-04-20T11:08:57.891 INFO:tasks.workunit.client.0.smithi117.stdout:5/260: creat de/d41/d47/f5c x:0 0 0 2024-04-20T11:08:57.896 INFO:tasks.workunit.client.0.smithi117.stdout:5/261: truncate de/d41/d47/f5c 998284 0 2024-04-20T11:08:57.900 INFO:tasks.workunit.client.0.smithi117.stdout:5/262: unlink fb 0 2024-04-20T11:08:57.901 INFO:tasks.workunit.client.0.smithi117.stdout:2/236: dwrite d1/d4/dd/de/f34 [0,4194304] 0 2024-04-20T11:08:57.902 INFO:tasks.workunit.client.0.smithi117.stdout:5/263: chown de/d21/d35/f36 0 1 2024-04-20T11:08:57.908 INFO:tasks.workunit.client.0.smithi117.stdout:5/264: mkdir de/d16/d5d 0 2024-04-20T11:08:57.908 INFO:tasks.workunit.client.0.smithi117.stdout:2/237: mkdir d1/d4/dd/d3e/d46 0 2024-04-20T11:08:57.909 INFO:tasks.workunit.client.0.smithi117.stdout:5/265: dread - de/d41/d4b/f5a zero size 2024-04-20T11:08:57.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:57 smithi173 ceph-mon[23800]: pgmap v8: 65 pgs: 65 active+clean; 268 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 376 KiB/s rd, 17 MiB/s wr, 111 op/s 2024-04-20T11:08:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:57 smithi173 ceph-mon[23800]: Standby manager daemon smithi173.gnrjof started 2024-04-20T11:08:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:57 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/712987045' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T11:08:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:57 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/712987045' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:08:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:57 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/712987045' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T11:08:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:57 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/712987045' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:08:57.915 INFO:tasks.workunit.client.0.smithi117.stdout:2/238: fdatasync d1/d4/dd/d17/f1e 0 2024-04-20T11:08:57.919 INFO:tasks.workunit.client.0.smithi117.stdout:2/239: truncate d1/d20/d26/f37 2119814 0 2024-04-20T11:08:57.920 INFO:tasks.workunit.client.0.smithi117.stdout:3/191: dread d0/d5/d7/dd/f10 [0,4194304] 0 2024-04-20T11:08:57.922 INFO:tasks.workunit.client.0.smithi117.stdout:2/240: write d1/d4/dd/de/f23 [4438176,20186] 0 2024-04-20T11:08:57.924 INFO:tasks.workunit.client.0.smithi117.stdout:3/192: write d0/d5/d7/f20 [1168783,80154] 0 2024-04-20T11:08:57.929 INFO:tasks.workunit.client.0.smithi117.stdout:2/241: rename d1/d15/l2d to d1/d4/dd/d17/d3b/l47 0 2024-04-20T11:08:57.932 INFO:tasks.workunit.client.0.smithi117.stdout:2/242: write d1/d4/dd/f19 [1732921,128719] 0 2024-04-20T11:08:57.934 INFO:tasks.workunit.client.0.smithi117.stdout:5/266: dwrite de/d21/d45/d48/f4a [0,4194304] 0 2024-04-20T11:08:57.935 INFO:tasks.workunit.client.0.smithi117.stdout:3/193: link d0/d5/d7/dd/f10 d0/d5/de/d1d/d36/f45 0 2024-04-20T11:08:57.937 INFO:tasks.workunit.client.0.smithi117.stdout:3/194: fdatasync d0/d5/de/d1d/d36/f33 0 2024-04-20T11:08:57.940 INFO:tasks.workunit.client.0.smithi117.stdout:3/195: write d0/d2a/f3b [577159,122897] 0 2024-04-20T11:08:57.942 INFO:tasks.workunit.client.0.smithi117.stdout:2/243: creat d1/f48 x:0 0 0 2024-04-20T11:08:57.945 INFO:tasks.workunit.client.0.smithi117.stdout:3/196: unlink d0/d5/d7/dd/f2b 0 2024-04-20T11:08:57.946 INFO:tasks.workunit.client.0.smithi117.stdout:2/244: mkdir d1/d20/d26/d49 0 2024-04-20T11:08:57.948 INFO:tasks.workunit.client.0.smithi117.stdout:3/197: mkdir d0/d5/d11/d46 0 2024-04-20T11:08:57.950 INFO:tasks.workunit.client.0.smithi117.stdout:2/245: write d1/d4/f5 [354280,102619] 0 2024-04-20T11:08:57.950 INFO:tasks.workunit.client.0.smithi117.stdout:3/198: readlink d0/d5/d21/l39 0 2024-04-20T11:08:57.954 INFO:tasks.workunit.client.0.smithi117.stdout:3/199: dread - d0/d5/d7/f1e zero size 2024-04-20T11:08:57.955 INFO:tasks.workunit.client.0.smithi117.stdout:5/267: dwrite de/d21/f31 [0,4194304] 0 2024-04-20T11:08:57.955 INFO:tasks.workunit.client.0.smithi117.stdout:2/246: dread d1/d4/dd/d17/f1e [0,4194304] 0 2024-04-20T11:08:57.956 INFO:tasks.workunit.client.0.smithi117.stdout:3/200: dread - d0/d5/de/d1d/f44 zero size 2024-04-20T11:08:57.962 INFO:tasks.workunit.client.0.smithi117.stdout:3/201: creat d0/d5/d21/f47 x:0 0 0 2024-04-20T11:08:57.962 INFO:tasks.workunit.client.0.smithi117.stdout:5/268: rmdir de/d16/d5d 0 2024-04-20T11:08:57.967 INFO:tasks.workunit.client.0.smithi117.stdout:5/269: creat de/d41/d47/f5e x:0 0 0 2024-04-20T11:08:57.972 INFO:tasks.workunit.client.0.smithi117.stdout:5/270: truncate de/f14 204623 0 2024-04-20T11:08:57.972 INFO:tasks.workunit.client.0.smithi117.stdout:3/202: dread d0/d5/d7/f20 [0,4194304] 0 2024-04-20T11:08:57.975 INFO:tasks.workunit.client.0.smithi117.stdout:5/271: write de/f18 [819681,112476] 0 2024-04-20T11:08:57.976 INFO:tasks.workunit.client.0.smithi117.stdout:2/247: dwrite d1/d4/f7 [0,4194304] 0 2024-04-20T11:08:57.978 INFO:tasks.workunit.client.0.smithi117.stdout:3/203: symlink d0/d5/d11/d46/l48 0 2024-04-20T11:08:57.983 INFO:tasks.workunit.client.0.smithi117.stdout:5/272: mkdir de/d21/d5f 0 2024-04-20T11:08:57.986 INFO:tasks.workunit.client.0.smithi117.stdout:3/204: rename d0/d5/de/d1d/d36/f45 to d0/d5/d11/f49 0 2024-04-20T11:08:57.988 INFO:tasks.workunit.client.0.smithi117.stdout:5/273: rmdir de/d21/d45 39 2024-04-20T11:08:57.990 INFO:tasks.workunit.client.0.smithi117.stdout:3/205: creat d0/d5/d21/f4a x:0 0 0 2024-04-20T11:08:57.992 INFO:tasks.workunit.client.0.smithi117.stdout:3/206: read d0/d5/d7/f20 [633114,11714] 0 2024-04-20T11:08:57.992 INFO:tasks.workunit.client.0.smithi117.stdout:5/274: fsync de/d21/d45/d48/f4a 0 2024-04-20T11:08:57.997 INFO:tasks.workunit.client.0.smithi117.stdout:3/207: symlink d0/d5/d7/dd/l4b 0 2024-04-20T11:08:57.999 INFO:tasks.workunit.client.0.smithi117.stdout:3/208: chown d0/d5/de/d1d/f1f 4802 1 2024-04-20T11:08:58.001 INFO:tasks.workunit.client.0.smithi117.stdout:5/275: dread de/f29 [0,4194304] 0 2024-04-20T11:08:58.005 INFO:tasks.workunit.client.0.smithi117.stdout:5/276: unlink de/d21/d35/f36 0 2024-04-20T11:08:58.006 INFO:tasks.workunit.client.0.smithi117.stdout:3/209: truncate d0/d5/d7/dd/ff 2971715 0 2024-04-20T11:08:58.008 INFO:tasks.workunit.client.0.smithi117.stdout:5/277: chown de/l1a 98423173 1 2024-04-20T11:08:58.011 INFO:tasks.workunit.client.0.smithi117.stdout:3/210: mknod d0/c4c 0 2024-04-20T11:08:58.013 INFO:tasks.workunit.client.0.smithi117.stdout:2/248: dwrite d1/d4/f8 [0,4194304] 0 2024-04-20T11:08:58.015 INFO:tasks.workunit.client.0.smithi117.stdout:3/211: write d0/d5/d21/f41 [40170,114174] 0 2024-04-20T11:08:58.015 INFO:tasks.workunit.client.0.smithi117.stdout:5/278: symlink de/d21/d35/l60 0 2024-04-20T11:08:58.018 INFO:tasks.workunit.client.0.smithi117.stdout:5/279: write de/d41/d47/f5e [109056,60750] 0 2024-04-20T11:08:58.025 INFO:tasks.workunit.client.0.smithi117.stdout:5/280: mknod de/d21/d45/d48/c61 0 2024-04-20T11:08:58.027 INFO:tasks.workunit.client.0.smithi117.stdout:3/212: truncate d0/d5/d21/f28 411998 0 2024-04-20T11:08:58.029 INFO:tasks.workunit.client.0.smithi117.stdout:2/249: link d1/l3d d1/l4a 0 2024-04-20T11:08:58.031 INFO:tasks.workunit.client.0.smithi117.stdout:2/250: chown d1/d4/dd 0 1 2024-04-20T11:08:58.032 INFO:tasks.workunit.client.0.smithi117.stdout:5/281: rename de/f28 to de/d21/d35/d57/f62 0 2024-04-20T11:08:58.033 INFO:tasks.workunit.client.0.smithi117.stdout:2/251: symlink d1/d4/dd/de/l4b 0 2024-04-20T11:08:58.034 INFO:tasks.workunit.client.0.smithi117.stdout:2/252: chown d1 2531 1 2024-04-20T11:08:58.034 INFO:tasks.workunit.client.0.smithi117.stdout:2/253: stat d1/d4/f45 0 2024-04-20T11:08:58.035 INFO:tasks.workunit.client.0.smithi117.stdout:7/207: sync 2024-04-20T11:08:58.037 INFO:tasks.workunit.client.0.smithi117.stdout:2/254: rmdir d1/d15/d1d 39 2024-04-20T11:08:58.038 INFO:tasks.workunit.client.0.smithi117.stdout:7/208: mkdir d1/d4/d6/d17/d3f 0 2024-04-20T11:08:58.042 INFO:tasks.workunit.client.0.smithi117.stdout:7/209: symlink d1/d4/d39/l40 0 2024-04-20T11:08:58.050 INFO:tasks.workunit.client.0.smithi117.stdout:2/255: dread d1/d4/f7 [0,4194304] 0 2024-04-20T11:08:58.056 INFO:tasks.workunit.client.0.smithi117.stdout:2/256: creat d1/d4/dd/de/d29/f4c x:0 0 0 2024-04-20T11:08:58.060 INFO:tasks.workunit.client.0.smithi117.stdout:2/257: mknod d1/d20/c4d 0 2024-04-20T11:08:58.065 INFO:tasks.workunit.client.0.smithi117.stdout:2/258: mknod d1/d15/d1d/c4e 0 2024-04-20T11:08:58.067 INFO:tasks.workunit.client.0.smithi117.stdout:2/259: chown d1/d4/dd 50577316 1 2024-04-20T11:08:58.067 INFO:tasks.workunit.client.0.smithi117.stdout:3/213: dwrite d0/d5/de/d1d/d29/f31 [0,4194304] 0 2024-04-20T11:08:58.068 INFO:tasks.workunit.client.0.smithi117.stdout:5/282: dwrite de/f1f [0,4194304] 0 2024-04-20T11:08:58.074 INFO:tasks.workunit.client.0.smithi117.stdout:2/260: symlink d1/d4/dd/d17/l4f 0 2024-04-20T11:08:58.074 INFO:tasks.workunit.client.0.smithi117.stdout:5/283: fsync de/f34 0 2024-04-20T11:08:58.078 INFO:tasks.workunit.client.0.smithi117.stdout:2/261: write d1/f48 [637693,122224] 0 2024-04-20T11:08:58.082 INFO:tasks.workunit.client.0.smithi117.stdout:2/262: write d1/f6 [4621745,47043] 0 2024-04-20T11:08:58.084 INFO:tasks.workunit.client.0.smithi117.stdout:5/284: dread de/d41/d47/f5c [0,4194304] 0 2024-04-20T11:08:58.093 INFO:tasks.workunit.client.0.smithi117.stdout:5/285: truncate de/d21/d35/f43 676461 0 2024-04-20T11:08:58.094 INFO:tasks.workunit.client.0.smithi117.stdout:3/214: dwrite d0/d5/f25 [0,4194304] 0 2024-04-20T11:08:58.098 INFO:tasks.workunit.client.0.smithi117.stdout:3/215: creat d0/d5/d21/f4d x:0 0 0 2024-04-20T11:08:58.099 INFO:tasks.workunit.client.0.smithi117.stdout:5/286: getdents de/d41/d47 0 2024-04-20T11:08:58.100 INFO:tasks.workunit.client.0.smithi117.stdout:3/216: read d0/d5/f13 [1386001,32097] 0 2024-04-20T11:08:58.105 INFO:tasks.workunit.client.0.smithi117.stdout:2/263: dwrite d1/d20/d26/f37 [0,4194304] 0 2024-04-20T11:08:58.111 INFO:tasks.workunit.client.0.smithi117.stdout:2/264: creat d1/d15/d1d/f50 x:0 0 0 2024-04-20T11:08:58.117 INFO:tasks.workunit.client.0.smithi117.stdout:2/265: symlink d1/d20/d26/d49/l51 0 2024-04-20T11:08:58.122 INFO:tasks.workunit.client.0.smithi117.stdout:5/287: dwrite de/d41/d47/f5c [0,4194304] 0 2024-04-20T11:08:58.133 INFO:tasks.workunit.client.0.smithi117.stdout:5/288: dread f5 [0,4194304] 0 2024-04-20T11:08:58.134 INFO:tasks.workunit.client.0.smithi117.stdout:2/266: dwrite d1/f6 [0,4194304] 0 2024-04-20T11:08:58.140 INFO:tasks.workunit.client.0.smithi117.stdout:5/289: rename de/d21/d35/c40 to de/d21/d45/d48/c63 0 2024-04-20T11:08:58.141 INFO:tasks.workunit.client.0.smithi117.stdout:2/267: mkdir d1/d4/dd/de/d29/d2e/d52 0 2024-04-20T11:08:58.143 INFO:tasks.workunit.client.0.smithi117.stdout:5/290: write de/d41/d4b/f5a [829411,115736] 0 2024-04-20T11:08:58.146 INFO:tasks.workunit.client.0.smithi117.stdout:2/268: mkdir d1/d4/dd/de/d29/d2e/d53 0 2024-04-20T11:08:58.149 INFO:tasks.workunit.client.0.smithi117.stdout:2/269: write d1/f3 [3143324,20293] 0 2024-04-20T11:08:58.151 INFO:tasks.workunit.client.0.smithi117.stdout:5/291: rename c8 to de/d21/d35/d57/c64 0 2024-04-20T11:08:58.152 INFO:tasks.workunit.client.0.smithi117.stdout:3/217: dread d0/fb [0,4194304] 0 2024-04-20T11:08:58.155 INFO:tasks.workunit.client.0.smithi117.stdout:2/270: creat d1/d20/d26/d49/f54 x:0 0 0 2024-04-20T11:08:58.155 INFO:tasks.workunit.client.0.smithi117.stdout:5/292: write de/d16/f2d [2745600,8923] 0 2024-04-20T11:08:58.157 INFO:tasks.workunit.client.0.smithi117.stdout:3/218: mkdir d0/d5/de/d4e 0 2024-04-20T11:08:58.160 INFO:tasks.workunit.client.0.smithi117.stdout:3/219: symlink d0/d5/d21/l4f 0 2024-04-20T11:08:58.169 INFO:tasks.workunit.client.0.smithi117.stdout:2/271: dread d1/d4/dd/d17/f30 [0,4194304] 0 2024-04-20T11:08:58.171 INFO:tasks.workunit.client.0.smithi117.stdout:2/272: chown d1/d15/d1d 2040 1 2024-04-20T11:08:58.174 INFO:tasks.workunit.client.0.smithi117.stdout:2/273: fdatasync d1/d4/f5 0 2024-04-20T11:08:58.174 INFO:tasks.workunit.client.0.smithi117.stdout:5/293: dwrite de/d21/d45/d48/d4e/f50 [0,4194304] 0 2024-04-20T11:08:58.180 INFO:tasks.workunit.client.0.smithi117.stdout:5/294: readlink de/d16/l54 0 2024-04-20T11:08:58.183 INFO:tasks.workunit.client.0.smithi117.stdout:5/295: dread - de/d21/d35/f3d zero size 2024-04-20T11:08:58.187 INFO:tasks.workunit.client.0.smithi117.stdout:5/296: mkdir de/d5b/d65 0 2024-04-20T11:08:58.188 INFO:tasks.workunit.client.0.smithi117.stdout:3/220: dwrite d0/f1a [0,4194304] 0 2024-04-20T11:08:58.190 INFO:tasks.workunit.client.0.smithi117.stdout:5/297: creat de/d16/f66 x:0 0 0 2024-04-20T11:08:58.191 INFO:tasks.workunit.client.0.smithi117.stdout:3/221: dread - d0/d5/de/d1d/d36/f33 zero size 2024-04-20T11:08:58.192 INFO:tasks.workunit.client.0.smithi117.stdout:2/274: dread d1/d4/fb [0,4194304] 0 2024-04-20T11:08:58.196 INFO:tasks.workunit.client.0.smithi117.stdout:5/298: dread de/f29 [0,4194304] 0 2024-04-20T11:08:58.200 INFO:tasks.workunit.client.0.smithi117.stdout:5/299: truncate de/f19 1230147 0 2024-04-20T11:08:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:5/300: write de/f46 [226677,56517] 0 2024-04-20T11:08:58.207 INFO:tasks.workunit.client.0.smithi117.stdout:2/275: dread d1/d4/dd/de/f23 [0,4194304] 0 2024-04-20T11:08:58.210 INFO:tasks.workunit.client.0.smithi117.stdout:2/276: write d1/d4/dd/d17/f1f [5285779,68975] 0 2024-04-20T11:08:58.212 INFO:tasks.workunit.client.0.smithi117.stdout:5/301: dread de/d21/d35/f3b [0,4194304] 0 2024-04-20T11:08:58.214 INFO:tasks.workunit.client.0.smithi117.stdout:5/302: chown de/f1f 489971 1 2024-04-20T11:08:58.217 INFO:tasks.workunit.client.0.smithi117.stdout:5/303: rmdir de/d41 39 2024-04-20T11:08:58.221 INFO:tasks.workunit.client.0.smithi117.stdout:3/222: dwrite d0/d5/f40 [0,4194304] 0 2024-04-20T11:08:58.225 INFO:tasks.workunit.client.0.smithi117.stdout:2/277: truncate d1/d4/dd/de/f18 4019096 0 2024-04-20T11:08:58.227 INFO:tasks.workunit.client.0.smithi117.stdout:5/304: mknod de/d21/d35/d57/c67 0 2024-04-20T11:08:58.228 INFO:tasks.workunit.client.0.smithi117.stdout:3/223: creat d0/d5/d7/f50 x:0 0 0 2024-04-20T11:08:58.228 INFO:tasks.workunit.client.0.smithi117.stdout:2/278: read d1/d4/dd/f14 [1851852,77237] 0 2024-04-20T11:08:58.232 INFO:tasks.workunit.client.0.smithi117.stdout:3/224: mknod d0/d5/de/d1d/d36/c51 0 2024-04-20T11:08:58.234 INFO:tasks.workunit.client.0.smithi117.stdout:3/225: mknod d0/d5/d7/c52 0 2024-04-20T11:08:58.242 INFO:tasks.workunit.client.0.smithi117.stdout:2/279: dwrite d1/f48 [0,4194304] 0 2024-04-20T11:08:58.275 INFO:tasks.workunit.client.0.smithi117.stdout:8/146: sync 2024-04-20T11:08:58.423 INFO:tasks.workunit.client.0.smithi117.stdout:7/210: sync 2024-04-20T11:08:58.423 INFO:tasks.workunit.client.0.smithi117.stdout:6/252: sync 2024-04-20T11:08:58.423 INFO:tasks.workunit.client.0.smithi117.stdout:0/178: sync 2024-04-20T11:08:58.425 INFO:tasks.workunit.client.0.smithi117.stdout:1/220: sync 2024-04-20T11:08:58.425 INFO:tasks.workunit.client.0.smithi117.stdout:9/241: sync 2024-04-20T11:08:58.425 INFO:tasks.workunit.client.0.smithi117.stdout:4/265: sync 2024-04-20T11:08:58.429 INFO:tasks.workunit.client.0.smithi117.stdout:9/242: stat d4/d5/da/l27 0 2024-04-20T11:08:58.433 INFO:tasks.workunit.client.0.smithi117.stdout:1/221: write d6/f9 [4665536,3466] 0 2024-04-20T11:08:58.434 INFO:tasks.workunit.client.0.smithi117.stdout:6/253: fdatasync d3/df/d39/f4b 0 2024-04-20T11:08:58.434 INFO:tasks.workunit.client.0.smithi117.stdout:4/266: fsync d1/d9/d1f/f39 0 2024-04-20T11:08:58.438 INFO:tasks.workunit.client.0.smithi117.stdout:1/222: chown d6/d21/l29 14 1 2024-04-20T11:08:58.440 INFO:tasks.workunit.client.0.smithi117.stdout:6/254: dread d3/f4c [0,4194304] 0 2024-04-20T11:08:58.453 INFO:tasks.workunit.client.0.smithi117.stdout:6/255: unlink d3/d4/c22 0 2024-04-20T11:08:58.456 INFO:tasks.workunit.client.0.smithi117.stdout:9/243: dwrite d4/f45 [0,4194304] 0 2024-04-20T11:08:58.456 INFO:tasks.workunit.client.0.smithi117.stdout:4/267: link d1/f10 d1/dd/d1a/d43/d4e/f62 0 2024-04-20T11:08:58.462 INFO:tasks.workunit.client.0.smithi117.stdout:1/223: mkdir d6/d1f/d30/d49 0 2024-04-20T11:08:58.470 INFO:tasks.workunit.client.0.smithi117.stdout:9/244: mknod d4/d35/c5b 0 2024-04-20T11:08:58.472 INFO:tasks.workunit.client.0.smithi117.stdout:4/268: mknod d1/d45/c63 0 2024-04-20T11:08:58.476 INFO:tasks.workunit.client.0.smithi117.stdout:1/224: chown d6/c28 176 1 2024-04-20T11:08:58.483 INFO:tasks.workunit.client.0.smithi117.stdout:0/179: rmdir d3/d7/d8/d3c 0 2024-04-20T11:08:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:6/256: dread d3/f28 [0,4194304] 0 2024-04-20T11:08:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:6/257: truncate d3/f18 3185070 0 2024-04-20T11:08:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:1/225: dread d6/d35/f36 [0,4194304] 0 2024-04-20T11:08:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:9/245: creat d4/de/d10/f5c x:0 0 0 2024-04-20T11:08:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:6/258: mkdir d3/d4/d2e/d55 0 2024-04-20T11:08:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:6/259: readlink d3/df/d39/l54 0 2024-04-20T11:08:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:6/260: write d3/df/f31 [3387696,50474] 0 2024-04-20T11:08:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:9/246: creat d4/d30/d31/f5d x:0 0 0 2024-04-20T11:08:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:4/269: getdents d1/dd/d52 0 2024-04-20T11:08:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:1/226: rmdir d6/d35/d48 0 2024-04-20T11:08:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:4/270: mknod d1/dd/d1a/c64 0 2024-04-20T11:08:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:7/211: symlink d1/l41 0 2024-04-20T11:08:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:4/271: dread - d1/d45/f4f zero size 2024-04-20T11:08:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:7/212: write d1/d4/d6/d2b/f33 [218737,93605] 0 2024-04-20T11:08:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:4/272: creat d1/dd/d1a/d43/f65 x:0 0 0 2024-04-20T11:08:58.539 INFO:tasks.workunit.client.0.smithi117.stdout:7/213: rename d1/d4/d39/d1f to d1/d1d/d42 0 2024-04-20T11:08:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:4/273: rename d1/dd/d1a/f48 to d1/d9/f66 0 2024-04-20T11:08:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:7/214: rmdir d1/d4 39 2024-04-20T11:08:58.545 INFO:tasks.workunit.client.0.smithi117.stdout:4/274: chown d1/dd/d1a/c28 21831 1 2024-04-20T11:08:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:7/215: mkdir d1/d43 0 2024-04-20T11:08:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:7/216: chown d1/d4/d3d 2322597 1 2024-04-20T11:08:58.563 INFO:tasks.workunit.client.0.smithi117.stdout:0/180: dwrite d3/d7/f9 [0,4194304] 0 2024-04-20T11:08:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:0/181: symlink d3/d7/d16/l45 0 2024-04-20T11:08:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:9/247: truncate d4/d5/f2a 3446635 0 2024-04-20T11:08:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:0/182: chown d3/d7/d16/f2e 832128235 1 2024-04-20T11:08:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:1/227: write d6/d21/f2d [266160,55661] 0 2024-04-20T11:08:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:0/183: write d3/d7/d20/f25 [3228701,78159] 0 2024-04-20T11:08:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:9/248: truncate d4/d5/f28 1242815 0 2024-04-20T11:08:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:0/184: dread d3/d7/d16/f2b [0,4194304] 0 2024-04-20T11:08:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:1/228: mkdir d6/d16/d4a 0 2024-04-20T11:08:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:7/217: dread d1/f10 [0,4194304] 0 2024-04-20T11:08:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:4/275: dwrite d1/f2 [0,4194304] 0 2024-04-20T11:08:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:7/218: rename d1/d4/c5 to d1/d1d/d42/c44 0 2024-04-20T11:08:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:4/276: write d1/f7 [1687229,99871] 0 2024-04-20T11:08:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:5/305: sync 2024-04-20T11:08:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:2/280: sync 2024-04-20T11:08:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:3/226: sync 2024-04-20T11:08:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:7/219: mknod d1/d1d/d42/d2d/c45 0 2024-04-20T11:08:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:2/281: stat d1/d15/l1b 0 2024-04-20T11:08:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:1/229: dwrite d6/d21/f31 [0,4194304] 0 2024-04-20T11:08:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:0/185: dwrite d3/d7/f3b [0,4194304] 0 2024-04-20T11:08:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:9/249: dwrite d4/de/d10/f5c [0,4194304] 0 2024-04-20T11:08:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:4/277: write d1/d9/d1f/f55 [572846,124744] 0 2024-04-20T11:08:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:7/220: creat d1/d4/d6/d13/f46 x:0 0 0 2024-04-20T11:08:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:4/278: chown d1/c27 11050 1 2024-04-20T11:08:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:3/227: creat d0/d5/de/d4e/f53 x:0 0 0 2024-04-20T11:08:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:7/221: write d1/d4/d39/f34 [1469410,111351] 0 2024-04-20T11:08:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:3/228: fsync d0/d5/d7/f50 0 2024-04-20T11:08:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:0/186: creat d3/d7/d8/df/f46 x:0 0 0 2024-04-20T11:08:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:9/250: mkdir d4/d4a/d5e 0 2024-04-20T11:08:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:0/187: mkdir d3/d7/d20/d47 0 2024-04-20T11:08:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:4/279: getdents d1/d9 0 2024-04-20T11:08:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:1/230: dwrite d6/f2e [0,4194304] 0 2024-04-20T11:08:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:7/222: link d1/d1d/c35 d1/d4/d6/d2b/c47 0 2024-04-20T11:08:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:0/188: truncate d3/f21 568288 0 2024-04-20T11:08:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:3/229: dwrite d0/d5/d7/dd/f10 [4194304,4194304] 0 2024-04-20T11:08:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:7/223: unlink d1/d4/f1a 0 2024-04-20T11:08:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:5/306: truncate de/d21/d35/f3a 392699 0 2024-04-20T11:08:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:2/282: write d1/d4/dd/d17/f1e [1630413,21396] 0 2024-04-20T11:08:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:7/224: write d1/d1d/f21 [4742658,34626] 0 2024-04-20T11:08:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:9/251: dwrite d4/d35/f56 [0,4194304] 0 2024-04-20T11:08:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:5/307: creat de/f68 x:0 0 0 2024-04-20T11:08:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:7/225: fsync d1/f10 0 2024-04-20T11:08:58.684 INFO:tasks.workunit.client.0.smithi117.stdout:7/226: fsync d1/f22 0 2024-04-20T11:08:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:9/252: truncate d4/d5/d3c/f46 330608 0 2024-04-20T11:08:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:2/283: truncate d1/f6 160249 0 2024-04-20T11:08:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:1/231: dwrite d6/f18 [0,4194304] 0 2024-04-20T11:08:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:5/308: mknod de/d5b/d65/c69 0 2024-04-20T11:08:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:2/284: fsync d1/d4/dd/f19 0 2024-04-20T11:08:58.704 INFO:tasks.workunit.client.0.smithi117.stdout:7/227: truncate d1/d4/d6/d17/f3a 456915 0 2024-04-20T11:08:58.705 INFO:tasks.workunit.client.0.smithi117.stdout:3/230: dwrite d0/f35 [0,4194304] 0 2024-04-20T11:08:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:5/309: chown de/f18 595924 1 2024-04-20T11:08:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:3/231: dread - d0/d5/d7/dd/f3a zero size 2024-04-20T11:08:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:7/228: creat d1/f48 x:0 0 0 2024-04-20T11:08:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:3/232: rmdir d0/d5/d7 39 2024-04-20T11:08:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:2/285: creat d1/d4/dd/f55 x:0 0 0 2024-04-20T11:08:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:7/229: unlink d1/l41 0 2024-04-20T11:08:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:2/286: creat d1/d4/dd/d3e/f56 x:0 0 0 2024-04-20T11:08:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:3/233: write d0/d5/de/d1d/d36/f33 [998100,76275] 0 2024-04-20T11:08:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:1/232: dwrite f2 [4194304,4194304] 0 2024-04-20T11:08:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:9/253: dwrite f0 [0,4194304] 0 2024-04-20T11:08:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:1/233: write d6/f9 [285981,60581] 0 2024-04-20T11:08:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:2/287: symlink d1/d15/d1d/l57 0 2024-04-20T11:08:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:2/288: chown d1/d4/dd/de 43034 1 2024-04-20T11:08:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:9/254: stat d4/d5/da/fb 0 2024-04-20T11:08:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:3/234: mknod d0/c54 0 2024-04-20T11:08:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:1/234: symlink d6/d21/l4b 0 2024-04-20T11:08:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:2/289: write d1/d4/f7 [3334342,3475] 0 2024-04-20T11:08:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:9/255: read d4/d5/da/fc [750811,41010] 0 2024-04-20T11:08:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:5/310: dwrite de/d41/d4b/f5a [0,4194304] 0 2024-04-20T11:08:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:3/235: symlink d0/d5/d7/l55 0 2024-04-20T11:08:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:7/230: dwrite d1/d4/d6/d13/f28 [0,4194304] 0 2024-04-20T11:08:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:3/236: stat d0/d5/d7/dd/c43 0 2024-04-20T11:08:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:7/231: mknod d1/d1d/c49 0 2024-04-20T11:08:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:2/290: unlink d1/d20/d26/d49/l51 0 2024-04-20T11:08:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:7/232: write d1/f22 [2691184,102683] 0 2024-04-20T11:08:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:5/311: link lc de/d16/l6a 0 2024-04-20T11:08:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:3/237: link d0/d5/f19 d0/d5/d7/dd/f56 0 2024-04-20T11:08:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:6/261: sync 2024-04-20T11:08:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:8/147: sync 2024-04-20T11:08:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:1/235: dwrite d6/d21/f33 [0,4194304] 0 2024-04-20T11:08:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:3/238: unlink d0/d5/d11/f49 0 2024-04-20T11:08:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:5/312: fsync de/d21/d35/f43 0 2024-04-20T11:08:58.790 INFO:tasks.workunit.client.0.smithi117.stdout:2/291: mkdir d1/d4/dd/de/d29/d2e/d53/d58 0 2024-04-20T11:08:58.791 INFO:tasks.workunit.client.0.smithi117.stdout:9/256: dwrite d4/de/d10/f5c [0,4194304] 0 2024-04-20T11:08:58.792 INFO:tasks.workunit.client.0.smithi117.stdout:8/148: creat d0/d9/f31 x:0 0 0 2024-04-20T11:08:58.796 INFO:tasks.workunit.client.0.smithi117.stdout:6/262: symlink d3/d4/d2e/d37/d4a/l56 0 2024-04-20T11:08:58.796 INFO:tasks.workunit.client.0.smithi117.stdout:5/313: symlink de/d16/l6b 0 2024-04-20T11:08:58.800 INFO:tasks.workunit.client.0.smithi117.stdout:8/149: write d0/f20 [658281,88699] 0 2024-04-20T11:08:58.801 INFO:tasks.workunit.client.0.smithi117.stdout:3/239: symlink d0/d5/de/d1d/d29/l57 0 2024-04-20T11:08:58.806 INFO:tasks.workunit.client.0.smithi117.stdout:3/240: dread d0/d5/d21/f41 [0,4194304] 0 2024-04-20T11:08:58.806 INFO:tasks.workunit.client.0.smithi117.stdout:2/292: mknod d1/d4/dd/d3e/c59 0 2024-04-20T11:08:58.811 INFO:tasks.workunit.client.0.smithi117.stdout:2/293: write d1/d4/f7 [874275,52863] 0 2024-04-20T11:08:58.815 INFO:tasks.workunit.client.0.smithi117.stdout:8/150: truncate d0/d8/d11/d24/d2b/f2c 905437 0 2024-04-20T11:08:58.819 INFO:tasks.workunit.client.0.smithi117.stdout:6/263: mknod d3/c57 0 2024-04-20T11:08:58.824 INFO:tasks.workunit.client.0.smithi117.stdout:3/241: write d0/fb [246025,27385] 0 2024-04-20T11:08:58.824 INFO:tasks.workunit.client.0.smithi117.stdout:6/264: read d3/d4/f1b [1917251,103173] 0 2024-04-20T11:08:58.826 INFO:tasks.workunit.client.0.smithi117.stdout:9/257: link d4/d5/f2b d4/d35/f5f 0 2024-04-20T11:08:58.826 INFO:tasks.workunit.client.0.smithi117.stdout:6/265: chown d3/d4/f1b 965 1 2024-04-20T11:08:58.827 INFO:tasks.workunit.client.0.smithi117.stdout:2/294: mknod d1/d4/dd/de/d29/c5a 0 2024-04-20T11:08:58.830 INFO:tasks.workunit.client.0.smithi117.stdout:8/151: creat d0/d9/f32 x:0 0 0 2024-04-20T11:08:58.835 INFO:tasks.workunit.client.0.smithi117.stdout:9/258: rmdir d4/d5/d3c 39 2024-04-20T11:08:58.838 INFO:tasks.workunit.client.0.smithi117.stdout:9/259: write d4/d5/da/f43 [716073,41628] 0 2024-04-20T11:08:58.841 INFO:tasks.workunit.client.0.smithi117.stdout:6/266: write d3/d4/f2f [437002,34329] 0 2024-04-20T11:08:58.843 INFO:tasks.workunit.client.0.smithi117.stdout:8/152: rename d0/d8/d11/f19 to d0/d9/f33 0 2024-04-20T11:08:58.844 INFO:tasks.workunit.client.0.smithi117.stdout:5/314: link de/d21/d35/d57/c64 de/d41/d47/d4c/c6c 0 2024-04-20T11:08:58.844 INFO:tasks.workunit.client.0.smithi117.stdout:3/242: dwrite d0/d5/d7/dd/f3a [0,4194304] 0 2024-04-20T11:08:58.849 INFO:tasks.workunit.client.0.smithi117.stdout:7/233: truncate d1/d4/d39/f34 439464 0 2024-04-20T11:08:58.849 INFO:tasks.workunit.client.0.smithi117.stdout:2/295: dwrite d1/d4/dd/d17/f1e [0,4194304] 0 2024-04-20T11:08:58.849 INFO:tasks.workunit.client.0.smithi117.stdout:6/267: write d3/d3a/f4d [141492,14485] 0 2024-04-20T11:08:58.850 INFO:tasks.workunit.client.0.smithi117.stdout:8/153: chown d0/d9/fe 482110 1 2024-04-20T11:08:58.851 INFO:tasks.workunit.client.0.smithi117.stdout:9/260: read d4/d5/da/f19 [124430,92268] 0 2024-04-20T11:08:58.852 INFO:tasks.workunit.client.0.smithi117.stdout:7/234: readlink d1/d4/l2e 0 2024-04-20T11:08:58.854 INFO:tasks.workunit.client.0.smithi117.stdout:3/243: mkdir d0/d2a/d58 0 2024-04-20T11:08:58.854 INFO:tasks.workunit.client.0.smithi117.stdout:2/296: dread - d1/d4/dd/f55 zero size 2024-04-20T11:08:58.857 INFO:tasks.workunit.client.0.smithi117.stdout:8/154: chown d0/d9/f31 792528 1 2024-04-20T11:08:58.867 INFO:tasks.workunit.client.0.smithi117.stdout:9/261: rename d4/de/f2c to d4/d4a/f60 0 2024-04-20T11:08:58.876 INFO:tasks.workunit.client.0.smithi117.stdout:5/315: dread de/d41/d4b/f5a [0,4194304] 0 2024-04-20T11:08:58.876 INFO:tasks.workunit.client.0.smithi117.stdout:9/262: fsync d4/de/f1b 0 2024-04-20T11:08:58.878 INFO:tasks.workunit.client.0.smithi117.stdout:3/244: link d0/d5/fc d0/d5/d7/dd/f59 0 2024-04-20T11:08:58.880 INFO:tasks.workunit.client.0.smithi117.stdout:6/268: dwrite d3/d4/d2e/d37/f47 [0,4194304] 0 2024-04-20T11:08:58.885 INFO:tasks.workunit.client.0.smithi117.stdout:9/263: symlink d4/d5/d57/l61 0 2024-04-20T11:08:58.889 INFO:tasks.workunit.client.0.smithi117.stdout:3/245: truncate d0/f3 1179971 0 2024-04-20T11:08:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:58 smithi117 ceph-mon[15659]: mgrmap e32: smithi117.ivnklj(active, since 11s), standbys: smithi173.gnrjof 2024-04-20T11:08:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:58 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi173.gnrjof", "id": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:08:58.891 INFO:tasks.workunit.client.0.smithi117.stdout:3/246: read d0/f35 [55380,60832] 0 2024-04-20T11:08:58.891 INFO:tasks.workunit.client.0.smithi117.stdout:6/269: mknod d3/df/c58 0 2024-04-20T11:08:58.894 INFO:tasks.workunit.client.0.smithi117.stdout:6/270: dread - d3/df/d39/f4b zero size 2024-04-20T11:08:58.896 INFO:tasks.workunit.client.0.smithi117.stdout:6/271: write d3/f18 [1007541,128618] 0 2024-04-20T11:08:58.898 INFO:tasks.workunit.client.0.smithi117.stdout:3/247: creat d0/d5/d7/dd/f5a x:0 0 0 2024-04-20T11:08:58.900 INFO:tasks.workunit.client.0.smithi117.stdout:6/272: fdatasync d3/d4/d2e/f40 0 2024-04-20T11:08:58.901 INFO:tasks.workunit.client.0.smithi117.stdout:5/316: dwrite de/f1f [0,4194304] 0 2024-04-20T11:08:58.902 INFO:tasks.workunit.client.0.smithi117.stdout:2/297: truncate d1/d4/dd/de/f34 1397254 0 2024-04-20T11:08:58.906 INFO:tasks.workunit.client.0.smithi117.stdout:9/264: link d4/de/d4b/f58 d4/d5/da/f62 0 2024-04-20T11:08:58.907 INFO:tasks.workunit.client.0.smithi117.stdout:8/155: dread d0/f5 [0,4194304] 0 2024-04-20T11:08:58.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:58 smithi173 ceph-mon[23800]: mgrmap e32: smithi117.ivnklj(active, since 11s), standbys: smithi173.gnrjof 2024-04-20T11:08:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:58 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi173.gnrjof", "id": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:08:58.913 INFO:tasks.workunit.client.0.smithi117.stdout:8/156: fdatasync d0/d8/d11/d24/d2b/f2c 0 2024-04-20T11:08:58.913 INFO:tasks.workunit.client.0.smithi117.stdout:7/235: dwrite d1/f10 [4194304,4194304] 0 2024-04-20T11:08:58.915 INFO:tasks.workunit.client.0.smithi117.stdout:9/265: mkdir d4/d4a/d63 0 2024-04-20T11:08:58.915 INFO:tasks.workunit.client.0.smithi117.stdout:3/248: symlink d0/d5/d7/l5b 0 2024-04-20T11:08:58.917 INFO:tasks.workunit.client.0.smithi117.stdout:2/298: dread d1/f6 [0,4194304] 0 2024-04-20T11:08:58.918 INFO:tasks.workunit.client.0.smithi117.stdout:6/273: creat d3/d50/f59 x:0 0 0 2024-04-20T11:08:58.921 INFO:tasks.workunit.client.0.smithi117.stdout:2/299: write d1/d4/f8 [656776,65196] 0 2024-04-20T11:08:58.924 INFO:tasks.workunit.client.0.smithi117.stdout:6/274: stat d3/c2d 0 2024-04-20T11:08:58.926 INFO:tasks.workunit.client.0.smithi117.stdout:2/300: mknod d1/d4/dd/d3e/c5b 0 2024-04-20T11:08:58.927 INFO:tasks.workunit.client.0.smithi117.stdout:6/275: readlink d3/d4/d2e/l45 0 2024-04-20T11:08:58.928 INFO:tasks.workunit.client.0.smithi117.stdout:9/266: rmdir d4/d5/da/d51/d59 0 2024-04-20T11:08:58.929 INFO:tasks.workunit.client.0.smithi117.stdout:2/301: dread - d1/d20/d26/d49/f54 zero size 2024-04-20T11:08:58.931 INFO:tasks.workunit.client.0.smithi117.stdout:9/267: chown d4/d35/c44 172254 1 2024-04-20T11:08:58.937 INFO:tasks.workunit.client.0.smithi117.stdout:9/268: mknod d4/d30/d31/c64 0 2024-04-20T11:08:58.937 INFO:tasks.workunit.client.0.smithi117.stdout:3/249: dwrite d0/d5/de/d1d/d29/f31 [0,4194304] 0 2024-04-20T11:08:58.940 INFO:tasks.workunit.client.0.smithi117.stdout:6/276: mkdir d3/d5a 0 2024-04-20T11:08:58.942 INFO:tasks.workunit.client.0.smithi117.stdout:3/250: rmdir d0 39 2024-04-20T11:08:58.944 INFO:tasks.workunit.client.0.smithi117.stdout:6/277: dread - d3/d4/f46 zero size 2024-04-20T11:08:58.948 INFO:tasks.workunit.client.0.smithi117.stdout:9/269: mknod d4/d5/d3c/c65 0 2024-04-20T11:08:58.950 INFO:tasks.workunit.client.0.smithi117.stdout:8/157: write d0/dc/f1a [971658,116952] 0 2024-04-20T11:08:58.966 INFO:tasks.workunit.client.0.smithi117.stdout:7/236: dwrite d1/f7 [0,4194304] 0 2024-04-20T11:08:58.967 INFO:tasks.workunit.client.0.smithi117.stdout:3/251: write d0/d5/d21/f41 [732549,5286] 0 2024-04-20T11:08:58.970 INFO:tasks.workunit.client.0.smithi117.stdout:7/237: mknod d1/d1d/d42/d2d/c4a 0 2024-04-20T11:08:58.972 INFO:tasks.workunit.client.0.smithi117.stdout:3/252: symlink d0/d5/de/d1d/d3c/l5c 0 2024-04-20T11:08:58.974 INFO:tasks.workunit.client.0.smithi117.stdout:7/238: mknod d1/d4/d39/c4b 0 2024-04-20T11:08:58.976 INFO:tasks.workunit.client.0.smithi117.stdout:2/302: dwrite d1/f11 [0,4194304] 0 2024-04-20T11:08:58.979 INFO:tasks.workunit.client.0.smithi117.stdout:3/253: symlink d0/l5d 0 2024-04-20T11:08:58.982 INFO:tasks.workunit.client.0.smithi117.stdout:7/239: dread - d1/d4/d6/d13/f46 zero size 2024-04-20T11:08:58.982 INFO:tasks.workunit.client.0.smithi117.stdout:8/158: dwrite d0/f3 [0,4194304] 0 2024-04-20T11:08:58.983 INFO:tasks.workunit.client.0.smithi117.stdout:2/303: rename d1/d4 to d1/d4/dd/de/d29/d2e/d53/d58/d5c 22 2024-04-20T11:08:58.985 INFO:tasks.workunit.client.0.smithi117.stdout:8/159: write d0/f23 [43197,11693] 0 2024-04-20T11:08:58.993 INFO:tasks.workunit.client.0.smithi117.stdout:9/270: dread d4/d30/f4e [0,4194304] 0 2024-04-20T11:08:58.994 INFO:tasks.workunit.client.0.smithi117.stdout:7/240: rename d1/f48 to d1/d4/d6/d17/f4c 0 2024-04-20T11:08:58.995 INFO:tasks.workunit.client.0.smithi117.stdout:8/160: truncate d0/f7 1075401 0 2024-04-20T11:08:58.996 INFO:tasks.workunit.client.0.smithi117.stdout:2/304: link d1/d20/d26/c44 d1/d20/d26/c5d 0 2024-04-20T11:08:58.997 INFO:tasks.workunit.client.0.smithi117.stdout:6/278: dread d3/f2c [0,4194304] 0 2024-04-20T11:08:58.998 INFO:tasks.workunit.client.0.smithi117.stdout:7/241: dread - d1/d4/d6/d17/f4c zero size 2024-04-20T11:08:59.000 INFO:tasks.workunit.client.0.smithi117.stdout:6/279: chown d3/d4/d2e/d3f 1008852 1 2024-04-20T11:08:59.000 INFO:tasks.workunit.client.0.smithi117.stdout:3/254: dread d0/f2c [0,4194304] 0 2024-04-20T11:08:59.004 INFO:tasks.workunit.client.0.smithi117.stdout:8/161: creat d0/f34 x:0 0 0 2024-04-20T11:08:59.006 INFO:tasks.workunit.client.0.smithi117.stdout:7/242: rename d1/d4/d6/d13/f46 to d1/d4/d39/f4d 0 2024-04-20T11:08:59.010 INFO:tasks.workunit.client.0.smithi117.stdout:8/162: fsync d0/f1 0 2024-04-20T11:08:59.018 INFO:tasks.workunit.client.0.smithi117.stdout:9/271: truncate d4/d5/da/f29 4101773 0 2024-04-20T11:08:59.020 INFO:tasks.workunit.client.0.smithi117.stdout:6/280: dwrite d3/d3a/f4d [0,4194304] 0 2024-04-20T11:08:59.020 INFO:tasks.workunit.client.0.smithi117.stdout:2/305: dwrite d1/d20/d26/f37 [0,4194304] 0 2024-04-20T11:08:59.022 INFO:tasks.workunit.client.0.smithi117.stdout:7/243: getdents d1/d4/d6 0 2024-04-20T11:08:59.024 INFO:tasks.workunit.client.0.smithi117.stdout:9/272: mknod d4/d5/da/d51/c66 0 2024-04-20T11:08:59.036 INFO:tasks.workunit.client.0.smithi117.stdout:2/306: write d1/d20/d26/d49/f54 [1000723,53298] 0 2024-04-20T11:08:59.036 INFO:tasks.workunit.client.0.smithi117.stdout:9/273: read d4/d35/f56 [925943,42205] 0 2024-04-20T11:08:59.037 INFO:tasks.workunit.client.0.smithi117.stdout:7/244: mkdir d1/d1d/d4e 0 2024-04-20T11:08:59.037 INFO:tasks.workunit.client.0.smithi117.stdout:8/163: dwrite d0/d8/d11/f1c [0,4194304] 0 2024-04-20T11:08:59.037 INFO:tasks.workunit.client.0.smithi117.stdout:8/164: write d0/f34 [946136,49889] 0 2024-04-20T11:08:59.037 INFO:tasks.workunit.client.0.smithi117.stdout:6/281: mknod d3/c5b 0 2024-04-20T11:08:59.038 INFO:tasks.workunit.client.0.smithi117.stdout:8/165: fdatasync d0/d8/f16 0 2024-04-20T11:08:59.043 INFO:tasks.workunit.client.0.smithi117.stdout:7/245: mknod d1/d4/d6/c4f 0 2024-04-20T11:08:59.048 INFO:tasks.workunit.client.0.smithi117.stdout:8/166: unlink d0/d9/fe 0 2024-04-20T11:08:59.048 INFO:tasks.workunit.client.0.smithi117.stdout:7/246: symlink d1/d1d/d42/d2d/l50 0 2024-04-20T11:08:59.051 INFO:tasks.workunit.client.0.smithi117.stdout:7/247: write d1/d4/d6/f26 [3179327,50634] 0 2024-04-20T11:08:59.051 INFO:tasks.workunit.client.0.smithi117.stdout:8/167: mknod d0/d8/d11/d24/d2b/c35 0 2024-04-20T11:08:59.053 INFO:tasks.workunit.client.0.smithi117.stdout:2/307: rename d1/d4/dd/l13 to d1/d15/l5e 0 2024-04-20T11:08:59.056 INFO:tasks.workunit.client.0.smithi117.stdout:2/308: write d1/d4/dd/f55 [852665,40857] 0 2024-04-20T11:08:59.060 INFO:tasks.workunit.client.0.smithi117.stdout:9/274: write d4/d5/f2b [456199,16923] 0 2024-04-20T11:08:59.061 INFO:tasks.workunit.client.0.smithi117.stdout:6/282: link d3/d4/c23 d3/d3a/c5c 0 2024-04-20T11:08:59.062 INFO:tasks.workunit.client.0.smithi117.stdout:2/309: rmdir d1/d4/dd/d17 39 2024-04-20T11:08:59.064 INFO:tasks.workunit.client.0.smithi117.stdout:2/310: read - d1/d4/dd/de/f3a zero size 2024-04-20T11:08:59.066 INFO:tasks.workunit.client.0.smithi117.stdout:6/283: mknod d3/df/c5d 0 2024-04-20T11:08:59.068 INFO:tasks.workunit.client.0.smithi117.stdout:9/275: symlink d4/de/d10/l67 0 2024-04-20T11:08:59.068 INFO:tasks.workunit.client.0.smithi117.stdout:8/168: getdents d0/d8/d11/d24 0 2024-04-20T11:08:59.070 INFO:tasks.workunit.client.0.smithi117.stdout:8/169: chown d0/d8/d11 14506 1 2024-04-20T11:08:59.070 INFO:tasks.workunit.client.0.smithi117.stdout:9/276: readlink d4/d5/da/l15 0 2024-04-20T11:08:59.072 INFO:tasks.workunit.client.0.smithi117.stdout:9/277: write d4/d30/d31/f33 [1897054,30550] 0 2024-04-20T11:08:59.073 INFO:tasks.workunit.client.0.smithi117.stdout:6/284: rename d3/c38 to d3/d4/d2e/d3f/c5e 0 2024-04-20T11:08:59.073 INFO:tasks.workunit.client.0.smithi117.stdout:8/170: dread d0/f23 [0,4194304] 0 2024-04-20T11:08:59.078 INFO:tasks.workunit.client.0.smithi117.stdout:9/278: chown d4/d5/da/fb 59425816 1 2024-04-20T11:08:59.078 INFO:tasks.workunit.client.0.smithi117.stdout:8/171: chown d0/f20 194 1 2024-04-20T11:08:59.078 INFO:tasks.workunit.client.0.smithi117.stdout:9/279: stat d4/d5/f2a 0 2024-04-20T11:08:59.079 INFO:tasks.workunit.client.0.smithi117.stdout:6/285: mkdir d3/d4/d2e/d5f 0 2024-04-20T11:08:59.083 INFO:tasks.workunit.client.0.smithi117.stdout:2/311: dwrite d1/d4/f7 [0,4194304] 0 2024-04-20T11:08:59.084 INFO:tasks.workunit.client.0.smithi117.stdout:8/172: truncate d0/f1 2125004 0 2024-04-20T11:08:59.085 INFO:tasks.workunit.client.0.smithi117.stdout:2/312: chown d1/d4/dd/de/f3a 13070 1 2024-04-20T11:08:59.086 INFO:tasks.workunit.client.0.smithi117.stdout:9/280: mkdir d4/d68 0 2024-04-20T11:08:59.090 INFO:tasks.workunit.client.0.smithi117.stdout:8/173: symlink d0/d8/d11/l36 0 2024-04-20T11:08:59.091 INFO:tasks.workunit.client.0.smithi117.stdout:7/248: truncate d1/d4/d6/d13/f15 754664 0 2024-04-20T11:08:59.093 INFO:tasks.workunit.client.0.smithi117.stdout:6/286: creat d3/d4/d2e/d55/f60 x:0 0 0 2024-04-20T11:08:59.098 INFO:tasks.workunit.client.0.smithi117.stdout:9/281: dread d4/de/f2d [0,4194304] 0 2024-04-20T11:08:59.101 INFO:tasks.workunit.client.0.smithi117.stdout:2/313: mknod d1/d4/dd/de/d29/d2e/d53/d58/c5f 0 2024-04-20T11:08:59.103 INFO:tasks.workunit.client.0.smithi117.stdout:2/314: chown d1/f48 2979 1 2024-04-20T11:08:59.104 INFO:tasks.workunit.client.0.smithi117.stdout:7/249: rename d1/d1d/d42/c44 to d1/d4/d6/d17/c51 0 2024-04-20T11:08:59.106 INFO:tasks.workunit.client.0.smithi117.stdout:0/189: sync 2024-04-20T11:08:59.106 INFO:tasks.workunit.client.0.smithi117.stdout:2/315: fsync d1/d4/dd/de/d29/f4c 0 2024-04-20T11:08:59.107 INFO:tasks.workunit.client.0.smithi117.stdout:4/280: sync 2024-04-20T11:08:59.107 INFO:tasks.workunit.client.0.smithi117.stdout:1/236: sync 2024-04-20T11:08:59.108 INFO:tasks.workunit.client.0.smithi117.stdout:9/282: mknod d4/d5/c69 0 2024-04-20T11:08:59.111 INFO:tasks.workunit.client.0.smithi117.stdout:4/281: dread - d1/d45/d4a/f50 zero size 2024-04-20T11:08:59.111 INFO:tasks.workunit.client.0.smithi117.stdout:8/174: dwrite d0/d8/d11/f1c [0,4194304] 0 2024-04-20T11:08:59.113 INFO:tasks.workunit.client.0.smithi117.stdout:2/316: stat d1/d15/d1d 0 2024-04-20T11:08:59.115 INFO:tasks.workunit.client.0.smithi117.stdout:7/250: rename d1/d4/d39/c4b to d1/d4/d6/d17/d3f/c52 0 2024-04-20T11:08:59.117 INFO:tasks.workunit.client.0.smithi117.stdout:0/190: mknod d3/d7/d20/d39/c48 0 2024-04-20T11:08:59.118 INFO:tasks.workunit.client.0.smithi117.stdout:2/317: truncate d1/d4/dd/de/d29/f4c 371676 0 2024-04-20T11:08:59.119 INFO:tasks.workunit.client.0.smithi117.stdout:6/287: getdents d3/d4/d2e/d55 0 2024-04-20T11:08:59.120 INFO:tasks.workunit.client.0.smithi117.stdout:6/288: chown d3 5 1 2024-04-20T11:08:59.121 INFO:tasks.workunit.client.0.smithi117.stdout:0/191: truncate d3/d7/f3f 854388 0 2024-04-20T11:08:59.123 INFO:tasks.workunit.client.0.smithi117.stdout:6/289: read d3/df/f14 [6543485,87915] 0 2024-04-20T11:08:59.123 INFO:tasks.workunit.client.0.smithi117.stdout:4/282: creat d1/f67 x:0 0 0 2024-04-20T11:08:59.124 INFO:tasks.workunit.client.0.smithi117.stdout:8/175: symlink d0/d8/d11/d24/d2b/l37 0 2024-04-20T11:08:59.125 INFO:tasks.workunit.client.0.smithi117.stdout:4/283: dread - d1/f67 zero size 2024-04-20T11:08:59.126 INFO:tasks.workunit.client.0.smithi117.stdout:4/284: chown d1/d9/d1f/f55 8486 1 2024-04-20T11:08:59.129 INFO:tasks.workunit.client.0.smithi117.stdout:1/237: symlink d6/d1f/d30/l4c 0 2024-04-20T11:08:59.132 INFO:tasks.workunit.client.0.smithi117.stdout:0/192: mknod d3/d7/d8/df/d22/c49 0 2024-04-20T11:08:59.134 INFO:tasks.workunit.client.0.smithi117.stdout:6/290: rmdir d3/d4/d2e/d37 39 2024-04-20T11:08:59.138 INFO:tasks.workunit.client.0.smithi117.stdout:2/318: mknod d1/d4/c60 0 2024-04-20T11:08:59.142 INFO:tasks.workunit.client.0.smithi117.stdout:0/193: mkdir d3/d7/d8/df/d4a 0 2024-04-20T11:08:59.143 INFO:tasks.workunit.client.0.smithi117.stdout:1/238: symlink d6/d1f/d30/l4d 0 2024-04-20T11:08:59.143 INFO:tasks.workunit.client.0.smithi117.stdout:6/291: chown d3/d4/f8 61485 1 2024-04-20T11:08:59.144 INFO:tasks.workunit.client.0.smithi117.stdout:2/319: write d1/d4/dd/de/f23 [2988549,2002] 0 2024-04-20T11:08:59.146 INFO:tasks.workunit.client.0.smithi117.stdout:0/194: fdatasync d3/d5/f26 0 2024-04-20T11:08:59.146 INFO:tasks.workunit.client.0.smithi117.stdout:4/285: creat d1/dd/f68 x:0 0 0 2024-04-20T11:08:59.147 INFO:tasks.workunit.client.0.smithi117.stdout:2/320: fdatasync d1/d4/dd/de/d29/f4c 0 2024-04-20T11:08:59.148 INFO:tasks.workunit.client.0.smithi117.stdout:4/286: chown d1/dd/d1a/c28 71692265 1 2024-04-20T11:08:59.149 INFO:tasks.workunit.client.0.smithi117.stdout:6/292: creat d3/d3a/f61 x:0 0 0 2024-04-20T11:08:59.153 INFO:tasks.workunit.client.0.smithi117.stdout:4/287: mknod d1/d45/d4a/c69 0 2024-04-20T11:08:59.154 INFO:tasks.workunit.client.0.smithi117.stdout:8/176: truncate d0/dc/f1e 641940 0 2024-04-20T11:08:59.235 INFO:tasks.workunit.client.0.smithi117.stdout:5/317: sync 2024-04-20T11:08:59.236 INFO:tasks.workunit.client.0.smithi117.stdout:2/321: symlink d1/l61 0 2024-04-20T11:08:59.237 INFO:tasks.workunit.client.0.smithi117.stdout:1/239: creat d6/d1f/d30/f4e x:0 0 0 2024-04-20T11:08:59.240 INFO:tasks.workunit.client.0.smithi117.stdout:5/318: write de/d41/f52 [115682,84819] 0 2024-04-20T11:08:59.241 INFO:tasks.workunit.client.0.smithi117.stdout:6/293: creat d3/d4/d2e/d37/f62 x:0 0 0 2024-04-20T11:08:59.241 INFO:tasks.workunit.client.0.smithi117.stdout:1/240: fsync d6/d1f/f3e 0 2024-04-20T11:08:59.241 INFO:tasks.workunit.client.0.smithi117.stdout:4/288: creat d1/d45/d4a/d59/f6a x:0 0 0 2024-04-20T11:08:59.243 INFO:tasks.workunit.client.0.smithi117.stdout:7/251: dwrite d1/d4/d39/f34 [0,4194304] 0 2024-04-20T11:08:59.245 INFO:tasks.workunit.client.0.smithi117.stdout:2/322: mkdir d1/d4/dd/de/d29/d62 0 2024-04-20T11:08:59.245 INFO:tasks.workunit.client.0.smithi117.stdout:6/294: creat d3/df/f63 x:0 0 0 2024-04-20T11:08:59.248 INFO:tasks.workunit.client.0.smithi117.stdout:1/241: rmdir d6/d35 39 2024-04-20T11:08:59.250 INFO:tasks.workunit.client.0.smithi117.stdout:6/295: stat d3/d4/l51 0 2024-04-20T11:08:59.252 INFO:tasks.workunit.client.0.smithi117.stdout:7/252: truncate d1/d4/d6/d17/f4c 417898 0 2024-04-20T11:08:59.252 INFO:tasks.workunit.client.0.smithi117.stdout:2/323: write d1/d4/dd/f55 [1743831,62060] 0 2024-04-20T11:08:59.270 INFO:tasks.workunit.client.0.smithi117.stdout:6/296: link d3/df/f63 d3/d4/d2e/d55/f64 0 2024-04-20T11:08:59.273 INFO:tasks.workunit.client.0.smithi117.stdout:1/242: mknod d6/d35/c4f 0 2024-04-20T11:08:59.282 INFO:tasks.workunit.client.0.smithi117.stdout:7/253: dwrite d1/d4/d6/d2b/f33 [0,4194304] 0 2024-04-20T11:08:59.282 INFO:tasks.workunit.client.0.smithi117.stdout:4/289: truncate d1/f35 2866759 0 2024-04-20T11:08:59.283 INFO:tasks.workunit.client.0.smithi117.stdout:2/324: dwrite d1/d4/dd/f55 [0,4194304] 0 2024-04-20T11:08:59.290 INFO:tasks.workunit.client.0.smithi117.stdout:7/254: mkdir d1/d1d/d4e/d53 0 2024-04-20T11:08:59.292 INFO:tasks.workunit.client.0.smithi117.stdout:4/290: truncate d1/dd/d1a/f1e 1205058 0 2024-04-20T11:08:59.292 INFO:tasks.workunit.client.0.smithi117.stdout:7/255: chown d1/d4/d6 0 1 2024-04-20T11:08:59.292 INFO:tasks.workunit.client.0.smithi117.stdout:2/325: symlink d1/d4/dd/de/d29/d2e/d52/l63 0 2024-04-20T11:08:59.298 INFO:tasks.workunit.client.0.smithi117.stdout:1/243: dread d6/d35/f36 [0,4194304] 0 2024-04-20T11:08:59.416 INFO:tasks.workunit.client.0.smithi117.stdout:4/291: mkdir d1/d9/d6b 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:6/297: dwrite d3/d4/d2e/d55/f64 [0,4194304] 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:1/244: read d6/d21/f33 [1601561,107968] 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:6/298: mknod d3/d5a/c65 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:1/245: readlink d6/d21/l4b 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:6/299: dread - d3/d4/d2e/d55/f60 zero size 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:1/246: chown d6/l15 66084 1 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:6/300: stat d3/d4/f1b 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:4/292: dread d1/d9/d1f/f36 [0,4194304] 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:2/326: dwrite d1/f11 [0,4194304] 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:7/256: dwrite d1/d4/f30 [0,4194304] 0 2024-04-20T11:08:59.417 INFO:tasks.workunit.client.0.smithi117.stdout:2/327: creat d1/d4/dd/de/d29/d2e/f64 x:0 0 0 2024-04-20T11:08:59.418 INFO:tasks.workunit.client.0.smithi117.stdout:1/247: rename d6/l15 to d6/d1f/d30/d49/l50 0 2024-04-20T11:08:59.418 INFO:tasks.workunit.client.0.smithi117.stdout:1/248: dwrite d6/d1f/f45 [0,4194304] 0 2024-04-20T11:08:59.418 INFO:tasks.workunit.client.0.smithi117.stdout:1/249: getdents d6 0 2024-04-20T11:08:59.418 INFO:tasks.workunit.client.0.smithi117.stdout:1/250: fsync d6/f9 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:1/251: unlink f2 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:7/257: dread d1/d4/d6/d13/f15 [0,4194304] 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:7/258: fdatasync d1/f22 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:7/259: mkdir d1/d4/d6/d54 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:6/301: dread d3/d4/f19 [0,4194304] 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:6/302: stat d3/df/l3b 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:7/260: dread d1/d4/d6/d13/f15 [0,4194304] 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:6/303: write d3/d4/d2e/f40 [453567,46753] 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:6/304: chown d3/d4/d2e/d5f 0 1 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:1/252: dwrite d6/d1f/f3d [0,4194304] 0 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:6/305: read - d3/d4/d2e/d37/f62 zero size 2024-04-20T11:08:59.419 INFO:tasks.workunit.client.0.smithi117.stdout:6/306: stat d3/f18 0 2024-04-20T11:08:59.420 INFO:tasks.workunit.client.0.smithi117.stdout:6/307: symlink d3/d4/l66 0 2024-04-20T11:08:59.420 INFO:tasks.workunit.client.0.smithi117.stdout:6/308: dread - d3/d4/d2e/d55/f60 zero size 2024-04-20T11:08:59.421 INFO:tasks.workunit.client.0.smithi117.stdout:7/261: dread d1/f10 [0,4194304] 0 2024-04-20T11:08:59.421 INFO:tasks.workunit.client.0.smithi117.stdout:7/262: dread d1/d4/d6/d17/f4c [0,4194304] 0 2024-04-20T11:08:59.422 INFO:tasks.workunit.client.0.smithi117.stdout:7/263: stat d1/d4/d6/d13 0 2024-04-20T11:08:59.422 INFO:tasks.workunit.client.0.smithi117.stdout:7/264: mkdir d1/d1d/d4e/d53/d55 0 2024-04-20T11:08:59.422 INFO:tasks.workunit.client.0.smithi117.stdout:3/255: sync 2024-04-20T11:08:59.422 INFO:tasks.workunit.client.0.smithi117.stdout:1/253: dwrite d6/d21/f39 [0,4194304] 0 2024-04-20T11:08:59.422 INFO:tasks.workunit.client.0.smithi117.stdout:7/265: symlink d1/d1d/d42/d2d/l56 0 2024-04-20T11:08:59.422 INFO:tasks.workunit.client.0.smithi117.stdout:6/309: dwrite d3/d4/f19 [0,4194304] 0 2024-04-20T11:08:59.422 INFO:tasks.workunit.client.0.smithi117.stdout:7/266: write d1/f7 [2830516,122806] 0 2024-04-20T11:08:59.423 INFO:tasks.workunit.client.0.smithi117.stdout:1/254: chown d6/d1f/d30/d49/l50 6092 1 2024-04-20T11:08:59.423 INFO:tasks.workunit.client.0.smithi117.stdout:6/310: dread - d3/d4/d2e/d37/f62 zero size 2024-04-20T11:08:59.424 INFO:tasks.workunit.client.0.smithi117.stdout:7/267: mknod d1/d4/d6/d17/d3f/c57 0 2024-04-20T11:08:59.426 INFO:tasks.workunit.client.0.smithi117.stdout:6/311: creat d3/f67 x:0 0 0 2024-04-20T11:08:59.429 INFO:tasks.workunit.client.0.smithi117.stdout:7/268: mknod d1/d4/d39/c58 0 2024-04-20T11:08:59.432 INFO:tasks.workunit.client.0.smithi117.stdout:7/269: unlink d1/d4/l2e 0 2024-04-20T11:08:59.434 INFO:tasks.workunit.client.0.smithi117.stdout:6/312: write d3/d4/d2e/d55/f60 [540465,63410] 0 2024-04-20T11:08:59.437 INFO:tasks.workunit.client.0.smithi117.stdout:6/313: chown d3/df/d39/l52 4186992 1 2024-04-20T11:08:59.441 INFO:tasks.workunit.client.0.smithi117.stdout:6/314: fdatasync d3/df/d39/f3e 0 2024-04-20T11:08:59.441 INFO:tasks.workunit.client.0.smithi117.stdout:7/270: mkdir d1/d43/d59 0 2024-04-20T11:08:59.441 INFO:tasks.workunit.client.0.smithi117.stdout:3/256: truncate d0/d5/d21/f42 3558159 0 2024-04-20T11:08:59.441 INFO:tasks.workunit.client.0.smithi117.stdout:1/255: dwrite d6/d1f/d30/f4e [0,4194304] 0 2024-04-20T11:08:59.444 INFO:tasks.workunit.client.0.smithi117.stdout:7/271: write d1/d4/d6/d13/f28 [2304019,91198] 0 2024-04-20T11:08:59.445 INFO:tasks.workunit.client.0.smithi117.stdout:6/315: mknod d3/df/d39/c68 0 2024-04-20T11:08:59.447 INFO:tasks.workunit.client.0.smithi117.stdout:6/316: stat d3/df/d39/l52 0 2024-04-20T11:08:59.450 INFO:tasks.workunit.client.0.smithi117.stdout:1/256: symlink d6/l51 0 2024-04-20T11:08:59.456 INFO:tasks.workunit.client.0.smithi117.stdout:3/257: fsync d0/d5/d7/dd/f10 0 2024-04-20T11:08:59.460 INFO:tasks.workunit.client.0.smithi117.stdout:7/272: getdents d1/d1d/d4e/d53 0 2024-04-20T11:08:59.462 INFO:tasks.workunit.client.0.smithi117.stdout:3/258: rename d0/d5/d7/l8 to d0/d5/de/l5e 0 2024-04-20T11:08:59.464 INFO:tasks.workunit.client.0.smithi117.stdout:6/317: dwrite d3/d4/f46 [0,4194304] 0 2024-04-20T11:08:59.469 INFO:tasks.workunit.client.0.smithi117.stdout:6/318: rename d3/d4/l25 to d3/d5a/l69 0 2024-04-20T11:08:59.470 INFO:tasks.workunit.client.0.smithi117.stdout:6/319: rmdir d3/d5a 39 2024-04-20T11:08:59.471 INFO:tasks.workunit.client.0.smithi117.stdout:6/320: chown d3/df/c53 25553 1 2024-04-20T11:08:59.490 INFO:tasks.workunit.client.0.smithi117.stdout:6/321: dwrite d3/d50/f59 [0,4194304] 0 2024-04-20T11:08:59.495 INFO:tasks.workunit.client.0.smithi117.stdout:6/322: dread d3/d4/f8 [0,4194304] 0 2024-04-20T11:08:59.512 INFO:tasks.workunit.client.0.smithi117.stdout:1/257: chown d6/d16/l37 1082333 1 2024-04-20T11:08:59.515 INFO:tasks.workunit.client.0.smithi117.stdout:6/323: dwrite d3/fc [0,4194304] 0 2024-04-20T11:08:59.518 INFO:tasks.workunit.client.0.smithi117.stdout:1/258: write d6/d1f/f2b [661302,126302] 0 2024-04-20T11:08:59.518 INFO:tasks.workunit.client.0.smithi117.stdout:6/324: truncate d3/f18 3387296 0 2024-04-20T11:08:59.519 INFO:tasks.workunit.client.0.smithi117.stdout:1/259: fsync d6/d1f/d30/d32/f46 0 2024-04-20T11:08:59.520 INFO:tasks.workunit.client.0.smithi117.stdout:6/325: write d3/f18 [2382656,110226] 0 2024-04-20T11:08:59.527 INFO:tasks.workunit.client.0.smithi117.stdout:1/260: truncate d6/f13 2422751 0 2024-04-20T11:08:59.535 INFO:tasks.workunit.client.0.smithi117.stdout:1/261: mknod d6/d35/c52 0 2024-04-20T11:08:59.536 INFO:tasks.workunit.client.0.smithi117.stdout:1/262: dread - d6/d35/f44 zero size 2024-04-20T11:08:59.536 INFO:tasks.workunit.client.0.smithi117.stdout:6/326: dwrite d3/d4/d2e/d37/f47 [0,4194304] 0 2024-04-20T11:08:59.540 INFO:tasks.workunit.client.0.smithi117.stdout:6/327: read d3/d4/d2e/d3f/f42 [1125779,48569] 0 2024-04-20T11:08:59.543 INFO:tasks.workunit.client.0.smithi117.stdout:6/328: chown d3/d4/d2e/d37/d4a 0 1 2024-04-20T11:08:59.551 INFO:tasks.workunit.client.0.smithi117.stdout:1/263: rename d6/d1f/f24 to d6/d16/d4a/f53 0 2024-04-20T11:08:59.555 INFO:tasks.workunit.client.0.smithi117.stdout:6/329: dread d3/d3a/f4d [0,4194304] 0 2024-04-20T11:08:59.557 INFO:tasks.workunit.client.0.smithi117.stdout:1/264: link d6/d1f/l47 d6/d16/l54 0 2024-04-20T11:08:59.576 INFO:tasks.workunit.client.0.smithi117.stdout:9/283: sync 2024-04-20T11:08:59.576 INFO:tasks.workunit.client.0.smithi117.stdout:6/330: truncate d3/df/f14 4190919 0 2024-04-20T11:08:59.578 INFO:tasks.workunit.client.0.smithi117.stdout:1/265: truncate d6/d21/f2d 348148 0 2024-04-20T11:08:59.580 INFO:tasks.workunit.client.0.smithi117.stdout:1/266: mknod d6/d21/c55 0 2024-04-20T11:08:59.584 INFO:tasks.workunit.client.0.smithi117.stdout:1/267: rename d6/d21/f33 to d6/d35/f56 0 2024-04-20T11:08:59.592 INFO:tasks.workunit.client.0.smithi117.stdout:6/331: dwrite d3/d4/f46 [0,4194304] 0 2024-04-20T11:08:59.597 INFO:tasks.workunit.client.0.smithi117.stdout:9/284: dwrite d4/d5/f2a [0,4194304] 0 2024-04-20T11:08:59.597 INFO:tasks.workunit.client.0.smithi117.stdout:6/332: creat d3/d4/d2e/d37/d4a/f6a x:0 0 0 2024-04-20T11:08:59.599 INFO:tasks.workunit.client.0.smithi117.stdout:9/285: dread - d4/de/d10/f40 zero size 2024-04-20T11:08:59.601 INFO:tasks.workunit.client.0.smithi117.stdout:6/333: fsync d3/f2c 0 2024-04-20T11:08:59.602 INFO:tasks.workunit.client.0.smithi117.stdout:9/286: write d4/de/d4b/f58 [2154154,38605] 0 2024-04-20T11:08:59.610 INFO:tasks.workunit.client.0.smithi117.stdout:1/268: unlink d6/d21/f2d 0 2024-04-20T11:08:59.614 INFO:tasks.workunit.client.0.smithi117.stdout:1/269: mkdir d6/d1f/d30/d32/d57 0 2024-04-20T11:08:59.620 INFO:tasks.workunit.client.0.smithi117.stdout:6/334: dwrite d3/d4/f46 [0,4194304] 0 2024-04-20T11:08:59.621 INFO:tasks.workunit.client.0.smithi117.stdout:1/270: unlink d6/d1f/c2c 0 2024-04-20T11:08:59.623 INFO:tasks.workunit.client.0.smithi117.stdout:6/335: truncate d3/f28 4284899 0 2024-04-20T11:08:59.624 INFO:tasks.workunit.client.0.smithi117.stdout:9/287: dwrite d4/d5/d3c/f47 [4194304,4194304] 0 2024-04-20T11:08:59.630 INFO:tasks.workunit.client.0.smithi117.stdout:9/288: mknod d4/d30/c6a 0 2024-04-20T11:08:59.636 INFO:tasks.workunit.client.0.smithi117.stdout:6/336: link d3/df/f31 d3/d3a/f6b 0 2024-04-20T11:08:59.637 INFO:tasks.workunit.client.0.smithi117.stdout:9/289: truncate d4/d30/d31/f5d 784427 0 2024-04-20T11:08:59.643 INFO:tasks.workunit.client.0.smithi117.stdout:6/337: creat d3/d4/d2e/d5f/f6c x:0 0 0 2024-04-20T11:08:59.645 INFO:tasks.workunit.client.0.smithi117.stdout:6/338: write d3/f2c [4873413,24019] 0 2024-04-20T11:08:59.650 INFO:tasks.workunit.client.0.smithi117.stdout:1/271: dwrite d6/f12 [0,4194304] 0 2024-04-20T11:08:59.652 INFO:tasks.workunit.client.0.smithi117.stdout:1/272: read - d6/d35/f44 zero size 2024-04-20T11:08:59.661 INFO:tasks.workunit.client.0.smithi117.stdout:9/290: truncate d4/d5/da/f32 2130012 0 2024-04-20T11:08:59.667 INFO:tasks.workunit.client.0.smithi117.stdout:6/339: dwrite d3/d50/f59 [0,4194304] 0 2024-04-20T11:08:59.668 INFO:tasks.workunit.client.0.smithi117.stdout:9/291: rename d4/d5/f7 to d4/d30/f6b 0 2024-04-20T11:08:59.676 INFO:tasks.workunit.client.0.smithi117.stdout:1/273: dwrite d6/f18 [4194304,4194304] 0 2024-04-20T11:08:59.676 INFO:tasks.workunit.client.0.smithi117.stdout:6/340: mknod d3/d4/d2e/d37/d4a/c6d 0 2024-04-20T11:08:59.677 INFO:tasks.workunit.client.0.smithi117.stdout:9/292: creat d4/d30/d31/f6c x:0 0 0 2024-04-20T11:08:59.677 INFO:tasks.workunit.client.0.smithi117.stdout:6/341: truncate d3/d3a/f61 194173 0 2024-04-20T11:08:59.680 INFO:tasks.workunit.client.0.smithi117.stdout:6/342: write d3/d50/f59 [4936858,81894] 0 2024-04-20T11:08:59.692 INFO:tasks.workunit.client.0.smithi117.stdout:6/343: creat d3/d4/f6e x:0 0 0 2024-04-20T11:08:59.695 INFO:tasks.workunit.client.0.smithi117.stdout:1/274: dread d6/d1f/f45 [0,4194304] 0 2024-04-20T11:08:59.695 INFO:tasks.workunit.client.0.smithi117.stdout:6/344: write d3/df/f63 [68529,83729] 0 2024-04-20T11:08:59.698 INFO:tasks.workunit.client.0.smithi117.stdout:1/275: symlink d6/d21/l58 0 2024-04-20T11:08:59.701 INFO:tasks.workunit.client.0.smithi117.stdout:1/276: chown d6/l51 10569935 1 2024-04-20T11:08:59.701 INFO:tasks.workunit.client.0.smithi117.stdout:6/345: creat d3/d4/d2e/d37/d4a/f6f x:0 0 0 2024-04-20T11:08:59.702 INFO:tasks.workunit.client.0.smithi117.stdout:9/293: dwrite d4/d30/f6b [0,4194304] 0 2024-04-20T11:08:59.704 INFO:tasks.workunit.client.0.smithi117.stdout:6/346: unlink d3/f4c 0 2024-04-20T11:08:59.707 INFO:tasks.workunit.client.0.smithi117.stdout:1/277: rename d6/d1f/d30/c34 to d6/d16/d4a/c59 0 2024-04-20T11:08:59.708 INFO:tasks.workunit.client.0.smithi117.stdout:9/294: creat d4/de/d4b/f6d x:0 0 0 2024-04-20T11:08:59.714 INFO:tasks.workunit.client.0.smithi117.stdout:1/278: write d6/d35/f56 [473891,74758] 0 2024-04-20T11:08:59.716 INFO:tasks.workunit.client.0.smithi117.stdout:6/347: creat d3/d4/d2e/d4e/f70 x:0 0 0 2024-04-20T11:08:59.717 INFO:tasks.workunit.client.0.smithi117.stdout:1/279: chown d6/d16/l1e 4328541 1 2024-04-20T11:08:59.719 INFO:tasks.workunit.client.0.smithi117.stdout:6/348: write d3/d4/f46 [814794,32109] 0 2024-04-20T11:08:59.725 INFO:tasks.workunit.client.0.smithi117.stdout:1/280: getdents d6/d1f/d30/d32/d57 0 2024-04-20T11:08:59.726 INFO:tasks.workunit.client.0.smithi117.stdout:9/295: dwrite d4/d5/d3c/f47 [4194304,4194304] 0 2024-04-20T11:08:59.731 INFO:tasks.workunit.client.0.smithi117.stdout:0/195: sync 2024-04-20T11:08:59.733 INFO:tasks.workunit.client.0.smithi117.stdout:1/281: rename d6/d16/d4a to d6/d16/d3a/d5a 0 2024-04-20T11:08:59.735 INFO:tasks.workunit.client.0.smithi117.stdout:1/282: fdatasync d6/d35/f56 0 2024-04-20T11:08:59.738 INFO:tasks.workunit.client.0.smithi117.stdout:6/349: write d3/df/f14 [2798704,26699] 0 2024-04-20T11:08:59.739 INFO:tasks.workunit.client.0.smithi117.stdout:1/283: fsync d6/f10 0 2024-04-20T11:08:59.744 INFO:tasks.workunit.client.0.smithi117.stdout:1/284: unlink d6/d21/l2f 0 2024-04-20T11:08:59.748 INFO:tasks.workunit.client.0.smithi117.stdout:1/285: mknod d6/c5b 0 2024-04-20T11:08:59.750 INFO:tasks.workunit.client.0.smithi117.stdout:0/196: dwrite d3/d5/f26 [0,4194304] 0 2024-04-20T11:08:59.753 INFO:tasks.workunit.client.0.smithi117.stdout:1/286: write d6/d1f/f40 [779284,106946] 0 2024-04-20T11:08:59.760 INFO:tasks.workunit.client.0.smithi117.stdout:9/296: dread d4/de/f3e [0,4194304] 0 2024-04-20T11:08:59.761 INFO:tasks.workunit.client.0.smithi117.stdout:0/197: rename d3/d7/d8/df/d4a to d3/d7/d20/d39/d4b 0 2024-04-20T11:08:59.762 INFO:tasks.workunit.client.0.smithi117.stdout:6/350: dwrite d3/d3a/f4d [0,4194304] 0 2024-04-20T11:08:59.765 INFO:tasks.workunit.client.0.smithi117.stdout:6/351: truncate d3/d4/f19 4613684 0 2024-04-20T11:08:59.770 INFO:tasks.workunit.client.0.smithi117.stdout:0/198: dread d3/d7/d16/f2b [0,4194304] 0 2024-04-20T11:08:59.773 INFO:tasks.workunit.client.0.smithi117.stdout:0/199: chown d3/d7/le 3385790 1 2024-04-20T11:08:59.773 INFO:tasks.workunit.client.0.smithi117.stdout:6/352: mkdir d3/d4/d2e/d4e/d71 0 2024-04-20T11:08:59.779 INFO:tasks.workunit.client.0.smithi117.stdout:0/200: link d3/d7/f3f d3/d7/d20/f4c 0 2024-04-20T11:08:59.780 INFO:tasks.workunit.client.0.smithi117.stdout:1/287: dread d6/d21/f31 [0,4194304] 0 2024-04-20T11:08:59.780 INFO:tasks.workunit.client.0.smithi117.stdout:6/353: getdents d3/d4/d2e/d55 0 2024-04-20T11:08:59.783 INFO:tasks.workunit.client.0.smithi117.stdout:6/354: rename d3/df to d3/df/d39/d72 22 2024-04-20T11:08:59.784 INFO:tasks.workunit.client.0.smithi117.stdout:0/201: creat d3/d7/f4d x:0 0 0 2024-04-20T11:08:59.785 INFO:tasks.workunit.client.0.smithi117.stdout:1/288: mkdir d6/d16/d3a/d5a/d5c 0 2024-04-20T11:08:59.786 INFO:tasks.workunit.client.0.smithi117.stdout:6/355: write d3/d4/d2e/f40 [852906,129382] 0 2024-04-20T11:08:59.787 INFO:tasks.workunit.client.0.smithi117.stdout:0/202: readlink d3/l1d 0 2024-04-20T11:08:59.791 INFO:tasks.workunit.client.0.smithi117.stdout:1/289: creat d6/d16/d3a/d5a/f5d x:0 0 0 2024-04-20T11:08:59.793 INFO:tasks.workunit.client.0.smithi117.stdout:0/203: rename d3/d7/d20/l30 to d3/d7/d8/d2a/l4e 0 2024-04-20T11:08:59.794 INFO:tasks.workunit.client.0.smithi117.stdout:1/290: mknod d6/d16/d3a/d5a/d5c/c5e 0 2024-04-20T11:08:59.795 INFO:tasks.workunit.client.0.smithi117.stdout:1/291: stat d6/f12 0 2024-04-20T11:08:59.796 INFO:tasks.workunit.client.0.smithi117.stdout:0/204: symlink d3/d7/d20/d47/l4f 0 2024-04-20T11:08:59.800 INFO:tasks.workunit.client.0.smithi117.stdout:5/319: sync 2024-04-20T11:08:59.801 INFO:tasks.workunit.client.0.smithi117.stdout:8/177: sync 2024-04-20T11:08:59.802 INFO:tasks.workunit.client.0.smithi117.stdout:1/292: creat d6/d35/f5f x:0 0 0 2024-04-20T11:08:59.803 INFO:tasks.workunit.client.0.smithi117.stdout:0/205: creat d3/d7/d8/df/f50 x:0 0 0 2024-04-20T11:08:59.803 INFO:tasks.workunit.client.0.smithi117.stdout:1/293: readlink d6/d21/l3f 0 2024-04-20T11:08:59.804 INFO:tasks.workunit.client.0.smithi117.stdout:0/206: fdatasync d3/d7/f19 0 2024-04-20T11:08:59.804 INFO:tasks.workunit.client.0.smithi117.stdout:8/178: truncate d0/d9/f33 1037089 0 2024-04-20T11:08:59.805 INFO:tasks.workunit.client.0.smithi117.stdout:5/320: dread de/d21/d35/f3b [0,4194304] 0 2024-04-20T11:08:59.808 INFO:tasks.workunit.client.0.smithi117.stdout:8/179: creat d0/d8/f38 x:0 0 0 2024-04-20T11:08:59.809 INFO:tasks.workunit.client.0.smithi117.stdout:5/321: unlink de/d21/d35/f43 0 2024-04-20T11:08:59.810 INFO:tasks.workunit.client.0.smithi117.stdout:1/294: mknod d6/d1f/d30/c60 0 2024-04-20T11:08:59.812 INFO:tasks.workunit.client.0.smithi117.stdout:8/180: mkdir d0/d8/d39 0 2024-04-20T11:08:59.814 INFO:tasks.workunit.client.0.smithi117.stdout:8/181: dread - d0/d8/d11/f1f zero size 2024-04-20T11:08:59.816 INFO:tasks.workunit.client.0.smithi117.stdout:8/182: fsync d0/fa 0 2024-04-20T11:08:59.817 INFO:tasks.workunit.client.0.smithi117.stdout:1/295: getdents d6/d1f/d30/d49 0 2024-04-20T11:08:59.822 INFO:tasks.workunit.client.0.smithi117.stdout:0/207: dwrite d3/d7/d8/df/f50 [0,4194304] 0 2024-04-20T11:08:59.826 INFO:tasks.workunit.client.0.smithi117.stdout:5/322: dread f5 [0,4194304] 0 2024-04-20T11:08:59.827 INFO:tasks.workunit.client.0.smithi117.stdout:5/323: chown de/d41 3 1 2024-04-20T11:08:59.831 INFO:tasks.workunit.client.0.smithi117.stdout:6/356: dread d3/f28 [0,4194304] 0 2024-04-20T11:08:59.833 INFO:tasks.workunit.client.0.smithi117.stdout:1/296: dread d6/f3c [0,4194304] 0 2024-04-20T11:08:59.835 INFO:tasks.workunit.client.0.smithi117.stdout:5/324: symlink de/d21/d45/d48/l6d 0 2024-04-20T11:08:59.837 INFO:tasks.workunit.client.0.smithi117.stdout:5/325: write de/f18 [280222,13612] 0 2024-04-20T11:08:59.839 INFO:tasks.workunit.client.0.smithi117.stdout:0/208: readlink d3/d7/d8/d2a/l4e 0 2024-04-20T11:08:59.839 INFO:tasks.workunit.client.0.smithi117.stdout:5/326: write de/d16/f1e [4789301,115374] 0 2024-04-20T11:08:59.841 INFO:tasks.workunit.client.0.smithi117.stdout:1/297: mkdir d6/d16/d3a/d5a/d5c/d61 0 2024-04-20T11:08:59.850 INFO:tasks.workunit.client.0.smithi117.stdout:8/183: dread d0/fd [0,4194304] 0 2024-04-20T11:08:59.853 INFO:tasks.workunit.client.0.smithi117.stdout:5/327: read de/f19 [267123,16671] 0 2024-04-20T11:08:59.857 INFO:tasks.workunit.client.0.smithi117.stdout:6/357: write d3/d3a/f6b [1684935,79641] 0 2024-04-20T11:08:59.859 INFO:tasks.workunit.client.0.smithi117.stdout:8/184: fsync d0/dc/f1a 0 2024-04-20T11:08:59.860 INFO:tasks.workunit.client.0.smithi117.stdout:5/328: creat de/d41/d47/d4c/f6e x:0 0 0 2024-04-20T11:08:59.861 INFO:tasks.workunit.client.0.smithi117.stdout:8/185: write d0/f22 [425108,20040] 0 2024-04-20T11:08:59.863 INFO:tasks.workunit.client.0.smithi117.stdout:0/209: truncate d3/d7/d8/f13 2532457 0 2024-04-20T11:08:59.864 INFO:tasks.workunit.client.0.smithi117.stdout:1/298: truncate d6/f13 1904349 0 2024-04-20T11:08:59.866 INFO:tasks.workunit.client.0.smithi117.stdout:5/329: unlink de/d21/d35/d57/c67 0 2024-04-20T11:08:59.867 INFO:tasks.workunit.client.0.smithi117.stdout:0/210: read d3/d7/f19 [3674701,116052] 0 2024-04-20T11:08:59.869 INFO:tasks.workunit.client.0.smithi117.stdout:8/186: mkdir d0/d9/d3a 0 2024-04-20T11:08:59.870 INFO:tasks.workunit.client.0.smithi117.stdout:1/299: chown d6/d16/c17 23577 1 2024-04-20T11:08:59.870 INFO:tasks.workunit.client.0.smithi117.stdout:8/187: chown d0/d9/f32 1 1 2024-04-20T11:08:59.872 INFO:tasks.workunit.client.0.smithi117.stdout:1/300: creat d6/d16/f62 x:0 0 0 2024-04-20T11:08:59.873 INFO:tasks.workunit.client.0.smithi117.stdout:0/211: truncate d3/d7/d16/f2b 585407 0 2024-04-20T11:08:59.876 INFO:tasks.workunit.client.0.smithi117.stdout:8/188: mknod d0/dc/d2e/c3b 0 2024-04-20T11:08:59.876 INFO:tasks.workunit.client.0.smithi117.stdout:5/330: dread de/f29 [0,4194304] 0 2024-04-20T11:08:59.881 INFO:tasks.workunit.client.0.smithi117.stdout:5/331: creat de/d41/d47/f6f x:0 0 0 2024-04-20T11:08:59.888 INFO:tasks.workunit.client.0.smithi117.stdout:5/332: creat de/d21/f70 x:0 0 0 2024-04-20T11:08:59.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:08:59 smithi117 ceph-mon[15659]: pgmap v9: 65 pgs: 65 active+clean; 502 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 57 MiB/s wr, 194 op/s 2024-04-20T11:08:59.890 INFO:tasks.workunit.client.0.smithi117.stdout:0/212: read d3/d7/f18 [2016511,29946] 0 2024-04-20T11:08:59.894 INFO:tasks.workunit.client.0.smithi117.stdout:1/301: truncate d6/d1f/f45 1131472 0 2024-04-20T11:08:59.894 INFO:tasks.workunit.client.0.smithi117.stdout:5/333: write de/f34 [607697,6556] 0 2024-04-20T11:08:59.894 INFO:tasks.workunit.client.0.smithi117.stdout:8/189: dwrite d0/d9/f32 [0,4194304] 0 2024-04-20T11:08:59.896 INFO:tasks.workunit.client.0.smithi117.stdout:0/213: symlink d3/d7/d20/d47/l51 0 2024-04-20T11:08:59.896 INFO:tasks.workunit.client.0.smithi117.stdout:1/302: rmdir d6/d16/d3a 39 2024-04-20T11:08:59.898 INFO:tasks.workunit.client.0.smithi117.stdout:0/214: read d3/d7/f3f [441203,11726] 0 2024-04-20T11:08:59.899 INFO:tasks.workunit.client.0.smithi117.stdout:5/334: symlink de/d21/d45/d48/l71 0 2024-04-20T11:08:59.901 INFO:tasks.workunit.client.0.smithi117.stdout:1/303: rmdir d6/d21 39 2024-04-20T11:08:59.903 INFO:tasks.workunit.client.0.smithi117.stdout:8/190: rename d0/dc/d2e/c3b to d0/d8/d11/c3c 0 2024-04-20T11:08:59.904 INFO:tasks.workunit.client.0.smithi117.stdout:0/215: mkdir d3/d7/d2c/d52 0 2024-04-20T11:08:59.904 INFO:tasks.workunit.client.0.smithi117.stdout:8/191: read d0/f23 [15041,128248] 0 2024-04-20T11:08:59.905 INFO:tasks.workunit.client.0.smithi117.stdout:1/304: mkdir d6/d35/d63 0 2024-04-20T11:08:59.906 INFO:tasks.workunit.client.0.smithi117.stdout:5/335: rename cd to de/d5b/c72 0 2024-04-20T11:08:59.909 INFO:tasks.workunit.client.0.smithi117.stdout:0/216: mkdir d3/d7/d20/d39/d53 0 2024-04-20T11:08:59.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:08:59 smithi173 ceph-mon[23800]: pgmap v9: 65 pgs: 65 active+clean; 502 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 57 MiB/s wr, 194 op/s 2024-04-20T11:08:59.915 INFO:tasks.workunit.client.0.smithi117.stdout:0/217: rmdir d3/d7/d20/d39/d4b 0 2024-04-20T11:08:59.921 INFO:tasks.workunit.client.0.smithi117.stdout:0/218: mkdir d3/d7/d8/d2a/d54 0 2024-04-20T11:08:59.926 INFO:tasks.workunit.client.0.smithi117.stdout:5/336: dwrite de/d21/d45/d48/d4e/f4f [4194304,4194304] 0 2024-04-20T11:08:59.926 INFO:tasks.workunit.client.0.smithi117.stdout:0/219: write d3/d7/f4d [63450,94456] 0 2024-04-20T11:08:59.929 INFO:tasks.workunit.client.0.smithi117.stdout:0/220: chown d3/d7/d20/d39 2794 1 2024-04-20T11:08:59.929 INFO:tasks.workunit.client.0.smithi117.stdout:8/192: dwrite d0/d9/f33 [0,4194304] 0 2024-04-20T11:08:59.932 INFO:tasks.workunit.client.0.smithi117.stdout:0/221: write d3/d7/d8/df/f50 [56936,95647] 0 2024-04-20T11:08:59.933 INFO:tasks.workunit.client.0.smithi117.stdout:5/337: chown de/c3f 1 1 2024-04-20T11:08:59.938 INFO:tasks.workunit.client.0.smithi117.stdout:5/338: unlink f5 0 2024-04-20T11:08:59.938 INFO:tasks.workunit.client.0.smithi117.stdout:8/193: mknod d0/d8/d39/c3d 0 2024-04-20T11:08:59.944 INFO:tasks.workunit.client.0.smithi117.stdout:8/194: mkdir d0/dc/d2e/d3e 0 2024-04-20T11:08:59.953 INFO:tasks.workunit.client.0.smithi117.stdout:5/339: getdents de/d41/d47/d55 0 2024-04-20T11:08:59.953 INFO:tasks.workunit.client.0.smithi117.stdout:1/305: dwrite d6/d1f/f45 [0,4194304] 0 2024-04-20T11:08:59.958 INFO:tasks.workunit.client.0.smithi117.stdout:5/340: write de/d41/d47/f5e [401590,122976] 0 2024-04-20T11:08:59.959 INFO:tasks.workunit.client.0.smithi117.stdout:1/306: write d6/f2e [2037369,98331] 0 2024-04-20T11:08:59.959 INFO:tasks.workunit.client.0.smithi117.stdout:0/222: dwrite d3/d5/f26 [0,4194304] 0 2024-04-20T11:08:59.966 INFO:tasks.workunit.client.0.smithi117.stdout:0/223: fdatasync d3/d7/f4d 0 2024-04-20T11:08:59.966 INFO:tasks.workunit.client.0.smithi117.stdout:8/195: dread d0/d9/f32 [0,4194304] 0 2024-04-20T11:08:59.966 INFO:tasks.workunit.client.0.smithi117.stdout:1/307: truncate d6/d1f/d30/d32/f46 720269 0 2024-04-20T11:08:59.966 INFO:tasks.workunit.client.0.smithi117.stdout:5/341: unlink de/d21/d35/f3d 0 2024-04-20T11:08:59.969 INFO:tasks.workunit.client.0.smithi117.stdout:5/342: readlink de/d16/l54 0 2024-04-20T11:08:59.969 INFO:tasks.workunit.client.0.smithi117.stdout:8/196: truncate d0/d9/f31 636753 0 2024-04-20T11:08:59.971 INFO:tasks.workunit.client.0.smithi117.stdout:1/308: write d6/d35/f36 [3693219,36774] 0 2024-04-20T11:08:59.974 INFO:tasks.workunit.client.0.smithi117.stdout:8/197: mknod d0/d8/d11/d24/d2b/c3f 0 2024-04-20T11:08:59.975 INFO:tasks.workunit.client.0.smithi117.stdout:8/198: fsync d0/d9/f2a 0 2024-04-20T11:08:59.976 INFO:tasks.workunit.client.0.smithi117.stdout:0/224: link l2 d3/d7/d16/l55 0 2024-04-20T11:08:59.980 INFO:tasks.workunit.client.0.smithi117.stdout:8/199: dread - d0/d8/f16 zero size 2024-04-20T11:08:59.980 INFO:tasks.workunit.client.0.smithi117.stdout:1/309: unlink d6/d21/c22 0 2024-04-20T11:08:59.983 INFO:tasks.workunit.client.0.smithi117.stdout:8/200: chown d0/f20 41054 1 2024-04-20T11:08:59.983 INFO:tasks.workunit.client.0.smithi117.stdout:0/225: stat d3/d7/d8/df/c27 0 2024-04-20T11:08:59.986 INFO:tasks.workunit.client.0.smithi117.stdout:8/201: mkdir d0/d8/d39/d40 0 2024-04-20T11:08:59.986 INFO:tasks.workunit.client.0.smithi117.stdout:0/226: write d3/d7/f4d [1199129,16933] 0 2024-04-20T11:08:59.990 INFO:tasks.workunit.client.0.smithi117.stdout:0/227: creat d3/d7/d16/f56 x:0 0 0 2024-04-20T11:08:59.990 INFO:tasks.workunit.client.0.smithi117.stdout:8/202: rename d0/d8/f16 to d0/d8/f41 0 2024-04-20T11:08:59.991 INFO:tasks.workunit.client.0.smithi117.stdout:8/203: dread - d0/d9/f2a zero size 2024-04-20T11:08:59.994 INFO:tasks.workunit.client.0.smithi117.stdout:8/204: truncate d0/d9/f31 1327788 0 2024-04-20T11:08:59.999 INFO:tasks.workunit.client.0.smithi117.stdout:8/205: mknod d0/d8/d39/d40/c42 0 2024-04-20T11:09:00.000 INFO:tasks.workunit.client.0.smithi117.stdout:8/206: chown d0/d8 4153 1 2024-04-20T11:09:00.002 INFO:tasks.workunit.client.0.smithi117.stdout:8/207: fdatasync d0/d9/f33 0 2024-04-20T11:09:00.005 INFO:tasks.workunit.client.0.smithi117.stdout:8/208: stat d0/d8/d11/d24/d2b/c35 0 2024-04-20T11:09:00.006 INFO:tasks.workunit.client.0.smithi117.stdout:8/209: stat d0/d8/c30 0 2024-04-20T11:09:00.007 INFO:tasks.workunit.client.0.smithi117.stdout:0/228: dwrite d3/d7/d8/f31 [0,4194304] 0 2024-04-20T11:09:00.007 INFO:tasks.workunit.client.0.smithi117.stdout:8/210: chown d0/d9/f2a 101317 1 2024-04-20T11:09:00.011 INFO:tasks.workunit.client.0.smithi117.stdout:0/229: mknod d3/d7/d8/df/d22/c57 0 2024-04-20T11:09:00.013 INFO:tasks.workunit.client.0.smithi117.stdout:8/211: creat d0/d8/d39/d40/f43 x:0 0 0 2024-04-20T11:09:00.018 INFO:tasks.workunit.client.0.smithi117.stdout:8/212: dread d0/f22 [0,4194304] 0 2024-04-20T11:09:00.018 INFO:tasks.workunit.client.0.smithi117.stdout:7/273: sync 2024-04-20T11:09:00.018 INFO:tasks.workunit.client.0.smithi117.stdout:2/328: sync 2024-04-20T11:09:00.019 INFO:tasks.workunit.client.0.smithi117.stdout:4/293: sync 2024-04-20T11:09:00.019 INFO:tasks.workunit.client.0.smithi117.stdout:3/259: sync 2024-04-20T11:09:00.020 INFO:tasks.workunit.client.0.smithi117.stdout:5/343: dread de/d21/f2e [0,4194304] 0 2024-04-20T11:09:00.021 INFO:tasks.workunit.client.0.smithi117.stdout:0/230: rename d3/d7/d8/df/d22/c1e to d3/d7/d2c/d52/c58 0 2024-04-20T11:09:00.026 INFO:tasks.workunit.client.0.smithi117.stdout:7/274: creat d1/d4/d3d/f5a x:0 0 0 2024-04-20T11:09:00.027 INFO:tasks.workunit.client.0.smithi117.stdout:2/329: unlink d1/d4/dd/de/d29/d3f/c43 0 2024-04-20T11:09:00.032 INFO:tasks.workunit.client.0.smithi117.stdout:8/213: dread d0/f20 [0,4194304] 0 2024-04-20T11:09:00.040 INFO:tasks.workunit.client.0.smithi117.stdout:5/344: rmdir de/d21/d5f 0 2024-04-20T11:09:00.040 INFO:tasks.workunit.client.0.smithi117.stdout:8/214: unlink d0/l2 0 2024-04-20T11:09:00.040 INFO:tasks.workunit.client.0.smithi117.stdout:5/345: write de/d41/d47/f5e [992557,81530] 0 2024-04-20T11:09:00.041 INFO:tasks.workunit.client.0.smithi117.stdout:7/275: symlink d1/d4/d6/d54/l5b 0 2024-04-20T11:09:00.042 INFO:tasks.workunit.client.0.smithi117.stdout:4/294: write d1/f20 [567618,57376] 0 2024-04-20T11:09:00.042 INFO:tasks.workunit.client.0.smithi117.stdout:5/346: chown de/d41/d47/l53 475456 1 2024-04-20T11:09:00.045 INFO:tasks.workunit.client.0.smithi117.stdout:4/295: dread - d1/d45/d4a/f50 zero size 2024-04-20T11:09:00.049 INFO:tasks.workunit.client.0.smithi117.stdout:5/347: symlink de/d21/d45/l73 0 2024-04-20T11:09:00.054 INFO:tasks.workunit.client.0.smithi117.stdout:2/330: dwrite d1/d4/dd/d17/f1e [0,4194304] 0 2024-04-20T11:09:00.054 INFO:tasks.workunit.client.0.smithi117.stdout:8/215: symlink d0/l44 0 2024-04-20T11:09:00.055 INFO:tasks.workunit.client.0.smithi117.stdout:2/331: stat d1/l39 0 2024-04-20T11:09:00.057 INFO:tasks.workunit.client.0.smithi117.stdout:3/260: dwrite d0/d5/d7/dd/f10 [4194304,4194304] 0 2024-04-20T11:09:00.060 INFO:tasks.workunit.client.0.smithi117.stdout:0/231: truncate d3/d7/d8/df/f1b 2487218 0 2024-04-20T11:09:00.063 INFO:tasks.workunit.client.0.smithi117.stdout:0/232: dread - d3/d7/f14 zero size 2024-04-20T11:09:00.064 INFO:tasks.workunit.client.0.smithi117.stdout:4/296: dread d1/f30 [0,4194304] 0 2024-04-20T11:09:00.066 INFO:tasks.workunit.client.0.smithi117.stdout:4/297: fdatasync d1/f2 0 2024-04-20T11:09:00.070 INFO:tasks.workunit.client.0.smithi117.stdout:7/276: dwrite d1/d4/d39/f4d [0,4194304] 0 2024-04-20T11:09:00.075 INFO:tasks.workunit.client.0.smithi117.stdout:0/233: creat d3/d5/f59 x:0 0 0 2024-04-20T11:09:00.077 INFO:tasks.workunit.client.0.smithi117.stdout:3/261: symlink d0/d5/l5f 0 2024-04-20T11:09:00.079 INFO:tasks.workunit.client.0.smithi117.stdout:3/262: fdatasync d0/d5/d21/f41 0 2024-04-20T11:09:00.083 INFO:tasks.workunit.client.0.smithi117.stdout:5/348: rename de/d21/f49 to de/f74 0 2024-04-20T11:09:00.085 INFO:tasks.workunit.client.0.smithi117.stdout:2/332: rename d1/d4/dd/de/d29 to d1/d4/dd/de/d29/d65 22 2024-04-20T11:09:00.090 INFO:tasks.workunit.client.0.smithi117.stdout:7/277: getdents d1/d1d 0 2024-04-20T11:09:00.098 INFO:tasks.workunit.client.0.smithi117.stdout:4/298: dwrite d1/dd/f4b [0,4194304] 0 2024-04-20T11:09:00.100 INFO:tasks.workunit.client.0.smithi117.stdout:3/263: link d0/d5/l5f d0/d5/de/d1d/l60 0 2024-04-20T11:09:00.101 INFO:tasks.workunit.client.0.smithi117.stdout:0/234: dwrite d3/d5/f59 [0,4194304] 0 2024-04-20T11:09:00.104 INFO:tasks.workunit.client.0.smithi117.stdout:0/235: truncate d3/d7/f14 428168 0 2024-04-20T11:09:00.112 INFO:tasks.workunit.client.0.smithi117.stdout:3/264: fdatasync d0/d5/d7/dd/f59 0 2024-04-20T11:09:00.113 INFO:tasks.workunit.client.0.smithi117.stdout:0/236: write d3/d7/f3b [2874053,66884] 0 2024-04-20T11:09:00.117 INFO:tasks.workunit.client.0.smithi117.stdout:5/349: dwrite de/d21/f31 [4194304,4194304] 0 2024-04-20T11:09:00.121 INFO:tasks.workunit.client.0.smithi117.stdout:0/237: creat d3/d5/f5a x:0 0 0 2024-04-20T11:09:00.122 INFO:tasks.workunit.client.0.smithi117.stdout:4/299: getdents d1/d45 0 2024-04-20T11:09:00.122 INFO:tasks.workunit.client.0.smithi117.stdout:3/265: mknod d0/d5/d7/dd/c61 0 2024-04-20T11:09:00.123 INFO:tasks.workunit.client.0.smithi117.stdout:7/278: dwrite d1/d1d/f21 [4194304,4194304] 0 2024-04-20T11:09:00.123 INFO:tasks.workunit.client.0.smithi117.stdout:4/300: dread - d1/f67 zero size 2024-04-20T11:09:00.125 INFO:tasks.workunit.client.0.smithi117.stdout:5/350: write de/f74 [2177787,66405] 0 2024-04-20T11:09:00.125 INFO:tasks.workunit.client.0.smithi117.stdout:0/238: chown d3/d5/f37 8 1 2024-04-20T11:09:00.133 INFO:tasks.workunit.client.0.smithi117.stdout:3/266: creat d0/d5/de/d1d/f62 x:0 0 0 2024-04-20T11:09:00.136 INFO:tasks.workunit.client.0.smithi117.stdout:0/239: mknod d3/d7/d8/c5b 0 2024-04-20T11:09:00.137 INFO:tasks.workunit.client.0.smithi117.stdout:3/267: fdatasync d0/d5/f19 0 2024-04-20T11:09:00.143 INFO:tasks.workunit.client.0.smithi117.stdout:7/279: dread d1/d1d/f21 [4194304,4194304] 0 2024-04-20T11:09:00.145 INFO:tasks.workunit.client.0.smithi117.stdout:3/268: chown d0/d5/fc 240550 1 2024-04-20T11:09:00.149 INFO:tasks.workunit.client.0.smithi117.stdout:7/280: creat d1/d1d/f5c x:0 0 0 2024-04-20T11:09:00.149 INFO:tasks.workunit.client.0.smithi117.stdout:4/301: dwrite d1/dd/f13 [0,4194304] 0 2024-04-20T11:09:00.152 INFO:tasks.workunit.client.0.smithi117.stdout:5/351: dwrite de/d21/f70 [0,4194304] 0 2024-04-20T11:09:00.156 INFO:tasks.workunit.client.0.smithi117.stdout:7/281: chown d1/d4/d6/d17 1822527 1 2024-04-20T11:09:00.156 INFO:tasks.workunit.client.0.smithi117.stdout:3/269: rename d0/d2a/d58 to d0/d5/d11/d46/d63 0 2024-04-20T11:09:00.156 INFO:tasks.workunit.client.0.smithi117.stdout:5/352: stat de/d21/d35/l60 0 2024-04-20T11:09:00.160 INFO:tasks.workunit.client.0.smithi117.stdout:7/282: creat d1/f5d x:0 0 0 2024-04-20T11:09:00.160 INFO:tasks.workunit.client.0.smithi117.stdout:5/353: creat de/d41/d47/d4c/f75 x:0 0 0 2024-04-20T11:09:00.163 INFO:tasks.workunit.client.0.smithi117.stdout:7/283: mknod d1/d1d/d4e/c5e 0 2024-04-20T11:09:00.166 INFO:tasks.workunit.client.0.smithi117.stdout:5/354: mknod de/d21/d45/d48/d4e/c76 0 2024-04-20T11:09:00.166 INFO:tasks.workunit.client.0.smithi117.stdout:7/284: write d1/f5d [807434,50313] 0 2024-04-20T11:09:00.171 INFO:tasks.workunit.client.0.smithi117.stdout:7/285: write d1/d4/d39/f34 [2839187,39494] 0 2024-04-20T11:09:00.178 INFO:tasks.workunit.client.0.smithi117.stdout:7/286: read d1/f5d [625601,74656] 0 2024-04-20T11:09:00.179 INFO:tasks.workunit.client.0.smithi117.stdout:5/355: truncate de/d21/d45/d48/f4a 496084 0 2024-04-20T11:09:00.180 INFO:tasks.workunit.client.0.smithi117.stdout:3/270: dwrite d0/d5/d7/f20 [0,4194304] 0 2024-04-20T11:09:00.183 INFO:tasks.workunit.client.0.smithi117.stdout:5/356: stat de/d41/d47/d55/d56 0 2024-04-20T11:09:00.185 INFO:tasks.workunit.client.0.smithi117.stdout:4/302: dwrite d1/d9/f12 [0,4194304] 0 2024-04-20T11:09:00.187 INFO:tasks.workunit.client.0.smithi117.stdout:7/287: creat d1/d1d/d42/d2d/f5f x:0 0 0 2024-04-20T11:09:00.187 INFO:tasks.workunit.client.0.smithi117.stdout:3/271: write d0/d5/d7/f20 [3167013,36188] 0 2024-04-20T11:09:00.191 INFO:tasks.workunit.client.0.smithi117.stdout:4/303: fsync d1/dd/f53 0 2024-04-20T11:09:00.191 INFO:tasks.workunit.client.0.smithi117.stdout:7/288: stat d1/d4/d6/d13/f15 0 2024-04-20T11:09:00.194 INFO:tasks.workunit.client.0.smithi117.stdout:4/304: read d1/d9/d1f/f5c [825464,53423] 0 2024-04-20T11:09:00.200 INFO:tasks.workunit.client.0.smithi117.stdout:5/357: dread de/d21/d35/f3a [0,4194304] 0 2024-04-20T11:09:00.200 INFO:tasks.workunit.client.0.smithi117.stdout:5/358: chown de/d16 1538499620 1 2024-04-20T11:09:00.202 INFO:tasks.workunit.client.0.smithi117.stdout:7/289: dread d1/d4/d39/f34 [0,4194304] 0 2024-04-20T11:09:00.203 INFO:tasks.workunit.client.0.smithi117.stdout:7/290: chown d1/d4/d6/d13/l19 238260 1 2024-04-20T11:09:00.234 INFO:tasks.workunit.client.0.smithi117.stdout:9/297: sync 2024-04-20T11:09:00.234 INFO:tasks.workunit.client.0.smithi117.stdout:6/358: sync 2024-04-20T11:09:00.247 INFO:tasks.workunit.client.0.smithi117.stdout:6/359: dwrite d3/df/f14 [0,4194304] 0 2024-04-20T11:09:00.266 INFO:tasks.workunit.client.0.smithi117.stdout:6/360: dwrite d3/d3a/f61 [0,4194304] 0 2024-04-20T11:09:00.267 INFO:tasks.workunit.client.0.smithi117.stdout:4/305: getdents d1/dd 0 2024-04-20T11:09:00.271 INFO:tasks.workunit.client.0.smithi117.stdout:9/298: fdatasync d4/de/f3e 0 2024-04-20T11:09:00.276 INFO:tasks.workunit.client.0.smithi117.stdout:4/306: creat d1/d45/d4a/f6c x:0 0 0 2024-04-20T11:09:00.280 INFO:tasks.workunit.client.0.smithi117.stdout:9/299: write d4/d4a/f60 [1684903,116939] 0 2024-04-20T11:09:00.284 INFO:tasks.workunit.client.0.smithi117.stdout:4/307: symlink d1/d9/d1f/l6d 0 2024-04-20T11:09:00.285 INFO:tasks.workunit.client.0.smithi117.stdout:5/359: getdents de/d41/d47/d4c 0 2024-04-20T11:09:00.287 INFO:tasks.workunit.client.0.smithi117.stdout:7/291: getdents d1/d4/d6/d17/d3f 0 2024-04-20T11:09:00.288 INFO:tasks.workunit.client.0.smithi117.stdout:9/300: mkdir d4/d68/d6e 0 2024-04-20T11:09:00.293 INFO:tasks.workunit.client.0.smithi117.stdout:7/292: chown d1/d1d/c35 1432 1 2024-04-20T11:09:00.293 INFO:tasks.workunit.client.0.smithi117.stdout:6/361: truncate d3/d4/f19 3039572 0 2024-04-20T11:09:00.295 INFO:tasks.workunit.client.0.smithi117.stdout:5/360: dread de/f18 [0,4194304] 0 2024-04-20T11:09:00.297 INFO:tasks.workunit.client.0.smithi117.stdout:4/308: write d1/f35 [897771,122208] 0 2024-04-20T11:09:00.298 INFO:tasks.workunit.client.0.smithi117.stdout:5/361: read de/d21/f31 [8114583,91778] 0 2024-04-20T11:09:00.303 INFO:tasks.workunit.client.0.smithi117.stdout:7/293: unlink d1/d1d/d42/d2d/c45 0 2024-04-20T11:09:00.303 INFO:tasks.workunit.client.0.smithi117.stdout:9/301: getdents d4/d68 0 2024-04-20T11:09:00.303 INFO:tasks.workunit.client.0.smithi117.stdout:4/309: chown d1/d9/d1f/c3c 2814781 1 2024-04-20T11:09:00.303 INFO:tasks.workunit.client.0.smithi117.stdout:4/310: fsync d1/d9/d1f/f39 0 2024-04-20T11:09:00.304 INFO:tasks.workunit.client.0.smithi117.stdout:4/311: write d1/f57 [225733,39130] 0 2024-04-20T11:09:00.306 INFO:tasks.workunit.client.0.smithi117.stdout:9/302: creat d4/d5/da/f6f x:0 0 0 2024-04-20T11:09:00.307 INFO:tasks.workunit.client.0.smithi117.stdout:6/362: getdents d3/d4/d2e/d37 0 2024-04-20T11:09:00.309 INFO:tasks.workunit.client.0.smithi117.stdout:5/362: fdatasync de/f14 0 2024-04-20T11:09:00.311 INFO:tasks.workunit.client.0.smithi117.stdout:7/294: rename d1/d4/d6/d17/d3f/c52 to d1/d1d/d42/d2d/c60 0 2024-04-20T11:09:00.317 INFO:tasks.workunit.client.0.smithi117.stdout:5/363: mkdir de/d21/d45/d77 0 2024-04-20T11:09:00.321 INFO:tasks.workunit.client.0.smithi117.stdout:7/295: creat d1/d43/f61 x:0 0 0 2024-04-20T11:09:00.324 INFO:tasks.workunit.client.0.smithi117.stdout:6/363: dwrite d3/d3a/f6b [0,4194304] 0 2024-04-20T11:09:00.328 INFO:tasks.workunit.client.0.smithi117.stdout:7/296: dread d1/f7 [4194304,4194304] 0 2024-04-20T11:09:00.329 INFO:tasks.workunit.client.0.smithi117.stdout:5/364: rename de/d41/d47/f5e to de/d41/d47/d55/d56/f78 0 2024-04-20T11:09:00.333 INFO:tasks.workunit.client.0.smithi117.stdout:4/312: write d1/dd/d1a/d2b/f46 [2651718,8871] 0 2024-04-20T11:09:00.335 INFO:tasks.workunit.client.0.smithi117.stdout:9/303: write d4/de/d10/f38 [614933,41192] 0 2024-04-20T11:09:00.338 INFO:tasks.workunit.client.0.smithi117.stdout:6/364: creat d3/f73 x:0 0 0 2024-04-20T11:09:00.339 INFO:tasks.workunit.client.0.smithi117.stdout:4/313: mkdir d1/dd/d1a/d2b/d6e 0 2024-04-20T11:09:00.340 INFO:tasks.workunit.client.0.smithi117.stdout:7/297: getdents d1 0 2024-04-20T11:09:00.343 INFO:tasks.workunit.client.0.smithi117.stdout:4/314: mknod d1/d9/d6b/c6f 0 2024-04-20T11:09:00.345 INFO:tasks.workunit.client.0.smithi117.stdout:7/298: getdents d1/d1d/d4e/d53/d55 0 2024-04-20T11:09:00.349 INFO:tasks.workunit.client.0.smithi117.stdout:7/299: symlink d1/d4/d6/d17/l62 0 2024-04-20T11:09:00.349 INFO:tasks.workunit.client.0.smithi117.stdout:4/315: dread d1/f30 [0,4194304] 0 2024-04-20T11:09:00.351 INFO:tasks.workunit.client.0.smithi117.stdout:9/304: write d4/f2e [392997,36133] 0 2024-04-20T11:09:00.353 INFO:tasks.workunit.client.0.smithi117.stdout:7/300: mkdir d1/d4/d6/d54/d63 0 2024-04-20T11:09:00.360 INFO:tasks.workunit.client.0.smithi117.stdout:9/305: mkdir d4/d70 0 2024-04-20T11:09:00.363 INFO:tasks.workunit.client.0.smithi117.stdout:7/301: mkdir d1/d4/d6/d2b/d64 0 2024-04-20T11:09:00.365 INFO:tasks.workunit.client.0.smithi117.stdout:9/306: unlink d4/de/d10/f40 0 2024-04-20T11:09:00.366 INFO:tasks.workunit.client.0.smithi117.stdout:6/365: dwrite d3/d4/f8 [0,4194304] 0 2024-04-20T11:09:00.366 INFO:tasks.workunit.client.0.smithi117.stdout:5/365: dread de/f1f [0,4194304] 0 2024-04-20T11:09:00.367 INFO:tasks.workunit.client.0.smithi117.stdout:9/307: fsync d4/d35/f3d 0 2024-04-20T11:09:00.369 INFO:tasks.workunit.client.0.smithi117.stdout:7/302: dread d1/d4/d6/d13/f15 [0,4194304] 0 2024-04-20T11:09:00.373 INFO:tasks.workunit.client.0.smithi117.stdout:5/366: creat de/d41/d47/d55/d56/f79 x:0 0 0 2024-04-20T11:09:00.376 INFO:tasks.workunit.client.0.smithi117.stdout:9/308: symlink d4/de/l71 0 2024-04-20T11:09:00.379 INFO:tasks.workunit.client.0.smithi117.stdout:7/303: rename d1/f5d to d1/d4/d6/d17/d3f/f65 0 2024-04-20T11:09:00.380 INFO:tasks.workunit.client.0.smithi117.stdout:5/367: mknod de/d41/d47/d4c/c7a 0 2024-04-20T11:09:00.380 INFO:tasks.workunit.client.0.smithi117.stdout:9/309: mkdir d4/d4a/d72 0 2024-04-20T11:09:00.382 INFO:tasks.workunit.client.0.smithi117.stdout:9/310: chown d4/d5/d3c/c65 24 1 2024-04-20T11:09:00.384 INFO:tasks.workunit.client.0.smithi117.stdout:9/311: truncate d4/de/f3e 4905281 0 2024-04-20T11:09:00.385 INFO:tasks.workunit.client.0.smithi117.stdout:5/368: mkdir de/d21/d7b 0 2024-04-20T11:09:00.388 INFO:tasks.workunit.client.0.smithi117.stdout:6/366: write d3/d4/f19 [1567055,27509] 0 2024-04-20T11:09:00.391 INFO:tasks.workunit.client.0.smithi117.stdout:6/367: read - d3/d4/d2e/d4e/f70 zero size 2024-04-20T11:09:00.392 INFO:tasks.workunit.client.0.smithi117.stdout:5/369: write de/d41/d47/d55/d56/f78 [1299245,72137] 0 2024-04-20T11:09:00.393 INFO:tasks.workunit.client.0.smithi117.stdout:6/368: chown d3/df/f21 356 1 2024-04-20T11:09:00.396 INFO:tasks.workunit.client.0.smithi117.stdout:6/369: mkdir d3/d5a/d74 0 2024-04-20T11:09:00.399 INFO:tasks.workunit.client.0.smithi117.stdout:9/312: truncate d4/de/f3e 1004029 0 2024-04-20T11:09:00.400 INFO:tasks.workunit.client.0.smithi117.stdout:6/370: write d3/df/f63 [3515247,34575] 0 2024-04-20T11:09:00.402 INFO:tasks.workunit.client.0.smithi117.stdout:5/370: getdents de 0 2024-04-20T11:09:00.405 INFO:tasks.workunit.client.0.smithi117.stdout:9/313: mkdir d4/d30/d31/d73 0 2024-04-20T11:09:00.406 INFO:tasks.workunit.client.0.smithi117.stdout:5/371: mknod de/d21/d35/c7c 0 2024-04-20T11:09:00.407 INFO:tasks.workunit.client.0.smithi117.stdout:6/371: symlink d3/d5a/d74/l75 0 2024-04-20T11:09:00.408 INFO:tasks.workunit.client.0.smithi117.stdout:9/314: read d4/de/d4b/f58 [408128,75230] 0 2024-04-20T11:09:00.409 INFO:tasks.workunit.client.0.smithi117.stdout:7/304: dread d1/d4/d39/f34 [0,4194304] 0 2024-04-20T11:09:00.410 INFO:tasks.workunit.client.0.smithi117.stdout:5/372: fsync de/d41/d47/d55/d56/f78 0 2024-04-20T11:09:00.414 INFO:tasks.workunit.client.0.smithi117.stdout:6/372: mkdir d3/df/d39/d76 0 2024-04-20T11:09:00.414 INFO:tasks.workunit.client.0.smithi117.stdout:7/305: write d1/d4/d6/d2b/f33 [3977516,66485] 0 2024-04-20T11:09:00.417 INFO:tasks.workunit.client.0.smithi117.stdout:6/373: creat d3/d4/d2e/d3f/f77 x:0 0 0 2024-04-20T11:09:00.418 INFO:tasks.workunit.client.0.smithi117.stdout:5/373: creat de/d21/d45/d77/f7d x:0 0 0 2024-04-20T11:09:00.427 INFO:tasks.workunit.client.0.smithi117.stdout:6/374: rename d3/df/c5d to d3/d5a/c78 0 2024-04-20T11:09:00.432 INFO:tasks.workunit.client.0.smithi117.stdout:7/306: dread d1/f10 [4194304,4194304] 0 2024-04-20T11:09:00.435 INFO:tasks.workunit.client.0.smithi117.stdout:5/374: dread de/d21/d45/d48/f4a [0,4194304] 0 2024-04-20T11:09:00.436 INFO:tasks.workunit.client.0.smithi117.stdout:6/375: rename d3/d4/d2e/d37/d4a to d3/d5a/d79 0 2024-04-20T11:09:00.443 INFO:tasks.workunit.client.0.smithi117.stdout:9/315: dwrite d4/de/d4b/f58 [4194304,4194304] 0 2024-04-20T11:09:00.446 INFO:tasks.workunit.client.0.smithi117.stdout:9/316: stat d4/d5/da/c26 0 2024-04-20T11:09:00.446 INFO:tasks.workunit.client.0.smithi117.stdout:5/375: mknod de/d21/d45/c7e 0 2024-04-20T11:09:00.450 INFO:tasks.workunit.client.0.smithi117.stdout:6/376: rename d3/f30 to d3/df/d39/d76/f7a 0 2024-04-20T11:09:00.450 INFO:tasks.workunit.client.0.smithi117.stdout:9/317: mkdir d4/d4a/d74 0 2024-04-20T11:09:00.451 INFO:tasks.workunit.client.0.smithi117.stdout:6/377: fdatasync d3/d4/f8 0 2024-04-20T11:09:00.455 INFO:tasks.workunit.client.0.smithi117.stdout:7/307: dwrite d1/d4/d6/d17/d3f/f65 [0,4194304] 0 2024-04-20T11:09:00.464 INFO:tasks.workunit.client.0.smithi117.stdout:7/308: rmdir d1/d43 39 2024-04-20T11:09:00.466 INFO:tasks.workunit.client.0.smithi117.stdout:6/378: getdents d3/d3a 0 2024-04-20T11:09:00.466 INFO:tasks.workunit.client.0.smithi117.stdout:1/310: sync 2024-04-20T11:09:00.466 INFO:tasks.workunit.client.0.smithi117.stdout:8/216: sync 2024-04-20T11:09:00.474 INFO:tasks.workunit.client.0.smithi117.stdout:5/376: dread de/f19 [0,4194304] 0 2024-04-20T11:09:00.475 INFO:tasks.workunit.client.0.smithi117.stdout:7/309: write d1/d4/d3d/f5a [441739,305] 0 2024-04-20T11:09:00.475 INFO:tasks.workunit.client.0.smithi117.stdout:6/379: mknod d3/d50/c7b 0 2024-04-20T11:09:00.478 INFO:tasks.workunit.client.0.smithi117.stdout:1/311: mkdir d6/d1f/d30/d49/d64 0 2024-04-20T11:09:00.480 INFO:tasks.workunit.client.0.smithi117.stdout:9/318: write d4/f13 [356785,108758] 0 2024-04-20T11:09:00.485 INFO:tasks.workunit.client.0.smithi117.stdout:6/380: symlink d3/d4/d2e/d4e/l7c 0 2024-04-20T11:09:00.487 INFO:tasks.workunit.client.0.smithi117.stdout:8/217: rmdir d0/d9/d29 0 2024-04-20T11:09:00.489 INFO:tasks.workunit.client.0.smithi117.stdout:6/381: write d3/df/f4f [188486,42844] 0 2024-04-20T11:09:00.495 INFO:tasks.workunit.client.0.smithi117.stdout:6/382: readlink d3/df/l27 0 2024-04-20T11:09:00.496 INFO:tasks.workunit.client.0.smithi117.stdout:9/319: rename d4/d30/d31/f33 to d4/d4a/d5e/f75 0 2024-04-20T11:09:00.496 INFO:tasks.workunit.client.0.smithi117.stdout:6/383: stat d3/df/l33 0 2024-04-20T11:09:00.499 INFO:tasks.workunit.client.0.smithi117.stdout:5/377: write de/f14 [60945,31581] 0 2024-04-20T11:09:00.500 INFO:tasks.workunit.client.0.smithi117.stdout:1/312: mknod d6/d16/d3a/d5a/c65 0 2024-04-20T11:09:00.505 INFO:tasks.workunit.client.0.smithi117.stdout:6/384: mknod d3/d5a/d79/c7d 0 2024-04-20T11:09:00.507 INFO:tasks.workunit.client.0.smithi117.stdout:6/385: read d3/d4/f8 [3358465,55936] 0 2024-04-20T11:09:00.511 INFO:tasks.workunit.client.0.smithi117.stdout:7/310: rename d1/l24 to d1/d1d/d4e/l66 0 2024-04-20T11:09:00.514 INFO:tasks.workunit.client.0.smithi117.stdout:5/378: symlink de/d41/d47/l7f 0 2024-04-20T11:09:00.515 INFO:tasks.workunit.client.0.smithi117.stdout:9/320: creat d4/d4a/d63/f76 x:0 0 0 2024-04-20T11:09:00.516 INFO:tasks.workunit.client.0.smithi117.stdout:5/379: stat de/c15 0 2024-04-20T11:09:00.517 INFO:tasks.workunit.client.0.smithi117.stdout:1/313: symlink d6/d35/d63/l66 0 2024-04-20T11:09:00.518 INFO:tasks.workunit.client.0.smithi117.stdout:7/311: mkdir d1/d4/d6/d17/d67 0 2024-04-20T11:09:00.521 INFO:tasks.workunit.client.0.smithi117.stdout:1/314: chown d6/d21/l3f 110975997 1 2024-04-20T11:09:00.522 INFO:tasks.workunit.client.0.smithi117.stdout:9/321: creat d4/d68/d6e/f77 x:0 0 0 2024-04-20T11:09:00.527 INFO:tasks.workunit.client.0.smithi117.stdout:1/315: fdatasync d6/d1f/d30/f43 0 2024-04-20T11:09:00.527 INFO:tasks.workunit.client.0.smithi117.stdout:6/386: dread d3/f28 [0,4194304] 0 2024-04-20T11:09:00.530 INFO:tasks.workunit.client.0.smithi117.stdout:9/322: mkdir d4/d30/d78 0 2024-04-20T11:09:00.541 INFO:tasks.workunit.client.0.smithi117.stdout:7/312: dwrite d1/d4/d6/f26 [4194304,4194304] 0 2024-04-20T11:09:00.544 INFO:tasks.workunit.client.0.smithi117.stdout:7/313: chown d1/d1d/c29 0 1 2024-04-20T11:09:00.548 INFO:tasks.workunit.client.0.smithi117.stdout:9/323: dread d4/d30/f6b [0,4194304] 0 2024-04-20T11:09:00.554 INFO:tasks.workunit.client.0.smithi117.stdout:1/316: dwrite d6/d21/f39 [0,4194304] 0 2024-04-20T11:09:00.557 INFO:tasks.workunit.client.0.smithi117.stdout:7/314: symlink d1/d4/d6/d54/d63/l68 0 2024-04-20T11:09:00.564 INFO:tasks.workunit.client.0.smithi117.stdout:9/324: mknod d4/d4a/c79 0 2024-04-20T11:09:00.566 INFO:tasks.workunit.client.0.smithi117.stdout:9/325: read - d4/de/d4b/f6d zero size 2024-04-20T11:09:00.575 INFO:tasks.workunit.client.0.smithi117.stdout:1/317: creat d6/d1f/d30/f67 x:0 0 0 2024-04-20T11:09:00.591 INFO:tasks.workunit.client.0.smithi117.stdout:2/333: sync 2024-04-20T11:09:00.591 INFO:tasks.workunit.client.0.smithi117.stdout:0/240: sync 2024-04-20T11:09:00.591 INFO:tasks.workunit.client.0.smithi117.stdout:4/316: sync 2024-04-20T11:09:00.591 INFO:tasks.workunit.client.0.smithi117.stdout:3/272: sync 2024-04-20T11:09:00.592 INFO:tasks.workunit.client.0.smithi117.stdout:5/380: dwrite de/d21/d35/f3b [0,4194304] 0 2024-04-20T11:09:00.598 INFO:tasks.workunit.client.0.smithi117.stdout:7/315: dwrite d1/f7 [0,4194304] 0 2024-04-20T11:09:00.599 INFO:tasks.workunit.client.0.smithi117.stdout:5/381: truncate de/d41/f52 333408 0 2024-04-20T11:09:00.600 INFO:tasks.workunit.client.0.smithi117.stdout:7/316: fdatasync d1/d1d/f5c 0 2024-04-20T11:09:00.611 INFO:tasks.workunit.client.0.smithi117.stdout:9/326: dread d4/d4a/d5e/f75 [0,4194304] 0 2024-04-20T11:09:00.615 INFO:tasks.workunit.client.0.smithi117.stdout:0/241: dwrite d3/d7/f9 [0,4194304] 0 2024-04-20T11:09:00.617 INFO:tasks.workunit.client.0.smithi117.stdout:0/242: chown d3/d7/d8/df/d22/c49 276569578 1 2024-04-20T11:09:00.617 INFO:tasks.workunit.client.0.smithi117.stdout:2/334: symlink d1/d4/dd/de/d29/d2e/l66 0 2024-04-20T11:09:00.620 INFO:tasks.workunit.client.0.smithi117.stdout:0/243: chown d3/d7/f4d 16 1 2024-04-20T11:09:00.620 INFO:tasks.workunit.client.0.smithi117.stdout:2/335: truncate d1/d4/f8 4831774 0 2024-04-20T11:09:00.625 INFO:tasks.workunit.client.0.smithi117.stdout:9/327: mkdir d4/d5/d3c/d7a 0 2024-04-20T11:09:00.631 INFO:tasks.workunit.client.0.smithi117.stdout:2/336: rmdir d1/d20/d26 39 2024-04-20T11:09:00.641 INFO:tasks.workunit.client.0.smithi117.stdout:5/382: rmdir de/d21/d7b 0 2024-04-20T11:09:00.645 INFO:tasks.workunit.client.0.smithi117.stdout:4/317: dread d1/f25 [0,4194304] 0 2024-04-20T11:09:00.645 INFO:tasks.workunit.client.0.smithi117.stdout:3/273: getdents d0/d2a 0 2024-04-20T11:09:00.648 INFO:tasks.workunit.client.0.smithi117.stdout:4/318: readlink d1/dd/d1a/d2b/l3f 0 2024-04-20T11:09:00.650 INFO:tasks.workunit.client.0.smithi117.stdout:5/383: mknod de/c80 0 2024-04-20T11:09:00.650 INFO:tasks.workunit.client.0.smithi117.stdout:1/318: truncate d6/d35/f36 2770241 0 2024-04-20T11:09:00.659 INFO:tasks.workunit.client.0.smithi117.stdout:2/337: rename d1/d4/c60 to d1/d20/c67 0 2024-04-20T11:09:00.661 INFO:tasks.workunit.client.0.smithi117.stdout:3/274: symlink d0/d5/d7/l64 0 2024-04-20T11:09:00.662 INFO:tasks.workunit.client.0.smithi117.stdout:0/244: getdents d3/d7 0 2024-04-20T11:09:00.667 INFO:tasks.workunit.client.0.smithi117.stdout:3/275: unlink d0/d5/d7/f20 0 2024-04-20T11:09:00.669 INFO:tasks.workunit.client.0.smithi117.stdout:3/276: chown d0/d2a/f3b 11239912 1 2024-04-20T11:09:00.670 INFO:tasks.workunit.client.0.smithi117.stdout:2/338: mknod d1/d20/c68 0 2024-04-20T11:09:00.672 INFO:tasks.workunit.client.0.smithi117.stdout:0/245: mknod d3/d7/d8/df/d22/c5c 0 2024-04-20T11:09:00.679 INFO:tasks.workunit.client.0.smithi117.stdout:2/339: mkdir d1/d4/d69 0 2024-04-20T11:09:00.681 INFO:tasks.workunit.client.0.smithi117.stdout:4/319: dwrite d1/dd/f4b [0,4194304] 0 2024-04-20T11:09:00.693 INFO:tasks.workunit.client.0.smithi117.stdout:2/340: truncate d1/d15/d1d/f31 800885 0 2024-04-20T11:09:00.695 INFO:tasks.workunit.client.0.smithi117.stdout:4/320: write d1/f19 [1064552,110577] 0 2024-04-20T11:09:00.698 INFO:tasks.workunit.client.0.smithi117.stdout:5/384: chown de/d21/d45/d48/l6d 1 1 2024-04-20T11:09:00.705 INFO:tasks.workunit.client.0.smithi117.stdout:5/385: chown de/d21/d35 0 1 2024-04-20T11:09:00.706 INFO:tasks.workunit.client.0.smithi117.stdout:9/328: dwrite d4/d5/da/f19 [0,4194304] 0 2024-04-20T11:09:00.710 INFO:tasks.workunit.client.0.smithi117.stdout:1/319: dwrite d6/f10 [0,4194304] 0 2024-04-20T11:09:00.711 INFO:tasks.workunit.client.0.smithi117.stdout:0/246: dread d3/d5/f26 [0,4194304] 0 2024-04-20T11:09:00.713 INFO:tasks.workunit.client.0.smithi117.stdout:5/386: stat de/d21/d45/d48/c61 0 2024-04-20T11:09:00.713 INFO:tasks.workunit.client.0.smithi117.stdout:9/329: chown d4/d5/f2a 35605679 1 2024-04-20T11:09:00.713 INFO:tasks.workunit.client.0.smithi117.stdout:4/321: symlink d1/d9/d1f/l70 0 2024-04-20T11:09:00.717 INFO:tasks.workunit.client.0.smithi117.stdout:2/341: rename d1/d20/c67 to d1/d4/dd/de/c6a 0 2024-04-20T11:09:00.720 INFO:tasks.workunit.client.0.smithi117.stdout:3/277: dwrite d0/d5/de/d1d/f1f [0,4194304] 0 2024-04-20T11:09:00.721 INFO:tasks.workunit.client.0.smithi117.stdout:4/322: write d1/f19 [5185762,71823] 0 2024-04-20T11:09:00.721 INFO:tasks.workunit.client.0.smithi117.stdout:9/330: chown d4/de/d10/l67 22182 1 2024-04-20T11:09:00.722 INFO:tasks.workunit.client.0.smithi117.stdout:0/247: mknod d3/d7/d8/d2a/c5d 0 2024-04-20T11:09:00.724 INFO:tasks.workunit.client.0.smithi117.stdout:5/387: readlink de/d41/d47/l53 0 2024-04-20T11:09:00.724 INFO:tasks.workunit.client.0.smithi117.stdout:2/342: rmdir d1/d4/dd/d17 39 2024-04-20T11:09:00.726 INFO:tasks.workunit.client.0.smithi117.stdout:4/323: fsync d1/dd/d1a/d43/f60 0 2024-04-20T11:09:00.728 INFO:tasks.workunit.client.0.smithi117.stdout:0/248: fsync d3/d5/f37 0 2024-04-20T11:09:00.728 INFO:tasks.workunit.client.0.smithi117.stdout:3/278: write d0/f1a [6769927,9960] 0 2024-04-20T11:09:00.729 INFO:tasks.workunit.client.0.smithi117.stdout:5/388: creat de/d5b/f81 x:0 0 0 2024-04-20T11:09:00.730 INFO:tasks.workunit.client.0.smithi117.stdout:2/343: mkdir d1/d15/d1d/d6b 0 2024-04-20T11:09:00.732 INFO:tasks.workunit.client.0.smithi117.stdout:4/324: mknod d1/dd/d52/c71 0 2024-04-20T11:09:00.735 INFO:tasks.workunit.client.0.smithi117.stdout:4/325: chown d1/d45/d4a/d59 504 1 2024-04-20T11:09:00.735 INFO:tasks.workunit.client.0.smithi117.stdout:5/389: rename de/d21/d45/d48 to de/d21/d45/d48/d82 22 2024-04-20T11:09:00.737 INFO:tasks.workunit.client.0.smithi117.stdout:0/249: truncate d3/d7/f19 2560414 0 2024-04-20T11:09:00.743 INFO:tasks.workunit.client.0.smithi117.stdout:5/390: rmdir de/d16 39 2024-04-20T11:09:00.745 INFO:tasks.workunit.client.0.smithi117.stdout:9/331: dwrite d4/d30/f36 [0,4194304] 0 2024-04-20T11:09:00.752 INFO:tasks.workunit.client.0.smithi117.stdout:0/250: fdatasync d3/d7/d8/df/f1b 0 2024-04-20T11:09:00.755 INFO:tasks.workunit.client.0.smithi117.stdout:8/218: sync 2024-04-20T11:09:00.755 INFO:tasks.workunit.client.0.smithi117.stdout:9/332: mkdir d4/de/d7b 0 2024-04-20T11:09:00.755 INFO:tasks.workunit.client.0.smithi117.stdout:8/219: stat d0/l21 0 2024-04-20T11:09:00.756 INFO:tasks.workunit.client.0.smithi117.stdout:5/391: fdatasync de/f18 0 2024-04-20T11:09:00.760 INFO:tasks.workunit.client.0.smithi117.stdout:4/326: dread d1/f25 [0,4194304] 0 2024-04-20T11:09:00.766 INFO:tasks.workunit.client.0.smithi117.stdout:9/333: symlink d4/d4a/d72/l7c 0 2024-04-20T11:09:00.767 INFO:tasks.workunit.client.0.smithi117.stdout:4/327: chown d1/d9/d1f/f36 15866 1 2024-04-20T11:09:00.770 INFO:tasks.workunit.client.0.smithi117.stdout:9/334: fsync d4/d5/da/f6f 0 2024-04-20T11:09:00.770 INFO:tasks.workunit.client.0.smithi117.stdout:0/251: truncate d3/d7/d8/df/f50 3318885 0 2024-04-20T11:09:00.775 INFO:tasks.workunit.client.0.smithi117.stdout:0/252: truncate d3/f3e 984262 0 2024-04-20T11:09:00.777 INFO:tasks.workunit.client.0.smithi117.stdout:5/392: link de/d21/d45/d48/l71 de/d5b/l83 0 2024-04-20T11:09:00.777 INFO:tasks.workunit.client.0.smithi117.stdout:2/344: dread d1/f3 [0,4194304] 0 2024-04-20T11:09:00.778 INFO:tasks.workunit.client.0.smithi117.stdout:0/253: mknod d3/d7/d8/c5e 0 2024-04-20T11:09:00.778 INFO:tasks.workunit.client.0.smithi117.stdout:9/335: creat d4/d5/d3c/f7d x:0 0 0 2024-04-20T11:09:00.780 INFO:tasks.workunit.client.0.smithi117.stdout:3/279: dread d0/d5/d7/dd/ff [0,4194304] 0 2024-04-20T11:09:00.783 INFO:tasks.workunit.client.0.smithi117.stdout:5/393: write de/d41/d47/d4c/f75 [254729,31914] 0 2024-04-20T11:09:00.784 INFO:tasks.workunit.client.0.smithi117.stdout:9/336: creat d4/d4a/f7e x:0 0 0 2024-04-20T11:09:00.786 INFO:tasks.workunit.client.0.smithi117.stdout:2/345: creat d1/d4/dd/d17/d3b/f6c x:0 0 0 2024-04-20T11:09:00.787 INFO:tasks.workunit.client.0.smithi117.stdout:8/220: dwrite d0/d9/f2a [0,4194304] 0 2024-04-20T11:09:00.791 INFO:tasks.workunit.client.0.smithi117.stdout:8/221: chown d0/l44 16 1 2024-04-20T11:09:00.791 INFO:tasks.workunit.client.0.smithi117.stdout:9/337: dread d4/d5/d3c/f46 [0,4194304] 0 2024-04-20T11:09:00.791 INFO:tasks.workunit.client.0.smithi117.stdout:0/254: symlink d3/d7/d8/d2a/d54/l5f 0 2024-04-20T11:09:00.793 INFO:tasks.workunit.client.0.smithi117.stdout:0/255: fsync d3/d5/f37 0 2024-04-20T11:09:00.795 INFO:tasks.workunit.client.0.smithi117.stdout:3/280: mknod d0/d5/de/d1d/c65 0 2024-04-20T11:09:00.798 INFO:tasks.workunit.client.0.smithi117.stdout:2/346: mknod d1/d4/dd/d17/c6d 0 2024-04-20T11:09:00.800 INFO:tasks.workunit.client.0.smithi117.stdout:8/222: symlink d0/dc/l45 0 2024-04-20T11:09:00.803 INFO:tasks.workunit.client.0.smithi117.stdout:9/338: symlink d4/d30/l7f 0 2024-04-20T11:09:00.804 INFO:tasks.workunit.client.0.smithi117.stdout:5/394: mkdir de/d21/d84 0 2024-04-20T11:09:00.806 INFO:tasks.workunit.client.0.smithi117.stdout:0/256: fsync d3/d7/d20/f4c 0 2024-04-20T11:09:00.808 INFO:tasks.workunit.client.0.smithi117.stdout:2/347: mknod d1/d4/dd/de/d29/d2e/c6e 0 2024-04-20T11:09:00.810 INFO:tasks.workunit.client.0.smithi117.stdout:9/339: rmdir d4/de 39 2024-04-20T11:09:00.816 INFO:tasks.workunit.client.0.smithi117.stdout:8/223: truncate d0/d9/f33 2668302 0 2024-04-20T11:09:00.816 INFO:tasks.workunit.client.0.smithi117.stdout:3/281: dwrite d0/d5/de/d1d/f44 [0,4194304] 0 2024-04-20T11:09:00.817 INFO:tasks.workunit.client.0.smithi117.stdout:5/395: dread de/d41/d47/d55/d56/f78 [0,4194304] 0 2024-04-20T11:09:00.820 INFO:tasks.workunit.client.0.smithi117.stdout:5/396: fdatasync de/d41/d47/d55/d56/f79 0 2024-04-20T11:09:00.823 INFO:tasks.workunit.client.0.smithi117.stdout:2/348: unlink d1/d20/d26/f37 0 2024-04-20T11:09:00.825 INFO:tasks.workunit.client.0.smithi117.stdout:2/349: chown d1/d15/d1d/c4e 10387 1 2024-04-20T11:09:00.826 INFO:tasks.workunit.client.0.smithi117.stdout:5/397: stat de/d21/c23 0 2024-04-20T11:09:00.827 INFO:tasks.workunit.client.0.smithi117.stdout:8/224: rmdir d0/d8/d11/d24 39 2024-04-20T11:09:00.831 INFO:tasks.workunit.client.0.smithi117.stdout:0/257: getdents d3/d7/d2c/d52 0 2024-04-20T11:09:00.833 INFO:tasks.workunit.client.0.smithi117.stdout:5/398: chown de/d5b/l83 0 1 2024-04-20T11:09:00.834 INFO:tasks.workunit.client.0.smithi117.stdout:0/258: chown d3/d7/d2c/c43 27 1 2024-04-20T11:09:00.834 INFO:tasks.workunit.client.0.smithi117.stdout:2/350: mkdir d1/d4/dd/de/d29/d2e/d53/d58/d6f 0 2024-04-20T11:09:00.835 INFO:tasks.workunit.client.0.smithi117.stdout:9/340: getdents d4/d4a/d5e 0 2024-04-20T11:09:00.836 INFO:tasks.workunit.client.0.smithi117.stdout:5/399: creat de/d21/d45/d77/f85 x:0 0 0 2024-04-20T11:09:00.838 INFO:tasks.workunit.client.0.smithi117.stdout:0/259: creat d3/d5/f60 x:0 0 0 2024-04-20T11:09:00.839 INFO:tasks.workunit.client.0.smithi117.stdout:9/341: readlink d4/de/d10/l1d 0 2024-04-20T11:09:00.842 INFO:tasks.workunit.client.0.smithi117.stdout:0/260: dread d3/f3e [0,4194304] 0 2024-04-20T11:09:00.843 INFO:tasks.workunit.client.0.smithi117.stdout:2/351: rename d1/d15/d1d/l57 to d1/d4/dd/de/d29/d62/l70 0 2024-04-20T11:09:00.854 INFO:tasks.workunit.client.0.smithi117.stdout:9/342: symlink d4/d5/d3c/d7a/l80 0 2024-04-20T11:09:00.857 INFO:tasks.workunit.client.0.smithi117.stdout:2/352: symlink d1/d20/l71 0 2024-04-20T11:09:00.859 INFO:tasks.workunit.client.0.smithi117.stdout:9/343: fsync d4/d30/f6b 0 2024-04-20T11:09:00.862 INFO:tasks.workunit.client.0.smithi117.stdout:3/282: dread d0/d5/f19 [0,4194304] 0 2024-04-20T11:09:00.865 INFO:tasks.workunit.client.0.smithi117.stdout:2/353: creat d1/d15/f72 x:0 0 0 2024-04-20T11:09:00.865 INFO:tasks.workunit.client.0.smithi117.stdout:0/261: dwrite d3/d7/f4d [0,4194304] 0 2024-04-20T11:09:00.867 INFO:tasks.workunit.client.0.smithi117.stdout:5/400: dwrite de/d21/d35/f3a [0,4194304] 0 2024-04-20T11:09:00.867 INFO:tasks.workunit.client.0.smithi117.stdout:9/344: creat d4/d35/f81 x:0 0 0 2024-04-20T11:09:00.869 INFO:tasks.workunit.client.0.smithi117.stdout:0/262: truncate d3/d7/d8/df/f46 358607 0 2024-04-20T11:09:00.870 INFO:tasks.workunit.client.0.smithi117.stdout:0/263: stat d3/d5/f59 0 2024-04-20T11:09:00.881 INFO:tasks.workunit.client.0.smithi117.stdout:2/354: mknod d1/d4/dd/de/d29/d2e/d53/c73 0 2024-04-20T11:09:00.885 INFO:tasks.workunit.client.0.smithi117.stdout:5/401: mknod de/d16/c86 0 2024-04-20T11:09:00.887 INFO:tasks.workunit.client.0.smithi117.stdout:2/355: write d1/f11 [5046573,94247] 0 2024-04-20T11:09:00.888 INFO:tasks.workunit.client.0.smithi117.stdout:9/345: dwrite d4/de/d4b/f6d [0,4194304] 0 2024-04-20T11:09:00.890 INFO:tasks.workunit.client.0.smithi117.stdout:2/356: chown d1/d4/dd/d17/d3b 168102078 1 2024-04-20T11:09:00.892 INFO:tasks.workunit.client.0.smithi117.stdout:9/346: mkdir d4/d5/d57/d82 0 2024-04-20T11:09:00.892 INFO:tasks.workunit.client.0.smithi117.stdout:0/264: dread d3/d7/d8/df/f50 [0,4194304] 0 2024-04-20T11:09:00.893 INFO:tasks.workunit.client.0.smithi117.stdout:0/265: fsync d3/d7/f3b 0 2024-04-20T11:09:00.894 INFO:tasks.workunit.client.0.smithi117.stdout:9/347: chown d4/d5/d3c/d7a/l80 16 1 2024-04-20T11:09:00.894 INFO:tasks.workunit.client.0.smithi117.stdout:2/357: creat d1/d4/dd/de/d29/d2e/f74 x:0 0 0 2024-04-20T11:09:00.965 INFO:tasks.workunit.client.0.smithi117.stdout:6/387: sync 2024-04-20T11:09:00.966 INFO:tasks.workunit.client.0.smithi117.stdout:7/317: sync 2024-04-20T11:09:00.969 INFO:tasks.workunit.client.0.smithi117.stdout:2/358: write d1/d4/f45 [286520,50737] 0 2024-04-20T11:09:00.971 INFO:tasks.workunit.client.0.smithi117.stdout:9/348: write d4/d4a/d63/f76 [579620,63652] 0 2024-04-20T11:09:00.971 INFO:tasks.workunit.client.0.smithi117.stdout:6/388: read - d3/d4/d2e/d37/f62 zero size 2024-04-20T11:09:00.975 INFO:tasks.workunit.client.0.smithi117.stdout:6/389: write d3/d5a/d79/f6f [623918,79576] 0 2024-04-20T11:09:00.979 INFO:tasks.workunit.client.0.smithi117.stdout:9/349: rename d4/d5/d3c/d7a/l80 to d4/de/d4b/l83 0 2024-04-20T11:09:00.980 INFO:tasks.workunit.client.0.smithi117.stdout:6/390: write d3/df/d39/d76/f7a [728836,93669] 0 2024-04-20T11:09:00.983 INFO:tasks.workunit.client.0.smithi117.stdout:2/359: link d1/d4/dd/d3e/l42 d1/d4/dd/de/d29/d2e/l75 0 2024-04-20T11:09:00.984 INFO:tasks.workunit.client.0.smithi117.stdout:3/283: dread d0/d5/d7/dd/f10 [0,4194304] 0 2024-04-20T11:09:00.985 INFO:tasks.workunit.client.0.smithi117.stdout:2/360: rmdir d1/d4 39 2024-04-20T11:09:00.986 INFO:tasks.workunit.client.0.smithi117.stdout:9/350: getdents d4/d30 0 2024-04-20T11:09:00.988 INFO:tasks.workunit.client.0.smithi117.stdout:9/351: fdatasync d4/d5/da/f6f 0 2024-04-20T11:09:00.991 INFO:tasks.workunit.client.0.smithi117.stdout:3/284: rename d0/d5/d11/d46 to d0/d5/de/d1d/d29/d66 0 2024-04-20T11:09:00.999 INFO:tasks.workunit.client.0.smithi117.stdout:9/352: truncate d4/d5/f21 2337133 0 2024-04-20T11:09:01.002 INFO:tasks.workunit.client.0.smithi117.stdout:9/353: read d4/f45 [234543,35800] 0 2024-04-20T11:09:01.003 INFO:tasks.workunit.client.0.smithi117.stdout:9/354: dread - d4/d30/d31/f6c zero size 2024-04-20T11:09:01.004 INFO:tasks.workunit.client.0.smithi117.stdout:2/361: rmdir d1/d4/dd/de/d29/d2e/d53/d58/d6f 0 2024-04-20T11:09:01.006 INFO:tasks.workunit.client.0.smithi117.stdout:3/285: write d0/f3 [1317165,85704] 0 2024-04-20T11:09:01.008 INFO:tasks.workunit.client.0.smithi117.stdout:2/362: rmdir d1/d4/dd/de/d29/d2e/d52 39 2024-04-20T11:09:01.015 INFO:tasks.workunit.client.0.smithi117.stdout:2/363: truncate d1/d4/dd/de/d29/d2e/f40 787405 0 2024-04-20T11:09:01.021 INFO:tasks.workunit.client.0.smithi117.stdout:2/364: creat d1/d4/dd/de/d29/d3f/f76 x:0 0 0 2024-04-20T11:09:01.021 INFO:tasks.workunit.client.0.smithi117.stdout:9/355: dwrite d4/de/d4b/f6d [0,4194304] 0 2024-04-20T11:09:01.023 INFO:tasks.workunit.client.0.smithi117.stdout:2/365: write d1/d4/dd/f55 [3068640,19201] 0 2024-04-20T11:09:01.024 INFO:tasks.workunit.client.0.smithi117.stdout:3/286: dwrite d0/fb [0,4194304] 0 2024-04-20T11:09:01.026 INFO:tasks.workunit.client.0.smithi117.stdout:3/287: dread - d0/d5/d21/f4a zero size 2024-04-20T11:09:01.026 INFO:tasks.workunit.client.0.smithi117.stdout:2/366: truncate d1/d4/dd/d17/f1f 5637359 0 2024-04-20T11:09:01.033 INFO:tasks.workunit.client.0.smithi117.stdout:2/367: creat d1/d4/dd/f77 x:0 0 0 2024-04-20T11:09:01.036 INFO:tasks.workunit.client.0.smithi117.stdout:2/368: write d1/d4/dd/de/d29/d3f/f76 [99881,95054] 0 2024-04-20T11:09:01.038 INFO:tasks.workunit.client.0.smithi117.stdout:9/356: dwrite d4/d68/d6e/f77 [0,4194304] 0 2024-04-20T11:09:01.040 INFO:tasks.workunit.client.0.smithi117.stdout:3/288: rename d0/d5/d7/dd/c61 to d0/d5/de/d1d/d29/d66/d63/c67 0 2024-04-20T11:09:01.051 INFO:tasks.workunit.client.0.smithi117.stdout:9/357: rename d4/de/d10/f1a to d4/de/f84 0 2024-04-20T11:09:01.055 INFO:tasks.workunit.client.0.smithi117.stdout:3/289: dread d0/d5/d7/dd/f10 [4194304,4194304] 0 2024-04-20T11:09:01.056 INFO:tasks.workunit.client.0.smithi117.stdout:3/290: chown d0 128880 1 2024-04-20T11:09:01.058 INFO:tasks.workunit.client.0.smithi117.stdout:2/369: dwrite d1/d4/dd/f55 [4194304,4194304] 0 2024-04-20T11:09:01.062 INFO:tasks.workunit.client.0.smithi117.stdout:3/291: mknod d0/d2a/c68 0 2024-04-20T11:09:01.066 INFO:tasks.workunit.client.0.smithi117.stdout:3/292: fdatasync d0/d5/de/d1d/d29/f31 0 2024-04-20T11:09:01.071 INFO:tasks.workunit.client.0.smithi117.stdout:9/358: dwrite d4/de/d10/f5c [0,4194304] 0 2024-04-20T11:09:01.080 INFO:tasks.workunit.client.0.smithi117.stdout:9/359: creat d4/d4a/d74/f85 x:0 0 0 2024-04-20T11:09:01.086 INFO:tasks.workunit.client.0.smithi117.stdout:2/370: dwrite d1/d4/dd/de/f3a [0,4194304] 0 2024-04-20T11:09:01.091 INFO:tasks.workunit.client.0.smithi117.stdout:2/371: mknod d1/d20/d26/d49/c78 0 2024-04-20T11:09:01.091 INFO:tasks.workunit.client.0.smithi117.stdout:3/293: dwrite d0/d5/de/d1d/f44 [4194304,4194304] 0 2024-04-20T11:09:01.095 INFO:tasks.workunit.client.0.smithi117.stdout:3/294: rmdir d0/d5/d7 39 2024-04-20T11:09:01.097 INFO:tasks.workunit.client.0.smithi117.stdout:9/360: dread d4/de/d10/f38 [0,4194304] 0 2024-04-20T11:09:01.099 INFO:tasks.workunit.client.0.smithi117.stdout:9/361: write d4/d4a/f60 [3905680,82144] 0 2024-04-20T11:09:01.103 INFO:tasks.workunit.client.0.smithi117.stdout:9/362: chown d4/d35/f3d 602 1 2024-04-20T11:09:01.113 INFO:tasks.workunit.client.0.smithi117.stdout:2/372: dwrite d1/d4/dd/d17/f1f [4194304,4194304] 0 2024-04-20T11:09:01.193 INFO:tasks.workunit.client.0.smithi117.stdout:9/363: creat d4/de/d4b/f86 x:0 0 0 2024-04-20T11:09:01.193 INFO:tasks.workunit.client.0.smithi117.stdout:3/295: truncate d0/d5/d7/dd/f59 3317386 0 2024-04-20T11:09:01.195 INFO:tasks.workunit.client.0.smithi117.stdout:3/296: readlink d0/d5/d7/l2e 0 2024-04-20T11:09:01.196 INFO:tasks.workunit.client.0.smithi117.stdout:2/373: fsync d1/d4/fb 0 2024-04-20T11:09:01.200 INFO:tasks.workunit.client.0.smithi117.stdout:3/297: truncate d0/d5/d21/f47 69711 0 2024-04-20T11:09:01.202 INFO:tasks.workunit.client.0.smithi117.stdout:3/298: stat d0/d5/de/d1d/d29/l57 0 2024-04-20T11:09:01.210 INFO:tasks.workunit.client.0.smithi117.stdout:3/299: truncate d0/d5/d21/f41 213420 0 2024-04-20T11:09:01.211 INFO:tasks.workunit.client.0.smithi117.stdout:2/374: dread d1/d4/dd/d17/f30 [0,4194304] 0 2024-04-20T11:09:01.213 INFO:tasks.workunit.client.0.smithi117.stdout:3/300: mkdir d0/d5/de/d1d/d69 0 2024-04-20T11:09:01.216 INFO:tasks.workunit.client.0.smithi117.stdout:2/375: dread d1/d15/d1d/f31 [0,4194304] 0 2024-04-20T11:09:01.228 INFO:tasks.workunit.client.0.smithi117.stdout:3/301: dread d0/d5/de/d1d/f44 [0,4194304] 0 2024-04-20T11:09:01.229 INFO:tasks.workunit.client.0.smithi117.stdout:2/376: dread d1/d4/f5 [0,4194304] 0 2024-04-20T11:09:01.232 INFO:tasks.workunit.client.0.smithi117.stdout:2/377: write d1/d4/dd/d17/f1e [3872700,81138] 0 2024-04-20T11:09:01.237 INFO:tasks.workunit.client.0.smithi117.stdout:2/378: creat d1/d4/dd/de/f79 x:0 0 0 2024-04-20T11:09:01.238 INFO:tasks.workunit.client.0.smithi117.stdout:3/302: link d0/d5/d21/l4f d0/d5/de/d4e/l6a 0 2024-04-20T11:09:01.244 INFO:tasks.workunit.client.0.smithi117.stdout:3/303: link d0/c54 d0/d5/d11/c6b 0 2024-04-20T11:09:01.250 INFO:tasks.workunit.client.0.smithi117.stdout:3/304: creat d0/d5/de/d1d/d29/d66/f6c x:0 0 0 2024-04-20T11:09:01.253 INFO:tasks.workunit.client.0.smithi117.stdout:3/305: unlink d0/d5/d7/l64 0 2024-04-20T11:09:01.256 INFO:tasks.workunit.client.0.smithi117.stdout:3/306: creat d0/d5/de/d1d/d69/f6d x:0 0 0 2024-04-20T11:09:01.257 INFO:tasks.workunit.client.0.smithi117.stdout:2/379: dwrite d1/f6 [4194304,4194304] 0 2024-04-20T11:09:01.263 INFO:tasks.workunit.client.0.smithi117.stdout:3/307: write d0/d5/de/d1d/f44 [228425,60555] 0 2024-04-20T11:09:01.268 INFO:tasks.workunit.client.0.smithi117.stdout:3/308: unlink d0/d5/de/d1d/l60 0 2024-04-20T11:09:01.271 INFO:tasks.workunit.client.0.smithi117.stdout:3/309: dread - d0/d5/de/d1d/d69/f6d zero size 2024-04-20T11:09:01.272 INFO:tasks.workunit.client.0.smithi117.stdout:3/310: fdatasync d0/d5/d21/f4d 0 2024-04-20T11:09:01.288 INFO:tasks.workunit.client.0.smithi117.stdout:2/380: fsync d1/d4/f5 0 2024-04-20T11:09:01.297 INFO:tasks.workunit.client.0.smithi117.stdout:2/381: rename d1/d20/d26 to d1/d4/dd/d17/d7a 0 2024-04-20T11:09:01.298 INFO:tasks.workunit.client.0.smithi117.stdout:2/382: readlink d1/d4/dd/l32 0 2024-04-20T11:09:01.302 INFO:tasks.workunit.client.0.smithi117.stdout:2/383: write d1/d4/f45 [908432,104510] 0 2024-04-20T11:09:01.306 INFO:tasks.workunit.client.0.smithi117.stdout:3/311: truncate d0/d5/d7/dd/f10 6140932 0 2024-04-20T11:09:01.308 INFO:tasks.workunit.client.0.smithi117.stdout:2/384: creat d1/f7b x:0 0 0 2024-04-20T11:09:01.309 INFO:tasks.workunit.client.0.smithi117.stdout:1/320: sync 2024-04-20T11:09:01.324 INFO:tasks.workunit.client.0.smithi117.stdout:2/385: dread d1/d4/dd/d17/f1f [4194304,4194304] 0 2024-04-20T11:09:01.330 INFO:tasks.workunit.client.0.smithi117.stdout:2/386: getdents d1/d4/dd/de/d29/d2e/d53/d58 0 2024-04-20T11:09:01.331 INFO:tasks.workunit.client.0.smithi117.stdout:1/321: dwrite d6/d16/d3a/d5a/f53 [0,4194304] 0 2024-04-20T11:09:01.334 INFO:tasks.workunit.client.0.smithi117.stdout:1/322: write d6/d1f/f3d [1869754,77979] 0 2024-04-20T11:09:01.342 INFO:tasks.workunit.client.0.smithi117.stdout:1/323: link d6/d1f/d30/d32/f46 d6/f68 0 2024-04-20T11:09:01.346 INFO:tasks.workunit.client.0.smithi117.stdout:1/324: mkdir d6/d1f/d30/d49/d69 0 2024-04-20T11:09:01.348 INFO:tasks.workunit.client.0.smithi117.stdout:2/387: dwrite d1/d15/d1d/f50 [0,4194304] 0 2024-04-20T11:09:01.355 INFO:tasks.workunit.client.0.smithi117.stdout:1/325: rename d6/d1f/d30/d32/f46 to d6/d1f/d30/d32/f6a 0 2024-04-20T11:09:01.359 INFO:tasks.workunit.client.0.smithi117.stdout:1/326: mkdir d6/d1f/d30/d32/d57/d6b 0 2024-04-20T11:09:01.360 INFO:tasks.workunit.client.0.smithi117.stdout:1/327: chown d6/d21/f31 33539521 1 2024-04-20T11:09:01.363 INFO:tasks.workunit.client.0.smithi117.stdout:2/388: dread d1/d4/dd/d17/f30 [0,4194304] 0 2024-04-20T11:09:01.364 INFO:tasks.workunit.client.0.smithi117.stdout:2/389: readlink d1/d20/l71 0 2024-04-20T11:09:01.364 INFO:tasks.workunit.client.0.smithi117.stdout:1/328: creat d6/d1f/d30/d32/d57/d6b/f6c x:0 0 0 2024-04-20T11:09:01.365 INFO:tasks.workunit.client.0.smithi117.stdout:2/390: fdatasync d1/f11 0 2024-04-20T11:09:01.365 INFO:tasks.workunit.client.0.smithi117.stdout:1/329: chown d6/d16/l1e 4 1 2024-04-20T11:09:01.368 INFO:tasks.workunit.client.0.smithi117.stdout:2/391: mknod d1/d4/dd/de/c7c 0 2024-04-20T11:09:01.380 INFO:tasks.workunit.client.0.smithi117.stdout:1/330: dwrite d6/d1f/f3e [0,4194304] 0 2024-04-20T11:09:01.383 INFO:tasks.workunit.client.0.smithi117.stdout:2/392: dwrite d1/d4/dd/d3e/f56 [0,4194304] 0 2024-04-20T11:09:01.386 INFO:tasks.workunit.client.0.smithi117.stdout:1/331: mknod d6/d1f/d30/d49/d69/c6d 0 2024-04-20T11:09:01.392 INFO:tasks.workunit.client.0.smithi117.stdout:1/332: chown d6/d1f/l47 920036719 1 2024-04-20T11:09:01.395 INFO:tasks.workunit.client.0.smithi117.stdout:1/333: mkdir d6/d1f/d30/d49/d69/d6e 0 2024-04-20T11:09:01.398 INFO:tasks.workunit.client.0.smithi117.stdout:1/334: mkdir d6/d1f/d30/d49/d69/d6e/d6f 0 2024-04-20T11:09:01.404 INFO:tasks.workunit.client.0.smithi117.stdout:1/335: rename d6/d16/c42 to d6/d16/d3a/d5a/d5c/c70 0 2024-04-20T11:09:01.416 INFO:tasks.workunit.client.0.smithi117.stdout:2/393: truncate d1/d4/dd/d17/f1e 1585272 0 2024-04-20T11:09:01.432 INFO:tasks.workunit.client.0.smithi117.stdout:2/394: dwrite d1/d4/dd/de/f18 [0,4194304] 0 2024-04-20T11:09:01.436 INFO:tasks.workunit.client.0.smithi117.stdout:2/395: fdatasync d1/d15/d1d/f31 0 2024-04-20T11:09:01.454 INFO:tasks.workunit.client.0.smithi117.stdout:4/328: sync 2024-04-20T11:09:01.458 INFO:tasks.workunit.client.0.smithi117.stdout:4/329: unlink d1/f6 0 2024-04-20T11:09:01.481 INFO:tasks.workunit.client.0.smithi117.stdout:4/330: dwrite d1/f25 [4194304,4194304] 0 2024-04-20T11:09:01.497 INFO:tasks.workunit.client.0.smithi117.stdout:4/331: dwrite d1/d9/d1f/f55 [0,4194304] 0 2024-04-20T11:09:01.500 INFO:tasks.workunit.client.0.smithi117.stdout:4/332: creat d1/f72 x:0 0 0 2024-04-20T11:09:01.503 INFO:tasks.workunit.client.0.smithi117.stdout:4/333: write d1/d9/d1f/f39 [913680,127630] 0 2024-04-20T11:09:01.505 INFO:tasks.workunit.client.0.smithi117.stdout:4/334: read - d1/dd/d1a/f37 zero size 2024-04-20T11:09:01.507 INFO:tasks.workunit.client.0.smithi117.stdout:4/335: chown d1/dd/d1a/f37 56 1 2024-04-20T11:09:01.514 INFO:tasks.workunit.client.0.smithi117.stdout:4/336: getdents d1/d45/d4a 0 2024-04-20T11:09:01.529 INFO:tasks.workunit.client.0.smithi117.stdout:4/337: dwrite d1/d45/d4a/f50 [0,4194304] 0 2024-04-20T11:09:01.538 INFO:tasks.workunit.client.0.smithi117.stdout:4/338: creat d1/dd/d1a/d43/d4e/f73 x:0 0 0 2024-04-20T11:09:01.558 INFO:tasks.workunit.client.0.smithi117.stdout:4/339: dwrite d1/d9/d1f/f36 [0,4194304] 0 2024-04-20T11:09:01.560 INFO:tasks.workunit.client.0.smithi117.stdout:4/340: chown d1/d9/d1f/l70 16201 1 2024-04-20T11:09:01.562 INFO:tasks.workunit.client.0.smithi117.stdout:4/341: fsync d1/dd/d1a/d2b/f46 0 2024-04-20T11:09:01.563 INFO:tasks.workunit.client.0.smithi117.stdout:4/342: fdatasync d1/f25 0 2024-04-20T11:09:01.570 INFO:tasks.workunit.client.0.smithi117.stdout:4/343: getdents d1/d9 0 2024-04-20T11:09:01.573 INFO:tasks.workunit.client.0.smithi117.stdout:4/344: creat d1/d45/f74 x:0 0 0 2024-04-20T11:09:01.575 INFO:tasks.workunit.client.0.smithi117.stdout:4/345: mknod d1/d9/c75 0 2024-04-20T11:09:01.577 INFO:tasks.workunit.client.0.smithi117.stdout:4/346: creat d1/d9/f76 x:0 0 0 2024-04-20T11:09:01.579 INFO:tasks.workunit.client.0.smithi117.stdout:4/347: mknod d1/d9/d6b/c77 0 2024-04-20T11:09:01.581 INFO:tasks.workunit.client.0.smithi117.stdout:4/348: truncate d1/d9/f66 728724 0 2024-04-20T11:09:01.583 INFO:tasks.workunit.client.0.smithi117.stdout:4/349: write d1/d45/d4a/f6c [526309,81141] 0 2024-04-20T11:09:01.585 INFO:tasks.workunit.client.0.smithi117.stdout:4/350: symlink d1/d9/d1f/l78 0 2024-04-20T11:09:01.601 INFO:tasks.workunit.client.0.smithi117.stdout:4/351: dwrite d1/d45/d4a/f50 [4194304,4194304] 0 2024-04-20T11:09:01.612 INFO:tasks.workunit.client.0.smithi117.stdout:4/352: dread d1/d9/d1f/f36 [0,4194304] 0 2024-04-20T11:09:01.614 INFO:tasks.workunit.client.0.smithi117.stdout:4/353: truncate d1/dd/f53 439908 0 2024-04-20T11:09:01.627 INFO:tasks.workunit.client.0.smithi117.stdout:4/354: write d1/d9/d1f/f5c [336187,102606] 0 2024-04-20T11:09:01.628 INFO:tasks.workunit.client.0.smithi117.stdout:4/355: fsync d1/d45/f4f 0 2024-04-20T11:09:01.631 INFO:tasks.workunit.client.0.smithi117.stdout:4/356: mkdir d1/d9/d79 0 2024-04-20T11:09:01.632 INFO:tasks.workunit.client.0.smithi117.stdout:4/357: chown d1/f57 29062965 1 2024-04-20T11:09:01.635 INFO:tasks.workunit.client.0.smithi117.stdout:4/358: symlink d1/dd/d1a/d43/d4e/l7a 0 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:3/312: sync 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:6/391: sync 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:2/396: sync 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:5/402: sync 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:1/336: sync 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:8/225: sync 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:0/266: sync 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:7/318: sync 2024-04-20T11:09:01.637 INFO:tasks.workunit.client.0.smithi117.stdout:9/364: sync 2024-04-20T11:09:01.644 INFO:tasks.workunit.client.0.smithi117.stdout:5/403: chown de/d21/d45/d48/l6d 9831250 1 2024-04-20T11:09:01.644 INFO:tasks.workunit.client.0.smithi117.stdout:0/267: chown d3/d7/d20/l2f 95614 1 2024-04-20T11:09:01.644 INFO:tasks.workunit.client.0.smithi117.stdout:4/359: dread d1/d9/f66 [0,4194304] 0 2024-04-20T11:09:01.644 INFO:tasks.workunit.client.0.smithi117.stdout:8/226: dread d0/f23 [0,4194304] 0 2024-04-20T11:09:01.644 INFO:tasks.workunit.client.0.smithi117.stdout:6/392: write d3/d4/f19 [2797688,120617] 0 2024-04-20T11:09:01.645 INFO:tasks.workunit.client.0.smithi117.stdout:3/313: stat d0/d5/de/d1d/d29/f31 0 2024-04-20T11:09:01.647 INFO:tasks.workunit.client.0.smithi117.stdout:5/404: dread - de/d5b/f81 zero size 2024-04-20T11:09:01.650 INFO:tasks.workunit.client.0.smithi117.stdout:7/319: rmdir d1/d4/d6/d13 39 2024-04-20T11:09:01.651 INFO:tasks.workunit.client.0.smithi117.stdout:6/393: rmdir d3/d50 39 2024-04-20T11:09:01.652 INFO:tasks.workunit.client.0.smithi117.stdout:3/314: write d0/d5/d21/f4a [956918,76218] 0 2024-04-20T11:09:01.657 INFO:tasks.workunit.client.0.smithi117.stdout:6/394: write d3/f67 [627132,49736] 0 2024-04-20T11:09:01.659 INFO:tasks.workunit.client.0.smithi117.stdout:0/268: mknod d3/d7/d8/d2a/d54/c61 0 2024-04-20T11:09:01.659 INFO:tasks.workunit.client.0.smithi117.stdout:2/397: dread d1/f11 [4194304,4194304] 0 2024-04-20T11:09:01.662 INFO:tasks.workunit.client.0.smithi117.stdout:5/405: creat de/d21/d84/f87 x:0 0 0 2024-04-20T11:09:01.668 INFO:tasks.workunit.client.0.smithi117.stdout:8/227: getdents d0/d8 0 2024-04-20T11:09:01.671 INFO:tasks.workunit.client.0.smithi117.stdout:9/365: dwrite d4/d35/f5f [0,4194304] 0 2024-04-20T11:09:01.672 INFO:tasks.workunit.client.0.smithi117.stdout:7/320: creat d1/d43/d59/f69 x:0 0 0 2024-04-20T11:09:01.674 INFO:tasks.workunit.client.0.smithi117.stdout:2/398: readlink d1/d4/dd/de/d29/d2e/l66 0 2024-04-20T11:09:01.675 INFO:tasks.workunit.client.0.smithi117.stdout:6/395: creat d3/d4/d2e/d37/f7e x:0 0 0 2024-04-20T11:09:01.676 INFO:tasks.workunit.client.0.smithi117.stdout:0/269: creat d3/d7/d20/d39/f62 x:0 0 0 2024-04-20T11:09:01.678 INFO:tasks.workunit.client.0.smithi117.stdout:1/337: dwrite d6/f1a [0,4194304] 0 2024-04-20T11:09:01.678 INFO:tasks.workunit.client.0.smithi117.stdout:3/315: write d0/d5/fc [301401,55491] 0 2024-04-20T11:09:01.678 INFO:tasks.workunit.client.0.smithi117.stdout:5/406: symlink de/d21/d84/l88 0 2024-04-20T11:09:01.685 INFO:tasks.workunit.client.0.smithi117.stdout:2/399: creat d1/d4/dd/de/d29/d3f/f7d x:0 0 0 2024-04-20T11:09:01.691 INFO:tasks.workunit.client.0.smithi117.stdout:6/396: rename d3/df/d39/d76 to d3/d3a/d7f 0 2024-04-20T11:09:01.691 INFO:tasks.workunit.client.0.smithi117.stdout:7/321: write d1/f10 [5322455,63695] 0 2024-04-20T11:09:01.692 INFO:tasks.workunit.client.0.smithi117.stdout:3/316: mknod d0/d5/de/d1d/d29/c6e 0 2024-04-20T11:09:01.692 INFO:tasks.workunit.client.0.smithi117.stdout:5/407: truncate de/d21/d35/f3b 4776776 0 2024-04-20T11:09:01.696 INFO:tasks.workunit.client.0.smithi117.stdout:5/408: chown de/d16/f2d 69557 1 2024-04-20T11:09:01.697 INFO:tasks.workunit.client.0.smithi117.stdout:8/228: link d0/f23 d0/d8/d11/d24/f46 0 2024-04-20T11:09:01.699 INFO:tasks.workunit.client.0.smithi117.stdout:6/397: write d3/df/f4f [1274415,34581] 0 2024-04-20T11:09:01.701 INFO:tasks.workunit.client.0.smithi117.stdout:2/400: unlink d1/d4/dd/c28 0 2024-04-20T11:09:01.706 INFO:tasks.workunit.client.0.smithi117.stdout:9/366: dwrite d4/de/d4b/f58 [0,4194304] 0 2024-04-20T11:09:01.707 INFO:tasks.workunit.client.0.smithi117.stdout:7/322: mknod d1/d4/d3d/c6a 0 2024-04-20T11:09:01.709 INFO:tasks.workunit.client.0.smithi117.stdout:4/360: dread d1/dd/d1a/f1d [0,4194304] 0 2024-04-20T11:09:01.710 INFO:tasks.workunit.client.0.smithi117.stdout:1/338: dwrite d6/f9 [0,4194304] 0 2024-04-20T11:09:01.711 INFO:tasks.workunit.client.0.smithi117.stdout:5/409: truncate de/d21/d45/d77/f85 769334 0 2024-04-20T11:09:01.711 INFO:tasks.workunit.client.0.smithi117.stdout:8/229: readlink d0/l6 0 2024-04-20T11:09:01.712 INFO:tasks.workunit.client.0.smithi117.stdout:0/270: rename d3/d7/d8/df/f50 to d3/d7/d16/f63 0 2024-04-20T11:09:01.716 INFO:tasks.workunit.client.0.smithi117.stdout:0/271: write d3/d7/d8/df/f46 [244634,55926] 0 2024-04-20T11:09:01.717 INFO:tasks.workunit.client.0.smithi117.stdout:3/317: mknod d0/d5/de/d1d/d29/c6f 0 2024-04-20T11:09:01.717 INFO:tasks.workunit.client.0.smithi117.stdout:2/401: unlink d1/d4/dd/l32 0 2024-04-20T11:09:01.720 INFO:tasks.workunit.client.0.smithi117.stdout:9/367: rmdir d4/d4a 39 2024-04-20T11:09:01.722 INFO:tasks.workunit.client.0.smithi117.stdout:7/323: symlink d1/d4/d6/d2b/l6b 0 2024-04-20T11:09:01.726 INFO:tasks.workunit.client.0.smithi117.stdout:4/361: unlink d1/d9/f16 0 2024-04-20T11:09:01.730 INFO:tasks.workunit.client.0.smithi117.stdout:5/410: creat de/d41/d47/d4c/f89 x:0 0 0 2024-04-20T11:09:01.731 INFO:tasks.workunit.client.0.smithi117.stdout:6/398: rename d3/df/l33 to d3/d5a/d79/l80 0 2024-04-20T11:09:01.732 INFO:tasks.workunit.client.0.smithi117.stdout:9/368: creat d4/d5/d3c/f87 x:0 0 0 2024-04-20T11:09:01.734 INFO:tasks.workunit.client.0.smithi117.stdout:3/318: creat d0/d5/de/d1d/f70 x:0 0 0 2024-04-20T11:09:01.736 INFO:tasks.workunit.client.0.smithi117.stdout:9/369: write d4/d30/d31/f6c [895493,116183] 0 2024-04-20T11:09:01.738 INFO:tasks.workunit.client.0.smithi117.stdout:3/319: chown d0/d5/d7/dd/f5a 1 1 2024-04-20T11:09:01.740 INFO:tasks.workunit.client.0.smithi117.stdout:3/320: fsync d0/d5/fc 0 2024-04-20T11:09:01.742 INFO:tasks.workunit.client.0.smithi117.stdout:1/339: rename d6/d1f/d30/d32/d57 to d6/d16/d3a/d5a/d5c/d61/d71 0 2024-04-20T11:09:01.745 INFO:tasks.workunit.client.0.smithi117.stdout:7/324: dwrite d1/f7 [0,4194304] 0 2024-04-20T11:09:01.746 INFO:tasks.workunit.client.0.smithi117.stdout:1/340: truncate d6/d35/f44 1001151 0 2024-04-20T11:09:01.748 INFO:tasks.workunit.client.0.smithi117.stdout:6/399: creat d3/d4/d2e/d5f/f81 x:0 0 0 2024-04-20T11:09:01.749 INFO:tasks.workunit.client.0.smithi117.stdout:9/370: readlink d4/l52 0 2024-04-20T11:09:01.751 INFO:tasks.workunit.client.0.smithi117.stdout:6/400: fsync d3/d5a/d79/f6f 0 2024-04-20T11:09:01.752 INFO:tasks.workunit.client.0.smithi117.stdout:0/272: dwrite d3/d5/f26 [0,4194304] 0 2024-04-20T11:09:01.753 INFO:tasks.workunit.client.0.smithi117.stdout:2/402: truncate d1/d4/dd/d17/f30 3325817 0 2024-04-20T11:09:01.758 INFO:tasks.workunit.client.0.smithi117.stdout:5/411: unlink de/f1f 0 2024-04-20T11:09:01.767 INFO:tasks.workunit.client.0.smithi117.stdout:3/321: dread d0/d5/d21/f41 [0,4194304] 0 2024-04-20T11:09:01.768 INFO:tasks.workunit.client.0.smithi117.stdout:5/412: readlink de/d21/d45/d48/l6d 0 2024-04-20T11:09:01.771 INFO:tasks.workunit.client.0.smithi117.stdout:3/322: fsync d0/d5/d21/f4a 0 2024-04-20T11:09:01.772 INFO:tasks.workunit.client.0.smithi117.stdout:7/325: write d1/d4/d6/d13/f28 [779380,109978] 0 2024-04-20T11:09:01.772 INFO:tasks.workunit.client.0.smithi117.stdout:5/413: stat de/d41/d47/d55/d56 0 2024-04-20T11:09:01.775 INFO:tasks.workunit.client.0.smithi117.stdout:0/273: unlink d3/f21 0 2024-04-20T11:09:01.777 INFO:tasks.workunit.client.0.smithi117.stdout:4/362: link d1/dd/l3d d1/dd/l7b 0 2024-04-20T11:09:01.781 INFO:tasks.workunit.client.0.smithi117.stdout:8/230: mkdir d0/d8/d11/d24/d27/d47 0 2024-04-20T11:09:01.781 INFO:tasks.workunit.client.0.smithi117.stdout:2/403: dwrite d1/d4/f8 [0,4194304] 0 2024-04-20T11:09:01.782 INFO:tasks.workunit.client.0.smithi117.stdout:8/231: dread - d0/d8/f41 zero size 2024-04-20T11:09:01.785 INFO:tasks.workunit.client.0.smithi117.stdout:6/401: rmdir d3/d4/d2e 39 2024-04-20T11:09:01.789 INFO:tasks.workunit.client.0.smithi117.stdout:8/232: truncate d0/d8/d39/d40/f43 484205 0 2024-04-20T11:09:01.789 INFO:tasks.workunit.client.0.smithi117.stdout:9/371: creat d4/d4a/d74/f88 x:0 0 0 2024-04-20T11:09:01.795 INFO:tasks.workunit.client.0.smithi117.stdout:5/414: symlink de/d41/d47/l8a 0 2024-04-20T11:09:01.796 INFO:tasks.workunit.client.0.smithi117.stdout:3/323: dread d0/d5/de/d1d/f44 [0,4194304] 0 2024-04-20T11:09:01.797 INFO:tasks.workunit.client.0.smithi117.stdout:5/415: readlink de/d21/d45/l73 0 2024-04-20T11:09:01.798 INFO:tasks.workunit.client.0.smithi117.stdout:7/326: mkdir d1/d1d/d42/d2d/d6c 0 2024-04-20T11:09:01.800 INFO:tasks.workunit.client.0.smithi117.stdout:0/274: creat d3/d7/d8/f64 x:0 0 0 2024-04-20T11:09:01.803 INFO:tasks.workunit.client.0.smithi117.stdout:2/404: creat d1/d4/dd/de/f7e x:0 0 0 2024-04-20T11:09:01.808 INFO:tasks.workunit.client.0.smithi117.stdout:4/363: dwrite d1/d45/f5f [0,4194304] 0 2024-04-20T11:09:01.808 INFO:tasks.workunit.client.0.smithi117.stdout:6/402: chown d3/d4/d2e/d4e/f70 39 1 2024-04-20T11:09:01.811 INFO:tasks.workunit.client.0.smithi117.stdout:5/416: mknod de/d5b/c8b 0 2024-04-20T11:09:01.812 INFO:tasks.workunit.client.0.smithi117.stdout:7/327: mknod d1/d4/d6/d17/d3f/c6d 0 2024-04-20T11:09:01.812 INFO:tasks.workunit.client.0.smithi117.stdout:0/275: fsync d3/d7/d8/f13 0 2024-04-20T11:09:01.819 INFO:tasks.workunit.client.0.smithi117.stdout:1/341: dwrite d6/f13 [0,4194304] 0 2024-04-20T11:09:01.819 INFO:tasks.workunit.client.0.smithi117.stdout:6/403: chown d3/d4/d2e/d3f/f77 246763669 1 2024-04-20T11:09:01.820 INFO:tasks.workunit.client.0.smithi117.stdout:8/233: fsync d0/f1 0 2024-04-20T11:09:01.821 INFO:tasks.workunit.client.0.smithi117.stdout:6/404: chown d3/d3a 131798264 1 2024-04-20T11:09:01.825 INFO:tasks.workunit.client.0.smithi117.stdout:2/405: stat d1/d4/dd/f19 0 2024-04-20T11:09:01.826 INFO:tasks.workunit.client.0.smithi117.stdout:1/342: write d6/d1f/d30/f4e [3842689,49548] 0 2024-04-20T11:09:01.830 INFO:tasks.workunit.client.0.smithi117.stdout:5/417: write de/f46 [303682,53868] 0 2024-04-20T11:09:01.830 INFO:tasks.workunit.client.0.smithi117.stdout:4/364: readlink d1/dd/l7b 0 2024-04-20T11:09:01.833 INFO:tasks.workunit.client.0.smithi117.stdout:3/324: creat d0/f71 x:0 0 0 2024-04-20T11:09:01.839 INFO:tasks.workunit.client.0.smithi117.stdout:7/328: truncate d1/d4/d6/d13/f15 88729 0 2024-04-20T11:09:01.845 INFO:tasks.workunit.client.0.smithi117.stdout:3/325: symlink d0/d5/d11/l72 0 2024-04-20T11:09:01.846 INFO:tasks.workunit.client.0.smithi117.stdout:5/418: chown de/l1a 84260275 1 2024-04-20T11:09:01.848 INFO:tasks.workunit.client.0.smithi117.stdout:0/276: dwrite d3/d7/d8/df/f46 [0,4194304] 0 2024-04-20T11:09:01.848 INFO:tasks.workunit.client.0.smithi117.stdout:9/372: truncate d4/d5/da/f19 3467350 0 2024-04-20T11:09:01.855 INFO:tasks.workunit.client.0.smithi117.stdout:2/406: creat d1/d15/d1d/d6b/f7f x:0 0 0 2024-04-20T11:09:01.863 INFO:tasks.workunit.client.0.smithi117.stdout:9/373: unlink d4/f45 0 2024-04-20T11:09:01.866 INFO:tasks.workunit.client.0.smithi117.stdout:2/407: read - d1/f7b zero size 2024-04-20T11:09:01.866 INFO:tasks.workunit.client.0.smithi117.stdout:6/405: getdents d3/d3a 0 2024-04-20T11:09:01.877 INFO:tasks.workunit.client.0.smithi117.stdout:6/406: read - d3/d5a/d79/f6a zero size 2024-04-20T11:09:01.877 INFO:tasks.workunit.client.0.smithi117.stdout:4/365: dwrite d1/f19 [4194304,4194304] 0 2024-04-20T11:09:01.881 INFO:tasks.workunit.client.0.smithi117.stdout:1/343: dread d6/d35/f36 [0,4194304] 0 2024-04-20T11:09:01.884 INFO:tasks.workunit.client.0.smithi117.stdout:9/374: creat d4/d4a/d72/f89 x:0 0 0 2024-04-20T11:09:01.888 INFO:tasks.workunit.client.0.smithi117.stdout:8/234: dwrite d0/d8/d39/d40/f43 [0,4194304] 0 2024-04-20T11:09:01.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:01 smithi117 ceph-mon[15659]: pgmap v10: 65 pgs: 65 active+clean; 502 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 52 MiB/s wr, 178 op/s 2024-04-20T11:09:01.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:01 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:09:01.893 INFO:tasks.workunit.client.0.smithi117.stdout:1/344: write d6/d1f/d30/f4e [3899617,78001] 0 2024-04-20T11:09:01.896 INFO:tasks.workunit.client.0.smithi117.stdout:6/407: creat d3/d4/d2e/d3f/f82 x:0 0 0 2024-04-20T11:09:01.897 INFO:tasks.workunit.client.0.smithi117.stdout:7/329: dwrite d1/d43/d59/f69 [0,4194304] 0 2024-04-20T11:09:01.899 INFO:tasks.workunit.client.0.smithi117.stdout:6/408: stat d3/df/f31 0 2024-04-20T11:09:01.900 INFO:tasks.workunit.client.0.smithi117.stdout:2/408: mkdir d1/d4/dd/de/d29/d2e/d80 0 2024-04-20T11:09:01.902 INFO:tasks.workunit.client.0.smithi117.stdout:0/277: getdents d3/d7/d8/d2a 0 2024-04-20T11:09:01.905 INFO:tasks.workunit.client.0.smithi117.stdout:1/345: truncate d6/d16/d3a/d5a/f5d 242202 0 2024-04-20T11:09:01.908 INFO:tasks.workunit.client.0.smithi117.stdout:1/346: chown d6/d1f/d30/c38 65439 1 2024-04-20T11:09:01.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:01 smithi173 ceph-mon[23800]: pgmap v10: 65 pgs: 65 active+clean; 502 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 52 MiB/s wr, 178 op/s 2024-04-20T11:09:01.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:01 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:09:01.913 INFO:tasks.workunit.client.0.smithi117.stdout:1/347: dread - d6/d1f/d30/f67 zero size 2024-04-20T11:09:01.923 INFO:tasks.workunit.client.0.smithi117.stdout:3/326: dwrite d0/d5/d21/f41 [0,4194304] 0 2024-04-20T11:09:01.924 INFO:tasks.workunit.client.0.smithi117.stdout:5/419: dread de/f34 [0,4194304] 0 2024-04-20T11:09:01.924 INFO:tasks.workunit.client.0.smithi117.stdout:0/278: creat d3/d7/d2c/f65 x:0 0 0 2024-04-20T11:09:01.929 INFO:tasks.workunit.client.0.smithi117.stdout:1/348: write d6/d35/f5f [80313,7019] 0 2024-04-20T11:09:01.929 INFO:tasks.workunit.client.0.smithi117.stdout:2/409: chown d1/d4/dd/d17/d7a/c44 1016 1 2024-04-20T11:09:01.929 INFO:tasks.workunit.client.0.smithi117.stdout:9/375: dread d4/d30/d31/f6c [0,4194304] 0 2024-04-20T11:09:01.930 INFO:tasks.workunit.client.0.smithi117.stdout:8/235: creat d0/dc/d2e/d3e/f48 x:0 0 0 2024-04-20T11:09:01.934 INFO:tasks.workunit.client.0.smithi117.stdout:4/366: dwrite d1/d45/d4a/f50 [0,4194304] 0 2024-04-20T11:09:01.934 INFO:tasks.workunit.client.0.smithi117.stdout:8/236: stat d0/f23 0 2024-04-20T11:09:01.940 INFO:tasks.workunit.client.0.smithi117.stdout:4/367: fsync d1/d9/f3b 0 2024-04-20T11:09:01.942 INFO:tasks.workunit.client.0.smithi117.stdout:6/409: getdents d3 0 2024-04-20T11:09:01.944 INFO:tasks.workunit.client.0.smithi117.stdout:9/376: stat d4/d5/d3c/c65 0 2024-04-20T11:09:01.948 INFO:tasks.workunit.client.0.smithi117.stdout:6/410: stat d3/d4/l66 0 2024-04-20T11:09:01.951 INFO:tasks.workunit.client.0.smithi117.stdout:2/410: symlink d1/d4/dd/d17/d3b/l81 0 2024-04-20T11:09:01.951 INFO:tasks.workunit.client.0.smithi117.stdout:9/377: chown d4/d4a/d63 3865 1 2024-04-20T11:09:01.953 INFO:tasks.workunit.client.0.smithi117.stdout:8/237: mkdir d0/dc/d49 0 2024-04-20T11:09:01.955 INFO:tasks.workunit.client.0.smithi117.stdout:5/420: creat de/d21/d45/d48/d4e/f8c x:0 0 0 2024-04-20T11:09:01.960 INFO:tasks.workunit.client.0.smithi117.stdout:2/411: symlink d1/d4/dd/de/l82 0 2024-04-20T11:09:01.961 INFO:tasks.workunit.client.0.smithi117.stdout:9/378: fsync d4/d68/d6e/f77 0 2024-04-20T11:09:01.961 INFO:tasks.workunit.client.0.smithi117.stdout:6/411: symlink d3/d3a/d7f/l83 0 2024-04-20T11:09:01.967 INFO:tasks.workunit.client.0.smithi117.stdout:4/368: truncate d1/dd/f4b 4243525 0 2024-04-20T11:09:01.967 INFO:tasks.workunit.client.0.smithi117.stdout:5/421: rename de/f46 to de/d21/d35/f8d 0 2024-04-20T11:09:01.967 INFO:tasks.workunit.client.0.smithi117.stdout:7/330: dwrite d1/d1d/f21 [4194304,4194304] 0 2024-04-20T11:09:01.974 INFO:tasks.workunit.client.0.smithi117.stdout:6/412: write d3/d4/d2e/d3f/f82 [818752,91376] 0 2024-04-20T11:09:01.974 INFO:tasks.workunit.client.0.smithi117.stdout:0/279: dwrite d3/d7/d8/df/f46 [0,4194304] 0 2024-04-20T11:09:01.984 INFO:tasks.workunit.client.0.smithi117.stdout:7/331: write d1/d1d/f5c [837929,108423] 0 2024-04-20T11:09:01.988 INFO:tasks.workunit.client.0.smithi117.stdout:8/238: link d0/l21 d0/d8/d11/d24/d27/l4a 0 2024-04-20T11:09:01.988 INFO:tasks.workunit.client.0.smithi117.stdout:6/413: unlink d3/d4/d2e/d4e/f70 0 2024-04-20T11:09:01.989 INFO:tasks.workunit.client.0.smithi117.stdout:0/280: read d3/d5/f59 [3573694,13326] 0 2024-04-20T11:09:01.990 INFO:tasks.workunit.client.0.smithi117.stdout:1/349: link d6/c28 d6/d16/d3a/d5a/d5c/d61/d71/d6b/c72 0 2024-04-20T11:09:01.991 INFO:tasks.workunit.client.0.smithi117.stdout:7/332: fdatasync d1/d4/d6/f26 0 2024-04-20T11:09:01.997 INFO:tasks.workunit.client.0.smithi117.stdout:7/333: dread - d1/d43/f61 zero size 2024-04-20T11:09:02.004 INFO:tasks.workunit.client.0.smithi117.stdout:5/422: dwrite de/d41/d47/f6f [0,4194304] 0 2024-04-20T11:09:02.005 INFO:tasks.workunit.client.0.smithi117.stdout:3/327: dwrite d0/d5/d21/f42 [0,4194304] 0 2024-04-20T11:09:02.011 INFO:tasks.workunit.client.0.smithi117.stdout:6/414: truncate d3/d4/d2e/d3f/f82 1289114 0 2024-04-20T11:09:02.012 INFO:tasks.workunit.client.0.smithi117.stdout:8/239: write d0/d8/d11/f1f [597175,33848] 0 2024-04-20T11:09:02.015 INFO:tasks.workunit.client.0.smithi117.stdout:2/412: dwrite d1/f7b [0,4194304] 0 2024-04-20T11:09:02.016 INFO:tasks.workunit.client.0.smithi117.stdout:4/369: rename d1/d45/f74 to d1/dd/d1a/f7c 0 2024-04-20T11:09:02.021 INFO:tasks.workunit.client.0.smithi117.stdout:0/281: mkdir d3/d7/d20/d39/d3a/d66 0 2024-04-20T11:09:02.021 INFO:tasks.workunit.client.0.smithi117.stdout:5/423: truncate de/d21/d45/d48/f58 276301 0 2024-04-20T11:09:02.022 INFO:tasks.workunit.client.0.smithi117.stdout:8/240: write d0/d8/f38 [27460,113119] 0 2024-04-20T11:09:02.022 INFO:tasks.workunit.client.0.smithi117.stdout:6/415: rmdir d3/d3a/d7f 39 2024-04-20T11:09:02.026 INFO:tasks.workunit.client.0.smithi117.stdout:1/350: mkdir d6/d1f/d30/d49/d69/d6e/d6f/d73 0 2024-04-20T11:09:02.026 INFO:tasks.workunit.client.0.smithi117.stdout:7/334: creat d1/d4/d6/d54/d63/f6e x:0 0 0 2024-04-20T11:09:02.033 INFO:tasks.workunit.client.0.smithi117.stdout:3/328: write d0/d5/de/d1d/f44 [4390876,14439] 0 2024-04-20T11:09:02.035 INFO:tasks.workunit.client.0.smithi117.stdout:4/370: mknod d1/d9/d79/c7d 0 2024-04-20T11:09:02.035 INFO:tasks.workunit.client.0.smithi117.stdout:2/413: chown d1/d4/dd/de/l33 1033 1 2024-04-20T11:09:02.036 INFO:tasks.workunit.client.0.smithi117.stdout:0/282: creat d3/d7/d8/d2a/d54/f67 x:0 0 0 2024-04-20T11:09:02.036 INFO:tasks.workunit.client.0.smithi117.stdout:5/424: unlink de/l17 0 2024-04-20T11:09:02.039 INFO:tasks.workunit.client.0.smithi117.stdout:4/371: chown d1/d45/d4a/d59 71 1 2024-04-20T11:09:02.041 INFO:tasks.workunit.client.0.smithi117.stdout:9/379: dwrite d4/d5/f28 [0,4194304] 0 2024-04-20T11:09:02.042 INFO:tasks.workunit.client.0.smithi117.stdout:6/416: dread d3/d4/d2e/d55/f60 [0,4194304] 0 2024-04-20T11:09:02.047 INFO:tasks.workunit.client.0.smithi117.stdout:5/425: write de/f34 [3176784,111365] 0 2024-04-20T11:09:02.047 INFO:tasks.workunit.client.0.smithi117.stdout:7/335: mknod d1/d1d/d42/d2d/d6c/c6f 0 2024-04-20T11:09:02.049 INFO:tasks.workunit.client.0.smithi117.stdout:6/417: fsync d3/d4/f1b 0 2024-04-20T11:09:02.051 INFO:tasks.workunit.client.0.smithi117.stdout:7/336: write d1/d4/d39/f4d [19417,3750] 0 2024-04-20T11:09:02.051 INFO:tasks.workunit.client.0.smithi117.stdout:0/283: rmdir d3/d7/d8/df 39 2024-04-20T11:09:02.058 INFO:tasks.workunit.client.0.smithi117.stdout:8/241: dwrite d0/d8/d11/f1c [0,4194304] 0 2024-04-20T11:09:02.060 INFO:tasks.workunit.client.0.smithi117.stdout:3/329: mkdir d0/d5/d73 0 2024-04-20T11:09:02.061 INFO:tasks.workunit.client.0.smithi117.stdout:1/351: creat d6/d1f/d30/f74 x:0 0 0 2024-04-20T11:09:02.063 INFO:tasks.workunit.client.0.smithi117.stdout:4/372: symlink d1/d9/d6b/l7e 0 2024-04-20T11:09:02.065 INFO:tasks.workunit.client.0.smithi117.stdout:9/380: read d4/d5/f21 [437830,84171] 0 2024-04-20T11:09:02.068 INFO:tasks.workunit.client.0.smithi117.stdout:1/352: dread - d6/d16/d3a/d5a/d5c/d61/d71/d6b/f6c zero size 2024-04-20T11:09:02.069 INFO:tasks.workunit.client.0.smithi117.stdout:4/373: read d1/d9/f2a [2097780,83493] 0 2024-04-20T11:09:02.071 INFO:tasks.workunit.client.0.smithi117.stdout:7/337: creat d1/d1d/f70 x:0 0 0 2024-04-20T11:09:02.080 INFO:tasks.workunit.client.0.smithi117.stdout:1/353: rmdir d6/d16/d3a/d5a/d5c/d61/d71 39 2024-04-20T11:09:02.080 INFO:tasks.workunit.client.0.smithi117.stdout:4/374: creat d1/d45/d4a/d59/f7f x:0 0 0 2024-04-20T11:09:02.083 INFO:tasks.workunit.client.0.smithi117.stdout:2/414: rename d1/d4/dd/de to d1/d4/d83 0 2024-04-20T11:09:02.084 INFO:tasks.workunit.client.0.smithi117.stdout:1/354: write d6/f13 [3714532,120280] 0 2024-04-20T11:09:02.089 INFO:tasks.workunit.client.0.smithi117.stdout:7/338: dread d1/d1d/f5c [0,4194304] 0 2024-04-20T11:09:02.096 INFO:tasks.workunit.client.0.smithi117.stdout:3/330: dread d0/d5/d7/dd/f59 [0,4194304] 0 2024-04-20T11:09:02.103 INFO:tasks.workunit.client.0.smithi117.stdout:5/426: getdents de/d41/d47/d4c 0 2024-04-20T11:09:02.103 INFO:tasks.workunit.client.0.smithi117.stdout:0/284: write d3/d7/f19 [1605692,94811] 0 2024-04-20T11:09:02.105 INFO:tasks.workunit.client.0.smithi117.stdout:8/242: dwrite d0/f20 [0,4194304] 0 2024-04-20T11:09:02.105 INFO:tasks.workunit.client.0.smithi117.stdout:6/418: link d3/d4/d2e/l45 d3/d3a/d7f/l84 0 2024-04-20T11:09:02.110 INFO:tasks.workunit.client.0.smithi117.stdout:4/375: dread d1/f25 [4194304,4194304] 0 2024-04-20T11:09:02.117 INFO:tasks.workunit.client.0.smithi117.stdout:2/415: link d1/d15/c35 d1/d4/dd/d17/d3b/c84 0 2024-04-20T11:09:02.119 INFO:tasks.workunit.client.0.smithi117.stdout:3/331: rename d0/d5/de/d1d/d29/f31 to d0/d5/d11/f74 0 2024-04-20T11:09:02.121 INFO:tasks.workunit.client.0.smithi117.stdout:6/419: mkdir d3/d4/d2e/d85 0 2024-04-20T11:09:02.121 INFO:tasks.workunit.client.0.smithi117.stdout:5/427: creat de/f8e x:0 0 0 2024-04-20T11:09:02.122 INFO:tasks.workunit.client.0.smithi117.stdout:0/285: write d3/d7/d20/f25 [4716097,79181] 0 2024-04-20T11:09:02.123 INFO:tasks.workunit.client.0.smithi117.stdout:4/376: mknod d1/d45/c80 0 2024-04-20T11:09:02.124 INFO:tasks.workunit.client.0.smithi117.stdout:2/416: readlink d1/d4/dd/d17/l4f 0 2024-04-20T11:09:02.127 INFO:tasks.workunit.client.0.smithi117.stdout:3/332: mkdir d0/d5/d21/d75 0 2024-04-20T11:09:02.132 INFO:tasks.workunit.client.0.smithi117.stdout:8/243: rename d0/d8/c30 to d0/d8/d11/d24/d27/d47/c4b 0 2024-04-20T11:09:02.133 INFO:tasks.workunit.client.0.smithi117.stdout:6/420: write d3/d4/f19 [2920533,75498] 0 2024-04-20T11:09:02.134 INFO:tasks.workunit.client.0.smithi117.stdout:5/428: creat de/d41/d47/d4c/f8f x:0 0 0 2024-04-20T11:09:02.135 INFO:tasks.workunit.client.0.smithi117.stdout:1/355: dwrite d6/d1f/f2b [0,4194304] 0 2024-04-20T11:09:02.137 INFO:tasks.workunit.client.0.smithi117.stdout:4/377: creat d1/f81 x:0 0 0 2024-04-20T11:09:02.138 INFO:tasks.workunit.client.0.smithi117.stdout:7/339: dwrite d1/d4/d39/f4d [4194304,4194304] 0 2024-04-20T11:09:02.142 INFO:tasks.workunit.client.0.smithi117.stdout:9/381: dwrite d4/d5/da/f19 [4194304,4194304] 0 2024-04-20T11:09:02.142 INFO:tasks.workunit.client.0.smithi117.stdout:8/244: write d0/dc/f25 [693377,55060] 0 2024-04-20T11:09:02.142 INFO:tasks.workunit.client.0.smithi117.stdout:2/417: dread - d1/d4/d83/d29/d2e/f74 zero size 2024-04-20T11:09:02.146 INFO:tasks.workunit.client.0.smithi117.stdout:3/333: dread - d0/d5/de/d1d/f70 zero size 2024-04-20T11:09:02.146 INFO:tasks.workunit.client.0.smithi117.stdout:6/421: mknod d3/d4/d2e/d3f/c86 0 2024-04-20T11:09:02.148 INFO:tasks.workunit.client.0.smithi117.stdout:9/382: stat d4/d30/d31/c42 0 2024-04-20T11:09:02.148 INFO:tasks.workunit.client.0.smithi117.stdout:7/340: write d1/d1d/f5c [1824509,126062] 0 2024-04-20T11:09:02.149 INFO:tasks.workunit.client.0.smithi117.stdout:0/286: rename d3/d7/d8/d2a/d54/f67 to d3/d7/d20/f68 0 2024-04-20T11:09:02.155 INFO:tasks.workunit.client.0.smithi117.stdout:0/287: stat d3/d5/c1f 0 2024-04-20T11:09:02.157 INFO:tasks.workunit.client.0.smithi117.stdout:9/383: truncate d4/d35/f3d 851374 0 2024-04-20T11:09:02.157 INFO:tasks.workunit.client.0.smithi117.stdout:3/334: symlink d0/d5/d21/l76 0 2024-04-20T11:09:02.159 INFO:tasks.workunit.client.0.smithi117.stdout:4/378: mknod d1/dd/c82 0 2024-04-20T11:09:02.167 INFO:tasks.workunit.client.0.smithi117.stdout:8/245: symlink d0/d8/d11/d24/d27/d47/l4c 0 2024-04-20T11:09:02.169 INFO:tasks.workunit.client.0.smithi117.stdout:3/335: mkdir d0/d5/d21/d77 0 2024-04-20T11:09:02.175 INFO:tasks.workunit.client.0.smithi117.stdout:0/288: dread d3/d7/d16/f2b [0,4194304] 0 2024-04-20T11:09:02.178 INFO:tasks.workunit.client.0.smithi117.stdout:1/356: truncate d6/f9 4330003 0 2024-04-20T11:09:02.181 INFO:tasks.workunit.client.0.smithi117.stdout:9/384: mknod d4/d5/d57/d82/c8a 0 2024-04-20T11:09:02.184 INFO:tasks.workunit.client.0.smithi117.stdout:8/246: fsync d0/dc/f1a 0 2024-04-20T11:09:02.186 INFO:tasks.workunit.client.0.smithi117.stdout:5/429: dwrite de/d16/f66 [0,4194304] 0 2024-04-20T11:09:02.188 INFO:tasks.workunit.client.0.smithi117.stdout:6/422: dwrite d3/f28 [4194304,4194304] 0 2024-04-20T11:09:02.192 INFO:tasks.workunit.client.0.smithi117.stdout:3/336: mknod d0/d5/d7/dd/c78 0 2024-04-20T11:09:02.192 INFO:tasks.workunit.client.0.smithi117.stdout:4/379: dread d1/dd/f53 [0,4194304] 0 2024-04-20T11:09:02.196 INFO:tasks.workunit.client.0.smithi117.stdout:2/418: dwrite d1/d4/d83/f18 [0,4194304] 0 2024-04-20T11:09:02.197 INFO:tasks.workunit.client.0.smithi117.stdout:0/289: creat d3/d7/d2c/f69 x:0 0 0 2024-04-20T11:09:02.198 INFO:tasks.workunit.client.0.smithi117.stdout:3/337: chown d0/fb 54270 1 2024-04-20T11:09:02.199 INFO:tasks.workunit.client.0.smithi117.stdout:6/423: dread - d3/df/d39/f4b zero size 2024-04-20T11:09:02.203 INFO:tasks.workunit.client.0.smithi117.stdout:0/290: read d3/d7/d20/f4c [455601,75961] 0 2024-04-20T11:09:02.204 INFO:tasks.workunit.client.0.smithi117.stdout:8/247: symlink d0/dc/l4d 0 2024-04-20T11:09:02.208 INFO:tasks.workunit.client.0.smithi117.stdout:4/380: symlink d1/dd/d52/l83 0 2024-04-20T11:09:02.208 INFO:tasks.workunit.client.0.smithi117.stdout:7/341: dread d1/d4/d6/d2b/f33 [0,4194304] 0 2024-04-20T11:09:02.208 INFO:tasks.workunit.client.0.smithi117.stdout:4/381: chown d1/l3 0 1 2024-04-20T11:09:02.209 INFO:tasks.workunit.client.0.smithi117.stdout:1/357: creat d6/d1f/d30/d49/d69/d6e/d6f/d73/f75 x:0 0 0 2024-04-20T11:09:02.212 INFO:tasks.workunit.client.0.smithi117.stdout:6/424: creat d3/d4/d2e/d85/f87 x:0 0 0 2024-04-20T11:09:02.212 INFO:tasks.workunit.client.0.smithi117.stdout:0/291: fsync d3/d7/d8/df/f46 0 2024-04-20T11:09:02.214 INFO:tasks.workunit.client.0.smithi117.stdout:5/430: link de/f14 de/d16/f90 0 2024-04-20T11:09:02.219 INFO:tasks.workunit.client.0.smithi117.stdout:8/248: rename d0/d8/d39/c3d to d0/d8/d11/d24/d27/d47/c4e 0 2024-04-20T11:09:02.219 INFO:tasks.workunit.client.0.smithi117.stdout:3/338: mknod d0/d5/d7/c79 0 2024-04-20T11:09:02.224 INFO:tasks.workunit.client.0.smithi117.stdout:6/425: dread d3/d4/f2f [0,4194304] 0 2024-04-20T11:09:02.228 INFO:tasks.workunit.client.0.smithi117.stdout:4/382: link d1/d45/c58 d1/d9/c84 0 2024-04-20T11:09:02.233 INFO:tasks.workunit.client.0.smithi117.stdout:3/339: creat d0/d5/de/d1d/d29/d66/d63/f7a x:0 0 0 2024-04-20T11:09:02.233 INFO:tasks.workunit.client.0.smithi117.stdout:4/383: dread - d1/d45/d4a/d59/f6a zero size 2024-04-20T11:09:02.233 INFO:tasks.workunit.client.0.smithi117.stdout:2/419: dwrite d1/d4/d83/f3a [4194304,4194304] 0 2024-04-20T11:09:02.234 INFO:tasks.workunit.client.0.smithi117.stdout:7/342: link d1/d4/d3d/f5a d1/d4/d6/d13/f71 0 2024-04-20T11:09:02.234 INFO:tasks.workunit.client.0.smithi117.stdout:8/249: creat d0/d8/d11/d24/d27/d47/f4f x:0 0 0 2024-04-20T11:09:02.236 INFO:tasks.workunit.client.0.smithi117.stdout:6/426: mknod d3/d4/d2e/d5f/c88 0 2024-04-20T11:09:02.237 INFO:tasks.workunit.client.0.smithi117.stdout:7/343: chown d1/d1d/d4e/d53 27 1 2024-04-20T11:09:02.238 INFO:tasks.workunit.client.0.smithi117.stdout:2/420: readlink d1/d4/dd/d17/d3b/l81 0 2024-04-20T11:09:02.239 INFO:tasks.workunit.client.0.smithi117.stdout:6/427: write d3/f73 [889451,71658] 0 2024-04-20T11:09:02.240 INFO:tasks.workunit.client.0.smithi117.stdout:8/250: symlink d0/l50 0 2024-04-20T11:09:02.240 INFO:tasks.workunit.client.0.smithi117.stdout:1/358: dwrite d6/d35/f5f [0,4194304] 0 2024-04-20T11:09:02.254 INFO:tasks.workunit.client.0.smithi117.stdout:2/421: chown d1/d4/d83/d29/d2e/d53 2 1 2024-04-20T11:09:02.256 INFO:tasks.workunit.client.0.smithi117.stdout:6/428: creat d3/f89 x:0 0 0 2024-04-20T11:09:02.257 INFO:tasks.workunit.client.0.smithi117.stdout:5/431: truncate de/d21/d35/f3a 2035822 0 2024-04-20T11:09:02.257 INFO:tasks.workunit.client.0.smithi117.stdout:9/385: dread d4/de/f84 [0,4194304] 0 2024-04-20T11:09:02.260 INFO:tasks.workunit.client.0.smithi117.stdout:2/422: mknod d1/d4/dd/d17/d7a/d49/c85 0 2024-04-20T11:09:02.263 INFO:tasks.workunit.client.0.smithi117.stdout:7/344: rename d1/d4/d6/d17/d67 to d1/d1d/d4e/d72 0 2024-04-20T11:09:02.265 INFO:tasks.workunit.client.0.smithi117.stdout:4/384: write d1/dd/f53 [984218,41456] 0 2024-04-20T11:09:02.268 INFO:tasks.workunit.client.0.smithi117.stdout:4/385: write d1/f72 [194079,85187] 0 2024-04-20T11:09:02.268 INFO:tasks.workunit.client.0.smithi117.stdout:5/432: dread - de/f68 zero size 2024-04-20T11:09:02.273 INFO:tasks.workunit.client.0.smithi117.stdout:7/345: stat d1/c38 0 2024-04-20T11:09:02.274 INFO:tasks.workunit.client.0.smithi117.stdout:2/423: symlink d1/d4/d83/d29/d2e/d52/l86 0 2024-04-20T11:09:02.275 INFO:tasks.workunit.client.0.smithi117.stdout:1/359: getdents d6/d1f/d30/d49/d69/d6e/d6f 0 2024-04-20T11:09:02.276 INFO:tasks.workunit.client.0.smithi117.stdout:4/386: mknod d1/d9/d6b/c85 0 2024-04-20T11:09:02.276 INFO:tasks.workunit.client.0.smithi117.stdout:0/292: dread d3/d7/d8/df/f1b [0,4194304] 0 2024-04-20T11:09:02.277 INFO:tasks.workunit.client.0.smithi117.stdout:5/433: symlink de/d41/d47/d55/d56/l91 0 2024-04-20T11:09:02.280 INFO:tasks.workunit.client.0.smithi117.stdout:4/387: stat d1/f35 0 2024-04-20T11:09:02.283 INFO:tasks.workunit.client.0.smithi117.stdout:5/434: write de/d41/d47/d4c/f89 [482262,36586] 0 2024-04-20T11:09:02.284 INFO:tasks.workunit.client.0.smithi117.stdout:5/435: chown de/d21/d35 16877 1 2024-04-20T11:09:02.286 INFO:tasks.workunit.client.0.smithi117.stdout:0/293: rmdir d3/d7/d2c/d52 39 2024-04-20T11:09:02.287 INFO:tasks.workunit.client.0.smithi117.stdout:5/436: fdatasync de/d41/d47/d55/d56/f79 0 2024-04-20T11:09:02.289 INFO:tasks.workunit.client.0.smithi117.stdout:0/294: readlink d3/d7/d16/l45 0 2024-04-20T11:09:02.294 INFO:tasks.workunit.client.0.smithi117.stdout:0/295: stat d3/d7/d2c/c43 0 2024-04-20T11:09:02.297 INFO:tasks.workunit.client.0.smithi117.stdout:2/424: link d1/d15/d1d/d6b/f7f d1/d4/dd/d17/d7a/d49/f87 0 2024-04-20T11:09:02.298 INFO:tasks.workunit.client.0.smithi117.stdout:9/386: dread d4/d30/f36 [0,4194304] 0 2024-04-20T11:09:02.304 INFO:tasks.workunit.client.0.smithi117.stdout:8/251: dwrite d0/f7 [0,4194304] 0 2024-04-20T11:09:02.310 INFO:tasks.workunit.client.0.smithi117.stdout:7/346: dread d1/d4/d6/d13/f28 [0,4194304] 0 2024-04-20T11:09:02.310 INFO:tasks.workunit.client.0.smithi117.stdout:9/387: write d4/d5/d3c/f46 [1257555,57379] 0 2024-04-20T11:09:02.313 INFO:tasks.workunit.client.0.smithi117.stdout:8/252: creat d0/d8/d39/d40/f51 x:0 0 0 2024-04-20T11:09:02.313 INFO:tasks.workunit.client.0.smithi117.stdout:9/388: chown d4/de/d10/f24 12200172 1 2024-04-20T11:09:02.314 INFO:tasks.workunit.client.0.smithi117.stdout:1/360: link d6/c28 d6/d16/d3a/d5a/d5c/d61/d71/d6b/c76 0 2024-04-20T11:09:02.314 INFO:tasks.workunit.client.0.smithi117.stdout:0/296: link d3/d5/f37 d3/d5/f6a 0 2024-04-20T11:09:02.315 INFO:tasks.workunit.client.0.smithi117.stdout:7/347: creat d1/d43/f73 x:0 0 0 2024-04-20T11:09:02.318 INFO:tasks.workunit.client.0.smithi117.stdout:8/253: read d0/dc/f1a [11310,7050] 0 2024-04-20T11:09:02.318 INFO:tasks.workunit.client.0.smithi117.stdout:7/348: readlink d1/d1d/d42/d2d/l50 0 2024-04-20T11:09:02.320 INFO:tasks.workunit.client.0.smithi117.stdout:2/425: rename d1/d15/d1d/d6b to d1/d4/dd/d17/d7a/d88 0 2024-04-20T11:09:02.323 INFO:tasks.workunit.client.0.smithi117.stdout:1/361: unlink d6/ca 0 2024-04-20T11:09:02.326 INFO:tasks.workunit.client.0.smithi117.stdout:9/389: creat d4/d35/d54/f8b x:0 0 0 2024-04-20T11:09:02.331 INFO:tasks.workunit.client.0.smithi117.stdout:5/437: dread de/d41/d47/f5c [0,4194304] 0 2024-04-20T11:09:02.335 INFO:tasks.workunit.client.0.smithi117.stdout:0/297: rename d3/d5/f60 to d3/d7/d2c/f6b 0 2024-04-20T11:09:02.340 INFO:tasks.workunit.client.0.smithi117.stdout:5/438: symlink de/d5b/d65/l92 0 2024-04-20T11:09:02.342 INFO:tasks.workunit.client.0.smithi117.stdout:0/298: truncate d3/d7/d16/f56 127782 0 2024-04-20T11:09:02.342 INFO:tasks.workunit.client.0.smithi117.stdout:8/254: dwrite d0/d8/d11/d24/d27/d47/f4f [0,4194304] 0 2024-04-20T11:09:02.343 INFO:tasks.workunit.client.0.smithi117.stdout:1/362: dwrite d6/d1f/f2b [0,4194304] 0 2024-04-20T11:09:02.345 INFO:tasks.workunit.client.0.smithi117.stdout:2/426: dread d1/d4/d83/f34 [0,4194304] 0 2024-04-20T11:09:02.345 INFO:tasks.workunit.client.0.smithi117.stdout:9/390: rename d4/de/d10/l1d to d4/d4a/d5e/l8c 0 2024-04-20T11:09:02.354 INFO:tasks.workunit.client.0.smithi117.stdout:8/255: symlink d0/d8/d39/l52 0 2024-04-20T11:09:02.359 INFO:tasks.workunit.client.0.smithi117.stdout:5/439: write de/d21/d45/d48/f58 [611848,51196] 0 2024-04-20T11:09:02.360 INFO:tasks.workunit.client.0.smithi117.stdout:2/427: chown d1/d4/dd/d17/d3b/l81 9 1 2024-04-20T11:09:02.364 INFO:tasks.workunit.client.0.smithi117.stdout:8/256: mkdir d0/d8/d11/d24/d53 0 2024-04-20T11:09:02.366 INFO:tasks.workunit.client.0.smithi117.stdout:7/349: dwrite d1/d4/d6/d13/f71 [0,4194304] 0 2024-04-20T11:09:02.368 INFO:tasks.workunit.client.0.smithi117.stdout:5/440: dread - de/d21/d45/d48/d4e/f8c zero size 2024-04-20T11:09:02.371 INFO:tasks.workunit.client.0.smithi117.stdout:2/428: symlink d1/d4/d83/d29/d2e/l89 0 2024-04-20T11:09:02.378 INFO:tasks.workunit.client.0.smithi117.stdout:9/391: dwrite d4/d5/da/f19 [0,4194304] 0 2024-04-20T11:09:02.379 INFO:tasks.workunit.client.0.smithi117.stdout:7/350: truncate d1/d4/d6/d17/f4c 1179551 0 2024-04-20T11:09:02.385 INFO:tasks.workunit.client.0.smithi117.stdout:0/299: dwrite d3/f3e [0,4194304] 0 2024-04-20T11:09:02.385 INFO:tasks.workunit.client.0.smithi117.stdout:5/441: rename de/d21/d35/l60 to de/d41/d47/d55/l93 0 2024-04-20T11:09:02.390 INFO:tasks.workunit.client.0.smithi117.stdout:7/351: symlink d1/d4/d6/d13/l74 0 2024-04-20T11:09:02.390 INFO:tasks.workunit.client.0.smithi117.stdout:0/300: chown d3/f38 1546228 1 2024-04-20T11:09:02.392 INFO:tasks.workunit.client.0.smithi117.stdout:0/301: stat d3/d7/d2c/f65 0 2024-04-20T11:09:02.398 INFO:tasks.workunit.client.0.smithi117.stdout:9/392: mknod d4/c8d 0 2024-04-20T11:09:02.398 INFO:tasks.workunit.client.0.smithi117.stdout:8/257: write d0/d9/f33 [203915,48426] 0 2024-04-20T11:09:02.402 INFO:tasks.workunit.client.0.smithi117.stdout:5/442: symlink de/l94 0 2024-04-20T11:09:02.406 INFO:tasks.workunit.client.0.smithi117.stdout:9/393: creat d4/d4a/d72/f8e x:0 0 0 2024-04-20T11:09:02.410 INFO:tasks.workunit.client.0.smithi117.stdout:5/443: rmdir de/d21/d51 39 2024-04-20T11:09:02.413 INFO:tasks.workunit.client.0.smithi117.stdout:9/394: symlink d4/d5/d3c/d7a/l8f 0 2024-04-20T11:09:02.413 INFO:tasks.workunit.client.0.smithi117.stdout:8/258: readlink d0/d8/d11/d24/d27/l4a 0 2024-04-20T11:09:02.418 INFO:tasks.workunit.client.0.smithi117.stdout:5/444: chown de/d21/d35/f3e 0 1 2024-04-20T11:09:02.418 INFO:tasks.workunit.client.0.smithi117.stdout:7/352: dwrite d1/d43/f61 [0,4194304] 0 2024-04-20T11:09:02.418 INFO:tasks.workunit.client.0.smithi117.stdout:9/395: write d4/d5/d3c/f46 [1041325,17585] 0 2024-04-20T11:09:02.425 INFO:tasks.workunit.client.0.smithi117.stdout:2/429: dread d1/d4/f7 [0,4194304] 0 2024-04-20T11:09:02.425 INFO:tasks.workunit.client.0.smithi117.stdout:9/396: stat d4/de/l71 0 2024-04-20T11:09:02.426 INFO:tasks.workunit.client.0.smithi117.stdout:0/302: dread d3/d7/d16/f2e [0,4194304] 0 2024-04-20T11:09:02.427 INFO:tasks.workunit.client.0.smithi117.stdout:0/303: read - d3/f38 zero size 2024-04-20T11:09:02.432 INFO:tasks.workunit.client.0.smithi117.stdout:2/430: creat d1/d4/d83/d29/d62/f8a x:0 0 0 2024-04-20T11:09:02.434 INFO:tasks.workunit.client.0.smithi117.stdout:5/445: dread de/f18 [0,4194304] 0 2024-04-20T11:09:02.435 INFO:tasks.workunit.client.0.smithi117.stdout:0/304: truncate d3/d5/f37 1195824 0 2024-04-20T11:09:02.439 INFO:tasks.workunit.client.0.smithi117.stdout:0/305: truncate d3/d7/d8/f44 649949 0 2024-04-20T11:09:02.439 INFO:tasks.workunit.client.0.smithi117.stdout:7/353: fsync d1/d43/f61 0 2024-04-20T11:09:02.441 INFO:tasks.workunit.client.0.smithi117.stdout:5/446: creat de/d21/f95 x:0 0 0 2024-04-20T11:09:02.451 INFO:tasks.workunit.client.0.smithi117.stdout:2/431: truncate d1/d4/dd/d17/f1f 7241855 0 2024-04-20T11:09:02.454 INFO:tasks.workunit.client.0.smithi117.stdout:5/447: creat de/d21/f96 x:0 0 0 2024-04-20T11:09:02.457 INFO:tasks.workunit.client.0.smithi117.stdout:7/354: link d1/d4/d39/l40 d1/d43/d59/l75 0 2024-04-20T11:09:02.462 INFO:tasks.workunit.client.0.smithi117.stdout:0/306: dwrite d3/d7/d16/f2e [0,4194304] 0 2024-04-20T11:09:02.466 INFO:tasks.workunit.client.0.smithi117.stdout:0/307: rename d3/d7/d20 to d3/d7/d20/d39/d6c 22 2024-04-20T11:09:02.469 INFO:tasks.workunit.client.0.smithi117.stdout:7/355: rmdir d1/d43 39 2024-04-20T11:09:02.470 INFO:tasks.workunit.client.0.smithi117.stdout:5/448: link de/f11 de/d16/f97 0 2024-04-20T11:09:02.472 INFO:tasks.workunit.client.0.smithi117.stdout:7/356: write d1/d4/d6/d54/d63/f6e [155336,91864] 0 2024-04-20T11:09:02.473 INFO:tasks.workunit.client.0.smithi117.stdout:5/449: fsync de/d21/d84/f87 0 2024-04-20T11:09:02.477 INFO:tasks.workunit.client.0.smithi117.stdout:5/450: fdatasync de/d21/d45/d77/f7d 0 2024-04-20T11:09:02.479 INFO:tasks.workunit.client.0.smithi117.stdout:7/357: mknod d1/d1d/d42/d2d/c76 0 2024-04-20T11:09:02.488 INFO:tasks.workunit.client.0.smithi117.stdout:7/358: read d1/d4/d6/d13/f71 [1272407,96582] 0 2024-04-20T11:09:02.493 INFO:tasks.workunit.client.0.smithi117.stdout:7/359: fdatasync d1/f22 0 2024-04-20T11:09:02.494 INFO:tasks.workunit.client.0.smithi117.stdout:0/308: dwrite d3/d7/d20/d39/f62 [0,4194304] 0 2024-04-20T11:09:02.497 INFO:tasks.workunit.client.0.smithi117.stdout:5/451: write de/f19 [356480,109373] 0 2024-04-20T11:09:02.499 INFO:tasks.workunit.client.0.smithi117.stdout:7/360: mkdir d1/d1d/d77 0 2024-04-20T11:09:02.503 INFO:tasks.workunit.client.0.smithi117.stdout:0/309: rename d3/d7/d20/d39/d3a/d66 to d3/d7/d8/df/d6d 0 2024-04-20T11:09:02.506 INFO:tasks.workunit.client.0.smithi117.stdout:5/452: rmdir de/d21/d45/d48 39 2024-04-20T11:09:02.515 INFO:tasks.workunit.client.0.smithi117.stdout:5/453: rename de/d16/c1c to de/d41/c98 0 2024-04-20T11:09:02.516 INFO:tasks.workunit.client.0.smithi117.stdout:7/361: creat d1/d4/d6/d2b/d64/f78 x:0 0 0 2024-04-20T11:09:02.522 INFO:tasks.workunit.client.0.smithi117.stdout:7/362: fdatasync d1/d4/d6/d17/f4c 0 2024-04-20T11:09:02.531 INFO:tasks.workunit.client.0.smithi117.stdout:0/310: write d3/d7/d16/f63 [330065,61748] 0 2024-04-20T11:09:02.538 INFO:tasks.workunit.client.0.smithi117.stdout:5/454: dwrite de/d21/d45/d48/f58 [0,4194304] 0 2024-04-20T11:09:02.542 INFO:tasks.workunit.client.0.smithi117.stdout:7/363: dwrite d1/f22 [0,4194304] 0 2024-04-20T11:09:02.547 INFO:tasks.workunit.client.0.smithi117.stdout:7/364: read d1/f10 [7038036,120858] 0 2024-04-20T11:09:02.554 INFO:tasks.workunit.client.0.smithi117.stdout:0/311: dread d3/d7/f18 [0,4194304] 0 2024-04-20T11:09:02.556 INFO:tasks.workunit.client.0.smithi117.stdout:5/455: rename de/d21/d45/d48/c61 to de/d21/d51/c99 0 2024-04-20T11:09:02.556 INFO:tasks.workunit.client.0.smithi117.stdout:7/365: chown d1/d4/d6/d2b/c47 2548 1 2024-04-20T11:09:02.557 INFO:tasks.workunit.client.0.smithi117.stdout:0/312: chown d3/d7/d8/l10 950320 1 2024-04-20T11:09:02.562 INFO:tasks.workunit.client.0.smithi117.stdout:5/456: mknod de/d21/d51/c9a 0 2024-04-20T11:09:02.565 INFO:tasks.workunit.client.0.smithi117.stdout:0/313: unlink d3/d7/d16/f63 0 2024-04-20T11:09:02.566 INFO:tasks.workunit.client.0.smithi117.stdout:0/314: stat d3/d7/d8/f64 0 2024-04-20T11:09:02.573 INFO:tasks.workunit.client.0.smithi117.stdout:0/315: mkdir d3/d7/d20/d6e 0 2024-04-20T11:09:02.575 INFO:tasks.workunit.client.0.smithi117.stdout:0/316: read d3/d7/d8/df/f46 [1832226,10920] 0 2024-04-20T11:09:02.581 INFO:tasks.workunit.client.0.smithi117.stdout:0/317: creat d3/d7/d8/df/d22/f6f x:0 0 0 2024-04-20T11:09:02.585 INFO:tasks.workunit.client.0.smithi117.stdout:5/457: dread de/d21/d35/f3a [0,4194304] 0 2024-04-20T11:09:02.587 INFO:tasks.workunit.client.0.smithi117.stdout:5/458: chown de/d41/d47/f5c 901296 1 2024-04-20T11:09:02.589 INFO:tasks.workunit.client.0.smithi117.stdout:0/318: creat d3/d7/d2c/d52/f70 x:0 0 0 2024-04-20T11:09:02.593 INFO:tasks.workunit.client.0.smithi117.stdout:7/366: dread d1/d4/d39/f4d [0,4194304] 0 2024-04-20T11:09:02.599 INFO:tasks.workunit.client.0.smithi117.stdout:0/319: getdents d3/d7/d8/d2a/d54 0 2024-04-20T11:09:02.599 INFO:tasks.workunit.client.0.smithi117.stdout:7/367: fsync d1/d4/f30 0 2024-04-20T11:09:02.603 INFO:tasks.workunit.client.0.smithi117.stdout:0/320: creat d3/d7/d8/df/d6d/f71 x:0 0 0 2024-04-20T11:09:02.606 INFO:tasks.workunit.client.0.smithi117.stdout:7/368: creat d1/d1d/d77/f79 x:0 0 0 2024-04-20T11:09:02.610 INFO:tasks.workunit.client.0.smithi117.stdout:7/369: creat d1/d4/d39/f7a x:0 0 0 2024-04-20T11:09:02.622 INFO:tasks.workunit.client.0.smithi117.stdout:7/370: dread d1/d4/d39/f4d [4194304,4194304] 0 2024-04-20T11:09:02.623 INFO:tasks.workunit.client.0.smithi117.stdout:7/371: chown d1/d4/d6 23 1 2024-04-20T11:09:02.624 INFO:tasks.workunit.client.0.smithi117.stdout:0/321: write d3/d7/d8/df/f1b [947346,58087] 0 2024-04-20T11:09:02.628 INFO:tasks.workunit.client.0.smithi117.stdout:7/372: read d1/d1d/f21 [4008137,47334] 0 2024-04-20T11:09:02.629 INFO:tasks.workunit.client.0.smithi117.stdout:0/322: mknod d3/d7/d2c/d52/c72 0 2024-04-20T11:09:02.630 INFO:tasks.workunit.client.0.smithi117.stdout:7/373: write d1/d1d/f5c [1580320,84696] 0 2024-04-20T11:09:02.631 INFO:tasks.workunit.client.0.smithi117.stdout:5/459: dwrite de/f11 [0,4194304] 0 2024-04-20T11:09:02.638 INFO:tasks.workunit.client.0.smithi117.stdout:7/374: creat d1/d43/f7b x:0 0 0 2024-04-20T11:09:02.648 INFO:tasks.workunit.client.0.smithi117.stdout:7/375: dread d1/f7 [4194304,4194304] 0 2024-04-20T11:09:02.653 INFO:tasks.workunit.client.0.smithi117.stdout:7/376: creat d1/d4/d3d/f7c x:0 0 0 2024-04-20T11:09:02.656 INFO:tasks.workunit.client.0.smithi117.stdout:5/460: dwrite de/d21/d84/f87 [0,4194304] 0 2024-04-20T11:09:02.656 INFO:tasks.workunit.client.0.smithi117.stdout:0/323: dwrite d3/d7/d8/df/f1b [0,4194304] 0 2024-04-20T11:09:02.658 INFO:tasks.workunit.client.0.smithi117.stdout:0/324: readlink l2 0 2024-04-20T11:09:02.659 INFO:tasks.workunit.client.0.smithi117.stdout:7/377: creat d1/d1d/d42/d2d/d6c/f7d x:0 0 0 2024-04-20T11:09:02.666 INFO:tasks.workunit.client.0.smithi117.stdout:0/325: truncate d3/d5/f5a 842826 0 2024-04-20T11:09:02.667 INFO:tasks.workunit.client.0.smithi117.stdout:5/461: dread - de/d21/d45/d48/d4e/f8c zero size 2024-04-20T11:09:02.667 INFO:tasks.workunit.client.0.smithi117.stdout:7/378: mknod d1/d1d/d42/c7e 0 2024-04-20T11:09:02.670 INFO:tasks.workunit.client.0.smithi117.stdout:0/326: unlink d3/d5/f26 0 2024-04-20T11:09:02.670 INFO:tasks.workunit.client.0.smithi117.stdout:7/379: chown d1/d1d/f70 22 1 2024-04-20T11:09:02.673 INFO:tasks.workunit.client.0.smithi117.stdout:0/327: write d3/d7/d2c/f69 [516393,71837] 0 2024-04-20T11:09:02.674 INFO:tasks.workunit.client.0.smithi117.stdout:7/380: write d1/d4/d3d/f7c [1024331,35523] 0 2024-04-20T11:09:02.678 INFO:tasks.workunit.client.0.smithi117.stdout:5/462: getdents de/d21/d84 0 2024-04-20T11:09:02.679 INFO:tasks.workunit.client.0.smithi117.stdout:0/328: dread d3/d7/d16/f2b [0,4194304] 0 2024-04-20T11:09:02.680 INFO:tasks.workunit.client.0.smithi117.stdout:7/381: rename d1/d1d/d77/f79 to d1/d1d/d42/d2d/f7f 0 2024-04-20T11:09:02.692 INFO:tasks.workunit.client.0.smithi117.stdout:5/463: dwrite de/d21/d45/d77/f85 [0,4194304] 0 2024-04-20T11:09:02.764 INFO:tasks.workunit.client.0.smithi117.stdout:5/464: unlink de/d21/d45/c7e 0 2024-04-20T11:09:02.768 INFO:tasks.workunit.client.0.smithi117.stdout:5/465: creat de/d41/f9b x:0 0 0 2024-04-20T11:09:02.772 INFO:tasks.workunit.client.0.smithi117.stdout:5/466: dread de/d41/d47/d4c/f89 [0,4194304] 0 2024-04-20T11:09:02.773 INFO:tasks.workunit.client.0.smithi117.stdout:5/467: fsync de/d21/d45/d48/d4e/f4f 0 2024-04-20T11:09:02.775 INFO:tasks.workunit.client.0.smithi117.stdout:5/468: chown de/d41/d47/d4c/f75 996672 1 2024-04-20T11:09:02.793 INFO:tasks.workunit.client.0.smithi117.stdout:0/329: dread d3/d7/d20/f25 [0,4194304] 0 2024-04-20T11:09:02.795 INFO:tasks.workunit.client.0.smithi117.stdout:0/330: write d3/d7/f19 [3103310,64669] 0 2024-04-20T11:09:02.920 INFO:tasks.workunit.client.0.smithi117.stdout:5/469: symlink de/d21/d84/l9c 0 2024-04-20T11:09:02.924 INFO:tasks.workunit.client.0.smithi117.stdout:0/331: rename d3/d7/d8/df/d22/c5c to d3/d7/d8/df/d6d/c73 0 2024-04-20T11:09:02.929 INFO:tasks.workunit.client.0.smithi117.stdout:5/470: mkdir de/d16/d9d 0 2024-04-20T11:09:02.929 INFO:tasks.workunit.client.0.smithi117.stdout:0/332: dread - d3/d7/d8/df/d6d/f71 zero size 2024-04-20T11:09:02.934 INFO:tasks.workunit.client.0.smithi117.stdout:0/333: readlink d3/d7/d20/d47/l4f 0 2024-04-20T11:09:02.945 INFO:tasks.workunit.client.0.smithi117.stdout:5/471: link de/d41/d47/d4c/f89 de/d41/d47/f9e 0 2024-04-20T11:09:02.952 INFO:tasks.workunit.client.0.smithi117.stdout:3/340: sync 2024-04-20T11:09:02.961 INFO:tasks.workunit.client.0.smithi117.stdout:5/472: creat de/d21/f9f x:0 0 0 2024-04-20T11:09:02.972 INFO:tasks.workunit.client.0.smithi117.stdout:5/473: dread de/d41/f52 [0,4194304] 0 2024-04-20T11:09:02.977 INFO:tasks.workunit.client.0.smithi117.stdout:0/334: dwrite d3/d7/d20/d39/f62 [0,4194304] 0 2024-04-20T11:09:02.977 INFO:tasks.workunit.client.0.smithi117.stdout:5/474: symlink de/d21/d45/d48/la0 0 2024-04-20T11:09:02.981 INFO:tasks.workunit.client.0.smithi117.stdout:0/335: write d3/d7/f4d [2035677,8493] 0 2024-04-20T11:09:02.982 INFO:tasks.workunit.client.0.smithi117.stdout:3/341: dread d0/d5/f40 [0,4194304] 0 2024-04-20T11:09:02.983 INFO:tasks.workunit.client.0.smithi117.stdout:5/475: read de/d41/d4b/f5a [1269924,102662] 0 2024-04-20T11:09:02.984 INFO:tasks.workunit.client.0.smithi117.stdout:0/336: chown d3/d7/c23 12487998 1 2024-04-20T11:09:02.984 INFO:tasks.workunit.client.0.smithi117.stdout:5/476: chown de 385086225 1 2024-04-20T11:09:02.989 INFO:tasks.workunit.client.0.smithi117.stdout:0/337: write d3/d7/d8/df/f1b [2766021,63270] 0 2024-04-20T11:09:02.990 INFO:tasks.workunit.client.0.smithi117.stdout:5/477: write de/d41/f52 [1352087,49969] 0 2024-04-20T11:09:02.994 INFO:tasks.workunit.client.0.smithi117.stdout:5/478: chown de/d41/d47/l7f 6 1 2024-04-20T11:09:02.994 INFO:tasks.workunit.client.0.smithi117.stdout:3/342: link d0/d5/d7/c79 d0/d5/de/d1d/d29/d66/d63/c7b 0 2024-04-20T11:09:02.997 INFO:tasks.workunit.client.0.smithi117.stdout:5/479: fsync de/d41/d47/d4c/f75 0 2024-04-20T11:09:02.997 INFO:tasks.workunit.client.0.smithi117.stdout:3/343: write d0/f3 [1995100,108607] 0 2024-04-20T11:09:03.002 INFO:tasks.workunit.client.0.smithi117.stdout:3/344: write d0/d5/de/d1d/d29/d66/f6c [947844,103498] 0 2024-04-20T11:09:03.015 INFO:tasks.workunit.client.0.smithi117.stdout:5/480: dwrite de/d21/d84/f87 [0,4194304] 0 2024-04-20T11:09:03.021 INFO:tasks.workunit.client.0.smithi117.stdout:5/481: mkdir de/d21/d45/d48/d4e/da1 0 2024-04-20T11:09:03.025 INFO:tasks.workunit.client.0.smithi117.stdout:5/482: write de/d41/d47/f6f [178312,59464] 0 2024-04-20T11:09:03.028 INFO:tasks.workunit.client.0.smithi117.stdout:5/483: write de/f34 [2275192,82064] 0 2024-04-20T11:09:03.031 INFO:tasks.workunit.client.0.smithi117.stdout:3/345: dwrite d0/d5/d11/f74 [0,4194304] 0 2024-04-20T11:09:03.037 INFO:tasks.workunit.client.0.smithi117.stdout:3/346: symlink d0/d5/de/d4e/l7c 0 2024-04-20T11:09:03.039 INFO:tasks.workunit.client.0.smithi117.stdout:5/484: dread de/d21/d35/f3a [0,4194304] 0 2024-04-20T11:09:03.044 INFO:tasks.workunit.client.0.smithi117.stdout:0/338: dwrite d3/d5/f37 [0,4194304] 0 2024-04-20T11:09:03.045 INFO:tasks.workunit.client.0.smithi117.stdout:5/485: creat de/d5b/d65/fa2 x:0 0 0 2024-04-20T11:09:03.061 INFO:tasks.workunit.client.0.smithi117.stdout:3/347: dwrite d0/d5/de/d1d/d29/d66/d63/f7a [0,4194304] 0 2024-04-20T11:09:03.067 INFO:tasks.workunit.client.0.smithi117.stdout:5/486: dwrite de/d41/f52 [0,4194304] 0 2024-04-20T11:09:03.070 INFO:tasks.workunit.client.0.smithi117.stdout:3/348: mknod d0/d5/de/d1d/d36/c7d 0 2024-04-20T11:09:03.071 INFO:tasks.workunit.client.0.smithi117.stdout:0/339: truncate d3/d7/d20/f25 2829701 0 2024-04-20T11:09:03.073 INFO:tasks.workunit.client.0.smithi117.stdout:0/340: write d3/d5/f6a [4008396,90575] 0 2024-04-20T11:09:03.075 INFO:tasks.workunit.client.0.smithi117.stdout:3/349: creat d0/d5/d11/f7e x:0 0 0 2024-04-20T11:09:03.077 INFO:tasks.workunit.client.0.smithi117.stdout:0/341: mkdir d3/d5/d74 0 2024-04-20T11:09:03.079 INFO:tasks.workunit.client.0.smithi117.stdout:3/350: symlink d0/d5/de/d4e/l7f 0 2024-04-20T11:09:03.081 INFO:tasks.workunit.client.0.smithi117.stdout:0/342: creat d3/d7/d8/df/d22/f75 x:0 0 0 2024-04-20T11:09:03.082 INFO:tasks.workunit.client.0.smithi117.stdout:3/351: write d0/d5/d7/dd/ff [3418002,41555] 0 2024-04-20T11:09:03.089 INFO:tasks.workunit.client.0.smithi117.stdout:5/487: write de/d21/d45/d48/f4a [461147,5213] 0 2024-04-20T11:09:03.089 INFO:tasks.workunit.client.0.smithi117.stdout:3/352: mkdir d0/d5/d11/d80 0 2024-04-20T11:09:03.091 INFO:tasks.workunit.client.0.smithi117.stdout:3/353: fsync d0/d5/d7/f1e 0 2024-04-20T11:09:03.095 INFO:tasks.workunit.client.0.smithi117.stdout:3/354: write d0/d5/f40 [1880023,2338] 0 2024-04-20T11:09:03.103 INFO:tasks.workunit.client.0.smithi117.stdout:0/343: dwrite d3/d7/d8/df/f1b [0,4194304] 0 2024-04-20T11:09:03.106 INFO:tasks.workunit.client.0.smithi117.stdout:3/355: link d0/d5/de/d4e/l7c d0/d5/de/d1d/d29/l81 0 2024-04-20T11:09:03.107 INFO:tasks.workunit.client.0.smithi117.stdout:5/488: dwrite de/d41/d47/d4c/f8f [0,4194304] 0 2024-04-20T11:09:03.108 INFO:tasks.workunit.client.0.smithi117.stdout:3/356: read - d0/d5/de/d1d/f70 zero size 2024-04-20T11:09:03.124 INFO:tasks.workunit.client.0.smithi117.stdout:0/344: dwrite d3/d7/d8/df/d6d/f71 [0,4194304] 0 2024-04-20T11:09:03.130 INFO:tasks.workunit.client.0.smithi117.stdout:5/489: dwrite de/f74 [0,4194304] 0 2024-04-20T11:09:03.135 INFO:tasks.workunit.client.0.smithi117.stdout:5/490: getdents de/d21 0 2024-04-20T11:09:03.143 INFO:tasks.workunit.client.0.smithi117.stdout:5/491: getdents de/d21 0 2024-04-20T11:09:03.143 INFO:tasks.workunit.client.0.smithi117.stdout:0/345: dwrite d3/d7/d8/f44 [0,4194304] 0 2024-04-20T11:09:03.156 INFO:tasks.workunit.client.0.smithi117.stdout:5/492: dread de/f11 [0,4194304] 0 2024-04-20T11:09:03.157 INFO:tasks.workunit.client.0.smithi117.stdout:5/493: stat de/d16/f1e 0 2024-04-20T11:09:03.162 INFO:tasks.workunit.client.0.smithi117.stdout:5/494: creat de/d21/d45/d48/d4e/fa3 x:0 0 0 2024-04-20T11:09:03.163 INFO:tasks.workunit.client.0.smithi117.stdout:0/346: dwrite d3/d7/f18 [0,4194304] 0 2024-04-20T11:09:03.180 INFO:tasks.workunit.client.0.smithi117.stdout:5/495: dwrite de/d21/d45/d77/f7d [0,4194304] 0 2024-04-20T11:09:03.180 INFO:tasks.workunit.client.0.smithi117.stdout:0/347: dwrite d3/d7/d8/f31 [0,4194304] 0 2024-04-20T11:09:03.193 INFO:tasks.workunit.client.0.smithi117.stdout:5/496: rmdir de 39 2024-04-20T11:09:03.198 INFO:tasks.workunit.client.0.smithi117.stdout:0/348: creat d3/d7/d2c/f76 x:0 0 0 2024-04-20T11:09:03.200 INFO:tasks.workunit.client.0.smithi117.stdout:5/497: symlink de/d21/d45/la4 0 2024-04-20T11:09:03.213 INFO:tasks.workunit.client.0.smithi117.stdout:0/349: dwrite d3/d7/f14 [0,4194304] 0 2024-04-20T11:09:03.219 INFO:tasks.workunit.client.0.smithi117.stdout:0/350: creat d3/d7/d20/d47/f77 x:0 0 0 2024-04-20T11:09:03.219 INFO:tasks.workunit.client.0.smithi117.stdout:5/498: truncate de/d16/f1e 4375562 0 2024-04-20T11:09:03.223 INFO:tasks.workunit.client.0.smithi117.stdout:5/499: creat de/d21/d45/d77/fa5 x:0 0 0 2024-04-20T11:09:03.224 INFO:tasks.workunit.client.0.smithi117.stdout:0/351: mkdir d3/d7/d20/d47/d78 0 2024-04-20T11:09:03.226 INFO:tasks.workunit.client.0.smithi117.stdout:5/500: rename de/d16/f90 to de/d21/d45/d77/fa6 0 2024-04-20T11:09:03.227 INFO:tasks.workunit.client.0.smithi117.stdout:0/352: symlink d3/d7/d2c/d52/l79 0 2024-04-20T11:09:03.229 INFO:tasks.workunit.client.0.smithi117.stdout:5/501: readlink de/d41/d47/d55/l93 0 2024-04-20T11:09:03.229 INFO:tasks.workunit.client.0.smithi117.stdout:0/353: chown d3/d7/d8/d2a/l4e 82 1 2024-04-20T11:09:03.230 INFO:tasks.workunit.client.0.smithi117.stdout:5/502: read de/d16/f66 [1098994,42168] 0 2024-04-20T11:09:03.230 INFO:tasks.workunit.client.0.smithi117.stdout:0/354: chown d3/d7/d16/l55 18 1 2024-04-20T11:09:03.232 INFO:tasks.workunit.client.0.smithi117.stdout:1/363: sync 2024-04-20T11:09:03.232 INFO:tasks.workunit.client.0.smithi117.stdout:6/429: sync 2024-04-20T11:09:03.232 INFO:tasks.workunit.client.0.smithi117.stdout:8/259: sync 2024-04-20T11:09:03.232 INFO:tasks.workunit.client.0.smithi117.stdout:4/388: sync 2024-04-20T11:09:03.232 INFO:tasks.workunit.client.0.smithi117.stdout:0/355: write d3/d7/d20/d47/f77 [292537,11908] 0 2024-04-20T11:09:03.233 INFO:tasks.workunit.client.0.smithi117.stdout:5/503: mkdir de/d21/d45/d48/da7 0 2024-04-20T11:09:03.235 INFO:tasks.workunit.client.0.smithi117.stdout:8/260: write d0/d9/f31 [681616,100069] 0 2024-04-20T11:09:03.236 INFO:tasks.workunit.client.0.smithi117.stdout:0/356: dread - d3/d7/d2c/f65 zero size 2024-04-20T11:09:03.239 INFO:tasks.workunit.client.0.smithi117.stdout:4/389: mkdir d1/dd/d1a/d2b/d86 0 2024-04-20T11:09:03.242 INFO:tasks.workunit.client.0.smithi117.stdout:5/504: symlink de/d41/d4b/la8 0 2024-04-20T11:09:03.243 INFO:tasks.workunit.client.0.smithi117.stdout:6/430: truncate d3/d4/d2e/d55/f60 1496771 0 2024-04-20T11:09:03.244 INFO:tasks.workunit.client.0.smithi117.stdout:8/261: creat d0/d8/d39/f54 x:0 0 0 2024-04-20T11:09:03.246 INFO:tasks.workunit.client.0.smithi117.stdout:5/505: truncate de/d41/d47/d4c/f75 1158878 0 2024-04-20T11:09:03.247 INFO:tasks.workunit.client.0.smithi117.stdout:5/506: chown de/d16/f97 790 1 2024-04-20T11:09:03.251 INFO:tasks.workunit.client.0.smithi117.stdout:6/431: unlink d3/d4/d2e/d5f/c88 0 2024-04-20T11:09:03.252 INFO:tasks.workunit.client.0.smithi117.stdout:6/432: fsync d3/d4/f46 0 2024-04-20T11:09:03.252 INFO:tasks.workunit.client.0.smithi117.stdout:0/357: link d3/d7/d8/f44 d3/d7/d8/df/d6d/f7a 0 2024-04-20T11:09:03.255 INFO:tasks.workunit.client.0.smithi117.stdout:0/358: write d3/f3e [1099502,77792] 0 2024-04-20T11:09:03.256 INFO:tasks.workunit.client.0.smithi117.stdout:8/262: rename d0/f22 to d0/d8/d11/d24/d27/d47/f55 0 2024-04-20T11:09:03.258 INFO:tasks.workunit.client.0.smithi117.stdout:4/390: truncate d1/f2 3293205 0 2024-04-20T11:09:03.261 INFO:tasks.workunit.client.0.smithi117.stdout:4/391: write d1/dd/f53 [902249,78742] 0 2024-04-20T11:09:03.261 INFO:tasks.workunit.client.0.smithi117.stdout:5/507: unlink de/d41/d47/f9e 0 2024-04-20T11:09:03.263 INFO:tasks.workunit.client.0.smithi117.stdout:4/392: readlink d1/dd/d52/l83 0 2024-04-20T11:09:03.266 INFO:tasks.workunit.client.0.smithi117.stdout:6/433: unlink d3/d4/d2e/d55/f60 0 2024-04-20T11:09:03.268 INFO:tasks.workunit.client.0.smithi117.stdout:7/382: sync 2024-04-20T11:09:03.268 INFO:tasks.workunit.client.0.smithi117.stdout:2/432: sync 2024-04-20T11:09:03.268 INFO:tasks.workunit.client.0.smithi117.stdout:9/397: sync 2024-04-20T11:09:03.268 INFO:tasks.workunit.client.0.smithi117.stdout:3/357: sync 2024-04-20T11:09:03.268 INFO:tasks.workunit.client.0.smithi117.stdout:1/364: dwrite d6/d1f/d30/d32/f6a [0,4194304] 0 2024-04-20T11:09:03.272 INFO:tasks.workunit.client.0.smithi117.stdout:0/359: unlink d3/d7/d2c/f76 0 2024-04-20T11:09:03.276 INFO:tasks.workunit.client.0.smithi117.stdout:8/263: unlink d0/dc/l45 0 2024-04-20T11:09:03.279 INFO:tasks.workunit.client.0.smithi117.stdout:0/360: truncate d3/d7/d20/d47/f77 1338129 0 2024-04-20T11:09:03.283 INFO:tasks.workunit.client.0.smithi117.stdout:0/361: write d3/d7/f4d [2112438,112455] 0 2024-04-20T11:09:03.286 INFO:tasks.workunit.client.0.smithi117.stdout:5/508: dwrite de/d21/d45/d77/f85 [0,4194304] 0 2024-04-20T11:09:03.286 INFO:tasks.workunit.client.0.smithi117.stdout:4/393: chown d1/c27 21618 1 2024-04-20T11:09:03.294 INFO:tasks.workunit.client.0.smithi117.stdout:9/398: truncate d4/f39 705342 0 2024-04-20T11:09:03.298 INFO:tasks.workunit.client.0.smithi117.stdout:9/399: readlink d4/d5/da/l5a 0 2024-04-20T11:09:03.299 INFO:tasks.workunit.client.0.smithi117.stdout:5/509: truncate de/d41/d47/d55/d56/f78 1582279 0 2024-04-20T11:09:03.301 INFO:tasks.workunit.client.0.smithi117.stdout:6/434: mkdir d3/d50/d8a 0 2024-04-20T11:09:03.304 INFO:tasks.workunit.client.0.smithi117.stdout:8/264: dwrite d0/d8/d11/d24/d27/d47/f4f [0,4194304] 0 2024-04-20T11:09:03.305 INFO:tasks.workunit.client.0.smithi117.stdout:9/400: write d4/d5/da/f6f [923606,120159] 0 2024-04-20T11:09:03.309 INFO:tasks.workunit.client.0.smithi117.stdout:3/358: symlink d0/d5/d7/l82 0 2024-04-20T11:09:03.309 INFO:tasks.workunit.client.0.smithi117.stdout:4/394: mkdir d1/d9/d1f/d87 0 2024-04-20T11:09:03.314 INFO:tasks.workunit.client.0.smithi117.stdout:0/362: dwrite d3/d7/f14 [4194304,4194304] 0 2024-04-20T11:09:03.316 INFO:tasks.workunit.client.0.smithi117.stdout:6/435: chown d3/l1d 157949341 1 2024-04-20T11:09:03.316 INFO:tasks.workunit.client.0.smithi117.stdout:9/401: write d4/d68/d6e/f77 [3817365,95808] 0 2024-04-20T11:09:03.317 INFO:tasks.workunit.client.0.smithi117.stdout:5/510: mknod de/d21/d45/d48/d4e/da1/ca9 0 2024-04-20T11:09:03.318 INFO:tasks.workunit.client.0.smithi117.stdout:1/365: dread d6/f12 [0,4194304] 0 2024-04-20T11:09:03.320 INFO:tasks.workunit.client.0.smithi117.stdout:4/395: stat d1/d45/d4a/f50 0 2024-04-20T11:09:03.322 INFO:tasks.workunit.client.0.smithi117.stdout:9/402: readlink d4/d30/l7f 0 2024-04-20T11:09:03.322 INFO:tasks.workunit.client.0.smithi117.stdout:1/366: stat d6/d16/d3a 0 2024-04-20T11:09:03.325 INFO:tasks.workunit.client.0.smithi117.stdout:0/363: rmdir d3/d7/d8/d2a/d54 39 2024-04-20T11:09:03.325 INFO:tasks.workunit.client.0.smithi117.stdout:7/383: dwrite d1/d4/d39/f34 [0,4194304] 0 2024-04-20T11:09:03.325 INFO:tasks.workunit.client.0.smithi117.stdout:5/511: chown de/d21/f9f 64683 1 2024-04-20T11:09:03.325 INFO:tasks.workunit.client.0.smithi117.stdout:1/367: chown d6/d35 471513 1 2024-04-20T11:09:03.327 INFO:tasks.workunit.client.0.smithi117.stdout:4/396: dread - d1/d45/d4a/d59/f7f zero size 2024-04-20T11:09:03.331 INFO:tasks.workunit.client.0.smithi117.stdout:6/436: creat d3/d4/d2e/d3f/f8b x:0 0 0 2024-04-20T11:09:03.333 INFO:tasks.workunit.client.0.smithi117.stdout:0/364: mknod d3/d7/d16/c7b 0 2024-04-20T11:09:03.341 INFO:tasks.workunit.client.0.smithi117.stdout:8/265: creat d0/d8/d11/d24/d53/f56 x:0 0 0 2024-04-20T11:09:03.341 INFO:tasks.workunit.client.0.smithi117.stdout:4/397: truncate d1/dd/d52/f54 99009 0 2024-04-20T11:09:03.349 INFO:tasks.workunit.client.0.smithi117.stdout:9/403: creat d4/de/f90 x:0 0 0 2024-04-20T11:09:03.349 INFO:tasks.workunit.client.0.smithi117.stdout:7/384: link d1/d43/d59/f69 d1/d43/f80 0 2024-04-20T11:09:03.349 INFO:tasks.workunit.client.0.smithi117.stdout:5/512: symlink de/d21/d45/laa 0 2024-04-20T11:09:03.350 INFO:tasks.workunit.client.0.smithi117.stdout:3/359: dwrite d0/f2c [0,4194304] 0 2024-04-20T11:09:03.351 INFO:tasks.workunit.client.0.smithi117.stdout:6/437: creat d3/d4/d2e/d55/f8c x:0 0 0 2024-04-20T11:09:03.356 INFO:tasks.workunit.client.0.smithi117.stdout:4/398: truncate d1/d45/d4a/d59/f7f 514321 0 2024-04-20T11:09:03.362 INFO:tasks.workunit.client.0.smithi117.stdout:4/399: readlink d1/d9/l22 0 2024-04-20T11:09:03.362 INFO:tasks.workunit.client.0.smithi117.stdout:4/400: chown d1/d9/d1f/f55 17198640 1 2024-04-20T11:09:03.367 INFO:tasks.workunit.client.0.smithi117.stdout:8/266: dwrite d0/d8/f38 [0,4194304] 0 2024-04-20T11:09:03.386 INFO:tasks.workunit.client.0.smithi117.stdout:1/368: link d6/d1f/d30/f74 d6/d16/d3a/d5a/d5c/d61/f77 0 2024-04-20T11:09:03.388 INFO:tasks.workunit.client.0.smithi117.stdout:1/369: write d6/d1f/f45 [5097125,115114] 0 2024-04-20T11:09:03.391 INFO:tasks.workunit.client.0.smithi117.stdout:1/370: dread - d6/d16/d3a/d5a/d5c/d61/d71/d6b/f6c zero size 2024-04-20T11:09:03.392 INFO:tasks.workunit.client.0.smithi117.stdout:5/513: mkdir de/d41/d47/d55/dab 0 2024-04-20T11:09:03.393 INFO:tasks.workunit.client.0.smithi117.stdout:9/404: mknod d4/d30/d31/c91 0 2024-04-20T11:09:03.395 INFO:tasks.workunit.client.0.smithi117.stdout:9/405: chown d4/d30/d31/c91 1723 1 2024-04-20T11:09:03.395 INFO:tasks.workunit.client.0.smithi117.stdout:3/360: unlink d0/d5/d7/dd/c43 0 2024-04-20T11:09:03.396 INFO:tasks.workunit.client.0.smithi117.stdout:3/361: chown d0/f35 0 1 2024-04-20T11:09:03.398 INFO:tasks.workunit.client.0.smithi117.stdout:4/401: mknod d1/d45/d4a/d59/c88 0 2024-04-20T11:09:03.399 INFO:tasks.workunit.client.0.smithi117.stdout:4/402: fdatasync d1/f57 0 2024-04-20T11:09:03.402 INFO:tasks.workunit.client.0.smithi117.stdout:9/406: dread d4/de/d10/f38 [0,4194304] 0 2024-04-20T11:09:03.405 INFO:tasks.workunit.client.0.smithi117.stdout:4/403: dread d1/f30 [0,4194304] 0 2024-04-20T11:09:03.407 INFO:tasks.workunit.client.0.smithi117.stdout:4/404: readlink d1/dd/d52/l83 0 2024-04-20T11:09:03.410 INFO:tasks.workunit.client.0.smithi117.stdout:7/385: truncate d1/d4/d6/f26 6334602 0 2024-04-20T11:09:03.418 INFO:tasks.workunit.client.0.smithi117.stdout:9/407: dwrite d4/d5/f2a [0,4194304] 0 2024-04-20T11:09:03.421 INFO:tasks.workunit.client.0.smithi117.stdout:9/408: write d4/d35/f3d [1714176,28656] 0 2024-04-20T11:09:03.421 INFO:tasks.workunit.client.0.smithi117.stdout:8/267: creat d0/f57 x:0 0 0 2024-04-20T11:09:03.422 INFO:tasks.workunit.client.0.smithi117.stdout:3/362: creat d0/d5/d7/f83 x:0 0 0 2024-04-20T11:09:03.426 INFO:tasks.workunit.client.0.smithi117.stdout:4/405: dwrite d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:03.426 INFO:tasks.workunit.client.0.smithi117.stdout:8/268: chown d0/d8/d11/d24/d27/d47/f4f 45090 1 2024-04-20T11:09:03.428 INFO:tasks.workunit.client.0.smithi117.stdout:4/406: chown d1/dd/d1a/f37 10 1 2024-04-20T11:09:03.431 INFO:tasks.workunit.client.0.smithi117.stdout:8/269: mkdir d0/dc/d2e/d3e/d58 0 2024-04-20T11:09:03.440 INFO:tasks.workunit.client.0.smithi117.stdout:7/386: dwrite d1/d1d/f21 [8388608,4194304] 0 2024-04-20T11:09:03.448 INFO:tasks.workunit.client.0.smithi117.stdout:3/363: dwrite d0/d5/de/d1d/f62 [0,4194304] 0 2024-04-20T11:09:03.449 INFO:tasks.workunit.client.0.smithi117.stdout:7/387: getdents d1/d4 0 2024-04-20T11:09:03.451 INFO:tasks.workunit.client.0.smithi117.stdout:4/407: dwrite d1/f19 [4194304,4194304] 0 2024-04-20T11:09:03.458 INFO:tasks.workunit.client.0.smithi117.stdout:3/364: symlink d0/d5/d73/l84 0 2024-04-20T11:09:03.466 INFO:tasks.workunit.client.0.smithi117.stdout:7/388: write d1/d4/d6/d13/f28 [1939125,106999] 0 2024-04-20T11:09:03.476 INFO:tasks.workunit.client.0.smithi117.stdout:3/365: dread d0/d5/d7/dd/ff [0,4194304] 0 2024-04-20T11:09:03.478 INFO:tasks.workunit.client.0.smithi117.stdout:3/366: fsync d0/d5/de/d1d/f70 0 2024-04-20T11:09:03.480 INFO:tasks.workunit.client.0.smithi117.stdout:1/371: unlink d6/f25 0 2024-04-20T11:09:03.483 INFO:tasks.workunit.client.0.smithi117.stdout:0/365: rename d3/d7/d8/df/f46 to d3/d7/d20/d47/d78/f7c 0 2024-04-20T11:09:03.484 INFO:tasks.workunit.client.0.smithi117.stdout:3/367: chown d0/d5/d7/dd/ff 5 1 2024-04-20T11:09:03.485 INFO:tasks.workunit.client.0.smithi117.stdout:2/433: sync 2024-04-20T11:09:03.486 INFO:tasks.workunit.client.0.smithi117.stdout:1/372: stat d6/d16/d3a/d5a/d5c/d61 0 2024-04-20T11:09:03.490 INFO:tasks.workunit.client.0.smithi117.stdout:0/366: creat d3/d7/f7d x:0 0 0 2024-04-20T11:09:03.491 INFO:tasks.workunit.client.0.smithi117.stdout:5/514: rename de/f68 to de/d41/d47/d55/d56/fac 0 2024-04-20T11:09:03.496 INFO:tasks.workunit.client.0.smithi117.stdout:2/434: rmdir d1/d4/d83/d29/d2e/d52 39 2024-04-20T11:09:03.497 INFO:tasks.workunit.client.0.smithi117.stdout:9/409: dread d4/d5/f2a [4194304,4194304] 0 2024-04-20T11:09:03.498 INFO:tasks.workunit.client.0.smithi117.stdout:6/438: creat d3/d4/f8d x:0 0 0 2024-04-20T11:09:03.504 INFO:tasks.workunit.client.0.smithi117.stdout:4/408: dread d1/f2 [0,4194304] 0 2024-04-20T11:09:03.504 INFO:tasks.workunit.client.0.smithi117.stdout:8/270: rename d0/d9/f32 to d0/d8/d11/d24/d2b/f59 0 2024-04-20T11:09:03.505 INFO:tasks.workunit.client.0.smithi117.stdout:3/368: dwrite d0/d2a/f3b [0,4194304] 0 2024-04-20T11:09:03.507 INFO:tasks.workunit.client.0.smithi117.stdout:0/367: dread d3/d7/d8/f13 [0,4194304] 0 2024-04-20T11:09:03.507 INFO:tasks.workunit.client.0.smithi117.stdout:5/515: creat de/d21/d45/d48/fad x:0 0 0 2024-04-20T11:09:03.508 INFO:tasks.workunit.client.0.smithi117.stdout:4/409: write d1/f81 [641224,17796] 0 2024-04-20T11:09:03.510 INFO:tasks.workunit.client.0.smithi117.stdout:1/373: dread d6/f3c [0,4194304] 0 2024-04-20T11:09:03.527 INFO:tasks.workunit.client.0.smithi117.stdout:3/369: mkdir d0/d5/de/d1d/d85 0 2024-04-20T11:09:03.528 INFO:tasks.workunit.client.0.smithi117.stdout:5/516: dwrite de/d21/d45/d77/f7d [0,4194304] 0 2024-04-20T11:09:03.529 INFO:tasks.workunit.client.0.smithi117.stdout:3/370: chown d0/d5/de/d4e 14588068 1 2024-04-20T11:09:03.531 INFO:tasks.workunit.client.0.smithi117.stdout:9/410: dread d4/d5/f21 [0,4194304] 0 2024-04-20T11:09:03.531 INFO:tasks.workunit.client.0.smithi117.stdout:5/517: chown de/d21/d45/d48 253 1 2024-04-20T11:09:03.533 INFO:tasks.workunit.client.0.smithi117.stdout:7/389: dwrite d1/d43/d59/f69 [0,4194304] 0 2024-04-20T11:09:03.535 INFO:tasks.workunit.client.0.smithi117.stdout:6/439: readlink d3/d3a/d7f/l84 0 2024-04-20T11:09:03.540 INFO:tasks.workunit.client.0.smithi117.stdout:8/271: rmdir d0 39 2024-04-20T11:09:03.548 INFO:tasks.workunit.client.0.smithi117.stdout:4/410: symlink d1/dd/d52/l89 0 2024-04-20T11:09:03.548 INFO:tasks.workunit.client.0.smithi117.stdout:0/368: mkdir d3/d7e 0 2024-04-20T11:09:03.551 INFO:tasks.workunit.client.0.smithi117.stdout:9/411: mkdir d4/d4a/d92 0 2024-04-20T11:09:03.554 INFO:tasks.workunit.client.0.smithi117.stdout:3/371: fsync d0/d5/d11/f74 0 2024-04-20T11:09:03.556 INFO:tasks.workunit.client.0.smithi117.stdout:3/372: dread - d0/d5/de/d4e/f53 zero size 2024-04-20T11:09:03.559 INFO:tasks.workunit.client.0.smithi117.stdout:6/440: unlink d3/d50/c7b 0 2024-04-20T11:09:03.560 INFO:tasks.workunit.client.0.smithi117.stdout:5/518: dread de/d21/f2e [0,4194304] 0 2024-04-20T11:09:03.562 INFO:tasks.workunit.client.0.smithi117.stdout:4/411: symlink d1/dd/d52/l8a 0 2024-04-20T11:09:03.566 INFO:tasks.workunit.client.0.smithi117.stdout:9/412: rename d4/d5/da/fb to d4/d5/da/f93 0 2024-04-20T11:09:03.566 INFO:tasks.workunit.client.0.smithi117.stdout:1/374: dwrite d6/d35/f36 [0,4194304] 0 2024-04-20T11:09:03.570 INFO:tasks.workunit.client.0.smithi117.stdout:5/519: creat de/d5b/fae x:0 0 0 2024-04-20T11:09:03.571 INFO:tasks.workunit.client.0.smithi117.stdout:8/272: mkdir d0/d8/d39/d5a 0 2024-04-20T11:09:03.573 INFO:tasks.workunit.client.0.smithi117.stdout:6/441: rmdir d3/d4/d2e 39 2024-04-20T11:09:03.574 INFO:tasks.workunit.client.0.smithi117.stdout:0/369: mknod d3/d7e/c7f 0 2024-04-20T11:09:03.574 INFO:tasks.workunit.client.0.smithi117.stdout:5/520: write de/d21/d45/d77/f85 [579750,106386] 0 2024-04-20T11:09:03.574 INFO:tasks.workunit.client.0.smithi117.stdout:7/390: dwrite d1/f10 [0,4194304] 0 2024-04-20T11:09:03.577 INFO:tasks.workunit.client.0.smithi117.stdout:4/412: unlink d1/d9/d1f/f55 0 2024-04-20T11:09:03.581 INFO:tasks.workunit.client.0.smithi117.stdout:2/435: link d1/d4/d83/d29/d2e/l75 d1/l8b 0 2024-04-20T11:09:03.584 INFO:tasks.workunit.client.0.smithi117.stdout:5/521: write de/d21/d45/d48/d4e/f50 [5162609,60512] 0 2024-04-20T11:09:03.584 INFO:tasks.workunit.client.0.smithi117.stdout:2/436: readlink d1/l61 0 2024-04-20T11:09:03.587 INFO:tasks.workunit.client.0.smithi117.stdout:5/522: chown de/d21/d35/c7c 1862 1 2024-04-20T11:09:03.590 INFO:tasks.workunit.client.0.smithi117.stdout:9/413: mkdir d4/de/d10/d94 0 2024-04-20T11:09:03.594 INFO:tasks.workunit.client.0.smithi117.stdout:3/373: creat d0/f86 x:0 0 0 2024-04-20T11:09:03.595 INFO:tasks.workunit.client.0.smithi117.stdout:1/375: mknod d6/d1f/d30/c78 0 2024-04-20T11:09:03.598 INFO:tasks.workunit.client.0.smithi117.stdout:8/273: mkdir d0/dc/d49/d5b 0 2024-04-20T11:09:03.603 INFO:tasks.workunit.client.0.smithi117.stdout:7/391: dwrite d1/d4/d6/d17/f3a [0,4194304] 0 2024-04-20T11:09:03.610 INFO:tasks.workunit.client.0.smithi117.stdout:6/442: dwrite d3/df/f4f [0,4194304] 0 2024-04-20T11:09:03.612 INFO:tasks.workunit.client.0.smithi117.stdout:3/374: dread d0/d5/de/d1d/d29/d66/f6c [0,4194304] 0 2024-04-20T11:09:03.625 INFO:tasks.workunit.client.0.smithi117.stdout:5/523: dread de/d16/f66 [0,4194304] 0 2024-04-20T11:09:03.630 INFO:tasks.workunit.client.0.smithi117.stdout:8/274: write d0/d8/d11/d24/d2b/f59 [849572,125107] 0 2024-04-20T11:09:03.632 INFO:tasks.workunit.client.0.smithi117.stdout:9/414: dread d4/d35/f5f [0,4194304] 0 2024-04-20T11:09:03.635 INFO:tasks.workunit.client.0.smithi117.stdout:9/415: chown d4/d4a/d72/f8e 174844 1 2024-04-20T11:09:03.636 INFO:tasks.workunit.client.0.smithi117.stdout:3/375: dwrite d0/d5/de/d1d/d36/f33 [0,4194304] 0 2024-04-20T11:09:03.638 INFO:tasks.workunit.client.0.smithi117.stdout:6/443: write d3/f18 [892695,78624] 0 2024-04-20T11:09:03.644 INFO:tasks.workunit.client.0.smithi117.stdout:4/413: rename d1/d45/d4a/d59/f6a to d1/dd/d1a/f8b 0 2024-04-20T11:09:03.658 INFO:tasks.workunit.client.0.smithi117.stdout:7/392: dwrite d1/d1d/d42/d2d/f7f [0,4194304] 0 2024-04-20T11:09:03.660 INFO:tasks.workunit.client.0.smithi117.stdout:7/393: fsync d1/d1d/f5c 0 2024-04-20T11:09:03.667 INFO:tasks.workunit.client.0.smithi117.stdout:0/370: truncate d3/d5/f37 52675 0 2024-04-20T11:09:03.670 INFO:tasks.workunit.client.0.smithi117.stdout:5/524: dread de/d16/f2d [0,4194304] 0 2024-04-20T11:09:03.671 INFO:tasks.workunit.client.0.smithi117.stdout:4/414: creat d1/d9/d79/f8c x:0 0 0 2024-04-20T11:09:03.673 INFO:tasks.workunit.client.0.smithi117.stdout:8/275: mkdir d0/d8/d39/d5a/d5c 0 2024-04-20T11:09:03.674 INFO:tasks.workunit.client.0.smithi117.stdout:8/276: readlink d0/d8/l2f 0 2024-04-20T11:09:03.675 INFO:tasks.workunit.client.0.smithi117.stdout:2/437: getdents d1/d4/dd 0 2024-04-20T11:09:03.677 INFO:tasks.workunit.client.0.smithi117.stdout:6/444: mkdir d3/d4/d2e/d37/d8e 0 2024-04-20T11:09:03.684 INFO:tasks.workunit.client.0.smithi117.stdout:7/394: creat d1/d4/d6/d2b/f81 x:0 0 0 2024-04-20T11:09:03.687 INFO:tasks.workunit.client.0.smithi117.stdout:7/395: write d1/d4/d6/d17/d3f/f65 [3978496,23656] 0 2024-04-20T11:09:03.688 INFO:tasks.workunit.client.0.smithi117.stdout:8/277: dread d0/dc/f25 [0,4194304] 0 2024-04-20T11:09:03.688 INFO:tasks.workunit.client.0.smithi117.stdout:5/525: rmdir de/d41/d47 39 2024-04-20T11:09:03.689 INFO:tasks.workunit.client.0.smithi117.stdout:4/415: dread - d1/dd/f68 zero size 2024-04-20T11:09:03.690 INFO:tasks.workunit.client.0.smithi117.stdout:0/371: dwrite d3/d7/d8/df/d6d/f71 [0,4194304] 0 2024-04-20T11:09:03.692 INFO:tasks.workunit.client.0.smithi117.stdout:4/416: dread - d1/d45/f4f zero size 2024-04-20T11:09:03.693 INFO:tasks.workunit.client.0.smithi117.stdout:2/438: dread d1/d4/f45 [0,4194304] 0 2024-04-20T11:09:03.696 INFO:tasks.workunit.client.0.smithi117.stdout:3/376: rename d0/d5/d7/f1e to d0/d5/d7/f87 0 2024-04-20T11:09:03.698 INFO:tasks.workunit.client.0.smithi117.stdout:7/396: truncate d1/d4/d6/d2b/d64/f78 1003722 0 2024-04-20T11:09:03.701 INFO:tasks.workunit.client.0.smithi117.stdout:2/439: read d1/d4/d83/d29/d3f/f76 [136089,52129] 0 2024-04-20T11:09:03.703 INFO:tasks.workunit.client.0.smithi117.stdout:6/445: creat d3/df/d39/f8f x:0 0 0 2024-04-20T11:09:03.705 INFO:tasks.workunit.client.0.smithi117.stdout:1/376: rename d6/d21/f39 to d6/f79 0 2024-04-20T11:09:03.705 INFO:tasks.workunit.client.0.smithi117.stdout:1/377: stat d6/f3c 0 2024-04-20T11:09:03.706 INFO:tasks.workunit.client.0.smithi117.stdout:9/416: getdents d4/d30 0 2024-04-20T11:09:03.706 INFO:tasks.workunit.client.0.smithi117.stdout:2/440: fdatasync d1/d4/dd/f77 0 2024-04-20T11:09:03.710 INFO:tasks.workunit.client.0.smithi117.stdout:2/441: readlink d1/d4/d83/d29/d2e/l3c 0 2024-04-20T11:09:03.711 INFO:tasks.workunit.client.0.smithi117.stdout:1/378: read d6/d1f/d30/d32/f6a [4141714,81747] 0 2024-04-20T11:09:03.712 INFO:tasks.workunit.client.0.smithi117.stdout:8/278: creat d0/d8/d11/d24/d2b/f5d x:0 0 0 2024-04-20T11:09:03.713 INFO:tasks.workunit.client.0.smithi117.stdout:4/417: mknod d1/dd/d1a/d43/c8d 0 2024-04-20T11:09:03.716 INFO:tasks.workunit.client.0.smithi117.stdout:0/372: dread d3/d7/d16/f2e [0,4194304] 0 2024-04-20T11:09:03.718 INFO:tasks.workunit.client.0.smithi117.stdout:0/373: dread - d3/d7/f7d zero size 2024-04-20T11:09:03.728 INFO:tasks.workunit.client.0.smithi117.stdout:3/377: rename d0/d5/de/d1d to d0/d5/d21/d88 0 2024-04-20T11:09:03.731 INFO:tasks.workunit.client.0.smithi117.stdout:9/417: dwrite d4/d35/f81 [0,4194304] 0 2024-04-20T11:09:03.731 INFO:tasks.workunit.client.0.smithi117.stdout:6/446: mknod d3/d4/d2e/d55/c90 0 2024-04-20T11:09:03.731 INFO:tasks.workunit.client.0.smithi117.stdout:4/418: dwrite d1/d9/f76 [0,4194304] 0 2024-04-20T11:09:03.739 INFO:tasks.workunit.client.0.smithi117.stdout:5/526: truncate de/d41/d47/d4c/f89 1296789 0 2024-04-20T11:09:03.739 INFO:tasks.workunit.client.0.smithi117.stdout:0/374: dread d3/d7/d20/d39/f62 [0,4194304] 0 2024-04-20T11:09:03.748 INFO:tasks.workunit.client.0.smithi117.stdout:1/379: symlink d6/d21/l7a 0 2024-04-20T11:09:03.753 INFO:tasks.workunit.client.0.smithi117.stdout:8/279: symlink d0/d8/d11/d24/d27/d47/l5e 0 2024-04-20T11:09:03.754 INFO:tasks.workunit.client.0.smithi117.stdout:8/280: read - d0/f57 zero size 2024-04-20T11:09:03.761 INFO:tasks.workunit.client.0.smithi117.stdout:4/419: unlink d1/c18 0 2024-04-20T11:09:03.769 INFO:tasks.workunit.client.0.smithi117.stdout:6/447: rename d3/d3a/d7f/l83 to d3/df/d39/l91 0 2024-04-20T11:09:03.771 INFO:tasks.workunit.client.0.smithi117.stdout:4/420: read d1/dd/d1a/d43/d4e/f62 [800482,98770] 0 2024-04-20T11:09:03.772 INFO:tasks.workunit.client.0.smithi117.stdout:2/442: creat d1/d4/d69/f8c x:0 0 0 2024-04-20T11:09:03.772 INFO:tasks.workunit.client.0.smithi117.stdout:0/375: creat d3/d7/f80 x:0 0 0 2024-04-20T11:09:03.774 INFO:tasks.workunit.client.0.smithi117.stdout:1/380: rmdir d6/d16/d3a 39 2024-04-20T11:09:03.776 INFO:tasks.workunit.client.0.smithi117.stdout:0/376: readlink d3/d7/d20/d47/l4f 0 2024-04-20T11:09:03.779 INFO:tasks.workunit.client.0.smithi117.stdout:3/378: mkdir d0/d5/d21/d75/d89 0 2024-04-20T11:09:03.780 INFO:tasks.workunit.client.0.smithi117.stdout:0/377: dread d3/d7/f3f [0,4194304] 0 2024-04-20T11:09:03.782 INFO:tasks.workunit.client.0.smithi117.stdout:0/378: chown d3/d5/f5a 120 1 2024-04-20T11:09:03.791 INFO:tasks.workunit.client.0.smithi117.stdout:5/527: rename de/d41/d4b/la8 to de/d16/laf 0 2024-04-20T11:09:03.792 INFO:tasks.workunit.client.0.smithi117.stdout:0/379: dread d3/d7/d8/f44 [0,4194304] 0 2024-04-20T11:09:03.793 INFO:tasks.workunit.client.0.smithi117.stdout:7/397: truncate d1/f10 3573901 0 2024-04-20T11:09:03.795 INFO:tasks.workunit.client.0.smithi117.stdout:1/381: write d6/d35/f5f [2988225,101473] 0 2024-04-20T11:09:03.798 INFO:tasks.workunit.client.0.smithi117.stdout:3/379: chown d0/d5/d7/c79 899554 1 2024-04-20T11:09:03.801 INFO:tasks.workunit.client.0.smithi117.stdout:3/380: write d0/d5/d7/f83 [734440,129382] 0 2024-04-20T11:09:03.803 INFO:tasks.workunit.client.0.smithi117.stdout:1/382: dread d6/d1f/f40 [0,4194304] 0 2024-04-20T11:09:03.804 INFO:tasks.workunit.client.0.smithi117.stdout:8/281: rename d0/f3 to d0/d8/d39/d40/f5f 0 2024-04-20T11:09:03.805 INFO:tasks.workunit.client.0.smithi117.stdout:0/380: unlink d3/d7/d8/c42 0 2024-04-20T11:09:03.807 INFO:tasks.workunit.client.0.smithi117.stdout:7/398: symlink d1/d4/d3d/l82 0 2024-04-20T11:09:03.810 INFO:tasks.workunit.client.0.smithi117.stdout:7/399: chown d1/d1d/d4e/d72 237083306 1 2024-04-20T11:09:03.810 INFO:tasks.workunit.client.0.smithi117.stdout:5/528: unlink de/c3f 0 2024-04-20T11:09:03.814 INFO:tasks.workunit.client.0.smithi117.stdout:0/381: truncate d3/f38 522158 0 2024-04-20T11:09:03.814 INFO:tasks.workunit.client.0.smithi117.stdout:6/448: link d3/df/d39/l54 d3/d4/d2e/d4e/d71/l92 0 2024-04-20T11:09:03.817 INFO:tasks.workunit.client.0.smithi117.stdout:7/400: creat d1/d4/d39/f83 x:0 0 0 2024-04-20T11:09:03.818 INFO:tasks.workunit.client.0.smithi117.stdout:1/383: creat d6/d16/d3a/d5a/d5c/f7b x:0 0 0 2024-04-20T11:09:03.821 INFO:tasks.workunit.client.0.smithi117.stdout:5/529: creat de/d16/d9d/fb0 x:0 0 0 2024-04-20T11:09:03.825 INFO:tasks.workunit.client.0.smithi117.stdout:9/418: truncate d4/d30/f4e 1817239 0 2024-04-20T11:09:03.826 INFO:tasks.workunit.client.0.smithi117.stdout:7/401: mknod d1/d1d/d4e/d72/c84 0 2024-04-20T11:09:03.827 INFO:tasks.workunit.client.0.smithi117.stdout:9/419: stat d4/d68/d6e/f77 0 2024-04-20T11:09:03.828 INFO:tasks.workunit.client.0.smithi117.stdout:6/449: creat d3/d5a/d74/f93 x:0 0 0 2024-04-20T11:09:03.836 INFO:tasks.workunit.client.0.smithi117.stdout:0/382: rename d3/d7/d8/df/d6d/c73 to d3/d7e/c81 0 2024-04-20T11:09:03.841 INFO:tasks.workunit.client.0.smithi117.stdout:9/420: unlink d4/d4a/c79 0 2024-04-20T11:09:03.842 INFO:tasks.workunit.client.0.smithi117.stdout:5/530: dwrite de/f19 [0,4194304] 0 2024-04-20T11:09:03.847 INFO:tasks.workunit.client.0.smithi117.stdout:1/384: getdents d6/d1f/d30/d49/d64 0 2024-04-20T11:09:03.851 INFO:tasks.workunit.client.0.smithi117.stdout:6/450: mkdir d3/d94 0 2024-04-20T11:09:03.851 INFO:tasks.workunit.client.0.smithi117.stdout:7/402: dwrite d1/d4/d6/d2b/d64/f78 [0,4194304] 0 2024-04-20T11:09:03.855 INFO:tasks.workunit.client.0.smithi117.stdout:5/531: creat de/d41/d47/d4c/fb1 x:0 0 0 2024-04-20T11:09:03.867 INFO:tasks.workunit.client.0.smithi117.stdout:5/532: write de/d41/d47/f6f [4622185,83134] 0 2024-04-20T11:09:03.872 INFO:tasks.workunit.client.0.smithi117.stdout:4/421: dwrite d1/dd/f38 [0,4194304] 0 2024-04-20T11:09:03.876 INFO:tasks.workunit.client.0.smithi117.stdout:7/403: creat d1/d4/d6/d54/d63/f85 x:0 0 0 2024-04-20T11:09:03.882 INFO:tasks.workunit.client.0.smithi117.stdout:6/451: creat d3/d5a/d79/f95 x:0 0 0 2024-04-20T11:09:03.883 INFO:tasks.workunit.client.0.smithi117.stdout:8/282: dread d0/d8/d39/d40/f5f [0,4194304] 0 2024-04-20T11:09:03.889 INFO:tasks.workunit.client.0.smithi117.stdout:0/383: dwrite d3/d7/f3f [0,4194304] 0 2024-04-20T11:09:03.889 INFO:tasks.workunit.client.0.smithi117.stdout:5/533: symlink de/d21/d51/lb2 0 2024-04-20T11:09:03.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:03 smithi117 ceph-mon[15659]: pgmap v11: 65 pgs: 65 active+clean; 588 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 69 MiB/s wr, 204 op/s 2024-04-20T11:09:03.890 INFO:tasks.workunit.client.0.smithi117.stdout:4/422: fdatasync d1/d9/d1f/f36 0 2024-04-20T11:09:03.891 INFO:tasks.workunit.client.0.smithi117.stdout:9/421: dwrite d4/d4a/d63/f76 [0,4194304] 0 2024-04-20T11:09:03.906 INFO:tasks.workunit.client.0.smithi117.stdout:1/385: link d6/d16/c17 d6/d1f/d30/d49/d69/d6e/d6f/d73/c7c 0 2024-04-20T11:09:03.907 INFO:tasks.workunit.client.0.smithi117.stdout:4/423: rmdir d1 39 2024-04-20T11:09:03.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:03 smithi173 ceph-mon[23800]: pgmap v11: 65 pgs: 65 active+clean; 588 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 69 MiB/s wr, 204 op/s 2024-04-20T11:09:03.913 INFO:tasks.workunit.client.0.smithi117.stdout:8/283: rename d0/d8/d11/d24/d27/d47/l5e to d0/dc/d2e/l60 0 2024-04-20T11:09:03.920 INFO:tasks.workunit.client.0.smithi117.stdout:8/284: chown d0/d8/d11/d24/d27/d47/l4c 126253 1 2024-04-20T11:09:03.920 INFO:tasks.workunit.client.0.smithi117.stdout:4/424: dread - d1/dd/d1a/d43/d4e/f73 zero size 2024-04-20T11:09:03.922 INFO:tasks.workunit.client.0.smithi117.stdout:9/422: truncate d4/d30/f6b 4127298 0 2024-04-20T11:09:03.928 INFO:tasks.workunit.client.0.smithi117.stdout:9/423: chown d4/d68/d6e 17536759 1 2024-04-20T11:09:03.930 INFO:tasks.workunit.client.0.smithi117.stdout:0/384: dwrite d3/d7/d8/df/d22/f75 [0,4194304] 0 2024-04-20T11:09:03.934 INFO:tasks.workunit.client.0.smithi117.stdout:7/404: dwrite d1/d43/d59/f69 [4194304,4194304] 0 2024-04-20T11:09:03.935 INFO:tasks.workunit.client.0.smithi117.stdout:5/534: dwrite de/d21/d45/d77/f7d [0,4194304] 0 2024-04-20T11:09:03.936 INFO:tasks.workunit.client.0.smithi117.stdout:4/425: unlink d1/dd/d1a/d43/f60 0 2024-04-20T11:09:03.938 INFO:tasks.workunit.client.0.smithi117.stdout:9/424: symlink d4/d35/l95 0 2024-04-20T11:09:03.942 INFO:tasks.workunit.client.0.smithi117.stdout:6/452: dread d3/d3a/d7f/f7a [0,4194304] 0 2024-04-20T11:09:03.949 INFO:tasks.workunit.client.0.smithi117.stdout:0/385: write d3/d7/d2c/f6b [365053,105843] 0 2024-04-20T11:09:03.949 INFO:tasks.workunit.client.0.smithi117.stdout:4/426: mknod d1/d45/c8e 0 2024-04-20T11:09:03.950 INFO:tasks.workunit.client.0.smithi117.stdout:5/535: symlink de/d5b/d65/lb3 0 2024-04-20T11:09:03.957 INFO:tasks.workunit.client.0.smithi117.stdout:5/536: mkdir de/d16/d9d/db4 0 2024-04-20T11:09:03.962 INFO:tasks.workunit.client.0.smithi117.stdout:4/427: rename d1/dd/d1a/f37 to d1/dd/d1a/d2b/f8f 0 2024-04-20T11:09:03.968 INFO:tasks.workunit.client.0.smithi117.stdout:7/405: getdents d1/d1d/d77 0 2024-04-20T11:09:03.968 INFO:tasks.workunit.client.0.smithi117.stdout:7/406: fsync d1/d4/f30 0 2024-04-20T11:09:03.969 INFO:tasks.workunit.client.0.smithi117.stdout:6/453: unlink d3/d4/c2a 0 2024-04-20T11:09:03.972 INFO:tasks.workunit.client.0.smithi117.stdout:4/428: mkdir d1/d9/d1f/d90 0 2024-04-20T11:09:03.977 INFO:tasks.workunit.client.0.smithi117.stdout:8/285: truncate d0/d8/d11/d24/d2b/f59 2197754 0 2024-04-20T11:09:03.977 INFO:tasks.workunit.client.0.smithi117.stdout:9/425: write d4/d5/f21 [554807,47119] 0 2024-04-20T11:09:03.979 INFO:tasks.workunit.client.0.smithi117.stdout:5/537: symlink de/d21/d45/d48/d4e/da1/lb5 0 2024-04-20T11:09:03.983 INFO:tasks.workunit.client.0.smithi117.stdout:8/286: chown d0/d8/d11/d24/d2b/f5d 0 1 2024-04-20T11:09:03.985 INFO:tasks.workunit.client.0.smithi117.stdout:7/407: rename d1/d1d/f21 to d1/f86 0 2024-04-20T11:09:03.987 INFO:tasks.workunit.client.0.smithi117.stdout:9/426: write d4/d30/f36 [3018140,90294] 0 2024-04-20T11:09:03.996 INFO:tasks.workunit.client.0.smithi117.stdout:5/538: rename de/d21/f2e to de/d41/fb6 0 2024-04-20T11:09:03.998 INFO:tasks.workunit.client.0.smithi117.stdout:6/454: dwrite d3/d5a/d79/f6f [0,4194304] 0 2024-04-20T11:09:04.000 INFO:tasks.workunit.client.0.smithi117.stdout:0/386: dwrite d3/d5/f37 [0,4194304] 0 2024-04-20T11:09:04.003 INFO:tasks.workunit.client.0.smithi117.stdout:7/408: dread d1/f7 [4194304,4194304] 0 2024-04-20T11:09:04.006 INFO:tasks.workunit.client.0.smithi117.stdout:9/427: rename d4/d70 to d4/de/d7b/d96 0 2024-04-20T11:09:04.009 INFO:tasks.workunit.client.0.smithi117.stdout:5/539: rmdir de/d41/d47/d4c 39 2024-04-20T11:09:04.011 INFO:tasks.workunit.client.0.smithi117.stdout:8/287: dwrite d0/d9/f33 [0,4194304] 0 2024-04-20T11:09:04.019 INFO:tasks.workunit.client.0.smithi117.stdout:7/409: creat d1/d1d/f87 x:0 0 0 2024-04-20T11:09:04.023 INFO:tasks.workunit.client.0.smithi117.stdout:9/428: mknod d4/de/d10/d94/c97 0 2024-04-20T11:09:04.026 INFO:tasks.workunit.client.0.smithi117.stdout:4/429: dread d1/dd/d1a/f1d [0,4194304] 0 2024-04-20T11:09:04.027 INFO:tasks.workunit.client.0.smithi117.stdout:5/540: mkdir de/db7 0 2024-04-20T11:09:04.029 INFO:tasks.workunit.client.0.smithi117.stdout:8/288: mknod d0/c61 0 2024-04-20T11:09:04.030 INFO:tasks.workunit.client.0.smithi117.stdout:8/289: stat d0/c61 0 2024-04-20T11:09:04.033 INFO:tasks.workunit.client.0.smithi117.stdout:7/410: dread d1/d4/d6/d13/f15 [0,4194304] 0 2024-04-20T11:09:04.036 INFO:tasks.workunit.client.0.smithi117.stdout:4/430: rename d1/f72 to d1/dd/d1a/d2b/d86/f91 0 2024-04-20T11:09:04.039 INFO:tasks.workunit.client.0.smithi117.stdout:6/455: dread d3/df/f14 [0,4194304] 0 2024-04-20T11:09:04.041 INFO:tasks.workunit.client.0.smithi117.stdout:7/411: creat d1/d1d/d42/d2d/d6c/f88 x:0 0 0 2024-04-20T11:09:04.044 INFO:tasks.workunit.client.0.smithi117.stdout:7/412: mkdir d1/d1d/d42/d89 0 2024-04-20T11:09:04.054 INFO:tasks.workunit.client.0.smithi117.stdout:5/541: dwrite de/d41/d47/f6f [4194304,4194304] 0 2024-04-20T11:09:04.062 INFO:tasks.workunit.client.0.smithi117.stdout:5/542: stat de/d21/d45/d48/d4e/da1 0 2024-04-20T11:09:04.065 INFO:tasks.workunit.client.0.smithi117.stdout:7/413: dread d1/f86 [8388608,4194304] 0 2024-04-20T11:09:04.067 INFO:tasks.workunit.client.0.smithi117.stdout:5/543: fdatasync de/d5b/d65/fa2 0 2024-04-20T11:09:04.067 INFO:tasks.workunit.client.0.smithi117.stdout:9/429: dwrite d4/d5/f1e [4194304,4194304] 0 2024-04-20T11:09:04.069 INFO:tasks.workunit.client.0.smithi117.stdout:8/290: dwrite d0/d8/d11/d24/f46 [0,4194304] 0 2024-04-20T11:09:04.074 INFO:tasks.workunit.client.0.smithi117.stdout:7/414: mkdir d1/d43/d59/d8a 0 2024-04-20T11:09:04.079 INFO:tasks.workunit.client.0.smithi117.stdout:5/544: rmdir de/d21/d45/d48 39 2024-04-20T11:09:04.079 INFO:tasks.workunit.client.0.smithi117.stdout:7/415: write d1/d4/d39/f7a [392993,93183] 0 2024-04-20T11:09:04.085 INFO:tasks.workunit.client.0.smithi117.stdout:7/416: fdatasync d1/d43/f80 0 2024-04-20T11:09:04.088 INFO:tasks.workunit.client.0.smithi117.stdout:8/291: rename d0/d8/c1d to d0/d8/d39/d5a/d5c/c62 0 2024-04-20T11:09:04.090 INFO:tasks.workunit.client.0.smithi117.stdout:8/292: readlink d0/d8/d11/l36 0 2024-04-20T11:09:04.091 INFO:tasks.workunit.client.0.smithi117.stdout:9/430: mkdir d4/d4a/d63/d98 0 2024-04-20T11:09:04.096 INFO:tasks.workunit.client.0.smithi117.stdout:4/431: dread d1/d9/d1f/f36 [0,4194304] 0 2024-04-20T11:09:04.096 INFO:tasks.workunit.client.0.smithi117.stdout:5/545: mkdir de/d21/d84/db8 0 2024-04-20T11:09:04.100 INFO:tasks.workunit.client.0.smithi117.stdout:9/431: rename d4/d4a/f60 to d4/d5/d57/f99 0 2024-04-20T11:09:04.102 INFO:tasks.workunit.client.0.smithi117.stdout:8/293: dread d0/d8/d11/d24/d2b/f2c [0,4194304] 0 2024-04-20T11:09:04.104 INFO:tasks.workunit.client.0.smithi117.stdout:7/417: getdents d1/d1d/d42/d89 0 2024-04-20T11:09:04.106 INFO:tasks.workunit.client.0.smithi117.stdout:7/418: dread - d1/d4/d6/d54/d63/f85 zero size 2024-04-20T11:09:04.110 INFO:tasks.workunit.client.0.smithi117.stdout:4/432: mknod d1/dd/d1a/d2b/c92 0 2024-04-20T11:09:04.111 INFO:tasks.workunit.client.0.smithi117.stdout:8/294: mkdir d0/dc/d49/d63 0 2024-04-20T11:09:04.112 INFO:tasks.workunit.client.0.smithi117.stdout:5/546: rmdir de/d21/d45/d48/d4e/da1 39 2024-04-20T11:09:04.114 INFO:tasks.workunit.client.0.smithi117.stdout:7/419: creat d1/d1d/d42/f8b x:0 0 0 2024-04-20T11:09:04.116 INFO:tasks.workunit.client.0.smithi117.stdout:7/420: readlink d1/d4/d6/d54/d63/l68 0 2024-04-20T11:09:04.117 INFO:tasks.workunit.client.0.smithi117.stdout:5/547: truncate de/d41/d47/d55/d56/f79 22431 0 2024-04-20T11:09:04.118 INFO:tasks.workunit.client.0.smithi117.stdout:5/548: write de/d21/f96 [915001,56068] 0 2024-04-20T11:09:04.119 INFO:tasks.workunit.client.0.smithi117.stdout:7/421: creat d1/d4/d6/d2b/d64/f8c x:0 0 0 2024-04-20T11:09:04.122 INFO:tasks.workunit.client.0.smithi117.stdout:5/549: stat de/d41/d47/d4c 0 2024-04-20T11:09:04.122 INFO:tasks.workunit.client.0.smithi117.stdout:7/422: chown d1/d1d/d42/d2d/d6c/f88 8194257 1 2024-04-20T11:09:04.126 INFO:tasks.workunit.client.0.smithi117.stdout:7/423: creat d1/d43/f8d x:0 0 0 2024-04-20T11:09:04.130 INFO:tasks.workunit.client.0.smithi117.stdout:7/424: mknod d1/d4/d6/c8e 0 2024-04-20T11:09:04.130 INFO:tasks.workunit.client.0.smithi117.stdout:9/432: write d4/d30/f6b [2371109,97125] 0 2024-04-20T11:09:04.132 INFO:tasks.workunit.client.0.smithi117.stdout:8/295: write d0/dc/f25 [1073930,89512] 0 2024-04-20T11:09:04.132 INFO:tasks.workunit.client.0.smithi117.stdout:5/550: write de/d21/d35/f3a [947118,65595] 0 2024-04-20T11:09:04.133 INFO:tasks.workunit.client.0.smithi117.stdout:7/425: write d1/d1d/f87 [987537,19088] 0 2024-04-20T11:09:04.134 INFO:tasks.workunit.client.0.smithi117.stdout:8/296: fdatasync d0/fa 0 2024-04-20T11:09:04.136 INFO:tasks.workunit.client.0.smithi117.stdout:7/426: readlink d1/d4/d6/d13/l37 0 2024-04-20T11:09:04.138 INFO:tasks.workunit.client.0.smithi117.stdout:5/551: write de/d21/d45/d77/f85 [3659574,67460] 0 2024-04-20T11:09:04.143 INFO:tasks.workunit.client.0.smithi117.stdout:5/552: mkdir de/d21/d45/db9 0 2024-04-20T11:09:04.145 INFO:tasks.workunit.client.0.smithi117.stdout:8/297: truncate d0/d8/d11/d24/d2b/f59 529344 0 2024-04-20T11:09:04.146 INFO:tasks.workunit.client.0.smithi117.stdout:5/553: chown de/d41/d47/d4c/f8f 4721928 1 2024-04-20T11:09:04.146 INFO:tasks.workunit.client.0.smithi117.stdout:9/433: truncate d4/d4a/d5e/f75 3866669 0 2024-04-20T11:09:04.149 INFO:tasks.workunit.client.0.smithi117.stdout:9/434: creat d4/d35/f9a x:0 0 0 2024-04-20T11:09:04.150 INFO:tasks.workunit.client.0.smithi117.stdout:8/298: mknod d0/dc/d2e/d3e/d58/c64 0 2024-04-20T11:09:04.160 INFO:tasks.workunit.client.0.smithi117.stdout:7/427: dwrite d1/d43/f73 [0,4194304] 0 2024-04-20T11:09:04.164 INFO:tasks.workunit.client.0.smithi117.stdout:8/299: truncate d0/d8/d11/d24/f46 3293131 0 2024-04-20T11:09:04.164 INFO:tasks.workunit.client.0.smithi117.stdout:5/554: dwrite de/d21/f31 [4194304,4194304] 0 2024-04-20T11:09:04.167 INFO:tasks.workunit.client.0.smithi117.stdout:5/555: readlink de/d21/d51/lb2 0 2024-04-20T11:09:04.173 INFO:tasks.workunit.client.0.smithi117.stdout:8/300: rename d0/d8/d11/f1c to d0/d8/f65 0 2024-04-20T11:09:04.178 INFO:tasks.workunit.client.0.smithi117.stdout:8/301: fdatasync d0/dc/f25 0 2024-04-20T11:09:04.178 INFO:tasks.workunit.client.0.smithi117.stdout:8/302: stat d0/dc/d2e/d3e/d58/c64 0 2024-04-20T11:09:04.180 INFO:tasks.workunit.client.0.smithi117.stdout:7/428: dread d1/d1d/d42/d2d/f7f [0,4194304] 0 2024-04-20T11:09:04.182 INFO:tasks.workunit.client.0.smithi117.stdout:7/429: rename d1/d4 to d1/d4/d3d/d8f 22 2024-04-20T11:09:04.183 INFO:tasks.workunit.client.0.smithi117.stdout:5/556: dread de/d41/fb6 [0,4194304] 0 2024-04-20T11:09:04.183 INFO:tasks.workunit.client.0.smithi117.stdout:8/303: creat d0/d8/d11/d24/f66 x:0 0 0 2024-04-20T11:09:04.185 INFO:tasks.workunit.client.0.smithi117.stdout:5/557: write de/f74 [1380522,31148] 0 2024-04-20T11:09:04.191 INFO:tasks.workunit.client.0.smithi117.stdout:7/430: rename d1/d1d/d42/d2d/f7f to d1/d4/d6/d17/d3f/f90 0 2024-04-20T11:09:04.194 INFO:tasks.workunit.client.0.smithi117.stdout:8/304: rename d0/d8/d11/d24/d27/d47/f55 to d0/dc/d49/d5b/f67 0 2024-04-20T11:09:04.196 INFO:tasks.workunit.client.0.smithi117.stdout:7/431: getdents d1/d1d/d77 0 2024-04-20T11:09:04.198 INFO:tasks.workunit.client.0.smithi117.stdout:7/432: chown d1/d4/d6/d13/f71 6657 1 2024-04-20T11:09:04.198 INFO:tasks.workunit.client.0.smithi117.stdout:8/305: unlink d0/d8/d11/d24/d2b/c35 0 2024-04-20T11:09:04.199 INFO:tasks.workunit.client.0.smithi117.stdout:5/558: dread de/d21/d45/d77/f7d [0,4194304] 0 2024-04-20T11:09:04.200 INFO:tasks.workunit.client.0.smithi117.stdout:7/433: readlink d1/d4/d3d/l82 0 2024-04-20T11:09:04.201 INFO:tasks.workunit.client.0.smithi117.stdout:5/559: mknod de/d21/cba 0 2024-04-20T11:09:04.202 INFO:tasks.workunit.client.0.smithi117.stdout:7/434: chown d1/d43/d59/l75 12 1 2024-04-20T11:09:04.205 INFO:tasks.workunit.client.0.smithi117.stdout:7/435: creat d1/d43/f91 x:0 0 0 2024-04-20T11:09:04.206 INFO:tasks.workunit.client.0.smithi117.stdout:7/436: dread - d1/d1d/d42/d2d/f5f zero size 2024-04-20T11:09:04.220 INFO:tasks.workunit.client.0.smithi117.stdout:5/560: dwrite de/d21/d45/d77/fa6 [0,4194304] 0 2024-04-20T11:09:04.224 INFO:tasks.workunit.client.0.smithi117.stdout:5/561: rmdir de/d16/d9d 39 2024-04-20T11:09:04.227 INFO:tasks.workunit.client.0.smithi117.stdout:5/562: mknod de/d21/d51/cbb 0 2024-04-20T11:09:04.229 INFO:tasks.workunit.client.0.smithi117.stdout:5/563: write de/d21/d35/f3a [2687329,79150] 0 2024-04-20T11:09:04.231 INFO:tasks.workunit.client.0.smithi117.stdout:5/564: fsync de/d16/f2d 0 2024-04-20T11:09:04.234 INFO:tasks.workunit.client.0.smithi117.stdout:5/565: write de/d16/f97 [2888224,65100] 0 2024-04-20T11:09:04.237 INFO:tasks.workunit.client.0.smithi117.stdout:5/566: symlink de/d21/d45/db9/lbc 0 2024-04-20T11:09:04.238 INFO:tasks.workunit.client.0.smithi117.stdout:5/567: chown de/f19 83161565 1 2024-04-20T11:09:04.240 INFO:tasks.workunit.client.0.smithi117.stdout:5/568: write de/d21/d45/d77/f7d [2457107,4621] 0 2024-04-20T11:09:04.242 INFO:tasks.workunit.client.0.smithi117.stdout:2/443: sync 2024-04-20T11:09:04.242 INFO:tasks.workunit.client.0.smithi117.stdout:5/569: creat de/db7/fbd x:0 0 0 2024-04-20T11:09:04.244 INFO:tasks.workunit.client.0.smithi117.stdout:5/570: dread - de/d21/d45/d77/fa5 zero size 2024-04-20T11:09:04.245 INFO:tasks.workunit.client.0.smithi117.stdout:2/444: readlink d1/d4/d83/d29/d62/l70 0 2024-04-20T11:09:04.258 INFO:tasks.workunit.client.0.smithi117.stdout:2/445: dwrite d1/d4/dd/f19 [0,4194304] 0 2024-04-20T11:09:04.261 INFO:tasks.workunit.client.0.smithi117.stdout:2/446: write d1/d4/dd/d17/d3b/f6c [985263,110262] 0 2024-04-20T11:09:04.297 INFO:tasks.workunit.client.0.smithi117.stdout:2/447: dread d1/d4/d83/f23 [0,4194304] 0 2024-04-20T11:09:04.299 INFO:tasks.workunit.client.0.smithi117.stdout:2/448: dread - d1/d4/d83/d29/d3f/f7d zero size 2024-04-20T11:09:04.411 INFO:tasks.workunit.client.0.smithi117.stdout:2/449: dwrite d1/d4/dd/d17/f1e [0,4194304] 0 2024-04-20T11:09:04.415 INFO:tasks.workunit.client.0.smithi117.stdout:2/450: dread - d1/d4/dd/f77 zero size 2024-04-20T11:09:04.419 INFO:tasks.workunit.client.0.smithi117.stdout:2/451: readlink d1/d4/d83/d29/d62/l70 0 2024-04-20T11:09:04.423 INFO:tasks.workunit.client.0.smithi117.stdout:2/452: write d1/d4/dd/d17/f1e [1802846,75304] 0 2024-04-20T11:09:04.443 INFO:tasks.workunit.client.0.smithi117.stdout:2/453: truncate d1/d4/dd/d3e/f56 3133580 0 2024-04-20T11:09:04.453 INFO:tasks.workunit.client.0.smithi117.stdout:2/454: rename d1/d15/c35 to d1/d4/d83/d29/d62/c8d 0 2024-04-20T11:09:04.454 INFO:tasks.workunit.client.0.smithi117.stdout:3/381: sync 2024-04-20T11:09:04.456 INFO:tasks.workunit.client.0.smithi117.stdout:1/386: sync 2024-04-20T11:09:04.457 INFO:tasks.workunit.client.0.smithi117.stdout:2/455: fdatasync d1/d4/d69/f8c 0 2024-04-20T11:09:04.459 INFO:tasks.workunit.client.0.smithi117.stdout:3/382: dread - d0/d5/d21/f4d zero size 2024-04-20T11:09:04.461 INFO:tasks.workunit.client.0.smithi117.stdout:3/383: chown d0/d2a 62436 1 2024-04-20T11:09:04.461 INFO:tasks.workunit.client.0.smithi117.stdout:1/387: dread - d6/d16/d3a/d5a/d5c/d61/d71/d6b/f6c zero size 2024-04-20T11:09:04.465 INFO:tasks.workunit.client.0.smithi117.stdout:2/456: chown d1/d4/d83/d29/d2e/f74 1919 1 2024-04-20T11:09:04.469 INFO:tasks.workunit.client.0.smithi117.stdout:1/388: mkdir d6/d16/d3a/d5a/d5c/d61/d71/d7d 0 2024-04-20T11:09:04.472 INFO:tasks.workunit.client.0.smithi117.stdout:3/384: creat d0/d5/d21/d77/f8a x:0 0 0 2024-04-20T11:09:04.477 INFO:tasks.workunit.client.0.smithi117.stdout:1/389: dread - d6/d16/d3a/d5a/d5c/d61/d71/d6b/f6c zero size 2024-04-20T11:09:04.477 INFO:tasks.workunit.client.0.smithi117.stdout:2/457: getdents d1/d15/d1d 0 2024-04-20T11:09:04.480 INFO:tasks.workunit.client.0.smithi117.stdout:3/385: fdatasync d0/d5/d21/d88/d29/d66/f6c 0 2024-04-20T11:09:04.481 INFO:tasks.workunit.client.0.smithi117.stdout:1/390: creat d6/d16/d3a/f7e x:0 0 0 2024-04-20T11:09:04.482 INFO:tasks.workunit.client.0.smithi117.stdout:2/458: mkdir d1/d4/d83/d8e 0 2024-04-20T11:09:04.487 INFO:tasks.workunit.client.0.smithi117.stdout:3/386: getdents d0/d5/d11/d80 0 2024-04-20T11:09:04.488 INFO:tasks.workunit.client.0.smithi117.stdout:2/459: fdatasync d1/d4/d83/f38 0 2024-04-20T11:09:04.491 INFO:tasks.workunit.client.0.smithi117.stdout:3/387: creat d0/d5/d73/f8b x:0 0 0 2024-04-20T11:09:04.493 INFO:tasks.workunit.client.0.smithi117.stdout:2/460: symlink d1/d4/d83/d29/l8f 0 2024-04-20T11:09:04.499 INFO:tasks.workunit.client.0.smithi117.stdout:2/461: write d1/d4/d83/f23 [2563269,54440] 0 2024-04-20T11:09:04.506 INFO:tasks.workunit.client.0.smithi117.stdout:2/462: getdents d1/d4/dd/d3e 0 2024-04-20T11:09:04.509 INFO:tasks.workunit.client.0.smithi117.stdout:3/388: dwrite d0/f3 [4194304,4194304] 0 2024-04-20T11:09:04.512 INFO:tasks.workunit.client.0.smithi117.stdout:1/391: write d6/d1f/d30/f74 [223623,79430] 0 2024-04-20T11:09:04.512 INFO:tasks.workunit.client.0.smithi117.stdout:3/389: chown d0/d5/d73/f8b 471270266 1 2024-04-20T11:09:04.529 INFO:tasks.workunit.client.0.smithi117.stdout:1/392: dwrite d6/d35/f36 [0,4194304] 0 2024-04-20T11:09:04.531 INFO:tasks.workunit.client.0.smithi117.stdout:3/390: dwrite d0/d5/d7/f83 [0,4194304] 0 2024-04-20T11:09:04.531 INFO:tasks.workunit.client.0.smithi117.stdout:2/463: write d1/f48 [909211,64844] 0 2024-04-20T11:09:04.537 INFO:tasks.workunit.client.0.smithi117.stdout:3/391: stat d0/d5/de/l5e 0 2024-04-20T11:09:04.546 INFO:tasks.workunit.client.0.smithi117.stdout:2/464: rename d1/d4/dd/d17/d3b/l47 to d1/d20/l90 0 2024-04-20T11:09:04.549 INFO:tasks.workunit.client.0.smithi117.stdout:2/465: chown d1/d4/dd/d17/l4f 232312734 1 2024-04-20T11:09:04.556 INFO:tasks.workunit.client.0.smithi117.stdout:3/392: dwrite d0/d5/de/d4e/f53 [0,4194304] 0 2024-04-20T11:09:04.562 INFO:tasks.workunit.client.0.smithi117.stdout:2/466: mknod d1/d20/c91 0 2024-04-20T11:09:04.564 INFO:tasks.workunit.client.0.smithi117.stdout:3/393: truncate d0/d5/f25 592264 0 2024-04-20T11:09:04.573 INFO:tasks.workunit.client.0.smithi117.stdout:3/394: link d0/d5/d7/dd/f5a d0/d5/d21/d88/d69/f8c 0 2024-04-20T11:09:04.588 INFO:tasks.workunit.client.0.smithi117.stdout:3/395: dread d0/d5/de/d4e/f53 [0,4194304] 0 2024-04-20T11:09:04.602 INFO:tasks.workunit.client.0.smithi117.stdout:3/396: dwrite d0/d5/d21/f4a [0,4194304] 0 2024-04-20T11:09:04.612 INFO:tasks.workunit.client.0.smithi117.stdout:3/397: creat d0/d5/d11/f8d x:0 0 0 2024-04-20T11:09:04.616 INFO:tasks.workunit.client.0.smithi117.stdout:3/398: creat d0/d5/d21/d88/d69/f8e x:0 0 0 2024-04-20T11:09:04.617 INFO:tasks.workunit.client.0.smithi117.stdout:3/399: creat d0/d5/d11/f8f x:0 0 0 2024-04-20T11:09:04.617 INFO:tasks.workunit.client.0.smithi117.stdout:3/400: fsync d0/d5/d21/d88/f44 0 2024-04-20T11:09:04.619 INFO:tasks.workunit.client.0.smithi117.stdout:2/467: dwrite d1/f3 [0,4194304] 0 2024-04-20T11:09:04.628 INFO:tasks.workunit.client.0.smithi117.stdout:2/468: symlink d1/d4/d69/l92 0 2024-04-20T11:09:04.629 INFO:tasks.workunit.client.0.smithi117.stdout:2/469: fdatasync d1/d4/dd/f19 0 2024-04-20T11:09:04.632 INFO:tasks.workunit.client.0.smithi117.stdout:3/401: dwrite d0/d5/d21/d88/f62 [0,4194304] 0 2024-04-20T11:09:04.639 INFO:tasks.workunit.client.0.smithi117.stdout:3/402: fsync d0/d5/d7/dd/f59 0 2024-04-20T11:09:04.640 INFO:tasks.workunit.client.0.smithi117.stdout:3/403: rmdir d0/d5/d21/d88/d3c 39 2024-04-20T11:09:04.641 INFO:tasks.workunit.client.0.smithi117.stdout:2/470: dwrite d1/d4/d83/f79 [0,4194304] 0 2024-04-20T11:09:04.653 INFO:tasks.workunit.client.0.smithi117.stdout:0/387: sync 2024-04-20T11:09:04.654 INFO:tasks.workunit.client.0.smithi117.stdout:3/404: getdents d0/d5/d21/d88/d36 0 2024-04-20T11:09:04.661 INFO:tasks.workunit.client.0.smithi117.stdout:0/388: fdatasync d3/d7/d20/f25 0 2024-04-20T11:09:04.671 INFO:tasks.workunit.client.0.smithi117.stdout:3/405: dwrite d0/d5/d21/f4a [0,4194304] 0 2024-04-20T11:09:04.676 INFO:tasks.workunit.client.0.smithi117.stdout:0/389: dread d3/d7/d8/df/d6d/f71 [0,4194304] 0 2024-04-20T11:09:04.680 INFO:tasks.workunit.client.0.smithi117.stdout:0/390: rename d3/d7/d20/l34 to d3/d7/d8/df/l82 0 2024-04-20T11:09:04.687 INFO:tasks.workunit.client.0.smithi117.stdout:2/471: dread d1/d4/dd/d3e/f56 [0,4194304] 0 2024-04-20T11:09:04.689 INFO:tasks.workunit.client.0.smithi117.stdout:2/472: mkdir d1/d4/d83/d29/d2e/d53/d93 0 2024-04-20T11:09:04.692 INFO:tasks.workunit.client.0.smithi117.stdout:3/406: dwrite d0/d5/d11/f7e [0,4194304] 0 2024-04-20T11:09:04.692 INFO:tasks.workunit.client.0.smithi117.stdout:2/473: chown d1/l3d 570315350 1 2024-04-20T11:09:04.694 INFO:tasks.workunit.client.0.smithi117.stdout:2/474: chown d1/d4/dd/d17/f1e 763 1 2024-04-20T11:09:04.696 INFO:tasks.workunit.client.0.smithi117.stdout:3/407: truncate d0/d5/d7/dd/f56 3494068 0 2024-04-20T11:09:04.698 INFO:tasks.workunit.client.0.smithi117.stdout:2/475: symlink d1/d4/dd/l94 0 2024-04-20T11:09:04.700 INFO:tasks.workunit.client.0.smithi117.stdout:2/476: truncate d1/d4/f7 4624035 0 2024-04-20T11:09:04.703 INFO:tasks.workunit.client.0.smithi117.stdout:3/408: creat d0/d5/d21/d88/d3c/f90 x:0 0 0 2024-04-20T11:09:04.707 INFO:tasks.workunit.client.0.smithi117.stdout:3/409: rmdir d0/d5/d7/dd 39 2024-04-20T11:09:04.709 INFO:tasks.workunit.client.0.smithi117.stdout:2/477: link d1/d4/dd/d3e/f56 d1/d4/dd/d17/d7a/f95 0 2024-04-20T11:09:04.712 INFO:tasks.workunit.client.0.smithi117.stdout:2/478: unlink d1/l4a 0 2024-04-20T11:09:04.714 INFO:tasks.workunit.client.0.smithi117.stdout:0/391: dread d3/d7/f3b [0,4194304] 0 2024-04-20T11:09:04.714 INFO:tasks.workunit.client.0.smithi117.stdout:2/479: read d1/d4/f8 [3825645,32836] 0 2024-04-20T11:09:04.719 INFO:tasks.workunit.client.0.smithi117.stdout:2/480: chown d1/d4/d83/d29/d2e/d52 49 1 2024-04-20T11:09:04.721 INFO:tasks.workunit.client.0.smithi117.stdout:2/481: readlink d1/d4/d83/d29/d2e/d52/l86 0 2024-04-20T11:09:04.724 INFO:tasks.workunit.client.0.smithi117.stdout:2/482: creat d1/d4/d83/d8e/f96 x:0 0 0 2024-04-20T11:09:04.726 INFO:tasks.workunit.client.0.smithi117.stdout:2/483: creat d1/d4/d83/f97 x:0 0 0 2024-04-20T11:09:04.729 INFO:tasks.workunit.client.0.smithi117.stdout:2/484: creat d1/d4/dd/d3e/f98 x:0 0 0 2024-04-20T11:09:04.733 INFO:tasks.workunit.client.0.smithi117.stdout:2/485: creat d1/d4/d83/d29/d2e/d53/d93/f99 x:0 0 0 2024-04-20T11:09:04.742 INFO:tasks.workunit.client.0.smithi117.stdout:2/486: dread d1/f6 [0,4194304] 0 2024-04-20T11:09:04.746 INFO:tasks.workunit.client.0.smithi117.stdout:2/487: rename d1/d4/f45 to d1/d4/d83/d29/d62/f9a 0 2024-04-20T11:09:04.748 INFO:tasks.workunit.client.0.smithi117.stdout:2/488: fdatasync d1/d4/dd/f14 0 2024-04-20T11:09:04.750 INFO:tasks.workunit.client.0.smithi117.stdout:2/489: mkdir d1/d9b 0 2024-04-20T11:09:04.760 INFO:tasks.workunit.client.0.smithi117.stdout:2/490: dread d1/d4/f5 [0,4194304] 0 2024-04-20T11:09:04.762 INFO:tasks.workunit.client.0.smithi117.stdout:2/491: mknod d1/d4/dd/d17/d7a/d88/c9c 0 2024-04-20T11:09:04.768 INFO:tasks.workunit.client.0.smithi117.stdout:2/492: fdatasync d1/d4/dd/d17/d7a/f95 0 2024-04-20T11:09:04.773 INFO:tasks.workunit.client.0.smithi117.stdout:2/493: unlink d1/d4/f7 0 2024-04-20T11:09:04.774 INFO:tasks.workunit.client.0.smithi117.stdout:2/494: chown d1/d9b 410 1 2024-04-20T11:09:04.792 INFO:tasks.workunit.client.0.smithi117.stdout:2/495: unlink d1/d4/dd/d17/c6d 0 2024-04-20T11:09:04.792 INFO:tasks.workunit.client.0.smithi117.stdout:2/496: chown d1/d4/d83/f34 113175 1 2024-04-20T11:09:04.792 INFO:tasks.workunit.client.0.smithi117.stdout:2/497: creat d1/d4/d83/d29/d2e/d53/f9d x:0 0 0 2024-04-20T11:09:04.793 INFO:tasks.workunit.client.0.smithi117.stdout:2/498: write d1/d4/dd/d3e/f98 [67240,15526] 0 2024-04-20T11:09:04.793 INFO:tasks.workunit.client.0.smithi117.stdout:2/499: fdatasync d1/d4/d83/f97 0 2024-04-20T11:09:04.795 INFO:tasks.workunit.client.0.smithi117.stdout:2/500: chown d1/d15/l1b 1 1 2024-04-20T11:09:04.795 INFO:tasks.workunit.client.0.smithi117.stdout:2/501: stat d1/d20/c91 0 2024-04-20T11:09:04.799 INFO:tasks.workunit.client.0.smithi117.stdout:2/502: creat d1/d4/d83/d29/d2e/d53/d93/f9e x:0 0 0 2024-04-20T11:09:04.802 INFO:tasks.workunit.client.0.smithi117.stdout:2/503: truncate d1/d4/d83/d29/d2e/f74 593763 0 2024-04-20T11:09:04.804 INFO:tasks.workunit.client.0.smithi117.stdout:2/504: write d1/d15/f72 [777578,77285] 0 2024-04-20T11:09:04.807 INFO:tasks.workunit.client.0.smithi117.stdout:2/505: mknod d1/d4/dd/c9f 0 2024-04-20T11:09:04.809 INFO:tasks.workunit.client.0.smithi117.stdout:2/506: dread - d1/d4/dd/f77 zero size 2024-04-20T11:09:04.814 INFO:tasks.workunit.client.0.smithi117.stdout:2/507: creat d1/d4/fa0 x:0 0 0 2024-04-20T11:09:04.821 INFO:tasks.workunit.client.0.smithi117.stdout:2/508: rename d1/d4/d83/d29/f4c to d1/d15/d1d/fa1 0 2024-04-20T11:09:04.821 INFO:tasks.workunit.client.0.smithi117.stdout:6/456: sync 2024-04-20T11:09:04.823 INFO:tasks.workunit.client.0.smithi117.stdout:2/509: fdatasync d1/d15/d1d/f31 0 2024-04-20T11:09:04.824 INFO:tasks.workunit.client.0.smithi117.stdout:2/510: stat d1/d4/dd 0 2024-04-20T11:09:04.840 INFO:tasks.workunit.client.0.smithi117.stdout:2/511: dread d1/d4/dd/d17/d7a/f95 [0,4194304] 0 2024-04-20T11:09:04.845 INFO:tasks.workunit.client.0.smithi117.stdout:2/512: truncate d1/d4/dd/f14 2678486 0 2024-04-20T11:09:04.848 INFO:tasks.workunit.client.0.smithi117.stdout:6/457: dwrite d3/d4/d2e/d3f/f82 [0,4194304] 0 2024-04-20T11:09:04.850 INFO:tasks.workunit.client.0.smithi117.stdout:6/458: chown d3/f67 2 1 2024-04-20T11:09:04.851 INFO:tasks.workunit.client.0.smithi117.stdout:6/459: write d3/df/f14 [2003741,71123] 0 2024-04-20T11:09:04.856 INFO:tasks.workunit.client.0.smithi117.stdout:6/460: symlink d3/d4/d2e/d37/l96 0 2024-04-20T11:09:04.857 INFO:tasks.workunit.client.0.smithi117.stdout:6/461: readlink d3/d4/l36 0 2024-04-20T11:09:04.859 INFO:tasks.workunit.client.0.smithi117.stdout:6/462: symlink d3/d4/d2e/d5f/l97 0 2024-04-20T11:09:04.863 INFO:tasks.workunit.client.0.smithi117.stdout:6/463: creat d3/d4/d2e/d4e/f98 x:0 0 0 2024-04-20T11:09:04.867 INFO:tasks.workunit.client.0.smithi117.stdout:2/513: dwrite d1/d20/f21 [0,4194304] 0 2024-04-20T11:09:04.884 INFO:tasks.workunit.client.0.smithi117.stdout:2/514: dwrite d1/d4/d83/f7e [0,4194304] 0 2024-04-20T11:09:04.886 INFO:tasks.workunit.client.0.smithi117.stdout:2/515: write d1/d4/d83/d29/d62/f8a [984618,83013] 0 2024-04-20T11:09:04.900 INFO:tasks.workunit.client.0.smithi117.stdout:2/516: dread d1/f3 [0,4194304] 0 2024-04-20T11:09:04.903 INFO:tasks.workunit.client.0.smithi117.stdout:2/517: truncate d1/d4/f8 787514 0 2024-04-20T11:09:04.905 INFO:tasks.workunit.client.0.smithi117.stdout:2/518: dread - d1/d4/dd/d17/d7a/d49/f87 zero size 2024-04-20T11:09:04.907 INFO:tasks.workunit.client.0.smithi117.stdout:5/571: sync 2024-04-20T11:09:04.907 INFO:tasks.workunit.client.0.smithi117.stdout:3/410: sync 2024-04-20T11:09:04.907 INFO:tasks.workunit.client.0.smithi117.stdout:4/433: sync 2024-04-20T11:09:04.907 INFO:tasks.workunit.client.0.smithi117.stdout:8/306: sync 2024-04-20T11:09:04.907 INFO:tasks.workunit.client.0.smithi117.stdout:0/392: sync 2024-04-20T11:09:04.907 INFO:tasks.workunit.client.0.smithi117.stdout:7/437: sync 2024-04-20T11:09:04.907 INFO:tasks.workunit.client.0.smithi117.stdout:9/435: sync 2024-04-20T11:09:04.907 INFO:tasks.workunit.client.0.smithi117.stdout:1/393: sync 2024-04-20T11:09:04.908 INFO:tasks.workunit.client.0.smithi117.stdout:6/464: dread d3/df/f31 [0,4194304] 0 2024-04-20T11:09:04.911 INFO:tasks.workunit.client.0.smithi117.stdout:7/438: chown d1/d4/d39/f83 85732 1 2024-04-20T11:09:04.911 INFO:tasks.workunit.client.0.smithi117.stdout:4/434: unlink d1/dd/d1a/d2b/c41 0 2024-04-20T11:09:04.913 INFO:tasks.workunit.client.0.smithi117.stdout:9/436: rmdir d4/d4a/d5e 39 2024-04-20T11:09:04.914 INFO:tasks.workunit.client.0.smithi117.stdout:0/393: write d3/d7/d16/f56 [381286,53316] 0 2024-04-20T11:09:04.916 INFO:tasks.workunit.client.0.smithi117.stdout:7/439: chown d1/d4/d39/f7a 254 1 2024-04-20T11:09:04.917 INFO:tasks.workunit.client.0.smithi117.stdout:8/307: creat d0/d8/d39/d5a/d5c/f68 x:0 0 0 2024-04-20T11:09:04.921 INFO:tasks.workunit.client.0.smithi117.stdout:5/572: creat de/d21/d35/fbe x:0 0 0 2024-04-20T11:09:04.921 INFO:tasks.workunit.client.0.smithi117.stdout:9/437: truncate d4/d30/f6b 4446184 0 2024-04-20T11:09:04.921 INFO:tasks.workunit.client.0.smithi117.stdout:7/440: readlink d1/d4/d6/d13/l19 0 2024-04-20T11:09:04.923 INFO:tasks.workunit.client.0.smithi117.stdout:6/465: rename d3/d4/d2e/d55 to d3/df/d39/d99 0 2024-04-20T11:09:04.924 INFO:tasks.workunit.client.0.smithi117.stdout:5/573: chown de/d41/d47/d55 0 1 2024-04-20T11:09:04.924 INFO:tasks.workunit.client.0.smithi117.stdout:7/441: chown d1/d43/f8d 304 1 2024-04-20T11:09:04.926 INFO:tasks.workunit.client.0.smithi117.stdout:7/442: dread - d1/d1d/d42/d2d/d6c/f7d zero size 2024-04-20T11:09:04.927 INFO:tasks.workunit.client.0.smithi117.stdout:4/435: rename d1/d9/d1f/l6d to d1/d9/d1f/l93 0 2024-04-20T11:09:04.930 INFO:tasks.workunit.client.0.smithi117.stdout:7/443: dread - d1/d4/d6/d2b/f81 zero size 2024-04-20T11:09:04.931 INFO:tasks.workunit.client.0.smithi117.stdout:2/519: dwrite d1/d4/d83/f7e [0,4194304] 0 2024-04-20T11:09:04.932 INFO:tasks.workunit.client.0.smithi117.stdout:3/411: dwrite d0/d5/d21/f42 [0,4194304] 0 2024-04-20T11:09:04.934 INFO:tasks.workunit.client.0.smithi117.stdout:7/444: stat d1/d1d/d42/d2d/d6c/c6f 0 2024-04-20T11:09:04.935 INFO:tasks.workunit.client.0.smithi117.stdout:2/520: chown d1/d15/f72 6709912 1 2024-04-20T11:09:04.943 INFO:tasks.workunit.client.0.smithi117.stdout:0/394: rename d3/d7/d20/d39/l41 to d3/d5/d74/l83 0 2024-04-20T11:09:04.943 INFO:tasks.workunit.client.0.smithi117.stdout:4/436: dread d1/dd/f53 [0,4194304] 0 2024-04-20T11:09:04.949 INFO:tasks.workunit.client.0.smithi117.stdout:6/466: mknod d3/c9a 0 2024-04-20T11:09:04.952 INFO:tasks.workunit.client.0.smithi117.stdout:7/445: creat d1/d1d/d42/d2d/f92 x:0 0 0 2024-04-20T11:09:04.954 INFO:tasks.workunit.client.0.smithi117.stdout:5/574: symlink de/d41/lbf 0 2024-04-20T11:09:04.957 INFO:tasks.workunit.client.0.smithi117.stdout:2/521: mknod d1/d4/dd/d3e/d46/ca2 0 2024-04-20T11:09:04.958 INFO:tasks.workunit.client.0.smithi117.stdout:3/412: rename d0/d5/d11/f8d to d0/d5/d7/dd/f91 0 2024-04-20T11:09:04.960 INFO:tasks.workunit.client.0.smithi117.stdout:2/522: chown d1/d4/dd/d3e/d46 5924 1 2024-04-20T11:09:04.962 INFO:tasks.workunit.client.0.smithi117.stdout:0/395: creat d3/d7/d20/d6e/f84 x:0 0 0 2024-04-20T11:09:04.964 INFO:tasks.workunit.client.0.smithi117.stdout:4/437: dwrite d1/f25 [0,4194304] 0 2024-04-20T11:09:04.969 INFO:tasks.workunit.client.0.smithi117.stdout:8/308: write d0/f23 [3729255,38403] 0 2024-04-20T11:09:04.970 INFO:tasks.workunit.client.0.smithi117.stdout:5/575: unlink de/c10 0 2024-04-20T11:09:04.975 INFO:tasks.workunit.client.0.smithi117.stdout:3/413: creat d0/d5/de/f92 x:0 0 0 2024-04-20T11:09:04.977 INFO:tasks.workunit.client.0.smithi117.stdout:0/396: mkdir d3/d7/d8/df/d22/d85 0 2024-04-20T11:09:04.977 INFO:tasks.workunit.client.0.smithi117.stdout:6/467: dwrite d3/f67 [0,4194304] 0 2024-04-20T11:09:04.981 INFO:tasks.workunit.client.0.smithi117.stdout:2/523: symlink d1/d4/d83/d29/d2e/d80/la3 0 2024-04-20T11:09:04.986 INFO:tasks.workunit.client.0.smithi117.stdout:4/438: mknod d1/dd/d1a/d2b/d6e/c94 0 2024-04-20T11:09:04.995 INFO:tasks.workunit.client.0.smithi117.stdout:2/524: mknod d1/d4/d83/d29/d3f/ca4 0 2024-04-20T11:09:04.996 INFO:tasks.workunit.client.0.smithi117.stdout:5/576: creat de/d16/fc0 x:0 0 0 2024-04-20T11:09:04.996 INFO:tasks.workunit.client.0.smithi117.stdout:9/438: dwrite d4/de/f3e [0,4194304] 0 2024-04-20T11:09:05.000 INFO:tasks.workunit.client.0.smithi117.stdout:5/577: stat de/d21/l33 0 2024-04-20T11:09:05.002 INFO:tasks.workunit.client.0.smithi117.stdout:2/525: read d1/d15/f72 [716625,119488] 0 2024-04-20T11:09:05.006 INFO:tasks.workunit.client.0.smithi117.stdout:4/439: rename d1/dd/d1a/c4d to d1/d9/d1f/d90/c95 0 2024-04-20T11:09:05.006 INFO:tasks.workunit.client.0.smithi117.stdout:9/439: rmdir d4/d35/d54 39 2024-04-20T11:09:05.007 INFO:tasks.workunit.client.0.smithi117.stdout:7/446: dwrite d1/d4/d3d/f5a [0,4194304] 0 2024-04-20T11:09:05.008 INFO:tasks.workunit.client.0.smithi117.stdout:8/309: dread d0/d9/f33 [0,4194304] 0 2024-04-20T11:09:05.009 INFO:tasks.workunit.client.0.smithi117.stdout:2/526: stat d1/d4/d83/d29/d2e/d52/l63 0 2024-04-20T11:09:05.012 INFO:tasks.workunit.client.0.smithi117.stdout:6/468: dread d3/df/f21 [0,4194304] 0 2024-04-20T11:09:05.013 INFO:tasks.workunit.client.0.smithi117.stdout:4/440: truncate d1/f25 8740662 0 2024-04-20T11:09:05.013 INFO:tasks.workunit.client.0.smithi117.stdout:8/310: fsync d0/d8/d39/d40/f51 0 2024-04-20T11:09:05.015 INFO:tasks.workunit.client.0.smithi117.stdout:5/578: mkdir de/d21/d45/d48/d4e/dc1 0 2024-04-20T11:09:05.016 INFO:tasks.workunit.client.0.smithi117.stdout:7/447: mkdir d1/d4/d3d/d93 0 2024-04-20T11:09:05.019 INFO:tasks.workunit.client.0.smithi117.stdout:9/440: mkdir d4/d4a/d5e/d9b 0 2024-04-20T11:09:05.021 INFO:tasks.workunit.client.0.smithi117.stdout:6/469: mkdir d3/d5a/d79/d9b 0 2024-04-20T11:09:05.026 INFO:tasks.workunit.client.0.smithi117.stdout:2/527: mkdir d1/da5 0 2024-04-20T11:09:05.026 INFO:tasks.workunit.client.0.smithi117.stdout:8/311: write d0/d8/d11/d24/d27/d47/f4f [2799644,90842] 0 2024-04-20T11:09:05.027 INFO:tasks.workunit.client.0.smithi117.stdout:4/441: rename d1/d9/d1f/f39 to d1/d9/f96 0 2024-04-20T11:09:05.033 INFO:tasks.workunit.client.0.smithi117.stdout:2/528: chown d1/d4/dd/d17/f1e 13300 1 2024-04-20T11:09:05.034 INFO:tasks.workunit.client.0.smithi117.stdout:5/579: write de/d21/d45/d48/fad [985940,6172] 0 2024-04-20T11:09:05.035 INFO:tasks.workunit.client.0.smithi117.stdout:8/312: mkdir d0/dc/d2e/d3e/d69 0 2024-04-20T11:09:05.040 INFO:tasks.workunit.client.0.smithi117.stdout:3/414: dread d0/d5/d21/d88/d36/f33 [0,4194304] 0 2024-04-20T11:09:05.040 INFO:tasks.workunit.client.0.smithi117.stdout:0/397: dread d3/f3e [0,4194304] 0 2024-04-20T11:09:05.040 INFO:tasks.workunit.client.0.smithi117.stdout:9/441: symlink d4/de/d10/l9c 0 2024-04-20T11:09:05.040 INFO:tasks.workunit.client.0.smithi117.stdout:2/529: read d1/f6 [999792,19312] 0 2024-04-20T11:09:05.041 INFO:tasks.workunit.client.0.smithi117.stdout:5/580: chown de/db7/fbd 99494440 1 2024-04-20T11:09:05.046 INFO:tasks.workunit.client.0.smithi117.stdout:6/470: dread d3/d4/d2e/f40 [0,4194304] 0 2024-04-20T11:09:05.047 INFO:tasks.workunit.client.0.smithi117.stdout:3/415: dread - d0/d5/d21/d77/f8a zero size 2024-04-20T11:09:05.048 INFO:tasks.workunit.client.0.smithi117.stdout:7/448: dwrite d1/d1d/f70 [0,4194304] 0 2024-04-20T11:09:05.049 INFO:tasks.workunit.client.0.smithi117.stdout:8/313: creat d0/d9/d3a/f6a x:0 0 0 2024-04-20T11:09:05.052 INFO:tasks.workunit.client.0.smithi117.stdout:8/314: readlink d0/d8/l2f 0 2024-04-20T11:09:05.052 INFO:tasks.workunit.client.0.smithi117.stdout:2/530: mkdir d1/d4/d83/d29/d62/da6 0 2024-04-20T11:09:05.057 INFO:tasks.workunit.client.0.smithi117.stdout:5/581: mkdir de/d41/dc2 0 2024-04-20T11:09:05.060 INFO:tasks.workunit.client.0.smithi117.stdout:6/471: read d3/f18 [1269944,98054] 0 2024-04-20T11:09:05.060 INFO:tasks.workunit.client.0.smithi117.stdout:3/416: rename d0/d5/d7/l82 to d0/d5/d21/d75/d89/l93 0 2024-04-20T11:09:05.061 INFO:tasks.workunit.client.0.smithi117.stdout:8/315: mknod d0/dc/d2e/d3e/d58/c6b 0 2024-04-20T11:09:05.061 INFO:tasks.workunit.client.0.smithi117.stdout:2/531: truncate d1/d4/d83/d29/d2e/d53/f9d 436680 0 2024-04-20T11:09:05.064 INFO:tasks.workunit.client.0.smithi117.stdout:1/394: sync 2024-04-20T11:09:05.068 INFO:tasks.workunit.client.0.smithi117.stdout:9/442: link d4/d30/f36 d4/de/d10/f9d 0 2024-04-20T11:09:05.068 INFO:tasks.workunit.client.0.smithi117.stdout:7/449: getdents d1/d1d/d77 0 2024-04-20T11:09:05.072 INFO:tasks.workunit.client.0.smithi117.stdout:9/443: write d4/de/f90 [476691,118508] 0 2024-04-20T11:09:05.074 INFO:tasks.workunit.client.0.smithi117.stdout:9/444: dread - d4/d4a/d74/f85 zero size 2024-04-20T11:09:05.075 INFO:tasks.workunit.client.0.smithi117.stdout:0/398: truncate d3/d7/f19 2745993 0 2024-04-20T11:09:05.077 INFO:tasks.workunit.client.0.smithi117.stdout:2/532: mknod d1/d4/d69/ca7 0 2024-04-20T11:09:05.079 INFO:tasks.workunit.client.0.smithi117.stdout:6/472: rename d3/df/l26 to d3/d4/d2e/d37/d8e/l9c 0 2024-04-20T11:09:05.081 INFO:tasks.workunit.client.0.smithi117.stdout:4/442: dread d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:05.082 INFO:tasks.workunit.client.0.smithi117.stdout:5/582: mkdir de/d21/d45/d48/d4e/dc3 0 2024-04-20T11:09:05.084 INFO:tasks.workunit.client.0.smithi117.stdout:7/450: write d1/d4/d6/d2b/f33 [1410401,125348] 0 2024-04-20T11:09:05.085 INFO:tasks.workunit.client.0.smithi117.stdout:9/445: mknod d4/d5/d3c/d7a/c9e 0 2024-04-20T11:09:05.088 INFO:tasks.workunit.client.0.smithi117.stdout:2/533: fsync d1/d15/f72 0 2024-04-20T11:09:05.088 INFO:tasks.workunit.client.0.smithi117.stdout:4/443: mknod d1/dd/d1a/d2b/c97 0 2024-04-20T11:09:05.088 INFO:tasks.workunit.client.0.smithi117.stdout:6/473: creat d3/d4/d2e/d37/d8e/f9d x:0 0 0 2024-04-20T11:09:05.093 INFO:tasks.workunit.client.0.smithi117.stdout:7/451: fdatasync d1/d43/f91 0 2024-04-20T11:09:05.093 INFO:tasks.workunit.client.0.smithi117.stdout:5/583: creat de/d21/d45/d48/d4e/dc1/fc4 x:0 0 0 2024-04-20T11:09:05.094 INFO:tasks.workunit.client.0.smithi117.stdout:6/474: stat d3/d3a/f6b 0 2024-04-20T11:09:05.097 INFO:tasks.workunit.client.0.smithi117.stdout:1/395: dread d6/d1f/f40 [0,4194304] 0 2024-04-20T11:09:05.099 INFO:tasks.workunit.client.0.smithi117.stdout:8/316: dwrite d0/d8/d11/f1f [0,4194304] 0 2024-04-20T11:09:05.102 INFO:tasks.workunit.client.0.smithi117.stdout:3/417: dwrite d0/d5/d21/d88/f62 [0,4194304] 0 2024-04-20T11:09:05.102 INFO:tasks.workunit.client.0.smithi117.stdout:5/584: dread - de/d21/d45/d77/fa5 zero size 2024-04-20T11:09:05.103 INFO:tasks.workunit.client.0.smithi117.stdout:6/475: chown d3/d3a 1 1 2024-04-20T11:09:05.103 INFO:tasks.workunit.client.0.smithi117.stdout:2/534: fdatasync d1/d4/dd/d17/d3b/f6c 0 2024-04-20T11:09:05.108 INFO:tasks.workunit.client.0.smithi117.stdout:3/418: readlink d0/d5/de/l15 0 2024-04-20T11:09:05.112 INFO:tasks.workunit.client.0.smithi117.stdout:0/399: dwrite d3/d7/d2c/f69 [0,4194304] 0 2024-04-20T11:09:05.114 INFO:tasks.workunit.client.0.smithi117.stdout:1/396: mknod d6/d35/d63/c7f 0 2024-04-20T11:09:05.116 INFO:tasks.workunit.client.0.smithi117.stdout:1/397: stat d6/d16/c1b 0 2024-04-20T11:09:05.119 INFO:tasks.workunit.client.0.smithi117.stdout:1/398: write d6/d16/d3a/f7e [442512,64681] 0 2024-04-20T11:09:05.120 INFO:tasks.workunit.client.0.smithi117.stdout:3/419: mkdir d0/d5/d21/d94 0 2024-04-20T11:09:05.123 INFO:tasks.workunit.client.0.smithi117.stdout:9/446: getdents d4/de/d7b 0 2024-04-20T11:09:05.123 INFO:tasks.workunit.client.0.smithi117.stdout:3/420: dread - d0/d5/de/f92 zero size 2024-04-20T11:09:05.127 INFO:tasks.workunit.client.0.smithi117.stdout:0/400: symlink d3/d7/d20/d39/d3a/l86 0 2024-04-20T11:09:05.128 INFO:tasks.workunit.client.0.smithi117.stdout:6/476: symlink d3/d50/d8a/l9e 0 2024-04-20T11:09:05.132 INFO:tasks.workunit.client.0.smithi117.stdout:7/452: link d1/d4/d6/d13/f15 d1/d1d/d4e/d53/d55/f94 0 2024-04-20T11:09:05.133 INFO:tasks.workunit.client.0.smithi117.stdout:8/317: dread d0/d8/d11/f1f [0,4194304] 0 2024-04-20T11:09:05.133 INFO:tasks.workunit.client.0.smithi117.stdout:8/318: stat d0/f1 0 2024-04-20T11:09:05.134 INFO:tasks.workunit.client.0.smithi117.stdout:5/585: rename de/d41/d47/d4c/f6e to de/d21/d45/fc5 0 2024-04-20T11:09:05.136 INFO:tasks.workunit.client.0.smithi117.stdout:2/535: getdents d1/d15 0 2024-04-20T11:09:05.137 INFO:tasks.workunit.client.0.smithi117.stdout:0/401: mknod d3/d7/c87 0 2024-04-20T11:09:05.138 INFO:tasks.workunit.client.0.smithi117.stdout:6/477: rmdir d3/d3a 39 2024-04-20T11:09:05.144 INFO:tasks.workunit.client.0.smithi117.stdout:9/447: dread d4/d4a/d63/f76 [0,4194304] 0 2024-04-20T11:09:05.150 INFO:tasks.workunit.client.0.smithi117.stdout:6/478: mkdir d3/d4/d2e/d3f/d9f 0 2024-04-20T11:09:05.152 INFO:tasks.workunit.client.0.smithi117.stdout:1/399: getdents d6/d1f/d30/d32 0 2024-04-20T11:09:05.156 INFO:tasks.workunit.client.0.smithi117.stdout:3/421: rename d0/d5/d7/dd/f56 to d0/d5/d73/f95 0 2024-04-20T11:09:05.156 INFO:tasks.workunit.client.0.smithi117.stdout:7/453: truncate d1/f10 2962275 0 2024-04-20T11:09:05.157 INFO:tasks.workunit.client.0.smithi117.stdout:8/319: dwrite d0/d8/d39/d5a/d5c/f68 [0,4194304] 0 2024-04-20T11:09:05.160 INFO:tasks.workunit.client.0.smithi117.stdout:3/422: stat d0/d5/d21/d88/d29/c6e 0 2024-04-20T11:09:05.161 INFO:tasks.workunit.client.0.smithi117.stdout:0/402: dread d3/d7/d8/df/d6d/f71 [0,4194304] 0 2024-04-20T11:09:05.161 INFO:tasks.workunit.client.0.smithi117.stdout:1/400: truncate d6/d1f/d30/f67 325313 0 2024-04-20T11:09:05.162 INFO:tasks.workunit.client.0.smithi117.stdout:2/536: write d1/d4/d83/f34 [2001487,112050] 0 2024-04-20T11:09:05.163 INFO:tasks.workunit.client.0.smithi117.stdout:9/448: rename d4/d5/d3c/f87 to d4/d4a/d92/f9f 0 2024-04-20T11:09:05.165 INFO:tasks.workunit.client.0.smithi117.stdout:5/586: creat de/d41/d47/fc6 x:0 0 0 2024-04-20T11:09:05.166 INFO:tasks.workunit.client.0.smithi117.stdout:8/320: truncate d0/dc/d2e/d3e/f48 983005 0 2024-04-20T11:09:05.167 INFO:tasks.workunit.client.0.smithi117.stdout:5/587: readlink de/d21/d84/l88 0 2024-04-20T11:09:05.168 INFO:tasks.workunit.client.0.smithi117.stdout:2/537: dread - d1/d4/d83/d8e/f96 zero size 2024-04-20T11:09:05.172 INFO:tasks.workunit.client.0.smithi117.stdout:6/479: creat d3/d5a/d79/d9b/fa0 x:0 0 0 2024-04-20T11:09:05.172 INFO:tasks.workunit.client.0.smithi117.stdout:2/538: chown d1/d15/d1d/c2f 34691835 1 2024-04-20T11:09:05.173 INFO:tasks.workunit.client.0.smithi117.stdout:0/403: symlink d3/d7/l88 0 2024-04-20T11:09:05.176 INFO:tasks.workunit.client.0.smithi117.stdout:4/444: dread d1/d45/f5f [0,4194304] 0 2024-04-20T11:09:05.177 INFO:tasks.workunit.client.0.smithi117.stdout:3/423: creat d0/d5/d7/dd/f96 x:0 0 0 2024-04-20T11:09:05.182 INFO:tasks.workunit.client.0.smithi117.stdout:5/588: dread de/f18 [0,4194304] 0 2024-04-20T11:09:05.182 INFO:tasks.workunit.client.0.smithi117.stdout:9/449: symlink d4/d68/la0 0 2024-04-20T11:09:05.182 INFO:tasks.workunit.client.0.smithi117.stdout:3/424: dread - d0/d5/d21/f4d zero size 2024-04-20T11:09:05.182 INFO:tasks.workunit.client.0.smithi117.stdout:8/321: creat d0/d8/d11/d24/d27/d47/f6c x:0 0 0 2024-04-20T11:09:05.186 INFO:tasks.workunit.client.0.smithi117.stdout:2/539: mkdir d1/d4/d83/d29/d2e/d53/d58/da8 0 2024-04-20T11:09:05.187 INFO:tasks.workunit.client.0.smithi117.stdout:2/540: chown d1/d4/fa0 554 1 2024-04-20T11:09:05.189 INFO:tasks.workunit.client.0.smithi117.stdout:4/445: mknod d1/d45/d4a/d59/c98 0 2024-04-20T11:09:05.191 INFO:tasks.workunit.client.0.smithi117.stdout:4/446: readlink d1/d9/d1f/l70 0 2024-04-20T11:09:05.192 INFO:tasks.workunit.client.0.smithi117.stdout:1/401: creat d6/d1f/f80 x:0 0 0 2024-04-20T11:09:05.194 INFO:tasks.workunit.client.0.smithi117.stdout:3/425: creat d0/d5/d21/d88/d29/f97 x:0 0 0 2024-04-20T11:09:05.195 INFO:tasks.workunit.client.0.smithi117.stdout:2/541: truncate d1/d4/d83/f3a 9293571 0 2024-04-20T11:09:05.196 INFO:tasks.workunit.client.0.smithi117.stdout:5/589: creat de/d16/d9d/fc7 x:0 0 0 2024-04-20T11:09:05.197 INFO:tasks.workunit.client.0.smithi117.stdout:8/322: write d0/d9/f33 [1370309,18238] 0 2024-04-20T11:09:05.199 INFO:tasks.workunit.client.0.smithi117.stdout:0/404: rename d3/d7/d2c/d52/c58 to d3/d7/d20/d47/c89 0 2024-04-20T11:09:05.199 INFO:tasks.workunit.client.0.smithi117.stdout:9/450: truncate d4/d5/da/f32 1883931 0 2024-04-20T11:09:05.203 INFO:tasks.workunit.client.0.smithi117.stdout:6/480: dwrite d3/d4/f2f [0,4194304] 0 2024-04-20T11:09:05.203 INFO:tasks.workunit.client.0.smithi117.stdout:7/454: link d1/l20 d1/d1d/d4e/l95 0 2024-04-20T11:09:05.203 INFO:tasks.workunit.client.0.smithi117.stdout:2/542: chown d1/d4/dd/d17/d7a/c44 5 1 2024-04-20T11:09:05.204 INFO:tasks.workunit.client.0.smithi117.stdout:8/323: write d0/d9/f2a [3692599,19088] 0 2024-04-20T11:09:05.211 INFO:tasks.workunit.client.0.smithi117.stdout:4/447: dread - d1/dd/d1a/d43/f65 zero size 2024-04-20T11:09:05.214 INFO:tasks.workunit.client.0.smithi117.stdout:5/590: mkdir de/d41/d47/d55/d56/dc8 0 2024-04-20T11:09:05.219 INFO:tasks.workunit.client.0.smithi117.stdout:7/455: mknod d1/d43/d59/c96 0 2024-04-20T11:09:05.225 INFO:tasks.workunit.client.0.smithi117.stdout:9/451: dwrite d4/d68/d6e/f77 [0,4194304] 0 2024-04-20T11:09:05.225 INFO:tasks.workunit.client.0.smithi117.stdout:3/426: dwrite d0/d5/d21/d88/f44 [4194304,4194304] 0 2024-04-20T11:09:05.230 INFO:tasks.workunit.client.0.smithi117.stdout:2/543: write d1/f3 [4825000,80896] 0 2024-04-20T11:09:05.233 INFO:tasks.workunit.client.0.smithi117.stdout:5/591: stat de/d16/l6a 0 2024-04-20T11:09:05.240 INFO:tasks.workunit.client.0.smithi117.stdout:5/592: dread - de/d5b/d65/fa2 zero size 2024-04-20T11:09:05.240 INFO:tasks.workunit.client.0.smithi117.stdout:8/324: rename d0/dc/d2e/d3e to d0/d8/d11/d6d 0 2024-04-20T11:09:05.244 INFO:tasks.workunit.client.0.smithi117.stdout:2/544: unlink d1/d4/dd/d17/d7a/d88/f7f 0 2024-04-20T11:09:05.250 INFO:tasks.workunit.client.0.smithi117.stdout:9/452: getdents d4/d5/da/d51 0 2024-04-20T11:09:05.253 INFO:tasks.workunit.client.0.smithi117.stdout:5/593: dread - de/d21/d45/fc5 zero size 2024-04-20T11:09:05.254 INFO:tasks.workunit.client.0.smithi117.stdout:8/325: readlink d0/dc/d2e/l60 0 2024-04-20T11:09:05.255 INFO:tasks.workunit.client.0.smithi117.stdout:0/405: dwrite d3/d7/d8/f13 [4194304,4194304] 0 2024-04-20T11:09:05.257 INFO:tasks.workunit.client.0.smithi117.stdout:4/448: rename d1/dd/d1a/d2b/d6e to d1/d45/d4a/d99 0 2024-04-20T11:09:05.261 INFO:tasks.workunit.client.0.smithi117.stdout:5/594: fdatasync de/d21/d45/d48/d4e/f8c 0 2024-04-20T11:09:05.261 INFO:tasks.workunit.client.0.smithi117.stdout:1/402: dwrite d6/d1f/d30/d32/f6a [4194304,4194304] 0 2024-04-20T11:09:05.261 INFO:tasks.workunit.client.0.smithi117.stdout:8/326: stat d0/d9/f2a 0 2024-04-20T11:09:05.264 INFO:tasks.workunit.client.0.smithi117.stdout:1/403: stat d6/f12 0 2024-04-20T11:09:05.265 INFO:tasks.workunit.client.0.smithi117.stdout:6/481: dwrite d3/d4/d2e/d37/f47 [0,4194304] 0 2024-04-20T11:09:05.266 INFO:tasks.workunit.client.0.smithi117.stdout:2/545: write d1/d4/d83/d29/d2e/f74 [1258423,115307] 0 2024-04-20T11:09:05.266 INFO:tasks.workunit.client.0.smithi117.stdout:3/427: write d0/d5/d21/d88/d69/f8c [375179,55586] 0 2024-04-20T11:09:05.269 INFO:tasks.workunit.client.0.smithi117.stdout:9/453: truncate d4/de/d4b/f6d 4838530 0 2024-04-20T11:09:05.272 INFO:tasks.workunit.client.0.smithi117.stdout:1/404: unlink d6/d1f/d30/l4d 0 2024-04-20T11:09:05.272 INFO:tasks.workunit.client.0.smithi117.stdout:7/456: dwrite d1/d1d/d42/d2d/d6c/f88 [0,4194304] 0 2024-04-20T11:09:05.273 INFO:tasks.workunit.client.0.smithi117.stdout:6/482: readlink d3/df/d39/l52 0 2024-04-20T11:09:05.273 INFO:tasks.workunit.client.0.smithi117.stdout:8/327: chown d0/d8/d11/l36 171 1 2024-04-20T11:09:05.276 INFO:tasks.workunit.client.0.smithi117.stdout:9/454: write d4/de/f3e [4521318,22964] 0 2024-04-20T11:09:05.281 INFO:tasks.workunit.client.0.smithi117.stdout:9/455: rmdir d4/d5/d3c 39 2024-04-20T11:09:05.285 INFO:tasks.workunit.client.0.smithi117.stdout:0/406: rename d3/d7/d8/d2a to d3/d7/d2c/d8a 0 2024-04-20T11:09:05.286 INFO:tasks.workunit.client.0.smithi117.stdout:7/457: write d1/d1d/d42/f8b [155643,14856] 0 2024-04-20T11:09:05.292 INFO:tasks.workunit.client.0.smithi117.stdout:3/428: creat d0/f98 x:0 0 0 2024-04-20T11:09:05.292 INFO:tasks.workunit.client.0.smithi117.stdout:2/546: truncate d1/d4/d83/d29/d2e/d53/f9d 586021 0 2024-04-20T11:09:05.295 INFO:tasks.workunit.client.0.smithi117.stdout:4/449: dwrite d1/f57 [0,4194304] 0 2024-04-20T11:09:05.295 INFO:tasks.workunit.client.0.smithi117.stdout:7/458: creat d1/d4/f97 x:0 0 0 2024-04-20T11:09:05.296 INFO:tasks.workunit.client.0.smithi117.stdout:0/407: write d3/d7/d20/d6e/f84 [572986,35324] 0 2024-04-20T11:09:05.297 INFO:tasks.workunit.client.0.smithi117.stdout:8/328: rename d0/d8/f38 to d0/d8/d39/d40/f6e 0 2024-04-20T11:09:05.299 INFO:tasks.workunit.client.0.smithi117.stdout:9/456: mkdir d4/d5/da1 0 2024-04-20T11:09:05.305 INFO:tasks.workunit.client.0.smithi117.stdout:4/450: fdatasync d1/dd/d1a/f7c 0 2024-04-20T11:09:05.309 INFO:tasks.workunit.client.0.smithi117.stdout:5/595: link de/d21/d45/d48/l71 de/d41/d4b/lc9 0 2024-04-20T11:09:05.310 INFO:tasks.workunit.client.0.smithi117.stdout:6/483: dwrite d3/d4/f8d [0,4194304] 0 2024-04-20T11:09:05.313 INFO:tasks.workunit.client.0.smithi117.stdout:5/596: readlink de/d16/l6b 0 2024-04-20T11:09:05.313 INFO:tasks.workunit.client.0.smithi117.stdout:6/484: fsync d3/d5a/d79/f6a 0 2024-04-20T11:09:05.313 INFO:tasks.workunit.client.0.smithi117.stdout:5/597: fsync de/d21/f70 0 2024-04-20T11:09:05.315 INFO:tasks.workunit.client.0.smithi117.stdout:1/405: getdents d6/d1f/d30 0 2024-04-20T11:09:05.319 INFO:tasks.workunit.client.0.smithi117.stdout:9/457: symlink d4/d5/la2 0 2024-04-20T11:09:05.320 INFO:tasks.workunit.client.0.smithi117.stdout:4/451: mkdir d1/dd/d1a/d43/d9a 0 2024-04-20T11:09:05.329 INFO:tasks.workunit.client.0.smithi117.stdout:2/547: creat d1/d20/fa9 x:0 0 0 2024-04-20T11:09:05.334 INFO:tasks.workunit.client.0.smithi117.stdout:7/459: write d1/d4/d6/d17/d3f/f65 [1619599,42493] 0 2024-04-20T11:09:05.334 INFO:tasks.workunit.client.0.smithi117.stdout:1/406: symlink d6/d16/d3a/d5a/l81 0 2024-04-20T11:09:05.337 INFO:tasks.workunit.client.0.smithi117.stdout:3/429: link d0/d5/d11/l72 d0/d5/d11/d80/l99 0 2024-04-20T11:09:05.338 INFO:tasks.workunit.client.0.smithi117.stdout:4/452: creat d1/dd/d1a/d43/d4e/f9b x:0 0 0 2024-04-20T11:09:05.343 INFO:tasks.workunit.client.0.smithi117.stdout:3/430: dread - d0/d5/d21/f4d zero size 2024-04-20T11:09:05.346 INFO:tasks.workunit.client.0.smithi117.stdout:5/598: dwrite de/d41/d47/d4c/f8f [0,4194304] 0 2024-04-20T11:09:05.346 INFO:tasks.workunit.client.0.smithi117.stdout:8/329: dread d0/d8/d11/d24/f46 [0,4194304] 0 2024-04-20T11:09:05.347 INFO:tasks.workunit.client.0.smithi117.stdout:5/599: chown c7 63268460 1 2024-04-20T11:09:05.353 INFO:tasks.workunit.client.0.smithi117.stdout:8/330: fdatasync d0/d8/d11/d24/d27/d47/f6c 0 2024-04-20T11:09:05.353 INFO:tasks.workunit.client.0.smithi117.stdout:5/600: write de/d21/d84/f87 [2699747,121856] 0 2024-04-20T11:09:05.353 INFO:tasks.workunit.client.0.smithi117.stdout:9/458: dwrite f2 [4194304,4194304] 0 2024-04-20T11:09:05.356 INFO:tasks.workunit.client.0.smithi117.stdout:8/331: read d0/f7 [3868354,56089] 0 2024-04-20T11:09:05.363 INFO:tasks.workunit.client.0.smithi117.stdout:6/485: truncate d3/d5a/d79/f6f 2669970 0 2024-04-20T11:09:05.363 INFO:tasks.workunit.client.0.smithi117.stdout:0/408: link d3/d7/d2c/d8a/l4e d3/d7/d2c/d8a/d54/l8b 0 2024-04-20T11:09:05.364 INFO:tasks.workunit.client.0.smithi117.stdout:3/431: readlink d0/d5/l5f 0 2024-04-20T11:09:05.366 INFO:tasks.workunit.client.0.smithi117.stdout:1/407: chown d6/d16/d3a/d5a/d5c/d61/d71/d6b/c76 3744 1 2024-04-20T11:09:05.367 INFO:tasks.workunit.client.0.smithi117.stdout:4/453: write d1/d9/f66 [1255092,97163] 0 2024-04-20T11:09:05.368 INFO:tasks.workunit.client.0.smithi117.stdout:3/432: fdatasync d0/d5/d7/dd/ff 0 2024-04-20T11:09:05.373 INFO:tasks.workunit.client.0.smithi117.stdout:3/433: fdatasync d0/d5/d21/d88/d69/f6d 0 2024-04-20T11:09:05.374 INFO:tasks.workunit.client.0.smithi117.stdout:1/408: fsync d6/d16/d3a/d5a/d5c/d61/f77 0 2024-04-20T11:09:05.376 INFO:tasks.workunit.client.0.smithi117.stdout:3/434: stat d0/d5/d7/dd/f3a 0 2024-04-20T11:09:05.376 INFO:tasks.workunit.client.0.smithi117.stdout:2/548: dwrite d1/d20/fa9 [0,4194304] 0 2024-04-20T11:09:05.377 INFO:tasks.workunit.client.0.smithi117.stdout:9/459: unlink d4/d5/da/l27 0 2024-04-20T11:09:05.385 INFO:tasks.workunit.client.0.smithi117.stdout:8/332: symlink d0/d8/d11/d6d/l6f 0 2024-04-20T11:09:05.397 INFO:tasks.workunit.client.0.smithi117.stdout:5/601: dwrite de/f14 [0,4194304] 0 2024-04-20T11:09:05.398 INFO:tasks.workunit.client.0.smithi117.stdout:9/460: mknod d4/d30/d31/ca3 0 2024-04-20T11:09:05.402 INFO:tasks.workunit.client.0.smithi117.stdout:8/333: creat d0/d8/d39/d40/f70 x:0 0 0 2024-04-20T11:09:05.402 INFO:tasks.workunit.client.0.smithi117.stdout:3/435: dwrite d0/d5/de/f92 [0,4194304] 0 2024-04-20T11:09:05.403 INFO:tasks.workunit.client.0.smithi117.stdout:8/334: readlink d0/l50 0 2024-04-20T11:09:05.404 INFO:tasks.workunit.client.0.smithi117.stdout:3/436: fsync d0/d5/d7/dd/f96 0 2024-04-20T11:09:05.408 INFO:tasks.workunit.client.0.smithi117.stdout:8/335: fdatasync d0/d8/d11/d24/d27/d47/f4f 0 2024-04-20T11:09:05.417 INFO:tasks.workunit.client.0.smithi117.stdout:9/461: truncate d4/d5/da/f43 1229612 0 2024-04-20T11:09:05.418 INFO:tasks.workunit.client.0.smithi117.stdout:3/437: unlink d0/d5/d7/f50 0 2024-04-20T11:09:05.420 INFO:tasks.workunit.client.0.smithi117.stdout:8/336: rename d0/f7 to d0/d8/d11/d6d/d58/f71 0 2024-04-20T11:09:05.425 INFO:tasks.workunit.client.0.smithi117.stdout:3/438: chown d0/c4c 174 1 2024-04-20T11:09:05.426 INFO:tasks.workunit.client.0.smithi117.stdout:5/602: getdents de/d16 0 2024-04-20T11:09:05.435 INFO:tasks.workunit.client.0.smithi117.stdout:4/454: dwrite d1/dd/f53 [0,4194304] 0 2024-04-20T11:09:05.441 INFO:tasks.workunit.client.0.smithi117.stdout:9/462: dread d4/de/f1b [0,4194304] 0 2024-04-20T11:09:05.441 INFO:tasks.workunit.client.0.smithi117.stdout:8/337: rmdir d0/d8/d11/d24/d2b 39 2024-04-20T11:09:05.441 INFO:tasks.workunit.client.0.smithi117.stdout:6/486: dwrite d3/df/f21 [0,4194304] 0 2024-04-20T11:09:05.447 INFO:tasks.workunit.client.0.smithi117.stdout:0/409: dwrite d3/d7/d20/f68 [0,4194304] 0 2024-04-20T11:09:05.453 INFO:tasks.workunit.client.0.smithi117.stdout:0/410: chown d3/d7/d16/f2b 95 1 2024-04-20T11:09:05.455 INFO:tasks.workunit.client.0.smithi117.stdout:1/409: dread d6/d35/f56 [0,4194304] 0 2024-04-20T11:09:05.464 INFO:tasks.workunit.client.0.smithi117.stdout:0/411: symlink d3/d7/d16/l8c 0 2024-04-20T11:09:05.466 INFO:tasks.workunit.client.0.smithi117.stdout:8/338: write d0/d8/d11/d24/d2b/f2c [261562,48722] 0 2024-04-20T11:09:05.466 INFO:tasks.workunit.client.0.smithi117.stdout:9/463: symlink d4/d4a/d5e/d9b/la4 0 2024-04-20T11:09:05.469 INFO:tasks.workunit.client.0.smithi117.stdout:0/412: fsync d3/d5/f37 0 2024-04-20T11:09:05.470 INFO:tasks.workunit.client.0.smithi117.stdout:6/487: mknod d3/d4/ca1 0 2024-04-20T11:09:05.473 INFO:tasks.workunit.client.0.smithi117.stdout:1/410: creat d6/d1f/d30/d49/d69/f82 x:0 0 0 2024-04-20T11:09:05.476 INFO:tasks.workunit.client.0.smithi117.stdout:0/413: write d3/d7/d20/d6e/f84 [1133809,15584] 0 2024-04-20T11:09:05.476 INFO:tasks.workunit.client.0.smithi117.stdout:5/603: rename de/d21/d35/d57/c64 to de/d21/d45/d48/d4e/cca 0 2024-04-20T11:09:05.484 INFO:tasks.workunit.client.0.smithi117.stdout:6/488: creat d3/d4/d2e/d4e/fa2 x:0 0 0 2024-04-20T11:09:05.487 INFO:tasks.workunit.client.0.smithi117.stdout:4/455: link d1/d45/d4a/c69 d1/dd/d1a/c9c 0 2024-04-20T11:09:05.488 INFO:tasks.workunit.client.0.smithi117.stdout:3/439: dwrite d0/d5/d7/dd/f10 [0,4194304] 0 2024-04-20T11:09:05.490 INFO:tasks.workunit.client.0.smithi117.stdout:9/464: dread d4/d5/da/f93 [0,4194304] 0 2024-04-20T11:09:05.491 INFO:tasks.workunit.client.0.smithi117.stdout:5/604: creat de/d21/d45/d77/fcb x:0 0 0 2024-04-20T11:09:05.496 INFO:tasks.workunit.client.0.smithi117.stdout:8/339: write d0/d8/d11/d24/d2b/f59 [1466341,107390] 0 2024-04-20T11:09:05.506 INFO:tasks.workunit.client.0.smithi117.stdout:8/340: mkdir d0/d8/d11/d24/d2b/d72 0 2024-04-20T11:09:05.508 INFO:tasks.workunit.client.0.smithi117.stdout:3/440: mkdir d0/d5/d21/d94/d9a 0 2024-04-20T11:09:05.518 INFO:tasks.workunit.client.0.smithi117.stdout:6/489: mkdir d3/d4/d2e/d4e/d71/da3 0 2024-04-20T11:09:05.518 INFO:tasks.workunit.client.0.smithi117.stdout:5/605: dwrite de/d16/f97 [4194304,4194304] 0 2024-04-20T11:09:05.531 INFO:tasks.workunit.client.0.smithi117.stdout:0/414: truncate d3/d7/d8/df/d22/f75 2796 0 2024-04-20T11:09:05.538 INFO:tasks.workunit.client.0.smithi117.stdout:9/465: dread d4/d5/f2b [0,4194304] 0 2024-04-20T11:09:05.544 INFO:tasks.workunit.client.0.smithi117.stdout:8/341: dwrite d0/d8/d11/d24/d53/f56 [0,4194304] 0 2024-04-20T11:09:05.545 INFO:tasks.workunit.client.0.smithi117.stdout:1/411: dread d6/d1f/f3e [0,4194304] 0 2024-04-20T11:09:05.548 INFO:tasks.workunit.client.0.smithi117.stdout:5/606: dwrite de/d41/d47/d4c/fb1 [0,4194304] 0 2024-04-20T11:09:05.548 INFO:tasks.workunit.client.0.smithi117.stdout:1/412: write d6/d16/f62 [689559,32057] 0 2024-04-20T11:09:05.559 INFO:tasks.workunit.client.0.smithi117.stdout:6/490: creat d3/d3a/d7f/fa4 x:0 0 0 2024-04-20T11:09:05.564 INFO:tasks.workunit.client.0.smithi117.stdout:5/607: creat de/d21/d51/fcc x:0 0 0 2024-04-20T11:09:05.564 INFO:tasks.workunit.client.0.smithi117.stdout:3/441: write d0/d5/d21/f28 [386562,14849] 0 2024-04-20T11:09:05.568 INFO:tasks.workunit.client.0.smithi117.stdout:8/342: dwrite d0/d8/d39/d40/f70 [0,4194304] 0 2024-04-20T11:09:05.589 INFO:tasks.workunit.client.0.smithi117.stdout:3/442: dwrite d0/d5/de/f92 [0,4194304] 0 2024-04-20T11:09:05.591 INFO:tasks.workunit.client.0.smithi117.stdout:0/415: dwrite d3/d7/d20/f25 [0,4194304] 0 2024-04-20T11:09:05.593 INFO:tasks.workunit.client.0.smithi117.stdout:3/443: write d0/d5/d7/dd/ff [4429600,17845] 0 2024-04-20T11:09:05.598 INFO:tasks.workunit.client.0.smithi117.stdout:9/466: truncate d4/d5/d57/f99 3671783 0 2024-04-20T11:09:05.611 INFO:tasks.workunit.client.0.smithi117.stdout:3/444: creat d0/d5/d21/d88/d69/f9b x:0 0 0 2024-04-20T11:09:05.611 INFO:tasks.workunit.client.0.smithi117.stdout:1/413: dwrite d6/d35/f56 [4194304,4194304] 0 2024-04-20T11:09:05.613 INFO:tasks.workunit.client.0.smithi117.stdout:5/608: creat de/d21/d45/d48/da7/fcd x:0 0 0 2024-04-20T11:09:05.615 INFO:tasks.workunit.client.0.smithi117.stdout:1/414: chown d6/d21 3555 1 2024-04-20T11:09:05.617 INFO:tasks.workunit.client.0.smithi117.stdout:5/609: write de/d41/d47/d55/d56/f78 [2136170,59394] 0 2024-04-20T11:09:05.619 INFO:tasks.workunit.client.0.smithi117.stdout:6/491: truncate d3/d4/f2f 487980 0 2024-04-20T11:09:05.620 INFO:tasks.workunit.client.0.smithi117.stdout:0/416: creat d3/d7/d20/d47/d78/f8d x:0 0 0 2024-04-20T11:09:05.621 INFO:tasks.workunit.client.0.smithi117.stdout:6/492: fdatasync d3/df/f14 0 2024-04-20T11:09:05.627 INFO:tasks.workunit.client.0.smithi117.stdout:9/467: readlink d4/d5/l55 0 2024-04-20T11:09:05.628 INFO:tasks.workunit.client.0.smithi117.stdout:8/343: getdents d0/d8/d11/d24 0 2024-04-20T11:09:05.635 INFO:tasks.workunit.client.0.smithi117.stdout:3/445: creat d0/d5/d21/f9c x:0 0 0 2024-04-20T11:09:05.639 INFO:tasks.workunit.client.0.smithi117.stdout:6/493: write d3/d3a/f6b [715344,36026] 0 2024-04-20T11:09:05.656 INFO:tasks.workunit.client.0.smithi117.stdout:6/494: symlink d3/d4/d2e/d3f/la5 0 2024-04-20T11:09:05.662 INFO:tasks.workunit.client.0.smithi117.stdout:8/344: fdatasync d0/d8/d11/d6d/d58/f71 0 2024-04-20T11:09:05.668 INFO:tasks.workunit.client.0.smithi117.stdout:1/415: rename d6/d16/d3a/d5a/d5c/d61/f77 to d6/d16/d3a/d5a/d5c/d61/f83 0 2024-04-20T11:09:05.673 INFO:tasks.workunit.client.0.smithi117.stdout:6/495: readlink d3/d4/d2e/d37/d8e/l9c 0 2024-04-20T11:09:05.674 INFO:tasks.workunit.client.0.smithi117.stdout:3/446: dread d0/d5/d21/f41 [0,4194304] 0 2024-04-20T11:09:05.677 INFO:tasks.workunit.client.0.smithi117.stdout:0/417: truncate d3/d7/d8/df/f1b 4022123 0 2024-04-20T11:09:05.683 INFO:tasks.workunit.client.0.smithi117.stdout:9/468: getdents d4/d4a 0 2024-04-20T11:09:05.692 INFO:tasks.workunit.client.0.smithi117.stdout:0/418: symlink d3/d7/d8/l8e 0 2024-04-20T11:09:05.696 INFO:tasks.workunit.client.0.smithi117.stdout:0/419: symlink d3/d7/d20/l8f 0 2024-04-20T11:09:05.697 INFO:tasks.workunit.client.0.smithi117.stdout:9/469: dwrite d4/d35/f3d [0,4194304] 0 2024-04-20T11:09:05.703 INFO:tasks.workunit.client.0.smithi117.stdout:9/470: write d4/de/d4b/f58 [8663662,32396] 0 2024-04-20T11:09:05.705 INFO:tasks.workunit.client.0.smithi117.stdout:6/496: truncate d3/d4/f19 199041 0 2024-04-20T11:09:05.711 INFO:tasks.workunit.client.0.smithi117.stdout:0/420: dwrite d3/d7/d16/f2e [0,4194304] 0 2024-04-20T11:09:05.716 INFO:tasks.workunit.client.0.smithi117.stdout:0/421: creat d3/d7/d8/df/d6d/f90 x:0 0 0 2024-04-20T11:09:05.722 INFO:tasks.workunit.client.0.smithi117.stdout:9/471: dwrite d4/d4a/d92/f9f [0,4194304] 0 2024-04-20T11:09:05.730 INFO:tasks.workunit.client.0.smithi117.stdout:0/422: dread d3/d7/d20/f68 [0,4194304] 0 2024-04-20T11:09:05.732 INFO:tasks.workunit.client.0.smithi117.stdout:0/423: read d3/d5/f5a [384744,110858] 0 2024-04-20T11:09:05.752 INFO:tasks.workunit.client.0.smithi117.stdout:9/472: dwrite d4/d30/f4e [0,4194304] 0 2024-04-20T11:09:05.814 INFO:tasks.workunit.client.0.smithi117.stdout:4/456: sync 2024-04-20T11:09:05.814 INFO:tasks.workunit.client.0.smithi117.stdout:2/549: sync 2024-04-20T11:09:05.814 INFO:tasks.workunit.client.0.smithi117.stdout:7/460: sync 2024-04-20T11:09:05.825 INFO:tasks.workunit.client.0.smithi117.stdout:2/550: chown d1/d4/d83/l33 1721 1 2024-04-20T11:09:05.826 INFO:tasks.workunit.client.0.smithi117.stdout:0/424: dwrite d3/d7/d20/d47/d78/f7c [0,4194304] 0 2024-04-20T11:09:05.827 INFO:tasks.workunit.client.0.smithi117.stdout:2/551: fdatasync d1/d4/d83/f23 0 2024-04-20T11:09:05.833 INFO:tasks.workunit.client.0.smithi117.stdout:2/552: mkdir d1/d4/d83/d29/daa 0 2024-04-20T11:09:05.835 INFO:tasks.workunit.client.0.smithi117.stdout:7/461: getdents d1/d43/d59 0 2024-04-20T11:09:05.838 INFO:tasks.workunit.client.0.smithi117.stdout:0/425: dread d3/d7/d2c/f6b [0,4194304] 0 2024-04-20T11:09:05.839 INFO:tasks.workunit.client.0.smithi117.stdout:7/462: truncate d1/d1d/d42/f8b 1132198 0 2024-04-20T11:09:05.840 INFO:tasks.workunit.client.0.smithi117.stdout:0/426: chown d3/d5/d74 3142 1 2024-04-20T11:09:05.843 INFO:tasks.workunit.client.0.smithi117.stdout:6/497: dread d3/d4/f46 [0,4194304] 0 2024-04-20T11:09:05.845 INFO:tasks.workunit.client.0.smithi117.stdout:6/498: readlink d3/d5a/d74/l75 0 2024-04-20T11:09:05.849 INFO:tasks.workunit.client.0.smithi117.stdout:0/427: link d3/d7/d8/df/d22/c57 d3/d7/d8/df/d22/d85/c91 0 2024-04-20T11:09:05.858 INFO:tasks.workunit.client.0.smithi117.stdout:0/428: unlink d3/d7/d20/d39/c48 0 2024-04-20T11:09:05.861 INFO:tasks.workunit.client.0.smithi117.stdout:2/553: dread d1/d4/d83/f18 [0,4194304] 0 2024-04-20T11:09:05.863 INFO:tasks.workunit.client.0.smithi117.stdout:0/429: truncate d3/f38 64588 0 2024-04-20T11:09:05.866 INFO:tasks.workunit.client.0.smithi117.stdout:7/463: dread d1/d43/f73 [0,4194304] 0 2024-04-20T11:09:05.868 INFO:tasks.workunit.client.0.smithi117.stdout:9/473: truncate d4/f13 704752 0 2024-04-20T11:09:05.869 INFO:tasks.workunit.client.0.smithi117.stdout:7/464: fsync d1/d4/d6/d17/f3a 0 2024-04-20T11:09:05.872 INFO:tasks.workunit.client.0.smithi117.stdout:0/430: link d3/d7/d8/df/d22/f6f d3/d7/d20/d39/f92 0 2024-04-20T11:09:05.875 INFO:tasks.workunit.client.0.smithi117.stdout:9/474: mkdir d4/d5/d57/d82/da5 0 2024-04-20T11:09:05.877 INFO:tasks.workunit.client.0.smithi117.stdout:0/431: symlink d3/d7/d16/l93 0 2024-04-20T11:09:05.878 INFO:tasks.workunit.client.0.smithi117.stdout:7/465: creat d1/d1d/d4e/d53/d55/f98 x:0 0 0 2024-04-20T11:09:05.880 INFO:tasks.workunit.client.0.smithi117.stdout:6/499: dread d3/df/f14 [0,4194304] 0 2024-04-20T11:09:05.883 INFO:tasks.workunit.client.0.smithi117.stdout:0/432: unlink d3/d7/f3b 0 2024-04-20T11:09:05.886 INFO:tasks.workunit.client.0.smithi117.stdout:7/466: unlink d1/d1d/f70 0 2024-04-20T11:09:05.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:05 smithi117 ceph-mon[15659]: pgmap v12: 65 pgs: 65 active+clean; 856 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 110 MiB/s wr, 312 op/s 2024-04-20T11:09:05.894 INFO:tasks.workunit.client.0.smithi117.stdout:0/433: rename d3/d7/d20/d39/d53 to d3/d7/d94 0 2024-04-20T11:09:05.894 INFO:tasks.workunit.client.0.smithi117.stdout:4/457: write d1/dd/f13 [431094,36199] 0 2024-04-20T11:09:05.897 INFO:tasks.workunit.client.0.smithi117.stdout:9/475: read d4/f13 [482943,104262] 0 2024-04-20T11:09:05.898 INFO:tasks.workunit.client.0.smithi117.stdout:0/434: truncate d3/d7/d16/f2e 9383044 0 2024-04-20T11:09:05.899 INFO:tasks.workunit.client.0.smithi117.stdout:6/500: creat d3/d4/d2e/fa6 x:0 0 0 2024-04-20T11:09:05.902 INFO:tasks.workunit.client.0.smithi117.stdout:0/435: read d3/d7/d20/d47/d78/f7c [1389200,13702] 0 2024-04-20T11:09:05.902 INFO:tasks.workunit.client.0.smithi117.stdout:6/501: readlink d3/d5a/d79/l80 0 2024-04-20T11:09:05.905 INFO:tasks.workunit.client.0.smithi117.stdout:7/467: dwrite d1/d1d/d42/d2d/d6c/f88 [0,4194304] 0 2024-04-20T11:09:05.906 INFO:tasks.workunit.client.0.smithi117.stdout:6/502: write d3/d5a/d74/f93 [279558,16861] 0 2024-04-20T11:09:05.908 INFO:tasks.workunit.client.0.smithi117.stdout:0/436: read d3/d5/f37 [883126,131015] 0 2024-04-20T11:09:05.909 INFO:tasks.workunit.client.0.smithi117.stdout:7/468: write d1/d4/d6/d17/f3a [3803241,89692] 0 2024-04-20T11:09:05.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:05 smithi173 ceph-mon[23800]: pgmap v12: 65 pgs: 65 active+clean; 856 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 110 MiB/s wr, 312 op/s 2024-04-20T11:09:05.912 INFO:tasks.workunit.client.0.smithi117.stdout:7/469: fdatasync d1/d4/d6/d2b/f33 0 2024-04-20T11:09:05.914 INFO:tasks.workunit.client.0.smithi117.stdout:6/503: rename d3/d4/d2e/d4e/f98 to d3/d5a/d79/fa7 0 2024-04-20T11:09:05.915 INFO:tasks.workunit.client.0.smithi117.stdout:0/437: truncate d3/d7/f9 2347491 0 2024-04-20T11:09:05.916 INFO:tasks.workunit.client.0.smithi117.stdout:7/470: truncate d1/d4/d6/d17/f4c 1910095 0 2024-04-20T11:09:05.917 INFO:tasks.workunit.client.0.smithi117.stdout:7/471: fdatasync d1/d43/f91 0 2024-04-20T11:09:05.917 INFO:tasks.workunit.client.0.smithi117.stdout:0/438: chown d3/d7/f7d 17928 1 2024-04-20T11:09:05.918 INFO:tasks.workunit.client.0.smithi117.stdout:9/476: dwrite d4/d68/d6e/f77 [0,4194304] 0 2024-04-20T11:09:05.920 INFO:tasks.workunit.client.0.smithi117.stdout:9/477: fsync d4/d5/f1e 0 2024-04-20T11:09:05.922 INFO:tasks.workunit.client.0.smithi117.stdout:0/439: creat d3/d7/d20/d47/d78/f95 x:0 0 0 2024-04-20T11:09:05.925 INFO:tasks.workunit.client.0.smithi117.stdout:0/440: creat d3/d7/f96 x:0 0 0 2024-04-20T11:09:05.926 INFO:tasks.workunit.client.0.smithi117.stdout:9/478: symlink d4/d5/d3c/la6 0 2024-04-20T11:09:05.928 INFO:tasks.workunit.client.0.smithi117.stdout:9/479: chown c3 7307945 1 2024-04-20T11:09:05.941 INFO:tasks.workunit.client.0.smithi117.stdout:6/504: write d3/df/d39/f3e [657690,30259] 0 2024-04-20T11:09:05.949 INFO:tasks.workunit.client.0.smithi117.stdout:0/441: truncate d3/d7/d20/f25 4156199 0 2024-04-20T11:09:05.955 INFO:tasks.workunit.client.0.smithi117.stdout:7/472: dwrite d1/f10 [0,4194304] 0 2024-04-20T11:09:05.955 INFO:tasks.workunit.client.0.smithi117.stdout:0/442: write d3/d7/d2c/f69 [5240870,57828] 0 2024-04-20T11:09:05.961 INFO:tasks.workunit.client.0.smithi117.stdout:0/443: creat d3/d7/d8/df/d6d/f97 x:0 0 0 2024-04-20T11:09:05.963 INFO:tasks.workunit.client.0.smithi117.stdout:9/480: dwrite d4/d5/da/f49 [0,4194304] 0 2024-04-20T11:09:05.965 INFO:tasks.workunit.client.0.smithi117.stdout:0/444: rename d3/d7 to d3/d7/d8/df/d6d/d98 22 2024-04-20T11:09:05.977 INFO:tasks.workunit.client.0.smithi117.stdout:6/505: dread d3/d3a/f6b [0,4194304] 0 2024-04-20T11:09:05.979 INFO:tasks.workunit.client.0.smithi117.stdout:9/481: getdents d4/d4a 0 2024-04-20T11:09:05.981 INFO:tasks.workunit.client.0.smithi117.stdout:6/506: creat d3/d4/d2e/d85/fa8 x:0 0 0 2024-04-20T11:09:05.981 INFO:tasks.workunit.client.0.smithi117.stdout:9/482: chown d4/de/d10/f38 69413 1 2024-04-20T11:09:05.983 INFO:tasks.workunit.client.0.smithi117.stdout:0/445: write d3/d7/f19 [1667250,69971] 0 2024-04-20T11:09:05.986 INFO:tasks.workunit.client.0.smithi117.stdout:9/483: read f0 [5642557,69185] 0 2024-04-20T11:09:05.986 INFO:tasks.workunit.client.0.smithi117.stdout:0/446: unlink d3/d7/le 0 2024-04-20T11:09:05.989 INFO:tasks.workunit.client.0.smithi117.stdout:9/484: unlink d4/de/l71 0 2024-04-20T11:09:05.993 INFO:tasks.workunit.client.0.smithi117.stdout:9/485: fsync d4/d30/f36 0 2024-04-20T11:09:05.997 INFO:tasks.workunit.client.0.smithi117.stdout:0/447: link d3/d7/d2c/d52/l79 d3/d7/d94/l99 0 2024-04-20T11:09:06.001 INFO:tasks.workunit.client.0.smithi117.stdout:9/486: mknod d4/d30/d31/ca7 0 2024-04-20T11:09:06.002 INFO:tasks.workunit.client.0.smithi117.stdout:7/473: dread d1/d4/d6/d2b/f33 [0,4194304] 0 2024-04-20T11:09:06.006 INFO:tasks.workunit.client.0.smithi117.stdout:9/487: symlink d4/d4a/d5e/la8 0 2024-04-20T11:09:06.008 INFO:tasks.workunit.client.0.smithi117.stdout:7/474: symlink d1/d4/d6/d54/d63/l99 0 2024-04-20T11:09:06.010 INFO:tasks.workunit.client.0.smithi117.stdout:6/507: dwrite d3/d4/d2e/d5f/f6c [0,4194304] 0 2024-04-20T11:09:06.014 INFO:tasks.workunit.client.0.smithi117.stdout:7/475: creat d1/d1d/d4e/f9a x:0 0 0 2024-04-20T11:09:06.019 INFO:tasks.workunit.client.0.smithi117.stdout:0/448: write d3/d5/f5a [43829,128272] 0 2024-04-20T11:09:06.019 INFO:tasks.workunit.client.0.smithi117.stdout:6/508: mknod d3/d5a/d79/ca9 0 2024-04-20T11:09:06.019 INFO:tasks.workunit.client.0.smithi117.stdout:6/509: mkdir d3/df/d39/daa 0 2024-04-20T11:09:06.020 INFO:tasks.workunit.client.0.smithi117.stdout:6/510: chown d3/d4/d2e/d37/f7e 21019091 1 2024-04-20T11:09:06.031 INFO:tasks.workunit.client.0.smithi117.stdout:9/488: write d4/f39 [421199,79667] 0 2024-04-20T11:09:06.034 INFO:tasks.workunit.client.0.smithi117.stdout:9/489: mknod d4/d30/d78/ca9 0 2024-04-20T11:09:06.037 INFO:tasks.workunit.client.0.smithi117.stdout:6/511: dwrite d3/d5a/d74/f93 [0,4194304] 0 2024-04-20T11:09:06.038 INFO:tasks.workunit.client.0.smithi117.stdout:7/476: dread d1/d4/d39/f7a [0,4194304] 0 2024-04-20T11:09:06.040 INFO:tasks.workunit.client.0.smithi117.stdout:7/477: fdatasync d1/d1d/f87 0 2024-04-20T11:09:06.042 INFO:tasks.workunit.client.0.smithi117.stdout:9/490: getdents d4/de 0 2024-04-20T11:09:06.044 INFO:tasks.workunit.client.0.smithi117.stdout:7/478: write d1/d1d/f5c [652003,57036] 0 2024-04-20T11:09:06.051 INFO:tasks.workunit.client.0.smithi117.stdout:6/512: truncate d3/d4/d2e/d3f/f82 2865650 0 2024-04-20T11:09:06.061 INFO:tasks.workunit.client.0.smithi117.stdout:0/449: dread d3/d5/f37 [0,4194304] 0 2024-04-20T11:09:06.066 INFO:tasks.workunit.client.0.smithi117.stdout:6/513: truncate d3/df/f31 2510833 0 2024-04-20T11:09:06.068 INFO:tasks.workunit.client.0.smithi117.stdout:0/450: mkdir d3/d7/d20/d39/d9a 0 2024-04-20T11:09:06.072 INFO:tasks.workunit.client.0.smithi117.stdout:6/514: unlink d3/d5a/d79/f6f 0 2024-04-20T11:09:06.085 INFO:tasks.workunit.client.0.smithi117.stdout:6/515: chown d3/df/f31 13 1 2024-04-20T11:09:06.089 INFO:tasks.workunit.client.0.smithi117.stdout:6/516: mknod d3/df/d39/daa/cab 0 2024-04-20T11:09:06.093 INFO:tasks.workunit.client.0.smithi117.stdout:6/517: getdents d3/d50 0 2024-04-20T11:09:06.095 INFO:tasks.workunit.client.0.smithi117.stdout:6/518: write d3/d5a/d79/f95 [734153,78171] 0 2024-04-20T11:09:06.095 INFO:tasks.workunit.client.0.smithi117.stdout:0/451: dread d3/d7/f3f [0,4194304] 0 2024-04-20T11:09:06.099 INFO:tasks.workunit.client.0.smithi117.stdout:0/452: creat d3/d7/d8/df/d6d/f9b x:0 0 0 2024-04-20T11:09:06.102 INFO:tasks.workunit.client.0.smithi117.stdout:0/453: creat d3/d7/d8/df/d6d/f9c x:0 0 0 2024-04-20T11:09:06.106 INFO:tasks.workunit.client.0.smithi117.stdout:6/519: dread d3/d3a/d7f/f7a [0,4194304] 0 2024-04-20T11:09:06.108 INFO:tasks.workunit.client.0.smithi117.stdout:6/520: readlink d3/df/d39/l52 0 2024-04-20T11:09:06.108 INFO:tasks.workunit.client.0.smithi117.stdout:0/454: link d3/d7/d8/df/d22/d85/c91 d3/d5/d74/c9d 0 2024-04-20T11:09:06.109 INFO:tasks.workunit.client.0.smithi117.stdout:6/521: chown d3/d50/d8a/l9e 24534 1 2024-04-20T11:09:06.111 INFO:tasks.workunit.client.0.smithi117.stdout:6/522: write d3/f89 [177214,87594] 0 2024-04-20T11:09:06.111 INFO:tasks.workunit.client.0.smithi117.stdout:0/455: truncate d3/d7/d16/f2b 178410 0 2024-04-20T11:09:06.112 INFO:tasks.workunit.client.0.smithi117.stdout:0/456: chown d3 236801 1 2024-04-20T11:09:06.115 INFO:tasks.workunit.client.0.smithi117.stdout:6/523: link d3/f18 d3/d3a/fac 0 2024-04-20T11:09:06.126 INFO:tasks.workunit.client.0.smithi117.stdout:0/457: dwrite d3/d7/d20/d47/d78/f8d [0,4194304] 0 2024-04-20T11:09:06.128 INFO:tasks.workunit.client.0.smithi117.stdout:6/524: write d3/d4/f2f [3079,24143] 0 2024-04-20T11:09:06.128 INFO:tasks.workunit.client.0.smithi117.stdout:0/458: fdatasync d3/d7/d20/d6e/f84 0 2024-04-20T11:09:06.132 INFO:tasks.workunit.client.0.smithi117.stdout:0/459: mknod d3/d5/c9e 0 2024-04-20T11:09:06.134 INFO:tasks.workunit.client.0.smithi117.stdout:6/525: creat d3/d4/d2e/d4e/d71/da3/fad x:0 0 0 2024-04-20T11:09:06.134 INFO:tasks.workunit.client.0.smithi117.stdout:0/460: dread - d3/d7/f7d zero size 2024-04-20T11:09:06.136 INFO:tasks.workunit.client.0.smithi117.stdout:6/526: chown d3/df/d39/d99/f8c 881 1 2024-04-20T11:09:06.139 INFO:tasks.workunit.client.0.smithi117.stdout:6/527: dread - d3/d4/d2e/d3f/f77 zero size 2024-04-20T11:09:06.141 INFO:tasks.workunit.client.0.smithi117.stdout:6/528: chown d3/f2c 408738614 1 2024-04-20T11:09:06.149 INFO:tasks.workunit.client.0.smithi117.stdout:0/461: truncate d3/d5/f5a 689972 0 2024-04-20T11:09:06.158 INFO:tasks.workunit.client.0.smithi117.stdout:6/529: dwrite d3/d4/d2e/d3f/f8b [0,4194304] 0 2024-04-20T11:09:06.177 INFO:tasks.workunit.client.0.smithi117.stdout:6/530: dwrite d3/d4/d2e/f40 [0,4194304] 0 2024-04-20T11:09:06.180 INFO:tasks.workunit.client.0.smithi117.stdout:0/462: dread d3/d5/f59 [0,4194304] 0 2024-04-20T11:09:06.183 INFO:tasks.workunit.client.0.smithi117.stdout:0/463: fdatasync d3/d7/d2c/f6b 0 2024-04-20T11:09:06.185 INFO:tasks.workunit.client.0.smithi117.stdout:6/531: dread d3/d3a/f6b [0,4194304] 0 2024-04-20T11:09:06.188 INFO:tasks.workunit.client.0.smithi117.stdout:0/464: fsync d3/d5/f5a 0 2024-04-20T11:09:06.195 INFO:tasks.workunit.client.0.smithi117.stdout:0/465: rename d3/d7/d20/d47/d78/f95 to d3/d7/d20/d39/d3a/f9f 0 2024-04-20T11:09:06.197 INFO:tasks.workunit.client.0.smithi117.stdout:0/466: stat d3/d7/d8/df/l82 0 2024-04-20T11:09:06.200 INFO:tasks.workunit.client.0.smithi117.stdout:6/532: dwrite d3/d4/d2e/d3f/f8b [0,4194304] 0 2024-04-20T11:09:06.206 INFO:tasks.workunit.client.0.smithi117.stdout:6/533: link d3/d4/ca1 d3/d4/d2e/d4e/d71/da3/cae 0 2024-04-20T11:09:06.213 INFO:tasks.workunit.client.0.smithi117.stdout:0/467: dwrite d3/d7/d8/df/d6d/f9b [0,4194304] 0 2024-04-20T11:09:06.214 INFO:tasks.workunit.client.0.smithi117.stdout:6/534: dread d3/d4/f19 [0,4194304] 0 2024-04-20T11:09:06.215 INFO:tasks.workunit.client.0.smithi117.stdout:0/468: dread - d3/d7/d2c/f65 zero size 2024-04-20T11:09:06.221 INFO:tasks.workunit.client.0.smithi117.stdout:5/610: sync 2024-04-20T11:09:06.229 INFO:tasks.workunit.client.0.smithi117.stdout:5/611: unlink de/d41/d47/d4c/f8f 0 2024-04-20T11:09:06.230 INFO:tasks.workunit.client.0.smithi117.stdout:5/612: chown de/d5b/d65 1025905 1 2024-04-20T11:09:06.232 INFO:tasks.workunit.client.0.smithi117.stdout:6/535: link d3/d4/d2e/d37/l96 d3/d4/d2e/d5f/laf 0 2024-04-20T11:09:06.232 INFO:tasks.workunit.client.0.smithi117.stdout:0/469: rename d3/d5/c1f to d3/ca0 0 2024-04-20T11:09:06.232 INFO:tasks.workunit.client.0.smithi117.stdout:5/613: readlink de/d16/l6b 0 2024-04-20T11:09:06.233 INFO:tasks.workunit.client.0.smithi117.stdout:6/536: mknod d3/df/d39/daa/cb0 0 2024-04-20T11:09:06.234 INFO:tasks.workunit.client.0.smithi117.stdout:0/470: mknod d3/d7/d94/ca1 0 2024-04-20T11:09:06.235 INFO:tasks.workunit.client.0.smithi117.stdout:5/614: mkdir de/d21/d45/d48/d4e/dc1/dce 0 2024-04-20T11:09:06.236 INFO:tasks.workunit.client.0.smithi117.stdout:0/471: read - d3/d7/d20/d39/f92 zero size 2024-04-20T11:09:06.237 INFO:tasks.workunit.client.0.smithi117.stdout:6/537: unlink d3/d4/f2f 0 2024-04-20T11:09:06.240 INFO:tasks.workunit.client.0.smithi117.stdout:5/615: unlink de/d41/d47/d55/d56/l91 0 2024-04-20T11:09:06.242 INFO:tasks.workunit.client.0.smithi117.stdout:5/616: read - de/d21/d45/d48/da7/fcd zero size 2024-04-20T11:09:06.242 INFO:tasks.workunit.client.0.smithi117.stdout:6/538: mknod d3/d4/d2e/cb1 0 2024-04-20T11:09:06.252 INFO:tasks.workunit.client.0.smithi117.stdout:6/539: dread d3/df/f31 [0,4194304] 0 2024-04-20T11:09:06.253 INFO:tasks.workunit.client.0.smithi117.stdout:0/472: dwrite d3/d7/d2c/d52/f70 [0,4194304] 0 2024-04-20T11:09:06.253 INFO:tasks.workunit.client.0.smithi117.stdout:5/617: rename de/d21/d45/d48/d4e/cca to de/d21/d35/d57/ccf 0 2024-04-20T11:09:06.260 INFO:tasks.workunit.client.0.smithi117.stdout:6/540: truncate d3/d4/d2e/d4e/d71/da3/fad 1002403 0 2024-04-20T11:09:06.262 INFO:tasks.workunit.client.0.smithi117.stdout:0/473: symlink d3/d7/d94/la2 0 2024-04-20T11:09:06.265 INFO:tasks.workunit.client.0.smithi117.stdout:6/541: dread d3/d4/d2e/d4e/d71/da3/fad [0,4194304] 0 2024-04-20T11:09:06.266 INFO:tasks.workunit.client.0.smithi117.stdout:6/542: readlink d3/d4/l51 0 2024-04-20T11:09:06.268 INFO:tasks.workunit.client.0.smithi117.stdout:6/543: mkdir d3/d4/d2e/d37/d8e/db2 0 2024-04-20T11:09:06.272 INFO:tasks.workunit.client.0.smithi117.stdout:6/544: rename d3/d4/d2e/d5f to d3/d5a/d74/db3 0 2024-04-20T11:09:06.284 INFO:tasks.workunit.client.0.smithi117.stdout:0/474: truncate d3/d7/d20/d47/d78/f8d 2184214 0 2024-04-20T11:09:06.287 INFO:tasks.workunit.client.0.smithi117.stdout:0/475: rmdir d3/d7/d20/d39 39 2024-04-20T11:09:06.292 INFO:tasks.workunit.client.0.smithi117.stdout:6/545: dread d3/f18 [0,4194304] 0 2024-04-20T11:09:06.294 INFO:tasks.workunit.client.0.smithi117.stdout:0/476: dread d3/d7/d16/f56 [0,4194304] 0 2024-04-20T11:09:06.297 INFO:tasks.workunit.client.0.smithi117.stdout:6/546: creat d3/d4/fb4 x:0 0 0 2024-04-20T11:09:06.300 INFO:tasks.workunit.client.0.smithi117.stdout:0/477: getdents d3/d7/d2c 0 2024-04-20T11:09:06.304 INFO:tasks.workunit.client.0.smithi117.stdout:0/478: creat d3/d7/d2c/d8a/d54/fa3 x:0 0 0 2024-04-20T11:09:06.305 INFO:tasks.workunit.client.0.smithi117.stdout:6/547: link d3/d4/d2e/l45 d3/d4/d2e/d4e/d71/da3/lb5 0 2024-04-20T11:09:06.309 INFO:tasks.workunit.client.0.smithi117.stdout:6/548: rename d3/df/d39/f4b to d3/d5a/d74/fb6 0 2024-04-20T11:09:06.313 INFO:tasks.workunit.client.0.smithi117.stdout:0/479: truncate d3/d5/f59 158580 0 2024-04-20T11:09:06.315 INFO:tasks.workunit.client.0.smithi117.stdout:0/480: chown d3/d7/d8/df/l82 478993128 1 2024-04-20T11:09:06.317 INFO:tasks.workunit.client.0.smithi117.stdout:0/481: dread - d3/d7/f7d zero size 2024-04-20T11:09:06.318 INFO:tasks.workunit.client.0.smithi117.stdout:0/482: stat d3/d7/d16/f56 0 2024-04-20T11:09:06.324 INFO:tasks.workunit.client.0.smithi117.stdout:6/549: dwrite d3/df/d39/f8f [0,4194304] 0 2024-04-20T11:09:06.327 INFO:tasks.workunit.client.0.smithi117.stdout:6/550: write d3/d5a/d74/db3/f6c [163283,80964] 0 2024-04-20T11:09:06.330 INFO:tasks.workunit.client.0.smithi117.stdout:6/551: write d3/d5a/d79/f6a [726907,11889] 0 2024-04-20T11:09:06.335 INFO:tasks.workunit.client.0.smithi117.stdout:0/483: dwrite d3/d7/d20/f68 [0,4194304] 0 2024-04-20T11:09:06.342 INFO:tasks.workunit.client.0.smithi117.stdout:0/484: link d3/d7/d16/c36 d3/d5/ca4 0 2024-04-20T11:09:06.353 INFO:tasks.workunit.client.0.smithi117.stdout:6/552: dwrite d3/f18 [0,4194304] 0 2024-04-20T11:09:06.358 INFO:tasks.workunit.client.0.smithi117.stdout:6/553: rename d3/d4/d2e/d85 to d3/d4/d2e/d4e/d71/db7 0 2024-04-20T11:09:06.387 INFO:tasks.workunit.client.0.smithi117.stdout:0/485: dread d3/d7/d8/f31 [0,4194304] 0 2024-04-20T11:09:06.391 INFO:tasks.workunit.client.0.smithi117.stdout:0/486: creat d3/d7/d94/fa5 x:0 0 0 2024-04-20T11:09:06.396 INFO:tasks.workunit.client.0.smithi117.stdout:0/487: rename d3/d7/d20/d39/d3a/l86 to d3/d7/d8/df/d22/d85/la6 0 2024-04-20T11:09:06.397 INFO:tasks.workunit.client.0.smithi117.stdout:0/488: chown d3/d7/d2c/d52/f70 1748780 1 2024-04-20T11:09:06.398 INFO:tasks.workunit.client.0.smithi117.stdout:0/489: readlink d3/d7/d20/d39/d3a/l3d 0 2024-04-20T11:09:06.411 INFO:tasks.workunit.client.0.smithi117.stdout:0/490: dwrite d3/d7/d8/f31 [4194304,4194304] 0 2024-04-20T11:09:06.427 INFO:tasks.workunit.client.0.smithi117.stdout:0/491: dread d3/d7/d20/f25 [0,4194304] 0 2024-04-20T11:09:06.427 INFO:tasks.workunit.client.0.smithi117.stdout:8/345: sync 2024-04-20T11:09:06.427 INFO:tasks.workunit.client.0.smithi117.stdout:1/416: sync 2024-04-20T11:09:06.430 INFO:tasks.workunit.client.0.smithi117.stdout:0/492: creat d3/d5/fa7 x:0 0 0 2024-04-20T11:09:06.433 INFO:tasks.workunit.client.0.smithi117.stdout:0/493: chown d3/d7/d2c/d8a/l4e 34 1 2024-04-20T11:09:06.434 INFO:tasks.workunit.client.0.smithi117.stdout:1/417: getdents d6/d1f/d30/d49/d69/d6e/d6f 0 2024-04-20T11:09:06.434 INFO:tasks.workunit.client.0.smithi117.stdout:0/494: stat d3/d7/d16/l45 0 2024-04-20T11:09:06.436 INFO:tasks.workunit.client.0.smithi117.stdout:1/418: chown d6/d16/d3a/d5a/d5c/d61/d71 2 1 2024-04-20T11:09:06.437 INFO:tasks.workunit.client.0.smithi117.stdout:8/346: write d0/d8/d11/d6d/d58/f71 [1867876,47530] 0 2024-04-20T11:09:06.439 INFO:tasks.workunit.client.0.smithi117.stdout:0/495: dread d3/d7/d16/f56 [0,4194304] 0 2024-04-20T11:09:06.442 INFO:tasks.workunit.client.0.smithi117.stdout:0/496: dread - d3/d7/d20/d39/d3a/f9f zero size 2024-04-20T11:09:06.443 INFO:tasks.workunit.client.0.smithi117.stdout:8/347: getdents d0/d8/d11/d24/d2b 0 2024-04-20T11:09:06.443 INFO:tasks.workunit.client.0.smithi117.stdout:1/419: dread d6/d1f/f40 [0,4194304] 0 2024-04-20T11:09:06.444 INFO:tasks.workunit.client.0.smithi117.stdout:0/497: chown d3/d7/d8/df/d6d/f90 1043176 1 2024-04-20T11:09:06.447 INFO:tasks.workunit.client.0.smithi117.stdout:0/498: write d3/d7/d94/fa5 [163967,116338] 0 2024-04-20T11:09:06.448 INFO:tasks.workunit.client.0.smithi117.stdout:8/348: creat d0/dc/d49/f73 x:0 0 0 2024-04-20T11:09:06.449 INFO:tasks.workunit.client.0.smithi117.stdout:0/499: write d3/d7/f7d [577744,51446] 0 2024-04-20T11:09:06.452 INFO:tasks.workunit.client.0.smithi117.stdout:1/420: getdents d6/d1f/d30/d49/d69 0 2024-04-20T11:09:06.452 INFO:tasks.workunit.client.0.smithi117.stdout:8/349: write d0/d8/d11/d24/d2b/f2c [1769014,98936] 0 2024-04-20T11:09:06.455 INFO:tasks.workunit.client.0.smithi117.stdout:0/500: truncate d3/d7/d20/d6e/f84 1707454 0 2024-04-20T11:09:06.455 INFO:tasks.workunit.client.0.smithi117.stdout:1/421: rmdir d6/d35/d63 39 2024-04-20T11:09:06.457 INFO:tasks.workunit.client.0.smithi117.stdout:8/350: truncate d0/d8/d11/d24/f46 4447026 0 2024-04-20T11:09:06.458 INFO:tasks.workunit.client.0.smithi117.stdout:1/422: chown d6/d16/d3a/d5a/d5c/d61/f83 79788 1 2024-04-20T11:09:06.459 INFO:tasks.workunit.client.0.smithi117.stdout:0/501: unlink d3/d5/ca4 0 2024-04-20T11:09:06.461 INFO:tasks.workunit.client.0.smithi117.stdout:1/423: write d6/f79 [4295344,101803] 0 2024-04-20T11:09:06.461 INFO:tasks.workunit.client.0.smithi117.stdout:8/351: mknod d0/d8/d11/d24/d53/c74 0 2024-04-20T11:09:06.464 INFO:tasks.workunit.client.0.smithi117.stdout:0/502: creat d3/d7/d2c/d8a/fa8 x:0 0 0 2024-04-20T11:09:06.465 INFO:tasks.workunit.client.0.smithi117.stdout:1/424: getdents d6/d1f/d30/d32 0 2024-04-20T11:09:06.467 INFO:tasks.workunit.client.0.smithi117.stdout:8/352: rename d0/d8/d11/d6d/f48 to d0/d8/d11/d24/d2b/d72/f75 0 2024-04-20T11:09:06.473 INFO:tasks.workunit.client.0.smithi117.stdout:8/353: mkdir d0/d8/d11/d24/d76 0 2024-04-20T11:09:06.477 INFO:tasks.workunit.client.0.smithi117.stdout:8/354: symlink d0/dc/d2e/l77 0 2024-04-20T11:09:06.478 INFO:tasks.workunit.client.0.smithi117.stdout:0/503: getdents d3/d7e 0 2024-04-20T11:09:06.479 INFO:tasks.workunit.client.0.smithi117.stdout:8/355: dread - d0/d8/d39/f54 zero size 2024-04-20T11:09:06.481 INFO:tasks.workunit.client.0.smithi117.stdout:0/504: rmdir d3/d7/d20/d39 39 2024-04-20T11:09:06.484 INFO:tasks.workunit.client.0.smithi117.stdout:0/505: chown d3/d7/d16/f56 199291 1 2024-04-20T11:09:06.487 INFO:tasks.workunit.client.0.smithi117.stdout:0/506: creat d3/d5/d74/fa9 x:0 0 0 2024-04-20T11:09:06.490 INFO:tasks.workunit.client.0.smithi117.stdout:0/507: mknod d3/d7/d8/df/caa 0 2024-04-20T11:09:06.493 INFO:tasks.workunit.client.0.smithi117.stdout:0/508: write d3/d7/f80 [700635,119782] 0 2024-04-20T11:09:06.497 INFO:tasks.workunit.client.0.smithi117.stdout:8/356: dwrite d0/d8/d39/d40/f70 [0,4194304] 0 2024-04-20T11:09:06.501 INFO:tasks.workunit.client.0.smithi117.stdout:0/509: rename d3/d7/d2c/d8a to d3/dab 0 2024-04-20T11:09:06.501 INFO:tasks.workunit.client.0.smithi117.stdout:1/425: dread d6/f79 [0,4194304] 0 2024-04-20T11:09:06.504 INFO:tasks.workunit.client.0.smithi117.stdout:8/357: rename d0/d8/d39/d5a to d0/d8/d39/d40/d78 0 2024-04-20T11:09:06.509 INFO:tasks.workunit.client.0.smithi117.stdout:0/510: creat d3/dab/fac x:0 0 0 2024-04-20T11:09:06.509 INFO:tasks.workunit.client.0.smithi117.stdout:8/358: symlink d0/d8/d11/d24/d53/l79 0 2024-04-20T11:09:06.510 INFO:tasks.workunit.client.0.smithi117.stdout:1/426: link d6/d16/d3a/d5a/l81 d6/d16/d3a/l84 0 2024-04-20T11:09:06.511 INFO:tasks.workunit.client.0.smithi117.stdout:8/359: dread - d0/d9/d3a/f6a zero size 2024-04-20T11:09:06.511 INFO:tasks.workunit.client.0.smithi117.stdout:0/511: dread - d3/dab/d54/fa3 zero size 2024-04-20T11:09:06.513 INFO:tasks.workunit.client.0.smithi117.stdout:1/427: dread d6/d1f/d30/f67 [0,4194304] 0 2024-04-20T11:09:06.513 INFO:tasks.workunit.client.0.smithi117.stdout:0/512: read d3/d7/f3f [3553155,119638] 0 2024-04-20T11:09:06.514 INFO:tasks.workunit.client.0.smithi117.stdout:1/428: chown d6 16302984 1 2024-04-20T11:09:06.515 INFO:tasks.workunit.client.0.smithi117.stdout:8/360: mknod d0/d8/d11/d6d/c7a 0 2024-04-20T11:09:06.516 INFO:tasks.workunit.client.0.smithi117.stdout:0/513: read d3/d7/f3f [124117,99799] 0 2024-04-20T11:09:06.516 INFO:tasks.workunit.client.0.smithi117.stdout:1/429: chown d6/d21/l58 26 1 2024-04-20T11:09:06.517 INFO:tasks.workunit.client.0.smithi117.stdout:8/361: stat d0/f5 0 2024-04-20T11:09:06.517 INFO:tasks.workunit.client.0.smithi117.stdout:4/458: sync 2024-04-20T11:09:06.518 INFO:tasks.workunit.client.0.smithi117.stdout:2/554: sync 2024-04-20T11:09:06.518 INFO:tasks.workunit.client.0.smithi117.stdout:7/479: sync 2024-04-20T11:09:06.522 INFO:tasks.workunit.client.0.smithi117.stdout:3/447: sync 2024-04-20T11:09:06.522 INFO:tasks.workunit.client.0.smithi117.stdout:9/491: sync 2024-04-20T11:09:06.522 INFO:tasks.workunit.client.0.smithi117.stdout:8/362: stat d0/d8/d11/d24/d53 0 2024-04-20T11:09:06.527 INFO:tasks.workunit.client.0.smithi117.stdout:7/480: creat d1/d4/d6/d17/d3f/f9b x:0 0 0 2024-04-20T11:09:06.530 INFO:tasks.workunit.client.0.smithi117.stdout:2/555: mknod d1/d4/cab 0 2024-04-20T11:09:06.531 INFO:tasks.workunit.client.0.smithi117.stdout:4/459: mknod d1/dd/d1a/c9d 0 2024-04-20T11:09:06.535 INFO:tasks.workunit.client.0.smithi117.stdout:0/514: dwrite d3/d7/d8/f13 [8388608,4194304] 0 2024-04-20T11:09:06.537 INFO:tasks.workunit.client.0.smithi117.stdout:7/481: mknod d1/d4/d6/d13/c9c 0 2024-04-20T11:09:06.538 INFO:tasks.workunit.client.0.smithi117.stdout:2/556: mknod d1/d4/d69/cac 0 2024-04-20T11:09:06.540 INFO:tasks.workunit.client.0.smithi117.stdout:2/557: chown d1/d4/fa0 3041768 1 2024-04-20T11:09:06.543 INFO:tasks.workunit.client.0.smithi117.stdout:2/558: write d1/d4/dd/f77 [354070,7715] 0 2024-04-20T11:09:06.546 INFO:tasks.workunit.client.0.smithi117.stdout:4/460: mkdir d1/d45/d4a/d99/d9e 0 2024-04-20T11:09:06.550 INFO:tasks.workunit.client.0.smithi117.stdout:7/482: unlink d1/d4/d6/d2b/c47 0 2024-04-20T11:09:06.552 INFO:tasks.workunit.client.0.smithi117.stdout:7/483: write d1/d1d/d42/d2d/f92 [542674,78345] 0 2024-04-20T11:09:06.552 INFO:tasks.workunit.client.0.smithi117.stdout:8/363: link d0/dc/f1a d0/d8/d11/d24/d53/f7b 0 2024-04-20T11:09:06.554 INFO:tasks.workunit.client.0.smithi117.stdout:7/484: write d1/d43/f91 [849106,109158] 0 2024-04-20T11:09:06.555 INFO:tasks.workunit.client.0.smithi117.stdout:8/364: truncate d0/d8/f65 4392287 0 2024-04-20T11:09:06.555 INFO:tasks.workunit.client.0.smithi117.stdout:2/559: symlink d1/d4/d83/lad 0 2024-04-20T11:09:06.557 INFO:tasks.workunit.client.0.smithi117.stdout:7/485: write d1/d1d/f87 [1648514,46569] 0 2024-04-20T11:09:06.560 INFO:tasks.workunit.client.0.smithi117.stdout:4/461: chown d1/c42 28459994 1 2024-04-20T11:09:06.561 INFO:tasks.workunit.client.0.smithi117.stdout:1/430: link d6/d16/d3a/d5a/d5c/d61/d71/d6b/c76 d6/d16/c85 0 2024-04-20T11:09:06.561 INFO:tasks.workunit.client.0.smithi117.stdout:4/462: dread - d1/dd/d1a/f7c zero size 2024-04-20T11:09:06.563 INFO:tasks.workunit.client.0.smithi117.stdout:8/365: mkdir d0/d9/d3a/d7c 0 2024-04-20T11:09:06.565 INFO:tasks.workunit.client.0.smithi117.stdout:2/560: mknod d1/d4/d83/d29/d3f/cae 0 2024-04-20T11:09:06.567 INFO:tasks.workunit.client.0.smithi117.stdout:7/486: mknod d1/d4/d39/c9d 0 2024-04-20T11:09:06.569 INFO:tasks.workunit.client.0.smithi117.stdout:9/492: truncate d4/d30/f4e 1991957 0 2024-04-20T11:09:06.570 INFO:tasks.workunit.client.0.smithi117.stdout:4/463: fdatasync d1/f10 0 2024-04-20T11:09:06.572 INFO:tasks.workunit.client.0.smithi117.stdout:1/431: creat d6/d21/f86 x:0 0 0 2024-04-20T11:09:06.575 INFO:tasks.workunit.client.0.smithi117.stdout:9/493: mknod d4/d30/caa 0 2024-04-20T11:09:06.577 INFO:tasks.workunit.client.0.smithi117.stdout:9/494: chown d4/d5/da/fc 1061407 1 2024-04-20T11:09:06.578 INFO:tasks.workunit.client.0.smithi117.stdout:8/366: dwrite d0/d9/d3a/f6a [0,4194304] 0 2024-04-20T11:09:06.582 INFO:tasks.workunit.client.0.smithi117.stdout:2/561: creat d1/d9b/faf x:0 0 0 2024-04-20T11:09:06.585 INFO:tasks.workunit.client.0.smithi117.stdout:1/432: unlink d6/d1f/d30/d49/d69/c6d 0 2024-04-20T11:09:06.587 INFO:tasks.workunit.client.0.smithi117.stdout:1/433: read - d6/d16/d3a/d5a/d5c/f7b zero size 2024-04-20T11:09:06.588 INFO:tasks.workunit.client.0.smithi117.stdout:7/487: creat d1/d4/d3d/f9e x:0 0 0 2024-04-20T11:09:06.597 INFO:tasks.workunit.client.0.smithi117.stdout:4/464: truncate d1/dd/d52/f54 465564 0 2024-04-20T11:09:06.600 INFO:tasks.workunit.client.0.smithi117.stdout:2/562: truncate d1/d4/d83/d29/d2e/f40 527066 0 2024-04-20T11:09:06.603 INFO:tasks.workunit.client.0.smithi117.stdout:1/434: readlink d6/lb 0 2024-04-20T11:09:06.606 INFO:tasks.workunit.client.0.smithi117.stdout:9/495: dwrite d4/d5/f2a [0,4194304] 0 2024-04-20T11:09:06.609 INFO:tasks.workunit.client.0.smithi117.stdout:1/435: mknod d6/d16/d3a/d5a/c87 0 2024-04-20T11:09:06.610 INFO:tasks.workunit.client.0.smithi117.stdout:8/367: dwrite d0/d8/d11/f1f [0,4194304] 0 2024-04-20T11:09:06.618 INFO:tasks.workunit.client.0.smithi117.stdout:9/496: unlink d4/d5/da/f29 0 2024-04-20T11:09:06.620 INFO:tasks.workunit.client.0.smithi117.stdout:9/497: write d4/d5/da/fc [5768154,84411] 0 2024-04-20T11:09:06.623 INFO:tasks.workunit.client.0.smithi117.stdout:8/368: creat d0/d8/d11/d24/d27/d47/f7d x:0 0 0 2024-04-20T11:09:06.630 INFO:tasks.workunit.client.0.smithi117.stdout:1/436: link d6/f79 d6/d1f/d30/d49/d69/d6e/d6f/d73/f88 0 2024-04-20T11:09:06.630 INFO:tasks.workunit.client.0.smithi117.stdout:2/563: getdents d1/d4/dd/d17/d7a 0 2024-04-20T11:09:06.630 INFO:tasks.workunit.client.0.smithi117.stdout:2/564: dread - d1/d4/d69/f8c zero size 2024-04-20T11:09:06.631 INFO:tasks.workunit.client.0.smithi117.stdout:2/565: chown d1/d4/d69 0 1 2024-04-20T11:09:06.631 INFO:tasks.workunit.client.0.smithi117.stdout:9/498: unlink d4/d4a/d72/f8e 0 2024-04-20T11:09:06.655 INFO:tasks.workunit.client.0.smithi117.stdout:8/369: dwrite d0/d8/d39/d40/f6e [4194304,4194304] 0 2024-04-20T11:09:06.657 INFO:tasks.workunit.client.0.smithi117.stdout:1/437: dwrite d6/d1f/d30/f74 [0,4194304] 0 2024-04-20T11:09:06.657 INFO:tasks.workunit.client.0.smithi117.stdout:6/554: sync 2024-04-20T11:09:06.658 INFO:tasks.workunit.client.0.smithi117.stdout:5/618: sync 2024-04-20T11:09:06.662 INFO:tasks.workunit.client.0.smithi117.stdout:4/465: dwrite d1/dd/d1a/d43/f65 [0,4194304] 0 2024-04-20T11:09:06.673 INFO:tasks.workunit.client.0.smithi117.stdout:4/466: rename d1/c4 to d1/dd/d1a/d43/c9f 0 2024-04-20T11:09:06.677 INFO:tasks.workunit.client.0.smithi117.stdout:9/499: dwrite d4/d35/d54/f8b [0,4194304] 0 2024-04-20T11:09:06.693 INFO:tasks.workunit.client.0.smithi117.stdout:6/555: dread d3/f73 [0,4194304] 0 2024-04-20T11:09:06.705 INFO:tasks.workunit.client.0.smithi117.stdout:6/556: creat d3/d50/fb8 x:0 0 0 2024-04-20T11:09:06.708 INFO:tasks.workunit.client.0.smithi117.stdout:4/467: dwrite d1/dd/d1a/f1e [0,4194304] 0 2024-04-20T11:09:06.710 INFO:tasks.workunit.client.0.smithi117.stdout:1/438: dwrite d6/d1f/f80 [0,4194304] 0 2024-04-20T11:09:06.710 INFO:tasks.workunit.client.0.smithi117.stdout:8/370: dread d0/d8/f65 [0,4194304] 0 2024-04-20T11:09:06.711 INFO:tasks.workunit.client.0.smithi117.stdout:4/468: chown d1/d9/c75 3591 1 2024-04-20T11:09:06.712 INFO:tasks.workunit.client.0.smithi117.stdout:6/557: mknod d3/d4/d2e/d37/cb9 0 2024-04-20T11:09:06.715 INFO:tasks.workunit.client.0.smithi117.stdout:1/439: creat d6/d1f/d30/d32/f89 x:0 0 0 2024-04-20T11:09:06.721 INFO:tasks.workunit.client.0.smithi117.stdout:8/371: rename d0/d8/d11/d24/d27/d47/l4c to d0/d8/d11/d6d/l7e 0 2024-04-20T11:09:06.721 INFO:tasks.workunit.client.0.smithi117.stdout:1/440: fsync d6/d1f/d30/f4e 0 2024-04-20T11:09:06.725 INFO:tasks.workunit.client.0.smithi117.stdout:5/619: dread de/d16/f66 [0,4194304] 0 2024-04-20T11:09:06.727 INFO:tasks.workunit.client.0.smithi117.stdout:5/620: chown de/d5b/f81 3665 1 2024-04-20T11:09:06.727 INFO:tasks.workunit.client.0.smithi117.stdout:8/372: mkdir d0/d8/d39/d7f 0 2024-04-20T11:09:06.729 INFO:tasks.workunit.client.0.smithi117.stdout:1/441: mknod d6/d1f/d30/d49/c8a 0 2024-04-20T11:09:06.729 INFO:tasks.workunit.client.0.smithi117.stdout:5/621: dread - de/d41/d47/fc6 zero size 2024-04-20T11:09:06.731 INFO:tasks.workunit.client.0.smithi117.stdout:6/558: link d3/df/d39/daa/cab d3/d4/cba 0 2024-04-20T11:09:06.732 INFO:tasks.workunit.client.0.smithi117.stdout:8/373: dread - d0/d8/d11/d24/d27/d47/f7d zero size 2024-04-20T11:09:06.734 INFO:tasks.workunit.client.0.smithi117.stdout:1/442: chown d6/d16/d3a/d5a/d5c/c70 0 1 2024-04-20T11:09:06.735 INFO:tasks.workunit.client.0.smithi117.stdout:4/469: dwrite d1/d9/f96 [0,4194304] 0 2024-04-20T11:09:06.737 INFO:tasks.workunit.client.0.smithi117.stdout:6/559: write d3/d4/d2e/d4e/d71/db7/fa8 [228734,112433] 0 2024-04-20T11:09:06.738 INFO:tasks.workunit.client.0.smithi117.stdout:8/374: mknod d0/d8/d11/d6d/c80 0 2024-04-20T11:09:06.741 INFO:tasks.workunit.client.0.smithi117.stdout:8/375: stat d0/d8/d11/d24/d2b/f59 0 2024-04-20T11:09:06.742 INFO:tasks.workunit.client.0.smithi117.stdout:8/376: write d0/d9/f33 [505072,53918] 0 2024-04-20T11:09:06.754 INFO:tasks.workunit.client.0.smithi117.stdout:5/622: truncate de/d21/d84/f87 3475033 0 2024-04-20T11:09:06.756 INFO:tasks.workunit.client.0.smithi117.stdout:1/443: write d6/d1f/d30/d49/d69/d6e/d6f/d73/f88 [4486579,94941] 0 2024-04-20T11:09:06.756 INFO:tasks.workunit.client.0.smithi117.stdout:6/560: write d3/d3a/f4d [875440,79409] 0 2024-04-20T11:09:06.759 INFO:tasks.workunit.client.0.smithi117.stdout:6/561: stat d3/d5a/d74/fb6 0 2024-04-20T11:09:06.760 INFO:tasks.workunit.client.0.smithi117.stdout:1/444: unlink d6/l51 0 2024-04-20T11:09:06.766 INFO:tasks.workunit.client.0.smithi117.stdout:1/445: mknod d6/d1f/d30/d49/d69/d6e/c8b 0 2024-04-20T11:09:06.768 INFO:tasks.workunit.client.0.smithi117.stdout:4/470: dread d1/f25 [0,4194304] 0 2024-04-20T11:09:06.777 INFO:tasks.workunit.client.0.smithi117.stdout:8/377: dread d0/d8/d39/d40/f6e [0,4194304] 0 2024-04-20T11:09:06.782 INFO:tasks.workunit.client.0.smithi117.stdout:6/562: dwrite d3/d5a/d74/fb6 [0,4194304] 0 2024-04-20T11:09:06.782 INFO:tasks.workunit.client.0.smithi117.stdout:4/471: creat d1/d45/d4a/d99/fa0 x:0 0 0 2024-04-20T11:09:06.784 INFO:tasks.workunit.client.0.smithi117.stdout:4/472: dread - d1/dd/d1a/d2b/f8f zero size 2024-04-20T11:09:06.786 INFO:tasks.workunit.client.0.smithi117.stdout:6/563: creat d3/d4/d2e/d4e/fbb x:0 0 0 2024-04-20T11:09:06.788 INFO:tasks.workunit.client.0.smithi117.stdout:6/564: chown d3/df/d39/d99/f8c 41715400 1 2024-04-20T11:09:06.792 INFO:tasks.workunit.client.0.smithi117.stdout:4/473: link d1/dd/d1a/f3e d1/d45/fa1 0 2024-04-20T11:09:06.795 INFO:tasks.workunit.client.0.smithi117.stdout:4/474: write d1/dd/d1a/f7c [147817,123413] 0 2024-04-20T11:09:06.796 INFO:tasks.workunit.client.0.smithi117.stdout:5/623: dread de/d16/f97 [4194304,4194304] 0 2024-04-20T11:09:06.796 INFO:tasks.workunit.client.0.smithi117.stdout:4/475: fsync d1/d45/f4f 0 2024-04-20T11:09:06.801 INFO:tasks.workunit.client.0.smithi117.stdout:5/624: getdents de/d16/d9d 0 2024-04-20T11:09:06.802 INFO:tasks.workunit.client.0.smithi117.stdout:5/625: dread - de/d5b/fae zero size 2024-04-20T11:09:06.806 INFO:tasks.workunit.client.0.smithi117.stdout:5/626: unlink de/d21/d45/d48/d4e/da1/lb5 0 2024-04-20T11:09:06.808 INFO:tasks.workunit.client.0.smithi117.stdout:5/627: mkdir de/dd0 0 2024-04-20T11:09:06.811 INFO:tasks.workunit.client.0.smithi117.stdout:5/628: getdents de/d21/d45/d48/d4e/dc3 0 2024-04-20T11:09:06.811 INFO:tasks.workunit.client.0.smithi117.stdout:5/629: stat de/db7 0 2024-04-20T11:09:06.813 INFO:tasks.workunit.client.0.smithi117.stdout:5/630: write de/d41/d47/f6f [7635377,6460] 0 2024-04-20T11:09:06.816 INFO:tasks.workunit.client.0.smithi117.stdout:5/631: creat de/d16/fd1 x:0 0 0 2024-04-20T11:09:06.816 INFO:tasks.workunit.client.0.smithi117.stdout:4/476: write d1/f10 [1530961,45947] 0 2024-04-20T11:09:06.826 INFO:tasks.workunit.client.0.smithi117.stdout:4/477: read d1/dd/d1a/d43/d4e/f62 [3303377,24260] 0 2024-04-20T11:09:06.828 INFO:tasks.workunit.client.0.smithi117.stdout:4/478: readlink d1/dd/d52/l83 0 2024-04-20T11:09:06.830 INFO:tasks.workunit.client.0.smithi117.stdout:4/479: readlink d1/d9/d1f/l70 0 2024-04-20T11:09:06.832 INFO:tasks.workunit.client.0.smithi117.stdout:6/565: dread d3/d3a/f61 [0,4194304] 0 2024-04-20T11:09:06.832 INFO:tasks.workunit.client.0.smithi117.stdout:4/480: fsync d1/dd/d1a/d43/d4e/f9b 0 2024-04-20T11:09:06.835 INFO:tasks.workunit.client.0.smithi117.stdout:4/481: write d1/d9/d79/f8c [620611,16804] 0 2024-04-20T11:09:06.840 INFO:tasks.workunit.client.0.smithi117.stdout:4/482: mkdir d1/dd/da2 0 2024-04-20T11:09:06.844 INFO:tasks.workunit.client.0.smithi117.stdout:4/483: rename d1/d45/c58 to d1/d45/d4a/d99/ca3 0 2024-04-20T11:09:06.847 INFO:tasks.workunit.client.0.smithi117.stdout:4/484: symlink d1/d9/d1f/la4 0 2024-04-20T11:09:06.848 INFO:tasks.workunit.client.0.smithi117.stdout:4/485: write d1/dd/f53 [2429449,68972] 0 2024-04-20T11:09:06.857 INFO:tasks.workunit.client.0.smithi117.stdout:5/632: dread de/d41/d47/d4c/fb1 [0,4194304] 0 2024-04-20T11:09:06.859 INFO:tasks.workunit.client.0.smithi117.stdout:5/633: dread - de/d21/d45/d77/fa5 zero size 2024-04-20T11:09:06.874 INFO:tasks.workunit.client.0.smithi117.stdout:5/634: dwrite de/d21/f95 [0,4194304] 0 2024-04-20T11:09:06.875 INFO:tasks.workunit.client.0.smithi117.stdout:4/486: dread d1/dd/d1a/d2b/f46 [0,4194304] 0 2024-04-20T11:09:06.878 INFO:tasks.workunit.client.0.smithi117.stdout:5/635: mknod de/d21/d84/cd2 0 2024-04-20T11:09:06.879 INFO:tasks.workunit.client.0.smithi117.stdout:4/487: rename d1/f2 to d1/fa5 0 2024-04-20T11:09:06.884 INFO:tasks.workunit.client.0.smithi117.stdout:5/636: rmdir de/d21/d45/d48/d4e/dc3 0 2024-04-20T11:09:06.886 INFO:tasks.workunit.client.0.smithi117.stdout:5/637: chown de/c15 2 1 2024-04-20T11:09:06.889 INFO:tasks.workunit.client.0.smithi117.stdout:5/638: creat de/d41/dc2/fd3 x:0 0 0 2024-04-20T11:09:06.890 INFO:tasks.workunit.client.0.smithi117.stdout:5/639: chown de/d5b/d65 26151166 1 2024-04-20T11:09:06.891 INFO:tasks.workunit.client.0.smithi117.stdout:5/640: read - de/d21/f9f zero size 2024-04-20T11:09:06.892 INFO:tasks.workunit.client.0.smithi117.stdout:5/641: dread - de/d21/d51/fcc zero size 2024-04-20T11:09:06.896 INFO:tasks.workunit.client.0.smithi117.stdout:5/642: symlink de/dd0/ld4 0 2024-04-20T11:09:06.896 INFO:tasks.workunit.client.0.smithi117.stdout:4/488: truncate d1/d9/d1f/f5c 2033609 0 2024-04-20T11:09:06.897 INFO:tasks.workunit.client.0.smithi117.stdout:5/643: chown de/f18 225 1 2024-04-20T11:09:06.898 INFO:tasks.workunit.client.0.smithi117.stdout:5/644: dread - de/d21/d45/fc5 zero size 2024-04-20T11:09:06.910 INFO:tasks.workunit.client.0.smithi117.stdout:5/645: dwrite de/d16/d9d/fb0 [0,4194304] 0 2024-04-20T11:09:06.916 INFO:tasks.workunit.client.0.smithi117.stdout:5/646: rename de/d21/d45/d48/d4e/dc1 to de/d21/d35/d57/dd5 0 2024-04-20T11:09:06.922 INFO:tasks.workunit.client.0.smithi117.stdout:5/647: getdents de/dd0 0 2024-04-20T11:09:06.929 INFO:tasks.workunit.client.0.smithi117.stdout:5/648: rename de/d21/d45/d48/d4e/f8c to de/d5b/d65/fd6 0 2024-04-20T11:09:06.946 INFO:tasks.workunit.client.0.smithi117.stdout:5/649: truncate de/d41/d47/d4c/fb1 2493435 0 2024-04-20T11:09:06.947 INFO:tasks.workunit.client.0.smithi117.stdout:5/650: fsync de/f14 0 2024-04-20T11:09:06.948 INFO:tasks.workunit.client.0.smithi117.stdout:5/651: write de/d16/f2d [1585501,111410] 0 2024-04-20T11:09:06.951 INFO:tasks.workunit.client.0.smithi117.stdout:5/652: creat de/d21/fd7 x:0 0 0 2024-04-20T11:09:06.952 INFO:tasks.workunit.client.0.smithi117.stdout:5/653: read - de/d41/d47/d55/d56/fac zero size 2024-04-20T11:09:06.954 INFO:tasks.workunit.client.0.smithi117.stdout:5/654: chown de/d41/dc2/fd3 6484594 1 2024-04-20T11:09:06.957 INFO:tasks.workunit.client.0.smithi117.stdout:5/655: mkdir de/d41/d47/d55/d56/dc8/dd8 0 2024-04-20T11:09:06.960 INFO:tasks.workunit.client.0.smithi117.stdout:5/656: write de/d41/d47/d55/d56/f79 [429758,112593] 0 2024-04-20T11:09:06.961 INFO:tasks.workunit.client.0.smithi117.stdout:5/657: chown de/d41/d47/d4c 29057329 1 2024-04-20T11:09:06.965 INFO:tasks.workunit.client.0.smithi117.stdout:5/658: creat de/d21/d35/d57/dd5/dce/fd9 x:0 0 0 2024-04-20T11:09:06.965 INFO:tasks.workunit.client.0.smithi117.stdout:5/659: readlink lc 0 2024-04-20T11:09:06.968 INFO:tasks.workunit.client.0.smithi117.stdout:5/660: mkdir de/d21/d45/db9/dda 0 2024-04-20T11:09:06.970 INFO:tasks.workunit.client.0.smithi117.stdout:5/661: readlink de/d41/d47/d55/l93 0 2024-04-20T11:09:06.983 INFO:tasks.workunit.client.0.smithi117.stdout:5/662: truncate de/f74 1379718 0 2024-04-20T11:09:06.992 INFO:tasks.workunit.client.0.smithi117.stdout:5/663: write de/d21/d35/f3b [2055870,13856] 0 2024-04-20T11:09:06.995 INFO:tasks.workunit.client.0.smithi117.stdout:5/664: dread - de/d5b/f81 zero size 2024-04-20T11:09:06.999 INFO:tasks.workunit.client.0.smithi117.stdout:5/665: creat de/d21/d45/d48/d4e/fdb x:0 0 0 2024-04-20T11:09:07.000 INFO:tasks.workunit.client.0.smithi117.stdout:5/666: chown l4 10970 1 2024-04-20T11:09:07.002 INFO:tasks.workunit.client.0.smithi117.stdout:5/667: creat de/d21/d35/d57/dd5/dce/fdc x:0 0 0 2024-04-20T11:09:07.004 INFO:tasks.workunit.client.0.smithi117.stdout:5/668: chown de/d21/d45/d48 988 1 2024-04-20T11:09:07.027 INFO:tasks.workunit.client.0.smithi117.stdout:3/448: sync 2024-04-20T11:09:07.031 INFO:tasks.workunit.client.0.smithi117.stdout:5/669: dread de/d21/d84/f87 [0,4194304] 0 2024-04-20T11:09:07.034 INFO:tasks.workunit.client.0.smithi117.stdout:3/449: getdents d0/d5/d21/d88/d29 0 2024-04-20T11:09:07.037 INFO:tasks.workunit.client.0.smithi117.stdout:5/670: rename de/d21/d45/laa to de/d41/dc2/ldd 0 2024-04-20T11:09:07.040 INFO:tasks.workunit.client.0.smithi117.stdout:3/450: rename d0/d5/d11/c14 to d0/d5/d21/d77/c9d 0 2024-04-20T11:09:07.046 INFO:tasks.workunit.client.0.smithi117.stdout:5/671: dread de/f18 [0,4194304] 0 2024-04-20T11:09:07.048 INFO:tasks.workunit.client.0.smithi117.stdout:5/672: chown de/l1a 1139 1 2024-04-20T11:09:07.053 INFO:tasks.workunit.client.0.smithi117.stdout:5/673: dread de/d21/d35/f8d [0,4194304] 0 2024-04-20T11:09:07.056 INFO:tasks.workunit.client.0.smithi117.stdout:5/674: symlink de/d21/d35/d57/dd5/dce/lde 0 2024-04-20T11:09:07.060 INFO:tasks.workunit.client.0.smithi117.stdout:5/675: mknod de/d21/d45/d48/d4e/da1/cdf 0 2024-04-20T11:09:07.062 INFO:tasks.workunit.client.0.smithi117.stdout:5/676: dread - de/d21/d51/fcc zero size 2024-04-20T11:09:07.066 INFO:tasks.workunit.client.0.smithi117.stdout:5/677: creat de/d21/d35/d57/fe0 x:0 0 0 2024-04-20T11:09:07.067 INFO:tasks.workunit.client.0.smithi117.stdout:3/451: dread d0/d5/d7/dd/f3a [0,4194304] 0 2024-04-20T11:09:07.069 INFO:tasks.workunit.client.0.smithi117.stdout:3/452: write d0/d5/d7/dd/f5a [1459492,13109] 0 2024-04-20T11:09:07.072 INFO:tasks.workunit.client.0.smithi117.stdout:5/678: rename de/d16/f2d to de/d21/d35/fe1 0 2024-04-20T11:09:07.072 INFO:tasks.workunit.client.0.smithi117.stdout:3/453: write d0/d5/d21/f28 [1390606,93110] 0 2024-04-20T11:09:07.074 INFO:tasks.workunit.client.0.smithi117.stdout:3/454: dread - d0/f86 zero size 2024-04-20T11:09:07.074 INFO:tasks.workunit.client.0.smithi117.stdout:5/679: dread - de/d21/d35/d57/fe0 zero size 2024-04-20T11:09:07.077 INFO:tasks.workunit.client.0.smithi117.stdout:3/455: creat d0/d5/de/d4e/f9e x:0 0 0 2024-04-20T11:09:07.078 INFO:tasks.workunit.client.0.smithi117.stdout:5/680: creat de/d41/d47/fe2 x:0 0 0 2024-04-20T11:09:07.079 INFO:tasks.workunit.client.0.smithi117.stdout:3/456: chown d0/d5/d21/d88/d29/c6f 39032 1 2024-04-20T11:09:07.080 INFO:tasks.workunit.client.0.smithi117.stdout:5/681: chown de/d21/d35/c7c 29148 1 2024-04-20T11:09:07.082 INFO:tasks.workunit.client.0.smithi117.stdout:5/682: truncate de/f8e 175709 0 2024-04-20T11:09:07.082 INFO:tasks.workunit.client.0.smithi117.stdout:3/457: fdatasync d0/d5/f19 0 2024-04-20T11:09:07.084 INFO:tasks.workunit.client.0.smithi117.stdout:5/683: write de/d41/dc2/fd3 [1016233,127731] 0 2024-04-20T11:09:07.086 INFO:tasks.workunit.client.0.smithi117.stdout:3/458: symlink d0/d5/d21/d75/d89/l9f 0 2024-04-20T11:09:07.087 INFO:tasks.workunit.client.0.smithi117.stdout:3/459: fsync d0/d5/d7/dd/f96 0 2024-04-20T11:09:07.091 INFO:tasks.workunit.client.0.smithi117.stdout:3/460: creat d0/d5/d21/d94/d9a/fa0 x:0 0 0 2024-04-20T11:09:07.095 INFO:tasks.workunit.client.0.smithi117.stdout:3/461: read d0/d5/d21/d88/d69/f8c [678706,69592] 0 2024-04-20T11:09:07.100 INFO:tasks.workunit.client.0.smithi117.stdout:3/462: fsync d0/d5/d7/dd/f5a 0 2024-04-20T11:09:07.102 INFO:tasks.workunit.client.0.smithi117.stdout:5/684: dwrite de/db7/fbd [0,4194304] 0 2024-04-20T11:09:07.104 INFO:tasks.workunit.client.0.smithi117.stdout:3/463: mknod d0/ca1 0 2024-04-20T11:09:07.110 INFO:tasks.workunit.client.0.smithi117.stdout:3/464: mknod d0/d5/d11/d80/ca2 0 2024-04-20T11:09:07.112 INFO:tasks.workunit.client.0.smithi117.stdout:3/465: write d0/d5/d11/f74 [4953349,48763] 0 2024-04-20T11:09:07.115 INFO:tasks.workunit.client.0.smithi117.stdout:3/466: creat d0/d5/d73/fa3 x:0 0 0 2024-04-20T11:09:07.119 INFO:tasks.workunit.client.0.smithi117.stdout:3/467: write d0/d5/d21/d88/d29/d66/d63/f7a [2023509,35627] 0 2024-04-20T11:09:07.121 INFO:tasks.workunit.client.0.smithi117.stdout:3/468: rmdir d0/d5/d21/d88/d29 39 2024-04-20T11:09:07.123 INFO:tasks.workunit.client.0.smithi117.stdout:3/469: fdatasync d0/d5/de/d4e/f53 0 2024-04-20T11:09:07.141 INFO:tasks.workunit.client.0.smithi117.stdout:3/470: rmdir d0/d5/d73 39 2024-04-20T11:09:07.147 INFO:tasks.workunit.client.0.smithi117.stdout:3/471: link d0/d5/d7/f87 d0/d5/d21/d88/d29/d66/d63/fa4 0 2024-04-20T11:09:07.166 INFO:tasks.workunit.client.0.smithi117.stdout:3/472: dread d0/d5/d7/dd/f59 [0,4194304] 0 2024-04-20T11:09:07.170 INFO:tasks.workunit.client.0.smithi117.stdout:3/473: truncate d0/d5/d7/dd/f91 592518 0 2024-04-20T11:09:07.177 INFO:tasks.workunit.client.0.smithi117.stdout:3/474: link d0/d5/de/d4e/f9e d0/d5/d21/d94/d9a/fa5 0 2024-04-20T11:09:07.204 INFO:tasks.workunit.client.0.smithi117.stdout:3/475: dwrite d0/f86 [0,4194304] 0 2024-04-20T11:09:07.209 INFO:tasks.workunit.client.0.smithi117.stdout:3/476: chown d0/d5/d7/dd/ff 943 1 2024-04-20T11:09:07.210 INFO:tasks.workunit.client.0.smithi117.stdout:3/477: stat d0/d5/l5f 0 2024-04-20T11:09:07.218 INFO:tasks.workunit.client.0.smithi117.stdout:3/478: write d0/d5/d21/d88/d29/f30 [427085,114341] 0 2024-04-20T11:09:07.225 INFO:tasks.workunit.client.0.smithi117.stdout:3/479: truncate d0/d5/d21/f47 789114 0 2024-04-20T11:09:07.231 INFO:tasks.workunit.client.0.smithi117.stdout:3/480: symlink d0/d2a/la6 0 2024-04-20T11:09:07.246 INFO:tasks.workunit.client.0.smithi117.stdout:3/481: write d0/f35 [4234715,81000] 0 2024-04-20T11:09:07.339 INFO:tasks.workunit.client.0.smithi117.stdout:7/488: sync 2024-04-20T11:09:07.339 INFO:tasks.workunit.client.0.smithi117.stdout:0/515: sync 2024-04-20T11:09:07.349 INFO:tasks.workunit.client.0.smithi117.stdout:0/516: read d3/d7/d2c/f6b [249669,39530] 0 2024-04-20T11:09:07.356 INFO:tasks.workunit.client.0.smithi117.stdout:0/517: creat d3/d5/d74/fad x:0 0 0 2024-04-20T11:09:07.360 INFO:tasks.workunit.client.0.smithi117.stdout:0/518: write d3/d7/f4d [2778199,44050] 0 2024-04-20T11:09:07.399 INFO:tasks.workunit.client.0.smithi117.stdout:0/519: dwrite d3/dab/d54/fa3 [0,4194304] 0 2024-04-20T11:09:07.409 INFO:tasks.workunit.client.0.smithi117.stdout:0/520: symlink d3/d7/lae 0 2024-04-20T11:09:07.411 INFO:tasks.workunit.client.0.smithi117.stdout:0/521: stat d3/d7e/c81 0 2024-04-20T11:09:07.422 INFO:tasks.workunit.client.0.smithi117.stdout:0/522: readlink d3/d7/d20/d39/d3a/l3d 0 2024-04-20T11:09:07.427 INFO:tasks.workunit.client.0.smithi117.stdout:2/566: sync 2024-04-20T11:09:07.428 INFO:tasks.workunit.client.0.smithi117.stdout:4/489: sync 2024-04-20T11:09:07.428 INFO:tasks.workunit.client.0.smithi117.stdout:9/500: sync 2024-04-20T11:09:07.428 INFO:tasks.workunit.client.0.smithi117.stdout:1/446: sync 2024-04-20T11:09:07.428 INFO:tasks.workunit.client.0.smithi117.stdout:5/685: sync 2024-04-20T11:09:07.428 INFO:tasks.workunit.client.0.smithi117.stdout:8/378: sync 2024-04-20T11:09:07.428 INFO:tasks.workunit.client.0.smithi117.stdout:6/566: sync 2024-04-20T11:09:07.430 INFO:tasks.workunit.client.0.smithi117.stdout:0/523: write d3/f3e [366963,51423] 0 2024-04-20T11:09:07.438 INFO:tasks.workunit.client.0.smithi117.stdout:4/490: unlink d1/d9/d1f/l78 0 2024-04-20T11:09:07.438 INFO:tasks.workunit.client.0.smithi117.stdout:0/524: truncate d3/d5/d74/fad 928933 0 2024-04-20T11:09:07.441 INFO:tasks.workunit.client.0.smithi117.stdout:1/447: creat d6/d1f/d30/d49/d69/d6e/d6f/d73/f8c x:0 0 0 2024-04-20T11:09:07.442 INFO:tasks.workunit.client.0.smithi117.stdout:5/686: symlink de/d16/d9d/db4/le3 0 2024-04-20T11:09:07.446 INFO:tasks.workunit.client.0.smithi117.stdout:6/567: rename d3/d5a/d79 to d3/d5a/d74/db3/dbc 0 2024-04-20T11:09:07.448 INFO:tasks.workunit.client.0.smithi117.stdout:2/567: getdents d1/d4/d83/d29/d2e/d53/d58/da8 0 2024-04-20T11:09:07.448 INFO:tasks.workunit.client.0.smithi117.stdout:1/448: dread d6/d16/d3a/f7e [0,4194304] 0 2024-04-20T11:09:07.450 INFO:tasks.workunit.client.0.smithi117.stdout:0/525: creat d3/dab/d54/faf x:0 0 0 2024-04-20T11:09:07.453 INFO:tasks.workunit.client.0.smithi117.stdout:4/491: rename d1/d9/d1f/c49 to d1/d9/ca6 0 2024-04-20T11:09:07.454 INFO:tasks.workunit.client.0.smithi117.stdout:8/379: link d0/d9/f17 d0/d8/d39/d7f/f81 0 2024-04-20T11:09:07.455 INFO:tasks.workunit.client.0.smithi117.stdout:7/489: sync 2024-04-20T11:09:07.455 INFO:tasks.workunit.client.0.smithi117.stdout:3/482: sync 2024-04-20T11:09:07.456 INFO:tasks.workunit.client.0.smithi117.stdout:9/501: getdents d4/d4a/d92 0 2024-04-20T11:09:07.457 INFO:tasks.workunit.client.0.smithi117.stdout:4/492: readlink d1/dd/d1a/d43/d4e/l7a 0 2024-04-20T11:09:07.461 INFO:tasks.workunit.client.0.smithi117.stdout:2/568: mknod d1/d15/d1d/cb0 0 2024-04-20T11:09:07.463 INFO:tasks.workunit.client.0.smithi117.stdout:4/493: chown d1/d9/d79/c7d 1155 1 2024-04-20T11:09:07.464 INFO:tasks.workunit.client.0.smithi117.stdout:5/687: dwrite de/d21/f96 [0,4194304] 0 2024-04-20T11:09:07.464 INFO:tasks.workunit.client.0.smithi117.stdout:4/494: stat d1/f57 0 2024-04-20T11:09:07.466 INFO:tasks.workunit.client.0.smithi117.stdout:0/526: rename d3/d7/d8/l8e to d3/d7/d8/df/lb0 0 2024-04-20T11:09:07.467 INFO:tasks.workunit.client.0.smithi117.stdout:3/483: mkdir d0/d5/d21/d94/da7 0 2024-04-20T11:09:07.469 INFO:tasks.workunit.client.0.smithi117.stdout:0/527: readlink d3/d7/l88 0 2024-04-20T11:09:07.469 INFO:tasks.workunit.client.0.smithi117.stdout:9/502: unlink d4/d5/f2b 0 2024-04-20T11:09:07.477 INFO:tasks.workunit.client.0.smithi117.stdout:7/490: dwrite d1/d43/f61 [0,4194304] 0 2024-04-20T11:09:07.486 INFO:tasks.workunit.client.0.smithi117.stdout:9/503: dwrite d4/d5/da/f6f [0,4194304] 0 2024-04-20T11:09:07.501 INFO:tasks.workunit.client.0.smithi117.stdout:9/504: dwrite d4/d5/da/f19 [4194304,4194304] 0 2024-04-20T11:09:07.542 INFO:tasks.workunit.client.0.smithi117.stdout:8/380: rename d0/d8/d11/d24/f66 to d0/dc/d49/d63/f82 0 2024-04-20T11:09:07.545 INFO:tasks.workunit.client.0.smithi117.stdout:6/568: link d3/d5a/d74/db3/f81 d3/d4/d2e/d4e/fbd 0 2024-04-20T11:09:07.550 INFO:tasks.workunit.client.0.smithi117.stdout:2/569: link d1/d4/d83/d29/d62/f9a d1/d4/dd/d3e/fb1 0 2024-04-20T11:09:07.553 INFO:tasks.workunit.client.0.smithi117.stdout:0/528: mkdir d3/d7/d20/d39/d3a/db1 0 2024-04-20T11:09:07.553 INFO:tasks.workunit.client.0.smithi117.stdout:2/570: chown d1/d4/d83/d29/d62 1 1 2024-04-20T11:09:07.559 INFO:tasks.workunit.client.0.smithi117.stdout:1/449: getdents d6/d1f/d30 0 2024-04-20T11:09:07.561 INFO:tasks.workunit.client.0.smithi117.stdout:9/505: rename d4/d30/d31/c37 to d4/d4a/d5e/cab 0 2024-04-20T11:09:07.563 INFO:tasks.workunit.client.0.smithi117.stdout:9/506: readlink d4/de/l22 0 2024-04-20T11:09:07.563 INFO:tasks.workunit.client.0.smithi117.stdout:6/569: creat d3/d5a/fbe x:0 0 0 2024-04-20T11:09:07.564 INFO:tasks.workunit.client.0.smithi117.stdout:8/381: dwrite d0/d8/d11/d24/d27/d47/f7d [0,4194304] 0 2024-04-20T11:09:07.576 INFO:tasks.workunit.client.0.smithi117.stdout:4/495: link d1/d45/fa1 d1/fa7 0 2024-04-20T11:09:07.579 INFO:tasks.workunit.client.0.smithi117.stdout:9/507: rename d4/d4a/d72/f89 to d4/d30/d31/fac 0 2024-04-20T11:09:07.581 INFO:tasks.workunit.client.0.smithi117.stdout:9/508: dread - d4/d35/f9a zero size 2024-04-20T11:09:07.603 INFO:tasks.workunit.client.0.smithi117.stdout:9/509: symlink d4/de/d7b/d96/lad 0 2024-04-20T11:09:07.609 INFO:tasks.workunit.client.0.smithi117.stdout:9/510: write f0 [8631317,31462] 0 2024-04-20T11:09:07.610 INFO:tasks.workunit.client.0.smithi117.stdout:7/491: link d1/l31 d1/d4/d6/d17/d3f/l9f 0 2024-04-20T11:09:07.615 INFO:tasks.workunit.client.0.smithi117.stdout:8/382: creat d0/f83 x:0 0 0 2024-04-20T11:09:07.616 INFO:tasks.workunit.client.0.smithi117.stdout:8/383: chown d0/c15 423727 1 2024-04-20T11:09:07.618 INFO:tasks.workunit.client.0.smithi117.stdout:8/384: chown d0/dc/d49/d63/f82 24602740 1 2024-04-20T11:09:07.621 INFO:tasks.workunit.client.0.smithi117.stdout:0/529: getdents d3/d7 0 2024-04-20T11:09:07.627 INFO:tasks.workunit.client.0.smithi117.stdout:7/492: creat d1/d43/fa0 x:0 0 0 2024-04-20T11:09:07.629 INFO:tasks.workunit.client.0.smithi117.stdout:4/496: creat d1/dd/fa8 x:0 0 0 2024-04-20T11:09:07.631 INFO:tasks.workunit.client.0.smithi117.stdout:4/497: chown d1/f19 19 1 2024-04-20T11:09:07.637 INFO:tasks.workunit.client.0.smithi117.stdout:8/385: read d0/d8/d11/f14 [205848,120224] 0 2024-04-20T11:09:07.641 INFO:tasks.workunit.client.0.smithi117.stdout:2/571: dwrite d1/d4/dd/f55 [4194304,4194304] 0 2024-04-20T11:09:07.646 INFO:tasks.workunit.client.0.smithi117.stdout:4/498: creat d1/d45/d4a/fa9 x:0 0 0 2024-04-20T11:09:07.653 INFO:tasks.workunit.client.0.smithi117.stdout:7/493: dread d1/f22 [0,4194304] 0 2024-04-20T11:09:07.654 INFO:tasks.workunit.client.0.smithi117.stdout:1/450: truncate d6/d1f/f2b 1906028 0 2024-04-20T11:09:07.656 INFO:tasks.workunit.client.0.smithi117.stdout:7/494: chown d1/d1d/d42/d2d/l50 4667941 1 2024-04-20T11:09:07.657 INFO:tasks.workunit.client.0.smithi117.stdout:8/386: mkdir d0/d8/d11/d6d/d84 0 2024-04-20T11:09:07.659 INFO:tasks.workunit.client.0.smithi117.stdout:3/484: sync 2024-04-20T11:09:07.659 INFO:tasks.workunit.client.0.smithi117.stdout:5/688: sync 2024-04-20T11:09:07.660 INFO:tasks.workunit.client.0.smithi117.stdout:2/572: creat d1/d4/d83/d29/d2e/d80/fb2 x:0 0 0 2024-04-20T11:09:07.665 INFO:tasks.workunit.client.0.smithi117.stdout:2/573: write d1/d4/d83/d29/d3f/f7d [956338,20310] 0 2024-04-20T11:09:07.665 INFO:tasks.workunit.client.0.smithi117.stdout:6/570: truncate d3/d5a/d74/f93 761487 0 2024-04-20T11:09:07.667 INFO:tasks.workunit.client.0.smithi117.stdout:4/499: rmdir d1/d45 39 2024-04-20T11:09:07.672 INFO:tasks.workunit.client.0.smithi117.stdout:2/574: read d1/f7b [259650,84309] 0 2024-04-20T11:09:07.679 INFO:tasks.workunit.client.0.smithi117.stdout:2/575: read d1/d4/d83/f34 [116442,54727] 0 2024-04-20T11:09:07.682 INFO:tasks.workunit.client.0.smithi117.stdout:8/387: creat d0/d8/d11/d6d/d69/f85 x:0 0 0 2024-04-20T11:09:07.684 INFO:tasks.workunit.client.0.smithi117.stdout:7/495: link d1/d4/d6/d17/d3f/f90 d1/d4/d6/d17/fa1 0 2024-04-20T11:09:07.685 INFO:tasks.workunit.client.0.smithi117.stdout:0/530: write d3/d7/d8/f44 [1014707,78356] 0 2024-04-20T11:09:07.688 INFO:tasks.workunit.client.0.smithi117.stdout:6/571: dwrite d3/d4/d2e/d37/f47 [0,4194304] 0 2024-04-20T11:09:07.688 INFO:tasks.workunit.client.0.smithi117.stdout:3/485: symlink d0/d5/d21/d94/da7/la8 0 2024-04-20T11:09:07.689 INFO:tasks.workunit.client.0.smithi117.stdout:9/511: write d4/d30/f4e [2428031,24174] 0 2024-04-20T11:09:07.694 INFO:tasks.workunit.client.0.smithi117.stdout:6/572: write d3/df/f21 [2251588,122979] 0 2024-04-20T11:09:07.695 INFO:tasks.workunit.client.0.smithi117.stdout:7/496: mknod d1/d1d/ca2 0 2024-04-20T11:09:07.696 INFO:tasks.workunit.client.0.smithi117.stdout:8/388: write d0/f23 [3394641,39172] 0 2024-04-20T11:09:07.699 INFO:tasks.workunit.client.0.smithi117.stdout:7/497: fsync d1/d43/f80 0 2024-04-20T11:09:07.721 INFO:tasks.workunit.client.0.smithi117.stdout:8/389: mkdir d0/d8/d11/d24/d86 0 2024-04-20T11:09:07.726 INFO:tasks.workunit.client.0.smithi117.stdout:2/576: dread d1/d4/dd/d3e/f56 [0,4194304] 0 2024-04-20T11:09:07.727 INFO:tasks.workunit.client.0.smithi117.stdout:7/498: creat d1/d4/d3d/fa3 x:0 0 0 2024-04-20T11:09:07.734 INFO:tasks.workunit.client.0.smithi117.stdout:8/390: stat d0/d8/d11/d24/d53/f7b 0 2024-04-20T11:09:07.735 INFO:tasks.workunit.client.0.smithi117.stdout:7/499: write d1/d1d/f87 [2346010,77681] 0 2024-04-20T11:09:07.738 INFO:tasks.workunit.client.0.smithi117.stdout:2/577: readlink d1/d4/dd/d3e/l42 0 2024-04-20T11:09:07.743 INFO:tasks.workunit.client.0.smithi117.stdout:0/531: dread d3/d7/f4d [0,4194304] 0 2024-04-20T11:09:07.745 INFO:tasks.workunit.client.0.smithi117.stdout:8/391: creat d0/d8/d11/d24/d53/f87 x:0 0 0 2024-04-20T11:09:07.745 INFO:tasks.workunit.client.0.smithi117.stdout:0/532: chown d3/d7/d20/l8f 1 1 2024-04-20T11:09:07.749 INFO:tasks.workunit.client.0.smithi117.stdout:6/573: dread d3/d3a/f4d [0,4194304] 0 2024-04-20T11:09:07.751 INFO:tasks.workunit.client.0.smithi117.stdout:2/578: mkdir d1/d15/d1d/db3 0 2024-04-20T11:09:07.751 INFO:tasks.workunit.client.0.smithi117.stdout:0/533: write d3/d7/d8/f44 [428553,94293] 0 2024-04-20T11:09:07.752 INFO:tasks.workunit.client.0.smithi117.stdout:6/574: fdatasync d3/d50/fb8 0 2024-04-20T11:09:07.755 INFO:tasks.workunit.client.0.smithi117.stdout:0/534: stat d3/d7/d20/c35 0 2024-04-20T11:09:07.758 INFO:tasks.workunit.client.0.smithi117.stdout:2/579: creat d1/d9b/fb4 x:0 0 0 2024-04-20T11:09:07.764 INFO:tasks.workunit.client.0.smithi117.stdout:0/535: readlink l1 0 2024-04-20T11:09:07.764 INFO:tasks.workunit.client.0.smithi117.stdout:6/575: symlink d3/d4/lbf 0 2024-04-20T11:09:07.768 INFO:tasks.workunit.client.0.smithi117.stdout:0/536: mkdir d3/d7/d8/df/db2 0 2024-04-20T11:09:07.771 INFO:tasks.workunit.client.0.smithi117.stdout:9/512: truncate f0 7694078 0 2024-04-20T11:09:07.771 INFO:tasks.workunit.client.0.smithi117.stdout:3/486: dwrite d0/d5/d21/d94/d9a/fa5 [0,4194304] 0 2024-04-20T11:09:07.772 INFO:tasks.workunit.client.0.smithi117.stdout:8/392: dread d0/f34 [0,4194304] 0 2024-04-20T11:09:07.772 INFO:tasks.workunit.client.0.smithi117.stdout:2/580: creat d1/d4/d83/d29/d2e/d53/d93/fb5 x:0 0 0 2024-04-20T11:09:07.776 INFO:tasks.workunit.client.0.smithi117.stdout:7/500: dread d1/d4/d6/d13/f28 [0,4194304] 0 2024-04-20T11:09:07.776 INFO:tasks.workunit.client.0.smithi117.stdout:8/393: write d0/dc/f25 [1894602,109516] 0 2024-04-20T11:09:07.778 INFO:tasks.workunit.client.0.smithi117.stdout:0/537: symlink d3/d7/d20/d39/d9a/lb3 0 2024-04-20T11:09:07.779 INFO:tasks.workunit.client.0.smithi117.stdout:7/501: dread - d1/d4/d6/d54/d63/f85 zero size 2024-04-20T11:09:07.780 INFO:tasks.workunit.client.0.smithi117.stdout:8/394: write d0/d8/d11/d24/d53/f87 [421587,101938] 0 2024-04-20T11:09:07.781 INFO:tasks.workunit.client.0.smithi117.stdout:2/581: creat d1/d9b/fb6 x:0 0 0 2024-04-20T11:09:07.782 INFO:tasks.workunit.client.0.smithi117.stdout:3/487: creat d0/d5/d21/d88/d29/d66/d63/fa9 x:0 0 0 2024-04-20T11:09:07.783 INFO:tasks.workunit.client.0.smithi117.stdout:2/582: chown d1/d20 137870100 1 2024-04-20T11:09:07.783 INFO:tasks.workunit.client.0.smithi117.stdout:0/538: creat d3/d7/d8/df/d6d/fb4 x:0 0 0 2024-04-20T11:09:07.786 INFO:tasks.workunit.client.0.smithi117.stdout:7/502: creat d1/d43/d59/fa4 x:0 0 0 2024-04-20T11:09:07.788 INFO:tasks.workunit.client.0.smithi117.stdout:6/576: dread d3/d4/d2e/d37/f47 [0,4194304] 0 2024-04-20T11:09:07.790 INFO:tasks.workunit.client.0.smithi117.stdout:7/503: chown d1/d43/f7b 3 1 2024-04-20T11:09:07.792 INFO:tasks.workunit.client.0.smithi117.stdout:8/395: truncate d0/d8/d11/d24/d2b/f2c 2749731 0 2024-04-20T11:09:07.792 INFO:tasks.workunit.client.0.smithi117.stdout:2/583: creat d1/d4/d83/d29/daa/fb7 x:0 0 0 2024-04-20T11:09:07.797 INFO:tasks.workunit.client.0.smithi117.stdout:7/504: truncate d1/d4/d6/d54/d63/f6e 820311 0 2024-04-20T11:09:07.797 INFO:tasks.workunit.client.0.smithi117.stdout:2/584: chown d1/d4/d83/d29/d2e/d53/d93/fb5 5462 1 2024-04-20T11:09:07.798 INFO:tasks.workunit.client.0.smithi117.stdout:8/396: write d0/d8/d11/d24/d2b/f5d [27205,71241] 0 2024-04-20T11:09:07.798 INFO:tasks.workunit.client.0.smithi117.stdout:0/539: link d3/d7/d94/la2 d3/d5/d74/lb5 0 2024-04-20T11:09:07.800 INFO:tasks.workunit.client.0.smithi117.stdout:9/513: dread d4/de/d10/f38 [0,4194304] 0 2024-04-20T11:09:07.801 INFO:tasks.workunit.client.0.smithi117.stdout:2/585: mknod d1/d4/dd/d3e/d46/cb8 0 2024-04-20T11:09:07.801 INFO:tasks.workunit.client.0.smithi117.stdout:8/397: fdatasync d0/d8/d39/d40/f51 0 2024-04-20T11:09:07.803 INFO:tasks.workunit.client.0.smithi117.stdout:9/514: chown d4/de/d4b/f86 194561631 1 2024-04-20T11:09:07.806 INFO:tasks.workunit.client.0.smithi117.stdout:7/505: link d1/d4/d6/d13/f71 d1/fa5 0 2024-04-20T11:09:07.808 INFO:tasks.workunit.client.0.smithi117.stdout:7/506: chown d1/d1d/d42/d2d/l50 26 1 2024-04-20T11:09:07.808 INFO:tasks.workunit.client.0.smithi117.stdout:0/540: getdents d3/d7/d20/d47 0 2024-04-20T11:09:07.811 INFO:tasks.workunit.client.0.smithi117.stdout:8/398: getdents d0/d8/d11/d24/d53 0 2024-04-20T11:09:07.814 INFO:tasks.workunit.client.0.smithi117.stdout:7/507: getdents d1/d4/d6 0 2024-04-20T11:09:07.817 INFO:tasks.workunit.client.0.smithi117.stdout:8/399: rename d0/dc/d2e/l60 to d0/d8/d11/d24/d53/l88 0 2024-04-20T11:09:07.823 INFO:tasks.workunit.client.0.smithi117.stdout:8/400: unlink d0/d9/f31 0 2024-04-20T11:09:07.832 INFO:tasks.workunit.client.0.smithi117.stdout:3/488: write d0/d5/d7/f87 [906353,96227] 0 2024-04-20T11:09:07.833 INFO:tasks.workunit.client.0.smithi117.stdout:6/577: write d3/f2c [4497304,63248] 0 2024-04-20T11:09:07.842 INFO:tasks.workunit.client.0.smithi117.stdout:3/489: write d0/d5/d21/d88/d69/f9b [789783,67157] 0 2024-04-20T11:09:07.842 INFO:tasks.workunit.client.0.smithi117.stdout:2/586: write d1/d15/d1d/fa1 [506843,2691] 0 2024-04-20T11:09:07.842 INFO:tasks.workunit.client.0.smithi117.stdout:9/515: dread d4/d30/f36 [0,4194304] 0 2024-04-20T11:09:07.842 INFO:tasks.workunit.client.0.smithi117.stdout:7/508: dwrite d1/d43/fa0 [0,4194304] 0 2024-04-20T11:09:07.852 INFO:tasks.workunit.client.0.smithi117.stdout:8/401: write d0/f1 [1559522,76004] 0 2024-04-20T11:09:07.853 INFO:tasks.workunit.client.0.smithi117.stdout:8/402: chown d0/d9 1198 1 2024-04-20T11:09:07.857 INFO:tasks.workunit.client.0.smithi117.stdout:7/509: unlink d1/d4/d3d/fa3 0 2024-04-20T11:09:07.860 INFO:tasks.workunit.client.0.smithi117.stdout:9/516: creat d4/d4a/d63/d98/fae x:0 0 0 2024-04-20T11:09:07.860 INFO:tasks.workunit.client.0.smithi117.stdout:3/490: dwrite d0/d5/fc [0,4194304] 0 2024-04-20T11:09:07.863 INFO:tasks.workunit.client.0.smithi117.stdout:8/403: mkdir d0/dc/d2e/d89 0 2024-04-20T11:09:07.867 INFO:tasks.workunit.client.0.smithi117.stdout:6/578: dread d3/d5a/d74/f93 [0,4194304] 0 2024-04-20T11:09:07.869 INFO:tasks.workunit.client.0.smithi117.stdout:7/510: creat d1/d4/d3d/d93/fa6 x:0 0 0 2024-04-20T11:09:07.871 INFO:tasks.workunit.client.0.smithi117.stdout:8/404: creat d0/d8/d39/f8a x:0 0 0 2024-04-20T11:09:07.873 INFO:tasks.workunit.client.0.smithi117.stdout:3/491: read d0/d5/d21/d88/d29/d66/d63/f7a [3072548,81131] 0 2024-04-20T11:09:07.873 INFO:tasks.workunit.client.0.smithi117.stdout:8/405: chown d0/fa 1905914 1 2024-04-20T11:09:07.876 INFO:tasks.workunit.client.0.smithi117.stdout:7/511: fsync d1/d4/f30 0 2024-04-20T11:09:07.877 INFO:tasks.workunit.client.0.smithi117.stdout:8/406: write d0/dc/d49/f73 [135189,28938] 0 2024-04-20T11:09:07.881 INFO:tasks.workunit.client.0.smithi117.stdout:7/512: mknod d1/d1d/d4e/d72/ca7 0 2024-04-20T11:09:07.884 INFO:tasks.workunit.client.0.smithi117.stdout:8/407: dread d0/d8/d11/f14 [0,4194304] 0 2024-04-20T11:09:07.884 INFO:tasks.workunit.client.0.smithi117.stdout:3/492: symlink d0/d5/d73/laa 0 2024-04-20T11:09:07.886 INFO:tasks.workunit.client.0.smithi117.stdout:7/513: creat d1/d1d/d77/fa8 x:0 0 0 2024-04-20T11:09:07.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:07 smithi117 ceph-mon[15659]: pgmap v13: 65 pgs: 65 active+clean; 856 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 102 MiB/s wr, 238 op/s 2024-04-20T11:09:07.890 INFO:tasks.workunit.client.0.smithi117.stdout:7/514: chown d1/d1d/d42/d2d/c76 95 1 2024-04-20T11:09:07.890 INFO:tasks.workunit.client.0.smithi117.stdout:3/493: fsync d0/d5/d73/f95 0 2024-04-20T11:09:07.891 INFO:tasks.workunit.client.0.smithi117.stdout:8/408: chown d0/d8/d11/d24/d2b/c3f 3367590 1 2024-04-20T11:09:07.895 INFO:tasks.workunit.client.0.smithi117.stdout:7/515: truncate d1/d4/d6/d2b/f81 63424 0 2024-04-20T11:09:07.898 INFO:tasks.workunit.client.0.smithi117.stdout:8/409: creat d0/d9/d3a/f8b x:0 0 0 2024-04-20T11:09:07.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:07 smithi173 ceph-mon[23800]: pgmap v13: 65 pgs: 65 active+clean; 856 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 102 MiB/s wr, 238 op/s 2024-04-20T11:09:08.012 INFO:tasks.workunit.client.0.smithi117.stdout:2/587: truncate d1/d4/d83/d29/d2e/d53/f9d 134389 0 2024-04-20T11:09:08.013 INFO:tasks.workunit.client.0.smithi117.stdout:9/517: truncate d4/d4a/d92/f9f 4009589 0 2024-04-20T11:09:08.014 INFO:tasks.workunit.client.0.smithi117.stdout:7/516: chown d1/d4/d6/d54/d63/f85 252834 1 2024-04-20T11:09:08.018 INFO:tasks.workunit.client.0.smithi117.stdout:9/518: readlink d4/d4a/d5e/la8 0 2024-04-20T11:09:08.018 INFO:tasks.workunit.client.0.smithi117.stdout:8/410: symlink d0/d8/d11/d24/d2b/d72/l8c 0 2024-04-20T11:09:08.022 INFO:tasks.workunit.client.0.smithi117.stdout:6/579: dread d3/f18 [0,4194304] 0 2024-04-20T11:09:08.025 INFO:tasks.workunit.client.0.smithi117.stdout:8/411: symlink d0/d8/d11/d6d/d58/l8d 0 2024-04-20T11:09:08.029 INFO:tasks.workunit.client.0.smithi117.stdout:8/412: dread - d0/d8/d39/f8a zero size 2024-04-20T11:09:08.031 INFO:tasks.workunit.client.0.smithi117.stdout:9/519: rename d4/d5/da/l15 to d4/laf 0 2024-04-20T11:09:08.034 INFO:tasks.workunit.client.0.smithi117.stdout:6/580: mkdir d3/d4/d2e/dc0 0 2024-04-20T11:09:08.042 INFO:tasks.workunit.client.0.smithi117.stdout:2/588: dwrite d1/d20/f21 [0,4194304] 0 2024-04-20T11:09:08.051 INFO:tasks.workunit.client.0.smithi117.stdout:2/589: creat d1/d4/d83/d29/d2e/d80/fb9 x:0 0 0 2024-04-20T11:09:08.053 INFO:tasks.workunit.client.0.smithi117.stdout:7/517: dwrite d1/d1d/f87 [0,4194304] 0 2024-04-20T11:09:08.054 INFO:tasks.workunit.client.0.smithi117.stdout:3/494: dread d0/d5/de/d4e/f53 [0,4194304] 0 2024-04-20T11:09:08.064 INFO:tasks.workunit.client.0.smithi117.stdout:3/495: symlink d0/d2a/lab 0 2024-04-20T11:09:08.064 INFO:tasks.workunit.client.0.smithi117.stdout:8/413: write d0/f5 [4464716,78396] 0 2024-04-20T11:09:08.065 INFO:tasks.workunit.client.0.smithi117.stdout:2/590: fdatasync d1/d15/d1d/f50 0 2024-04-20T11:09:08.065 INFO:tasks.workunit.client.0.smithi117.stdout:9/520: dwrite d4/d30/f4e [0,4194304] 0 2024-04-20T11:09:08.070 INFO:tasks.workunit.client.0.smithi117.stdout:7/518: write d1/f22 [1834816,27998] 0 2024-04-20T11:09:08.071 INFO:tasks.workunit.client.0.smithi117.stdout:6/581: dwrite d3/d3a/d7f/fa4 [0,4194304] 0 2024-04-20T11:09:08.071 INFO:tasks.workunit.client.0.smithi117.stdout:9/521: read - d4/d30/d31/fac zero size 2024-04-20T11:09:08.073 INFO:tasks.workunit.client.0.smithi117.stdout:3/496: chown d0/d5/d11/f7e 0 1 2024-04-20T11:09:08.074 INFO:tasks.workunit.client.0.smithi117.stdout:8/414: creat d0/d8/d11/d24/d27/d47/f8e x:0 0 0 2024-04-20T11:09:08.077 INFO:tasks.workunit.client.0.smithi117.stdout:2/591: mknod d1/d4/dd/cba 0 2024-04-20T11:09:08.084 INFO:tasks.workunit.client.0.smithi117.stdout:8/415: chown d0/d8/d11/d24/d2b 0 1 2024-04-20T11:09:08.084 INFO:tasks.workunit.client.0.smithi117.stdout:6/582: stat d3/d4/l16 0 2024-04-20T11:09:08.085 INFO:tasks.workunit.client.0.smithi117.stdout:9/522: rename d4/d68/d6e to d4/de/d7b/d96/db0 0 2024-04-20T11:09:08.088 INFO:tasks.workunit.client.0.smithi117.stdout:2/592: chown d1/d4/d83/d29/d62/c8d 3689 1 2024-04-20T11:09:08.088 INFO:tasks.workunit.client.0.smithi117.stdout:8/416: dread d0/dc/d49/f73 [0,4194304] 0 2024-04-20T11:09:08.090 INFO:tasks.workunit.client.0.smithi117.stdout:6/583: unlink d3/d5a/d74/db3/dbc/f95 0 2024-04-20T11:09:08.097 INFO:tasks.workunit.client.0.smithi117.stdout:6/584: chown d3/d3a/d7f/l84 150 1 2024-04-20T11:09:08.098 INFO:tasks.workunit.client.0.smithi117.stdout:2/593: creat d1/d20/fbb x:0 0 0 2024-04-20T11:09:08.100 INFO:tasks.workunit.client.0.smithi117.stdout:6/585: creat d3/d4/d2e/d3f/fc1 x:0 0 0 2024-04-20T11:09:08.102 INFO:tasks.workunit.client.0.smithi117.stdout:8/417: link d0/d8/d39/d40/d78/d5c/f68 d0/f8f 0 2024-04-20T11:09:08.104 INFO:tasks.workunit.client.0.smithi117.stdout:8/418: chown d0/d8/d39/d40/d78/d5c 295720 1 2024-04-20T11:09:08.107 INFO:tasks.workunit.client.0.smithi117.stdout:6/586: unlink d3/d4/d2e/d37/l49 0 2024-04-20T11:09:08.108 INFO:tasks.workunit.client.0.smithi117.stdout:8/419: rmdir d0/d8/d11/d24/d27 39 2024-04-20T11:09:08.110 INFO:tasks.workunit.client.0.smithi117.stdout:2/594: getdents d1/d4/dd/d17 0 2024-04-20T11:09:08.115 INFO:tasks.workunit.client.0.smithi117.stdout:8/420: dread d0/d8/d11/d24/d53/f7b [0,4194304] 0 2024-04-20T11:09:08.123 INFO:tasks.workunit.client.0.smithi117.stdout:9/523: write d4/d5/f2a [6102548,61534] 0 2024-04-20T11:09:08.129 INFO:tasks.workunit.client.0.smithi117.stdout:6/587: getdents d3/d4/d2e/d3f 0 2024-04-20T11:09:08.129 INFO:tasks.workunit.client.0.smithi117.stdout:9/524: mkdir d4/de/db1 0 2024-04-20T11:09:08.131 INFO:tasks.workunit.client.0.smithi117.stdout:3/497: dwrite d0/d5/d7/f83 [4194304,4194304] 0 2024-04-20T11:09:08.136 INFO:tasks.workunit.client.0.smithi117.stdout:9/525: creat d4/d4a/d72/fb2 x:0 0 0 2024-04-20T11:09:08.136 INFO:tasks.workunit.client.0.smithi117.stdout:2/595: dwrite d1/d20/f21 [0,4194304] 0 2024-04-20T11:09:08.140 INFO:tasks.workunit.client.0.smithi117.stdout:7/519: dread d1/fa5 [0,4194304] 0 2024-04-20T11:09:08.148 INFO:tasks.workunit.client.0.smithi117.stdout:8/421: dwrite d0/d8/d11/d24/d27/d47/f8e [0,4194304] 0 2024-04-20T11:09:08.154 INFO:tasks.workunit.client.0.smithi117.stdout:2/596: fdatasync d1/d4/dd/d17/d7a/f95 0 2024-04-20T11:09:08.156 INFO:tasks.workunit.client.0.smithi117.stdout:9/526: rename d4/d5/da/f43 to d4/d68/fb3 0 2024-04-20T11:09:08.157 INFO:tasks.workunit.client.0.smithi117.stdout:0/541: sync 2024-04-20T11:09:08.158 INFO:tasks.workunit.client.0.smithi117.stdout:5/689: sync 2024-04-20T11:09:08.159 INFO:tasks.workunit.client.0.smithi117.stdout:1/451: sync 2024-04-20T11:09:08.159 INFO:tasks.workunit.client.0.smithi117.stdout:4/500: sync 2024-04-20T11:09:08.160 INFO:tasks.workunit.client.0.smithi117.stdout:5/690: chown de/d16/f1e 1430982136 1 2024-04-20T11:09:08.162 INFO:tasks.workunit.client.0.smithi117.stdout:6/588: write d3/d4/d2e/d4e/fbd [558216,70433] 0 2024-04-20T11:09:08.162 INFO:tasks.workunit.client.0.smithi117.stdout:8/422: symlink d0/d8/d39/l90 0 2024-04-20T11:09:08.164 INFO:tasks.workunit.client.0.smithi117.stdout:3/498: getdents d0/d5/d21/d88/d29/d66/d63 0 2024-04-20T11:09:08.167 INFO:tasks.workunit.client.0.smithi117.stdout:7/520: truncate d1/d4/d6/d2b/f33 1401118 0 2024-04-20T11:09:08.169 INFO:tasks.workunit.client.0.smithi117.stdout:9/527: mkdir d4/d5/d57/db4 0 2024-04-20T11:09:08.169 INFO:tasks.workunit.client.0.smithi117.stdout:4/501: read d1/d9/f76 [2761390,55986] 0 2024-04-20T11:09:08.173 INFO:tasks.workunit.client.0.smithi117.stdout:1/452: mknod d6/d16/d3a/d5a/d5c/c8d 0 2024-04-20T11:09:08.173 INFO:tasks.workunit.client.0.smithi117.stdout:7/521: readlink d1/d1d/d42/d2d/l50 0 2024-04-20T11:09:08.174 INFO:tasks.workunit.client.0.smithi117.stdout:6/589: symlink d3/d50/d8a/lc2 0 2024-04-20T11:09:08.175 INFO:tasks.workunit.client.0.smithi117.stdout:8/423: symlink d0/d8/d39/d40/d78/l91 0 2024-04-20T11:09:08.180 INFO:tasks.workunit.client.0.smithi117.stdout:9/528: creat d4/d5/da/fb5 x:0 0 0 2024-04-20T11:09:08.181 INFO:tasks.workunit.client.0.smithi117.stdout:5/691: dread de/d21/f96 [0,4194304] 0 2024-04-20T11:09:08.181 INFO:tasks.workunit.client.0.smithi117.stdout:2/597: creat d1/d4/fbc x:0 0 0 2024-04-20T11:09:08.182 INFO:tasks.workunit.client.0.smithi117.stdout:0/542: truncate d3/d7/d8/df/f1b 3560017 0 2024-04-20T11:09:08.183 INFO:tasks.workunit.client.0.smithi117.stdout:5/692: fsync de/f34 0 2024-04-20T11:09:08.186 INFO:tasks.workunit.client.0.smithi117.stdout:1/453: mknod d6/d16/d3a/d5a/d5c/d61/d71/d6b/c8e 0 2024-04-20T11:09:08.189 INFO:tasks.workunit.client.0.smithi117.stdout:3/499: getdents d0/d5/d21/d88/d29 0 2024-04-20T11:09:08.190 INFO:tasks.workunit.client.0.smithi117.stdout:7/522: mkdir d1/d1d/d42/d89/da9 0 2024-04-20T11:09:08.192 INFO:tasks.workunit.client.0.smithi117.stdout:7/523: fdatasync d1/d43/d59/fa4 0 2024-04-20T11:09:08.194 INFO:tasks.workunit.client.0.smithi117.stdout:9/529: rename d4/d5/da/d51/c66 to d4/d30/d31/d73/cb6 0 2024-04-20T11:09:08.195 INFO:tasks.workunit.client.0.smithi117.stdout:4/502: dread d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:08.196 INFO:tasks.workunit.client.0.smithi117.stdout:2/598: chown d1/d4/dd/d17/d7a/d49/f87 736290318 1 2024-04-20T11:09:08.198 INFO:tasks.workunit.client.0.smithi117.stdout:2/599: readlink d1/d4/d83/d29/d2e/l66 0 2024-04-20T11:09:08.202 INFO:tasks.workunit.client.0.smithi117.stdout:2/600: write d1/f6 [1836040,70546] 0 2024-04-20T11:09:08.204 INFO:tasks.workunit.client.0.smithi117.stdout:7/524: symlink d1/d1d/d42/laa 0 2024-04-20T11:09:08.208 INFO:tasks.workunit.client.0.smithi117.stdout:2/601: mkdir d1/d15/dbd 0 2024-04-20T11:09:08.208 INFO:tasks.workunit.client.0.smithi117.stdout:5/693: creat de/fe4 x:0 0 0 2024-04-20T11:09:08.215 INFO:tasks.workunit.client.0.smithi117.stdout:7/525: readlink d1/d4/d6/d2b/l6b 0 2024-04-20T11:09:08.215 INFO:tasks.workunit.client.0.smithi117.stdout:4/503: symlink d1/d45/d4a/laa 0 2024-04-20T11:09:08.216 INFO:tasks.workunit.client.0.smithi117.stdout:0/543: link d3/ca d3/dab/cb6 0 2024-04-20T11:09:08.216 INFO:tasks.workunit.client.0.smithi117.stdout:1/454: dwrite d6/d1f/f45 [4194304,4194304] 0 2024-04-20T11:09:08.221 INFO:tasks.workunit.client.0.smithi117.stdout:4/504: stat d1/dd/l3d 0 2024-04-20T11:09:08.226 INFO:tasks.workunit.client.0.smithi117.stdout:2/602: mkdir d1/d4/dd/d3e/d46/dbe 0 2024-04-20T11:09:08.227 INFO:tasks.workunit.client.0.smithi117.stdout:6/590: dwrite d3/d4/f6e [0,4194304] 0 2024-04-20T11:09:08.227 INFO:tasks.workunit.client.0.smithi117.stdout:8/424: dread d0/fd [0,4194304] 0 2024-04-20T11:09:08.228 INFO:tasks.workunit.client.0.smithi117.stdout:0/544: write d3/d5/d74/fa9 [938528,29352] 0 2024-04-20T11:09:08.228 INFO:tasks.workunit.client.0.smithi117.stdout:7/526: creat d1/d1d/d77/fab x:0 0 0 2024-04-20T11:09:08.231 INFO:tasks.workunit.client.0.smithi117.stdout:4/505: chown d1/d9/l22 23 1 2024-04-20T11:09:08.239 INFO:tasks.workunit.client.0.smithi117.stdout:7/527: creat d1/d1d/d42/fac x:0 0 0 2024-04-20T11:09:08.240 INFO:tasks.workunit.client.0.smithi117.stdout:6/591: creat d3/d5a/d74/fc3 x:0 0 0 2024-04-20T11:09:08.241 INFO:tasks.workunit.client.0.smithi117.stdout:0/545: rmdir d3/d7/d20/d39/d3a 39 2024-04-20T11:09:08.243 INFO:tasks.workunit.client.0.smithi117.stdout:2/603: creat d1/d15/d1d/fbf x:0 0 0 2024-04-20T11:09:08.248 INFO:tasks.workunit.client.0.smithi117.stdout:7/528: fsync d1/d4/d6/d54/d63/f85 0 2024-04-20T11:09:08.249 INFO:tasks.workunit.client.0.smithi117.stdout:1/455: dwrite d6/d1f/d30/f4e [0,4194304] 0 2024-04-20T11:09:08.255 INFO:tasks.workunit.client.0.smithi117.stdout:2/604: creat d1/d4/fc0 x:0 0 0 2024-04-20T11:09:08.263 INFO:tasks.workunit.client.0.smithi117.stdout:0/546: write d3/d7/d20/f4c [2822066,80450] 0 2024-04-20T11:09:08.263 INFO:tasks.workunit.client.0.smithi117.stdout:3/500: dread d0/d5/d21/d88/d36/f33 [0,4194304] 0 2024-04-20T11:09:08.263 INFO:tasks.workunit.client.0.smithi117.stdout:7/529: read - d1/d1d/d77/fab zero size 2024-04-20T11:09:08.263 INFO:tasks.workunit.client.0.smithi117.stdout:2/605: creat d1/d4/dd/fc1 x:0 0 0 2024-04-20T11:09:08.264 INFO:tasks.workunit.client.0.smithi117.stdout:6/592: getdents d3/d5a/d74/db3/dbc/d9b 0 2024-04-20T11:09:08.277 INFO:tasks.workunit.client.0.smithi117.stdout:4/506: dwrite d1/dd/d1a/f7c [0,4194304] 0 2024-04-20T11:09:08.282 INFO:tasks.workunit.client.0.smithi117.stdout:3/501: mkdir d0/d5/d21/d88/d29/dac 0 2024-04-20T11:09:08.288 INFO:tasks.workunit.client.0.smithi117.stdout:5/694: dread de/d21/d45/d77/f85 [0,4194304] 0 2024-04-20T11:09:08.292 INFO:tasks.workunit.client.0.smithi117.stdout:8/425: dwrite d0/d8/d39/d40/f6e [4194304,4194304] 0 2024-04-20T11:09:08.294 INFO:tasks.workunit.client.0.smithi117.stdout:4/507: creat d1/d45/d4a/d99/fab x:0 0 0 2024-04-20T11:09:08.295 INFO:tasks.workunit.client.0.smithi117.stdout:6/593: rename d3/d5a/d74/db3/laf to d3/d4/d2e/d4e/d71/da3/lc4 0 2024-04-20T11:09:08.303 INFO:tasks.workunit.client.0.smithi117.stdout:0/547: dwrite d3/d7/f80 [0,4194304] 0 2024-04-20T11:09:08.306 INFO:tasks.workunit.client.0.smithi117.stdout:8/426: write d0/f1 [1911398,6878] 0 2024-04-20T11:09:08.313 INFO:tasks.workunit.client.0.smithi117.stdout:1/456: dwrite d6/d1f/d30/d32/f6a [4194304,4194304] 0 2024-04-20T11:09:08.313 INFO:tasks.workunit.client.0.smithi117.stdout:7/530: dwrite d1/d1d/d77/fa8 [0,4194304] 0 2024-04-20T11:09:08.314 INFO:tasks.workunit.client.0.smithi117.stdout:5/695: creat de/d21/d45/d48/d4e/da1/fe5 x:0 0 0 2024-04-20T11:09:08.318 INFO:tasks.workunit.client.0.smithi117.stdout:2/606: dwrite d1/d20/f21 [0,4194304] 0 2024-04-20T11:09:08.319 INFO:tasks.workunit.client.0.smithi117.stdout:4/508: rename d1/dd/d52/c71 to d1/d9/cac 0 2024-04-20T11:09:08.319 INFO:tasks.workunit.client.0.smithi117.stdout:3/502: mkdir d0/d5/d21/d88/d29/d66/d63/dad 0 2024-04-20T11:09:08.321 INFO:tasks.workunit.client.0.smithi117.stdout:6/594: symlink d3/d5a/d74/db3/lc5 0 2024-04-20T11:09:08.322 INFO:tasks.workunit.client.0.smithi117.stdout:1/457: fsync d6/d35/f5f 0 2024-04-20T11:09:08.325 INFO:tasks.workunit.client.0.smithi117.stdout:5/696: creat de/d16/d9d/db4/fe6 x:0 0 0 2024-04-20T11:09:08.330 INFO:tasks.workunit.client.0.smithi117.stdout:7/531: mknod d1/d1d/d42/d2d/cad 0 2024-04-20T11:09:08.332 INFO:tasks.workunit.client.0.smithi117.stdout:7/532: readlink d1/d4/d6/d13/l74 0 2024-04-20T11:09:08.333 INFO:tasks.workunit.client.0.smithi117.stdout:2/607: chown d1/d4/dd/d17/d7a/d49/c85 6511870 1 2024-04-20T11:09:08.336 INFO:tasks.workunit.client.0.smithi117.stdout:3/503: unlink d0/d5/d7/f83 0 2024-04-20T11:09:08.338 INFO:tasks.workunit.client.0.smithi117.stdout:0/548: truncate d3/d7/d20/d39/f92 624888 0 2024-04-20T11:09:08.339 INFO:tasks.workunit.client.0.smithi117.stdout:2/608: write d1/f48 [1584969,106722] 0 2024-04-20T11:09:08.342 INFO:tasks.workunit.client.0.smithi117.stdout:5/697: mkdir de/d21/d45/d48/d4e/da1/de7 0 2024-04-20T11:09:08.345 INFO:tasks.workunit.client.0.smithi117.stdout:7/533: mkdir d1/d1d/d42/d2d/dae 0 2024-04-20T11:09:08.348 INFO:tasks.workunit.client.0.smithi117.stdout:3/504: chown d0/d5/d21/d94/d9a/fa0 309672 1 2024-04-20T11:09:08.350 INFO:tasks.workunit.client.0.smithi117.stdout:1/458: creat d6/d1f/d30/d49/d64/f8f x:0 0 0 2024-04-20T11:09:08.350 INFO:tasks.workunit.client.0.smithi117.stdout:2/609: chown d1/d4/d83/d29/d3f/c27 121125917 1 2024-04-20T11:09:08.350 INFO:tasks.workunit.client.0.smithi117.stdout:0/549: write d3/d7/f80 [218838,75005] 0 2024-04-20T11:09:08.353 INFO:tasks.workunit.client.0.smithi117.stdout:7/534: dread - d1/d1d/d77/fab zero size 2024-04-20T11:09:08.353 INFO:tasks.workunit.client.0.smithi117.stdout:5/698: chown de/d21/d45/d48/d4e/da1/fe5 242 1 2024-04-20T11:09:08.353 INFO:tasks.workunit.client.0.smithi117.stdout:8/427: dwrite d0/d8/d11/d24/d2b/f5d [0,4194304] 0 2024-04-20T11:09:08.375 INFO:tasks.workunit.client.0.smithi117.stdout:9/530: sync 2024-04-20T11:09:08.376 INFO:tasks.workunit.client.0.smithi117.stdout:0/550: dwrite d3/d7/d8/df/d6d/fb4 [0,4194304] 0 2024-04-20T11:09:08.381 INFO:tasks.workunit.client.0.smithi117.stdout:0/551: chown d3/d5/d74/fad 40 1 2024-04-20T11:09:08.383 INFO:tasks.workunit.client.0.smithi117.stdout:9/531: write d4/d4a/d74/f85 [584037,91760] 0 2024-04-20T11:09:08.389 INFO:tasks.workunit.client.0.smithi117.stdout:5/699: dwrite de/f34 [0,4194304] 0 2024-04-20T11:09:08.411 INFO:tasks.workunit.client.0.smithi117.stdout:8/428: dread d0/d8/f65 [0,4194304] 0 2024-04-20T11:09:08.421 INFO:tasks.workunit.client.0.smithi117.stdout:4/509: getdents d1/dd 0 2024-04-20T11:09:08.428 INFO:tasks.workunit.client.0.smithi117.stdout:3/505: mknod d0/d5/d21/d94/da7/cae 0 2024-04-20T11:09:08.429 INFO:tasks.workunit.client.0.smithi117.stdout:2/610: mknod d1/d4/dd/d3e/d46/cc2 0 2024-04-20T11:09:08.433 INFO:tasks.workunit.client.0.smithi117.stdout:2/611: readlink d1/d4/dd/d3e/l42 0 2024-04-20T11:09:08.439 INFO:tasks.workunit.client.0.smithi117.stdout:8/429: dwrite d0/f5 [4194304,4194304] 0 2024-04-20T11:09:08.445 INFO:tasks.workunit.client.0.smithi117.stdout:6/595: write d3/d5a/d74/f93 [898917,122033] 0 2024-04-20T11:09:08.445 INFO:tasks.workunit.client.0.smithi117.stdout:9/532: mknod d4/d5/d57/cb7 0 2024-04-20T11:09:08.462 INFO:tasks.workunit.client.0.smithi117.stdout:8/430: symlink d0/d8/d11/d6d/d58/l92 0 2024-04-20T11:09:08.472 INFO:tasks.workunit.client.0.smithi117.stdout:5/700: mkdir de/d21/d45/d48/de8 0 2024-04-20T11:09:08.478 INFO:tasks.workunit.client.0.smithi117.stdout:3/506: symlink d0/d5/d21/d88/d29/d66/d63/dad/laf 0 2024-04-20T11:09:08.495 INFO:tasks.workunit.client.0.smithi117.stdout:3/507: creat d0/d2a/fb0 x:0 0 0 2024-04-20T11:09:08.496 INFO:tasks.workunit.client.0.smithi117.stdout:7/535: link d1/d1d/d4e/d53/d55/f94 d1/d1d/d42/faf 0 2024-04-20T11:09:08.498 INFO:tasks.workunit.client.0.smithi117.stdout:3/508: chown d0/d5/d7/dd/f91 252611160 1 2024-04-20T11:09:08.499 INFO:tasks.workunit.client.0.smithi117.stdout:9/533: mknod d4/d35/cb8 0 2024-04-20T11:09:08.500 INFO:tasks.workunit.client.0.smithi117.stdout:7/536: truncate d1/d4/d6/d17/f4c 2241224 0 2024-04-20T11:09:08.500 INFO:tasks.workunit.client.0.smithi117.stdout:9/534: fsync f2 0 2024-04-20T11:09:08.500 INFO:tasks.workunit.client.0.smithi117.stdout:5/701: mknod de/d21/d45/ce9 0 2024-04-20T11:09:08.502 INFO:tasks.workunit.client.0.smithi117.stdout:7/537: fdatasync d1/d1d/d42/d2d/f92 0 2024-04-20T11:09:08.502 INFO:tasks.workunit.client.0.smithi117.stdout:9/535: write d4/d4a/d72/fb2 [936807,37929] 0 2024-04-20T11:09:08.503 INFO:tasks.workunit.client.0.smithi117.stdout:2/612: getdents d1/d4/d83/d8e 0 2024-04-20T11:09:08.504 INFO:tasks.workunit.client.0.smithi117.stdout:7/538: dread - d1/d4/d39/f83 zero size 2024-04-20T11:09:08.507 INFO:tasks.workunit.client.0.smithi117.stdout:2/613: write d1/d4/dd/f19 [3088506,52516] 0 2024-04-20T11:09:08.512 INFO:tasks.workunit.client.0.smithi117.stdout:4/510: mknod d1/dd/d1a/d43/cad 0 2024-04-20T11:09:08.515 INFO:tasks.workunit.client.0.smithi117.stdout:7/539: mknod d1/d4/d6/d54/d63/cb0 0 2024-04-20T11:09:08.517 INFO:tasks.workunit.client.0.smithi117.stdout:2/614: creat d1/d4/d83/fc3 x:0 0 0 2024-04-20T11:09:08.533 INFO:tasks.workunit.client.0.smithi117.stdout:2/615: getdents d1/d4/d83/d29/d2e/d80 0 2024-04-20T11:09:08.544 INFO:tasks.workunit.client.0.smithi117.stdout:2/616: link d1/d4/d83/d29/d2e/d53/d93/f9e d1/d9b/fc4 0 2024-04-20T11:09:08.546 INFO:tasks.workunit.client.0.smithi117.stdout:2/617: dread - d1/d9b/fb4 zero size 2024-04-20T11:09:08.550 INFO:tasks.workunit.client.0.smithi117.stdout:7/540: dread d1/d4/d39/f34 [0,4194304] 0 2024-04-20T11:09:08.552 INFO:tasks.workunit.client.0.smithi117.stdout:7/541: chown d1/d1d/d4e/c5e 725441145 1 2024-04-20T11:09:08.552 INFO:tasks.workunit.client.0.smithi117.stdout:2/618: dread d1/d4/d83/d29/d2e/f40 [0,4194304] 0 2024-04-20T11:09:08.567 INFO:tasks.workunit.client.0.smithi117.stdout:2/619: rmdir d1/d4/dd/d3e/d46/dbe 0 2024-04-20T11:09:08.571 INFO:tasks.workunit.client.0.smithi117.stdout:0/552: truncate d3/d7/d8/f13 2981615 0 2024-04-20T11:09:08.577 INFO:tasks.workunit.client.0.smithi117.stdout:2/620: mknod d1/d4/cc5 0 2024-04-20T11:09:08.581 INFO:tasks.workunit.client.0.smithi117.stdout:0/553: creat d3/d7/d2c/d52/fb7 x:0 0 0 2024-04-20T11:09:08.585 INFO:tasks.workunit.client.0.smithi117.stdout:6/596: write d3/d3a/f61 [1494640,79649] 0 2024-04-20T11:09:08.586 INFO:tasks.workunit.client.0.smithi117.stdout:6/597: chown d3/d4/d2e/dc0 65 1 2024-04-20T11:09:08.591 INFO:tasks.workunit.client.0.smithi117.stdout:6/598: fsync d3/d4/d2e/d3f/f77 0 2024-04-20T11:09:08.595 INFO:tasks.workunit.client.0.smithi117.stdout:8/431: write d0/d8/d11/d24/d53/f7b [111785,81389] 0 2024-04-20T11:09:08.602 INFO:tasks.workunit.client.0.smithi117.stdout:0/554: dwrite d3/dab/fa8 [0,4194304] 0 2024-04-20T11:09:08.603 INFO:tasks.workunit.client.0.smithi117.stdout:8/432: symlink d0/d8/d11/d24/d2b/d72/l93 0 2024-04-20T11:09:08.615 INFO:tasks.workunit.client.0.smithi117.stdout:3/509: write d0/d5/d7/dd/f3a [4891692,61524] 0 2024-04-20T11:09:08.623 INFO:tasks.workunit.client.0.smithi117.stdout:0/555: dwrite d3/d5/f37 [0,4194304] 0 2024-04-20T11:09:08.627 INFO:tasks.workunit.client.0.smithi117.stdout:7/542: rmdir d1/d4/d6/d54/d63 39 2024-04-20T11:09:08.629 INFO:tasks.workunit.client.0.smithi117.stdout:8/433: dwrite d0/dc/d49/f73 [0,4194304] 0 2024-04-20T11:09:08.630 INFO:tasks.workunit.client.0.smithi117.stdout:0/556: read d3/d7/d20/d47/f77 [148379,64788] 0 2024-04-20T11:09:08.630 INFO:tasks.workunit.client.0.smithi117.stdout:4/511: write d1/f35 [3045420,122782] 0 2024-04-20T11:09:08.632 INFO:tasks.workunit.client.0.smithi117.stdout:1/459: symlink d6/d16/l90 0 2024-04-20T11:09:08.635 INFO:tasks.workunit.client.0.smithi117.stdout:7/543: write d1/d43/f8d [399269,6292] 0 2024-04-20T11:09:08.642 INFO:tasks.workunit.client.0.smithi117.stdout:0/557: chown d3/d7/d8/l33 12 1 2024-04-20T11:09:08.645 INFO:tasks.workunit.client.0.smithi117.stdout:1/460: chown d6/d16/d3a/l84 2098097 1 2024-04-20T11:09:08.646 INFO:tasks.workunit.client.0.smithi117.stdout:5/702: creat de/fea x:0 0 0 2024-04-20T11:09:08.647 INFO:tasks.workunit.client.0.smithi117.stdout:4/512: creat d1/dd/d1a/d43/d9a/fae x:0 0 0 2024-04-20T11:09:08.648 INFO:tasks.workunit.client.0.smithi117.stdout:0/558: stat d3/d7/d20/f68 0 2024-04-20T11:09:08.649 INFO:tasks.workunit.client.0.smithi117.stdout:9/536: dwrite d4/d5/d3c/f47 [0,4194304] 0 2024-04-20T11:09:08.649 INFO:tasks.workunit.client.0.smithi117.stdout:3/510: dwrite d0/d5/d73/fa3 [0,4194304] 0 2024-04-20T11:09:08.658 INFO:tasks.workunit.client.0.smithi117.stdout:9/537: chown c3 9 1 2024-04-20T11:09:08.665 INFO:tasks.workunit.client.0.smithi117.stdout:0/559: write d3/d7/d8/df/d6d/f7a [1881684,61093] 0 2024-04-20T11:09:08.670 INFO:tasks.workunit.client.0.smithi117.stdout:4/513: mknod d1/dd/d1a/d2b/caf 0 2024-04-20T11:09:08.677 INFO:tasks.workunit.client.0.smithi117.stdout:3/511: creat d0/d5/d21/d94/fb1 x:0 0 0 2024-04-20T11:09:08.680 INFO:tasks.workunit.client.0.smithi117.stdout:3/512: fdatasync d0/f35 0 2024-04-20T11:09:08.681 INFO:tasks.workunit.client.0.smithi117.stdout:2/621: rename d1/l39 to d1/d4/dd/d17/d7a/d88/lc6 0 2024-04-20T11:09:08.685 INFO:tasks.workunit.client.0.smithi117.stdout:9/538: getdents d4/d5/d57/d82/da5 0 2024-04-20T11:09:08.686 INFO:tasks.workunit.client.0.smithi117.stdout:1/461: dwrite d6/d16/d3a/d5a/d5c/f7b [0,4194304] 0 2024-04-20T11:09:08.695 INFO:tasks.workunit.client.0.smithi117.stdout:5/703: dwrite de/d21/d45/d77/f85 [0,4194304] 0 2024-04-20T11:09:08.695 INFO:tasks.workunit.client.0.smithi117.stdout:3/513: creat d0/d5/d21/d88/d36/fb2 x:0 0 0 2024-04-20T11:09:08.697 INFO:tasks.workunit.client.0.smithi117.stdout:1/462: fdatasync d6/d16/d3a/d5a/d5c/d61/d71/d6b/f6c 0 2024-04-20T11:09:08.704 INFO:tasks.workunit.client.0.smithi117.stdout:3/514: mkdir d0/d5/d21/d88/d29/db3 0 2024-04-20T11:09:08.712 INFO:tasks.workunit.client.0.smithi117.stdout:5/704: rename de/d41/d47/fc6 to de/d21/d45/d48/da7/feb 0 2024-04-20T11:09:08.714 INFO:tasks.workunit.client.0.smithi117.stdout:2/622: dwrite d1/d9b/fb6 [0,4194304] 0 2024-04-20T11:09:08.719 INFO:tasks.workunit.client.0.smithi117.stdout:4/514: dwrite d1/dd/d1a/d2b/f46 [0,4194304] 0 2024-04-20T11:09:08.727 INFO:tasks.workunit.client.0.smithi117.stdout:5/705: creat de/d41/d47/d55/fec x:0 0 0 2024-04-20T11:09:08.727 INFO:tasks.workunit.client.0.smithi117.stdout:2/623: rmdir d1/d4/d83 39 2024-04-20T11:09:08.727 INFO:tasks.workunit.client.0.smithi117.stdout:6/599: write d3/d4/f8 [1525478,120785] 0 2024-04-20T11:09:08.730 INFO:tasks.workunit.client.0.smithi117.stdout:0/560: dread d3/d7/d20/f68 [0,4194304] 0 2024-04-20T11:09:08.732 INFO:tasks.workunit.client.0.smithi117.stdout:3/515: creat d0/d5/d21/d88/d85/fb4 x:0 0 0 2024-04-20T11:09:08.733 INFO:tasks.workunit.client.0.smithi117.stdout:9/539: dread d4/d68/fb3 [0,4194304] 0 2024-04-20T11:09:08.734 INFO:tasks.workunit.client.0.smithi117.stdout:5/706: symlink de/d16/d9d/led 0 2024-04-20T11:09:08.746 INFO:tasks.workunit.client.0.smithi117.stdout:3/516: creat d0/d5/d21/fb5 x:0 0 0 2024-04-20T11:09:08.750 INFO:tasks.workunit.client.0.smithi117.stdout:5/707: write de/d21/d45/fc5 [674103,64181] 0 2024-04-20T11:09:08.752 INFO:tasks.workunit.client.0.smithi117.stdout:3/517: chown d0/d5/d21/d88/d69/f8e 13586363 1 2024-04-20T11:09:08.753 INFO:tasks.workunit.client.0.smithi117.stdout:0/561: truncate d3/d7/d20/d47/d78/f7c 2809955 0 2024-04-20T11:09:08.756 INFO:tasks.workunit.client.0.smithi117.stdout:8/434: write d0/f8f [3892613,84263] 0 2024-04-20T11:09:08.758 INFO:tasks.workunit.client.0.smithi117.stdout:6/600: dwrite d3/d5a/d74/db3/dbc/f6a [0,4194304] 0 2024-04-20T11:09:08.759 INFO:tasks.workunit.client.0.smithi117.stdout:5/708: dread - de/d5b/f81 zero size 2024-04-20T11:09:08.760 INFO:tasks.workunit.client.0.smithi117.stdout:9/540: creat d4/de/db1/fb9 x:0 0 0 2024-04-20T11:09:08.761 INFO:tasks.workunit.client.0.smithi117.stdout:2/624: dread d1/d4/d83/d29/d62/f9a [0,4194304] 0 2024-04-20T11:09:08.763 INFO:tasks.workunit.client.0.smithi117.stdout:0/562: chown d3/dab/d54/c61 27243141 1 2024-04-20T11:09:08.764 INFO:tasks.workunit.client.0.smithi117.stdout:9/541: chown d4/d35 1798 1 2024-04-20T11:09:08.765 INFO:tasks.workunit.client.0.smithi117.stdout:8/435: write d0/d8/d11/d6d/d58/f71 [3638497,4897] 0 2024-04-20T11:09:08.768 INFO:tasks.workunit.client.0.smithi117.stdout:0/563: write d3/d7/f19 [652238,20083] 0 2024-04-20T11:09:08.769 INFO:tasks.workunit.client.0.smithi117.stdout:5/709: write de/d21/d45/d48/d4e/f50 [1728407,32612] 0 2024-04-20T11:09:08.771 INFO:tasks.workunit.client.0.smithi117.stdout:4/515: dread d1/dd/d1a/f1e [0,4194304] 0 2024-04-20T11:09:08.773 INFO:tasks.workunit.client.0.smithi117.stdout:6/601: dread d3/df/d39/f3e [0,4194304] 0 2024-04-20T11:09:08.774 INFO:tasks.workunit.client.0.smithi117.stdout:5/710: fdatasync de/d41/dc2/fd3 0 2024-04-20T11:09:08.777 INFO:tasks.workunit.client.0.smithi117.stdout:9/542: mknod d4/d4a/d5e/cba 0 2024-04-20T11:09:08.777 INFO:tasks.workunit.client.0.smithi117.stdout:6/602: chown d3/d4/l51 73 1 2024-04-20T11:09:08.777 INFO:tasks.workunit.client.0.smithi117.stdout:5/711: chown de/d21/f31 8862579 1 2024-04-20T11:09:08.780 INFO:tasks.workunit.client.0.smithi117.stdout:3/518: creat d0/d5/d21/d88/d29/db3/fb6 x:0 0 0 2024-04-20T11:09:08.783 INFO:tasks.workunit.client.0.smithi117.stdout:8/436: rename d0/d8/d11/d6d/d58/l8d to d0/d9/d3a/l94 0 2024-04-20T11:09:08.789 INFO:tasks.workunit.client.0.smithi117.stdout:5/712: creat de/d41/d47/d55/d56/dc8/fee x:0 0 0 2024-04-20T11:09:08.791 INFO:tasks.workunit.client.0.smithi117.stdout:0/564: rename d3/d7/d2c/f6b to d3/d7/d8/df/d22/fb8 0 2024-04-20T11:09:08.794 INFO:tasks.workunit.client.0.smithi117.stdout:5/713: mknod de/d21/d45/db9/cef 0 2024-04-20T11:09:08.798 INFO:tasks.workunit.client.0.smithi117.stdout:0/565: rmdir d3/d7/d20/d39 39 2024-04-20T11:09:08.801 INFO:tasks.workunit.client.0.smithi117.stdout:6/603: rename d3/df/d39/l91 to d3/d50/lc6 0 2024-04-20T11:09:08.802 INFO:tasks.workunit.client.0.smithi117.stdout:8/437: link d0/d8/d39/d40/f43 d0/d9/f95 0 2024-04-20T11:09:08.805 INFO:tasks.workunit.client.0.smithi117.stdout:6/604: dread - d3/d4/d2e/d4e/fbb zero size 2024-04-20T11:09:08.806 INFO:tasks.workunit.client.0.smithi117.stdout:0/566: mknod d3/d7/d20/d39/d9a/cb9 0 2024-04-20T11:09:08.810 INFO:tasks.workunit.client.0.smithi117.stdout:5/714: dwrite de/d21/d45/d77/fa6 [0,4194304] 0 2024-04-20T11:09:08.810 INFO:tasks.workunit.client.0.smithi117.stdout:8/438: link d0/d9/d3a/f8b d0/d8/d11/d24/d76/f96 0 2024-04-20T11:09:08.811 INFO:tasks.workunit.client.0.smithi117.stdout:6/605: truncate d3/df/d39/d99/f64 1619484 0 2024-04-20T11:09:08.817 INFO:tasks.workunit.client.0.smithi117.stdout:6/606: mknod d3/d5a/d74/db3/cc7 0 2024-04-20T11:09:08.820 INFO:tasks.workunit.client.0.smithi117.stdout:9/543: write d4/d4a/d74/f88 [678552,120862] 0 2024-04-20T11:09:08.822 INFO:tasks.workunit.client.0.smithi117.stdout:0/567: link d3/dab/d54/l5f d3/d7/d8/df/d22/d85/lba 0 2024-04-20T11:09:08.822 INFO:tasks.workunit.client.0.smithi117.stdout:6/607: creat d3/df/fc8 x:0 0 0 2024-04-20T11:09:08.825 INFO:tasks.workunit.client.0.smithi117.stdout:4/516: truncate d1/dd/d1a/f1d 1467199 0 2024-04-20T11:09:08.825 INFO:tasks.workunit.client.0.smithi117.stdout:9/544: dread - d4/d4a/f7e zero size 2024-04-20T11:09:08.827 INFO:tasks.workunit.client.0.smithi117.stdout:6/608: creat d3/d4/d2e/d3f/fc9 x:0 0 0 2024-04-20T11:09:08.827 INFO:tasks.workunit.client.0.smithi117.stdout:5/715: dwrite de/d41/d47/f6f [4194304,4194304] 0 2024-04-20T11:09:08.829 INFO:tasks.workunit.client.0.smithi117.stdout:4/517: creat d1/d9/fb0 x:0 0 0 2024-04-20T11:09:08.829 INFO:tasks.workunit.client.0.smithi117.stdout:0/568: read d3/d7/d20/f25 [752061,8953] 0 2024-04-20T11:09:08.844 INFO:tasks.workunit.client.0.smithi117.stdout:8/439: write d0/d8/d11/f14 [347271,114949] 0 2024-04-20T11:09:08.863 INFO:tasks.workunit.client.0.smithi117.stdout:8/440: dwrite d0/d8/d11/d6d/d69/f85 [0,4194304] 0 2024-04-20T11:09:08.865 INFO:tasks.workunit.client.0.smithi117.stdout:9/545: dwrite d4/d30/d31/f5d [0,4194304] 0 2024-04-20T11:09:08.867 INFO:tasks.workunit.client.0.smithi117.stdout:5/716: rename de/d21/d45/d77/f85 to de/d21/d84/db8/ff0 0 2024-04-20T11:09:08.867 INFO:tasks.workunit.client.0.smithi117.stdout:9/546: chown c3 337127853 1 2024-04-20T11:09:08.875 INFO:tasks.workunit.client.0.smithi117.stdout:6/609: creat d3/fca x:0 0 0 2024-04-20T11:09:08.879 INFO:tasks.workunit.client.0.smithi117.stdout:0/569: mkdir d3/d7/d20/d39/dbb 0 2024-04-20T11:09:08.883 INFO:tasks.workunit.client.0.smithi117.stdout:5/717: mkdir de/d16/d9d/db4/df1 0 2024-04-20T11:09:08.886 INFO:tasks.workunit.client.0.smithi117.stdout:4/518: rename d1/f35 to d1/d9/d1f/d90/fb1 0 2024-04-20T11:09:08.898 INFO:tasks.workunit.client.0.smithi117.stdout:0/570: dwrite d3/d7/d8/df/d6d/f9c [0,4194304] 0 2024-04-20T11:09:08.898 INFO:tasks.workunit.client.0.smithi117.stdout:6/610: dread d3/d3a/f6b [0,4194304] 0 2024-04-20T11:09:08.901 INFO:tasks.workunit.client.0.smithi117.stdout:8/441: getdents d0/d8/d11/d6d 0 2024-04-20T11:09:08.901 INFO:tasks.workunit.client.0.smithi117.stdout:0/571: readlink d3/d7/lae 0 2024-04-20T11:09:08.903 INFO:tasks.workunit.client.0.smithi117.stdout:8/442: write d0/d8/d39/f8a [577610,31547] 0 2024-04-20T11:09:08.905 INFO:tasks.workunit.client.0.smithi117.stdout:5/718: creat de/d21/d45/d48/d4e/ff2 x:0 0 0 2024-04-20T11:09:08.909 INFO:tasks.workunit.client.0.smithi117.stdout:8/443: write d0/d8/d11/d24/d27/d47/f6c [844980,122657] 0 2024-04-20T11:09:08.909 INFO:tasks.workunit.client.0.smithi117.stdout:5/719: chown de/d21/d45/d48/d4e/c76 0 1 2024-04-20T11:09:08.912 INFO:tasks.workunit.client.0.smithi117.stdout:8/444: chown d0/l50 36709797 1 2024-04-20T11:09:08.913 INFO:tasks.workunit.client.0.smithi117.stdout:0/572: rename d3/ca to d3/d5/d74/cbc 0 2024-04-20T11:09:08.914 INFO:tasks.workunit.client.0.smithi117.stdout:9/547: write d4/d5/f28 [1575392,72546] 0 2024-04-20T11:09:08.917 INFO:tasks.workunit.client.0.smithi117.stdout:5/720: creat de/d41/d47/d55/d56/dc8/ff3 x:0 0 0 2024-04-20T11:09:08.917 INFO:tasks.workunit.client.0.smithi117.stdout:8/445: write d0/d8/d11/d24/d27/d47/f8e [1580282,62377] 0 2024-04-20T11:09:08.919 INFO:tasks.workunit.client.0.smithi117.stdout:0/573: unlink d3/d7/d20/d47/l4f 0 2024-04-20T11:09:08.923 INFO:tasks.workunit.client.0.smithi117.stdout:4/519: write d1/d45/fa1 [789967,123020] 0 2024-04-20T11:09:08.926 INFO:tasks.workunit.client.0.smithi117.stdout:4/520: fsync d1/dd/d1a/d2b/f46 0 2024-04-20T11:09:08.927 INFO:tasks.workunit.client.0.smithi117.stdout:4/521: dread - d1/d9/fb0 zero size 2024-04-20T11:09:08.928 INFO:tasks.workunit.client.0.smithi117.stdout:5/721: dread de/d41/dc2/fd3 [0,4194304] 0 2024-04-20T11:09:08.929 INFO:tasks.workunit.client.0.smithi117.stdout:5/722: readlink de/d21/d45/l73 0 2024-04-20T11:09:08.931 INFO:tasks.workunit.client.0.smithi117.stdout:5/723: fdatasync de/d21/d45/d48/d4e/f50 0 2024-04-20T11:09:09.040 INFO:tasks.workunit.client.0.smithi117.stdout:9/548: creat d4/de/d7b/fbb x:0 0 0 2024-04-20T11:09:09.041 INFO:tasks.workunit.client.0.smithi117.stdout:6/611: write d3/df/f4f [4660518,29527] 0 2024-04-20T11:09:09.044 INFO:tasks.workunit.client.0.smithi117.stdout:9/549: chown d4/d5/da/l3f 3590013 1 2024-04-20T11:09:09.045 INFO:tasks.workunit.client.0.smithi117.stdout:6/612: chown d3/df/d39/d99/c90 87199 1 2024-04-20T11:09:09.047 INFO:tasks.workunit.client.0.smithi117.stdout:6/613: fsync d3/f67 0 2024-04-20T11:09:09.048 INFO:tasks.workunit.client.0.smithi117.stdout:0/574: chown d3/d5/d74/cbc 29 1 2024-04-20T11:09:09.049 INFO:tasks.workunit.client.0.smithi117.stdout:4/522: creat d1/dd/d1a/d43/d9a/fb2 x:0 0 0 2024-04-20T11:09:09.052 INFO:tasks.workunit.client.0.smithi117.stdout:5/724: mknod de/d21/d45/db9/cf4 0 2024-04-20T11:09:09.053 INFO:tasks.workunit.client.0.smithi117.stdout:4/523: write d1/d45/d4a/d99/fa0 [465644,82994] 0 2024-04-20T11:09:09.060 INFO:tasks.workunit.client.0.smithi117.stdout:5/725: rename de/d21/d35 to de/d21/df5 0 2024-04-20T11:09:09.062 INFO:tasks.workunit.client.0.smithi117.stdout:4/524: stat d1/d45/c5b 0 2024-04-20T11:09:09.063 INFO:tasks.workunit.client.0.smithi117.stdout:6/614: mknod d3/d4/d2e/d37/d8e/db2/ccb 0 2024-04-20T11:09:09.065 INFO:tasks.workunit.client.0.smithi117.stdout:8/446: dwrite d0/f20 [0,4194304] 0 2024-04-20T11:09:09.071 INFO:tasks.workunit.client.0.smithi117.stdout:5/726: creat de/d21/ff6 x:0 0 0 2024-04-20T11:09:09.072 INFO:tasks.workunit.client.0.smithi117.stdout:0/575: write d3/d7/d20/f25 [4586044,61446] 0 2024-04-20T11:09:09.073 INFO:tasks.workunit.client.0.smithi117.stdout:4/525: rename d1/dd/f4b to d1/d9/d1f/d87/fb3 0 2024-04-20T11:09:09.078 INFO:tasks.workunit.client.0.smithi117.stdout:9/550: dwrite d4/d30/f4e [0,4194304] 0 2024-04-20T11:09:09.078 INFO:tasks.workunit.client.0.smithi117.stdout:8/447: creat d0/d8/d39/d40/d78/f97 x:0 0 0 2024-04-20T11:09:09.080 INFO:tasks.workunit.client.0.smithi117.stdout:5/727: fdatasync de/d16/d9d/fb0 0 2024-04-20T11:09:09.081 INFO:tasks.workunit.client.0.smithi117.stdout:4/526: fdatasync d1/d45/d4a/d99/fab 0 2024-04-20T11:09:09.086 INFO:tasks.workunit.client.0.smithi117.stdout:5/728: stat de/d21/d45/d48/l6d 0 2024-04-20T11:09:09.089 INFO:tasks.workunit.client.0.smithi117.stdout:9/551: creat d4/fbc x:0 0 0 2024-04-20T11:09:09.093 INFO:tasks.workunit.client.0.smithi117.stdout:6/615: dwrite d3/df/f14 [4194304,4194304] 0 2024-04-20T11:09:09.096 INFO:tasks.workunit.client.0.smithi117.stdout:8/448: link d0/f57 d0/d8/d39/d40/d78/f98 0 2024-04-20T11:09:09.100 INFO:tasks.workunit.client.0.smithi117.stdout:9/552: unlink d4/d35/d54/f8b 0 2024-04-20T11:09:09.101 INFO:tasks.workunit.client.0.smithi117.stdout:8/449: readlink d0/d8/l2f 0 2024-04-20T11:09:09.107 INFO:tasks.workunit.client.0.smithi117.stdout:6/616: read - d3/d4/d2e/d4e/d71/db7/f87 zero size 2024-04-20T11:09:09.110 INFO:tasks.workunit.client.0.smithi117.stdout:0/576: dwrite d3/d7/d2c/f65 [0,4194304] 0 2024-04-20T11:09:09.112 INFO:tasks.workunit.client.0.smithi117.stdout:6/617: read - d3/fca zero size 2024-04-20T11:09:09.118 INFO:tasks.workunit.client.0.smithi117.stdout:5/729: dwrite de/f18 [0,4194304] 0 2024-04-20T11:09:09.120 INFO:tasks.workunit.client.0.smithi117.stdout:0/577: write d3/d7/d16/f56 [566434,85215] 0 2024-04-20T11:09:09.120 INFO:tasks.workunit.client.0.smithi117.stdout:6/618: rmdir d3/d4/d2e/d4e 39 2024-04-20T11:09:09.123 INFO:tasks.workunit.client.0.smithi117.stdout:4/527: creat d1/dd/d1a/d2b/d86/fb4 x:0 0 0 2024-04-20T11:09:09.123 INFO:tasks.workunit.client.0.smithi117.stdout:6/619: readlink d3/d4/d2e/d3f/la5 0 2024-04-20T11:09:09.128 INFO:tasks.workunit.client.0.smithi117.stdout:5/730: getdents de/d21/d45/d48/d4e 0 2024-04-20T11:09:09.129 INFO:tasks.workunit.client.0.smithi117.stdout:4/528: creat d1/dd/fb5 x:0 0 0 2024-04-20T11:09:09.131 INFO:tasks.workunit.client.0.smithi117.stdout:5/731: chown de/d21/d45/d48/da7/feb 1001 1 2024-04-20T11:09:09.132 INFO:tasks.workunit.client.0.smithi117.stdout:8/450: write d0/d8/d39/d40/f43 [2329996,96767] 0 2024-04-20T11:09:09.133 INFO:tasks.workunit.client.0.smithi117.stdout:4/529: mknod d1/d45/d4a/d99/cb6 0 2024-04-20T11:09:09.134 INFO:tasks.workunit.client.0.smithi117.stdout:8/451: write d0/d8/d39/f54 [684168,80045] 0 2024-04-20T11:09:09.137 INFO:tasks.workunit.client.0.smithi117.stdout:5/732: unlink de/d21/df5/d57/dd5/dce/fd9 0 2024-04-20T11:09:09.138 INFO:tasks.workunit.client.0.smithi117.stdout:8/452: truncate d0/d9/f17 1599965 0 2024-04-20T11:09:09.139 INFO:tasks.workunit.client.0.smithi117.stdout:5/733: fdatasync de/d21/d45/d48/d4e/ff2 0 2024-04-20T11:09:09.140 INFO:tasks.workunit.client.0.smithi117.stdout:8/453: write d0/dc/f1e [1223444,34395] 0 2024-04-20T11:09:09.146 INFO:tasks.workunit.client.0.smithi117.stdout:8/454: mkdir d0/d8/d11/d24/d27/d99 0 2024-04-20T11:09:09.147 INFO:tasks.workunit.client.0.smithi117.stdout:5/734: link de/d41/f52 de/ff7 0 2024-04-20T11:09:09.153 INFO:tasks.workunit.client.0.smithi117.stdout:5/735: truncate de/d41/f52 2951248 0 2024-04-20T11:09:09.161 INFO:tasks.workunit.client.0.smithi117.stdout:5/736: creat de/d21/df5/d57/dd5/dce/ff8 x:0 0 0 2024-04-20T11:09:09.164 INFO:tasks.workunit.client.0.smithi117.stdout:9/553: dread d4/d4a/d92/f9f [0,4194304] 0 2024-04-20T11:09:09.167 INFO:tasks.workunit.client.0.smithi117.stdout:5/737: rename de/d21/d45 to de/d41/d4b/df9 0 2024-04-20T11:09:09.168 INFO:tasks.workunit.client.0.smithi117.stdout:9/554: creat d4/de/d7b/fbd x:0 0 0 2024-04-20T11:09:09.172 INFO:tasks.workunit.client.0.smithi117.stdout:5/738: symlink de/d41/d47/d55/dab/lfa 0 2024-04-20T11:09:09.236 INFO:tasks.workunit.client.0.smithi117.stdout:9/555: creat d4/d5/d57/d82/da5/fbe x:0 0 0 2024-04-20T11:09:09.236 INFO:tasks.workunit.client.0.smithi117.stdout:5/739: mknod de/d21/d84/cfb 0 2024-04-20T11:09:09.238 INFO:tasks.workunit.client.0.smithi117.stdout:0/578: dread d3/d7/f18 [0,4194304] 0 2024-04-20T11:09:09.239 INFO:tasks.workunit.client.0.smithi117.stdout:0/579: chown d3/d7/f18 2101 1 2024-04-20T11:09:09.240 INFO:tasks.workunit.client.0.smithi117.stdout:5/740: write de/d41/fb6 [1086362,120208] 0 2024-04-20T11:09:09.242 INFO:tasks.workunit.client.0.smithi117.stdout:9/556: creat d4/d30/d31/fbf x:0 0 0 2024-04-20T11:09:09.243 INFO:tasks.workunit.client.0.smithi117.stdout:5/741: dread - de/d41/d47/fe2 zero size 2024-04-20T11:09:09.245 INFO:tasks.workunit.client.0.smithi117.stdout:5/742: fdatasync de/d41/d4b/df9/d48/d4e/f50 0 2024-04-20T11:09:09.252 INFO:tasks.workunit.client.0.smithi117.stdout:5/743: creat de/d16/d9d/db4/df1/ffc x:0 0 0 2024-04-20T11:09:09.253 INFO:tasks.workunit.client.0.smithi117.stdout:5/744: stat de/f8e 0 2024-04-20T11:09:09.261 INFO:tasks.workunit.client.0.smithi117.stdout:5/745: dread de/d21/df5/f8d [0,4194304] 0 2024-04-20T11:09:09.261 INFO:tasks.workunit.client.0.smithi117.stdout:9/557: dwrite d4/d4a/d72/fb2 [0,4194304] 0 2024-04-20T11:09:09.264 INFO:tasks.workunit.client.0.smithi117.stdout:5/746: fdatasync de/d41/d47/d55/fec 0 2024-04-20T11:09:09.264 INFO:tasks.workunit.client.0.smithi117.stdout:9/558: readlink d4/d35/l95 0 2024-04-20T11:09:09.270 INFO:tasks.workunit.client.0.smithi117.stdout:0/580: dwrite d3/d7/f14 [8388608,4194304] 0 2024-04-20T11:09:09.273 INFO:tasks.workunit.client.0.smithi117.stdout:0/581: fdatasync d3/d7/f80 0 2024-04-20T11:09:09.282 INFO:tasks.workunit.client.0.smithi117.stdout:0/582: mkdir d3/d7/d8/df/db2/dbd 0 2024-04-20T11:09:09.285 INFO:tasks.workunit.client.0.smithi117.stdout:0/583: chown d3/d7/d8/df/c27 47 1 2024-04-20T11:09:09.285 INFO:tasks.workunit.client.0.smithi117.stdout:5/747: dread de/d41/d4b/df9/d48/f4a [0,4194304] 0 2024-04-20T11:09:09.289 INFO:tasks.workunit.client.0.smithi117.stdout:9/559: dwrite d4/d4a/d74/f85 [0,4194304] 0 2024-04-20T11:09:09.291 INFO:tasks.workunit.client.0.smithi117.stdout:5/748: mkdir de/d41/d4b/df9/dfd 0 2024-04-20T11:09:09.297 INFO:tasks.workunit.client.0.smithi117.stdout:0/584: link d3/d7/d8/f64 d3/d7/d20/d39/d3a/db1/fbe 0 2024-04-20T11:09:09.301 INFO:tasks.workunit.client.0.smithi117.stdout:0/585: symlink d3/d7e/lbf 0 2024-04-20T11:09:09.306 INFO:tasks.workunit.client.0.smithi117.stdout:0/586: rename d3/d7/l88 to d3/d7/d8/df/lc0 0 2024-04-20T11:09:09.310 INFO:tasks.workunit.client.0.smithi117.stdout:0/587: unlink d3/d7/d20/f25 0 2024-04-20T11:09:09.311 INFO:tasks.workunit.client.0.smithi117.stdout:0/588: chown d3/d5/d74/cbc 22 1 2024-04-20T11:09:09.314 INFO:tasks.workunit.client.0.smithi117.stdout:5/749: dread de/f74 [0,4194304] 0 2024-04-20T11:09:09.318 INFO:tasks.workunit.client.0.smithi117.stdout:7/544: sync 2024-04-20T11:09:09.320 INFO:tasks.workunit.client.0.smithi117.stdout:7/545: fsync d1/d1d/d4e/d53/d55/f98 0 2024-04-20T11:09:09.323 INFO:tasks.workunit.client.0.smithi117.stdout:9/560: dread d4/d35/f81 [0,4194304] 0 2024-04-20T11:09:09.332 INFO:tasks.workunit.client.0.smithi117.stdout:0/589: dwrite d3/d7/f80 [0,4194304] 0 2024-04-20T11:09:09.337 INFO:tasks.workunit.client.0.smithi117.stdout:5/750: write de/d21/d84/f87 [4477191,13276] 0 2024-04-20T11:09:09.340 INFO:tasks.workunit.client.0.smithi117.stdout:7/546: dwrite d1/d1d/d77/fa8 [0,4194304] 0 2024-04-20T11:09:09.342 INFO:tasks.workunit.client.0.smithi117.stdout:5/751: mkdir de/d21/df5/d57/dd5/dfe 0 2024-04-20T11:09:09.345 INFO:tasks.workunit.client.0.smithi117.stdout:7/547: mkdir d1/d1d/d4e/d53/db1 0 2024-04-20T11:09:09.357 INFO:tasks.workunit.client.0.smithi117.stdout:5/752: rename de/d41/d4b/df9/d48/l6d to de/d41/d47/d55/d56/dc8/dd8/lff 0 2024-04-20T11:09:09.359 INFO:tasks.workunit.client.0.smithi117.stdout:0/590: dwrite d3/f3e [4194304,4194304] 0 2024-04-20T11:09:09.363 INFO:tasks.workunit.client.0.smithi117.stdout:0/591: chown d3/dab/l4e 153762 1 2024-04-20T11:09:09.370 INFO:tasks.workunit.client.0.smithi117.stdout:0/592: symlink d3/dab/d54/lc1 0 2024-04-20T11:09:09.373 INFO:tasks.workunit.client.0.smithi117.stdout:0/593: chown d3/d7/d20/d47/d78 64597 1 2024-04-20T11:09:09.375 INFO:tasks.workunit.client.0.smithi117.stdout:7/548: write d1/d4/d6/d17/d3f/f90 [57785,77104] 0 2024-04-20T11:09:09.381 INFO:tasks.workunit.client.0.smithi117.stdout:0/594: fsync d3/d7/d8/f13 0 2024-04-20T11:09:09.385 INFO:tasks.workunit.client.0.smithi117.stdout:9/561: dread d4/d5/f2a [0,4194304] 0 2024-04-20T11:09:09.386 INFO:tasks.workunit.client.0.smithi117.stdout:0/595: chown d3/d7/d2c/f69 1 1 2024-04-20T11:09:09.386 INFO:tasks.workunit.client.0.smithi117.stdout:1/463: sync 2024-04-20T11:09:09.387 INFO:tasks.workunit.client.0.smithi117.stdout:7/549: read d1/d1d/d42/faf [42780,86550] 0 2024-04-20T11:09:09.388 INFO:tasks.workunit.client.0.smithi117.stdout:9/562: chown d4/d30/d31/d73 61 1 2024-04-20T11:09:09.392 INFO:tasks.workunit.client.0.smithi117.stdout:0/596: creat d3/d7/d20/d47/fc2 x:0 0 0 2024-04-20T11:09:09.394 INFO:tasks.workunit.client.0.smithi117.stdout:9/563: readlink d4/de/d10/l67 0 2024-04-20T11:09:09.399 INFO:tasks.workunit.client.0.smithi117.stdout:5/753: dwrite de/d41/f9b [0,4194304] 0 2024-04-20T11:09:09.404 INFO:tasks.workunit.client.0.smithi117.stdout:1/464: getdents d6/d1f/d30/d49/d69/d6e/d6f/d73 0 2024-04-20T11:09:09.405 INFO:tasks.workunit.client.0.smithi117.stdout:7/550: write d1/d4/d6/d2b/f33 [1427992,79602] 0 2024-04-20T11:09:09.405 INFO:tasks.workunit.client.0.smithi117.stdout:0/597: dread d3/d7/f7d [0,4194304] 0 2024-04-20T11:09:09.409 INFO:tasks.workunit.client.0.smithi117.stdout:7/551: dread - d1/d43/f7b zero size 2024-04-20T11:09:09.410 INFO:tasks.workunit.client.0.smithi117.stdout:1/465: mkdir d6/d16/d3a/d5a/d5c/d61/d91 0 2024-04-20T11:09:09.419 INFO:tasks.workunit.client.0.smithi117.stdout:7/552: dread d1/d43/f8d [0,4194304] 0 2024-04-20T11:09:09.422 INFO:tasks.workunit.client.0.smithi117.stdout:1/466: creat d6/d1f/d30/d49/d69/f92 x:0 0 0 2024-04-20T11:09:09.428 INFO:tasks.workunit.client.0.smithi117.stdout:7/553: creat d1/d4/d6/fb2 x:0 0 0 2024-04-20T11:09:09.431 INFO:tasks.workunit.client.0.smithi117.stdout:1/467: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/f93 x:0 0 0 2024-04-20T11:09:09.435 INFO:tasks.workunit.client.0.smithi117.stdout:5/754: dwrite de/d41/d4b/df9/d48/da7/feb [0,4194304] 0 2024-04-20T11:09:09.437 INFO:tasks.workunit.client.0.smithi117.stdout:0/598: dwrite d3/d7/d20/d47/d78/f8d [0,4194304] 0 2024-04-20T11:09:09.437 INFO:tasks.workunit.client.0.smithi117.stdout:1/468: unlink d6/d1f/f40 0 2024-04-20T11:09:09.437 INFO:tasks.workunit.client.0.smithi117.stdout:9/564: dread d4/d30/f36 [0,4194304] 0 2024-04-20T11:09:09.444 INFO:tasks.workunit.client.0.smithi117.stdout:8/455: sync 2024-04-20T11:09:09.444 INFO:tasks.workunit.client.0.smithi117.stdout:6/620: sync 2024-04-20T11:09:09.444 INFO:tasks.workunit.client.0.smithi117.stdout:3/519: sync 2024-04-20T11:09:09.445 INFO:tasks.workunit.client.0.smithi117.stdout:2/625: sync 2024-04-20T11:09:09.448 INFO:tasks.workunit.client.0.smithi117.stdout:5/755: creat de/d41/d4b/df9/db9/dda/f100 x:0 0 0 2024-04-20T11:09:09.448 INFO:tasks.workunit.client.0.smithi117.stdout:4/530: sync 2024-04-20T11:09:09.449 INFO:tasks.workunit.client.0.smithi117.stdout:0/599: truncate d3/d7/d8/f13 3162298 0 2024-04-20T11:09:09.453 INFO:tasks.workunit.client.0.smithi117.stdout:3/520: stat d0/d5/d21/d88/d29/d66/f6c 0 2024-04-20T11:09:09.456 INFO:tasks.workunit.client.0.smithi117.stdout:2/626: creat d1/d9b/fc7 x:0 0 0 2024-04-20T11:09:09.457 INFO:tasks.workunit.client.0.smithi117.stdout:6/621: read d3/df/f63 [136760,92790] 0 2024-04-20T11:09:09.458 INFO:tasks.workunit.client.0.smithi117.stdout:7/554: write d1/d4/d6/d54/d63/f85 [201650,125480] 0 2024-04-20T11:09:09.459 INFO:tasks.workunit.client.0.smithi117.stdout:2/627: stat d1/d4/dd/d17/d3b/f6c 0 2024-04-20T11:09:09.460 INFO:tasks.workunit.client.0.smithi117.stdout:0/600: symlink d3/d7/d8/df/lc3 0 2024-04-20T11:09:09.466 INFO:tasks.workunit.client.0.smithi117.stdout:0/601: readlink d3/d7/d16/l55 0 2024-04-20T11:09:09.468 INFO:tasks.workunit.client.0.smithi117.stdout:7/555: write d1/d1d/d4e/d53/d55/f98 [230538,9115] 0 2024-04-20T11:09:09.468 INFO:tasks.workunit.client.0.smithi117.stdout:1/469: getdents d6/d1f 0 2024-04-20T11:09:09.474 INFO:tasks.workunit.client.0.smithi117.stdout:5/756: rename de/d41/d4b/df9/d48/d4e/f4f to de/d16/d9d/db4/df1/f101 0 2024-04-20T11:09:09.478 INFO:tasks.workunit.client.0.smithi117.stdout:5/757: chown de/d41/d47/d55 4849002 1 2024-04-20T11:09:09.488 INFO:tasks.workunit.client.0.smithi117.stdout:4/531: symlink d1/dd/d1a/d43/d9a/lb7 0 2024-04-20T11:09:09.489 INFO:tasks.workunit.client.0.smithi117.stdout:7/556: mkdir d1/d1d/d42/d2d/d6c/db3 0 2024-04-20T11:09:09.494 INFO:tasks.workunit.client.0.smithi117.stdout:9/565: dwrite d4/d30/f36 [0,4194304] 0 2024-04-20T11:09:09.502 INFO:tasks.workunit.client.0.smithi117.stdout:4/532: fdatasync d1/f30 0 2024-04-20T11:09:09.504 INFO:tasks.workunit.client.0.smithi117.stdout:8/456: dwrite d0/dc/f1e [0,4194304] 0 2024-04-20T11:09:09.505 INFO:tasks.workunit.client.0.smithi117.stdout:0/602: creat d3/d7/d20/d39/d3a/db1/fc4 x:0 0 0 2024-04-20T11:09:09.506 INFO:tasks.workunit.client.0.smithi117.stdout:5/758: creat de/d16/d9d/db4/f102 x:0 0 0 2024-04-20T11:09:09.524 INFO:tasks.workunit.client.0.smithi117.stdout:4/533: chown d1/d45/f5f 29202588 1 2024-04-20T11:09:09.528 INFO:tasks.workunit.client.0.smithi117.stdout:3/521: dread d0/f86 [0,4194304] 0 2024-04-20T11:09:09.535 INFO:tasks.workunit.client.0.smithi117.stdout:7/557: mkdir d1/d4/d6/d54/db4 0 2024-04-20T11:09:09.538 INFO:tasks.workunit.client.0.smithi117.stdout:2/628: dwrite d1/d4/d83/d29/d2e/f64 [0,4194304] 0 2024-04-20T11:09:09.539 INFO:tasks.workunit.client.0.smithi117.stdout:5/759: rename de/d16/f97 to de/d16/d9d/db4/df1/f103 0 2024-04-20T11:09:09.539 INFO:tasks.workunit.client.0.smithi117.stdout:1/470: dwrite d6/f13 [4194304,4194304] 0 2024-04-20T11:09:09.548 INFO:tasks.workunit.client.0.smithi117.stdout:6/622: dwrite d3/df/d39/d99/f64 [0,4194304] 0 2024-04-20T11:09:09.557 INFO:tasks.workunit.client.0.smithi117.stdout:9/566: getdents d4/d5/d57 0 2024-04-20T11:09:09.557 INFO:tasks.workunit.client.0.smithi117.stdout:8/457: dread d0/dc/f25 [0,4194304] 0 2024-04-20T11:09:09.558 INFO:tasks.workunit.client.0.smithi117.stdout:3/522: truncate d0/d5/d21/d94/d9a/fa0 842536 0 2024-04-20T11:09:09.563 INFO:tasks.workunit.client.0.smithi117.stdout:8/458: read d0/d9/f33 [1106094,3467] 0 2024-04-20T11:09:09.567 INFO:tasks.workunit.client.0.smithi117.stdout:7/558: rename d1/d1d/d42/d2d/cad to d1/d1d/d4e/d53/db1/cb5 0 2024-04-20T11:09:09.568 INFO:tasks.workunit.client.0.smithi117.stdout:1/471: creat d6/d16/d3a/f94 x:0 0 0 2024-04-20T11:09:09.575 INFO:tasks.workunit.client.0.smithi117.stdout:3/523: symlink d0/d5/d21/d75/lb7 0 2024-04-20T11:09:09.580 INFO:tasks.workunit.client.0.smithi117.stdout:7/559: read - d1/d4/d6/d17/d3f/f9b zero size 2024-04-20T11:09:09.586 INFO:tasks.workunit.client.0.smithi117.stdout:9/567: rename d4/d4a/d92 to d4/d4a/d63/d98/dc0 0 2024-04-20T11:09:09.590 INFO:tasks.workunit.client.0.smithi117.stdout:9/568: write d4/d5/f28 [1106963,33707] 0 2024-04-20T11:09:09.590 INFO:tasks.workunit.client.0.smithi117.stdout:2/629: mknod d1/d4/d83/d29/d2e/cc8 0 2024-04-20T11:09:09.590 INFO:tasks.workunit.client.0.smithi117.stdout:6/623: dwrite d3/d4/d2e/d4e/fa2 [0,4194304] 0 2024-04-20T11:09:09.593 INFO:tasks.workunit.client.0.smithi117.stdout:9/569: chown d4/d35/d54 1432 1 2024-04-20T11:09:09.596 INFO:tasks.workunit.client.0.smithi117.stdout:5/760: creat de/d41/d4b/df9/d48/f104 x:0 0 0 2024-04-20T11:09:09.597 INFO:tasks.workunit.client.0.smithi117.stdout:6/624: truncate d3/d4/d2e/d3f/fc9 649649 0 2024-04-20T11:09:09.598 INFO:tasks.workunit.client.0.smithi117.stdout:0/603: dread d3/dab/d54/fa3 [0,4194304] 0 2024-04-20T11:09:09.614 INFO:tasks.workunit.client.0.smithi117.stdout:3/524: dread d0/f2c [4194304,4194304] 0 2024-04-20T11:09:09.616 INFO:tasks.workunit.client.0.smithi117.stdout:3/525: fsync d0/d2a/fb0 0 2024-04-20T11:09:09.619 INFO:tasks.workunit.client.0.smithi117.stdout:3/526: write d0/d5/d11/f8f [345532,79172] 0 2024-04-20T11:09:09.625 INFO:tasks.workunit.client.0.smithi117.stdout:3/527: fdatasync d0/d5/d21/d88/d69/f6d 0 2024-04-20T11:09:09.625 INFO:tasks.workunit.client.0.smithi117.stdout:4/534: truncate d1/dd/f13 2842139 0 2024-04-20T11:09:09.630 INFO:tasks.workunit.client.0.smithi117.stdout:4/535: dread - d1/d45/d4a/d99/fab zero size 2024-04-20T11:09:09.630 INFO:tasks.workunit.client.0.smithi117.stdout:0/604: creat d3/d7/d2c/d52/fc5 x:0 0 0 2024-04-20T11:09:09.634 INFO:tasks.workunit.client.0.smithi117.stdout:2/630: rename d1/d4/d83/d29/d2e/d53/f9d to d1/d4/dd/d17/d3b/fc9 0 2024-04-20T11:09:09.636 INFO:tasks.workunit.client.0.smithi117.stdout:2/631: chown d1/d9b 209858 1 2024-04-20T11:09:09.636 INFO:tasks.workunit.client.0.smithi117.stdout:6/625: creat d3/d94/fcc x:0 0 0 2024-04-20T11:09:09.640 INFO:tasks.workunit.client.0.smithi117.stdout:0/605: truncate d3/d7/d8/df/d6d/f90 641102 0 2024-04-20T11:09:09.640 INFO:tasks.workunit.client.0.smithi117.stdout:3/528: fdatasync d0/d5/d21/d88/f70 0 2024-04-20T11:09:09.642 INFO:tasks.workunit.client.0.smithi117.stdout:9/570: dwrite d4/de/d4b/f6d [0,4194304] 0 2024-04-20T11:09:09.645 INFO:tasks.workunit.client.0.smithi117.stdout:1/472: write d6/f9 [2959484,115083] 0 2024-04-20T11:09:09.645 INFO:tasks.workunit.client.0.smithi117.stdout:8/459: truncate d0/d8/d11/d24/d27/d47/f6c 281147 0 2024-04-20T11:09:09.656 INFO:tasks.workunit.client.0.smithi117.stdout:7/560: getdents d1/d1d 0 2024-04-20T11:09:09.659 INFO:tasks.workunit.client.0.smithi117.stdout:5/761: dwrite de/d21/d84/db8/ff0 [0,4194304] 0 2024-04-20T11:09:09.667 INFO:tasks.workunit.client.0.smithi117.stdout:0/606: mkdir d3/d7/dc6 0 2024-04-20T11:09:09.670 INFO:tasks.workunit.client.0.smithi117.stdout:0/607: chown d3/d7/d94/ca1 124 1 2024-04-20T11:09:09.673 INFO:tasks.workunit.client.0.smithi117.stdout:2/632: truncate d1/d4/dd/d17/d7a/f95 2602059 0 2024-04-20T11:09:09.675 INFO:tasks.workunit.client.0.smithi117.stdout:8/460: mknod d0/d8/d39/d40/d78/c9a 0 2024-04-20T11:09:09.675 INFO:tasks.workunit.client.0.smithi117.stdout:1/473: mkdir d6/d21/d95 0 2024-04-20T11:09:09.676 INFO:tasks.workunit.client.0.smithi117.stdout:4/536: dwrite d1/f19 [8388608,4194304] 0 2024-04-20T11:09:09.678 INFO:tasks.workunit.client.0.smithi117.stdout:4/537: stat d1/fa7 0 2024-04-20T11:09:09.690 INFO:tasks.workunit.client.0.smithi117.stdout:7/561: fdatasync d1/d4/d6/d2b/d64/f78 0 2024-04-20T11:09:09.690 INFO:tasks.workunit.client.0.smithi117.stdout:5/762: write de/d21/df5/f8d [1284819,46669] 0 2024-04-20T11:09:09.695 INFO:tasks.workunit.client.0.smithi117.stdout:7/562: chown d1/d4/d3d/f7c 185379871 1 2024-04-20T11:09:09.695 INFO:tasks.workunit.client.0.smithi117.stdout:0/608: unlink d3/d7/d2c/f69 0 2024-04-20T11:09:09.699 INFO:tasks.workunit.client.0.smithi117.stdout:7/563: dread - d1/d4/d3d/f9e zero size 2024-04-20T11:09:09.702 INFO:tasks.workunit.client.0.smithi117.stdout:4/538: mknod d1/d9/d1f/d90/cb8 0 2024-04-20T11:09:09.703 INFO:tasks.workunit.client.0.smithi117.stdout:3/529: dwrite d0/f71 [0,4194304] 0 2024-04-20T11:09:09.705 INFO:tasks.workunit.client.0.smithi117.stdout:9/571: write d4/d68/fb3 [2123575,18169] 0 2024-04-20T11:09:09.706 INFO:tasks.workunit.client.0.smithi117.stdout:3/530: readlink d0/d5/de/l5e 0 2024-04-20T11:09:09.707 INFO:tasks.workunit.client.0.smithi117.stdout:7/564: write d1/d43/fa0 [1271555,118365] 0 2024-04-20T11:09:09.709 INFO:tasks.workunit.client.0.smithi117.stdout:6/626: dread d3/d5a/d74/fb6 [0,4194304] 0 2024-04-20T11:09:09.711 INFO:tasks.workunit.client.0.smithi117.stdout:7/565: chown d1/d1d/d42/f8b 9645 1 2024-04-20T11:09:09.721 INFO:tasks.workunit.client.0.smithi117.stdout:1/474: creat d6/d16/d3a/d5a/d5c/d61/d91/f96 x:0 0 0 2024-04-20T11:09:09.722 INFO:tasks.workunit.client.0.smithi117.stdout:0/609: chown d3/d7/d16/f2e 442 1 2024-04-20T11:09:09.726 INFO:tasks.workunit.client.0.smithi117.stdout:3/531: mknod d0/d5/d11/d80/cb8 0 2024-04-20T11:09:09.729 INFO:tasks.workunit.client.0.smithi117.stdout:3/532: chown d0/d5/d21/d94/d9a/fa0 17136231 1 2024-04-20T11:09:09.733 INFO:tasks.workunit.client.0.smithi117.stdout:4/539: dwrite d1/d45/d4a/d99/fab [0,4194304] 0 2024-04-20T11:09:09.733 INFO:tasks.workunit.client.0.smithi117.stdout:7/566: mkdir d1/d1d/d4e/d53/d55/db6 0 2024-04-20T11:09:09.737 INFO:tasks.workunit.client.0.smithi117.stdout:6/627: dwrite d3/d4/f46 [0,4194304] 0 2024-04-20T11:09:09.739 INFO:tasks.workunit.client.0.smithi117.stdout:4/540: chown d1/d9/f66 507504154 1 2024-04-20T11:09:09.742 INFO:tasks.workunit.client.0.smithi117.stdout:9/572: symlink d4/d4a/d72/lc1 0 2024-04-20T11:09:09.744 INFO:tasks.workunit.client.0.smithi117.stdout:5/763: fdatasync de/d41/d47/d4c/fb1 0 2024-04-20T11:09:09.745 INFO:tasks.workunit.client.0.smithi117.stdout:2/633: link d1/d20/c91 d1/d4/d83/d29/d2e/d52/cca 0 2024-04-20T11:09:09.746 INFO:tasks.workunit.client.0.smithi117.stdout:3/533: write d0/f3 [3695072,92631] 0 2024-04-20T11:09:09.750 INFO:tasks.workunit.client.0.smithi117.stdout:8/461: getdents d0/d8/d39/d40/d78/d5c 0 2024-04-20T11:09:09.751 INFO:tasks.workunit.client.0.smithi117.stdout:2/634: write d1/d4/dd/d17/d3b/f6c [1497062,3798] 0 2024-04-20T11:09:09.755 INFO:tasks.workunit.client.0.smithi117.stdout:2/635: write d1/d4/fa0 [783989,76071] 0 2024-04-20T11:09:09.757 INFO:tasks.workunit.client.0.smithi117.stdout:8/462: write d0/d8/d11/d6d/d58/f71 [4111763,61982] 0 2024-04-20T11:09:09.762 INFO:tasks.workunit.client.0.smithi117.stdout:4/541: chown d1/dd/d52/f54 113 1 2024-04-20T11:09:09.767 INFO:tasks.workunit.client.0.smithi117.stdout:9/573: symlink d4/d5/lc2 0 2024-04-20T11:09:09.771 INFO:tasks.workunit.client.0.smithi117.stdout:9/574: chown d4/d5/da/d51 81940 1 2024-04-20T11:09:09.775 INFO:tasks.workunit.client.0.smithi117.stdout:1/475: rename d6/d35/f5f to d6/d35/f97 0 2024-04-20T11:09:09.776 INFO:tasks.workunit.client.0.smithi117.stdout:2/636: mknod d1/d4/d83/d29/d2e/d53/ccb 0 2024-04-20T11:09:09.780 INFO:tasks.workunit.client.0.smithi117.stdout:7/567: dread d1/f86 [0,4194304] 0 2024-04-20T11:09:09.783 INFO:tasks.workunit.client.0.smithi117.stdout:5/764: stat de/d41/d4b/df9/d48/d4e/da1 0 2024-04-20T11:09:09.790 INFO:tasks.workunit.client.0.smithi117.stdout:0/610: dread d3/d7/d8/df/d6d/f7a [0,4194304] 0 2024-04-20T11:09:09.799 INFO:tasks.workunit.client.0.smithi117.stdout:8/463: link d0/d8/d11/d24/d27/d47/f8e d0/d8/d39/d40/d78/f9b 0 2024-04-20T11:09:09.800 INFO:tasks.workunit.client.0.smithi117.stdout:7/568: creat d1/d1d/d42/d2d/fb7 x:0 0 0 2024-04-20T11:09:09.800 INFO:tasks.workunit.client.0.smithi117.stdout:3/534: link d0/d2a/c68 d0/d5/d21/d88/d85/cb9 0 2024-04-20T11:09:09.800 INFO:tasks.workunit.client.0.smithi117.stdout:6/628: write d3/d4/d2e/d3f/f77 [939740,71696] 0 2024-04-20T11:09:09.800 INFO:tasks.workunit.client.0.smithi117.stdout:5/765: symlink de/d41/d47/d55/d56/dc8/dd8/l105 0 2024-04-20T11:09:09.803 INFO:tasks.workunit.client.0.smithi117.stdout:3/535: dread - d0/d5/d21/d88/d36/fb2 zero size 2024-04-20T11:09:09.804 INFO:tasks.workunit.client.0.smithi117.stdout:5/766: fsync de/d16/d9d/db4/f102 0 2024-04-20T11:09:09.808 INFO:tasks.workunit.client.0.smithi117.stdout:9/575: fsync d4/d35/f56 0 2024-04-20T11:09:09.809 INFO:tasks.workunit.client.0.smithi117.stdout:3/536: readlink d0/d5/d21/d88/d29/d66/d63/dad/laf 0 2024-04-20T11:09:09.812 INFO:tasks.workunit.client.0.smithi117.stdout:4/542: write d1/dd/d1a/f1d [2365360,90976] 0 2024-04-20T11:09:09.815 INFO:tasks.workunit.client.0.smithi117.stdout:3/537: truncate d0/d5/d7/dd/f5a 1921275 0 2024-04-20T11:09:09.821 INFO:tasks.workunit.client.0.smithi117.stdout:2/637: creat d1/d4/dd/d17/d7a/d49/fcc x:0 0 0 2024-04-20T11:09:09.834 INFO:tasks.workunit.client.0.smithi117.stdout:5/767: dwrite de/d41/d4b/df9/d77/fa6 [0,4194304] 0 2024-04-20T11:09:09.839 INFO:tasks.workunit.client.0.smithi117.stdout:8/464: truncate d0/d8/d39/d40/f5f 1565513 0 2024-04-20T11:09:09.841 INFO:tasks.workunit.client.0.smithi117.stdout:5/768: chown de/dd0 1904769 1 2024-04-20T11:09:09.841 INFO:tasks.workunit.client.0.smithi117.stdout:6/629: truncate d3/d4/d2e/d37/f47 1480937 0 2024-04-20T11:09:09.844 INFO:tasks.workunit.client.0.smithi117.stdout:2/638: creat d1/d4/dd/d3e/d46/fcd x:0 0 0 2024-04-20T11:09:09.845 INFO:tasks.workunit.client.0.smithi117.stdout:9/576: dwrite d4/d4a/d74/f88 [0,4194304] 0 2024-04-20T11:09:09.848 INFO:tasks.workunit.client.0.smithi117.stdout:8/465: stat d0/d8/d11/d6d/d58 0 2024-04-20T11:09:09.848 INFO:tasks.workunit.client.0.smithi117.stdout:4/543: dwrite d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:09.853 INFO:tasks.workunit.client.0.smithi117.stdout:0/611: dwrite d3/d7/d16/f2b [0,4194304] 0 2024-04-20T11:09:09.867 INFO:tasks.workunit.client.0.smithi117.stdout:4/544: fsync d1/dd/d1a/d2b/f8f 0 2024-04-20T11:09:09.868 INFO:tasks.workunit.client.0.smithi117.stdout:8/466: mkdir d0/d8/d11/d6d/d69/d9c 0 2024-04-20T11:09:09.869 INFO:tasks.workunit.client.0.smithi117.stdout:0/612: rename d3/d7/f19 to d3/d7e/fc7 0 2024-04-20T11:09:09.873 INFO:tasks.workunit.client.0.smithi117.stdout:2/639: symlink d1/d4/dd/d17/d7a/d49/lce 0 2024-04-20T11:09:09.881 INFO:tasks.workunit.client.0.smithi117.stdout:4/545: creat d1/dd/d1a/d43/d9a/fb9 x:0 0 0 2024-04-20T11:09:09.882 INFO:tasks.workunit.client.0.smithi117.stdout:3/538: dread d0/d5/de/d4e/f53 [0,4194304] 0 2024-04-20T11:09:09.885 INFO:tasks.workunit.client.0.smithi117.stdout:3/539: stat d0/d5/de/d4e 0 2024-04-20T11:09:09.886 INFO:tasks.workunit.client.0.smithi117.stdout:0/613: symlink d3/d7/d2c/d52/lc8 0 2024-04-20T11:09:09.887 INFO:tasks.workunit.client.0.smithi117.stdout:0/614: fsync d3/d7/f14 0 2024-04-20T11:09:09.889 INFO:tasks.workunit.client.0.smithi117.stdout:3/540: chown d0/d5/d21/d75/d89/l9f 174552 1 2024-04-20T11:09:09.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:09 smithi117 ceph-mon[15659]: pgmap v14: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 31 MiB/s rd, 163 MiB/s wr, 385 op/s 2024-04-20T11:09:09.890 INFO:tasks.workunit.client.0.smithi117.stdout:0/615: readlink d3/d7/d16/l8c 0 2024-04-20T11:09:09.893 INFO:tasks.workunit.client.0.smithi117.stdout:4/546: creat d1/d45/d4a/fba x:0 0 0 2024-04-20T11:09:09.894 INFO:tasks.workunit.client.0.smithi117.stdout:8/467: creat d0/d8/d11/d24/d27/d99/f9d x:0 0 0 2024-04-20T11:09:09.899 INFO:tasks.workunit.client.0.smithi117.stdout:8/468: write d0/d8/d39/d40/f6e [6072148,22713] 0 2024-04-20T11:09:09.901 INFO:tasks.workunit.client.0.smithi117.stdout:2/640: mknod d1/d4/dd/d17/d7a/d49/ccf 0 2024-04-20T11:09:09.906 INFO:tasks.workunit.client.0.smithi117.stdout:3/541: creat d0/d5/d21/d94/d9a/fba x:0 0 0 2024-04-20T11:09:09.907 INFO:tasks.workunit.client.0.smithi117.stdout:2/641: write d1/d4/d83/d29/daa/fb7 [119584,56530] 0 2024-04-20T11:09:09.908 INFO:tasks.workunit.client.0.smithi117.stdout:0/616: creat d3/d7/dc6/fc9 x:0 0 0 2024-04-20T11:09:09.910 INFO:tasks.workunit.client.0.smithi117.stdout:2/642: chown d1/d9b 0 1 2024-04-20T11:09:09.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:09 smithi173 ceph-mon[23800]: pgmap v14: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 31 MiB/s rd, 163 MiB/s wr, 385 op/s 2024-04-20T11:09:09.912 INFO:tasks.workunit.client.0.smithi117.stdout:3/542: truncate d0/d5/d21/f28 1866527 0 2024-04-20T11:09:09.913 INFO:tasks.workunit.client.0.smithi117.stdout:2/643: stat d1/d4/d83/d29/c5a 0 2024-04-20T11:09:09.914 INFO:tasks.workunit.client.0.smithi117.stdout:4/547: dwrite d1/fa7 [0,4194304] 0 2024-04-20T11:09:09.918 INFO:tasks.workunit.client.0.smithi117.stdout:2/644: read - d1/d9b/fb4 zero size 2024-04-20T11:09:09.919 INFO:tasks.workunit.client.0.smithi117.stdout:2/645: readlink d1/d4/dd/l94 0 2024-04-20T11:09:09.919 INFO:tasks.workunit.client.0.smithi117.stdout:3/543: fdatasync d0/d5/d11/f7e 0 2024-04-20T11:09:09.922 INFO:tasks.workunit.client.0.smithi117.stdout:0/617: link d3/d7/f3f d3/d7/d2c/d52/fca 0 2024-04-20T11:09:09.926 INFO:tasks.workunit.client.0.smithi117.stdout:4/548: mknod d1/dd/d1a/d2b/d86/cbb 0 2024-04-20T11:09:09.929 INFO:tasks.workunit.client.0.smithi117.stdout:5/769: write de/d41/d47/f5c [2732820,15538] 0 2024-04-20T11:09:09.934 INFO:tasks.workunit.client.0.smithi117.stdout:9/577: write d4/d4a/d5e/f75 [4747626,25491] 0 2024-04-20T11:09:09.940 INFO:tasks.workunit.client.0.smithi117.stdout:6/630: write d3/d4/d2e/d37/f47 [160328,28760] 0 2024-04-20T11:09:09.945 INFO:tasks.workunit.client.0.smithi117.stdout:8/469: dread d0/d9/f95 [0,4194304] 0 2024-04-20T11:09:09.945 INFO:tasks.workunit.client.0.smithi117.stdout:2/646: link d1/d4/d83/d29/d3f/f76 d1/da5/fd0 0 2024-04-20T11:09:09.952 INFO:tasks.workunit.client.0.smithi117.stdout:9/578: creat d4/d4a/d63/fc3 x:0 0 0 2024-04-20T11:09:09.953 INFO:tasks.workunit.client.0.smithi117.stdout:6/631: rmdir d3/d4/d2e/d37/d8e/db2 39 2024-04-20T11:09:09.953 INFO:tasks.workunit.client.0.smithi117.stdout:8/470: write d0/d8/d11/d24/d2b/f5d [3701422,109376] 0 2024-04-20T11:09:09.955 INFO:tasks.workunit.client.0.smithi117.stdout:8/471: stat d0/f83 0 2024-04-20T11:09:09.955 INFO:tasks.workunit.client.0.smithi117.stdout:0/618: dwrite d3/d7/dc6/fc9 [0,4194304] 0 2024-04-20T11:09:09.958 INFO:tasks.workunit.client.0.smithi117.stdout:4/549: rmdir d1/dd/da2 0 2024-04-20T11:09:09.958 INFO:tasks.workunit.client.0.smithi117.stdout:9/579: write d4/de/d10/f9d [1190695,82226] 0 2024-04-20T11:09:09.959 INFO:tasks.workunit.client.0.smithi117.stdout:8/472: chown d0/d8/d11/d24/d27/d99 1315968 1 2024-04-20T11:09:09.961 INFO:tasks.workunit.client.0.smithi117.stdout:3/544: dread d0/d5/de/f92 [0,4194304] 0 2024-04-20T11:09:09.971 INFO:tasks.workunit.client.0.smithi117.stdout:4/550: rename d1/f30 to d1/d45/d4a/d59/fbc 0 2024-04-20T11:09:09.971 INFO:tasks.workunit.client.0.smithi117.stdout:8/473: symlink d0/d8/d11/d6d/l9e 0 2024-04-20T11:09:09.972 INFO:tasks.workunit.client.0.smithi117.stdout:4/551: readlink d1/dd/d52/l8a 0 2024-04-20T11:09:09.974 INFO:tasks.workunit.client.0.smithi117.stdout:6/632: rmdir d3/d4/d2e/d3f/d9f 0 2024-04-20T11:09:09.976 INFO:tasks.workunit.client.0.smithi117.stdout:4/552: unlink d1/d45/d4a/d59/f7f 0 2024-04-20T11:09:09.979 INFO:tasks.workunit.client.0.smithi117.stdout:8/474: creat d0/d8/d39/d7f/f9f x:0 0 0 2024-04-20T11:09:09.979 INFO:tasks.workunit.client.0.smithi117.stdout:5/770: write de/d41/f52 [167751,82899] 0 2024-04-20T11:09:09.981 INFO:tasks.workunit.client.0.smithi117.stdout:8/475: readlink d0/dc/l4d 0 2024-04-20T11:09:09.982 INFO:tasks.workunit.client.0.smithi117.stdout:6/633: creat d3/df/d39/d99/fcd x:0 0 0 2024-04-20T11:09:09.982 INFO:tasks.workunit.client.0.smithi117.stdout:9/580: dwrite d4/f39 [0,4194304] 0 2024-04-20T11:09:09.985 INFO:tasks.workunit.client.0.smithi117.stdout:5/771: mknod de/d41/d4b/df9/d77/c106 0 2024-04-20T11:09:09.987 INFO:tasks.workunit.client.0.smithi117.stdout:9/581: symlink d4/d30/lc4 0 2024-04-20T11:09:09.988 INFO:tasks.workunit.client.0.smithi117.stdout:6/634: creat d3/df/fce x:0 0 0 2024-04-20T11:09:09.988 INFO:tasks.workunit.client.0.smithi117.stdout:8/476: mknod d0/d8/d11/d24/d86/ca0 0 2024-04-20T11:09:09.989 INFO:tasks.workunit.client.0.smithi117.stdout:6/635: readlink d3/d4/l16 0 2024-04-20T11:09:09.991 INFO:tasks.workunit.client.0.smithi117.stdout:4/553: rename d1/d45/d4a/laa to d1/dd/d1a/lbd 0 2024-04-20T11:09:09.992 INFO:tasks.workunit.client.0.smithi117.stdout:6/636: chown d3/d3a/f61 5613 1 2024-04-20T11:09:09.992 INFO:tasks.workunit.client.0.smithi117.stdout:5/772: mknod de/d41/d4b/df9/db9/c107 0 2024-04-20T11:09:09.993 INFO:tasks.workunit.client.0.smithi117.stdout:9/582: mkdir d4/d5/d57/d82/da5/dc5 0 2024-04-20T11:09:09.995 INFO:tasks.workunit.client.0.smithi117.stdout:9/583: dread - d4/de/d7b/fbd zero size 2024-04-20T11:09:09.998 INFO:tasks.workunit.client.0.smithi117.stdout:6/637: creat d3/d4/d2e/d4e/d71/fcf x:0 0 0 2024-04-20T11:09:09.999 INFO:tasks.workunit.client.0.smithi117.stdout:5/773: creat de/d21/d51/f108 x:0 0 0 2024-04-20T11:09:09.999 INFO:tasks.workunit.client.0.smithi117.stdout:4/554: creat d1/d9/d1f/d87/fbe x:0 0 0 2024-04-20T11:09:09.999 INFO:tasks.workunit.client.0.smithi117.stdout:9/584: mknod d4/de/d7b/d96/cc6 0 2024-04-20T11:09:10.004 INFO:tasks.workunit.client.0.smithi117.stdout:5/774: chown de/d41/d47/d4c/f89 547289581 1 2024-04-20T11:09:10.004 INFO:tasks.workunit.client.0.smithi117.stdout:9/585: write d4/d5/d57/d82/da5/fbe [178222,110797] 0 2024-04-20T11:09:10.004 INFO:tasks.workunit.client.0.smithi117.stdout:6/638: mkdir d3/d5a/d74/db3/dd0 0 2024-04-20T11:09:10.005 INFO:tasks.workunit.client.0.smithi117.stdout:6/639: fsync d3/f89 0 2024-04-20T11:09:10.009 INFO:tasks.workunit.client.0.smithi117.stdout:4/555: fdatasync d1/d45/d4a/f6c 0 2024-04-20T11:09:10.011 INFO:tasks.workunit.client.0.smithi117.stdout:9/586: creat d4/d4a/d63/d98/dc0/fc7 x:0 0 0 2024-04-20T11:09:10.020 INFO:tasks.workunit.client.0.smithi117.stdout:6/640: rename d3/d5a/d74/db3/dbc/fa7 to d3/df/d39/daa/fd1 0 2024-04-20T11:09:10.020 INFO:tasks.workunit.client.0.smithi117.stdout:9/587: symlink d4/d4a/d72/lc8 0 2024-04-20T11:09:10.023 INFO:tasks.workunit.client.0.smithi117.stdout:8/477: write d0/d9/f33 [1322167,10442] 0 2024-04-20T11:09:10.028 INFO:tasks.workunit.client.0.smithi117.stdout:6/641: write d3/d4/d2e/d4e/d71/db7/fa8 [364574,124377] 0 2024-04-20T11:09:10.029 INFO:tasks.workunit.client.0.smithi117.stdout:9/588: mknod d4/d5/d57/d82/cc9 0 2024-04-20T11:09:10.034 INFO:tasks.workunit.client.0.smithi117.stdout:6/642: write d3/d4/d2e/f40 [1783782,64480] 0 2024-04-20T11:09:10.036 INFO:tasks.workunit.client.0.smithi117.stdout:9/589: creat d4/d5/d57/fca x:0 0 0 2024-04-20T11:09:10.036 INFO:tasks.workunit.client.0.smithi117.stdout:4/556: dwrite d1/d9/f66 [0,4194304] 0 2024-04-20T11:09:10.040 INFO:tasks.workunit.client.0.smithi117.stdout:5/775: dwrite de/d21/df5/d57/dd5/fc4 [0,4194304] 0 2024-04-20T11:09:10.043 INFO:tasks.workunit.client.0.smithi117.stdout:4/557: fsync d1/f20 0 2024-04-20T11:09:10.047 INFO:tasks.workunit.client.0.smithi117.stdout:9/590: mkdir d4/d4a/d63/d98/dc0/dcb 0 2024-04-20T11:09:10.048 INFO:tasks.workunit.client.0.smithi117.stdout:5/776: creat de/db7/f109 x:0 0 0 2024-04-20T11:09:10.049 INFO:tasks.workunit.client.0.smithi117.stdout:8/478: dread d0/f20 [0,4194304] 0 2024-04-20T11:09:10.050 INFO:tasks.workunit.client.0.smithi117.stdout:4/558: symlink d1/d9/d1f/d90/lbf 0 2024-04-20T11:09:10.051 INFO:tasks.workunit.client.0.smithi117.stdout:8/479: readlink d0/d8/l2f 0 2024-04-20T11:09:10.052 INFO:tasks.workunit.client.0.smithi117.stdout:6/643: creat d3/d4/d2e/d37/d8e/fd2 x:0 0 0 2024-04-20T11:09:10.052 INFO:tasks.workunit.client.0.smithi117.stdout:4/559: stat d1/dd/d52/l8a 0 2024-04-20T11:09:10.053 INFO:tasks.workunit.client.0.smithi117.stdout:6/644: fdatasync d3/d4/d2e/d3f/fc1 0 2024-04-20T11:09:10.054 INFO:tasks.workunit.client.0.smithi117.stdout:4/560: chown d1/d9/f66 1196267 1 2024-04-20T11:09:10.056 INFO:tasks.workunit.client.0.smithi117.stdout:9/591: creat d4/d4a/d74/fcc x:0 0 0 2024-04-20T11:09:10.058 INFO:tasks.workunit.client.0.smithi117.stdout:9/592: write d4/de/d7b/d96/db0/f77 [688482,20500] 0 2024-04-20T11:09:10.062 INFO:tasks.workunit.client.0.smithi117.stdout:6/645: mknod d3/d5a/d74/cd3 0 2024-04-20T11:09:10.064 INFO:tasks.workunit.client.0.smithi117.stdout:5/777: creat de/d41/d4b/df9/dfd/f10a x:0 0 0 2024-04-20T11:09:10.067 INFO:tasks.workunit.client.0.smithi117.stdout:6/646: creat d3/d4/d2e/d4e/fd4 x:0 0 0 2024-04-20T11:09:10.069 INFO:tasks.workunit.client.0.smithi117.stdout:5/778: truncate de/d41/d47/d55/d56/fac 304428 0 2024-04-20T11:09:10.071 INFO:tasks.workunit.client.0.smithi117.stdout:9/593: dread d4/de/f90 [0,4194304] 0 2024-04-20T11:09:10.075 INFO:tasks.workunit.client.0.smithi117.stdout:5/779: creat de/d21/df5/f10b x:0 0 0 2024-04-20T11:09:10.075 INFO:tasks.workunit.client.0.smithi117.stdout:9/594: write d4/d4a/d63/fc3 [403815,7395] 0 2024-04-20T11:09:10.078 INFO:tasks.workunit.client.0.smithi117.stdout:5/780: mkdir de/dd0/d10c 0 2024-04-20T11:09:10.080 INFO:tasks.workunit.client.0.smithi117.stdout:9/595: write d4/d5/d57/d82/da5/fbe [1034996,52390] 0 2024-04-20T11:09:10.081 INFO:tasks.workunit.client.0.smithi117.stdout:5/781: stat de/d41/d47/d55/d56/dc8/ff3 0 2024-04-20T11:09:10.081 INFO:tasks.workunit.client.0.smithi117.stdout:4/561: write d1/d9/f3b [562525,11259] 0 2024-04-20T11:09:10.086 INFO:tasks.workunit.client.0.smithi117.stdout:5/782: write de/db7/fbd [1697049,35364] 0 2024-04-20T11:09:10.089 INFO:tasks.workunit.client.0.smithi117.stdout:9/596: mkdir d4/d4a/d5e/d9b/dcd 0 2024-04-20T11:09:10.090 INFO:tasks.workunit.client.0.smithi117.stdout:4/562: write d1/dd/d1a/f1e [3538036,122910] 0 2024-04-20T11:09:10.090 INFO:tasks.workunit.client.0.smithi117.stdout:9/597: readlink d4/de/d10/l67 0 2024-04-20T11:09:10.091 INFO:tasks.workunit.client.0.smithi117.stdout:5/783: stat de/d41/d4b/df9/d48/l71 0 2024-04-20T11:09:10.094 INFO:tasks.workunit.client.0.smithi117.stdout:9/598: chown d4/d5/d57/cb7 86 1 2024-04-20T11:09:10.095 INFO:tasks.workunit.client.0.smithi117.stdout:5/784: mkdir de/d21/d10d 0 2024-04-20T11:09:10.096 INFO:tasks.workunit.client.0.smithi117.stdout:6/647: dread d3/d3a/f6b [0,4194304] 0 2024-04-20T11:09:10.102 INFO:tasks.workunit.client.0.smithi117.stdout:5/785: write de/d21/f96 [570295,127877] 0 2024-04-20T11:09:10.105 INFO:tasks.workunit.client.0.smithi117.stdout:9/599: creat d4/d4a/d5e/d9b/dcd/fce x:0 0 0 2024-04-20T11:09:10.107 INFO:tasks.workunit.client.0.smithi117.stdout:6/648: link d3/d5a/d74/db3/dbc/d9b/fa0 d3/df/d39/fd5 0 2024-04-20T11:09:10.108 INFO:tasks.workunit.client.0.smithi117.stdout:6/649: read - d3/df/d39/fd5 zero size 2024-04-20T11:09:10.111 INFO:tasks.workunit.client.0.smithi117.stdout:9/600: readlink d4/d35/l3b 0 2024-04-20T11:09:10.112 INFO:tasks.workunit.client.0.smithi117.stdout:9/601: read d4/f13 [490901,11174] 0 2024-04-20T11:09:10.113 INFO:tasks.workunit.client.0.smithi117.stdout:4/563: dwrite d1/dd/d1a/d43/d9a/fb9 [0,4194304] 0 2024-04-20T11:09:10.114 INFO:tasks.workunit.client.0.smithi117.stdout:6/650: write d3/df/f21 [1239041,8231] 0 2024-04-20T11:09:10.124 INFO:tasks.workunit.client.0.smithi117.stdout:5/786: dwrite de/d41/f52 [0,4194304] 0 2024-04-20T11:09:10.129 INFO:tasks.workunit.client.0.smithi117.stdout:4/564: rename d1/dd/d1a/d43/d4e/f73 to d1/d45/d4a/d99/fc0 0 2024-04-20T11:09:10.132 INFO:tasks.workunit.client.0.smithi117.stdout:6/651: creat d3/d3a/fd6 x:0 0 0 2024-04-20T11:09:10.135 INFO:tasks.workunit.client.0.smithi117.stdout:5/787: rename de/d41/d4b/df9/d48 to de/dd0/d10e 0 2024-04-20T11:09:10.138 INFO:tasks.workunit.client.0.smithi117.stdout:5/788: fsync de/d41/d4b/df9/dfd/f10a 0 2024-04-20T11:09:10.142 INFO:tasks.workunit.client.0.smithi117.stdout:5/789: unlink de/cf 0 2024-04-20T11:09:10.143 INFO:tasks.workunit.client.0.smithi117.stdout:9/602: truncate d4/d35/f5f 2788019 0 2024-04-20T11:09:10.148 INFO:tasks.workunit.client.0.smithi117.stdout:5/790: truncate de/dd0/d10e/da7/feb 4608172 0 2024-04-20T11:09:10.149 INFO:tasks.workunit.client.0.smithi117.stdout:9/603: unlink d4/d4a/d63/d98/dc0/fc7 0 2024-04-20T11:09:10.153 INFO:tasks.workunit.client.0.smithi117.stdout:5/791: creat de/d5b/f10f x:0 0 0 2024-04-20T11:09:10.158 INFO:tasks.workunit.client.0.smithi117.stdout:9/604: write d4/d5/da/fc [1571540,116471] 0 2024-04-20T11:09:10.159 INFO:tasks.workunit.client.0.smithi117.stdout:4/565: dwrite d1/dd/d1a/d2b/d86/f91 [0,4194304] 0 2024-04-20T11:09:10.161 INFO:tasks.workunit.client.0.smithi117.stdout:5/792: creat de/d21/df5/d57/dd5/dce/f110 x:0 0 0 2024-04-20T11:09:10.165 INFO:tasks.workunit.client.0.smithi117.stdout:9/605: fsync d4/d5/da/f93 0 2024-04-20T11:09:10.168 INFO:tasks.workunit.client.0.smithi117.stdout:5/793: creat de/d21/df5/d57/dd5/dce/f111 x:0 0 0 2024-04-20T11:09:10.168 INFO:tasks.workunit.client.0.smithi117.stdout:6/652: dread d3/d3a/f4d [0,4194304] 0 2024-04-20T11:09:10.171 INFO:tasks.workunit.client.0.smithi117.stdout:5/794: read - de/d16/d9d/db4/f102 zero size 2024-04-20T11:09:10.172 INFO:tasks.workunit.client.0.smithi117.stdout:6/653: write d3/d4/f6e [1062677,103920] 0 2024-04-20T11:09:10.175 INFO:tasks.workunit.client.0.smithi117.stdout:9/606: mknod d4/d5/d57/d82/da5/ccf 0 2024-04-20T11:09:10.179 INFO:tasks.workunit.client.0.smithi117.stdout:5/795: creat de/d41/d47/d55/dab/f112 x:0 0 0 2024-04-20T11:09:10.181 INFO:tasks.workunit.client.0.smithi117.stdout:5/796: creat de/d16/f113 x:0 0 0 2024-04-20T11:09:10.187 INFO:tasks.workunit.client.0.smithi117.stdout:5/797: creat de/d21/df5/d57/dd5/f114 x:0 0 0 2024-04-20T11:09:10.191 INFO:tasks.workunit.client.0.smithi117.stdout:5/798: mknod de/dd0/d10e/da7/c115 0 2024-04-20T11:09:10.193 INFO:tasks.workunit.client.0.smithi117.stdout:4/566: truncate d1/d9/f96 2788292 0 2024-04-20T11:09:10.195 INFO:tasks.workunit.client.0.smithi117.stdout:5/799: fdatasync de/d21/f31 0 2024-04-20T11:09:10.196 INFO:tasks.workunit.client.0.smithi117.stdout:4/567: dread - d1/dd/d1a/d43/d9a/fb2 zero size 2024-04-20T11:09:10.198 INFO:tasks.workunit.client.0.smithi117.stdout:5/800: read - de/d5b/d65/fd6 zero size 2024-04-20T11:09:10.199 INFO:tasks.workunit.client.0.smithi117.stdout:6/654: write d3/df/f31 [669940,84644] 0 2024-04-20T11:09:10.200 INFO:tasks.workunit.client.0.smithi117.stdout:4/568: readlink d1/dd/d1a/lbd 0 2024-04-20T11:09:10.203 INFO:tasks.workunit.client.0.smithi117.stdout:4/569: dread - d1/dd/d1a/d2b/d86/fb4 zero size 2024-04-20T11:09:10.203 INFO:tasks.workunit.client.0.smithi117.stdout:4/570: chown d1/dd/d1a 1931 1 2024-04-20T11:09:10.205 INFO:tasks.workunit.client.0.smithi117.stdout:5/801: fsync de/d21/df5/fe1 0 2024-04-20T11:09:10.206 INFO:tasks.workunit.client.0.smithi117.stdout:6/655: symlink d3/d4/d2e/d4e/d71/da3/ld7 0 2024-04-20T11:09:10.207 INFO:tasks.workunit.client.0.smithi117.stdout:5/802: readlink de/d16/l38 0 2024-04-20T11:09:10.229 INFO:tasks.workunit.client.0.smithi117.stdout:4/571: dwrite d1/d45/d4a/d99/fab [4194304,4194304] 0 2024-04-20T11:09:10.229 INFO:tasks.workunit.client.0.smithi117.stdout:5/803: dwrite de/d5b/fae [0,4194304] 0 2024-04-20T11:09:10.235 INFO:tasks.workunit.client.0.smithi117.stdout:5/804: mknod de/dd0/d10e/c116 0 2024-04-20T11:09:10.238 INFO:tasks.workunit.client.0.smithi117.stdout:5/805: fsync de/d21/df5/fe1 0 2024-04-20T11:09:10.238 INFO:tasks.workunit.client.0.smithi117.stdout:4/572: link d1/d9/d6b/c85 d1/cc1 0 2024-04-20T11:09:10.245 INFO:tasks.workunit.client.0.smithi117.stdout:4/573: creat d1/dd/d1a/d43/d9a/fc2 x:0 0 0 2024-04-20T11:09:10.245 INFO:tasks.workunit.client.0.smithi117.stdout:6/656: dread d3/df/f14 [4194304,4194304] 0 2024-04-20T11:09:10.250 INFO:tasks.workunit.client.0.smithi117.stdout:4/574: creat d1/fc3 x:0 0 0 2024-04-20T11:09:10.254 INFO:tasks.workunit.client.0.smithi117.stdout:4/575: symlink d1/dd/lc4 0 2024-04-20T11:09:10.260 INFO:tasks.workunit.client.0.smithi117.stdout:5/806: dwrite de/d21/df5/d57/dd5/f114 [0,4194304] 0 2024-04-20T11:09:10.262 INFO:tasks.workunit.client.0.smithi117.stdout:6/657: dread d3/df/f14 [4194304,4194304] 0 2024-04-20T11:09:10.267 INFO:tasks.workunit.client.0.smithi117.stdout:6/658: symlink d3/d50/ld8 0 2024-04-20T11:09:10.269 INFO:tasks.workunit.client.0.smithi117.stdout:6/659: write d3/d4/f6e [993601,12565] 0 2024-04-20T11:09:10.270 INFO:tasks.workunit.client.0.smithi117.stdout:4/576: truncate d1/d9/f17 2635804 0 2024-04-20T11:09:10.274 INFO:tasks.workunit.client.0.smithi117.stdout:6/660: rmdir d3/d5a 39 2024-04-20T11:09:10.280 INFO:tasks.workunit.client.0.smithi117.stdout:4/577: link d1/dd/d1a/d2b/c97 d1/d9/d79/cc5 0 2024-04-20T11:09:10.284 INFO:tasks.workunit.client.0.smithi117.stdout:5/807: dwrite de/d5b/d65/fd6 [0,4194304] 0 2024-04-20T11:09:10.284 INFO:tasks.workunit.client.0.smithi117.stdout:4/578: symlink d1/d9/d1f/d90/lc6 0 2024-04-20T11:09:10.286 INFO:tasks.workunit.client.0.smithi117.stdout:4/579: chown d1/dd/d1a/d2b/l56 2 1 2024-04-20T11:09:10.287 INFO:tasks.workunit.client.0.smithi117.stdout:4/580: chown d1/f81 20793 1 2024-04-20T11:09:10.289 INFO:tasks.workunit.client.0.smithi117.stdout:5/808: creat de/dd0/f117 x:0 0 0 2024-04-20T11:09:10.291 INFO:tasks.workunit.client.0.smithi117.stdout:5/809: chown de/d41/d4b/df9/db9/dda 7 1 2024-04-20T11:09:10.292 INFO:tasks.workunit.client.0.smithi117.stdout:2/647: sync 2024-04-20T11:09:10.292 INFO:tasks.workunit.client.0.smithi117.stdout:1/476: sync 2024-04-20T11:09:10.292 INFO:tasks.workunit.client.0.smithi117.stdout:0/619: sync 2024-04-20T11:09:10.294 INFO:tasks.workunit.client.0.smithi117.stdout:0/620: write d3/dab/fac [39236,126547] 0 2024-04-20T11:09:10.296 INFO:tasks.workunit.client.0.smithi117.stdout:0/621: write d3/d7/d16/f2b [1515336,89474] 0 2024-04-20T11:09:10.302 INFO:tasks.workunit.client.0.smithi117.stdout:2/648: unlink d1/f48 0 2024-04-20T11:09:10.306 INFO:tasks.workunit.client.0.smithi117.stdout:1/477: rmdir d6/d16/d3a/d5a/d5c/d61 39 2024-04-20T11:09:10.306 INFO:tasks.workunit.client.0.smithi117.stdout:4/581: link d1/dd/d1a/d43/c8d d1/d9/d1f/d87/cc7 0 2024-04-20T11:09:10.311 INFO:tasks.workunit.client.0.smithi117.stdout:4/582: rename d1/dd/d1a/c32 to d1/d45/d4a/cc8 0 2024-04-20T11:09:10.313 INFO:tasks.workunit.client.0.smithi117.stdout:5/810: creat de/d41/d47/f118 x:0 0 0 2024-04-20T11:09:10.317 INFO:tasks.workunit.client.0.smithi117.stdout:4/583: symlink d1/d45/d4a/lc9 0 2024-04-20T11:09:10.319 INFO:tasks.workunit.client.0.smithi117.stdout:5/811: write de/d41/d4b/df9/d77/fa6 [2131613,16443] 0 2024-04-20T11:09:10.320 INFO:tasks.workunit.client.0.smithi117.stdout:0/622: write d3/d7/f18 [2025624,95753] 0 2024-04-20T11:09:10.323 INFO:tasks.workunit.client.0.smithi117.stdout:5/812: write de/fe4 [175961,110770] 0 2024-04-20T11:09:10.326 INFO:tasks.workunit.client.0.smithi117.stdout:0/623: write d3/d7/d20/d47/d78/f8d [5134374,83296] 0 2024-04-20T11:09:10.328 INFO:tasks.workunit.client.0.smithi117.stdout:5/813: stat de/d41/d47/d55/d56/f79 0 2024-04-20T11:09:10.333 INFO:tasks.workunit.client.0.smithi117.stdout:4/584: truncate d1/d9/d1f/d87/fb3 3258360 0 2024-04-20T11:09:10.335 INFO:tasks.workunit.client.0.smithi117.stdout:2/649: dwrite d1/d4/d83/d29/d2e/f74 [0,4194304] 0 2024-04-20T11:09:10.340 INFO:tasks.workunit.client.0.smithi117.stdout:4/585: mkdir d1/d9/d6b/dca 0 2024-04-20T11:09:10.341 INFO:tasks.workunit.client.0.smithi117.stdout:1/478: dread d6/d16/d3a/d5a/f53 [0,4194304] 0 2024-04-20T11:09:10.341 INFO:tasks.workunit.client.0.smithi117.stdout:4/586: stat d1/d9/d6b/c6f 0 2024-04-20T11:09:10.342 INFO:tasks.workunit.client.0.smithi117.stdout:5/814: rename de/d41/d47/d4c/c7a to de/d21/df5/d57/dd5/c119 0 2024-04-20T11:09:10.343 INFO:tasks.workunit.client.0.smithi117.stdout:1/479: write d6/f68 [8095851,28550] 0 2024-04-20T11:09:10.349 INFO:tasks.workunit.client.0.smithi117.stdout:0/624: symlink d3/d7/d20/d39/lcb 0 2024-04-20T11:09:10.354 INFO:tasks.workunit.client.0.smithi117.stdout:4/587: symlink d1/d45/lcb 0 2024-04-20T11:09:10.360 INFO:tasks.workunit.client.0.smithi117.stdout:4/588: mkdir d1/dd/d1a/d43/dcc 0 2024-04-20T11:09:10.361 INFO:tasks.workunit.client.0.smithi117.stdout:4/589: readlink d1/lb 0 2024-04-20T11:09:10.361 INFO:tasks.workunit.client.0.smithi117.stdout:1/480: mkdir d6/d16/d3a/d5a/d5c/d61/d98 0 2024-04-20T11:09:10.365 INFO:tasks.workunit.client.0.smithi117.stdout:4/590: truncate d1/f20 1318674 0 2024-04-20T11:09:10.368 INFO:tasks.workunit.client.0.smithi117.stdout:1/481: mknod d6/d1f/d30/d49/d64/c99 0 2024-04-20T11:09:10.375 INFO:tasks.workunit.client.0.smithi117.stdout:0/625: truncate d3/d7/d20/d47/d78/f8d 698823 0 2024-04-20T11:09:10.375 INFO:tasks.workunit.client.0.smithi117.stdout:4/591: fsync d1/d45/d4a/d59/fbc 0 2024-04-20T11:09:10.380 INFO:tasks.workunit.client.0.smithi117.stdout:2/650: dread d1/d4/dd/d17/f1e [0,4194304] 0 2024-04-20T11:09:10.380 INFO:tasks.workunit.client.0.smithi117.stdout:0/626: unlink d3/d7/d2c/d52/fca 0 2024-04-20T11:09:10.382 INFO:tasks.workunit.client.0.smithi117.stdout:4/592: chown d1/d45/d4a/c69 9 1 2024-04-20T11:09:10.384 INFO:tasks.workunit.client.0.smithi117.stdout:1/482: write d6/f3c [4678398,50755] 0 2024-04-20T11:09:10.386 INFO:tasks.workunit.client.0.smithi117.stdout:2/651: mknod d1/d4/d83/d29/d62/cd1 0 2024-04-20T11:09:10.388 INFO:tasks.workunit.client.0.smithi117.stdout:4/593: mkdir d1/d45/dcd 0 2024-04-20T11:09:10.391 INFO:tasks.workunit.client.0.smithi117.stdout:0/627: creat d3/d7/d20/fcc x:0 0 0 2024-04-20T11:09:10.392 INFO:tasks.workunit.client.0.smithi117.stdout:2/652: mknod d1/d20/cd2 0 2024-04-20T11:09:10.394 INFO:tasks.workunit.client.0.smithi117.stdout:4/594: creat d1/d45/d4a/d59/fce x:0 0 0 2024-04-20T11:09:10.396 INFO:tasks.workunit.client.0.smithi117.stdout:4/595: fdatasync d1/d45/d4a/d59/fce 0 2024-04-20T11:09:10.400 INFO:tasks.workunit.client.0.smithi117.stdout:4/596: mknod d1/d45/d4a/ccf 0 2024-04-20T11:09:10.405 INFO:tasks.workunit.client.0.smithi117.stdout:1/483: link d6/d16/d3a/d5a/c59 d6/d16/d3a/d5a/d5c/d61/c9a 0 2024-04-20T11:09:10.407 INFO:tasks.workunit.client.0.smithi117.stdout:0/628: getdents d3/d7/d20/d39/d3a 0 2024-04-20T11:09:10.409 INFO:tasks.workunit.client.0.smithi117.stdout:0/629: fsync d3/d7/d2c/d52/fb7 0 2024-04-20T11:09:10.411 INFO:tasks.workunit.client.0.smithi117.stdout:2/653: rename d1/d4/dd/d17/d7a/f95 to d1/d20/fd3 0 2024-04-20T11:09:10.413 INFO:tasks.workunit.client.0.smithi117.stdout:1/484: mknod d6/d1f/d30/d49/d64/c9b 0 2024-04-20T11:09:10.420 INFO:tasks.workunit.client.0.smithi117.stdout:2/654: rename d1/d4/d83/d29/d2e/l3c to d1/d4/dd/d17/d7a/ld4 0 2024-04-20T11:09:10.423 INFO:tasks.workunit.client.0.smithi117.stdout:2/655: write d1/d4/d83/d29/daa/fb7 [10527,80080] 0 2024-04-20T11:09:10.426 INFO:tasks.workunit.client.0.smithi117.stdout:0/630: truncate d3/d7/d8/f31 297706 0 2024-04-20T11:09:10.430 INFO:tasks.workunit.client.0.smithi117.stdout:2/656: symlink d1/d4/dd/d17/d7a/d49/ld5 0 2024-04-20T11:09:10.434 INFO:tasks.workunit.client.0.smithi117.stdout:4/597: dwrite d1/dd/f38 [0,4194304] 0 2024-04-20T11:09:10.435 INFO:tasks.workunit.client.0.smithi117.stdout:2/657: write d1/d4/dd/f19 [2668187,67898] 0 2024-04-20T11:09:10.435 INFO:tasks.workunit.client.0.smithi117.stdout:1/485: dwrite d6/f12 [0,4194304] 0 2024-04-20T11:09:10.439 INFO:tasks.workunit.client.0.smithi117.stdout:2/658: chown d1/d4/dd/f14 3391 1 2024-04-20T11:09:10.444 INFO:tasks.workunit.client.0.smithi117.stdout:1/486: fdatasync d6/f18 0 2024-04-20T11:09:10.446 INFO:tasks.workunit.client.0.smithi117.stdout:2/659: fsync d1/d4/dd/fc1 0 2024-04-20T11:09:10.454 INFO:tasks.workunit.client.0.smithi117.stdout:0/631: dwrite d3/d7/d8/df/d6d/f9b [4194304,4194304] 0 2024-04-20T11:09:10.455 INFO:tasks.workunit.client.0.smithi117.stdout:2/660: rmdir d1/d4/d83/d29 39 2024-04-20T11:09:10.460 INFO:tasks.workunit.client.0.smithi117.stdout:1/487: link d6/lc d6/d35/l9c 0 2024-04-20T11:09:10.467 INFO:tasks.workunit.client.0.smithi117.stdout:7/569: sync 2024-04-20T11:09:10.467 INFO:tasks.workunit.client.0.smithi117.stdout:3/545: sync 2024-04-20T11:09:10.467 INFO:tasks.workunit.client.0.smithi117.stdout:0/632: write d3/d7/d20/f68 [461682,51072] 0 2024-04-20T11:09:10.470 INFO:tasks.workunit.client.0.smithi117.stdout:3/546: fsync d0/f2c 0 2024-04-20T11:09:10.471 INFO:tasks.workunit.client.0.smithi117.stdout:7/570: chown d1/d43/d59/l75 7108 1 2024-04-20T11:09:10.474 INFO:tasks.workunit.client.0.smithi117.stdout:3/547: chown d0/d5/d21/f42 252023 1 2024-04-20T11:09:10.479 INFO:tasks.workunit.client.0.smithi117.stdout:0/633: unlink d3/d7/d16/l45 0 2024-04-20T11:09:10.484 INFO:tasks.workunit.client.0.smithi117.stdout:3/548: creat d0/d5/d21/d94/d9a/fbb x:0 0 0 2024-04-20T11:09:10.491 INFO:tasks.workunit.client.0.smithi117.stdout:2/661: dwrite d1/d4/d83/d29/d2e/d80/fb2 [0,4194304] 0 2024-04-20T11:09:10.492 INFO:tasks.workunit.client.0.smithi117.stdout:0/634: creat d3/d7/d8/df/d22/d85/fcd x:0 0 0 2024-04-20T11:09:10.498 INFO:tasks.workunit.client.0.smithi117.stdout:1/488: dread d6/d1f/f2b [0,4194304] 0 2024-04-20T11:09:10.508 INFO:tasks.workunit.client.0.smithi117.stdout:1/489: truncate d6/d1f/f45 8541178 0 2024-04-20T11:09:10.508 INFO:tasks.workunit.client.0.smithi117.stdout:7/571: dwrite d1/d43/f91 [0,4194304] 0 2024-04-20T11:09:10.512 INFO:tasks.workunit.client.0.smithi117.stdout:3/549: dread d0/d5/d21/d88/d29/d66/f6c [0,4194304] 0 2024-04-20T11:09:10.519 INFO:tasks.workunit.client.0.smithi117.stdout:1/490: fdatasync d6/d1f/d30/d49/d64/f8f 0 2024-04-20T11:09:10.523 INFO:tasks.workunit.client.0.smithi117.stdout:7/572: symlink d1/lb8 0 2024-04-20T11:09:10.525 INFO:tasks.workunit.client.0.smithi117.stdout:3/550: mkdir d0/d5/d21/d88/d29/db3/dbc 0 2024-04-20T11:09:10.526 INFO:tasks.workunit.client.0.smithi117.stdout:2/662: dwrite d1/d4/d83/fc3 [0,4194304] 0 2024-04-20T11:09:10.533 INFO:tasks.workunit.client.0.smithi117.stdout:0/635: dwrite d3/d7/f3f [0,4194304] 0 2024-04-20T11:09:10.534 INFO:tasks.workunit.client.0.smithi117.stdout:3/551: symlink d0/d5/d21/d88/d29/lbd 0 2024-04-20T11:09:10.537 INFO:tasks.workunit.client.0.smithi117.stdout:1/491: mknod d6/d35/c9d 0 2024-04-20T11:09:10.543 INFO:tasks.workunit.client.0.smithi117.stdout:1/492: fdatasync d6/d1f/d30/f43 0 2024-04-20T11:09:10.550 INFO:tasks.workunit.client.0.smithi117.stdout:1/493: symlink d6/d21/l9e 0 2024-04-20T11:09:10.553 INFO:tasks.workunit.client.0.smithi117.stdout:1/494: fdatasync d6/d1f/d30/d32/f89 0 2024-04-20T11:09:10.557 INFO:tasks.workunit.client.0.smithi117.stdout:3/552: dwrite d0/d5/de/d4e/f9e [4194304,4194304] 0 2024-04-20T11:09:10.564 INFO:tasks.workunit.client.0.smithi117.stdout:0/636: dwrite d3/d7e/fc7 [0,4194304] 0 2024-04-20T11:09:10.564 INFO:tasks.workunit.client.0.smithi117.stdout:0/637: chown d3/d7/dc6/fc9 70651299 1 2024-04-20T11:09:10.565 INFO:tasks.workunit.client.0.smithi117.stdout:1/495: symlink d6/d35/l9f 0 2024-04-20T11:09:10.571 INFO:tasks.workunit.client.0.smithi117.stdout:3/553: unlink d0/d5/d21/d88/d29/d66/d63/c7b 0 2024-04-20T11:09:10.575 INFO:tasks.workunit.client.0.smithi117.stdout:0/638: creat d3/d7/d20/d39/d3a/db1/fce x:0 0 0 2024-04-20T11:09:10.579 INFO:tasks.workunit.client.0.smithi117.stdout:0/639: read d3/d7/d20/f4c [1480323,128142] 0 2024-04-20T11:09:10.579 INFO:tasks.workunit.client.0.smithi117.stdout:7/573: dwrite d1/d4/d39/f4d [0,4194304] 0 2024-04-20T11:09:10.590 INFO:tasks.workunit.client.0.smithi117.stdout:1/496: link d6/d1f/d30/f67 d6/d16/d3a/d5a/d5c/fa0 0 2024-04-20T11:09:10.594 INFO:tasks.workunit.client.0.smithi117.stdout:0/640: creat d3/d7/d8/df/d6d/fcf x:0 0 0 2024-04-20T11:09:10.595 INFO:tasks.workunit.client.0.smithi117.stdout:8/480: sync 2024-04-20T11:09:10.598 INFO:tasks.workunit.client.0.smithi117.stdout:8/481: chown d0/d8/d39/d40/d78/d5c/c62 930 1 2024-04-20T11:09:10.598 INFO:tasks.workunit.client.0.smithi117.stdout:2/663: dread d1/d4/dd/d17/f30 [0,4194304] 0 2024-04-20T11:09:10.599 INFO:tasks.workunit.client.0.smithi117.stdout:1/497: mknod d6/d1f/d30/ca1 0 2024-04-20T11:09:10.606 INFO:tasks.workunit.client.0.smithi117.stdout:0/641: fdatasync d3/d7/d8/df/d6d/f71 0 2024-04-20T11:09:10.610 INFO:tasks.workunit.client.0.smithi117.stdout:0/642: read d3/d7/f4d [3617889,105943] 0 2024-04-20T11:09:10.610 INFO:tasks.workunit.client.0.smithi117.stdout:2/664: unlink d1/l8b 0 2024-04-20T11:09:10.615 INFO:tasks.workunit.client.0.smithi117.stdout:3/554: write d0/d5/d21/f4a [131786,95672] 0 2024-04-20T11:09:10.622 INFO:tasks.workunit.client.0.smithi117.stdout:0/643: rmdir d3/d7/d8/df/d22/d85 39 2024-04-20T11:09:10.626 INFO:tasks.workunit.client.0.smithi117.stdout:1/498: dread d6/d1f/d30/f67 [0,4194304] 0 2024-04-20T11:09:10.627 INFO:tasks.workunit.client.0.smithi117.stdout:2/665: dread d1/d4/d83/d29/d2e/f74 [0,4194304] 0 2024-04-20T11:09:10.628 INFO:tasks.workunit.client.0.smithi117.stdout:7/574: getdents d1/d4/d6/d54/d63 0 2024-04-20T11:09:10.630 INFO:tasks.workunit.client.0.smithi117.stdout:7/575: write d1/d1d/f5c [477416,43448] 0 2024-04-20T11:09:10.630 INFO:tasks.workunit.client.0.smithi117.stdout:3/555: mkdir d0/d5/d21/d88/d29/d66/dbe 0 2024-04-20T11:09:10.636 INFO:tasks.workunit.client.0.smithi117.stdout:0/644: unlink d3/d7/f9 0 2024-04-20T11:09:10.639 INFO:tasks.workunit.client.0.smithi117.stdout:1/499: rmdir d6/d1f/d30/d49/d64 39 2024-04-20T11:09:10.640 INFO:tasks.workunit.client.0.smithi117.stdout:8/482: write d0/f34 [993326,3220] 0 2024-04-20T11:09:10.643 INFO:tasks.workunit.client.0.smithi117.stdout:6/661: sync 2024-04-20T11:09:10.643 INFO:tasks.workunit.client.0.smithi117.stdout:5/815: sync 2024-04-20T11:09:10.643 INFO:tasks.workunit.client.0.smithi117.stdout:9/607: sync 2024-04-20T11:09:10.644 INFO:tasks.workunit.client.0.smithi117.stdout:4/598: sync 2024-04-20T11:09:10.648 INFO:tasks.workunit.client.0.smithi117.stdout:8/483: fdatasync d0/d8/d39/d40/f6e 0 2024-04-20T11:09:10.652 INFO:tasks.workunit.client.0.smithi117.stdout:3/556: creat d0/d5/d21/d94/d9a/fbf x:0 0 0 2024-04-20T11:09:10.653 INFO:tasks.workunit.client.0.smithi117.stdout:2/666: dread d1/d9b/fb6 [0,4194304] 0 2024-04-20T11:09:10.658 INFO:tasks.workunit.client.0.smithi117.stdout:0/645: rename d3/d7/d8/df to d3/d7/d20/d39/d9a/dd0 0 2024-04-20T11:09:10.658 INFO:tasks.workunit.client.0.smithi117.stdout:3/557: chown d0/d5/d21/d88/d29/f97 1473 1 2024-04-20T11:09:10.659 INFO:tasks.workunit.client.0.smithi117.stdout:2/667: read d1/d4/d83/d29/d2e/d80/fb2 [2310428,26773] 0 2024-04-20T11:09:10.665 INFO:tasks.workunit.client.0.smithi117.stdout:3/558: read d0/d5/d7/dd/f59 [4085846,9168] 0 2024-04-20T11:09:10.667 INFO:tasks.workunit.client.0.smithi117.stdout:3/559: chown d0/d5/d21/d75/d89 10 1 2024-04-20T11:09:10.670 INFO:tasks.workunit.client.0.smithi117.stdout:7/576: link d1/d4/f97 d1/d1d/d4e/fb9 0 2024-04-20T11:09:10.674 INFO:tasks.workunit.client.0.smithi117.stdout:4/599: creat d1/dd/d1a/d43/d4e/fd0 x:0 0 0 2024-04-20T11:09:10.675 INFO:tasks.workunit.client.0.smithi117.stdout:7/577: read d1/d4/d6/d17/f4c [2126978,95848] 0 2024-04-20T11:09:10.680 INFO:tasks.workunit.client.0.smithi117.stdout:2/668: write d1/f3 [5249436,67535] 0 2024-04-20T11:09:10.680 INFO:tasks.workunit.client.0.smithi117.stdout:7/578: fsync d1/d1d/d42/f8b 0 2024-04-20T11:09:10.680 INFO:tasks.workunit.client.0.smithi117.stdout:0/646: symlink d3/d7/d20/d39/d9a/dd0/db2/ld1 0 2024-04-20T11:09:10.684 INFO:tasks.workunit.client.0.smithi117.stdout:5/816: dwrite de/d41/d47/d55/d56/fac [0,4194304] 0 2024-04-20T11:09:10.687 INFO:tasks.workunit.client.0.smithi117.stdout:6/662: symlink d3/d4/ld9 0 2024-04-20T11:09:10.689 INFO:tasks.workunit.client.0.smithi117.stdout:7/579: chown d1/d4/d3d/c6a 3565 1 2024-04-20T11:09:10.691 INFO:tasks.workunit.client.0.smithi117.stdout:1/500: mkdir d6/d16/d3a/d5a/d5c/d61/d71/d7d/da2 0 2024-04-20T11:09:10.696 INFO:tasks.workunit.client.0.smithi117.stdout:8/484: creat d0/d9/d3a/d7c/fa1 x:0 0 0 2024-04-20T11:09:10.703 INFO:tasks.workunit.client.0.smithi117.stdout:8/485: read d0/dc/f25 [1880200,36867] 0 2024-04-20T11:09:10.704 INFO:tasks.workunit.client.0.smithi117.stdout:5/817: dwrite de/dd0/d10e/fad [0,4194304] 0 2024-04-20T11:09:10.708 INFO:tasks.workunit.client.0.smithi117.stdout:4/600: unlink d1/d9/d1f/d90/fb1 0 2024-04-20T11:09:10.715 INFO:tasks.workunit.client.0.smithi117.stdout:0/647: creat d3/d7/d20/d39/d9a/fd2 x:0 0 0 2024-04-20T11:09:10.721 INFO:tasks.workunit.client.0.smithi117.stdout:8/486: dwrite d0/dc/f1e [0,4194304] 0 2024-04-20T11:09:10.721 INFO:tasks.workunit.client.0.smithi117.stdout:2/669: mknod d1/d4/d83/d29/d2e/d53/cd6 0 2024-04-20T11:09:10.724 INFO:tasks.workunit.client.0.smithi117.stdout:2/670: write d1/d4/d83/d29/d2e/f64 [3521913,31299] 0 2024-04-20T11:09:10.726 INFO:tasks.workunit.client.0.smithi117.stdout:6/663: readlink d3/d4/l20 0 2024-04-20T11:09:10.731 INFO:tasks.workunit.client.0.smithi117.stdout:7/580: creat d1/d4/d3d/d93/fba x:0 0 0 2024-04-20T11:09:10.744 INFO:tasks.workunit.client.0.smithi117.stdout:4/601: mknod d1/dd/d1a/d43/d9a/cd1 0 2024-04-20T11:09:10.744 INFO:tasks.workunit.client.0.smithi117.stdout:5/818: rename de/fe4 to de/d5b/f11a 0 2024-04-20T11:09:10.746 INFO:tasks.workunit.client.0.smithi117.stdout:8/487: symlink d0/dc/d2e/la2 0 2024-04-20T11:09:10.752 INFO:tasks.workunit.client.0.smithi117.stdout:9/608: getdents d4/d4a 0 2024-04-20T11:09:10.754 INFO:tasks.workunit.client.0.smithi117.stdout:2/671: creat d1/d4/dd/fd7 x:0 0 0 2024-04-20T11:09:10.760 INFO:tasks.workunit.client.0.smithi117.stdout:0/648: dread d3/dab/d54/fa3 [0,4194304] 0 2024-04-20T11:09:10.764 INFO:tasks.workunit.client.0.smithi117.stdout:1/501: mknod d6/d16/d3a/d5a/d5c/d61/d98/ca3 0 2024-04-20T11:09:10.767 INFO:tasks.workunit.client.0.smithi117.stdout:8/488: symlink d0/dc/d49/la3 0 2024-04-20T11:09:10.767 INFO:tasks.workunit.client.0.smithi117.stdout:1/502: write d6/d1f/d30/d49/d69/d6e/d6f/d73/f88 [5067385,125791] 0 2024-04-20T11:09:10.775 INFO:tasks.workunit.client.0.smithi117.stdout:2/672: symlink d1/d15/d1d/ld8 0 2024-04-20T11:09:10.776 INFO:tasks.workunit.client.0.smithi117.stdout:0/649: creat d3/d7/d2c/d52/fd3 x:0 0 0 2024-04-20T11:09:10.784 INFO:tasks.workunit.client.0.smithi117.stdout:1/503: chown d6/c20 646847 1 2024-04-20T11:09:10.785 INFO:tasks.workunit.client.0.smithi117.stdout:8/489: mknod d0/dc/d49/ca4 0 2024-04-20T11:09:10.786 INFO:tasks.workunit.client.0.smithi117.stdout:8/490: chown d0 2187378 1 2024-04-20T11:09:10.790 INFO:tasks.workunit.client.0.smithi117.stdout:7/581: dread d1/d1d/d77/fa8 [0,4194304] 0 2024-04-20T11:09:10.793 INFO:tasks.workunit.client.0.smithi117.stdout:2/673: dwrite d1/d4/dd/f55 [0,4194304] 0 2024-04-20T11:09:10.797 INFO:tasks.workunit.client.0.smithi117.stdout:2/674: fsync d1/d4/d83/d29/d62/f8a 0 2024-04-20T11:09:10.801 INFO:tasks.workunit.client.0.smithi117.stdout:8/491: creat d0/d8/d39/d7f/fa5 x:0 0 0 2024-04-20T11:09:10.805 INFO:tasks.workunit.client.0.smithi117.stdout:2/675: read d1/d4/d83/d29/d2e/d80/fb2 [4064187,49996] 0 2024-04-20T11:09:10.805 INFO:tasks.workunit.client.0.smithi117.stdout:7/582: write d1/fa5 [5153632,36148] 0 2024-04-20T11:09:10.815 INFO:tasks.workunit.client.0.smithi117.stdout:2/676: readlink d1/d4/d83/d29/l8f 0 2024-04-20T11:09:10.816 INFO:tasks.workunit.client.0.smithi117.stdout:7/583: creat d1/d1d/fbb x:0 0 0 2024-04-20T11:09:10.818 INFO:tasks.workunit.client.0.smithi117.stdout:4/602: write d1/d9/f96 [3433040,21127] 0 2024-04-20T11:09:10.819 INFO:tasks.workunit.client.0.smithi117.stdout:8/492: getdents d0/d8/d39/d40/d78/d5c 0 2024-04-20T11:09:10.821 INFO:tasks.workunit.client.0.smithi117.stdout:2/677: stat d1/d4/dd 0 2024-04-20T11:09:10.828 INFO:tasks.workunit.client.0.smithi117.stdout:6/664: dwrite d3/df/d39/d99/f8c [0,4194304] 0 2024-04-20T11:09:10.831 INFO:tasks.workunit.client.0.smithi117.stdout:4/603: mknod d1/cd2 0 2024-04-20T11:09:10.849 INFO:tasks.workunit.client.0.smithi117.stdout:4/604: unlink d1/c27 0 2024-04-20T11:09:10.851 INFO:tasks.workunit.client.0.smithi117.stdout:2/678: read - d1/d4/d83/d29/d2e/d80/fb9 zero size 2024-04-20T11:09:10.853 INFO:tasks.workunit.client.0.smithi117.stdout:4/605: stat d1/d45/f5f 0 2024-04-20T11:09:10.854 INFO:tasks.workunit.client.0.smithi117.stdout:2/679: chown d1 26610 1 2024-04-20T11:09:10.855 INFO:tasks.workunit.client.0.smithi117.stdout:1/504: dwrite d6/f1a [0,4194304] 0 2024-04-20T11:09:10.855 INFO:tasks.workunit.client.0.smithi117.stdout:9/609: dwrite d4/d4a/f7e [0,4194304] 0 2024-04-20T11:09:10.857 INFO:tasks.workunit.client.0.smithi117.stdout:0/650: dwrite d3/d7/d8/f64 [0,4194304] 0 2024-04-20T11:09:10.860 INFO:tasks.workunit.client.0.smithi117.stdout:8/493: dread d0/dc/f1a [0,4194304] 0 2024-04-20T11:09:10.861 INFO:tasks.workunit.client.0.smithi117.stdout:4/606: chown d1/d9/c51 1 1 2024-04-20T11:09:10.883 INFO:tasks.workunit.client.0.smithi117.stdout:9/610: mkdir d4/de/d7b/dd0 0 2024-04-20T11:09:10.885 INFO:tasks.workunit.client.0.smithi117.stdout:8/494: symlink d0/d8/d39/d40/la6 0 2024-04-20T11:09:10.887 INFO:tasks.workunit.client.0.smithi117.stdout:9/611: readlink d4/d30/l7f 0 2024-04-20T11:09:10.890 INFO:tasks.workunit.client.0.smithi117.stdout:6/665: symlink d3/d4/d2e/d37/d8e/db2/lda 0 2024-04-20T11:09:10.894 INFO:tasks.workunit.client.0.smithi117.stdout:9/612: mknod d4/d4a/d63/cd1 0 2024-04-20T11:09:10.894 INFO:tasks.workunit.client.0.smithi117.stdout:0/651: creat d3/d7/d20/d39/d9a/dd0/d22/d85/fd4 x:0 0 0 2024-04-20T11:09:10.894 INFO:tasks.workunit.client.0.smithi117.stdout:1/505: getdents d6/d1f 0 2024-04-20T11:09:10.897 INFO:tasks.workunit.client.0.smithi117.stdout:6/666: creat d3/d4/d2e/d37/fdb x:0 0 0 2024-04-20T11:09:10.957 INFO:tasks.workunit.client.0.smithi117.stdout:9/613: mkdir d4/d4a/d72/dd2 0 2024-04-20T11:09:10.959 INFO:tasks.workunit.client.0.smithi117.stdout:4/607: getdents d1/dd/d1a 0 2024-04-20T11:09:10.961 INFO:tasks.workunit.client.0.smithi117.stdout:1/506: getdents d6/d1f 0 2024-04-20T11:09:10.961 INFO:tasks.workunit.client.0.smithi117.stdout:9/614: write d4/d5/f1e [5592724,35755] 0 2024-04-20T11:09:10.964 INFO:tasks.workunit.client.0.smithi117.stdout:1/507: read d6/d35/f44 [38277,102992] 0 2024-04-20T11:09:10.964 INFO:tasks.workunit.client.0.smithi117.stdout:2/680: dread d1/d4/d83/f7e [0,4194304] 0 2024-04-20T11:09:10.969 INFO:tasks.workunit.client.0.smithi117.stdout:2/681: stat d1/d4/dd/d3e/d46/cc2 0 2024-04-20T11:09:10.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:10 smithi117 ceph-mon[15659]: pgmap v15: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 29 MiB/s rd, 123 MiB/s wr, 290 op/s 2024-04-20T11:09:10.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:10 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:10.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:10 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:10.973 INFO:tasks.workunit.client.0.smithi117.stdout:1/508: rmdir d6/d16/d3a/d5a/d5c/d61/d98 39 2024-04-20T11:09:10.980 INFO:tasks.workunit.client.0.smithi117.stdout:0/652: dwrite d3/d7/d20/d39/d9a/dd0/d22/d85/fd4 [0,4194304] 0 2024-04-20T11:09:10.991 INFO:tasks.workunit.client.0.smithi117.stdout:1/509: symlink d6/d35/d63/la4 0 2024-04-20T11:09:11.001 INFO:tasks.workunit.client.0.smithi117.stdout:6/667: dread d3/d4/f46 [0,4194304] 0 2024-04-20T11:09:11.002 INFO:tasks.workunit.client.0.smithi117.stdout:1/510: symlink d6/d16/d3a/d5a/d5c/d61/d71/la5 0 2024-04-20T11:09:11.005 INFO:tasks.workunit.client.0.smithi117.stdout:4/608: dread d1/d9/d1f/f5c [0,4194304] 0 2024-04-20T11:09:11.009 INFO:tasks.workunit.client.0.smithi117.stdout:0/653: dwrite d3/d7/dc6/fc9 [0,4194304] 0 2024-04-20T11:09:11.012 INFO:tasks.workunit.client.0.smithi117.stdout:0/654: chown d3/dab/d54 154 1 2024-04-20T11:09:11.016 INFO:tasks.workunit.client.0.smithi117.stdout:6/668: rename d3/df/d39/l54 to d3/df/d39/d99/ldc 0 2024-04-20T11:09:11.016 INFO:tasks.workunit.client.0.smithi117.stdout:0/655: mknod d3/d5/d74/cd5 0 2024-04-20T11:09:11.017 INFO:tasks.workunit.client.0.smithi117.stdout:6/669: dread - d3/d4/d2e/d4e/fd4 zero size 2024-04-20T11:09:11.019 INFO:tasks.workunit.client.0.smithi117.stdout:4/609: rename d1/dd/d1a/d2b/caf to d1/dd/d1a/d43/d4e/cd3 0 2024-04-20T11:09:11.020 INFO:tasks.workunit.client.0.smithi117.stdout:0/656: chown d3/dab/d54/c61 6 1 2024-04-20T11:09:11.020 INFO:tasks.workunit.client.0.smithi117.stdout:9/615: write d4/d35/f5f [2626022,748] 0 2024-04-20T11:09:11.022 INFO:tasks.workunit.client.0.smithi117.stdout:9/616: dread - d4/d5/d57/fca zero size 2024-04-20T11:09:11.022 INFO:tasks.workunit.client.0.smithi117.stdout:0/657: dread - d3/d7/d2c/d52/fb7 zero size 2024-04-20T11:09:11.023 INFO:tasks.workunit.client.0.smithi117.stdout:9/617: fdatasync d4/d5/d57/fca 0 2024-04-20T11:09:11.024 INFO:tasks.workunit.client.0.smithi117.stdout:4/610: mknod d1/cd4 0 2024-04-20T11:09:11.029 INFO:tasks.workunit.client.0.smithi117.stdout:1/511: rmdir d6 39 2024-04-20T11:09:11.029 INFO:tasks.workunit.client.0.smithi117.stdout:0/658: read d3/d7/d20/d47/d78/f7c [600926,97486] 0 2024-04-20T11:09:11.032 INFO:tasks.workunit.client.0.smithi117.stdout:9/618: symlink d4/d5/da/d51/ld3 0 2024-04-20T11:09:11.037 INFO:tasks.workunit.client.0.smithi117.stdout:4/611: chown d1/d9/d6b/c6f 1896477017 1 2024-04-20T11:09:11.037 INFO:tasks.workunit.client.0.smithi117.stdout:6/670: rename d3/d4/f8d to d3/d4/d2e/fdd 0 2024-04-20T11:09:11.037 INFO:tasks.workunit.client.0.smithi117.stdout:2/682: dwrite d1/d4/dd/d17/d7a/d49/f54 [0,4194304] 0 2024-04-20T11:09:11.037 INFO:tasks.workunit.client.0.smithi117.stdout:9/619: chown d4/d30/caa 3961 1 2024-04-20T11:09:11.040 INFO:tasks.workunit.client.0.smithi117.stdout:5/819: sync 2024-04-20T11:09:11.040 INFO:tasks.workunit.client.0.smithi117.stdout:3/560: sync 2024-04-20T11:09:11.043 INFO:tasks.workunit.client.0.smithi117.stdout:0/659: symlink d3/dab/ld6 0 2024-04-20T11:09:11.049 INFO:tasks.workunit.client.0.smithi117.stdout:1/512: mkdir d6/d16/d3a/d5a/d5c/da6 0 2024-04-20T11:09:11.056 INFO:tasks.workunit.client.0.smithi117.stdout:2/683: creat d1/d20/fd9 x:0 0 0 2024-04-20T11:09:11.057 INFO:tasks.workunit.client.0.smithi117.stdout:3/561: symlink d0/d5/d21/d88/lc0 0 2024-04-20T11:09:11.058 INFO:tasks.workunit.client.0.smithi117.stdout:9/620: getdents d4/de/d7b 0 2024-04-20T11:09:11.059 INFO:tasks.workunit.client.0.smithi117.stdout:1/513: symlink d6/d16/d3a/d5a/d5c/d61/d91/la7 0 2024-04-20T11:09:11.064 INFO:tasks.workunit.client.0.smithi117.stdout:3/562: mkdir d0/d5/d21/d75/dc1 0 2024-04-20T11:09:11.065 INFO:tasks.workunit.client.0.smithi117.stdout:6/671: dwrite d3/d4/d2e/d3f/fc9 [0,4194304] 0 2024-04-20T11:09:11.069 INFO:tasks.workunit.client.0.smithi117.stdout:1/514: mkdir d6/d16/d3a/da8 0 2024-04-20T11:09:11.071 INFO:tasks.workunit.client.0.smithi117.stdout:9/621: creat d4/d5/d57/fd4 x:0 0 0 2024-04-20T11:09:11.072 INFO:tasks.workunit.client.0.smithi117.stdout:2/684: truncate d1/d4/d83/f34 755117 0 2024-04-20T11:09:11.075 INFO:tasks.workunit.client.0.smithi117.stdout:5/820: dread de/d21/df5/f3a [0,4194304] 0 2024-04-20T11:09:11.079 INFO:tasks.workunit.client.0.smithi117.stdout:7/584: sync 2024-04-20T11:09:11.080 INFO:tasks.workunit.client.0.smithi117.stdout:8/495: sync 2024-04-20T11:09:11.080 INFO:tasks.workunit.client.0.smithi117.stdout:0/660: dwrite d3/d7/d20/d39/d9a/dd0/d6d/f71 [0,4194304] 0 2024-04-20T11:09:11.083 INFO:tasks.workunit.client.0.smithi117.stdout:7/585: readlink d1/d1d/d42/d2d/l56 0 2024-04-20T11:09:11.084 INFO:tasks.workunit.client.0.smithi117.stdout:6/672: dread d3/d4/d2e/d4e/fbd [0,4194304] 0 2024-04-20T11:09:11.085 INFO:tasks.workunit.client.0.smithi117.stdout:5/821: read de/d21/df5/f8d [482040,96659] 0 2024-04-20T11:09:11.086 INFO:tasks.workunit.client.0.smithi117.stdout:2/685: creat d1/d4/d83/d29/fda x:0 0 0 2024-04-20T11:09:11.088 INFO:tasks.workunit.client.0.smithi117.stdout:4/612: write d1/d9/d1f/d87/fb3 [940595,3295] 0 2024-04-20T11:09:11.091 INFO:tasks.workunit.client.0.smithi117.stdout:6/673: chown d3/d50/d8a/l9e 235871 1 2024-04-20T11:09:11.091 INFO:tasks.workunit.client.0.smithi117.stdout:1/515: creat d6/d21/d95/fa9 x:0 0 0 2024-04-20T11:09:11.097 INFO:tasks.workunit.client.0.smithi117.stdout:2/686: write d1/d4/d83/d29/d2e/d53/d93/fb5 [535544,68897] 0 2024-04-20T11:09:11.101 INFO:tasks.workunit.client.0.smithi117.stdout:7/586: mkdir d1/d4/d3d/dbc 0 2024-04-20T11:09:11.103 INFO:tasks.workunit.client.0.smithi117.stdout:8/496: symlink d0/la7 0 2024-04-20T11:09:11.112 INFO:tasks.workunit.client.0.smithi117.stdout:0/661: dwrite d3/d7e/fc7 [0,4194304] 0 2024-04-20T11:09:11.117 INFO:tasks.workunit.client.0.smithi117.stdout:6/674: rename d3/df/c58 to d3/d50/cde 0 2024-04-20T11:09:11.118 INFO:tasks.workunit.client.0.smithi117.stdout:9/622: write d4/d5/da/f93 [133679,24082] 0 2024-04-20T11:09:11.124 INFO:tasks.workunit.client.0.smithi117.stdout:9/623: chown d4/de/d7b/d96/db0/f77 0 1 2024-04-20T11:09:11.128 INFO:tasks.workunit.client.0.smithi117.stdout:3/563: dread d0/d5/d7/dd/ff [0,4194304] 0 2024-04-20T11:09:11.128 INFO:tasks.workunit.client.0.smithi117.stdout:7/587: unlink d1/d4/d6/d2b/c32 0 2024-04-20T11:09:11.129 INFO:tasks.workunit.client.0.smithi117.stdout:1/516: write d6/d1f/f3d [435310,100417] 0 2024-04-20T11:09:11.130 INFO:tasks.workunit.client.0.smithi117.stdout:0/662: fdatasync d3/d7/d20/d39/d9a/dd0/d6d/f71 0 2024-04-20T11:09:11.130 INFO:tasks.workunit.client.0.smithi117.stdout:2/687: dread d1/d4/d83/f3a [8388608,4194304] 0 2024-04-20T11:09:11.133 INFO:tasks.workunit.client.0.smithi117.stdout:7/588: chown d1/d43/f61 73 1 2024-04-20T11:09:11.136 INFO:tasks.workunit.client.0.smithi117.stdout:6/675: write d3/f18 [1382737,51087] 0 2024-04-20T11:09:11.137 INFO:tasks.workunit.client.0.smithi117.stdout:8/497: unlink d0/fd 0 2024-04-20T11:09:11.138 INFO:tasks.workunit.client.0.smithi117.stdout:9/624: rmdir d4/d4a/d74 39 2024-04-20T11:09:11.140 INFO:tasks.workunit.client.0.smithi117.stdout:5/822: dwrite de/d5b/f81 [0,4194304] 0 2024-04-20T11:09:11.145 INFO:tasks.workunit.client.0.smithi117.stdout:1/517: chown d6/d35/l9c 5 1 2024-04-20T11:09:11.148 INFO:tasks.workunit.client.0.smithi117.stdout:6/676: dread d3/d4/f19 [0,4194304] 0 2024-04-20T11:09:11.152 INFO:tasks.workunit.client.0.smithi117.stdout:4/613: link d1/dd/d1a/d2b/d86/cbb d1/d9/d79/cd5 0 2024-04-20T11:09:11.154 INFO:tasks.workunit.client.0.smithi117.stdout:0/663: rename d3/d7/d20/fcc to d3/d7/d20/d39/d9a/dd0/db2/fd7 0 2024-04-20T11:09:11.157 INFO:tasks.workunit.client.0.smithi117.stdout:9/625: symlink d4/d4a/d72/ld5 0 2024-04-20T11:09:11.164 INFO:tasks.workunit.client.0.smithi117.stdout:2/688: symlink d1/d4/d83/ldb 0 2024-04-20T11:09:11.165 INFO:tasks.workunit.client.0.smithi117.stdout:6/677: symlink d3/d4/d2e/d37/d8e/db2/ldf 0 2024-04-20T11:09:11.165 INFO:tasks.workunit.client.0.smithi117.stdout:7/589: dread d1/d4/d6/d17/f4c [0,4194304] 0 2024-04-20T11:09:11.165 INFO:tasks.workunit.client.0.smithi117.stdout:1/518: mknod d6/d16/d3a/d5a/caa 0 2024-04-20T11:09:11.169 INFO:tasks.workunit.client.0.smithi117.stdout:4/614: chown d1/d9/c51 550 1 2024-04-20T11:09:11.170 INFO:tasks.workunit.client.0.smithi117.stdout:8/498: symlink d0/d8/d11/d24/la8 0 2024-04-20T11:09:11.171 INFO:tasks.workunit.client.0.smithi117.stdout:5/823: dwrite de/d41/d47/d55/fec [0,4194304] 0 2024-04-20T11:09:11.172 INFO:tasks.workunit.client.0.smithi117.stdout:7/590: chown d1/f10 4 1 2024-04-20T11:09:11.181 INFO:tasks.workunit.client.0.smithi117.stdout:4/615: creat d1/d45/d4a/d59/fd6 x:0 0 0 2024-04-20T11:09:11.185 INFO:tasks.workunit.client.0.smithi117.stdout:8/499: mkdir d0/dc/d49/d5b/da9 0 2024-04-20T11:09:11.185 INFO:tasks.workunit.client.0.smithi117.stdout:2/689: read d1/d4/d83/d29/d2e/d53/d93/fb5 [273620,64243] 0 2024-04-20T11:09:11.188 INFO:tasks.workunit.client.0.smithi117.stdout:1/519: symlink d6/d1f/d30/d49/lab 0 2024-04-20T11:09:11.191 INFO:tasks.workunit.client.0.smithi117.stdout:3/564: write d0/d5/d21/d88/d69/f8e [598279,77426] 0 2024-04-20T11:09:11.192 INFO:tasks.workunit.client.0.smithi117.stdout:7/591: rename d1/d4/d39/c58 to d1/d43/cbd 0 2024-04-20T11:09:11.208 INFO:tasks.workunit.client.0.smithi117.stdout:8/500: write d0/d8/d11/d24/d27/d47/f7d [1046807,35043] 0 2024-04-20T11:09:11.209 INFO:tasks.workunit.client.0.smithi117.stdout:2/690: dread - d1/d4/d83/d8e/f96 zero size 2024-04-20T11:09:11.211 INFO:tasks.workunit.client.0.smithi117.stdout:7/592: mknod d1/d4/d39/cbe 0 2024-04-20T11:09:11.213 INFO:tasks.workunit.client.0.smithi117.stdout:9/626: dwrite d4/de/d7b/fbd [0,4194304] 0 2024-04-20T11:09:11.215 INFO:tasks.workunit.client.0.smithi117.stdout:4/616: write d1/dd/f68 [490105,54910] 0 2024-04-20T11:09:11.218 INFO:tasks.workunit.client.0.smithi117.stdout:1/520: dread d6/d1f/f2b [0,4194304] 0 2024-04-20T11:09:11.218 INFO:tasks.workunit.client.0.smithi117.stdout:6/678: dwrite d3/d5a/fbe [0,4194304] 0 2024-04-20T11:09:11.221 INFO:tasks.workunit.client.0.smithi117.stdout:7/593: chown d1/d4/d39/f7a 64278 1 2024-04-20T11:09:11.222 INFO:tasks.workunit.client.0.smithi117.stdout:0/664: dread d3/d7/d20/d39/d9a/dd0/d6d/fb4 [0,4194304] 0 2024-04-20T11:09:11.222 INFO:tasks.workunit.client.0.smithi117.stdout:3/565: mkdir d0/d5/d21/d88/dc2 0 2024-04-20T11:09:11.227 INFO:tasks.workunit.client.0.smithi117.stdout:9/627: symlink d4/de/d10/d94/ld6 0 2024-04-20T11:09:11.227 INFO:tasks.workunit.client.0.smithi117.stdout:0/665: readlink d3/d7/d20/d39/lcb 0 2024-04-20T11:09:11.229 INFO:tasks.workunit.client.0.smithi117.stdout:1/521: symlink d6/d35/d63/lac 0 2024-04-20T11:09:11.230 INFO:tasks.workunit.client.0.smithi117.stdout:7/594: rename d1/d4/d6/fb2 to d1/d43/fbf 0 2024-04-20T11:09:11.234 INFO:tasks.workunit.client.0.smithi117.stdout:2/691: mkdir d1/d4/d83/d29/d2e/ddc 0 2024-04-20T11:09:11.234 INFO:tasks.workunit.client.0.smithi117.stdout:7/595: write d1/d4/d39/f83 [616816,51779] 0 2024-04-20T11:09:11.235 INFO:tasks.workunit.client.0.smithi117.stdout:5/824: dwrite de/fea [0,4194304] 0 2024-04-20T11:09:11.235 INFO:tasks.workunit.client.0.smithi117.stdout:1/522: dread - d6/d1f/d30/d49/d69/f92 zero size 2024-04-20T11:09:11.240 INFO:tasks.workunit.client.0.smithi117.stdout:9/628: mkdir d4/d4a/d72/dd7 0 2024-04-20T11:09:11.243 INFO:tasks.workunit.client.0.smithi117.stdout:0/666: symlink d3/d7/d20/d39/d9a/dd0/d22/d85/ld8 0 2024-04-20T11:09:11.250 INFO:tasks.workunit.client.0.smithi117.stdout:0/667: chown d3/d7/d2c/d52/fc5 173632383 1 2024-04-20T11:09:11.254 INFO:tasks.workunit.client.0.smithi117.stdout:7/596: rmdir d1/d1d 39 2024-04-20T11:09:11.254 INFO:tasks.workunit.client.0.smithi117.stdout:0/668: chown d3/d7/f18 274 1 2024-04-20T11:09:11.259 INFO:tasks.workunit.client.0.smithi117.stdout:0/669: write d3/dab/fac [455244,82219] 0 2024-04-20T11:09:11.270 INFO:tasks.workunit.client.0.smithi117.stdout:8/501: link d0/d8/d11/d24/d27/c28 d0/d8/d11/d6d/d58/caa 0 2024-04-20T11:09:11.271 INFO:tasks.workunit.client.0.smithi117.stdout:3/566: write d0/d5/d11/f7e [1591943,31738] 0 2024-04-20T11:09:11.275 INFO:tasks.workunit.client.0.smithi117.stdout:9/629: dwrite d4/d68/fb3 [0,4194304] 0 2024-04-20T11:09:11.275 INFO:tasks.workunit.client.0.smithi117.stdout:1/523: creat d6/d16/fad x:0 0 0 2024-04-20T11:09:11.275 INFO:tasks.workunit.client.0.smithi117.stdout:5/825: dwrite de/d5b/f81 [0,4194304] 0 2024-04-20T11:09:11.275 INFO:tasks.workunit.client.0.smithi117.stdout:7/597: symlink d1/lc0 0 2024-04-20T11:09:11.280 INFO:tasks.workunit.client.0.smithi117.stdout:9/630: write d4/d4a/d72/fb2 [2885372,6411] 0 2024-04-20T11:09:11.281 INFO:tasks.workunit.client.0.smithi117.stdout:2/692: dwrite d1/d4/dd/fc1 [0,4194304] 0 2024-04-20T11:09:11.297 INFO:tasks.workunit.client.0.smithi117.stdout:9/631: read d4/d5/da/fc [1425086,128652] 0 2024-04-20T11:09:11.300 INFO:tasks.workunit.client.0.smithi117.stdout:5/826: mknod de/d41/d47/d4c/c11b 0 2024-04-20T11:09:11.304 INFO:tasks.workunit.client.0.smithi117.stdout:3/567: read d0/d5/de/d4e/f53 [1967745,39365] 0 2024-04-20T11:09:11.306 INFO:tasks.workunit.client.0.smithi117.stdout:0/670: rename d3/dab/d54/l5f to d3/d7/d20/d39/d9a/dd0/db2/ld9 0 2024-04-20T11:09:11.312 INFO:tasks.workunit.client.0.smithi117.stdout:1/524: creat d6/d16/d3a/da8/fae x:0 0 0 2024-04-20T11:09:11.314 INFO:tasks.workunit.client.0.smithi117.stdout:8/502: dwrite d0/d8/d11/d24/d27/d47/f7d [0,4194304] 0 2024-04-20T11:09:11.316 INFO:tasks.workunit.client.0.smithi117.stdout:5/827: fsync de/d21/df5/d57/f62 0 2024-04-20T11:09:11.318 INFO:tasks.workunit.client.0.smithi117.stdout:3/568: creat d0/d2a/fc3 x:0 0 0 2024-04-20T11:09:11.318 INFO:tasks.workunit.client.0.smithi117.stdout:0/671: symlink d3/d7/d2c/d52/lda 0 2024-04-20T11:09:11.322 INFO:tasks.workunit.client.0.smithi117.stdout:1/525: rmdir d6/d16/d3a/d5a/d5c/d61/d71 39 2024-04-20T11:09:11.326 INFO:tasks.workunit.client.0.smithi117.stdout:7/598: write d1/d1d/d4e/d53/d55/f94 [838619,47886] 0 2024-04-20T11:09:11.326 INFO:tasks.workunit.client.0.smithi117.stdout:8/503: creat d0/d8/d11/d24/d2b/fab x:0 0 0 2024-04-20T11:09:11.326 INFO:tasks.workunit.client.0.smithi117.stdout:5/828: mkdir de/d41/d47/d55/d56/dc8/d11c 0 2024-04-20T11:09:11.332 INFO:tasks.workunit.client.0.smithi117.stdout:1/526: rmdir d6/d1f/d30/d32 39 2024-04-20T11:09:11.337 INFO:tasks.workunit.client.0.smithi117.stdout:1/527: readlink d6/d35/d63/l66 0 2024-04-20T11:09:11.339 INFO:tasks.workunit.client.0.smithi117.stdout:0/672: rename d3/d7/dc6 to d3/d7/d20/d39/d9a/dd0/d22/d85/ddb 0 2024-04-20T11:09:11.344 INFO:tasks.workunit.client.0.smithi117.stdout:2/693: dwrite d1/d4/dd/d17/d7a/d49/f87 [0,4194304] 0 2024-04-20T11:09:11.345 INFO:tasks.workunit.client.0.smithi117.stdout:9/632: dread d4/d5/da/f32 [0,4194304] 0 2024-04-20T11:09:11.350 INFO:tasks.workunit.client.0.smithi117.stdout:7/599: link d1/d4/d39/c9d d1/d43/d59/d8a/cc1 0 2024-04-20T11:09:11.354 INFO:tasks.workunit.client.0.smithi117.stdout:3/569: dread d0/d5/d11/f74 [0,4194304] 0 2024-04-20T11:09:11.356 INFO:tasks.workunit.client.0.smithi117.stdout:9/633: dread - d4/de/db1/fb9 zero size 2024-04-20T11:09:11.356 INFO:tasks.workunit.client.0.smithi117.stdout:1/528: mkdir d6/d16/d3a/d5a/daf 0 2024-04-20T11:09:11.358 INFO:tasks.workunit.client.0.smithi117.stdout:0/673: creat d3/d7/d8/fdc x:0 0 0 2024-04-20T11:09:11.367 INFO:tasks.workunit.client.0.smithi117.stdout:9/634: dread - d4/d30/d31/fbf zero size 2024-04-20T11:09:11.368 INFO:tasks.workunit.client.0.smithi117.stdout:5/829: dwrite de/d21/df5/d57/f62 [0,4194304] 0 2024-04-20T11:09:11.373 INFO:tasks.workunit.client.0.smithi117.stdout:3/570: creat d0/d5/d21/d88/d29/d66/d63/fc4 x:0 0 0 2024-04-20T11:09:11.378 INFO:tasks.workunit.client.0.smithi117.stdout:0/674: getdents d3/d7/d16 0 2024-04-20T11:09:11.381 INFO:tasks.workunit.client.0.smithi117.stdout:9/635: write d4/d30/d31/fbf [385775,31681] 0 2024-04-20T11:09:11.383 INFO:tasks.workunit.client.0.smithi117.stdout:8/504: dwrite d0/d8/f41 [0,4194304] 0 2024-04-20T11:09:11.385 INFO:tasks.workunit.client.0.smithi117.stdout:2/694: dread d1/d4/dd/fc1 [0,4194304] 0 2024-04-20T11:09:11.387 INFO:tasks.workunit.client.0.smithi117.stdout:7/600: dwrite d1/d1d/fbb [0,4194304] 0 2024-04-20T11:09:11.387 INFO:tasks.workunit.client.0.smithi117.stdout:0/675: write d3/d7/d8/f13 [2098054,126111] 0 2024-04-20T11:09:11.390 INFO:tasks.workunit.client.0.smithi117.stdout:5/830: rename de/d21/d51 to de/d21/df5/d57/dd5/d11d 0 2024-04-20T11:09:11.390 INFO:tasks.workunit.client.0.smithi117.stdout:3/571: write d0/d5/d21/d94/d9a/fa5 [5196882,116501] 0 2024-04-20T11:09:11.400 INFO:tasks.workunit.client.0.smithi117.stdout:9/636: read - d4/d4a/d74/fcc zero size 2024-04-20T11:09:11.403 INFO:tasks.workunit.client.0.smithi117.stdout:5/831: fdatasync de/d41/f9b 0 2024-04-20T11:09:11.404 INFO:tasks.workunit.client.0.smithi117.stdout:7/601: unlink d1/d4/d39/f83 0 2024-04-20T11:09:11.404 INFO:tasks.workunit.client.0.smithi117.stdout:2/695: fdatasync d1/d4/d83/d29/d2e/d53/d93/f9e 0 2024-04-20T11:09:11.411 INFO:tasks.workunit.client.0.smithi117.stdout:3/572: fsync d0/d5/d21/d88/f1f 0 2024-04-20T11:09:11.411 INFO:tasks.workunit.client.0.smithi117.stdout:9/637: chown d4/d35/f81 16565178 1 2024-04-20T11:09:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:10 smithi173 ceph-mon[23800]: pgmap v15: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 29 MiB/s rd, 123 MiB/s wr, 290 op/s 2024-04-20T11:09:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:10 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:11.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:10 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:11.416 INFO:tasks.workunit.client.0.smithi117.stdout:8/505: creat d0/d8/d11/fac x:0 0 0 2024-04-20T11:09:11.424 INFO:tasks.workunit.client.0.smithi117.stdout:2/696: mknod d1/d20/cdd 0 2024-04-20T11:09:11.424 INFO:tasks.workunit.client.0.smithi117.stdout:3/573: chown d0/c1 418944943 1 2024-04-20T11:09:11.435 INFO:tasks.workunit.client.0.smithi117.stdout:7/602: symlink d1/d4/d6/d54/d63/lc2 0 2024-04-20T11:09:11.438 INFO:tasks.workunit.client.0.smithi117.stdout:0/676: dwrite d3/d5/fa7 [0,4194304] 0 2024-04-20T11:09:11.442 INFO:tasks.workunit.client.0.smithi117.stdout:5/832: dwrite de/dd0/d10e/da7/fcd [0,4194304] 0 2024-04-20T11:09:11.446 INFO:tasks.workunit.client.0.smithi117.stdout:2/697: dread d1/d4/dd/d3e/f98 [0,4194304] 0 2024-04-20T11:09:11.451 INFO:tasks.workunit.client.0.smithi117.stdout:5/833: stat de/dd0/d10e/f104 0 2024-04-20T11:09:11.452 INFO:tasks.workunit.client.0.smithi117.stdout:7/603: fsync d1/d4/d6/f26 0 2024-04-20T11:09:11.457 INFO:tasks.workunit.client.0.smithi117.stdout:8/506: dwrite d0/f8f [0,4194304] 0 2024-04-20T11:09:11.461 INFO:tasks.workunit.client.0.smithi117.stdout:9/638: dwrite d4/de/f1b [0,4194304] 0 2024-04-20T11:09:11.467 INFO:tasks.workunit.client.0.smithi117.stdout:0/677: getdents d3/d7e 0 2024-04-20T11:09:11.470 INFO:tasks.workunit.client.0.smithi117.stdout:9/639: mkdir d4/de/d10/d94/dd8 0 2024-04-20T11:09:11.471 INFO:tasks.workunit.client.0.smithi117.stdout:8/507: chown d0/d9/f33 1578515021 1 2024-04-20T11:09:11.471 INFO:tasks.workunit.client.0.smithi117.stdout:9/640: fdatasync d4/d35/f5f 0 2024-04-20T11:09:11.474 INFO:tasks.workunit.client.0.smithi117.stdout:8/508: chown d0/d8/d11/d6d/d69/d9c 368 1 2024-04-20T11:09:11.478 INFO:tasks.workunit.client.0.smithi117.stdout:9/641: truncate d4/d4a/d74/fcc 713286 0 2024-04-20T11:09:11.479 INFO:tasks.workunit.client.0.smithi117.stdout:5/834: dread de/d5b/f81 [0,4194304] 0 2024-04-20T11:09:11.479 INFO:tasks.workunit.client.0.smithi117.stdout:7/604: getdents d1/d1d/d77 0 2024-04-20T11:09:11.482 INFO:tasks.workunit.client.0.smithi117.stdout:8/509: write d0/dc/f1a [1641528,99260] 0 2024-04-20T11:09:11.482 INFO:tasks.workunit.client.0.smithi117.stdout:2/698: dread d1/d4/dd/d17/d7a/d49/f54 [0,4194304] 0 2024-04-20T11:09:11.483 INFO:tasks.workunit.client.0.smithi117.stdout:5/835: readlink de/d21/d84/l9c 0 2024-04-20T11:09:11.488 INFO:tasks.workunit.client.0.smithi117.stdout:3/574: dread d0/d5/d21/d94/d9a/fa5 [0,4194304] 0 2024-04-20T11:09:11.495 INFO:tasks.workunit.client.0.smithi117.stdout:8/510: write d0/d9/d3a/f6a [3383158,107913] 0 2024-04-20T11:09:11.497 INFO:tasks.workunit.client.0.smithi117.stdout:0/678: getdents d3/d7/d20 0 2024-04-20T11:09:11.503 INFO:tasks.workunit.client.0.smithi117.stdout:4/617: sync 2024-04-20T11:09:11.504 INFO:tasks.workunit.client.0.smithi117.stdout:6/679: sync 2024-04-20T11:09:11.505 INFO:tasks.workunit.client.0.smithi117.stdout:0/679: chown d3/d7/d20/d39/d9a/dd0/db2/fd7 2488445 1 2024-04-20T11:09:11.505 INFO:tasks.workunit.client.0.smithi117.stdout:9/642: getdents d4/d5/d57/d82/da5 0 2024-04-20T11:09:11.511 INFO:tasks.workunit.client.0.smithi117.stdout:7/605: rmdir d1/d1d/d4e/d53/d55/db6 0 2024-04-20T11:09:11.511 INFO:tasks.workunit.client.0.smithi117.stdout:2/699: dread d1/d4/dd/d17/d7a/d49/f87 [0,4194304] 0 2024-04-20T11:09:11.514 INFO:tasks.workunit.client.0.smithi117.stdout:4/618: mknod d1/dd/d1a/d2b/cd7 0 2024-04-20T11:09:11.517 INFO:tasks.workunit.client.0.smithi117.stdout:9/643: dread d4/d5/da/f32 [0,4194304] 0 2024-04-20T11:09:11.520 INFO:tasks.workunit.client.0.smithi117.stdout:6/680: unlink d3/d4/d2e/d4e/fd4 0 2024-04-20T11:09:11.521 INFO:tasks.workunit.client.0.smithi117.stdout:6/681: dread - d3/d5a/d74/fc3 zero size 2024-04-20T11:09:11.524 INFO:tasks.workunit.client.0.smithi117.stdout:7/606: fdatasync d1/d4/f97 0 2024-04-20T11:09:11.548 INFO:tasks.workunit.client.0.smithi117.stdout:3/575: rename d0/d5/d11/f8f to d0/d5/fc5 0 2024-04-20T11:09:11.550 INFO:tasks.workunit.client.0.smithi117.stdout:2/700: creat d1/d4/dd/d3e/fde x:0 0 0 2024-04-20T11:09:11.551 INFO:tasks.workunit.client.0.smithi117.stdout:9/644: mkdir d4/de/d10/d94/dd9 0 2024-04-20T11:09:11.559 INFO:tasks.workunit.client.0.smithi117.stdout:5/836: dread de/db7/fbd [0,4194304] 0 2024-04-20T11:09:11.561 INFO:tasks.workunit.client.0.smithi117.stdout:5/837: stat de/l94 0 2024-04-20T11:09:11.564 INFO:tasks.workunit.client.0.smithi117.stdout:8/511: rename d0/d8/d39/d7f/f9f to d0/fad 0 2024-04-20T11:09:11.564 INFO:tasks.workunit.client.0.smithi117.stdout:5/838: chown de/d21/d84/cd2 103 1 2024-04-20T11:09:11.565 INFO:tasks.workunit.client.0.smithi117.stdout:3/576: write d0/d5/f40 [3217536,80632] 0 2024-04-20T11:09:11.566 INFO:tasks.workunit.client.0.smithi117.stdout:6/682: dwrite d3/df/f21 [0,4194304] 0 2024-04-20T11:09:11.567 INFO:tasks.workunit.client.0.smithi117.stdout:2/701: mkdir d1/d4/ddf 0 2024-04-20T11:09:11.571 INFO:tasks.workunit.client.0.smithi117.stdout:9/645: symlink d4/de/d7b/d96/db0/lda 0 2024-04-20T11:09:11.578 INFO:tasks.workunit.client.0.smithi117.stdout:7/607: dwrite d1/d1d/d42/d2d/f5f [0,4194304] 0 2024-04-20T11:09:11.579 INFO:tasks.workunit.client.0.smithi117.stdout:8/512: truncate d0/d8/d39/d7f/fa5 86062 0 2024-04-20T11:09:11.582 INFO:tasks.workunit.client.0.smithi117.stdout:9/646: chown d4/d35/cb8 56 1 2024-04-20T11:09:11.584 INFO:tasks.workunit.client.0.smithi117.stdout:5/839: symlink de/d21/d84/db8/l11e 0 2024-04-20T11:09:11.588 INFO:tasks.workunit.client.0.smithi117.stdout:4/619: dread d1/fa5 [0,4194304] 0 2024-04-20T11:09:11.597 INFO:tasks.workunit.client.0.smithi117.stdout:8/513: creat d0/d8/d39/d40/fae x:0 0 0 2024-04-20T11:09:11.598 INFO:tasks.workunit.client.0.smithi117.stdout:2/702: dread d1/da5/fd0 [0,4194304] 0 2024-04-20T11:09:11.600 INFO:tasks.workunit.client.0.smithi117.stdout:9/647: mkdir d4/d5/da/ddb 0 2024-04-20T11:09:11.602 INFO:tasks.workunit.client.0.smithi117.stdout:3/577: creat d0/d5/d21/d88/d29/d66/dbe/fc6 x:0 0 0 2024-04-20T11:09:11.610 INFO:tasks.workunit.client.0.smithi117.stdout:6/683: dwrite d3/df/f14 [4194304,4194304] 0 2024-04-20T11:09:11.614 INFO:tasks.workunit.client.0.smithi117.stdout:8/514: symlink d0/d8/d11/d24/d86/laf 0 2024-04-20T11:09:11.615 INFO:tasks.workunit.client.0.smithi117.stdout:9/648: dread - d4/de/d4b/f86 zero size 2024-04-20T11:09:11.615 INFO:tasks.workunit.client.0.smithi117.stdout:5/840: mkdir de/dd0/d10e/d11f 0 2024-04-20T11:09:11.616 INFO:tasks.workunit.client.0.smithi117.stdout:8/515: fdatasync d0/f83 0 2024-04-20T11:09:11.617 INFO:tasks.workunit.client.0.smithi117.stdout:9/649: chown d4/d5/da/ddb 9636 1 2024-04-20T11:09:11.618 INFO:tasks.workunit.client.0.smithi117.stdout:3/578: rmdir d0/d5/d21/d88/d29/d66/d63 39 2024-04-20T11:09:11.619 INFO:tasks.workunit.client.0.smithi117.stdout:2/703: dread d1/d4/f8 [0,4194304] 0 2024-04-20T11:09:11.621 INFO:tasks.workunit.client.0.smithi117.stdout:4/620: rename d1/c3a to d1/dd/d1a/d2b/cd8 0 2024-04-20T11:09:11.622 INFO:tasks.workunit.client.0.smithi117.stdout:2/704: chown d1/d4/d83/d29/d3f/cae 2507893 1 2024-04-20T11:09:11.624 INFO:tasks.workunit.client.0.smithi117.stdout:2/705: dread - d1/d9b/fb4 zero size 2024-04-20T11:09:11.627 INFO:tasks.workunit.client.0.smithi117.stdout:6/684: creat d3/df/d39/d99/fe0 x:0 0 0 2024-04-20T11:09:11.627 INFO:tasks.workunit.client.0.smithi117.stdout:2/706: chown d1/d4/dd/fc1 112655 1 2024-04-20T11:09:11.630 INFO:tasks.workunit.client.0.smithi117.stdout:6/685: chown d3/d4/d2e/d4e/d71/db7/f87 4624802 1 2024-04-20T11:09:11.630 INFO:tasks.workunit.client.0.smithi117.stdout:5/841: creat de/d41/dc2/f120 x:0 0 0 2024-04-20T11:09:11.633 INFO:tasks.workunit.client.0.smithi117.stdout:5/842: chown de/d41/d47/l7f 903 1 2024-04-20T11:09:11.637 INFO:tasks.workunit.client.0.smithi117.stdout:9/650: read d4/d5/f21 [1500978,106634] 0 2024-04-20T11:09:11.637 INFO:tasks.workunit.client.0.smithi117.stdout:1/529: sync 2024-04-20T11:09:11.638 INFO:tasks.workunit.client.0.smithi117.stdout:3/579: unlink d0/d5/l5f 0 2024-04-20T11:09:11.641 INFO:tasks.workunit.client.0.smithi117.stdout:4/621: dwrite d1/dd/d1a/d2b/d86/fb4 [0,4194304] 0 2024-04-20T11:09:11.643 INFO:tasks.workunit.client.0.smithi117.stdout:2/707: truncate d1/d15/d1d/f31 127801 0 2024-04-20T11:09:11.647 INFO:tasks.workunit.client.0.smithi117.stdout:8/516: creat d0/d8/d11/d24/d76/fb0 x:0 0 0 2024-04-20T11:09:11.647 INFO:tasks.workunit.client.0.smithi117.stdout:2/708: fsync d1/d4/fbc 0 2024-04-20T11:09:11.651 INFO:tasks.workunit.client.0.smithi117.stdout:6/686: mknod d3/d5a/d74/ce1 0 2024-04-20T11:09:11.660 INFO:tasks.workunit.client.0.smithi117.stdout:7/608: link d1/d4/d6/d17/d3f/c6d d1/d4/d6/d54/cc3 0 2024-04-20T11:09:11.661 INFO:tasks.workunit.client.0.smithi117.stdout:4/622: symlink d1/d45/d4a/d99/ld9 0 2024-04-20T11:09:11.661 INFO:tasks.workunit.client.0.smithi117.stdout:1/530: rename d6/d21/l4b to d6/d1f/d30/d49/d69/lb0 0 2024-04-20T11:09:11.665 INFO:tasks.workunit.client.0.smithi117.stdout:9/651: mkdir d4/d4a/d63/d98/dc0/dcb/ddc 0 2024-04-20T11:09:11.666 INFO:tasks.workunit.client.0.smithi117.stdout:8/517: read d0/d8/d11/d24/d27/d47/f8e [3427626,128328] 0 2024-04-20T11:09:11.668 INFO:tasks.workunit.client.0.smithi117.stdout:2/709: mkdir d1/d20/de0 0 2024-04-20T11:09:11.671 INFO:tasks.workunit.client.0.smithi117.stdout:1/531: rmdir d6/d21/d95 39 2024-04-20T11:09:11.675 INFO:tasks.workunit.client.0.smithi117.stdout:6/687: dwrite d3/d4/d2e/d4e/fa2 [0,4194304] 0 2024-04-20T11:09:11.678 INFO:tasks.workunit.client.0.smithi117.stdout:4/623: mkdir d1/dd/d1a/d2b/d86/dda 0 2024-04-20T11:09:11.678 INFO:tasks.workunit.client.0.smithi117.stdout:6/688: fsync d3/df/d39/f8f 0 2024-04-20T11:09:11.681 INFO:tasks.workunit.client.0.smithi117.stdout:6/689: fdatasync d3/d4/d2e/d4e/d71/fcf 0 2024-04-20T11:09:11.683 INFO:tasks.workunit.client.0.smithi117.stdout:4/624: truncate d1/dd/d1a/d43/d4e/f9b 624278 0 2024-04-20T11:09:11.684 INFO:tasks.workunit.client.0.smithi117.stdout:9/652: fsync d4/de/d10/f5c 0 2024-04-20T11:09:11.700 INFO:tasks.workunit.client.0.smithi117.stdout:6/690: mknod d3/d4/d2e/d4e/d71/ce2 0 2024-04-20T11:09:11.703 INFO:tasks.workunit.client.0.smithi117.stdout:6/691: write d3/d3a/fac [2080364,41039] 0 2024-04-20T11:09:11.708 INFO:tasks.workunit.client.0.smithi117.stdout:9/653: fdatasync d4/de/d4b/f86 0 2024-04-20T11:09:11.710 INFO:tasks.workunit.client.0.smithi117.stdout:1/532: symlink d6/d1f/d30/d32/lb1 0 2024-04-20T11:09:11.712 INFO:tasks.workunit.client.0.smithi117.stdout:5/843: dwrite de/d21/df5/fbe [0,4194304] 0 2024-04-20T11:09:11.728 INFO:tasks.workunit.client.0.smithi117.stdout:7/609: write d1/f7 [4254047,82227] 0 2024-04-20T11:09:11.735 INFO:tasks.workunit.client.0.smithi117.stdout:1/533: dwrite d6/d16/d3a/da8/fae [0,4194304] 0 2024-04-20T11:09:11.738 INFO:tasks.workunit.client.0.smithi117.stdout:6/692: dwrite d3/df/d39/d99/f64 [0,4194304] 0 2024-04-20T11:09:11.750 INFO:tasks.workunit.client.0.smithi117.stdout:4/625: write d1/d45/d4a/d99/fc0 [123590,5965] 0 2024-04-20T11:09:11.751 INFO:tasks.workunit.client.0.smithi117.stdout:7/610: unlink d1/d4/d6/d54/l5b 0 2024-04-20T11:09:11.751 INFO:tasks.workunit.client.0.smithi117.stdout:2/710: dwrite d1/d4/dd/d3e/f98 [0,4194304] 0 2024-04-20T11:09:11.755 INFO:tasks.workunit.client.0.smithi117.stdout:4/626: truncate d1/d9/d1f/d87/fbe 267468 0 2024-04-20T11:09:11.759 INFO:tasks.workunit.client.0.smithi117.stdout:1/534: symlink d6/d1f/lb2 0 2024-04-20T11:09:11.771 INFO:tasks.workunit.client.0.smithi117.stdout:0/680: sync 2024-04-20T11:09:11.771 INFO:tasks.workunit.client.0.smithi117.stdout:9/654: truncate d4/f2e 4006399 0 2024-04-20T11:09:11.773 INFO:tasks.workunit.client.0.smithi117.stdout:5/844: link de/dd0/d10e/d4e/ff2 de/d41/d4b/df9/db9/f121 0 2024-04-20T11:09:11.778 INFO:tasks.workunit.client.0.smithi117.stdout:6/693: dwrite d3/df/d39/daa/fd1 [0,4194304] 0 2024-04-20T11:09:11.778 INFO:tasks.workunit.client.0.smithi117.stdout:0/681: creat d3/d5/fdd x:0 0 0 2024-04-20T11:09:11.780 INFO:tasks.workunit.client.0.smithi117.stdout:6/694: write d3/df/f63 [4255546,43387] 0 2024-04-20T11:09:11.784 INFO:tasks.workunit.client.0.smithi117.stdout:4/627: write d1/fa7 [4337977,49442] 0 2024-04-20T11:09:11.784 INFO:tasks.workunit.client.0.smithi117.stdout:9/655: mknod d4/d5/d3c/cdd 0 2024-04-20T11:09:11.788 INFO:tasks.workunit.client.0.smithi117.stdout:2/711: rmdir d1/d15/dbd 0 2024-04-20T11:09:11.797 INFO:tasks.workunit.client.0.smithi117.stdout:0/682: dwrite d3/d5/fa7 [4194304,4194304] 0 2024-04-20T11:09:11.800 INFO:tasks.workunit.client.0.smithi117.stdout:1/535: creat d6/d35/fb3 x:0 0 0 2024-04-20T11:09:11.800 INFO:tasks.workunit.client.0.smithi117.stdout:6/695: chown d3/d5a/l69 1199292 1 2024-04-20T11:09:11.800 INFO:tasks.workunit.client.0.smithi117.stdout:5/845: symlink de/dd0/d10e/d4e/da1/de7/l122 0 2024-04-20T11:09:11.800 INFO:tasks.workunit.client.0.smithi117.stdout:4/628: creat d1/d45/fdb x:0 0 0 2024-04-20T11:09:11.803 INFO:tasks.workunit.client.0.smithi117.stdout:6/696: write d3/df/f21 [4432029,47382] 0 2024-04-20T11:09:11.804 INFO:tasks.workunit.client.0.smithi117.stdout:5/846: chown de/d21/df5/c7c 175 1 2024-04-20T11:09:11.805 INFO:tasks.workunit.client.0.smithi117.stdout:4/629: stat d1/dd/d1a/d2b/cd7 0 2024-04-20T11:09:11.808 INFO:tasks.workunit.client.0.smithi117.stdout:5/847: stat de/dd0/d10e/d4e/da1/de7 0 2024-04-20T11:09:11.809 INFO:tasks.workunit.client.0.smithi117.stdout:5/848: dread - de/d21/ff6 zero size 2024-04-20T11:09:11.810 INFO:tasks.workunit.client.0.smithi117.stdout:9/656: rename d4/d4a/d63/d98/dc0/f9f to d4/d5/da/fde 0 2024-04-20T11:09:11.815 INFO:tasks.workunit.client.0.smithi117.stdout:2/712: creat d1/d20/de0/fe1 x:0 0 0 2024-04-20T11:09:11.816 INFO:tasks.workunit.client.0.smithi117.stdout:0/683: creat d3/d7/d2c/d52/fde x:0 0 0 2024-04-20T11:09:11.823 INFO:tasks.workunit.client.0.smithi117.stdout:4/630: rename d1/d9/d1f/f5c to d1/dd/d1a/d2b/d86/fdc 0 2024-04-20T11:09:11.824 INFO:tasks.workunit.client.0.smithi117.stdout:9/657: rmdir d4/d5/d3c 39 2024-04-20T11:09:11.825 INFO:tasks.workunit.client.0.smithi117.stdout:4/631: readlink d1/dd/d52/l83 0 2024-04-20T11:09:11.827 INFO:tasks.workunit.client.0.smithi117.stdout:1/536: creat d6/fb4 x:0 0 0 2024-04-20T11:09:11.830 INFO:tasks.workunit.client.0.smithi117.stdout:0/684: creat d3/d7/d20/d39/d3a/db1/fdf x:0 0 0 2024-04-20T11:09:11.831 INFO:tasks.workunit.client.0.smithi117.stdout:5/849: rename de/d41/d4b/df9 to de/dd0/d10e/d4e/da1/de7/d123 0 2024-04-20T11:09:11.836 INFO:tasks.workunit.client.0.smithi117.stdout:9/658: unlink d4/d4a/d63/fc3 0 2024-04-20T11:09:11.837 INFO:tasks.workunit.client.0.smithi117.stdout:9/659: readlink d4/d5/la2 0 2024-04-20T11:09:11.842 INFO:tasks.workunit.client.0.smithi117.stdout:4/632: rename d1/fa5 to d1/dd/d1a/d2b/d86/dda/fdd 0 2024-04-20T11:09:11.842 INFO:tasks.workunit.client.0.smithi117.stdout:9/660: chown d4/d5/d57/d82/da5 476768 1 2024-04-20T11:09:11.843 INFO:tasks.workunit.client.0.smithi117.stdout:1/537: read d6/f2e [1802001,88322] 0 2024-04-20T11:09:11.847 INFO:tasks.workunit.client.0.smithi117.stdout:0/685: creat d3/dab/d54/fe0 x:0 0 0 2024-04-20T11:09:11.851 INFO:tasks.workunit.client.0.smithi117.stdout:6/697: dwrite d3/d3a/f4d [0,4194304] 0 2024-04-20T11:09:11.854 INFO:tasks.workunit.client.0.smithi117.stdout:5/850: creat de/dd0/d10e/d11f/f124 x:0 0 0 2024-04-20T11:09:11.855 INFO:tasks.workunit.client.0.smithi117.stdout:0/686: write d3/d5/d74/fad [172181,100927] 0 2024-04-20T11:09:11.861 INFO:tasks.workunit.client.0.smithi117.stdout:2/713: dread d1/d4/dd/d17/f1f [4194304,4194304] 0 2024-04-20T11:09:11.863 INFO:tasks.workunit.client.0.smithi117.stdout:1/538: dread - d6/d1f/d30/d49/d69/d6e/d6f/d73/f75 zero size 2024-04-20T11:09:11.863 INFO:tasks.workunit.client.0.smithi117.stdout:2/714: chown d1/d4/dd/d17/d7a 98 1 2024-04-20T11:09:11.875 INFO:tasks.workunit.client.0.smithi117.stdout:0/687: dwrite d3/d7/d16/f2e [0,4194304] 0 2024-04-20T11:09:11.889 INFO:tasks.workunit.client.0.smithi117.stdout:9/661: rename d4/d30 to d4/d5/d3c/ddf 0 2024-04-20T11:09:11.892 INFO:tasks.workunit.client.0.smithi117.stdout:5/851: rmdir de/d41/d47/d55/dab 39 2024-04-20T11:09:11.893 INFO:tasks.workunit.client.0.smithi117.stdout:6/698: mknod d3/d5a/d74/db3/dbc/d9b/ce3 0 2024-04-20T11:09:11.893 INFO:tasks.workunit.client.0.smithi117.stdout:1/539: stat c1 0 2024-04-20T11:09:11.893 INFO:tasks.workunit.client.0.smithi117.stdout:6/699: chown d3 797 1 2024-04-20T11:09:11.896 INFO:tasks.workunit.client.0.smithi117.stdout:0/688: creat d3/dab/d54/fe1 x:0 0 0 2024-04-20T11:09:11.901 INFO:tasks.workunit.client.0.smithi117.stdout:6/700: read - d3/d4/d2e/d4e/d71/db7/f87 zero size 2024-04-20T11:09:11.902 INFO:tasks.workunit.client.0.smithi117.stdout:0/689: unlink d3/d7/d16/c7b 0 2024-04-20T11:09:11.903 INFO:tasks.workunit.client.0.smithi117.stdout:0/690: readlink d3/d7/d20/d39/d3a/l3d 0 2024-04-20T11:09:11.905 INFO:tasks.workunit.client.0.smithi117.stdout:1/540: mknod d6/d1f/cb5 0 2024-04-20T11:09:11.906 INFO:tasks.workunit.client.0.smithi117.stdout:0/691: chown d3/d7/d20/d39/d3a/db1/fc4 39877 1 2024-04-20T11:09:11.907 INFO:tasks.workunit.client.0.smithi117.stdout:2/715: truncate d1/d4/dd/d17/d3b/fc9 621915 0 2024-04-20T11:09:11.909 INFO:tasks.workunit.client.0.smithi117.stdout:0/692: chown d3/l1d 34 1 2024-04-20T11:09:11.911 INFO:tasks.workunit.client.0.smithi117.stdout:6/701: unlink d3/d4/f8 0 2024-04-20T11:09:11.914 INFO:tasks.workunit.client.0.smithi117.stdout:5/852: creat de/d41/d47/f125 x:0 0 0 2024-04-20T11:09:11.916 INFO:tasks.workunit.client.0.smithi117.stdout:2/716: creat d1/d20/de0/fe2 x:0 0 0 2024-04-20T11:09:11.919 INFO:tasks.workunit.client.0.smithi117.stdout:0/693: symlink d3/d7/d20/d39/d9a/dd0/d22/d85/le2 0 2024-04-20T11:09:11.920 INFO:tasks.workunit.client.0.smithi117.stdout:6/702: unlink d3/d4/f19 0 2024-04-20T11:09:11.923 INFO:tasks.workunit.client.0.smithi117.stdout:5/853: mkdir de/dd0/d10e/d4e/da1/d126 0 2024-04-20T11:09:11.924 INFO:tasks.workunit.client.0.smithi117.stdout:2/717: mknod d1/d4/d83/d29/d62/ce3 0 2024-04-20T11:09:11.927 INFO:tasks.workunit.client.0.smithi117.stdout:6/703: symlink d3/df/d39/le4 0 2024-04-20T11:09:11.928 INFO:tasks.workunit.client.0.smithi117.stdout:5/854: creat de/d16/f127 x:0 0 0 2024-04-20T11:09:11.931 INFO:tasks.workunit.client.0.smithi117.stdout:6/704: chown d3/d4/d2e/d37/fdb 905 1 2024-04-20T11:09:11.932 INFO:tasks.workunit.client.0.smithi117.stdout:2/718: rename d1/d4/dd/f14 to d1/d4/dd/d3e/d46/fe4 0 2024-04-20T11:09:11.932 INFO:tasks.workunit.client.0.smithi117.stdout:6/705: chown d3/d50/d8a/l9e 8616 1 2024-04-20T11:09:11.935 INFO:tasks.workunit.client.0.smithi117.stdout:5/855: fdatasync de/d16/d9d/db4/df1/f103 0 2024-04-20T11:09:11.935 INFO:tasks.workunit.client.0.smithi117.stdout:6/706: write d3/d4/d2e/d3f/f77 [657152,102381] 0 2024-04-20T11:09:11.935 INFO:tasks.workunit.client.0.smithi117.stdout:0/694: truncate d3/d7/d94/fa5 817433 0 2024-04-20T11:09:11.938 INFO:tasks.workunit.client.0.smithi117.stdout:2/719: unlink d1/d9b/fc7 0 2024-04-20T11:09:11.938 INFO:tasks.workunit.client.0.smithi117.stdout:6/707: write d3/df/d39/d99/f64 [456375,106373] 0 2024-04-20T11:09:11.939 INFO:tasks.workunit.client.0.smithi117.stdout:0/695: write d3/d7/d2c/d52/fd3 [303640,9706] 0 2024-04-20T11:09:11.946 INFO:tasks.workunit.client.0.smithi117.stdout:2/720: symlink d1/d4/dd/d3e/le5 0 2024-04-20T11:09:11.947 INFO:tasks.workunit.client.0.smithi117.stdout:3/580: sync 2024-04-20T11:09:11.947 INFO:tasks.workunit.client.0.smithi117.stdout:4/633: dread d1/dd/d1a/d2b/f46 [0,4194304] 0 2024-04-20T11:09:11.949 INFO:tasks.workunit.client.0.smithi117.stdout:6/708: symlink d3/d4/d2e/d4e/d71/db7/le5 0 2024-04-20T11:09:11.952 INFO:tasks.workunit.client.0.smithi117.stdout:5/856: symlink de/l128 0 2024-04-20T11:09:11.952 INFO:tasks.workunit.client.0.smithi117.stdout:2/721: chown d1/d20/l71 2012 1 2024-04-20T11:09:11.954 INFO:tasks.workunit.client.0.smithi117.stdout:6/709: chown d3/f18 417864088 1 2024-04-20T11:09:11.956 INFO:tasks.workunit.client.0.smithi117.stdout:3/581: truncate d0/d5/d7/dd/f91 679812 0 2024-04-20T11:09:11.957 INFO:tasks.workunit.client.0.smithi117.stdout:4/634: mkdir d1/d9/d1f/d90/dde 0 2024-04-20T11:09:11.959 INFO:tasks.workunit.client.0.smithi117.stdout:1/541: dread d6/f18 [4194304,4194304] 0 2024-04-20T11:09:11.960 INFO:tasks.workunit.client.0.smithi117.stdout:3/582: readlink d0/d5/d21/d75/lb7 0 2024-04-20T11:09:11.961 INFO:tasks.workunit.client.0.smithi117.stdout:4/635: dread - d1/dd/d1a/f8b zero size 2024-04-20T11:09:11.962 INFO:tasks.workunit.client.0.smithi117.stdout:6/710: chown d3/d4/d2e/d37/l96 1 1 2024-04-20T11:09:11.963 INFO:tasks.workunit.client.0.smithi117.stdout:1/542: chown d6/d35/c4f 0 1 2024-04-20T11:09:11.965 INFO:tasks.workunit.client.0.smithi117.stdout:3/583: fdatasync d0/d5/d21/f41 0 2024-04-20T11:09:11.966 INFO:tasks.workunit.client.0.smithi117.stdout:4/636: chown d1/dd/d52/l8a 3 1 2024-04-20T11:09:11.970 INFO:tasks.workunit.client.0.smithi117.stdout:5/857: truncate de/d16/d9d/db4/df1/f101 188273 0 2024-04-20T11:09:11.973 INFO:tasks.workunit.client.0.smithi117.stdout:1/543: symlink d6/d1f/d30/d32/lb6 0 2024-04-20T11:09:11.976 INFO:tasks.workunit.client.0.smithi117.stdout:4/637: truncate d1/dd/fb5 153409 0 2024-04-20T11:09:11.978 INFO:tasks.workunit.client.0.smithi117.stdout:0/696: dread d3/d7/d20/d39/d9a/dd0/d6d/f9c [0,4194304] 0 2024-04-20T11:09:11.984 INFO:tasks.workunit.client.0.smithi117.stdout:0/697: chown d3/d7/d20/d39/d9a/dd0 943718106 1 2024-04-20T11:09:11.989 INFO:tasks.workunit.client.0.smithi117.stdout:9/662: dwrite d4/d5/d3c/f46 [0,4194304] 0 2024-04-20T11:09:11.993 INFO:tasks.workunit.client.0.smithi117.stdout:5/858: rename de/d5b/d65/fa2 to de/d41/d47/d55/f129 0 2024-04-20T11:09:11.993 INFO:tasks.workunit.client.0.smithi117.stdout:0/698: write d3/d7/d2c/f65 [2499477,59651] 0 2024-04-20T11:09:11.994 INFO:tasks.workunit.client.0.smithi117.stdout:8/518: sync 2024-04-20T11:09:11.994 INFO:tasks.workunit.client.0.smithi117.stdout:7/611: sync 2024-04-20T11:09:11.997 INFO:tasks.workunit.client.0.smithi117.stdout:6/711: dwrite d3/d3a/fd6 [0,4194304] 0 2024-04-20T11:09:11.997 INFO:tasks.workunit.client.0.smithi117.stdout:2/722: write d1/f7b [2477422,36649] 0 2024-04-20T11:09:12.009 INFO:tasks.workunit.client.0.smithi117.stdout:1/544: link d6/d1f/d30/d49/d69/f82 d6/d16/d3a/d5a/d5c/d61/d91/fb7 0 2024-04-20T11:09:12.009 INFO:tasks.workunit.client.0.smithi117.stdout:8/519: readlink d0/d8/lf 0 2024-04-20T11:09:12.013 INFO:tasks.workunit.client.0.smithi117.stdout:0/699: write d3/d5/d74/fad [92896,122717] 0 2024-04-20T11:09:12.015 INFO:tasks.workunit.client.0.smithi117.stdout:6/712: unlink d3/d4/d2e/d4e/d71/da3/lc4 0 2024-04-20T11:09:12.018 INFO:tasks.workunit.client.0.smithi117.stdout:2/723: creat d1/d4/ddf/fe6 x:0 0 0 2024-04-20T11:09:12.022 INFO:tasks.workunit.client.0.smithi117.stdout:7/612: read d1/f22 [3366697,79751] 0 2024-04-20T11:09:12.024 INFO:tasks.workunit.client.0.smithi117.stdout:2/724: dread - d1/d4/fc0 zero size 2024-04-20T11:09:12.024 INFO:tasks.workunit.client.0.smithi117.stdout:8/520: rmdir d0/d8/d11/d24/d27 39 2024-04-20T11:09:12.024 INFO:tasks.workunit.client.0.smithi117.stdout:4/638: dwrite d1/dd/d1a/d43/f65 [0,4194304] 0 2024-04-20T11:09:12.032 INFO:tasks.workunit.client.0.smithi117.stdout:7/613: mkdir d1/d43/dc4 0 2024-04-20T11:09:12.032 INFO:tasks.workunit.client.0.smithi117.stdout:8/521: write d0/d8/d11/d24/d2b/fab [896943,69657] 0 2024-04-20T11:09:12.039 INFO:tasks.workunit.client.0.smithi117.stdout:2/725: mkdir d1/d4/d83/d29/d2e/d53/de7 0 2024-04-20T11:09:12.039 INFO:tasks.workunit.client.0.smithi117.stdout:1/545: creat d6/d21/fb8 x:0 0 0 2024-04-20T11:09:12.039 INFO:tasks.workunit.client.0.smithi117.stdout:4/639: mknod d1/d9/d6b/dca/cdf 0 2024-04-20T11:09:12.040 INFO:tasks.workunit.client.0.smithi117.stdout:5/859: link de/d41/d47/d4c/fb1 de/d41/dc2/f12a 0 2024-04-20T11:09:12.045 INFO:tasks.workunit.client.0.smithi117.stdout:5/860: chown de/d41/d47/f125 6067 1 2024-04-20T11:09:12.048 INFO:tasks.workunit.client.0.smithi117.stdout:1/546: read d6/d1f/d30/f43 [165767,112226] 0 2024-04-20T11:09:12.048 INFO:tasks.workunit.client.0.smithi117.stdout:9/663: dread d4/d5/d3c/ddf/f4e [0,4194304] 0 2024-04-20T11:09:12.055 INFO:tasks.workunit.client.0.smithi117.stdout:8/522: rename d0/dc/f1e to d0/d8/d39/d40/d78/fb1 0 2024-04-20T11:09:12.055 INFO:tasks.workunit.client.0.smithi117.stdout:2/726: truncate d1/d4/d83/d29/d3f/f76 560937 0 2024-04-20T11:09:12.056 INFO:tasks.workunit.client.0.smithi117.stdout:9/664: stat d4/f2e 0 2024-04-20T11:09:12.059 INFO:tasks.workunit.client.0.smithi117.stdout:8/523: write d0/d8/d39/d40/fae [141933,41502] 0 2024-04-20T11:09:12.066 INFO:tasks.workunit.client.0.smithi117.stdout:9/665: symlink d4/d4a/d5e/d9b/dcd/le0 0 2024-04-20T11:09:12.066 INFO:tasks.workunit.client.0.smithi117.stdout:8/524: chown d0/d8/d11/d24/d2b/d72/f75 1911203 1 2024-04-20T11:09:12.068 INFO:tasks.workunit.client.0.smithi117.stdout:0/700: write d3/d7/d20/d39/d9a/dd0/d22/f75 [997290,47903] 0 2024-04-20T11:09:12.069 INFO:tasks.workunit.client.0.smithi117.stdout:7/614: write d1/f10 [432654,125909] 0 2024-04-20T11:09:12.070 INFO:tasks.workunit.client.0.smithi117.stdout:7/615: stat d1 0 2024-04-20T11:09:12.073 INFO:tasks.workunit.client.0.smithi117.stdout:8/525: read d0/d8/d39/d40/f5f [1443847,33103] 0 2024-04-20T11:09:12.073 INFO:tasks.workunit.client.0.smithi117.stdout:1/547: write d6/d16/d3a/d5a/d5c/d61/f83 [2951392,121362] 0 2024-04-20T11:09:12.076 INFO:tasks.workunit.client.0.smithi117.stdout:2/727: write d1/d15/d1d/f50 [311234,97546] 0 2024-04-20T11:09:12.077 INFO:tasks.workunit.client.0.smithi117.stdout:0/701: symlink d3/d7/d20/d39/d9a/le3 0 2024-04-20T11:09:12.079 INFO:tasks.workunit.client.0.smithi117.stdout:7/616: creat d1/d1d/d77/fc5 x:0 0 0 2024-04-20T11:09:12.081 INFO:tasks.workunit.client.0.smithi117.stdout:7/617: dread - d1/d43/fbf zero size 2024-04-20T11:09:12.083 INFO:tasks.workunit.client.0.smithi117.stdout:8/526: dread d0/d8/d39/d40/fae [0,4194304] 0 2024-04-20T11:09:12.084 INFO:tasks.workunit.client.0.smithi117.stdout:2/728: unlink d1/d4/dd/d17/d3b/c84 0 2024-04-20T11:09:12.086 INFO:tasks.workunit.client.0.smithi117.stdout:9/666: getdents d4/d4a/d63/d98/dc0 0 2024-04-20T11:09:12.089 INFO:tasks.workunit.client.0.smithi117.stdout:0/702: read - d3/d7/d20/d39/d3a/db1/fc4 zero size 2024-04-20T11:09:12.095 INFO:tasks.workunit.client.0.smithi117.stdout:9/667: symlink d4/d5/d3c/ddf/d31/le1 0 2024-04-20T11:09:12.095 INFO:tasks.workunit.client.0.smithi117.stdout:2/729: symlink d1/da5/le8 0 2024-04-20T11:09:12.101 INFO:tasks.workunit.client.0.smithi117.stdout:8/527: getdents d0/d8/d39/d40/d78 0 2024-04-20T11:09:12.102 INFO:tasks.workunit.client.0.smithi117.stdout:5/861: dread de/d21/df5/d57/dd5/f114 [0,4194304] 0 2024-04-20T11:09:12.103 INFO:tasks.workunit.client.0.smithi117.stdout:4/640: dread d1/dd/f53 [0,4194304] 0 2024-04-20T11:09:12.106 INFO:tasks.workunit.client.0.smithi117.stdout:9/668: rename d4/d5/d3c/cdd to d4/de/d10/d94/dd8/ce2 0 2024-04-20T11:09:12.108 INFO:tasks.workunit.client.0.smithi117.stdout:2/730: creat d1/d4/d83/d29/d2e/fe9 x:0 0 0 2024-04-20T11:09:12.109 INFO:tasks.workunit.client.0.smithi117.stdout:0/703: write d3/d7/d8/f31 [600277,26269] 0 2024-04-20T11:09:12.111 INFO:tasks.workunit.client.0.smithi117.stdout:1/548: dread d6/d1f/d30/f74 [0,4194304] 0 2024-04-20T11:09:12.112 INFO:tasks.workunit.client.0.smithi117.stdout:0/704: dread - d3/d7/d2c/d52/fb7 zero size 2024-04-20T11:09:12.112 INFO:tasks.workunit.client.0.smithi117.stdout:5/862: mkdir de/db7/d12b 0 2024-04-20T11:09:12.114 INFO:tasks.workunit.client.0.smithi117.stdout:4/641: creat d1/dd/d1a/fe0 x:0 0 0 2024-04-20T11:09:12.115 INFO:tasks.workunit.client.0.smithi117.stdout:9/669: mknod d4/de/d7b/dd0/ce3 0 2024-04-20T11:09:12.117 INFO:tasks.workunit.client.0.smithi117.stdout:1/549: mkdir d6/d21/db9 0 2024-04-20T11:09:12.119 INFO:tasks.workunit.client.0.smithi117.stdout:5/863: creat de/d16/d9d/db4/f12c x:0 0 0 2024-04-20T11:09:12.120 INFO:tasks.workunit.client.0.smithi117.stdout:9/670: unlink c3 0 2024-04-20T11:09:12.123 INFO:tasks.workunit.client.0.smithi117.stdout:9/671: chown d4/d5/d3c/ddf/d31/ca7 104437 1 2024-04-20T11:09:12.124 INFO:tasks.workunit.client.0.smithi117.stdout:2/731: link d1/d4/d83/d29/d2e/d53/d93/fb5 d1/d4/dd/d17/d7a/d49/fea 0 2024-04-20T11:09:12.125 INFO:tasks.workunit.client.0.smithi117.stdout:9/672: readlink d4/d4a/d5e/d9b/la4 0 2024-04-20T11:09:12.128 INFO:tasks.workunit.client.0.smithi117.stdout:5/864: symlink de/db7/d12b/l12d 0 2024-04-20T11:09:12.130 INFO:tasks.workunit.client.0.smithi117.stdout:9/673: mkdir d4/de4 0 2024-04-20T11:09:12.133 INFO:tasks.workunit.client.0.smithi117.stdout:5/865: unlink de/d21/df5/d57/dd5/fc4 0 2024-04-20T11:09:12.140 INFO:tasks.workunit.client.0.smithi117.stdout:5/866: symlink de/dd0/d10e/d4e/da1/d126/l12e 0 2024-04-20T11:09:12.142 INFO:tasks.workunit.client.0.smithi117.stdout:2/732: write d1/d4/d83/d29/d2e/f40 [1448452,75567] 0 2024-04-20T11:09:12.148 INFO:tasks.workunit.client.0.smithi117.stdout:5/867: rename de/d21/df5/f3a to de/f12f 0 2024-04-20T11:09:12.149 INFO:tasks.workunit.client.0.smithi117.stdout:2/733: truncate d1/d4/d83/f3a 10095110 0 2024-04-20T11:09:12.152 INFO:tasks.workunit.client.0.smithi117.stdout:1/550: dread d6/d35/f56 [4194304,4194304] 0 2024-04-20T11:09:12.153 INFO:tasks.workunit.client.0.smithi117.stdout:5/868: truncate de/d16/d9d/db4/fe6 80770 0 2024-04-20T11:09:12.154 INFO:tasks.workunit.client.0.smithi117.stdout:2/734: unlink d1/d20/cd2 0 2024-04-20T11:09:12.161 INFO:tasks.workunit.client.0.smithi117.stdout:5/869: mkdir de/d16/d9d/db4/d130 0 2024-04-20T11:09:12.161 INFO:tasks.workunit.client.0.smithi117.stdout:0/705: dread d3/d7/d8/f64 [0,4194304] 0 2024-04-20T11:09:12.172 INFO:tasks.workunit.client.0.smithi117.stdout:9/674: dread d4/d5/da/f62 [0,4194304] 0 2024-04-20T11:09:12.172 INFO:tasks.workunit.client.0.smithi117.stdout:0/706: rename d3/d7/d20/l2f to d3/d7/d16/le4 0 2024-04-20T11:09:12.177 INFO:tasks.workunit.client.0.smithi117.stdout:2/735: dwrite d1/d15/d1d/f50 [0,4194304] 0 2024-04-20T11:09:12.179 INFO:tasks.workunit.client.0.smithi117.stdout:9/675: creat d4/de/d4b/fe5 x:0 0 0 2024-04-20T11:09:12.183 INFO:tasks.workunit.client.0.smithi117.stdout:2/736: fdatasync d1/d4/d83/d29/d3f/f7d 0 2024-04-20T11:09:12.183 INFO:tasks.workunit.client.0.smithi117.stdout:9/676: rmdir d4/d4a 39 2024-04-20T11:09:12.187 INFO:tasks.workunit.client.0.smithi117.stdout:9/677: chown d4/de/d7b/d96/db0 4 1 2024-04-20T11:09:12.188 INFO:tasks.workunit.client.0.smithi117.stdout:1/551: dwrite d6/d1f/d30/d49/d69/f82 [0,4194304] 0 2024-04-20T11:09:12.193 INFO:tasks.workunit.client.0.smithi117.stdout:2/737: dread d1/da5/fd0 [0,4194304] 0 2024-04-20T11:09:12.193 INFO:tasks.workunit.client.0.smithi117.stdout:9/678: mknod d4/d5/d3c/ce6 0 2024-04-20T11:09:12.195 INFO:tasks.workunit.client.0.smithi117.stdout:5/870: dwrite de/f74 [0,4194304] 0 2024-04-20T11:09:12.198 INFO:tasks.workunit.client.0.smithi117.stdout:5/871: fsync de/ff7 0 2024-04-20T11:09:12.206 INFO:tasks.workunit.client.0.smithi117.stdout:5/872: truncate de/f11 2274912 0 2024-04-20T11:09:12.208 INFO:tasks.workunit.client.0.smithi117.stdout:2/738: dread d1/d4/dd/d17/f1f [4194304,4194304] 0 2024-04-20T11:09:12.210 INFO:tasks.workunit.client.0.smithi117.stdout:5/873: creat de/db7/d12b/f131 x:0 0 0 2024-04-20T11:09:12.220 INFO:tasks.workunit.client.0.smithi117.stdout:5/874: truncate de/d41/d4b/f5a 2862767 0 2024-04-20T11:09:12.221 INFO:tasks.workunit.client.0.smithi117.stdout:2/739: link d1/d4/c41 d1/d4/dd/d3e/ceb 0 2024-04-20T11:09:12.221 INFO:tasks.workunit.client.0.smithi117.stdout:1/552: dwrite d6/d21/d95/fa9 [0,4194304] 0 2024-04-20T11:09:12.222 INFO:tasks.workunit.client.0.smithi117.stdout:9/679: dwrite d4/d5/d3c/ddf/d31/fac [0,4194304] 0 2024-04-20T11:09:12.227 INFO:tasks.workunit.client.0.smithi117.stdout:1/553: dread d6/d16/d3a/d5a/d5c/fa0 [0,4194304] 0 2024-04-20T11:09:12.232 INFO:tasks.workunit.client.0.smithi117.stdout:1/554: readlink d6/d1f/d30/l4c 0 2024-04-20T11:09:12.233 INFO:tasks.workunit.client.0.smithi117.stdout:2/740: creat d1/d4/fec x:0 0 0 2024-04-20T11:09:12.234 INFO:tasks.workunit.client.0.smithi117.stdout:5/875: link de/d41/d47/d55/d56/dc8/dd8/l105 de/dd0/d10e/d4e/da1/de7/d123/db9/dda/l132 0 2024-04-20T11:09:12.239 INFO:tasks.workunit.client.0.smithi117.stdout:9/680: symlink d4/d4a/d63/d98/dc0/dcb/le7 0 2024-04-20T11:09:12.241 INFO:tasks.workunit.client.0.smithi117.stdout:5/876: rmdir de/d21/df5/d57/dd5/d11d 39 2024-04-20T11:09:12.245 INFO:tasks.workunit.client.0.smithi117.stdout:5/877: dread - de/d21/df5/d57/dd5/dce/f110 zero size 2024-04-20T11:09:12.251 INFO:tasks.workunit.client.0.smithi117.stdout:5/878: creat de/d41/d47/f133 x:0 0 0 2024-04-20T11:09:12.252 INFO:tasks.workunit.client.0.smithi117.stdout:3/584: sync 2024-04-20T11:09:12.258 INFO:tasks.workunit.client.0.smithi117.stdout:3/585: symlink d0/d5/d21/d75/dc1/lc7 0 2024-04-20T11:09:12.259 INFO:tasks.workunit.client.0.smithi117.stdout:1/555: dwrite d6/d21/d95/fa9 [0,4194304] 0 2024-04-20T11:09:12.262 INFO:tasks.workunit.client.0.smithi117.stdout:2/741: dwrite d1/d4/dd/d17/d7a/d49/f54 [0,4194304] 0 2024-04-20T11:09:12.272 INFO:tasks.workunit.client.0.smithi117.stdout:3/586: creat d0/d5/fc8 x:0 0 0 2024-04-20T11:09:12.272 INFO:tasks.workunit.client.0.smithi117.stdout:5/879: getdents de/dd0/d10e/d4e/da1/de7 0 2024-04-20T11:09:12.277 INFO:tasks.workunit.client.0.smithi117.stdout:1/556: truncate d6/d1f/f2b 1558831 0 2024-04-20T11:09:12.357 INFO:tasks.workunit.client.0.smithi117.stdout:3/587: unlink d0/d5/d7/dd/f59 0 2024-04-20T11:09:12.358 INFO:tasks.workunit.client.0.smithi117.stdout:1/557: unlink d6/d21/l29 0 2024-04-20T11:09:12.360 INFO:tasks.workunit.client.0.smithi117.stdout:9/681: truncate d4/de/d7b/d96/db0/f77 2576028 0 2024-04-20T11:09:12.361 INFO:tasks.workunit.client.0.smithi117.stdout:3/588: chown d0/d5/de 224 1 2024-04-20T11:09:12.366 INFO:tasks.workunit.client.0.smithi117.stdout:1/558: mknod d6/d35/d63/cba 0 2024-04-20T11:09:12.367 INFO:tasks.workunit.client.0.smithi117.stdout:2/742: dwrite d1/d4/dd/d17/d7a/d49/fcc [0,4194304] 0 2024-04-20T11:09:12.370 INFO:tasks.workunit.client.0.smithi117.stdout:3/589: mkdir d0/d5/d21/d94/d9a/dc9 0 2024-04-20T11:09:12.371 INFO:tasks.workunit.client.0.smithi117.stdout:5/880: dread de/d21/df5/fbe [0,4194304] 0 2024-04-20T11:09:12.373 INFO:tasks.workunit.client.0.smithi117.stdout:1/559: dread d6/d35/f44 [0,4194304] 0 2024-04-20T11:09:12.375 INFO:tasks.workunit.client.0.smithi117.stdout:9/682: rename d4/de/d4b to d4/d4a/d5e/de8 0 2024-04-20T11:09:12.382 INFO:tasks.workunit.client.0.smithi117.stdout:9/683: creat d4/d68/fe9 x:0 0 0 2024-04-20T11:09:12.384 INFO:tasks.workunit.client.0.smithi117.stdout:3/590: creat d0/d5/d21/d94/d9a/dc9/fca x:0 0 0 2024-04-20T11:09:12.386 INFO:tasks.workunit.client.0.smithi117.stdout:5/881: symlink de/d21/df5/d57/dd5/d11d/l134 0 2024-04-20T11:09:12.389 INFO:tasks.workunit.client.0.smithi117.stdout:1/560: creat d6/d16/fbb x:0 0 0 2024-04-20T11:09:12.390 INFO:tasks.workunit.client.0.smithi117.stdout:3/591: creat d0/d5/d21/d88/d85/fcb x:0 0 0 2024-04-20T11:09:12.394 INFO:tasks.workunit.client.0.smithi117.stdout:5/882: unlink de/d41/d47/l53 0 2024-04-20T11:09:12.395 INFO:tasks.workunit.client.0.smithi117.stdout:1/561: creat d6/d16/d3a/da8/fbc x:0 0 0 2024-04-20T11:09:12.396 INFO:tasks.workunit.client.0.smithi117.stdout:3/592: mknod d0/d5/d21/d75/ccc 0 2024-04-20T11:09:12.397 INFO:tasks.workunit.client.0.smithi117.stdout:5/883: read - de/d16/fd1 zero size 2024-04-20T11:09:12.406 INFO:tasks.workunit.client.0.smithi117.stdout:1/562: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/fbd x:0 0 0 2024-04-20T11:09:12.407 INFO:tasks.workunit.client.0.smithi117.stdout:2/743: dwrite d1/d4/dd/f77 [0,4194304] 0 2024-04-20T11:09:12.411 INFO:tasks.workunit.client.0.smithi117.stdout:2/744: fsync d1/f3 0 2024-04-20T11:09:12.422 INFO:tasks.workunit.client.0.smithi117.stdout:6/713: sync 2024-04-20T11:09:12.426 INFO:tasks.workunit.client.0.smithi117.stdout:2/745: creat d1/d20/fed x:0 0 0 2024-04-20T11:09:12.426 INFO:tasks.workunit.client.0.smithi117.stdout:5/884: dread de/d41/dc2/fd3 [0,4194304] 0 2024-04-20T11:09:12.428 INFO:tasks.workunit.client.0.smithi117.stdout:3/593: dwrite d0/d5/d21/f28 [0,4194304] 0 2024-04-20T11:09:12.429 INFO:tasks.workunit.client.0.smithi117.stdout:6/714: mknod d3/df/d39/daa/ce6 0 2024-04-20T11:09:12.432 INFO:tasks.workunit.client.0.smithi117.stdout:9/684: dread d4/d4a/d5e/de8/f6d [0,4194304] 0 2024-04-20T11:09:12.435 INFO:tasks.workunit.client.0.smithi117.stdout:3/594: read - d0/d5/d21/d94/fb1 zero size 2024-04-20T11:09:12.436 INFO:tasks.workunit.client.0.smithi117.stdout:2/746: fsync d1/d4/d83/f3a 0 2024-04-20T11:09:12.439 INFO:tasks.workunit.client.0.smithi117.stdout:3/595: read d0/d5/d11/f74 [2485834,49383] 0 2024-04-20T11:09:12.439 INFO:tasks.workunit.client.0.smithi117.stdout:6/715: stat d3/d5a/c78 0 2024-04-20T11:09:12.441 INFO:tasks.workunit.client.0.smithi117.stdout:9/685: write d4/d5/d57/d82/da5/fbe [350272,44904] 0 2024-04-20T11:09:12.447 INFO:tasks.workunit.client.0.smithi117.stdout:1/563: dwrite d6/f2e [0,4194304] 0 2024-04-20T11:09:12.447 INFO:tasks.workunit.client.0.smithi117.stdout:2/747: creat d1/d4/d83/d29/d2e/d53/d58/fee x:0 0 0 2024-04-20T11:09:12.449 INFO:tasks.workunit.client.0.smithi117.stdout:5/885: write de/d16/fc0 [117140,86100] 0 2024-04-20T11:09:12.452 INFO:tasks.workunit.client.0.smithi117.stdout:6/716: symlink d3/d4/d2e/d4e/d71/db7/le7 0 2024-04-20T11:09:12.457 INFO:tasks.workunit.client.0.smithi117.stdout:1/564: write d6/d1f/d30/f74 [1523353,104522] 0 2024-04-20T11:09:12.458 INFO:tasks.workunit.client.0.smithi117.stdout:6/717: unlink d3/d5a/d74/db3/f6c 0 2024-04-20T11:09:12.460 INFO:tasks.workunit.client.0.smithi117.stdout:1/565: readlink d6/d1f/d30/d49/lab 0 2024-04-20T11:09:12.463 INFO:tasks.workunit.client.0.smithi117.stdout:3/596: dread d0/d5/de/d4e/f9e [4194304,4194304] 0 2024-04-20T11:09:12.464 INFO:tasks.workunit.client.0.smithi117.stdout:5/886: rename de/d16/f127 to de/d21/d10d/f135 0 2024-04-20T11:09:12.470 INFO:tasks.workunit.client.0.smithi117.stdout:5/887: unlink de/d41/f9b 0 2024-04-20T11:09:12.471 INFO:tasks.workunit.client.0.smithi117.stdout:6/718: mknod d3/d3a/ce8 0 2024-04-20T11:09:12.473 INFO:tasks.workunit.client.0.smithi117.stdout:5/888: chown de/d16/l38 322 1 2024-04-20T11:09:12.475 INFO:tasks.workunit.client.0.smithi117.stdout:3/597: creat d0/d5/d21/d75/dc1/fcd x:0 0 0 2024-04-20T11:09:12.478 INFO:tasks.workunit.client.0.smithi117.stdout:6/719: dread - d3/d4/d2e/d37/d8e/f9d zero size 2024-04-20T11:09:12.480 INFO:tasks.workunit.client.0.smithi117.stdout:6/720: fdatasync d3/d4/f1b 0 2024-04-20T11:09:12.481 INFO:tasks.workunit.client.0.smithi117.stdout:3/598: write d0/d5/d21/d88/d29/d66/d63/fa4 [755320,70046] 0 2024-04-20T11:09:12.482 INFO:tasks.workunit.client.0.smithi117.stdout:5/889: mknod de/dd0/d10e/d4e/da1/de7/d123/c136 0 2024-04-20T11:09:12.484 INFO:tasks.workunit.client.0.smithi117.stdout:3/599: write d0/d5/d7/dd/f3a [2354533,57504] 0 2024-04-20T11:09:12.490 INFO:tasks.workunit.client.0.smithi117.stdout:1/566: getdents d6/d1f/d30/d49/d64 0 2024-04-20T11:09:12.491 INFO:tasks.workunit.client.0.smithi117.stdout:6/721: mknod d3/d4/d2e/d4e/d71/db7/ce9 0 2024-04-20T11:09:12.492 INFO:tasks.workunit.client.0.smithi117.stdout:5/890: unlink de/dd0/d10e/d4e/da1/de7/d123/d77/fa6 0 2024-04-20T11:09:12.492 INFO:tasks.workunit.client.0.smithi117.stdout:1/567: chown d6/d16/d3a/d5a 1 1 2024-04-20T11:09:12.494 INFO:tasks.workunit.client.0.smithi117.stdout:3/600: mknod d0/d5/d21/d88/d29/d66/d63/cce 0 2024-04-20T11:09:12.495 INFO:tasks.workunit.client.0.smithi117.stdout:5/891: stat de/d21/df5/c37 0 2024-04-20T11:09:12.498 INFO:tasks.workunit.client.0.smithi117.stdout:5/892: read de/f8e [162260,82837] 0 2024-04-20T11:09:12.504 INFO:tasks.workunit.client.0.smithi117.stdout:2/748: dread d1/d4/d83/fc3 [0,4194304] 0 2024-04-20T11:09:12.510 INFO:tasks.workunit.client.0.smithi117.stdout:3/601: mknod d0/d5/d21/d88/d29/d66/d63/dad/ccf 0 2024-04-20T11:09:12.515 INFO:tasks.workunit.client.0.smithi117.stdout:5/893: rmdir de/d21/df5/d57/dd5/dfe 0 2024-04-20T11:09:12.516 INFO:tasks.workunit.client.0.smithi117.stdout:2/749: symlink d1/d4/d83/d29/d62/da6/lef 0 2024-04-20T11:09:12.519 INFO:tasks.workunit.client.0.smithi117.stdout:5/894: symlink de/d5b/l137 0 2024-04-20T11:09:12.522 INFO:tasks.workunit.client.0.smithi117.stdout:5/895: write de/dd0/d10e/da7/feb [2761412,127269] 0 2024-04-20T11:09:12.523 INFO:tasks.workunit.client.0.smithi117.stdout:6/722: write d3/d4/d2e/d4e/d71/db7/f87 [656125,59515] 0 2024-04-20T11:09:12.527 INFO:tasks.workunit.client.0.smithi117.stdout:6/723: dread - d3/df/fc8 zero size 2024-04-20T11:09:12.528 INFO:tasks.workunit.client.0.smithi117.stdout:5/896: symlink de/d41/d4b/l138 0 2024-04-20T11:09:12.535 INFO:tasks.workunit.client.0.smithi117.stdout:7/618: sync 2024-04-20T11:09:12.535 INFO:tasks.workunit.client.0.smithi117.stdout:6/724: mknod d3/d5a/d74/db3/dbc/d9b/cea 0 2024-04-20T11:09:12.545 INFO:tasks.workunit.client.0.smithi117.stdout:1/568: dread d6/d1f/d30/d49/d69/d6e/d6f/d73/f88 [0,4194304] 0 2024-04-20T11:09:12.551 INFO:tasks.workunit.client.0.smithi117.stdout:5/897: dwrite de/d41/d47/f125 [0,4194304] 0 2024-04-20T11:09:12.553 INFO:tasks.workunit.client.0.smithi117.stdout:3/602: dread d0/d5/d21/d88/f44 [4194304,4194304] 0 2024-04-20T11:09:12.553 INFO:tasks.workunit.client.0.smithi117.stdout:1/569: creat d6/d16/d3a/d5a/d5c/d61/d98/fbe x:0 0 0 2024-04-20T11:09:12.562 INFO:tasks.workunit.client.0.smithi117.stdout:7/619: dwrite d1/d1d/d42/fac [0,4194304] 0 2024-04-20T11:09:12.562 INFO:tasks.workunit.client.0.smithi117.stdout:1/570: symlink d6/d1f/lbf 0 2024-04-20T11:09:12.569 INFO:tasks.workunit.client.0.smithi117.stdout:7/620: dread - d1/d4/d3d/d93/fba zero size 2024-04-20T11:09:12.569 INFO:tasks.workunit.client.0.smithi117.stdout:5/898: fdatasync de/d41/d47/d55/f129 0 2024-04-20T11:09:12.580 INFO:tasks.workunit.client.0.smithi117.stdout:1/571: unlink d6/d16/d3a/d5a/d5c/c5e 0 2024-04-20T11:09:12.585 INFO:tasks.workunit.client.0.smithi117.stdout:6/725: dread d3/fc [0,4194304] 0 2024-04-20T11:09:12.592 INFO:tasks.workunit.client.0.smithi117.stdout:3/603: truncate d0/d5/d21/d88/d36/f33 2082050 0 2024-04-20T11:09:12.594 INFO:tasks.workunit.client.0.smithi117.stdout:1/572: mknod d6/d1f/d30/d49/d69/d6e/d6f/cc0 0 2024-04-20T11:09:12.597 INFO:tasks.workunit.client.0.smithi117.stdout:7/621: dwrite d1/d4/d3d/f5a [4194304,4194304] 0 2024-04-20T11:09:12.601 INFO:tasks.workunit.client.0.smithi117.stdout:3/604: mkdir d0/d5/d21/d88/d29/d66/dbe/dd0 0 2024-04-20T11:09:12.602 INFO:tasks.workunit.client.0.smithi117.stdout:5/899: write de/dd0/d10e/d4e/ff2 [40243,64278] 0 2024-04-20T11:09:12.604 INFO:tasks.workunit.client.0.smithi117.stdout:5/900: chown de/d41/f52 330 1 2024-04-20T11:09:12.604 INFO:tasks.workunit.client.0.smithi117.stdout:3/605: read - d0/d5/d21/f9c zero size 2024-04-20T11:09:12.608 INFO:tasks.workunit.client.0.smithi117.stdout:6/726: dread d3/d3a/f6b [0,4194304] 0 2024-04-20T11:09:12.612 INFO:tasks.workunit.client.0.smithi117.stdout:3/606: truncate d0/d5/d21/d94/d9a/dc9/fca 765781 0 2024-04-20T11:09:12.614 INFO:tasks.workunit.client.0.smithi117.stdout:3/607: dread - d0/d5/d21/d88/d29/d66/d63/fc4 zero size 2024-04-20T11:09:12.621 INFO:tasks.workunit.client.0.smithi117.stdout:6/727: mknod d3/d4/d2e/dc0/ceb 0 2024-04-20T11:09:12.621 INFO:tasks.workunit.client.0.smithi117.stdout:3/608: truncate d0/d5/d21/d88/d29/db3/fb6 332148 0 2024-04-20T11:09:12.624 INFO:tasks.workunit.client.0.smithi117.stdout:6/728: read - d3/d4/d2e/d3f/fc1 zero size 2024-04-20T11:09:12.626 INFO:tasks.workunit.client.0.smithi117.stdout:6/729: write d3/d50/fb8 [944185,127934] 0 2024-04-20T11:09:12.629 INFO:tasks.workunit.client.0.smithi117.stdout:6/730: write d3/d3a/d7f/fa4 [4880329,330] 0 2024-04-20T11:09:12.641 INFO:tasks.workunit.client.0.smithi117.stdout:7/622: dwrite d1/d43/f73 [0,4194304] 0 2024-04-20T11:09:12.642 INFO:tasks.workunit.client.0.smithi117.stdout:7/623: readlink d1/lb8 0 2024-04-20T11:09:12.651 INFO:tasks.workunit.client.0.smithi117.stdout:7/624: dread d1/d4/d6/d54/d63/f85 [0,4194304] 0 2024-04-20T11:09:12.654 INFO:tasks.workunit.client.0.smithi117.stdout:3/609: write d0/d5/d21/d88/d36/f33 [2116531,6930] 0 2024-04-20T11:09:12.660 INFO:tasks.workunit.client.0.smithi117.stdout:6/731: dwrite d3/d4/d2e/d3f/fc1 [0,4194304] 0 2024-04-20T11:09:12.669 INFO:tasks.workunit.client.0.smithi117.stdout:6/732: truncate d3/df/d39/daa/fd1 4680437 0 2024-04-20T11:09:12.672 INFO:tasks.workunit.client.0.smithi117.stdout:0/707: sync 2024-04-20T11:09:12.672 INFO:tasks.workunit.client.0.smithi117.stdout:9/686: sync 2024-04-20T11:09:12.672 INFO:tasks.workunit.client.0.smithi117.stdout:8/528: sync 2024-04-20T11:09:12.672 INFO:tasks.workunit.client.0.smithi117.stdout:4/642: sync 2024-04-20T11:09:12.677 INFO:tasks.workunit.client.0.smithi117.stdout:8/529: chown d0/d8/d39/d40/f51 2390 1 2024-04-20T11:09:12.680 INFO:tasks.workunit.client.0.smithi117.stdout:7/625: dwrite d1/d4/d6/d2b/f81 [0,4194304] 0 2024-04-20T11:09:12.681 INFO:tasks.workunit.client.0.smithi117.stdout:8/530: write d0/d9/d3a/f6a [4650625,119367] 0 2024-04-20T11:09:12.681 INFO:tasks.workunit.client.0.smithi117.stdout:0/708: readlink d3/d5/d74/lb5 0 2024-04-20T11:09:12.698 INFO:tasks.workunit.client.0.smithi117.stdout:7/626: creat d1/d1d/d4e/d72/fc6 x:0 0 0 2024-04-20T11:09:12.700 INFO:tasks.workunit.client.0.smithi117.stdout:6/733: fsync d3/d4/d2e/d3f/fc1 0 2024-04-20T11:09:12.701 INFO:tasks.workunit.client.0.smithi117.stdout:3/610: dwrite d0/d5/f25 [0,4194304] 0 2024-04-20T11:09:12.716 INFO:tasks.workunit.client.0.smithi117.stdout:9/687: link d4/d4a/d5e/de8/f86 d4/d4a/d63/d98/fea 0 2024-04-20T11:09:12.716 INFO:tasks.workunit.client.0.smithi117.stdout:6/734: chown d3/f28 2014733772 1 2024-04-20T11:09:12.718 INFO:tasks.workunit.client.0.smithi117.stdout:9/688: fsync d4/d4a/d63/d98/fae 0 2024-04-20T11:09:12.722 INFO:tasks.workunit.client.0.smithi117.stdout:3/611: mkdir d0/d5/d21/d75/d89/dd1 0 2024-04-20T11:09:12.734 INFO:tasks.workunit.client.0.smithi117.stdout:4/643: rename d1/d9/d6b/c85 to d1/dd/d1a/d43/d4e/ce1 0 2024-04-20T11:09:12.737 INFO:tasks.workunit.client.0.smithi117.stdout:7/627: dread d1/f7 [0,4194304] 0 2024-04-20T11:09:12.737 INFO:tasks.workunit.client.0.smithi117.stdout:0/709: truncate d3/f3e 749739 0 2024-04-20T11:09:12.739 INFO:tasks.workunit.client.0.smithi117.stdout:0/710: stat d3/d7/f18 0 2024-04-20T11:09:12.742 INFO:tasks.workunit.client.0.smithi117.stdout:6/735: dwrite d3/df/d39/d99/f8c [4194304,4194304] 0 2024-04-20T11:09:12.743 INFO:tasks.workunit.client.0.smithi117.stdout:3/612: mkdir d0/d5/d21/d88/d29/dac/dd2 0 2024-04-20T11:09:12.747 INFO:tasks.workunit.client.0.smithi117.stdout:6/736: truncate d3/d5a/d74/f93 1828595 0 2024-04-20T11:09:12.748 INFO:tasks.workunit.client.0.smithi117.stdout:6/737: dread - d3/d94/fcc zero size 2024-04-20T11:09:12.751 INFO:tasks.workunit.client.0.smithi117.stdout:7/628: fsync d1/d1d/d4e/f9a 0 2024-04-20T11:09:12.757 INFO:tasks.workunit.client.0.smithi117.stdout:4/644: creat d1/dd/d1a/d2b/d86/fe2 x:0 0 0 2024-04-20T11:09:12.759 INFO:tasks.workunit.client.0.smithi117.stdout:6/738: dread d3/df/f31 [0,4194304] 0 2024-04-20T11:09:12.759 INFO:tasks.workunit.client.0.smithi117.stdout:4/645: fdatasync d1/dd/d1a/d43/f65 0 2024-04-20T11:09:12.759 INFO:tasks.workunit.client.0.smithi117.stdout:7/629: mknod d1/d43/d59/cc7 0 2024-04-20T11:09:12.763 INFO:tasks.workunit.client.0.smithi117.stdout:0/711: stat d3/d7/d20/d39/d9a/dd0/d22 0 2024-04-20T11:09:12.764 INFO:tasks.workunit.client.0.smithi117.stdout:9/689: truncate d4/d68/fb3 3493188 0 2024-04-20T11:09:12.764 INFO:tasks.workunit.client.0.smithi117.stdout:6/739: stat d3/l1e 0 2024-04-20T11:09:12.765 INFO:tasks.workunit.client.0.smithi117.stdout:7/630: unlink d1/d4/f97 0 2024-04-20T11:09:12.766 INFO:tasks.workunit.client.0.smithi117.stdout:0/712: write d3/d7/f18 [1206613,27239] 0 2024-04-20T11:09:12.767 INFO:tasks.workunit.client.0.smithi117.stdout:4/646: truncate d1/f81 1138613 0 2024-04-20T11:09:12.768 INFO:tasks.workunit.client.0.smithi117.stdout:0/713: fdatasync d3/dab/fac 0 2024-04-20T11:09:12.773 INFO:tasks.workunit.client.0.smithi117.stdout:4/647: stat d1/d9/d79/cc5 0 2024-04-20T11:09:12.775 INFO:tasks.workunit.client.0.smithi117.stdout:9/690: symlink d4/d5/da1/leb 0 2024-04-20T11:09:12.780 INFO:tasks.workunit.client.0.smithi117.stdout:7/631: creat d1/fc8 x:0 0 0 2024-04-20T11:09:12.784 INFO:tasks.workunit.client.0.smithi117.stdout:6/740: dwrite d3/d3a/d7f/f7a [4194304,4194304] 0 2024-04-20T11:09:12.790 INFO:tasks.workunit.client.0.smithi117.stdout:6/741: chown d3/d4/d2e/d3f/f42 1171 1 2024-04-20T11:09:12.791 INFO:tasks.workunit.client.0.smithi117.stdout:0/714: link d3/d7/d16/c28 d3/d7/d20/d39/d9a/dd0/d22/ce5 0 2024-04-20T11:09:12.791 INFO:tasks.workunit.client.0.smithi117.stdout:3/613: dread d0/f35 [0,4194304] 0 2024-04-20T11:09:12.796 INFO:tasks.workunit.client.0.smithi117.stdout:9/691: getdents d4/d4a/d63/d98/dc0/dcb 0 2024-04-20T11:09:12.797 INFO:tasks.workunit.client.0.smithi117.stdout:4/648: dwrite d1/fc3 [0,4194304] 0 2024-04-20T11:09:12.798 INFO:tasks.workunit.client.0.smithi117.stdout:6/742: truncate d3/d4/d2e/d3f/f42 3749297 0 2024-04-20T11:09:12.802 INFO:tasks.workunit.client.0.smithi117.stdout:4/649: chown d1/dd/d1a/d2b/d86/dda 13825 1 2024-04-20T11:09:12.804 INFO:tasks.workunit.client.0.smithi117.stdout:4/650: fsync d1/d9/fb0 0 2024-04-20T11:09:12.806 INFO:tasks.workunit.client.0.smithi117.stdout:4/651: chown d1/dd/d1a/fe0 75492015 1 2024-04-20T11:09:12.808 INFO:tasks.workunit.client.0.smithi117.stdout:4/652: rmdir d1/dd/d1a/d43/d9a 39 2024-04-20T11:09:12.810 INFO:tasks.workunit.client.0.smithi117.stdout:4/653: dread - d1/dd/d1a/f8b zero size 2024-04-20T11:09:12.812 INFO:tasks.workunit.client.0.smithi117.stdout:2/750: sync 2024-04-20T11:09:12.814 INFO:tasks.workunit.client.0.smithi117.stdout:2/751: dread - d1/d9b/fb4 zero size 2024-04-20T11:09:12.816 INFO:tasks.workunit.client.0.smithi117.stdout:4/654: mknod d1/dd/d1a/ce3 0 2024-04-20T11:09:12.818 INFO:tasks.workunit.client.0.smithi117.stdout:7/632: truncate d1/fa5 2062640 0 2024-04-20T11:09:12.820 INFO:tasks.workunit.client.0.smithi117.stdout:2/752: mkdir d1/d4/d83/d29/d2e/d53/df0 0 2024-04-20T11:09:12.827 INFO:tasks.workunit.client.0.smithi117.stdout:9/692: write d4/d4a/d63/f76 [3254892,91100] 0 2024-04-20T11:09:12.830 INFO:tasks.workunit.client.0.smithi117.stdout:0/715: truncate d3/dab/d54/fa3 974199 0 2024-04-20T11:09:12.836 INFO:tasks.workunit.client.0.smithi117.stdout:3/614: dwrite d0/d5/d21/d77/f8a [0,4194304] 0 2024-04-20T11:09:12.847 INFO:tasks.workunit.client.0.smithi117.stdout:7/633: truncate d1/d43/fa0 875476 0 2024-04-20T11:09:12.850 INFO:tasks.workunit.client.0.smithi117.stdout:6/743: dread d3/f67 [0,4194304] 0 2024-04-20T11:09:12.850 INFO:tasks.workunit.client.0.smithi117.stdout:3/615: mknod d0/d5/d21/d75/d89/dd1/cd3 0 2024-04-20T11:09:12.851 INFO:tasks.workunit.client.0.smithi117.stdout:7/634: chown d1/d1d/d42/d2d/c4a 102 1 2024-04-20T11:09:12.864 INFO:tasks.workunit.client.0.smithi117.stdout:6/744: creat d3/df/d39/fec x:0 0 0 2024-04-20T11:09:12.864 INFO:tasks.workunit.client.0.smithi117.stdout:3/616: creat d0/d5/de/fd4 x:0 0 0 2024-04-20T11:09:12.866 INFO:tasks.workunit.client.0.smithi117.stdout:2/753: dwrite d1/d4/dd/d17/f1e [4194304,4194304] 0 2024-04-20T11:09:12.871 INFO:tasks.workunit.client.0.smithi117.stdout:6/745: truncate d3/d4/f46 5185197 0 2024-04-20T11:09:12.874 INFO:tasks.workunit.client.0.smithi117.stdout:0/716: dwrite d3/d7/d20/d39/d9a/dd0/d22/fb8 [0,4194304] 0 2024-04-20T11:09:12.877 INFO:tasks.workunit.client.0.smithi117.stdout:4/655: dread d1/dd/d1a/d2b/d86/fb4 [0,4194304] 0 2024-04-20T11:09:12.892 INFO:tasks.workunit.client.0.smithi117.stdout:3/617: rmdir d0/d5/d21/d77 39 2024-04-20T11:09:12.901 INFO:tasks.workunit.client.0.smithi117.stdout:6/746: dwrite d3/d4/d2e/d3f/f77 [0,4194304] 0 2024-04-20T11:09:12.905 INFO:tasks.workunit.client.0.smithi117.stdout:3/618: rename d0/d5/d11/f74 to d0/d5/d73/fd5 0 2024-04-20T11:09:12.909 INFO:tasks.workunit.client.0.smithi117.stdout:0/717: dwrite d3/d7/d20/d39/d9a/dd0/d6d/fcf [0,4194304] 0 2024-04-20T11:09:12.911 INFO:tasks.workunit.client.0.smithi117.stdout:0/718: readlink d3/d7/d20/d39/d9a/dd0/d22/d85/ld8 0 2024-04-20T11:09:12.914 INFO:tasks.workunit.client.0.smithi117.stdout:2/754: getdents d1/d4/dd/d17/d7a 0 2024-04-20T11:09:12.915 INFO:tasks.workunit.client.0.smithi117.stdout:4/656: getdents d1/d9/d6b 0 2024-04-20T11:09:12.919 INFO:tasks.workunit.client.0.smithi117.stdout:3/619: chown d0/d5/d21/f47 8338 1 2024-04-20T11:09:12.922 INFO:tasks.workunit.client.0.smithi117.stdout:6/747: symlink d3/d5a/d74/db3/dd0/led 0 2024-04-20T11:09:12.925 INFO:tasks.workunit.client.0.smithi117.stdout:2/755: creat d1/d4/d83/d29/d2e/d53/d58/ff1 x:0 0 0 2024-04-20T11:09:12.930 INFO:tasks.workunit.client.0.smithi117.stdout:6/748: mkdir d3/df/d39/daa/dee 0 2024-04-20T11:09:12.930 INFO:tasks.workunit.client.0.smithi117.stdout:2/756: chown d1/d4/d83/ldb 2259 1 2024-04-20T11:09:12.931 INFO:tasks.workunit.client.0.smithi117.stdout:4/657: symlink d1/d45/dcd/le4 0 2024-04-20T11:09:12.934 INFO:tasks.workunit.client.0.smithi117.stdout:2/757: unlink d1/d4/d83/d29/d2e/d53/cd6 0 2024-04-20T11:09:12.936 INFO:tasks.workunit.client.0.smithi117.stdout:6/749: unlink d3/d5a/d74/db3/dbc/d9b/ce3 0 2024-04-20T11:09:12.940 INFO:tasks.workunit.client.0.smithi117.stdout:4/658: creat d1/dd/d1a/d2b/d86/fe5 x:0 0 0 2024-04-20T11:09:12.941 INFO:tasks.workunit.client.0.smithi117.stdout:0/719: dwrite d3/d5/d74/fa9 [0,4194304] 0 2024-04-20T11:09:12.948 INFO:tasks.workunit.client.0.smithi117.stdout:2/758: rename d1/d4/dd/d3e to d1/da5/df2 0 2024-04-20T11:09:12.948 INFO:tasks.workunit.client.0.smithi117.stdout:6/750: creat d3/d4/d2e/d37/fef x:0 0 0 2024-04-20T11:09:12.948 INFO:tasks.workunit.client.0.smithi117.stdout:0/720: stat d3/d7/d20/d39/d9a/dd0/d22/d85/ddb 0 2024-04-20T11:09:12.954 INFO:tasks.workunit.client.0.smithi117.stdout:2/759: symlink d1/d4/d83/d29/d2e/d52/lf3 0 2024-04-20T11:09:12.957 INFO:tasks.workunit.client.0.smithi117.stdout:0/721: symlink d3/d7/d20/d39/d3a/le6 0 2024-04-20T11:09:12.959 INFO:tasks.workunit.client.0.smithi117.stdout:2/760: write d1/d4/dd/d17/d3b/f6c [51,8003] 0 2024-04-20T11:09:12.965 INFO:tasks.workunit.client.0.smithi117.stdout:0/722: creat d3/d7/d16/fe7 x:0 0 0 2024-04-20T11:09:12.967 INFO:tasks.workunit.client.0.smithi117.stdout:4/659: dwrite d1/d45/d4a/d99/fa0 [0,4194304] 0 2024-04-20T11:09:12.974 INFO:tasks.workunit.client.0.smithi117.stdout:1/573: sync 2024-04-20T11:09:12.974 INFO:tasks.workunit.client.0.smithi117.stdout:5/901: sync 2024-04-20T11:09:12.974 INFO:tasks.workunit.client.0.smithi117.stdout:8/531: sync 2024-04-20T11:09:12.974 INFO:tasks.workunit.client.0.smithi117.stdout:4/660: write d1/f7 [2178146,88595] 0 2024-04-20T11:09:12.976 INFO:tasks.workunit.client.0.smithi117.stdout:7/635: dwrite d1/d4/d6/d2b/d64/f8c [0,4194304] 0 2024-04-20T11:09:12.986 INFO:tasks.workunit.client.0.smithi117.stdout:6/751: dwrite d3/d4/d2e/d3f/fc9 [0,4194304] 0 2024-04-20T11:09:12.986 INFO:tasks.workunit.client.0.smithi117.stdout:2/761: dread d1/d4/d83/d29/d62/f9a [0,4194304] 0 2024-04-20T11:09:12.990 INFO:tasks.workunit.client.0.smithi117.stdout:4/661: readlink d1/dd/d1a/l2f 0 2024-04-20T11:09:12.992 INFO:tasks.workunit.client.0.smithi117.stdout:4/662: chown d1/dd 5389067 1 2024-04-20T11:09:12.994 INFO:tasks.workunit.client.0.smithi117.stdout:8/532: mknod d0/dc/d49/d5b/cb2 0 2024-04-20T11:09:12.995 INFO:tasks.workunit.client.0.smithi117.stdout:2/762: stat d1/d4/d83/d29/d2e/d53/df0 0 2024-04-20T11:09:13.003 INFO:tasks.workunit.client.0.smithi117.stdout:6/752: rename d3/d4/d2e/d4e/d71 to d3/d5a/d74/db3/df0 0 2024-04-20T11:09:13.006 INFO:tasks.workunit.client.0.smithi117.stdout:8/533: fsync d0/dc/d49/d5b/f67 0 2024-04-20T11:09:13.007 INFO:tasks.workunit.client.0.smithi117.stdout:6/753: write d3/df/fce [657397,17545] 0 2024-04-20T11:09:13.009 INFO:tasks.workunit.client.0.smithi117.stdout:4/663: fsync d1/d9/d79/f8c 0 2024-04-20T11:09:13.012 INFO:tasks.workunit.client.0.smithi117.stdout:0/723: write d3/d5/d74/fa9 [5100897,14177] 0 2024-04-20T11:09:13.012 INFO:tasks.workunit.client.0.smithi117.stdout:6/754: write d3/d4/f6e [2949636,30632] 0 2024-04-20T11:09:13.015 INFO:tasks.workunit.client.0.smithi117.stdout:5/902: write de/d41/d47/d55/d56/f78 [1058683,101592] 0 2024-04-20T11:09:13.024 INFO:tasks.workunit.client.0.smithi117.stdout:1/574: truncate d6/f2e 28967 0 2024-04-20T11:09:13.026 INFO:tasks.workunit.client.0.smithi117.stdout:1/575: read - d6/d35/fb3 zero size 2024-04-20T11:09:13.026 INFO:tasks.workunit.client.0.smithi117.stdout:4/664: creat d1/dd/d1a/d2b/fe6 x:0 0 0 2024-04-20T11:09:13.030 INFO:tasks.workunit.client.0.smithi117.stdout:7/636: dread d1/d1d/d42/d2d/f5f [0,4194304] 0 2024-04-20T11:09:13.032 INFO:tasks.workunit.client.0.smithi117.stdout:6/755: read d3/d4/d2e/d3f/f8b [3643210,17786] 0 2024-04-20T11:09:13.034 INFO:tasks.workunit.client.0.smithi117.stdout:6/756: dread - d3/d4/d2e/d37/fdb zero size 2024-04-20T11:09:13.036 INFO:tasks.workunit.client.0.smithi117.stdout:4/665: fsync d1/f67 0 2024-04-20T11:09:13.037 INFO:tasks.workunit.client.0.smithi117.stdout:2/763: write d1/d15/d1d/f31 [1131590,78206] 0 2024-04-20T11:09:13.038 INFO:tasks.workunit.client.0.smithi117.stdout:5/903: read de/f19 [3910587,104536] 0 2024-04-20T11:09:13.041 INFO:tasks.workunit.client.0.smithi117.stdout:1/576: mknod d6/d16/d3a/d5a/d5c/d61/d91/cc1 0 2024-04-20T11:09:13.048 INFO:tasks.workunit.client.0.smithi117.stdout:4/666: mkdir d1/d9/d6b/dca/de7 0 2024-04-20T11:09:13.048 INFO:tasks.workunit.client.0.smithi117.stdout:6/757: rename d3/df/d39/le4 to d3/d94/lf1 0 2024-04-20T11:09:13.048 INFO:tasks.workunit.client.0.smithi117.stdout:7/637: write d1/f7 [5609174,105786] 0 2024-04-20T11:09:13.048 INFO:tasks.workunit.client.0.smithi117.stdout:8/534: link d0/c61 d0/cb3 0 2024-04-20T11:09:13.051 INFO:tasks.workunit.client.0.smithi117.stdout:2/764: dread d1/da5/fd0 [0,4194304] 0 2024-04-20T11:09:13.052 INFO:tasks.workunit.client.0.smithi117.stdout:5/904: creat de/dd0/d10e/d4e/da1/de7/f139 x:0 0 0 2024-04-20T11:09:13.054 INFO:tasks.workunit.client.0.smithi117.stdout:6/758: chown d3/d50/d8a 5756 1 2024-04-20T11:09:13.057 INFO:tasks.workunit.client.0.smithi117.stdout:6/759: stat d3/d4/lbf 0 2024-04-20T11:09:13.057 INFO:tasks.workunit.client.0.smithi117.stdout:1/577: mkdir d6/d1f/d30/d49/d69/d6e/d6f/d73/dc2 0 2024-04-20T11:09:13.059 INFO:tasks.workunit.client.0.smithi117.stdout:8/535: creat d0/d8/d39/d40/d78/fb4 x:0 0 0 2024-04-20T11:09:13.062 INFO:tasks.workunit.client.0.smithi117.stdout:5/905: mknod de/d21/df5/c13a 0 2024-04-20T11:09:13.065 INFO:tasks.workunit.client.0.smithi117.stdout:4/667: truncate d1/d45/d4a/d59/fd6 188461 0 2024-04-20T11:09:13.070 INFO:tasks.workunit.client.0.smithi117.stdout:7/638: link d1/d43/f61 d1/d43/d59/fc9 0 2024-04-20T11:09:13.073 INFO:tasks.workunit.client.0.smithi117.stdout:0/724: dwrite d3/d7/d20/d39/d9a/dd0/d22/f6f [0,4194304] 0 2024-04-20T11:09:13.073 INFO:tasks.workunit.client.0.smithi117.stdout:8/536: creat d0/dc/d49/d63/fb5 x:0 0 0 2024-04-20T11:09:13.074 INFO:tasks.workunit.client.0.smithi117.stdout:5/906: fdatasync de/d21/df5/f8d 0 2024-04-20T11:09:13.075 INFO:tasks.workunit.client.0.smithi117.stdout:9/693: sync 2024-04-20T11:09:13.079 INFO:tasks.workunit.client.0.smithi117.stdout:7/639: fsync d1/d43/f7b 0 2024-04-20T11:09:13.083 INFO:tasks.workunit.client.0.smithi117.stdout:8/537: mknod d0/dc/cb6 0 2024-04-20T11:09:13.084 INFO:tasks.workunit.client.0.smithi117.stdout:0/725: dread - d3/d7/f96 zero size 2024-04-20T11:09:13.084 INFO:tasks.workunit.client.0.smithi117.stdout:9/694: mkdir d4/d5/d57/d82/dec 0 2024-04-20T11:09:13.085 INFO:tasks.workunit.client.0.smithi117.stdout:0/726: stat d3/dab 0 2024-04-20T11:09:13.088 INFO:tasks.workunit.client.0.smithi117.stdout:2/765: truncate d1/d4/dd/fc1 2756337 0 2024-04-20T11:09:13.091 INFO:tasks.workunit.client.0.smithi117.stdout:4/668: write d1/d9/f76 [341528,85050] 0 2024-04-20T11:09:13.096 INFO:tasks.workunit.client.0.smithi117.stdout:4/669: dread d1/dd/d52/f54 [0,4194304] 0 2024-04-20T11:09:13.097 INFO:tasks.workunit.client.0.smithi117.stdout:5/907: mkdir de/d16/d9d/db4/d130/d13b 0 2024-04-20T11:09:13.100 INFO:tasks.workunit.client.0.smithi117.stdout:5/908: stat de/db7/d12b 0 2024-04-20T11:09:13.103 INFO:tasks.workunit.client.0.smithi117.stdout:2/766: unlink d1/f6 0 2024-04-20T11:09:13.103 INFO:tasks.workunit.client.0.smithi117.stdout:0/727: fdatasync d3/d7/d8/f44 0 2024-04-20T11:09:13.104 INFO:tasks.workunit.client.0.smithi117.stdout:7/640: symlink d1/d4/d6/d54/lca 0 2024-04-20T11:09:13.108 INFO:tasks.workunit.client.0.smithi117.stdout:4/670: rename d1/d9/d1f/d87/fbe to d1/d45/d4a/d99/fe8 0 2024-04-20T11:09:13.109 INFO:tasks.workunit.client.0.smithi117.stdout:5/909: fdatasync de/d21/df5/d57/dd5/dce/f111 0 2024-04-20T11:09:13.111 INFO:tasks.workunit.client.0.smithi117.stdout:0/728: rmdir d3/dab 39 2024-04-20T11:09:13.112 INFO:tasks.workunit.client.0.smithi117.stdout:7/641: readlink d1/d4/d6/d13/l19 0 2024-04-20T11:09:13.113 INFO:tasks.workunit.client.0.smithi117.stdout:9/695: dwrite d4/d68/fe9 [0,4194304] 0 2024-04-20T11:09:13.116 INFO:tasks.workunit.client.0.smithi117.stdout:4/671: chown d1/dd/d1a/d2b/c92 0 1 2024-04-20T11:09:13.117 INFO:tasks.workunit.client.0.smithi117.stdout:3/620: sync 2024-04-20T11:09:13.122 INFO:tasks.workunit.client.0.smithi117.stdout:6/760: dread d3/df/f4f [0,4194304] 0 2024-04-20T11:09:13.123 INFO:tasks.workunit.client.0.smithi117.stdout:1/578: dread d6/f68 [0,4194304] 0 2024-04-20T11:09:13.124 INFO:tasks.workunit.client.0.smithi117.stdout:6/761: readlink d3/d50/ld8 0 2024-04-20T11:09:13.124 INFO:tasks.workunit.client.0.smithi117.stdout:0/729: unlink d3/d7/c23 0 2024-04-20T11:09:13.125 INFO:tasks.workunit.client.0.smithi117.stdout:9/696: symlink d4/d35/d54/led 0 2024-04-20T11:09:13.131 INFO:tasks.workunit.client.0.smithi117.stdout:6/762: write d3/f89 [571387,14642] 0 2024-04-20T11:09:13.145 INFO:tasks.workunit.client.0.smithi117.stdout:2/767: dwrite d1/d4/dd/f55 [4194304,4194304] 0 2024-04-20T11:09:13.145 INFO:tasks.workunit.client.0.smithi117.stdout:1/579: mknod d6/d16/d3a/d5a/d5c/d61/d91/cc3 0 2024-04-20T11:09:13.150 INFO:tasks.workunit.client.0.smithi117.stdout:8/538: dread d0/d8/d39/d40/f70 [0,4194304] 0 2024-04-20T11:09:13.154 INFO:tasks.workunit.client.0.smithi117.stdout:6/763: dread - d3/d4/d2e/d37/f7e zero size 2024-04-20T11:09:13.157 INFO:tasks.workunit.client.0.smithi117.stdout:5/910: creat de/dd0/d10e/d4e/f13c x:0 0 0 2024-04-20T11:09:13.159 INFO:tasks.workunit.client.0.smithi117.stdout:9/697: write d4/de/d10/f24 [4395817,44250] 0 2024-04-20T11:09:13.172 INFO:tasks.workunit.client.0.smithi117.stdout:3/621: dwrite d0/d5/f13 [0,4194304] 0 2024-04-20T11:09:13.173 INFO:tasks.workunit.client.0.smithi117.stdout:7/642: dwrite d1/d4/d39/f7a [0,4194304] 0 2024-04-20T11:09:13.175 INFO:tasks.workunit.client.0.smithi117.stdout:6/764: unlink d3/d4/d2e/d3f/f82 0 2024-04-20T11:09:13.182 INFO:tasks.workunit.client.0.smithi117.stdout:6/765: read d3/f89 [556681,22200] 0 2024-04-20T11:09:13.189 INFO:tasks.workunit.client.0.smithi117.stdout:5/911: unlink de/d41/d47/f133 0 2024-04-20T11:09:13.191 INFO:tasks.workunit.client.0.smithi117.stdout:4/672: dread d1/dd/d1a/f7c [0,4194304] 0 2024-04-20T11:09:13.198 INFO:tasks.workunit.client.0.smithi117.stdout:5/912: creat de/d5b/d65/f13d x:0 0 0 2024-04-20T11:09:13.203 INFO:tasks.workunit.client.0.smithi117.stdout:2/768: rename d1/d4/d83/d29 to d1/d4/d83/d29/df4 22 2024-04-20T11:09:13.204 INFO:tasks.workunit.client.0.smithi117.stdout:9/698: creat d4/d4a/fee x:0 0 0 2024-04-20T11:09:13.209 INFO:tasks.workunit.client.0.smithi117.stdout:1/580: dwrite d6/d21/f31 [0,4194304] 0 2024-04-20T11:09:13.212 INFO:tasks.workunit.client.0.smithi117.stdout:2/769: mknod d1/d4/dd/d17/d7a/d88/cf5 0 2024-04-20T11:09:13.216 INFO:tasks.workunit.client.0.smithi117.stdout:8/539: dwrite d0/f1 [0,4194304] 0 2024-04-20T11:09:13.217 INFO:tasks.workunit.client.0.smithi117.stdout:9/699: mknod d4/d4a/d72/dd2/cef 0 2024-04-20T11:09:13.219 INFO:tasks.workunit.client.0.smithi117.stdout:6/766: dwrite d3/df/f63 [0,4194304] 0 2024-04-20T11:09:13.225 INFO:tasks.workunit.client.0.smithi117.stdout:3/622: write d0/d5/d21/d88/d69/f9b [1568943,36088] 0 2024-04-20T11:09:13.225 INFO:tasks.workunit.client.0.smithi117.stdout:2/770: mkdir d1/d4/d83/d29/d3f/df6 0 2024-04-20T11:09:13.231 INFO:tasks.workunit.client.0.smithi117.stdout:8/540: mknod d0/dc/d49/d5b/da9/cb7 0 2024-04-20T11:09:13.233 INFO:tasks.workunit.client.0.smithi117.stdout:2/771: rmdir d1/d4/dd/d17 39 2024-04-20T11:09:13.238 INFO:tasks.workunit.client.0.smithi117.stdout:6/767: rename d3/d3a/c5c to d3/d5a/d74/db3/cf2 0 2024-04-20T11:09:13.240 INFO:tasks.workunit.client.0.smithi117.stdout:9/700: link d4/d5/d57/d82/da5/ccf d4/d4a/d74/cf0 0 2024-04-20T11:09:13.244 INFO:tasks.workunit.client.0.smithi117.stdout:6/768: fdatasync d3/d4/d2e/d3f/f77 0 2024-04-20T11:09:13.252 INFO:tasks.workunit.client.0.smithi117.stdout:2/772: mkdir d1/d4/d83/d29/d62/df7 0 2024-04-20T11:09:13.253 INFO:tasks.workunit.client.0.smithi117.stdout:9/701: write d4/d4a/d5e/de8/fe5 [132410,128248] 0 2024-04-20T11:09:13.256 INFO:tasks.workunit.client.0.smithi117.stdout:7/643: dwrite d1/d43/fa0 [0,4194304] 0 2024-04-20T11:09:13.261 INFO:tasks.workunit.client.0.smithi117.stdout:3/623: dread d0/d5/d7/dd/f5a [0,4194304] 0 2024-04-20T11:09:13.270 INFO:tasks.workunit.client.0.smithi117.stdout:7/644: truncate d1/d43/f80 3587640 0 2024-04-20T11:09:13.279 INFO:tasks.workunit.client.0.smithi117.stdout:1/581: truncate d6/d21/d95/fa9 2770028 0 2024-04-20T11:09:13.279 INFO:tasks.workunit.client.0.smithi117.stdout:5/913: dread de/d5b/f81 [0,4194304] 0 2024-04-20T11:09:13.280 INFO:tasks.workunit.client.0.smithi117.stdout:8/541: dwrite d0/f5 [0,4194304] 0 2024-04-20T11:09:13.287 INFO:tasks.workunit.client.0.smithi117.stdout:6/769: truncate d3/df/d39/d99/f64 1768174 0 2024-04-20T11:09:13.288 INFO:tasks.workunit.client.0.smithi117.stdout:7/645: mknod d1/d1d/ccb 0 2024-04-20T11:09:13.293 INFO:tasks.workunit.client.0.smithi117.stdout:5/914: rmdir de/d41/d47/d55/d56/dc8/dd8 39 2024-04-20T11:09:13.293 INFO:tasks.workunit.client.0.smithi117.stdout:2/773: dwrite d1/d4/dd/d17/d7a/d49/f87 [0,4194304] 0 2024-04-20T11:09:13.303 INFO:tasks.workunit.client.0.smithi117.stdout:1/582: creat d6/d16/d3a/d5a/d5c/fc4 x:0 0 0 2024-04-20T11:09:13.308 INFO:tasks.workunit.client.0.smithi117.stdout:9/702: dwrite d4/de/d7b/fbb [0,4194304] 0 2024-04-20T11:09:13.309 INFO:tasks.workunit.client.0.smithi117.stdout:8/542: rename d0/d8/d39/d40/d78/d5c/c62 to d0/cb8 0 2024-04-20T11:09:13.311 INFO:tasks.workunit.client.0.smithi117.stdout:1/583: fsync d6/d16/d3a/d5a/d5c/d61/d91/fb7 0 2024-04-20T11:09:13.316 INFO:tasks.workunit.client.0.smithi117.stdout:5/915: mknod de/dd0/d10e/d4e/da1/de7/d123/db9/c13e 0 2024-04-20T11:09:13.318 INFO:tasks.workunit.client.0.smithi117.stdout:7/646: dread d1/d4/d6/d17/f4c [0,4194304] 0 2024-04-20T11:09:13.320 INFO:tasks.workunit.client.0.smithi117.stdout:3/624: dwrite d0/d5/d73/f95 [0,4194304] 0 2024-04-20T11:09:13.323 INFO:tasks.workunit.client.0.smithi117.stdout:8/543: truncate d0/d8/d39/d40/f5f 4231889 0 2024-04-20T11:09:13.334 INFO:tasks.workunit.client.0.smithi117.stdout:9/703: truncate d4/d5/da/fde 3304824 0 2024-04-20T11:09:13.342 INFO:tasks.workunit.client.0.smithi117.stdout:2/774: dwrite d1/d4/dd/d17/d7a/d49/f87 [4194304,4194304] 0 2024-04-20T11:09:13.342 INFO:tasks.workunit.client.0.smithi117.stdout:5/916: mkdir de/dd0/d10e/d4e/da1/de7/d13f 0 2024-04-20T11:09:13.342 INFO:tasks.workunit.client.0.smithi117.stdout:5/917: chown de/d16/c86 806606 1 2024-04-20T11:09:13.342 INFO:tasks.workunit.client.0.smithi117.stdout:6/770: dread d3/d3a/f4d [0,4194304] 0 2024-04-20T11:09:13.342 INFO:tasks.workunit.client.0.smithi117.stdout:6/771: stat d3/d5a/d74/db3/cc7 0 2024-04-20T11:09:13.344 INFO:tasks.workunit.client.0.smithi117.stdout:6/772: readlink d3/d5a/d74/db3/l97 0 2024-04-20T11:09:13.352 INFO:tasks.workunit.client.0.smithi117.stdout:2/775: unlink d1/d4/dd/d17/d7a/d49/f87 0 2024-04-20T11:09:13.359 INFO:tasks.workunit.client.0.smithi117.stdout:8/544: dwrite d0/d8/d39/f54 [0,4194304] 0 2024-04-20T11:09:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:12 smithi117 ceph-mon[15659]: pgmap v16: 65 pgs: 65 active+clean; 1.3 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 35 MiB/s rd, 136 MiB/s wr, 348 op/s 2024-04-20T11:09:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:12 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:12 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:13.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:12 smithi173 ceph-mon[23800]: pgmap v16: 65 pgs: 65 active+clean; 1.3 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 35 MiB/s rd, 136 MiB/s wr, 348 op/s 2024-04-20T11:09:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:12 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:12 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:13.428 INFO:tasks.workunit.client.0.smithi117.stdout:6/773: fsync d3/d4/d2e/fdd 0 2024-04-20T11:09:13.433 INFO:tasks.workunit.client.0.smithi117.stdout:7/647: getdents d1/d1d/d42/d89 0 2024-04-20T11:09:13.436 INFO:tasks.workunit.client.0.smithi117.stdout:7/648: fsync d1/d4/d6/d17/d3f/f9b 0 2024-04-20T11:09:13.439 INFO:tasks.workunit.client.0.smithi117.stdout:7/649: read d1/d1d/fbb [3971485,114306] 0 2024-04-20T11:09:13.442 INFO:tasks.workunit.client.0.smithi117.stdout:2/776: creat d1/d4/d83/d29/d2e/d53/de7/ff8 x:0 0 0 2024-04-20T11:09:13.446 INFO:tasks.workunit.client.0.smithi117.stdout:8/545: link d0/d9/d3a/f6a d0/d8/d39/d7f/fb9 0 2024-04-20T11:09:13.455 INFO:tasks.workunit.client.0.smithi117.stdout:5/918: link de/dd0/d10e/d4e/da1/de7/d123/db9/dda/l132 de/d5b/l140 0 2024-04-20T11:09:13.459 INFO:tasks.workunit.client.0.smithi117.stdout:7/650: dread - d1/d4/d3d/d93/fa6 zero size 2024-04-20T11:09:13.459 INFO:tasks.workunit.client.0.smithi117.stdout:1/584: dwrite d6/d1f/d30/d49/d69/d6e/d6f/d73/f8c [0,4194304] 0 2024-04-20T11:09:13.465 INFO:tasks.workunit.client.0.smithi117.stdout:1/585: readlink d6/d21/l58 0 2024-04-20T11:09:13.467 INFO:tasks.workunit.client.0.smithi117.stdout:2/777: dread d1/d4/dd/f55 [4194304,4194304] 0 2024-04-20T11:09:13.470 INFO:tasks.workunit.client.0.smithi117.stdout:5/919: symlink de/d21/df5/d57/dd5/d11d/l141 0 2024-04-20T11:09:13.473 INFO:tasks.workunit.client.0.smithi117.stdout:7/651: creat d1/d1d/d42/fcc x:0 0 0 2024-04-20T11:09:13.474 INFO:tasks.workunit.client.0.smithi117.stdout:6/774: dread d3/df/d39/d99/f8c [0,4194304] 0 2024-04-20T11:09:13.477 INFO:tasks.workunit.client.0.smithi117.stdout:8/546: link d0/d8/d39/d40/d78/f97 d0/d8/d11/d6d/fba 0 2024-04-20T11:09:13.478 INFO:tasks.workunit.client.0.smithi117.stdout:0/730: sync 2024-04-20T11:09:13.478 INFO:tasks.workunit.client.0.smithi117.stdout:5/920: mkdir de/d21/df5/d57/d142 0 2024-04-20T11:09:13.483 INFO:tasks.workunit.client.0.smithi117.stdout:7/652: symlink d1/d1d/d42/d2d/lcd 0 2024-04-20T11:09:13.487 INFO:tasks.workunit.client.0.smithi117.stdout:7/653: readlink d1/d4/d6/d54/d63/l68 0 2024-04-20T11:09:13.487 INFO:tasks.workunit.client.0.smithi117.stdout:6/775: fdatasync d3/d4/d2e/d37/f62 0 2024-04-20T11:09:13.490 INFO:tasks.workunit.client.0.smithi117.stdout:6/776: chown d3/d4/d2e/fa6 14737 1 2024-04-20T11:09:13.493 INFO:tasks.workunit.client.0.smithi117.stdout:6/777: readlink d3/d5a/d74/db3/df0/db7/le5 0 2024-04-20T11:09:13.499 INFO:tasks.workunit.client.0.smithi117.stdout:5/921: write de/db7/fbd [1155318,130416] 0 2024-04-20T11:09:13.502 INFO:tasks.workunit.client.0.smithi117.stdout:5/922: chown de/d21/df5/f3b 7607797 1 2024-04-20T11:09:13.510 INFO:tasks.workunit.client.0.smithi117.stdout:1/586: dread d6/d35/f56 [0,4194304] 0 2024-04-20T11:09:13.516 INFO:tasks.workunit.client.0.smithi117.stdout:2/778: dread d1/d4/d83/f3a [0,4194304] 0 2024-04-20T11:09:13.523 INFO:tasks.workunit.client.0.smithi117.stdout:5/923: mknod de/dd0/d10e/d4e/da1/de7/d123/db9/c143 0 2024-04-20T11:09:13.527 INFO:tasks.workunit.client.0.smithi117.stdout:5/924: fdatasync de/d16/d9d/db4/f12c 0 2024-04-20T11:09:13.528 INFO:tasks.workunit.client.0.smithi117.stdout:2/779: fdatasync d1/d20/de0/fe1 0 2024-04-20T11:09:13.534 INFO:tasks.workunit.client.0.smithi117.stdout:1/587: creat d6/d16/d3a/d5a/daf/fc5 x:0 0 0 2024-04-20T11:09:13.537 INFO:tasks.workunit.client.0.smithi117.stdout:2/780: rename d1/d4/dd/d17/d7a/d49/ccf to d1/d4/d83/d29/d2e/d53/d58/cf9 0 2024-04-20T11:09:13.540 INFO:tasks.workunit.client.0.smithi117.stdout:6/778: getdents d3/d5a/d74/db3/dd0 0 2024-04-20T11:09:13.551 INFO:tasks.workunit.client.0.smithi117.stdout:3/625: sync 2024-04-20T11:09:13.551 INFO:tasks.workunit.client.0.smithi117.stdout:4/673: sync 2024-04-20T11:09:13.554 INFO:tasks.workunit.client.0.smithi117.stdout:4/674: chown d1/d45/d4a/d99/fc0 0 1 2024-04-20T11:09:13.558 INFO:tasks.workunit.client.0.smithi117.stdout:3/626: fsync d0/d5/d21/d88/f62 0 2024-04-20T11:09:13.565 INFO:tasks.workunit.client.0.smithi117.stdout:9/704: write d4/d4a/d63/d98/fea [441714,17908] 0 2024-04-20T11:09:13.574 INFO:tasks.workunit.client.0.smithi117.stdout:3/627: creat d0/d5/d7/fd6 x:0 0 0 2024-04-20T11:09:13.576 INFO:tasks.workunit.client.0.smithi117.stdout:3/628: write d0/d5/d73/f95 [2595706,34537] 0 2024-04-20T11:09:13.588 INFO:tasks.workunit.client.0.smithi117.stdout:3/629: link d0/d5/d73/f8b d0/d5/de/fd7 0 2024-04-20T11:09:13.594 INFO:tasks.workunit.client.0.smithi117.stdout:3/630: mkdir d0/d5/d73/dd8 0 2024-04-20T11:09:13.595 INFO:tasks.workunit.client.0.smithi117.stdout:4/675: rename d1/d45/d4a/d99 to d1/d9/de9 0 2024-04-20T11:09:13.596 INFO:tasks.workunit.client.0.smithi117.stdout:3/631: chown d0/d5/d21/d88/d29/l57 467 1 2024-04-20T11:09:13.600 INFO:tasks.workunit.client.0.smithi117.stdout:3/632: write d0/d2a/fb0 [999989,82634] 0 2024-04-20T11:09:13.606 INFO:tasks.workunit.client.0.smithi117.stdout:0/731: rmdir d3/d7/d20/d6e 39 2024-04-20T11:09:13.608 INFO:tasks.workunit.client.0.smithi117.stdout:2/781: creat d1/d4/d83/ffa x:0 0 0 2024-04-20T11:09:13.613 INFO:tasks.workunit.client.0.smithi117.stdout:2/782: chown d1/d4/d83/d29/d2e/d53/d58/c5f 100317201 1 2024-04-20T11:09:13.616 INFO:tasks.workunit.client.0.smithi117.stdout:4/676: dwrite d1/d45/d4a/d59/fce [0,4194304] 0 2024-04-20T11:09:13.618 INFO:tasks.workunit.client.0.smithi117.stdout:8/547: mkdir d0/d9/dbb 0 2024-04-20T11:09:13.634 INFO:tasks.workunit.client.0.smithi117.stdout:8/548: creat d0/d8/d39/d7f/fbc x:0 0 0 2024-04-20T11:09:13.637 INFO:tasks.workunit.client.0.smithi117.stdout:2/783: dwrite d1/d4/dd/d17/d7a/d49/fcc [0,4194304] 0 2024-04-20T11:09:13.640 INFO:tasks.workunit.client.0.smithi117.stdout:5/925: creat de/d16/f144 x:0 0 0 2024-04-20T11:09:13.647 INFO:tasks.workunit.client.0.smithi117.stdout:7/654: dwrite d1/d1d/d42/d2d/d6c/f88 [0,4194304] 0 2024-04-20T11:09:13.651 INFO:tasks.workunit.client.0.smithi117.stdout:2/784: symlink d1/d4/d83/d29/lfb 0 2024-04-20T11:09:13.653 INFO:tasks.workunit.client.0.smithi117.stdout:6/779: write d3/d4/fb4 [711987,34128] 0 2024-04-20T11:09:13.653 INFO:tasks.workunit.client.0.smithi117.stdout:1/588: write d6/d35/f36 [3260008,123893] 0 2024-04-20T11:09:13.658 INFO:tasks.workunit.client.0.smithi117.stdout:2/785: write d1/d20/de0/fe1 [584290,50889] 0 2024-04-20T11:09:13.659 INFO:tasks.workunit.client.0.smithi117.stdout:7/655: read d1/d43/fa0 [3389131,126948] 0 2024-04-20T11:09:13.661 INFO:tasks.workunit.client.0.smithi117.stdout:1/589: stat d6/d1f/d30/d49/c8a 0 2024-04-20T11:09:13.661 INFO:tasks.workunit.client.0.smithi117.stdout:4/677: dwrite d1/dd/d1a/d43/d4e/f9b [0,4194304] 0 2024-04-20T11:09:13.663 INFO:tasks.workunit.client.0.smithi117.stdout:1/590: chown d6/d1f/d30/c60 37456686 1 2024-04-20T11:09:13.669 INFO:tasks.workunit.client.0.smithi117.stdout:6/780: symlink d3/d4/d2e/d3f/lf3 0 2024-04-20T11:09:13.671 INFO:tasks.workunit.client.0.smithi117.stdout:2/786: write d1/da5/df2/fde [376136,107011] 0 2024-04-20T11:09:13.672 INFO:tasks.workunit.client.0.smithi117.stdout:1/591: chown d6/d35/d63/l66 743367 1 2024-04-20T11:09:13.676 INFO:tasks.workunit.client.0.smithi117.stdout:4/678: write d1/dd/d1a/f7c [446975,8041] 0 2024-04-20T11:09:13.684 INFO:tasks.workunit.client.0.smithi117.stdout:6/781: rename d3/d4/d2e/l45 to d3/df/d39/daa/lf4 0 2024-04-20T11:09:13.685 INFO:tasks.workunit.client.0.smithi117.stdout:1/592: truncate d6/d16/d3a/d5a/d5c/d61/d91/f96 53796 0 2024-04-20T11:09:13.688 INFO:tasks.workunit.client.0.smithi117.stdout:1/593: chown d6/f79 3322 1 2024-04-20T11:09:13.691 INFO:tasks.workunit.client.0.smithi117.stdout:1/594: fdatasync d6/d16/d3a/d5a/d5c/d61/f83 0 2024-04-20T11:09:13.694 INFO:tasks.workunit.client.0.smithi117.stdout:2/787: symlink d1/d4/d83/d29/d2e/d53/d58/lfc 0 2024-04-20T11:09:13.695 INFO:tasks.workunit.client.0.smithi117.stdout:1/595: write d6/d16/d3a/d5a/d5c/f7b [4694447,78413] 0 2024-04-20T11:09:13.696 INFO:tasks.workunit.client.0.smithi117.stdout:2/788: stat d1/d4/d83/fc3 0 2024-04-20T11:09:13.698 INFO:tasks.workunit.client.0.smithi117.stdout:8/549: dread d0/d9/f2a [0,4194304] 0 2024-04-20T11:09:13.700 INFO:tasks.workunit.client.0.smithi117.stdout:6/782: unlink d3/d5a/d74/db3/df0/db7/ce9 0 2024-04-20T11:09:13.701 INFO:tasks.workunit.client.0.smithi117.stdout:3/633: write d0/d5/d21/f47 [407772,61975] 0 2024-04-20T11:09:13.703 INFO:tasks.workunit.client.0.smithi117.stdout:5/926: dread de/f11 [0,4194304] 0 2024-04-20T11:09:13.704 INFO:tasks.workunit.client.0.smithi117.stdout:8/550: read d0/d8/d11/d24/d53/f7b [1289643,82626] 0 2024-04-20T11:09:13.704 INFO:tasks.workunit.client.0.smithi117.stdout:3/634: fdatasync d0/d5/d21/d94/d9a/fbb 0 2024-04-20T11:09:13.708 INFO:tasks.workunit.client.0.smithi117.stdout:8/551: dread - d0/d8/d39/d7f/fbc zero size 2024-04-20T11:09:13.711 INFO:tasks.workunit.client.0.smithi117.stdout:2/789: symlink d1/d4/d83/d29/daa/lfd 0 2024-04-20T11:09:13.711 INFO:tasks.workunit.client.0.smithi117.stdout:8/552: dread - d0/d8/d11/d24/d76/fb0 zero size 2024-04-20T11:09:13.713 INFO:tasks.workunit.client.0.smithi117.stdout:6/783: creat d3/d5a/d74/db3/df0/ff5 x:0 0 0 2024-04-20T11:09:13.715 INFO:tasks.workunit.client.0.smithi117.stdout:8/553: chown d0/dc/d49/d5b 160338731 1 2024-04-20T11:09:13.716 INFO:tasks.workunit.client.0.smithi117.stdout:6/784: chown d3/d4 4315 1 2024-04-20T11:09:13.722 INFO:tasks.workunit.client.0.smithi117.stdout:5/927: rename de/d41/d47/d55/d56/dc8/dd8/l105 to de/d21/d84/l145 0 2024-04-20T11:09:13.722 INFO:tasks.workunit.client.0.smithi117.stdout:0/732: dwrite d3/d7/d20/d47/d78/f7c [0,4194304] 0 2024-04-20T11:09:13.726 INFO:tasks.workunit.client.0.smithi117.stdout:6/785: mkdir d3/d5a/d74/db3/dbc/df6 0 2024-04-20T11:09:13.730 INFO:tasks.workunit.client.0.smithi117.stdout:6/786: write d3/df/d39/d99/fcd [462890,66667] 0 2024-04-20T11:09:13.732 INFO:tasks.workunit.client.0.smithi117.stdout:0/733: rmdir d3/d5/d74 39 2024-04-20T11:09:13.735 INFO:tasks.workunit.client.0.smithi117.stdout:5/928: creat de/dd0/d10e/d4e/da1/de7/d123/db9/dda/f146 x:0 0 0 2024-04-20T11:09:13.738 INFO:tasks.workunit.client.0.smithi117.stdout:1/596: dread d6/d1f/d30/f43 [0,4194304] 0 2024-04-20T11:09:13.740 INFO:tasks.workunit.client.0.smithi117.stdout:2/790: dread d1/d4/dd/d17/d7a/d49/fcc [0,4194304] 0 2024-04-20T11:09:13.740 INFO:tasks.workunit.client.0.smithi117.stdout:7/656: truncate d1/d4/d39/f4d 6166927 0 2024-04-20T11:09:13.741 INFO:tasks.workunit.client.0.smithi117.stdout:5/929: dread de/dd0/d10e/d4e/ff2 [0,4194304] 0 2024-04-20T11:09:13.744 INFO:tasks.workunit.client.0.smithi117.stdout:6/787: mkdir d3/d4/d2e/dc0/df7 0 2024-04-20T11:09:13.748 INFO:tasks.workunit.client.0.smithi117.stdout:4/679: write d1/dd/d1a/d43/d4e/f62 [928081,37316] 0 2024-04-20T11:09:13.750 INFO:tasks.workunit.client.0.smithi117.stdout:7/657: mkdir d1/d4/d3d/dce 0 2024-04-20T11:09:13.756 INFO:tasks.workunit.client.0.smithi117.stdout:1/597: unlink d6/d16/d3a/d5a/f53 0 2024-04-20T11:09:13.758 INFO:tasks.workunit.client.0.smithi117.stdout:8/554: dread d0/d9/f95 [0,4194304] 0 2024-04-20T11:09:13.760 INFO:tasks.workunit.client.0.smithi117.stdout:5/930: rename de/d21/df5/d57/dd5/d11d/l59 to de/d41/dc2/l147 0 2024-04-20T11:09:13.762 INFO:tasks.workunit.client.0.smithi117.stdout:6/788: fsync d3/d4/d2e/d37/f62 0 2024-04-20T11:09:13.763 INFO:tasks.workunit.client.0.smithi117.stdout:3/635: write d0/d5/d21/d88/f70 [353736,83291] 0 2024-04-20T11:09:13.765 INFO:tasks.workunit.client.0.smithi117.stdout:8/555: readlink d0/d8/d11/d24/d2b/d72/l8c 0 2024-04-20T11:09:13.768 INFO:tasks.workunit.client.0.smithi117.stdout:5/931: truncate de/d16/d9d/db4/df1/ffc 1001847 0 2024-04-20T11:09:13.768 INFO:tasks.workunit.client.0.smithi117.stdout:9/705: sync 2024-04-20T11:09:13.770 INFO:tasks.workunit.client.0.smithi117.stdout:0/734: creat d3/d7/d20/d39/fe8 x:0 0 0 2024-04-20T11:09:13.781 INFO:tasks.workunit.client.0.smithi117.stdout:9/706: symlink d4/d5/d3c/ddf/d31/lf1 0 2024-04-20T11:09:13.782 INFO:tasks.workunit.client.0.smithi117.stdout:9/707: readlink d4/l52 0 2024-04-20T11:09:13.786 INFO:tasks.workunit.client.0.smithi117.stdout:8/556: mknod d0/cbd 0 2024-04-20T11:09:13.786 INFO:tasks.workunit.client.0.smithi117.stdout:9/708: write d4/d5/d57/fd4 [708592,25835] 0 2024-04-20T11:09:13.788 INFO:tasks.workunit.client.0.smithi117.stdout:0/735: readlink d3/d7/d20/d39/d9a/dd0/l82 0 2024-04-20T11:09:13.791 INFO:tasks.workunit.client.0.smithi117.stdout:3/636: creat d0/d5/d7/fd9 x:0 0 0 2024-04-20T11:09:13.791 INFO:tasks.workunit.client.0.smithi117.stdout:0/736: chown d3/d7e/fc7 429964 1 2024-04-20T11:09:13.794 INFO:tasks.workunit.client.0.smithi117.stdout:8/557: truncate d0/d8/d11/d6d/d58/f71 4363962 0 2024-04-20T11:09:13.794 INFO:tasks.workunit.client.0.smithi117.stdout:1/598: dwrite d6/f13 [4194304,4194304] 0 2024-04-20T11:09:13.799 INFO:tasks.workunit.client.0.smithi117.stdout:3/637: mknod d0/d5/d21/d88/d29/db3/cda 0 2024-04-20T11:09:13.800 INFO:tasks.workunit.client.0.smithi117.stdout:7/658: write d1/d4/d6/f26 [5672934,9388] 0 2024-04-20T11:09:13.802 INFO:tasks.workunit.client.0.smithi117.stdout:1/599: chown d6/f10 715584583 1 2024-04-20T11:09:13.803 INFO:tasks.workunit.client.0.smithi117.stdout:8/558: unlink d0/dc/d49/d5b/da9/cb7 0 2024-04-20T11:09:13.808 INFO:tasks.workunit.client.0.smithi117.stdout:1/600: chown d6/f79 109712932 1 2024-04-20T11:09:13.811 INFO:tasks.workunit.client.0.smithi117.stdout:1/601: chown d6 354750997 1 2024-04-20T11:09:13.811 INFO:tasks.workunit.client.0.smithi117.stdout:3/638: chown d0/d5/d11/d80/cb8 40244597 1 2024-04-20T11:09:13.817 INFO:tasks.workunit.client.0.smithi117.stdout:6/789: dwrite d3/df/f4f [0,4194304] 0 2024-04-20T11:09:13.819 INFO:tasks.workunit.client.0.smithi117.stdout:7/659: symlink d1/d1d/d4e/d53/d55/lcf 0 2024-04-20T11:09:13.821 INFO:tasks.workunit.client.0.smithi117.stdout:8/559: chown d0/d8/d39/d7f/fb9 417243386 1 2024-04-20T11:09:13.823 INFO:tasks.workunit.client.0.smithi117.stdout:1/602: read d6/f13 [7503238,92114] 0 2024-04-20T11:09:13.823 INFO:tasks.workunit.client.0.smithi117.stdout:5/932: write de/d41/d47/d4c/f75 [25978,67020] 0 2024-04-20T11:09:13.838 INFO:tasks.workunit.client.0.smithi117.stdout:7/660: write d1/d1d/d4e/d72/fc6 [541679,29735] 0 2024-04-20T11:09:13.838 INFO:tasks.workunit.client.0.smithi117.stdout:4/680: dwrite d1/dd/d1a/d2b/f46 [0,4194304] 0 2024-04-20T11:09:13.842 INFO:tasks.workunit.client.0.smithi117.stdout:2/791: dwrite d1/f11 [0,4194304] 0 2024-04-20T11:09:13.845 INFO:tasks.workunit.client.0.smithi117.stdout:9/709: dread d4/d5/f1e [4194304,4194304] 0 2024-04-20T11:09:13.846 INFO:tasks.workunit.client.0.smithi117.stdout:8/560: rename d0/f5 to d0/dc/d2e/d89/fbe 0 2024-04-20T11:09:13.850 INFO:tasks.workunit.client.0.smithi117.stdout:8/561: chown d0/d8/d11/c1b 5 1 2024-04-20T11:09:13.856 INFO:tasks.workunit.client.0.smithi117.stdout:3/639: dread d0/d5/f13 [0,4194304] 0 2024-04-20T11:09:13.856 INFO:tasks.workunit.client.0.smithi117.stdout:9/710: chown d4/de/f84 6 1 2024-04-20T11:09:13.857 INFO:tasks.workunit.client.0.smithi117.stdout:4/681: rmdir d1/dd/d1a/d43/d4e 39 2024-04-20T11:09:13.859 INFO:tasks.workunit.client.0.smithi117.stdout:8/562: fsync d0/d8/d39/d7f/fa5 0 2024-04-20T11:09:13.876 INFO:tasks.workunit.client.0.smithi117.stdout:0/737: dwrite d3/d5/f59 [0,4194304] 0 2024-04-20T11:09:13.876 INFO:tasks.workunit.client.0.smithi117.stdout:3/640: stat d0/d5/d21/d88/d29/d66 0 2024-04-20T11:09:13.881 INFO:tasks.workunit.client.0.smithi117.stdout:6/790: dwrite d3/f73 [0,4194304] 0 2024-04-20T11:09:13.902 INFO:tasks.workunit.client.0.smithi117.stdout:2/792: dwrite d1/d4/d83/d29/d3f/f76 [0,4194304] 0 2024-04-20T11:09:13.904 INFO:tasks.workunit.client.0.smithi117.stdout:7/661: creat d1/d4/d6/fd0 x:0 0 0 2024-04-20T11:09:13.906 INFO:tasks.workunit.client.0.smithi117.stdout:2/793: readlink d1/d4/d83/d29/d2e/d53/d58/lfc 0 2024-04-20T11:09:13.907 INFO:tasks.workunit.client.0.smithi117.stdout:1/603: dread d6/d35/f97 [0,4194304] 0 2024-04-20T11:09:13.907 INFO:tasks.workunit.client.0.smithi117.stdout:4/682: unlink d1/f20 0 2024-04-20T11:09:13.911 INFO:tasks.workunit.client.0.smithi117.stdout:5/933: dread de/d21/d84/f87 [0,4194304] 0 2024-04-20T11:09:13.911 INFO:tasks.workunit.client.0.smithi117.stdout:6/791: rename d3/d4/d2e/d3f/c5e to d3/d5a/d74/cf8 0 2024-04-20T11:09:13.913 INFO:tasks.workunit.client.0.smithi117.stdout:9/711: symlink d4/de/d10/lf2 0 2024-04-20T11:09:13.916 INFO:tasks.workunit.client.0.smithi117.stdout:7/662: chown d1/d4/d6/l3e 345 1 2024-04-20T11:09:13.919 INFO:tasks.workunit.client.0.smithi117.stdout:1/604: chown d6/d1f/f80 119 1 2024-04-20T11:09:13.924 INFO:tasks.workunit.client.0.smithi117.stdout:8/563: read d0/d8/d39/d7f/f81 [10989,68726] 0 2024-04-20T11:09:13.924 INFO:tasks.workunit.client.0.smithi117.stdout:2/794: dread - d1/d4/d83/d29/d2e/d53/d93/f99 zero size 2024-04-20T11:09:13.927 INFO:tasks.workunit.client.0.smithi117.stdout:3/641: creat d0/d5/fdb x:0 0 0 2024-04-20T11:09:13.928 INFO:tasks.workunit.client.0.smithi117.stdout:1/605: fdatasync d6/d35/f56 0 2024-04-20T11:09:13.930 INFO:tasks.workunit.client.0.smithi117.stdout:3/642: fdatasync d0/d5/d21/f4a 0 2024-04-20T11:09:13.933 INFO:tasks.workunit.client.0.smithi117.stdout:7/663: mkdir d1/dd1 0 2024-04-20T11:09:13.935 INFO:tasks.workunit.client.0.smithi117.stdout:2/795: write d1/d4/d83/d29/d2e/d53/d58/fee [63151,109850] 0 2024-04-20T11:09:13.936 INFO:tasks.workunit.client.0.smithi117.stdout:6/792: link d3/d4/l36 d3/df/d39/daa/lf9 0 2024-04-20T11:09:13.937 INFO:tasks.workunit.client.0.smithi117.stdout:8/564: write d0/d8/d39/d40/f43 [751016,22209] 0 2024-04-20T11:09:13.939 INFO:tasks.workunit.client.0.smithi117.stdout:7/664: truncate d1/d43/f8d 534223 0 2024-04-20T11:09:13.945 INFO:tasks.workunit.client.0.smithi117.stdout:1/606: symlink d6/d16/d3a/lc6 0 2024-04-20T11:09:13.949 INFO:tasks.workunit.client.0.smithi117.stdout:2/796: rmdir d1/d4/d83/d29/d3f 39 2024-04-20T11:09:13.957 INFO:tasks.workunit.client.0.smithi117.stdout:8/565: symlink d0/d8/d39/lbf 0 2024-04-20T11:09:13.958 INFO:tasks.workunit.client.0.smithi117.stdout:9/712: getdents d4/d4a/d5e/d9b/dcd 0 2024-04-20T11:09:13.961 INFO:tasks.workunit.client.0.smithi117.stdout:4/683: dread d1/dd/d1a/d2b/d86/dda/fdd [0,4194304] 0 2024-04-20T11:09:13.961 INFO:tasks.workunit.client.0.smithi117.stdout:1/607: mkdir d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7 0 2024-04-20T11:09:13.962 INFO:tasks.workunit.client.0.smithi117.stdout:6/793: dread d3/d50/fb8 [0,4194304] 0 2024-04-20T11:09:13.962 INFO:tasks.workunit.client.0.smithi117.stdout:2/797: dread - d1/d20/fbb zero size 2024-04-20T11:09:13.963 INFO:tasks.workunit.client.0.smithi117.stdout:3/643: creat d0/d5/d21/d88/d29/dac/dd2/fdc x:0 0 0 2024-04-20T11:09:13.966 INFO:tasks.workunit.client.0.smithi117.stdout:5/934: dread de/d21/df5/fbe [0,4194304] 0 2024-04-20T11:09:13.973 INFO:tasks.workunit.client.0.smithi117.stdout:6/794: write d3/d4/d2e/d37/d8e/fd2 [42059,44056] 0 2024-04-20T11:09:13.973 INFO:tasks.workunit.client.0.smithi117.stdout:9/713: symlink d4/d4a/d5e/d9b/dcd/lf3 0 2024-04-20T11:09:13.973 INFO:tasks.workunit.client.0.smithi117.stdout:3/644: dread - d0/d5/de/fd7 zero size 2024-04-20T11:09:13.974 INFO:tasks.workunit.client.0.smithi117.stdout:6/795: chown d3/d5a/d74/db3/dbc/df6 286105 1 2024-04-20T11:09:13.976 INFO:tasks.workunit.client.0.smithi117.stdout:3/645: stat d0/d5/fdb 0 2024-04-20T11:09:13.976 INFO:tasks.workunit.client.0.smithi117.stdout:4/684: mkdir d1/d45/d4a/dea 0 2024-04-20T11:09:13.980 INFO:tasks.workunit.client.0.smithi117.stdout:7/665: rename d1/d1d/d4e/fb9 to d1/d4/d6/fd2 0 2024-04-20T11:09:13.985 INFO:tasks.workunit.client.0.smithi117.stdout:1/608: getdents d6/d1f/d30/d49/d64 0 2024-04-20T11:09:13.988 INFO:tasks.workunit.client.0.smithi117.stdout:1/609: chown d6/d16/l37 54528391 1 2024-04-20T11:09:13.988 INFO:tasks.workunit.client.0.smithi117.stdout:7/666: rmdir d1/d4/d39 39 2024-04-20T11:09:13.989 INFO:tasks.workunit.client.0.smithi117.stdout:2/798: dwrite d1/d4/d83/d29/d2e/fe9 [0,4194304] 0 2024-04-20T11:09:13.990 INFO:tasks.workunit.client.0.smithi117.stdout:7/667: chown d1/d43 5494 1 2024-04-20T11:09:13.995 INFO:tasks.workunit.client.0.smithi117.stdout:3/646: link d0/d5/d21/f28 d0/d2a/fdd 0 2024-04-20T11:09:13.995 INFO:tasks.workunit.client.0.smithi117.stdout:2/799: chown d1/d4/fa0 24528745 1 2024-04-20T11:09:14.001 INFO:tasks.workunit.client.0.smithi117.stdout:6/796: getdents d3/d3a 0 2024-04-20T11:09:14.012 INFO:tasks.workunit.client.0.smithi117.stdout:2/800: rename d1/d4/d83/d29/d2e/f64 to d1/d4/dd/d17/ffe 0 2024-04-20T11:09:14.020 INFO:tasks.workunit.client.0.smithi117.stdout:2/801: read d1/d4/f5 [5067670,8122] 0 2024-04-20T11:09:14.023 INFO:tasks.workunit.client.0.smithi117.stdout:9/714: write d4/de/d10/f3a [768299,64631] 0 2024-04-20T11:09:14.033 INFO:tasks.workunit.client.0.smithi117.stdout:2/802: chown d1/d4/d83/d29/c5a 29751379 1 2024-04-20T11:09:14.036 INFO:tasks.workunit.client.0.smithi117.stdout:8/566: dwrite d0/d8/d39/d7f/f81 [0,4194304] 0 2024-04-20T11:09:14.038 INFO:tasks.workunit.client.0.smithi117.stdout:3/647: dwrite d0/d5/d21/d94/d9a/fa0 [0,4194304] 0 2024-04-20T11:09:14.039 INFO:tasks.workunit.client.0.smithi117.stdout:7/668: dwrite d1/d1d/d42/fac [4194304,4194304] 0 2024-04-20T11:09:14.040 INFO:tasks.workunit.client.0.smithi117.stdout:3/648: chown d0/d5/d21/d88/d3c 2626 1 2024-04-20T11:09:14.048 INFO:tasks.workunit.client.0.smithi117.stdout:9/715: truncate d4/de/d10/f38 296269 0 2024-04-20T11:09:14.053 INFO:tasks.workunit.client.0.smithi117.stdout:7/669: unlink d1/d43/d59/cc7 0 2024-04-20T11:09:14.067 INFO:tasks.workunit.client.0.smithi117.stdout:9/716: creat d4/d5/da/d51/ff4 x:0 0 0 2024-04-20T11:09:14.070 INFO:tasks.workunit.client.0.smithi117.stdout:7/670: truncate d1/d4/d6/d13/f15 1600403 0 2024-04-20T11:09:14.073 INFO:tasks.workunit.client.0.smithi117.stdout:3/649: dwrite d0/f35 [0,4194304] 0 2024-04-20T11:09:14.075 INFO:tasks.workunit.client.0.smithi117.stdout:6/797: dread d3/d3a/fd6 [0,4194304] 0 2024-04-20T11:09:14.079 INFO:tasks.workunit.client.0.smithi117.stdout:9/717: symlink d4/d4a/d5e/d9b/lf5 0 2024-04-20T11:09:14.086 INFO:tasks.workunit.client.0.smithi117.stdout:6/798: creat d3/d94/ffa x:0 0 0 2024-04-20T11:09:14.087 INFO:tasks.workunit.client.0.smithi117.stdout:8/567: dwrite d0/d8/d11/f14 [0,4194304] 0 2024-04-20T11:09:14.088 INFO:tasks.workunit.client.0.smithi117.stdout:7/671: mkdir d1/d1d/d4e/dd3 0 2024-04-20T11:09:14.090 INFO:tasks.workunit.client.0.smithi117.stdout:2/803: dwrite d1/d4/dd/d17/f30 [0,4194304] 0 2024-04-20T11:09:14.092 INFO:tasks.workunit.client.0.smithi117.stdout:1/610: dread d6/d1f/f45 [0,4194304] 0 2024-04-20T11:09:14.094 INFO:tasks.workunit.client.0.smithi117.stdout:6/799: chown d3/d5a/d74/db3/df0/da3 105 1 2024-04-20T11:09:14.094 INFO:tasks.workunit.client.0.smithi117.stdout:2/804: stat d1/d4/ddf 0 2024-04-20T11:09:14.101 INFO:tasks.workunit.client.0.smithi117.stdout:9/718: rename d4/de/d10/d94/ld6 to d4/lf6 0 2024-04-20T11:09:14.102 INFO:tasks.workunit.client.0.smithi117.stdout:8/568: fsync d0/d8/d39/d40/d78/f9b 0 2024-04-20T11:09:14.106 INFO:tasks.workunit.client.0.smithi117.stdout:7/672: truncate d1/d4/d6/d13/f71 2142442 0 2024-04-20T11:09:14.124 INFO:tasks.workunit.client.0.smithi117.stdout:1/611: mknod d6/d16/d3a/d5a/d5c/da6/cc8 0 2024-04-20T11:09:14.126 INFO:tasks.workunit.client.0.smithi117.stdout:3/650: getdents d0/d5/d21/d88/d29/d66 0 2024-04-20T11:09:14.134 INFO:tasks.workunit.client.0.smithi117.stdout:6/800: rename d3/d5a/d74/cf8 to d3/df/d39/daa/dee/cfb 0 2024-04-20T11:09:14.135 INFO:tasks.workunit.client.0.smithi117.stdout:1/612: write d6/d16/d3a/da8/fae [5077936,12623] 0 2024-04-20T11:09:14.138 INFO:tasks.workunit.client.0.smithi117.stdout:7/673: fdatasync d1/d43/d59/f69 0 2024-04-20T11:09:14.138 INFO:tasks.workunit.client.0.smithi117.stdout:6/801: dread - d3/d4/d2e/d37/fdb zero size 2024-04-20T11:09:14.143 INFO:tasks.workunit.client.0.smithi117.stdout:3/651: mknod d0/d5/d21/d88/d29/db3/cde 0 2024-04-20T11:09:14.144 INFO:tasks.workunit.client.0.smithi117.stdout:6/802: write d3/f67 [2522088,62630] 0 2024-04-20T11:09:14.152 INFO:tasks.workunit.client.0.smithi117.stdout:3/652: dread d0/d5/d21/d94/d9a/dc9/fca [0,4194304] 0 2024-04-20T11:09:14.153 INFO:tasks.workunit.client.0.smithi117.stdout:7/674: creat d1/d1d/d4e/d53/fd4 x:0 0 0 2024-04-20T11:09:14.159 INFO:tasks.workunit.client.0.smithi117.stdout:3/653: read d0/d5/d11/f7e [3499289,127806] 0 2024-04-20T11:09:14.162 INFO:tasks.workunit.client.0.smithi117.stdout:8/569: dwrite d0/d8/f65 [4194304,4194304] 0 2024-04-20T11:09:14.172 INFO:tasks.workunit.client.0.smithi117.stdout:8/570: chown d0/d8/d11/d24/d86 89848 1 2024-04-20T11:09:14.172 INFO:tasks.workunit.client.0.smithi117.stdout:9/719: dwrite d4/d5/d57/f99 [0,4194304] 0 2024-04-20T11:09:14.172 INFO:tasks.workunit.client.0.smithi117.stdout:6/803: mknod d3/cfc 0 2024-04-20T11:09:14.172 INFO:tasks.workunit.client.0.smithi117.stdout:1/613: creat d6/d1f/d30/fc9 x:0 0 0 2024-04-20T11:09:14.174 INFO:tasks.workunit.client.0.smithi117.stdout:1/614: stat d6/d16/d3a/d5a/c87 0 2024-04-20T11:09:14.176 INFO:tasks.workunit.client.0.smithi117.stdout:1/615: dread - d6/d1f/d30/fc9 zero size 2024-04-20T11:09:14.179 INFO:tasks.workunit.client.0.smithi117.stdout:6/804: creat d3/df/d39/ffd x:0 0 0 2024-04-20T11:09:14.183 INFO:tasks.workunit.client.0.smithi117.stdout:8/571: link d0/l50 d0/d8/lc0 0 2024-04-20T11:09:14.184 INFO:tasks.workunit.client.0.smithi117.stdout:6/805: truncate d3/df/f63 1874437 0 2024-04-20T11:09:14.188 INFO:tasks.workunit.client.0.smithi117.stdout:3/654: dwrite d0/d5/d21/d88/d69/f9b [0,4194304] 0 2024-04-20T11:09:14.190 INFO:tasks.workunit.client.0.smithi117.stdout:0/738: sync 2024-04-20T11:09:14.198 INFO:tasks.workunit.client.0.smithi117.stdout:8/572: creat d0/d8/d11/d6d/d84/fc1 x:0 0 0 2024-04-20T11:09:14.198 INFO:tasks.workunit.client.0.smithi117.stdout:6/806: truncate d3/d4/d2e/d37/f62 551143 0 2024-04-20T11:09:14.205 INFO:tasks.workunit.client.0.smithi117.stdout:8/573: mknod d0/dc/d49/cc2 0 2024-04-20T11:09:14.206 INFO:tasks.workunit.client.0.smithi117.stdout:9/720: dread d4/d35/f81 [0,4194304] 0 2024-04-20T11:09:14.211 INFO:tasks.workunit.client.0.smithi117.stdout:8/574: fsync d0/d8/d11/fac 0 2024-04-20T11:09:14.213 INFO:tasks.workunit.client.0.smithi117.stdout:8/575: read - d0/d9/d3a/f8b zero size 2024-04-20T11:09:14.220 INFO:tasks.workunit.client.0.smithi117.stdout:0/739: dwrite d3/d7/d8/f44 [0,4194304] 0 2024-04-20T11:09:14.224 INFO:tasks.workunit.client.0.smithi117.stdout:3/655: write d0/d5/fc [1711096,73821] 0 2024-04-20T11:09:14.225 INFO:tasks.workunit.client.0.smithi117.stdout:0/740: mknod d3/d7e/ce9 0 2024-04-20T11:09:14.230 INFO:tasks.workunit.client.0.smithi117.stdout:7/675: dread d1/d4/d6/d17/f3a [0,4194304] 0 2024-04-20T11:09:14.230 INFO:tasks.workunit.client.0.smithi117.stdout:8/576: creat d0/d8/d11/d6d/d84/fc3 x:0 0 0 2024-04-20T11:09:14.231 INFO:tasks.workunit.client.0.smithi117.stdout:6/807: dwrite d3/d4/d2e/d37/f47 [0,4194304] 0 2024-04-20T11:09:14.238 INFO:tasks.workunit.client.0.smithi117.stdout:3/656: mknod d0/d5/d73/cdf 0 2024-04-20T11:09:14.240 INFO:tasks.workunit.client.0.smithi117.stdout:8/577: symlink d0/d8/d11/d6d/lc4 0 2024-04-20T11:09:14.241 INFO:tasks.workunit.client.0.smithi117.stdout:3/657: dread - d0/d5/fc8 zero size 2024-04-20T11:09:14.241 INFO:tasks.workunit.client.0.smithi117.stdout:7/676: creat d1/d4/d6/d13/fd5 x:0 0 0 2024-04-20T11:09:14.242 INFO:tasks.workunit.client.0.smithi117.stdout:6/808: creat d3/d94/ffe x:0 0 0 2024-04-20T11:09:14.248 INFO:tasks.workunit.client.0.smithi117.stdout:6/809: dread - d3/d94/ffe zero size 2024-04-20T11:09:14.250 INFO:tasks.workunit.client.0.smithi117.stdout:0/741: rename d3/d5/d74/cbc to d3/d7/d94/cea 0 2024-04-20T11:09:14.251 INFO:tasks.workunit.client.0.smithi117.stdout:7/677: mknod d1/d4/d3d/d93/cd6 0 2024-04-20T11:09:14.256 INFO:tasks.workunit.client.0.smithi117.stdout:6/810: rename d3/d5a/d74/db3/df0/db7/fa8 to d3/d5a/d74/db3/fff 0 2024-04-20T11:09:14.262 INFO:tasks.workunit.client.0.smithi117.stdout:9/721: dread d4/d4a/d5e/f75 [0,4194304] 0 2024-04-20T11:09:14.263 INFO:tasks.workunit.client.0.smithi117.stdout:7/678: mkdir d1/d43/d59/d8a/dd7 0 2024-04-20T11:09:14.264 INFO:tasks.workunit.client.0.smithi117.stdout:6/811: write d3/df/f4f [5209186,98299] 0 2024-04-20T11:09:14.265 INFO:tasks.workunit.client.0.smithi117.stdout:0/742: getdents d3/d5 0 2024-04-20T11:09:14.267 INFO:tasks.workunit.client.0.smithi117.stdout:8/578: dwrite d0/f8f [0,4194304] 0 2024-04-20T11:09:14.270 INFO:tasks.workunit.client.0.smithi117.stdout:0/743: fsync d3/d7/d8/f31 0 2024-04-20T11:09:14.271 INFO:tasks.workunit.client.0.smithi117.stdout:9/722: mkdir d4/d35/df7 0 2024-04-20T11:09:14.272 INFO:tasks.workunit.client.0.smithi117.stdout:7/679: chown d1/d4/d39/f4d 27019 1 2024-04-20T11:09:14.283 INFO:tasks.workunit.client.0.smithi117.stdout:9/723: symlink d4/d5/d3c/ddf/d78/lf8 0 2024-04-20T11:09:14.285 INFO:tasks.workunit.client.0.smithi117.stdout:3/658: dwrite d0/f98 [0,4194304] 0 2024-04-20T11:09:14.293 INFO:tasks.workunit.client.0.smithi117.stdout:9/724: write f2 [6531217,20525] 0 2024-04-20T11:09:14.294 INFO:tasks.workunit.client.0.smithi117.stdout:5/935: sync 2024-04-20T11:09:14.306 INFO:tasks.workunit.client.0.smithi117.stdout:9/725: mkdir d4/de/db1/df9 0 2024-04-20T11:09:14.311 INFO:tasks.workunit.client.0.smithi117.stdout:3/659: write d0/d5/d7/dd/f5a [2956807,28716] 0 2024-04-20T11:09:14.313 INFO:tasks.workunit.client.0.smithi117.stdout:6/812: dwrite d3/df/f63 [0,4194304] 0 2024-04-20T11:09:14.316 INFO:tasks.workunit.client.0.smithi117.stdout:9/726: mkdir d4/de/d10/d94/dfa 0 2024-04-20T11:09:14.322 INFO:tasks.workunit.client.0.smithi117.stdout:9/727: fsync d4/d4a/d5e/de8/f6d 0 2024-04-20T11:09:14.322 INFO:tasks.workunit.client.0.smithi117.stdout:6/813: mkdir d3/df/d39/daa/d100 0 2024-04-20T11:09:14.323 INFO:tasks.workunit.client.0.smithi117.stdout:3/660: mknod d0/d5/d21/d88/d29/d66/d63/ce0 0 2024-04-20T11:09:14.324 INFO:tasks.workunit.client.0.smithi117.stdout:0/744: dwrite d3/d5/f59 [0,4194304] 0 2024-04-20T11:09:14.324 INFO:tasks.workunit.client.0.smithi117.stdout:7/680: dwrite d1/d1d/f87 [0,4194304] 0 2024-04-20T11:09:14.329 INFO:tasks.workunit.client.0.smithi117.stdout:8/579: dread d0/dc/f1a [0,4194304] 0 2024-04-20T11:09:14.329 INFO:tasks.workunit.client.0.smithi117.stdout:3/661: write d0/d5/fc [1764322,58044] 0 2024-04-20T11:09:14.331 INFO:tasks.workunit.client.0.smithi117.stdout:0/745: fdatasync d3/d7/d2c/d52/fde 0 2024-04-20T11:09:14.333 INFO:tasks.workunit.client.0.smithi117.stdout:6/814: write d3/df/f63 [4476733,49380] 0 2024-04-20T11:09:14.334 INFO:tasks.workunit.client.0.smithi117.stdout:7/681: unlink d1/d4/d6/f26 0 2024-04-20T11:09:14.335 INFO:tasks.workunit.client.0.smithi117.stdout:7/682: chown d1/d1d 310060365 1 2024-04-20T11:09:14.338 INFO:tasks.workunit.client.0.smithi117.stdout:2/805: sync 2024-04-20T11:09:14.338 INFO:tasks.workunit.client.0.smithi117.stdout:1/616: sync 2024-04-20T11:09:14.339 INFO:tasks.workunit.client.0.smithi117.stdout:4/685: sync 2024-04-20T11:09:14.345 INFO:tasks.workunit.client.0.smithi117.stdout:3/662: chown d0/d2a/c68 1 1 2024-04-20T11:09:14.352 INFO:tasks.workunit.client.0.smithi117.stdout:8/580: read d0/d8/d39/f8a [555733,114446] 0 2024-04-20T11:09:14.356 INFO:tasks.workunit.client.0.smithi117.stdout:5/936: dread de/d21/f70 [0,4194304] 0 2024-04-20T11:09:14.356 INFO:tasks.workunit.client.0.smithi117.stdout:2/806: link d1/f11 d1/d4/d83/d29/d62/df7/fff 0 2024-04-20T11:09:14.357 INFO:tasks.workunit.client.0.smithi117.stdout:7/683: getdents d1/d1d 0 2024-04-20T11:09:14.359 INFO:tasks.workunit.client.0.smithi117.stdout:8/581: creat d0/d8/d39/d7f/fc5 x:0 0 0 2024-04-20T11:09:14.360 INFO:tasks.workunit.client.0.smithi117.stdout:5/937: chown de/d41/dc2/ldd 0 1 2024-04-20T11:09:14.366 INFO:tasks.workunit.client.0.smithi117.stdout:4/686: link d1/dd/d1a/d2b/cd8 d1/dd/d1a/ceb 0 2024-04-20T11:09:14.368 INFO:tasks.workunit.client.0.smithi117.stdout:2/807: dread d1/d4/d83/d29/daa/fb7 [0,4194304] 0 2024-04-20T11:09:14.379 INFO:tasks.workunit.client.0.smithi117.stdout:4/687: creat d1/d45/dcd/fec x:0 0 0 2024-04-20T11:09:14.383 INFO:tasks.workunit.client.0.smithi117.stdout:7/684: link d1/d1d/d42/fac d1/fd8 0 2024-04-20T11:09:14.388 INFO:tasks.workunit.client.0.smithi117.stdout:0/746: dwrite d3/d7/f4d [4194304,4194304] 0 2024-04-20T11:09:14.397 INFO:tasks.workunit.client.0.smithi117.stdout:6/815: dwrite d3/df/d39/d99/fcd [0,4194304] 0 2024-04-20T11:09:14.401 INFO:tasks.workunit.client.0.smithi117.stdout:0/747: readlink d3/d7/d16/l8c 0 2024-04-20T11:09:14.402 INFO:tasks.workunit.client.0.smithi117.stdout:1/617: dread d6/f79 [0,4194304] 0 2024-04-20T11:09:14.404 INFO:tasks.workunit.client.0.smithi117.stdout:3/663: dwrite d0/d5/d21/d88/d69/f9b [0,4194304] 0 2024-04-20T11:09:14.406 INFO:tasks.workunit.client.0.smithi117.stdout:7/685: rename d1/d4/d39/f7a to d1/d4/d3d/dce/fd9 0 2024-04-20T11:09:14.411 INFO:tasks.workunit.client.0.smithi117.stdout:1/618: chown d6/d35/d63/c7f 408 1 2024-04-20T11:09:14.412 INFO:tasks.workunit.client.0.smithi117.stdout:2/808: creat d1/d4/d83/d29/f100 x:0 0 0 2024-04-20T11:09:14.414 INFO:tasks.workunit.client.0.smithi117.stdout:5/938: dwrite de/d21/d84/f87 [0,4194304] 0 2024-04-20T11:09:14.415 INFO:tasks.workunit.client.0.smithi117.stdout:8/582: dread d0/d8/d39/d7f/fb9 [0,4194304] 0 2024-04-20T11:09:14.418 INFO:tasks.workunit.client.0.smithi117.stdout:0/748: creat d3/d7/d16/feb x:0 0 0 2024-04-20T11:09:14.418 INFO:tasks.workunit.client.0.smithi117.stdout:4/688: creat d1/dd/d1a/d43/fed x:0 0 0 2024-04-20T11:09:14.422 INFO:tasks.workunit.client.0.smithi117.stdout:0/749: fdatasync d3/d7/d8/f31 0 2024-04-20T11:09:14.429 INFO:tasks.workunit.client.0.smithi117.stdout:6/816: creat d3/d4/d2e/d37/d8e/f101 x:0 0 0 2024-04-20T11:09:14.429 INFO:tasks.workunit.client.0.smithi117.stdout:2/809: fdatasync d1/d15/d1d/fa1 0 2024-04-20T11:09:14.429 INFO:tasks.workunit.client.0.smithi117.stdout:1/619: mknod d6/d16/d3a/d5a/d5c/d61/d71/cca 0 2024-04-20T11:09:14.431 INFO:tasks.workunit.client.0.smithi117.stdout:9/728: dread d4/f2e [0,4194304] 0 2024-04-20T11:09:14.433 INFO:tasks.workunit.client.0.smithi117.stdout:7/686: rename d1/d4/d39/c27 to d1/d4/d6/d2b/d64/cda 0 2024-04-20T11:09:14.435 INFO:tasks.workunit.client.0.smithi117.stdout:4/689: stat d1/d9/f96 0 2024-04-20T11:09:14.435 INFO:tasks.workunit.client.0.smithi117.stdout:2/810: read d1/d4/d83/f7e [4084992,57191] 0 2024-04-20T11:09:14.437 INFO:tasks.workunit.client.0.smithi117.stdout:1/620: creat d6/d1f/d30/d32/fcb x:0 0 0 2024-04-20T11:09:14.437 INFO:tasks.workunit.client.0.smithi117.stdout:3/664: getdents d0/d5/d21/d88/d29/d66/dbe/dd0 0 2024-04-20T11:09:14.439 INFO:tasks.workunit.client.0.smithi117.stdout:1/621: dread - d6/d16/fad zero size 2024-04-20T11:09:14.440 INFO:tasks.workunit.client.0.smithi117.stdout:0/750: rename d3/d7/d16/fe7 to d3/d7/d94/fec 0 2024-04-20T11:09:14.442 INFO:tasks.workunit.client.0.smithi117.stdout:9/729: write d4/d5/d57/f99 [369913,1215] 0 2024-04-20T11:09:14.443 INFO:tasks.workunit.client.0.smithi117.stdout:6/817: mknod d3/d4/d2e/d37/d8e/db2/c102 0 2024-04-20T11:09:14.443 INFO:tasks.workunit.client.0.smithi117.stdout:1/622: chown d6/d1f/d30/d49/d69/d6e/d6f/cc0 1048876 1 2024-04-20T11:09:14.445 INFO:tasks.workunit.client.0.smithi117.stdout:3/665: creat d0/d5/d21/d88/d29/d66/dbe/fe1 x:0 0 0 2024-04-20T11:09:14.450 INFO:tasks.workunit.client.0.smithi117.stdout:9/730: mknod d4/d5/d3c/ddf/cfb 0 2024-04-20T11:09:14.451 INFO:tasks.workunit.client.0.smithi117.stdout:2/811: link d1/d15/d1d/c2f d1/da5/c101 0 2024-04-20T11:09:14.454 INFO:tasks.workunit.client.0.smithi117.stdout:3/666: stat d0/d5/d21/d88/d29/db3/cde 0 2024-04-20T11:09:14.454 INFO:tasks.workunit.client.0.smithi117.stdout:7/687: rmdir d1/d43/dc4 0 2024-04-20T11:09:14.454 INFO:tasks.workunit.client.0.smithi117.stdout:9/731: chown d4/d4a/d72 459962009 1 2024-04-20T11:09:14.459 INFO:tasks.workunit.client.0.smithi117.stdout:0/751: read d3/dab/d54/fa3 [672710,79439] 0 2024-04-20T11:09:14.462 INFO:tasks.workunit.client.0.smithi117.stdout:4/690: write d1/d45/d4a/f6c [965394,61444] 0 2024-04-20T11:09:14.462 INFO:tasks.workunit.client.0.smithi117.stdout:8/583: dwrite d0/d8/f65 [4194304,4194304] 0 2024-04-20T11:09:14.462 INFO:tasks.workunit.client.0.smithi117.stdout:2/812: read d1/d4/fa0 [782279,97914] 0 2024-04-20T11:09:14.463 INFO:tasks.workunit.client.0.smithi117.stdout:6/818: write d3/f28 [440576,58102] 0 2024-04-20T11:09:14.468 INFO:tasks.workunit.client.0.smithi117.stdout:5/939: dwrite de/d41/d47/d55/fec [0,4194304] 0 2024-04-20T11:09:14.469 INFO:tasks.workunit.client.0.smithi117.stdout:3/667: rename d0/d5/f25 to d0/d5/fe2 0 2024-04-20T11:09:14.473 INFO:tasks.workunit.client.0.smithi117.stdout:3/668: symlink d0/d5/d21/d94/da7/le3 0 2024-04-20T11:09:14.483 INFO:tasks.workunit.client.0.smithi117.stdout:1/623: dwrite d6/d35/f97 [0,4194304] 0 2024-04-20T11:09:14.484 INFO:tasks.workunit.client.0.smithi117.stdout:5/940: write de/d41/d47/d55/d56/dc8/ff3 [105294,27351] 0 2024-04-20T11:09:14.486 INFO:tasks.workunit.client.0.smithi117.stdout:7/688: rename d1/d4/d6/d13/l37 to d1/d1d/d42/d2d/ldb 0 2024-04-20T11:09:14.488 INFO:tasks.workunit.client.0.smithi117.stdout:9/732: link d4/d4a/d72/lc1 d4/d5/d57/lfc 0 2024-04-20T11:09:14.488 INFO:tasks.workunit.client.0.smithi117.stdout:2/813: symlink d1/d4/d83/d29/daa/l102 0 2024-04-20T11:09:14.489 INFO:tasks.workunit.client.0.smithi117.stdout:1/624: read - d6/d16/d3a/d5a/d5c/d61/d98/fbe zero size 2024-04-20T11:09:14.490 INFO:tasks.workunit.client.0.smithi117.stdout:6/819: mkdir d3/df/d39/daa/d100/d103 0 2024-04-20T11:09:14.492 INFO:tasks.workunit.client.0.smithi117.stdout:1/625: fsync d6/d16/d3a/d5a/daf/fc5 0 2024-04-20T11:09:14.493 INFO:tasks.workunit.client.0.smithi117.stdout:2/814: chown d1/d4/dd/d17/d7a/d49/fea 31 1 2024-04-20T11:09:14.495 INFO:tasks.workunit.client.0.smithi117.stdout:8/584: rename d0/dc/d2e/la2 to d0/d8/d11/d6d/d58/lc6 0 2024-04-20T11:09:14.498 INFO:tasks.workunit.client.0.smithi117.stdout:5/941: mknod de/d16/d9d/c148 0 2024-04-20T11:09:14.499 INFO:tasks.workunit.client.0.smithi117.stdout:9/733: truncate d4/de/f1b 3461481 0 2024-04-20T11:09:14.501 INFO:tasks.workunit.client.0.smithi117.stdout:8/585: dread - d0/d8/d11/d24/d76/f96 zero size 2024-04-20T11:09:14.501 INFO:tasks.workunit.client.0.smithi117.stdout:2/815: dread - d1/d4/d83/d29/d2e/d53/d93/f9e zero size 2024-04-20T11:09:14.507 INFO:tasks.workunit.client.0.smithi117.stdout:5/942: mkdir de/d21/df5/d57/d149 0 2024-04-20T11:09:14.515 INFO:tasks.workunit.client.0.smithi117.stdout:8/586: mkdir d0/dc/d49/d5b/da9/dc7 0 2024-04-20T11:09:14.521 INFO:tasks.workunit.client.0.smithi117.stdout:8/587: dread - d0/d8/d39/d7f/fc5 zero size 2024-04-20T11:09:14.526 INFO:tasks.workunit.client.0.smithi117.stdout:7/689: dwrite d1/d4/d6/d13/fd5 [0,4194304] 0 2024-04-20T11:09:14.526 INFO:tasks.workunit.client.0.smithi117.stdout:8/588: readlink d0/d8/d39/l52 0 2024-04-20T11:09:14.531 INFO:tasks.workunit.client.0.smithi117.stdout:6/820: dwrite d3/d5a/d74/db3/df0/fcf [0,4194304] 0 2024-04-20T11:09:14.531 INFO:tasks.workunit.client.0.smithi117.stdout:5/943: rename de/d16/d9d/db4/d130 to de/d41/d47/d14a 0 2024-04-20T11:09:14.534 INFO:tasks.workunit.client.0.smithi117.stdout:2/816: link d1/da5/df2/fb1 d1/d4/d83/d29/d2e/d53/f103 0 2024-04-20T11:09:14.537 INFO:tasks.workunit.client.0.smithi117.stdout:8/589: dread - d0/d8/d11/d6d/fba zero size 2024-04-20T11:09:14.538 INFO:tasks.workunit.client.0.smithi117.stdout:3/669: truncate d0/d5/d73/fd5 4188714 0 2024-04-20T11:09:14.546 INFO:tasks.workunit.client.0.smithi117.stdout:8/590: chown d0/f20 15386219 1 2024-04-20T11:09:14.548 INFO:tasks.workunit.client.0.smithi117.stdout:7/690: mknod d1/d1d/d42/d89/da9/cdc 0 2024-04-20T11:09:14.548 INFO:tasks.workunit.client.0.smithi117.stdout:6/821: creat d3/d5a/d74/db3/df0/db7/f104 x:0 0 0 2024-04-20T11:09:14.550 INFO:tasks.workunit.client.0.smithi117.stdout:2/817: symlink d1/d20/de0/l104 0 2024-04-20T11:09:14.552 INFO:tasks.workunit.client.0.smithi117.stdout:5/944: truncate de/d16/d9d/db4/df1/f103 1864263 0 2024-04-20T11:09:14.554 INFO:tasks.workunit.client.0.smithi117.stdout:0/752: dread d3/d7/d20/f68 [0,4194304] 0 2024-04-20T11:09:14.555 INFO:tasks.workunit.client.0.smithi117.stdout:9/734: write d4/d5/d3c/ddf/f6b [3728129,46651] 0 2024-04-20T11:09:14.557 INFO:tasks.workunit.client.0.smithi117.stdout:1/626: dread d6/d1f/f3d [0,4194304] 0 2024-04-20T11:09:14.559 INFO:tasks.workunit.client.0.smithi117.stdout:7/691: creat d1/d1d/fdd x:0 0 0 2024-04-20T11:09:14.562 INFO:tasks.workunit.client.0.smithi117.stdout:3/670: rename d0/d5/d21/d88/d69/f6d to d0/d5/d21/d88/d29/d66/d63/fe4 0 2024-04-20T11:09:14.563 INFO:tasks.workunit.client.0.smithi117.stdout:6/822: mkdir d3/d5a/d74/db3/df0/db7/d105 0 2024-04-20T11:09:14.567 INFO:tasks.workunit.client.0.smithi117.stdout:5/945: symlink de/d21/df5/l14b 0 2024-04-20T11:09:14.569 INFO:tasks.workunit.client.0.smithi117.stdout:0/753: stat d3/ca0 0 2024-04-20T11:09:14.572 INFO:tasks.workunit.client.0.smithi117.stdout:1/627: mknod d6/d1f/d30/d49/d69/d6e/d6f/d73/ccc 0 2024-04-20T11:09:14.580 INFO:tasks.workunit.client.0.smithi117.stdout:2/818: creat d1/d4/d83/d29/d3f/f105 x:0 0 0 2024-04-20T11:09:14.580 INFO:tasks.workunit.client.0.smithi117.stdout:6/823: mkdir d3/d5a/d74/db3/df0/d106 0 2024-04-20T11:09:14.582 INFO:tasks.workunit.client.0.smithi117.stdout:9/735: symlink d4/d35/df7/lfd 0 2024-04-20T11:09:14.583 INFO:tasks.workunit.client.0.smithi117.stdout:8/591: write d0/d8/d39/f54 [4872493,96569] 0 2024-04-20T11:09:14.585 INFO:tasks.workunit.client.0.smithi117.stdout:5/946: mkdir de/d21/df5/d57/d14c 0 2024-04-20T11:09:14.587 INFO:tasks.workunit.client.0.smithi117.stdout:7/692: creat d1/d4/d6/d54/db4/fde x:0 0 0 2024-04-20T11:09:14.587 INFO:tasks.workunit.client.0.smithi117.stdout:0/754: mknod d3/d7/d94/ced 0 2024-04-20T11:09:14.591 INFO:tasks.workunit.client.0.smithi117.stdout:3/671: dwrite d0/d5/d21/d88/d29/d66/dbe/fe1 [0,4194304] 0 2024-04-20T11:09:14.592 INFO:tasks.workunit.client.0.smithi117.stdout:1/628: creat d6/d1f/d30/d49/d69/d6e/d6f/d73/fcd x:0 0 0 2024-04-20T11:09:14.594 INFO:tasks.workunit.client.0.smithi117.stdout:9/736: chown d4/de/f2d 38131622 1 2024-04-20T11:09:14.595 INFO:tasks.workunit.client.0.smithi117.stdout:2/819: symlink d1/d4/d83/d29/d2e/d53/l106 0 2024-04-20T11:09:14.603 INFO:tasks.workunit.client.0.smithi117.stdout:6/824: rename d3/d5a/d74/db3/dbc/l56 to d3/d4/d2e/dc0/l107 0 2024-04-20T11:09:14.608 INFO:tasks.workunit.client.0.smithi117.stdout:0/755: mknod d3/d7/d20/d39/d9a/cee 0 2024-04-20T11:09:14.618 INFO:tasks.workunit.client.0.smithi117.stdout:8/592: dread d0/d8/d11/d24/d2b/f2c [0,4194304] 0 2024-04-20T11:09:14.622 INFO:tasks.workunit.client.0.smithi117.stdout:6/825: dwrite d3/d5a/d74/fc3 [0,4194304] 0 2024-04-20T11:09:14.624 INFO:tasks.workunit.client.0.smithi117.stdout:1/629: mkdir d6/d16/dce 0 2024-04-20T11:09:14.625 INFO:tasks.workunit.client.0.smithi117.stdout:2/820: mknod d1/d15/d1d/db3/c107 0 2024-04-20T11:09:14.629 INFO:tasks.workunit.client.0.smithi117.stdout:0/756: write d3/d7/d20/d39/d9a/dd0/d22/f6f [126442,64418] 0 2024-04-20T11:09:14.629 INFO:tasks.workunit.client.0.smithi117.stdout:5/947: creat de/d41/d47/d55/f14d x:0 0 0 2024-04-20T11:09:14.629 INFO:tasks.workunit.client.0.smithi117.stdout:2/821: chown d1/d4/dd/d17/ffe 28335206 1 2024-04-20T11:09:14.630 INFO:tasks.workunit.client.0.smithi117.stdout:8/593: truncate d0/d8/d11/d24/d53/f87 1261507 0 2024-04-20T11:09:14.632 INFO:tasks.workunit.client.0.smithi117.stdout:6/826: mknod d3/df/c108 0 2024-04-20T11:09:14.633 INFO:tasks.workunit.client.0.smithi117.stdout:7/693: dread d1/d4/d6/d17/d3f/f65 [0,4194304] 0 2024-04-20T11:09:14.636 INFO:tasks.workunit.client.0.smithi117.stdout:6/827: truncate d3/d3a/f6b 2757707 0 2024-04-20T11:09:14.636 INFO:tasks.workunit.client.0.smithi117.stdout:8/594: truncate d0/d8/d39/d7f/fc5 379036 0 2024-04-20T11:09:14.638 INFO:tasks.workunit.client.0.smithi117.stdout:0/757: truncate d3/dab/fa8 4305102 0 2024-04-20T11:09:14.639 INFO:tasks.workunit.client.0.smithi117.stdout:2/822: read d1/d9b/fb6 [2848068,72104] 0 2024-04-20T11:09:14.640 INFO:tasks.workunit.client.0.smithi117.stdout:7/694: dread - d1/d1d/d42/d2d/d6c/f7d zero size 2024-04-20T11:09:14.645 INFO:tasks.workunit.client.0.smithi117.stdout:2/823: chown d1/l3d 34027 1 2024-04-20T11:09:14.647 INFO:tasks.workunit.client.0.smithi117.stdout:8/595: rename d0/dc/d49/d63/f82 to d0/d9/dbb/fc8 0 2024-04-20T11:09:14.649 INFO:tasks.workunit.client.0.smithi117.stdout:0/758: fdatasync d3/d7/d2c/d52/f70 0 2024-04-20T11:09:14.650 INFO:tasks.workunit.client.0.smithi117.stdout:8/596: readlink d0/d8/d11/d24/d86/laf 0 2024-04-20T11:09:14.653 INFO:tasks.workunit.client.0.smithi117.stdout:0/759: dread - d3/dab/d54/fe1 zero size 2024-04-20T11:09:14.654 INFO:tasks.workunit.client.0.smithi117.stdout:5/948: creat de/d21/df5/d57/dd5/f14e x:0 0 0 2024-04-20T11:09:14.659 INFO:tasks.workunit.client.0.smithi117.stdout:6/828: dwrite d3/d5a/d74/db3/df0/db7/f87 [0,4194304] 0 2024-04-20T11:09:14.662 INFO:tasks.workunit.client.0.smithi117.stdout:6/829: readlink d3/df/l3b 0 2024-04-20T11:09:14.663 INFO:tasks.workunit.client.0.smithi117.stdout:1/630: dread d6/d1f/f80 [0,4194304] 0 2024-04-20T11:09:14.668 INFO:tasks.workunit.client.0.smithi117.stdout:8/597: creat d0/d8/d39/fc9 x:0 0 0 2024-04-20T11:09:14.677 INFO:tasks.workunit.client.0.smithi117.stdout:0/760: rename d3/d7/d8/c5e to d3/d7/d20/d47/d78/cef 0 2024-04-20T11:09:14.677 INFO:tasks.workunit.client.0.smithi117.stdout:1/631: fsync d6/d1f/f80 0 2024-04-20T11:09:14.699 INFO:tasks.workunit.client.0.smithi117.stdout:7/695: dwrite d1/d43/d59/f69 [0,4194304] 0 2024-04-20T11:09:14.700 INFO:tasks.workunit.client.0.smithi117.stdout:8/598: mknod d0/d9/cca 0 2024-04-20T11:09:14.700 INFO:tasks.workunit.client.0.smithi117.stdout:1/632: mknod d6/d16/d3a/ccf 0 2024-04-20T11:09:14.701 INFO:tasks.workunit.client.0.smithi117.stdout:2/824: dwrite d1/d4/f5 [4194304,4194304] 0 2024-04-20T11:09:14.703 INFO:tasks.workunit.client.0.smithi117.stdout:0/761: truncate d3/d7/d20/f4c 5198394 0 2024-04-20T11:09:14.705 INFO:tasks.workunit.client.0.smithi117.stdout:7/696: readlink d1/d4/d6/d17/l62 0 2024-04-20T11:09:14.710 INFO:tasks.workunit.client.0.smithi117.stdout:7/697: truncate d1/d1d/fdd 88909 0 2024-04-20T11:09:14.711 INFO:tasks.workunit.client.0.smithi117.stdout:5/949: dwrite de/d21/df5/d57/fe0 [0,4194304] 0 2024-04-20T11:09:14.717 INFO:tasks.workunit.client.0.smithi117.stdout:5/950: dread - de/d41/d47/fe2 zero size 2024-04-20T11:09:14.722 INFO:tasks.workunit.client.0.smithi117.stdout:6/830: truncate d3/d3a/d7f/fa4 3630185 0 2024-04-20T11:09:14.724 INFO:tasks.workunit.client.0.smithi117.stdout:6/831: dread - d3/d4/d2e/d37/fef zero size 2024-04-20T11:09:14.728 INFO:tasks.workunit.client.0.smithi117.stdout:5/951: creat de/d21/d84/f14f x:0 0 0 2024-04-20T11:09:14.730 INFO:tasks.workunit.client.0.smithi117.stdout:2/825: link d1/d4/dd/d17/f1e d1/d4/d83/d29/d3f/df6/f108 0 2024-04-20T11:09:14.740 INFO:tasks.workunit.client.0.smithi117.stdout:5/952: mkdir de/dd0/d10e/d4e/da1/de7/d123/d77/d150 0 2024-04-20T11:09:14.753 INFO:tasks.workunit.client.0.smithi117.stdout:5/953: symlink de/dd0/d10c/l151 0 2024-04-20T11:09:14.756 INFO:tasks.workunit.client.0.smithi117.stdout:1/633: dwrite d6/d1f/f3e [0,4194304] 0 2024-04-20T11:09:14.758 INFO:tasks.workunit.client.0.smithi117.stdout:7/698: dread d1/d4/d3d/f5a [0,4194304] 0 2024-04-20T11:09:14.763 INFO:tasks.workunit.client.0.smithi117.stdout:0/762: truncate d3/d7/d20/d39/d9a/dd0/d22/d85/ddb/fc9 3825493 0 2024-04-20T11:09:14.767 INFO:tasks.workunit.client.0.smithi117.stdout:8/599: dwrite d0/d8/d11/d24/d27/d47/f6c [0,4194304] 0 2024-04-20T11:09:14.770 INFO:tasks.workunit.client.0.smithi117.stdout:7/699: rmdir d1/d4/d3d/dbc 0 2024-04-20T11:09:14.773 INFO:tasks.workunit.client.0.smithi117.stdout:7/700: fsync d1/d1d/d42/d2d/fb7 0 2024-04-20T11:09:14.774 INFO:tasks.workunit.client.0.smithi117.stdout:1/634: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/fd0 x:0 0 0 2024-04-20T11:09:14.774 INFO:tasks.workunit.client.0.smithi117.stdout:0/763: symlink d3/d7/d20/d47/d78/lf0 0 2024-04-20T11:09:14.777 INFO:tasks.workunit.client.0.smithi117.stdout:2/826: write d1/d20/fbb [901338,34035] 0 2024-04-20T11:09:14.784 INFO:tasks.workunit.client.0.smithi117.stdout:6/832: dwrite d3/d4/d2e/fa6 [0,4194304] 0 2024-04-20T11:09:14.785 INFO:tasks.workunit.client.0.smithi117.stdout:8/600: getdents d0/d8/d11/d24/d53 0 2024-04-20T11:09:14.791 INFO:tasks.workunit.client.0.smithi117.stdout:7/701: mknod d1/d4/d6/d2b/cdf 0 2024-04-20T11:09:14.799 INFO:tasks.workunit.client.0.smithi117.stdout:6/833: symlink d3/d5a/d74/db3/dd0/l109 0 2024-04-20T11:09:14.803 INFO:tasks.workunit.client.0.smithi117.stdout:8/601: truncate d0/d9/f2a 2543527 0 2024-04-20T11:09:14.803 INFO:tasks.workunit.client.0.smithi117.stdout:5/954: dread de/d21/df5/d57/f62 [0,4194304] 0 2024-04-20T11:09:14.804 INFO:tasks.workunit.client.0.smithi117.stdout:1/635: creat d6/d21/fd1 x:0 0 0 2024-04-20T11:09:14.805 INFO:tasks.workunit.client.0.smithi117.stdout:8/602: write d0/d9/f33 [3792812,79885] 0 2024-04-20T11:09:14.807 INFO:tasks.workunit.client.0.smithi117.stdout:2/827: unlink d1/d4/dd/d17/d7a/d88/lc6 0 2024-04-20T11:09:14.810 INFO:tasks.workunit.client.0.smithi117.stdout:6/834: rename d3/d5a/d74/db3/lc5 to d3/d4/d2e/d37/d8e/db2/l10a 0 2024-04-20T11:09:14.813 INFO:tasks.workunit.client.0.smithi117.stdout:5/955: mknod de/d16/d9d/db4/df1/c152 0 2024-04-20T11:09:14.824 INFO:tasks.workunit.client.0.smithi117.stdout:5/956: rename de/dd0/d10e/d4e/da1/de7/f139 to de/d21/df5/d57/dd5/dce/f153 0 2024-04-20T11:09:14.826 INFO:tasks.workunit.client.0.smithi117.stdout:0/764: dwrite d3/d5/f37 [0,4194304] 0 2024-04-20T11:09:14.829 INFO:tasks.workunit.client.0.smithi117.stdout:1/636: dwrite d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/fd0 [0,4194304] 0 2024-04-20T11:09:14.833 INFO:tasks.workunit.client.0.smithi117.stdout:6/835: symlink d3/df/d39/daa/d100/d103/l10b 0 2024-04-20T11:09:14.833 INFO:tasks.workunit.client.0.smithi117.stdout:0/765: write d3/d5/d74/fa9 [1475849,119029] 0 2024-04-20T11:09:14.835 INFO:tasks.workunit.client.0.smithi117.stdout:2/828: rmdir d1/d4 39 2024-04-20T11:09:14.838 INFO:tasks.workunit.client.0.smithi117.stdout:1/637: read d6/d1f/d30/d32/f6a [2099217,57170] 0 2024-04-20T11:09:14.842 INFO:tasks.workunit.client.0.smithi117.stdout:7/702: truncate d1/d43/f91 1220669 0 2024-04-20T11:09:14.843 INFO:tasks.workunit.client.0.smithi117.stdout:8/603: dread d0/f20 [0,4194304] 0 2024-04-20T11:09:14.851 INFO:tasks.workunit.client.0.smithi117.stdout:5/957: link de/d21/d84/l9c de/d5b/d65/l154 0 2024-04-20T11:09:14.854 INFO:tasks.workunit.client.0.smithi117.stdout:6/836: read d3/d4/d2e/d4e/fa2 [3510574,78367] 0 2024-04-20T11:09:14.854 INFO:tasks.workunit.client.0.smithi117.stdout:7/703: mknod d1/d1d/d4e/d72/ce0 0 2024-04-20T11:09:14.855 INFO:tasks.workunit.client.0.smithi117.stdout:8/604: mknod d0/d8/d39/ccb 0 2024-04-20T11:09:14.857 INFO:tasks.workunit.client.0.smithi117.stdout:2/829: mkdir d1/d4/d83/d29/d62/d109 0 2024-04-20T11:09:14.859 INFO:tasks.workunit.client.0.smithi117.stdout:0/766: dwrite d3/d7/d2c/d52/fde [0,4194304] 0 2024-04-20T11:09:14.860 INFO:tasks.workunit.client.0.smithi117.stdout:8/605: dread - d0/dc/d49/d63/fb5 zero size 2024-04-20T11:09:14.862 INFO:tasks.workunit.client.0.smithi117.stdout:1/638: rmdir d6/d1f/d30/d49/d69/d6e/d6f/d73/dc2 0 2024-04-20T11:09:14.863 INFO:tasks.workunit.client.0.smithi117.stdout:2/830: fsync d1/d4/fa0 0 2024-04-20T11:09:14.865 INFO:tasks.workunit.client.0.smithi117.stdout:0/767: chown d3/d7/d16/le4 1058131661 1 2024-04-20T11:09:14.880 INFO:tasks.workunit.client.0.smithi117.stdout:7/704: truncate d1/fd8 2589200 0 2024-04-20T11:09:14.881 INFO:tasks.workunit.client.0.smithi117.stdout:2/831: mknod d1/d4/d83/d29/daa/c10a 0 2024-04-20T11:09:14.887 INFO:tasks.workunit.client.0.smithi117.stdout:8/606: write d0/d8/d11/d6d/fba [534572,74291] 0 2024-04-20T11:09:14.890 INFO:tasks.workunit.client.0.smithi117.stdout:0/768: dwrite d3/d7/d8/f13 [0,4194304] 0 2024-04-20T11:09:14.891 INFO:tasks.workunit.client.0.smithi117.stdout:8/607: write d0/d8/d11/d24/d27/d99/f9d [988511,122057] 0 2024-04-20T11:09:14.894 INFO:tasks.workunit.client.0.smithi117.stdout:8/608: write d0/d8/d11/fac [383840,520] 0 2024-04-20T11:09:14.896 INFO:tasks.workunit.client.0.smithi117.stdout:5/958: dwrite de/d16/d9d/db4/df1/f101 [0,4194304] 0 2024-04-20T11:09:14.898 INFO:tasks.workunit.client.0.smithi117.stdout:5/959: chown de/d21/df5/c37 420676677 1 2024-04-20T11:09:14.900 INFO:tasks.workunit.client.0.smithi117.stdout:5/960: write de/db7/f109 [243399,126486] 0 2024-04-20T11:09:14.912 INFO:tasks.workunit.client.0.smithi117.stdout:6/837: link d3/d3a/fd6 d3/d4/d2e/f10c 0 2024-04-20T11:09:14.916 INFO:tasks.workunit.client.0.smithi117.stdout:2/832: creat d1/d4/dd/d17/d7a/f10b x:0 0 0 2024-04-20T11:09:14.919 INFO:tasks.workunit.client.0.smithi117.stdout:5/961: creat de/d41/d4b/f155 x:0 0 0 2024-04-20T11:09:14.919 INFO:tasks.workunit.client.0.smithi117.stdout:6/838: unlink d3/d4/d2e/d37/d8e/db2/c102 0 2024-04-20T11:09:14.926 INFO:tasks.workunit.client.0.smithi117.stdout:1/639: rename d6/d16/d3a/d5a/d5c/d61/d91 to d6/dd2 0 2024-04-20T11:09:14.926 INFO:tasks.workunit.client.0.smithi117.stdout:6/839: truncate d3/d4/d2e/d3f/f77 4988928 0 2024-04-20T11:09:14.929 INFO:tasks.workunit.client.0.smithi117.stdout:5/962: mkdir de/dd0/d10e/d4e/da1/de7/d123/dfd/d156 0 2024-04-20T11:09:14.929 INFO:tasks.workunit.client.0.smithi117.stdout:2/833: fdatasync d1/da5/df2/f98 0 2024-04-20T11:09:14.931 INFO:tasks.workunit.client.0.smithi117.stdout:0/769: rename d3/d7/d16/l8c to d3/d7/d20/d39/d9a/dd0/db2/dbd/lf1 0 2024-04-20T11:09:14.933 INFO:tasks.workunit.client.0.smithi117.stdout:8/609: dread d0/dc/d2e/d89/fbe [4194304,4194304] 0 2024-04-20T11:09:14.936 INFO:tasks.workunit.client.0.smithi117.stdout:8/610: write d0/d8/d39/f54 [1930528,23782] 0 2024-04-20T11:09:14.939 INFO:tasks.workunit.client.0.smithi117.stdout:2/834: mknod d1/d4/d83/d8e/c10c 0 2024-04-20T11:09:14.943 INFO:tasks.workunit.client.0.smithi117.stdout:2/835: write d1/d4/d83/d29/fda [642266,97268] 0 2024-04-20T11:09:14.943 INFO:tasks.workunit.client.0.smithi117.stdout:8/611: mkdir d0/dc/d49/d5b/dcc 0 2024-04-20T11:09:14.952 INFO:tasks.workunit.client.0.smithi117.stdout:8/612: rename d0/dc/d49/d5b/dcc to d0/dc/d49/d5b/dcd 0 2024-04-20T11:09:14.953 INFO:tasks.workunit.client.0.smithi117.stdout:2/836: mknod d1/d4/d83/d29/d2e/d53/d58/da8/c10d 0 2024-04-20T11:09:14.959 INFO:tasks.workunit.client.0.smithi117.stdout:6/840: dread d3/d4/d2e/d3f/fc9 [0,4194304] 0 2024-04-20T11:09:14.962 INFO:tasks.workunit.client.0.smithi117.stdout:8/613: mkdir d0/d9/d3a/d7c/dce 0 2024-04-20T11:09:14.967 INFO:tasks.workunit.client.0.smithi117.stdout:7/705: write d1/f86 [12581964,95408] 0 2024-04-20T11:09:14.969 INFO:tasks.workunit.client.0.smithi117.stdout:1/640: dread d6/f13 [4194304,4194304] 0 2024-04-20T11:09:14.969 INFO:tasks.workunit.client.0.smithi117.stdout:8/614: mknod d0/d8/d11/ccf 0 2024-04-20T11:09:14.971 INFO:tasks.workunit.client.0.smithi117.stdout:5/963: dread de/f34 [0,4194304] 0 2024-04-20T11:09:14.972 INFO:tasks.workunit.client.0.smithi117.stdout:7/706: chown d1/d1d/d77/fab 1980451 1 2024-04-20T11:09:14.973 INFO:tasks.workunit.client.0.smithi117.stdout:1/641: read d6/d35/f97 [2398348,51901] 0 2024-04-20T11:09:14.974 INFO:tasks.workunit.client.0.smithi117.stdout:6/841: getdents d3/d5a/d74/db3/dbc/d9b 0 2024-04-20T11:09:14.975 INFO:tasks.workunit.client.0.smithi117.stdout:7/707: write d1/f10 [4439828,64157] 0 2024-04-20T11:09:14.975 INFO:tasks.workunit.client.0.smithi117.stdout:5/964: fdatasync de/dd0/d10e/da7/feb 0 2024-04-20T11:09:14.977 INFO:tasks.workunit.client.0.smithi117.stdout:8/615: unlink d0/l6 0 2024-04-20T11:09:14.982 INFO:tasks.workunit.client.0.smithi117.stdout:7/708: creat d1/d4/d6/d2b/d64/fe1 x:0 0 0 2024-04-20T11:09:14.983 INFO:tasks.workunit.client.0.smithi117.stdout:6/842: symlink d3/df/d39/daa/dee/l10d 0 2024-04-20T11:09:14.984 INFO:tasks.workunit.client.0.smithi117.stdout:6/843: stat d3/f28 0 2024-04-20T11:09:14.984 INFO:tasks.workunit.client.0.smithi117.stdout:7/709: chown d1/d1d/d42/faf 204387520 1 2024-04-20T11:09:14.986 INFO:tasks.workunit.client.0.smithi117.stdout:1/642: fdatasync d6/d16/d3a/d5a/d5c/d61/d71/d6b/f6c 0 2024-04-20T11:09:14.988 INFO:tasks.workunit.client.0.smithi117.stdout:8/616: unlink d0/d8/d11/d6d/d58/l92 0 2024-04-20T11:09:14.990 INFO:tasks.workunit.client.0.smithi117.stdout:8/617: write d0/d8/d39/fc9 [567779,54500] 0 2024-04-20T11:09:14.992 INFO:tasks.workunit.client.0.smithi117.stdout:5/965: rename de/d21/df5/d57/d142 to de/dd0/d10e/d4e/da1/d157 0 2024-04-20T11:09:14.993 INFO:tasks.workunit.client.0.smithi117.stdout:6/844: mknod d3/d4/d2e/dc0/c10e 0 2024-04-20T11:09:14.995 INFO:tasks.workunit.client.0.smithi117.stdout:1/643: write d6/d1f/f45 [8072185,45298] 0 2024-04-20T11:09:14.999 INFO:tasks.workunit.client.0.smithi117.stdout:7/710: mknod d1/d43/d59/d8a/dd7/ce2 0 2024-04-20T11:09:15.000 INFO:tasks.workunit.client.0.smithi117.stdout:2/837: dread d1/d15/d1d/f50 [0,4194304] 0 2024-04-20T11:09:15.006 INFO:tasks.workunit.client.0.smithi117.stdout:7/711: symlink d1/d43/d59/d8a/le3 0 2024-04-20T11:09:15.008 INFO:tasks.workunit.client.0.smithi117.stdout:7/712: chown d1/c38 1263 1 2024-04-20T11:09:15.009 INFO:tasks.workunit.client.0.smithi117.stdout:1/644: mknod d6/d16/d3a/d5a/d5c/d61/d71/d7d/da2/cd3 0 2024-04-20T11:09:15.011 INFO:tasks.workunit.client.0.smithi117.stdout:6/845: dwrite d3/df/d39/d99/fcd [4194304,4194304] 0 2024-04-20T11:09:15.019 INFO:tasks.workunit.client.0.smithi117.stdout:1/645: rename d6/d1f/f80 to d6/fd4 0 2024-04-20T11:09:15.020 INFO:tasks.workunit.client.0.smithi117.stdout:1/646: dread - d6/d16/fbb zero size 2024-04-20T11:09:15.021 INFO:tasks.workunit.client.0.smithi117.stdout:6/846: getdents d3/d3a 0 2024-04-20T11:09:15.029 INFO:tasks.workunit.client.0.smithi117.stdout:1/647: write d6/d16/d3a/d5a/d5c/d61/d71/d6b/f6c [700972,27306] 0 2024-04-20T11:09:15.033 INFO:tasks.workunit.client.0.smithi117.stdout:8/618: write d0/d8/d11/d24/f46 [5070963,77899] 0 2024-04-20T11:09:15.039 INFO:tasks.workunit.client.0.smithi117.stdout:5/966: truncate de/d5b/d65/fd6 421671 0 2024-04-20T11:09:15.039 INFO:tasks.workunit.client.0.smithi117.stdout:2/838: write d1/d4/d83/f18 [4727413,102224] 0 2024-04-20T11:09:15.042 INFO:tasks.workunit.client.0.smithi117.stdout:5/967: chown de/c80 126903 1 2024-04-20T11:09:15.051 INFO:tasks.workunit.client.0.smithi117.stdout:2/839: creat d1/d4/d83/d29/d62/df7/f10e x:0 0 0 2024-04-20T11:09:15.052 INFO:tasks.workunit.client.0.smithi117.stdout:8/619: mknod d0/d8/d11/d6d/d84/cd0 0 2024-04-20T11:09:15.055 INFO:tasks.workunit.client.0.smithi117.stdout:5/968: creat de/d41/f158 x:0 0 0 2024-04-20T11:09:15.058 INFO:tasks.workunit.client.0.smithi117.stdout:7/713: dwrite d1/d4/d3d/f7c [0,4194304] 0 2024-04-20T11:09:15.059 INFO:tasks.workunit.client.0.smithi117.stdout:6/847: dwrite d3/d4/d2e/d37/f7e [0,4194304] 0 2024-04-20T11:09:15.060 INFO:tasks.workunit.client.0.smithi117.stdout:8/620: readlink d0/d8/lf 0 2024-04-20T11:09:15.064 INFO:tasks.workunit.client.0.smithi117.stdout:7/714: chown d1/d4/d39/f4d 7259203 1 2024-04-20T11:09:15.064 INFO:tasks.workunit.client.0.smithi117.stdout:1/648: dwrite d6/d16/fad [0,4194304] 0 2024-04-20T11:09:15.069 INFO:tasks.workunit.client.0.smithi117.stdout:7/715: chown d1/d4/d6/d2b/f81 3692154 1 2024-04-20T11:09:15.071 INFO:tasks.workunit.client.0.smithi117.stdout:6/848: mkdir d3/d4/d2e/d37/d8e/db2/d10f 0 2024-04-20T11:09:15.073 INFO:tasks.workunit.client.0.smithi117.stdout:1/649: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/fd5 x:0 0 0 2024-04-20T11:09:15.074 INFO:tasks.workunit.client.0.smithi117.stdout:6/849: fdatasync d3/df/f14 0 2024-04-20T11:09:15.076 INFO:tasks.workunit.client.0.smithi117.stdout:7/716: rmdir d1/d4/d6/d17 39 2024-04-20T11:09:15.080 INFO:tasks.workunit.client.0.smithi117.stdout:1/650: unlink d6/d16/l37 0 2024-04-20T11:09:15.083 INFO:tasks.workunit.client.0.smithi117.stdout:8/621: dread d0/dc/d2e/d89/fbe [4194304,4194304] 0 2024-04-20T11:09:15.084 INFO:tasks.workunit.client.0.smithi117.stdout:1/651: write d6/d35/fb3 [980713,16407] 0 2024-04-20T11:09:15.085 INFO:tasks.workunit.client.0.smithi117.stdout:7/717: chown d1/d4/d6/d17/d3f/f90 59907113 1 2024-04-20T11:09:15.086 INFO:tasks.workunit.client.0.smithi117.stdout:5/969: dwrite de/f29 [0,4194304] 0 2024-04-20T11:09:15.087 INFO:tasks.workunit.client.0.smithi117.stdout:6/850: rename d3/d4/d2e/d3f/fc1 to d3/df/d39/daa/d100/f110 0 2024-04-20T11:09:15.093 INFO:tasks.workunit.client.0.smithi117.stdout:6/851: write d3/d5a/d74/db3/df0/fcf [3641027,93490] 0 2024-04-20T11:09:15.103 INFO:tasks.workunit.client.0.smithi117.stdout:8/622: dwrite d0/f23 [0,4194304] 0 2024-04-20T11:09:15.105 INFO:tasks.workunit.client.0.smithi117.stdout:5/970: dwrite de/fea [0,4194304] 0 2024-04-20T11:09:15.119 INFO:tasks.workunit.client.0.smithi117.stdout:5/971: write de/dd0/d10e/d4e/da1/de7/d123/d77/f7d [5102219,2619] 0 2024-04-20T11:09:15.121 INFO:tasks.workunit.client.0.smithi117.stdout:5/972: readlink de/dd0/d10e/la0 0 2024-04-20T11:09:15.125 INFO:tasks.workunit.client.0.smithi117.stdout:5/973: mkdir de/dd0/d10e/d4e/d159 0 2024-04-20T11:09:15.128 INFO:tasks.workunit.client.0.smithi117.stdout:8/623: dwrite d0/d9/dbb/fc8 [0,4194304] 0 2024-04-20T11:09:15.143 INFO:tasks.workunit.client.0.smithi117.stdout:5/974: dwrite de/d5b/d65/f13d [0,4194304] 0 2024-04-20T11:09:15.153 INFO:tasks.workunit.client.0.smithi117.stdout:8/624: dwrite d0/d8/d11/d24/d53/f7b [0,4194304] 0 2024-04-20T11:09:15.162 INFO:tasks.workunit.client.0.smithi117.stdout:8/625: creat d0/d8/d39/d40/d78/d5c/fd1 x:0 0 0 2024-04-20T11:09:15.163 INFO:tasks.workunit.client.0.smithi117.stdout:5/975: write de/f8e [770604,122295] 0 2024-04-20T11:09:15.166 INFO:tasks.workunit.client.0.smithi117.stdout:8/626: write d0/f83 [238470,55759] 0 2024-04-20T11:09:15.169 INFO:tasks.workunit.client.0.smithi117.stdout:8/627: mknod d0/d9/dbb/cd2 0 2024-04-20T11:09:15.173 INFO:tasks.workunit.client.0.smithi117.stdout:8/628: rename d0/dc/d49/d5b/dcd to d0/dd3 0 2024-04-20T11:09:15.193 INFO:tasks.workunit.client.0.smithi117.stdout:8/629: dread d0/f1 [0,4194304] 0 2024-04-20T11:09:15.204 INFO:tasks.workunit.client.0.smithi117.stdout:8/630: dread d0/d8/d11/d24/d27/d47/f6c [0,4194304] 0 2024-04-20T11:09:15.206 INFO:tasks.workunit.client.0.smithi117.stdout:8/631: creat d0/d9/d3a/fd4 x:0 0 0 2024-04-20T11:09:15.220 INFO:tasks.workunit.client.0.smithi117.stdout:8/632: dwrite d0/d8/d39/d40/d78/f9b [0,4194304] 0 2024-04-20T11:09:15.230 INFO:tasks.workunit.client.0.smithi117.stdout:8/633: truncate d0/d8/d39/d7f/fb9 4506820 0 2024-04-20T11:09:15.232 INFO:tasks.workunit.client.0.smithi117.stdout:8/634: mkdir d0/dc/d49/dd5 0 2024-04-20T11:09:15.234 INFO:tasks.workunit.client.0.smithi117.stdout:8/635: write d0/f23 [1813268,49397] 0 2024-04-20T11:09:15.235 INFO:tasks.workunit.client.0.smithi117.stdout:8/636: stat d0/d8/d11/d6d/fba 0 2024-04-20T11:09:15.239 INFO:tasks.workunit.client.0.smithi117.stdout:8/637: rmdir d0/d8/d11/d6d/d84 39 2024-04-20T11:09:15.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:14 smithi117 ceph-mon[15659]: pgmap v17: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 47 MiB/s rd, 156 MiB/s wr, 383 op/s 2024-04-20T11:09:15.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:14 smithi173 ceph-mon[23800]: pgmap v17: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 47 MiB/s rd, 156 MiB/s wr, 383 op/s 2024-04-20T11:09:15.437 INFO:tasks.workunit.client.0.smithi117.stdout:4/691: sync 2024-04-20T11:09:15.452 INFO:tasks.workunit.client.0.smithi117.stdout:4/692: link d1/d9/c11 d1/d9/d1f/d90/cee 0 2024-04-20T11:09:15.493 INFO:tasks.workunit.client.0.smithi117.stdout:4/693: dwrite d1/dd/d1a/d43/d4e/f62 [0,4194304] 0 2024-04-20T11:09:15.509 INFO:tasks.workunit.client.0.smithi117.stdout:4/694: rmdir d1/d45/d4a/dea 0 2024-04-20T11:09:15.517 INFO:tasks.workunit.client.0.smithi117.stdout:4/695: write d1/dd/d1a/f1d [3488230,128] 0 2024-04-20T11:09:15.526 INFO:tasks.workunit.client.0.smithi117.stdout:4/696: creat d1/dd/d1a/d2b/d86/dda/fef x:0 0 0 2024-04-20T11:09:15.549 INFO:tasks.workunit.client.0.smithi117.stdout:3/672: sync 2024-04-20T11:09:15.557 INFO:tasks.workunit.client.0.smithi117.stdout:4/697: dwrite d1/dd/d1a/d43/d9a/fc2 [0,4194304] 0 2024-04-20T11:09:15.561 INFO:tasks.workunit.client.0.smithi117.stdout:4/698: chown d1/dd/d1a/d2b/d86/f91 145 1 2024-04-20T11:09:15.588 INFO:tasks.workunit.client.0.smithi117.stdout:3/673: dwrite d0/d5/d21/d88/d29/f97 [0,4194304] 0 2024-04-20T11:09:15.596 INFO:tasks.workunit.client.0.smithi117.stdout:4/699: dread d1/dd/d1a/f3e [0,4194304] 0 2024-04-20T11:09:15.598 INFO:tasks.workunit.client.0.smithi117.stdout:3/674: dread d0/d5/d21/d88/d29/f30 [0,4194304] 0 2024-04-20T11:09:15.598 INFO:tasks.workunit.client.0.smithi117.stdout:4/700: dread - d1/dd/d1a/d2b/d86/fe2 zero size 2024-04-20T11:09:15.601 INFO:tasks.workunit.client.0.smithi117.stdout:4/701: creat d1/d9/d6b/ff0 x:0 0 0 2024-04-20T11:09:15.603 INFO:tasks.workunit.client.0.smithi117.stdout:3/675: creat d0/d5/d21/d88/d29/db3/dbc/fe5 x:0 0 0 2024-04-20T11:09:15.603 INFO:tasks.workunit.client.0.smithi117.stdout:4/702: fdatasync d1/dd/d1a/d2b/d86/fe5 0 2024-04-20T11:09:15.606 INFO:tasks.workunit.client.0.smithi117.stdout:3/676: read d0/d5/d7/dd/f5a [233990,16774] 0 2024-04-20T11:09:15.609 INFO:tasks.workunit.client.0.smithi117.stdout:4/703: mknod d1/dd/d52/cf1 0 2024-04-20T11:09:15.624 INFO:tasks.workunit.client.0.smithi117.stdout:9/737: sync 2024-04-20T11:09:15.624 INFO:tasks.workunit.client.0.smithi117.stdout:0/770: sync 2024-04-20T11:09:15.628 INFO:tasks.workunit.client.0.smithi117.stdout:3/677: dwrite d0/d5/d7/dd/f5a [0,4194304] 0 2024-04-20T11:09:15.631 INFO:tasks.workunit.client.0.smithi117.stdout:3/678: stat d0/d5/d21/d88/d29/f97 0 2024-04-20T11:09:15.633 INFO:tasks.workunit.client.0.smithi117.stdout:9/738: mknod d4/d5/d57/d82/da5/cfe 0 2024-04-20T11:09:15.640 INFO:tasks.workunit.client.0.smithi117.stdout:4/704: truncate d1/d9/f2a 525591 0 2024-04-20T11:09:15.642 INFO:tasks.workunit.client.0.smithi117.stdout:0/771: dwrite d3/d7e/fc7 [0,4194304] 0 2024-04-20T11:09:15.649 INFO:tasks.workunit.client.0.smithi117.stdout:9/739: read d4/d35/f56 [2736436,112111] 0 2024-04-20T11:09:15.651 INFO:tasks.workunit.client.0.smithi117.stdout:9/740: fsync d4/d5/d3c/ddf/f36 0 2024-04-20T11:09:15.652 INFO:tasks.workunit.client.0.smithi117.stdout:0/772: creat d3/d7/d20/d47/ff2 x:0 0 0 2024-04-20T11:09:15.654 INFO:tasks.workunit.client.0.smithi117.stdout:3/679: write d0/d5/d21/f28 [206706,97551] 0 2024-04-20T11:09:15.657 INFO:tasks.workunit.client.0.smithi117.stdout:9/741: chown d4/d35/c44 2206827 1 2024-04-20T11:09:15.663 INFO:tasks.workunit.client.0.smithi117.stdout:3/680: creat d0/d5/d21/d75/d89/fe6 x:0 0 0 2024-04-20T11:09:15.665 INFO:tasks.workunit.client.0.smithi117.stdout:9/742: creat d4/de/db1/fff x:0 0 0 2024-04-20T11:09:15.667 INFO:tasks.workunit.client.0.smithi117.stdout:0/773: mkdir d3/d7/d20/d6e/df3 0 2024-04-20T11:09:15.671 INFO:tasks.workunit.client.0.smithi117.stdout:3/681: rename d0/d5/fc to d0/d5/d21/d88/d29/d66/fe7 0 2024-04-20T11:09:15.674 INFO:tasks.workunit.client.0.smithi117.stdout:9/743: unlink d4/de/d7b/fbb 0 2024-04-20T11:09:15.675 INFO:tasks.workunit.client.0.smithi117.stdout:0/774: rmdir d3/d7e 39 2024-04-20T11:09:15.682 INFO:tasks.workunit.client.0.smithi117.stdout:9/744: dread d4/f13 [0,4194304] 0 2024-04-20T11:09:15.687 INFO:tasks.workunit.client.0.smithi117.stdout:4/705: dwrite d1/d9/f2a [0,4194304] 0 2024-04-20T11:09:15.689 INFO:tasks.workunit.client.0.smithi117.stdout:3/682: dwrite d0/d2a/fc3 [0,4194304] 0 2024-04-20T11:09:15.692 INFO:tasks.workunit.client.0.smithi117.stdout:2/840: sync 2024-04-20T11:09:15.693 INFO:tasks.workunit.client.0.smithi117.stdout:7/718: sync 2024-04-20T11:09:15.693 INFO:tasks.workunit.client.0.smithi117.stdout:8/638: sync 2024-04-20T11:09:15.693 INFO:tasks.workunit.client.0.smithi117.stdout:5/976: sync 2024-04-20T11:09:15.693 INFO:tasks.workunit.client.0.smithi117.stdout:1/652: sync 2024-04-20T11:09:15.694 INFO:tasks.workunit.client.0.smithi117.stdout:9/745: fsync d4/d5/f2a 0 2024-04-20T11:09:15.695 INFO:tasks.workunit.client.0.smithi117.stdout:4/706: readlink d1/dd/d1a/lbd 0 2024-04-20T11:09:15.695 INFO:tasks.workunit.client.0.smithi117.stdout:6/852: sync 2024-04-20T11:09:15.701 INFO:tasks.workunit.client.0.smithi117.stdout:4/707: fdatasync d1/dd/d1a/d2b/f46 0 2024-04-20T11:09:15.712 INFO:tasks.workunit.client.0.smithi117.stdout:2/841: creat d1/d4/d83/d29/d2e/d53/d93/f10f x:0 0 0 2024-04-20T11:09:15.720 INFO:tasks.workunit.client.0.smithi117.stdout:8/639: dwrite d0/d8/d11/d24/d27/d99/f9d [0,4194304] 0 2024-04-20T11:09:15.722 INFO:tasks.workunit.client.0.smithi117.stdout:7/719: creat d1/d4/d6/d54/db4/fe4 x:0 0 0 2024-04-20T11:09:15.726 INFO:tasks.workunit.client.0.smithi117.stdout:6/853: creat d3/df/d39/daa/d100/d103/f111 x:0 0 0 2024-04-20T11:09:15.730 INFO:tasks.workunit.client.0.smithi117.stdout:6/854: write d3/d4/f1b [6540097,102782] 0 2024-04-20T11:09:15.735 INFO:tasks.workunit.client.0.smithi117.stdout:5/977: mknod de/dd0/d10e/d4e/d159/c15a 0 2024-04-20T11:09:15.736 INFO:tasks.workunit.client.0.smithi117.stdout:5/978: dread - de/d16/fd1 zero size 2024-04-20T11:09:15.737 INFO:tasks.workunit.client.0.smithi117.stdout:7/720: dread d1/d4/d3d/f5a [0,4194304] 0 2024-04-20T11:09:15.738 INFO:tasks.workunit.client.0.smithi117.stdout:0/775: write d3/d7/d2c/d52/fc5 [514447,74850] 0 2024-04-20T11:09:15.742 INFO:tasks.workunit.client.0.smithi117.stdout:7/721: dread - d1/d4/d6/fd0 zero size 2024-04-20T11:09:15.748 INFO:tasks.workunit.client.0.smithi117.stdout:8/640: mknod d0/d8/d11/d24/d53/cd6 0 2024-04-20T11:09:15.750 INFO:tasks.workunit.client.0.smithi117.stdout:9/746: dread d4/d5/d3c/ddf/f36 [0,4194304] 0 2024-04-20T11:09:15.750 INFO:tasks.workunit.client.0.smithi117.stdout:8/641: chown d0/d8/d11/d24/d27/d47/f8e 41 1 2024-04-20T11:09:15.751 INFO:tasks.workunit.client.0.smithi117.stdout:6/855: unlink d3/d50/f59 0 2024-04-20T11:09:15.752 INFO:tasks.workunit.client.0.smithi117.stdout:8/642: fdatasync d0/d8/d39/f54 0 2024-04-20T11:09:15.754 INFO:tasks.workunit.client.0.smithi117.stdout:4/708: getdents d1/d9/d6b/dca/de7 0 2024-04-20T11:09:15.757 INFO:tasks.workunit.client.0.smithi117.stdout:3/683: write d0/d5/d21/d88/d29/d66/d63/fe4 [697039,32577] 0 2024-04-20T11:09:15.762 INFO:tasks.workunit.client.0.smithi117.stdout:0/776: symlink d3/d7/d20/d39/d3a/lf4 0 2024-04-20T11:09:15.764 INFO:tasks.workunit.client.0.smithi117.stdout:1/653: write d6/d16/d3a/d5a/d5c/d61/d71/d6b/f93 [529813,35117] 0 2024-04-20T11:09:15.765 INFO:tasks.workunit.client.0.smithi117.stdout:2/842: truncate d1/da5/df2/f56 1632870 0 2024-04-20T11:09:15.773 INFO:tasks.workunit.client.0.smithi117.stdout:9/747: rename d4/d4a/d72/lc8 to d4/d5/d57/db4/l100 0 2024-04-20T11:09:15.775 INFO:tasks.workunit.client.0.smithi117.stdout:4/709: truncate d1/dd/d1a/d43/d9a/fae 179365 0 2024-04-20T11:09:15.776 INFO:tasks.workunit.client.0.smithi117.stdout:9/748: truncate d4/d4a/d5e/de8/f86 1488848 0 2024-04-20T11:09:15.777 INFO:tasks.workunit.client.0.smithi117.stdout:5/979: creat de/d41/d47/d14a/d13b/f15b x:0 0 0 2024-04-20T11:09:15.779 INFO:tasks.workunit.client.0.smithi117.stdout:3/684: symlink d0/d5/d21/d94/da7/le8 0 2024-04-20T11:09:15.791 INFO:tasks.workunit.client.0.smithi117.stdout:9/749: creat d4/d4a/d5e/d9b/dcd/f101 x:0 0 0 2024-04-20T11:09:15.792 INFO:tasks.workunit.client.0.smithi117.stdout:5/980: chown de/d41/dc2/f12a 15234807 1 2024-04-20T11:09:15.793 INFO:tasks.workunit.client.0.smithi117.stdout:3/685: rmdir d0 39 2024-04-20T11:09:15.800 INFO:tasks.workunit.client.0.smithi117.stdout:6/856: link d3/d4/l20 d3/d3a/l112 0 2024-04-20T11:09:15.803 INFO:tasks.workunit.client.0.smithi117.stdout:9/750: fdatasync d4/d5/da/f49 0 2024-04-20T11:09:15.807 INFO:tasks.workunit.client.0.smithi117.stdout:4/710: dwrite d1/fa7 [0,4194304] 0 2024-04-20T11:09:15.812 INFO:tasks.workunit.client.0.smithi117.stdout:5/981: truncate de/d41/dc2/f12a 795869 0 2024-04-20T11:09:15.821 INFO:tasks.workunit.client.0.smithi117.stdout:8/643: getdents d0 0 2024-04-20T11:09:15.824 INFO:tasks.workunit.client.0.smithi117.stdout:3/686: fsync d0/fb 0 2024-04-20T11:09:15.826 INFO:tasks.workunit.client.0.smithi117.stdout:1/654: getdents d6/d16/d3a/d5a/d5c/d61/d71/d7d/da2 0 2024-04-20T11:09:15.826 INFO:tasks.workunit.client.0.smithi117.stdout:6/857: mkdir d3/d4/d2e/d113 0 2024-04-20T11:09:15.829 INFO:tasks.workunit.client.0.smithi117.stdout:5/982: chown de/d21/d84/l9c 14872 1 2024-04-20T11:09:15.830 INFO:tasks.workunit.client.0.smithi117.stdout:8/644: read d0/d8/d39/d40/f43 [1417471,46363] 0 2024-04-20T11:09:15.836 INFO:tasks.workunit.client.0.smithi117.stdout:5/983: chown de/d21/df5/d57/d14c 2885445 1 2024-04-20T11:09:15.836 INFO:tasks.workunit.client.0.smithi117.stdout:9/751: dwrite d4/d5/d57/fca [0,4194304] 0 2024-04-20T11:09:15.837 INFO:tasks.workunit.client.0.smithi117.stdout:6/858: mkdir d3/d3a/d114 0 2024-04-20T11:09:15.841 INFO:tasks.workunit.client.0.smithi117.stdout:4/711: dwrite d1/dd/d1a/d43/d4e/f62 [0,4194304] 0 2024-04-20T11:09:15.841 INFO:tasks.workunit.client.0.smithi117.stdout:8/645: read d0/d8/d39/d40/d78/f97 [63559,89724] 0 2024-04-20T11:09:15.844 INFO:tasks.workunit.client.0.smithi117.stdout:7/722: truncate d1/d4/d6/d2b/d64/f8c 1765218 0 2024-04-20T11:09:15.848 INFO:tasks.workunit.client.0.smithi117.stdout:2/843: write d1/d15/f72 [624174,72988] 0 2024-04-20T11:09:15.850 INFO:tasks.workunit.client.0.smithi117.stdout:2/844: stat d1/da5/le8 0 2024-04-20T11:09:15.853 INFO:tasks.workunit.client.0.smithi117.stdout:2/845: stat d1/d4/d83/d29/d2e/cc8 0 2024-04-20T11:09:15.855 INFO:tasks.workunit.client.0.smithi117.stdout:0/777: truncate d3/d7/d20/d39/d9a/dd0/d6d/fcf 1946691 0 2024-04-20T11:09:15.857 INFO:tasks.workunit.client.0.smithi117.stdout:0/778: fdatasync d3/d7/d2c/d52/fde 0 2024-04-20T11:09:15.866 INFO:tasks.workunit.client.0.smithi117.stdout:1/655: dwrite d6/d16/fad [0,4194304] 0 2024-04-20T11:09:15.867 INFO:tasks.workunit.client.0.smithi117.stdout:5/984: dwrite de/d21/df5/d57/dd5/d11d/f108 [0,4194304] 0 2024-04-20T11:09:15.869 INFO:tasks.workunit.client.0.smithi117.stdout:1/656: readlink d6/lb 0 2024-04-20T11:09:15.870 INFO:tasks.workunit.client.0.smithi117.stdout:4/712: unlink d1/dd/d1a/c64 0 2024-04-20T11:09:15.872 INFO:tasks.workunit.client.0.smithi117.stdout:4/713: fdatasync d1/d9/f76 0 2024-04-20T11:09:15.873 INFO:tasks.workunit.client.0.smithi117.stdout:4/714: stat d1/c42 0 2024-04-20T11:09:15.893 INFO:tasks.workunit.client.0.smithi117.stdout:2/846: dread - d1/da5/df2/d46/fcd zero size 2024-04-20T11:09:15.895 INFO:tasks.workunit.client.0.smithi117.stdout:5/985: mknod de/d16/d9d/c15c 0 2024-04-20T11:09:15.897 INFO:tasks.workunit.client.0.smithi117.stdout:7/723: creat d1/d4/d6/d54/fe5 x:0 0 0 2024-04-20T11:09:15.897 INFO:tasks.workunit.client.0.smithi117.stdout:3/687: write d0/d5/d21/f4d [77497,69177] 0 2024-04-20T11:09:15.899 INFO:tasks.workunit.client.0.smithi117.stdout:6/859: truncate d3/df/fce 302695 0 2024-04-20T11:09:15.901 INFO:tasks.workunit.client.0.smithi117.stdout:9/752: write d4/de/f1b [3962867,57856] 0 2024-04-20T11:09:15.938 INFO:tasks.workunit.client.0.smithi117.stdout:9/753: dread d4/d4a/d5e/de8/f58 [0,4194304] 0 2024-04-20T11:09:15.964 INFO:tasks.workunit.client.0.smithi117.stdout:2/847: symlink d1/d4/d69/l110 0 2024-04-20T11:09:15.965 INFO:tasks.workunit.client.0.smithi117.stdout:1/657: mkdir d6/d1f/dd6 0 2024-04-20T11:09:15.967 INFO:tasks.workunit.client.0.smithi117.stdout:5/986: write de/d5b/f11a [1306935,86424] 0 2024-04-20T11:09:15.967 INFO:tasks.workunit.client.0.smithi117.stdout:4/715: mknod d1/dd/d1a/d43/dcc/cf2 0 2024-04-20T11:09:15.969 INFO:tasks.workunit.client.0.smithi117.stdout:8/646: getdents d0/dc/d49/d5b/da9 0 2024-04-20T11:09:15.972 INFO:tasks.workunit.client.0.smithi117.stdout:6/860: mknod d3/d5a/d74/db3/df0/db7/c115 0 2024-04-20T11:09:15.978 INFO:tasks.workunit.client.0.smithi117.stdout:7/724: symlink d1/le6 0 2024-04-20T11:09:15.984 INFO:tasks.workunit.client.0.smithi117.stdout:1/658: dwrite d6/d1f/d30/d49/d69/d6e/d6f/d73/f8c [4194304,4194304] 0 2024-04-20T11:09:15.984 INFO:tasks.workunit.client.0.smithi117.stdout:4/716: unlink d1/dd/lc4 0 2024-04-20T11:09:15.987 INFO:tasks.workunit.client.0.smithi117.stdout:0/779: getdents d3/d7/d20/d6e 0 2024-04-20T11:09:15.990 INFO:tasks.workunit.client.0.smithi117.stdout:6/861: read d3/d5a/d74/db3/df0/da3/fad [74240,7069] 0 2024-04-20T11:09:15.992 INFO:tasks.workunit.client.0.smithi117.stdout:9/754: rename d4/de/d10/f38 to d4/d35/df7/f102 0 2024-04-20T11:09:15.994 INFO:tasks.workunit.client.0.smithi117.stdout:8/647: dread d0/d8/d11/d24/d2b/f2c [0,4194304] 0 2024-04-20T11:09:15.997 INFO:tasks.workunit.client.0.smithi117.stdout:5/987: dread - de/d41/d47/d55/dab/f112 zero size 2024-04-20T11:09:15.998 INFO:tasks.workunit.client.0.smithi117.stdout:7/725: chown d1/d4/d6/d13/c25 1999287 1 2024-04-20T11:09:16.000 INFO:tasks.workunit.client.0.smithi117.stdout:7/726: write d1/d1d/d4e/d53/fd4 [221771,1082] 0 2024-04-20T11:09:16.001 INFO:tasks.workunit.client.0.smithi117.stdout:4/717: rmdir d1/dd/d1a/d43/dcc 39 2024-04-20T11:09:16.003 INFO:tasks.workunit.client.0.smithi117.stdout:0/780: symlink d3/d7/d20/d39/d9a/dd0/db2/dbd/lf5 0 2024-04-20T11:09:16.010 INFO:tasks.workunit.client.0.smithi117.stdout:1/659: symlink d6/d21/db9/ld7 0 2024-04-20T11:09:16.012 INFO:tasks.workunit.client.0.smithi117.stdout:7/727: symlink d1/d4/d39/le7 0 2024-04-20T11:09:16.017 INFO:tasks.workunit.client.0.smithi117.stdout:6/862: mkdir d3/d4/d2e/d37/d8e/db2/d10f/d116 0 2024-04-20T11:09:16.025 INFO:tasks.workunit.client.0.smithi117.stdout:7/728: mkdir d1/d4/d6/d17/de8 0 2024-04-20T11:09:16.025 INFO:tasks.workunit.client.0.smithi117.stdout:5/988: dread de/f11 [0,4194304] 0 2024-04-20T11:09:16.025 INFO:tasks.workunit.client.0.smithi117.stdout:0/781: truncate d3/d7/d94/fa5 2073 0 2024-04-20T11:09:16.025 INFO:tasks.workunit.client.0.smithi117.stdout:1/660: fsync d6/d1f/f2b 0 2024-04-20T11:09:16.028 INFO:tasks.workunit.client.0.smithi117.stdout:7/729: creat d1/d1d/d77/fe9 x:0 0 0 2024-04-20T11:09:16.032 INFO:tasks.workunit.client.0.smithi117.stdout:1/661: rmdir d6/d1f/d30/d49/d69/d6e 39 2024-04-20T11:09:16.034 INFO:tasks.workunit.client.0.smithi117.stdout:7/730: read d1/d1d/d42/faf [68223,103880] 0 2024-04-20T11:09:16.037 INFO:tasks.workunit.client.0.smithi117.stdout:1/662: rmdir d6/d1f 39 2024-04-20T11:09:16.039 INFO:tasks.workunit.client.0.smithi117.stdout:6/863: dwrite d3/d4/d2e/d3f/f77 [0,4194304] 0 2024-04-20T11:09:16.040 INFO:tasks.workunit.client.0.smithi117.stdout:7/731: rmdir d1/d4/d3d/dce 39 2024-04-20T11:09:16.041 INFO:tasks.workunit.client.0.smithi117.stdout:0/782: dwrite d3/d7/f4d [0,4194304] 0 2024-04-20T11:09:16.042 INFO:tasks.workunit.client.0.smithi117.stdout:3/688: write d0/d5/d73/f8b [482120,64176] 0 2024-04-20T11:09:16.050 INFO:tasks.workunit.client.0.smithi117.stdout:6/864: creat d3/d50/d8a/f117 x:0 0 0 2024-04-20T11:09:16.051 INFO:tasks.workunit.client.0.smithi117.stdout:4/718: dread d1/d9/de9/fab [4194304,4194304] 0 2024-04-20T11:09:16.053 INFO:tasks.workunit.client.0.smithi117.stdout:0/783: chown d3/f38 6911 1 2024-04-20T11:09:16.056 INFO:tasks.workunit.client.0.smithi117.stdout:1/663: creat d6/d16/dce/fd8 x:0 0 0 2024-04-20T11:09:16.056 INFO:tasks.workunit.client.0.smithi117.stdout:1/664: read - d6/fb4 zero size 2024-04-20T11:09:16.063 INFO:tasks.workunit.client.0.smithi117.stdout:6/865: fdatasync d3/d4/d2e/f10c 0 2024-04-20T11:09:16.063 INFO:tasks.workunit.client.0.smithi117.stdout:2/848: write d1/d9b/faf [784941,52098] 0 2024-04-20T11:09:16.065 INFO:tasks.workunit.client.0.smithi117.stdout:4/719: truncate d1/f67 956955 0 2024-04-20T11:09:16.067 INFO:tasks.workunit.client.0.smithi117.stdout:2/849: dread - d1/d4/d83/d29/d62/df7/f10e zero size 2024-04-20T11:09:16.069 INFO:tasks.workunit.client.0.smithi117.stdout:8/648: write d0/dc/d49/f73 [827824,124899] 0 2024-04-20T11:09:16.071 INFO:tasks.workunit.client.0.smithi117.stdout:7/732: creat d1/fea x:0 0 0 2024-04-20T11:09:16.073 INFO:tasks.workunit.client.0.smithi117.stdout:1/665: rename d6/d16/d3a/d5a/caa to d6/d16/d3a/d5a/d5c/d61/d98/cd9 0 2024-04-20T11:09:16.075 INFO:tasks.workunit.client.0.smithi117.stdout:9/755: truncate f2 8198679 0 2024-04-20T11:09:16.079 INFO:tasks.workunit.client.0.smithi117.stdout:4/720: mknod d1/d45/d4a/cf3 0 2024-04-20T11:09:16.082 INFO:tasks.workunit.client.0.smithi117.stdout:2/850: rmdir d1/da5/df2 39 2024-04-20T11:09:16.086 INFO:tasks.workunit.client.0.smithi117.stdout:1/666: read d6/f10 [1152836,7315] 0 2024-04-20T11:09:16.086 INFO:tasks.workunit.client.0.smithi117.stdout:5/989: write de/dd0/d10e/d4e/da1/fe5 [64733,70842] 0 2024-04-20T11:09:16.087 INFO:tasks.workunit.client.0.smithi117.stdout:6/866: creat d3/d3a/d114/f118 x:0 0 0 2024-04-20T11:09:16.087 INFO:tasks.workunit.client.0.smithi117.stdout:1/667: fsync d6/d16/fbb 0 2024-04-20T11:09:16.090 INFO:tasks.workunit.client.0.smithi117.stdout:9/756: rename d4/d4a/d72/fb2 to d4/d5/d57/f103 0 2024-04-20T11:09:16.091 INFO:tasks.workunit.client.0.smithi117.stdout:4/721: mknod d1/d9/d6b/dca/cf4 0 2024-04-20T11:09:16.091 INFO:tasks.workunit.client.0.smithi117.stdout:9/757: fsync d4/d4a/d63/d98/fae 0 2024-04-20T11:09:16.092 INFO:tasks.workunit.client.0.smithi117.stdout:8/649: creat d0/d9/d3a/d7c/dce/fd7 x:0 0 0 2024-04-20T11:09:16.094 INFO:tasks.workunit.client.0.smithi117.stdout:7/733: mkdir d1/d4/d3d/dce/deb 0 2024-04-20T11:09:16.096 INFO:tasks.workunit.client.0.smithi117.stdout:7/734: write d1/d4/d3d/f7c [2737370,112238] 0 2024-04-20T11:09:16.099 INFO:tasks.workunit.client.0.smithi117.stdout:6/867: symlink d3/d94/l119 0 2024-04-20T11:09:16.100 INFO:tasks.workunit.client.0.smithi117.stdout:3/689: write d0/d5/d21/f41 [5241963,45674] 0 2024-04-20T11:09:16.107 INFO:tasks.workunit.client.0.smithi117.stdout:1/668: dwrite d6/d16/d3a/da8/fae [4194304,4194304] 0 2024-04-20T11:09:16.113 INFO:tasks.workunit.client.0.smithi117.stdout:9/758: truncate d4/d5/da/f32 660271 0 2024-04-20T11:09:16.114 INFO:tasks.workunit.client.0.smithi117.stdout:5/990: mknod de/dd0/d10e/d4e/c15d 0 2024-04-20T11:09:16.115 INFO:tasks.workunit.client.0.smithi117.stdout:0/784: truncate d3/d7/d20/d39/d9a/dd0/d22/f75 673455 0 2024-04-20T11:09:16.115 INFO:tasks.workunit.client.0.smithi117.stdout:9/759: chown d4/d4a/d5e/d9b/dcd/le0 5 1 2024-04-20T11:09:16.118 INFO:tasks.workunit.client.0.smithi117.stdout:3/690: rmdir d0/d5/de 39 2024-04-20T11:09:16.123 INFO:tasks.workunit.client.0.smithi117.stdout:4/722: truncate d1/d9/f12 1146817 0 2024-04-20T11:09:16.123 INFO:tasks.workunit.client.0.smithi117.stdout:8/650: unlink d0/d8/lc0 0 2024-04-20T11:09:16.125 INFO:tasks.workunit.client.0.smithi117.stdout:6/868: dwrite d3/f28 [4194304,4194304] 0 2024-04-20T11:09:16.126 INFO:tasks.workunit.client.0.smithi117.stdout:4/723: chown d1/d9/de9/ld9 0 1 2024-04-20T11:09:16.128 INFO:tasks.workunit.client.0.smithi117.stdout:5/991: dread - de/dd0/f117 zero size 2024-04-20T11:09:16.133 INFO:tasks.workunit.client.0.smithi117.stdout:0/785: creat d3/d7/d20/d39/d3a/db1/ff6 x:0 0 0 2024-04-20T11:09:16.137 INFO:tasks.workunit.client.0.smithi117.stdout:3/691: rename d0/d5/d21/f28 to d0/d5/d73/fe9 0 2024-04-20T11:09:16.137 INFO:tasks.workunit.client.0.smithi117.stdout:1/669: mknod d6/d16/cda 0 2024-04-20T11:09:16.138 INFO:tasks.workunit.client.0.smithi117.stdout:2/851: truncate d1/d4/dd/d17/ffe 1268831 0 2024-04-20T11:09:16.151 INFO:tasks.workunit.client.0.smithi117.stdout:0/786: read d3/d7/d20/d39/f92 [2122347,49812] 0 2024-04-20T11:09:16.157 INFO:tasks.workunit.client.0.smithi117.stdout:5/992: getdents de/dd0/d10c 0 2024-04-20T11:09:16.160 INFO:tasks.workunit.client.0.smithi117.stdout:2/852: symlink d1/d4/d83/d29/d2e/l111 0 2024-04-20T11:09:16.160 INFO:tasks.workunit.client.0.smithi117.stdout:4/724: dwrite d1/d9/de9/fc0 [0,4194304] 0 2024-04-20T11:09:16.161 INFO:tasks.workunit.client.0.smithi117.stdout:1/670: link d6/d16/d3a/d5a/d5c/d61/d71/la5 d6/d16/dce/ldb 0 2024-04-20T11:09:16.163 INFO:tasks.workunit.client.0.smithi117.stdout:0/787: dread - d3/d7/d20/d47/fc2 zero size 2024-04-20T11:09:16.167 INFO:tasks.workunit.client.0.smithi117.stdout:5/993: truncate de/d41/dc2/f120 601423 0 2024-04-20T11:09:16.167 INFO:tasks.workunit.client.0.smithi117.stdout:8/651: dwrite d0/d8/d39/d7f/fbc [0,4194304] 0 2024-04-20T11:09:16.171 INFO:tasks.workunit.client.0.smithi117.stdout:5/994: chown de/db7/fbd 5392603 1 2024-04-20T11:09:16.172 INFO:tasks.workunit.client.0.smithi117.stdout:7/735: write d1/d1d/d4e/f9a [1028777,125508] 0 2024-04-20T11:09:16.178 INFO:tasks.workunit.client.0.smithi117.stdout:1/671: chown d6/d1f/d30/ca1 16776 1 2024-04-20T11:09:16.178 INFO:tasks.workunit.client.0.smithi117.stdout:0/788: mknod d3/d7/d20/d39/d9a/dd0/db2/cf7 0 2024-04-20T11:09:16.185 INFO:tasks.workunit.client.0.smithi117.stdout:3/692: write d0/d5/d21/d88/f62 [3285991,35646] 0 2024-04-20T11:09:16.193 INFO:tasks.workunit.client.0.smithi117.stdout:2/853: creat d1/d4/d83/d29/d62/d109/f112 x:0 0 0 2024-04-20T11:09:16.196 INFO:tasks.workunit.client.0.smithi117.stdout:0/789: symlink d3/d7/d20/d47/d78/lf8 0 2024-04-20T11:09:16.198 INFO:tasks.workunit.client.0.smithi117.stdout:8/652: mknod d0/d8/d11/d24/d86/cd8 0 2024-04-20T11:09:16.201 INFO:tasks.workunit.client.0.smithi117.stdout:2/854: write d1/d4/d83/d29/fda [730871,42523] 0 2024-04-20T11:09:16.201 INFO:tasks.workunit.client.0.smithi117.stdout:1/672: rename d6/c28 to d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/cdc 0 2024-04-20T11:09:16.207 INFO:tasks.workunit.client.0.smithi117.stdout:5/995: dwrite de/db7/d12b/f131 [0,4194304] 0 2024-04-20T11:09:16.211 INFO:tasks.workunit.client.0.smithi117.stdout:9/760: dwrite d4/d35/f9a [0,4194304] 0 2024-04-20T11:09:16.215 INFO:tasks.workunit.client.0.smithi117.stdout:3/693: mknod d0/d5/d21/cea 0 2024-04-20T11:09:16.215 INFO:tasks.workunit.client.0.smithi117.stdout:5/996: chown de/f18 1679299317 1 2024-04-20T11:09:16.217 INFO:tasks.workunit.client.0.smithi117.stdout:8/653: read d0/d8/d39/d40/d78/d5c/f68 [245682,1290] 0 2024-04-20T11:09:16.218 INFO:tasks.workunit.client.0.smithi117.stdout:4/725: dread d1/d45/d4a/d59/fce [0,4194304] 0 2024-04-20T11:09:16.221 INFO:tasks.workunit.client.0.smithi117.stdout:9/761: rmdir d4/d4a/d63 39 2024-04-20T11:09:16.224 INFO:tasks.workunit.client.0.smithi117.stdout:2/855: mknod d1/d4/d83/d29/d3f/df6/c113 0 2024-04-20T11:09:16.231 INFO:tasks.workunit.client.0.smithi117.stdout:0/790: dwrite d3/d5/f37 [0,4194304] 0 2024-04-20T11:09:16.232 INFO:tasks.workunit.client.0.smithi117.stdout:7/736: dwrite d1/d1d/d42/faf [0,4194304] 0 2024-04-20T11:09:16.236 INFO:tasks.workunit.client.0.smithi117.stdout:9/762: readlink d4/d5/d57/lfc 0 2024-04-20T11:09:16.240 INFO:tasks.workunit.client.0.smithi117.stdout:4/726: read d1/d9/f96 [1154260,32258] 0 2024-04-20T11:09:16.240 INFO:tasks.workunit.client.0.smithi117.stdout:0/791: write d3/d7/d20/d39/d3a/db1/fdf [135499,55021] 0 2024-04-20T11:09:16.242 INFO:tasks.workunit.client.0.smithi117.stdout:9/763: chown d4/de/d7b/fbd 5 1 2024-04-20T11:09:16.246 INFO:tasks.workunit.client.0.smithi117.stdout:7/737: dread d1/d4/d6/d54/d63/f85 [0,4194304] 0 2024-04-20T11:09:16.252 INFO:tasks.workunit.client.0.smithi117.stdout:2/856: link d1/d15/f72 d1/d20/de0/f114 0 2024-04-20T11:09:16.255 INFO:tasks.workunit.client.0.smithi117.stdout:2/857: dread - d1/d4/fbc zero size 2024-04-20T11:09:16.257 INFO:tasks.workunit.client.0.smithi117.stdout:9/764: unlink d4/d5/d3c/la6 0 2024-04-20T11:09:16.261 INFO:tasks.workunit.client.0.smithi117.stdout:0/792: read - d3/d7/d20/d39/d9a/dd0/d6d/f97 zero size 2024-04-20T11:09:16.261 INFO:tasks.workunit.client.0.smithi117.stdout:9/765: fdatasync d4/de/db1/fb9 0 2024-04-20T11:09:16.262 INFO:tasks.workunit.client.0.smithi117.stdout:4/727: dread d1/d9/d1f/d87/fb3 [0,4194304] 0 2024-04-20T11:09:16.263 INFO:tasks.workunit.client.0.smithi117.stdout:7/738: creat d1/d1d/d4e/dd3/fec x:0 0 0 2024-04-20T11:09:16.268 INFO:tasks.workunit.client.0.smithi117.stdout:7/739: chown d1/d4/d3d/dce/deb 2032147 1 2024-04-20T11:09:16.271 INFO:tasks.workunit.client.0.smithi117.stdout:5/997: dwrite de/d41/d47/d55/d56/f79 [0,4194304] 0 2024-04-20T11:09:16.272 INFO:tasks.workunit.client.0.smithi117.stdout:3/694: dread d0/d5/d21/f42 [0,4194304] 0 2024-04-20T11:09:16.275 INFO:tasks.workunit.client.0.smithi117.stdout:7/740: mkdir d1/d1d/ded 0 2024-04-20T11:09:16.276 INFO:tasks.workunit.client.0.smithi117.stdout:0/793: rename d3/f3e to d3/d7/d20/d47/d78/ff9 0 2024-04-20T11:09:16.283 INFO:tasks.workunit.client.0.smithi117.stdout:3/695: creat d0/d5/d21/d94/d9a/feb x:0 0 0 2024-04-20T11:09:16.284 INFO:tasks.workunit.client.0.smithi117.stdout:0/794: chown d3/d5/c15 283 1 2024-04-20T11:09:16.285 INFO:tasks.workunit.client.0.smithi117.stdout:8/654: dread d0/d8/f41 [0,4194304] 0 2024-04-20T11:09:16.286 INFO:tasks.workunit.client.0.smithi117.stdout:9/766: rename d4/d5/d3c/ddf/d31/f6c to d4/d35/df7/f104 0 2024-04-20T11:09:16.290 INFO:tasks.workunit.client.0.smithi117.stdout:0/795: truncate d3/d7/f18 4501172 0 2024-04-20T11:09:16.291 INFO:tasks.workunit.client.0.smithi117.stdout:3/696: fsync d0/d5/d21/d88/d29/d66/fe7 0 2024-04-20T11:09:16.296 INFO:tasks.workunit.client.0.smithi117.stdout:5/998: rename de/d41/d4b to de/d21/df5/d57/dd5/d15e 0 2024-04-20T11:09:16.297 INFO:tasks.workunit.client.0.smithi117.stdout:4/728: dwrite d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:16.298 INFO:tasks.workunit.client.0.smithi117.stdout:8/655: symlink d0/d8/d11/d24/d27/d47/ld9 0 2024-04-20T11:09:16.299 INFO:tasks.workunit.client.0.smithi117.stdout:0/796: fsync d3/d7/d16/f2b 0 2024-04-20T11:09:16.302 INFO:tasks.workunit.client.0.smithi117.stdout:4/729: fdatasync d1/dd/fb5 0 2024-04-20T11:09:16.304 INFO:tasks.workunit.client.0.smithi117.stdout:3/697: unlink d0/d5/d21/d94/d9a/fa5 0 2024-04-20T11:09:16.306 INFO:tasks.workunit.client.0.smithi117.stdout:9/767: rename d4/d5/da to d4/d35/df7/d105 0 2024-04-20T11:09:16.309 INFO:tasks.workunit.client.0.smithi117.stdout:7/741: dwrite d1/d4/d3d/f7c [0,4194304] 0 2024-04-20T11:09:16.313 INFO:tasks.workunit.client.0.smithi117.stdout:7/742: fsync d1/d1d/d42/fcc 0 2024-04-20T11:09:16.313 INFO:tasks.workunit.client.0.smithi117.stdout:7/743: stat d1/d1d/d42/c3b 0 2024-04-20T11:09:16.317 INFO:tasks.workunit.client.0.smithi117.stdout:0/797: dread d3/d7/d2c/d52/fc5 [0,4194304] 0 2024-04-20T11:09:16.320 INFO:tasks.workunit.client.0.smithi117.stdout:8/656: mkdir d0/d8/d39/d40/d78/dda 0 2024-04-20T11:09:16.322 INFO:tasks.workunit.client.0.smithi117.stdout:5/999: dread de/f29 [0,4194304] 0 2024-04-20T11:09:16.324 INFO:tasks.workunit.client.0.smithi117.stdout:3/698: mkdir d0/d5/de/d4e/dec 0 2024-04-20T11:09:16.325 INFO:tasks.workunit.client.0.smithi117.stdout:7/744: rename d1/d1d/d42/d89/da9 to d1/d1d/d77/dee 0 2024-04-20T11:09:16.329 INFO:tasks.workunit.client.0.smithi117.stdout:0/798: rename d3/d7/d20/d39/f62 to d3/d7/d20/d6e/df3/ffa 0 2024-04-20T11:09:16.330 INFO:tasks.workunit.client.0.smithi117.stdout:8/657: creat d0/dc/d49/dd5/fdb x:0 0 0 2024-04-20T11:09:16.335 INFO:tasks.workunit.client.0.smithi117.stdout:3/699: rename d0/d5/d21/d88/d29 to d0/d5/d73/ded 0 2024-04-20T11:09:16.338 INFO:tasks.workunit.client.0.smithi117.stdout:8/658: creat d0/fdc x:0 0 0 2024-04-20T11:09:16.343 INFO:tasks.workunit.client.0.smithi117.stdout:3/700: unlink d0/d5/f19 0 2024-04-20T11:09:16.346 INFO:tasks.workunit.client.0.smithi117.stdout:7/745: dwrite d1/d1d/d42/d2d/fb7 [0,4194304] 0 2024-04-20T11:09:16.350 INFO:tasks.workunit.client.0.smithi117.stdout:0/799: rename d3/dab/d54/l8b to d3/dab/lfb 0 2024-04-20T11:09:16.351 INFO:tasks.workunit.client.0.smithi117.stdout:7/746: write d1/d1d/d4e/d72/fc6 [444099,84719] 0 2024-04-20T11:09:16.352 INFO:tasks.workunit.client.0.smithi117.stdout:3/701: mknod d0/d5/d73/ded/d66/d63/cee 0 2024-04-20T11:09:16.355 INFO:tasks.workunit.client.0.smithi117.stdout:4/730: dwrite d1/d45/d4a/fa9 [0,4194304] 0 2024-04-20T11:09:16.358 INFO:tasks.workunit.client.0.smithi117.stdout:3/702: write d0/d5/d73/ded/d66/d63/fe4 [504760,32824] 0 2024-04-20T11:09:16.360 INFO:tasks.workunit.client.0.smithi117.stdout:4/731: truncate d1/d9/fb0 125031 0 2024-04-20T11:09:16.363 INFO:tasks.workunit.client.0.smithi117.stdout:7/747: chown d1/d4/d6/d2b/d64/cda 46906 1 2024-04-20T11:09:16.363 INFO:tasks.workunit.client.0.smithi117.stdout:0/800: rename d3/d7/d20 to d3/d7/d20/d39/d9a/dd0/db2/dbd/dfc 22 2024-04-20T11:09:16.363 INFO:tasks.workunit.client.0.smithi117.stdout:4/732: readlink d1/dd/d52/l89 0 2024-04-20T11:09:16.367 INFO:tasks.workunit.client.0.smithi117.stdout:7/748: dread - d1/d4/d6/d54/db4/fde zero size 2024-04-20T11:09:16.367 INFO:tasks.workunit.client.0.smithi117.stdout:9/768: dread d4/d35/f3d [0,4194304] 0 2024-04-20T11:09:16.367 INFO:tasks.workunit.client.0.smithi117.stdout:3/703: write d0/d5/d21/d77/f8a [707743,107931] 0 2024-04-20T11:09:16.369 INFO:tasks.workunit.client.0.smithi117.stdout:8/659: mkdir d0/d8/d11/d24/d2b/ddd 0 2024-04-20T11:09:16.371 INFO:tasks.workunit.client.0.smithi117.stdout:3/704: fsync d0/d5/d21/f4d 0 2024-04-20T11:09:16.373 INFO:tasks.workunit.client.0.smithi117.stdout:0/801: mknod d3/d7/d20/d39/d9a/dd0/db2/cfd 0 2024-04-20T11:09:16.374 INFO:tasks.workunit.client.0.smithi117.stdout:8/660: readlink d0/d8/d11/d24/d86/laf 0 2024-04-20T11:09:16.375 INFO:tasks.workunit.client.0.smithi117.stdout:4/733: symlink d1/dd/d1a/d43/dcc/lf5 0 2024-04-20T11:09:16.379 INFO:tasks.workunit.client.0.smithi117.stdout:9/769: mkdir d4/d4a/d72/dd2/d106 0 2024-04-20T11:09:16.380 INFO:tasks.workunit.client.0.smithi117.stdout:3/705: unlink d0/d5/f13 0 2024-04-20T11:09:16.383 INFO:tasks.workunit.client.0.smithi117.stdout:0/802: creat d3/d7/d20/d6e/ffe x:0 0 0 2024-04-20T11:09:16.392 INFO:tasks.workunit.client.0.smithi117.stdout:9/770: dread d4/d35/df7/d105/fc [0,4194304] 0 2024-04-20T11:09:16.394 INFO:tasks.workunit.client.0.smithi117.stdout:0/803: rename d3/d7/d20/d39/d9a/dd0/d22 to d3/d7/d20/d6e/df3/dff 0 2024-04-20T11:09:16.395 INFO:tasks.workunit.client.0.smithi117.stdout:9/771: chown d4/d35/df7/d105/f93 4630 1 2024-04-20T11:09:16.397 INFO:tasks.workunit.client.0.smithi117.stdout:8/661: unlink d0/d9/f2a 0 2024-04-20T11:09:16.400 INFO:tasks.workunit.client.0.smithi117.stdout:0/804: fsync d3/d5/f5a 0 2024-04-20T11:09:16.402 INFO:tasks.workunit.client.0.smithi117.stdout:7/749: write d1/d43/fa0 [2396134,54454] 0 2024-04-20T11:09:16.403 INFO:tasks.workunit.client.0.smithi117.stdout:9/772: fdatasync d4/d4a/d5e/d9b/dcd/fce 0 2024-04-20T11:09:16.412 INFO:tasks.workunit.client.0.smithi117.stdout:7/750: rename d1/d43/d59/f69 to d1/d4/d6/d13/fef 0 2024-04-20T11:09:16.413 INFO:tasks.workunit.client.0.smithi117.stdout:8/662: creat d0/dc/d49/d5b/da9/dc7/fde x:0 0 0 2024-04-20T11:09:16.417 INFO:tasks.workunit.client.0.smithi117.stdout:4/734: dwrite d1/dd/f38 [0,4194304] 0 2024-04-20T11:09:16.419 INFO:tasks.workunit.client.0.smithi117.stdout:3/706: dread d0/d2a/f3b [0,4194304] 0 2024-04-20T11:09:16.421 INFO:tasks.workunit.client.0.smithi117.stdout:8/663: dread d0/d8/d11/d24/d2b/d72/f75 [0,4194304] 0 2024-04-20T11:09:16.422 INFO:tasks.workunit.client.0.smithi117.stdout:9/773: rename d4/d35/df7/d105/c26 to d4/d5/d57/d82/c107 0 2024-04-20T11:09:16.423 INFO:tasks.workunit.client.0.smithi117.stdout:4/735: chown d1/d45/c63 17 1 2024-04-20T11:09:16.426 INFO:tasks.workunit.client.0.smithi117.stdout:4/736: read d1/d9/de9/fc0 [3302826,96895] 0 2024-04-20T11:09:16.430 INFO:tasks.workunit.client.0.smithi117.stdout:9/774: dread - d4/fbc zero size 2024-04-20T11:09:16.433 INFO:tasks.workunit.client.0.smithi117.stdout:4/737: unlink d1/dd/d1a/d43/l47 0 2024-04-20T11:09:16.434 INFO:tasks.workunit.client.0.smithi117.stdout:8/664: truncate d0/d8/d39/d7f/fb9 1148625 0 2024-04-20T11:09:16.436 INFO:tasks.workunit.client.0.smithi117.stdout:3/707: mknod d0/d5/d73/ded/d66/dbe/cef 0 2024-04-20T11:09:16.438 INFO:tasks.workunit.client.0.smithi117.stdout:4/738: truncate d1/dd/d1a/d2b/d86/fe5 136251 0 2024-04-20T11:09:16.440 INFO:tasks.workunit.client.0.smithi117.stdout:4/739: chown d1/dd/f68 17952 1 2024-04-20T11:09:16.440 INFO:tasks.workunit.client.0.smithi117.stdout:8/665: rmdir d0/d8/d11/d24/d86 39 2024-04-20T11:09:16.443 INFO:tasks.workunit.client.0.smithi117.stdout:0/805: truncate d3/d7/d20/d39/d3a/db1/fbe 998973 0 2024-04-20T11:09:16.444 INFO:tasks.workunit.client.0.smithi117.stdout:4/740: stat d1/d9/d1f/l93 0 2024-04-20T11:09:16.446 INFO:tasks.workunit.client.0.smithi117.stdout:8/666: fsync d0/d8/d39/d40/f51 0 2024-04-20T11:09:16.447 INFO:tasks.workunit.client.0.smithi117.stdout:0/806: dread - d3/d5/fdd zero size 2024-04-20T11:09:16.453 INFO:tasks.workunit.client.0.smithi117.stdout:8/667: read d0/d8/d11/d24/d27/d47/f8e [2518380,50711] 0 2024-04-20T11:09:16.454 INFO:tasks.workunit.client.0.smithi117.stdout:4/741: creat d1/dd/d1a/d2b/d86/ff6 x:0 0 0 2024-04-20T11:09:16.457 INFO:tasks.workunit.client.0.smithi117.stdout:3/708: dread d0/d5/d7/dd/f5a [0,4194304] 0 2024-04-20T11:09:16.459 INFO:tasks.workunit.client.0.smithi117.stdout:8/668: stat d0/d8/d11/d24/d2b/d72 0 2024-04-20T11:09:16.462 INFO:tasks.workunit.client.0.smithi117.stdout:0/807: dread d3/dab/d54/fa3 [0,4194304] 0 2024-04-20T11:09:16.462 INFO:tasks.workunit.client.0.smithi117.stdout:4/742: creat d1/d9/d79/ff7 x:0 0 0 2024-04-20T11:09:16.464 INFO:tasks.workunit.client.0.smithi117.stdout:9/775: dwrite d4/d5/d3c/f47 [4194304,4194304] 0 2024-04-20T11:09:16.464 INFO:tasks.workunit.client.0.smithi117.stdout:3/709: mkdir d0/d5/d73/ded/db3/df0 0 2024-04-20T11:09:16.464 INFO:tasks.workunit.client.0.smithi117.stdout:4/743: fsync d1/d9/d6b/ff0 0 2024-04-20T11:09:16.466 INFO:tasks.workunit.client.0.smithi117.stdout:8/669: mkdir d0/d9/dbb/ddf 0 2024-04-20T11:09:16.470 INFO:tasks.workunit.client.0.smithi117.stdout:0/808: chown d3/d7/d20/d6e/df3/dff/d85/c91 2166202 1 2024-04-20T11:09:16.472 INFO:tasks.workunit.client.0.smithi117.stdout:9/776: symlink d4/d5/d3c/l108 0 2024-04-20T11:09:16.475 INFO:tasks.workunit.client.0.smithi117.stdout:4/744: rename d1/d45/d4a/f50 to d1/d9/d6b/dca/ff8 0 2024-04-20T11:09:16.476 INFO:tasks.workunit.client.0.smithi117.stdout:3/710: mknod d0/d5/d7/cf1 0 2024-04-20T11:09:16.481 INFO:tasks.workunit.client.0.smithi117.stdout:0/809: stat d3/d7/d20/f4c 0 2024-04-20T11:09:16.481 INFO:tasks.workunit.client.0.smithi117.stdout:4/745: mknod d1/dd/d1a/d43/dcc/cf9 0 2024-04-20T11:09:16.482 INFO:tasks.workunit.client.0.smithi117.stdout:3/711: chown d0/d5/d73/ded/db3 1344 1 2024-04-20T11:09:16.482 INFO:tasks.workunit.client.0.smithi117.stdout:4/746: read d1/f10 [2282211,121132] 0 2024-04-20T11:09:16.483 INFO:tasks.workunit.client.0.smithi117.stdout:0/810: write d3/d7/d16/f2e [3285714,11779] 0 2024-04-20T11:09:16.487 INFO:tasks.workunit.client.0.smithi117.stdout:3/712: write d0/d5/d21/d94/d9a/feb [276911,80279] 0 2024-04-20T11:09:16.489 INFO:tasks.workunit.client.0.smithi117.stdout:4/747: read - d1/dd/d1a/d43/d9a/fb2 zero size 2024-04-20T11:09:16.490 INFO:tasks.workunit.client.0.smithi117.stdout:9/777: write d4/d35/df7/d105/f49 [1167172,18095] 0 2024-04-20T11:09:16.493 INFO:tasks.workunit.client.0.smithi117.stdout:3/713: creat d0/d5/d73/ded/d66/dbe/ff2 x:0 0 0 2024-04-20T11:09:16.493 INFO:tasks.workunit.client.0.smithi117.stdout:8/670: dwrite d0/d8/d39/d40/d78/d5c/f68 [0,4194304] 0 2024-04-20T11:09:16.495 INFO:tasks.workunit.client.0.smithi117.stdout:3/714: stat d0/d5/d73/ded/d66/dbe/fc6 0 2024-04-20T11:09:16.495 INFO:tasks.workunit.client.0.smithi117.stdout:8/671: readlink d0/dc/d49/la3 0 2024-04-20T11:09:16.501 INFO:tasks.workunit.client.0.smithi117.stdout:9/778: rmdir d4/d68 39 2024-04-20T11:09:16.505 INFO:tasks.workunit.client.0.smithi117.stdout:3/715: readlink d0/d5/d21/d75/d89/l93 0 2024-04-20T11:09:16.508 INFO:tasks.workunit.client.0.smithi117.stdout:9/779: rename d4/d4a/d74 to d4/d4a/d5e/d109 0 2024-04-20T11:09:16.510 INFO:tasks.workunit.client.0.smithi117.stdout:3/716: mkdir d0/d5/d21/d77/df3 0 2024-04-20T11:09:16.511 INFO:tasks.workunit.client.0.smithi117.stdout:4/748: write d1/d45/d4a/fba [718257,126661] 0 2024-04-20T11:09:16.512 INFO:tasks.workunit.client.0.smithi117.stdout:9/780: dread - d4/d35/df7/d105/d51/ff4 zero size 2024-04-20T11:09:16.517 INFO:tasks.workunit.client.0.smithi117.stdout:4/749: write d1/dd/d1a/d2b/fe6 [423684,86029] 0 2024-04-20T11:09:16.518 INFO:tasks.workunit.client.0.smithi117.stdout:3/717: mkdir d0/d5/d21/d75/d89/dd1/df4 0 2024-04-20T11:09:16.522 INFO:tasks.workunit.client.0.smithi117.stdout:3/718: chown d0/d5/d21/d88/d69/f8e 664 1 2024-04-20T11:09:16.525 INFO:tasks.workunit.client.0.smithi117.stdout:3/719: read d0/d2a/fdd [3316078,47731] 0 2024-04-20T11:09:16.526 INFO:tasks.workunit.client.0.smithi117.stdout:4/750: rename d1/d9/c2c to d1/d45/dcd/cfa 0 2024-04-20T11:09:16.528 INFO:tasks.workunit.client.0.smithi117.stdout:8/672: write d0/d8/d39/d40/d78/f98 [442417,7950] 0 2024-04-20T11:09:16.529 INFO:tasks.workunit.client.0.smithi117.stdout:3/720: write d0/d5/de/fd7 [1568891,116330] 0 2024-04-20T11:09:16.535 INFO:tasks.workunit.client.0.smithi117.stdout:4/751: truncate d1/d9/f12 2090901 0 2024-04-20T11:09:16.536 INFO:tasks.workunit.client.0.smithi117.stdout:3/721: stat d0/d5/d11/d80/cb8 0 2024-04-20T11:09:16.540 INFO:tasks.workunit.client.0.smithi117.stdout:4/752: symlink d1/dd/d1a/d43/d9a/lfb 0 2024-04-20T11:09:16.542 INFO:tasks.workunit.client.0.smithi117.stdout:0/811: dread d3/d7/d20/d6e/df3/dff/fb8 [0,4194304] 0 2024-04-20T11:09:16.544 INFO:tasks.workunit.client.0.smithi117.stdout:3/722: rename d0/d5/d73/ded/db3/dbc/fe5 to d0/d5/d21/d88/ff5 0 2024-04-20T11:09:16.545 INFO:tasks.workunit.client.0.smithi117.stdout:9/781: dread d4/d5/d3c/ddf/f4e [0,4194304] 0 2024-04-20T11:09:16.550 INFO:tasks.workunit.client.0.smithi117.stdout:4/753: rename d1/d9/d6b/c6f to d1/dd/d1a/d43/dcc/cfc 0 2024-04-20T11:09:16.550 INFO:tasks.workunit.client.0.smithi117.stdout:3/723: creat d0/d5/d7/dd/ff6 x:0 0 0 2024-04-20T11:09:16.554 INFO:tasks.workunit.client.0.smithi117.stdout:4/754: mkdir d1/d9/d6b/dfd 0 2024-04-20T11:09:16.560 INFO:tasks.workunit.client.0.smithi117.stdout:3/724: creat d0/d5/d73/ff7 x:0 0 0 2024-04-20T11:09:16.562 INFO:tasks.workunit.client.0.smithi117.stdout:4/755: mknod d1/d45/cfe 0 2024-04-20T11:09:16.565 INFO:tasks.workunit.client.0.smithi117.stdout:3/725: chown d0/d2a/fb0 3 1 2024-04-20T11:09:16.567 INFO:tasks.workunit.client.0.smithi117.stdout:4/756: write d1/d45/dcd/fec [55570,88279] 0 2024-04-20T11:09:16.568 INFO:tasks.workunit.client.0.smithi117.stdout:9/782: dwrite d4/d35/f5f [0,4194304] 0 2024-04-20T11:09:16.570 INFO:tasks.workunit.client.0.smithi117.stdout:0/812: dwrite d3/d7/d20/d6e/df3/dff/d85/fd4 [0,4194304] 0 2024-04-20T11:09:16.571 INFO:tasks.workunit.client.0.smithi117.stdout:8/673: dread d0/d8/d11/d24/d53/f56 [0,4194304] 0 2024-04-20T11:09:16.575 INFO:tasks.workunit.client.0.smithi117.stdout:3/726: rename d0/d5/d73/ded/d66/fe7 to d0/d5/d73/ded/dac/ff8 0 2024-04-20T11:09:16.579 INFO:tasks.workunit.client.0.smithi117.stdout:3/727: chown d0/d5/de/f92 7911 1 2024-04-20T11:09:16.579 INFO:tasks.workunit.client.0.smithi117.stdout:9/783: getdents d4 0 2024-04-20T11:09:16.580 INFO:tasks.workunit.client.0.smithi117.stdout:3/728: chown d0/d5 3872 1 2024-04-20T11:09:16.582 INFO:tasks.workunit.client.0.smithi117.stdout:3/729: fdatasync d0/f3 0 2024-04-20T11:09:16.585 INFO:tasks.workunit.client.0.smithi117.stdout:9/784: mkdir d4/de/d10/d94/dd8/d10a 0 2024-04-20T11:09:16.587 INFO:tasks.workunit.client.0.smithi117.stdout:3/730: creat d0/d5/d21/d94/d9a/dc9/ff9 x:0 0 0 2024-04-20T11:09:16.593 INFO:tasks.workunit.client.0.smithi117.stdout:3/731: link d0/d5/d21/d94/fb1 d0/d5/d21/d75/d89/dd1/df4/ffa 0 2024-04-20T11:09:16.597 INFO:tasks.workunit.client.0.smithi117.stdout:3/732: write d0/d5/d21/d88/d36/f33 [2852490,84923] 0 2024-04-20T11:09:16.597 INFO:tasks.workunit.client.0.smithi117.stdout:8/674: dwrite d0/d9/d3a/f8b [0,4194304] 0 2024-04-20T11:09:16.600 INFO:tasks.workunit.client.0.smithi117.stdout:8/675: chown d0/d8/d11/d6d/d69/f85 118 1 2024-04-20T11:09:16.604 INFO:tasks.workunit.client.0.smithi117.stdout:8/676: symlink d0/d8/d39/d7f/le0 0 2024-04-20T11:09:16.606 INFO:tasks.workunit.client.0.smithi117.stdout:3/733: getdents d0/d5/d21/d77 0 2024-04-20T11:09:16.610 INFO:tasks.workunit.client.0.smithi117.stdout:3/734: write d0/d5/d21/f42 [4773310,24729] 0 2024-04-20T11:09:16.618 INFO:tasks.workunit.client.0.smithi117.stdout:3/735: link d0/d5/d21/d88/d3c/c3d d0/d5/d21/d75/dc1/cfb 0 2024-04-20T11:09:16.622 INFO:tasks.workunit.client.0.smithi117.stdout:3/736: symlink d0/d5/d21/d77/lfc 0 2024-04-20T11:09:16.626 INFO:tasks.workunit.client.0.smithi117.stdout:8/677: dread d0/d8/d11/d24/d2b/fab [0,4194304] 0 2024-04-20T11:09:16.628 INFO:tasks.workunit.client.0.smithi117.stdout:3/737: unlink d0/d5/d21/d88/f1f 0 2024-04-20T11:09:16.633 INFO:tasks.workunit.client.0.smithi117.stdout:8/678: symlink d0/d8/d11/d24/d27/d47/le1 0 2024-04-20T11:09:16.634 INFO:tasks.workunit.client.0.smithi117.stdout:3/738: rmdir d0/d5/d21/d94 39 2024-04-20T11:09:16.639 INFO:tasks.workunit.client.0.smithi117.stdout:3/739: unlink d0/d5/d11/l3f 0 2024-04-20T11:09:16.642 INFO:tasks.workunit.client.0.smithi117.stdout:3/740: fdatasync d0/f3 0 2024-04-20T11:09:16.649 INFO:tasks.workunit.client.0.smithi117.stdout:3/741: creat d0/d5/d21/d75/d89/dd1/df4/ffd x:0 0 0 2024-04-20T11:09:16.653 INFO:tasks.workunit.client.0.smithi117.stdout:4/757: dread d1/fc3 [0,4194304] 0 2024-04-20T11:09:16.655 INFO:tasks.workunit.client.0.smithi117.stdout:9/785: dread d4/d35/f56 [0,4194304] 0 2024-04-20T11:09:16.658 INFO:tasks.workunit.client.0.smithi117.stdout:3/742: mknod d0/d5/d21/cfe 0 2024-04-20T11:09:16.659 INFO:tasks.workunit.client.0.smithi117.stdout:3/743: chown d0/d5/d7/cf1 0 1 2024-04-20T11:09:16.659 INFO:tasks.workunit.client.0.smithi117.stdout:0/813: dread d3/d7/d20/d6e/df3/dff/f6f [0,4194304] 0 2024-04-20T11:09:16.660 INFO:tasks.workunit.client.0.smithi117.stdout:4/758: stat d1/dd/d1a/d43/d9a/fae 0 2024-04-20T11:09:16.664 INFO:tasks.workunit.client.0.smithi117.stdout:3/744: fsync d0/d5/fe2 0 2024-04-20T11:09:16.664 INFO:tasks.workunit.client.0.smithi117.stdout:4/759: read d1/dd/d1a/d43/d9a/fc2 [2717644,10005] 0 2024-04-20T11:09:16.671 INFO:tasks.workunit.client.0.smithi117.stdout:3/745: mkdir d0/d5/d7/dd/dff 0 2024-04-20T11:09:16.677 INFO:tasks.workunit.client.0.smithi117.stdout:4/760: symlink d1/d9/de9/d9e/lff 0 2024-04-20T11:09:16.683 INFO:tasks.workunit.client.0.smithi117.stdout:3/746: getdents d0/d5/d73/ded/db3/df0 0 2024-04-20T11:09:16.686 INFO:tasks.workunit.client.0.smithi117.stdout:0/814: dread d3/d7/f14 [8388608,4194304] 0 2024-04-20T11:09:16.693 INFO:tasks.workunit.client.0.smithi117.stdout:4/761: rename d1/dd/fa8 to d1/dd/d1a/f100 0 2024-04-20T11:09:16.696 INFO:tasks.workunit.client.0.smithi117.stdout:3/747: rename d0/d5/d21/d94/d9a/dc9/fca to d0/d5/d21/d77/df3/f100 0 2024-04-20T11:09:16.702 INFO:tasks.workunit.client.0.smithi117.stdout:3/748: creat d0/d5/d11/f101 x:0 0 0 2024-04-20T11:09:16.703 INFO:tasks.workunit.client.0.smithi117.stdout:9/786: write d4/d5/d3c/ddf/d31/f5d [3258876,10255] 0 2024-04-20T11:09:16.704 INFO:tasks.workunit.client.0.smithi117.stdout:3/749: write d0/d5/d21/d88/f70 [1182271,4818] 0 2024-04-20T11:09:16.708 INFO:tasks.workunit.client.0.smithi117.stdout:9/787: unlink d4/d4a/f7e 0 2024-04-20T11:09:16.710 INFO:tasks.workunit.client.0.smithi117.stdout:9/788: write d4/d5/d57/d82/da5/fbe [1771360,57181] 0 2024-04-20T11:09:16.710 INFO:tasks.workunit.client.0.smithi117.stdout:3/750: mknod d0/d5/d73/ded/db3/dbc/c102 0 2024-04-20T11:09:16.716 INFO:tasks.workunit.client.0.smithi117.stdout:3/751: mkdir d0/d5/d21/d77/df3/d103 0 2024-04-20T11:09:16.722 INFO:tasks.workunit.client.0.smithi117.stdout:9/789: rename d4/l20 to d4/d35/l10b 0 2024-04-20T11:09:16.724 INFO:tasks.workunit.client.0.smithi117.stdout:0/815: write d3/d7/f3f [890690,11440] 0 2024-04-20T11:09:16.726 INFO:tasks.workunit.client.0.smithi117.stdout:0/816: stat d3/dab/c40 0 2024-04-20T11:09:16.730 INFO:tasks.workunit.client.0.smithi117.stdout:4/762: write d1/d45/d4a/d59/fbc [1195910,2549] 0 2024-04-20T11:09:16.733 INFO:tasks.workunit.client.0.smithi117.stdout:0/817: rename d3/d7/d20/d6e/df3/dff/d85/ld8 to d3/d5/l100 0 2024-04-20T11:09:16.734 INFO:tasks.workunit.client.0.smithi117.stdout:4/763: truncate d1/d9/de9/fe8 888524 0 2024-04-20T11:09:16.738 INFO:tasks.workunit.client.0.smithi117.stdout:0/818: read d3/d7/f18 [2619777,105462] 0 2024-04-20T11:09:16.740 INFO:tasks.workunit.client.0.smithi117.stdout:3/752: dread d0/f98 [0,4194304] 0 2024-04-20T11:09:16.742 INFO:tasks.workunit.client.0.smithi117.stdout:0/819: truncate d3/d7/d20/d47/ff2 768235 0 2024-04-20T11:09:16.746 INFO:tasks.workunit.client.0.smithi117.stdout:0/820: symlink d3/d7/d20/d6e/df3/dff/d85/ddb/l101 0 2024-04-20T11:09:16.752 INFO:tasks.workunit.client.0.smithi117.stdout:9/790: dread d4/d5/d3c/ddf/f4e [0,4194304] 0 2024-04-20T11:09:16.755 INFO:tasks.workunit.client.0.smithi117.stdout:9/791: read d4/d5/d57/d82/da5/fbe [9761,116530] 0 2024-04-20T11:09:16.758 INFO:tasks.workunit.client.0.smithi117.stdout:9/792: readlink d4/de/d7b/d96/db0/lda 0 2024-04-20T11:09:16.761 INFO:tasks.workunit.client.0.smithi117.stdout:9/793: creat d4/d5/d3c/d7a/f10c x:0 0 0 2024-04-20T11:09:16.763 INFO:tasks.workunit.client.0.smithi117.stdout:9/794: rmdir d4 39 2024-04-20T11:09:16.766 INFO:tasks.workunit.client.0.smithi117.stdout:4/764: dread d1/d9/f96 [0,4194304] 0 2024-04-20T11:09:16.769 INFO:tasks.workunit.client.0.smithi117.stdout:4/765: chown d1/dd/d1a/d43/d9a/lfb 14171 1 2024-04-20T11:09:16.771 INFO:tasks.workunit.client.0.smithi117.stdout:4/766: mknod d1/dd/d1a/d2b/d86/dda/c101 0 2024-04-20T11:09:16.775 INFO:tasks.workunit.client.0.smithi117.stdout:4/767: rmdir d1/d9/d6b 39 2024-04-20T11:09:16.780 INFO:tasks.workunit.client.0.smithi117.stdout:4/768: rename d1/f7 to d1/dd/d1a/d2b/d86/f102 0 2024-04-20T11:09:16.782 INFO:tasks.workunit.client.0.smithi117.stdout:0/821: dread d3/d7/f4d [4194304,4194304] 0 2024-04-20T11:09:16.783 INFO:tasks.workunit.client.0.smithi117.stdout:4/769: creat d1/d9/d1f/f103 x:0 0 0 2024-04-20T11:09:16.785 INFO:tasks.workunit.client.0.smithi117.stdout:9/795: dwrite f2 [4194304,4194304] 0 2024-04-20T11:09:16.786 INFO:tasks.workunit.client.0.smithi117.stdout:4/770: creat d1/dd/d1a/d43/dcc/f104 x:0 0 0 2024-04-20T11:09:16.794 INFO:tasks.workunit.client.0.smithi117.stdout:9/796: getdents d4/d35/df7/d105/d51 0 2024-04-20T11:09:16.820 INFO:tasks.workunit.client.0.smithi117.stdout:9/797: dread d4/d4a/d63/f76 [0,4194304] 0 2024-04-20T11:09:16.823 INFO:tasks.workunit.client.0.smithi117.stdout:9/798: read d4/d5/f28 [2670815,117231] 0 2024-04-20T11:09:16.825 INFO:tasks.workunit.client.0.smithi117.stdout:9/799: chown d4/de/d10/f9d 210 1 2024-04-20T11:09:17.024 INFO:tasks.workunit.client.0.smithi117.stdout:6/869: sync 2024-04-20T11:09:17.063 INFO:tasks.workunit.client.0.smithi117.stdout:6/870: dwrite d3/df/f14 [0,4194304] 0 2024-04-20T11:09:17.105 INFO:tasks.workunit.client.0.smithi117.stdout:6/871: dwrite d3/d4/f46 [0,4194304] 0 2024-04-20T11:09:17.119 INFO:tasks.workunit.client.0.smithi117.stdout:6/872: creat d3/f11a x:0 0 0 2024-04-20T11:09:17.122 INFO:tasks.workunit.client.0.smithi117.stdout:1/673: sync 2024-04-20T11:09:17.130 INFO:tasks.workunit.client.0.smithi117.stdout:6/873: fdatasync d3/d5a/d74/db3/dbc/d9b/fa0 0 2024-04-20T11:09:17.146 INFO:tasks.workunit.client.0.smithi117.stdout:1/674: unlink d6/d1f/d30/d49/d69/d6e/c8b 0 2024-04-20T11:09:17.148 INFO:tasks.workunit.client.0.smithi117.stdout:6/874: mkdir d3/df/d39/daa/d11b 0 2024-04-20T11:09:17.150 INFO:tasks.workunit.client.0.smithi117.stdout:6/875: chown d3/d50/d8a 301970051 1 2024-04-20T11:09:17.156 INFO:tasks.workunit.client.0.smithi117.stdout:1/675: creat d6/d1f/d30/d32/fdd x:0 0 0 2024-04-20T11:09:17.158 INFO:tasks.workunit.client.0.smithi117.stdout:6/876: creat d3/d4/d2e/dc0/f11c x:0 0 0 2024-04-20T11:09:17.162 INFO:tasks.workunit.client.0.smithi117.stdout:2/858: sync 2024-04-20T11:09:17.162 INFO:tasks.workunit.client.0.smithi117.stdout:7/751: sync 2024-04-20T11:09:17.162 INFO:tasks.workunit.client.0.smithi117.stdout:9/800: sync 2024-04-20T11:09:17.162 INFO:tasks.workunit.client.0.smithi117.stdout:3/753: sync 2024-04-20T11:09:17.163 INFO:tasks.workunit.client.0.smithi117.stdout:4/771: sync 2024-04-20T11:09:17.163 INFO:tasks.workunit.client.0.smithi117.stdout:0/822: sync 2024-04-20T11:09:17.163 INFO:tasks.workunit.client.0.smithi117.stdout:8/679: sync 2024-04-20T11:09:17.164 INFO:tasks.workunit.client.0.smithi117.stdout:6/877: creat d3/d3a/d7f/f11d x:0 0 0 2024-04-20T11:09:17.166 INFO:tasks.workunit.client.0.smithi117.stdout:4/772: stat d1/d9/de9/c94 0 2024-04-20T11:09:17.168 INFO:tasks.workunit.client.0.smithi117.stdout:3/754: chown d0/d5/d21/d88/d3c/c3e 417 1 2024-04-20T11:09:17.168 INFO:tasks.workunit.client.0.smithi117.stdout:4/773: stat d1/d45/dcd/fec 0 2024-04-20T11:09:17.170 INFO:tasks.workunit.client.0.smithi117.stdout:0/823: dread - d3/d7/d20/d39/d9a/dd0/d6d/f97 zero size 2024-04-20T11:09:17.176 INFO:tasks.workunit.client.0.smithi117.stdout:7/752: truncate d1/d4/d39/f4d 5484719 0 2024-04-20T11:09:17.179 INFO:tasks.workunit.client.0.smithi117.stdout:3/755: fdatasync d0/f98 0 2024-04-20T11:09:17.183 INFO:tasks.workunit.client.0.smithi117.stdout:6/878: mknod d3/d4/d2e/d113/c11e 0 2024-04-20T11:09:17.183 INFO:tasks.workunit.client.0.smithi117.stdout:3/756: readlink d0/d5/d21/d75/d89/l9f 0 2024-04-20T11:09:17.183 INFO:tasks.workunit.client.0.smithi117.stdout:6/879: dread - d3/d94/fcc zero size 2024-04-20T11:09:17.185 INFO:tasks.workunit.client.0.smithi117.stdout:6/880: chown d3/df/fc8 82413 1 2024-04-20T11:09:17.186 INFO:tasks.workunit.client.0.smithi117.stdout:4/774: rename d1/d45/f4f to d1/d9/d79/f105 0 2024-04-20T11:09:17.191 INFO:tasks.workunit.client.0.smithi117.stdout:2/859: creat d1/d4/d83/d29/f115 x:0 0 0 2024-04-20T11:09:17.198 INFO:tasks.workunit.client.0.smithi117.stdout:8/680: dread d0/d9/f17 [0,4194304] 0 2024-04-20T11:09:17.201 INFO:tasks.workunit.client.0.smithi117.stdout:9/801: dread d4/d68/fb3 [0,4194304] 0 2024-04-20T11:09:17.202 INFO:tasks.workunit.client.0.smithi117.stdout:6/881: unlink d3/d4/d2e/d3f/lf3 0 2024-04-20T11:09:17.205 INFO:tasks.workunit.client.0.smithi117.stdout:8/681: read d0/d8/d11/d24/f46 [1844446,87106] 0 2024-04-20T11:09:17.215 INFO:tasks.workunit.client.0.smithi117.stdout:6/882: symlink d3/d3a/l11f 0 2024-04-20T11:09:17.216 INFO:tasks.workunit.client.0.smithi117.stdout:9/802: dread f0 [4194304,4194304] 0 2024-04-20T11:09:17.217 INFO:tasks.workunit.client.0.smithi117.stdout:6/883: stat d3/d5a/l69 0 2024-04-20T11:09:17.218 INFO:tasks.workunit.client.0.smithi117.stdout:3/757: creat d0/d5/d21/f104 x:0 0 0 2024-04-20T11:09:17.219 INFO:tasks.workunit.client.0.smithi117.stdout:8/682: truncate d0/d8/d39/d40/f43 2949857 0 2024-04-20T11:09:17.220 INFO:tasks.workunit.client.0.smithi117.stdout:8/683: chown d0/d8/d39/l52 98776 1 2024-04-20T11:09:17.224 INFO:tasks.workunit.client.0.smithi117.stdout:6/884: mkdir d3/df/d39/daa/d120 0 2024-04-20T11:09:17.226 INFO:tasks.workunit.client.0.smithi117.stdout:4/775: rename d1/dd/d1a/c28 to d1/c106 0 2024-04-20T11:09:17.230 INFO:tasks.workunit.client.0.smithi117.stdout:4/776: mkdir d1/d9/d1f/d90/d107 0 2024-04-20T11:09:17.252 INFO:tasks.workunit.client.0.smithi117.stdout:2/860: rename d1/da5/df2/fb1 to d1/d4/dd/d17/f116 0 2024-04-20T11:09:17.257 INFO:tasks.workunit.client.0.smithi117.stdout:6/885: link d3/l1e d3/df/d39/daa/l121 0 2024-04-20T11:09:17.258 INFO:tasks.workunit.client.0.smithi117.stdout:9/803: dwrite d4/d5/d3c/ddf/f4e [4194304,4194304] 0 2024-04-20T11:09:17.264 INFO:tasks.workunit.client.0.smithi117.stdout:4/777: write d1/d9/de9/fe8 [1662648,80160] 0 2024-04-20T11:09:17.266 INFO:tasks.workunit.client.0.smithi117.stdout:2/861: fsync d1/d4/d83/d29/d62/f9a 0 2024-04-20T11:09:17.268 INFO:tasks.workunit.client.0.smithi117.stdout:6/886: write d3/d3a/d114/f118 [666206,52340] 0 2024-04-20T11:09:17.270 INFO:tasks.workunit.client.0.smithi117.stdout:7/753: dwrite d1/d4/d3d/f5a [0,4194304] 0 2024-04-20T11:09:17.274 INFO:tasks.workunit.client.0.smithi117.stdout:0/824: dwrite d3/dab/fa8 [4194304,4194304] 0 2024-04-20T11:09:17.275 INFO:tasks.workunit.client.0.smithi117.stdout:7/754: readlink d1/l20 0 2024-04-20T11:09:17.275 INFO:tasks.workunit.client.0.smithi117.stdout:4/778: mkdir d1/dd/d1a/d43/d108 0 2024-04-20T11:09:17.276 INFO:tasks.workunit.client.0.smithi117.stdout:2/862: symlink d1/d4/d83/d29/d62/df7/l117 0 2024-04-20T11:09:17.279 INFO:tasks.workunit.client.0.smithi117.stdout:9/804: getdents d4/de/d10/d94 0 2024-04-20T11:09:17.279 INFO:tasks.workunit.client.0.smithi117.stdout:6/887: creat d3/d4/d2e/dc0/df7/f122 x:0 0 0 2024-04-20T11:09:17.281 INFO:tasks.workunit.client.0.smithi117.stdout:7/755: rename d1/f7 to d1/dd1/ff0 0 2024-04-20T11:09:17.282 INFO:tasks.workunit.client.0.smithi117.stdout:6/888: dread - d3/d4/d2e/dc0/df7/f122 zero size 2024-04-20T11:09:17.286 INFO:tasks.workunit.client.0.smithi117.stdout:2/863: mkdir d1/d15/d118 0 2024-04-20T11:09:17.287 INFO:tasks.workunit.client.0.smithi117.stdout:7/756: creat d1/d4/d6/d13/ff1 x:0 0 0 2024-04-20T11:09:17.291 INFO:tasks.workunit.client.0.smithi117.stdout:2/864: mkdir d1/d4/d83/d29/d3f/df6/d119 0 2024-04-20T11:09:17.293 INFO:tasks.workunit.client.0.smithi117.stdout:0/825: rename d3/d7/d20/d39/d9a/dd0/db2/fd7 to d3/d7/d20/d39/d9a/dd0/f102 0 2024-04-20T11:09:17.295 INFO:tasks.workunit.client.0.smithi117.stdout:0/826: chown d3/dab 1292206 1 2024-04-20T11:09:17.296 INFO:tasks.workunit.client.0.smithi117.stdout:2/865: mkdir d1/d15/d118/d11a 0 2024-04-20T11:09:17.301 INFO:tasks.workunit.client.0.smithi117.stdout:4/779: write d1/dd/d1a/d2b/f8f [964629,8667] 0 2024-04-20T11:09:17.304 INFO:tasks.workunit.client.0.smithi117.stdout:6/889: write d3/df/d39/d99/fe0 [887843,16028] 0 2024-04-20T11:09:17.306 INFO:tasks.workunit.client.0.smithi117.stdout:9/805: dwrite d4/de/d10/f9d [0,4194304] 0 2024-04-20T11:09:17.311 INFO:tasks.workunit.client.0.smithi117.stdout:0/827: dread d3/d5/d74/fad [0,4194304] 0 2024-04-20T11:09:17.317 INFO:tasks.workunit.client.0.smithi117.stdout:9/806: mkdir d4/d4a/d72/dd2/d106/d10d 0 2024-04-20T11:09:17.320 INFO:tasks.workunit.client.0.smithi117.stdout:0/828: mknod d3/d7/d20/d39/d9a/dd0/c103 0 2024-04-20T11:09:17.322 INFO:tasks.workunit.client.0.smithi117.stdout:9/807: chown d4/d4a/d5e/de8/fe5 1896 1 2024-04-20T11:09:17.322 INFO:tasks.workunit.client.0.smithi117.stdout:7/757: dwrite d1/d4/d3d/dce/fd9 [4194304,4194304] 0 2024-04-20T11:09:17.322 INFO:tasks.workunit.client.0.smithi117.stdout:1/676: sync 2024-04-20T11:09:17.326 INFO:tasks.workunit.client.0.smithi117.stdout:1/677: read - d6/d16/d3a/da8/fbc zero size 2024-04-20T11:09:17.332 INFO:tasks.workunit.client.0.smithi117.stdout:1/678: stat d6/d16/dce/ldb 0 2024-04-20T11:09:17.333 INFO:tasks.workunit.client.0.smithi117.stdout:6/890: dread d3/f89 [0,4194304] 0 2024-04-20T11:09:17.336 INFO:tasks.workunit.client.0.smithi117.stdout:1/679: chown d6/d21/l3f 39 1 2024-04-20T11:09:17.339 INFO:tasks.workunit.client.0.smithi117.stdout:8/684: dread d0/d8/d11/d24/d2b/f5d [0,4194304] 0 2024-04-20T11:09:17.347 INFO:tasks.workunit.client.0.smithi117.stdout:7/758: dwrite d1/d1d/d42/faf [0,4194304] 0 2024-04-20T11:09:17.350 INFO:tasks.workunit.client.0.smithi117.stdout:7/759: stat d1/d1d/d42/f8b 0 2024-04-20T11:09:17.355 INFO:tasks.workunit.client.0.smithi117.stdout:0/829: truncate d3/dab/fa8 5890023 0 2024-04-20T11:09:17.355 INFO:tasks.workunit.client.0.smithi117.stdout:4/780: write d1/dd/d1a/f100 [933636,63731] 0 2024-04-20T11:09:17.356 INFO:tasks.workunit.client.0.smithi117.stdout:9/808: write d4/d35/df7/d105/fde [2223494,25662] 0 2024-04-20T11:09:17.356 INFO:tasks.workunit.client.0.smithi117.stdout:2/866: dread d1/da5/df2/f98 [0,4194304] 0 2024-04-20T11:09:17.357 INFO:tasks.workunit.client.0.smithi117.stdout:1/680: mknod d6/d1f/d30/d49/d69/cde 0 2024-04-20T11:09:17.357 INFO:tasks.workunit.client.0.smithi117.stdout:8/685: dread - d0/fad zero size 2024-04-20T11:09:17.362 INFO:tasks.workunit.client.0.smithi117.stdout:1/681: chown d6/d35/f56 9736848 1 2024-04-20T11:09:17.363 INFO:tasks.workunit.client.0.smithi117.stdout:8/686: write d0/d8/d39/fc9 [381137,26032] 0 2024-04-20T11:09:17.365 INFO:tasks.workunit.client.0.smithi117.stdout:4/781: creat d1/d9/d6b/f109 x:0 0 0 2024-04-20T11:09:17.366 INFO:tasks.workunit.client.0.smithi117.stdout:9/809: truncate d4/d4a/d5e/de8/fe5 1149161 0 2024-04-20T11:09:17.368 INFO:tasks.workunit.client.0.smithi117.stdout:0/830: mkdir d3/d7/d20/d39/d9a/dd0/db2/dbd/d104 0 2024-04-20T11:09:17.369 INFO:tasks.workunit.client.0.smithi117.stdout:1/682: read d6/d16/fad [2115933,30839] 0 2024-04-20T11:09:17.373 INFO:tasks.workunit.client.0.smithi117.stdout:1/683: chown d6/d16/d3a/d5a/d5c/d61/d71/d6b 44537664 1 2024-04-20T11:09:17.375 INFO:tasks.workunit.client.0.smithi117.stdout:2/867: truncate d1/d4/d83/d29/d2e/d53/d93/f99 100951 0 2024-04-20T11:09:17.375 INFO:tasks.workunit.client.0.smithi117.stdout:8/687: rename d0/fad to d0/d8/d39/fe2 0 2024-04-20T11:09:17.376 INFO:tasks.workunit.client.0.smithi117.stdout:7/760: link d1/d1d/d4e/d53/db1/cb5 d1/d4/d6/d54/d63/cf2 0 2024-04-20T11:09:17.378 INFO:tasks.workunit.client.0.smithi117.stdout:3/758: sync 2024-04-20T11:09:17.382 INFO:tasks.workunit.client.0.smithi117.stdout:6/891: write d3/d3a/d7f/fa4 [501033,58861] 0 2024-04-20T11:09:17.386 INFO:tasks.workunit.client.0.smithi117.stdout:0/831: read d3/d7/d20/d39/d9a/dd0/d6d/f9b [7226264,105763] 0 2024-04-20T11:09:17.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:16 smithi117 ceph-mon[15659]: pgmap v18: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 111 MiB/s wr, 265 op/s 2024-04-20T11:09:17.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:16 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:09:17.391 INFO:tasks.workunit.client.0.smithi117.stdout:9/810: mknod d4/d4a/d63/d98/dc0/dcb/ddc/c10e 0 2024-04-20T11:09:17.395 INFO:tasks.workunit.client.0.smithi117.stdout:3/759: mknod d0/d5/d21/d88/d85/c105 0 2024-04-20T11:09:17.405 INFO:tasks.workunit.client.0.smithi117.stdout:1/684: write d6/d1f/d30/d49/d69/f92 [787116,26244] 0 2024-04-20T11:09:17.409 INFO:tasks.workunit.client.0.smithi117.stdout:2/868: dread d1/d4/dd/fc1 [0,4194304] 0 2024-04-20T11:09:17.410 INFO:tasks.workunit.client.0.smithi117.stdout:6/892: write d3/d4/d2e/f40 [574454,114437] 0 2024-04-20T11:09:17.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:16 smithi173 ceph-mon[23800]: pgmap v18: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 111 MiB/s wr, 265 op/s 2024-04-20T11:09:17.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:16 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:09:17.414 INFO:tasks.workunit.client.0.smithi117.stdout:8/688: mkdir d0/d9/d3a/de3 0 2024-04-20T11:09:17.416 INFO:tasks.workunit.client.0.smithi117.stdout:1/685: chown d6/d1f/l47 3855001 1 2024-04-20T11:09:17.416 INFO:tasks.workunit.client.0.smithi117.stdout:4/782: dwrite d1/dd/d1a/d43/d4e/f62 [4194304,4194304] 0 2024-04-20T11:09:17.417 INFO:tasks.workunit.client.0.smithi117.stdout:6/893: creat d3/df/d39/f123 x:0 0 0 2024-04-20T11:09:17.418 INFO:tasks.workunit.client.0.smithi117.stdout:3/760: creat d0/d5/d73/ded/d66/dbe/f106 x:0 0 0 2024-04-20T11:09:17.419 INFO:tasks.workunit.client.0.smithi117.stdout:0/832: link d3/d5/f5a d3/d7/d20/f105 0 2024-04-20T11:09:17.424 INFO:tasks.workunit.client.0.smithi117.stdout:4/783: stat d1/dd/d52/l8a 0 2024-04-20T11:09:17.425 INFO:tasks.workunit.client.0.smithi117.stdout:7/761: dwrite d1/d4/d6/d2b/f33 [0,4194304] 0 2024-04-20T11:09:17.430 INFO:tasks.workunit.client.0.smithi117.stdout:1/686: mkdir d6/d1f/d30/d49/d69/d6e/d6f/d73/ddf 0 2024-04-20T11:09:17.432 INFO:tasks.workunit.client.0.smithi117.stdout:0/833: rename d3/d7/d20/d39/lcb to d3/d7/d94/l106 0 2024-04-20T11:09:17.435 INFO:tasks.workunit.client.0.smithi117.stdout:7/762: creat d1/dd1/ff3 x:0 0 0 2024-04-20T11:09:17.438 INFO:tasks.workunit.client.0.smithi117.stdout:4/784: mknod d1/dd/d1a/d43/d4e/c10a 0 2024-04-20T11:09:17.445 INFO:tasks.workunit.client.0.smithi117.stdout:6/894: dwrite d3/d3a/f6b [0,4194304] 0 2024-04-20T11:09:17.445 INFO:tasks.workunit.client.0.smithi117.stdout:2/869: dread d1/d4/d83/d29/fda [0,4194304] 0 2024-04-20T11:09:17.453 INFO:tasks.workunit.client.0.smithi117.stdout:6/895: truncate d3/d50/fb8 1331018 0 2024-04-20T11:09:17.453 INFO:tasks.workunit.client.0.smithi117.stdout:0/834: chown d3/d7/d94/l99 39131 1 2024-04-20T11:09:17.454 INFO:tasks.workunit.client.0.smithi117.stdout:7/763: creat d1/d1d/d42/d2d/d6c/ff4 x:0 0 0 2024-04-20T11:09:17.457 INFO:tasks.workunit.client.0.smithi117.stdout:9/811: dread d4/de/d7b/fbd [0,4194304] 0 2024-04-20T11:09:17.457 INFO:tasks.workunit.client.0.smithi117.stdout:4/785: chown d1/d9/f12 3514926 1 2024-04-20T11:09:17.458 INFO:tasks.workunit.client.0.smithi117.stdout:8/689: link d0/d8/d11/d24/d53/c74 d0/dd3/ce4 0 2024-04-20T11:09:17.459 INFO:tasks.workunit.client.0.smithi117.stdout:3/761: dread d0/d5/d73/ded/dac/ff8 [0,4194304] 0 2024-04-20T11:09:17.462 INFO:tasks.workunit.client.0.smithi117.stdout:2/870: symlink d1/da5/df2/d46/l11b 0 2024-04-20T11:09:17.463 INFO:tasks.workunit.client.0.smithi117.stdout:7/764: dread - d1/fea zero size 2024-04-20T11:09:17.463 INFO:tasks.workunit.client.0.smithi117.stdout:1/687: write d6/f1a [2095629,58252] 0 2024-04-20T11:09:17.463 INFO:tasks.workunit.client.0.smithi117.stdout:3/762: chown d0/d5 1295 1 2024-04-20T11:09:17.464 INFO:tasks.workunit.client.0.smithi117.stdout:0/835: stat d3/d7/d20/d6e/df3/ffa 0 2024-04-20T11:09:17.464 INFO:tasks.workunit.client.0.smithi117.stdout:6/896: mknod d3/df/d39/daa/d120/c124 0 2024-04-20T11:09:17.468 INFO:tasks.workunit.client.0.smithi117.stdout:8/690: write d0/d8/d11/d24/d53/f7b [4598538,24844] 0 2024-04-20T11:09:17.471 INFO:tasks.workunit.client.0.smithi117.stdout:9/812: rename d4/d5/da1/leb to d4/de4/l10f 0 2024-04-20T11:09:17.472 INFO:tasks.workunit.client.0.smithi117.stdout:6/897: write d3/df/d39/d99/fcd [268789,66331] 0 2024-04-20T11:09:17.474 INFO:tasks.workunit.client.0.smithi117.stdout:3/763: dread - d0/d5/d73/ded/d66/dbe/fc6 zero size 2024-04-20T11:09:17.477 INFO:tasks.workunit.client.0.smithi117.stdout:2/871: mknod d1/d4/d83/d29/d2e/d53/df0/c11c 0 2024-04-20T11:09:17.477 INFO:tasks.workunit.client.0.smithi117.stdout:0/836: creat d3/d7/d20/d39/dbb/f107 x:0 0 0 2024-04-20T11:09:17.478 INFO:tasks.workunit.client.0.smithi117.stdout:3/764: write d0/d2a/fb0 [1798391,11060] 0 2024-04-20T11:09:17.478 INFO:tasks.workunit.client.0.smithi117.stdout:1/688: unlink d6/d16/d3a/d5a/d5c/d61/d71/cca 0 2024-04-20T11:09:17.483 INFO:tasks.workunit.client.0.smithi117.stdout:9/813: creat d4/de/d7b/d96/f110 x:0 0 0 2024-04-20T11:09:17.484 INFO:tasks.workunit.client.0.smithi117.stdout:8/691: write d0/d8/d11/d6d/d84/fc1 [152986,44996] 0 2024-04-20T11:09:17.484 INFO:tasks.workunit.client.0.smithi117.stdout:2/872: chown d1/da5/le8 204023734 1 2024-04-20T11:09:17.485 INFO:tasks.workunit.client.0.smithi117.stdout:7/765: rename d1/d43/f7b to d1/d1d/d42/d2d/d6c/db3/ff5 0 2024-04-20T11:09:17.486 INFO:tasks.workunit.client.0.smithi117.stdout:0/837: dread - d3/d7/d20/d39/d3a/db1/fc4 zero size 2024-04-20T11:09:17.488 INFO:tasks.workunit.client.0.smithi117.stdout:6/898: creat d3/d94/f125 x:0 0 0 2024-04-20T11:09:17.495 INFO:tasks.workunit.client.0.smithi117.stdout:8/692: mkdir d0/d8/d39/d40/de5 0 2024-04-20T11:09:17.496 INFO:tasks.workunit.client.0.smithi117.stdout:0/838: creat d3/d7/f108 x:0 0 0 2024-04-20T11:09:17.496 INFO:tasks.workunit.client.0.smithi117.stdout:4/786: dwrite d1/d45/d4a/f6c [0,4194304] 0 2024-04-20T11:09:17.499 INFO:tasks.workunit.client.0.smithi117.stdout:3/765: mknod d0/d5/d7/dd/dff/c107 0 2024-04-20T11:09:17.500 INFO:tasks.workunit.client.0.smithi117.stdout:7/766: mknod d1/d4/d6/d54/db4/cf6 0 2024-04-20T11:09:17.502 INFO:tasks.workunit.client.0.smithi117.stdout:9/814: truncate d4/d35/df7/d105/f62 3926460 0 2024-04-20T11:09:17.505 INFO:tasks.workunit.client.0.smithi117.stdout:9/815: fsync d4/d4a/d63/d98/fea 0 2024-04-20T11:09:17.506 INFO:tasks.workunit.client.0.smithi117.stdout:6/899: rename d3/d5a/d74/db3/df0/db7/le7 to d3/d94/l126 0 2024-04-20T11:09:17.508 INFO:tasks.workunit.client.0.smithi117.stdout:4/787: mknod d1/d45/c10b 0 2024-04-20T11:09:17.510 INFO:tasks.workunit.client.0.smithi117.stdout:0/839: rmdir d3/d7/d20/d47/d78 39 2024-04-20T11:09:17.518 INFO:tasks.workunit.client.0.smithi117.stdout:7/767: mkdir d1/d43/d59/d8a/dd7/df7 0 2024-04-20T11:09:17.518 INFO:tasks.workunit.client.0.smithi117.stdout:9/816: write d4/d5/d3c/ddf/f4e [5473862,54915] 0 2024-04-20T11:09:17.519 INFO:tasks.workunit.client.0.smithi117.stdout:8/693: rename d0/d8/d39/d7f/fbc to d0/d8/d39/d40/d78/d5c/fe6 0 2024-04-20T11:09:17.525 INFO:tasks.workunit.client.0.smithi117.stdout:6/900: readlink d3/d4/d2e/d37/l96 0 2024-04-20T11:09:17.529 INFO:tasks.workunit.client.0.smithi117.stdout:3/766: creat d0/d5/d73/ded/d66/d63/dad/f108 x:0 0 0 2024-04-20T11:09:17.531 INFO:tasks.workunit.client.0.smithi117.stdout:1/689: dwrite d6/d1f/f2b [0,4194304] 0 2024-04-20T11:09:17.536 INFO:tasks.workunit.client.0.smithi117.stdout:2/873: dwrite d1/d4/dd/d17/f1e [4194304,4194304] 0 2024-04-20T11:09:17.537 INFO:tasks.workunit.client.0.smithi117.stdout:0/840: creat d3/d7/d20/d6e/df3/dff/d85/ddb/f109 x:0 0 0 2024-04-20T11:09:17.547 INFO:tasks.workunit.client.0.smithi117.stdout:7/768: rmdir d1/d4/d6 39 2024-04-20T11:09:17.552 INFO:tasks.workunit.client.0.smithi117.stdout:6/901: dwrite d3/d4/d2e/d37/fdb [0,4194304] 0 2024-04-20T11:09:17.552 INFO:tasks.workunit.client.0.smithi117.stdout:4/788: truncate d1/dd/d1a/d43/d4e/fd0 680842 0 2024-04-20T11:09:17.555 INFO:tasks.workunit.client.0.smithi117.stdout:8/694: read d0/d8/d11/f1f [2365970,56187] 0 2024-04-20T11:09:17.557 INFO:tasks.workunit.client.0.smithi117.stdout:1/690: mkdir d6/d16/dce/de0 0 2024-04-20T11:09:17.558 INFO:tasks.workunit.client.0.smithi117.stdout:2/874: mkdir d1/d4/d83/d29/d2e/d52/d11d 0 2024-04-20T11:09:17.562 INFO:tasks.workunit.client.0.smithi117.stdout:9/817: unlink d4/f13 0 2024-04-20T11:09:17.565 INFO:tasks.workunit.client.0.smithi117.stdout:0/841: unlink d3/d7/d2c/d52/l79 0 2024-04-20T11:09:17.571 INFO:tasks.workunit.client.0.smithi117.stdout:4/789: creat d1/dd/d1a/d43/d4e/f10c x:0 0 0 2024-04-20T11:09:17.574 INFO:tasks.workunit.client.0.smithi117.stdout:2/875: rmdir d1/d4/d83/d29/d2e/d53/df0 39 2024-04-20T11:09:17.575 INFO:tasks.workunit.client.0.smithi117.stdout:9/818: mkdir d4/d5/d3c/ddf/d78/d111 0 2024-04-20T11:09:17.577 INFO:tasks.workunit.client.0.smithi117.stdout:0/842: mknod d3/d5/c10a 0 2024-04-20T11:09:17.581 INFO:tasks.workunit.client.0.smithi117.stdout:7/769: creat d1/d4/d6/d54/db4/ff8 x:0 0 0 2024-04-20T11:09:17.583 INFO:tasks.workunit.client.0.smithi117.stdout:8/695: mknod d0/dc/d49/ce7 0 2024-04-20T11:09:17.586 INFO:tasks.workunit.client.0.smithi117.stdout:8/696: readlink d0/dc/d49/la3 0 2024-04-20T11:09:17.591 INFO:tasks.workunit.client.0.smithi117.stdout:3/767: getdents d0/d5/d7 0 2024-04-20T11:09:17.597 INFO:tasks.workunit.client.0.smithi117.stdout:1/691: creat d6/d35/fe1 x:0 0 0 2024-04-20T11:09:17.599 INFO:tasks.workunit.client.0.smithi117.stdout:7/770: symlink d1/d4/d6/d2b/lf9 0 2024-04-20T11:09:17.601 INFO:tasks.workunit.client.0.smithi117.stdout:2/876: mkdir d1/d4/d83/d29/d3f/df6/d119/d11e 0 2024-04-20T11:09:17.603 INFO:tasks.workunit.client.0.smithi117.stdout:6/902: dread d3/d5a/d74/fc3 [0,4194304] 0 2024-04-20T11:09:17.608 INFO:tasks.workunit.client.0.smithi117.stdout:6/903: fdatasync d3/d4/d2e/d37/fdb 0 2024-04-20T11:09:17.611 INFO:tasks.workunit.client.0.smithi117.stdout:2/877: fdatasync d1/d4/d83/d29/d2e/d80/fb9 0 2024-04-20T11:09:17.619 INFO:tasks.workunit.client.0.smithi117.stdout:7/771: creat d1/d4/d3d/dce/deb/ffa x:0 0 0 2024-04-20T11:09:17.619 INFO:tasks.workunit.client.0.smithi117.stdout:9/819: dwrite d4/d4a/d5e/d9b/dcd/fce [0,4194304] 0 2024-04-20T11:09:17.620 INFO:tasks.workunit.client.0.smithi117.stdout:6/904: truncate d3/d4/d2e/fdd 2297691 0 2024-04-20T11:09:17.621 INFO:tasks.workunit.client.0.smithi117.stdout:4/790: dread d1/d9/d1f/d87/fb3 [0,4194304] 0 2024-04-20T11:09:17.626 INFO:tasks.workunit.client.0.smithi117.stdout:2/878: rename d1/d4/fa0 to d1/d20/de0/f11f 0 2024-04-20T11:09:17.627 INFO:tasks.workunit.client.0.smithi117.stdout:6/905: write d3/d4/d2e/d37/fdb [2239900,22462] 0 2024-04-20T11:09:17.636 INFO:tasks.workunit.client.0.smithi117.stdout:7/772: rename d1/d1d/d42/f8b to d1/d4/d6/d17/d3f/ffb 0 2024-04-20T11:09:17.636 INFO:tasks.workunit.client.0.smithi117.stdout:1/692: dwrite d6/d35/fb3 [0,4194304] 0 2024-04-20T11:09:17.639 INFO:tasks.workunit.client.0.smithi117.stdout:0/843: dread d3/d7/d20/d6e/df3/dff/d85/ddb/fc9 [0,4194304] 0 2024-04-20T11:09:17.640 INFO:tasks.workunit.client.0.smithi117.stdout:9/820: creat d4/de/d10/d94/dd8/d10a/f112 x:0 0 0 2024-04-20T11:09:17.645 INFO:tasks.workunit.client.0.smithi117.stdout:2/879: creat d1/d4/d83/d29/d2e/d53/de7/f120 x:0 0 0 2024-04-20T11:09:17.648 INFO:tasks.workunit.client.0.smithi117.stdout:4/791: write d1/d9/f12 [332162,71486] 0 2024-04-20T11:09:17.649 INFO:tasks.workunit.client.0.smithi117.stdout:8/697: dread d0/d8/f41 [0,4194304] 0 2024-04-20T11:09:17.650 INFO:tasks.workunit.client.0.smithi117.stdout:7/773: truncate d1/d4/d6/d2b/d64/f78 601032 0 2024-04-20T11:09:17.651 INFO:tasks.workunit.client.0.smithi117.stdout:9/821: mkdir d4/d4a/d5e/d113 0 2024-04-20T11:09:17.652 INFO:tasks.workunit.client.0.smithi117.stdout:8/698: read - d0/d8/d39/fe2 zero size 2024-04-20T11:09:17.654 INFO:tasks.workunit.client.0.smithi117.stdout:2/880: truncate d1/d4/d83/d29/d3f/f105 119263 0 2024-04-20T11:09:17.660 INFO:tasks.workunit.client.0.smithi117.stdout:7/774: creat d1/d1d/d42/d89/ffc x:0 0 0 2024-04-20T11:09:17.666 INFO:tasks.workunit.client.0.smithi117.stdout:8/699: write d0/d8/d39/f8a [814976,64314] 0 2024-04-20T11:09:17.668 INFO:tasks.workunit.client.0.smithi117.stdout:9/822: mknod d4/d5/d57/d82/c114 0 2024-04-20T11:09:17.669 INFO:tasks.workunit.client.0.smithi117.stdout:4/792: dwrite d1/dd/d1a/d2b/f8f [0,4194304] 0 2024-04-20T11:09:17.673 INFO:tasks.workunit.client.0.smithi117.stdout:7/775: creat d1/d43/d59/ffd x:0 0 0 2024-04-20T11:09:17.682 INFO:tasks.workunit.client.0.smithi117.stdout:0/844: getdents d3/dab 0 2024-04-20T11:09:17.684 INFO:tasks.workunit.client.0.smithi117.stdout:8/700: chown d0/d8/lf 208 1 2024-04-20T11:09:17.686 INFO:tasks.workunit.client.0.smithi117.stdout:4/793: unlink d1/dd/d1a/d43/d9a/cd1 0 2024-04-20T11:09:17.689 INFO:tasks.workunit.client.0.smithi117.stdout:9/823: rename d4/d4a/d63/d98/dc0/dcb/ddc/c10e to d4/d35/df7/d105/d51/c115 0 2024-04-20T11:09:17.690 INFO:tasks.workunit.client.0.smithi117.stdout:6/906: dread d3/d5a/d74/fb6 [0,4194304] 0 2024-04-20T11:09:17.692 INFO:tasks.workunit.client.0.smithi117.stdout:1/693: truncate d6/d1f/d30/d49/d69/f92 805756 0 2024-04-20T11:09:17.693 INFO:tasks.workunit.client.0.smithi117.stdout:0/845: dread - d3/dab/d54/fe0 zero size 2024-04-20T11:09:17.694 INFO:tasks.workunit.client.0.smithi117.stdout:8/701: symlink d0/d8/d11/d24/d27/d99/le8 0 2024-04-20T11:09:17.698 INFO:tasks.workunit.client.0.smithi117.stdout:7/776: rename d1/d1d/d42/d2d/d6c/c6f to d1/d43/d59/cfe 0 2024-04-20T11:09:17.700 INFO:tasks.workunit.client.0.smithi117.stdout:7/777: write d1/d43/d59/ffd [162120,35475] 0 2024-04-20T11:09:17.703 INFO:tasks.workunit.client.0.smithi117.stdout:2/881: link d1/d4/d83/d29/f115 d1/d4/dd/f121 0 2024-04-20T11:09:17.703 INFO:tasks.workunit.client.0.smithi117.stdout:9/824: mkdir d4/d5/d57/d82/d116 0 2024-04-20T11:09:17.704 INFO:tasks.workunit.client.0.smithi117.stdout:1/694: symlink d6/d21/d95/le2 0 2024-04-20T11:09:17.706 INFO:tasks.workunit.client.0.smithi117.stdout:2/882: chown d1/d15/d1d/db3/c107 319403980 1 2024-04-20T11:09:17.706 INFO:tasks.workunit.client.0.smithi117.stdout:4/794: symlink d1/dd/l10d 0 2024-04-20T11:09:17.709 INFO:tasks.workunit.client.0.smithi117.stdout:4/795: write d1/dd/d1a/d2b/d86/ff6 [1013110,103636] 0 2024-04-20T11:09:17.710 INFO:tasks.workunit.client.0.smithi117.stdout:6/907: rename d3/d5a/l69 to d3/d5a/d74/db3/df0/da3/l127 0 2024-04-20T11:09:17.711 INFO:tasks.workunit.client.0.smithi117.stdout:6/908: dread - d3/df/d39/fec zero size 2024-04-20T11:09:17.713 INFO:tasks.workunit.client.0.smithi117.stdout:6/909: write d3/df/f4f [2117158,98776] 0 2024-04-20T11:09:17.714 INFO:tasks.workunit.client.0.smithi117.stdout:1/695: creat d6/d35/d63/fe3 x:0 0 0 2024-04-20T11:09:17.716 INFO:tasks.workunit.client.0.smithi117.stdout:6/910: write d3/df/f63 [4891589,59077] 0 2024-04-20T11:09:17.720 INFO:tasks.workunit.client.0.smithi117.stdout:6/911: read d3/df/d39/daa/d100/f110 [1373052,33742] 0 2024-04-20T11:09:17.727 INFO:tasks.workunit.client.0.smithi117.stdout:4/796: chown d1/d9/c84 154034793 1 2024-04-20T11:09:17.729 INFO:tasks.workunit.client.0.smithi117.stdout:2/883: rename d1/d4/d83/d29/d62/d109/f112 to d1/d4/d83/d29/d62/da6/f122 0 2024-04-20T11:09:17.737 INFO:tasks.workunit.client.0.smithi117.stdout:1/696: mkdir d6/d16/d3a/d5a/daf/de4 0 2024-04-20T11:09:17.741 INFO:tasks.workunit.client.0.smithi117.stdout:4/797: dwrite d1/dd/d1a/d2b/d86/fe5 [0,4194304] 0 2024-04-20T11:09:17.743 INFO:tasks.workunit.client.0.smithi117.stdout:6/912: mknod d3/d4/d2e/d3f/c128 0 2024-04-20T11:09:17.745 INFO:tasks.workunit.client.0.smithi117.stdout:8/702: link d0/d8/d11/d24/d2b/d72/l93 d0/d8/d39/d40/d78/d5c/le9 0 2024-04-20T11:09:17.752 INFO:tasks.workunit.client.0.smithi117.stdout:2/884: symlink d1/d4/d83/d29/daa/l123 0 2024-04-20T11:09:17.757 INFO:tasks.workunit.client.0.smithi117.stdout:0/846: write d3/d7/d20/d39/d9a/dd0/f1b [2003530,50003] 0 2024-04-20T11:09:17.758 INFO:tasks.workunit.client.0.smithi117.stdout:9/825: link d4/d35/df7/d105/f93 d4/d5/da1/f117 0 2024-04-20T11:09:17.762 INFO:tasks.workunit.client.0.smithi117.stdout:6/913: unlink d3/df/f63 0 2024-04-20T11:09:17.764 INFO:tasks.workunit.client.0.smithi117.stdout:4/798: truncate d1/dd/d1a/d43/d9a/fb9 1409470 0 2024-04-20T11:09:17.766 INFO:tasks.workunit.client.0.smithi117.stdout:8/703: symlink d0/dc/d49/lea 0 2024-04-20T11:09:17.770 INFO:tasks.workunit.client.0.smithi117.stdout:2/885: chown d1/d15/d1d/c2f 181026 1 2024-04-20T11:09:17.772 INFO:tasks.workunit.client.0.smithi117.stdout:1/697: creat d6/d1f/dd6/fe5 x:0 0 0 2024-04-20T11:09:17.775 INFO:tasks.workunit.client.0.smithi117.stdout:0/847: readlink d3/d7/d20/d39/d9a/dd0/db2/ld9 0 2024-04-20T11:09:17.776 INFO:tasks.workunit.client.0.smithi117.stdout:9/826: fsync d4/d4a/d5e/f75 0 2024-04-20T11:09:17.781 INFO:tasks.workunit.client.0.smithi117.stdout:6/914: truncate d3/df/f21 4445602 0 2024-04-20T11:09:17.785 INFO:tasks.workunit.client.0.smithi117.stdout:8/704: symlink d0/dc/leb 0 2024-04-20T11:09:17.794 INFO:tasks.workunit.client.0.smithi117.stdout:9/827: unlink d4/d5/d57/fd4 0 2024-04-20T11:09:17.795 INFO:tasks.workunit.client.0.smithi117.stdout:6/915: creat d3/d4/d2e/d37/d8e/db2/f129 x:0 0 0 2024-04-20T11:09:17.798 INFO:tasks.workunit.client.0.smithi117.stdout:6/916: truncate d3/d94/ffa 111678 0 2024-04-20T11:09:17.808 INFO:tasks.workunit.client.0.smithi117.stdout:7/778: dwrite d1/d4/d6/d2b/d64/f78 [0,4194304] 0 2024-04-20T11:09:17.812 INFO:tasks.workunit.client.0.smithi117.stdout:6/917: symlink d3/d4/d2e/d37/d8e/db2/d10f/d116/l12a 0 2024-04-20T11:09:17.815 INFO:tasks.workunit.client.0.smithi117.stdout:8/705: getdents d0/d8/d11/d24/d2b/d72 0 2024-04-20T11:09:17.817 INFO:tasks.workunit.client.0.smithi117.stdout:4/799: write d1/dd/d1a/d2b/d86/f91 [3881952,125705] 0 2024-04-20T11:09:17.817 INFO:tasks.workunit.client.0.smithi117.stdout:7/779: mknod d1/d4/d6/d2b/d64/cff 0 2024-04-20T11:09:17.823 INFO:tasks.workunit.client.0.smithi117.stdout:8/706: fdatasync d0/d8/d39/d40/d78/d5c/fe6 0 2024-04-20T11:09:17.823 INFO:tasks.workunit.client.0.smithi117.stdout:2/886: write d1/d4/dd/d17/f116 [551438,123769] 0 2024-04-20T11:09:17.826 INFO:tasks.workunit.client.0.smithi117.stdout:1/698: write d6/d16/f62 [1042540,125857] 0 2024-04-20T11:09:17.826 INFO:tasks.workunit.client.0.smithi117.stdout:8/707: stat d0/d8/d39/d40/la6 0 2024-04-20T11:09:17.830 INFO:tasks.workunit.client.0.smithi117.stdout:8/708: chown d0/d8/d39/fc9 5 1 2024-04-20T11:09:17.846 INFO:tasks.workunit.client.0.smithi117.stdout:7/780: dwrite d1/fea [0,4194304] 0 2024-04-20T11:09:17.846 INFO:tasks.workunit.client.0.smithi117.stdout:6/918: dwrite d3/d4/d2e/d37/d8e/fd2 [0,4194304] 0 2024-04-20T11:09:17.846 INFO:tasks.workunit.client.0.smithi117.stdout:1/699: getdents d6/d16/d3a 0 2024-04-20T11:09:17.846 INFO:tasks.workunit.client.0.smithi117.stdout:8/709: rename d0/d8/f41 to d0/d9/d3a/d7c/dce/fec 0 2024-04-20T11:09:17.846 INFO:tasks.workunit.client.0.smithi117.stdout:7/781: dread - d1/d43/fbf zero size 2024-04-20T11:09:17.847 INFO:tasks.workunit.client.0.smithi117.stdout:9/828: dwrite d4/d4a/d5e/d109/f85 [4194304,4194304] 0 2024-04-20T11:09:17.848 INFO:tasks.workunit.client.0.smithi117.stdout:9/829: chown d4/d5 174 1 2024-04-20T11:09:17.849 INFO:tasks.workunit.client.0.smithi117.stdout:1/700: write d6/d1f/f45 [2419922,82063] 0 2024-04-20T11:09:17.849 INFO:tasks.workunit.client.0.smithi117.stdout:7/782: readlink d1/d1d/d42/laa 0 2024-04-20T11:09:17.851 INFO:tasks.workunit.client.0.smithi117.stdout:2/887: write d1/da5/df2/f56 [2071565,130041] 0 2024-04-20T11:09:17.857 INFO:tasks.workunit.client.0.smithi117.stdout:2/888: write d1/d4/d83/d29/d2e/d53/f103 [1927303,130910] 0 2024-04-20T11:09:17.861 INFO:tasks.workunit.client.0.smithi117.stdout:7/783: creat d1/d1d/ded/f100 x:0 0 0 2024-04-20T11:09:17.863 INFO:tasks.workunit.client.0.smithi117.stdout:1/701: truncate d6/d1f/d30/d49/d64/f8f 1035413 0 2024-04-20T11:09:17.864 INFO:tasks.workunit.client.0.smithi117.stdout:6/919: truncate d3/d4/d2e/d37/d8e/f9d 564224 0 2024-04-20T11:09:17.866 INFO:tasks.workunit.client.0.smithi117.stdout:1/702: write d6/d1f/d30/d32/fdd [1016294,121331] 0 2024-04-20T11:09:17.868 INFO:tasks.workunit.client.0.smithi117.stdout:1/703: chown d6/c5b 0 1 2024-04-20T11:09:17.868 INFO:tasks.workunit.client.0.smithi117.stdout:8/710: truncate d0/d9/dbb/fc8 3853173 0 2024-04-20T11:09:17.869 INFO:tasks.workunit.client.0.smithi117.stdout:6/920: chown d3/df/d39/d99 14240 1 2024-04-20T11:09:17.872 INFO:tasks.workunit.client.0.smithi117.stdout:8/711: mknod d0/d9/d3a/d7c/dce/ced 0 2024-04-20T11:09:17.875 INFO:tasks.workunit.client.0.smithi117.stdout:1/704: rename d6/d1f/d30/d49/d69/cde to d6/d35/d63/ce6 0 2024-04-20T11:09:17.876 INFO:tasks.workunit.client.0.smithi117.stdout:8/712: chown d0/d8/d11/d6d/c7a 424358 1 2024-04-20T11:09:17.883 INFO:tasks.workunit.client.0.smithi117.stdout:8/713: fsync d0/d8/d11/d24/d2b/d72/f75 0 2024-04-20T11:09:17.884 INFO:tasks.workunit.client.0.smithi117.stdout:6/921: write d3/d5a/d74/db3/df0/da3/fad [1688620,14497] 0 2024-04-20T11:09:17.886 INFO:tasks.workunit.client.0.smithi117.stdout:1/705: getdents d6/d21 0 2024-04-20T11:09:17.889 INFO:tasks.workunit.client.0.smithi117.stdout:6/922: readlink d3/df/d39/daa/lf4 0 2024-04-20T11:09:17.893 INFO:tasks.workunit.client.0.smithi117.stdout:8/714: rename d0/cb3 to d0/dc/cee 0 2024-04-20T11:09:17.894 INFO:tasks.workunit.client.0.smithi117.stdout:1/706: mknod d6/d16/d3a/d5a/ce7 0 2024-04-20T11:09:17.896 INFO:tasks.workunit.client.0.smithi117.stdout:1/707: readlink d6/d1f/lbf 0 2024-04-20T11:09:17.898 INFO:tasks.workunit.client.0.smithi117.stdout:1/708: truncate d6/d21/fd1 961956 0 2024-04-20T11:09:17.900 INFO:tasks.workunit.client.0.smithi117.stdout:9/830: dread d4/d35/df7/d105/f6f [0,4194304] 0 2024-04-20T11:09:17.906 INFO:tasks.workunit.client.0.smithi117.stdout:1/709: creat d6/d16/d3a/d5a/daf/fe8 x:0 0 0 2024-04-20T11:09:17.907 INFO:tasks.workunit.client.0.smithi117.stdout:6/923: write d3/d4/d2e/d3f/f8b [658830,34438] 0 2024-04-20T11:09:17.907 INFO:tasks.workunit.client.0.smithi117.stdout:9/831: fsync d4/d5/d3c/f47 0 2024-04-20T11:09:17.911 INFO:tasks.workunit.client.0.smithi117.stdout:9/832: read d4/d35/df7/d105/f49 [917623,85671] 0 2024-04-20T11:09:17.913 INFO:tasks.workunit.client.0.smithi117.stdout:9/833: chown d4/d5/d3c/ddf/d31/c91 1 1 2024-04-20T11:09:17.914 INFO:tasks.workunit.client.0.smithi117.stdout:9/834: write d4/d35/df7/d105/f49 [3063391,58981] 0 2024-04-20T11:09:17.934 INFO:tasks.workunit.client.0.smithi117.stdout:1/710: dread d6/d35/f36 [0,4194304] 0 2024-04-20T11:09:17.949 INFO:tasks.workunit.client.0.smithi117.stdout:6/924: creat d3/d3a/f12b x:0 0 0 2024-04-20T11:09:17.951 INFO:tasks.workunit.client.0.smithi117.stdout:6/925: write d3/f11a [253969,126523] 0 2024-04-20T11:09:17.953 INFO:tasks.workunit.client.0.smithi117.stdout:9/835: rename d4/de/d10/f3a to d4/d4a/d63/d98/dc0/dcb/ddc/f118 0 2024-04-20T11:09:17.954 INFO:tasks.workunit.client.0.smithi117.stdout:1/711: mkdir d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/de9 0 2024-04-20T11:09:17.955 INFO:tasks.workunit.client.0.smithi117.stdout:1/712: readlink d6/d1f/l47 0 2024-04-20T11:09:17.956 INFO:tasks.workunit.client.0.smithi117.stdout:8/715: creat d0/d8/d39/d40/de5/fef x:0 0 0 2024-04-20T11:09:17.958 INFO:tasks.workunit.client.0.smithi117.stdout:8/716: fdatasync d0/d8/d39/d40/de5/fef 0 2024-04-20T11:09:17.960 INFO:tasks.workunit.client.0.smithi117.stdout:6/926: mkdir d3/d4/d12c 0 2024-04-20T11:09:17.964 INFO:tasks.workunit.client.0.smithi117.stdout:8/717: rename d0 to d0/d8/d11/d24/df0 22 2024-04-20T11:09:17.966 INFO:tasks.workunit.client.0.smithi117.stdout:6/927: creat d3/d5a/d74/db3/df0/db7/f12d x:0 0 0 2024-04-20T11:09:17.980 INFO:tasks.workunit.client.0.smithi117.stdout:8/718: dwrite d0/d8/d11/d6d/d58/f71 [0,4194304] 0 2024-04-20T11:09:17.984 INFO:tasks.workunit.client.0.smithi117.stdout:8/719: chown d0/d8/d11/d24/d53/l88 192 1 2024-04-20T11:09:17.989 INFO:tasks.workunit.client.0.smithi117.stdout:9/836: write d4/d35/f3d [536349,43439] 0 2024-04-20T11:09:17.992 INFO:tasks.workunit.client.0.smithi117.stdout:6/928: write d3/df/d39/f3e [555096,59089] 0 2024-04-20T11:09:17.996 INFO:tasks.workunit.client.0.smithi117.stdout:9/837: creat d4/d5/f119 x:0 0 0 2024-04-20T11:09:18.001 INFO:tasks.workunit.client.0.smithi117.stdout:1/713: dwrite d6/fd4 [0,4194304] 0 2024-04-20T11:09:18.001 INFO:tasks.workunit.client.0.smithi117.stdout:6/929: dread d3/d4/fb4 [0,4194304] 0 2024-04-20T11:09:18.007 INFO:tasks.workunit.client.0.smithi117.stdout:8/720: dwrite d0/f57 [0,4194304] 0 2024-04-20T11:09:18.011 INFO:tasks.workunit.client.0.smithi117.stdout:6/930: rename d3/d4/d2e/dc0/ceb to d3/d5a/d74/db3/df0/db7/d105/c12e 0 2024-04-20T11:09:18.017 INFO:tasks.workunit.client.0.smithi117.stdout:6/931: chown d3/d5a/d74/db3/df0/fcf 22167578 1 2024-04-20T11:09:18.021 INFO:tasks.workunit.client.0.smithi117.stdout:8/721: getdents d0/d8/d39/d40/de5 0 2024-04-20T11:09:18.028 INFO:tasks.workunit.client.0.smithi117.stdout:1/714: dwrite d6/f2e [0,4194304] 0 2024-04-20T11:09:18.030 INFO:tasks.workunit.client.0.smithi117.stdout:1/715: chown d6/d1f/d30/f67 242 1 2024-04-20T11:09:18.032 INFO:tasks.workunit.client.0.smithi117.stdout:6/932: rename d3/d5a/d74/db3/df0/db7 to d3/df/d39/d12f 0 2024-04-20T11:09:18.098 INFO:tasks.workunit.client.0.smithi117.stdout:6/933: symlink d3/d4/d12c/l130 0 2024-04-20T11:09:18.098 INFO:tasks.workunit.client.0.smithi117.stdout:8/722: write d0/d8/d11/d24/d2b/f5d [1741620,10109] 0 2024-04-20T11:09:18.098 INFO:tasks.workunit.client.0.smithi117.stdout:1/716: write d6/d16/d3a/f7e [915126,57383] 0 2024-04-20T11:09:18.102 INFO:tasks.workunit.client.0.smithi117.stdout:1/717: mkdir d6/d16/d3a/d5a/dea 0 2024-04-20T11:09:18.104 INFO:tasks.workunit.client.0.smithi117.stdout:8/723: unlink d0/d8/d39/d40/f70 0 2024-04-20T11:09:18.105 INFO:tasks.workunit.client.0.smithi117.stdout:1/718: truncate d6/d16/fbb 101799 0 2024-04-20T11:09:18.116 INFO:tasks.workunit.client.0.smithi117.stdout:9/838: dread d4/d5/d3c/ddf/d31/fac [0,4194304] 0 2024-04-20T11:09:18.117 INFO:tasks.workunit.client.0.smithi117.stdout:6/934: dwrite d3/d4/d2e/d3f/f77 [0,4194304] 0 2024-04-20T11:09:18.117 INFO:tasks.workunit.client.0.smithi117.stdout:8/724: dread d0/f57 [0,4194304] 0 2024-04-20T11:09:18.126 INFO:tasks.workunit.client.0.smithi117.stdout:8/725: rename d0/d8/d11/d24/d2b/ddd to d0/d9/d3a/de3/df1 0 2024-04-20T11:09:18.130 INFO:tasks.workunit.client.0.smithi117.stdout:8/726: symlink d0/dc/d2e/d89/lf2 0 2024-04-20T11:09:18.131 INFO:tasks.workunit.client.0.smithi117.stdout:9/839: write d4/fbc [972212,126727] 0 2024-04-20T11:09:18.131 INFO:tasks.workunit.client.0.smithi117.stdout:6/935: write d3/df/d39/d12f/f87 [4729923,39896] 0 2024-04-20T11:09:18.133 INFO:tasks.workunit.client.0.smithi117.stdout:9/840: stat d4/de/d7b/d96/db0/lda 0 2024-04-20T11:09:18.136 INFO:tasks.workunit.client.0.smithi117.stdout:6/936: unlink d3/d4/d2e/d37/d8e/fd2 0 2024-04-20T11:09:18.137 INFO:tasks.workunit.client.0.smithi117.stdout:9/841: mknod d4/d68/c11a 0 2024-04-20T11:09:18.138 INFO:tasks.workunit.client.0.smithi117.stdout:8/727: truncate d0/d9/dbb/fc8 2456859 0 2024-04-20T11:09:18.138 INFO:tasks.workunit.client.0.smithi117.stdout:6/937: truncate d3/d94/ffe 499110 0 2024-04-20T11:09:18.140 INFO:tasks.workunit.client.0.smithi117.stdout:9/842: creat d4/d5/f11b x:0 0 0 2024-04-20T11:09:18.142 INFO:tasks.workunit.client.0.smithi117.stdout:6/938: readlink d3/d4/d2e/d37/d8e/db2/l10a 0 2024-04-20T11:09:18.145 INFO:tasks.workunit.client.0.smithi117.stdout:6/939: symlink d3/d4/d2e/dc0/df7/l131 0 2024-04-20T11:09:18.146 INFO:tasks.workunit.client.0.smithi117.stdout:8/728: getdents d0/dc/d49/d5b 0 2024-04-20T11:09:18.148 INFO:tasks.workunit.client.0.smithi117.stdout:8/729: fdatasync d0/d9/d3a/d7c/dce/fec 0 2024-04-20T11:09:18.151 INFO:tasks.workunit.client.0.smithi117.stdout:8/730: write d0/d8/d39/d40/d78/d5c/fd1 [194991,81457] 0 2024-04-20T11:09:18.152 INFO:tasks.workunit.client.0.smithi117.stdout:9/843: write d4/d35/df7/d105/f49 [3735286,23735] 0 2024-04-20T11:09:18.154 INFO:tasks.workunit.client.0.smithi117.stdout:8/731: chown d0/d8/d39/l52 30560971 1 2024-04-20T11:09:18.159 INFO:tasks.workunit.client.0.smithi117.stdout:9/844: link d4/de/d7b/dd0/ce3 d4/de/db1/c11c 0 2024-04-20T11:09:18.161 INFO:tasks.workunit.client.0.smithi117.stdout:8/732: rmdir d0/d8/d39/d40/d78/dda 0 2024-04-20T11:09:18.164 INFO:tasks.workunit.client.0.smithi117.stdout:8/733: rmdir d0/d8/d39/d7f 39 2024-04-20T11:09:18.166 INFO:tasks.workunit.client.0.smithi117.stdout:9/845: getdents d4/de/d7b/dd0 0 2024-04-20T11:09:18.168 INFO:tasks.workunit.client.0.smithi117.stdout:8/734: mkdir d0/d8/d11/df3 0 2024-04-20T11:09:18.170 INFO:tasks.workunit.client.0.smithi117.stdout:8/735: chown d0/d8/d11/d24/d53/l88 2577207 1 2024-04-20T11:09:18.183 INFO:tasks.workunit.client.0.smithi117.stdout:9/846: dread d4/de/d10/f5c [0,4194304] 0 2024-04-20T11:09:18.184 INFO:tasks.workunit.client.0.smithi117.stdout:9/847: chown d4/d35/d54 67513666 1 2024-04-20T11:09:18.192 INFO:tasks.workunit.client.0.smithi117.stdout:9/848: write d4/d5/d3c/f7d [60368,50099] 0 2024-04-20T11:09:18.192 INFO:tasks.workunit.client.0.smithi117.stdout:8/736: dread d0/d9/dbb/fc8 [0,4194304] 0 2024-04-20T11:09:18.193 INFO:tasks.workunit.client.0.smithi117.stdout:8/737: fsync d0/d9/d3a/f8b 0 2024-04-20T11:09:18.194 INFO:tasks.workunit.client.0.smithi117.stdout:9/849: creat d4/d5/d57/db4/f11d x:0 0 0 2024-04-20T11:09:18.202 INFO:tasks.workunit.client.0.smithi117.stdout:9/850: link d4/d35/df7/d105/l5a d4/d35/df7/d105/ddb/l11e 0 2024-04-20T11:09:18.208 INFO:tasks.workunit.client.0.smithi117.stdout:8/738: truncate d0/d8/d11/d24/d2b/fab 489989 0 2024-04-20T11:09:18.211 INFO:tasks.workunit.client.0.smithi117.stdout:8/739: mknod d0/d8/d11/d6d/d58/cf4 0 2024-04-20T11:09:18.216 INFO:tasks.workunit.client.0.smithi117.stdout:9/851: dwrite d4/de/f84 [0,4194304] 0 2024-04-20T11:09:18.221 INFO:tasks.workunit.client.0.smithi117.stdout:9/852: creat d4/d35/df7/d105/d51/f11f x:0 0 0 2024-04-20T11:09:18.225 INFO:tasks.workunit.client.0.smithi117.stdout:9/853: creat d4/d4a/d5e/de8/f120 x:0 0 0 2024-04-20T11:09:18.225 INFO:tasks.workunit.client.0.smithi117.stdout:8/740: dwrite d0/d8/d39/d40/d78/d5c/fd1 [0,4194304] 0 2024-04-20T11:09:18.229 INFO:tasks.workunit.client.0.smithi117.stdout:8/741: creat d0/d8/d11/d24/d27/ff5 x:0 0 0 2024-04-20T11:09:18.231 INFO:tasks.workunit.client.0.smithi117.stdout:8/742: readlink d0/l50 0 2024-04-20T11:09:18.245 INFO:tasks.workunit.client.0.smithi117.stdout:9/854: dwrite d4/d5/f2a [0,4194304] 0 2024-04-20T11:09:18.331 INFO:tasks.workunit.client.0.smithi117.stdout:8/743: dwrite d0/d8/d11/d24/d53/f7b [0,4194304] 0 2024-04-20T11:09:18.336 INFO:tasks.workunit.client.0.smithi117.stdout:8/744: dread - d0/d8/d11/d6d/d84/fc3 zero size 2024-04-20T11:09:18.341 INFO:tasks.workunit.client.0.smithi117.stdout:9/855: link d4/f39 d4/d4a/d63/d98/f121 0 2024-04-20T11:09:18.344 INFO:tasks.workunit.client.0.smithi117.stdout:8/745: unlink d0/d9/dbb/fc8 0 2024-04-20T11:09:18.348 INFO:tasks.workunit.client.0.smithi117.stdout:9/856: mkdir d4/d4a/d5e/d9b/d122 0 2024-04-20T11:09:18.353 INFO:tasks.workunit.client.0.smithi117.stdout:9/857: write d4/de/d10/d94/dd8/d10a/f112 [556919,87622] 0 2024-04-20T11:09:18.353 INFO:tasks.workunit.client.0.smithi117.stdout:8/746: mknod d0/d8/d11/d6d/d69/d9c/cf6 0 2024-04-20T11:09:18.430 INFO:tasks.workunit.client.0.smithi117.stdout:3/768: sync 2024-04-20T11:09:18.442 INFO:tasks.workunit.client.0.smithi117.stdout:8/747: creat d0/dc/d49/d63/ff7 x:0 0 0 2024-04-20T11:09:18.465 INFO:tasks.workunit.client.0.smithi117.stdout:3/769: dwrite d0/d5/fc5 [0,4194304] 0 2024-04-20T11:09:18.471 INFO:tasks.workunit.client.0.smithi117.stdout:3/770: mkdir d0/d5/d21/d94/da7/d109 0 2024-04-20T11:09:18.480 INFO:tasks.workunit.client.0.smithi117.stdout:8/748: dread d0/d8/d39/d40/d78/d5c/fe6 [0,4194304] 0 2024-04-20T11:09:18.483 INFO:tasks.workunit.client.0.smithi117.stdout:3/771: dread d0/d5/d21/f47 [0,4194304] 0 2024-04-20T11:09:18.487 INFO:tasks.workunit.client.0.smithi117.stdout:9/858: dread d4/f39 [0,4194304] 0 2024-04-20T11:09:18.496 INFO:tasks.workunit.client.0.smithi117.stdout:3/772: rmdir d0/d5/d73/ded/d66/dbe/dd0 0 2024-04-20T11:09:18.502 INFO:tasks.workunit.client.0.smithi117.stdout:3/773: write d0/d5/d73/ded/d66/d63/dad/f108 [1011186,64515] 0 2024-04-20T11:09:18.503 INFO:tasks.workunit.client.0.smithi117.stdout:9/859: dread d4/d35/df7/d105/fc [0,4194304] 0 2024-04-20T11:09:18.513 INFO:tasks.workunit.client.0.smithi117.stdout:9/860: truncate d4/de/d7b/fbd 234944 0 2024-04-20T11:09:18.522 INFO:tasks.workunit.client.0.smithi117.stdout:9/861: readlink d4/lf6 0 2024-04-20T11:09:18.535 INFO:tasks.workunit.client.0.smithi117.stdout:0/848: sync 2024-04-20T11:09:18.546 INFO:tasks.workunit.client.0.smithi117.stdout:9/862: dwrite d4/d4a/d5e/de8/f58 [0,4194304] 0 2024-04-20T11:09:18.550 INFO:tasks.workunit.client.0.smithi117.stdout:0/849: dread d3/d7/d20/d39/d3a/db1/fdf [0,4194304] 0 2024-04-20T11:09:18.556 INFO:tasks.workunit.client.0.smithi117.stdout:0/850: creat d3/d7/d20/d6e/df3/dff/d85/f10b x:0 0 0 2024-04-20T11:09:18.562 INFO:tasks.workunit.client.0.smithi117.stdout:0/851: unlink d3/d7/d20/d39/d9a/dd0/d6d/f9c 0 2024-04-20T11:09:18.568 INFO:tasks.workunit.client.0.smithi117.stdout:0/852: mkdir d3/d7/d20/d39/d9a/dd0/d6d/d10c 0 2024-04-20T11:09:18.581 INFO:tasks.workunit.client.0.smithi117.stdout:9/863: dread d4/d5/f1e [4194304,4194304] 0 2024-04-20T11:09:18.581 INFO:tasks.workunit.client.0.smithi117.stdout:0/853: rename d3/d7/d20/d39/d9a/dd0/d6d/f9b to d3/d7/d20/d47/f10d 0 2024-04-20T11:09:18.587 INFO:tasks.workunit.client.0.smithi117.stdout:9/864: getdents d4/de/db1/df9 0 2024-04-20T11:09:18.589 INFO:tasks.workunit.client.0.smithi117.stdout:0/854: creat d3/d7/d2c/f10e x:0 0 0 2024-04-20T11:09:18.593 INFO:tasks.workunit.client.0.smithi117.stdout:8/749: dwrite d0/d8/d11/d24/d27/d47/f7d [0,4194304] 0 2024-04-20T11:09:18.598 INFO:tasks.workunit.client.0.smithi117.stdout:9/865: getdents d4/d4a/d63 0 2024-04-20T11:09:18.602 INFO:tasks.workunit.client.0.smithi117.stdout:8/750: creat d0/d9/ff8 x:0 0 0 2024-04-20T11:09:18.605 INFO:tasks.workunit.client.0.smithi117.stdout:9/866: creat d4/d5/d57/d82/da5/f123 x:0 0 0 2024-04-20T11:09:18.609 INFO:tasks.workunit.client.0.smithi117.stdout:9/867: readlink d4/d5/d3c/ddf/lc4 0 2024-04-20T11:09:18.613 INFO:tasks.workunit.client.0.smithi117.stdout:8/751: getdents d0/d8/d11/d24/d2b/d72 0 2024-04-20T11:09:18.616 INFO:tasks.workunit.client.0.smithi117.stdout:3/774: dwrite d0/d5/de/fd4 [0,4194304] 0 2024-04-20T11:09:18.617 INFO:tasks.workunit.client.0.smithi117.stdout:3/775: chown d0/d5/d21/d88/d36 1 1 2024-04-20T11:09:18.618 INFO:tasks.workunit.client.0.smithi117.stdout:9/868: mkdir d4/d4a/d72/dd2/d106/d124 0 2024-04-20T11:09:18.620 INFO:tasks.workunit.client.0.smithi117.stdout:4/800: sync 2024-04-20T11:09:18.620 INFO:tasks.workunit.client.0.smithi117.stdout:2/889: sync 2024-04-20T11:09:18.620 INFO:tasks.workunit.client.0.smithi117.stdout:7/784: sync 2024-04-20T11:09:18.620 INFO:tasks.workunit.client.0.smithi117.stdout:6/940: sync 2024-04-20T11:09:18.620 INFO:tasks.workunit.client.0.smithi117.stdout:1/719: sync 2024-04-20T11:09:18.636 INFO:tasks.workunit.client.0.smithi117.stdout:9/869: dread d4/d4a/d63/d98/dc0/dcb/ddc/f118 [0,4194304] 0 2024-04-20T11:09:18.644 INFO:tasks.workunit.client.0.smithi117.stdout:8/752: dwrite d0/d9/f17 [0,4194304] 0 2024-04-20T11:09:18.645 INFO:tasks.workunit.client.0.smithi117.stdout:1/720: mkdir d6/d16/d3a/d5a/d5c/d61/d98/deb 0 2024-04-20T11:09:18.649 INFO:tasks.workunit.client.0.smithi117.stdout:2/890: truncate d1/d4/fec 875745 0 2024-04-20T11:09:18.650 INFO:tasks.workunit.client.0.smithi117.stdout:1/721: chown d6/d16/d3a/d5a/c65 3775806 1 2024-04-20T11:09:18.664 INFO:tasks.workunit.client.0.smithi117.stdout:4/801: dread d1/dd/d1a/f7c [0,4194304] 0 2024-04-20T11:09:18.667 INFO:tasks.workunit.client.0.smithi117.stdout:9/870: dread d4/d5/da1/f117 [0,4194304] 0 2024-04-20T11:09:18.667 INFO:tasks.workunit.client.0.smithi117.stdout:6/941: dread d3/d4/f46 [0,4194304] 0 2024-04-20T11:09:18.667 INFO:tasks.workunit.client.0.smithi117.stdout:4/802: chown d1/d9/f2a 751 1 2024-04-20T11:09:18.673 INFO:tasks.workunit.client.0.smithi117.stdout:1/722: dwrite d6/d1f/f2b [0,4194304] 0 2024-04-20T11:09:18.677 INFO:tasks.workunit.client.0.smithi117.stdout:9/871: mknod d4/d4a/d63/d98/dc0/c125 0 2024-04-20T11:09:18.680 INFO:tasks.workunit.client.0.smithi117.stdout:6/942: mkdir d3/d5a/d74/d132 0 2024-04-20T11:09:18.682 INFO:tasks.workunit.client.0.smithi117.stdout:9/872: truncate d4/d5/f119 843349 0 2024-04-20T11:09:18.682 INFO:tasks.workunit.client.0.smithi117.stdout:4/803: rename d1/d9/d1f/d90/lbf to d1/d9/d1f/d87/l10e 0 2024-04-20T11:09:18.684 INFO:tasks.workunit.client.0.smithi117.stdout:1/723: rmdir d6/d21/db9 39 2024-04-20T11:09:18.686 INFO:tasks.workunit.client.0.smithi117.stdout:6/943: write d3/d4/d2e/d37/fdb [2169805,119707] 0 2024-04-20T11:09:18.690 INFO:tasks.workunit.client.0.smithi117.stdout:8/753: dread d0/dc/f25 [0,4194304] 0 2024-04-20T11:09:18.690 INFO:tasks.workunit.client.0.smithi117.stdout:0/855: dwrite d3/d7/d2c/d52/f70 [0,4194304] 0 2024-04-20T11:09:18.695 INFO:tasks.workunit.client.0.smithi117.stdout:2/891: dwrite d1/d9b/fb6 [4194304,4194304] 0 2024-04-20T11:09:18.695 INFO:tasks.workunit.client.0.smithi117.stdout:4/804: unlink d1/d9/d79/c7d 0 2024-04-20T11:09:18.712 INFO:tasks.workunit.client.0.smithi117.stdout:3/776: dwrite d0/d5/d7/dd/f3a [0,4194304] 0 2024-04-20T11:09:18.716 INFO:tasks.workunit.client.0.smithi117.stdout:2/892: truncate d1/d4/dd/d17/d3b/f6c 2268863 0 2024-04-20T11:09:18.719 INFO:tasks.workunit.client.0.smithi117.stdout:7/785: dwrite d1/d43/d59/fa4 [0,4194304] 0 2024-04-20T11:09:18.720 INFO:tasks.workunit.client.0.smithi117.stdout:4/805: symlink d1/dd/d1a/d2b/d86/l10f 0 2024-04-20T11:09:18.725 INFO:tasks.workunit.client.0.smithi117.stdout:2/893: write d1/d4/dd/d17/f1e [8868204,43309] 0 2024-04-20T11:09:18.725 INFO:tasks.workunit.client.0.smithi117.stdout:3/777: truncate d0/d5/d11/f101 791804 0 2024-04-20T11:09:18.728 INFO:tasks.workunit.client.0.smithi117.stdout:9/873: dwrite d4/d5/f1e [0,4194304] 0 2024-04-20T11:09:18.734 INFO:tasks.workunit.client.0.smithi117.stdout:0/856: rename d3/d7e/ce9 to d3/dab/c10f 0 2024-04-20T11:09:18.737 INFO:tasks.workunit.client.0.smithi117.stdout:6/944: dwrite d3/d3a/f4d [0,4194304] 0 2024-04-20T11:09:18.743 INFO:tasks.workunit.client.0.smithi117.stdout:6/945: stat d3/df/f4f 0 2024-04-20T11:09:18.743 INFO:tasks.workunit.client.0.smithi117.stdout:7/786: mkdir d1/d43/d59/d8a/d101 0 2024-04-20T11:09:18.747 INFO:tasks.workunit.client.0.smithi117.stdout:4/806: mkdir d1/d45/d110 0 2024-04-20T11:09:18.751 INFO:tasks.workunit.client.0.smithi117.stdout:1/724: dread d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/fd0 [0,4194304] 0 2024-04-20T11:09:18.757 INFO:tasks.workunit.client.0.smithi117.stdout:2/894: read d1/f11 [4001876,120861] 0 2024-04-20T11:09:18.757 INFO:tasks.workunit.client.0.smithi117.stdout:8/754: creat d0/d8/d11/ff9 x:0 0 0 2024-04-20T11:09:18.776 INFO:tasks.workunit.client.0.smithi117.stdout:7/787: symlink d1/d4/d6/d54/db4/l102 0 2024-04-20T11:09:18.780 INFO:tasks.workunit.client.0.smithi117.stdout:1/725: creat d6/d16/d3a/d5a/fec x:0 0 0 2024-04-20T11:09:18.781 INFO:tasks.workunit.client.0.smithi117.stdout:1/726: chown d6/d16/d3a/da8 0 1 2024-04-20T11:09:18.784 INFO:tasks.workunit.client.0.smithi117.stdout:1/727: fdatasync d6/d16/d3a/d5a/daf/fe8 0 2024-04-20T11:09:18.789 INFO:tasks.workunit.client.0.smithi117.stdout:2/895: creat d1/d4/dd/d17/d7a/d88/f124 x:0 0 0 2024-04-20T11:09:18.792 INFO:tasks.workunit.client.0.smithi117.stdout:2/896: read - d1/da5/df2/d46/fcd zero size 2024-04-20T11:09:18.795 INFO:tasks.workunit.client.0.smithi117.stdout:6/946: dread d3/d4/d2e/d37/f47 [0,4194304] 0 2024-04-20T11:09:18.796 INFO:tasks.workunit.client.0.smithi117.stdout:4/807: dwrite d1/d9/d1f/d87/fb3 [0,4194304] 0 2024-04-20T11:09:18.807 INFO:tasks.workunit.client.0.smithi117.stdout:3/778: link d0/d5/de/f92 d0/d5/d73/ded/db3/df0/f10a 0 2024-04-20T11:09:18.808 INFO:tasks.workunit.client.0.smithi117.stdout:0/857: dread d3/d7/d20/d47/f10d [0,4194304] 0 2024-04-20T11:09:18.811 INFO:tasks.workunit.client.0.smithi117.stdout:8/755: link d0/d8/d39/d40/d78/f98 d0/d9/d3a/d7c/dce/ffa 0 2024-04-20T11:09:18.812 INFO:tasks.workunit.client.0.smithi117.stdout:2/897: symlink d1/d15/d1d/l125 0 2024-04-20T11:09:18.815 INFO:tasks.workunit.client.0.smithi117.stdout:0/858: dread - d3/d7/d20/d39/d9a/dd0/d6d/f97 zero size 2024-04-20T11:09:18.819 INFO:tasks.workunit.client.0.smithi117.stdout:1/728: chown d6/d21/d95 105775639 1 2024-04-20T11:09:18.821 INFO:tasks.workunit.client.0.smithi117.stdout:4/808: dwrite d1/dd/d1a/d2b/d86/fe5 [0,4194304] 0 2024-04-20T11:09:18.824 INFO:tasks.workunit.client.0.smithi117.stdout:9/874: write d4/de/d7b/d96/db0/f77 [302987,4071] 0 2024-04-20T11:09:18.828 INFO:tasks.workunit.client.0.smithi117.stdout:9/875: chown d4/f39 83 1 2024-04-20T11:09:18.828 INFO:tasks.workunit.client.0.smithi117.stdout:2/898: write d1/f11 [2267154,107810] 0 2024-04-20T11:09:18.831 INFO:tasks.workunit.client.0.smithi117.stdout:3/779: fdatasync d0/d5/d73/fe9 0 2024-04-20T11:09:18.834 INFO:tasks.workunit.client.0.smithi117.stdout:9/876: write d4/d5/d3c/ddf/d31/f5d [194963,30817] 0 2024-04-20T11:09:18.836 INFO:tasks.workunit.client.0.smithi117.stdout:3/780: chown d0/d5/d73/ded/d66/d63/dad/laf 34 1 2024-04-20T11:09:18.837 INFO:tasks.workunit.client.0.smithi117.stdout:0/859: creat d3/d7/d20/d47/d78/f110 x:0 0 0 2024-04-20T11:09:18.838 INFO:tasks.workunit.client.0.smithi117.stdout:8/756: read d0/d8/d11/d6d/fba [497801,125376] 0 2024-04-20T11:09:18.845 INFO:tasks.workunit.client.0.smithi117.stdout:6/947: getdents d3/df/d39 0 2024-04-20T11:09:18.846 INFO:tasks.workunit.client.0.smithi117.stdout:9/877: mknod d4/d4a/d5e/de8/c126 0 2024-04-20T11:09:18.847 INFO:tasks.workunit.client.0.smithi117.stdout:1/729: read d6/d1f/d30/d49/d69/d6e/d6f/d73/f88 [263951,92548] 0 2024-04-20T11:09:18.848 INFO:tasks.workunit.client.0.smithi117.stdout:7/788: dread d1/d43/f61 [0,4194304] 0 2024-04-20T11:09:18.851 INFO:tasks.workunit.client.0.smithi117.stdout:3/781: mkdir d0/d5/d21/d88/d69/d10b 0 2024-04-20T11:09:18.856 INFO:tasks.workunit.client.0.smithi117.stdout:6/948: creat d3/d5a/d74/db3/dd0/f133 x:0 0 0 2024-04-20T11:09:18.859 INFO:tasks.workunit.client.0.smithi117.stdout:7/789: creat d1/d1d/d42/d2d/f103 x:0 0 0 2024-04-20T11:09:18.861 INFO:tasks.workunit.client.0.smithi117.stdout:3/782: unlink d0/d5/de/fd4 0 2024-04-20T11:09:18.864 INFO:tasks.workunit.client.0.smithi117.stdout:6/949: write d3/d4/d2e/d37/fdb [3672362,69628] 0 2024-04-20T11:09:18.866 INFO:tasks.workunit.client.0.smithi117.stdout:0/860: getdents d3/d7/d20/d6e/df3/dff/d85 0 2024-04-20T11:09:18.867 INFO:tasks.workunit.client.0.smithi117.stdout:1/730: truncate d6/d1f/d30/d32/fcb 62915 0 2024-04-20T11:09:18.870 INFO:tasks.workunit.client.0.smithi117.stdout:2/899: dwrite d1/f3 [4194304,4194304] 0 2024-04-20T11:09:18.875 INFO:tasks.workunit.client.0.smithi117.stdout:4/809: dwrite d1/f67 [0,4194304] 0 2024-04-20T11:09:18.879 INFO:tasks.workunit.client.0.smithi117.stdout:0/861: chown d3/d7e/fc7 54 1 2024-04-20T11:09:18.879 INFO:tasks.workunit.client.0.smithi117.stdout:2/900: fdatasync d1/d4/d83/f18 0 2024-04-20T11:09:18.880 INFO:tasks.workunit.client.0.smithi117.stdout:1/731: write d6/d1f/f3e [2948730,57762] 0 2024-04-20T11:09:18.880 INFO:tasks.workunit.client.0.smithi117.stdout:4/810: chown d1/d9/d79 152174 1 2024-04-20T11:09:18.886 INFO:tasks.workunit.client.0.smithi117.stdout:8/757: dwrite d0/d9/ff8 [0,4194304] 0 2024-04-20T11:09:18.890 INFO:tasks.workunit.client.0.smithi117.stdout:6/950: mknod d3/d5a/d74/db3/dbc/df6/c134 0 2024-04-20T11:09:18.893 INFO:tasks.workunit.client.0.smithi117.stdout:9/878: dread d4/de/f1b [0,4194304] 0 2024-04-20T11:09:18.897 INFO:tasks.workunit.client.0.smithi117.stdout:2/901: rmdir d1/d15/d1d/db3 39 2024-04-20T11:09:18.899 INFO:tasks.workunit.client.0.smithi117.stdout:1/732: creat d6/d16/d3a/d5a/d5c/d61/d98/fed x:0 0 0 2024-04-20T11:09:18.901 INFO:tasks.workunit.client.0.smithi117.stdout:8/758: readlink d0/la7 0 2024-04-20T11:09:18.902 INFO:tasks.workunit.client.0.smithi117.stdout:3/783: dwrite d0/d5/d7/fd6 [0,4194304] 0 2024-04-20T11:09:18.918 INFO:tasks.workunit.client.0.smithi117.stdout:4/811: rmdir d1/d9/d6b/dfd 0 2024-04-20T11:09:18.919 INFO:tasks.workunit.client.0.smithi117.stdout:7/790: dwrite d1/d1d/d77/fa8 [0,4194304] 0 2024-04-20T11:09:18.924 INFO:tasks.workunit.client.0.smithi117.stdout:7/791: write d1/d4/d6/d54/db4/fde [173535,2931] 0 2024-04-20T11:09:18.931 INFO:tasks.workunit.client.0.smithi117.stdout:0/862: dwrite d3/dab/d54/fe1 [0,4194304] 0 2024-04-20T11:09:18.954 INFO:tasks.workunit.client.0.smithi117.stdout:2/902: dread d1/f3 [0,4194304] 0 2024-04-20T11:09:18.958 INFO:tasks.workunit.client.0.smithi117.stdout:1/733: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/fee x:0 0 0 2024-04-20T11:09:18.961 INFO:tasks.workunit.client.0.smithi117.stdout:3/784: mknod d0/d5/d21/d94/da7/c10c 0 2024-04-20T11:09:18.967 INFO:tasks.workunit.client.0.smithi117.stdout:7/792: rename d1/d4/d6/d17/f3a to d1/dd1/f104 0 2024-04-20T11:09:18.967 INFO:tasks.workunit.client.0.smithi117.stdout:6/951: dread d3/df/f14 [0,4194304] 0 2024-04-20T11:09:18.971 INFO:tasks.workunit.client.0.smithi117.stdout:8/759: creat d0/d8/d39/d7f/ffb x:0 0 0 2024-04-20T11:09:18.971 INFO:tasks.workunit.client.0.smithi117.stdout:9/879: creat d4/de/d10/f127 x:0 0 0 2024-04-20T11:09:18.973 INFO:tasks.workunit.client.0.smithi117.stdout:1/734: mkdir d6/d16/d3a/d5a/daf/def 0 2024-04-20T11:09:18.975 INFO:tasks.workunit.client.0.smithi117.stdout:4/812: truncate d1/dd/d1a/d2b/d86/fdc 856185 0 2024-04-20T11:09:18.979 INFO:tasks.workunit.client.0.smithi117.stdout:3/785: creat d0/d5/d21/d75/d89/dd1/f10d x:0 0 0 2024-04-20T11:09:18.979 INFO:tasks.workunit.client.0.smithi117.stdout:0/863: mknod d3/d7/d20/d39/d9a/dd0/d6d/d10c/c111 0 2024-04-20T11:09:18.985 INFO:tasks.workunit.client.0.smithi117.stdout:2/903: link d1/d4/dd/f55 d1/d4/d83/d29/d2e/d53/d58/da8/f126 0 2024-04-20T11:09:18.989 INFO:tasks.workunit.client.0.smithi117.stdout:8/760: fdatasync d0/d8/d39/d40/fae 0 2024-04-20T11:09:18.990 INFO:tasks.workunit.client.0.smithi117.stdout:9/880: mkdir d4/d4a/d63/d98/dc0/d128 0 2024-04-20T11:09:18.992 INFO:tasks.workunit.client.0.smithi117.stdout:9/881: read - d4/d4a/d63/d98/fae zero size 2024-04-20T11:09:18.998 INFO:tasks.workunit.client.0.smithi117.stdout:9/882: write d4/de/d7b/d96/db0/f77 [3069445,53301] 0 2024-04-20T11:09:19.001 INFO:tasks.workunit.client.0.smithi117.stdout:2/904: dread d1/d20/de0/f114 [0,4194304] 0 2024-04-20T11:09:19.001 INFO:tasks.workunit.client.0.smithi117.stdout:3/786: rename d0/d5/d73/ded/d66/d63/f7a to d0/d5/d73/ded/dac/dd2/f10e 0 2024-04-20T11:09:19.010 INFO:tasks.workunit.client.0.smithi117.stdout:2/905: dread - d1/d4/d83/d29/d2e/d53/d93/f9e zero size 2024-04-20T11:09:19.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/793: dread d1/d1d/f5c [0,4194304] 0 2024-04-20T11:09:19.014 INFO:tasks.workunit.client.0.smithi117.stdout:2/906: read - d1/d4/d83/d29/d2e/d53/d93/f9e zero size 2024-04-20T11:09:19.020 INFO:tasks.workunit.client.0.smithi117.stdout:4/813: dread d1/dd/d1a/d2b/f46 [0,4194304] 0 2024-04-20T11:09:19.022 INFO:tasks.workunit.client.0.smithi117.stdout:1/735: mkdir d6/d16/d3a/d5a/d5c/df0 0 2024-04-20T11:09:19.023 INFO:tasks.workunit.client.0.smithi117.stdout:1/736: readlink d6/d1f/d30/d49/lab 0 2024-04-20T11:09:19.024 INFO:tasks.workunit.client.0.smithi117.stdout:1/737: truncate d6/f13 9180509 0 2024-04-20T11:09:19.027 INFO:tasks.workunit.client.0.smithi117.stdout:8/761: creat d0/d9/d3a/de3/ffc x:0 0 0 2024-04-20T11:09:19.031 INFO:tasks.workunit.client.0.smithi117.stdout:3/787: fdatasync d0/d5/d73/ded/d66/d63/fc4 0 2024-04-20T11:09:19.032 INFO:tasks.workunit.client.0.smithi117.stdout:9/883: mknod d4/d5/d57/db4/c129 0 2024-04-20T11:09:19.035 INFO:tasks.workunit.client.0.smithi117.stdout:3/788: chown d0/d5/d73/ded/d66/dbe 0 1 2024-04-20T11:09:19.041 INFO:tasks.workunit.client.0.smithi117.stdout:2/907: rename d1/d4/d83/d29/fda to d1/d9b/f127 0 2024-04-20T11:09:19.049 INFO:tasks.workunit.client.0.smithi117.stdout:6/952: dread d3/df/d39/d99/f8c [0,4194304] 0 2024-04-20T11:09:19.066 INFO:tasks.workunit.client.0.smithi117.stdout:8/762: unlink d0/d8/d39/d40/f43 0 2024-04-20T11:09:19.069 INFO:tasks.workunit.client.0.smithi117.stdout:9/884: unlink d4/d35/l3b 0 2024-04-20T11:09:19.071 INFO:tasks.workunit.client.0.smithi117.stdout:0/864: dwrite d3/d7/d20/d47/f77 [0,4194304] 0 2024-04-20T11:09:19.078 INFO:tasks.workunit.client.0.smithi117.stdout:9/885: symlink d4/d35/df7/d105/l12a 0 2024-04-20T11:09:19.079 INFO:tasks.workunit.client.0.smithi117.stdout:1/738: write d6/d16/fad [1618763,83703] 0 2024-04-20T11:09:19.082 INFO:tasks.workunit.client.0.smithi117.stdout:9/886: write d4/d4a/d5e/de8/f6d [4298588,108682] 0 2024-04-20T11:09:19.086 INFO:tasks.workunit.client.0.smithi117.stdout:3/789: write d0/d5/d21/d88/f44 [9175489,82654] 0 2024-04-20T11:09:19.088 INFO:tasks.workunit.client.0.smithi117.stdout:4/814: truncate d1/dd/d1a/f1d 2194206 0 2024-04-20T11:09:19.090 INFO:tasks.workunit.client.0.smithi117.stdout:6/953: getdents d3/d4/d2e/dc0/df7 0 2024-04-20T11:09:19.092 INFO:tasks.workunit.client.0.smithi117.stdout:6/954: chown d3/df/c11 21 1 2024-04-20T11:09:19.097 INFO:tasks.workunit.client.0.smithi117.stdout:0/865: creat d3/d7/d8/f112 x:0 0 0 2024-04-20T11:09:19.097 INFO:tasks.workunit.client.0.smithi117.stdout:7/794: dwrite d1/d1d/d42/fac [0,4194304] 0 2024-04-20T11:09:19.099 INFO:tasks.workunit.client.0.smithi117.stdout:2/908: dread d1/d4/dd/d17/d7a/d49/fcc [0,4194304] 0 2024-04-20T11:09:19.101 INFO:tasks.workunit.client.0.smithi117.stdout:3/790: creat d0/d5/d7/f10f x:0 0 0 2024-04-20T11:09:19.104 INFO:tasks.workunit.client.0.smithi117.stdout:0/866: readlink d3/d7/d20/d6e/df3/dff/l1a 0 2024-04-20T11:09:19.106 INFO:tasks.workunit.client.0.smithi117.stdout:8/763: getdents d0/d8/d39/d40/d78 0 2024-04-20T11:09:19.111 INFO:tasks.workunit.client.0.smithi117.stdout:1/739: truncate d6/d21/d95/fa9 1237902 0 2024-04-20T11:09:19.114 INFO:tasks.workunit.client.0.smithi117.stdout:6/955: unlink d3/df/d39/daa/d100/f110 0 2024-04-20T11:09:19.114 INFO:tasks.workunit.client.0.smithi117.stdout:9/887: mkdir d4/d4a/d5e/d113/d12b 0 2024-04-20T11:09:19.119 INFO:tasks.workunit.client.0.smithi117.stdout:9/888: write d4/d5/f119 [1350095,10093] 0 2024-04-20T11:09:19.141 INFO:tasks.workunit.client.0.smithi117.stdout:9/889: creat d4/d4a/d5e/de8/f12c x:0 0 0 2024-04-20T11:09:19.142 INFO:tasks.workunit.client.0.smithi117.stdout:7/795: rename d1/d4/d6/d17/c51 to d1/d4/d3d/dce/deb/c105 0 2024-04-20T11:09:19.147 INFO:tasks.workunit.client.0.smithi117.stdout:4/815: dwrite d1/dd/d1a/f1e [0,4194304] 0 2024-04-20T11:09:19.147 INFO:tasks.workunit.client.0.smithi117.stdout:2/909: write d1/d4/dd/f77 [4302588,38429] 0 2024-04-20T11:09:19.148 INFO:tasks.workunit.client.0.smithi117.stdout:0/867: fdatasync d3/d7/d8/f64 0 2024-04-20T11:09:19.149 INFO:tasks.workunit.client.0.smithi117.stdout:6/956: write d3/fc [326907,120582] 0 2024-04-20T11:09:19.153 INFO:tasks.workunit.client.0.smithi117.stdout:7/796: unlink d1/d4/d6/d17/f4c 0 2024-04-20T11:09:19.155 INFO:tasks.workunit.client.0.smithi117.stdout:0/868: readlink d3/d7/d20/d39/d9a/dd0/db2/dbd/lf5 0 2024-04-20T11:09:19.157 INFO:tasks.workunit.client.0.smithi117.stdout:7/797: fdatasync d1/d43/d59/ffd 0 2024-04-20T11:09:19.165 INFO:tasks.workunit.client.0.smithi117.stdout:4/816: dread d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:19.169 INFO:tasks.workunit.client.0.smithi117.stdout:1/740: link d6/d35/l9f d6/d16/d3a/d5a/d5c/d61/d98/deb/lf1 0 2024-04-20T11:09:19.169 INFO:tasks.workunit.client.0.smithi117.stdout:3/791: getdents d0/d5/d73/ded/d66/d63 0 2024-04-20T11:09:19.170 INFO:tasks.workunit.client.0.smithi117.stdout:2/910: creat d1/d15/d118/f128 x:0 0 0 2024-04-20T11:09:19.172 INFO:tasks.workunit.client.0.smithi117.stdout:4/817: write d1/d9/de9/fe8 [859397,109815] 0 2024-04-20T11:09:19.180 INFO:tasks.workunit.client.0.smithi117.stdout:7/798: symlink d1/d4/d6/d13/l106 0 2024-04-20T11:09:19.180 INFO:tasks.workunit.client.0.smithi117.stdout:4/818: truncate d1/d9/fb0 784910 0 2024-04-20T11:09:19.181 INFO:tasks.workunit.client.0.smithi117.stdout:6/957: rename d3/c5b to d3/d50/c135 0 2024-04-20T11:09:19.181 INFO:tasks.workunit.client.0.smithi117.stdout:7/799: chown d1/d1d/d42/d2d/lcd 650 1 2024-04-20T11:09:19.183 INFO:tasks.workunit.client.0.smithi117.stdout:8/764: dread d0/d8/d11/d6d/d69/f85 [0,4194304] 0 2024-04-20T11:09:19.183 INFO:tasks.workunit.client.0.smithi117.stdout:9/890: link d4/de/d7b/d96/db0/lda d4/d5/d3c/ddf/d78/d111/l12d 0 2024-04-20T11:09:19.186 INFO:tasks.workunit.client.0.smithi117.stdout:4/819: creat d1/dd/d52/f111 x:0 0 0 2024-04-20T11:09:19.190 INFO:tasks.workunit.client.0.smithi117.stdout:3/792: rename d0/d5/de/d4e/l7c to d0/d5/d73/ded/d66/dbe/l110 0 2024-04-20T11:09:19.190 INFO:tasks.workunit.client.0.smithi117.stdout:1/741: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/de9/ff2 x:0 0 0 2024-04-20T11:09:19.191 INFO:tasks.workunit.client.0.smithi117.stdout:4/820: fdatasync d1/f25 0 2024-04-20T11:09:19.194 INFO:tasks.workunit.client.0.smithi117.stdout:6/958: getdents d3/d5a/d74/db3/dbc/d9b 0 2024-04-20T11:09:19.200 INFO:tasks.workunit.client.0.smithi117.stdout:6/959: write d3/d4/d2e/f40 [5034774,69823] 0 2024-04-20T11:09:19.200 INFO:tasks.workunit.client.0.smithi117.stdout:9/891: symlink d4/d5/d57/d82/dec/l12e 0 2024-04-20T11:09:19.201 INFO:tasks.workunit.client.0.smithi117.stdout:8/765: creat d0/dc/d49/d5b/da9/dc7/ffd x:0 0 0 2024-04-20T11:09:19.206 INFO:tasks.workunit.client.0.smithi117.stdout:6/960: mkdir d3/d50/d8a/d136 0 2024-04-20T11:09:19.206 INFO:tasks.workunit.client.0.smithi117.stdout:8/766: dread - d0/d8/d11/ff9 zero size 2024-04-20T11:09:19.211 INFO:tasks.workunit.client.0.smithi117.stdout:7/800: dread d1/d43/f61 [0,4194304] 0 2024-04-20T11:09:19.213 INFO:tasks.workunit.client.0.smithi117.stdout:9/892: truncate d4/d68/fb3 4081004 0 2024-04-20T11:09:19.217 INFO:tasks.workunit.client.0.smithi117.stdout:9/893: fsync d4/de/db1/fb9 0 2024-04-20T11:09:19.217 INFO:tasks.workunit.client.0.smithi117.stdout:0/869: dwrite d3/d7/d2c/d52/fc5 [0,4194304] 0 2024-04-20T11:09:19.221 INFO:tasks.workunit.client.0.smithi117.stdout:2/911: dwrite d1/d4/d83/d29/d62/df7/fff [0,4194304] 0 2024-04-20T11:09:19.224 INFO:tasks.workunit.client.0.smithi117.stdout:6/961: dread d3/d4/d2e/d37/d8e/f9d [0,4194304] 0 2024-04-20T11:09:19.228 INFO:tasks.workunit.client.0.smithi117.stdout:7/801: write d1/d43/f8d [1307076,85251] 0 2024-04-20T11:09:19.229 INFO:tasks.workunit.client.0.smithi117.stdout:3/793: dwrite d0/d5/d21/f41 [0,4194304] 0 2024-04-20T11:09:19.232 INFO:tasks.workunit.client.0.smithi117.stdout:8/767: rename d0/d8/d39/d40/d78/c9a to d0/d9/d3a/de3/cfe 0 2024-04-20T11:09:19.233 INFO:tasks.workunit.client.0.smithi117.stdout:3/794: chown d0/d5/d21/d88/d69/d10b 365630590 1 2024-04-20T11:09:19.237 INFO:tasks.workunit.client.0.smithi117.stdout:4/821: dwrite d1/dd/d1a/d2b/f46 [0,4194304] 0 2024-04-20T11:09:19.240 INFO:tasks.workunit.client.0.smithi117.stdout:4/822: read d1/dd/d1a/f1e [2160670,40819] 0 2024-04-20T11:09:19.240 INFO:tasks.workunit.client.0.smithi117.stdout:6/962: stat d3/df/f21 0 2024-04-20T11:09:19.243 INFO:tasks.workunit.client.0.smithi117.stdout:4/823: chown d1/dd/d1a/d2b/c97 31 1 2024-04-20T11:09:19.244 INFO:tasks.workunit.client.0.smithi117.stdout:1/742: write d6/d16/d3a/d5a/f5d [1140752,100777] 0 2024-04-20T11:09:19.245 INFO:tasks.workunit.client.0.smithi117.stdout:8/768: mkdir d0/d9/d3a/de3/dff 0 2024-04-20T11:09:19.250 INFO:tasks.workunit.client.0.smithi117.stdout:3/795: dread - d0/d5/d21/d88/d36/fb2 zero size 2024-04-20T11:09:19.252 INFO:tasks.workunit.client.0.smithi117.stdout:4/824: dread - d1/dd/d1a/d43/d9a/fb2 zero size 2024-04-20T11:09:19.256 INFO:tasks.workunit.client.0.smithi117.stdout:4/825: stat d1/dd/d1a/d43/dcc/cf9 0 2024-04-20T11:09:19.267 INFO:tasks.workunit.client.0.smithi117.stdout:2/912: write d1/d4/d83/d29/d62/f8a [1112931,71507] 0 2024-04-20T11:09:19.268 INFO:tasks.workunit.client.0.smithi117.stdout:3/796: mkdir d0/d5/d21/d88/d36/d111 0 2024-04-20T11:09:19.270 INFO:tasks.workunit.client.0.smithi117.stdout:1/743: truncate d6/d1f/d30/f67 1179874 0 2024-04-20T11:09:19.270 INFO:tasks.workunit.client.0.smithi117.stdout:2/913: chown d1/d15/d1d 931289 1 2024-04-20T11:09:19.272 INFO:tasks.workunit.client.0.smithi117.stdout:0/870: write d3/d7/d2c/d52/fd3 [1112981,104263] 0 2024-04-20T11:09:19.276 INFO:tasks.workunit.client.0.smithi117.stdout:4/826: creat d1/dd/d1a/d2b/d86/f112 x:0 0 0 2024-04-20T11:09:19.277 INFO:tasks.workunit.client.0.smithi117.stdout:9/894: getdents d4/d5/d3c/ddf/d31 0 2024-04-20T11:09:19.278 INFO:tasks.workunit.client.0.smithi117.stdout:1/744: stat d6/d21/db9 0 2024-04-20T11:09:19.282 INFO:tasks.workunit.client.0.smithi117.stdout:9/895: read - d4/d4a/d5e/de8/f120 zero size 2024-04-20T11:09:19.282 INFO:tasks.workunit.client.0.smithi117.stdout:1/745: fdatasync d6/d1f/f3e 0 2024-04-20T11:09:19.294 INFO:tasks.workunit.client.0.smithi117.stdout:9/896: rename d4/d5/f28 to d4/d4a/d5e/f12f 0 2024-04-20T11:09:19.294 INFO:tasks.workunit.client.0.smithi117.stdout:7/802: dread d1/d43/f73 [0,4194304] 0 2024-04-20T11:09:19.300 INFO:tasks.workunit.client.0.smithi117.stdout:1/746: symlink d6/d1f/d30/d49/lf3 0 2024-04-20T11:09:19.307 INFO:tasks.workunit.client.0.smithi117.stdout:7/803: mkdir d1/d1d/ded/d107 0 2024-04-20T11:09:19.307 INFO:tasks.workunit.client.0.smithi117.stdout:1/747: fdatasync d6/d1f/f45 0 2024-04-20T11:09:19.311 INFO:tasks.workunit.client.0.smithi117.stdout:7/804: chown d1/d43/f80 156023 1 2024-04-20T11:09:19.314 INFO:tasks.workunit.client.0.smithi117.stdout:7/805: chown d1/d1d 126087 1 2024-04-20T11:09:19.319 INFO:tasks.workunit.client.0.smithi117.stdout:4/827: write d1/dd/f13 [3575639,2269] 0 2024-04-20T11:09:19.324 INFO:tasks.workunit.client.0.smithi117.stdout:9/897: dwrite d4/d35/f9a [0,4194304] 0 2024-04-20T11:09:19.331 INFO:tasks.workunit.client.0.smithi117.stdout:3/797: dwrite d0/d5/d73/ded/d66/d63/fa9 [0,4194304] 0 2024-04-20T11:09:19.331 INFO:tasks.workunit.client.0.smithi117.stdout:1/748: rename d6/d16/d3a/da8/fbc to d6/d16/d3a/d5a/d5c/da6/ff4 0 2024-04-20T11:09:19.332 INFO:tasks.workunit.client.0.smithi117.stdout:9/898: chown d4/d5/d3c/ddf/d31/c64 91379 1 2024-04-20T11:09:19.336 INFO:tasks.workunit.client.0.smithi117.stdout:9/899: chown d4/d4a 28863000 1 2024-04-20T11:09:19.338 INFO:tasks.workunit.client.0.smithi117.stdout:9/900: read f0 [1203065,91856] 0 2024-04-20T11:09:19.342 INFO:tasks.workunit.client.0.smithi117.stdout:2/914: dread d1/d4/d83/f23 [0,4194304] 0 2024-04-20T11:09:19.343 INFO:tasks.workunit.client.0.smithi117.stdout:7/806: rename d1/fd8 to d1/d1d/d4e/d53/d55/f108 0 2024-04-20T11:09:19.346 INFO:tasks.workunit.client.0.smithi117.stdout:0/871: dread d3/d7/d20/d6e/df3/dff/d85/fd4 [0,4194304] 0 2024-04-20T11:09:19.359 INFO:tasks.workunit.client.0.smithi117.stdout:6/963: dread d3/d4/d2e/d4e/fa2 [0,4194304] 0 2024-04-20T11:09:19.359 INFO:tasks.workunit.client.0.smithi117.stdout:1/749: rename d6/d1f/f3e to d6/d16/d3a/d5a/daf/ff5 0 2024-04-20T11:09:19.361 INFO:tasks.workunit.client.0.smithi117.stdout:4/828: dwrite d1/dd/d1a/d2b/f46 [0,4194304] 0 2024-04-20T11:09:19.361 INFO:tasks.workunit.client.0.smithi117.stdout:7/807: symlink d1/d1d/d42/d2d/d6c/l109 0 2024-04-20T11:09:19.361 INFO:tasks.workunit.client.0.smithi117.stdout:2/915: rmdir d1/d4/d83/d29/d2e/d80 39 2024-04-20T11:09:19.365 INFO:tasks.workunit.client.0.smithi117.stdout:7/808: stat d1/d43 0 2024-04-20T11:09:19.365 INFO:tasks.workunit.client.0.smithi117.stdout:6/964: truncate d3/d94/ffa 631931 0 2024-04-20T11:09:19.367 INFO:tasks.workunit.client.0.smithi117.stdout:6/965: readlink d3/d3a/d7f/l84 0 2024-04-20T11:09:19.369 INFO:tasks.workunit.client.0.smithi117.stdout:9/901: mknod d4/d5/d3c/ddf/d78/d111/c130 0 2024-04-20T11:09:19.371 INFO:tasks.workunit.client.0.smithi117.stdout:1/750: read - d6/d21/f86 zero size 2024-04-20T11:09:19.371 INFO:tasks.workunit.client.0.smithi117.stdout:3/798: write d0/d5/d21/d94/d9a/fbb [665020,19554] 0 2024-04-20T11:09:19.373 INFO:tasks.workunit.client.0.smithi117.stdout:2/916: dread - d1/d4/d69/f8c zero size 2024-04-20T11:09:19.373 INFO:tasks.workunit.client.0.smithi117.stdout:0/872: mknod d3/d7/d20/c113 0 2024-04-20T11:09:19.375 INFO:tasks.workunit.client.0.smithi117.stdout:2/917: stat d1/d4/cab 0 2024-04-20T11:09:19.381 INFO:tasks.workunit.client.0.smithi117.stdout:7/809: dread d1/d43/f8d [0,4194304] 0 2024-04-20T11:09:19.387 INFO:tasks.workunit.client.0.smithi117.stdout:1/751: mkdir d6/d16/d3a/d5a/daf/df6 0 2024-04-20T11:09:19.388 INFO:tasks.workunit.client.0.smithi117.stdout:0/873: creat d3/d5/f114 x:0 0 0 2024-04-20T11:09:19.389 INFO:tasks.workunit.client.0.smithi117.stdout:2/918: dread - d1/d4/fc0 zero size 2024-04-20T11:09:19.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:18 smithi117 ceph-mon[15659]: pgmap v19: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 69 MiB/s rd, 161 MiB/s wr, 412 op/s 2024-04-20T11:09:19.391 INFO:tasks.workunit.client.0.smithi117.stdout:2/919: fsync d1/d4/fb 0 2024-04-20T11:09:19.393 INFO:tasks.workunit.client.0.smithi117.stdout:4/829: truncate d1/dd/d1a/d43/d9a/fb9 703996 0 2024-04-20T11:09:19.396 INFO:tasks.workunit.client.0.smithi117.stdout:1/752: chown d6/d16/d3a/d5a/d5c/d61/d71/d6b/c76 422 1 2024-04-20T11:09:19.396 INFO:tasks.workunit.client.0.smithi117.stdout:9/902: creat d4/f131 x:0 0 0 2024-04-20T11:09:19.399 INFO:tasks.workunit.client.0.smithi117.stdout:1/753: dread - d6/d1f/dd6/fe5 zero size 2024-04-20T11:09:19.401 INFO:tasks.workunit.client.0.smithi117.stdout:3/799: creat d0/d5/d21/d88/f112 x:0 0 0 2024-04-20T11:09:19.403 INFO:tasks.workunit.client.0.smithi117.stdout:6/966: dread d3/df/f4f [0,4194304] 0 2024-04-20T11:09:19.409 INFO:tasks.workunit.client.0.smithi117.stdout:1/754: mknod d6/d1f/d30/d49/d69/d6e/d6f/cf7 0 2024-04-20T11:09:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:19 smithi173 ceph-mon[23800]: pgmap v19: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 69 MiB/s rd, 161 MiB/s wr, 412 op/s 2024-04-20T11:09:19.411 INFO:tasks.workunit.client.0.smithi117.stdout:3/800: creat d0/d5/d11/f113 x:0 0 0 2024-04-20T11:09:19.412 INFO:tasks.workunit.client.0.smithi117.stdout:1/755: readlink d6/lb 0 2024-04-20T11:09:19.412 INFO:tasks.workunit.client.0.smithi117.stdout:9/903: mknod d4/d35/c132 0 2024-04-20T11:09:19.417 INFO:tasks.workunit.client.0.smithi117.stdout:6/967: dread d3/d3a/d114/f118 [0,4194304] 0 2024-04-20T11:09:19.417 INFO:tasks.workunit.client.0.smithi117.stdout:2/920: rename d1/d4/cc5 to d1/d4/d83/d29/d2e/d53/de7/c129 0 2024-04-20T11:09:19.421 INFO:tasks.workunit.client.0.smithi117.stdout:9/904: chown d4/de4/l10f 489 1 2024-04-20T11:09:19.423 INFO:tasks.workunit.client.0.smithi117.stdout:4/830: dwrite d1/d45/d4a/d59/fbc [0,4194304] 0 2024-04-20T11:09:19.426 INFO:tasks.workunit.client.0.smithi117.stdout:1/756: mkdir d6/d16/d3a/d5a/daf/def/df8 0 2024-04-20T11:09:19.428 INFO:tasks.workunit.client.0.smithi117.stdout:2/921: creat d1/d4/d83/d29/d2e/d53/d58/da8/f12a x:0 0 0 2024-04-20T11:09:19.430 INFO:tasks.workunit.client.0.smithi117.stdout:3/801: truncate d0/d5/de/d4e/f9e 5137856 0 2024-04-20T11:09:19.434 INFO:tasks.workunit.client.0.smithi117.stdout:2/922: rmdir d1/d4/d83 39 2024-04-20T11:09:19.436 INFO:tasks.workunit.client.0.smithi117.stdout:4/831: write d1/d45/d4a/d59/fce [4568648,30648] 0 2024-04-20T11:09:19.442 INFO:tasks.workunit.client.0.smithi117.stdout:0/874: dread d3/d7/d20/d39/d9a/dd0/d6d/fb4 [0,4194304] 0 2024-04-20T11:09:19.442 INFO:tasks.workunit.client.0.smithi117.stdout:3/802: symlink d0/d5/d21/d94/d9a/l114 0 2024-04-20T11:09:19.442 INFO:tasks.workunit.client.0.smithi117.stdout:1/757: mkdir d6/d1f/d30/d49/d69/d6e/d6f/d73/df9 0 2024-04-20T11:09:19.445 INFO:tasks.workunit.client.0.smithi117.stdout:4/832: chown d1/d9/d6b/ff0 267972 1 2024-04-20T11:09:19.446 INFO:tasks.workunit.client.0.smithi117.stdout:3/803: readlink d0/d5/d11/l12 0 2024-04-20T11:09:19.446 INFO:tasks.workunit.client.0.smithi117.stdout:7/810: dwrite d1/d1d/d77/fc5 [0,4194304] 0 2024-04-20T11:09:19.446 INFO:tasks.workunit.client.0.smithi117.stdout:2/923: truncate d1/da5/df2/f98 4588282 0 2024-04-20T11:09:19.458 INFO:tasks.workunit.client.0.smithi117.stdout:3/804: unlink d0/d5/d21/d94/da7/la8 0 2024-04-20T11:09:19.466 INFO:tasks.workunit.client.0.smithi117.stdout:6/968: truncate d3/f28 2326098 0 2024-04-20T11:09:19.470 INFO:tasks.workunit.client.0.smithi117.stdout:4/833: creat d1/dd/d1a/d43/d108/f113 x:0 0 0 2024-04-20T11:09:19.473 INFO:tasks.workunit.client.0.smithi117.stdout:7/811: creat d1/d1d/d4e/f10a x:0 0 0 2024-04-20T11:09:19.475 INFO:tasks.workunit.client.0.smithi117.stdout:3/805: mknod d0/d5/d21/d94/da7/d109/c115 0 2024-04-20T11:09:19.478 INFO:tasks.workunit.client.0.smithi117.stdout:4/834: creat d1/dd/d52/f114 x:0 0 0 2024-04-20T11:09:19.479 INFO:tasks.workunit.client.0.smithi117.stdout:6/969: symlink d3/d4/d2e/d37/d8e/l137 0 2024-04-20T11:09:19.479 INFO:tasks.workunit.client.0.smithi117.stdout:9/905: dread d4/de/d10/f5c [0,4194304] 0 2024-04-20T11:09:19.483 INFO:tasks.workunit.client.0.smithi117.stdout:0/875: write d3/d7/d16/f56 [337446,61986] 0 2024-04-20T11:09:19.484 INFO:tasks.workunit.client.0.smithi117.stdout:2/924: dwrite d1/d20/de0/f11f [0,4194304] 0 2024-04-20T11:09:19.485 INFO:tasks.workunit.client.0.smithi117.stdout:0/876: fdatasync d3/d5/d74/fa9 0 2024-04-20T11:09:19.486 INFO:tasks.workunit.client.0.smithi117.stdout:4/835: dread - d1/dd/d1a/d2b/d86/f112 zero size 2024-04-20T11:09:19.487 INFO:tasks.workunit.client.0.smithi117.stdout:6/970: mkdir d3/d4/d2e/d37/d8e/db2/d138 0 2024-04-20T11:09:19.489 INFO:tasks.workunit.client.0.smithi117.stdout:2/925: chown d1/d20/fa9 29 1 2024-04-20T11:09:19.489 INFO:tasks.workunit.client.0.smithi117.stdout:4/836: readlink d1/d9/d6b/l7e 0 2024-04-20T11:09:19.501 INFO:tasks.workunit.client.0.smithi117.stdout:0/877: symlink d3/d7/l115 0 2024-04-20T11:09:19.504 INFO:tasks.workunit.client.0.smithi117.stdout:4/837: mknod d1/d9/c115 0 2024-04-20T11:09:19.505 INFO:tasks.workunit.client.0.smithi117.stdout:0/878: readlink d3/d7/d20/d6e/df3/dff/d85/lba 0 2024-04-20T11:09:19.507 INFO:tasks.workunit.client.0.smithi117.stdout:9/906: dwrite d4/d5/d57/db4/f11d [0,4194304] 0 2024-04-20T11:09:19.513 INFO:tasks.workunit.client.0.smithi117.stdout:2/926: rename d1/d4/d83/d29/daa/l123 to d1/d4/d83/d29/d3f/df6/d119/l12b 0 2024-04-20T11:09:19.515 INFO:tasks.workunit.client.0.smithi117.stdout:2/927: write d1/d15/d118/f128 [806483,99182] 0 2024-04-20T11:09:19.515 INFO:tasks.workunit.client.0.smithi117.stdout:0/879: mkdir d3/d5/d116 0 2024-04-20T11:09:19.518 INFO:tasks.workunit.client.0.smithi117.stdout:0/880: chown d3/d7/f108 8754 1 2024-04-20T11:09:19.518 INFO:tasks.workunit.client.0.smithi117.stdout:2/928: chown d1/d15/d1d/ld8 87508005 1 2024-04-20T11:09:19.525 INFO:tasks.workunit.client.0.smithi117.stdout:4/838: rename d1/dd/d1a/d43/d4e/l7a to d1/d9/d1f/d87/l116 0 2024-04-20T11:09:19.529 INFO:tasks.workunit.client.0.smithi117.stdout:6/971: rmdir d3/d4/d2e/d37/d8e/db2 39 2024-04-20T11:09:19.532 INFO:tasks.workunit.client.0.smithi117.stdout:3/806: dread d0/d5/d73/ded/dac/ff8 [0,4194304] 0 2024-04-20T11:09:19.534 INFO:tasks.workunit.client.0.smithi117.stdout:2/929: mknod d1/d4/d83/d29/d3f/df6/c12c 0 2024-04-20T11:09:19.536 INFO:tasks.workunit.client.0.smithi117.stdout:2/930: dread - d1/d4/d83/d29/d2e/d53/d93/f10f zero size 2024-04-20T11:09:19.555 INFO:tasks.workunit.client.0.smithi117.stdout:0/881: write d3/d7/d20/d6e/df3/dff/d85/fcd [762845,53590] 0 2024-04-20T11:09:19.559 INFO:tasks.workunit.client.0.smithi117.stdout:6/972: write d3/df/d39/fd5 [849203,99519] 0 2024-04-20T11:09:19.562 INFO:tasks.workunit.client.0.smithi117.stdout:2/931: write d1/d20/fed [778212,107531] 0 2024-04-20T11:09:19.565 INFO:tasks.workunit.client.0.smithi117.stdout:9/907: dwrite d4/d35/f56 [4194304,4194304] 0 2024-04-20T11:09:19.569 INFO:tasks.workunit.client.0.smithi117.stdout:6/973: dread - d3/d4/d2e/d4e/fbb zero size 2024-04-20T11:09:19.576 INFO:tasks.workunit.client.0.smithi117.stdout:2/932: creat d1/d4/d83/d29/d62/da6/f12d x:0 0 0 2024-04-20T11:09:19.578 INFO:tasks.workunit.client.0.smithi117.stdout:6/974: fdatasync d3/d4/d2e/d37/d8e/db2/f129 0 2024-04-20T11:09:19.579 INFO:tasks.workunit.client.0.smithi117.stdout:3/807: dwrite d0/d5/d21/fb5 [0,4194304] 0 2024-04-20T11:09:19.586 INFO:tasks.workunit.client.0.smithi117.stdout:4/839: dread d1/d45/fa1 [0,4194304] 0 2024-04-20T11:09:19.588 INFO:tasks.workunit.client.0.smithi117.stdout:9/908: link d4/d35/df7/d105/fc d4/de/d10/f133 0 2024-04-20T11:09:19.589 INFO:tasks.workunit.client.0.smithi117.stdout:3/808: fdatasync d0/d5/d21/d88/d85/fcb 0 2024-04-20T11:09:19.595 INFO:tasks.workunit.client.0.smithi117.stdout:4/840: rename d1/dd/d1a/d43/d9a to d1/dd/d52/d117 0 2024-04-20T11:09:19.597 INFO:tasks.workunit.client.0.smithi117.stdout:0/882: dwrite d3/d7/f80 [0,4194304] 0 2024-04-20T11:09:19.609 INFO:tasks.workunit.client.0.smithi117.stdout:4/841: rename d1/d45/c8e to d1/d9/d6b/dca/de7/c118 0 2024-04-20T11:09:19.614 INFO:tasks.workunit.client.0.smithi117.stdout:2/933: dwrite d1/d20/f21 [0,4194304] 0 2024-04-20T11:09:19.615 INFO:tasks.workunit.client.0.smithi117.stdout:0/883: rename d3/d7e/c81 to d3/d7/d20/d39/d9a/dd0/d6d/d10c/c117 0 2024-04-20T11:09:19.620 INFO:tasks.workunit.client.0.smithi117.stdout:2/934: write d1/d4/d83/d29/d2e/d53/d58/fee [279545,35436] 0 2024-04-20T11:09:19.628 INFO:tasks.workunit.client.0.smithi117.stdout:2/935: write d1/d4/d83/d29/d2e/d53/de7/f120 [807191,96301] 0 2024-04-20T11:09:19.633 INFO:tasks.workunit.client.0.smithi117.stdout:4/842: rename d1/d9/d1f/d87/l116 to d1/d45/d4a/d59/l119 0 2024-04-20T11:09:19.640 INFO:tasks.workunit.client.0.smithi117.stdout:4/843: getdents d1/d9/d79 0 2024-04-20T11:09:19.643 INFO:tasks.workunit.client.0.smithi117.stdout:6/975: dread d3/df/d39/d99/f64 [0,4194304] 0 2024-04-20T11:09:19.644 INFO:tasks.workunit.client.0.smithi117.stdout:4/844: chown d1/dd/d1a/d43/d4e/f9b 0 1 2024-04-20T11:09:19.646 INFO:tasks.workunit.client.0.smithi117.stdout:4/845: chown d1/d45/c63 0 1 2024-04-20T11:09:19.649 INFO:tasks.workunit.client.0.smithi117.stdout:4/846: write d1/dd/d1a/d43/d108/f113 [456084,76163] 0 2024-04-20T11:09:19.654 INFO:tasks.workunit.client.0.smithi117.stdout:4/847: write d1/d9/d1f/d87/fb3 [1848122,49397] 0 2024-04-20T11:09:19.661 INFO:tasks.workunit.client.0.smithi117.stdout:3/809: dread d0/d2a/fdd [0,4194304] 0 2024-04-20T11:09:19.663 INFO:tasks.workunit.client.0.smithi117.stdout:9/909: dread d4/d4a/d5e/f75 [0,4194304] 0 2024-04-20T11:09:19.675 INFO:tasks.workunit.client.0.smithi117.stdout:4/848: creat d1/d9/d1f/d90/d107/f11a x:0 0 0 2024-04-20T11:09:19.677 INFO:tasks.workunit.client.0.smithi117.stdout:9/910: mknod d4/d35/c134 0 2024-04-20T11:09:19.682 INFO:tasks.workunit.client.0.smithi117.stdout:0/884: dwrite d3/d7/d2c/f65 [4194304,4194304] 0 2024-04-20T11:09:19.686 INFO:tasks.workunit.client.0.smithi117.stdout:2/936: dwrite d1/d4/d83/d29/d3f/f76 [4194304,4194304] 0 2024-04-20T11:09:19.687 INFO:tasks.workunit.client.0.smithi117.stdout:6/976: dread d3/d4/d2e/d3f/fc9 [0,4194304] 0 2024-04-20T11:09:19.698 INFO:tasks.workunit.client.0.smithi117.stdout:9/911: creat d4/d4a/d72/dd2/d106/d10d/f135 x:0 0 0 2024-04-20T11:09:19.706 INFO:tasks.workunit.client.0.smithi117.stdout:0/885: write d3/d7/d94/fec [729317,75995] 0 2024-04-20T11:09:19.708 INFO:tasks.workunit.client.0.smithi117.stdout:0/886: dread - d3/d7/d16/feb zero size 2024-04-20T11:09:19.711 INFO:tasks.workunit.client.0.smithi117.stdout:0/887: fsync d3/d5/f37 0 2024-04-20T11:09:19.715 INFO:tasks.workunit.client.0.smithi117.stdout:9/912: dwrite d4/d5/d57/f99 [0,4194304] 0 2024-04-20T11:09:19.725 INFO:tasks.workunit.client.0.smithi117.stdout:6/977: rename d3/d4/d2e to d3/df/d39/d99/d139 0 2024-04-20T11:09:19.728 INFO:tasks.workunit.client.0.smithi117.stdout:9/913: symlink d4/d4a/d5e/de8/l136 0 2024-04-20T11:09:19.734 INFO:tasks.workunit.client.0.smithi117.stdout:9/914: creat d4/de/d7b/d96/db0/f137 x:0 0 0 2024-04-20T11:09:19.737 INFO:tasks.workunit.client.0.smithi117.stdout:6/978: mkdir d3/df/d39/daa/d11b/d13a 0 2024-04-20T11:09:19.739 INFO:tasks.workunit.client.0.smithi117.stdout:9/915: chown d4/d4a/d72 1 1 2024-04-20T11:09:19.740 INFO:tasks.workunit.client.0.smithi117.stdout:6/979: fsync d3/df/d39/d99/d139/d3f/f77 0 2024-04-20T11:09:19.758 INFO:tasks.workunit.client.0.smithi117.stdout:2/937: dwrite d1/d4/d83/f97 [0,4194304] 0 2024-04-20T11:09:19.762 INFO:tasks.workunit.client.0.smithi117.stdout:9/916: dwrite d4/f131 [0,4194304] 0 2024-04-20T11:09:19.771 INFO:tasks.workunit.client.0.smithi117.stdout:6/980: dwrite d3/df/d39/ffd [0,4194304] 0 2024-04-20T11:09:19.772 INFO:tasks.workunit.client.0.smithi117.stdout:0/888: dread d3/d5/f6a [0,4194304] 0 2024-04-20T11:09:19.780 INFO:tasks.workunit.client.0.smithi117.stdout:9/917: rename d4/de/d7b/d96/cc6 to d4/d4a/d5e/d113/c138 0 2024-04-20T11:09:19.788 INFO:tasks.workunit.client.0.smithi117.stdout:9/918: read d4/d5/d3c/f47 [3869144,82457] 0 2024-04-20T11:09:19.788 INFO:tasks.workunit.client.0.smithi117.stdout:2/938: creat d1/d4/d83/d29/d2e/d80/f12e x:0 0 0 2024-04-20T11:09:19.794 INFO:tasks.workunit.client.0.smithi117.stdout:6/981: chown d3/d4/l34 49519220 1 2024-04-20T11:09:19.796 INFO:tasks.workunit.client.0.smithi117.stdout:0/889: dwrite d3/d7e/fc7 [0,4194304] 0 2024-04-20T11:09:19.798 INFO:tasks.workunit.client.0.smithi117.stdout:0/890: chown d3/d7/lae 148932 1 2024-04-20T11:09:19.816 INFO:tasks.workunit.client.0.smithi117.stdout:9/919: getdents d4/d4a/d5e 0 2024-04-20T11:09:19.817 INFO:tasks.workunit.client.0.smithi117.stdout:6/982: getdents d3/df/d39/d99 0 2024-04-20T11:09:19.818 INFO:tasks.workunit.client.0.smithi117.stdout:9/920: fsync d4/de/d7b/d96/db0/f77 0 2024-04-20T11:09:19.821 INFO:tasks.workunit.client.0.smithi117.stdout:9/921: dread - d4/d4a/d5e/de8/f12c zero size 2024-04-20T11:09:19.822 INFO:tasks.workunit.client.0.smithi117.stdout:6/983: unlink d3/df/d39/c68 0 2024-04-20T11:09:19.827 INFO:tasks.workunit.client.0.smithi117.stdout:6/984: creat d3/df/d39/daa/d100/d103/f13b x:0 0 0 2024-04-20T11:09:19.829 INFO:tasks.workunit.client.0.smithi117.stdout:9/922: rename d4/d5/d3c/ddf/d78/d111/l12d to d4/d5/d57/d82/da5/l139 0 2024-04-20T11:09:19.832 INFO:tasks.workunit.client.0.smithi117.stdout:0/891: write d3/d7/f96 [235322,105861] 0 2024-04-20T11:09:19.834 INFO:tasks.workunit.client.0.smithi117.stdout:9/923: creat d4/de/d7b/dd0/f13a x:0 0 0 2024-04-20T11:09:19.839 INFO:tasks.workunit.client.0.smithi117.stdout:0/892: symlink d3/d7/d94/l118 0 2024-04-20T11:09:19.839 INFO:tasks.workunit.client.0.smithi117.stdout:2/939: dwrite d1/d20/de0/fe1 [0,4194304] 0 2024-04-20T11:09:19.843 INFO:tasks.workunit.client.0.smithi117.stdout:6/985: write d3/fca [581261,15108] 0 2024-04-20T11:09:19.846 INFO:tasks.workunit.client.0.smithi117.stdout:9/924: rename d4/d5/d3c/ddf/d31/lf1 to d4/d5/d57/l13b 0 2024-04-20T11:09:19.847 INFO:tasks.workunit.client.0.smithi117.stdout:2/940: read - d1/d4/d83/d29/d2e/d53/d58/ff1 zero size 2024-04-20T11:09:19.849 INFO:tasks.workunit.client.0.smithi117.stdout:6/986: readlink d3/d5a/d74/db3/df0/da3/l127 0 2024-04-20T11:09:19.851 INFO:tasks.workunit.client.0.smithi117.stdout:9/925: fsync d4/d4a/d72/dd2/d106/d10d/f135 0 2024-04-20T11:09:19.851 INFO:tasks.workunit.client.0.smithi117.stdout:6/987: chown d3/f2c 185 1 2024-04-20T11:09:19.852 INFO:tasks.workunit.client.0.smithi117.stdout:2/941: write d1/d4/d83/d29/d62/f8a [1152408,42239] 0 2024-04-20T11:09:19.854 INFO:tasks.workunit.client.0.smithi117.stdout:2/942: write d1/d9b/fb6 [9244202,7126] 0 2024-04-20T11:09:19.855 INFO:tasks.workunit.client.0.smithi117.stdout:0/893: link d3/d7/d94/cea d3/d7/d20/d39/d9a/dd0/db2/dbd/d104/c119 0 2024-04-20T11:09:19.857 INFO:tasks.workunit.client.0.smithi117.stdout:9/926: write d4/d5/d3c/ddf/d31/f5d [704920,96032] 0 2024-04-20T11:09:19.859 INFO:tasks.workunit.client.0.smithi117.stdout:0/894: fsync d3/d7/d20/d6e/df3/dff/d85/ddb/f109 0 2024-04-20T11:09:19.859 INFO:tasks.workunit.client.0.smithi117.stdout:2/943: dread - d1/d4/d83/d29/d2e/d53/de7/ff8 zero size 2024-04-20T11:09:19.862 INFO:tasks.workunit.client.0.smithi117.stdout:2/944: fdatasync d1/d4/d83/d29/f100 0 2024-04-20T11:09:19.864 INFO:tasks.workunit.client.0.smithi117.stdout:0/895: creat d3/d7/f11a x:0 0 0 2024-04-20T11:09:19.866 INFO:tasks.workunit.client.0.smithi117.stdout:9/927: creat d4/d4a/f13c x:0 0 0 2024-04-20T11:09:19.872 INFO:tasks.workunit.client.0.smithi117.stdout:6/988: dwrite d3/d4/f46 [0,4194304] 0 2024-04-20T11:09:19.874 INFO:tasks.workunit.client.0.smithi117.stdout:0/896: symlink d3/d7/d20/d39/l11b 0 2024-04-20T11:09:19.878 INFO:tasks.workunit.client.0.smithi117.stdout:2/945: dread d1/d4/f5 [4194304,4194304] 0 2024-04-20T11:09:19.880 INFO:tasks.workunit.client.0.smithi117.stdout:2/946: chown d1/da5/df2/c5b 524101 1 2024-04-20T11:09:19.887 INFO:tasks.workunit.client.0.smithi117.stdout:2/947: symlink d1/d4/d83/d29/d2e/d52/l12f 0 2024-04-20T11:09:19.888 INFO:tasks.workunit.client.0.smithi117.stdout:0/897: mknod d3/d7/d2c/c11c 0 2024-04-20T11:09:19.893 INFO:tasks.workunit.client.0.smithi117.stdout:0/898: chown d3/dab/ld6 369895 1 2024-04-20T11:09:19.897 INFO:tasks.workunit.client.0.smithi117.stdout:9/928: write d4/d35/df7/d105/fb5 [200887,56351] 0 2024-04-20T11:09:19.897 INFO:tasks.workunit.client.0.smithi117.stdout:0/899: write d3/d7/d20/d39/dbb/f107 [442176,44979] 0 2024-04-20T11:09:19.901 INFO:tasks.workunit.client.0.smithi117.stdout:9/929: chown d4/d4a/d72/dd2/d106/d10d/f135 7734 1 2024-04-20T11:09:19.910 INFO:tasks.workunit.client.0.smithi117.stdout:6/989: dwrite d3/df/d39/d99/f64 [0,4194304] 0 2024-04-20T11:09:19.919 INFO:tasks.workunit.client.0.smithi117.stdout:0/900: dread d3/d7/d20/d47/d78/ff9 [0,4194304] 0 2024-04-20T11:09:19.922 INFO:tasks.workunit.client.0.smithi117.stdout:9/930: dread d4/d4a/d5e/f75 [0,4194304] 0 2024-04-20T11:09:19.927 INFO:tasks.workunit.client.0.smithi117.stdout:2/948: dread d1/d15/d1d/f50 [0,4194304] 0 2024-04-20T11:09:19.931 INFO:tasks.workunit.client.0.smithi117.stdout:9/931: dread d4/d4a/d5e/de8/fe5 [0,4194304] 0 2024-04-20T11:09:19.936 INFO:tasks.workunit.client.0.smithi117.stdout:9/932: read d4/d4a/d63/d98/dc0/dcb/ddc/f118 [64359,92870] 0 2024-04-20T11:09:19.961 INFO:tasks.workunit.client.0.smithi117.stdout:6/990: write d3/df/d39/d99/d139/d37/f47 [4082251,121596] 0 2024-04-20T11:09:19.962 INFO:tasks.workunit.client.0.smithi117.stdout:0/901: mknod d3/d7/d20/d39/d9a/dd0/db2/dbd/c11d 0 2024-04-20T11:09:19.965 INFO:tasks.workunit.client.0.smithi117.stdout:0/902: write d3/d7/d2c/f10e [300405,96511] 0 2024-04-20T11:09:19.969 INFO:tasks.workunit.client.0.smithi117.stdout:9/933: creat d4/d4a/d5e/d9b/d122/f13d x:0 0 0 2024-04-20T11:09:19.969 INFO:tasks.workunit.client.0.smithi117.stdout:2/949: mkdir d1/d4/d83/d29/d2e/d130 0 2024-04-20T11:09:19.972 INFO:tasks.workunit.client.0.smithi117.stdout:0/903: unlink d3/d7/d20/d39/d9a/dd0/db2/dbd/lf5 0 2024-04-20T11:09:19.976 INFO:tasks.workunit.client.0.smithi117.stdout:9/934: rename d4/d5/d3c/ddf/l7f to d4/d5/d3c/ddf/l13e 0 2024-04-20T11:09:19.979 INFO:tasks.workunit.client.0.smithi117.stdout:0/904: mkdir d3/dab/d54/d11e 0 2024-04-20T11:09:19.982 INFO:tasks.workunit.client.0.smithi117.stdout:9/935: truncate d4/d4a/d72/dd2/d106/d10d/f135 860716 0 2024-04-20T11:09:19.982 INFO:tasks.workunit.client.0.smithi117.stdout:2/950: getdents d1/d4/dd 0 2024-04-20T11:09:19.984 INFO:tasks.workunit.client.0.smithi117.stdout:9/936: chown d4/d4a/d72/l7c 46136906 1 2024-04-20T11:09:19.985 INFO:tasks.workunit.client.0.smithi117.stdout:2/951: dread - d1/d4/d83/d29/d62/da6/f12d zero size 2024-04-20T11:09:19.987 INFO:tasks.workunit.client.0.smithi117.stdout:2/952: chown d1/d4/d83/d29/d3f/f105 8326676 1 2024-04-20T11:09:19.988 INFO:tasks.workunit.client.0.smithi117.stdout:9/937: read d4/d5/f119 [353311,123231] 0 2024-04-20T11:09:19.991 INFO:tasks.workunit.client.0.smithi117.stdout:9/938: mknod d4/de4/c13f 0 2024-04-20T11:09:20.001 INFO:tasks.workunit.client.0.smithi117.stdout:9/939: creat d4/d5/d3c/d7a/f140 x:0 0 0 2024-04-20T11:09:20.006 INFO:tasks.workunit.client.0.smithi117.stdout:9/940: rename d4/d4a/d63/d98/dc0/dcb/ddc/f118 to d4/d4a/d5e/d9b/f141 0 2024-04-20T11:09:20.007 INFO:tasks.workunit.client.0.smithi117.stdout:9/941: chown d4/d4a/d63 821589670 1 2024-04-20T11:09:20.010 INFO:tasks.workunit.client.0.smithi117.stdout:9/942: creat d4/d4a/d72/dd2/d106/f142 x:0 0 0 2024-04-20T11:09:20.015 INFO:tasks.workunit.client.0.smithi117.stdout:9/943: dread d4/d35/df7/d105/f93 [0,4194304] 0 2024-04-20T11:09:20.019 INFO:tasks.workunit.client.0.smithi117.stdout:9/944: chown d4/d4a/d5e/d109/f88 30827 1 2024-04-20T11:09:20.019 INFO:tasks.workunit.client.0.smithi117.stdout:6/991: dwrite d3/d3a/f61 [0,4194304] 0 2024-04-20T11:09:20.021 INFO:tasks.workunit.client.0.smithi117.stdout:6/992: fdatasync d3/df/d39/f123 0 2024-04-20T11:09:20.025 INFO:tasks.workunit.client.0.smithi117.stdout:2/953: dwrite d1/d4/d83/d29/d2e/d53/d93/f99 [0,4194304] 0 2024-04-20T11:09:20.029 INFO:tasks.workunit.client.0.smithi117.stdout:6/993: chown d3/d4/ca1 59 1 2024-04-20T11:09:20.029 INFO:tasks.workunit.client.0.smithi117.stdout:2/954: stat d1/d15/d1d/f50 0 2024-04-20T11:09:20.033 INFO:tasks.workunit.client.0.smithi117.stdout:2/955: creat d1/d9b/f131 x:0 0 0 2024-04-20T11:09:20.041 INFO:tasks.workunit.client.0.smithi117.stdout:2/956: symlink d1/d4/d83/d29/d2e/ddc/l132 0 2024-04-20T11:09:20.045 INFO:tasks.workunit.client.0.smithi117.stdout:6/994: read d3/d3a/d7f/fa4 [3583216,57901] 0 2024-04-20T11:09:20.047 INFO:tasks.workunit.client.0.smithi117.stdout:9/945: dwrite d4/d35/df7/d105/f93 [0,4194304] 0 2024-04-20T11:09:20.048 INFO:tasks.workunit.client.0.smithi117.stdout:2/957: read d1/d4/d83/d29/d2e/d53/d93/fb5 [207014,72095] 0 2024-04-20T11:09:20.051 INFO:tasks.workunit.client.0.smithi117.stdout:2/958: fdatasync d1/d20/de0/f11f 0 2024-04-20T11:09:20.065 INFO:tasks.workunit.client.0.smithi117.stdout:2/959: getdents d1/d4/d69 0 2024-04-20T11:09:20.067 INFO:tasks.workunit.client.0.smithi117.stdout:2/960: fdatasync d1/d20/de0/fe1 0 2024-04-20T11:09:20.070 INFO:tasks.workunit.client.0.smithi117.stdout:9/946: dread d4/d4a/d5e/f75 [0,4194304] 0 2024-04-20T11:09:20.073 INFO:tasks.workunit.client.0.smithi117.stdout:6/995: dwrite d3/df/d39/d99/d139/dc0/f11c [0,4194304] 0 2024-04-20T11:09:20.076 INFO:tasks.workunit.client.0.smithi117.stdout:2/961: rename d1/d4/dd/f19 to d1/d4/dd/d17/f133 0 2024-04-20T11:09:20.081 INFO:tasks.workunit.client.0.smithi117.stdout:6/996: mknod d3/c13c 0 2024-04-20T11:09:20.087 INFO:tasks.workunit.client.0.smithi117.stdout:6/997: write d3/d5a/d74/db3/dd0/f133 [445171,50178] 0 2024-04-20T11:09:20.088 INFO:tasks.workunit.client.0.smithi117.stdout:2/962: mkdir d1/d4/d134 0 2024-04-20T11:09:20.095 INFO:tasks.workunit.client.0.smithi117.stdout:9/947: dwrite d4/d35/f3d [4194304,4194304] 0 2024-04-20T11:09:20.095 INFO:tasks.workunit.client.0.smithi117.stdout:6/998: symlink d3/d5a/d74/db3/df0/da3/l13d 0 2024-04-20T11:09:20.096 INFO:tasks.workunit.client.0.smithi117.stdout:2/963: fdatasync d1/d4/d83/d29/daa/fb7 0 2024-04-20T11:09:20.101 INFO:tasks.workunit.client.0.smithi117.stdout:6/999: creat d3/df/d39/d99/f13e x:0 0 0 2024-04-20T11:09:20.104 INFO:tasks.workunit.client.0.smithi117.stdout:2/964: symlink d1/d20/de0/l135 0 2024-04-20T11:09:20.106 INFO:tasks.workunit.client.0.smithi117.stdout:9/948: fdatasync d4/d35/df7/d105/fc 0 2024-04-20T11:09:20.108 INFO:tasks.workunit.client.0.smithi117.stdout:2/965: rmdir d1/d4/d83/d29/d62 39 2024-04-20T11:09:20.110 INFO:tasks.workunit.client.0.smithi117.stdout:9/949: write d4/d4a/d72/dd2/d106/f142 [987075,37103] 0 2024-04-20T11:09:20.112 INFO:tasks.workunit.client.0.smithi117.stdout:9/950: readlink d4/d35/df7/d105/d51/ld3 0 2024-04-20T11:09:20.112 INFO:tasks.workunit.client.0.smithi117.stdout:2/966: symlink d1/d4/d83/d29/d2e/d80/l136 0 2024-04-20T11:09:20.115 INFO:tasks.workunit.client.0.smithi117.stdout:9/951: dread - d4/d5/d3c/d7a/f10c zero size 2024-04-20T11:09:20.116 INFO:tasks.workunit.client.0.smithi117.stdout:2/967: read - d1/d20/fd9 zero size 2024-04-20T11:09:20.116 INFO:tasks.workunit.client.0.smithi117.stdout:9/952: readlink d4/d5/d57/d82/dec/l12e 0 2024-04-20T11:09:20.131 INFO:tasks.workunit.client.0.smithi117.stdout:9/953: dwrite d4/d4a/d5e/f75 [0,4194304] 0 2024-04-20T11:09:20.131 INFO:tasks.workunit.client.0.smithi117.stdout:2/968: dwrite d1/f11 [4194304,4194304] 0 2024-04-20T11:09:20.135 INFO:tasks.workunit.client.0.smithi117.stdout:9/954: mknod d4/d4a/d63/d98/c143 0 2024-04-20T11:09:20.135 INFO:tasks.workunit.client.0.smithi117.stdout:2/969: creat d1/d4/ddf/f137 x:0 0 0 2024-04-20T11:09:20.137 INFO:tasks.workunit.client.0.smithi117.stdout:9/955: dread - d4/d5/f11b zero size 2024-04-20T11:09:20.141 INFO:tasks.workunit.client.0.smithi117.stdout:9/956: truncate d4/d35/f81 907202 0 2024-04-20T11:09:20.159 INFO:tasks.workunit.client.0.smithi117.stdout:9/957: read d4/f39 [436333,107926] 0 2024-04-20T11:09:20.160 INFO:tasks.workunit.client.0.smithi117.stdout:2/970: read d1/d4/d83/fc3 [250712,101845] 0 2024-04-20T11:09:20.161 INFO:tasks.workunit.client.0.smithi117.stdout:9/958: chown d4/de/d10/f9d 7 1 2024-04-20T11:09:20.163 INFO:tasks.workunit.client.0.smithi117.stdout:2/971: mknod d1/d4/d83/d29/d2e/d53/d93/c138 0 2024-04-20T11:09:20.168 INFO:tasks.workunit.client.0.smithi117.stdout:9/959: rename d4/d35/c5b to d4/d4a/d63/d98/dc0/dcb/c144 0 2024-04-20T11:09:20.170 INFO:tasks.workunit.client.0.smithi117.stdout:2/972: creat d1/d15/d1d/db3/f139 x:0 0 0 2024-04-20T11:09:20.173 INFO:tasks.workunit.client.0.smithi117.stdout:2/973: symlink d1/l13a 0 2024-04-20T11:09:20.192 INFO:tasks.workunit.client.0.smithi117.stdout:2/974: truncate d1/d4/d83/d29/d2e/d53/f103 1202339 0 2024-04-20T11:09:20.196 INFO:tasks.workunit.client.0.smithi117.stdout:9/960: dwrite d4/d68/fe9 [0,4194304] 0 2024-04-20T11:09:20.209 INFO:tasks.workunit.client.0.smithi117.stdout:2/975: dread d1/d4/d83/d29/d2e/f40 [0,4194304] 0 2024-04-20T11:09:20.216 INFO:tasks.workunit.client.0.smithi117.stdout:9/961: dread d4/f2e [0,4194304] 0 2024-04-20T11:09:20.230 INFO:tasks.workunit.client.0.smithi117.stdout:2/976: dwrite d1/d4/d83/d29/d2e/d53/de7/f120 [0,4194304] 0 2024-04-20T11:09:20.235 INFO:tasks.workunit.client.0.smithi117.stdout:2/977: mkdir d1/d15/d118/d13b 0 2024-04-20T11:09:20.241 INFO:tasks.workunit.client.0.smithi117.stdout:9/962: truncate d4/f2e 1496370 0 2024-04-20T11:09:20.242 INFO:tasks.workunit.client.0.smithi117.stdout:9/963: readlink d4/d35/l95 0 2024-04-20T11:09:20.245 INFO:tasks.workunit.client.0.smithi117.stdout:9/964: fsync d4/d5/d3c/ddf/d31/fac 0 2024-04-20T11:09:20.249 INFO:tasks.workunit.client.0.smithi117.stdout:9/965: mkdir d4/d35/df7/d145 0 2024-04-20T11:09:20.253 INFO:tasks.workunit.client.0.smithi117.stdout:9/966: rename d4/d5/f11b to d4/d4a/d72/f146 0 2024-04-20T11:09:20.258 INFO:tasks.workunit.client.0.smithi117.stdout:1/758: sync 2024-04-20T11:09:20.258 INFO:tasks.workunit.client.0.smithi117.stdout:7/812: sync 2024-04-20T11:09:20.258 INFO:tasks.workunit.client.0.smithi117.stdout:8/769: sync 2024-04-20T11:09:20.261 INFO:tasks.workunit.client.0.smithi117.stdout:9/967: symlink d4/d4a/d63/d98/dc0/dcb/l147 0 2024-04-20T11:09:20.263 INFO:tasks.workunit.client.0.smithi117.stdout:9/968: truncate d4/d5/f119 2403913 0 2024-04-20T11:09:20.265 INFO:tasks.workunit.client.0.smithi117.stdout:8/770: creat d0/d8/d39/d40/d78/f100 x:0 0 0 2024-04-20T11:09:20.269 INFO:tasks.workunit.client.0.smithi117.stdout:7/813: getdents d1/d43/d59 0 2024-04-20T11:09:20.269 INFO:tasks.workunit.client.0.smithi117.stdout:9/969: creat d4/d4a/d72/dd7/f148 x:0 0 0 2024-04-20T11:09:20.275 INFO:tasks.workunit.client.0.smithi117.stdout:7/814: rmdir d1/d4/d39 39 2024-04-20T11:09:20.276 INFO:tasks.workunit.client.0.smithi117.stdout:9/970: creat d4/d5/d3c/ddf/d31/f149 x:0 0 0 2024-04-20T11:09:20.278 INFO:tasks.workunit.client.0.smithi117.stdout:9/971: rmdir d4/d4a/d5e/d113 39 2024-04-20T11:09:20.279 INFO:tasks.workunit.client.0.smithi117.stdout:8/771: getdents d0/d8/d11/d6d 0 2024-04-20T11:09:20.282 INFO:tasks.workunit.client.0.smithi117.stdout:7/815: getdents d1/d1d/d4e/d53/db1 0 2024-04-20T11:09:20.287 INFO:tasks.workunit.client.0.smithi117.stdout:1/759: dread d6/d1f/d30/d32/f6a [0,4194304] 0 2024-04-20T11:09:20.289 INFO:tasks.workunit.client.0.smithi117.stdout:9/972: write d4/d5/d3c/ddf/d31/fac [3669025,23431] 0 2024-04-20T11:09:20.292 INFO:tasks.workunit.client.0.smithi117.stdout:1/760: getdents d6/d16/d3a/d5a/dea 0 2024-04-20T11:09:20.295 INFO:tasks.workunit.client.0.smithi117.stdout:9/973: creat d4/de/d10/d94/dd9/f14a x:0 0 0 2024-04-20T11:09:20.301 INFO:tasks.workunit.client.0.smithi117.stdout:1/761: rename d6/d35/d63/l66 to d6/d16/d3a/d5a/daf/def/df8/lfa 0 2024-04-20T11:09:20.303 INFO:tasks.workunit.client.0.smithi117.stdout:1/762: rmdir d6 39 2024-04-20T11:09:20.310 INFO:tasks.workunit.client.0.smithi117.stdout:7/816: dread d1/f10 [0,4194304] 0 2024-04-20T11:09:20.310 INFO:tasks.workunit.client.0.smithi117.stdout:9/974: dread d4/d35/f81 [0,4194304] 0 2024-04-20T11:09:20.317 INFO:tasks.workunit.client.0.smithi117.stdout:9/975: mknod d4/d35/c14b 0 2024-04-20T11:09:20.319 INFO:tasks.workunit.client.0.smithi117.stdout:8/772: dread d0/d8/d11/f1f [0,4194304] 0 2024-04-20T11:09:20.321 INFO:tasks.workunit.client.0.smithi117.stdout:9/976: unlink d4/d4a/d63/d98/dc0/c125 0 2024-04-20T11:09:20.323 INFO:tasks.workunit.client.0.smithi117.stdout:8/773: creat d0/dc/d49/f101 x:0 0 0 2024-04-20T11:09:20.325 INFO:tasks.workunit.client.0.smithi117.stdout:9/977: fsync d4/d4a/d63/f76 0 2024-04-20T11:09:20.327 INFO:tasks.workunit.client.0.smithi117.stdout:9/978: stat d4/de/d7b 0 2024-04-20T11:09:20.328 INFO:tasks.workunit.client.0.smithi117.stdout:7/817: dwrite d1/d1d/d42/faf [0,4194304] 0 2024-04-20T11:09:20.328 INFO:tasks.workunit.client.0.smithi117.stdout:8/774: mknod d0/dc/d49/dd5/c102 0 2024-04-20T11:09:20.332 INFO:tasks.workunit.client.0.smithi117.stdout:1/763: dwrite d6/d16/d3a/d5a/daf/fc5 [0,4194304] 0 2024-04-20T11:09:20.337 INFO:tasks.workunit.client.0.smithi117.stdout:7/818: unlink d1/d4/d6/d17/d3f/ffb 0 2024-04-20T11:09:20.342 INFO:tasks.workunit.client.0.smithi117.stdout:1/764: rename d6/f3c to d6/d1f/d30/d32/ffb 0 2024-04-20T11:09:20.348 INFO:tasks.workunit.client.0.smithi117.stdout:7/819: rename d1/d1d/d42/d2d/d6c/f7d to d1/d4/d6/d54/f10b 0 2024-04-20T11:09:20.351 INFO:tasks.workunit.client.0.smithi117.stdout:1/765: getdents d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7 0 2024-04-20T11:09:20.353 INFO:tasks.workunit.client.0.smithi117.stdout:7/820: rename d1/d4/d3d/l82 to d1/d1d/d77/dee/l10c 0 2024-04-20T11:09:20.356 INFO:tasks.workunit.client.0.smithi117.stdout:1/766: chown d6/d16/d3a/d5a/daf/def/df8/lfa 1585657753 1 2024-04-20T11:09:20.359 INFO:tasks.workunit.client.0.smithi117.stdout:8/775: dread d0/d8/d39/d40/d78/f97 [0,4194304] 0 2024-04-20T11:09:20.362 INFO:tasks.workunit.client.0.smithi117.stdout:8/776: write d0/dc/d49/d63/ff7 [340074,15623] 0 2024-04-20T11:09:20.367 INFO:tasks.workunit.client.0.smithi117.stdout:8/777: unlink d0/d8/d11/d24/d86/ca0 0 2024-04-20T11:09:20.370 INFO:tasks.workunit.client.0.smithi117.stdout:8/778: mknod d0/d8/d11/d6d/c103 0 2024-04-20T11:09:20.370 INFO:tasks.workunit.client.0.smithi117.stdout:1/767: write d6/d21/f86 [685164,11601] 0 2024-04-20T11:09:20.372 INFO:tasks.workunit.client.0.smithi117.stdout:1/768: truncate d6/d35/fe1 1034405 0 2024-04-20T11:09:20.374 INFO:tasks.workunit.client.0.smithi117.stdout:8/779: rename d0/dc/d49/ce7 to d0/dc/d49/c104 0 2024-04-20T11:09:20.535 INFO:tasks.workunit.client.0.smithi117.stdout:4/849: sync 2024-04-20T11:09:20.536 INFO:tasks.workunit.client.0.smithi117.stdout:3/810: sync 2024-04-20T11:09:20.539 INFO:tasks.workunit.client.0.smithi117.stdout:3/811: fsync d0/f71 0 2024-04-20T11:09:20.544 INFO:tasks.workunit.client.0.smithi117.stdout:3/812: chown d0/d5/d73/ded/d66/dbe/ff2 13 1 2024-04-20T11:09:20.549 INFO:tasks.workunit.client.0.smithi117.stdout:3/813: mkdir d0/d5/d21/d75/d89/d116 0 2024-04-20T11:09:20.558 INFO:tasks.workunit.client.0.smithi117.stdout:3/814: symlink d0/d5/d21/d88/dc2/l117 0 2024-04-20T11:09:20.581 INFO:tasks.workunit.client.0.smithi117.stdout:4/850: dread d1/d9/d6b/dca/ff8 [0,4194304] 0 2024-04-20T11:09:20.592 INFO:tasks.workunit.client.0.smithi117.stdout:0/905: sync 2024-04-20T11:09:20.592 INFO:tasks.workunit.client.0.smithi117.stdout:2/978: sync 2024-04-20T11:09:20.603 INFO:tasks.workunit.client.0.smithi117.stdout:0/906: creat d3/d7/f11f x:0 0 0 2024-04-20T11:09:20.607 INFO:tasks.workunit.client.0.smithi117.stdout:2/979: mknod d1/d4/dd/d17/c13c 0 2024-04-20T11:09:20.610 INFO:tasks.workunit.client.0.smithi117.stdout:0/907: mknod d3/d7/d20/d47/c120 0 2024-04-20T11:09:20.614 INFO:tasks.workunit.client.0.smithi117.stdout:4/851: dwrite d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:20.614 INFO:tasks.workunit.client.0.smithi117.stdout:3/815: dread d0/d5/d73/fa3 [0,4194304] 0 2024-04-20T11:09:20.616 INFO:tasks.workunit.client.0.smithi117.stdout:2/980: truncate d1/d4/fc0 950423 0 2024-04-20T11:09:20.619 INFO:tasks.workunit.client.0.smithi117.stdout:0/908: symlink d3/d5/d116/l121 0 2024-04-20T11:09:20.623 INFO:tasks.workunit.client.0.smithi117.stdout:4/852: mkdir d1/dd/d1a/d2b/d86/d11b 0 2024-04-20T11:09:20.625 INFO:tasks.workunit.client.0.smithi117.stdout:4/853: dread - d1/d9/d79/ff7 zero size 2024-04-20T11:09:20.625 INFO:tasks.workunit.client.0.smithi117.stdout:0/909: truncate d3/d7/d16/f2e 5472608 0 2024-04-20T11:09:20.630 INFO:tasks.workunit.client.0.smithi117.stdout:3/816: fdatasync d0/d5/d73/ded/dac/dd2/fdc 0 2024-04-20T11:09:20.633 INFO:tasks.workunit.client.0.smithi117.stdout:0/910: fdatasync d3/d7/d20/d39/f92 0 2024-04-20T11:09:20.637 INFO:tasks.workunit.client.0.smithi117.stdout:0/911: mkdir d3/d7e/d122 0 2024-04-20T11:09:20.639 INFO:tasks.workunit.client.0.smithi117.stdout:2/981: dread d1/d4/d83/d29/d2e/d53/d93/f99 [0,4194304] 0 2024-04-20T11:09:20.653 INFO:tasks.workunit.client.0.smithi117.stdout:2/982: write d1/d4/dd/d17/f1f [6893109,115255] 0 2024-04-20T11:09:20.659 INFO:tasks.workunit.client.0.smithi117.stdout:4/854: dwrite d1/d9/de9/fa0 [0,4194304] 0 2024-04-20T11:09:20.660 INFO:tasks.workunit.client.0.smithi117.stdout:0/912: dwrite d3/d7/d20/d39/d9a/dd0/d6d/fb4 [0,4194304] 0 2024-04-20T11:09:20.660 INFO:tasks.workunit.client.0.smithi117.stdout:2/983: creat d1/d4/d83/d29/d2e/f13d x:0 0 0 2024-04-20T11:09:20.661 INFO:tasks.workunit.client.0.smithi117.stdout:4/855: chown d1/d9/d6b 1589859 1 2024-04-20T11:09:20.664 INFO:tasks.workunit.client.0.smithi117.stdout:8/780: sync 2024-04-20T11:09:20.665 INFO:tasks.workunit.client.0.smithi117.stdout:9/979: sync 2024-04-20T11:09:20.665 INFO:tasks.workunit.client.0.smithi117.stdout:7/821: sync 2024-04-20T11:09:20.665 INFO:tasks.workunit.client.0.smithi117.stdout:1/769: sync 2024-04-20T11:09:20.668 INFO:tasks.workunit.client.0.smithi117.stdout:2/984: rmdir d1/d4/d83/d29/d2e/d53/df0 39 2024-04-20T11:09:20.676 INFO:tasks.workunit.client.0.smithi117.stdout:7/822: creat d1/d43/d59/d8a/f10d x:0 0 0 2024-04-20T11:09:20.676 INFO:tasks.workunit.client.0.smithi117.stdout:1/770: write d6/d16/d3a/d5a/d5c/d61/d71/d6b/fd5 [649882,116341] 0 2024-04-20T11:09:20.679 INFO:tasks.workunit.client.0.smithi117.stdout:0/913: unlink d3/d7/d20/d39/d9a/dd0/lc3 0 2024-04-20T11:09:20.679 INFO:tasks.workunit.client.0.smithi117.stdout:1/771: chown d6/d16/cda 161207934 1 2024-04-20T11:09:20.683 INFO:tasks.workunit.client.0.smithi117.stdout:2/985: symlink d1/d4/d83/d29/d3f/df6/l13e 0 2024-04-20T11:09:20.685 INFO:tasks.workunit.client.0.smithi117.stdout:7/823: fdatasync d1/d43/fbf 0 2024-04-20T11:09:20.685 INFO:tasks.workunit.client.0.smithi117.stdout:4/856: dwrite d1/f67 [0,4194304] 0 2024-04-20T11:09:20.688 INFO:tasks.workunit.client.0.smithi117.stdout:9/980: creat d4/d5/d57/d82/da5/dc5/f14c x:0 0 0 2024-04-20T11:09:20.690 INFO:tasks.workunit.client.0.smithi117.stdout:9/981: write d4/d4a/d5e/de8/f6d [3977758,110240] 0 2024-04-20T11:09:20.695 INFO:tasks.workunit.client.0.smithi117.stdout:2/986: chown d1/d4/d83/f34 6 1 2024-04-20T11:09:20.700 INFO:tasks.workunit.client.0.smithi117.stdout:7/824: chown d1/d4/d6/d2b/d64/f8c 582331 1 2024-04-20T11:09:20.701 INFO:tasks.workunit.client.0.smithi117.stdout:7/825: truncate d1/d1d/d77/fe9 769711 0 2024-04-20T11:09:20.703 INFO:tasks.workunit.client.0.smithi117.stdout:9/982: symlink d4/d5/d57/d82/da5/l14d 0 2024-04-20T11:09:20.712 INFO:tasks.workunit.client.0.smithi117.stdout:2/987: rename d1/d20/de0/f11f to d1/d4/d83/d29/d3f/df6/d119/f13f 0 2024-04-20T11:09:20.714 INFO:tasks.workunit.client.0.smithi117.stdout:7/826: unlink d1/d4/d6/d54/d63/f85 0 2024-04-20T11:09:20.719 INFO:tasks.workunit.client.0.smithi117.stdout:4/857: creat d1/dd/d1a/d43/f11c x:0 0 0 2024-04-20T11:09:20.727 INFO:tasks.workunit.client.0.smithi117.stdout:7/827: creat d1/d1d/d4e/d53/f10e x:0 0 0 2024-04-20T11:09:20.727 INFO:tasks.workunit.client.0.smithi117.stdout:8/781: truncate d0/d8/d11/d24/d27/d47/f6c 4045238 0 2024-04-20T11:09:20.731 INFO:tasks.workunit.client.0.smithi117.stdout:1/772: dwrite d6/d1f/d30/f4e [0,4194304] 0 2024-04-20T11:09:20.731 INFO:tasks.workunit.client.0.smithi117.stdout:0/914: write d3/d7/d94/fa5 [6322,35482] 0 2024-04-20T11:09:20.738 INFO:tasks.workunit.client.0.smithi117.stdout:4/858: creat d1/d45/d110/f11d x:0 0 0 2024-04-20T11:09:20.740 INFO:tasks.workunit.client.0.smithi117.stdout:4/859: readlink d1/dd/d1a/d43/dcc/lf5 0 2024-04-20T11:09:20.740 INFO:tasks.workunit.client.0.smithi117.stdout:7/828: symlink d1/d4/d3d/l10f 0 2024-04-20T11:09:20.743 INFO:tasks.workunit.client.0.smithi117.stdout:4/860: dread - d1/d9/d79/f105 zero size 2024-04-20T11:09:20.746 INFO:tasks.workunit.client.0.smithi117.stdout:4/861: write d1/d9/d1f/d87/fb3 [4940481,78538] 0 2024-04-20T11:09:20.748 INFO:tasks.workunit.client.0.smithi117.stdout:1/773: symlink d6/d16/d3a/d5a/daf/def/lfc 0 2024-04-20T11:09:20.755 INFO:tasks.workunit.client.0.smithi117.stdout:7/829: symlink d1/d1d/d4e/d53/d55/l110 0 2024-04-20T11:09:20.758 INFO:tasks.workunit.client.0.smithi117.stdout:7/830: chown d1/d4/d6/d54/d63/l68 0 1 2024-04-20T11:09:20.762 INFO:tasks.workunit.client.0.smithi117.stdout:2/988: write d1/d4/d83/d29/d2e/fe9 [3444505,57325] 0 2024-04-20T11:09:20.763 INFO:tasks.workunit.client.0.smithi117.stdout:7/831: read d1/d4/d3d/f5a [741892,681] 0 2024-04-20T11:09:20.765 INFO:tasks.workunit.client.0.smithi117.stdout:9/983: dwrite d4/d5/f2a [0,4194304] 0 2024-04-20T11:09:20.779 INFO:tasks.workunit.client.0.smithi117.stdout:9/984: rmdir d4/d4a/d5e/d9b/d122 39 2024-04-20T11:09:20.781 INFO:tasks.workunit.client.0.smithi117.stdout:8/782: dread d0/d9/ff8 [0,4194304] 0 2024-04-20T11:09:20.784 INFO:tasks.workunit.client.0.smithi117.stdout:1/774: dwrite d6/d1f/d30/d49/d69/d6e/d6f/d73/f88 [0,4194304] 0 2024-04-20T11:09:20.787 INFO:tasks.workunit.client.0.smithi117.stdout:1/775: write d6/d21/f86 [1428805,117836] 0 2024-04-20T11:09:20.789 INFO:tasks.workunit.client.0.smithi117.stdout:4/862: dread d1/dd/d1a/d2b/d86/f102 [0,4194304] 0 2024-04-20T11:09:20.789 INFO:tasks.workunit.client.0.smithi117.stdout:1/776: readlink d6/d16/l90 0 2024-04-20T11:09:20.789 INFO:tasks.workunit.client.0.smithi117.stdout:2/989: fdatasync d1/d4/dd/f55 0 2024-04-20T11:09:20.790 INFO:tasks.workunit.client.0.smithi117.stdout:4/863: stat d1/d45/c63 0 2024-04-20T11:09:20.798 INFO:tasks.workunit.client.0.smithi117.stdout:7/832: dread d1/d43/f73 [0,4194304] 0 2024-04-20T11:09:20.799 INFO:tasks.workunit.client.0.smithi117.stdout:8/783: getdents d0/d9/d3a/de3/df1 0 2024-04-20T11:09:20.800 INFO:tasks.workunit.client.0.smithi117.stdout:7/833: chown d1/d1d/d4e/d72/ce0 12 1 2024-04-20T11:09:20.801 INFO:tasks.workunit.client.0.smithi117.stdout:4/864: chown d1/dd/d1a/d43/dcc/cfc 96796233 1 2024-04-20T11:09:20.806 INFO:tasks.workunit.client.0.smithi117.stdout:3/817: sync 2024-04-20T11:09:20.811 INFO:tasks.workunit.client.0.smithi117.stdout:4/865: fsync d1/fa7 0 2024-04-20T11:09:20.823 INFO:tasks.workunit.client.0.smithi117.stdout:3/818: mkdir d0/d5/de/d4e/dec/d118 0 2024-04-20T11:09:20.829 INFO:tasks.workunit.client.0.smithi117.stdout:8/784: write d0/dc/f25 [1627724,93131] 0 2024-04-20T11:09:20.831 INFO:tasks.workunit.client.0.smithi117.stdout:1/777: dwrite d6/d1f/d30/fc9 [0,4194304] 0 2024-04-20T11:09:20.838 INFO:tasks.workunit.client.0.smithi117.stdout:2/990: dwrite d1/d15/d1d/fbf [0,4194304] 0 2024-04-20T11:09:20.838 INFO:tasks.workunit.client.0.smithi117.stdout:4/866: write d1/d9/de9/fc0 [328347,52393] 0 2024-04-20T11:09:20.844 INFO:tasks.workunit.client.0.smithi117.stdout:2/991: chown d1/d15/d1d/db3/c107 103339 1 2024-04-20T11:09:20.845 INFO:tasks.workunit.client.0.smithi117.stdout:1/778: mkdir d6/d1f/d30/dfd 0 2024-04-20T11:09:20.848 INFO:tasks.workunit.client.0.smithi117.stdout:4/867: symlink d1/d45/d110/l11e 0 2024-04-20T11:09:20.849 INFO:tasks.workunit.client.0.smithi117.stdout:7/834: dread d1/fa5 [0,4194304] 0 2024-04-20T11:09:20.856 INFO:tasks.workunit.client.0.smithi117.stdout:4/868: symlink d1/d45/d4a/d59/l11f 0 2024-04-20T11:09:20.858 INFO:tasks.workunit.client.0.smithi117.stdout:7/835: mkdir d1/d43/d59/d111 0 2024-04-20T11:09:20.859 INFO:tasks.workunit.client.0.smithi117.stdout:3/819: write d0/d5/d73/ded/f30 [242987,107170] 0 2024-04-20T11:09:20.862 INFO:tasks.workunit.client.0.smithi117.stdout:3/820: chown d0/f1a 11 1 2024-04-20T11:09:20.862 INFO:tasks.workunit.client.0.smithi117.stdout:8/785: rmdir d0/d9/d3a/de3/dff 0 2024-04-20T11:09:20.864 INFO:tasks.workunit.client.0.smithi117.stdout:7/836: read d1/d1d/d77/fc5 [1623306,38304] 0 2024-04-20T11:09:20.867 INFO:tasks.workunit.client.0.smithi117.stdout:2/992: rmdir d1/d4/d83/d29/d62/da6 39 2024-04-20T11:09:20.869 INFO:tasks.workunit.client.0.smithi117.stdout:2/993: chown d1/d4/d69/cac 0 1 2024-04-20T11:09:20.874 INFO:tasks.workunit.client.0.smithi117.stdout:8/786: rename d0/d8/d39/d7f to d0/d9/d3a/d105 0 2024-04-20T11:09:20.876 INFO:tasks.workunit.client.0.smithi117.stdout:1/779: creat d6/d16/d3a/d5a/d5c/ffe x:0 0 0 2024-04-20T11:09:20.882 INFO:tasks.workunit.client.0.smithi117.stdout:3/821: rename d0/d5/d21/d94/da7/le8 to d0/d5/d21/d88/d36/l119 0 2024-04-20T11:09:20.887 INFO:tasks.workunit.client.0.smithi117.stdout:7/837: creat d1/d4/f112 x:0 0 0 2024-04-20T11:09:20.890 INFO:tasks.workunit.client.0.smithi117.stdout:8/787: read d0/d8/d11/d6d/d69/f85 [2736837,1623] 0 2024-04-20T11:09:20.896 INFO:tasks.workunit.client.0.smithi117.stdout:4/869: write d1/d9/f96 [1607019,115388] 0 2024-04-20T11:09:20.899 INFO:tasks.workunit.client.0.smithi117.stdout:1/780: creat d6/d1f/d30/d49/d69/fff x:0 0 0 2024-04-20T11:09:20.900 INFO:tasks.workunit.client.0.smithi117.stdout:2/994: dread d1/da5/df2/d46/fe4 [0,4194304] 0 2024-04-20T11:09:20.902 INFO:tasks.workunit.client.0.smithi117.stdout:1/781: chown d6/d21/db9/ld7 7 1 2024-04-20T11:09:20.907 INFO:tasks.workunit.client.0.smithi117.stdout:3/822: dwrite d0/d5/d21/d94/d9a/feb [0,4194304] 0 2024-04-20T11:09:20.909 INFO:tasks.workunit.client.0.smithi117.stdout:3/823: chown d0/d2a/f3b 1 1 2024-04-20T11:09:20.912 INFO:tasks.workunit.client.0.smithi117.stdout:2/995: fsync d1/d4/d83/d29/f115 0 2024-04-20T11:09:20.924 INFO:tasks.workunit.client.0.smithi117.stdout:3/824: fdatasync d0/d5/d73/ded/d66/d63/fa4 0 2024-04-20T11:09:20.932 INFO:tasks.workunit.client.0.smithi117.stdout:4/870: dwrite d1/dd/d1a/d2b/d86/f112 [0,4194304] 0 2024-04-20T11:09:20.934 INFO:tasks.workunit.client.0.smithi117.stdout:1/782: write d6/f68 [7522017,123356] 0 2024-04-20T11:09:20.935 INFO:tasks.workunit.client.0.smithi117.stdout:7/838: dread d1/dd1/f104 [0,4194304] 0 2024-04-20T11:09:20.937 INFO:tasks.workunit.client.0.smithi117.stdout:0/915: sync 2024-04-20T11:09:20.937 INFO:tasks.workunit.client.0.smithi117.stdout:9/985: sync 2024-04-20T11:09:20.944 INFO:tasks.workunit.client.0.smithi117.stdout:8/788: dwrite d0/fa [0,4194304] 0 2024-04-20T11:09:20.951 INFO:tasks.workunit.client.0.smithi117.stdout:3/825: dread d0/d2a/fb0 [0,4194304] 0 2024-04-20T11:09:20.976 INFO:tasks.workunit.client.0.smithi117.stdout:7/839: creat d1/d4/d6/d54/db4/f113 x:0 0 0 2024-04-20T11:09:20.979 INFO:tasks.workunit.client.0.smithi117.stdout:8/789: fsync d0/d8/d39/d40/d78/f98 0 2024-04-20T11:09:20.982 INFO:tasks.workunit.client.0.smithi117.stdout:2/996: creat d1/d4/d134/f140 x:0 0 0 2024-04-20T11:09:20.985 INFO:tasks.workunit.client.0.smithi117.stdout:0/916: symlink d3/d7/d94/l123 0 2024-04-20T11:09:20.986 INFO:tasks.workunit.client.0.smithi117.stdout:1/783: getdents d6/d1f/d30/d49/d64 0 2024-04-20T11:09:20.988 INFO:tasks.workunit.client.0.smithi117.stdout:7/840: creat d1/d1d/d42/d2d/f114 x:0 0 0 2024-04-20T11:09:20.990 INFO:tasks.workunit.client.0.smithi117.stdout:7/841: chown d1/d4/d3d/dce 2023 1 2024-04-20T11:09:20.995 INFO:tasks.workunit.client.0.smithi117.stdout:8/790: truncate d0/d9/d3a/fd4 155801 0 2024-04-20T11:09:20.998 INFO:tasks.workunit.client.0.smithi117.stdout:9/986: creat d4/de/d10/f14e x:0 0 0 2024-04-20T11:09:21.005 INFO:tasks.workunit.client.0.smithi117.stdout:2/997: symlink d1/d15/d1d/l141 0 2024-04-20T11:09:21.007 INFO:tasks.workunit.client.0.smithi117.stdout:7/842: creat d1/d4/d3d/dce/f115 x:0 0 0 2024-04-20T11:09:21.008 INFO:tasks.workunit.client.0.smithi117.stdout:4/871: getdents d1/d9/d6b/dca/de7 0 2024-04-20T11:09:21.010 INFO:tasks.workunit.client.0.smithi117.stdout:7/843: readlink d1/d1d/d42/d2d/l50 0 2024-04-20T11:09:21.011 INFO:tasks.workunit.client.0.smithi117.stdout:1/784: dread d6/d16/d3a/da8/fae [0,4194304] 0 2024-04-20T11:09:21.011 INFO:tasks.workunit.client.0.smithi117.stdout:9/987: rmdir d4/de4 39 2024-04-20T11:09:21.020 INFO:tasks.workunit.client.0.smithi117.stdout:0/917: getdents d3/d5/d116 0 2024-04-20T11:09:21.022 INFO:tasks.workunit.client.0.smithi117.stdout:7/844: symlink d1/d1d/d77/dee/l116 0 2024-04-20T11:09:21.022 INFO:tasks.workunit.client.0.smithi117.stdout:1/785: rename d6/d1f/d30/f4e to d6/dd2/f100 0 2024-04-20T11:09:21.026 INFO:tasks.workunit.client.0.smithi117.stdout:0/918: stat d3/d7/d20/d39/d3a/db1/fc4 0 2024-04-20T11:09:21.042 INFO:tasks.workunit.client.0.smithi117.stdout:7/845: truncate d1/d43/f61 408946 0 2024-04-20T11:09:21.042 INFO:tasks.workunit.client.0.smithi117.stdout:4/872: getdents d1/d45/d4a/d59 0 2024-04-20T11:09:21.042 INFO:tasks.workunit.client.0.smithi117.stdout:8/791: dread d0/d8/d39/d40/d78/d5c/fe6 [0,4194304] 0 2024-04-20T11:09:21.043 INFO:tasks.workunit.client.0.smithi117.stdout:0/919: stat d3/d7/d94/la2 0 2024-04-20T11:09:21.044 INFO:tasks.workunit.client.0.smithi117.stdout:0/920: chown d3/d7/f11f 2 1 2024-04-20T11:09:21.046 INFO:tasks.workunit.client.0.smithi117.stdout:3/826: write d0/d5/de/d4e/f9e [1645504,103750] 0 2024-04-20T11:09:21.048 INFO:tasks.workunit.client.0.smithi117.stdout:2/998: dread d1/d9b/fb6 [4194304,4194304] 0 2024-04-20T11:09:21.049 INFO:tasks.workunit.client.0.smithi117.stdout:7/846: mkdir d1/d4/d3d/d93/d117 0 2024-04-20T11:09:21.055 INFO:tasks.workunit.client.0.smithi117.stdout:4/873: read d1/dd/d1a/d2b/d86/fb4 [102273,14746] 0 2024-04-20T11:09:21.058 INFO:tasks.workunit.client.0.smithi117.stdout:9/988: dread d4/de/d10/f5c [0,4194304] 0 2024-04-20T11:09:21.058 INFO:tasks.workunit.client.0.smithi117.stdout:2/999: chown d1/d15/d1d/db3 3697357 1 2024-04-20T11:09:21.058 INFO:tasks.workunit.client.0.smithi117.stdout:7/847: fdatasync d1/d1d/f5c 0 2024-04-20T11:09:21.061 INFO:tasks.workunit.client.0.smithi117.stdout:8/792: chown d0/d8/d11/d6d/l6f 116 1 2024-04-20T11:09:21.061 INFO:tasks.workunit.client.0.smithi117.stdout:3/827: truncate d0/d5/d21/f104 353333 0 2024-04-20T11:09:21.064 INFO:tasks.workunit.client.0.smithi117.stdout:1/786: dwrite d6/d16/d3a/d5a/d5c/d61/d98/fed [0,4194304] 0 2024-04-20T11:09:21.064 INFO:tasks.workunit.client.0.smithi117.stdout:9/989: dread - d4/d35/df7/d105/d51/ff4 zero size 2024-04-20T11:09:21.064 INFO:tasks.workunit.client.0.smithi117.stdout:0/921: truncate d3/d7/d20/d6e/df3/dff/d85/ddb/fc9 362024 0 2024-04-20T11:09:21.067 INFO:tasks.workunit.client.0.smithi117.stdout:4/874: write d1/dd/d1a/d43/f11c [152035,27902] 0 2024-04-20T11:09:21.074 INFO:tasks.workunit.client.0.smithi117.stdout:7/848: mkdir d1/d4/d6/d13/d118 0 2024-04-20T11:09:21.074 INFO:tasks.workunit.client.0.smithi117.stdout:7/849: chown d1/d1d/d42/c7e 96244 1 2024-04-20T11:09:21.083 INFO:tasks.workunit.client.0.smithi117.stdout:0/922: symlink d3/d7/d20/d39/d3a/l124 0 2024-04-20T11:09:21.087 INFO:tasks.workunit.client.0.smithi117.stdout:4/875: rmdir d1/d45/d4a 39 2024-04-20T11:09:21.088 INFO:tasks.workunit.client.0.smithi117.stdout:9/990: rename d4/d35/f81 to d4/d5/d57/d82/dec/f14f 0 2024-04-20T11:09:21.093 INFO:tasks.workunit.client.0.smithi117.stdout:7/850: mknod d1/d43/d59/d111/c119 0 2024-04-20T11:09:21.096 INFO:tasks.workunit.client.0.smithi117.stdout:4/876: rmdir d1/d9/d1f/d90/d107 39 2024-04-20T11:09:21.099 INFO:tasks.workunit.client.0.smithi117.stdout:7/851: creat d1/d4/d6/d17/f11a x:0 0 0 2024-04-20T11:09:21.101 INFO:tasks.workunit.client.0.smithi117.stdout:4/877: truncate d1/d9/d6b/f109 409899 0 2024-04-20T11:09:21.106 INFO:tasks.workunit.client.0.smithi117.stdout:7/852: creat d1/d1d/d77/f11b x:0 0 0 2024-04-20T11:09:21.113 INFO:tasks.workunit.client.0.smithi117.stdout:4/878: creat d1/d9/f120 x:0 0 0 2024-04-20T11:09:21.114 INFO:tasks.workunit.client.0.smithi117.stdout:3/828: write d0/d5/d73/ded/db3/fb6 [778443,90060] 0 2024-04-20T11:09:21.118 INFO:tasks.workunit.client.0.smithi117.stdout:7/853: mkdir d1/d43/d59/d8a/d101/d11c 0 2024-04-20T11:09:21.127 INFO:tasks.workunit.client.0.smithi117.stdout:3/829: mknod d0/d5/d21/d77/df3/c11a 0 2024-04-20T11:09:21.127 INFO:tasks.workunit.client.0.smithi117.stdout:4/879: mknod d1/dd/d1a/d43/d108/c121 0 2024-04-20T11:09:21.133 INFO:tasks.workunit.client.0.smithi117.stdout:7/854: dread - d1/d4/d6/fd2 zero size 2024-04-20T11:09:21.134 INFO:tasks.workunit.client.0.smithi117.stdout:8/793: dwrite d0/d8/d11/d6d/d69/f85 [0,4194304] 0 2024-04-20T11:09:21.139 INFO:tasks.workunit.client.0.smithi117.stdout:4/880: chown d1/dd/d1a/f8b 2921 1 2024-04-20T11:09:21.141 INFO:tasks.workunit.client.0.smithi117.stdout:1/787: dwrite d6/d1f/d30/d49/d69/f82 [0,4194304] 0 2024-04-20T11:09:21.142 INFO:tasks.workunit.client.0.smithi117.stdout:4/881: stat d1/d9/f120 0 2024-04-20T11:09:21.142 INFO:tasks.workunit.client.0.smithi117.stdout:7/855: fdatasync d1/d1d/d4e/d72/fc6 0 2024-04-20T11:09:21.146 INFO:tasks.workunit.client.0.smithi117.stdout:3/830: symlink d0/d5/d73/ded/d66/d63/l11b 0 2024-04-20T11:09:21.149 INFO:tasks.workunit.client.0.smithi117.stdout:0/923: dwrite d3/d7/d20/d47/fc2 [0,4194304] 0 2024-04-20T11:09:21.152 INFO:tasks.workunit.client.0.smithi117.stdout:1/788: chown d6/d16/c1b 54530695 1 2024-04-20T11:09:21.155 INFO:tasks.workunit.client.0.smithi117.stdout:9/991: dread d4/d5/d57/fca [0,4194304] 0 2024-04-20T11:09:21.158 INFO:tasks.workunit.client.0.smithi117.stdout:8/794: fdatasync d0/d8/d39/d40/d78/f9b 0 2024-04-20T11:09:21.159 INFO:tasks.workunit.client.0.smithi117.stdout:0/924: symlink d3/d5/l125 0 2024-04-20T11:09:21.160 INFO:tasks.workunit.client.0.smithi117.stdout:0/925: chown d3/dab/d54/fe0 127717 1 2024-04-20T11:09:21.166 INFO:tasks.workunit.client.0.smithi117.stdout:3/831: rmdir d0/d5/d21/d88 39 2024-04-20T11:09:21.174 INFO:tasks.workunit.client.0.smithi117.stdout:9/992: mknod d4/d5/d3c/ddf/d31/d73/c150 0 2024-04-20T11:09:21.178 INFO:tasks.workunit.client.0.smithi117.stdout:4/882: dwrite d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:21.181 INFO:tasks.workunit.client.0.smithi117.stdout:0/926: creat d3/d7/d20/d47/d78/f126 x:0 0 0 2024-04-20T11:09:21.184 INFO:tasks.workunit.client.0.smithi117.stdout:4/883: chown d1/dd/d1a/d43/d108/f113 3 1 2024-04-20T11:09:21.184 INFO:tasks.workunit.client.0.smithi117.stdout:0/927: write d3/d5/f114 [916449,103763] 0 2024-04-20T11:09:21.192 INFO:tasks.workunit.client.0.smithi117.stdout:3/832: dwrite d0/d5/d21/d94/d9a/dc9/ff9 [0,4194304] 0 2024-04-20T11:09:21.199 INFO:tasks.workunit.client.0.smithi117.stdout:0/928: symlink d3/d7/d20/d6e/l127 0 2024-04-20T11:09:21.199 INFO:tasks.workunit.client.0.smithi117.stdout:7/856: getdents d1/dd1 0 2024-04-20T11:09:21.201 INFO:tasks.workunit.client.0.smithi117.stdout:3/833: stat d0/d5/d21/d94 0 2024-04-20T11:09:21.216 INFO:tasks.workunit.client.0.smithi117.stdout:7/857: mkdir d1/d4/d3d/d93/d117/d11d 0 2024-04-20T11:09:21.224 INFO:tasks.workunit.client.0.smithi117.stdout:4/884: link d1/d9/c11 d1/dd/d1a/d43/d4e/c122 0 2024-04-20T11:09:21.224 INFO:tasks.workunit.client.0.smithi117.stdout:1/789: write d6/d1f/d30/f74 [3938296,4167] 0 2024-04-20T11:09:21.232 INFO:tasks.workunit.client.0.smithi117.stdout:0/929: write d3/d7/d20/d39/d3a/db1/fce [856390,121746] 0 2024-04-20T11:09:21.233 INFO:tasks.workunit.client.0.smithi117.stdout:8/795: dread d0/f8f [0,4194304] 0 2024-04-20T11:09:21.234 INFO:tasks.workunit.client.0.smithi117.stdout:4/885: rename d1/d9/d1f/f103 to d1/dd/d1a/d43/d4e/f123 0 2024-04-20T11:09:21.240 INFO:tasks.workunit.client.0.smithi117.stdout:8/796: chown d0/dc/d2e/l77 371438774 1 2024-04-20T11:09:21.240 INFO:tasks.workunit.client.0.smithi117.stdout:0/930: mknod d3/d7/d16/c128 0 2024-04-20T11:09:21.241 INFO:tasks.workunit.client.0.smithi117.stdout:4/886: creat d1/dd/d1a/d43/d108/f124 x:0 0 0 2024-04-20T11:09:21.243 INFO:tasks.workunit.client.0.smithi117.stdout:7/858: write d1/d4/f30 [5183687,5824] 0 2024-04-20T11:09:21.248 INFO:tasks.workunit.client.0.smithi117.stdout:9/993: dwrite d4/d5/d57/fca [4194304,4194304] 0 2024-04-20T11:09:21.249 INFO:tasks.workunit.client.0.smithi117.stdout:8/797: chown d0/d8/d11/d24/d2b/f2c 1 1 2024-04-20T11:09:21.255 INFO:tasks.workunit.client.0.smithi117.stdout:3/834: dwrite d0/d5/d21/d94/d9a/fba [0,4194304] 0 2024-04-20T11:09:21.256 INFO:tasks.workunit.client.0.smithi117.stdout:9/994: chown d4/d4a/d72/dd2/d106/d10d/f135 370098 1 2024-04-20T11:09:21.264 INFO:tasks.workunit.client.0.smithi117.stdout:1/790: dwrite d6/d16/d3a/d5a/d5c/ffe [0,4194304] 0 2024-04-20T11:09:21.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.264+0000 7f8dd3e00700 1 -- 172.21.15.117:0/3571723194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc10a810 msgr2=0x7f8dcc10cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.264+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/3571723194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc10a810 0x7f8dcc10cc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8db800baf0 tx=0x7f8db800be00 comp rx=0 tx=0).stop 2024-04-20T11:09:21.265 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:21 smithi117 ceph-mon[15659]: pgmap v20: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 49 MiB/s rd, 101 MiB/s wr, 265 op/s 2024-04-20T11:09:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.264+0000 7f8dd3e00700 1 -- 172.21.15.117:0/3571723194 shutdown_connections 2024-04-20T11:09:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.264+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/3571723194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc10a810 0x7f8dcc10cc60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.264+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/3571723194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dcc107ee0 0x7f8dcc10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.264+0000 7f8dd3e00700 1 -- 172.21.15.117:0/3571723194 >> 172.21.15.117:0/3571723194 conn(0x7f8dcc06de40 msgr2=0x7f8dcc070260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.266+0000 7f8dd3e00700 1 -- 172.21.15.117:0/3571723194 shutdown_connections 2024-04-20T11:09:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.266+0000 7f8dd3e00700 1 -- 172.21.15.117:0/3571723194 wait complete. 2024-04-20T11:09:21.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.270+0000 7f8dd3e00700 1 Processor -- start 2024-04-20T11:09:21.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.271+0000 7f8dd3e00700 1 -- start start 2024-04-20T11:09:21.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.272+0000 7f8dd3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc107ee0 0x7f8dcc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:21.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.272+0000 7f8dd3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dcc10a810 0x7f8dcc19d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:21.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.272+0000 7f8dd3e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc19d6d0 con 0x7f8dcc10a810 2024-04-20T11:09:21.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.272+0000 7f8dd3e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc19d810 con 0x7f8dcc107ee0 2024-04-20T11:09:21.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.273+0000 7f8dd0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dcc10a810 0x7f8dcc19d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:21.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.273+0000 7f8dd1400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc107ee0 0x7f8dcc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:21.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.273+0000 7f8dd0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dcc10a810 0x7f8dcc19d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50470/0 (socket says 172.21.15.117:50470) 2024-04-20T11:09:21.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.273+0000 7f8dd1400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc107ee0 0x7f8dcc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:47666/0 (socket says 172.21.15.117:47666) 2024-04-20T11:09:21.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.273+0000 7f8dd1400700 1 -- 172.21.15.117:0/1083632959 learned_addr learned my addr 172.21.15.117:0/1083632959 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:21.274 INFO:tasks.workunit.client.0.smithi117.stdout:0/931: dwrite d3/d7/d20/d6e/df3/dff/d85/f10b [0,4194304] 0 2024-04-20T11:09:21.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.274+0000 7f8dd0a00700 1 -- 172.21.15.117:0/1083632959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc107ee0 msgr2=0x7f8dcc19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:21.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.274+0000 7f8dd0a00700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc107ee0 0x7f8dcc19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:21.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.274+0000 7f8dd0a00700 1 -- 172.21.15.117:0/1083632959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db80095f0 con 0x7f8dcc10a810 2024-04-20T11:09:21.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.274+0000 7f8dd1400700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc107ee0 0x7f8dcc19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:09:21.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.276+0000 7f8dd0a00700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dcc10a810 0x7f8dcc19d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8db80098e0 tx=0x7f8db800b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:21.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.277+0000 7f8dca000700 1 -- 172.21.15.117:0/1083632959 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db801d070 con 0x7f8dcc10a810 2024-04-20T11:09:21.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.277+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc1b3280 con 0x7f8dcc10a810 2024-04-20T11:09:21.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.277+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc1b3770 con 0x7f8dcc10a810 2024-04-20T11:09:21.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.278+0000 7f8dca000700 1 -- 172.21.15.117:0/1083632959 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8db80056c0 con 0x7f8dcc10a810 2024-04-20T11:09:21.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.278+0000 7f8dca000700 1 -- 172.21.15.117:0/1083632959 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db8017630 con 0x7f8dcc10a810 2024-04-20T11:09:21.279 INFO:tasks.workunit.client.0.smithi117.stdout:3/835: unlink d0/d5/d7/l2e 0 2024-04-20T11:09:21.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.280+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac0052f0 con 0x7f8dcc10a810 2024-04-20T11:09:21.279 INFO:tasks.workunit.client.0.smithi117.stdout:8/798: write d0/d8/d39/d40/f51 [397195,49840] 0 2024-04-20T11:09:21.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.282+0000 7f8dca000700 1 -- 172.21.15.117:0/1083632959 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7f8db800fae0 con 0x7f8dcc10a810 2024-04-20T11:09:21.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.286+0000 7f8dca000700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8db406eac0 0x7f8db4070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:21.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.286+0000 7f8dca000700 1 -- 172.21.15.117:0/1083632959 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f8db808ff70 con 0x7f8dcc10a810 2024-04-20T11:09:21.287 INFO:tasks.workunit.client.0.smithi117.stdout:1/791: rename d6/d16/d3a/d5a/d5c/d61/d71/d7d to d6/d1f/d30/dfd/d101 0 2024-04-20T11:09:21.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.289+0000 7f8dd1400700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8db406eac0 0x7f8db4070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:21.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.291+0000 7f8dd1400700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8db406eac0 0x7f8db4070f80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000f4d0 tx=0x7f8dc0005f90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:21.291 INFO:tasks.workunit.client.0.smithi117.stdout:9/995: getdents d4/d35/df7/d105 0 2024-04-20T11:09:21.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.296+0000 7f8dca000700 1 -- 172.21.15.117:0/1083632959 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f8db80c4f60 con 0x7f8dcc10a810 2024-04-20T11:09:21.305 INFO:tasks.workunit.client.0.smithi117.stdout:7/859: truncate d1/d1d/d4e/d53/d55/f94 2590413 0 2024-04-20T11:09:21.307 INFO:tasks.workunit.client.0.smithi117.stdout:7/860: readlink d1/d1d/d42/d2d/l56 0 2024-04-20T11:09:21.309 INFO:tasks.workunit.client.0.smithi117.stdout:1/792: creat d6/d21/db9/f102 x:0 0 0 2024-04-20T11:09:21.310 INFO:tasks.workunit.client.0.smithi117.stdout:0/932: dread d3/d7/d20/d6e/df3/dff/d85/ddb/fc9 [0,4194304] 0 2024-04-20T11:09:21.314 INFO:tasks.workunit.client.0.smithi117.stdout:4/887: dread d1/dd/d52/d117/fc2 [0,4194304] 0 2024-04-20T11:09:21.315 INFO:tasks.workunit.client.0.smithi117.stdout:9/996: dread d4/d68/fe9 [0,4194304] 0 2024-04-20T11:09:21.318 INFO:tasks.workunit.client.0.smithi117.stdout:1/793: chown d6/d16/d3a/d5a/d5c/d61/d98/ca3 173 1 2024-04-20T11:09:21.320 INFO:tasks.workunit.client.0.smithi117.stdout:3/836: dwrite d0/d5/de/d4e/f9e [0,4194304] 0 2024-04-20T11:09:21.321 INFO:tasks.workunit.client.0.smithi117.stdout:1/794: chown d6/d1f/d30/dfd 12231 1 2024-04-20T11:09:21.322 INFO:tasks.workunit.client.0.smithi117.stdout:4/888: symlink d1/dd/d1a/d43/dcc/l125 0 2024-04-20T11:09:21.323 INFO:tasks.workunit.client.0.smithi117.stdout:7/861: creat d1/d4/f11e x:0 0 0 2024-04-20T11:09:21.324 INFO:tasks.workunit.client.0.smithi117.stdout:1/795: fdatasync d6/d1f/f45 0 2024-04-20T11:09:21.326 INFO:tasks.workunit.client.0.smithi117.stdout:8/799: link d0/d9/d3a/de3/cfe d0/d8/d11/d24/c106 0 2024-04-20T11:09:21.332 INFO:tasks.workunit.client.0.smithi117.stdout:4/889: symlink d1/d9/d1f/d90/l126 0 2024-04-20T11:09:21.335 INFO:tasks.workunit.client.0.smithi117.stdout:1/796: mknod d6/d16/d3a/d5a/daf/de4/c103 0 2024-04-20T11:09:21.336 INFO:tasks.workunit.client.0.smithi117.stdout:7/862: mkdir d1/d11f 0 2024-04-20T11:09:21.337 INFO:tasks.workunit.client.0.smithi117.stdout:3/837: mknod d0/d5/d21/d88/d85/c11c 0 2024-04-20T11:09:21.342 INFO:tasks.workunit.client.0.smithi117.stdout:9/997: getdents d4/d4a/d63/d98/dc0/dcb 0 2024-04-20T11:09:21.345 INFO:tasks.workunit.client.0.smithi117.stdout:7/863: readlink d1/d1d/d4e/d53/d55/l110 0 2024-04-20T11:09:21.345 INFO:tasks.workunit.client.0.smithi117.stdout:3/838: write d0/d5/d11/f113 [730070,13898] 0 2024-04-20T11:09:21.349 INFO:tasks.workunit.client.0.smithi117.stdout:0/933: rename d3/d7/d20/d47/d78/f8d to d3/d7/d20/d6e/df3/dff/f129 0 2024-04-20T11:09:21.353 INFO:tasks.workunit.client.0.smithi117.stdout:1/797: mknod d6/d16/d3a/d5a/d5c/d61/d71/c104 0 2024-04-20T11:09:21.355 INFO:tasks.workunit.client.0.smithi117.stdout:9/998: readlink d4/d4a/d5e/de8/l83 0 2024-04-20T11:09:21.355 INFO:tasks.workunit.client.0.smithi117.stdout:3/839: write d0/d5/fc5 [602550,107650] 0 2024-04-20T11:09:21.360 INFO:tasks.workunit.client.0.smithi117.stdout:1/798: write d6/d16/f62 [2105037,8974] 0 2024-04-20T11:09:21.361 INFO:tasks.workunit.client.0.smithi117.stdout:4/890: dwrite d1/f25 [4194304,4194304] 0 2024-04-20T11:09:21.367 INFO:tasks.workunit.client.0.smithi117.stdout:3/840: symlink d0/d5/d21/d88/d3c/l11d 0 2024-04-20T11:09:21.372 INFO:tasks.workunit.client.0.smithi117.stdout:0/934: creat d3/d7/d20/f12a x:0 0 0 2024-04-20T11:09:21.379 INFO:tasks.workunit.client.0.smithi117.stdout:3/841: dread d0/d5/d73/ded/f30 [0,4194304] 0 2024-04-20T11:09:21.382 INFO:tasks.workunit.client.0.smithi117.stdout:8/800: write d0/d9/d3a/fd4 [704067,117008] 0 2024-04-20T11:09:21.384 INFO:tasks.workunit.client.0.smithi117.stdout:0/935: mkdir d3/d7/d16/d12b 0 2024-04-20T11:09:21.387 INFO:tasks.workunit.client.0.smithi117.stdout:7/864: write d1/d4/d6/d17/d3f/f65 [1124479,52643] 0 2024-04-20T11:09:21.389 INFO:tasks.workunit.client.0.smithi117.stdout:1/799: creat d6/d16/f105 x:0 0 0 2024-04-20T11:09:21.390 INFO:tasks.workunit.client.0.smithi117.stdout:7/865: read - d1/d4/d3d/dce/deb/ffa zero size 2024-04-20T11:09:21.391 INFO:tasks.workunit.client.0.smithi117.stdout:4/891: fdatasync d1/f25 0 2024-04-20T11:09:21.394 INFO:tasks.workunit.client.0.smithi117.stdout:1/800: chown d6/d1f/d30/d49/d69/d6e/d6f 15 1 2024-04-20T11:09:21.395 INFO:tasks.workunit.client.0.smithi117.stdout:8/801: creat d0/d8/d11/d6d/d69/d9c/f107 x:0 0 0 2024-04-20T11:09:21.397 INFO:tasks.workunit.client.0.smithi117.stdout:1/801: chown d6/d16/d3a/d5a/d5c/c8d 43594182 1 2024-04-20T11:09:21.399 INFO:tasks.workunit.client.0.smithi117.stdout:9/999: truncate d4/d5/d57/db4/f11d 2445932 0 2024-04-20T11:09:21.401 INFO:tasks.workunit.client.0.smithi117.stdout:4/892: chown d1/dd/c82 766517 1 2024-04-20T11:09:21.404 INFO:tasks.workunit.client.0.smithi117.stdout:8/802: unlink d0/d8/d11/d24/f46 0 2024-04-20T11:09:21.404 INFO:tasks.workunit.client.0.smithi117.stdout:7/866: dread d1/d1d/fdd [0,4194304] 0 2024-04-20T11:09:21.410 INFO:tasks.workunit.client.0.smithi117.stdout:4/893: creat d1/dd/d52/d117/f127 x:0 0 0 2024-04-20T11:09:21.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:21 smithi173 ceph-mon[23800]: pgmap v20: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 49 MiB/s rd, 101 MiB/s wr, 265 op/s 2024-04-20T11:09:21.412 INFO:tasks.workunit.client.0.smithi117.stdout:3/842: link d0/d5/d7/c79 d0/d5/d21/c11e 0 2024-04-20T11:09:21.412 INFO:tasks.workunit.client.0.smithi117.stdout:0/936: getdents d3/d7/d20/d6e/df3 0 2024-04-20T11:09:21.414 INFO:tasks.workunit.client.0.smithi117.stdout:1/802: creat d6/d16/d3a/d5a/d5c/f106 x:0 0 0 2024-04-20T11:09:21.417 INFO:tasks.workunit.client.0.smithi117.stdout:4/894: stat d1/d9/cc 0 2024-04-20T11:09:21.417 INFO:tasks.workunit.client.0.smithi117.stdout:3/843: write d0/d5/d21/d77/f8a [1801223,47749] 0 2024-04-20T11:09:21.418 INFO:tasks.workunit.client.0.smithi117.stdout:4/895: fsync d1/fa7 0 2024-04-20T11:09:21.421 INFO:tasks.workunit.client.0.smithi117.stdout:7/867: dread d1/d43/f61 [0,4194304] 0 2024-04-20T11:09:21.427 INFO:tasks.workunit.client.0.smithi117.stdout:8/803: dwrite d0/dc/d49/f73 [0,4194304] 0 2024-04-20T11:09:21.428 INFO:tasks.workunit.client.0.smithi117.stdout:1/803: creat d6/d1f/d30/dfd/d101/da2/f107 x:0 0 0 2024-04-20T11:09:21.435 INFO:tasks.workunit.client.0.smithi117.stdout:3/844: rename d0/f71 to d0/d5/d21/d75/dc1/f11f 0 2024-04-20T11:09:21.439 INFO:tasks.workunit.client.0.smithi117.stdout:8/804: fdatasync d0/d9/d3a/d7c/fa1 0 2024-04-20T11:09:21.446 INFO:tasks.workunit.client.0.smithi117.stdout:1/804: rename d6/d1f/d30/d49/d69/d6e/d6f/d73/ccc to d6/c108 0 2024-04-20T11:09:21.452 INFO:tasks.workunit.client.0.smithi117.stdout:8/805: link d0/d8/d11/d24/d2b/f5d d0/dd3/f108 0 2024-04-20T11:09:21.459 INFO:tasks.workunit.client.0.smithi117.stdout:7/868: dwrite d1/d1d/d77/fa8 [0,4194304] 0 2024-04-20T11:09:21.459 INFO:tasks.workunit.client.0.smithi117.stdout:1/805: mknod d6/d16/d3a/d5a/d5c/d61/c109 0 2024-04-20T11:09:21.461 INFO:tasks.workunit.client.0.smithi117.stdout:0/937: write d3/d7/d20/d47/d78/f7c [4986246,102270] 0 2024-04-20T11:09:21.463 INFO:tasks.workunit.client.0.smithi117.stdout:8/806: mkdir d0/d8/d11/d24/d86/d109 0 2024-04-20T11:09:21.470 INFO:tasks.workunit.client.0.smithi117.stdout:7/869: write d1/d4/d3d/dce/f115 [425276,128660] 0 2024-04-20T11:09:21.473 INFO:tasks.workunit.client.0.smithi117.stdout:1/806: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/f10a x:0 0 0 2024-04-20T11:09:21.481 INFO:tasks.workunit.client.0.smithi117.stdout:8/807: symlink d0/d9/dbb/l10a 0 2024-04-20T11:09:21.486 INFO:tasks.workunit.client.0.smithi117.stdout:0/938: rename d3/d7/d2c/f65 to d3/d7/d8/f12c 0 2024-04-20T11:09:21.488 INFO:tasks.workunit.client.0.smithi117.stdout:3/845: dread d0/d5/d73/ded/d66/dbe/fe1 [0,4194304] 0 2024-04-20T11:09:21.491 INFO:tasks.workunit.client.0.smithi117.stdout:7/870: link d1/f86 d1/d43/d59/d111/f120 0 2024-04-20T11:09:21.495 INFO:tasks.workunit.client.0.smithi117.stdout:0/939: truncate d3/d7/d20/d6e/ffe 382587 0 2024-04-20T11:09:21.495 INFO:tasks.workunit.client.0.smithi117.stdout:3/846: symlink d0/d5/d21/d77/l120 0 2024-04-20T11:09:21.497 INFO:tasks.workunit.client.0.smithi117.stdout:1/807: dwrite d6/d35/d63/fe3 [0,4194304] 0 2024-04-20T11:09:21.502 INFO:tasks.workunit.client.0.smithi117.stdout:3/847: symlink d0/d5/d21/d75/d89/d116/l121 0 2024-04-20T11:09:21.503 INFO:tasks.workunit.client.0.smithi117.stdout:1/808: mknod d6/d1f/d30/d49/c10b 0 2024-04-20T11:09:21.505 INFO:tasks.workunit.client.0.smithi117.stdout:0/940: link d3/d7/d20/d39/d9a/cb9 d3/d7e/c12d 0 2024-04-20T11:09:21.507 INFO:tasks.workunit.client.0.smithi117.stdout:3/848: fdatasync d0/d5/d21/d94/d9a/fbb 0 2024-04-20T11:09:21.508 INFO:tasks.workunit.client.0.smithi117.stdout:1/809: mkdir d6/d16/d3a/d5a/d5c/df0/d10c 0 2024-04-20T11:09:21.510 INFO:tasks.workunit.client.0.smithi117.stdout:0/941: symlink d3/d7/d20/d39/d3a/db1/l12e 0 2024-04-20T11:09:21.516 INFO:tasks.workunit.client.0.smithi117.stdout:3/849: readlink d0/d5/d7/l5b 0 2024-04-20T11:09:21.517 INFO:tasks.workunit.client.0.smithi117.stdout:1/810: mknod d6/d1f/c10d 0 2024-04-20T11:09:21.518 INFO:tasks.workunit.client.0.smithi117.stdout:0/942: write d3/d7/f4d [1585772,73573] 0 2024-04-20T11:09:21.530 INFO:tasks.workunit.client.0.smithi117.stdout:7/871: write d1/d43/d59/d111/f120 [5320384,41472] 0 2024-04-20T11:09:21.533 INFO:tasks.workunit.client.0.smithi117.stdout:7/872: write d1/dd1/ff3 [124888,21118] 0 2024-04-20T11:09:21.540 INFO:tasks.workunit.client.0.smithi117.stdout:0/943: dread d3/dab/d54/fa3 [0,4194304] 0 2024-04-20T11:09:21.546 INFO:tasks.workunit.client.0.smithi117.stdout:1/811: write d6/d35/f36 [3456471,121531] 0 2024-04-20T11:09:21.547 INFO:tasks.workunit.client.0.smithi117.stdout:3/850: write d0/d5/d73/ded/f97 [3105340,79233] 0 2024-04-20T11:09:21.555 INFO:tasks.workunit.client.0.smithi117.stdout:3/851: mknod d0/d5/d21/d88/dc2/c122 0 2024-04-20T11:09:21.557 INFO:tasks.workunit.client.0.smithi117.stdout:1/812: rename d6/d16/d3a/d5a/d5c/d61/d71/d6b/c76 to d6/d16/d3a/d5a/daf/df6/c10e 0 2024-04-20T11:09:21.558 INFO:tasks.workunit.client.0.smithi117.stdout:0/944: write d3/d7/d20/f68 [5233879,7656] 0 2024-04-20T11:09:21.564 INFO:tasks.workunit.client.0.smithi117.stdout:3/852: read - d0/d5/d7/dd/f96 zero size 2024-04-20T11:09:21.568 INFO:tasks.workunit.client.0.smithi117.stdout:7/873: dwrite d1/d4/d6/d13/f15 [0,4194304] 0 2024-04-20T11:09:21.571 INFO:tasks.workunit.client.0.smithi117.stdout:7/874: chown d1/d11f 0 1 2024-04-20T11:09:21.580 INFO:tasks.workunit.client.0.smithi117.stdout:1/813: getdents d6/d16/d3a/d5a/daf/df6 0 2024-04-20T11:09:21.586 INFO:tasks.workunit.client.0.smithi117.stdout:0/945: dread d3/d7/d20/d39/d9a/dd0/d6d/fb4 [0,4194304] 0 2024-04-20T11:09:21.591 INFO:tasks.workunit.client.0.smithi117.stdout:1/814: rename d6/d35/d63/fe3 to d6/d1f/dd6/f10f 0 2024-04-20T11:09:21.593 INFO:tasks.workunit.client.0.smithi117.stdout:3/853: dwrite d0/d5/d21/d88/d36/f33 [0,4194304] 0 2024-04-20T11:09:21.598 INFO:tasks.workunit.client.0.smithi117.stdout:3/854: unlink d0/d5/d21/d88/d3c/c3e 0 2024-04-20T11:09:21.602 INFO:tasks.workunit.client.0.smithi117.stdout:0/946: creat d3/d7/d20/d39/d9a/dd0/f12f x:0 0 0 2024-04-20T11:09:21.606 INFO:tasks.workunit.client.0.smithi117.stdout:7/875: truncate d1/d4/d3d/dce/fd9 29247 0 2024-04-20T11:09:21.606 INFO:tasks.workunit.client.0.smithi117.stdout:0/947: write d3/d7/d20/f68 [4145167,48166] 0 2024-04-20T11:09:21.609 INFO:tasks.workunit.client.0.smithi117.stdout:3/855: write d0/d5/d21/f104 [406195,108278] 0 2024-04-20T11:09:21.613 INFO:tasks.workunit.client.0.smithi117.stdout:1/815: dwrite d6/d21/f86 [0,4194304] 0 2024-04-20T11:09:21.613 INFO:tasks.workunit.client.0.smithi117.stdout:7/876: fdatasync d1/d1d/f5c 0 2024-04-20T11:09:21.616 INFO:tasks.workunit.client.0.smithi117.stdout:3/856: write d0/d5/d73/ded/f97 [2632139,69318] 0 2024-04-20T11:09:21.620 INFO:tasks.workunit.client.0.smithi117.stdout:7/877: fsync d1/d43/f61 0 2024-04-20T11:09:21.629 INFO:tasks.workunit.client.0.smithi117.stdout:3/857: dread d0/d5/d73/ded/f30 [0,4194304] 0 2024-04-20T11:09:21.631 INFO:tasks.workunit.client.0.smithi117.stdout:7/878: rmdir d1/d1d/d42/d2d/d6c 39 2024-04-20T11:09:21.635 INFO:tasks.workunit.client.1.smithi173.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-20T11:09:21.643 INFO:tasks.workunit.client.0.smithi117.stdout:7/879: creat d1/d1d/d77/dee/f121 x:0 0 0 2024-04-20T11:09:21.646 INFO:tasks.workunit.client.0.smithi117.stdout:3/858: getdents d0/d5/d21/d77/df3 0 2024-04-20T11:09:21.649 INFO:tasks.workunit.client.0.smithi117.stdout:7/880: mknod d1/d1d/d42/d2d/d6c/db3/c122 0 2024-04-20T11:09:21.652 INFO:tasks.workunit.client.1.smithi173.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-20T11:09:21.652 INFO:tasks.workunit.client.1.smithi173.stderr:+ make 2024-04-20T11:09:21.652 INFO:tasks.workunit.client.0.smithi117.stdout:7/881: chown d1/d1d/d42/d2d/dae 330 1 2024-04-20T11:09:21.653 INFO:tasks.workunit.client.0.smithi117.stdout:1/816: dwrite d6/d35/f44 [0,4194304] 0 2024-04-20T11:09:21.658 INFO:tasks.workunit.client.0.smithi117.stdout:3/859: creat d0/d5/d73/ded/d66/d63/dad/f123 x:0 0 0 2024-04-20T11:09:21.660 INFO:tasks.workunit.client.0.smithi117.stdout:3/860: dread - d0/d5/d21/d75/d89/dd1/df4/ffa zero size 2024-04-20T11:09:21.661 INFO:tasks.workunit.client.0.smithi117.stdout:7/882: creat d1/d4/d3d/dce/deb/f123 x:0 0 0 2024-04-20T11:09:21.664 INFO:tasks.workunit.client.0.smithi117.stdout:0/948: dread d3/d7/d2c/d52/fc5 [0,4194304] 0 2024-04-20T11:09:21.675 INFO:tasks.workunit.client.0.smithi117.stdout:3/861: unlink d0/d5/d7/dd/f96 0 2024-04-20T11:09:21.676 INFO:tasks.workunit.client.0.smithi117.stdout:7/883: rename d1/d43/d59/d8a/d101/d11c to d1/d1d/d77/dee/d124 0 2024-04-20T11:09:21.679 INFO:tasks.workunit.client.0.smithi117.stdout:0/949: fdatasync d3/d7/d20/d39/fe8 0 2024-04-20T11:09:21.682 INFO:tasks.workunit.client.0.smithi117.stdout:0/950: write d3/d5/f114 [1692155,56570] 0 2024-04-20T11:09:21.683 INFO:tasks.workunit.client.0.smithi117.stdout:3/862: read d0/d5/d7/dd/f91 [24352,30095] 0 2024-04-20T11:09:21.691 INFO:tasks.workunit.client.0.smithi117.stdout:0/951: dread d3/d7/d20/d39/dbb/f107 [0,4194304] 0 2024-04-20T11:09:21.692 INFO:tasks.workunit.client.0.smithi117.stdout:7/884: dread d1/d4/d3d/dce/fd9 [0,4194304] 0 2024-04-20T11:09:21.696 INFO:tasks.workunit.client.0.smithi117.stdout:3/863: rmdir d0/d5/d73/ded/d66 39 2024-04-20T11:09:21.700 INFO:tasks.workunit.client.0.smithi117.stdout:1/817: dread d6/d35/f56 [0,4194304] 0 2024-04-20T11:09:21.701 INFO:tasks.workunit.client.0.smithi117.stdout:7/885: unlink d1/d1d/d42/faf 0 2024-04-20T11:09:21.704 INFO:tasks.workunit.client.0.smithi117.stdout:7/886: chown d1/d1d/d77/f11b 15 1 2024-04-20T11:09:21.705 INFO:tasks.workunit.client.0.smithi117.stdout:1/818: write d6/d1f/dd6/fe5 [720431,48003] 0 2024-04-20T11:09:21.705 INFO:tasks.workunit.client.0.smithi117.stdout:7/887: readlink d1/d1d/d42/d2d/lcd 0 2024-04-20T11:09:21.709 INFO:tasks.workunit.client.0.smithi117.stdout:1/819: truncate d6/fb4 771188 0 2024-04-20T11:09:21.714 INFO:tasks.workunit.client.0.smithi117.stdout:7/888: creat d1/d1d/d4e/d53/db1/f125 x:0 0 0 2024-04-20T11:09:21.715 INFO:tasks.workunit.client.0.smithi117.stdout:1/820: unlink d6/d1f/d30/d49/c8a 0 2024-04-20T11:09:21.716 INFO:tasks.workunit.client.0.smithi117.stdout:3/864: link d0/d5/d73/ded/db3/cda d0/d5/d21/d88/d69/d10b/c124 0 2024-04-20T11:09:21.723 INFO:tasks.workunit.client.0.smithi117.stdout:7/889: truncate d1/d4/d6/d13/f28 391840 0 2024-04-20T11:09:21.724 INFO:tasks.workunit.client.0.smithi117.stdout:3/865: rmdir d0/d5/d21/d88/d69/d10b 39 2024-04-20T11:09:21.727 INFO:tasks.workunit.client.0.smithi117.stdout:0/952: dwrite d3/dab/d54/faf [0,4194304] 0 2024-04-20T11:09:21.727 INFO:tasks.workunit.client.0.smithi117.stdout:1/821: rename d6/d35/f44 to d6/d16/d3a/d5a/d5c/da6/f110 0 2024-04-20T11:09:21.728 INFO:tasks.workunit.client.0.smithi117.stdout:3/866: write d0/d5/d21/f4d [238393,130320] 0 2024-04-20T11:09:21.730 INFO:tasks.workunit.client.0.smithi117.stdout:1/822: write d6/d16/d3a/d5a/fec [427411,33339] 0 2024-04-20T11:09:21.732 INFO:tasks.workunit.client.0.smithi117.stdout:1/823: chown d6/d1f 1955587 1 2024-04-20T11:09:21.732 INFO:tasks.workunit.client.0.smithi117.stdout:3/867: chown d0/d5/d7/f10f 36355 1 2024-04-20T11:09:21.739 INFO:tasks.workunit.client.0.smithi117.stdout:0/953: symlink d3/d7/d20/d39/d3a/db1/l130 0 2024-04-20T11:09:21.744 INFO:tasks.workunit.client.0.smithi117.stdout:1/824: chown d6/d21/l58 0 1 2024-04-20T11:09:21.745 INFO:tasks.workunit.client.0.smithi117.stdout:0/954: dread - d3/d7/d20/d39/d3a/db1/ff6 zero size 2024-04-20T11:09:21.747 INFO:tasks.workunit.client.0.smithi117.stdout:7/890: stat d1/d4/d39/f4d 0 2024-04-20T11:09:21.751 INFO:tasks.workunit.client.0.smithi117.stdout:3/868: truncate d0/d5/d73/ded/dac/dd2/fdc 686056 0 2024-04-20T11:09:21.754 INFO:tasks.workunit.client.0.smithi117.stdout:0/955: mkdir d3/d7/d20/d39/d9a/d131 0 2024-04-20T11:09:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.759+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dac000bc0 con 0x7f8db406eac0 2024-04-20T11:09:21.760 INFO:tasks.workunit.client.0.smithi117.stdout:1/825: rename d6/d1f/d30/d49/d69/d6e/d6f/d73/df9 to d6/d1f/d30/d49/d69/d6e/d6f/d73/d111 0 2024-04-20T11:09:21.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.761+0000 7f8dca000700 1 -- 172.21.15.117:0/1083632959 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f8dac000bc0 con 0x7f8db406eac0 2024-04-20T11:09:21.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.766+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8db406eac0 msgr2=0x7f8db4070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:21.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.766+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8db406eac0 0x7f8db4070f80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000f4d0 tx=0x7f8dc0005f90 comp rx=0 tx=0).stop 2024-04-20T11:09:21.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.766+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dcc10a810 msgr2=0x7f8dcc19d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:21.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.766+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dcc10a810 0x7f8dcc19d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8db80098e0 tx=0x7f8db800b800 comp rx=0 tx=0).stop 2024-04-20T11:09:21.766 INFO:tasks.workunit.client.0.smithi117.stdout:7/891: mknod d1/d1d/ded/d107/c126 0 2024-04-20T11:09:21.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.767+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 shutdown_connections 2024-04-20T11:09:21.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.767+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8db406eac0 0x7f8db4070f80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:21.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.767+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dcc107ee0 0x7f8dcc19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:21.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.767+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/1083632959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dcc10a810 0x7f8dcc19d190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:21.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.767+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 >> 172.21.15.117:0/1083632959 conn(0x7f8dcc06de40 msgr2=0x7f8dcc070230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:21.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.768+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 shutdown_connections 2024-04-20T11:09:21.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.769+0000 7f8dd3e00700 1 -- 172.21.15.117:0/1083632959 wait complete. 2024-04-20T11:09:21.777 INFO:tasks.workunit.client.0.smithi117.stdout:1/826: read - d6/d1f/d30/d32/f89 zero size 2024-04-20T11:09:21.785 INFO:tasks.workunit.client.0.smithi117.stdout:1/827: stat d6/d35/d63/ce6 0 2024-04-20T11:09:21.785 INFO:tasks.workunit.client.0.smithi117.stdout:7/892: link d1/d4/f11e d1/d4/d3d/f127 0 2024-04-20T11:09:21.787 INFO:tasks.workunit.client.0.smithi117.stdout:1/828: chown d6/d1f/d30/d49/lf3 3668159 1 2024-04-20T11:09:21.790 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:09:21.796 INFO:tasks.workunit.client.0.smithi117.stdout:1/829: chown d6/d1f/d30/d32/ffb 37 1 2024-04-20T11:09:21.800 INFO:tasks.workunit.client.0.smithi117.stdout:3/869: truncate d0/d5/d21/fb5 810808 0 2024-04-20T11:09:21.820 INFO:tasks.workunit.client.0.smithi117.stdout:7/893: dwrite d1/d4/f11e [0,4194304] 0 2024-04-20T11:09:21.825 INFO:tasks.workunit.client.0.smithi117.stdout:3/870: dwrite d0/d5/d21/d88/f44 [4194304,4194304] 0 2024-04-20T11:09:21.829 INFO:tasks.workunit.client.0.smithi117.stdout:7/894: truncate d1/d4/d6/d17/d3f/f90 5175989 0 2024-04-20T11:09:21.838 INFO:tasks.workunit.client.0.smithi117.stdout:1/830: write d6/d16/d3a/d5a/d5c/fc4 [1044010,56316] 0 2024-04-20T11:09:21.843 INFO:tasks.workunit.client.0.smithi117.stdout:3/871: read d0/f86 [3812370,19298] 0 2024-04-20T11:09:21.843 INFO:tasks.workunit.client.0.smithi117.stdout:7/895: mknod d1/d4/d6/d2b/d64/c128 0 2024-04-20T11:09:21.843 INFO:tasks.workunit.client.0.smithi117.stdout:1/831: write d6/d21/db9/f102 [837461,111696] 0 2024-04-20T11:09:21.855 INFO:tasks.workunit.client.0.smithi117.stdout:3/872: write d0/d5/d21/d88/d69/f8c [1291935,87335] 0 2024-04-20T11:09:21.858 INFO:tasks.workunit.client.0.smithi117.stdout:1/832: mknod d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/c112 0 2024-04-20T11:09:21.859 INFO:tasks.workunit.client.0.smithi117.stdout:3/873: write d0/d5/d21/d88/f44 [4935096,29428] 0 2024-04-20T11:09:21.867 INFO:tasks.workunit.client.0.smithi117.stdout:3/874: creat d0/d5/d21/d88/d36/d111/f125 x:0 0 0 2024-04-20T11:09:21.869 INFO:tasks.workunit.client.0.smithi117.stdout:7/896: rename d1/d1d/d42/d2d/f92 to d1/d4/d6/d54/f129 0 2024-04-20T11:09:21.878 INFO:tasks.workunit.client.0.smithi117.stdout:3/875: getdents d0/d5/d21/d75/d89/d116 0 2024-04-20T11:09:21.879 INFO:tasks.workunit.client.0.smithi117.stdout:1/833: dwrite d6/f79 [4194304,4194304] 0 2024-04-20T11:09:21.893 INFO:tasks.workunit.client.0.smithi117.stdout:3/876: getdents d0/d5/d11 0 2024-04-20T11:09:21.895 INFO:tasks.workunit.client.0.smithi117.stdout:3/877: creat d0/d5/de/d4e/dec/f126 x:0 0 0 2024-04-20T11:09:21.896 INFO:tasks.workunit.client.0.smithi117.stdout:7/897: write d1/d4/d3d/f5a [5053291,33427] 0 2024-04-20T11:09:21.902 INFO:tasks.workunit.client.0.smithi117.stdout:3/878: truncate d0/d5/d73/ff7 491369 0 2024-04-20T11:09:21.903 INFO:tasks.workunit.client.0.smithi117.stdout:1/834: dwrite d6/d1f/f2b [0,4194304] 0 2024-04-20T11:09:21.903 INFO:tasks.workunit.client.0.smithi117.stdout:7/898: fsync d1/d4/d3d/d93/fa6 0 2024-04-20T11:09:21.913 INFO:tasks.workunit.client.0.smithi117.stdout:3/879: rmdir d0/d5/d21/d75/d89 39 2024-04-20T11:09:21.913 INFO:tasks.workunit.client.0.smithi117.stdout:1/835: symlink d6/dd2/l113 0 2024-04-20T11:09:21.913 INFO:tasks.workunit.client.0.smithi117.stdout:4/896: sync 2024-04-20T11:09:21.917 INFO:tasks.workunit.client.0.smithi117.stdout:1/836: creat d6/d1f/dd6/f114 x:0 0 0 2024-04-20T11:09:21.918 INFO:tasks.workunit.client.0.smithi117.stdout:4/897: symlink d1/dd/d1a/d43/d4e/l128 0 2024-04-20T11:09:21.919 INFO:tasks.workunit.client.0.smithi117.stdout:3/880: creat d0/d5/d21/d77/df3/d103/f127 x:0 0 0 2024-04-20T11:09:21.922 INFO:tasks.workunit.client.0.smithi117.stdout:1/837: read d6/f18 [5064024,70535] 0 2024-04-20T11:09:21.933 INFO:tasks.workunit.client.0.smithi117.stdout:4/898: truncate d1/f19 8708475 0 2024-04-20T11:09:21.939 INFO:tasks.workunit.client.1.smithi173.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-20T11:09:21.940 INFO:tasks.workunit.client.0.smithi117.stdout:3/881: write d0/f86 [4550447,36779] 0 2024-04-20T11:09:21.945 INFO:tasks.workunit.client.0.smithi117.stdout:7/899: dread d1/d4/f30 [0,4194304] 0 2024-04-20T11:09:21.948 INFO:tasks.workunit.client.0.smithi117.stdout:4/899: dread d1/dd/d1a/d2b/d86/ff6 [0,4194304] 0 2024-04-20T11:09:21.953 INFO:tasks.workunit.client.0.smithi117.stdout:1/838: dwrite d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/de9/ff2 [0,4194304] 0 2024-04-20T11:09:21.955 INFO:tasks.workunit.client.0.smithi117.stdout:1/839: write d6/f1a [76221,84813] 0 2024-04-20T11:09:21.956 INFO:tasks.workunit.client.0.smithi117.stdout:7/900: read d1/d4/d3d/dce/fd9 [18233,123884] 0 2024-04-20T11:09:21.967 INFO:tasks.workunit.client.0.smithi117.stdout:7/901: mkdir d1/d4/d6/d17/d12a 0 2024-04-20T11:09:21.974 INFO:tasks.workunit.client.0.smithi117.stdout:4/900: dwrite d1/d9/f12 [0,4194304] 0 2024-04-20T11:09:21.979 INFO:tasks.workunit.client.0.smithi117.stdout:7/902: mknod d1/d1d/d4e/c12b 0 2024-04-20T11:09:21.984 INFO:tasks.workunit.client.0.smithi117.stdout:7/903: write d1/d1d/d4e/d72/fc6 [1193009,34563] 0 2024-04-20T11:09:21.989 INFO:tasks.workunit.client.0.smithi117.stdout:4/901: creat d1/dd/d1a/d43/f129 x:0 0 0 2024-04-20T11:09:21.990 INFO:tasks.workunit.client.0.smithi117.stdout:8/808: sync 2024-04-20T11:09:21.995 INFO:tasks.workunit.client.0.smithi117.stdout:8/809: write d0/d8/d39/d40/d78/f100 [628370,121577] 0 2024-04-20T11:09:21.995 INFO:tasks.workunit.client.0.smithi117.stdout:1/840: dwrite d6/d16/d3a/d5a/daf/ff5 [0,4194304] 0 2024-04-20T11:09:21.996 INFO:tasks.workunit.client.0.smithi117.stdout:3/882: dread d0/d5/d73/ded/f97 [0,4194304] 0 2024-04-20T11:09:21.997 INFO:tasks.workunit.client.0.smithi117.stdout:4/902: mknod d1/d9/d6b/dca/c12a 0 2024-04-20T11:09:21.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.998+0000 7efd90e00700 1 -- 172.21.15.117:0/2156368263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c103fa0 msgr2=0x7efd8c104420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:21.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:21.998+0000 7efd90e00700 1 --2- 172.21.15.117:0/2156368263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c103fa0 0x7efd8c104420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efd74009910 tx=0x7efd74009c20 comp rx=0 tx=0).stop 2024-04-20T11:09:22.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.000+0000 7efd90e00700 1 -- 172.21.15.117:0/2156368263 shutdown_connections 2024-04-20T11:09:22.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.000+0000 7efd90e00700 1 --2- 172.21.15.117:0/2156368263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c103fa0 0x7efd8c104420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.000+0000 7efd90e00700 1 --2- 172.21.15.117:0/2156368263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd8c102e40 0x7efd8c103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.000+0000 7efd90e00700 1 -- 172.21.15.117:0/2156368263 >> 172.21.15.117:0/2156368263 conn(0x7efd8c0fe420 msgr2=0x7efd8c100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:22.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.000+0000 7efd90e00700 1 -- 172.21.15.117:0/2156368263 shutdown_connections 2024-04-20T11:09:22.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.000+0000 7efd90e00700 1 -- 172.21.15.117:0/2156368263 wait complete. 2024-04-20T11:09:22.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.003+0000 7efd90e00700 1 Processor -- start 2024-04-20T11:09:22.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.004+0000 7efd90e00700 1 -- start start 2024-04-20T11:09:22.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.005+0000 7efd90e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c102e40 0x7efd8c078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:22.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.005+0000 7efd90e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd8c103fa0 0x7efd8c079180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:22.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.005+0000 7efd90e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c075780 con 0x7efd8c102e40 2024-04-20T11:09:22.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.005+0000 7efd90e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0758c0 con 0x7efd8c103fa0 2024-04-20T11:09:22.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.006+0000 7efd81e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c102e40 0x7efd8c078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:22.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.006+0000 7efd81e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c102e40 0x7efd8c078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50498/0 (socket says 172.21.15.117:50498) 2024-04-20T11:09:22.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.006+0000 7efd81e00700 1 -- 172.21.15.117:0/2416315390 learned_addr learned my addr 172.21.15.117:0/2416315390 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:22.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.007+0000 7efd81e00700 1 -- 172.21.15.117:0/2416315390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd8c103fa0 msgr2=0x7efd8c079180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:22.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.007+0000 7efd81e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd8c103fa0 0x7efd8c079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.007+0000 7efd81e00700 1 -- 172.21.15.117:0/2416315390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740095f0 con 0x7efd8c102e40 2024-04-20T11:09:22.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.007+0000 7efd81e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c102e40 0x7efd8c078c40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00ba50 tx=0x7efd7c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:22.008 INFO:tasks.workunit.client.0.smithi117.stdout:8/810: rename d0/l50 to d0/d8/d11/d24/d53/l10b 0 2024-04-20T11:09:22.018 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:22 smithi117 ceph-mon[15659]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:22.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.018+0000 7efd83400700 1 -- 172.21.15.117:0/2416315390 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd7c00ca00 con 0x7efd8c102e40 2024-04-20T11:09:22.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.018+0000 7efd83400700 1 -- 172.21.15.117:0/2416315390 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd7c00cb60 con 0x7efd8c102e40 2024-04-20T11:09:22.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.018+0000 7efd83400700 1 -- 172.21.15.117:0/2416315390 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd7c0075a0 con 0x7efd8c102e40 2024-04-20T11:09:22.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.019+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c075ba0 con 0x7efd8c102e40 2024-04-20T11:09:22.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.019+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c076140 con 0x7efd8c102e40 2024-04-20T11:09:22.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.021+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c1929c0 con 0x7efd8c102e40 2024-04-20T11:09:22.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.024+0000 7efd83400700 1 -- 172.21.15.117:0/2416315390 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7efd7c00ccd0 con 0x7efd8c102e40 2024-04-20T11:09:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.025+0000 7efd83400700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7efd7006ebe0 0x7efd700710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.025+0000 7efd83400700 1 -- 172.21.15.117:0/2416315390 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7efd7c006e40 con 0x7efd8c102e40 2024-04-20T11:09:22.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.027+0000 7efd89e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7efd7006ebe0 0x7efd700710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:22.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.030+0000 7efd89e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7efd7006ebe0 0x7efd700710a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd740098e0 tx=0x7efd740055c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:22.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.031+0000 7efd83400700 1 -- 172.21.15.117:0/2416315390 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7efd7c05b4a0 con 0x7efd8c102e40 2024-04-20T11:09:22.037 INFO:tasks.workunit.client.0.smithi117.stdout:3/883: write d0/d5/d21/f9c [940483,2823] 0 2024-04-20T11:09:22.039 INFO:tasks.workunit.client.0.smithi117.stdout:1/841: getdents d6/d1f/d30/d32 0 2024-04-20T11:09:22.040 INFO:tasks.workunit.client.0.smithi117.stdout:7/904: dread d1/f22 [0,4194304] 0 2024-04-20T11:09:22.042 INFO:tasks.workunit.client.0.smithi117.stdout:4/903: dwrite d1/dd/d1a/d43/dcc/f104 [0,4194304] 0 2024-04-20T11:09:22.042 INFO:tasks.workunit.client.0.smithi117.stdout:3/884: write d0/d5/d7/f10f [846108,95136] 0 2024-04-20T11:09:22.046 INFO:tasks.workunit.client.0.smithi117.stdout:1/842: rmdir d6/d35/d63 39 2024-04-20T11:09:22.048 INFO:tasks.workunit.client.0.smithi117.stdout:1/843: stat d6/d16/d3a/d5a/d5c/fc4 0 2024-04-20T11:09:22.055 INFO:tasks.workunit.client.0.smithi117.stdout:8/811: rename d0/d8/d39/d40/f5f to d0/d8/d11/d24/d76/f10c 0 2024-04-20T11:09:22.059 INFO:tasks.workunit.client.0.smithi117.stdout:4/904: fsync d1/f19 0 2024-04-20T11:09:22.060 INFO:tasks.workunit.client.0.smithi117.stdout:1/844: creat d6/d16/d3a/d5a/d5c/d61/d98/f115 x:0 0 0 2024-04-20T11:09:22.068 INFO:tasks.workunit.client.0.smithi117.stdout:0/956: sync 2024-04-20T11:09:22.069 INFO:tasks.workunit.client.0.smithi117.stdout:4/905: read d1/d9/f2a [2260036,1261] 0 2024-04-20T11:09:22.073 INFO:tasks.workunit.client.0.smithi117.stdout:4/906: chown d1/d45/cfe 934 1 2024-04-20T11:09:22.075 INFO:tasks.workunit.client.0.smithi117.stdout:0/957: chown d3/d7/d20/d6e/ffe 135792864 1 2024-04-20T11:09:22.077 INFO:tasks.workunit.client.0.smithi117.stdout:3/885: dwrite d0/d5/fdb [0,4194304] 0 2024-04-20T11:09:22.082 INFO:tasks.workunit.client.0.smithi117.stdout:7/905: dwrite d1/d43/f80 [4194304,4194304] 0 2024-04-20T11:09:22.084 INFO:tasks.workunit.client.0.smithi117.stdout:0/958: rmdir d3/d5/d74 39 2024-04-20T11:09:22.085 INFO:tasks.workunit.client.0.smithi117.stdout:4/907: mkdir d1/d9/d1f/d90/d107/d12b 0 2024-04-20T11:09:22.090 INFO:tasks.workunit.client.0.smithi117.stdout:3/886: unlink d0/d5/d21/d75/dc1/fcd 0 2024-04-20T11:09:22.091 INFO:tasks.workunit.client.0.smithi117.stdout:0/959: symlink d3/d7e/l132 0 2024-04-20T11:09:22.096 INFO:tasks.workunit.client.0.smithi117.stdout:0/960: mknod d3/d7/d20/d47/d78/c133 0 2024-04-20T11:09:22.096 INFO:tasks.workunit.client.0.smithi117.stdout:8/812: dwrite d0/d8/d11/d24/d27/d47/f4f [0,4194304] 0 2024-04-20T11:09:22.101 INFO:tasks.workunit.client.0.smithi117.stdout:4/908: read d1/f19 [8288495,122316] 0 2024-04-20T11:09:22.102 INFO:tasks.workunit.client.0.smithi117.stdout:0/961: stat d3/d7/d2c/d52/c72 0 2024-04-20T11:09:22.114 INFO:tasks.workunit.client.0.smithi117.stdout:3/887: rename d0/d5/d21/l39 to d0/d5/d73/ded/d66/dbe/l128 0 2024-04-20T11:09:22.114 INFO:tasks.workunit.client.0.smithi117.stdout:7/906: dwrite d1/d4/d6/d2b/f33 [0,4194304] 0 2024-04-20T11:09:22.122 INFO:tasks.workunit.client.0.smithi117.stdout:4/909: symlink d1/d45/d4a/l12c 0 2024-04-20T11:09:22.127 INFO:tasks.workunit.client.0.smithi117.stdout:1/845: dread d6/d16/d3a/d5a/daf/fc5 [0,4194304] 0 2024-04-20T11:09:22.128 INFO:tasks.workunit.client.0.smithi117.stdout:7/907: mknod d1/d43/d59/d8a/c12c 0 2024-04-20T11:09:22.130 INFO:tasks.workunit.client.0.smithi117.stdout:7/908: chown d1/d4/d6/d54 2622760 1 2024-04-20T11:09:22.131 INFO:tasks.workunit.client.0.smithi117.stdout:4/910: chown d1/d9/cac 5 1 2024-04-20T11:09:22.137 INFO:tasks.workunit.client.0.smithi117.stdout:8/813: write d0/f23 [5331713,38595] 0 2024-04-20T11:09:22.140 INFO:tasks.workunit.client.0.smithi117.stdout:1/846: truncate d6/d1f/d30/f43 3950387 0 2024-04-20T11:09:22.150 INFO:tasks.workunit.client.0.smithi117.stdout:8/814: mknod d0/d8/d11/d24/d27/d47/c10d 0 2024-04-20T11:09:22.151 INFO:tasks.workunit.client.0.smithi117.stdout:1/847: truncate d6/d35/f56 656120 0 2024-04-20T11:09:22.152 INFO:tasks.workunit.client.0.smithi117.stdout:3/888: write d0/d5/d21/d88/d85/fcb [533962,15550] 0 2024-04-20T11:09:22.156 INFO:tasks.workunit.client.0.smithi117.stdout:3/889: dread - d0/d5/d21/d88/ff5 zero size 2024-04-20T11:09:22.156 INFO:tasks.workunit.client.0.smithi117.stdout:7/909: dwrite d1/d43/d59/ffd [0,4194304] 0 2024-04-20T11:09:22.158 INFO:tasks.workunit.client.0.smithi117.stdout:1/848: write d6/d1f/d30/d32/fdd [1418832,24690] 0 2024-04-20T11:09:22.160 INFO:tasks.workunit.client.0.smithi117.stdout:8/815: chown d0/d9/d3a/d105/le0 175 1 2024-04-20T11:09:22.163 INFO:tasks.workunit.client.0.smithi117.stdout:0/962: truncate d3/d7/d20/d47/fc2 1918621 0 2024-04-20T11:09:22.164 INFO:tasks.workunit.client.0.smithi117.stdout:3/890: creat d0/d5/de/d4e/f129 x:0 0 0 2024-04-20T11:09:22.177 INFO:tasks.workunit.client.0.smithi117.stdout:8/816: dread - d0/dc/d49/d5b/da9/dc7/fde zero size 2024-04-20T11:09:22.199 INFO:tasks.workunit.client.0.smithi117.stdout:4/911: dwrite d1/dd/d1a/d43/d4e/f9b [0,4194304] 0 2024-04-20T11:09:22.202 INFO:tasks.workunit.client.0.smithi117.stdout:1/849: dwrite d6/d16/d3a/d5a/d5c/da6/f110 [0,4194304] 0 2024-04-20T11:09:22.204 INFO:tasks.workunit.client.0.smithi117.stdout:0/963: dwrite d3/d7/d20/d39/d9a/dd0/f12f [0,4194304] 0 2024-04-20T11:09:22.221 INFO:tasks.workunit.client.0.smithi117.stdout:0/964: chown d3/d7/f108 16484573 1 2024-04-20T11:09:22.224 INFO:tasks.workunit.client.0.smithi117.stdout:8/817: dwrite d0/d8/d39/d40/d78/f100 [0,4194304] 0 2024-04-20T11:09:22.225 INFO:tasks.workunit.client.0.smithi117.stdout:1/850: creat d6/d1f/d30/dfd/d101/f116 x:0 0 0 2024-04-20T11:09:22.228 INFO:tasks.workunit.client.0.smithi117.stdout:0/965: readlink d3/d5/d74/lb5 0 2024-04-20T11:09:22.229 INFO:tasks.workunit.client.0.smithi117.stdout:7/910: dwrite d1/d4/d6/d54/db4/fe4 [0,4194304] 0 2024-04-20T11:09:22.231 INFO:tasks.workunit.client.0.smithi117.stdout:1/851: readlink d6/d1f/d30/l4c 0 2024-04-20T11:09:22.232 INFO:tasks.workunit.client.0.smithi117.stdout:0/966: chown d3/d7/d8/f112 255810 1 2024-04-20T11:09:22.245 INFO:tasks.workunit.client.0.smithi117.stdout:8/818: unlink d0/d8/d11/d6d/c7a 0 2024-04-20T11:09:22.251 INFO:tasks.workunit.client.0.smithi117.stdout:1/852: rename d6/d16/d3a/d5a/f5d to d6/d16/d3a/d5a/d5c/da6/f117 0 2024-04-20T11:09:22.253 INFO:tasks.workunit.client.0.smithi117.stdout:3/891: dread d0/f35 [0,4194304] 0 2024-04-20T11:09:22.255 INFO:tasks.workunit.client.0.smithi117.stdout:8/819: symlink d0/d9/dbb/l10e 0 2024-04-20T11:09:22.258 INFO:tasks.workunit.client.0.smithi117.stdout:1/853: chown d6/f12 21432 1 2024-04-20T11:09:22.259 INFO:tasks.workunit.client.0.smithi117.stdout:4/912: dwrite d1/d45/d4a/fa9 [0,4194304] 0 2024-04-20T11:09:22.263 INFO:tasks.workunit.client.0.smithi117.stdout:1/854: truncate d6/d16/d3a/f7e 1282632 0 2024-04-20T11:09:22.263 INFO:tasks.workunit.client.0.smithi117.stdout:3/892: truncate d0/d5/de/d4e/f129 566209 0 2024-04-20T11:09:22.265 INFO:tasks.workunit.client.0.smithi117.stdout:0/967: dwrite d3/d7/d2c/f10e [0,4194304] 0 2024-04-20T11:09:22.268 INFO:tasks.workunit.client.0.smithi117.stdout:8/820: creat d0/d8/d11/d24/d27/d99/f10f x:0 0 0 2024-04-20T11:09:22.271 INFO:tasks.workunit.client.0.smithi117.stdout:4/913: fdatasync d1/d9/f3b 0 2024-04-20T11:09:22.273 INFO:tasks.workunit.client.0.smithi117.stdout:4/914: stat d1/dd/f38 0 2024-04-20T11:09:22.275 INFO:tasks.workunit.client.0.smithi117.stdout:0/968: creat d3/d7/d20/d39/d9a/dd0/d6d/d10c/f134 x:0 0 0 2024-04-20T11:09:22.281 INFO:tasks.workunit.client.0.smithi117.stdout:3/893: creat d0/d5/d21/d88/d69/d10b/f12a x:0 0 0 2024-04-20T11:09:22.283 INFO:tasks.workunit.client.0.smithi117.stdout:8/821: symlink d0/d8/d11/d24/d27/l110 0 2024-04-20T11:09:22.288 INFO:tasks.workunit.client.0.smithi117.stdout:0/969: rename d3/d7/d20/d39/d9a/dd0/d6d/d10c/c111 to d3/d7/d2c/d52/c135 0 2024-04-20T11:09:22.291 INFO:tasks.workunit.client.0.smithi117.stdout:3/894: fdatasync d0/d5/d21/d94/fb1 0 2024-04-20T11:09:22.293 INFO:tasks.workunit.client.0.smithi117.stdout:7/911: dread d1/d1d/fbb [0,4194304] 0 2024-04-20T11:09:22.293 INFO:tasks.workunit.client.0.smithi117.stdout:8/822: creat d0/d9/d3a/de3/f111 x:0 0 0 2024-04-20T11:09:22.297 INFO:tasks.workunit.client.0.smithi117.stdout:8/823: dread - d0/d8/d39/d40/d78/fb4 zero size 2024-04-20T11:09:22.298 INFO:tasks.workunit.client.0.smithi117.stdout:4/915: dread d1/dd/d1a/d2b/d86/fdc [0,4194304] 0 2024-04-20T11:09:22.299 INFO:tasks.workunit.client.0.smithi117.stdout:1/855: write d6/d35/f97 [3920310,25861] 0 2024-04-20T11:09:22.306 INFO:tasks.workunit.client.0.smithi117.stdout:0/970: creat d3/d7/d20/d6e/df3/dff/f136 x:0 0 0 2024-04-20T11:09:22.307 INFO:tasks.workunit.client.0.smithi117.stdout:3/895: link d0/d5/c38 d0/d5/d73/ded/dac/c12b 0 2024-04-20T11:09:22.312 INFO:tasks.workunit.client.0.smithi117.stdout:8/824: rename d0/d9/d3a/d105/f81 to d0/d8/d11/d6d/d84/f112 0 2024-04-20T11:09:22.316 INFO:tasks.workunit.client.0.smithi117.stdout:7/912: getdents d1/d1d/d77/dee 0 2024-04-20T11:09:22.317 INFO:tasks.workunit.client.0.smithi117.stdout:0/971: mknod d3/d7/d20/d39/dbb/c137 0 2024-04-20T11:09:22.321 INFO:tasks.workunit.client.0.smithi117.stdout:8/825: getdents d0/d8/d11/d24/d2b/d72 0 2024-04-20T11:09:22.327 INFO:tasks.workunit.client.0.smithi117.stdout:0/972: mknod d3/dab/c138 0 2024-04-20T11:09:22.328 INFO:tasks.workunit.client.0.smithi117.stdout:3/896: getdents d0/d5/d11/d80 0 2024-04-20T11:09:22.328 INFO:tasks.workunit.client.0.smithi117.stdout:8/826: mkdir d0/d9/dbb/d113 0 2024-04-20T11:09:22.332 INFO:tasks.workunit.client.0.smithi117.stdout:3/897: mknod d0/d5/d7/dd/dff/c12c 0 2024-04-20T11:09:22.335 INFO:tasks.workunit.client.0.smithi117.stdout:7/913: write d1/d4/d6/d54/d63/f6e [732740,89160] 0 2024-04-20T11:09:22.337 INFO:tasks.workunit.client.0.smithi117.stdout:8/827: creat d0/d8/d11/f114 x:0 0 0 2024-04-20T11:09:22.338 INFO:tasks.workunit.client.0.smithi117.stdout:0/973: rename d3/d5/cc to d3/d7/d20/c139 0 2024-04-20T11:09:22.341 INFO:tasks.workunit.client.0.smithi117.stdout:3/898: creat d0/d5/de/d4e/dec/f12d x:0 0 0 2024-04-20T11:09:22.342 INFO:tasks.workunit.client.0.smithi117.stdout:1/856: dread d6/f10 [0,4194304] 0 2024-04-20T11:09:22.347 INFO:tasks.workunit.client.0.smithi117.stdout:4/916: dread d1/dd/d1a/d2b/d86/fb4 [0,4194304] 0 2024-04-20T11:09:22.347 INFO:tasks.workunit.client.0.smithi117.stdout:3/899: symlink d0/d5/d11/d80/l12e 0 2024-04-20T11:09:22.350 INFO:tasks.workunit.client.0.smithi117.stdout:4/917: write d1/d9/f17 [1546945,61153] 0 2024-04-20T11:09:22.351 INFO:tasks.workunit.client.0.smithi117.stdout:3/900: chown d0/d5/d21/d88/f62 32 1 2024-04-20T11:09:22.353 INFO:tasks.workunit.client.0.smithi117.stdout:7/914: write d1/d43/d59/fc9 [566850,29702] 0 2024-04-20T11:09:22.353 INFO:tasks.workunit.client.0.smithi117.stdout:1/857: creat d6/d21/db9/f118 x:0 0 0 2024-04-20T11:09:22.355 INFO:tasks.workunit.client.0.smithi117.stdout:7/915: write d1/d4/d3d/f127 [2483073,16869] 0 2024-04-20T11:09:22.361 INFO:tasks.workunit.client.0.smithi117.stdout:1/858: dread - d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/fee zero size 2024-04-20T11:09:22.363 INFO:tasks.workunit.client.0.smithi117.stdout:3/901: chown d0/d5/d73/f8b 81707851 1 2024-04-20T11:09:22.363 INFO:tasks.workunit.client.0.smithi117.stdout:8/828: getdents d0/d9/d3a/de3 0 2024-04-20T11:09:22.365 INFO:tasks.workunit.client.0.smithi117.stdout:0/974: write d3/d7/d20/d39/dbb/f107 [1446397,109665] 0 2024-04-20T11:09:22.365 INFO:tasks.workunit.client.0.smithi117.stdout:3/902: chown d0/d5/d21/d75/dc1 404713487 1 2024-04-20T11:09:22.368 INFO:tasks.workunit.client.0.smithi117.stdout:4/918: link d1/dd/f68 d1/d45/f12d 0 2024-04-20T11:09:22.368 INFO:tasks.workunit.client.0.smithi117.stdout:8/829: chown d0/d9/dbb/l10a 1154 1 2024-04-20T11:09:22.369 INFO:tasks.workunit.client.0.smithi117.stdout:1/859: write d6/d1f/d30/d49/d69/d6e/d6f/d73/f88 [2495661,97072] 0 2024-04-20T11:09:22.370 INFO:tasks.workunit.client.0.smithi117.stdout:3/903: chown d0/d5/d21/d77/df3/f100 1758 1 2024-04-20T11:09:22.373 INFO:tasks.workunit.client.0.smithi117.stdout:8/830: fsync d0/d8/d39/f8a 0 2024-04-20T11:09:22.376 INFO:tasks.workunit.client.0.smithi117.stdout:0/975: read - d3/d7/d20/d39/d9a/dd0/d6d/f97 zero size 2024-04-20T11:09:22.379 INFO:tasks.workunit.client.0.smithi117.stdout:3/904: chown d0/d5/c38 1118587481 1 2024-04-20T11:09:22.379 INFO:tasks.workunit.client.0.smithi117.stdout:4/919: rmdir d1/dd/d1a/d43/d108 39 2024-04-20T11:09:22.381 INFO:tasks.workunit.client.0.smithi117.stdout:8/831: symlink d0/d9/d3a/de3/df1/l115 0 2024-04-20T11:09:22.383 INFO:tasks.workunit.client.0.smithi117.stdout:1/860: unlink d6/d16/d3a/d5a/daf/def/lfc 0 2024-04-20T11:09:22.384 INFO:tasks.workunit.client.0.smithi117.stdout:4/920: stat d1/dd/d1a/lbd 0 2024-04-20T11:09:22.388 INFO:tasks.workunit.client.0.smithi117.stdout:7/916: dwrite d1/d4/d6/d54/db4/ff8 [0,4194304] 0 2024-04-20T11:09:22.391 INFO:tasks.workunit.client.0.smithi117.stdout:0/976: chown d3/d7/d20/d39/d9a/dd0/l82 110471123 1 2024-04-20T11:09:22.398 INFO:tasks.workunit.client.0.smithi117.stdout:4/921: truncate d1/dd/d52/d117/f127 152128 0 2024-04-20T11:09:22.398 INFO:tasks.workunit.client.0.smithi117.stdout:0/977: chown d3/dab/d54/c61 4840898 1 2024-04-20T11:09:22.398 INFO:tasks.workunit.client.0.smithi117.stdout:8/832: write d0/f34 [1011354,34009] 0 2024-04-20T11:09:22.401 INFO:tasks.workunit.client.0.smithi117.stdout:0/978: stat d3/d7/d20/d47/f10d 0 2024-04-20T11:09:22.406 INFO:tasks.workunit.client.0.smithi117.stdout:1/861: getdents d6/d16/d3a/d5a/d5c/d61 0 2024-04-20T11:09:22.409 INFO:tasks.workunit.client.0.smithi117.stdout:1/862: chown d6/d1f/d30/dfd/d101/da2 72651629 1 2024-04-20T11:09:22.410 INFO:tasks.workunit.client.0.smithi117.stdout:3/905: dread d0/d5/d73/ded/f97 [0,4194304] 0 2024-04-20T11:09:22.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:22 smithi173 ceph-mon[23800]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:22.414 INFO:tasks.workunit.client.0.smithi117.stdout:8/833: rename d0/d8/d11/d24/d53/f87 to d0/d8/d11/d24/d2b/f116 0 2024-04-20T11:09:22.415 INFO:tasks.workunit.client.0.smithi117.stdout:0/979: truncate d3/d7/d20/d39/d9a/dd0/d6d/f71 1496195 0 2024-04-20T11:09:22.419 INFO:tasks.workunit.client.0.smithi117.stdout:7/917: dread d1/d43/d59/ffd [0,4194304] 0 2024-04-20T11:09:22.420 INFO:tasks.workunit.client.0.smithi117.stdout:8/834: read d0/f20 [2042011,76034] 0 2024-04-20T11:09:22.425 INFO:tasks.workunit.client.0.smithi117.stdout:3/906: link d0/d5/d73/ded/d66/d63/fe4 d0/d5/d21/d75/d89/f12f 0 2024-04-20T11:09:22.429 INFO:tasks.workunit.client.0.smithi117.stdout:8/835: mkdir d0/d8/d11/d24/d27/d117 0 2024-04-20T11:09:22.431 INFO:tasks.workunit.client.0.smithi117.stdout:7/918: truncate d1/d1d/d4e/d53/f10e 526918 0 2024-04-20T11:09:22.435 INFO:tasks.workunit.client.0.smithi117.stdout:7/919: chown d1/l31 78 1 2024-04-20T11:09:22.439 INFO:tasks.workunit.client.0.smithi117.stdout:1/863: dread d6/d16/d3a/d5a/daf/fc5 [0,4194304] 0 2024-04-20T11:09:22.439 INFO:tasks.workunit.client.0.smithi117.stdout:4/922: dwrite d1/d9/d79/f8c [0,4194304] 0 2024-04-20T11:09:22.439 INFO:tasks.workunit.client.0.smithi117.stdout:7/920: dread - d1/d4/f112 zero size 2024-04-20T11:09:22.444 INFO:tasks.workunit.client.0.smithi117.stdout:0/980: dwrite d3/d7/d2c/d52/fc5 [0,4194304] 0 2024-04-20T11:09:22.445 INFO:tasks.workunit.client.0.smithi117.stdout:4/923: symlink d1/dd/d52/d117/l12e 0 2024-04-20T11:09:22.448 INFO:tasks.workunit.client.0.smithi117.stdout:3/907: write d0/d5/fe2 [2911978,5465] 0 2024-04-20T11:09:22.450 INFO:tasks.workunit.client.0.smithi117.stdout:8/836: getdents d0/dd3 0 2024-04-20T11:09:22.455 INFO:tasks.workunit.client.0.smithi117.stdout:4/924: write d1/d45/d4a/d59/fce [5077893,6441] 0 2024-04-20T11:09:22.460 INFO:tasks.workunit.client.0.smithi117.stdout:8/837: chown d0/d8/d11/d24/d2b/d72/l8c 0 1 2024-04-20T11:09:22.463 INFO:tasks.workunit.client.0.smithi117.stdout:8/838: readlink d0/d8/d11/d24/d27/l4a 0 2024-04-20T11:09:22.467 INFO:tasks.workunit.client.0.smithi117.stdout:8/839: rmdir d0/dd3 39 2024-04-20T11:09:22.469 INFO:tasks.workunit.client.0.smithi117.stdout:1/864: dwrite d6/d16/d3a/f7e [0,4194304] 0 2024-04-20T11:09:22.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.474+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd8c02cde0 con 0x7efd7006ebe0 2024-04-20T11:09:22.476 INFO:tasks.workunit.client.0.smithi117.stdout:0/981: write d3/d7/d20/d6e/df3/dff/f129 [1718470,103408] 0 2024-04-20T11:09:22.476 INFO:tasks.workunit.client.0.smithi117.stdout:3/908: write d0/d5/d21/d75/d89/fe6 [716976,94460] 0 2024-04-20T11:09:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.478+0000 7efd83400700 1 -- 172.21.15.117:0/2416315390 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7efd8c02cde0 con 0x7efd7006ebe0 2024-04-20T11:09:22.481 INFO:tasks.workunit.client.0.smithi117.stdout:8/840: unlink d0/l21 0 2024-04-20T11:09:22.483 INFO:tasks.workunit.client.0.smithi117.stdout:1/865: creat d6/d21/f119 x:0 0 0 2024-04-20T11:09:22.484 INFO:tasks.workunit.client.0.smithi117.stdout:3/909: write d0/d5/d73/ded/db3/fb6 [536864,111300] 0 2024-04-20T11:09:22.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7efd7006ebe0 msgr2=0x7efd700710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:22.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7efd7006ebe0 0x7efd700710a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd740098e0 tx=0x7efd740055c0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c102e40 msgr2=0x7efd8c078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:22.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c102e40 0x7efd8c078c40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00ba50 tx=0x7efd7c00be10 comp rx=0 tx=0).stop 2024-04-20T11:09:22.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 shutdown_connections 2024-04-20T11:09:22.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7efd7006ebe0 0x7efd700710a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c102e40 0x7efd8c078c40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 --2- 172.21.15.117:0/2416315390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd8c103fa0 0x7efd8c079180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.485+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 >> 172.21.15.117:0/2416315390 conn(0x7efd8c0fe420 msgr2=0x7efd8c107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:22.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.486+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 shutdown_connections 2024-04-20T11:09:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.487+0000 7efd90e00700 1 -- 172.21.15.117:0/2416315390 wait complete. 2024-04-20T11:09:22.490 INFO:tasks.workunit.client.0.smithi117.stdout:0/982: fsync d3/d7/d20/d47/ff2 0 2024-04-20T11:09:22.497 INFO:tasks.workunit.client.0.smithi117.stdout:3/910: mknod d0/c130 0 2024-04-20T11:09:22.499 INFO:tasks.workunit.client.0.smithi117.stdout:4/925: dwrite d1/f57 [0,4194304] 0 2024-04-20T11:09:22.501 INFO:tasks.workunit.client.0.smithi117.stdout:0/983: mkdir d3/dab/d13a 0 2024-04-20T11:09:22.505 INFO:tasks.workunit.client.0.smithi117.stdout:1/866: rename d6/d16/dce/fd8 to d6/d16/d3a/d5a/d5c/d61/f11a 0 2024-04-20T11:09:22.508 INFO:tasks.workunit.client.0.smithi117.stdout:0/984: write d3/d7/d20/d6e/df3/dff/d85/ddb/f109 [160496,21074] 0 2024-04-20T11:09:22.511 INFO:tasks.workunit.client.0.smithi117.stdout:1/867: rmdir d6/d16/d3a/d5a/daf/def/df8 39 2024-04-20T11:09:22.516 INFO:tasks.workunit.client.0.smithi117.stdout:7/921: dread d1/d1d/f87 [0,4194304] 0 2024-04-20T11:09:22.521 INFO:tasks.workunit.client.0.smithi117.stdout:3/911: dread d0/d5/d21/fb5 [0,4194304] 0 2024-04-20T11:09:22.526 INFO:tasks.workunit.client.0.smithi117.stdout:7/922: dread d1/dd1/ff3 [0,4194304] 0 2024-04-20T11:09:22.528 INFO:tasks.workunit.client.0.smithi117.stdout:0/985: rename d3/d7/d20/d39/d3a/db1/l12e to d3/dab/l13b 0 2024-04-20T11:09:22.528 INFO:tasks.workunit.client.0.smithi117.stdout:4/926: dread d1/d45/d4a/fa9 [0,4194304] 0 2024-04-20T11:09:22.532 INFO:tasks.workunit.client.0.smithi117.stdout:8/841: dwrite d0/d8/d11/d24/d76/fb0 [0,4194304] 0 2024-04-20T11:09:22.542 INFO:tasks.workunit.client.0.smithi117.stdout:7/923: rename d1/d4/d6/d2b/d64/cda to d1/d11f/c12d 0 2024-04-20T11:09:22.543 INFO:tasks.workunit.client.0.smithi117.stdout:0/986: read d3/d7/d20/d6e/df3/dff/d85/ddb/f109 [69031,123350] 0 2024-04-20T11:09:22.548 INFO:tasks.workunit.client.0.smithi117.stdout:7/924: truncate d1/dd1/ff3 1017783 0 2024-04-20T11:09:22.552 INFO:tasks.workunit.client.0.smithi117.stdout:7/925: readlink d1/d4/d6/d54/lca 0 2024-04-20T11:09:22.554 INFO:tasks.workunit.client.0.smithi117.stdout:1/868: truncate d6/d1f/d30/d49/d69/d6e/d6f/d73/f88 8111155 0 2024-04-20T11:09:22.556 INFO:tasks.workunit.client.0.smithi117.stdout:8/842: write d0/d8/d11/d24/d27/d47/f7d [1983786,6584] 0 2024-04-20T11:09:22.557 INFO:tasks.workunit.client.0.smithi117.stdout:1/869: chown d6/d16/l54 4239450 1 2024-04-20T11:09:22.560 INFO:tasks.workunit.client.0.smithi117.stdout:0/987: link d3/d7/c87 d3/d7/d20/d39/d3a/db1/c13c 0 2024-04-20T11:09:22.560 INFO:tasks.workunit.client.0.smithi117.stdout:1/870: read - d6/d1f/d30/d49/d69/d6e/d6f/d73/f75 zero size 2024-04-20T11:09:22.567 INFO:tasks.workunit.client.0.smithi117.stdout:0/988: rmdir d3/dab 39 2024-04-20T11:09:22.570 INFO:tasks.workunit.client.0.smithi117.stdout:1/871: creat d6/d1f/d30/dfd/d101/f11b x:0 0 0 2024-04-20T11:09:22.576 INFO:tasks.workunit.client.0.smithi117.stdout:3/912: dwrite d0/d5/d73/ded/dac/dd2/f10e [0,4194304] 0 2024-04-20T11:09:22.581 INFO:tasks.workunit.client.0.smithi117.stdout:8/843: write d0/d8/d39/f54 [1239159,9787] 0 2024-04-20T11:09:22.582 INFO:tasks.workunit.client.0.smithi117.stdout:0/989: creat d3/dab/d54/f13d x:0 0 0 2024-04-20T11:09:22.583 INFO:tasks.workunit.client.0.smithi117.stdout:4/927: dwrite d1/dd/d52/d117/fb9 [0,4194304] 0 2024-04-20T11:09:22.587 INFO:tasks.workunit.client.0.smithi117.stdout:4/928: dread - d1/d9/d79/ff7 zero size 2024-04-20T11:09:22.590 INFO:tasks.workunit.client.0.smithi117.stdout:3/913: symlink d0/d5/d73/ded/dac/l131 0 2024-04-20T11:09:22.591 INFO:tasks.workunit.client.0.smithi117.stdout:7/926: link d1/d1d/d4e/d53/db1/cb5 d1/d1d/d42/d2d/c12e 0 2024-04-20T11:09:22.595 INFO:tasks.workunit.client.0.smithi117.stdout:4/929: chown d1/d9/ca6 56539 1 2024-04-20T11:09:22.596 INFO:tasks.workunit.client.0.smithi117.stdout:1/872: dwrite d6/d16/d3a/f7e [0,4194304] 0 2024-04-20T11:09:22.607 INFO:tasks.workunit.client.0.smithi117.stdout:1/873: stat d6/d21/fb8 0 2024-04-20T11:09:22.610 INFO:tasks.workunit.client.0.smithi117.stdout:0/990: dread d3/d7/f96 [0,4194304] 0 2024-04-20T11:09:22.613 INFO:tasks.workunit.client.0.smithi117.stdout:1/874: mknod d6/d16/d3a/d5a/d5c/df0/d10c/c11c 0 2024-04-20T11:09:22.621 INFO:tasks.workunit.client.0.smithi117.stdout:1/875: rename d6/d16/c85 to d6/d16/d3a/d5a/daf/de4/c11d 0 2024-04-20T11:09:22.627 INFO:tasks.workunit.client.0.smithi117.stdout:3/914: write d0/d2a/f3b [862026,4192] 0 2024-04-20T11:09:22.628 INFO:tasks.workunit.client.0.smithi117.stdout:1/876: mkdir d6/d16/d3a/d11e 0 2024-04-20T11:09:22.630 INFO:tasks.workunit.client.0.smithi117.stdout:8/844: truncate d0/d8/d39/d40/f51 130506 0 2024-04-20T11:09:22.637 INFO:tasks.workunit.client.0.smithi117.stdout:3/915: rmdir d0/d5/d21/d75/d89/d116 39 2024-04-20T11:09:22.641 INFO:tasks.workunit.client.0.smithi117.stdout:0/991: dwrite d3/d7/d20/d39/d9a/dd0/f102 [0,4194304] 0 2024-04-20T11:09:22.641 INFO:tasks.workunit.client.0.smithi117.stdout:3/916: stat d0/d5/d21/d94/d9a 0 2024-04-20T11:09:22.644 INFO:tasks.workunit.client.0.smithi117.stdout:0/992: stat d3/d7/d2c/d52/f70 0 2024-04-20T11:09:22.645 INFO:tasks.workunit.client.0.smithi117.stdout:7/927: dwrite d1/d1d/d42/d2d/f5f [4194304,4194304] 0 2024-04-20T11:09:22.646 INFO:tasks.workunit.client.0.smithi117.stdout:1/877: write d6/f18 [8046819,87888] 0 2024-04-20T11:09:22.648 INFO:tasks.workunit.client.0.smithi117.stdout:7/928: readlink d1/d1d/d42/d2d/lcd 0 2024-04-20T11:09:22.653 INFO:tasks.workunit.client.0.smithi117.stdout:4/930: dread d1/dd/f53 [0,4194304] 0 2024-04-20T11:09:22.653 INFO:tasks.workunit.client.0.smithi117.stdout:0/993: mknod d3/d7e/c13e 0 2024-04-20T11:09:22.656 INFO:tasks.workunit.client.0.smithi117.stdout:8/845: rename d0/d8/d11/d6d/d69 to d0/d9/d118 0 2024-04-20T11:09:22.659 INFO:tasks.workunit.client.0.smithi117.stdout:4/931: read d1/dd/d1a/d43/d4e/f62 [7605286,117445] 0 2024-04-20T11:09:22.664 INFO:tasks.workunit.client.0.smithi117.stdout:3/917: dread d0/d5/d21/f9c [0,4194304] 0 2024-04-20T11:09:22.668 INFO:tasks.workunit.client.0.smithi117.stdout:0/994: chown d3/dab/l13b 482 1 2024-04-20T11:09:22.673 INFO:tasks.workunit.client.0.smithi117.stdout:1/878: creat d6/d16/d3a/d5a/d5c/d61/d98/f11f x:0 0 0 2024-04-20T11:09:22.675 INFO:tasks.workunit.client.0.smithi117.stdout:3/918: mknod d0/d5/d21/d75/d89/dd1/df4/c132 0 2024-04-20T11:09:22.676 INFO:tasks.workunit.client.0.smithi117.stdout:4/932: symlink d1/l12f 0 2024-04-20T11:09:22.681 INFO:tasks.workunit.client.0.smithi117.stdout:4/933: read d1/dd/d52/d117/fae [128318,67413] 0 2024-04-20T11:09:22.682 INFO:tasks.workunit.client.0.smithi117.stdout:4/934: stat d1/cd4 0 2024-04-20T11:09:22.682 INFO:tasks.workunit.client.0.smithi117.stdout:7/929: dread d1/d1d/d4e/d53/d55/f94 [0,4194304] 0 2024-04-20T11:09:22.686 INFO:tasks.workunit.client.0.smithi117.stdout:1/879: creat d6/d1f/d30/d49/d69/d6e/d6f/d73/d111/f120 x:0 0 0 2024-04-20T11:09:22.690 INFO:tasks.workunit.client.0.smithi117.stdout:8/846: write d0/d8/d11/d24/d2b/fab [968202,97711] 0 2024-04-20T11:09:22.691 INFO:tasks.workunit.client.0.smithi117.stdout:4/935: mkdir d1/dd/d52/d130 0 2024-04-20T11:09:22.697 INFO:tasks.workunit.client.0.smithi117.stdout:1/880: symlink d6/d16/d3a/d5a/d5c/d61/d98/deb/l121 0 2024-04-20T11:09:22.701 INFO:tasks.workunit.client.0.smithi117.stdout:4/936: chown d1/d9/d1f/c5e 129923 1 2024-04-20T11:09:22.706 INFO:tasks.workunit.client.0.smithi117.stdout:3/919: dwrite d0/d5/d73/ded/d66/d63/dad/f108 [0,4194304] 0 2024-04-20T11:09:22.709 INFO:tasks.workunit.client.0.smithi117.stdout:7/930: dwrite d1/d4/f11e [0,4194304] 0 2024-04-20T11:09:22.712 INFO:tasks.workunit.client.0.smithi117.stdout:0/995: dread d3/d7/d20/d47/fc2 [0,4194304] 0 2024-04-20T11:09:22.712 INFO:tasks.workunit.client.0.smithi117.stdout:4/937: rename d1/d9/d1f/d90/dde to d1/dd/d1a/d2b/d86/d11b/d131 0 2024-04-20T11:09:22.715 INFO:tasks.workunit.client.0.smithi117.stdout:7/931: chown d1/d1d/d42/d2d/d6c 8493299 1 2024-04-20T11:09:22.716 INFO:tasks.workunit.client.0.smithi117.stdout:3/920: mknod d0/d5/de/d4e/dec/c133 0 2024-04-20T11:09:22.716 INFO:tasks.workunit.client.0.smithi117.stdout:0/996: write d3/d7/d2c/d52/fd3 [1527785,59117] 0 2024-04-20T11:09:22.720 INFO:tasks.workunit.client.0.smithi117.stdout:0/997: write d3/d7/f4d [4054716,95514] 0 2024-04-20T11:09:22.720 INFO:tasks.workunit.client.0.smithi117.stdout:4/938: read d1/d45/fa1 [2899152,128473] 0 2024-04-20T11:09:22.721 INFO:tasks.workunit.client.0.smithi117.stdout:8/847: rmdir d0/d8/d11/d24/d86/d109 0 2024-04-20T11:09:22.729 INFO:tasks.workunit.client.0.smithi117.stdout:3/921: truncate d0/fb 1154369 0 2024-04-20T11:09:22.734 INFO:tasks.workunit.client.0.smithi117.stdout:3/922: chown d0/d5/d21/d94/fb1 735728 1 2024-04-20T11:09:22.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.742+0000 7fdb2e800700 1 -- 172.21.15.117:0/2885570217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 msgr2=0x7fdb28073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:22.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.742+0000 7fdb2e800700 1 --2- 172.21.15.117:0/2885570217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 0x7fdb28073dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009910 tx=0x7fdb18009c20 comp rx=0 tx=0).stop 2024-04-20T11:09:22.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.743+0000 7fdb2e800700 1 -- 172.21.15.117:0/2885570217 shutdown_connections 2024-04-20T11:09:22.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.743+0000 7fdb2e800700 1 --2- 172.21.15.117:0/2885570217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 0x7fdb28073dc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.743+0000 7fdb2e800700 1 --2- 172.21.15.117:0/2885570217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb28074f10 0x7fdb28073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.743+0000 7fdb2e800700 1 -- 172.21.15.117:0/2885570217 >> 172.21.15.117:0/2885570217 conn(0x7fdb280fc1c0 msgr2=0x7fdb280fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:22.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.745+0000 7fdb2e800700 1 -- 172.21.15.117:0/2885570217 shutdown_connections 2024-04-20T11:09:22.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.746+0000 7fdb2e800700 1 -- 172.21.15.117:0/2885570217 wait complete. 2024-04-20T11:09:22.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.748+0000 7fdb2e800700 1 Processor -- start 2024-04-20T11:09:22.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.748+0000 7fdb2e800700 1 -- start start 2024-04-20T11:09:22.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.748+0000 7fdb2e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 0x7fdb28194500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:22.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.748+0000 7fdb2e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb28074f10 0x7fdb28194a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:22.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.749+0000 7fdb2e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28195060 con 0x7fdb28074f10 2024-04-20T11:09:22.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.749+0000 7fdb2e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281951a0 con 0x7fdb28073940 2024-04-20T11:09:22.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.749+0000 7fdb27400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 0x7fdb28194500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:22.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.749+0000 7fdb27400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 0x7fdb28194500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:47698/0 (socket says 172.21.15.117:47698) 2024-04-20T11:09:22.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.749+0000 7fdb27400700 1 -- 172.21.15.117:0/2622117065 learned_addr learned my addr 172.21.15.117:0/2622117065 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:22.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.750+0000 7fdb27400700 1 -- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb28074f10 msgr2=0x7fdb28194a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:22.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.750+0000 7fdb27400700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb28074f10 0x7fdb28194a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:22.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.750+0000 7fdb27400700 1 -- 172.21.15.117:0/2622117065 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb180095f0 con 0x7fdb28073940 2024-04-20T11:09:22.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.750+0000 7fdb27400700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 0x7fdb28194500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000b9c0 tx=0x7fdb2000bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:22.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.750+0000 7fdb1fe00700 1 -- 172.21.15.117:0/2622117065 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2000c980 con 0x7fdb28073940 2024-04-20T11:09:22.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.750+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb28199c00 con 0x7fdb28073940 2024-04-20T11:09:22.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.750+0000 7fdb1fe00700 1 -- 172.21.15.117:0/2622117065 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb20014440 con 0x7fdb28073940 2024-04-20T11:09:22.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.751+0000 7fdb1fe00700 1 -- 172.21.15.117:0/2622117065 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb20012570 con 0x7fdb28073940 2024-04-20T11:09:22.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.751+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2819a1d0 con 0x7fdb28073940 2024-04-20T11:09:22.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.754+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb08005320 con 0x7fdb28073940 2024-04-20T11:09:22.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.754+0000 7fdb1fe00700 1 -- 172.21.15.117:0/2622117065 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7fdb2000cae0 con 0x7fdb28073940 2024-04-20T11:09:22.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.755+0000 7fdb1fe00700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdb0c06eb10 0x7fdb0c070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:22.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.755+0000 7fdb1fe00700 1 -- 172.21.15.117:0/2622117065 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fdb20006e40 con 0x7fdb28073940 2024-04-20T11:09:22.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.756+0000 7fdb26a00700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdb0c06eb10 0x7fdb0c070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:22.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.757+0000 7fdb26a00700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdb0c06eb10 0x7fdb0c070fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb180098e0 tx=0x7fdb180050c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:22.757 INFO:tasks.workunit.client.0.smithi117.stdout:0/998: dwrite d3/d7/d20/d39/d9a/dd0/d6d/d10c/f134 [0,4194304] 0 2024-04-20T11:09:22.767 INFO:tasks.workunit.client.0.smithi117.stdout:0/999: mkdir d3/d7e/d13f 0 2024-04-20T11:09:22.769 INFO:tasks.workunit.client.0.smithi117.stdout:7/932: dread d1/d43/f80 [0,4194304] 0 2024-04-20T11:09:22.772 INFO:tasks.workunit.client.0.smithi117.stdout:8/848: dread d0/d9/f95 [0,4194304] 0 2024-04-20T11:09:22.776 INFO:tasks.workunit.client.0.smithi117.stdout:3/923: dwrite d0/d5/d21/d88/ff5 [0,4194304] 0 2024-04-20T11:09:22.778 INFO:tasks.workunit.client.0.smithi117.stdout:3/924: fsync d0/d5/d73/ded/d66/d63/fa9 0 2024-04-20T11:09:22.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:22.781+0000 7fdb1fe00700 1 -- 172.21.15.117:0/2622117065 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fdb2005ab40 con 0x7fdb28073940 2024-04-20T11:09:22.784 INFO:tasks.workunit.client.0.smithi117.stdout:8/849: mkdir d0/d8/d39/d40/de5/d119 0 2024-04-20T11:09:22.787 INFO:tasks.workunit.client.0.smithi117.stdout:8/850: chown d0/d9/f95 4 1 2024-04-20T11:09:22.803 INFO:tasks.workunit.client.0.smithi117.stdout:8/851: write d0/d8/d11/d6d/d84/f112 [2529175,74185] 0 2024-04-20T11:09:22.806 INFO:tasks.workunit.client.0.smithi117.stdout:3/925: dwrite d0/f98 [0,4194304] 0 2024-04-20T11:09:22.814 INFO:tasks.workunit.client.0.smithi117.stdout:8/852: chown d0/dc/d49/cc2 8581844 1 2024-04-20T11:09:22.819 INFO:tasks.workunit.client.0.smithi117.stdout:7/933: dread d1/f10 [0,4194304] 0 2024-04-20T11:09:22.824 INFO:tasks.workunit.client.0.smithi117.stdout:8/853: fdatasync d0/f8f 0 2024-04-20T11:09:22.826 INFO:tasks.workunit.client.0.smithi117.stdout:3/926: dread d0/d5/f40 [4194304,4194304] 0 2024-04-20T11:09:22.837 INFO:tasks.workunit.client.0.smithi117.stdout:3/927: truncate d0/d5/d21/d88/d69/f8e 695358 0 2024-04-20T11:09:22.838 INFO:tasks.workunit.client.0.smithi117.stdout:8/854: mknod d0/d9/dbb/d113/c11a 0 2024-04-20T11:09:22.848 INFO:tasks.workunit.client.0.smithi117.stdout:4/939: write d1/dd/d1a/d2b/d86/f102 [1231303,64538] 0 2024-04-20T11:09:22.849 INFO:tasks.workunit.client.0.smithi117.stdout:7/934: link d1/d4/d6/d54/d63/cf2 d1/d43/d59/c12f 0 2024-04-20T11:09:22.857 INFO:tasks.workunit.client.0.smithi117.stdout:7/935: mkdir d1/d4/d6/d13/d130 0 2024-04-20T11:09:22.860 INFO:tasks.workunit.client.0.smithi117.stdout:8/855: dwrite d0/d8/d39/fe2 [0,4194304] 0 2024-04-20T11:09:22.865 INFO:tasks.workunit.client.0.smithi117.stdout:7/936: chown d1/d1d/d4e/l95 12719006 1 2024-04-20T11:09:22.869 INFO:tasks.workunit.client.0.smithi117.stdout:4/940: creat d1/d9/f132 x:0 0 0 2024-04-20T11:09:22.873 INFO:tasks.workunit.client.0.smithi117.stdout:8/856: dread d0/d9/d3a/fd4 [0,4194304] 0 2024-04-20T11:09:22.879 INFO:tasks.workunit.client.0.smithi117.stdout:4/941: creat d1/dd/d1a/d43/dcc/f133 x:0 0 0 2024-04-20T11:09:22.882 INFO:tasks.workunit.client.0.smithi117.stdout:4/942: readlink d1/d9/de9/d9e/lff 0 2024-04-20T11:09:22.883 INFO:tasks.workunit.client.0.smithi117.stdout:8/857: unlink d0/d9/d3a/d7c/fa1 0 2024-04-20T11:09:22.889 INFO:tasks.workunit.client.0.smithi117.stdout:7/937: dwrite d1/d4/d3d/dce/deb/ffa [0,4194304] 0 2024-04-20T11:09:22.892 INFO:tasks.workunit.client.0.smithi117.stdout:8/858: chown d0/d8/l2f 2 1 2024-04-20T11:09:22.895 INFO:tasks.workunit.client.0.smithi117.stdout:4/943: link d1/f81 d1/d45/d4a/f134 0 2024-04-20T11:09:22.896 INFO:tasks.workunit.client.0.smithi117.stdout:7/938: mknod d1/d4/d6/d13/d118/c131 0 2024-04-20T11:09:22.901 INFO:tasks.workunit.client.0.smithi117.stdout:8/859: creat d0/d9/dbb/d113/f11b x:0 0 0 2024-04-20T11:09:22.902 INFO:tasks.workunit.client.0.smithi117.stdout:4/944: fsync d1/f19 0 2024-04-20T11:09:22.912 INFO:tasks.workunit.client.0.smithi117.stdout:7/939: link d1/d43/d59/d111/f120 d1/d4/d3d/d93/d117/f132 0 2024-04-20T11:09:22.928 INFO:tasks.workunit.client.0.smithi117.stdout:8/860: fsync d0/d9/dbb/d113/f11b 0 2024-04-20T11:09:22.936 INFO:tasks.workunit.client.0.smithi117.stdout:7/940: truncate d1/d43/d59/ffd 3495771 0 2024-04-20T11:09:22.937 INFO:tasks.workunit.client.0.smithi117.stdout:4/945: write d1/f81 [1688081,81281] 0 2024-04-20T11:09:22.946 INFO:tasks.workunit.client.0.smithi117.stdout:8/861: dread d0/d8/d39/d40/d78/f97 [0,4194304] 0 2024-04-20T11:09:22.960 INFO:tasks.workunit.client.0.smithi117.stdout:7/941: chown d1/d1d/d42/d2d/c60 0 1 2024-04-20T11:09:22.963 INFO:tasks.workunit.client.0.smithi117.stdout:7/942: chown d1/d4/d6/d13/fef 113503170 1 2024-04-20T11:09:22.965 INFO:tasks.workunit.client.0.smithi117.stdout:7/943: chown d1/d1d/d42/d2d/d6c 207 1 2024-04-20T11:09:22.970 INFO:tasks.workunit.client.0.smithi117.stdout:7/944: symlink d1/d4/d6/d13/d118/l133 0 2024-04-20T11:09:22.990 INFO:tasks.workunit.client.0.smithi117.stdout:4/946: write d1/dd/f53 [4217582,26559] 0 2024-04-20T11:09:22.991 INFO:tasks.workunit.client.0.smithi117.stdout:8/862: dwrite d0/d8/d11/d6d/d84/fc1 [0,4194304] 0 2024-04-20T11:09:22.996 INFO:tasks.workunit.client.0.smithi117.stdout:4/947: truncate d1/dd/d1a/fe0 48989 0 2024-04-20T11:09:22.999 INFO:tasks.workunit.client.0.smithi117.stdout:4/948: fsync d1/dd/d1a/d43/d4e/f9b 0 2024-04-20T11:09:23.000 INFO:tasks.workunit.client.0.smithi117.stdout:8/863: rename d0/c61 to d0/d9/d3a/d7c/c11c 0 2024-04-20T11:09:23.000 INFO:tasks.workunit.client.1.smithi173.stderr:++ readlink -f fsstress 2024-04-20T11:09:23.001 INFO:tasks.workunit.client.0.smithi117.stdout:7/945: dwrite d1/d4/d3d/f7c [0,4194304] 0 2024-04-20T11:09:23.005 INFO:tasks.workunit.client.0.smithi117.stdout:4/949: mkdir d1/d9/d1f/d90/d107/d12b/d135 0 2024-04-20T11:09:23.009 INFO:tasks.workunit.client.1.smithi173.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-20T11:09:23.009 INFO:tasks.workunit.client.1.smithi173.stderr:+ popd 2024-04-20T11:09:23.009 INFO:tasks.workunit.client.0.smithi117.stdout:4/950: write d1/dd/d1a/d43/f129 [633902,40001] 0 2024-04-20T11:09:23.016 INFO:tasks.workunit.client.1.smithi173.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-20T11:09:23.016 INFO:tasks.workunit.client.1.smithi173.stderr:+ popd 2024-04-20T11:09:23.017 INFO:tasks.workunit.client.0.smithi117.stdout:7/946: write d1/d4/d6/d2b/d64/f78 [3196199,85953] 0 2024-04-20T11:09:23.020 INFO:tasks.workunit.client.0.smithi117.stdout:7/947: write d1/f10 [5435996,60307] 0 2024-04-20T11:09:23.021 INFO:tasks.workunit.client.1.smithi173.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-20T11:09:23.022 INFO:tasks.workunit.client.1.smithi173.stderr:++ mktemp -d -p . 2024-04-20T11:09:23.025 INFO:tasks.workunit.client.0.smithi117.stdout:7/948: mknod d1/d1d/d4e/d53/db1/c134 0 2024-04-20T11:09:23.027 INFO:tasks.workunit.client.0.smithi117.stdout:8/864: dread d0/d8/d11/d24/d27/d47/f8e [0,4194304] 0 2024-04-20T11:09:23.030 INFO:tasks.workunit.client.0.smithi117.stdout:8/865: chown d0/d8/d11/d24/d27/d47/f8e 0 1 2024-04-20T11:09:23.031 INFO:tasks.workunit.client.0.smithi117.stdout:7/949: fsync d1/d4/d6/d54/fe5 0 2024-04-20T11:09:23.031 INFO:tasks.workunit.client.1.smithi173.stderr:+ T=./tmp.8cjNyXBYLx 2024-04-20T11:09:23.031 INFO:tasks.workunit.client.1.smithi173.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.8cjNyXBYLx -l 1 -n 1000 -p 10 -v 2024-04-20T11:09:23.035 INFO:tasks.workunit.client.0.smithi117.stdout:8/866: creat d0/dc/d49/dd5/f11d x:0 0 0 2024-04-20T11:09:23.037 INFO:tasks.workunit.client.0.smithi117.stdout:7/950: creat d1/d1d/d77/dee/d124/f135 x:0 0 0 2024-04-20T11:09:23.039 INFO:tasks.workunit.client.0.smithi117.stdout:7/951: readlink d1/d4/d6/d13/d118/l133 0 2024-04-20T11:09:23.041 INFO:tasks.workunit.client.0.smithi117.stdout:8/867: unlink d0/dc/d2e/d89/fbe 0 2024-04-20T11:09:23.043 INFO:tasks.workunit.client.0.smithi117.stdout:7/952: write d1/d1d/d4e/d53/d55/f108 [182413,106633] 0 2024-04-20T11:09:23.044 INFO:tasks.workunit.client.1.smithi173.stdout:seed = 1713574754 2024-04-20T11:09:23.051 INFO:tasks.workunit.client.0.smithi117.stdout:7/953: symlink d1/d4/d6/d54/l136 0 2024-04-20T11:09:23.057 INFO:tasks.workunit.client.0.smithi117.stdout:7/954: creat d1/d1d/d42/d2d/d6c/f137 x:0 0 0 2024-04-20T11:09:23.059 INFO:tasks.workunit.client.0.smithi117.stdout:8/868: write d0/fdc [584095,11911] 0 2024-04-20T11:09:23.059 INFO:tasks.workunit.client.1.smithi173.stdout:1/0: dread - no filename 2024-04-20T11:09:23.059 INFO:tasks.workunit.client.1.smithi173.stdout:1/1: dread - no filename 2024-04-20T11:09:23.059 INFO:tasks.workunit.client.1.smithi173.stdout:1/2: stat - no entries 2024-04-20T11:09:23.059 INFO:tasks.workunit.client.1.smithi173.stdout:1/3: dread - no filename 2024-04-20T11:09:23.059 INFO:tasks.workunit.client.1.smithi173.stdout:1/4: readlink - no filename 2024-04-20T11:09:23.062 INFO:tasks.workunit.client.1.smithi173.stdout:0/0: getdents . 0 2024-04-20T11:09:23.062 INFO:tasks.workunit.client.1.smithi173.stdout:0/1: rmdir - no directory 2024-04-20T11:09:23.062 INFO:tasks.workunit.client.1.smithi173.stdout:2/0: dread - no filename 2024-04-20T11:09:23.062 INFO:tasks.workunit.client.1.smithi173.stdout:2/1: link - no file 2024-04-20T11:09:23.062 INFO:tasks.workunit.client.1.smithi173.stdout:2/2: readlink - no filename 2024-04-20T11:09:23.062 INFO:tasks.workunit.client.1.smithi173.stdout:2/3: link - no file 2024-04-20T11:09:23.062 INFO:tasks.workunit.client.1.smithi173.stdout:2/4: dwrite - no filename 2024-04-20T11:09:23.062 INFO:tasks.workunit.client.1.smithi173.stdout:0/2: chown . 851 1 2024-04-20T11:09:23.064 INFO:tasks.workunit.client.0.smithi117.stdout:7/955: truncate d1/f86 2547159 0 2024-04-20T11:09:23.064 INFO:tasks.workunit.client.1.smithi173.stdout:1/5: creat f0 x:0 0 0 2024-04-20T11:09:23.066 INFO:tasks.workunit.client.1.smithi173.stdout:1/6: chown f0 3829 1 2024-04-20T11:09:23.066 INFO:tasks.workunit.client.0.smithi117.stdout:4/951: dread d1/d9/f96 [0,4194304] 0 2024-04-20T11:09:23.067 INFO:tasks.workunit.client.1.smithi173.stdout:3/0: dread - no filename 2024-04-20T11:09:23.067 INFO:tasks.workunit.client.1.smithi173.stdout:3/1: write - no filename 2024-04-20T11:09:23.067 INFO:tasks.workunit.client.1.smithi173.stdout:3/2: chown . 25753111 1 2024-04-20T11:09:23.067 INFO:tasks.workunit.client.1.smithi173.stdout:3/3: write - no filename 2024-04-20T11:09:23.068 INFO:tasks.workunit.client.1.smithi173.stdout:3/4: write - no filename 2024-04-20T11:09:23.068 INFO:tasks.workunit.client.0.smithi117.stdout:8/869: mkdir d0/d9/d11e 0 2024-04-20T11:09:23.069 INFO:tasks.workunit.client.1.smithi173.stdout:1/7: mknod c1 0 2024-04-20T11:09:23.072 INFO:tasks.workunit.client.1.smithi173.stdout:0/3: mknod c0 0 2024-04-20T11:09:23.075 INFO:tasks.workunit.client.0.smithi117.stdout:7/956: truncate d1/d4/d6/d13/ff1 732000 0 2024-04-20T11:09:23.078 INFO:tasks.workunit.client.1.smithi173.stdout:1/8: rename c1 to c2 0 2024-04-20T11:09:23.078 INFO:tasks.workunit.client.1.smithi173.stdout:1/9: readlink - no filename 2024-04-20T11:09:23.081 INFO:tasks.workunit.client.1.smithi173.stdout:5/0: rename - no filename 2024-04-20T11:09:23.082 INFO:tasks.workunit.client.0.smithi117.stdout:4/952: mknod d1/d9/c136 0 2024-04-20T11:09:23.085 INFO:tasks.workunit.client.1.smithi173.stdout:0/4: creat f1 x:0 0 0 2024-04-20T11:09:23.086 INFO:tasks.workunit.client.1.smithi173.stdout:3/5: symlink l0 0 2024-04-20T11:09:23.086 INFO:tasks.workunit.client.1.smithi173.stdout:3/6: truncate - no filename 2024-04-20T11:09:23.087 INFO:tasks.workunit.client.0.smithi117.stdout:8/870: creat d0/d8/d11/d6d/d58/f11f x:0 0 0 2024-04-20T11:09:23.087 INFO:tasks.workunit.client.1.smithi173.stdout:0/5: truncate f1 584971 0 2024-04-20T11:09:23.087 INFO:tasks.workunit.client.1.smithi173.stdout:3/7: chown l0 826600 1 2024-04-20T11:09:23.088 INFO:tasks.workunit.client.0.smithi117.stdout:7/957: symlink d1/d4/l138 0 2024-04-20T11:09:23.090 INFO:tasks.workunit.client.1.smithi173.stdout:6/0: dwrite - no filename 2024-04-20T11:09:23.090 INFO:tasks.workunit.client.1.smithi173.stdout:6/1: dwrite - no filename 2024-04-20T11:09:23.091 INFO:tasks.workunit.client.1.smithi173.stdout:4/0: mknod c0 0 2024-04-20T11:09:23.092 INFO:tasks.workunit.client.0.smithi117.stdout:4/953: creat d1/dd/d1a/d43/d108/f137 x:0 0 0 2024-04-20T11:09:23.093 INFO:tasks.workunit.client.1.smithi173.stdout:0/6: creat f2 x:0 0 0 2024-04-20T11:09:23.093 INFO:tasks.workunit.client.0.smithi117.stdout:7/958: truncate d1/d4/d3d/dce/deb/f123 581718 0 2024-04-20T11:09:23.094 INFO:tasks.workunit.client.1.smithi173.stdout:3/8: unlink l0 0 2024-04-20T11:09:23.094 INFO:tasks.workunit.client.1.smithi173.stdout:3/9: link - no file 2024-04-20T11:09:23.094 INFO:tasks.workunit.client.1.smithi173.stdout:3/10: truncate - no filename 2024-04-20T11:09:23.094 INFO:tasks.workunit.client.1.smithi173.stdout:3/11: write - no filename 2024-04-20T11:09:23.095 INFO:tasks.workunit.client.1.smithi173.stdout:3/12: write - no filename 2024-04-20T11:09:23.095 INFO:tasks.workunit.client.1.smithi173.stdout:3/13: dread - no filename 2024-04-20T11:09:23.095 INFO:tasks.workunit.client.1.smithi173.stdout:3/14: link - no file 2024-04-20T11:09:23.095 INFO:tasks.workunit.client.1.smithi173.stdout:3/15: truncate - no filename 2024-04-20T11:09:23.095 INFO:tasks.workunit.client.1.smithi173.stdout:3/16: rename - no filename 2024-04-20T11:09:23.095 INFO:tasks.workunit.client.1.smithi173.stdout:3/17: dwrite - no filename 2024-04-20T11:09:23.096 INFO:tasks.workunit.client.1.smithi173.stdout:7/0: write - no filename 2024-04-20T11:09:23.096 INFO:tasks.workunit.client.1.smithi173.stdout:7/1: write - no filename 2024-04-20T11:09:23.097 INFO:tasks.workunit.client.1.smithi173.stdout:7/2: rmdir - no directory 2024-04-20T11:09:23.098 INFO:tasks.workunit.client.1.smithi173.stdout:5/1: mkdir d0 0 2024-04-20T11:09:23.098 INFO:tasks.workunit.client.1.smithi173.stdout:5/2: truncate - no filename 2024-04-20T11:09:23.098 INFO:tasks.workunit.client.1.smithi173.stdout:5/3: fsync - no filename 2024-04-20T11:09:23.098 INFO:tasks.workunit.client.1.smithi173.stdout:5/4: readlink - no filename 2024-04-20T11:09:23.098 INFO:tasks.workunit.client.1.smithi173.stdout:5/5: dwrite - no filename 2024-04-20T11:09:23.099 INFO:tasks.workunit.client.1.smithi173.stdout:4/1: creat f1 x:0 0 0 2024-04-20T11:09:23.100 INFO:tasks.workunit.client.1.smithi173.stdout:3/18: mknod c1 0 2024-04-20T11:09:23.101 INFO:tasks.workunit.client.1.smithi173.stdout:6/2: mknod c0 0 2024-04-20T11:09:23.101 INFO:tasks.workunit.client.1.smithi173.stdout:6/3: dwrite - no filename 2024-04-20T11:09:23.101 INFO:tasks.workunit.client.1.smithi173.stdout:6/4: dread - no filename 2024-04-20T11:09:23.103 INFO:tasks.workunit.client.1.smithi173.stdout:6/5: stat c0 0 2024-04-20T11:09:23.103 INFO:tasks.workunit.client.1.smithi173.stdout:6/6: dread - no filename 2024-04-20T11:09:23.103 INFO:tasks.workunit.client.1.smithi173.stdout:6/7: dread - no filename 2024-04-20T11:09:23.105 INFO:tasks.workunit.client.1.smithi173.stdout:3/19: creat f2 x:0 0 0 2024-04-20T11:09:23.105 INFO:tasks.workunit.client.0.smithi117.stdout:4/954: dread d1/d45/d4a/fba [0,4194304] 0 2024-04-20T11:09:23.106 INFO:tasks.workunit.client.1.smithi173.stdout:7/3: mknod c0 0 2024-04-20T11:09:23.106 INFO:tasks.workunit.client.1.smithi173.stdout:7/4: dwrite - no filename 2024-04-20T11:09:23.106 INFO:tasks.workunit.client.1.smithi173.stdout:7/5: dwrite - no filename 2024-04-20T11:09:23.106 INFO:tasks.workunit.client.1.smithi173.stdout:7/6: write - no filename 2024-04-20T11:09:23.107 INFO:tasks.workunit.client.1.smithi173.stdout:6/8: symlink l1 0 2024-04-20T11:09:23.108 INFO:tasks.workunit.client.1.smithi173.stdout:6/9: write - no filename 2024-04-20T11:09:23.108 INFO:tasks.workunit.client.1.smithi173.stdout:6/10: dwrite - no filename 2024-04-20T11:09:23.109 INFO:tasks.workunit.client.0.smithi117.stdout:8/871: dwrite d0/d8/d11/d6d/d84/fc3 [0,4194304] 0 2024-04-20T11:09:23.110 INFO:tasks.workunit.client.1.smithi173.stdout:7/7: creat f1 x:0 0 0 2024-04-20T11:09:23.111 INFO:tasks.workunit.client.1.smithi173.stdout:6/11: mknod c2 0 2024-04-20T11:09:23.112 INFO:tasks.workunit.client.1.smithi173.stdout:6/12: write - no filename 2024-04-20T11:09:23.112 INFO:tasks.workunit.client.1.smithi173.stdout:6/13: fsync - no filename 2024-04-20T11:09:23.113 INFO:tasks.workunit.client.0.smithi117.stdout:4/955: truncate d1/dd/d1a/d2b/d86/f112 4585180 0 2024-04-20T11:09:23.114 INFO:tasks.workunit.client.1.smithi173.stdout:8/0: symlink l0 0 2024-04-20T11:09:23.114 INFO:tasks.workunit.client.1.smithi173.stdout:8/1: read - no filename 2024-04-20T11:09:23.114 INFO:tasks.workunit.client.1.smithi173.stdout:8/2: write - no filename 2024-04-20T11:09:23.114 INFO:tasks.workunit.client.1.smithi173.stdout:8/3: fdatasync - no filename 2024-04-20T11:09:23.114 INFO:tasks.workunit.client.1.smithi173.stdout:8/4: fsync - no filename 2024-04-20T11:09:23.114 INFO:tasks.workunit.client.1.smithi173.stdout:8/5: read - no filename 2024-04-20T11:09:23.114 INFO:tasks.workunit.client.1.smithi173.stdout:8/6: dread - no filename 2024-04-20T11:09:23.115 INFO:tasks.workunit.client.1.smithi173.stdout:3/20: link f2 f3 0 2024-04-20T11:09:23.116 INFO:tasks.workunit.client.1.smithi173.stdout:7/8: unlink c0 0 2024-04-20T11:09:23.117 INFO:tasks.workunit.client.1.smithi173.stdout:6/14: symlink l3 0 2024-04-20T11:09:23.118 INFO:tasks.workunit.client.1.smithi173.stdout:3/21: dread - f3 zero size 2024-04-20T11:09:23.120 INFO:tasks.workunit.client.1.smithi173.stdout:7/9: chown f1 6187781 1 2024-04-20T11:09:23.120 INFO:tasks.workunit.client.1.smithi173.stdout:9/0: creat f0 x:0 0 0 2024-04-20T11:09:23.121 INFO:tasks.workunit.client.1.smithi173.stdout:6/15: mkdir d4 0 2024-04-20T11:09:23.121 INFO:tasks.workunit.client.1.smithi173.stdout:6/16: fdatasync - no filename 2024-04-20T11:09:23.122 INFO:tasks.workunit.client.1.smithi173.stdout:6/17: truncate - no filename 2024-04-20T11:09:23.122 INFO:tasks.workunit.client.1.smithi173.stdout:6/18: dread - no filename 2024-04-20T11:09:23.122 INFO:tasks.workunit.client.1.smithi173.stdout:6/19: truncate - no filename 2024-04-20T11:09:23.122 INFO:tasks.workunit.client.1.smithi173.stdout:6/20: dwrite - no filename 2024-04-20T11:09:23.122 INFO:tasks.workunit.client.1.smithi173.stdout:0/7: dwrite f1 [0,4194304] 0 2024-04-20T11:09:23.123 INFO:tasks.workunit.client.1.smithi173.stdout:7/10: chown f1 5284 1 2024-04-20T11:09:23.123 INFO:tasks.workunit.client.0.smithi117.stdout:7/959: dread d1/d4/d39/f34 [0,4194304] 0 2024-04-20T11:09:23.124 INFO:tasks.workunit.client.1.smithi173.stdout:0/8: dread - f2 zero size 2024-04-20T11:09:23.126 INFO:tasks.workunit.client.0.smithi117.stdout:7/960: write d1/d1d/d77/dee/f121 [914125,38600] 0 2024-04-20T11:09:23.127 INFO:tasks.workunit.client.0.smithi117.stdout:4/956: link d1/dd/d52/d117/lb7 d1/dd/d1a/l138 0 2024-04-20T11:09:23.128 INFO:tasks.workunit.client.1.smithi173.stdout:9/1: mkdir d1 0 2024-04-20T11:09:23.128 INFO:tasks.workunit.client.1.smithi173.stdout:0/9: read - f2 zero size 2024-04-20T11:09:23.129 INFO:tasks.workunit.client.1.smithi173.stdout:9/2: stat f0 0 2024-04-20T11:09:23.129 INFO:tasks.workunit.client.0.smithi117.stdout:4/957: chown d1/d9/c51 44740474 1 2024-04-20T11:09:23.129 INFO:tasks.workunit.client.0.smithi117.stdout:7/961: dread - d1/d4/d6/d54/f10b zero size 2024-04-20T11:09:23.130 INFO:tasks.workunit.client.1.smithi173.stdout:4/2: dwrite f1 [0,4194304] 0 2024-04-20T11:09:23.130 INFO:tasks.workunit.client.1.smithi173.stdout:7/11: symlink l2 0 2024-04-20T11:09:23.131 INFO:tasks.workunit.client.1.smithi173.stdout:8/7: link l0 l1 0 2024-04-20T11:09:23.131 INFO:tasks.workunit.client.1.smithi173.stdout:8/8: write - no filename 2024-04-20T11:09:23.131 INFO:tasks.workunit.client.1.smithi173.stdout:8/9: write - no filename 2024-04-20T11:09:23.131 INFO:tasks.workunit.client.1.smithi173.stdout:8/10: read - no filename 2024-04-20T11:09:23.131 INFO:tasks.workunit.client.1.smithi173.stdout:8/11: write - no filename 2024-04-20T11:09:23.131 INFO:tasks.workunit.client.1.smithi173.stdout:8/12: truncate - no filename 2024-04-20T11:09:23.131 INFO:tasks.workunit.client.1.smithi173.stdout:8/13: dread - no filename 2024-04-20T11:09:23.132 INFO:tasks.workunit.client.1.smithi173.stdout:9/3: write f0 [376295,58505] 0 2024-04-20T11:09:23.132 INFO:tasks.workunit.client.1.smithi173.stdout:7/12: chown l2 41 1 2024-04-20T11:09:23.134 INFO:tasks.workunit.client.1.smithi173.stdout:4/3: symlink l2 0 2024-04-20T11:09:23.135 INFO:tasks.workunit.client.0.smithi117.stdout:4/958: write d1/d9/f2a [692648,84662] 0 2024-04-20T11:09:23.135 INFO:tasks.workunit.client.1.smithi173.stdout:6/21: link c0 d4/c5 0 2024-04-20T11:09:23.135 INFO:tasks.workunit.client.1.smithi173.stdout:6/22: dread - no filename 2024-04-20T11:09:23.136 INFO:tasks.workunit.client.1.smithi173.stdout:6/23: truncate - no filename 2024-04-20T11:09:23.136 INFO:tasks.workunit.client.1.smithi173.stdout:6/24: dread - no filename 2024-04-20T11:09:23.136 INFO:tasks.workunit.client.1.smithi173.stdout:8/14: creat f2 x:0 0 0 2024-04-20T11:09:23.136 INFO:tasks.workunit.client.1.smithi173.stdout:8/15: rmdir - no directory 2024-04-20T11:09:23.137 INFO:tasks.workunit.client.1.smithi173.stdout:7/13: creat f3 x:0 0 0 2024-04-20T11:09:23.139 INFO:tasks.workunit.client.0.smithi117.stdout:7/962: rename d1/d1d/d4e/d72/c84 to d1/d4/d3d/dce/deb/c139 0 2024-04-20T11:09:23.140 INFO:tasks.workunit.client.1.smithi173.stdout:0/10: link f1 f3 0 2024-04-20T11:09:23.140 INFO:tasks.workunit.client.1.smithi173.stdout:0/11: rmdir - no directory 2024-04-20T11:09:23.141 INFO:tasks.workunit.client.1.smithi173.stdout:4/4: mknod c3 0 2024-04-20T11:09:23.143 INFO:tasks.workunit.client.1.smithi173.stdout:6/25: unlink l1 0 2024-04-20T11:09:23.146 INFO:tasks.workunit.client.1.smithi173.stdout:9/4: rmdir d1 0 2024-04-20T11:09:23.146 INFO:tasks.workunit.client.1.smithi173.stdout:0/12: write f3 [2665646,4104] 0 2024-04-20T11:09:23.146 INFO:tasks.workunit.client.1.smithi173.stdout:6/26: rename d4 to d4/d6 22 2024-04-20T11:09:23.148 INFO:tasks.workunit.client.0.smithi117.stdout:7/963: symlink d1/d43/d59/d8a/dd7/l13a 0 2024-04-20T11:09:23.149 INFO:tasks.workunit.client.1.smithi173.stdout:9/5: symlink l2 0 2024-04-20T11:09:23.150 INFO:tasks.workunit.client.1.smithi173.stdout:0/13: unlink f1 0 2024-04-20T11:09:23.152 INFO:tasks.workunit.client.1.smithi173.stdout:9/6: write f0 [314406,123455] 0 2024-04-20T11:09:23.155 INFO:tasks.workunit.client.1.smithi173.stdout:0/14: unlink f3 0 2024-04-20T11:09:23.159 INFO:tasks.workunit.client.1.smithi173.stdout:8/16: dwrite f2 [0,4194304] 0 2024-04-20T11:09:23.161 INFO:tasks.workunit.client.0.smithi117.stdout:7/964: dread d1/d4/d6/d54/d63/f6e [0,4194304] 0 2024-04-20T11:09:23.161 INFO:tasks.workunit.client.1.smithi173.stdout:0/15: link f2 f4 0 2024-04-20T11:09:23.162 INFO:tasks.workunit.client.1.smithi173.stdout:0/16: dread - f2 zero size 2024-04-20T11:09:23.165 INFO:tasks.workunit.client.0.smithi117.stdout:7/965: write d1/d1d/d77/dee/f121 [415131,130655] 0 2024-04-20T11:09:23.167 INFO:tasks.workunit.client.0.smithi117.stdout:1/881: sync 2024-04-20T11:09:23.167 INFO:tasks.workunit.client.0.smithi117.stdout:3/928: sync 2024-04-20T11:09:23.167 INFO:tasks.workunit.client.1.smithi173.stdout:9/7: dread f0 [0,4194304] 0 2024-04-20T11:09:23.171 INFO:tasks.workunit.client.1.smithi173.stdout:9/8: creat f3 x:0 0 0 2024-04-20T11:09:23.174 INFO:tasks.workunit.client.1.smithi173.stdout:9/9: write f3 [687946,10616] 0 2024-04-20T11:09:23.177 INFO:tasks.workunit.client.0.smithi117.stdout:3/929: creat d0/d5/d21/d88/f134 x:0 0 0 2024-04-20T11:09:23.181 INFO:tasks.workunit.client.1.smithi173.stdout:9/10: rename l2 to l4 0 2024-04-20T11:09:23.184 INFO:tasks.workunit.client.1.smithi173.stdout:8/17: dwrite f2 [0,4194304] 0 2024-04-20T11:09:23.185 INFO:tasks.workunit.client.1.smithi173.stdout:9/11: mkdir d5 0 2024-04-20T11:09:23.187 INFO:tasks.workunit.client.0.smithi117.stdout:4/959: dread d1/f25 [0,4194304] 0 2024-04-20T11:09:23.198 INFO:tasks.workunit.client.0.smithi117.stdout:7/966: dwrite d1/d1d/d77/dee/d124/f135 [0,4194304] 0 2024-04-20T11:09:23.198 INFO:tasks.workunit.client.0.smithi117.stdout:3/930: link d0/d5/d21/d75/dc1/f11f d0/d5/d73/ded/d66/f135 0 2024-04-20T11:09:23.199 INFO:tasks.workunit.client.0.smithi117.stdout:4/960: symlink d1/d9/d1f/d90/d107/d12b/d135/l139 0 2024-04-20T11:09:23.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.203+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb08000bf0 con 0x7fdb0c06eb10 2024-04-20T11:09:23.212 INFO:tasks.workunit.client.1.smithi173.stdout:9/12: dwrite f0 [0,4194304] 0 2024-04-20T11:09:23.214 INFO:tasks.workunit.client.1.smithi173.stdout:1/10: chown c2 0 1 2024-04-20T11:09:23.214 INFO:tasks.workunit.client.0.smithi117.stdout:3/931: truncate d0/d5/d73/ded/d66/d63/fa4 227414 0 2024-04-20T11:09:23.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.218+0000 7fdb1fe00700 1 -- 172.21.15.117:0/2622117065 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fdb08000bf0 con 0x7fdb0c06eb10 2024-04-20T11:09:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:09:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (10m) 13s ago 11m 13.7M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:09:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (12m) 13s ago 12m 7948k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (10m) 32s ago 10m 7750k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (12m) 13s ago 12m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (10m) 32s ago 10m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (9m) 13s ago 11m 75.8M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (4m) 13s ago 4m 14.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (4m) 13s ago 4m 172M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (4m) 32s ago 4m 14.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (4m) 32s ago 4m 23.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (59s) 13s ago 13m 477M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (117s) 32s ago 10m 74.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 7199977e9e8d 2024-04-20T11:09:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (13m) 13s ago 13m 53.3M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (10m) 32s ago 10m 41.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (12m) 13s ago 12m 8162k - 1.5.0 0da6a335fe13 6b69ffbf445a 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (10m) 32s ago 10m 8074k - 1.5.0 0da6a335fe13 81909a8618f8 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (9m) 13s ago 9m 89.9M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (8m) 13s ago 8m 93.3M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (8m) 13s ago 8m 78.2M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (7m) 32s ago 7m 104M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (7m) 32s ago 7m 98.1M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (6m) 32s ago 6m 95.3M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:09:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (70s) 13s ago 11m 46.2M - 2.43.0 a07b618ecd1d ca77d508764b 2024-04-20T11:09:23.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.221+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdb0c06eb10 msgr2=0x7fdb0c070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:23.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.221+0000 7fdb2e800700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdb0c06eb10 0x7fdb0c070fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb180098e0 tx=0x7fdb180050c0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.221+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 msgr2=0x7fdb28194500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:23.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.221+0000 7fdb2e800700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 0x7fdb28194500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000b9c0 tx=0x7fdb2000bd80 comp rx=0 tx=0).stop 2024-04-20T11:09:23.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.222+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 shutdown_connections 2024-04-20T11:09:23.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.222+0000 7fdb2e800700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdb0c06eb10 0x7fdb0c070fd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.222+0000 7fdb2e800700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb28073940 0x7fdb28194500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.222+0000 7fdb2e800700 1 --2- 172.21.15.117:0/2622117065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb28074f10 0x7fdb28194a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.222+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 >> 172.21.15.117:0/2622117065 conn(0x7fdb280fc1c0 msgr2=0x7fdb28102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:23.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.222+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 shutdown_connections 2024-04-20T11:09:23.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.222+0000 7fdb2e800700 1 -- 172.21.15.117:0/2622117065 wait complete. 2024-04-20T11:09:23.227 INFO:tasks.workunit.client.1.smithi173.stdout:9/13: dread f0 [0,4194304] 0 2024-04-20T11:09:23.228 INFO:tasks.workunit.client.0.smithi117.stdout:3/932: mknod d0/d5/de/c136 0 2024-04-20T11:09:23.228 INFO:tasks.workunit.client.1.smithi173.stdout:1/11: link f0 f3 0 2024-04-20T11:09:23.229 INFO:tasks.workunit.client.1.smithi173.stdout:1/12: truncate f3 188571 0 2024-04-20T11:09:23.229 INFO:tasks.workunit.client.0.smithi117.stdout:3/933: chown d0/d5/f40 2936397 1 2024-04-20T11:09:23.230 INFO:tasks.workunit.client.1.smithi173.stdout:3/22: fsync f2 0 2024-04-20T11:09:23.230 INFO:tasks.workunit.client.1.smithi173.stdout:3/23: stat f3 0 2024-04-20T11:09:23.232 INFO:tasks.workunit.client.0.smithi117.stdout:7/967: getdents d1/d1d/d4e/d53 0 2024-04-20T11:09:23.234 INFO:tasks.workunit.client.0.smithi117.stdout:8/872: write d0/f1 [3912745,72840] 0 2024-04-20T11:09:23.235 INFO:tasks.workunit.client.1.smithi173.stdout:1/13: write f0 [844364,6499] 0 2024-04-20T11:09:23.242 INFO:tasks.workunit.client.0.smithi117.stdout:7/968: rename d1/d4/d39/cbe to d1/d1d/d77/c13b 0 2024-04-20T11:09:23.242 INFO:tasks.workunit.client.1.smithi173.stdout:4/5: write f1 [4774855,83015] 0 2024-04-20T11:09:23.243 INFO:tasks.workunit.client.1.smithi173.stdout:9/14: getdents d5 0 2024-04-20T11:09:23.246 INFO:tasks.workunit.client.1.smithi173.stdout:1/14: creat f4 x:0 0 0 2024-04-20T11:09:23.248 INFO:tasks.workunit.client.1.smithi173.stdout:9/15: dread f3 [0,4194304] 0 2024-04-20T11:09:23.249 INFO:tasks.workunit.client.1.smithi173.stdout:9/16: chown f3 79590 1 2024-04-20T11:09:23.250 INFO:tasks.workunit.client.0.smithi117.stdout:7/969: creat d1/d4/d3d/d93/f13c x:0 0 0 2024-04-20T11:09:23.251 INFO:tasks.workunit.client.1.smithi173.stdout:4/6: fdatasync f1 0 2024-04-20T11:09:23.254 INFO:tasks.workunit.client.1.smithi173.stdout:1/15: mkdir d5 0 2024-04-20T11:09:23.257 INFO:tasks.workunit.client.1.smithi173.stdout:1/16: write f3 [977486,97033] 0 2024-04-20T11:09:23.259 INFO:tasks.workunit.client.1.smithi173.stdout:3/24: dwrite f2 [0,4194304] 0 2024-04-20T11:09:23.260 INFO:tasks.workunit.client.0.smithi117.stdout:3/934: truncate d0/d5/d73/ded/d66/f135 3861329 0 2024-04-20T11:09:23.261 INFO:tasks.workunit.client.0.smithi117.stdout:8/873: dwrite d0/d8/d11/d24/d53/f7b [4194304,4194304] 0 2024-04-20T11:09:23.263 INFO:tasks.workunit.client.1.smithi173.stdout:4/7: creat f4 x:0 0 0 2024-04-20T11:09:23.264 INFO:tasks.workunit.client.1.smithi173.stdout:9/17: creat d5/f6 x:0 0 0 2024-04-20T11:09:23.271 INFO:tasks.workunit.client.0.smithi117.stdout:3/935: mkdir d0/d5/d21/d88/dc2/d137 0 2024-04-20T11:09:23.274 INFO:tasks.workunit.client.1.smithi173.stdout:4/8: creat f5 x:0 0 0 2024-04-20T11:09:23.278 INFO:tasks.workunit.client.1.smithi173.stdout:9/18: dread f3 [0,4194304] 0 2024-04-20T11:09:23.278 INFO:tasks.workunit.client.1.smithi173.stdout:3/25: dwrite f3 [0,4194304] 0 2024-04-20T11:09:23.279 INFO:tasks.workunit.client.1.smithi173.stdout:3/26: stat c1 0 2024-04-20T11:09:23.282 INFO:tasks.workunit.client.0.smithi117.stdout:8/874: rename d0/d8/d39/d40/de5/fef to d0/d8/d11/d24/d27/d47/f120 0 2024-04-20T11:09:23.287 INFO:tasks.workunit.client.1.smithi173.stdout:8/18: truncate f2 1798577 0 2024-04-20T11:09:23.289 INFO:tasks.workunit.client.0.smithi117.stdout:3/936: mknod d0/d5/d7/c138 0 2024-04-20T11:09:23.290 INFO:tasks.workunit.client.1.smithi173.stdout:3/27: dread f3 [0,4194304] 0 2024-04-20T11:09:23.294 INFO:tasks.workunit.client.0.smithi117.stdout:1/882: dwrite d6/d16/d3a/f94 [0,4194304] 0 2024-04-20T11:09:23.295 INFO:tasks.workunit.client.0.smithi117.stdout:4/961: write d1/d45/dcd/fec [807909,88328] 0 2024-04-20T11:09:23.298 INFO:tasks.workunit.client.0.smithi117.stdout:1/883: readlink d6/d16/dce/ldb 0 2024-04-20T11:09:23.300 INFO:tasks.workunit.client.1.smithi173.stdout:1/17: dread f0 [0,4194304] 0 2024-04-20T11:09:23.300 INFO:tasks.workunit.client.0.smithi117.stdout:1/884: chown d6/d1f/d30/fc9 0 1 2024-04-20T11:09:23.303 INFO:tasks.workunit.client.0.smithi117.stdout:1/885: write d6/d16/d3a/f7e [2001534,53562] 0 2024-04-20T11:09:23.308 INFO:tasks.workunit.client.1.smithi173.stdout:3/28: unlink c1 0 2024-04-20T11:09:23.310 INFO:tasks.workunit.client.0.smithi117.stdout:8/875: mknod d0/d8/d11/d24/c121 0 2024-04-20T11:09:23.317 INFO:tasks.workunit.client.0.smithi117.stdout:7/970: write d1/d1d/ded/f100 [438610,48587] 0 2024-04-20T11:09:23.331 INFO:tasks.workunit.client.0.smithi117.stdout:3/937: symlink d0/d5/d21/d75/d89/l139 0 2024-04-20T11:09:23.332 INFO:tasks.workunit.client.1.smithi173.stdout:8/19: write f2 [1160174,123815] 0 2024-04-20T11:09:23.333 INFO:tasks.workunit.client.1.smithi173.stdout:3/29: creat f4 x:0 0 0 2024-04-20T11:09:23.335 INFO:tasks.workunit.client.1.smithi173.stdout:3/30: chown f3 1387546160 1 2024-04-20T11:09:23.336 INFO:tasks.workunit.client.1.smithi173.stdout:4/9: getdents . 0 2024-04-20T11:09:23.338 INFO:tasks.workunit.client.0.smithi117.stdout:7/971: rename d1/d1d/d42/d2d/l50 to d1/d1d/d4e/dd3/l13d 0 2024-04-20T11:09:23.341 INFO:tasks.workunit.client.0.smithi117.stdout:7/972: write d1/d1d/d4e/f10a [469433,2437] 0 2024-04-20T11:09:23.345 INFO:tasks.workunit.client.0.smithi117.stdout:4/962: link d1/dd/d1a/d2b/d86/f112 d1/d9/de9/d9e/f13a 0 2024-04-20T11:09:23.346 INFO:tasks.workunit.client.1.smithi173.stdout:1/18: mkdir d5/d6 0 2024-04-20T11:09:23.349 INFO:tasks.workunit.client.1.smithi173.stdout:3/31: mknod c5 0 2024-04-20T11:09:23.350 INFO:tasks.workunit.client.1.smithi173.stdout:9/19: getdents d5 0 2024-04-20T11:09:23.353 INFO:tasks.workunit.client.0.smithi117.stdout:1/886: rename d6/d16/d3a/d5a/d5c/d61/d71/d6b/f93 to d6/d16/d3a/d5a/d5c/d61/d98/f122 0 2024-04-20T11:09:23.353 INFO:tasks.workunit.client.0.smithi117.stdout:4/963: truncate d1/f19 6186773 0 2024-04-20T11:09:23.373 INFO:tasks.workunit.client.0.smithi117.stdout:4/964: dwrite d1/dd/d52/d117/f127 [0,4194304] 0 2024-04-20T11:09:23.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:23 smithi117 ceph-mon[15659]: pgmap v21: 65 pgs: 65 active+clean; 1.8 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 52 MiB/s rd, 106 MiB/s wr, 300 op/s 2024-04-20T11:09:23.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:23 smithi117 ceph-mon[15659]: from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:23.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:23 smithi173 ceph-mon[23800]: pgmap v21: 65 pgs: 65 active+clean; 1.8 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 52 MiB/s rd, 106 MiB/s wr, 300 op/s 2024-04-20T11:09:23.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:23 smithi173 ceph-mon[23800]: from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:23.419 INFO:tasks.workunit.client.0.smithi117.stdout:8/876: getdents d0/d8/d11/d24/d86 0 2024-04-20T11:09:23.424 INFO:tasks.workunit.client.0.smithi117.stdout:8/877: chown d0/d8/d11/d24/d27/d117 7 1 2024-04-20T11:09:23.426 INFO:tasks.workunit.client.0.smithi117.stdout:8/878: symlink d0/d9/d3a/d7c/dce/l122 0 2024-04-20T11:09:23.427 INFO:tasks.workunit.client.0.smithi117.stdout:8/879: readlink d0/dc/d49/lea 0 2024-04-20T11:09:23.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.446+0000 7fada4400700 1 -- 172.21.15.117:0/2795166226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 msgr2=0x7fad9c103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:23.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.446+0000 7fada4400700 1 --2- 172.21.15.117:0/2795166226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 0x7fad9c103a70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fad90009910 tx=0x7fad90009c20 comp rx=0 tx=0).stop 2024-04-20T11:09:23.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.446+0000 7fada4400700 1 -- 172.21.15.117:0/2795166226 shutdown_connections 2024-04-20T11:09:23.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.446+0000 7fada4400700 1 --2- 172.21.15.117:0/2795166226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad9c103fb0 0x7fad9c1063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.446+0000 7fada4400700 1 --2- 172.21.15.117:0/2795166226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 0x7fad9c103a70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.446+0000 7fada4400700 1 -- 172.21.15.117:0/2795166226 >> 172.21.15.117:0/2795166226 conn(0x7fad9c0fb0d0 msgr2=0x7fad9c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:23.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.446+0000 7fada4400700 1 -- 172.21.15.117:0/2795166226 shutdown_connections 2024-04-20T11:09:23.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.447+0000 7fada4400700 1 -- 172.21.15.117:0/2795166226 wait complete. 2024-04-20T11:09:23.447 INFO:tasks.workunit.client.0.smithi117.stdout:8/880: dread d0/d8/d11/d24/d53/f56 [0,4194304] 0 2024-04-20T11:09:23.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.448+0000 7fada4400700 1 Processor -- start 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.448+0000 7fada4400700 1 -- start start 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.448+0000 7fada4400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 0x7fad9c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.448+0000 7fada4400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad9c103fb0 0x7fad9c198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.448+0000 7fada4400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c199420 con 0x7fad9c101680 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.448+0000 7fada4400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c199560 con 0x7fad9c103fb0 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.449+0000 7fada1a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 0x7fad9c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.449+0000 7fada1a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 0x7fad9c1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50526/0 (socket says 172.21.15.117:50526) 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.449+0000 7fada1a00700 1 -- 172.21.15.117:0/1298043032 learned_addr learned my addr 172.21.15.117:0/1298043032 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:23.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.449+0000 7fada1000700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad9c103fb0 0x7fad9c198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:23.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.450+0000 7fada1a00700 1 -- 172.21.15.117:0/1298043032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad9c103fb0 msgr2=0x7fad9c198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:23.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.450+0000 7fada1a00700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad9c103fb0 0x7fad9c198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.450+0000 7fada1a00700 1 -- 172.21.15.117:0/1298043032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad900095f0 con 0x7fad9c101680 2024-04-20T11:09:23.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.450+0000 7fada1a00700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 0x7fad9c1988c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fad9000bd70 tx=0x7fad9000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:23.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.450+0000 7fad9a000700 1 -- 172.21.15.117:0/1298043032 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad9001d070 con 0x7fad9c101680 2024-04-20T11:09:23.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.451+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c19df60 con 0x7fad9c101680 2024-04-20T11:09:23.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.451+0000 7fad9a000700 1 -- 172.21.15.117:0/1298043032 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad9000f460 con 0x7fad9c101680 2024-04-20T11:09:23.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.451+0000 7fad9a000700 1 -- 172.21.15.117:0/1298043032 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad90021640 con 0x7fad9c101680 2024-04-20T11:09:23.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.451+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c19e450 con 0x7fad9c101680 2024-04-20T11:09:23.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.452+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c0052f0 con 0x7fad9c101680 2024-04-20T11:09:23.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.453+0000 7fad9a000700 1 -- 172.21.15.117:0/1298043032 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7fad9002b440 con 0x7fad9c101680 2024-04-20T11:09:23.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.454+0000 7fad9a000700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fad8406ea70 0x7fad84070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:23.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.454+0000 7fad9a000700 1 -- 172.21.15.117:0/1298043032 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fad9008fda0 con 0x7fad9c101680 2024-04-20T11:09:23.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.455+0000 7fada1000700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fad8406ea70 0x7fad84070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:23.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.455+0000 7fada1000700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fad8406ea70 0x7fad84070f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fad88009a10 tx=0x7fad88009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:23.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.459+0000 7fad9a000700 1 -- 172.21.15.117:0/1298043032 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fad9005c9a0 con 0x7fad9c101680 2024-04-20T11:09:23.601 INFO:tasks.workunit.client.0.smithi117.stdout:7/973: sync 2024-04-20T11:09:23.632 INFO:tasks.workunit.client.0.smithi117.stdout:7/974: dwrite d1/d43/d59/d8a/f10d [0,4194304] 0 2024-04-20T11:09:23.635 INFO:tasks.workunit.client.0.smithi117.stdout:7/975: stat d1/d4/d3d/d93/f13c 0 2024-04-20T11:09:23.658 INFO:tasks.workunit.client.1.smithi173.stdout:8/20: rename l0 to l3 0 2024-04-20T11:09:23.663 INFO:tasks.workunit.client.1.smithi173.stdout:3/32: creat f6 x:0 0 0 2024-04-20T11:09:23.665 INFO:tasks.workunit.client.1.smithi173.stdout:1/19: dread f3 [0,4194304] 0 2024-04-20T11:09:23.668 INFO:tasks.workunit.client.1.smithi173.stdout:9/20: symlink d5/l7 0 2024-04-20T11:09:23.670 INFO:tasks.workunit.client.1.smithi173.stdout:3/33: mkdir d7 0 2024-04-20T11:09:23.671 INFO:tasks.workunit.client.1.smithi173.stdout:1/20: dread - f4 zero size 2024-04-20T11:09:23.675 INFO:tasks.workunit.client.1.smithi173.stdout:4/10: dwrite f4 [0,4194304] 0 2024-04-20T11:09:23.676 INFO:tasks.workunit.client.1.smithi173.stdout:9/21: write f0 [4667670,49667] 0 2024-04-20T11:09:23.678 INFO:tasks.workunit.client.0.smithi117.stdout:7/976: dwrite d1/d1d/d4e/d72/fc6 [0,4194304] 0 2024-04-20T11:09:23.679 INFO:tasks.workunit.client.1.smithi173.stdout:1/21: write f0 [468700,112646] 0 2024-04-20T11:09:23.681 INFO:tasks.workunit.client.0.smithi117.stdout:4/965: dread d1/d9/de9/fe8 [0,4194304] 0 2024-04-20T11:09:23.682 INFO:tasks.workunit.client.1.smithi173.stdout:1/22: write f4 [1039177,23675] 0 2024-04-20T11:09:23.685 INFO:tasks.workunit.client.1.smithi173.stdout:8/21: dread f2 [0,4194304] 0 2024-04-20T11:09:23.685 INFO:tasks.workunit.client.0.smithi117.stdout:7/977: unlink d1/d1d/d42/d2d/lcd 0 2024-04-20T11:09:23.688 INFO:tasks.workunit.client.1.smithi173.stdout:3/34: creat d7/f8 x:0 0 0 2024-04-20T11:09:23.695 INFO:tasks.workunit.client.0.smithi117.stdout:7/978: mkdir d1/d4/d39/d13e 0 2024-04-20T11:09:23.702 INFO:tasks.workunit.client.0.smithi117.stdout:7/979: write d1/d1d/d4e/d53/f10e [331016,42693] 0 2024-04-20T11:09:23.704 INFO:tasks.workunit.client.1.smithi173.stdout:1/23: fsync f0 0 2024-04-20T11:09:23.704 INFO:tasks.workunit.client.0.smithi117.stdout:7/980: stat d1/d43/d59/c96 0 2024-04-20T11:09:23.705 INFO:tasks.workunit.client.1.smithi173.stdout:3/35: mkdir d7/d9 0 2024-04-20T11:09:23.715 INFO:tasks.workunit.client.1.smithi173.stdout:4/11: dwrite f1 [4194304,4194304] 0 2024-04-20T11:09:23.715 INFO:tasks.workunit.client.0.smithi117.stdout:3/938: dwrite d0/d5/d7/dd/f91 [0,4194304] 0 2024-04-20T11:09:23.718 INFO:tasks.workunit.client.0.smithi117.stdout:7/981: dread d1/d4/d3d/dce/f115 [0,4194304] 0 2024-04-20T11:09:23.722 INFO:tasks.workunit.client.1.smithi173.stdout:8/22: dwrite f2 [0,4194304] 0 2024-04-20T11:09:23.783 INFO:tasks.workunit.client.1.smithi173.stdout:9/22: link d5/l7 d5/l8 0 2024-04-20T11:09:23.784 INFO:tasks.workunit.client.1.smithi173.stdout:1/24: mkdir d5/d7 0 2024-04-20T11:09:23.785 INFO:tasks.workunit.client.1.smithi173.stdout:3/36: unlink f2 0 2024-04-20T11:09:23.787 INFO:tasks.workunit.client.1.smithi173.stdout:8/23: mkdir d4 0 2024-04-20T11:09:23.789 INFO:tasks.workunit.client.1.smithi173.stdout:8/24: write f2 [3117382,15074] 0 2024-04-20T11:09:23.789 INFO:tasks.workunit.client.1.smithi173.stdout:9/23: write f3 [579530,84164] 0 2024-04-20T11:09:23.790 INFO:tasks.workunit.client.1.smithi173.stdout:1/25: dread f0 [0,4194304] 0 2024-04-20T11:09:23.792 INFO:tasks.workunit.client.1.smithi173.stdout:8/25: read f2 [3070079,4373] 0 2024-04-20T11:09:23.793 INFO:tasks.workunit.client.1.smithi173.stdout:1/26: rename d5/d7 to d5/d7/d8 22 2024-04-20T11:09:23.794 INFO:tasks.workunit.client.0.smithi117.stdout:8/881: rename d0/d8/d11/d6d/d58 to d0/d8/d39/d40/d78/d5c/d123 0 2024-04-20T11:09:23.796 INFO:tasks.workunit.client.1.smithi173.stdout:8/26: fsync f2 0 2024-04-20T11:09:23.797 INFO:tasks.workunit.client.1.smithi173.stdout:9/24: readlink l4 0 2024-04-20T11:09:23.800 INFO:tasks.workunit.client.0.smithi117.stdout:7/982: rename d1/d4/d6/d2b/d64/fe1 to d1/d4/d39/f13f 0 2024-04-20T11:09:23.802 INFO:tasks.workunit.client.1.smithi173.stdout:3/37: symlink d7/d9/la 0 2024-04-20T11:09:23.812 INFO:tasks.workunit.client.0.smithi117.stdout:8/882: rename d0/d8/d11/d6d/fba to d0/d8/d11/d24/d27/d47/f124 0 2024-04-20T11:09:23.814 INFO:tasks.workunit.client.1.smithi173.stdout:9/25: mkdir d5/d9 0 2024-04-20T11:09:23.815 INFO:tasks.workunit.client.0.smithi117.stdout:8/883: chown d0/d8/d11/d24/d76/fb0 8520264 1 2024-04-20T11:09:23.815 INFO:tasks.workunit.client.1.smithi173.stdout:1/27: creat d5/d6/f9 x:0 0 0 2024-04-20T11:09:23.817 INFO:tasks.workunit.client.1.smithi173.stdout:3/38: creat d7/d9/fb x:0 0 0 2024-04-20T11:09:23.818 INFO:tasks.workunit.client.1.smithi173.stdout:8/27: mknod d4/c5 0 2024-04-20T11:09:23.820 INFO:tasks.workunit.client.0.smithi117.stdout:8/884: symlink d0/d9/d3a/de3/df1/l125 0 2024-04-20T11:09:23.825 INFO:tasks.workunit.client.1.smithi173.stdout:3/39: symlink d7/d9/lc 0 2024-04-20T11:09:23.827 INFO:tasks.workunit.client.1.smithi173.stdout:8/28: creat d4/f6 x:0 0 0 2024-04-20T11:09:23.830 INFO:tasks.workunit.client.1.smithi173.stdout:9/26: creat d5/d9/fa x:0 0 0 2024-04-20T11:09:23.831 INFO:tasks.workunit.client.1.smithi173.stdout:1/28: mkdir d5/d7/da 0 2024-04-20T11:09:23.832 INFO:tasks.workunit.client.1.smithi173.stdout:9/27: rename d5/d9 to d5/d9/db 22 2024-04-20T11:09:23.833 INFO:tasks.workunit.client.1.smithi173.stdout:8/29: creat d4/f7 x:0 0 0 2024-04-20T11:09:23.833 INFO:tasks.workunit.client.1.smithi173.stdout:1/29: dread - d5/d6/f9 zero size 2024-04-20T11:09:23.838 INFO:tasks.workunit.client.0.smithi117.stdout:1/887: write d6/d1f/d30/f43 [4504415,1392] 0 2024-04-20T11:09:23.842 INFO:tasks.workunit.client.0.smithi117.stdout:1/888: creat d6/dd2/f123 x:0 0 0 2024-04-20T11:09:23.845 INFO:tasks.workunit.client.0.smithi117.stdout:4/966: write d1/fc3 [4025390,81993] 0 2024-04-20T11:09:23.848 INFO:tasks.workunit.client.1.smithi173.stdout:3/40: dwrite d7/f8 [0,4194304] 0 2024-04-20T11:09:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.854+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fad7c005870 con 0x7fad9c101680 2024-04-20T11:09:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.855+0000 7fad9a000700 1 -- 172.21.15.117:0/1298043032 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7fad9005c530 con 0x7fad9c101680 2024-04-20T11:09:23.855 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:09:23.855 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:09:23.855 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:09:23.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:09:23.857 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:09:23.857 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:09:23.857 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:09:23.857 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:09:23.857 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:09:23.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.858+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fad8406ea70 msgr2=0x7fad84070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:23.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.858+0000 7fada4400700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fad8406ea70 0x7fad84070f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fad88009a10 tx=0x7fad88009190 comp rx=0 tx=0).stop 2024-04-20T11:09:23.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.858+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 msgr2=0x7fad9c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:23.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.858+0000 7fada4400700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 0x7fad9c1988c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fad9000bd70 tx=0x7fad9000bda0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.859+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 shutdown_connections 2024-04-20T11:09:23.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.859+0000 7fada4400700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fad8406ea70 0x7fad84070f30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.859+0000 7fada4400700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c101680 0x7fad9c1988c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.859+0000 7fada4400700 1 --2- 172.21.15.117:0/1298043032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad9c103fb0 0x7fad9c198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:23.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.859+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 >> 172.21.15.117:0/1298043032 conn(0x7fad9c0fb0d0 msgr2=0x7fad9c104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:23.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.859+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 shutdown_connections 2024-04-20T11:09:23.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:23.861+0000 7fada4400700 1 -- 172.21.15.117:0/1298043032 wait complete. 2024-04-20T11:09:23.861 INFO:tasks.workunit.client.1.smithi173.stdout:9/28: dwrite d5/f6 [0,4194304] 0 2024-04-20T11:09:23.865 INFO:tasks.workunit.client.0.smithi117.stdout:4/967: dwrite d1/dd/f53 [0,4194304] 0 2024-04-20T11:09:23.867 INFO:tasks.workunit.client.1.smithi173.stdout:9/29: mknod d5/d9/cc 0 2024-04-20T11:09:23.869 INFO:tasks.workunit.client.1.smithi173.stdout:9/30: read d5/f6 [3402544,45416] 0 2024-04-20T11:09:23.871 INFO:tasks.workunit.client.0.smithi117.stdout:1/889: rename d6/d21 to d6/d1f/d30/d49/d124 0 2024-04-20T11:09:23.872 INFO:tasks.workunit.client.1.smithi173.stdout:9/31: write d5/f6 [92122,28518] 0 2024-04-20T11:09:23.873 INFO:tasks.workunit.client.0.smithi117.stdout:3/939: write d0/f3 [8276640,100315] 0 2024-04-20T11:09:23.875 INFO:tasks.workunit.client.0.smithi117.stdout:3/940: fdatasync d0/d5/d7/dd/f3a 0 2024-04-20T11:09:23.878 INFO:tasks.workunit.client.0.smithi117.stdout:1/890: dread d6/d1f/d30/d32/fcb [0,4194304] 0 2024-04-20T11:09:23.880 INFO:tasks.workunit.client.0.smithi117.stdout:3/941: stat d0/d5/d73/ded/d66/d63/dad/laf 0 2024-04-20T11:09:23.892 INFO:tasks.workunit.client.0.smithi117.stdout:1/891: dread d6/d1f/dd6/fe5 [0,4194304] 0 2024-04-20T11:09:23.895 INFO:tasks.workunit.client.0.smithi117.stdout:1/892: creat d6/d16/d3a/d5a/f125 x:0 0 0 2024-04-20T11:09:23.897 INFO:tasks.workunit.client.0.smithi117.stdout:3/942: getdents d0/d5/d7/dd 0 2024-04-20T11:09:23.899 INFO:tasks.workunit.client.0.smithi117.stdout:1/893: creat d6/d1f/d30/d49/d69/d6e/f126 x:0 0 0 2024-04-20T11:09:23.913 INFO:tasks.workunit.client.0.smithi117.stdout:3/943: dwrite d0/d5/d21/f4d [0,4194304] 0 2024-04-20T11:09:23.920 INFO:tasks.workunit.client.0.smithi117.stdout:3/944: dread d0/d5/d21/fb5 [0,4194304] 0 2024-04-20T11:09:23.922 INFO:tasks.workunit.client.0.smithi117.stdout:1/894: dread d6/f9 [0,4194304] 0 2024-04-20T11:09:23.923 INFO:tasks.workunit.client.1.smithi173.stdout:4/12: truncate f1 5172178 0 2024-04-20T11:09:23.926 INFO:tasks.workunit.client.0.smithi117.stdout:1/895: mknod d6/d1f/d30/d32/c127 0 2024-04-20T11:09:23.931 INFO:tasks.workunit.client.0.smithi117.stdout:7/983: write d1/d1d/d4e/d53/d55/f94 [1838974,129813] 0 2024-04-20T11:09:23.933 INFO:tasks.workunit.client.1.smithi173.stdout:9/32: getdents d5 0 2024-04-20T11:09:23.935 INFO:tasks.workunit.client.0.smithi117.stdout:7/984: unlink d1/le6 0 2024-04-20T11:09:23.936 INFO:tasks.workunit.client.1.smithi173.stdout:1/30: rmdir d5/d6 39 2024-04-20T11:09:23.947 INFO:tasks.workunit.client.1.smithi173.stdout:1/31: dread f4 [0,4194304] 0 2024-04-20T11:09:23.952 INFO:tasks.workunit.client.1.smithi173.stdout:8/30: rmdir d4 39 2024-04-20T11:09:23.955 INFO:tasks.workunit.client.0.smithi117.stdout:8/885: dwrite d0/dd3/f108 [0,4194304] 0 2024-04-20T11:09:23.957 INFO:tasks.workunit.client.1.smithi173.stdout:8/31: dread - d4/f7 zero size 2024-04-20T11:09:23.959 INFO:tasks.workunit.client.1.smithi173.stdout:8/32: dread - d4/f7 zero size 2024-04-20T11:09:23.960 INFO:tasks.workunit.client.1.smithi173.stdout:1/32: creat d5/fb x:0 0 0 2024-04-20T11:09:23.962 INFO:tasks.workunit.client.1.smithi173.stdout:8/33: write d4/f7 [121634,100545] 0 2024-04-20T11:09:23.966 INFO:tasks.workunit.client.0.smithi117.stdout:3/945: mknod d0/d5/d7/dd/c13a 0 2024-04-20T11:09:23.968 INFO:tasks.workunit.client.1.smithi173.stdout:8/34: truncate d4/f6 976653 0 2024-04-20T11:09:23.969 INFO:tasks.workunit.client.1.smithi173.stdout:9/33: write d5/f6 [5237178,113485] 0 2024-04-20T11:09:23.970 INFO:tasks.workunit.client.1.smithi173.stdout:9/34: stat d5/f6 0 2024-04-20T11:09:23.970 INFO:tasks.workunit.client.0.smithi117.stdout:4/968: rename d1/f25 to d1/dd/d1a/d43/f13b 0 2024-04-20T11:09:23.970 INFO:tasks.workunit.client.0.smithi117.stdout:4/969: readlink d1/lb 0 2024-04-20T11:09:23.972 INFO:tasks.workunit.client.1.smithi173.stdout:1/33: mkdir d5/d6/dc 0 2024-04-20T11:09:23.975 INFO:tasks.workunit.client.1.smithi173.stdout:8/35: creat d4/f8 x:0 0 0 2024-04-20T11:09:23.977 INFO:tasks.workunit.client.1.smithi173.stdout:9/35: creat d5/d9/fd x:0 0 0 2024-04-20T11:09:23.979 INFO:tasks.workunit.client.1.smithi173.stdout:1/34: creat d5/d7/fd x:0 0 0 2024-04-20T11:09:23.983 INFO:tasks.workunit.client.1.smithi173.stdout:9/36: mknod d5/ce 0 2024-04-20T11:09:23.992 INFO:tasks.workunit.client.0.smithi117.stdout:1/896: truncate d6/d16/d3a/d5a/d5c/d61/f83 1433313 0 2024-04-20T11:09:23.993 INFO:tasks.workunit.client.1.smithi173.stdout:1/35: link d5/d7/fd d5/d6/dc/fe 0 2024-04-20T11:09:23.994 INFO:tasks.workunit.client.0.smithi117.stdout:1/897: fsync d6/d1f/f45 0 2024-04-20T11:09:23.994 INFO:tasks.workunit.client.1.smithi173.stdout:1/36: chown d5/d6 47055442 1 2024-04-20T11:09:23.996 INFO:tasks.workunit.client.1.smithi173.stdout:1/37: truncate f4 1087563 0 2024-04-20T11:09:24.001 INFO:tasks.workunit.client.1.smithi173.stdout:1/38: write f4 [582137,49291] 0 2024-04-20T11:09:24.002 INFO:tasks.workunit.client.1.smithi173.stdout:1/39: stat d5/d6/dc 0 2024-04-20T11:09:24.003 INFO:tasks.workunit.client.1.smithi173.stdout:4/13: dread f1 [0,4194304] 0 2024-04-20T11:09:24.011 INFO:tasks.workunit.client.0.smithi117.stdout:7/985: creat d1/d4/d6/d54/db4/f140 x:0 0 0 2024-04-20T11:09:24.012 INFO:tasks.workunit.client.0.smithi117.stdout:8/886: fsync d0/dc/d49/d63/fb5 0 2024-04-20T11:09:24.017 INFO:tasks.workunit.client.0.smithi117.stdout:7/986: mkdir d1/d1d/ded/d107/d141 0 2024-04-20T11:09:24.017 INFO:tasks.workunit.client.1.smithi173.stdout:4/14: dread f4 [0,4194304] 0 2024-04-20T11:09:24.025 INFO:tasks.workunit.client.1.smithi173.stdout:8/36: write f2 [4099379,101833] 0 2024-04-20T11:09:24.025 INFO:tasks.workunit.client.0.smithi117.stdout:7/987: mknod d1/d4/d3d/dce/c142 0 2024-04-20T11:09:24.027 INFO:tasks.workunit.client.1.smithi173.stdout:4/15: getdents . 0 2024-04-20T11:09:24.028 INFO:tasks.workunit.client.1.smithi173.stdout:8/37: write f2 [4501355,22382] 0 2024-04-20T11:09:24.030 INFO:tasks.workunit.client.0.smithi117.stdout:4/970: getdents d1/d45/d4a 0 2024-04-20T11:09:24.030 INFO:tasks.workunit.client.0.smithi117.stdout:7/988: truncate d1/d4/d6/d17/f11a 225266 0 2024-04-20T11:09:24.031 INFO:tasks.workunit.client.1.smithi173.stdout:1/40: dwrite d5/fb [0,4194304] 0 2024-04-20T11:09:24.032 INFO:tasks.workunit.client.0.smithi117.stdout:1/898: rename d6/f79 to d6/d1f/f128 0 2024-04-20T11:09:24.033 INFO:tasks.workunit.client.1.smithi173.stdout:1/41: chown d5/d6/dc 368520 1 2024-04-20T11:09:24.033 INFO:tasks.workunit.client.1.smithi173.stdout:1/42: chown d5 2100352921 1 2024-04-20T11:09:24.034 INFO:tasks.workunit.client.1.smithi173.stdout:4/16: symlink l6 0 2024-04-20T11:09:24.037 INFO:tasks.workunit.client.1.smithi173.stdout:8/38: mknod d4/c9 0 2024-04-20T11:09:24.041 INFO:tasks.workunit.client.0.smithi117.stdout:4/971: mknod d1/dd/d1a/d2b/d86/d11b/d131/c13c 0 2024-04-20T11:09:24.042 INFO:tasks.workunit.client.0.smithi117.stdout:1/899: truncate d6/d16/d3a/d5a/d5c/d61/d98/fed 4759683 0 2024-04-20T11:09:24.044 INFO:tasks.workunit.client.1.smithi173.stdout:4/17: dread f4 [0,4194304] 0 2024-04-20T11:09:24.045 INFO:tasks.workunit.client.1.smithi173.stdout:8/39: readlink l3 0 2024-04-20T11:09:24.047 INFO:tasks.workunit.client.1.smithi173.stdout:9/37: truncate d5/f6 3449182 0 2024-04-20T11:09:24.048 INFO:tasks.workunit.client.0.smithi117.stdout:3/946: dread d0/d2a/fc3 [0,4194304] 0 2024-04-20T11:09:24.048 INFO:tasks.workunit.client.1.smithi173.stdout:4/18: symlink l7 0 2024-04-20T11:09:24.051 INFO:tasks.workunit.client.1.smithi173.stdout:8/40: unlink l1 0 2024-04-20T11:09:24.051 INFO:tasks.workunit.client.0.smithi117.stdout:1/900: truncate d6/d1f/d30/d49/d124/f86 4398454 0 2024-04-20T11:09:24.053 INFO:tasks.workunit.client.1.smithi173.stdout:9/38: readlink d5/l8 0 2024-04-20T11:09:24.054 INFO:tasks.workunit.client.1.smithi173.stdout:4/19: creat f8 x:0 0 0 2024-04-20T11:09:24.055 INFO:tasks.workunit.client.1.smithi173.stdout:4/20: fdatasync f5 0 2024-04-20T11:09:24.056 INFO:tasks.workunit.client.1.smithi173.stdout:4/21: readlink l2 0 2024-04-20T11:09:24.058 INFO:tasks.workunit.client.0.smithi117.stdout:7/989: rename d1/d4/d6/d54/d63/cf2 to d1/d1d/d42/d2d/c143 0 2024-04-20T11:09:24.060 INFO:tasks.workunit.client.1.smithi173.stdout:8/41: rename d4/f7 to d4/fa 0 2024-04-20T11:09:24.061 INFO:tasks.workunit.client.1.smithi173.stdout:8/42: write f2 [3979039,93756] 0 2024-04-20T11:09:24.062 INFO:tasks.workunit.client.0.smithi117.stdout:3/947: creat d0/d5/d21/d94/da7/f13b x:0 0 0 2024-04-20T11:09:24.063 INFO:tasks.workunit.client.1.smithi173.stdout:9/39: mkdir d5/df 0 2024-04-20T11:09:24.065 INFO:tasks.workunit.client.1.smithi173.stdout:9/40: chown d5/l7 392 1 2024-04-20T11:09:24.067 INFO:tasks.workunit.client.0.smithi117.stdout:3/948: read d0/d5/d21/d94/d9a/dc9/ff9 [1249240,25777] 0 2024-04-20T11:09:24.070 INFO:tasks.workunit.client.1.smithi173.stdout:4/22: unlink c0 0 2024-04-20T11:09:24.072 INFO:tasks.workunit.client.0.smithi117.stdout:1/901: dwrite d6/d16/d3a/d5a/d5c/f106 [0,4194304] 0 2024-04-20T11:09:24.086 INFO:tasks.workunit.client.0.smithi117.stdout:4/972: dread d1/dd/d1a/d43/d4e/f62 [4194304,4194304] 0 2024-04-20T11:09:24.088 INFO:tasks.workunit.client.1.smithi173.stdout:9/41: dwrite f3 [0,4194304] 0 2024-04-20T11:09:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.093+0000 7fa8f7a00700 1 -- 172.21.15.117:0/4029864242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 msgr2=0x7fa8f0073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.093+0000 7fa8f7a00700 1 --2- 172.21.15.117:0/4029864242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 0x7fa8f0073370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009910 tx=0x7fa8dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:09:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.093+0000 7fa8f7a00700 1 -- 172.21.15.117:0/4029864242 shutdown_connections 2024-04-20T11:09:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.093+0000 7fa8f7a00700 1 --2- 172.21.15.117:0/4029864242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f0073940 0x7fa8f0073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.093+0000 7fa8f7a00700 1 --2- 172.21.15.117:0/4029864242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 0x7fa8f0073370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.093+0000 7fa8f7a00700 1 -- 172.21.15.117:0/4029864242 >> 172.21.15.117:0/4029864242 conn(0x7fa8f00fc1c0 msgr2=0x7fa8f00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:24.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.093+0000 7fa8f7a00700 1 -- 172.21.15.117:0/4029864242 shutdown_connections 2024-04-20T11:09:24.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.093+0000 7fa8f7a00700 1 -- 172.21.15.117:0/4029864242 wait complete. 2024-04-20T11:09:24.094 INFO:tasks.workunit.client.1.smithi173.stdout:9/42: write d5/d9/fa [955793,53130] 0 2024-04-20T11:09:24.095 INFO:tasks.workunit.client.0.smithi117.stdout:1/902: mkdir d6/d16/d129 0 2024-04-20T11:09:24.095 INFO:tasks.workunit.client.1.smithi173.stdout:8/43: link l3 d4/lb 0 2024-04-20T11:09:24.098 INFO:tasks.workunit.client.1.smithi173.stdout:4/23: dwrite f8 [0,4194304] 0 2024-04-20T11:09:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.095+0000 7fa8f7a00700 1 Processor -- start 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.095+0000 7fa8f7a00700 1 -- start start 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.096+0000 7fa8f7a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f0073940 0x7fa8f01915e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.096+0000 7fa8f7a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 0x7fa8f0191b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.096+0000 7fa8f7a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01920b0 con 0x7fa8f0073940 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.096+0000 7fa8f7a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01921f0 con 0x7fa8f0074f10 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.096+0000 7fa8f5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f0073940 0x7fa8f01915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.097+0000 7fa8efe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 0x7fa8f0191b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.097+0000 7fa8efe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 0x7fa8f0191b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:52158/0 (socket says 172.21.15.117:52158) 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.097+0000 7fa8efe00700 1 -- 172.21.15.117:0/3531694754 learned_addr learned my addr 172.21.15.117:0/3531694754 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.097+0000 7fa8f5000700 1 -- 172.21.15.117:0/3531694754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 msgr2=0x7fa8f0191b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.097+0000 7fa8f5000700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 0x7fa8f0191b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.097+0000 7fa8f5000700 1 -- 172.21.15.117:0/3531694754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0095f0 con 0x7fa8f0073940 2024-04-20T11:09:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.098+0000 7fa8f5000700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f0073940 0x7fa8f01915e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000c00 tx=0x7fa8dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.098+0000 7fa8ed600700 1 -- 172.21.15.117:0/3531694754 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8dc01d070 con 0x7fa8f0073940 2024-04-20T11:09:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.098+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f0196c00 con 0x7fa8f0073940 2024-04-20T11:09:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.099+0000 7fa8ed600700 1 -- 172.21.15.117:0/3531694754 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8dc00f460 con 0x7fa8f0073940 2024-04-20T11:09:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.099+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f0197140 con 0x7fa8f0073940 2024-04-20T11:09:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.099+0000 7fa8ed600700 1 -- 172.21.15.117:0/3531694754 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8dc021640 con 0x7fa8f0073940 2024-04-20T11:09:24.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.103+0000 7fa8ed600700 1 -- 172.21.15.117:0/3531694754 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7fa8dc003a60 con 0x7fa8f0073940 2024-04-20T11:09:24.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.104+0000 7fa8ed600700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa8d806eac0 0x7fa8d8070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:24.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.104+0000 7fa8ed600700 1 -- 172.21.15.117:0/3531694754 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fa8dc08fde0 con 0x7fa8f0073940 2024-04-20T11:09:24.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.106+0000 7fa8efe00700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa8d806eac0 0x7fa8d8070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:24.106 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:24 smithi117 ceph-mon[15659]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:24.106 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:24 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1298043032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:09:24.107 INFO:tasks.workunit.client.0.smithi117.stdout:8/887: dwrite d0/d8/d39/d40/f51 [0,4194304] 0 2024-04-20T11:09:24.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.109+0000 7fa8efe00700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa8d806eac0 0x7fa8d8070f80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4005950 tx=0x7fa8e40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:24.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.110+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d0005320 con 0x7fa8f0073940 2024-04-20T11:09:24.117 INFO:tasks.workunit.client.1.smithi173.stdout:8/44: creat d4/fc x:0 0 0 2024-04-20T11:09:24.118 INFO:tasks.workunit.client.1.smithi173.stdout:8/45: write f2 [4358744,13247] 0 2024-04-20T11:09:24.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.119+0000 7fa8ed600700 1 -- 172.21.15.117:0/3531694754 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fa8dc05c9e0 con 0x7fa8f0073940 2024-04-20T11:09:24.120 INFO:tasks.workunit.client.0.smithi117.stdout:4/973: dwrite d1/d9/f17 [0,4194304] 0 2024-04-20T11:09:24.121 INFO:tasks.workunit.client.1.smithi173.stdout:9/43: dread d5/d9/fa [0,4194304] 0 2024-04-20T11:09:24.123 INFO:tasks.workunit.client.1.smithi173.stdout:4/24: symlink l9 0 2024-04-20T11:09:24.126 INFO:tasks.workunit.client.0.smithi117.stdout:7/990: rmdir d1/d4 39 2024-04-20T11:09:24.129 INFO:tasks.workunit.client.0.smithi117.stdout:8/888: creat d0/d8/d11/d24/d86/f126 x:0 0 0 2024-04-20T11:09:24.134 INFO:tasks.workunit.client.0.smithi117.stdout:8/889: chown d0/d8/d39/d40/de5/d119 2069982 1 2024-04-20T11:09:24.135 INFO:tasks.workunit.client.1.smithi173.stdout:8/46: dread f2 [0,4194304] 0 2024-04-20T11:09:24.135 INFO:tasks.workunit.client.1.smithi173.stdout:9/44: write d5/d9/fa [521130,96741] 0 2024-04-20T11:09:24.136 INFO:tasks.workunit.client.1.smithi173.stdout:4/25: mkdir da 0 2024-04-20T11:09:24.136 INFO:tasks.workunit.client.1.smithi173.stdout:8/47: stat d4/f8 0 2024-04-20T11:09:24.140 INFO:tasks.workunit.client.0.smithi117.stdout:7/991: truncate d1/d4/d3d/dce/deb/ffa 4732609 0 2024-04-20T11:09:24.142 INFO:tasks.workunit.client.0.smithi117.stdout:7/992: chown d1/d1d/d42/d2d/dae 3749412 1 2024-04-20T11:09:24.144 INFO:tasks.workunit.client.0.smithi117.stdout:3/949: write d0/d5/d21/d88/d85/fb4 [571757,74158] 0 2024-04-20T11:09:24.152 INFO:tasks.workunit.client.0.smithi117.stdout:7/993: fdatasync d1/d43/f8d 0 2024-04-20T11:09:24.158 INFO:tasks.workunit.client.0.smithi117.stdout:3/950: truncate d0/d2a/fc3 4229972 0 2024-04-20T11:09:24.159 INFO:tasks.workunit.client.0.smithi117.stdout:1/903: dread d6/d1f/d30/d49/d69/d6e/d6f/d73/f8c [4194304,4194304] 0 2024-04-20T11:09:24.160 INFO:tasks.workunit.client.0.smithi117.stdout:7/994: creat d1/d4/d3d/d93/f144 x:0 0 0 2024-04-20T11:09:24.163 INFO:tasks.workunit.client.0.smithi117.stdout:7/995: chown d1/d4/d6/d13/fd5 405074 1 2024-04-20T11:09:24.163 INFO:tasks.workunit.client.0.smithi117.stdout:4/974: dread d1/dd/d1a/d2b/d86/dda/fdd [0,4194304] 0 2024-04-20T11:09:24.165 INFO:tasks.workunit.client.0.smithi117.stdout:3/951: creat d0/d5/d73/ded/dac/dd2/f13c x:0 0 0 2024-04-20T11:09:24.167 INFO:tasks.workunit.client.0.smithi117.stdout:1/904: unlink d6/d1f/d30/d49/d124/d95/le2 0 2024-04-20T11:09:24.172 INFO:tasks.workunit.client.0.smithi117.stdout:3/952: rename d0/c130 to d0/d5/d21/d88/d36/c13d 0 2024-04-20T11:09:24.173 INFO:tasks.workunit.client.0.smithi117.stdout:7/996: creat d1/d4/d6/d17/de8/f145 x:0 0 0 2024-04-20T11:09:24.175 INFO:tasks.workunit.client.0.smithi117.stdout:3/953: read d0/f35 [103650,2626] 0 2024-04-20T11:09:24.177 INFO:tasks.workunit.client.0.smithi117.stdout:8/890: dwrite d0/d8/d11/f14 [0,4194304] 0 2024-04-20T11:09:24.180 INFO:tasks.workunit.client.1.smithi173.stdout:4/26: mknod da/cb 0 2024-04-20T11:09:24.181 INFO:tasks.workunit.client.0.smithi117.stdout:7/997: chown d1/d4/d6/c4f 1096 1 2024-04-20T11:09:24.182 INFO:tasks.workunit.client.1.smithi173.stdout:9/45: creat d5/df/f10 x:0 0 0 2024-04-20T11:09:24.182 INFO:tasks.workunit.client.0.smithi117.stdout:8/891: read d0/d9/f95 [2820843,55043] 0 2024-04-20T11:09:24.182 INFO:tasks.workunit.client.0.smithi117.stdout:3/954: fsync d0/d5/d73/ded/d66/dbe/fe1 0 2024-04-20T11:09:24.184 INFO:tasks.workunit.client.1.smithi173.stdout:4/27: mkdir da/dc 0 2024-04-20T11:09:24.184 INFO:tasks.workunit.client.1.smithi173.stdout:4/28: stat c3 0 2024-04-20T11:09:24.190 INFO:tasks.workunit.client.1.smithi173.stdout:9/46: link f3 d5/df/f11 0 2024-04-20T11:09:24.191 INFO:tasks.workunit.client.0.smithi117.stdout:4/975: dwrite d1/dd/d1a/d43/f13b [0,4194304] 0 2024-04-20T11:09:24.191 INFO:tasks.workunit.client.1.smithi173.stdout:9/47: stat d5/l8 0 2024-04-20T11:09:24.196 INFO:tasks.workunit.client.1.smithi173.stdout:8/48: dwrite f2 [0,4194304] 0 2024-04-20T11:09:24.197 INFO:tasks.workunit.client.0.smithi117.stdout:4/976: dread d1/dd/fb5 [0,4194304] 0 2024-04-20T11:09:24.199 INFO:tasks.workunit.client.1.smithi173.stdout:9/48: symlink d5/l12 0 2024-04-20T11:09:24.200 INFO:tasks.workunit.client.0.smithi117.stdout:3/955: truncate d0/d5/d21/d75/d89/dd1/df4/ffd 703015 0 2024-04-20T11:09:24.207 INFO:tasks.workunit.client.1.smithi173.stdout:8/49: creat d4/fd x:0 0 0 2024-04-20T11:09:24.208 INFO:tasks.workunit.client.1.smithi173.stdout:8/50: chown d4 444 1 2024-04-20T11:09:24.214 INFO:tasks.workunit.client.0.smithi117.stdout:1/905: dread d6/f13 [4194304,4194304] 0 2024-04-20T11:09:24.215 INFO:tasks.workunit.client.1.smithi173.stdout:9/49: dread d5/df/f11 [0,4194304] 0 2024-04-20T11:09:24.215 INFO:tasks.workunit.client.0.smithi117.stdout:8/892: rename d0/d8/d11/d24/d27/d47/c4b to d0/dc/d2e/d89/c127 0 2024-04-20T11:09:24.215 INFO:tasks.workunit.client.0.smithi117.stdout:7/998: link d1/d1d/d42/d2d/c3c d1/d1d/d42/d2d/d6c/db3/c146 0 2024-04-20T11:09:24.217 INFO:tasks.workunit.client.0.smithi117.stdout:4/977: dwrite d1/d9/d1f/d87/fb3 [4194304,4194304] 0 2024-04-20T11:09:24.219 INFO:tasks.workunit.client.1.smithi173.stdout:9/50: creat d5/d9/f13 x:0 0 0 2024-04-20T11:09:24.222 INFO:tasks.workunit.client.0.smithi117.stdout:1/906: write d6/d1f/d30/d49/d124/f86 [3677723,52325] 0 2024-04-20T11:09:24.223 INFO:tasks.workunit.client.1.smithi173.stdout:9/51: symlink d5/df/l14 0 2024-04-20T11:09:24.226 INFO:tasks.workunit.client.0.smithi117.stdout:3/956: dwrite d0/d5/d21/d94/da7/f13b [0,4194304] 0 2024-04-20T11:09:24.227 INFO:tasks.workunit.client.0.smithi117.stdout:8/893: rmdir d0/d9/d118/d9c 39 2024-04-20T11:09:24.227 INFO:tasks.workunit.client.0.smithi117.stdout:7/999: write d1/d4/d3d/dce/deb/ffa [2839099,130466] 0 2024-04-20T11:09:24.227 INFO:tasks.workunit.client.1.smithi173.stdout:8/51: rmdir d4 39 2024-04-20T11:09:24.233 INFO:tasks.workunit.client.1.smithi173.stdout:9/52: unlink f3 0 2024-04-20T11:09:24.237 INFO:tasks.workunit.client.1.smithi173.stdout:8/52: dread - d4/fc zero size 2024-04-20T11:09:24.242 INFO:tasks.workunit.client.1.smithi173.stdout:8/53: readlink d4/lb 0 2024-04-20T11:09:24.244 INFO:tasks.workunit.client.0.smithi117.stdout:8/894: unlink d0/d9/d3a/d105/fc5 0 2024-04-20T11:09:24.245 INFO:tasks.workunit.client.1.smithi173.stdout:8/54: chown d4/c9 1 1 2024-04-20T11:09:24.250 INFO:tasks.workunit.client.0.smithi117.stdout:8/895: write d0/dc/d49/f101 [274005,8941] 0 2024-04-20T11:09:24.251 INFO:tasks.workunit.client.1.smithi173.stdout:8/55: dread d4/fa [0,4194304] 0 2024-04-20T11:09:24.253 INFO:tasks.workunit.client.0.smithi117.stdout:1/907: dwrite d6/d1f/d30/dfd/d101/f116 [0,4194304] 0 2024-04-20T11:09:24.256 INFO:tasks.workunit.client.1.smithi173.stdout:8/56: rmdir d4 39 2024-04-20T11:09:24.257 INFO:tasks.workunit.client.1.smithi173.stdout:9/53: dwrite d5/d9/fa [0,4194304] 0 2024-04-20T11:09:24.261 INFO:tasks.workunit.client.0.smithi117.stdout:4/978: link d1/d9/c21 d1/dd/d1a/c13d 0 2024-04-20T11:09:24.262 INFO:tasks.workunit.client.1.smithi173.stdout:8/57: dread - d4/fc zero size 2024-04-20T11:09:24.266 INFO:tasks.workunit.client.0.smithi117.stdout:3/957: write d0/d5/d7/dd/ff [222871,109282] 0 2024-04-20T11:09:24.268 INFO:tasks.workunit.client.0.smithi117.stdout:1/908: fsync d6/d35/f56 0 2024-04-20T11:09:24.269 INFO:tasks.workunit.client.1.smithi173.stdout:8/58: mknod d4/ce 0 2024-04-20T11:09:24.271 INFO:tasks.workunit.client.1.smithi173.stdout:8/59: chown l3 75 1 2024-04-20T11:09:24.274 INFO:tasks.workunit.client.1.smithi173.stdout:8/60: truncate d4/f6 1559054 0 2024-04-20T11:09:24.275 INFO:tasks.workunit.client.0.smithi117.stdout:3/958: rmdir d0/d5/d73/ded/d66/dbe 39 2024-04-20T11:09:24.282 INFO:tasks.workunit.client.1.smithi173.stdout:9/54: dwrite d5/d9/fd [0,4194304] 0 2024-04-20T11:09:24.284 INFO:tasks.workunit.client.0.smithi117.stdout:1/909: dwrite d6/d1f/d30/d49/d124/f86 [0,4194304] 0 2024-04-20T11:09:24.287 INFO:tasks.workunit.client.0.smithi117.stdout:3/959: symlink d0/d5/d21/d75/dc1/l13e 0 2024-04-20T11:09:24.290 INFO:tasks.workunit.client.0.smithi117.stdout:8/896: write d0/d8/d11/d24/d27/d47/f6c [4419918,46804] 0 2024-04-20T11:09:24.293 INFO:tasks.workunit.client.1.smithi173.stdout:8/61: creat d4/ff x:0 0 0 2024-04-20T11:09:24.293 INFO:tasks.workunit.client.0.smithi117.stdout:8/897: fsync d0/d8/d39/d40/d78/fb1 0 2024-04-20T11:09:24.295 INFO:tasks.workunit.client.1.smithi173.stdout:8/62: stat d4/fc 0 2024-04-20T11:09:24.295 INFO:tasks.workunit.client.0.smithi117.stdout:1/910: symlink d6/d16/d3a/d5a/d5c/df0/l12a 0 2024-04-20T11:09:24.304 INFO:tasks.workunit.client.0.smithi117.stdout:3/960: rename d0/d5/d7/l26 to d0/d5/d21/d94/da7/d109/l13f 0 2024-04-20T11:09:24.305 INFO:tasks.workunit.client.1.smithi173.stdout:8/63: link d4/ff d4/f10 0 2024-04-20T11:09:24.306 INFO:tasks.workunit.client.0.smithi117.stdout:3/961: chown d0/d5/c38 513768667 1 2024-04-20T11:09:24.306 INFO:tasks.workunit.client.0.smithi117.stdout:1/911: mkdir d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/d12b 0 2024-04-20T11:09:24.307 INFO:tasks.workunit.client.1.smithi173.stdout:8/64: dread d4/fa [0,4194304] 0 2024-04-20T11:09:24.308 INFO:tasks.workunit.client.0.smithi117.stdout:8/898: unlink d0/c15 0 2024-04-20T11:09:24.310 INFO:tasks.workunit.client.1.smithi173.stdout:9/55: getdents d5/df 0 2024-04-20T11:09:24.312 INFO:tasks.workunit.client.0.smithi117.stdout:1/912: creat d6/d1f/d30/dfd/d101/f12c x:0 0 0 2024-04-20T11:09:24.315 INFO:tasks.workunit.client.0.smithi117.stdout:8/899: creat d0/d9/dbb/d113/f128 x:0 0 0 2024-04-20T11:09:24.315 INFO:tasks.workunit.client.0.smithi117.stdout:3/962: dread d0/d5/d21/f47 [0,4194304] 0 2024-04-20T11:09:24.317 INFO:tasks.workunit.client.0.smithi117.stdout:1/913: mkdir d6/d16/d3a/d12d 0 2024-04-20T11:09:24.320 INFO:tasks.workunit.client.0.smithi117.stdout:8/900: mknod d0/dc/c129 0 2024-04-20T11:09:24.321 INFO:tasks.workunit.client.1.smithi173.stdout:8/65: link d4/lb d4/l11 0 2024-04-20T11:09:24.322 INFO:tasks.workunit.client.0.smithi117.stdout:3/963: mknod d0/d5/de/d4e/c140 0 2024-04-20T11:09:24.325 INFO:tasks.workunit.client.0.smithi117.stdout:1/914: creat d6/d16/d3a/d5a/d5c/da6/f12e x:0 0 0 2024-04-20T11:09:24.326 INFO:tasks.workunit.client.1.smithi173.stdout:8/66: readlink l3 0 2024-04-20T11:09:24.327 INFO:tasks.workunit.client.0.smithi117.stdout:8/901: mkdir d0/d8/d39/d40/de5/d12a 0 2024-04-20T11:09:24.328 INFO:tasks.workunit.client.1.smithi173.stdout:8/67: truncate d4/fc 450452 0 2024-04-20T11:09:24.329 INFO:tasks.workunit.client.0.smithi117.stdout:8/902: dread - d0/dc/d49/d63/fb5 zero size 2024-04-20T11:09:24.329 INFO:tasks.workunit.client.1.smithi173.stdout:8/68: fdatasync f2 0 2024-04-20T11:09:24.331 INFO:tasks.workunit.client.0.smithi117.stdout:1/915: mknod d6/d1f/d30/d49/d69/d6e/d6f/d73/c12f 0 2024-04-20T11:09:24.336 INFO:tasks.workunit.client.1.smithi173.stdout:8/69: chown d4/l11 8019 1 2024-04-20T11:09:24.337 INFO:tasks.workunit.client.0.smithi117.stdout:3/964: truncate d0/d5/d73/f95 5034629 0 2024-04-20T11:09:24.339 INFO:tasks.workunit.client.1.smithi173.stdout:8/70: chown d4/f8 101 1 2024-04-20T11:09:24.340 INFO:tasks.workunit.client.0.smithi117.stdout:3/965: readlink d0/d5/d21/l4f 0 2024-04-20T11:09:24.342 INFO:tasks.workunit.client.1.smithi173.stdout:8/71: dread - d4/ff zero size 2024-04-20T11:09:24.344 INFO:tasks.workunit.client.0.smithi117.stdout:3/966: fdatasync d0/d5/d21/f4a 0 2024-04-20T11:09:24.347 INFO:tasks.workunit.client.1.smithi173.stdout:8/72: symlink d4/l12 0 2024-04-20T11:09:24.348 INFO:tasks.workunit.client.0.smithi117.stdout:3/967: dread - d0/d5/d21/d88/d36/d111/f125 zero size 2024-04-20T11:09:24.350 INFO:tasks.workunit.client.1.smithi173.stdout:9/56: dwrite d5/df/f11 [0,4194304] 0 2024-04-20T11:09:24.351 INFO:tasks.workunit.client.0.smithi117.stdout:8/903: dwrite d0/d9/d3a/f8b [0,4194304] 0 2024-04-20T11:09:24.351 INFO:tasks.workunit.client.1.smithi173.stdout:8/73: symlink d4/l13 0 2024-04-20T11:09:24.354 INFO:tasks.workunit.client.0.smithi117.stdout:1/916: dwrite d6/d16/d3a/d5a/daf/fc5 [0,4194304] 0 2024-04-20T11:09:24.356 INFO:tasks.workunit.client.0.smithi117.stdout:8/904: mknod d0/d9/dbb/c12b 0 2024-04-20T11:09:24.359 INFO:tasks.workunit.client.0.smithi117.stdout:3/968: mkdir d0/d5/d73/ded/db3/df0/d141 0 2024-04-20T11:09:24.359 INFO:tasks.workunit.client.0.smithi117.stdout:8/905: truncate d0/d8/d11/d24/d86/f126 199856 0 2024-04-20T11:09:24.362 INFO:tasks.workunit.client.1.smithi173.stdout:8/74: mknod d4/c14 0 2024-04-20T11:09:24.368 INFO:tasks.workunit.client.0.smithi117.stdout:8/906: creat d0/d8/d11/d24/d2b/f12c x:0 0 0 2024-04-20T11:09:24.369 INFO:tasks.workunit.client.1.smithi173.stdout:8/75: mknod d4/c15 0 2024-04-20T11:09:24.370 INFO:tasks.workunit.client.1.smithi173.stdout:9/57: link d5/l12 d5/df/l15 0 2024-04-20T11:09:24.372 INFO:tasks.workunit.client.0.smithi117.stdout:8/907: symlink d0/d9/d118/l12d 0 2024-04-20T11:09:24.373 INFO:tasks.workunit.client.1.smithi173.stdout:6/27: sync 2024-04-20T11:09:24.373 INFO:tasks.workunit.client.1.smithi173.stdout:6/28: dwrite - no filename 2024-04-20T11:09:24.373 INFO:tasks.workunit.client.1.smithi173.stdout:5/6: sync 2024-04-20T11:09:24.374 INFO:tasks.workunit.client.1.smithi173.stdout:5/7: dwrite - no filename 2024-04-20T11:09:24.374 INFO:tasks.workunit.client.1.smithi173.stdout:7/14: sync 2024-04-20T11:09:24.374 INFO:tasks.workunit.client.1.smithi173.stdout:9/58: mknod d5/c16 0 2024-04-20T11:09:24.375 INFO:tasks.workunit.client.1.smithi173.stdout:7/15: read - f3 zero size 2024-04-20T11:09:24.376 INFO:tasks.workunit.client.1.smithi173.stdout:9/59: chown f0 484658 1 2024-04-20T11:09:24.377 INFO:tasks.workunit.client.1.smithi173.stdout:2/5: sync 2024-04-20T11:09:24.377 INFO:tasks.workunit.client.1.smithi173.stdout:8/76: mknod d4/c16 0 2024-04-20T11:09:24.381 INFO:tasks.workunit.client.0.smithi117.stdout:1/917: dwrite d6/d16/d3a/d5a/d5c/d61/d98/f11f [0,4194304] 0 2024-04-20T11:09:24.386 INFO:tasks.workunit.client.0.smithi117.stdout:3/969: write d0/d5/d21/d77/df3/f100 [1049210,98068] 0 2024-04-20T11:09:24.387 INFO:tasks.workunit.client.1.smithi173.stdout:6/29: symlink d4/l7 0 2024-04-20T11:09:24.388 INFO:tasks.workunit.client.0.smithi117.stdout:1/918: truncate d6/d16/fbb 142591 0 2024-04-20T11:09:24.390 INFO:tasks.workunit.client.0.smithi117.stdout:3/970: write d0/f86 [423624,37954] 0 2024-04-20T11:09:24.391 INFO:tasks.workunit.client.1.smithi173.stdout:2/6: mknod c0 0 2024-04-20T11:09:24.392 INFO:tasks.workunit.client.1.smithi173.stdout:5/8: rmdir d0 0 2024-04-20T11:09:24.392 INFO:tasks.workunit.client.1.smithi173.stdout:5/9: readlink - no filename 2024-04-20T11:09:24.392 INFO:tasks.workunit.client.1.smithi173.stdout:5/10: readlink - no filename 2024-04-20T11:09:24.392 INFO:tasks.workunit.client.1.smithi173.stdout:5/11: stat - no entries 2024-04-20T11:09:24.392 INFO:tasks.workunit.client.1.smithi173.stdout:5/12: write - no filename 2024-04-20T11:09:24.393 INFO:tasks.workunit.client.0.smithi117.stdout:8/908: dread d0/d9/d3a/f8b [0,4194304] 0 2024-04-20T11:09:24.395 INFO:tasks.workunit.client.0.smithi117.stdout:8/909: stat d0/d8/d11/d24/d76/f96 0 2024-04-20T11:09:24.404 INFO:tasks.workunit.client.1.smithi173.stdout:7/16: dwrite f1 [0,4194304] 0 2024-04-20T11:09:24.405 INFO:tasks.workunit.client.1.smithi173.stdout:9/60: dwrite d5/df/f11 [0,4194304] 0 2024-04-20T11:09:24.406 INFO:tasks.workunit.client.1.smithi173.stdout:6/30: creat d4/f8 x:0 0 0 2024-04-20T11:09:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:24 smithi173 ceph-mon[23800]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:24 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1298043032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:09:24.411 INFO:tasks.workunit.client.0.smithi117.stdout:1/919: dread d6/d16/d3a/d5a/d5c/d61/d98/f11f [0,4194304] 0 2024-04-20T11:09:24.416 INFO:tasks.workunit.client.0.smithi117.stdout:8/910: dwrite d0/d8/d11/d6d/d84/fc3 [0,4194304] 0 2024-04-20T11:09:24.420 INFO:tasks.workunit.client.0.smithi117.stdout:3/971: write d0/d5/d73/fa3 [1116439,16468] 0 2024-04-20T11:09:24.423 INFO:tasks.workunit.client.0.smithi117.stdout:1/920: creat d6/d16/d3a/d11e/f130 x:0 0 0 2024-04-20T11:09:24.425 INFO:tasks.workunit.client.0.smithi117.stdout:8/911: mknod d0/dc/d2e/d89/c12e 0 2024-04-20T11:09:24.430 INFO:tasks.workunit.client.0.smithi117.stdout:3/972: creat d0/d5/d21/d75/d89/d116/f142 x:0 0 0 2024-04-20T11:09:24.431 INFO:tasks.workunit.client.0.smithi117.stdout:8/912: truncate d0/d8/d39/d40/f6e 8681102 0 2024-04-20T11:09:24.433 INFO:tasks.workunit.client.0.smithi117.stdout:3/973: fsync d0/d5/d73/ded/dac/ff8 0 2024-04-20T11:09:24.434 INFO:tasks.workunit.client.1.smithi173.stdout:1/43: sync 2024-04-20T11:09:24.434 INFO:tasks.workunit.client.1.smithi173.stdout:0/17: sync 2024-04-20T11:09:24.434 INFO:tasks.workunit.client.1.smithi173.stdout:3/41: sync 2024-04-20T11:09:24.434 INFO:tasks.workunit.client.1.smithi173.stdout:4/29: sync 2024-04-20T11:09:24.437 INFO:tasks.workunit.client.0.smithi117.stdout:8/913: link d0/d9/cca d0/dc/c12f 0 2024-04-20T11:09:24.444 INFO:tasks.workunit.client.0.smithi117.stdout:8/914: getdents d0/d8/d11/d24/d27/d47 0 2024-04-20T11:09:24.444 INFO:tasks.workunit.client.0.smithi117.stdout:1/921: dwrite d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/fee [0,4194304] 0 2024-04-20T11:09:24.449 INFO:tasks.workunit.client.1.smithi173.stdout:5/13: symlink l1 0 2024-04-20T11:09:24.450 INFO:tasks.workunit.client.1.smithi173.stdout:7/17: fdatasync f1 0 2024-04-20T11:09:24.452 INFO:tasks.workunit.client.1.smithi173.stdout:9/61: chown f0 1915 1 2024-04-20T11:09:24.454 INFO:tasks.workunit.client.1.smithi173.stdout:4/30: chown f5 6 1 2024-04-20T11:09:24.455 INFO:tasks.workunit.client.1.smithi173.stdout:8/77: dwrite d4/fd [0,4194304] 0 2024-04-20T11:09:24.455 INFO:tasks.workunit.client.1.smithi173.stdout:7/18: symlink l4 0 2024-04-20T11:09:24.456 INFO:tasks.workunit.client.1.smithi173.stdout:5/14: rename l1 to l2 0 2024-04-20T11:09:24.457 INFO:tasks.workunit.client.1.smithi173.stdout:9/62: write f0 [5643887,4544] 0 2024-04-20T11:09:24.458 INFO:tasks.workunit.client.1.smithi173.stdout:7/19: truncate f3 814610 0 2024-04-20T11:09:24.458 INFO:tasks.workunit.client.1.smithi173.stdout:1/44: mkdir d5/d7/da/df 0 2024-04-20T11:09:24.463 INFO:tasks.workunit.client.1.smithi173.stdout:8/78: write d4/fa [804029,101954] 0 2024-04-20T11:09:24.463 INFO:tasks.workunit.client.1.smithi173.stdout:7/20: symlink l5 0 2024-04-20T11:09:24.463 INFO:tasks.workunit.client.1.smithi173.stdout:1/45: chown c2 129061 1 2024-04-20T11:09:24.466 INFO:tasks.workunit.client.1.smithi173.stdout:3/42: rename f6 to d7/fd 0 2024-04-20T11:09:24.466 INFO:tasks.workunit.client.0.smithi117.stdout:8/915: write d0/d8/d11/d24/d2b/f116 [1130506,44504] 0 2024-04-20T11:09:24.470 INFO:tasks.workunit.client.0.smithi117.stdout:1/922: dwrite d6/d1f/d30/dfd/d101/f116 [0,4194304] 0 2024-04-20T11:09:24.472 INFO:tasks.workunit.client.0.smithi117.stdout:8/916: write d0/d8/d39/d40/d78/fb1 [3471024,64313] 0 2024-04-20T11:09:24.474 INFO:tasks.workunit.client.1.smithi173.stdout:7/21: rename f3 to f6 0 2024-04-20T11:09:24.478 INFO:tasks.workunit.client.1.smithi173.stdout:8/79: write d4/fa [1629365,66694] 0 2024-04-20T11:09:24.479 INFO:tasks.workunit.client.0.smithi117.stdout:1/923: read d6/d1f/d30/d32/f6a [6595215,50036] 0 2024-04-20T11:09:24.479 INFO:tasks.workunit.client.1.smithi173.stdout:1/46: truncate d5/d7/fd 263814 0 2024-04-20T11:09:24.480 INFO:tasks.workunit.client.0.smithi117.stdout:8/917: write d0/f23 [3902966,100651] 0 2024-04-20T11:09:24.483 INFO:tasks.workunit.client.0.smithi117.stdout:3/974: dread d0/d5/d21/d94/d9a/fba [0,4194304] 0 2024-04-20T11:09:24.484 INFO:tasks.workunit.client.0.smithi117.stdout:1/924: chown d6/d1f/d30/ca1 1990037 1 2024-04-20T11:09:24.485 INFO:tasks.workunit.client.1.smithi173.stdout:1/47: write f0 [826456,28227] 0 2024-04-20T11:09:24.487 INFO:tasks.workunit.client.1.smithi173.stdout:5/15: chown l2 3000 1 2024-04-20T11:09:24.492 INFO:tasks.workunit.client.0.smithi117.stdout:1/925: symlink d6/d16/d3a/l131 0 2024-04-20T11:09:24.493 INFO:tasks.workunit.client.1.smithi173.stdout:4/31: dwrite f8 [0,4194304] 0 2024-04-20T11:09:24.495 INFO:tasks.workunit.client.1.smithi173.stdout:5/16: creat f3 x:0 0 0 2024-04-20T11:09:24.496 INFO:tasks.workunit.client.0.smithi117.stdout:3/975: mkdir d0/d5/de/d4e/dec/d143 0 2024-04-20T11:09:24.498 INFO:tasks.workunit.client.1.smithi173.stdout:9/63: dwrite d5/d9/fd [4194304,4194304] 0 2024-04-20T11:09:24.498 INFO:tasks.workunit.client.0.smithi117.stdout:1/926: unlink d6/d16/d3a/d11e/f130 0 2024-04-20T11:09:24.501 INFO:tasks.workunit.client.0.smithi117.stdout:1/927: chown d6/d16/d3a/d5a/d5c/d61/d98 15047702 1 2024-04-20T11:09:24.503 INFO:tasks.workunit.client.1.smithi173.stdout:5/17: truncate f3 926754 0 2024-04-20T11:09:24.503 INFO:tasks.workunit.client.0.smithi117.stdout:8/918: link d0/d8/d11/d24/d53/l79 d0/d8/d11/df3/l130 0 2024-04-20T11:09:24.506 INFO:tasks.workunit.client.1.smithi173.stdout:9/64: symlink d5/d9/l17 0 2024-04-20T11:09:24.507 INFO:tasks.workunit.client.1.smithi173.stdout:8/80: getdents d4 0 2024-04-20T11:09:24.509 INFO:tasks.workunit.client.1.smithi173.stdout:4/32: symlink da/dc/ld 0 2024-04-20T11:09:24.510 INFO:tasks.workunit.client.1.smithi173.stdout:5/18: rename l2 to l4 0 2024-04-20T11:09:24.510 INFO:tasks.workunit.client.1.smithi173.stdout:1/48: getdents d5 0 2024-04-20T11:09:24.514 INFO:tasks.workunit.client.1.smithi173.stdout:3/43: dwrite d7/f8 [0,4194304] 0 2024-04-20T11:09:24.514 INFO:tasks.workunit.client.1.smithi173.stdout:9/65: write d5/df/f10 [777011,26571] 0 2024-04-20T11:09:24.515 INFO:tasks.workunit.client.1.smithi173.stdout:1/49: write f4 [1881144,125370] 0 2024-04-20T11:09:24.517 INFO:tasks.workunit.client.1.smithi173.stdout:9/66: write f0 [2734992,69794] 0 2024-04-20T11:09:24.518 INFO:tasks.workunit.client.1.smithi173.stdout:8/81: dread d4/f6 [0,4194304] 0 2024-04-20T11:09:24.520 INFO:tasks.workunit.client.1.smithi173.stdout:8/82: write d4/fc [528022,73075] 0 2024-04-20T11:09:24.526 INFO:tasks.workunit.client.1.smithi173.stdout:9/67: rmdir d5/df 39 2024-04-20T11:09:24.530 INFO:tasks.workunit.client.0.smithi117.stdout:1/928: dwrite d6/d16/d3a/f94 [0,4194304] 0 2024-04-20T11:09:24.536 INFO:tasks.workunit.client.0.smithi117.stdout:8/919: dwrite d0/d9/d3a/f8b [4194304,4194304] 0 2024-04-20T11:09:24.536 INFO:tasks.workunit.client.1.smithi173.stdout:1/50: dwrite d5/fb [0,4194304] 0 2024-04-20T11:09:24.537 INFO:tasks.workunit.client.1.smithi173.stdout:3/44: dwrite d7/d9/fb [0,4194304] 0 2024-04-20T11:09:24.537 INFO:tasks.workunit.client.1.smithi173.stdout:4/33: getdents da 0 2024-04-20T11:09:24.540 INFO:tasks.workunit.client.0.smithi117.stdout:1/929: chown d6/d16/d3a/d5a/d5c/d61/d71/la5 1294 1 2024-04-20T11:09:24.540 INFO:tasks.workunit.client.0.smithi117.stdout:3/976: dread d0/d2a/fdd [0,4194304] 0 2024-04-20T11:09:24.540 INFO:tasks.workunit.client.1.smithi173.stdout:9/68: mkdir d5/df/d18 0 2024-04-20T11:09:24.544 INFO:tasks.workunit.client.1.smithi173.stdout:5/19: rename l4 to l5 0 2024-04-20T11:09:24.547 INFO:tasks.workunit.client.1.smithi173.stdout:4/34: unlink f5 0 2024-04-20T11:09:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.547+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8d0000bf0 con 0x7fa8d806eac0 2024-04-20T11:09:24.549 INFO:tasks.workunit.client.0.smithi117.stdout:8/920: symlink d0/d8/d11/d24/d27/d47/l131 0 2024-04-20T11:09:24.551 INFO:tasks.workunit.client.0.smithi117.stdout:1/930: rename d6/d1f/d30/d49/lab to d6/d1f/d30/d49/d69/d6e/d6f/d73/ddf/l132 0 2024-04-20T11:09:24.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.552+0000 7fa8ed600700 1 -- 172.21.15.117:0/3531694754 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fa8d0000bf0 con 0x7fa8d806eac0 2024-04-20T11:09:24.555 INFO:tasks.workunit.client.1.smithi173.stdout:4/35: fsync f1 0 2024-04-20T11:09:24.556 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:09:24.556 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:09:24.556 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:09:24.557 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:09:24.557 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:09:24.557 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:09:24.557 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:09:24.557 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "2/2 daemons upgraded", 2024-04-20T11:09:24.557 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "", 2024-04-20T11:09:24.557 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:09:24.557 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:09:24.557 INFO:tasks.workunit.client.1.smithi173.stdout:4/36: stat f8 0 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.557+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa8d806eac0 msgr2=0x7fa8d8070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.557+0000 7fa8f7a00700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa8d806eac0 0x7fa8d8070f80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4005950 tx=0x7fa8e40058e0 comp rx=0 tx=0).stop 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.558+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f0073940 msgr2=0x7fa8f01915e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.558+0000 7fa8f7a00700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f0073940 0x7fa8f01915e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000c00 tx=0x7fa8dc00be20 comp rx=0 tx=0).stop 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.558+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 shutdown_connections 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.558+0000 7fa8f7a00700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa8d806eac0 0x7fa8d8070f80 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4005950 tx=0x7fa8e40058e0 comp rx=0 tx=0).stop 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.558+0000 7fa8f7a00700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f0073940 0x7fa8f01915e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.558+0000 7fa8f7a00700 1 --2- 172.21.15.117:0/3531694754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f0074f10 0x7fa8f0191b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.558+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 >> 172.21.15.117:0/3531694754 conn(0x7fa8f00fc1c0 msgr2=0x7fa8f0106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:24.559 INFO:tasks.workunit.client.1.smithi173.stdout:4/37: chown da/dc/ld 232 1 2024-04-20T11:09:24.559 INFO:tasks.workunit.client.1.smithi173.stdout:4/38: stat f1 0 2024-04-20T11:09:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.559+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 shutdown_connections 2024-04-20T11:09:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:24.559+0000 7fa8f7a00700 1 -- 172.21.15.117:0/3531694754 wait complete. 2024-04-20T11:09:24.569 INFO:tasks.workunit.client.0.smithi117.stdout:8/921: dread d0/d8/d11/d6d/d84/fc1 [0,4194304] 0 2024-04-20T11:09:24.570 INFO:tasks.workunit.client.1.smithi173.stdout:1/51: link c2 d5/d7/c10 0 2024-04-20T11:09:24.570 INFO:tasks.workunit.client.0.smithi117.stdout:1/931: unlink d6/d1f/d30/d49/d69/fff 0 2024-04-20T11:09:24.572 INFO:tasks.workunit.client.1.smithi173.stdout:4/39: creat da/dc/fe x:0 0 0 2024-04-20T11:09:24.572 INFO:tasks.workunit.client.0.smithi117.stdout:3/977: link d0/d5/d21/d88/d85/c105 d0/d5/de/d4e/dec/d118/c144 0 2024-04-20T11:09:24.574 INFO:tasks.workunit.client.1.smithi173.stdout:4/40: write f8 [1240498,43943] 0 2024-04-20T11:09:24.575 INFO:tasks.workunit.client.1.smithi173.stdout:1/52: creat d5/d7/f11 x:0 0 0 2024-04-20T11:09:24.576 INFO:tasks.workunit.client.0.smithi117.stdout:1/932: chown d6/d35/d63/la4 5444869 1 2024-04-20T11:09:24.578 INFO:tasks.workunit.client.1.smithi173.stdout:2/7: sync 2024-04-20T11:09:24.578 INFO:tasks.workunit.client.1.smithi173.stdout:3/45: fdatasync d7/d9/fb 0 2024-04-20T11:09:24.579 INFO:tasks.workunit.client.1.smithi173.stdout:2/8: chown c0 104812635 1 2024-04-20T11:09:24.581 INFO:tasks.workunit.client.1.smithi173.stdout:4/41: fdatasync f4 0 2024-04-20T11:09:24.583 INFO:tasks.workunit.client.0.smithi117.stdout:3/978: fsync d0/d5/d73/fd5 0 2024-04-20T11:09:24.584 INFO:tasks.workunit.client.1.smithi173.stdout:1/53: mkdir d5/d7/d12 0 2024-04-20T11:09:24.586 INFO:tasks.workunit.client.0.smithi117.stdout:3/979: dread - d0/d5/d21/d88/d36/d111/f125 zero size 2024-04-20T11:09:24.587 INFO:tasks.workunit.client.1.smithi173.stdout:5/20: readlink l5 0 2024-04-20T11:09:24.591 INFO:tasks.workunit.client.1.smithi173.stdout:4/42: dread f4 [0,4194304] 0 2024-04-20T11:09:24.591 INFO:tasks.workunit.client.0.smithi117.stdout:8/922: mknod d0/d8/d39/d40/de5/d119/c132 0 2024-04-20T11:09:24.594 INFO:tasks.workunit.client.1.smithi173.stdout:2/9: rename c0 to c1 0 2024-04-20T11:09:24.599 INFO:tasks.workunit.client.0.smithi117.stdout:1/933: dwrite d6/d35/f36 [0,4194304] 0 2024-04-20T11:09:24.602 INFO:tasks.workunit.client.0.smithi117.stdout:3/980: unlink d0/d5/de/d4e/l7f 0 2024-04-20T11:09:24.604 INFO:tasks.workunit.client.1.smithi173.stdout:5/21: creat f6 x:0 0 0 2024-04-20T11:09:24.604 INFO:tasks.workunit.client.1.smithi173.stdout:5/22: fsync f3 0 2024-04-20T11:09:24.609 INFO:tasks.workunit.client.1.smithi173.stdout:3/46: unlink c5 0 2024-04-20T11:09:24.610 INFO:tasks.workunit.client.1.smithi173.stdout:9/69: truncate d5/df/f11 1215150 0 2024-04-20T11:09:24.611 INFO:tasks.workunit.client.0.smithi117.stdout:1/934: creat d6/d16/d3a/d5a/d5c/d61/d98/deb/f133 x:0 0 0 2024-04-20T11:09:24.612 INFO:tasks.workunit.client.1.smithi173.stdout:3/47: write d7/d9/fb [1934539,106595] 0 2024-04-20T11:09:24.612 INFO:tasks.workunit.client.1.smithi173.stdout:4/43: creat da/dc/ff x:0 0 0 2024-04-20T11:09:24.614 INFO:tasks.workunit.client.1.smithi173.stdout:4/44: write f4 [764968,36364] 0 2024-04-20T11:09:24.616 INFO:tasks.workunit.client.1.smithi173.stdout:2/10: mkdir d2 0 2024-04-20T11:09:24.620 INFO:tasks.workunit.client.1.smithi173.stdout:5/23: creat f7 x:0 0 0 2024-04-20T11:09:24.623 INFO:tasks.workunit.client.1.smithi173.stdout:3/48: creat d7/d9/fe x:0 0 0 2024-04-20T11:09:24.625 INFO:tasks.workunit.client.0.smithi117.stdout:3/981: rename d0/d5/d73/ded/f97 to d0/d5/d7/f145 0 2024-04-20T11:09:24.632 INFO:tasks.workunit.client.1.smithi173.stdout:2/11: stat c1 0 2024-04-20T11:09:24.635 INFO:tasks.workunit.client.1.smithi173.stdout:2/12: rename d2 to d2/d3 22 2024-04-20T11:09:24.635 INFO:tasks.workunit.client.1.smithi173.stdout:2/13: dread - no filename 2024-04-20T11:09:24.635 INFO:tasks.workunit.client.1.smithi173.stdout:2/14: truncate - no filename 2024-04-20T11:09:24.638 INFO:tasks.workunit.client.1.smithi173.stdout:9/70: link d5/df/l14 d5/l19 0 2024-04-20T11:09:24.639 INFO:tasks.workunit.client.1.smithi173.stdout:2/15: chown c1 511095 1 2024-04-20T11:09:24.639 INFO:tasks.workunit.client.0.smithi117.stdout:3/982: getdents d0/d5/d73/ded/dac 0 2024-04-20T11:09:24.639 INFO:tasks.workunit.client.1.smithi173.stdout:6/31: sync 2024-04-20T11:09:24.642 INFO:tasks.workunit.client.1.smithi173.stdout:3/49: dread d7/f8 [0,4194304] 0 2024-04-20T11:09:24.643 INFO:tasks.workunit.client.0.smithi117.stdout:1/935: truncate d6/d16/d3a/d5a/d5c/d61/d98/fed 2314206 0 2024-04-20T11:09:24.643 INFO:tasks.workunit.client.1.smithi173.stdout:6/32: readlink l3 0 2024-04-20T11:09:24.643 INFO:tasks.workunit.client.0.smithi117.stdout:3/983: fdatasync d0/d5/d21/d75/d89/dd1/df4/ffa 0 2024-04-20T11:09:24.644 INFO:tasks.workunit.client.0.smithi117.stdout:8/923: dwrite d0/f57 [0,4194304] 0 2024-04-20T11:09:24.646 INFO:tasks.workunit.client.1.smithi173.stdout:4/45: truncate f8 1814186 0 2024-04-20T11:09:24.646 INFO:tasks.workunit.client.1.smithi173.stdout:5/24: dwrite f3 [0,4194304] 0 2024-04-20T11:09:24.648 INFO:tasks.workunit.client.1.smithi173.stdout:9/71: chown d5/df/l15 3731 1 2024-04-20T11:09:24.648 INFO:tasks.workunit.client.1.smithi173.stdout:1/54: dwrite d5/d6/dc/fe [0,4194304] 0 2024-04-20T11:09:24.649 INFO:tasks.workunit.client.1.smithi173.stdout:2/16: symlink d2/l4 0 2024-04-20T11:09:24.649 INFO:tasks.workunit.client.1.smithi173.stdout:2/17: dread - no filename 2024-04-20T11:09:24.653 INFO:tasks.workunit.client.0.smithi117.stdout:1/936: mkdir d6/d16/d3a/d5a/d5c/df0/d10c/d134 0 2024-04-20T11:09:24.661 INFO:tasks.workunit.client.1.smithi173.stdout:9/72: dread d5/df/f10 [0,4194304] 0 2024-04-20T11:09:24.665 INFO:tasks.workunit.client.0.smithi117.stdout:1/937: creat d6/d16/d3a/d12d/f135 x:0 0 0 2024-04-20T11:09:24.665 INFO:tasks.workunit.client.1.smithi173.stdout:2/18: mkdir d2/d5 0 2024-04-20T11:09:24.668 INFO:tasks.workunit.client.1.smithi173.stdout:3/50: fsync f3 0 2024-04-20T11:09:24.669 INFO:tasks.workunit.client.1.smithi173.stdout:3/51: chown d7/d9/lc 1 1 2024-04-20T11:09:24.671 INFO:tasks.workunit.client.0.smithi117.stdout:8/924: write d0/d8/d11/d24/d53/f56 [5089755,2211] 0 2024-04-20T11:09:24.673 INFO:tasks.workunit.client.1.smithi173.stdout:7/22: sync 2024-04-20T11:09:24.673 INFO:tasks.workunit.client.1.smithi173.stdout:8/83: sync 2024-04-20T11:09:24.673 INFO:tasks.workunit.client.1.smithi173.stdout:0/18: sync 2024-04-20T11:09:24.674 INFO:tasks.workunit.client.1.smithi173.stdout:8/84: fsync d4/f8 0 2024-04-20T11:09:24.675 INFO:tasks.workunit.client.1.smithi173.stdout:5/25: link f7 f8 0 2024-04-20T11:09:24.676 INFO:tasks.workunit.client.1.smithi173.stdout:9/73: creat d5/d9/f1a x:0 0 0 2024-04-20T11:09:24.681 INFO:tasks.workunit.client.1.smithi173.stdout:6/33: link d4/l7 d4/l9 0 2024-04-20T11:09:24.682 INFO:tasks.workunit.client.1.smithi173.stdout:3/52: write d7/fd [933528,28086] 0 2024-04-20T11:09:24.682 INFO:tasks.workunit.client.1.smithi173.stdout:0/19: fsync f2 0 2024-04-20T11:09:24.684 INFO:tasks.workunit.client.0.smithi117.stdout:3/984: dwrite d0/d5/d73/f95 [0,4194304] 0 2024-04-20T11:09:24.684 INFO:tasks.workunit.client.1.smithi173.stdout:7/23: rename l4 to l7 0 2024-04-20T11:09:24.687 INFO:tasks.workunit.client.1.smithi173.stdout:5/26: unlink f3 0 2024-04-20T11:09:24.689 INFO:tasks.workunit.client.0.smithi117.stdout:3/985: unlink d0/d5/d73/ded/d66/d63/dad/f123 0 2024-04-20T11:09:24.689 INFO:tasks.workunit.client.1.smithi173.stdout:2/19: symlink d2/d5/l6 0 2024-04-20T11:09:24.689 INFO:tasks.workunit.client.1.smithi173.stdout:5/27: read - f8 zero size 2024-04-20T11:09:24.690 INFO:tasks.workunit.client.1.smithi173.stdout:2/20: chown d2 15806897 1 2024-04-20T11:09:24.690 INFO:tasks.workunit.client.1.smithi173.stdout:2/21: dwrite - no filename 2024-04-20T11:09:24.690 INFO:tasks.workunit.client.1.smithi173.stdout:2/22: dread - no filename 2024-04-20T11:09:24.691 INFO:tasks.workunit.client.1.smithi173.stdout:5/28: chown f7 1 1 2024-04-20T11:09:24.695 INFO:tasks.workunit.client.0.smithi117.stdout:1/938: dread d6/f13 [0,4194304] 0 2024-04-20T11:09:24.703 INFO:tasks.workunit.client.1.smithi173.stdout:6/34: dwrite d4/f8 [0,4194304] 0 2024-04-20T11:09:24.790 INFO:tasks.workunit.client.1.smithi173.stdout:0/20: symlink l5 0 2024-04-20T11:09:24.791 INFO:tasks.workunit.client.1.smithi173.stdout:8/85: link f2 d4/f17 0 2024-04-20T11:09:24.793 INFO:tasks.workunit.client.1.smithi173.stdout:8/86: fdatasync d4/f8 0 2024-04-20T11:09:24.795 INFO:tasks.workunit.client.1.smithi173.stdout:4/46: truncate f4 2036960 0 2024-04-20T11:09:24.797 INFO:tasks.workunit.client.0.smithi117.stdout:1/939: fsync d6/d1f/d30/d32/fcb 0 2024-04-20T11:09:24.799 INFO:tasks.workunit.client.1.smithi173.stdout:8/87: dread d4/f17 [4194304,4194304] 0 2024-04-20T11:09:24.801 INFO:tasks.workunit.client.1.smithi173.stdout:3/53: creat d7/d9/ff x:0 0 0 2024-04-20T11:09:24.801 INFO:tasks.workunit.client.0.smithi117.stdout:3/986: truncate d0/d5/d73/ded/d66/f135 1386551 0 2024-04-20T11:09:24.802 INFO:tasks.workunit.client.1.smithi173.stdout:8/88: write d4/fd [4847210,7444] 0 2024-04-20T11:09:24.805 INFO:tasks.workunit.client.1.smithi173.stdout:8/89: fsync d4/f8 0 2024-04-20T11:09:24.807 INFO:tasks.workunit.client.0.smithi117.stdout:1/940: creat d6/d16/d3a/d5a/d5c/df0/f136 x:0 0 0 2024-04-20T11:09:24.809 INFO:tasks.workunit.client.1.smithi173.stdout:1/55: truncate d5/d6/dc/fe 1971068 0 2024-04-20T11:09:24.810 INFO:tasks.workunit.client.0.smithi117.stdout:1/941: write d6/d35/fe1 [1943572,20224] 0 2024-04-20T11:09:24.814 INFO:tasks.workunit.client.1.smithi173.stdout:2/23: symlink d2/l7 0 2024-04-20T11:09:24.815 INFO:tasks.workunit.client.0.smithi117.stdout:1/942: read d6/d16/d3a/d5a/d5c/da6/f110 [500271,75645] 0 2024-04-20T11:09:24.821 INFO:tasks.workunit.client.0.smithi117.stdout:3/987: symlink d0/d5/d73/dd8/l146 0 2024-04-20T11:09:24.826 INFO:tasks.workunit.client.0.smithi117.stdout:3/988: write d0/d5/d73/ded/dac/dd2/f13c [742165,105085] 0 2024-04-20T11:09:24.833 INFO:tasks.workunit.client.1.smithi173.stdout:9/74: write d5/df/f11 [2259627,43529] 0 2024-04-20T11:09:24.836 INFO:tasks.workunit.client.0.smithi117.stdout:1/943: dwrite d6/d16/f62 [0,4194304] 0 2024-04-20T11:09:24.839 INFO:tasks.workunit.client.0.smithi117.stdout:1/944: write d6/d16/d3a/d5a/fec [55388,22318] 0 2024-04-20T11:09:24.843 INFO:tasks.workunit.client.0.smithi117.stdout:3/989: link d0/d5/d21/d75/d89/d116/l121 d0/d5/d73/ded/dac/l147 0 2024-04-20T11:09:24.845 INFO:tasks.workunit.client.0.smithi117.stdout:1/945: mknod d6/d16/d3a/d5a/daf/df6/c137 0 2024-04-20T11:09:24.845 INFO:tasks.workunit.client.0.smithi117.stdout:3/990: chown d0/d5/d21/d75/d89/dd1/f10d 839 1 2024-04-20T11:09:24.853 INFO:tasks.workunit.client.0.smithi117.stdout:3/991: rename d0/d5/d11/d80/l12e to d0/d5/d73/ded/dac/dd2/l148 0 2024-04-20T11:09:24.859 INFO:tasks.workunit.client.1.smithi173.stdout:5/29: dwrite f7 [0,4194304] 0 2024-04-20T11:09:24.865 INFO:tasks.workunit.client.1.smithi173.stdout:4/47: sync 2024-04-20T11:09:24.892 INFO:tasks.workunit.client.0.smithi117.stdout:1/946: dwrite d6/d16/d3a/d5a/d5c/d61/f83 [0,4194304] 0 2024-04-20T11:09:24.896 INFO:tasks.workunit.client.0.smithi117.stdout:1/947: write d6/d1f/d30/d49/d124/db9/f102 [1671988,39967] 0 2024-04-20T11:09:24.898 INFO:tasks.workunit.client.0.smithi117.stdout:1/948: dread - d6/d1f/d30/d49/d124/fb8 zero size 2024-04-20T11:09:24.902 INFO:tasks.workunit.client.1.smithi173.stdout:0/21: mkdir d6 0 2024-04-20T11:09:24.903 INFO:tasks.workunit.client.0.smithi117.stdout:1/949: write d6/d1f/d30/d49/d124/db9/f118 [207235,107539] 0 2024-04-20T11:09:24.905 INFO:tasks.workunit.client.1.smithi173.stdout:0/22: read - f4 zero size 2024-04-20T11:09:24.908 INFO:tasks.workunit.client.1.smithi173.stdout:3/54: mknod d7/c10 0 2024-04-20T11:09:24.908 INFO:tasks.workunit.client.0.smithi117.stdout:1/950: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/de9/f138 x:0 0 0 2024-04-20T11:09:24.909 INFO:tasks.workunit.client.1.smithi173.stdout:0/23: chown f2 35 1 2024-04-20T11:09:24.911 INFO:tasks.workunit.client.1.smithi173.stdout:0/24: chown d6 8508 1 2024-04-20T11:09:24.911 INFO:tasks.workunit.client.1.smithi173.stdout:2/24: rmdir d2/d5 39 2024-04-20T11:09:24.911 INFO:tasks.workunit.client.1.smithi173.stdout:2/25: write - no filename 2024-04-20T11:09:24.911 INFO:tasks.workunit.client.1.smithi173.stdout:2/26: read - no filename 2024-04-20T11:09:24.912 INFO:tasks.workunit.client.1.smithi173.stdout:2/27: write - no filename 2024-04-20T11:09:24.913 INFO:tasks.workunit.client.1.smithi173.stdout:0/25: dread - f2 zero size 2024-04-20T11:09:24.914 INFO:tasks.workunit.client.1.smithi173.stdout:9/75: unlink d5/d9/fd 0 2024-04-20T11:09:24.919 INFO:tasks.workunit.client.1.smithi173.stdout:5/30: creat f9 x:0 0 0 2024-04-20T11:09:24.922 INFO:tasks.workunit.client.1.smithi173.stdout:7/24: link l5 l8 0 2024-04-20T11:09:24.924 INFO:tasks.workunit.client.1.smithi173.stdout:1/56: getdents d5/d7/d12 0 2024-04-20T11:09:24.927 INFO:tasks.workunit.client.1.smithi173.stdout:5/31: mkdir da 0 2024-04-20T11:09:24.928 INFO:tasks.workunit.client.1.smithi173.stdout:6/35: link c2 d4/ca 0 2024-04-20T11:09:24.931 INFO:tasks.workunit.client.1.smithi173.stdout:7/25: mkdir d9 0 2024-04-20T11:09:24.932 INFO:tasks.workunit.client.1.smithi173.stdout:0/26: creat d6/f7 x:0 0 0 2024-04-20T11:09:24.933 INFO:tasks.workunit.client.1.smithi173.stdout:2/28: mknod d2/d5/c8 0 2024-04-20T11:09:24.933 INFO:tasks.workunit.client.1.smithi173.stdout:2/29: dread - no filename 2024-04-20T11:09:24.933 INFO:tasks.workunit.client.1.smithi173.stdout:2/30: dwrite - no filename 2024-04-20T11:09:24.933 INFO:tasks.workunit.client.1.smithi173.stdout:2/31: dwrite - no filename 2024-04-20T11:09:24.936 INFO:tasks.workunit.client.1.smithi173.stdout:0/27: dread - f4 zero size 2024-04-20T11:09:24.938 INFO:tasks.workunit.client.1.smithi173.stdout:8/90: getdents d4 0 2024-04-20T11:09:24.939 INFO:tasks.workunit.client.1.smithi173.stdout:7/26: write f1 [861364,95307] 0 2024-04-20T11:09:24.940 INFO:tasks.workunit.client.1.smithi173.stdout:1/57: mknod d5/d6/dc/c13 0 2024-04-20T11:09:24.943 INFO:tasks.workunit.client.1.smithi173.stdout:2/32: symlink d2/d5/l9 0 2024-04-20T11:09:24.943 INFO:tasks.workunit.client.1.smithi173.stdout:2/33: dwrite - no filename 2024-04-20T11:09:24.947 INFO:tasks.workunit.client.1.smithi173.stdout:3/55: getdents d7 0 2024-04-20T11:09:24.949 INFO:tasks.workunit.client.1.smithi173.stdout:1/58: read - d5/d6/f9 zero size 2024-04-20T11:09:24.949 INFO:tasks.workunit.client.0.smithi117.stdout:1/951: write d6/d16/d3a/d5a/d5c/fa0 [1611109,96247] 0 2024-04-20T11:09:24.950 INFO:tasks.workunit.client.1.smithi173.stdout:6/36: rename d4/c5 to d4/cb 0 2024-04-20T11:09:24.952 INFO:tasks.workunit.client.0.smithi117.stdout:1/952: read d6/d1f/d30/d32/ffb [4498243,27694] 0 2024-04-20T11:09:24.953 INFO:tasks.workunit.client.1.smithi173.stdout:1/59: stat f0 0 2024-04-20T11:09:24.953 INFO:tasks.workunit.client.1.smithi173.stdout:0/28: mkdir d6/d8 0 2024-04-20T11:09:24.957 INFO:tasks.workunit.client.1.smithi173.stdout:3/56: chown d7/d9/ff 25323 1 2024-04-20T11:09:24.957 INFO:tasks.workunit.client.1.smithi173.stdout:8/91: mkdir d4/d18 0 2024-04-20T11:09:24.957 INFO:tasks.workunit.client.1.smithi173.stdout:7/27: dread f6 [0,4194304] 0 2024-04-20T11:09:24.959 INFO:tasks.workunit.client.1.smithi173.stdout:0/29: dread - d6/f7 zero size 2024-04-20T11:09:24.962 INFO:tasks.workunit.client.1.smithi173.stdout:6/37: creat d4/fc x:0 0 0 2024-04-20T11:09:24.963 INFO:tasks.workunit.client.1.smithi173.stdout:3/57: readlink d7/d9/la 0 2024-04-20T11:09:24.964 INFO:tasks.workunit.client.1.smithi173.stdout:0/30: fsync d6/f7 0 2024-04-20T11:09:24.966 INFO:tasks.workunit.client.1.smithi173.stdout:2/34: symlink d2/la 0 2024-04-20T11:09:24.967 INFO:tasks.workunit.client.1.smithi173.stdout:2/35: truncate - no filename 2024-04-20T11:09:24.968 INFO:tasks.workunit.client.1.smithi173.stdout:1/60: creat d5/d7/da/f14 x:0 0 0 2024-04-20T11:09:24.970 INFO:tasks.workunit.client.1.smithi173.stdout:2/36: readlink d2/d5/l6 0 2024-04-20T11:09:24.970 INFO:tasks.workunit.client.1.smithi173.stdout:2/37: dwrite - no filename 2024-04-20T11:09:24.970 INFO:tasks.workunit.client.1.smithi173.stdout:9/76: dwrite d5/d9/fa [0,4194304] 0 2024-04-20T11:09:24.971 INFO:tasks.workunit.client.1.smithi173.stdout:5/32: dwrite f7 [0,4194304] 0 2024-04-20T11:09:24.972 INFO:tasks.workunit.client.1.smithi173.stdout:2/38: stat d2/d5/l9 0 2024-04-20T11:09:24.972 INFO:tasks.workunit.client.1.smithi173.stdout:2/39: truncate - no filename 2024-04-20T11:09:24.977 INFO:tasks.workunit.client.1.smithi173.stdout:1/61: fdatasync d5/d7/fd 0 2024-04-20T11:09:24.977 INFO:tasks.workunit.client.1.smithi173.stdout:6/38: chown d4/l7 2 1 2024-04-20T11:09:24.978 INFO:tasks.workunit.client.1.smithi173.stdout:5/33: write f9 [907148,115884] 0 2024-04-20T11:09:24.979 INFO:tasks.workunit.client.1.smithi173.stdout:2/40: unlink d2/l7 0 2024-04-20T11:09:24.980 INFO:tasks.workunit.client.1.smithi173.stdout:0/31: mknod d6/d8/c9 0 2024-04-20T11:09:24.985 INFO:tasks.workunit.client.1.smithi173.stdout:1/62: write d5/d7/fd [1544092,113214] 0 2024-04-20T11:09:24.986 INFO:tasks.workunit.client.1.smithi173.stdout:6/39: creat d4/fd x:0 0 0 2024-04-20T11:09:24.989 INFO:tasks.workunit.client.1.smithi173.stdout:2/41: mkdir d2/d5/db 0 2024-04-20T11:09:24.992 INFO:tasks.workunit.client.1.smithi173.stdout:1/63: fdatasync d5/fb 0 2024-04-20T11:09:24.993 INFO:tasks.workunit.client.1.smithi173.stdout:0/32: creat d6/fa x:0 0 0 2024-04-20T11:09:25.001 INFO:tasks.workunit.client.1.smithi173.stdout:9/77: truncate d5/df/f10 374376 0 2024-04-20T11:09:25.004 INFO:tasks.workunit.client.1.smithi173.stdout:5/34: rename l5 to da/lb 0 2024-04-20T11:09:25.006 INFO:tasks.workunit.client.1.smithi173.stdout:6/40: rename d4/fc to d4/fe 0 2024-04-20T11:09:25.013 INFO:tasks.workunit.client.1.smithi173.stdout:2/42: rename d2/d5 to d2/d5/db/dc 22 2024-04-20T11:09:25.019 INFO:tasks.workunit.client.1.smithi173.stdout:4/48: dwrite f4 [0,4194304] 0 2024-04-20T11:09:25.019 INFO:tasks.workunit.client.1.smithi173.stdout:8/92: dwrite d4/f8 [0,4194304] 0 2024-04-20T11:09:25.019 INFO:tasks.workunit.client.1.smithi173.stdout:2/43: readlink d2/l4 0 2024-04-20T11:09:25.019 INFO:tasks.workunit.client.0.smithi117.stdout:4/979: sync 2024-04-20T11:09:25.020 INFO:tasks.workunit.client.1.smithi173.stdout:3/58: dwrite d7/d9/fb [0,4194304] 0 2024-04-20T11:09:25.022 INFO:tasks.workunit.client.1.smithi173.stdout:7/28: dwrite f6 [0,4194304] 0 2024-04-20T11:09:25.026 INFO:tasks.workunit.client.1.smithi173.stdout:6/41: truncate d4/fe 453899 0 2024-04-20T11:09:25.033 INFO:tasks.workunit.client.1.smithi173.stdout:4/49: chown f8 319887 1 2024-04-20T11:09:25.033 INFO:tasks.workunit.client.1.smithi173.stdout:2/44: chown d2/d5/l9 6 1 2024-04-20T11:09:25.035 INFO:tasks.workunit.client.1.smithi173.stdout:8/93: creat d4/f19 x:0 0 0 2024-04-20T11:09:25.038 INFO:tasks.workunit.client.1.smithi173.stdout:1/64: link d5/d7/fd d5/d7/d12/f15 0 2024-04-20T11:09:25.040 INFO:tasks.workunit.client.1.smithi173.stdout:2/45: readlink d2/d5/l6 0 2024-04-20T11:09:25.040 INFO:tasks.workunit.client.1.smithi173.stdout:2/46: fdatasync - no filename 2024-04-20T11:09:25.040 INFO:tasks.workunit.client.1.smithi173.stdout:0/33: getdents d6 0 2024-04-20T11:09:25.042 INFO:tasks.workunit.client.1.smithi173.stdout:9/78: getdents d5/df 0 2024-04-20T11:09:25.042 INFO:tasks.workunit.client.1.smithi173.stdout:8/94: chown d4/f6 72042 1 2024-04-20T11:09:25.047 INFO:tasks.workunit.client.1.smithi173.stdout:4/50: creat da/f10 x:0 0 0 2024-04-20T11:09:25.048 INFO:tasks.workunit.client.1.smithi173.stdout:8/95: dread - d4/f19 zero size 2024-04-20T11:09:25.052 INFO:tasks.workunit.client.1.smithi173.stdout:0/34: mknod d6/cb 0 2024-04-20T11:09:25.052 INFO:tasks.workunit.client.1.smithi173.stdout:5/35: dwrite f8 [0,4194304] 0 2024-04-20T11:09:25.053 INFO:tasks.workunit.client.1.smithi173.stdout:2/47: creat d2/d5/fd x:0 0 0 2024-04-20T11:09:25.056 INFO:tasks.workunit.client.1.smithi173.stdout:9/79: creat d5/d9/f1b x:0 0 0 2024-04-20T11:09:25.059 INFO:tasks.workunit.client.1.smithi173.stdout:4/51: mknod da/c11 0 2024-04-20T11:09:25.059 INFO:tasks.workunit.client.1.smithi173.stdout:3/59: dread d7/f8 [0,4194304] 0 2024-04-20T11:09:25.059 INFO:tasks.workunit.client.1.smithi173.stdout:4/52: readlink l7 0 2024-04-20T11:09:25.063 INFO:tasks.workunit.client.1.smithi173.stdout:4/53: dread f1 [4194304,4194304] 0 2024-04-20T11:09:25.064 INFO:tasks.workunit.client.0.smithi117.stdout:8/925: sync 2024-04-20T11:09:25.064 INFO:tasks.workunit.client.0.smithi117.stdout:3/992: sync 2024-04-20T11:09:25.066 INFO:tasks.workunit.client.1.smithi173.stdout:4/54: write da/dc/ff [738682,125199] 0 2024-04-20T11:09:25.066 INFO:tasks.workunit.client.1.smithi173.stdout:7/29: mkdir d9/da 0 2024-04-20T11:09:25.069 INFO:tasks.workunit.client.1.smithi173.stdout:6/42: getdents d4 0 2024-04-20T11:09:25.072 INFO:tasks.workunit.client.0.smithi117.stdout:1/953: creat d6/d16/f139 x:0 0 0 2024-04-20T11:09:25.073 INFO:tasks.workunit.client.1.smithi173.stdout:6/43: write d4/fd [616968,86935] 0 2024-04-20T11:09:25.073 INFO:tasks.workunit.client.1.smithi173.stdout:1/65: mknod d5/d7/d12/c16 0 2024-04-20T11:09:25.075 INFO:tasks.workunit.client.0.smithi117.stdout:8/926: creat d0/dc/d2e/f133 x:0 0 0 2024-04-20T11:09:25.076 INFO:tasks.workunit.client.1.smithi173.stdout:5/36: creat da/fc x:0 0 0 2024-04-20T11:09:25.077 INFO:tasks.workunit.client.0.smithi117.stdout:8/927: readlink d0/d8/d11/d24/d27/d47/le1 0 2024-04-20T11:09:25.077 INFO:tasks.workunit.client.1.smithi173.stdout:5/37: fdatasync f6 0 2024-04-20T11:09:25.077 INFO:tasks.workunit.client.1.smithi173.stdout:9/80: dwrite d5/d9/f1b [0,4194304] 0 2024-04-20T11:09:25.078 INFO:tasks.workunit.client.0.smithi117.stdout:1/954: truncate d6/d1f/d30/d49/d124/db9/f118 1103771 0 2024-04-20T11:09:25.081 INFO:tasks.workunit.client.1.smithi173.stdout:5/38: fsync f7 0 2024-04-20T11:09:25.082 INFO:tasks.workunit.client.1.smithi173.stdout:3/60: creat d7/f11 x:0 0 0 2024-04-20T11:09:25.083 INFO:tasks.workunit.client.1.smithi173.stdout:7/30: readlink l7 0 2024-04-20T11:09:25.083 INFO:tasks.workunit.client.1.smithi173.stdout:5/39: write da/fc [988484,14245] 0 2024-04-20T11:09:25.083 INFO:tasks.workunit.client.1.smithi173.stdout:7/31: chown d9/da 7316094 1 2024-04-20T11:09:25.086 INFO:tasks.workunit.client.0.smithi117.stdout:1/955: write d6/d16/d3a/d5a/d5c/d61/d98/f11f [2891850,104295] 0 2024-04-20T11:09:25.088 INFO:tasks.workunit.client.1.smithi173.stdout:6/44: rename d4/l7 to d4/lf 0 2024-04-20T11:09:25.090 INFO:tasks.workunit.client.1.smithi173.stdout:1/66: mkdir d5/d6/d17 0 2024-04-20T11:09:25.094 INFO:tasks.workunit.client.0.smithi117.stdout:1/956: creat d6/d1f/d30/d32/f13a x:0 0 0 2024-04-20T11:09:25.097 INFO:tasks.workunit.client.1.smithi173.stdout:3/61: symlink d7/d9/l12 0 2024-04-20T11:09:25.099 INFO:tasks.workunit.client.0.smithi117.stdout:1/957: mkdir d6/d16/d3a/d5a/daf/d13b 0 2024-04-20T11:09:25.100 INFO:tasks.workunit.client.0.smithi117.stdout:8/928: dwrite d0/d8/d11/d6d/d84/fc1 [0,4194304] 0 2024-04-20T11:09:25.101 INFO:tasks.workunit.client.1.smithi173.stdout:6/45: creat d4/f10 x:0 0 0 2024-04-20T11:09:25.101 INFO:tasks.workunit.client.1.smithi173.stdout:3/62: truncate d7/d9/fe 120827 0 2024-04-20T11:09:25.102 INFO:tasks.workunit.client.0.smithi117.stdout:1/958: read - d6/d16/d3a/d5a/d5c/d61/d98/deb/f133 zero size 2024-04-20T11:09:25.103 INFO:tasks.workunit.client.1.smithi173.stdout:6/46: fdatasync d4/fd 0 2024-04-20T11:09:25.104 INFO:tasks.workunit.client.1.smithi173.stdout:7/32: symlink d9/da/lb 0 2024-04-20T11:09:25.105 INFO:tasks.workunit.client.1.smithi173.stdout:3/63: chown d7/c10 219 1 2024-04-20T11:09:25.106 INFO:tasks.workunit.client.1.smithi173.stdout:7/33: write f1 [1569673,30017] 0 2024-04-20T11:09:25.107 INFO:tasks.workunit.client.1.smithi173.stdout:1/67: unlink d5/d7/fd 0 2024-04-20T11:09:25.108 INFO:tasks.workunit.client.1.smithi173.stdout:2/48: rmdir d2 39 2024-04-20T11:09:25.109 INFO:tasks.workunit.client.0.smithi117.stdout:8/929: mkdir d0/dc/d49/d134 0 2024-04-20T11:09:25.109 INFO:tasks.workunit.client.1.smithi173.stdout:6/47: truncate d4/fd 1065354 0 2024-04-20T11:09:25.110 INFO:tasks.workunit.client.1.smithi173.stdout:5/40: dwrite f6 [0,4194304] 0 2024-04-20T11:09:25.113 INFO:tasks.workunit.client.1.smithi173.stdout:1/68: truncate f3 2088151 0 2024-04-20T11:09:25.113 INFO:tasks.workunit.client.1.smithi173.stdout:7/34: truncate f1 5230343 0 2024-04-20T11:09:25.116 INFO:tasks.workunit.client.1.smithi173.stdout:9/81: rename d5/df/l15 to d5/l1c 0 2024-04-20T11:09:25.117 INFO:tasks.workunit.client.0.smithi117.stdout:4/980: write d1/dd/d1a/d43/d4e/fd0 [121126,93212] 0 2024-04-20T11:09:25.118 INFO:tasks.workunit.client.1.smithi173.stdout:8/96: write d4/f10 [647948,109026] 0 2024-04-20T11:09:25.121 INFO:tasks.workunit.client.1.smithi173.stdout:3/64: creat d7/d9/f13 x:0 0 0 2024-04-20T11:09:25.124 INFO:tasks.workunit.client.1.smithi173.stdout:5/41: mkdir da/dd 0 2024-04-20T11:09:25.126 INFO:tasks.workunit.client.1.smithi173.stdout:4/55: write f1 [5175452,92169] 0 2024-04-20T11:09:25.126 INFO:tasks.workunit.client.0.smithi117.stdout:8/930: getdents d0/dc/d49/d5b/da9/dc7 0 2024-04-20T11:09:25.130 INFO:tasks.workunit.client.1.smithi173.stdout:3/65: truncate d7/d9/fe 301819 0 2024-04-20T11:09:25.131 INFO:tasks.workunit.client.1.smithi173.stdout:9/82: creat d5/d9/f1d x:0 0 0 2024-04-20T11:09:25.132 INFO:tasks.workunit.client.1.smithi173.stdout:4/56: fsync da/f10 0 2024-04-20T11:09:25.140 INFO:tasks.workunit.client.1.smithi173.stdout:2/49: getdents d2/d5/db 0 2024-04-20T11:09:25.140 INFO:tasks.workunit.client.0.smithi117.stdout:8/931: dwrite d0/f1 [0,4194304] 0 2024-04-20T11:09:25.141 INFO:tasks.workunit.client.1.smithi173.stdout:4/57: write f1 [5401457,9974] 0 2024-04-20T11:09:25.143 INFO:tasks.workunit.client.0.smithi117.stdout:1/959: truncate d6/d1f/dd6/f10f 955029 0 2024-04-20T11:09:25.144 INFO:tasks.workunit.client.0.smithi117.stdout:8/932: chown d0/d8/d11/d24/d76/f10c 6238 1 2024-04-20T11:09:25.147 INFO:tasks.workunit.client.0.smithi117.stdout:8/933: mknod d0/dc/d49/c135 0 2024-04-20T11:09:25.149 INFO:tasks.workunit.client.1.smithi173.stdout:3/66: creat d7/d9/f14 x:0 0 0 2024-04-20T11:09:25.155 INFO:tasks.workunit.client.1.smithi173.stdout:2/50: write d2/d5/fd [283641,18314] 0 2024-04-20T11:09:25.155 INFO:tasks.workunit.client.1.smithi173.stdout:0/35: dwrite f4 [0,4194304] 0 2024-04-20T11:09:25.155 INFO:tasks.workunit.client.1.smithi173.stdout:1/69: getdents d5 0 2024-04-20T11:09:25.157 INFO:tasks.workunit.client.1.smithi173.stdout:3/67: readlink d7/d9/lc 0 2024-04-20T11:09:25.157 INFO:tasks.workunit.client.1.smithi173.stdout:7/35: dwrite f6 [0,4194304] 0 2024-04-20T11:09:25.159 INFO:tasks.workunit.client.0.smithi117.stdout:4/981: dwrite d1/d45/fdb [0,4194304] 0 2024-04-20T11:09:25.160 INFO:tasks.workunit.client.1.smithi173.stdout:4/58: rename da/f10 to da/f12 0 2024-04-20T11:09:25.163 INFO:tasks.workunit.client.1.smithi173.stdout:6/48: dwrite d4/fe [0,4194304] 0 2024-04-20T11:09:25.165 INFO:tasks.workunit.client.0.smithi117.stdout:8/934: rmdir d0/d9/d3a/de3/df1 39 2024-04-20T11:09:25.168 INFO:tasks.workunit.client.0.smithi117.stdout:8/935: readlink d0/d9/dbb/l10e 0 2024-04-20T11:09:25.171 INFO:tasks.workunit.client.1.smithi173.stdout:5/42: getdents da 0 2024-04-20T11:09:25.175 INFO:tasks.workunit.client.1.smithi173.stdout:0/36: symlink d6/lc 0 2024-04-20T11:09:25.177 INFO:tasks.workunit.client.1.smithi173.stdout:3/68: creat d7/f15 x:0 0 0 2024-04-20T11:09:25.179 INFO:tasks.workunit.client.1.smithi173.stdout:6/49: creat d4/f11 x:0 0 0 2024-04-20T11:09:25.180 INFO:tasks.workunit.client.0.smithi117.stdout:8/936: dread d0/d8/d39/fc9 [0,4194304] 0 2024-04-20T11:09:25.181 INFO:tasks.workunit.client.1.smithi173.stdout:4/59: mkdir da/dc/d13 0 2024-04-20T11:09:25.183 INFO:tasks.workunit.client.1.smithi173.stdout:3/69: write d7/fd [998922,10689] 0 2024-04-20T11:09:25.185 INFO:tasks.workunit.client.1.smithi173.stdout:4/60: dread - da/dc/fe zero size 2024-04-20T11:09:25.185 INFO:tasks.workunit.client.1.smithi173.stdout:2/51: mkdir d2/de 0 2024-04-20T11:09:25.188 INFO:tasks.workunit.client.0.smithi117.stdout:8/937: rename d0/d8/d39/d40/d78/d5c/fd1 to d0/d8/d39/d40/de5/d12a/f136 0 2024-04-20T11:09:25.193 INFO:tasks.workunit.client.0.smithi117.stdout:8/938: write d0/d8/d39/d40/d78/f100 [4970679,30512] 0 2024-04-20T11:09:25.193 INFO:tasks.workunit.client.1.smithi173.stdout:3/70: symlink d7/l16 0 2024-04-20T11:09:25.193 INFO:tasks.workunit.client.1.smithi173.stdout:4/61: chown c3 1054196975 1 2024-04-20T11:09:25.193 INFO:tasks.workunit.client.0.smithi117.stdout:1/960: dwrite d6/d16/d3a/d5a/d5c/d61/d98/fed [0,4194304] 0 2024-04-20T11:09:25.196 INFO:tasks.workunit.client.0.smithi117.stdout:1/961: read - d6/d1f/dd6/f114 zero size 2024-04-20T11:09:25.196 INFO:tasks.workunit.client.1.smithi173.stdout:0/37: write f2 [2321625,4845] 0 2024-04-20T11:09:25.199 INFO:tasks.workunit.client.1.smithi173.stdout:7/36: dwrite f1 [0,4194304] 0 2024-04-20T11:09:25.200 INFO:tasks.workunit.client.0.smithi117.stdout:8/939: dread d0/dc/d49/f101 [0,4194304] 0 2024-04-20T11:09:25.200 INFO:tasks.workunit.client.0.smithi117.stdout:4/982: write d1/f19 [5197181,20558] 0 2024-04-20T11:09:25.204 INFO:tasks.workunit.client.1.smithi173.stdout:4/62: chown da/dc/ff 22881663 1 2024-04-20T11:09:25.205 INFO:tasks.workunit.client.1.smithi173.stdout:7/37: chown f1 204993 1 2024-04-20T11:09:25.206 INFO:tasks.workunit.client.1.smithi173.stdout:2/52: dread d2/d5/fd [0,4194304] 0 2024-04-20T11:09:25.206 INFO:tasks.workunit.client.1.smithi173.stdout:3/71: mkdir d7/d17 0 2024-04-20T11:09:25.207 INFO:tasks.workunit.client.1.smithi173.stdout:0/38: write d6/fa [175617,67139] 0 2024-04-20T11:09:25.212 INFO:tasks.workunit.client.1.smithi173.stdout:9/83: dwrite d5/f6 [0,4194304] 0 2024-04-20T11:09:25.213 INFO:tasks.workunit.client.1.smithi173.stdout:4/63: truncate f8 1228837 0 2024-04-20T11:09:25.214 INFO:tasks.workunit.client.1.smithi173.stdout:7/38: rename d9 to d9/dc 22 2024-04-20T11:09:25.216 INFO:tasks.workunit.client.1.smithi173.stdout:0/39: fsync f2 0 2024-04-20T11:09:25.217 INFO:tasks.workunit.client.1.smithi173.stdout:9/84: chown d5/df 1 1 2024-04-20T11:09:25.220 INFO:tasks.workunit.client.1.smithi173.stdout:9/85: stat d5/d9/f1a 0 2024-04-20T11:09:25.222 INFO:tasks.workunit.client.1.smithi173.stdout:3/72: truncate d7/f11 508367 0 2024-04-20T11:09:25.224 INFO:tasks.workunit.client.1.smithi173.stdout:4/64: rename da/cb to da/dc/d13/c14 0 2024-04-20T11:09:25.226 INFO:tasks.workunit.client.1.smithi173.stdout:2/53: link d2/d5/l9 d2/de/lf 0 2024-04-20T11:09:25.226 INFO:tasks.workunit.client.0.smithi117.stdout:1/962: dwrite d6/d1f/d30/d49/d124/db9/f118 [0,4194304] 0 2024-04-20T11:09:25.230 INFO:tasks.workunit.client.1.smithi173.stdout:0/40: creat d6/fd x:0 0 0 2024-04-20T11:09:25.231 INFO:tasks.workunit.client.1.smithi173.stdout:6/50: dwrite d4/f8 [0,4194304] 0 2024-04-20T11:09:25.232 INFO:tasks.workunit.client.1.smithi173.stdout:4/65: mkdir da/dc/d15 0 2024-04-20T11:09:25.232 INFO:tasks.workunit.client.1.smithi173.stdout:3/73: chown d7/f8 10842452 1 2024-04-20T11:09:25.233 INFO:tasks.workunit.client.0.smithi117.stdout:8/940: dwrite d0/d8/d11/d24/d2b/f116 [0,4194304] 0 2024-04-20T11:09:25.234 INFO:tasks.workunit.client.0.smithi117.stdout:4/983: dwrite d1/dd/d1a/f1d [0,4194304] 0 2024-04-20T11:09:25.236 INFO:tasks.workunit.client.0.smithi117.stdout:8/941: read d0/d8/d39/fc9 [223452,51896] 0 2024-04-20T11:09:25.238 INFO:tasks.workunit.client.1.smithi173.stdout:0/41: creat d6/d8/fe x:0 0 0 2024-04-20T11:09:25.239 INFO:tasks.workunit.client.1.smithi173.stdout:2/54: creat d2/d5/f10 x:0 0 0 2024-04-20T11:09:25.240 INFO:tasks.workunit.client.1.smithi173.stdout:4/66: chown da/dc/fe 3831672 1 2024-04-20T11:09:25.240 INFO:tasks.workunit.client.1.smithi173.stdout:1/70: dread d5/d6/dc/fe [0,4194304] 0 2024-04-20T11:09:25.243 INFO:tasks.workunit.client.1.smithi173.stdout:2/55: fdatasync d2/d5/f10 0 2024-04-20T11:09:25.244 INFO:tasks.workunit.client.1.smithi173.stdout:4/67: fsync f1 0 2024-04-20T11:09:25.251 INFO:tasks.workunit.client.1.smithi173.stdout:0/42: mkdir d6/d8/df 0 2024-04-20T11:09:25.251 INFO:tasks.workunit.client.1.smithi173.stdout:6/51: rename d4/ca to d4/c12 0 2024-04-20T11:09:25.254 INFO:tasks.workunit.client.1.smithi173.stdout:4/68: mknod da/dc/d13/c16 0 2024-04-20T11:09:25.256 INFO:tasks.workunit.client.1.smithi173.stdout:0/43: mknod d6/c10 0 2024-04-20T11:09:25.257 INFO:tasks.workunit.client.1.smithi173.stdout:3/74: rmdir d7 39 2024-04-20T11:09:25.259 INFO:tasks.workunit.client.0.smithi117.stdout:8/942: dwrite d0/d9/d3a/d7c/dce/ffa [0,4194304] 0 2024-04-20T11:09:25.269 INFO:tasks.workunit.client.0.smithi117.stdout:8/943: write d0/d9/dbb/d113/f11b [425288,126044] 0 2024-04-20T11:09:25.270 INFO:tasks.workunit.client.0.smithi117.stdout:4/984: write d1/d45/d4a/d59/fd6 [775314,113288] 0 2024-04-20T11:09:25.270 INFO:tasks.workunit.client.0.smithi117.stdout:8/944: chown d0 3441 1 2024-04-20T11:09:25.271 INFO:tasks.workunit.client.1.smithi173.stdout:1/71: getdents d5/d7 0 2024-04-20T11:09:25.273 INFO:tasks.workunit.client.1.smithi173.stdout:0/44: read - d6/f7 zero size 2024-04-20T11:09:25.276 INFO:tasks.workunit.client.1.smithi173.stdout:9/86: dwrite d5/df/f11 [0,4194304] 0 2024-04-20T11:09:25.276 INFO:tasks.workunit.client.1.smithi173.stdout:7/39: dwrite f6 [0,4194304] 0 2024-04-20T11:09:25.276 INFO:tasks.workunit.client.1.smithi173.stdout:1/72: chown d5 335 1 2024-04-20T11:09:25.277 INFO:tasks.workunit.client.1.smithi173.stdout:2/56: rename d2/d5 to d2/de/d11 0 2024-04-20T11:09:25.277 INFO:tasks.workunit.client.0.smithi117.stdout:8/945: getdents d0 0 2024-04-20T11:09:25.278 INFO:tasks.workunit.client.0.smithi117.stdout:1/963: truncate d6/d16/d3a/d5a/d5c/f106 1690058 0 2024-04-20T11:09:25.281 INFO:tasks.workunit.client.0.smithi117.stdout:1/964: symlink d6/d1f/d30/dfd/l13c 0 2024-04-20T11:09:25.284 INFO:tasks.workunit.client.0.smithi117.stdout:1/965: unlink d6/d16/cda 0 2024-04-20T11:09:25.285 INFO:tasks.workunit.client.1.smithi173.stdout:4/69: symlink da/dc/l17 0 2024-04-20T11:09:25.286 INFO:tasks.workunit.client.0.smithi117.stdout:8/946: link d0/d8/d11/d24/d27/d47/f7d d0/d8/d11/d24/d76/f137 0 2024-04-20T11:09:25.288 INFO:tasks.workunit.client.0.smithi117.stdout:1/966: readlink d6/d1f/d30/d49/d124/l3f 0 2024-04-20T11:09:25.291 INFO:tasks.workunit.client.1.smithi173.stdout:1/73: chown d5/d7/c10 1 1 2024-04-20T11:09:25.295 INFO:tasks.workunit.client.1.smithi173.stdout:0/45: unlink d6/cb 0 2024-04-20T11:09:25.296 INFO:tasks.workunit.client.0.smithi117.stdout:3/993: sync 2024-04-20T11:09:25.298 INFO:tasks.workunit.client.1.smithi173.stdout:3/75: rename d7/d9/fb to d7/f18 0 2024-04-20T11:09:25.300 INFO:tasks.workunit.client.0.smithi117.stdout:8/947: symlink d0/dc/d49/d134/l138 0 2024-04-20T11:09:25.300 INFO:tasks.workunit.client.0.smithi117.stdout:3/994: chown d0/d5/d73/ded/c6f 11445 1 2024-04-20T11:09:25.301 INFO:tasks.workunit.client.1.smithi173.stdout:0/46: fdatasync d6/f7 0 2024-04-20T11:09:25.302 INFO:tasks.workunit.client.1.smithi173.stdout:6/52: dwrite d4/f11 [0,4194304] 0 2024-04-20T11:09:25.302 INFO:tasks.workunit.client.1.smithi173.stdout:2/57: symlink d2/de/d11/db/l12 0 2024-04-20T11:09:25.305 INFO:tasks.workunit.client.1.smithi173.stdout:8/97: sync 2024-04-20T11:09:25.305 INFO:tasks.workunit.client.0.smithi117.stdout:8/948: mknod d0/d8/d11/d24/d2b/c139 0 2024-04-20T11:09:25.307 INFO:tasks.workunit.client.1.smithi173.stdout:3/76: write d7/f18 [285736,16401] 0 2024-04-20T11:09:25.309 INFO:tasks.workunit.client.0.smithi117.stdout:8/949: write d0/d8/d11/d6d/d84/fc1 [2902953,7787] 0 2024-04-20T11:09:25.316 INFO:tasks.workunit.client.1.smithi173.stdout:2/58: write d2/de/d11/fd [67050,114470] 0 2024-04-20T11:09:25.318 INFO:tasks.workunit.client.1.smithi173.stdout:7/40: dread f6 [0,4194304] 0 2024-04-20T11:09:25.318 INFO:tasks.workunit.client.0.smithi117.stdout:4/985: dread d1/dd/d1a/d2b/d86/fb4 [0,4194304] 0 2024-04-20T11:09:25.319 INFO:tasks.workunit.client.1.smithi173.stdout:8/98: unlink d4/fa 0 2024-04-20T11:09:25.320 INFO:tasks.workunit.client.1.smithi173.stdout:9/87: dread d5/f6 [0,4194304] 0 2024-04-20T11:09:25.321 INFO:tasks.workunit.client.1.smithi173.stdout:4/70: getdents da/dc/d13 0 2024-04-20T11:09:25.321 INFO:tasks.workunit.client.1.smithi173.stdout:7/41: fsync f6 0 2024-04-20T11:09:25.321 INFO:tasks.workunit.client.1.smithi173.stdout:3/77: write d7/d9/fe [1254929,69343] 0 2024-04-20T11:09:25.323 INFO:tasks.workunit.client.1.smithi173.stdout:0/47: creat d6/d8/df/f11 x:0 0 0 2024-04-20T11:09:25.323 INFO:tasks.workunit.client.1.smithi173.stdout:6/53: rename d4/f8 to d4/f13 0 2024-04-20T11:09:25.324 INFO:tasks.workunit.client.1.smithi173.stdout:2/59: truncate d2/de/d11/fd 464064 0 2024-04-20T11:09:25.325 INFO:tasks.workunit.client.1.smithi173.stdout:0/48: write f2 [2697901,24923] 0 2024-04-20T11:09:25.326 INFO:tasks.workunit.client.1.smithi173.stdout:2/60: chown d2/de 2010046331 1 2024-04-20T11:09:25.326 INFO:tasks.workunit.client.0.smithi117.stdout:4/986: fsync d1/d9/f96 0 2024-04-20T11:09:25.327 INFO:tasks.workunit.client.1.smithi173.stdout:3/78: truncate d7/fd 1581492 0 2024-04-20T11:09:25.328 INFO:tasks.workunit.client.1.smithi173.stdout:0/49: chown d6/d8/df/f11 5191 1 2024-04-20T11:09:25.328 INFO:tasks.workunit.client.1.smithi173.stdout:9/88: fsync d5/df/f11 0 2024-04-20T11:09:25.330 INFO:tasks.workunit.client.1.smithi173.stdout:9/89: readlink d5/l7 0 2024-04-20T11:09:25.332 INFO:tasks.workunit.client.1.smithi173.stdout:9/90: write d5/df/f11 [5012207,121775] 0 2024-04-20T11:09:25.333 INFO:tasks.workunit.client.1.smithi173.stdout:2/61: read d2/de/d11/fd [310339,31470] 0 2024-04-20T11:09:25.334 INFO:tasks.workunit.client.0.smithi117.stdout:1/967: dwrite d6/f10 [0,4194304] 0 2024-04-20T11:09:25.337 INFO:tasks.workunit.client.1.smithi173.stdout:6/54: mkdir d4/d14 0 2024-04-20T11:09:25.340 INFO:tasks.workunit.client.1.smithi173.stdout:2/62: write d2/de/d11/f10 [1000522,54069] 0 2024-04-20T11:09:25.341 INFO:tasks.workunit.client.0.smithi117.stdout:1/968: truncate d6/d16/fad 4557346 0 2024-04-20T11:09:25.341 INFO:tasks.workunit.client.1.smithi173.stdout:8/99: creat d4/d18/f1a x:0 0 0 2024-04-20T11:09:25.346 INFO:tasks.workunit.client.1.smithi173.stdout:4/71: creat da/dc/d15/f18 x:0 0 0 2024-04-20T11:09:25.347 INFO:tasks.workunit.client.1.smithi173.stdout:7/42: rename l5 to d9/ld 0 2024-04-20T11:09:25.348 INFO:tasks.workunit.client.1.smithi173.stdout:8/100: truncate d4/fc 1019013 0 2024-04-20T11:09:25.350 INFO:tasks.workunit.client.1.smithi173.stdout:6/55: symlink d4/l15 0 2024-04-20T11:09:25.351 INFO:tasks.workunit.client.0.smithi117.stdout:3/995: dread d0/d5/d73/ded/d66/d63/dad/f108 [0,4194304] 0 2024-04-20T11:09:25.351 INFO:tasks.workunit.client.1.smithi173.stdout:9/91: mknod d5/c1e 0 2024-04-20T11:09:25.353 INFO:tasks.workunit.client.1.smithi173.stdout:8/101: read d4/ff [482787,29515] 0 2024-04-20T11:09:25.354 INFO:tasks.workunit.client.1.smithi173.stdout:6/56: creat d4/f16 x:0 0 0 2024-04-20T11:09:25.356 INFO:tasks.workunit.client.1.smithi173.stdout:3/79: dwrite f3 [0,4194304] 0 2024-04-20T11:09:25.356 INFO:tasks.workunit.client.0.smithi117.stdout:3/996: symlink d0/d5/d21/d75/d89/dd1/l149 0 2024-04-20T11:09:25.356 INFO:tasks.workunit.client.1.smithi173.stdout:0/50: getdents d6/d8 0 2024-04-20T11:09:25.357 INFO:tasks.workunit.client.1.smithi173.stdout:9/92: write d5/d9/f1b [5011187,39580] 0 2024-04-20T11:09:25.359 INFO:tasks.workunit.client.1.smithi173.stdout:0/51: write f2 [472075,81752] 0 2024-04-20T11:09:25.361 INFO:tasks.workunit.client.1.smithi173.stdout:4/72: dread da/dc/ff [0,4194304] 0 2024-04-20T11:09:25.361 INFO:tasks.workunit.client.1.smithi173.stdout:2/63: rename d2/de/d11/db to d2/d13 0 2024-04-20T11:09:25.362 INFO:tasks.workunit.client.1.smithi173.stdout:0/52: fdatasync d6/d8/df/f11 0 2024-04-20T11:09:25.362 INFO:tasks.workunit.client.1.smithi173.stdout:0/53: readlink l5 0 2024-04-20T11:09:25.362 INFO:tasks.workunit.client.1.smithi173.stdout:2/64: fsync d2/de/d11/f10 0 2024-04-20T11:09:25.363 INFO:tasks.workunit.client.1.smithi173.stdout:9/93: rmdir d5/df 39 2024-04-20T11:09:25.365 INFO:tasks.workunit.client.1.smithi173.stdout:9/94: truncate d5/d9/f1d 768124 0 2024-04-20T11:09:25.366 INFO:tasks.workunit.client.0.smithi117.stdout:1/969: dwrite d6/d16/d3a/d5a/d5c/da6/f117 [0,4194304] 0 2024-04-20T11:09:25.366 INFO:tasks.workunit.client.1.smithi173.stdout:6/57: symlink d4/d14/l17 0 2024-04-20T11:09:25.366 INFO:tasks.workunit.client.1.smithi173.stdout:9/95: dread - d5/d9/f1a zero size 2024-04-20T11:09:25.368 INFO:tasks.workunit.client.1.smithi173.stdout:9/96: stat d5/c16 0 2024-04-20T11:09:25.370 INFO:tasks.workunit.client.0.smithi117.stdout:4/987: dread d1/dd/d1a/d2b/f8f [0,4194304] 0 2024-04-20T11:09:25.372 INFO:tasks.workunit.client.0.smithi117.stdout:3/997: link d0/d5/d73/ded/db3/fb6 d0/d5/d21/d77/df3/d103/f14a 0 2024-04-20T11:09:25.380 INFO:tasks.workunit.client.1.smithi173.stdout:0/54: dwrite d6/f7 [0,4194304] 0 2024-04-20T11:09:25.380 INFO:tasks.workunit.client.1.smithi173.stdout:3/80: creat d7/d17/f19 x:0 0 0 2024-04-20T11:09:25.383 INFO:tasks.workunit.client.1.smithi173.stdout:3/81: write d7/d17/f19 [540292,130666] 0 2024-04-20T11:09:25.384 INFO:tasks.workunit.client.0.smithi117.stdout:4/988: mkdir d1/d9/d1f/d90/d107/d12b/d13e 0 2024-04-20T11:09:25.385 INFO:tasks.workunit.client.1.smithi173.stdout:6/58: mkdir d4/d18 0 2024-04-20T11:09:25.388 INFO:tasks.workunit.client.1.smithi173.stdout:7/43: rename l8 to d9/da/le 0 2024-04-20T11:09:25.389 INFO:tasks.workunit.client.1.smithi173.stdout:7/44: fdatasync f6 0 2024-04-20T11:09:25.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:25 smithi117 ceph-mon[15659]: pgmap v22: 65 pgs: 65 active+clean; 2.0 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 71 MiB/s rd, 131 MiB/s wr, 347 op/s 2024-04-20T11:09:25.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:25 smithi117 ceph-mon[15659]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:25.392 INFO:tasks.workunit.client.1.smithi173.stdout:0/55: creat d6/d8/f12 x:0 0 0 2024-04-20T11:09:25.395 INFO:tasks.workunit.client.1.smithi173.stdout:3/82: truncate f4 437116 0 2024-04-20T11:09:25.400 INFO:tasks.workunit.client.1.smithi173.stdout:8/102: rename d4/fc to d4/d18/f1b 0 2024-04-20T11:09:25.402 INFO:tasks.workunit.client.0.smithi117.stdout:3/998: getdents d0/d5/d21/d88/d36/d111 0 2024-04-20T11:09:25.402 INFO:tasks.workunit.client.1.smithi173.stdout:7/45: creat d9/da/ff x:0 0 0 2024-04-20T11:09:25.403 INFO:tasks.workunit.client.1.smithi173.stdout:0/56: creat d6/f13 x:0 0 0 2024-04-20T11:09:25.404 INFO:tasks.workunit.client.1.smithi173.stdout:9/97: creat d5/df/d18/f1f x:0 0 0 2024-04-20T11:09:25.405 INFO:tasks.workunit.client.1.smithi173.stdout:3/83: unlink d7/fd 0 2024-04-20T11:09:25.406 INFO:tasks.workunit.client.1.smithi173.stdout:7/46: write d9/da/ff [237585,38157] 0 2024-04-20T11:09:25.406 INFO:tasks.workunit.client.1.smithi173.stdout:6/59: dwrite d4/fe [0,4194304] 0 2024-04-20T11:09:25.407 INFO:tasks.workunit.client.1.smithi173.stdout:6/60: dread - d4/f10 zero size 2024-04-20T11:09:25.408 INFO:tasks.workunit.client.1.smithi173.stdout:3/84: readlink d7/d9/la 0 2024-04-20T11:09:25.409 INFO:tasks.workunit.client.1.smithi173.stdout:6/61: dread - d4/f16 zero size 2024-04-20T11:09:25.410 INFO:tasks.workunit.client.1.smithi173.stdout:3/85: chown f3 142468 1 2024-04-20T11:09:25.410 INFO:tasks.workunit.client.0.smithi117.stdout:3/999: symlink d0/d5/de/d4e/dec/d143/l14b 0 2024-04-20T11:09:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:25 smithi173 ceph-mon[23800]: pgmap v22: 65 pgs: 65 active+clean; 2.0 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 71 MiB/s rd, 131 MiB/s wr, 347 op/s 2024-04-20T11:09:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:25 smithi173 ceph-mon[23800]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:25.411 INFO:tasks.workunit.client.1.smithi173.stdout:6/62: chown d4/d14 209 1 2024-04-20T11:09:25.411 INFO:tasks.workunit.client.1.smithi173.stdout:8/103: symlink d4/l1c 0 2024-04-20T11:09:25.414 INFO:tasks.workunit.client.1.smithi173.stdout:9/98: mkdir d5/d9/d20 0 2024-04-20T11:09:25.414 INFO:tasks.workunit.client.1.smithi173.stdout:8/104: chown d4/fd 25574 1 2024-04-20T11:09:25.415 INFO:tasks.workunit.client.1.smithi173.stdout:2/65: write d2/de/d11/fd [1193584,69707] 0 2024-04-20T11:09:25.416 INFO:tasks.workunit.client.1.smithi173.stdout:7/47: creat d9/da/f10 x:0 0 0 2024-04-20T11:09:25.416 INFO:tasks.workunit.client.1.smithi173.stdout:3/86: symlink d7/l1a 0 2024-04-20T11:09:25.418 INFO:tasks.workunit.client.1.smithi173.stdout:8/105: read d4/f8 [3341685,98069] 0 2024-04-20T11:09:25.419 INFO:tasks.workunit.client.1.smithi173.stdout:9/99: read d5/d9/f1d [759255,104288] 0 2024-04-20T11:09:25.422 INFO:tasks.workunit.client.1.smithi173.stdout:6/63: creat d4/d18/f19 x:0 0 0 2024-04-20T11:09:25.422 INFO:tasks.workunit.client.1.smithi173.stdout:9/100: dread - d5/d9/f13 zero size 2024-04-20T11:09:25.424 INFO:tasks.workunit.client.0.smithi117.stdout:4/989: dwrite d1/d45/f12d [0,4194304] 0 2024-04-20T11:09:25.427 INFO:tasks.workunit.client.1.smithi173.stdout:7/48: mkdir d9/d11 0 2024-04-20T11:09:25.430 INFO:tasks.workunit.client.1.smithi173.stdout:0/57: dwrite d6/d8/fe [0,4194304] 0 2024-04-20T11:09:25.430 INFO:tasks.workunit.client.1.smithi173.stdout:3/87: mknod d7/c1b 0 2024-04-20T11:09:25.431 INFO:tasks.workunit.client.1.smithi173.stdout:8/106: mknod d4/c1d 0 2024-04-20T11:09:25.431 INFO:tasks.workunit.client.1.smithi173.stdout:0/58: fsync f4 0 2024-04-20T11:09:25.433 INFO:tasks.workunit.client.0.smithi117.stdout:4/990: mkdir d1/dd/d1a/d43/d108/d13f 0 2024-04-20T11:09:25.438 INFO:tasks.workunit.client.1.smithi173.stdout:2/66: mkdir d2/d14 0 2024-04-20T11:09:25.441 INFO:tasks.workunit.client.1.smithi173.stdout:6/64: write d4/f13 [4577854,12926] 0 2024-04-20T11:09:25.442 INFO:tasks.workunit.client.1.smithi173.stdout:3/88: mknod d7/c1c 0 2024-04-20T11:09:25.444 INFO:tasks.workunit.client.1.smithi173.stdout:0/59: symlink d6/d8/df/l14 0 2024-04-20T11:09:25.448 INFO:tasks.workunit.client.1.smithi173.stdout:2/67: rmdir d2/de 39 2024-04-20T11:09:25.451 INFO:tasks.workunit.client.1.smithi173.stdout:3/89: mknod d7/d9/c1d 0 2024-04-20T11:09:25.451 INFO:tasks.workunit.client.1.smithi173.stdout:0/60: mkdir d6/d8/df/d15 0 2024-04-20T11:09:25.452 INFO:tasks.workunit.client.1.smithi173.stdout:7/49: dread f1 [0,4194304] 0 2024-04-20T11:09:25.454 INFO:tasks.workunit.client.1.smithi173.stdout:3/90: write f3 [4561426,124206] 0 2024-04-20T11:09:25.455 INFO:tasks.workunit.client.1.smithi173.stdout:9/101: dwrite d5/d9/f1b [0,4194304] 0 2024-04-20T11:09:25.456 INFO:tasks.workunit.client.1.smithi173.stdout:0/61: creat d6/d8/df/f16 x:0 0 0 2024-04-20T11:09:25.458 INFO:tasks.workunit.client.1.smithi173.stdout:3/91: symlink d7/d9/l1e 0 2024-04-20T11:09:25.458 INFO:tasks.workunit.client.1.smithi173.stdout:0/62: fdatasync d6/f13 0 2024-04-20T11:09:25.459 INFO:tasks.workunit.client.1.smithi173.stdout:3/92: rename d7/d17 to d7/d17/d1f 22 2024-04-20T11:09:25.461 INFO:tasks.workunit.client.0.smithi117.stdout:4/991: dread d1/fa7 [0,4194304] 0 2024-04-20T11:09:25.465 INFO:tasks.workunit.client.0.smithi117.stdout:4/992: fdatasync d1/dd/d1a/d43/fed 0 2024-04-20T11:09:25.465 INFO:tasks.workunit.client.1.smithi173.stdout:5/43: sync 2024-04-20T11:09:25.468 INFO:tasks.workunit.client.0.smithi117.stdout:4/993: creat d1/dd/f140 x:0 0 0 2024-04-20T11:09:25.473 INFO:tasks.workunit.client.0.smithi117.stdout:4/994: rename d1/d9/f12 to d1/d9/d79/f141 0 2024-04-20T11:09:25.474 INFO:tasks.workunit.client.1.smithi173.stdout:8/107: truncate d4/f17 2371714 0 2024-04-20T11:09:25.476 INFO:tasks.workunit.client.1.smithi173.stdout:6/65: dread d4/f13 [0,4194304] 0 2024-04-20T11:09:25.476 INFO:tasks.workunit.client.1.smithi173.stdout:9/102: fdatasync d5/d9/f1b 0 2024-04-20T11:09:25.477 INFO:tasks.workunit.client.0.smithi117.stdout:4/995: truncate d1/d9/f66 2046613 0 2024-04-20T11:09:25.482 INFO:tasks.workunit.client.1.smithi173.stdout:3/93: rename d7/d9/fe to d7/d17/f20 0 2024-04-20T11:09:25.482 INFO:tasks.workunit.client.1.smithi173.stdout:5/44: getdents da 0 2024-04-20T11:09:25.482 INFO:tasks.workunit.client.1.smithi173.stdout:5/45: write f8 [2960986,31904] 0 2024-04-20T11:09:25.482 INFO:tasks.workunit.client.0.smithi117.stdout:4/996: stat d1/d9/f66 0 2024-04-20T11:09:25.484 INFO:tasks.workunit.client.1.smithi173.stdout:3/94: mknod d7/d9/c21 0 2024-04-20T11:09:25.486 INFO:tasks.workunit.client.1.smithi173.stdout:5/46: mkdir da/de 0 2024-04-20T11:09:25.487 INFO:tasks.workunit.client.0.smithi117.stdout:4/997: creat d1/d9/d6b/dca/f142 x:0 0 0 2024-04-20T11:09:25.489 INFO:tasks.workunit.client.1.smithi173.stdout:3/95: creat d7/f22 x:0 0 0 2024-04-20T11:09:25.490 INFO:tasks.workunit.client.0.smithi117.stdout:4/998: unlink d1/d45/dcd/cfa 0 2024-04-20T11:09:25.492 INFO:tasks.workunit.client.0.smithi117.stdout:4/999: symlink d1/dd/d1a/l143 0 2024-04-20T11:09:25.498 INFO:tasks.workunit.client.1.smithi173.stdout:5/47: link f9 da/de/ff 0 2024-04-20T11:09:25.500 INFO:tasks.workunit.client.1.smithi173.stdout:3/96: unlink d7/c10 0 2024-04-20T11:09:25.509 INFO:tasks.workunit.client.1.smithi173.stdout:0/63: dwrite d6/fd [0,4194304] 0 2024-04-20T11:09:25.519 INFO:tasks.workunit.client.1.smithi173.stdout:3/97: rename d7/l1a to d7/d9/l23 0 2024-04-20T11:09:25.521 INFO:tasks.workunit.client.1.smithi173.stdout:6/66: dwrite d4/fd [0,4194304] 0 2024-04-20T11:09:25.523 INFO:tasks.workunit.client.1.smithi173.stdout:8/108: dwrite d4/fd [0,4194304] 0 2024-04-20T11:09:25.525 INFO:tasks.workunit.client.1.smithi173.stdout:0/64: write d6/d8/f12 [986651,128889] 0 2024-04-20T11:09:25.526 INFO:tasks.workunit.client.1.smithi173.stdout:3/98: creat d7/d17/f24 x:0 0 0 2024-04-20T11:09:25.527 INFO:tasks.workunit.client.1.smithi173.stdout:9/103: dwrite d5/df/d18/f1f [0,4194304] 0 2024-04-20T11:09:25.535 INFO:tasks.workunit.client.1.smithi173.stdout:3/99: creat d7/f25 x:0 0 0 2024-04-20T11:09:25.537 INFO:tasks.workunit.client.1.smithi173.stdout:0/65: read d6/d8/fe [2328730,33740] 0 2024-04-20T11:09:25.538 INFO:tasks.workunit.client.1.smithi173.stdout:6/67: link d4/fe d4/d14/f1a 0 2024-04-20T11:09:25.538 INFO:tasks.workunit.client.1.smithi173.stdout:5/48: dread da/de/ff [0,4194304] 0 2024-04-20T11:09:25.540 INFO:tasks.workunit.client.1.smithi173.stdout:3/100: creat d7/d17/f26 x:0 0 0 2024-04-20T11:09:25.540 INFO:tasks.workunit.client.1.smithi173.stdout:0/66: readlink d6/lc 0 2024-04-20T11:09:25.543 INFO:tasks.workunit.client.1.smithi173.stdout:9/104: symlink d5/d9/d20/l21 0 2024-04-20T11:09:25.543 INFO:tasks.workunit.client.1.smithi173.stdout:3/101: mknod d7/d17/c27 0 2024-04-20T11:09:25.547 INFO:tasks.workunit.client.1.smithi173.stdout:0/67: dread - d6/d8/df/f16 zero size 2024-04-20T11:09:25.548 INFO:tasks.workunit.client.1.smithi173.stdout:9/105: rename l4 to d5/df/l22 0 2024-04-20T11:09:25.549 INFO:tasks.workunit.client.1.smithi173.stdout:3/102: write d7/f22 [114390,13030] 0 2024-04-20T11:09:25.554 INFO:tasks.workunit.client.1.smithi173.stdout:0/68: fdatasync d6/d8/f12 0 2024-04-20T11:09:25.556 INFO:tasks.workunit.client.1.smithi173.stdout:9/106: fdatasync d5/df/f10 0 2024-04-20T11:09:25.564 INFO:tasks.workunit.client.1.smithi173.stdout:1/74: sync 2024-04-20T11:09:25.564 INFO:tasks.workunit.client.1.smithi173.stdout:9/107: mknod d5/d9/d20/c23 0 2024-04-20T11:09:25.570 INFO:tasks.workunit.client.1.smithi173.stdout:8/109: dwrite d4/f10 [0,4194304] 0 2024-04-20T11:09:25.572 INFO:tasks.workunit.client.1.smithi173.stdout:6/68: dread d4/fe [0,4194304] 0 2024-04-20T11:09:25.573 INFO:tasks.workunit.client.1.smithi173.stdout:1/75: symlink d5/d7/l18 0 2024-04-20T11:09:25.573 INFO:tasks.workunit.client.1.smithi173.stdout:9/108: mknod d5/df/c24 0 2024-04-20T11:09:25.574 INFO:tasks.workunit.client.1.smithi173.stdout:0/69: link d6/d8/df/l14 d6/d8/df/l17 0 2024-04-20T11:09:25.576 INFO:tasks.workunit.client.0.smithi117.stdout:8/950: sync 2024-04-20T11:09:25.578 INFO:tasks.workunit.client.1.smithi173.stdout:6/69: stat c0 0 2024-04-20T11:09:25.581 INFO:tasks.workunit.client.1.smithi173.stdout:8/110: rename d4/f6 to d4/d18/f1e 0 2024-04-20T11:09:25.581 INFO:tasks.workunit.client.1.smithi173.stdout:5/49: dwrite f6 [4194304,4194304] 0 2024-04-20T11:09:25.581 INFO:tasks.workunit.client.1.smithi173.stdout:9/109: rmdir d5/d9/d20 39 2024-04-20T11:09:25.582 INFO:tasks.workunit.client.1.smithi173.stdout:6/70: chown l3 58 1 2024-04-20T11:09:25.589 INFO:tasks.workunit.client.1.smithi173.stdout:1/76: link f4 d5/d7/f19 0 2024-04-20T11:09:25.590 INFO:tasks.workunit.client.1.smithi173.stdout:3/103: dwrite d7/d9/f13 [0,4194304] 0 2024-04-20T11:09:25.592 INFO:tasks.workunit.client.1.smithi173.stdout:3/104: stat d7/d9/la 0 2024-04-20T11:09:25.594 INFO:tasks.workunit.client.1.smithi173.stdout:3/105: write d7/d17/f19 [1095832,119508] 0 2024-04-20T11:09:25.595 INFO:tasks.workunit.client.1.smithi173.stdout:8/111: creat d4/d18/f1f x:0 0 0 2024-04-20T11:09:25.596 INFO:tasks.workunit.client.1.smithi173.stdout:4/73: sync 2024-04-20T11:09:25.596 INFO:tasks.workunit.client.1.smithi173.stdout:2/68: sync 2024-04-20T11:09:25.596 INFO:tasks.workunit.client.1.smithi173.stdout:7/50: sync 2024-04-20T11:09:25.597 INFO:tasks.workunit.client.1.smithi173.stdout:6/71: rmdir d4/d14 39 2024-04-20T11:09:25.598 INFO:tasks.workunit.client.1.smithi173.stdout:0/70: creat d6/d8/df/d15/f18 x:0 0 0 2024-04-20T11:09:25.600 INFO:tasks.workunit.client.1.smithi173.stdout:7/51: readlink l7 0 2024-04-20T11:09:25.603 INFO:tasks.workunit.client.1.smithi173.stdout:6/72: unlink l3 0 2024-04-20T11:09:25.605 INFO:tasks.workunit.client.1.smithi173.stdout:8/112: chown d4/d18/f1a 30 1 2024-04-20T11:09:25.608 INFO:tasks.workunit.client.0.smithi117.stdout:8/951: dwrite d0/d8/d39/d40/de5/d12a/f136 [0,4194304] 0 2024-04-20T11:09:25.609 INFO:tasks.workunit.client.1.smithi173.stdout:4/74: rename l2 to da/l19 0 2024-04-20T11:09:25.612 INFO:tasks.workunit.client.1.smithi173.stdout:0/71: read d6/f7 [86952,122723] 0 2024-04-20T11:09:25.612 INFO:tasks.workunit.client.1.smithi173.stdout:7/52: chown d9/da/f10 938 1 2024-04-20T11:09:25.614 INFO:tasks.workunit.client.1.smithi173.stdout:6/73: truncate d4/f16 1041287 0 2024-04-20T11:09:25.614 INFO:tasks.workunit.client.1.smithi173.stdout:8/113: creat d4/f20 x:0 0 0 2024-04-20T11:09:25.615 INFO:tasks.workunit.client.1.smithi173.stdout:2/69: creat d2/d14/f15 x:0 0 0 2024-04-20T11:09:25.617 INFO:tasks.workunit.client.1.smithi173.stdout:3/106: rename d7/f11 to d7/d9/f28 0 2024-04-20T11:09:25.625 INFO:tasks.workunit.client.1.smithi173.stdout:6/74: symlink d4/d18/l1b 0 2024-04-20T11:09:25.625 INFO:tasks.workunit.client.1.smithi173.stdout:9/110: dwrite d5/df/f11 [0,4194304] 0 2024-04-20T11:09:25.628 INFO:tasks.workunit.client.1.smithi173.stdout:0/72: write d6/f7 [3178299,114831] 0 2024-04-20T11:09:25.630 INFO:tasks.workunit.client.0.smithi117.stdout:8/952: truncate d0/d8/d11/d6d/d84/fc1 3560242 0 2024-04-20T11:09:25.630 INFO:tasks.workunit.client.1.smithi173.stdout:5/50: dwrite f7 [0,4194304] 0 2024-04-20T11:09:25.632 INFO:tasks.workunit.client.1.smithi173.stdout:3/107: chown d7/d9/c1d 44 1 2024-04-20T11:09:25.632 INFO:tasks.workunit.client.1.smithi173.stdout:4/75: write da/dc/d15/f18 [993969,73308] 0 2024-04-20T11:09:25.634 INFO:tasks.workunit.client.1.smithi173.stdout:6/75: fdatasync d4/d18/f19 0 2024-04-20T11:09:25.634 INFO:tasks.workunit.client.1.smithi173.stdout:1/77: dwrite d5/d7/f11 [0,4194304] 0 2024-04-20T11:09:25.635 INFO:tasks.workunit.client.1.smithi173.stdout:3/108: read - d7/d9/f14 zero size 2024-04-20T11:09:25.635 INFO:tasks.workunit.client.1.smithi173.stdout:7/53: creat d9/da/f12 x:0 0 0 2024-04-20T11:09:25.637 INFO:tasks.workunit.client.0.smithi117.stdout:8/953: chown d0/d8/d11/d24/d27/d99 10 1 2024-04-20T11:09:25.638 INFO:tasks.workunit.client.1.smithi173.stdout:2/70: unlink d2/d13/l12 0 2024-04-20T11:09:25.639 INFO:tasks.workunit.client.1.smithi173.stdout:0/73: unlink f2 0 2024-04-20T11:09:25.640 INFO:tasks.workunit.client.1.smithi173.stdout:4/76: mkdir da/d1a 0 2024-04-20T11:09:25.642 INFO:tasks.workunit.client.1.smithi173.stdout:7/54: symlink d9/l13 0 2024-04-20T11:09:25.644 INFO:tasks.workunit.client.1.smithi173.stdout:8/114: dread d4/f17 [0,4194304] 0 2024-04-20T11:09:25.644 INFO:tasks.workunit.client.1.smithi173.stdout:0/74: dread - d6/d8/df/f16 zero size 2024-04-20T11:09:25.645 INFO:tasks.workunit.client.1.smithi173.stdout:3/109: creat d7/d17/f29 x:0 0 0 2024-04-20T11:09:25.647 INFO:tasks.workunit.client.0.smithi117.stdout:8/954: unlink d0/d8/d11/d24/d53/l10b 0 2024-04-20T11:09:25.648 INFO:tasks.workunit.client.1.smithi173.stdout:1/78: read - d5/d7/da/f14 zero size 2024-04-20T11:09:25.649 INFO:tasks.workunit.client.1.smithi173.stdout:5/51: creat da/de/f10 x:0 0 0 2024-04-20T11:09:25.650 INFO:tasks.workunit.client.1.smithi173.stdout:3/110: dread f4 [0,4194304] 0 2024-04-20T11:09:25.650 INFO:tasks.workunit.client.1.smithi173.stdout:4/77: symlink da/l1b 0 2024-04-20T11:09:25.652 INFO:tasks.workunit.client.1.smithi173.stdout:7/55: creat d9/da/f14 x:0 0 0 2024-04-20T11:09:25.652 INFO:tasks.workunit.client.1.smithi173.stdout:9/111: creat d5/d9/f25 x:0 0 0 2024-04-20T11:09:25.653 INFO:tasks.workunit.client.1.smithi173.stdout:8/115: unlink d4/l1c 0 2024-04-20T11:09:25.656 INFO:tasks.workunit.client.1.smithi173.stdout:1/79: unlink d5/d7/f11 0 2024-04-20T11:09:25.657 INFO:tasks.workunit.client.1.smithi173.stdout:2/71: dread d2/de/d11/fd [0,4194304] 0 2024-04-20T11:09:25.657 INFO:tasks.workunit.client.1.smithi173.stdout:3/111: mknod d7/d9/c2a 0 2024-04-20T11:09:25.657 INFO:tasks.workunit.client.1.smithi173.stdout:7/56: fdatasync d9/da/ff 0 2024-04-20T11:09:25.658 INFO:tasks.workunit.client.1.smithi173.stdout:4/78: unlink da/dc/d13/c16 0 2024-04-20T11:09:25.659 INFO:tasks.workunit.client.1.smithi173.stdout:1/80: stat d5/d6/f9 0 2024-04-20T11:09:25.661 INFO:tasks.workunit.client.0.smithi117.stdout:1/970: sync 2024-04-20T11:09:25.664 INFO:tasks.workunit.client.1.smithi173.stdout:8/116: write d4/f8 [197970,108346] 0 2024-04-20T11:09:25.664 INFO:tasks.workunit.client.1.smithi173.stdout:5/52: creat da/dd/f11 x:0 0 0 2024-04-20T11:09:25.666 INFO:tasks.workunit.client.1.smithi173.stdout:7/57: creat d9/f15 x:0 0 0 2024-04-20T11:09:25.668 INFO:tasks.workunit.client.1.smithi173.stdout:3/112: mkdir d7/d2b 0 2024-04-20T11:09:25.668 INFO:tasks.workunit.client.1.smithi173.stdout:8/117: dread - d4/f19 zero size 2024-04-20T11:09:25.669 INFO:tasks.workunit.client.1.smithi173.stdout:7/58: chown l7 163 1 2024-04-20T11:09:25.669 INFO:tasks.workunit.client.0.smithi117.stdout:8/955: write d0/d8/d39/fc9 [1068533,115554] 0 2024-04-20T11:09:25.670 INFO:tasks.workunit.client.1.smithi173.stdout:0/75: dwrite d6/fa [0,4194304] 0 2024-04-20T11:09:25.670 INFO:tasks.workunit.client.0.smithi117.stdout:1/971: link d6/d1f/d30/d49/d124/db9/f118 d6/d16/d3a/d5a/daf/de4/f13d 0 2024-04-20T11:09:25.671 INFO:tasks.workunit.client.1.smithi173.stdout:4/79: write da/dc/ff [972430,30241] 0 2024-04-20T11:09:25.681 INFO:tasks.workunit.client.1.smithi173.stdout:5/53: mkdir da/d12 0 2024-04-20T11:09:25.683 INFO:tasks.workunit.client.1.smithi173.stdout:6/76: getdents d4 0 2024-04-20T11:09:25.685 INFO:tasks.workunit.client.1.smithi173.stdout:3/113: creat d7/d17/f2c x:0 0 0 2024-04-20T11:09:25.686 INFO:tasks.workunit.client.0.smithi117.stdout:1/972: creat d6/d16/d3a/d5a/d5c/f13e x:0 0 0 2024-04-20T11:09:25.687 INFO:tasks.workunit.client.1.smithi173.stdout:0/76: read - d6/d8/df/f16 zero size 2024-04-20T11:09:25.688 INFO:tasks.workunit.client.1.smithi173.stdout:8/118: creat d4/d18/f21 x:0 0 0 2024-04-20T11:09:25.689 INFO:tasks.workunit.client.1.smithi173.stdout:0/77: dread - d6/f13 zero size 2024-04-20T11:09:25.690 INFO:tasks.workunit.client.1.smithi173.stdout:4/80: creat da/dc/d15/f1c x:0 0 0 2024-04-20T11:09:25.691 INFO:tasks.workunit.client.0.smithi117.stdout:1/973: creat d6/d35/d63/f13f x:0 0 0 2024-04-20T11:09:25.692 INFO:tasks.workunit.client.1.smithi173.stdout:2/72: dwrite d2/de/d11/fd [0,4194304] 0 2024-04-20T11:09:25.694 INFO:tasks.workunit.client.1.smithi173.stdout:4/81: truncate da/dc/d15/f1c 151468 0 2024-04-20T11:09:25.710 INFO:tasks.workunit.client.0.smithi117.stdout:1/974: dwrite d6/d16/d3a/d5a/d5c/d61/d98/f11f [4194304,4194304] 0 2024-04-20T11:09:25.712 INFO:tasks.workunit.client.0.smithi117.stdout:1/975: stat d6/d1f/d30/d49/d69/d6e/d6f/cc0 0 2024-04-20T11:09:25.718 INFO:tasks.workunit.client.0.smithi117.stdout:1/976: creat d6/d16/dce/de0/f140 x:0 0 0 2024-04-20T11:09:25.719 INFO:tasks.workunit.client.0.smithi117.stdout:1/977: readlink d6/dd2/la7 0 2024-04-20T11:09:25.731 INFO:tasks.workunit.client.1.smithi173.stdout:6/77: mkdir d4/d1c 0 2024-04-20T11:09:25.732 INFO:tasks.workunit.client.1.smithi173.stdout:7/59: creat d9/d11/f16 x:0 0 0 2024-04-20T11:09:25.732 INFO:tasks.workunit.client.1.smithi173.stdout:8/119: truncate d4/f17 86602 0 2024-04-20T11:09:25.733 INFO:tasks.workunit.client.1.smithi173.stdout:0/78: rmdir d6/d8/df/d15 39 2024-04-20T11:09:25.733 INFO:tasks.workunit.client.1.smithi173.stdout:1/81: truncate f3 1317395 0 2024-04-20T11:09:25.734 INFO:tasks.workunit.client.1.smithi173.stdout:7/60: stat d9/da/ff 0 2024-04-20T11:09:25.736 INFO:tasks.workunit.client.1.smithi173.stdout:7/61: write d9/da/f10 [991049,69539] 0 2024-04-20T11:09:25.741 INFO:tasks.workunit.client.1.smithi173.stdout:2/73: dread d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:25.742 INFO:tasks.workunit.client.1.smithi173.stdout:2/74: write d2/d14/f15 [781238,90684] 0 2024-04-20T11:09:25.747 INFO:tasks.workunit.client.0.smithi117.stdout:1/978: dread d6/d1f/d30/d49/d69/d6e/d6f/d73/f8c [0,4194304] 0 2024-04-20T11:09:25.767 INFO:tasks.workunit.client.1.smithi173.stdout:9/112: sync 2024-04-20T11:09:25.775 INFO:tasks.workunit.client.1.smithi173.stdout:4/82: creat da/dc/d15/f1d x:0 0 0 2024-04-20T11:09:25.776 INFO:tasks.workunit.client.1.smithi173.stdout:0/79: creat d6/d8/f19 x:0 0 0 2024-04-20T11:09:25.782 INFO:tasks.workunit.client.1.smithi173.stdout:7/62: rename d9/d11 to d9/da/d17 0 2024-04-20T11:09:25.784 INFO:tasks.workunit.client.1.smithi173.stdout:2/75: rename d2 to d2/de/d16 22 2024-04-20T11:09:25.785 INFO:tasks.workunit.client.1.smithi173.stdout:4/83: mkdir da/d1e 0 2024-04-20T11:09:25.786 INFO:tasks.workunit.client.1.smithi173.stdout:7/63: fsync d9/da/f10 0 2024-04-20T11:09:25.787 INFO:tasks.workunit.client.1.smithi173.stdout:5/54: getdents da/dd 0 2024-04-20T11:09:25.791 INFO:tasks.workunit.client.1.smithi173.stdout:9/113: creat d5/d9/f26 x:0 0 0 2024-04-20T11:09:25.792 INFO:tasks.workunit.client.1.smithi173.stdout:7/64: truncate d9/f15 545964 0 2024-04-20T11:09:25.793 INFO:tasks.workunit.client.1.smithi173.stdout:3/114: sync 2024-04-20T11:09:25.793 INFO:tasks.workunit.client.1.smithi173.stdout:4/84: symlink da/d1a/l1f 0 2024-04-20T11:09:25.794 INFO:tasks.workunit.client.1.smithi173.stdout:3/115: chown f3 31570661 1 2024-04-20T11:09:25.794 INFO:tasks.workunit.client.1.smithi173.stdout:6/78: rename d4/lf to d4/d18/l1d 0 2024-04-20T11:09:25.795 INFO:tasks.workunit.client.1.smithi173.stdout:5/55: symlink da/d12/l13 0 2024-04-20T11:09:25.796 INFO:tasks.workunit.client.1.smithi173.stdout:3/116: chown d7/d9/c1d 6 1 2024-04-20T11:09:25.798 INFO:tasks.workunit.client.1.smithi173.stdout:0/80: dread d6/f7 [0,4194304] 0 2024-04-20T11:09:25.798 INFO:tasks.workunit.client.1.smithi173.stdout:5/56: dread - da/de/f10 zero size 2024-04-20T11:09:25.799 INFO:tasks.workunit.client.1.smithi173.stdout:3/117: creat d7/f2d x:0 0 0 2024-04-20T11:09:25.800 INFO:tasks.workunit.client.1.smithi173.stdout:3/118: fsync d7/f22 0 2024-04-20T11:09:25.801 INFO:tasks.workunit.client.1.smithi173.stdout:6/79: write d4/fd [1930732,54267] 0 2024-04-20T11:09:25.801 INFO:tasks.workunit.client.1.smithi173.stdout:4/85: rename l7 to da/d1e/l20 0 2024-04-20T11:09:25.804 INFO:tasks.workunit.client.0.smithi117.stdout:1/979: dwrite d6/d1f/d30/d32/ffb [0,4194304] 0 2024-04-20T11:09:25.804 INFO:tasks.workunit.client.1.smithi173.stdout:0/81: write d6/d8/df/f16 [525739,33284] 0 2024-04-20T11:09:25.804 INFO:tasks.workunit.client.1.smithi173.stdout:5/57: symlink da/de/l14 0 2024-04-20T11:09:25.807 INFO:tasks.workunit.client.0.smithi117.stdout:1/980: write d6/d1f/d30/d32/fdd [1221090,53109] 0 2024-04-20T11:09:25.810 INFO:tasks.workunit.client.0.smithi117.stdout:1/981: write d6/d16/f62 [2517514,8643] 0 2024-04-20T11:09:25.811 INFO:tasks.workunit.client.1.smithi173.stdout:0/82: mknod d6/c1a 0 2024-04-20T11:09:25.812 INFO:tasks.workunit.client.1.smithi173.stdout:6/80: read d4/f13 [1161317,45199] 0 2024-04-20T11:09:25.813 INFO:tasks.workunit.client.1.smithi173.stdout:5/58: rmdir da/d12 39 2024-04-20T11:09:25.814 INFO:tasks.workunit.client.1.smithi173.stdout:4/86: mknod da/dc/d13/c21 0 2024-04-20T11:09:25.821 INFO:tasks.workunit.client.1.smithi173.stdout:6/81: fsync d4/fd 0 2024-04-20T11:09:25.821 INFO:tasks.workunit.client.1.smithi173.stdout:9/114: dwrite d5/df/f11 [0,4194304] 0 2024-04-20T11:09:25.829 INFO:tasks.workunit.client.1.smithi173.stdout:8/120: dread f2 [0,4194304] 0 2024-04-20T11:09:25.830 INFO:tasks.workunit.client.1.smithi173.stdout:2/76: dwrite d2/de/d11/fd [0,4194304] 0 2024-04-20T11:09:25.832 INFO:tasks.workunit.client.0.smithi117.stdout:1/982: write d6/d16/fad [3952409,100931] 0 2024-04-20T11:09:25.833 INFO:tasks.workunit.client.1.smithi173.stdout:4/87: mknod da/dc/c22 0 2024-04-20T11:09:25.836 INFO:tasks.workunit.client.1.smithi173.stdout:9/115: read d5/d9/f1b [986187,99108] 0 2024-04-20T11:09:25.836 INFO:tasks.workunit.client.1.smithi173.stdout:7/65: dwrite d9/da/f14 [0,4194304] 0 2024-04-20T11:09:25.837 INFO:tasks.workunit.client.1.smithi173.stdout:6/82: rmdir d4/d18 39 2024-04-20T11:09:25.840 INFO:tasks.workunit.client.0.smithi117.stdout:1/983: unlink d6/d16/d3a/d5a/d5c/d61/d71/d6b/c8e 0 2024-04-20T11:09:25.842 INFO:tasks.workunit.client.1.smithi173.stdout:8/121: unlink d4/d18/f1e 0 2024-04-20T11:09:25.847 INFO:tasks.workunit.client.1.smithi173.stdout:7/66: write d9/f15 [325231,53534] 0 2024-04-20T11:09:25.847 INFO:tasks.workunit.client.1.smithi173.stdout:6/83: symlink d4/l1e 0 2024-04-20T11:09:25.851 INFO:tasks.workunit.client.1.smithi173.stdout:0/83: rename d6/d8/df/l14 to d6/d8/df/l1b 0 2024-04-20T11:09:25.853 INFO:tasks.workunit.client.1.smithi173.stdout:7/67: write d9/f15 [47031,58811] 0 2024-04-20T11:09:25.855 INFO:tasks.workunit.client.1.smithi173.stdout:5/59: dwrite da/fc [0,4194304] 0 2024-04-20T11:09:25.857 INFO:tasks.workunit.client.1.smithi173.stdout:1/82: dwrite d5/d7/f19 [0,4194304] 0 2024-04-20T11:09:25.858 INFO:tasks.workunit.client.0.smithi117.stdout:1/984: dwrite d6/d16/d3a/d5a/daf/ff5 [4194304,4194304] 0 2024-04-20T11:09:25.859 INFO:tasks.workunit.client.1.smithi173.stdout:7/68: truncate f6 4663185 0 2024-04-20T11:09:25.865 INFO:tasks.workunit.client.1.smithi173.stdout:0/84: symlink d6/d8/l1c 0 2024-04-20T11:09:25.865 INFO:tasks.workunit.client.1.smithi173.stdout:3/119: dwrite f4 [0,4194304] 0 2024-04-20T11:09:25.867 INFO:tasks.workunit.client.1.smithi173.stdout:2/77: link d2/l4 d2/de/l17 0 2024-04-20T11:09:25.870 INFO:tasks.workunit.client.1.smithi173.stdout:4/88: dread f1 [0,4194304] 0 2024-04-20T11:09:25.870 INFO:tasks.workunit.client.1.smithi173.stdout:2/78: fsync d2/de/d11/fd 0 2024-04-20T11:09:25.871 INFO:tasks.workunit.client.1.smithi173.stdout:9/116: creat d5/d9/f27 x:0 0 0 2024-04-20T11:09:25.872 INFO:tasks.workunit.client.0.smithi117.stdout:1/985: mkdir d6/d1f/d30/d49/d64/d141 0 2024-04-20T11:09:25.872 INFO:tasks.workunit.client.1.smithi173.stdout:3/120: fsync d7/f22 0 2024-04-20T11:09:25.873 INFO:tasks.workunit.client.0.smithi117.stdout:1/986: write d6/f1a [871699,65191] 0 2024-04-20T11:09:25.877 INFO:tasks.workunit.client.0.smithi117.stdout:8/956: sync 2024-04-20T11:09:25.878 INFO:tasks.workunit.client.1.smithi173.stdout:4/89: write f1 [2871336,117871] 0 2024-04-20T11:09:25.879 INFO:tasks.workunit.client.1.smithi173.stdout:3/121: read d7/f8 [2476154,120508] 0 2024-04-20T11:09:25.880 INFO:tasks.workunit.client.1.smithi173.stdout:6/84: rename d4/c12 to d4/d14/c1f 0 2024-04-20T11:09:25.880 INFO:tasks.workunit.client.1.smithi173.stdout:8/122: dread d4/f10 [0,4194304] 0 2024-04-20T11:09:25.883 INFO:tasks.workunit.client.1.smithi173.stdout:4/90: stat da/dc/d15 0 2024-04-20T11:09:25.883 INFO:tasks.workunit.client.1.smithi173.stdout:1/83: getdents d5/d6/d17 0 2024-04-20T11:09:25.884 INFO:tasks.workunit.client.1.smithi173.stdout:3/122: chown d7/l16 84950 1 2024-04-20T11:09:25.889 INFO:tasks.workunit.client.0.smithi117.stdout:8/957: truncate d0/dc/d49/d5b/da9/dc7/fde 86951 0 2024-04-20T11:09:25.890 INFO:tasks.workunit.client.1.smithi173.stdout:2/79: dread d2/d14/f15 [0,4194304] 0 2024-04-20T11:09:25.891 INFO:tasks.workunit.client.1.smithi173.stdout:7/69: truncate d9/da/ff 102116 0 2024-04-20T11:09:25.892 INFO:tasks.workunit.client.1.smithi173.stdout:5/60: dwrite f8 [4194304,4194304] 0 2024-04-20T11:09:25.892 INFO:tasks.workunit.client.1.smithi173.stdout:8/123: unlink d4/c5 0 2024-04-20T11:09:25.893 INFO:tasks.workunit.client.1.smithi173.stdout:6/85: dread - d4/d18/f19 zero size 2024-04-20T11:09:25.895 INFO:tasks.workunit.client.1.smithi173.stdout:7/70: fdatasync d9/da/f10 0 2024-04-20T11:09:25.895 INFO:tasks.workunit.client.1.smithi173.stdout:9/117: mkdir d5/d9/d20/d28 0 2024-04-20T11:09:25.895 INFO:tasks.workunit.client.0.smithi117.stdout:1/987: dwrite d6/d16/d3a/d5a/d5c/f13e [0,4194304] 0 2024-04-20T11:09:25.896 INFO:tasks.workunit.client.1.smithi173.stdout:0/85: creat d6/d8/df/f1d x:0 0 0 2024-04-20T11:09:25.898 INFO:tasks.workunit.client.1.smithi173.stdout:8/124: symlink d4/l22 0 2024-04-20T11:09:25.898 INFO:tasks.workunit.client.1.smithi173.stdout:9/118: chown d5/d9/fa 7024848 1 2024-04-20T11:09:25.900 INFO:tasks.workunit.client.1.smithi173.stdout:9/119: chown d5 20507163 1 2024-04-20T11:09:25.900 INFO:tasks.workunit.client.0.smithi117.stdout:8/958: link d0/d9/d3a/d105/ffb d0/d9/d3a/d7c/dce/f13a 0 2024-04-20T11:09:25.902 INFO:tasks.workunit.client.0.smithi117.stdout:1/988: rmdir d6/d16/d3a/d5a/d5c/d61/d98/deb 39 2024-04-20T11:09:25.902 INFO:tasks.workunit.client.1.smithi173.stdout:5/61: creat da/dd/f15 x:0 0 0 2024-04-20T11:09:25.903 INFO:tasks.workunit.client.0.smithi117.stdout:8/959: dread - d0/dc/d49/d5b/da9/dc7/ffd zero size 2024-04-20T11:09:25.905 INFO:tasks.workunit.client.1.smithi173.stdout:1/84: symlink d5/d7/da/df/l1a 0 2024-04-20T11:09:25.906 INFO:tasks.workunit.client.0.smithi117.stdout:1/989: fdatasync d6/d16/d3a/d5a/d5c/ffe 0 2024-04-20T11:09:25.906 INFO:tasks.workunit.client.0.smithi117.stdout:8/960: read d0/dc/d49/f101 [258729,109961] 0 2024-04-20T11:09:25.907 INFO:tasks.workunit.client.1.smithi173.stdout:9/120: fsync d5/d9/f1a 0 2024-04-20T11:09:25.908 INFO:tasks.workunit.client.0.smithi117.stdout:1/990: dread - d6/d16/dce/de0/f140 zero size 2024-04-20T11:09:25.908 INFO:tasks.workunit.client.1.smithi173.stdout:2/80: dread d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:25.909 INFO:tasks.workunit.client.0.smithi117.stdout:8/961: chown d0/d8/d11/d24/d86/f126 77901909 1 2024-04-20T11:09:25.911 INFO:tasks.workunit.client.1.smithi173.stdout:9/121: fsync d5/d9/f25 0 2024-04-20T11:09:25.912 INFO:tasks.workunit.client.1.smithi173.stdout:7/71: rename d9/l13 to d9/da/d17/l18 0 2024-04-20T11:09:25.917 INFO:tasks.workunit.client.1.smithi173.stdout:6/86: truncate d4/fe 2956590 0 2024-04-20T11:09:25.917 INFO:tasks.workunit.client.0.smithi117.stdout:1/991: creat d6/d16/d3a/d5a/d5c/d61/d71/d6b/dc7/f142 x:0 0 0 2024-04-20T11:09:25.918 INFO:tasks.workunit.client.1.smithi173.stdout:0/86: unlink f4 0 2024-04-20T11:09:25.920 INFO:tasks.workunit.client.0.smithi117.stdout:8/962: creat d0/d9/d118/f13b x:0 0 0 2024-04-20T11:09:25.922 INFO:tasks.workunit.client.0.smithi117.stdout:1/992: mkdir d6/d35/d63/d143 0 2024-04-20T11:09:25.924 INFO:tasks.workunit.client.1.smithi173.stdout:3/123: dwrite d7/f25 [0,4194304] 0 2024-04-20T11:09:25.926 INFO:tasks.workunit.client.1.smithi173.stdout:2/81: write d2/de/d11/f10 [565566,79029] 0 2024-04-20T11:09:25.928 INFO:tasks.workunit.client.1.smithi173.stdout:4/91: dwrite da/dc/d15/f18 [0,4194304] 0 2024-04-20T11:09:25.929 INFO:tasks.workunit.client.1.smithi173.stdout:1/85: mkdir d5/d6/d17/d1b 0 2024-04-20T11:09:25.931 INFO:tasks.workunit.client.1.smithi173.stdout:5/62: symlink da/d12/l16 0 2024-04-20T11:09:25.931 INFO:tasks.workunit.client.0.smithi117.stdout:8/963: dread d0/d8/d11/fac [0,4194304] 0 2024-04-20T11:09:25.935 INFO:tasks.workunit.client.0.smithi117.stdout:1/993: link d6/d1f/d30/d49/d124/f86 d6/d16/d3a/d5a/d5c/df0/d10c/f144 0 2024-04-20T11:09:25.940 INFO:tasks.workunit.client.1.smithi173.stdout:0/87: read - d6/d8/df/d15/f18 zero size 2024-04-20T11:09:25.946 INFO:tasks.workunit.client.0.smithi117.stdout:1/994: chown d6/d1f/f128 3596 1 2024-04-20T11:09:25.950 INFO:tasks.workunit.client.1.smithi173.stdout:8/125: dwrite d4/d18/f1f [0,4194304] 0 2024-04-20T11:09:25.950 INFO:tasks.workunit.client.1.smithi173.stdout:4/92: dread f1 [4194304,4194304] 0 2024-04-20T11:09:25.951 INFO:tasks.workunit.client.1.smithi173.stdout:1/86: mkdir d5/d7/d12/d1c 0 2024-04-20T11:09:25.957 INFO:tasks.workunit.client.1.smithi173.stdout:1/87: fsync d5/fb 0 2024-04-20T11:09:25.957 INFO:tasks.workunit.client.1.smithi173.stdout:7/72: dread d9/da/f14 [0,4194304] 0 2024-04-20T11:09:25.965 INFO:tasks.workunit.client.1.smithi173.stdout:9/122: dwrite d5/d9/f25 [0,4194304] 0 2024-04-20T11:09:25.966 INFO:tasks.workunit.client.1.smithi173.stdout:6/87: getdents d4/d18 0 2024-04-20T11:09:25.967 INFO:tasks.workunit.client.1.smithi173.stdout:1/88: chown d5/d7/d12/f15 49095 1 2024-04-20T11:09:25.969 INFO:tasks.workunit.client.0.smithi117.stdout:8/964: dwrite d0/d8/d11/d6d/d84/f112 [0,4194304] 0 2024-04-20T11:09:25.970 INFO:tasks.workunit.client.1.smithi173.stdout:7/73: write d9/da/f10 [1752765,110026] 0 2024-04-20T11:09:25.972 INFO:tasks.workunit.client.1.smithi173.stdout:6/88: fdatasync d4/d18/f19 0 2024-04-20T11:09:25.972 INFO:tasks.workunit.client.1.smithi173.stdout:4/93: truncate da/dc/d15/f1c 618792 0 2024-04-20T11:09:25.978 INFO:tasks.workunit.client.1.smithi173.stdout:3/124: dwrite d7/d9/ff [0,4194304] 0 2024-04-20T11:09:25.978 INFO:tasks.workunit.client.1.smithi173.stdout:8/126: unlink d4/d18/f1b 0 2024-04-20T11:09:25.984 INFO:tasks.workunit.client.1.smithi173.stdout:2/82: dwrite d2/d14/f15 [0,4194304] 0 2024-04-20T11:09:25.984 INFO:tasks.workunit.client.1.smithi173.stdout:5/63: truncate f6 1566363 0 2024-04-20T11:09:25.985 INFO:tasks.workunit.client.1.smithi173.stdout:1/89: rmdir d5/d6/dc 39 2024-04-20T11:09:25.988 INFO:tasks.workunit.client.1.smithi173.stdout:7/74: rmdir d9/da 39 2024-04-20T11:09:25.990 INFO:tasks.workunit.client.1.smithi173.stdout:5/64: dread da/de/ff [0,4194304] 0 2024-04-20T11:09:25.992 INFO:tasks.workunit.client.1.smithi173.stdout:5/65: truncate da/dd/f15 240176 0 2024-04-20T11:09:25.995 INFO:tasks.workunit.client.1.smithi173.stdout:8/127: mkdir d4/d23 0 2024-04-20T11:09:25.997 INFO:tasks.workunit.client.1.smithi173.stdout:8/128: stat d4/l11 0 2024-04-20T11:09:25.997 INFO:tasks.workunit.client.1.smithi173.stdout:0/88: getdents d6/d8 0 2024-04-20T11:09:25.998 INFO:tasks.workunit.client.1.smithi173.stdout:8/129: readlink d4/l13 0 2024-04-20T11:09:25.999 INFO:tasks.workunit.client.1.smithi173.stdout:0/89: chown d6/d8/l1c 0 1 2024-04-20T11:09:26.002 INFO:tasks.workunit.client.1.smithi173.stdout:9/123: rename d5/df to d5/d9/d20/d29 0 2024-04-20T11:09:26.004 INFO:tasks.workunit.client.1.smithi173.stdout:9/124: write d5/d9/f26 [130331,126025] 0 2024-04-20T11:09:26.009 INFO:tasks.workunit.client.1.smithi173.stdout:3/125: creat d7/d9/f2e x:0 0 0 2024-04-20T11:09:26.011 INFO:tasks.workunit.client.1.smithi173.stdout:3/126: dread - d7/d9/f14 zero size 2024-04-20T11:09:26.011 INFO:tasks.workunit.client.1.smithi173.stdout:6/89: unlink d4/f10 0 2024-04-20T11:09:26.012 INFO:tasks.workunit.client.1.smithi173.stdout:6/90: chown d4/l1e 10519307 1 2024-04-20T11:09:26.014 INFO:tasks.workunit.client.1.smithi173.stdout:5/66: mknod da/d12/c17 0 2024-04-20T11:09:26.015 INFO:tasks.workunit.client.1.smithi173.stdout:8/130: rmdir d4/d18 39 2024-04-20T11:09:26.016 INFO:tasks.workunit.client.1.smithi173.stdout:8/131: dread - d4/f20 zero size 2024-04-20T11:09:26.021 INFO:tasks.workunit.client.1.smithi173.stdout:0/90: chown c0 437280536 1 2024-04-20T11:09:26.024 INFO:tasks.workunit.client.1.smithi173.stdout:9/125: fdatasync d5/f6 0 2024-04-20T11:09:26.026 INFO:tasks.workunit.client.1.smithi173.stdout:7/75: mkdir d9/d19 0 2024-04-20T11:09:26.026 INFO:tasks.workunit.client.1.smithi173.stdout:0/91: dread d6/d8/df/f16 [0,4194304] 0 2024-04-20T11:09:26.027 INFO:tasks.workunit.client.1.smithi173.stdout:0/92: chown d6/c1a 2139207268 1 2024-04-20T11:09:26.029 INFO:tasks.workunit.client.1.smithi173.stdout:0/93: write d6/d8/df/f11 [464878,59188] 0 2024-04-20T11:09:26.029 INFO:tasks.workunit.client.1.smithi173.stdout:4/94: truncate f1 5044021 0 2024-04-20T11:09:26.030 INFO:tasks.workunit.client.1.smithi173.stdout:3/127: creat d7/d17/f2f x:0 0 0 2024-04-20T11:09:26.031 INFO:tasks.workunit.client.1.smithi173.stdout:0/94: truncate d6/d8/f19 916284 0 2024-04-20T11:09:26.034 INFO:tasks.workunit.client.1.smithi173.stdout:1/90: truncate f4 1709354 0 2024-04-20T11:09:26.039 INFO:tasks.workunit.client.1.smithi173.stdout:9/126: symlink d5/d9/d20/l2a 0 2024-04-20T11:09:26.041 INFO:tasks.workunit.client.1.smithi173.stdout:9/127: write d5/d9/f25 [2670090,70054] 0 2024-04-20T11:09:26.042 INFO:tasks.workunit.client.1.smithi173.stdout:3/128: mkdir d7/d9/d30 0 2024-04-20T11:09:26.046 INFO:tasks.workunit.client.1.smithi173.stdout:0/95: chown d6/d8/df/l1b 75232 1 2024-04-20T11:09:26.053 INFO:tasks.workunit.client.1.smithi173.stdout:6/91: dwrite d4/f16 [0,4194304] 0 2024-04-20T11:09:26.056 INFO:tasks.workunit.client.1.smithi173.stdout:7/76: creat d9/da/d17/f1a x:0 0 0 2024-04-20T11:09:26.061 INFO:tasks.workunit.client.1.smithi173.stdout:8/132: dwrite d4/d18/f1f [0,4194304] 0 2024-04-20T11:09:26.062 INFO:tasks.workunit.client.1.smithi173.stdout:8/133: write d4/f19 [947811,96355] 0 2024-04-20T11:09:26.069 INFO:tasks.workunit.client.1.smithi173.stdout:9/128: rename d5/d9/d20/d28 to d5/d9/d20/d2b 0 2024-04-20T11:09:26.075 INFO:tasks.workunit.client.1.smithi173.stdout:1/91: mknod d5/d6/dc/c1d 0 2024-04-20T11:09:26.076 INFO:tasks.workunit.client.1.smithi173.stdout:8/134: rename d4/f20 to d4/d18/f24 0 2024-04-20T11:09:26.078 INFO:tasks.workunit.client.1.smithi173.stdout:8/135: chown d4/l13 919 1 2024-04-20T11:09:26.078 INFO:tasks.workunit.client.1.smithi173.stdout:3/129: mkdir d7/d9/d30/d31 0 2024-04-20T11:09:26.079 INFO:tasks.workunit.client.1.smithi173.stdout:8/136: chown d4/d18 175 1 2024-04-20T11:09:26.079 INFO:tasks.workunit.client.1.smithi173.stdout:8/137: readlink d4/l22 0 2024-04-20T11:09:26.080 INFO:tasks.workunit.client.1.smithi173.stdout:3/130: write d7/f18 [3737058,9740] 0 2024-04-20T11:09:26.080 INFO:tasks.workunit.client.1.smithi173.stdout:4/95: getdents da/d1e 0 2024-04-20T11:09:26.082 INFO:tasks.workunit.client.1.smithi173.stdout:6/92: truncate d4/fd 4099357 0 2024-04-20T11:09:26.082 INFO:tasks.workunit.client.1.smithi173.stdout:9/129: symlink d5/d9/d20/d29/l2c 0 2024-04-20T11:09:26.083 INFO:tasks.workunit.client.1.smithi173.stdout:8/138: write d4/f19 [1629806,19412] 0 2024-04-20T11:09:26.086 INFO:tasks.workunit.client.1.smithi173.stdout:4/96: truncate da/dc/fe 687018 0 2024-04-20T11:09:26.089 INFO:tasks.workunit.client.1.smithi173.stdout:9/130: write d5/d9/f13 [879047,105697] 0 2024-04-20T11:09:26.090 INFO:tasks.workunit.client.1.smithi173.stdout:3/131: mknod d7/d9/d30/d31/c32 0 2024-04-20T11:09:26.090 INFO:tasks.workunit.client.1.smithi173.stdout:8/139: unlink d4/d18/f1a 0 2024-04-20T11:09:26.092 INFO:tasks.workunit.client.1.smithi173.stdout:8/140: readlink d4/l12 0 2024-04-20T11:09:26.092 INFO:tasks.workunit.client.1.smithi173.stdout:4/97: write da/f12 [984910,113252] 0 2024-04-20T11:09:26.093 INFO:tasks.workunit.client.1.smithi173.stdout:8/141: chown d4/d18 6 1 2024-04-20T11:09:26.093 INFO:tasks.workunit.client.1.smithi173.stdout:3/132: mknod d7/d9/d30/c33 0 2024-04-20T11:09:26.094 INFO:tasks.workunit.client.1.smithi173.stdout:8/142: readlink d4/l13 0 2024-04-20T11:09:26.095 INFO:tasks.workunit.client.1.smithi173.stdout:3/133: truncate f4 4284663 0 2024-04-20T11:09:26.097 INFO:tasks.workunit.client.1.smithi173.stdout:3/134: dread - d7/f2d zero size 2024-04-20T11:09:26.097 INFO:tasks.workunit.client.1.smithi173.stdout:9/131: mkdir d5/d9/d20/d29/d2d 0 2024-04-20T11:09:26.098 INFO:tasks.workunit.client.1.smithi173.stdout:4/98: dread da/dc/ff [0,4194304] 0 2024-04-20T11:09:26.101 INFO:tasks.workunit.client.0.smithi117.stdout:8/965: dwrite d0/dc/d49/d63/fb5 [0,4194304] 0 2024-04-20T11:09:26.103 INFO:tasks.workunit.client.1.smithi173.stdout:3/135: read d7/d9/f13 [1478040,117672] 0 2024-04-20T11:09:26.103 INFO:tasks.workunit.client.0.smithi117.stdout:8/966: chown d0/d8/d39/d40/de5/d12a 112 1 2024-04-20T11:09:26.103 INFO:tasks.workunit.client.1.smithi173.stdout:9/132: fdatasync d5/f6 0 2024-04-20T11:09:26.104 INFO:tasks.workunit.client.1.smithi173.stdout:4/99: readlink da/dc/ld 0 2024-04-20T11:09:26.109 INFO:tasks.workunit.client.0.smithi117.stdout:8/967: chown d0/d8/d39/d40/d78/d5c/d123/caa 12 1 2024-04-20T11:09:26.109 INFO:tasks.workunit.client.0.smithi117.stdout:1/995: sync 2024-04-20T11:09:26.111 INFO:tasks.workunit.client.0.smithi117.stdout:1/996: stat d6/d1f/d30/d32/lb1 0 2024-04-20T11:09:26.111 INFO:tasks.workunit.client.0.smithi117.stdout:8/968: readlink d0/d8/d11/d24/d27/d99/le8 0 2024-04-20T11:09:26.111 INFO:tasks.workunit.client.1.smithi173.stdout:7/77: truncate f1 4428928 0 2024-04-20T11:09:26.116 INFO:tasks.workunit.client.1.smithi173.stdout:0/96: dwrite d6/fa [4194304,4194304] 0 2024-04-20T11:09:26.117 INFO:tasks.workunit.client.0.smithi117.stdout:1/997: rename d6/d35/c9d to d6/d16/d3a/d11e/c145 0 2024-04-20T11:09:26.118 INFO:tasks.workunit.client.1.smithi173.stdout:4/100: mknod da/dc/d15/c23 0 2024-04-20T11:09:26.121 INFO:tasks.workunit.client.1.smithi173.stdout:3/136: symlink d7/d2b/l34 0 2024-04-20T11:09:26.124 INFO:tasks.workunit.client.1.smithi173.stdout:6/93: truncate d4/f11 1978097 0 2024-04-20T11:09:26.124 INFO:tasks.workunit.client.0.smithi117.stdout:1/998: creat d6/d16/d3a/d5a/d5c/d61/d71/f146 x:0 0 0 2024-04-20T11:09:26.125 INFO:tasks.workunit.client.1.smithi173.stdout:6/94: chown d4/d14 43174 1 2024-04-20T11:09:26.128 INFO:tasks.workunit.client.1.smithi173.stdout:5/67: dwrite f6 [0,4194304] 0 2024-04-20T11:09:26.128 INFO:tasks.workunit.client.1.smithi173.stdout:1/92: read f4 [272433,88340] 0 2024-04-20T11:09:26.129 INFO:tasks.workunit.client.0.smithi117.stdout:1/999: chown d6/d16/f139 547349 1 2024-04-20T11:09:26.131 INFO:tasks.workunit.client.1.smithi173.stdout:0/97: creat d6/f1e x:0 0 0 2024-04-20T11:09:26.132 INFO:tasks.workunit.client.1.smithi173.stdout:4/101: rename da/dc/ff to da/dc/d15/f24 0 2024-04-20T11:09:26.134 INFO:tasks.workunit.client.1.smithi173.stdout:7/78: creat d9/da/d17/f1b x:0 0 0 2024-04-20T11:09:26.138 INFO:tasks.workunit.client.1.smithi173.stdout:5/68: symlink da/dd/l18 0 2024-04-20T11:09:26.141 INFO:tasks.workunit.client.1.smithi173.stdout:3/137: rename f3 to d7/d9/f35 0 2024-04-20T11:09:26.145 INFO:tasks.workunit.client.1.smithi173.stdout:2/83: sync 2024-04-20T11:09:26.147 INFO:tasks.workunit.client.1.smithi173.stdout:7/79: write d9/da/d17/f1b [845446,112510] 0 2024-04-20T11:09:26.151 INFO:tasks.workunit.client.0.smithi117.stdout:8/969: dwrite d0/d8/d11/f1f [0,4194304] 0 2024-04-20T11:09:26.153 INFO:tasks.workunit.client.1.smithi173.stdout:9/133: dwrite d5/d9/f27 [0,4194304] 0 2024-04-20T11:09:26.161 INFO:tasks.workunit.client.1.smithi173.stdout:3/138: write d7/f18 [1117693,97910] 0 2024-04-20T11:09:26.164 INFO:tasks.workunit.client.1.smithi173.stdout:5/69: dread f9 [0,4194304] 0 2024-04-20T11:09:26.173 INFO:tasks.workunit.client.1.smithi173.stdout:2/84: mknod d2/d13/c18 0 2024-04-20T11:09:26.173 INFO:tasks.workunit.client.1.smithi173.stdout:3/139: chown d7/d2b/l34 5 1 2024-04-20T11:09:26.177 INFO:tasks.workunit.client.1.smithi173.stdout:4/102: dread da/f12 [0,4194304] 0 2024-04-20T11:09:26.180 INFO:tasks.workunit.client.1.smithi173.stdout:9/134: rmdir d5/d9/d20/d29 39 2024-04-20T11:09:26.181 INFO:tasks.workunit.client.0.smithi117.stdout:8/970: dwrite d0/d8/d11/d24/d2b/f2c [0,4194304] 0 2024-04-20T11:09:26.182 INFO:tasks.workunit.client.1.smithi173.stdout:2/85: mknod d2/d14/c19 0 2024-04-20T11:09:26.183 INFO:tasks.workunit.client.0.smithi117.stdout:8/971: chown d0/d8/d39/d40/d78/d5c 5589 1 2024-04-20T11:09:26.183 INFO:tasks.workunit.client.1.smithi173.stdout:5/70: mkdir da/de/d19 0 2024-04-20T11:09:26.184 INFO:tasks.workunit.client.1.smithi173.stdout:8/143: dwrite d4/f17 [0,4194304] 0 2024-04-20T11:09:26.186 INFO:tasks.workunit.client.0.smithi117.stdout:8/972: mkdir d0/d8/d39/d40/d13c 0 2024-04-20T11:09:26.189 INFO:tasks.workunit.client.0.smithi117.stdout:8/973: symlink d0/d9/l13d 0 2024-04-20T11:09:26.197 INFO:tasks.workunit.client.1.smithi173.stdout:3/140: chown d7/d9/c21 116897 1 2024-04-20T11:09:26.198 INFO:tasks.workunit.client.1.smithi173.stdout:4/103: rename da/f12 to da/f25 0 2024-04-20T11:09:26.199 INFO:tasks.workunit.client.1.smithi173.stdout:5/71: chown f9 80325 1 2024-04-20T11:09:26.204 INFO:tasks.workunit.client.1.smithi173.stdout:3/141: creat d7/d2b/f36 x:0 0 0 2024-04-20T11:09:26.204 INFO:tasks.workunit.client.1.smithi173.stdout:8/144: mknod d4/c25 0 2024-04-20T11:09:26.206 INFO:tasks.workunit.client.1.smithi173.stdout:5/72: write f6 [3256117,12044] 0 2024-04-20T11:09:26.206 INFO:tasks.workunit.client.1.smithi173.stdout:9/135: symlink d5/d9/d20/d29/d2d/l2e 0 2024-04-20T11:09:26.207 INFO:tasks.workunit.client.1.smithi173.stdout:2/86: write d2/de/d11/fd [2257250,117599] 0 2024-04-20T11:09:26.210 INFO:tasks.workunit.client.0.smithi117.stdout:8/974: truncate d0/d8/d11/d24/d76/f96 1744060 0 2024-04-20T11:09:26.212 INFO:tasks.workunit.client.1.smithi173.stdout:8/145: rename d4 to d4/d23/d26 22 2024-04-20T11:09:26.214 INFO:tasks.workunit.client.1.smithi173.stdout:0/98: dwrite d6/d8/f19 [0,4194304] 0 2024-04-20T11:09:26.214 INFO:tasks.workunit.client.1.smithi173.stdout:7/80: dwrite d9/da/d17/f1a [0,4194304] 0 2024-04-20T11:09:26.216 INFO:tasks.workunit.client.1.smithi173.stdout:9/136: chown d5/l19 1 1 2024-04-20T11:09:26.219 INFO:tasks.workunit.client.1.smithi173.stdout:2/87: write d2/de/d11/fd [838065,60497] 0 2024-04-20T11:09:26.219 INFO:tasks.workunit.client.1.smithi173.stdout:8/146: fdatasync d4/f8 0 2024-04-20T11:09:26.220 INFO:tasks.workunit.client.1.smithi173.stdout:9/137: fdatasync d5/d9/f13 0 2024-04-20T11:09:26.222 INFO:tasks.workunit.client.1.smithi173.stdout:3/142: link d7/f15 d7/d9/d30/f37 0 2024-04-20T11:09:26.229 INFO:tasks.workunit.client.1.smithi173.stdout:2/88: fdatasync d2/de/d11/f10 0 2024-04-20T11:09:26.229 INFO:tasks.workunit.client.1.smithi173.stdout:3/143: stat d7/d17/c27 0 2024-04-20T11:09:26.229 INFO:tasks.workunit.client.1.smithi173.stdout:8/147: link f2 d4/f27 0 2024-04-20T11:09:26.236 INFO:tasks.workunit.client.1.smithi173.stdout:5/73: truncate f8 7661787 0 2024-04-20T11:09:26.241 INFO:tasks.workunit.client.1.smithi173.stdout:2/89: symlink d2/de/l1a 0 2024-04-20T11:09:26.249 INFO:tasks.workunit.client.1.smithi173.stdout:6/95: sync 2024-04-20T11:09:26.253 INFO:tasks.workunit.client.1.smithi173.stdout:0/99: truncate d6/fa 1479818 0 2024-04-20T11:09:26.255 INFO:tasks.workunit.client.1.smithi173.stdout:2/90: rename d2/de/d11/c8 to d2/de/c1b 0 2024-04-20T11:09:26.255 INFO:tasks.workunit.client.1.smithi173.stdout:7/81: dwrite d9/da/f10 [0,4194304] 0 2024-04-20T11:09:26.257 INFO:tasks.workunit.client.1.smithi173.stdout:3/144: dread d7/d17/f19 [0,4194304] 0 2024-04-20T11:09:26.258 INFO:tasks.workunit.client.1.smithi173.stdout:8/148: creat d4/f28 x:0 0 0 2024-04-20T11:09:26.260 INFO:tasks.workunit.client.1.smithi173.stdout:0/100: read d6/fd [1142473,95367] 0 2024-04-20T11:09:26.262 INFO:tasks.workunit.client.1.smithi173.stdout:9/138: dwrite d5/d9/f13 [0,4194304] 0 2024-04-20T11:09:26.263 INFO:tasks.workunit.client.1.smithi173.stdout:2/91: write d2/d14/f15 [2265958,56614] 0 2024-04-20T11:09:26.265 INFO:tasks.workunit.client.1.smithi173.stdout:0/101: write d6/f13 [767688,38902] 0 2024-04-20T11:09:26.266 INFO:tasks.workunit.client.1.smithi173.stdout:6/96: creat d4/f20 x:0 0 0 2024-04-20T11:09:26.269 INFO:tasks.workunit.client.1.smithi173.stdout:7/82: symlink d9/da/d17/l1c 0 2024-04-20T11:09:26.269 INFO:tasks.workunit.client.1.smithi173.stdout:5/74: dread f6 [0,4194304] 0 2024-04-20T11:09:26.270 INFO:tasks.workunit.client.1.smithi173.stdout:6/97: dread - d4/f20 zero size 2024-04-20T11:09:26.271 INFO:tasks.workunit.client.1.smithi173.stdout:3/145: creat d7/d9/d30/d31/f38 x:0 0 0 2024-04-20T11:09:26.277 INFO:tasks.workunit.client.1.smithi173.stdout:6/98: write d4/d18/f19 [914319,38815] 0 2024-04-20T11:09:26.279 INFO:tasks.workunit.client.1.smithi173.stdout:3/146: rename d7/c1b to d7/d9/d30/d31/c39 0 2024-04-20T11:09:26.284 INFO:tasks.workunit.client.1.smithi173.stdout:7/83: rmdir d9/d19 0 2024-04-20T11:09:26.285 INFO:tasks.workunit.client.1.smithi173.stdout:6/99: symlink d4/d18/l21 0 2024-04-20T11:09:26.286 INFO:tasks.workunit.client.1.smithi173.stdout:7/84: stat d9/f15 0 2024-04-20T11:09:26.287 INFO:tasks.workunit.client.1.smithi173.stdout:5/75: symlink da/de/d19/l1a 0 2024-04-20T11:09:26.288 INFO:tasks.workunit.client.1.smithi173.stdout:9/139: dread d5/d9/fa [0,4194304] 0 2024-04-20T11:09:26.290 INFO:tasks.workunit.client.1.smithi173.stdout:8/149: dread f2 [0,4194304] 0 2024-04-20T11:09:26.291 INFO:tasks.workunit.client.1.smithi173.stdout:9/140: read d5/d9/f26 [131469,64084] 0 2024-04-20T11:09:26.292 INFO:tasks.workunit.client.1.smithi173.stdout:6/100: creat d4/d14/f22 x:0 0 0 2024-04-20T11:09:26.294 INFO:tasks.workunit.client.1.smithi173.stdout:4/104: sync 2024-04-20T11:09:26.294 INFO:tasks.workunit.client.1.smithi173.stdout:1/93: sync 2024-04-20T11:09:26.299 INFO:tasks.workunit.client.1.smithi173.stdout:7/85: creat d9/f1d x:0 0 0 2024-04-20T11:09:26.299 INFO:tasks.workunit.client.1.smithi173.stdout:5/76: dread f9 [0,4194304] 0 2024-04-20T11:09:26.305 INFO:tasks.workunit.client.1.smithi173.stdout:2/92: dwrite d2/de/d11/fd [0,4194304] 0 2024-04-20T11:09:26.305 INFO:tasks.workunit.client.1.smithi173.stdout:5/77: chown da/d12/l13 2813 1 2024-04-20T11:09:26.312 INFO:tasks.workunit.client.1.smithi173.stdout:6/101: getdents d4/d14 0 2024-04-20T11:09:26.312 INFO:tasks.workunit.client.1.smithi173.stdout:7/86: mknod d9/da/d17/c1e 0 2024-04-20T11:09:26.315 INFO:tasks.workunit.client.1.smithi173.stdout:1/94: symlink d5/d7/d12/d1c/l1e 0 2024-04-20T11:09:26.320 INFO:tasks.workunit.client.1.smithi173.stdout:3/147: dwrite d7/f22 [0,4194304] 0 2024-04-20T11:09:26.320 INFO:tasks.workunit.client.1.smithi173.stdout:5/78: creat da/dd/f1b x:0 0 0 2024-04-20T11:09:26.325 INFO:tasks.workunit.client.1.smithi173.stdout:7/87: chown d9/da/d17 9 1 2024-04-20T11:09:26.325 INFO:tasks.workunit.client.1.smithi173.stdout:9/141: rename d5/d9/d20/d29/f10 to d5/f2f 0 2024-04-20T11:09:26.325 INFO:tasks.workunit.client.1.smithi173.stdout:6/102: truncate d4/d18/f19 1589668 0 2024-04-20T11:09:26.327 INFO:tasks.workunit.client.1.smithi173.stdout:1/95: mkdir d5/d7/d12/d1f 0 2024-04-20T11:09:26.332 INFO:tasks.workunit.client.1.smithi173.stdout:7/88: readlink d9/da/d17/l1c 0 2024-04-20T11:09:26.332 INFO:tasks.workunit.client.1.smithi173.stdout:8/150: dwrite d4/f28 [0,4194304] 0 2024-04-20T11:09:26.332 INFO:tasks.workunit.client.1.smithi173.stdout:0/102: dwrite d6/fa [0,4194304] 0 2024-04-20T11:09:26.335 INFO:tasks.workunit.client.1.smithi173.stdout:8/151: chown d4/c14 1 1 2024-04-20T11:09:26.336 INFO:tasks.workunit.client.1.smithi173.stdout:3/148: fdatasync d7/d9/d30/f37 0 2024-04-20T11:09:26.362 INFO:tasks.workunit.client.1.smithi173.stdout:5/79: dwrite da/fc [0,4194304] 0 2024-04-20T11:09:26.362 INFO:tasks.workunit.client.1.smithi173.stdout:2/93: truncate d2/d14/f15 1030917 0 2024-04-20T11:09:26.365 INFO:tasks.workunit.client.1.smithi173.stdout:3/149: creat d7/d9/f3a x:0 0 0 2024-04-20T11:09:26.368 INFO:tasks.workunit.client.1.smithi173.stdout:2/94: readlink d2/la 0 2024-04-20T11:09:26.369 INFO:tasks.workunit.client.1.smithi173.stdout:7/89: write f1 [4709915,122038] 0 2024-04-20T11:09:26.374 INFO:tasks.workunit.client.1.smithi173.stdout:0/103: truncate d6/d8/f19 3245800 0 2024-04-20T11:09:26.379 INFO:tasks.workunit.client.1.smithi173.stdout:8/152: dread d4/d18/f1f [0,4194304] 0 2024-04-20T11:09:26.380 INFO:tasks.workunit.client.1.smithi173.stdout:7/90: fsync d9/f15 0 2024-04-20T11:09:26.382 INFO:tasks.workunit.client.1.smithi173.stdout:0/104: creat d6/d8/f1f x:0 0 0 2024-04-20T11:09:26.383 INFO:tasks.workunit.client.0.smithi117.stdout:8/975: sync 2024-04-20T11:09:26.386 INFO:tasks.workunit.client.1.smithi173.stdout:6/103: dread d4/f13 [0,4194304] 0 2024-04-20T11:09:26.387 INFO:tasks.workunit.client.0.smithi117.stdout:8/976: rename d0/dc/d49/d5b/cb2 to d0/d8/d11/d24/d86/c13e 0 2024-04-20T11:09:26.387 INFO:tasks.workunit.client.1.smithi173.stdout:7/91: chown f1 1 1 2024-04-20T11:09:26.387 INFO:tasks.workunit.client.0.smithi117.stdout:8/977: readlink d0/la7 0 2024-04-20T11:09:26.388 INFO:tasks.workunit.client.1.smithi173.stdout:9/142: dwrite d5/d9/fa [0,4194304] 0 2024-04-20T11:09:26.390 INFO:tasks.workunit.client.1.smithi173.stdout:1/96: dwrite d5/d7/da/f14 [0,4194304] 0 2024-04-20T11:09:26.391 INFO:tasks.workunit.client.1.smithi173.stdout:2/95: truncate d2/de/d11/f10 1963597 0 2024-04-20T11:09:26.393 INFO:tasks.workunit.client.1.smithi173.stdout:0/105: symlink d6/d8/l20 0 2024-04-20T11:09:26.395 INFO:tasks.workunit.client.1.smithi173.stdout:6/104: mknod d4/d14/c23 0 2024-04-20T11:09:26.399 INFO:tasks.workunit.client.1.smithi173.stdout:1/97: read - d5/d6/f9 zero size 2024-04-20T11:09:26.400 INFO:tasks.workunit.client.1.smithi173.stdout:8/153: truncate d4/d18/f21 684246 0 2024-04-20T11:09:26.401 INFO:tasks.workunit.client.1.smithi173.stdout:7/92: mkdir d9/d1f 0 2024-04-20T11:09:26.409 INFO:tasks.workunit.client.1.smithi173.stdout:2/96: truncate d2/de/d11/fd 4542875 0 2024-04-20T11:09:26.410 INFO:tasks.workunit.client.1.smithi173.stdout:7/93: dread - d9/f1d zero size 2024-04-20T11:09:26.412 INFO:tasks.workunit.client.1.smithi173.stdout:6/105: truncate d4/fd 2207990 0 2024-04-20T11:09:26.413 INFO:tasks.workunit.client.1.smithi173.stdout:1/98: mkdir d5/d7/da/d20 0 2024-04-20T11:09:26.414 INFO:tasks.workunit.client.1.smithi173.stdout:7/94: write f1 [4274027,47248] 0 2024-04-20T11:09:26.416 INFO:tasks.workunit.client.1.smithi173.stdout:4/105: sync 2024-04-20T11:09:26.416 INFO:tasks.workunit.client.1.smithi173.stdout:2/97: write d2/de/d11/fd [4140092,24348] 0 2024-04-20T11:09:26.416 INFO:tasks.workunit.client.1.smithi173.stdout:3/150: dwrite d7/d9/f3a [0,4194304] 0 2024-04-20T11:09:26.417 INFO:tasks.workunit.client.1.smithi173.stdout:0/106: rename d6/d8/l1c to d6/d8/df/l21 0 2024-04-20T11:09:26.420 INFO:tasks.workunit.client.1.smithi173.stdout:6/106: unlink c2 0 2024-04-20T11:09:26.420 INFO:tasks.workunit.client.1.smithi173.stdout:5/80: dwrite da/dd/f11 [0,4194304] 0 2024-04-20T11:09:26.423 INFO:tasks.workunit.client.1.smithi173.stdout:0/107: dread - d6/f1e zero size 2024-04-20T11:09:26.427 INFO:tasks.workunit.client.0.smithi117.stdout:8/978: dread d0/d8/d11/d24/d27/d99/f9d [0,4194304] 0 2024-04-20T11:09:26.430 INFO:tasks.workunit.client.1.smithi173.stdout:2/98: read d2/de/d11/fd [2967709,869] 0 2024-04-20T11:09:26.430 INFO:tasks.workunit.client.1.smithi173.stdout:4/106: rename l6 to da/d1e/l26 0 2024-04-20T11:09:26.432 INFO:tasks.workunit.client.1.smithi173.stdout:7/95: read d9/da/ff [23445,55293] 0 2024-04-20T11:09:26.433 INFO:tasks.workunit.client.0.smithi117.stdout:8/979: chown d0/d9/d3a/d7c/dce/l122 8346952 1 2024-04-20T11:09:26.434 INFO:tasks.workunit.client.1.smithi173.stdout:3/151: write d7/d17/f29 [896391,19205] 0 2024-04-20T11:09:26.435 INFO:tasks.workunit.client.1.smithi173.stdout:5/81: read - da/de/f10 zero size 2024-04-20T11:09:26.436 INFO:tasks.workunit.client.1.smithi173.stdout:9/143: write d5/d9/f26 [482898,53296] 0 2024-04-20T11:09:26.437 INFO:tasks.workunit.client.1.smithi173.stdout:2/99: symlink d2/de/d11/l1c 0 2024-04-20T11:09:26.438 INFO:tasks.workunit.client.1.smithi173.stdout:0/108: chown d6/d8/df/l17 1753633747 1 2024-04-20T11:09:26.439 INFO:tasks.workunit.client.1.smithi173.stdout:4/107: creat da/dc/f27 x:0 0 0 2024-04-20T11:09:26.440 INFO:tasks.workunit.client.1.smithi173.stdout:7/96: mkdir d9/da/d20 0 2024-04-20T11:09:26.442 INFO:tasks.workunit.client.1.smithi173.stdout:8/154: dwrite d4/f8 [4194304,4194304] 0 2024-04-20T11:09:26.450 INFO:tasks.workunit.client.1.smithi173.stdout:7/97: write d9/da/f12 [766141,110259] 0 2024-04-20T11:09:26.450 INFO:tasks.workunit.client.1.smithi173.stdout:5/82: dread f9 [0,4194304] 0 2024-04-20T11:09:26.451 INFO:tasks.workunit.client.1.smithi173.stdout:1/99: dread d5/d6/dc/fe [0,4194304] 0 2024-04-20T11:09:26.451 INFO:tasks.workunit.client.1.smithi173.stdout:9/144: mknod d5/c30 0 2024-04-20T11:09:26.453 INFO:tasks.workunit.client.1.smithi173.stdout:3/152: rename d7/d9/d30/d31 to d7/d9/d3b 0 2024-04-20T11:09:26.453 INFO:tasks.workunit.client.1.smithi173.stdout:4/108: write da/f25 [356272,109704] 0 2024-04-20T11:09:26.454 INFO:tasks.workunit.client.1.smithi173.stdout:8/155: creat d4/d18/f29 x:0 0 0 2024-04-20T11:09:26.455 INFO:tasks.workunit.client.1.smithi173.stdout:5/83: rename da to da/de/d19/d1c 22 2024-04-20T11:09:26.456 INFO:tasks.workunit.client.1.smithi173.stdout:1/100: write d5/fb [2103396,27009] 0 2024-04-20T11:09:26.467 INFO:tasks.workunit.client.1.smithi173.stdout:4/109: unlink da/dc/d15/f24 0 2024-04-20T11:09:26.468 INFO:tasks.workunit.client.1.smithi173.stdout:2/100: dread d2/de/d11/fd [0,4194304] 0 2024-04-20T11:09:26.468 INFO:tasks.workunit.client.1.smithi173.stdout:0/109: dread d6/d8/f19 [0,4194304] 0 2024-04-20T11:09:26.469 INFO:tasks.workunit.client.1.smithi173.stdout:7/98: creat d9/f21 x:0 0 0 2024-04-20T11:09:26.469 INFO:tasks.workunit.client.1.smithi173.stdout:4/110: readlink da/dc/l17 0 2024-04-20T11:09:26.472 INFO:tasks.workunit.client.1.smithi173.stdout:5/84: rename da/dd/f1b to da/d12/f1d 0 2024-04-20T11:09:26.477 INFO:tasks.workunit.client.0.smithi117.stdout:8/980: dread d0/d8/d11/d24/d27/d47/f8e [0,4194304] 0 2024-04-20T11:09:26.477 INFO:tasks.workunit.client.1.smithi173.stdout:6/107: write d4/f11 [1126686,123544] 0 2024-04-20T11:09:26.479 INFO:tasks.workunit.client.0.smithi117.stdout:8/981: fdatasync d0/d8/d39/f54 0 2024-04-20T11:09:26.480 INFO:tasks.workunit.client.1.smithi173.stdout:7/99: dread d9/da/d17/f1b [0,4194304] 0 2024-04-20T11:09:26.480 INFO:tasks.workunit.client.1.smithi173.stdout:3/153: dwrite d7/f2d [0,4194304] 0 2024-04-20T11:09:26.480 INFO:tasks.workunit.client.1.smithi173.stdout:2/101: write d2/de/d11/fd [4223860,20908] 0 2024-04-20T11:09:26.482 INFO:tasks.workunit.client.1.smithi173.stdout:2/102: chown d2/d13/c18 474 1 2024-04-20T11:09:26.483 INFO:tasks.workunit.client.1.smithi173.stdout:0/110: creat d6/d8/df/d15/f22 x:0 0 0 2024-04-20T11:09:26.486 INFO:tasks.workunit.client.1.smithi173.stdout:9/145: write d5/d9/f1d [1461335,64819] 0 2024-04-20T11:09:26.487 INFO:tasks.workunit.client.0.smithi117.stdout:8/982: rmdir d0/d8/d11/d24/d76 39 2024-04-20T11:09:26.488 INFO:tasks.workunit.client.1.smithi173.stdout:8/156: link l3 d4/d18/l2a 0 2024-04-20T11:09:26.491 INFO:tasks.workunit.client.1.smithi173.stdout:1/101: creat d5/d6/f21 x:0 0 0 2024-04-20T11:09:26.496 INFO:tasks.workunit.client.1.smithi173.stdout:1/102: dread f4 [0,4194304] 0 2024-04-20T11:09:26.497 INFO:tasks.workunit.client.0.smithi117.stdout:8/983: chown d0/d9/d3a/de3/df1/l125 12179096 1 2024-04-20T11:09:26.497 INFO:tasks.workunit.client.0.smithi117.stdout:8/984: chown d0/la7 3031279 1 2024-04-20T11:09:26.499 INFO:tasks.workunit.client.0.smithi117.stdout:8/985: fsync d0/d8/d39/d40/d78/d5c/f68 0 2024-04-20T11:09:26.499 INFO:tasks.workunit.client.1.smithi173.stdout:0/111: unlink d6/fa 0 2024-04-20T11:09:26.501 INFO:tasks.workunit.client.1.smithi173.stdout:2/103: chown d2/d14/f15 202 1 2024-04-20T11:09:26.503 INFO:tasks.workunit.client.1.smithi173.stdout:7/100: mknod d9/d1f/c22 0 2024-04-20T11:09:26.504 INFO:tasks.workunit.client.0.smithi117.stdout:8/986: creat d0/d8/d39/d40/de5/f13f x:0 0 0 2024-04-20T11:09:26.505 INFO:tasks.workunit.client.1.smithi173.stdout:4/111: dread f4 [0,4194304] 0 2024-04-20T11:09:26.508 INFO:tasks.workunit.client.1.smithi173.stdout:2/104: creat d2/de/d11/f1d x:0 0 0 2024-04-20T11:09:26.509 INFO:tasks.workunit.client.1.smithi173.stdout:9/146: getdents d5/d9/d20 0 2024-04-20T11:09:26.510 INFO:tasks.workunit.client.1.smithi173.stdout:4/112: write da/dc/d15/f1d [451618,108868] 0 2024-04-20T11:09:26.511 INFO:tasks.workunit.client.1.smithi173.stdout:8/157: rename d4/d18/l2a to d4/l2b 0 2024-04-20T11:09:26.514 INFO:tasks.workunit.client.1.smithi173.stdout:9/147: mkdir d5/d9/d20/d2b/d31 0 2024-04-20T11:09:26.516 INFO:tasks.workunit.client.1.smithi173.stdout:4/113: write da/dc/f27 [218294,67741] 0 2024-04-20T11:09:26.518 INFO:tasks.workunit.client.1.smithi173.stdout:2/105: link d2/la d2/de/l1e 0 2024-04-20T11:09:26.519 INFO:tasks.workunit.client.1.smithi173.stdout:3/154: dwrite d7/d9/f35 [0,4194304] 0 2024-04-20T11:09:26.519 INFO:tasks.workunit.client.1.smithi173.stdout:7/101: truncate d9/da/f10 1774993 0 2024-04-20T11:09:26.520 INFO:tasks.workunit.client.1.smithi173.stdout:2/106: chown d2/d13 6383530 1 2024-04-20T11:09:26.521 INFO:tasks.workunit.client.1.smithi173.stdout:4/114: stat f1 0 2024-04-20T11:09:26.522 INFO:tasks.workunit.client.1.smithi173.stdout:7/102: chown f1 1036194 1 2024-04-20T11:09:26.522 INFO:tasks.workunit.client.1.smithi173.stdout:4/115: chown c3 0 1 2024-04-20T11:09:26.523 INFO:tasks.workunit.client.1.smithi173.stdout:8/158: rename d4/c15 to d4/c2c 0 2024-04-20T11:09:26.524 INFO:tasks.workunit.client.1.smithi173.stdout:2/107: read d2/d14/f15 [472072,35280] 0 2024-04-20T11:09:26.526 INFO:tasks.workunit.client.1.smithi173.stdout:3/155: rmdir d7/d9/d30 39 2024-04-20T11:09:26.528 INFO:tasks.workunit.client.1.smithi173.stdout:2/108: write d2/de/d11/fd [2500650,1745] 0 2024-04-20T11:09:26.528 INFO:tasks.workunit.client.1.smithi173.stdout:4/116: write f4 [1523244,117353] 0 2024-04-20T11:09:26.529 INFO:tasks.workunit.client.1.smithi173.stdout:6/108: dwrite d4/d14/f1a [0,4194304] 0 2024-04-20T11:09:26.531 INFO:tasks.workunit.client.1.smithi173.stdout:7/103: mknod d9/da/d20/c23 0 2024-04-20T11:09:26.533 INFO:tasks.workunit.client.1.smithi173.stdout:7/104: write d9/f15 [861509,51328] 0 2024-04-20T11:09:26.533 INFO:tasks.workunit.client.1.smithi173.stdout:9/148: dread d5/d9/f27 [0,4194304] 0 2024-04-20T11:09:26.536 INFO:tasks.workunit.client.1.smithi173.stdout:6/109: chown d4/f16 1 1 2024-04-20T11:09:26.536 INFO:tasks.workunit.client.1.smithi173.stdout:9/149: read d5/d9/f26 [167991,64796] 0 2024-04-20T11:09:26.538 INFO:tasks.workunit.client.1.smithi173.stdout:7/105: mknod d9/da/d17/c24 0 2024-04-20T11:09:26.539 INFO:tasks.workunit.client.1.smithi173.stdout:9/150: dread - d5/d9/f1a zero size 2024-04-20T11:09:26.540 INFO:tasks.workunit.client.1.smithi173.stdout:7/106: chown d9/da/d20 2 1 2024-04-20T11:09:26.541 INFO:tasks.workunit.client.1.smithi173.stdout:8/159: rmdir d4/d23 0 2024-04-20T11:09:26.542 INFO:tasks.workunit.client.1.smithi173.stdout:6/110: mkdir d4/d24 0 2024-04-20T11:09:26.544 INFO:tasks.workunit.client.1.smithi173.stdout:7/107: write d9/da/f14 [3612835,47707] 0 2024-04-20T11:09:26.544 INFO:tasks.workunit.client.1.smithi173.stdout:2/109: creat d2/f1f x:0 0 0 2024-04-20T11:09:26.545 INFO:tasks.workunit.client.1.smithi173.stdout:6/111: write d4/f20 [106056,44352] 0 2024-04-20T11:09:26.545 INFO:tasks.workunit.client.1.smithi173.stdout:3/156: rename d7/d9/d30/f37 to d7/d17/f3c 0 2024-04-20T11:09:26.547 INFO:tasks.workunit.client.1.smithi173.stdout:2/110: dread - d2/f1f zero size 2024-04-20T11:09:26.548 INFO:tasks.workunit.client.1.smithi173.stdout:9/151: mkdir d5/d9/d32 0 2024-04-20T11:09:26.559 INFO:tasks.workunit.client.1.smithi173.stdout:3/157: creat d7/d2b/f3d x:0 0 0 2024-04-20T11:09:26.560 INFO:tasks.workunit.client.1.smithi173.stdout:4/117: dwrite da/dc/d15/f1c [0,4194304] 0 2024-04-20T11:09:26.564 INFO:tasks.workunit.client.1.smithi173.stdout:6/112: creat d4/f25 x:0 0 0 2024-04-20T11:09:26.566 INFO:tasks.workunit.client.1.smithi173.stdout:7/108: rmdir d9 39 2024-04-20T11:09:26.568 INFO:tasks.workunit.client.1.smithi173.stdout:3/158: rename d7/d9/la to d7/l3e 0 2024-04-20T11:09:26.571 INFO:tasks.workunit.client.1.smithi173.stdout:4/118: read da/dc/d15/f1c [1650095,102023] 0 2024-04-20T11:09:26.577 INFO:tasks.workunit.client.1.smithi173.stdout:7/109: creat d9/da/f25 x:0 0 0 2024-04-20T11:09:26.579 INFO:tasks.workunit.client.1.smithi173.stdout:8/160: dwrite d4/f28 [4194304,4194304] 0 2024-04-20T11:09:26.579 INFO:tasks.workunit.client.1.smithi173.stdout:4/119: dread da/dc/f27 [0,4194304] 0 2024-04-20T11:09:26.581 INFO:tasks.workunit.client.1.smithi173.stdout:2/111: write d2/d14/f15 [564710,118215] 0 2024-04-20T11:09:26.584 INFO:tasks.workunit.client.1.smithi173.stdout:3/159: creat d7/d9/f3f x:0 0 0 2024-04-20T11:09:26.584 INFO:tasks.workunit.client.1.smithi173.stdout:4/120: chown l9 2 1 2024-04-20T11:09:26.588 INFO:tasks.workunit.client.1.smithi173.stdout:8/161: write d4/d18/f24 [509541,15860] 0 2024-04-20T11:09:26.588 INFO:tasks.workunit.client.1.smithi173.stdout:9/152: dwrite d5/d9/f1d [0,4194304] 0 2024-04-20T11:09:26.589 INFO:tasks.workunit.client.1.smithi173.stdout:6/113: dread d4/d18/f19 [0,4194304] 0 2024-04-20T11:09:26.591 INFO:tasks.workunit.client.1.smithi173.stdout:8/162: fsync d4/fd 0 2024-04-20T11:09:26.592 INFO:tasks.workunit.client.1.smithi173.stdout:3/160: readlink d7/l3e 0 2024-04-20T11:09:26.593 INFO:tasks.workunit.client.1.smithi173.stdout:2/112: mknod d2/d14/c20 0 2024-04-20T11:09:26.595 INFO:tasks.workunit.client.1.smithi173.stdout:8/163: unlink d4/d18/f1f 0 2024-04-20T11:09:26.600 INFO:tasks.workunit.client.1.smithi173.stdout:2/113: dread d2/d14/f15 [0,4194304] 0 2024-04-20T11:09:26.603 INFO:tasks.workunit.client.1.smithi173.stdout:3/161: symlink d7/d9/l40 0 2024-04-20T11:09:26.608 INFO:tasks.workunit.client.1.smithi173.stdout:2/114: creat d2/de/f21 x:0 0 0 2024-04-20T11:09:26.608 INFO:tasks.workunit.client.1.smithi173.stdout:7/110: truncate f1 3180928 0 2024-04-20T11:09:26.609 INFO:tasks.workunit.client.1.smithi173.stdout:2/115: write d2/f1f [469710,46973] 0 2024-04-20T11:09:26.609 INFO:tasks.workunit.client.1.smithi173.stdout:9/153: dread d5/d9/d20/d29/f11 [0,4194304] 0 2024-04-20T11:09:26.610 INFO:tasks.workunit.client.1.smithi173.stdout:3/162: link d7/d9/f3f d7/d2b/f41 0 2024-04-20T11:09:26.614 INFO:tasks.workunit.client.1.smithi173.stdout:6/114: truncate d4/f11 1845086 0 2024-04-20T11:09:26.615 INFO:tasks.workunit.client.1.smithi173.stdout:2/116: symlink d2/de/l22 0 2024-04-20T11:09:26.615 INFO:tasks.workunit.client.1.smithi173.stdout:7/111: rename d9/d1f/c22 to d9/da/d20/c26 0 2024-04-20T11:09:26.616 INFO:tasks.workunit.client.1.smithi173.stdout:9/154: mkdir d5/d9/d20/d29/d2d/d33 0 2024-04-20T11:09:26.618 INFO:tasks.workunit.client.1.smithi173.stdout:5/85: sync 2024-04-20T11:09:26.619 INFO:tasks.workunit.client.1.smithi173.stdout:6/115: truncate d4/f25 165839 0 2024-04-20T11:09:26.619 INFO:tasks.workunit.client.1.smithi173.stdout:2/117: fdatasync d2/de/d11/f10 0 2024-04-20T11:09:26.620 INFO:tasks.workunit.client.1.smithi173.stdout:8/164: dwrite d4/fd [4194304,4194304] 0 2024-04-20T11:09:26.620 INFO:tasks.workunit.client.1.smithi173.stdout:7/112: creat d9/da/f27 x:0 0 0 2024-04-20T11:09:26.620 INFO:tasks.workunit.client.1.smithi173.stdout:5/86: chown da/de 1891 1 2024-04-20T11:09:26.623 INFO:tasks.workunit.client.1.smithi173.stdout:9/155: truncate d5/d9/f26 948749 0 2024-04-20T11:09:26.626 INFO:tasks.workunit.client.1.smithi173.stdout:7/113: creat d9/d1f/f28 x:0 0 0 2024-04-20T11:09:26.628 INFO:tasks.workunit.client.1.smithi173.stdout:8/165: unlink d4/f27 0 2024-04-20T11:09:26.644 INFO:tasks.workunit.client.0.smithi117.stdout:8/987: sync 2024-04-20T11:09:26.648 INFO:tasks.workunit.client.0.smithi117.stdout:8/988: getdents d0/d9/d118 0 2024-04-20T11:09:26.650 INFO:tasks.workunit.client.0.smithi117.stdout:8/989: symlink d0/d8/d11/d24/l140 0 2024-04-20T11:09:26.654 INFO:tasks.workunit.client.0.smithi117.stdout:8/990: fsync d0/d8/d11/d24/d76/f96 0 2024-04-20T11:09:26.658 INFO:tasks.workunit.client.0.smithi117.stdout:8/991: symlink d0/d8/d11/d24/d2b/d72/l141 0 2024-04-20T11:09:26.660 INFO:tasks.workunit.client.0.smithi117.stdout:8/992: rmdir d0/d9/dbb 39 2024-04-20T11:09:26.663 INFO:tasks.workunit.client.0.smithi117.stdout:8/993: mknod d0/dc/d49/d134/c142 0 2024-04-20T11:09:26.677 INFO:tasks.workunit.client.0.smithi117.stdout:8/994: dwrite d0/dc/d49/dd5/fdb [0,4194304] 0 2024-04-20T11:09:26.679 INFO:tasks.workunit.client.1.smithi173.stdout:4/121: dwrite f8 [0,4194304] 0 2024-04-20T11:09:26.685 INFO:tasks.workunit.client.1.smithi173.stdout:4/122: chown da/dc/d15 546790752 1 2024-04-20T11:09:26.685 INFO:tasks.workunit.client.0.smithi117.stdout:8/995: truncate d0/d8/d39/d40/d78/d5c/f68 2183407 0 2024-04-20T11:09:26.690 INFO:tasks.workunit.client.1.smithi173.stdout:8/166: symlink d4/l2d 0 2024-04-20T11:09:26.692 INFO:tasks.workunit.client.1.smithi173.stdout:4/123: chown da/c11 34385991 1 2024-04-20T11:09:26.693 INFO:tasks.workunit.client.0.smithi117.stdout:8/996: rename d0/d8/d39/d40/d78/f9b to d0/dc/d49/f143 0 2024-04-20T11:09:26.694 INFO:tasks.workunit.client.1.smithi173.stdout:3/163: dwrite d7/d2b/f36 [0,4194304] 0 2024-04-20T11:09:26.699 INFO:tasks.workunit.client.1.smithi173.stdout:4/124: fsync da/dc/fe 0 2024-04-20T11:09:26.699 INFO:tasks.workunit.client.1.smithi173.stdout:3/164: chown f4 20801 1 2024-04-20T11:09:26.699 INFO:tasks.workunit.client.1.smithi173.stdout:6/116: dread d4/f16 [0,4194304] 0 2024-04-20T11:09:26.700 INFO:tasks.workunit.client.1.smithi173.stdout:8/167: mkdir d4/d2e 0 2024-04-20T11:09:26.704 INFO:tasks.workunit.client.1.smithi173.stdout:2/118: dwrite d2/de/d11/f1d [0,4194304] 0 2024-04-20T11:09:26.705 INFO:tasks.workunit.client.1.smithi173.stdout:4/125: symlink da/d1a/l28 0 2024-04-20T11:09:26.706 INFO:tasks.workunit.client.1.smithi173.stdout:9/156: dread d5/d9/f25 [0,4194304] 0 2024-04-20T11:09:26.706 INFO:tasks.workunit.client.1.smithi173.stdout:3/165: mknod d7/d17/c42 0 2024-04-20T11:09:26.707 INFO:tasks.workunit.client.1.smithi173.stdout:9/157: chown d5 2494 1 2024-04-20T11:09:26.708 INFO:tasks.workunit.client.1.smithi173.stdout:4/126: fsync da/dc/d15/f18 0 2024-04-20T11:09:26.709 INFO:tasks.workunit.client.1.smithi173.stdout:8/168: rename d4/c25 to d4/d18/c2f 0 2024-04-20T11:09:26.710 INFO:tasks.workunit.client.1.smithi173.stdout:3/166: rmdir d7/d9/d3b 39 2024-04-20T11:09:26.716 INFO:tasks.workunit.client.1.smithi173.stdout:4/127: creat da/dc/d13/f29 x:0 0 0 2024-04-20T11:09:26.722 INFO:tasks.workunit.client.1.smithi173.stdout:2/119: rename d2/f1f to d2/de/d11/f23 0 2024-04-20T11:09:26.722 INFO:tasks.workunit.client.1.smithi173.stdout:9/158: dread d5/d9/f1d [0,4194304] 0 2024-04-20T11:09:26.723 INFO:tasks.workunit.client.1.smithi173.stdout:1/103: sync 2024-04-20T11:09:26.723 INFO:tasks.workunit.client.1.smithi173.stdout:0/112: sync 2024-04-20T11:09:26.724 INFO:tasks.workunit.client.0.smithi117.stdout:8/997: truncate d0/d8/d11/d24/d2b/fab 944048 0 2024-04-20T11:09:26.725 INFO:tasks.workunit.client.1.smithi173.stdout:8/169: fdatasync d4/f17 0 2024-04-20T11:09:26.726 INFO:tasks.workunit.client.0.smithi117.stdout:8/998: rmdir d0 39 2024-04-20T11:09:26.730 INFO:tasks.workunit.client.1.smithi173.stdout:2/120: creat d2/de/f24 x:0 0 0 2024-04-20T11:09:26.731 INFO:tasks.workunit.client.1.smithi173.stdout:3/167: dwrite d7/d9/f3a [4194304,4194304] 0 2024-04-20T11:09:26.735 INFO:tasks.workunit.client.0.smithi117.stdout:8/999: mknod d0/dc/d2e/d89/c144 0 2024-04-20T11:09:26.737 INFO:tasks.workunit.client.1.smithi173.stdout:1/104: fdatasync d5/d6/dc/fe 0 2024-04-20T11:09:26.738 INFO:tasks.workunit.client.1.smithi173.stdout:9/159: read d5/f2f [209068,54999] 0 2024-04-20T11:09:26.738 INFO:tasks.workunit.client.1.smithi173.stdout:2/121: dread - d2/de/f24 zero size 2024-04-20T11:09:26.740 INFO:tasks.workunit.client.1.smithi173.stdout:9/160: chown d5/d9/fa 317 1 2024-04-20T11:09:26.745 INFO:tasks.workunit.client.0.smithi117.stderr:+ rm -rf -- ./tmp.akUEe7qXEm 2024-04-20T11:09:26.745 INFO:tasks.workunit.client.1.smithi173.stdout:4/128: truncate f4 3482330 0 2024-04-20T11:09:26.746 INFO:tasks.workunit.client.1.smithi173.stdout:8/170: dread d4/f10 [0,4194304] 0 2024-04-20T11:09:26.747 INFO:tasks.workunit.client.1.smithi173.stdout:4/129: readlink da/d1e/l20 0 2024-04-20T11:09:26.748 INFO:tasks.workunit.client.1.smithi173.stdout:4/130: read da/dc/d15/f1c [3916030,13630] 0 2024-04-20T11:09:26.751 INFO:tasks.workunit.client.1.smithi173.stdout:4/131: write da/dc/d15/f1d [1374315,114918] 0 2024-04-20T11:09:26.762 INFO:tasks.workunit.client.1.smithi173.stdout:4/132: dread da/dc/d15/f18 [0,4194304] 0 2024-04-20T11:09:26.788 INFO:tasks.workunit.client.1.smithi173.stdout:0/113: symlink d6/d8/df/l23 0 2024-04-20T11:09:26.790 INFO:tasks.workunit.client.1.smithi173.stdout:1/105: mknod d5/d7/d12/d1c/c22 0 2024-04-20T11:09:26.791 INFO:tasks.workunit.client.1.smithi173.stdout:8/171: readlink d4/l2b 0 2024-04-20T11:09:26.796 INFO:tasks.workunit.client.1.smithi173.stdout:8/172: unlink d4/d18/f29 0 2024-04-20T11:09:26.798 INFO:tasks.workunit.client.1.smithi173.stdout:3/168: link d7/f2d d7/d9/f43 0 2024-04-20T11:09:26.800 INFO:tasks.workunit.client.1.smithi173.stdout:1/106: symlink d5/d6/dc/l23 0 2024-04-20T11:09:26.802 INFO:tasks.workunit.client.1.smithi173.stdout:8/173: dread d4/d18/f24 [0,4194304] 0 2024-04-20T11:09:26.806 INFO:tasks.workunit.client.1.smithi173.stdout:3/169: read - d7/d17/f3c zero size 2024-04-20T11:09:26.807 INFO:tasks.workunit.client.1.smithi173.stdout:9/161: dwrite d5/d9/f27 [0,4194304] 0 2024-04-20T11:09:26.810 INFO:tasks.workunit.client.1.smithi173.stdout:0/114: getdents d6/d8/df/d15 0 2024-04-20T11:09:26.810 INFO:tasks.workunit.client.1.smithi173.stdout:8/174: creat d4/d18/f30 x:0 0 0 2024-04-20T11:09:26.812 INFO:tasks.workunit.client.1.smithi173.stdout:8/175: write d4/f8 [3597532,113144] 0 2024-04-20T11:09:26.815 INFO:tasks.workunit.client.1.smithi173.stdout:0/115: mkdir d6/d24 0 2024-04-20T11:09:26.817 INFO:tasks.workunit.client.1.smithi173.stdout:0/116: dread - d6/d8/f1f zero size 2024-04-20T11:09:26.823 INFO:tasks.workunit.client.1.smithi173.stdout:0/117: symlink d6/l25 0 2024-04-20T11:09:26.824 INFO:tasks.workunit.client.1.smithi173.stdout:8/176: mknod d4/c31 0 2024-04-20T11:09:26.827 INFO:tasks.workunit.client.1.smithi173.stdout:9/162: fdatasync d5/d9/f27 0 2024-04-20T11:09:26.829 INFO:tasks.workunit.client.1.smithi173.stdout:0/118: unlink d6/d8/df/d15/f18 0 2024-04-20T11:09:26.829 INFO:tasks.workunit.client.1.smithi173.stdout:9/163: chown d5/l7 168543430 1 2024-04-20T11:09:26.829 INFO:tasks.workunit.client.1.smithi173.stdout:1/107: dread f3 [0,4194304] 0 2024-04-20T11:09:26.831 INFO:tasks.workunit.client.1.smithi173.stdout:3/170: getdents d7/d2b 0 2024-04-20T11:09:26.834 INFO:tasks.workunit.client.1.smithi173.stdout:1/108: mknod d5/d7/da/c24 0 2024-04-20T11:09:26.835 INFO:tasks.workunit.client.1.smithi173.stdout:0/119: mknod d6/d24/c26 0 2024-04-20T11:09:26.837 INFO:tasks.workunit.client.1.smithi173.stdout:3/171: link d7/d2b/f3d d7/d9/d30/f44 0 2024-04-20T11:09:26.838 INFO:tasks.workunit.client.1.smithi173.stdout:8/177: dread f2 [0,4194304] 0 2024-04-20T11:09:26.840 INFO:tasks.workunit.client.1.smithi173.stdout:9/164: dread d5/d9/f26 [0,4194304] 0 2024-04-20T11:09:26.840 INFO:tasks.workunit.client.1.smithi173.stdout:1/109: write d5/d7/d12/f15 [1300337,66462] 0 2024-04-20T11:09:26.841 INFO:tasks.workunit.client.1.smithi173.stdout:5/87: sync 2024-04-20T11:09:26.845 INFO:tasks.workunit.client.1.smithi173.stdout:3/172: write d7/d2b/f36 [1168853,24437] 0 2024-04-20T11:09:26.845 INFO:tasks.workunit.client.1.smithi173.stdout:9/165: readlink d5/d9/d20/d29/d2d/l2e 0 2024-04-20T11:09:26.846 INFO:tasks.workunit.client.1.smithi173.stdout:8/178: creat d4/f32 x:0 0 0 2024-04-20T11:09:26.847 INFO:tasks.workunit.client.1.smithi173.stdout:9/166: fdatasync d5/d9/f1d 0 2024-04-20T11:09:26.849 INFO:tasks.workunit.client.1.smithi173.stdout:3/173: mkdir d7/d9/d3b/d45 0 2024-04-20T11:09:26.849 INFO:tasks.workunit.client.1.smithi173.stdout:9/167: chown d5/d9/d20/d2b/d31 14344419 1 2024-04-20T11:09:26.850 INFO:tasks.workunit.client.1.smithi173.stdout:1/110: dread d5/d7/f19 [0,4194304] 0 2024-04-20T11:09:26.851 INFO:tasks.workunit.client.1.smithi173.stdout:0/120: dread d6/f7 [0,4194304] 0 2024-04-20T11:09:26.853 INFO:tasks.workunit.client.1.smithi173.stdout:0/121: fdatasync d6/d8/f12 0 2024-04-20T11:09:26.854 INFO:tasks.workunit.client.1.smithi173.stdout:9/168: mknod d5/c34 0 2024-04-20T11:09:26.854 INFO:tasks.workunit.client.1.smithi173.stdout:1/111: write d5/d6/f9 [953696,62678] 0 2024-04-20T11:09:26.856 INFO:tasks.workunit.client.1.smithi173.stdout:2/122: truncate d2/d14/f15 143650 0 2024-04-20T11:09:26.861 INFO:tasks.workunit.client.1.smithi173.stdout:4/133: link f4 da/dc/d15/f2a 0 2024-04-20T11:09:26.863 INFO:tasks.workunit.client.1.smithi173.stdout:9/169: write d5/d9/d20/d29/d18/f1f [650956,128000] 0 2024-04-20T11:09:26.865 INFO:tasks.workunit.client.1.smithi173.stdout:1/112: mknod d5/d6/c25 0 2024-04-20T11:09:26.869 INFO:tasks.workunit.client.1.smithi173.stdout:8/179: dread d4/ff [0,4194304] 0 2024-04-20T11:09:26.870 INFO:tasks.workunit.client.1.smithi173.stdout:4/134: write da/dc/fe [587456,6083] 0 2024-04-20T11:09:26.872 INFO:tasks.workunit.client.1.smithi173.stdout:5/88: dwrite da/de/ff [0,4194304] 0 2024-04-20T11:09:26.873 INFO:tasks.workunit.client.1.smithi173.stdout:8/180: chown d4/l12 0 1 2024-04-20T11:09:26.877 INFO:tasks.workunit.client.1.smithi173.stdout:4/135: readlink da/d1a/l28 0 2024-04-20T11:09:26.878 INFO:tasks.workunit.client.1.smithi173.stdout:3/174: dwrite d7/d17/f29 [0,4194304] 0 2024-04-20T11:09:26.880 INFO:tasks.workunit.client.1.smithi173.stdout:6/117: sync 2024-04-20T11:09:26.880 INFO:tasks.workunit.client.1.smithi173.stdout:7/114: sync 2024-04-20T11:09:26.885 INFO:tasks.workunit.client.1.smithi173.stdout:1/113: creat d5/d7/d12/d1f/f26 x:0 0 0 2024-04-20T11:09:26.890 INFO:tasks.workunit.client.1.smithi173.stdout:8/181: rename d4/c16 to d4/d2e/c33 0 2024-04-20T11:09:26.890 INFO:tasks.workunit.client.1.smithi173.stdout:2/123: dwrite d2/de/d11/f23 [0,4194304] 0 2024-04-20T11:09:26.898 INFO:tasks.workunit.client.1.smithi173.stdout:6/118: creat d4/d18/f26 x:0 0 0 2024-04-20T11:09:26.900 INFO:tasks.workunit.client.1.smithi173.stdout:6/119: readlink d4/d18/l21 0 2024-04-20T11:09:26.905 INFO:tasks.workunit.client.1.smithi173.stdout:1/114: creat d5/d6/d17/f27 x:0 0 0 2024-04-20T11:09:26.906 INFO:tasks.workunit.client.1.smithi173.stdout:0/122: dwrite d6/fd [0,4194304] 0 2024-04-20T11:09:26.908 INFO:tasks.workunit.client.1.smithi173.stdout:7/115: dwrite d9/da/f14 [0,4194304] 0 2024-04-20T11:09:26.910 INFO:tasks.workunit.client.1.smithi173.stdout:7/116: dread - d9/d1f/f28 zero size 2024-04-20T11:09:26.910 INFO:tasks.workunit.client.1.smithi173.stdout:0/123: fdatasync d6/d8/df/d15/f22 0 2024-04-20T11:09:26.912 INFO:tasks.workunit.client.1.smithi173.stdout:9/170: dread d5/d9/d20/d29/d18/f1f [0,4194304] 0 2024-04-20T11:09:26.913 INFO:tasks.workunit.client.1.smithi173.stdout:5/89: mknod da/c1e 0 2024-04-20T11:09:26.920 INFO:tasks.workunit.client.1.smithi173.stdout:6/120: truncate d4/d14/f22 879332 0 2024-04-20T11:09:26.922 INFO:tasks.workunit.client.1.smithi173.stdout:0/124: dread d6/d8/f19 [0,4194304] 0 2024-04-20T11:09:26.923 INFO:tasks.workunit.client.1.smithi173.stdout:0/125: truncate d6/f13 1480952 0 2024-04-20T11:09:26.924 INFO:tasks.workunit.client.1.smithi173.stdout:7/117: mknod d9/d1f/c29 0 2024-04-20T11:09:26.925 INFO:tasks.workunit.client.1.smithi173.stdout:8/182: link d4/f32 d4/d2e/f34 0 2024-04-20T11:09:26.928 INFO:tasks.workunit.client.1.smithi173.stdout:8/183: read d4/f19 [1156036,86279] 0 2024-04-20T11:09:26.930 INFO:tasks.workunit.client.1.smithi173.stdout:3/175: creat d7/d9/f46 x:0 0 0 2024-04-20T11:09:26.936 INFO:tasks.workunit.client.1.smithi173.stdout:9/171: symlink d5/d9/d20/d29/l35 0 2024-04-20T11:09:26.938 INFO:tasks.workunit.client.1.smithi173.stdout:6/121: rmdir d4/d18 39 2024-04-20T11:09:26.949 INFO:tasks.workunit.client.1.smithi173.stdout:2/124: truncate d2/de/d11/f23 212343 0 2024-04-20T11:09:26.953 INFO:tasks.workunit.client.1.smithi173.stdout:0/126: dwrite d6/d8/df/d15/f22 [0,4194304] 0 2024-04-20T11:09:26.957 INFO:tasks.workunit.client.1.smithi173.stdout:8/184: dwrite d4/fd [0,4194304] 0 2024-04-20T11:09:26.959 INFO:tasks.workunit.client.1.smithi173.stdout:3/176: write d7/d17/f3c [271930,23711] 0 2024-04-20T11:09:26.960 INFO:tasks.workunit.client.1.smithi173.stdout:5/90: creat da/f1f x:0 0 0 2024-04-20T11:09:26.961 INFO:tasks.workunit.client.1.smithi173.stdout:2/125: symlink d2/de/d11/l25 0 2024-04-20T11:09:26.965 INFO:tasks.workunit.client.1.smithi173.stdout:0/127: creat d6/f27 x:0 0 0 2024-04-20T11:09:26.966 INFO:tasks.workunit.client.1.smithi173.stdout:4/136: dwrite f4 [0,4194304] 0 2024-04-20T11:09:26.976 INFO:tasks.workunit.client.1.smithi173.stdout:1/115: dwrite f3 [0,4194304] 0 2024-04-20T11:09:26.976 INFO:tasks.workunit.client.1.smithi173.stdout:7/118: getdents d9/da 0 2024-04-20T11:09:26.977 INFO:tasks.workunit.client.1.smithi173.stdout:8/185: write d4/d18/f24 [1166314,84073] 0 2024-04-20T11:09:26.978 INFO:tasks.workunit.client.1.smithi173.stdout:7/119: readlink d9/da/d17/l1c 0 2024-04-20T11:09:26.982 INFO:tasks.workunit.client.1.smithi173.stdout:8/186: chown d4/c14 246 1 2024-04-20T11:09:26.982 INFO:tasks.workunit.client.1.smithi173.stdout:8/187: readlink d4/l2d 0 2024-04-20T11:09:26.982 INFO:tasks.workunit.client.1.smithi173.stdout:1/116: fdatasync d5/d7/d12/d1f/f26 0 2024-04-20T11:09:26.984 INFO:tasks.workunit.client.1.smithi173.stdout:7/120: dread - d9/da/f27 zero size 2024-04-20T11:09:26.986 INFO:tasks.workunit.client.1.smithi173.stdout:6/122: creat d4/f27 x:0 0 0 2024-04-20T11:09:26.987 INFO:tasks.workunit.client.1.smithi173.stdout:0/128: unlink d6/c1a 0 2024-04-20T11:09:26.989 INFO:tasks.workunit.client.1.smithi173.stdout:1/117: write d5/d7/d12/d1f/f26 [566261,115295] 0 2024-04-20T11:09:26.989 INFO:tasks.workunit.client.1.smithi173.stdout:9/172: truncate d5/d9/f25 2514812 0 2024-04-20T11:09:26.989 INFO:tasks.workunit.client.1.smithi173.stdout:2/126: symlink d2/de/d11/l26 0 2024-04-20T11:09:26.991 INFO:tasks.workunit.client.1.smithi173.stdout:4/137: creat da/dc/d13/f2b x:0 0 0 2024-04-20T11:09:26.993 INFO:tasks.workunit.client.1.smithi173.stdout:7/121: creat d9/da/d20/f2a x:0 0 0 2024-04-20T11:09:26.994 INFO:tasks.workunit.client.1.smithi173.stdout:4/138: read f8 [2121110,119869] 0 2024-04-20T11:09:26.996 INFO:tasks.workunit.client.1.smithi173.stdout:0/129: write d6/d8/df/f1d [134951,66720] 0 2024-04-20T11:09:26.997 INFO:tasks.workunit.client.1.smithi173.stdout:1/118: rmdir d5/d7 39 2024-04-20T11:09:26.999 INFO:tasks.workunit.client.1.smithi173.stdout:0/130: fsync d6/f13 0 2024-04-20T11:09:27.000 INFO:tasks.workunit.client.1.smithi173.stdout:5/91: dread da/dd/f11 [0,4194304] 0 2024-04-20T11:09:27.001 INFO:tasks.workunit.client.1.smithi173.stdout:9/173: unlink d5/l19 0 2024-04-20T11:09:27.001 INFO:tasks.workunit.client.1.smithi173.stdout:7/122: chown d9/da/d17/l1c 617605211 1 2024-04-20T11:09:27.006 INFO:tasks.workunit.client.1.smithi173.stdout:5/92: write f9 [1182164,45270] 0 2024-04-20T11:09:27.007 INFO:tasks.workunit.client.1.smithi173.stdout:7/123: write d9/da/d17/f1a [338952,20759] 0 2024-04-20T11:09:27.008 INFO:tasks.workunit.client.1.smithi173.stdout:5/93: fdatasync da/de/f10 0 2024-04-20T11:09:27.008 INFO:tasks.workunit.client.1.smithi173.stdout:8/188: dread d4/f17 [0,4194304] 0 2024-04-20T11:09:27.009 INFO:tasks.workunit.client.1.smithi173.stdout:5/94: readlink da/de/l14 0 2024-04-20T11:09:27.010 INFO:tasks.workunit.client.1.smithi173.stdout:4/139: creat da/dc/f2c x:0 0 0 2024-04-20T11:09:27.012 INFO:tasks.workunit.client.1.smithi173.stdout:3/177: dwrite d7/d9/f3f [0,4194304] 0 2024-04-20T11:09:27.018 INFO:tasks.workunit.client.1.smithi173.stdout:8/189: write d4/d18/f21 [774910,48142] 0 2024-04-20T11:09:27.019 INFO:tasks.workunit.client.1.smithi173.stdout:6/123: mkdir d4/d1c/d28 0 2024-04-20T11:09:27.024 INFO:tasks.workunit.client.1.smithi173.stdout:3/178: truncate d7/d9/d3b/f38 64039 0 2024-04-20T11:09:27.028 INFO:tasks.workunit.client.1.smithi173.stdout:5/95: mknod da/de/d19/c20 0 2024-04-20T11:09:27.029 INFO:tasks.workunit.client.1.smithi173.stdout:1/119: dread d5/d7/f19 [0,4194304] 0 2024-04-20T11:09:27.029 INFO:tasks.workunit.client.1.smithi173.stdout:0/131: dwrite d6/d8/f1f [0,4194304] 0 2024-04-20T11:09:27.030 INFO:tasks.workunit.client.1.smithi173.stdout:3/179: dread - d7/d9/f46 zero size 2024-04-20T11:09:27.033 INFO:tasks.workunit.client.1.smithi173.stdout:1/120: stat d5/d7/da/df 0 2024-04-20T11:09:27.034 INFO:tasks.workunit.client.1.smithi173.stdout:3/180: write d7/f22 [3729408,43887] 0 2024-04-20T11:09:27.037 INFO:tasks.workunit.client.1.smithi173.stdout:9/174: mknod d5/d9/d20/d29/d2d/d33/c36 0 2024-04-20T11:09:27.038 INFO:tasks.workunit.client.1.smithi173.stdout:8/190: chown d4/c2c 57 1 2024-04-20T11:09:27.039 INFO:tasks.workunit.client.1.smithi173.stdout:4/140: dread f8 [0,4194304] 0 2024-04-20T11:09:27.043 INFO:tasks.workunit.client.1.smithi173.stdout:4/141: chown da/d1e/l20 743 1 2024-04-20T11:09:27.044 INFO:tasks.workunit.client.1.smithi173.stdout:5/96: rename da/d12/l16 to da/de/l21 0 2024-04-20T11:09:27.051 INFO:tasks.workunit.client.1.smithi173.stdout:9/175: dread d5/f2f [0,4194304] 0 2024-04-20T11:09:27.051 INFO:tasks.workunit.client.1.smithi173.stdout:3/181: creat d7/f47 x:0 0 0 2024-04-20T11:09:27.054 INFO:tasks.workunit.client.1.smithi173.stdout:4/142: mknod da/dc/c2d 0 2024-04-20T11:09:27.056 INFO:tasks.workunit.client.1.smithi173.stdout:5/97: rename da to da/d22 22 2024-04-20T11:09:27.057 INFO:tasks.workunit.client.1.smithi173.stdout:0/132: dread d6/d8/df/d15/f22 [0,4194304] 0 2024-04-20T11:09:27.057 INFO:tasks.workunit.client.1.smithi173.stdout:5/98: stat da/d12/l13 0 2024-04-20T11:09:27.065 INFO:tasks.workunit.client.1.smithi173.stdout:9/176: dread f0 [4194304,4194304] 0 2024-04-20T11:09:27.066 INFO:tasks.workunit.client.1.smithi173.stdout:3/182: dread d7/f22 [0,4194304] 0 2024-04-20T11:09:27.071 INFO:tasks.workunit.client.1.smithi173.stdout:7/124: dread f1 [0,4194304] 0 2024-04-20T11:09:27.072 INFO:tasks.workunit.client.1.smithi173.stdout:3/183: mknod d7/d2b/c48 0 2024-04-20T11:09:27.073 INFO:tasks.workunit.client.1.smithi173.stdout:6/124: write d4/f11 [2216144,65684] 0 2024-04-20T11:09:27.073 INFO:tasks.workunit.client.1.smithi173.stdout:0/133: dwrite d6/fd [0,4194304] 0 2024-04-20T11:09:27.075 INFO:tasks.workunit.client.1.smithi173.stdout:8/191: getdents d4/d18 0 2024-04-20T11:09:27.077 INFO:tasks.workunit.client.1.smithi173.stdout:1/121: write d5/d7/f19 [254928,46292] 0 2024-04-20T11:09:27.079 INFO:tasks.workunit.client.1.smithi173.stdout:7/125: mkdir d9/da/d17/d2b 0 2024-04-20T11:09:27.085 INFO:tasks.workunit.client.1.smithi173.stdout:8/192: fsync d4/d2e/f34 0 2024-04-20T11:09:27.086 INFO:tasks.workunit.client.1.smithi173.stdout:5/99: getdents da/d12 0 2024-04-20T11:09:27.088 INFO:tasks.workunit.client.1.smithi173.stdout:6/125: mknod d4/c29 0 2024-04-20T11:09:27.089 INFO:tasks.workunit.client.1.smithi173.stdout:9/177: getdents d5/d9/d20/d29 0 2024-04-20T11:09:27.090 INFO:tasks.workunit.client.1.smithi173.stdout:9/178: chown d5/d9/d32 120600854 1 2024-04-20T11:09:27.095 INFO:tasks.workunit.client.1.smithi173.stdout:5/100: creat da/dd/f23 x:0 0 0 2024-04-20T11:09:27.096 INFO:tasks.workunit.client.1.smithi173.stdout:5/101: write f9 [5082785,51688] 0 2024-04-20T11:09:27.100 INFO:tasks.workunit.client.1.smithi173.stdout:9/179: symlink d5/d9/d20/l37 0 2024-04-20T11:09:27.101 INFO:tasks.workunit.client.1.smithi173.stdout:9/180: fdatasync d5/d9/fa 0 2024-04-20T11:09:27.103 INFO:tasks.workunit.client.1.smithi173.stdout:7/126: creat d9/f2c x:0 0 0 2024-04-20T11:09:27.104 INFO:tasks.workunit.client.1.smithi173.stdout:7/127: fdatasync d9/f1d 0 2024-04-20T11:09:27.105 INFO:tasks.workunit.client.1.smithi173.stdout:6/126: mknod d4/d1c/d28/c2a 0 2024-04-20T11:09:27.105 INFO:tasks.workunit.client.1.smithi173.stdout:0/134: link l5 d6/d8/df/d15/l28 0 2024-04-20T11:09:27.106 INFO:tasks.workunit.client.1.smithi173.stdout:7/128: read - d9/da/f27 zero size 2024-04-20T11:09:27.106 INFO:tasks.workunit.client.1.smithi173.stdout:6/127: read - d4/f27 zero size 2024-04-20T11:09:27.106 INFO:tasks.workunit.client.1.smithi173.stdout:8/193: rename d4/c1d to d4/c35 0 2024-04-20T11:09:27.108 INFO:tasks.workunit.client.1.smithi173.stdout:5/102: dread da/dd/f11 [0,4194304] 0 2024-04-20T11:09:27.161 INFO:tasks.workunit.client.1.smithi173.stdout:2/127: sync 2024-04-20T11:09:27.162 INFO:tasks.workunit.client.1.smithi173.stdout:6/128: unlink d4/d14/f22 0 2024-04-20T11:09:27.162 INFO:tasks.workunit.client.1.smithi173.stdout:9/181: unlink d5/d9/f25 0 2024-04-20T11:09:27.166 INFO:tasks.workunit.client.1.smithi173.stdout:8/194: creat d4/f36 x:0 0 0 2024-04-20T11:09:27.167 INFO:tasks.workunit.client.1.smithi173.stdout:9/182: write d5/d9/f27 [5101151,45303] 0 2024-04-20T11:09:27.167 INFO:tasks.workunit.client.1.smithi173.stdout:8/195: readlink d4/l13 0 2024-04-20T11:09:27.171 INFO:tasks.workunit.client.1.smithi173.stdout:6/129: creat d4/d18/f2b x:0 0 0 2024-04-20T11:09:27.171 INFO:tasks.workunit.client.1.smithi173.stdout:9/183: stat d5/d9/cc 0 2024-04-20T11:09:27.171 INFO:tasks.workunit.client.1.smithi173.stdout:8/196: write d4/f36 [208719,34419] 0 2024-04-20T11:09:27.174 INFO:tasks.workunit.client.1.smithi173.stdout:2/128: rename d2/de/c1b to d2/d13/c27 0 2024-04-20T11:09:27.175 INFO:tasks.workunit.client.1.smithi173.stdout:8/197: dread - d4/f32 zero size 2024-04-20T11:09:27.175 INFO:tasks.workunit.client.1.smithi173.stdout:0/135: dread d6/f7 [0,4194304] 0 2024-04-20T11:09:27.175 INFO:tasks.workunit.client.1.smithi173.stdout:7/129: dwrite d9/da/f27 [0,4194304] 0 2024-04-20T11:09:27.178 INFO:tasks.workunit.client.1.smithi173.stdout:6/130: write d4/f16 [559013,48034] 0 2024-04-20T11:09:27.179 INFO:tasks.workunit.client.1.smithi173.stdout:5/103: dread da/fc [0,4194304] 0 2024-04-20T11:09:27.180 INFO:tasks.workunit.client.1.smithi173.stdout:9/184: creat d5/f38 x:0 0 0 2024-04-20T11:09:27.180 INFO:tasks.workunit.client.1.smithi173.stdout:8/198: truncate d4/f32 55739 0 2024-04-20T11:09:27.182 INFO:tasks.workunit.client.1.smithi173.stdout:5/104: stat da/de/f10 0 2024-04-20T11:09:27.183 INFO:tasks.workunit.client.1.smithi173.stdout:8/199: creat d4/d2e/f37 x:0 0 0 2024-04-20T11:09:27.185 INFO:tasks.workunit.client.1.smithi173.stdout:5/105: chown f8 3869286 1 2024-04-20T11:09:27.186 INFO:tasks.workunit.client.1.smithi173.stdout:6/131: symlink d4/d14/l2c 0 2024-04-20T11:09:27.186 INFO:tasks.workunit.client.1.smithi173.stdout:8/200: symlink d4/d2e/l38 0 2024-04-20T11:09:27.189 INFO:tasks.workunit.client.1.smithi173.stdout:0/136: dread d6/d8/f19 [0,4194304] 0 2024-04-20T11:09:27.190 INFO:tasks.workunit.client.1.smithi173.stdout:5/106: creat da/de/f24 x:0 0 0 2024-04-20T11:09:27.200 INFO:tasks.workunit.client.1.smithi173.stdout:0/137: getdents d6 0 2024-04-20T11:09:27.204 INFO:tasks.workunit.client.1.smithi173.stdout:7/130: dwrite d9/da/d17/f1b [0,4194304] 0 2024-04-20T11:09:27.204 INFO:tasks.workunit.client.1.smithi173.stdout:0/138: stat d6/f13 0 2024-04-20T11:09:27.207 INFO:tasks.workunit.client.1.smithi173.stdout:6/132: getdents d4/d14 0 2024-04-20T11:09:27.211 INFO:tasks.workunit.client.1.smithi173.stdout:7/131: fsync d9/d1f/f28 0 2024-04-20T11:09:27.213 INFO:tasks.workunit.client.1.smithi173.stdout:0/139: write d6/d8/f12 [815041,43742] 0 2024-04-20T11:09:27.213 INFO:tasks.workunit.client.1.smithi173.stdout:2/129: dwrite d2/de/d11/f23 [0,4194304] 0 2024-04-20T11:09:27.216 INFO:tasks.workunit.client.1.smithi173.stdout:5/107: write f7 [7547062,62380] 0 2024-04-20T11:09:27.218 INFO:tasks.workunit.client.1.smithi173.stdout:9/185: dwrite d5/d9/d20/d29/d18/f1f [4194304,4194304] 0 2024-04-20T11:09:27.219 INFO:tasks.workunit.client.1.smithi173.stdout:7/132: write d9/da/d17/f1b [1552123,40690] 0 2024-04-20T11:09:27.222 INFO:tasks.workunit.client.1.smithi173.stdout:0/140: creat d6/f29 x:0 0 0 2024-04-20T11:09:27.224 INFO:tasks.workunit.client.1.smithi173.stdout:5/108: mkdir da/d12/d25 0 2024-04-20T11:09:27.226 INFO:tasks.workunit.client.1.smithi173.stdout:9/186: write d5/d9/d20/d29/f11 [5527253,9229] 0 2024-04-20T11:09:27.227 INFO:tasks.workunit.client.1.smithi173.stdout:0/141: rmdir d6/d8 39 2024-04-20T11:09:27.229 INFO:tasks.workunit.client.1.smithi173.stdout:7/133: mknod d9/da/d17/c2d 0 2024-04-20T11:09:27.236 INFO:tasks.workunit.client.1.smithi173.stdout:5/109: mknod da/d12/c26 0 2024-04-20T11:09:27.237 INFO:tasks.workunit.client.1.smithi173.stdout:6/133: dwrite d4/f16 [0,4194304] 0 2024-04-20T11:09:27.239 INFO:tasks.workunit.client.1.smithi173.stdout:2/130: truncate d2/de/d11/fd 416947 0 2024-04-20T11:09:27.244 INFO:tasks.workunit.client.1.smithi173.stdout:0/142: truncate d6/d8/df/f11 1404431 0 2024-04-20T11:09:27.244 INFO:tasks.workunit.client.1.smithi173.stdout:7/134: rename d9/da/d20 to d9/da/d2e 0 2024-04-20T11:09:27.246 INFO:tasks.workunit.client.1.smithi173.stdout:6/134: symlink d4/d1c/l2d 0 2024-04-20T11:09:27.248 INFO:tasks.workunit.client.1.smithi173.stdout:6/135: read d4/d14/f1a [3883427,51944] 0 2024-04-20T11:09:27.256 INFO:tasks.workunit.client.1.smithi173.stdout:7/135: unlink d9/da/d2e/c26 0 2024-04-20T11:09:27.257 INFO:tasks.workunit.client.1.smithi173.stdout:7/136: rename d9 to d9/da/d17/d2b/d2f 22 2024-04-20T11:09:27.260 INFO:tasks.workunit.client.1.smithi173.stdout:7/137: chown d9/da/d17/f1a 2481154 1 2024-04-20T11:09:27.266 INFO:tasks.workunit.client.1.smithi173.stdout:7/138: unlink d9/da/d2e/c23 0 2024-04-20T11:09:27.267 INFO:tasks.workunit.client.1.smithi173.stdout:0/143: dwrite d6/f7 [0,4194304] 0 2024-04-20T11:09:27.268 INFO:tasks.workunit.client.1.smithi173.stdout:9/187: dread d5/d9/f1b [0,4194304] 0 2024-04-20T11:09:27.270 INFO:tasks.workunit.client.1.smithi173.stdout:6/136: dwrite d4/d18/f2b [0,4194304] 0 2024-04-20T11:09:27.273 INFO:tasks.workunit.client.1.smithi173.stdout:5/110: truncate f8 3865867 0 2024-04-20T11:09:27.274 INFO:tasks.workunit.client.1.smithi173.stdout:5/111: readlink da/lb 0 2024-04-20T11:09:27.275 INFO:tasks.workunit.client.1.smithi173.stdout:2/131: truncate d2/de/d11/fd 1007323 0 2024-04-20T11:09:27.277 INFO:tasks.workunit.client.1.smithi173.stdout:5/112: write da/dd/f15 [82867,35555] 0 2024-04-20T11:09:27.278 INFO:tasks.workunit.client.1.smithi173.stdout:6/137: chown d4/l9 11275179 1 2024-04-20T11:09:27.283 INFO:tasks.workunit.client.1.smithi173.stdout:9/188: rename d5/d9/d20/l21 to d5/d9/d20/d29/d2d/l39 0 2024-04-20T11:09:27.293 INFO:tasks.workunit.client.1.smithi173.stdout:0/144: symlink d6/l2a 0 2024-04-20T11:09:27.294 INFO:tasks.workunit.client.1.smithi173.stdout:5/113: mknod da/dd/c27 0 2024-04-20T11:09:27.295 INFO:tasks.workunit.client.1.smithi173.stdout:5/114: chown da/dd 0 1 2024-04-20T11:09:27.296 INFO:tasks.workunit.client.1.smithi173.stdout:5/115: stat da/de/d19 0 2024-04-20T11:09:27.296 INFO:tasks.workunit.client.1.smithi173.stdout:4/143: sync 2024-04-20T11:09:27.297 INFO:tasks.workunit.client.1.smithi173.stdout:4/144: write da/dc/f2c [545558,52546] 0 2024-04-20T11:09:27.298 INFO:tasks.workunit.client.1.smithi173.stdout:6/138: unlink d4/c29 0 2024-04-20T11:09:27.299 INFO:tasks.workunit.client.1.smithi173.stdout:4/145: fdatasync da/dc/fe 0 2024-04-20T11:09:27.301 INFO:tasks.workunit.client.1.smithi173.stdout:7/139: creat d9/f30 x:0 0 0 2024-04-20T11:09:27.303 INFO:tasks.workunit.client.1.smithi173.stdout:2/132: rename d2/d14 to d2/d13/d28 0 2024-04-20T11:09:27.305 INFO:tasks.workunit.client.1.smithi173.stdout:9/189: symlink d5/d9/d20/d29/d2d/d33/l3a 0 2024-04-20T11:09:27.311 INFO:tasks.workunit.client.1.smithi173.stdout:7/140: mkdir d9/da/d31 0 2024-04-20T11:09:27.312 INFO:tasks.workunit.client.1.smithi173.stdout:4/146: rename da/c11 to da/dc/d15/c2e 0 2024-04-20T11:09:27.314 INFO:tasks.workunit.client.1.smithi173.stdout:2/133: dread d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:27.314 INFO:tasks.workunit.client.1.smithi173.stdout:5/116: symlink da/l28 0 2024-04-20T11:09:27.315 INFO:tasks.workunit.client.1.smithi173.stdout:5/117: stat da/f1f 0 2024-04-20T11:09:27.316 INFO:tasks.workunit.client.1.smithi173.stdout:9/190: mknod d5/d9/d20/d29/d2d/c3b 0 2024-04-20T11:09:27.318 INFO:tasks.workunit.client.1.smithi173.stdout:6/139: creat d4/d24/f2e x:0 0 0 2024-04-20T11:09:27.319 INFO:tasks.workunit.client.1.smithi173.stdout:6/140: stat d4/f27 0 2024-04-20T11:09:27.319 INFO:tasks.workunit.client.1.smithi173.stdout:4/147: dread da/dc/fe [0,4194304] 0 2024-04-20T11:09:27.320 INFO:tasks.workunit.client.1.smithi173.stdout:7/141: mknod d9/d1f/c32 0 2024-04-20T11:09:27.321 INFO:tasks.workunit.client.1.smithi173.stdout:4/148: fsync f4 0 2024-04-20T11:09:27.323 INFO:tasks.workunit.client.1.smithi173.stdout:6/141: chown c0 674671238 1 2024-04-20T11:09:27.324 INFO:tasks.workunit.client.1.smithi173.stdout:2/134: mkdir d2/d29 0 2024-04-20T11:09:27.324 INFO:tasks.workunit.client.1.smithi173.stdout:0/145: dwrite d6/d8/df/d15/f22 [4194304,4194304] 0 2024-04-20T11:09:27.330 INFO:tasks.workunit.client.1.smithi173.stdout:6/142: creat d4/d14/f2f x:0 0 0 2024-04-20T11:09:27.333 INFO:tasks.workunit.client.1.smithi173.stdout:4/149: mknod da/c2f 0 2024-04-20T11:09:27.334 INFO:tasks.workunit.client.1.smithi173.stdout:5/118: dread da/dd/f11 [0,4194304] 0 2024-04-20T11:09:27.334 INFO:tasks.workunit.client.1.smithi173.stdout:7/142: mkdir d9/d33 0 2024-04-20T11:09:27.336 INFO:tasks.workunit.client.1.smithi173.stdout:2/135: mkdir d2/d13/d28/d2a 0 2024-04-20T11:09:27.337 INFO:tasks.workunit.client.1.smithi173.stdout:9/191: dwrite d5/f38 [0,4194304] 0 2024-04-20T11:09:27.340 INFO:tasks.workunit.client.1.smithi173.stdout:0/146: mknod d6/d8/df/d15/c2b 0 2024-04-20T11:09:27.341 INFO:tasks.workunit.client.1.smithi173.stdout:2/136: write d2/de/f24 [854052,12034] 0 2024-04-20T11:09:27.345 INFO:tasks.workunit.client.1.smithi173.stdout:6/143: rename d4/d18/l21 to d4/d1c/d28/l30 0 2024-04-20T11:09:27.345 INFO:tasks.workunit.client.1.smithi173.stdout:4/150: mkdir da/d1a/d30 0 2024-04-20T11:09:27.345 INFO:tasks.workunit.client.1.smithi173.stdout:8/201: sync 2024-04-20T11:09:27.346 INFO:tasks.workunit.client.1.smithi173.stdout:1/122: sync 2024-04-20T11:09:27.346 INFO:tasks.workunit.client.1.smithi173.stdout:3/184: sync 2024-04-20T11:09:27.346 INFO:tasks.workunit.client.1.smithi173.stdout:4/151: fdatasync da/dc/d13/f29 0 2024-04-20T11:09:27.347 INFO:tasks.workunit.client.1.smithi173.stdout:7/143: truncate d9/da/ff 468470 0 2024-04-20T11:09:27.347 INFO:tasks.workunit.client.1.smithi173.stdout:6/144: write d4/f27 [342031,77785] 0 2024-04-20T11:09:27.350 INFO:tasks.workunit.client.1.smithi173.stdout:4/152: write f4 [4316483,89153] 0 2024-04-20T11:09:27.351 INFO:tasks.workunit.client.1.smithi173.stdout:1/123: chown d5/d7/d12/c16 0 1 2024-04-20T11:09:27.355 INFO:tasks.workunit.client.1.smithi173.stdout:9/192: mknod d5/d9/d20/d29/d2d/c3c 0 2024-04-20T11:09:27.355 INFO:tasks.workunit.client.1.smithi173.stdout:0/147: mkdir d6/d8/df/d2c 0 2024-04-20T11:09:27.357 INFO:tasks.workunit.client.1.smithi173.stdout:0/148: chown d6/d8/f19 277 1 2024-04-20T11:09:27.358 INFO:tasks.workunit.client.1.smithi173.stdout:3/185: symlink d7/d17/l49 0 2024-04-20T11:09:27.359 INFO:tasks.workunit.client.1.smithi173.stdout:7/144: mknod d9/da/c34 0 2024-04-20T11:09:27.360 INFO:tasks.workunit.client.1.smithi173.stdout:5/119: creat da/f29 x:0 0 0 2024-04-20T11:09:27.366 INFO:tasks.workunit.client.1.smithi173.stdout:4/153: symlink da/dc/d13/l31 0 2024-04-20T11:09:27.369 INFO:tasks.workunit.client.1.smithi173.stdout:9/193: mknod d5/d9/d20/d29/d2d/c3d 0 2024-04-20T11:09:27.374 INFO:tasks.workunit.client.1.smithi173.stdout:0/149: dwrite d6/f27 [0,4194304] 0 2024-04-20T11:09:27.379 INFO:tasks.workunit.client.1.smithi173.stdout:0/150: write d6/f1e [581302,29457] 0 2024-04-20T11:09:27.381 INFO:tasks.workunit.client.1.smithi173.stdout:7/145: dwrite d9/d1f/f28 [0,4194304] 0 2024-04-20T11:09:27.383 INFO:tasks.workunit.client.1.smithi173.stdout:6/145: dread d4/fe [0,4194304] 0 2024-04-20T11:09:27.384 INFO:tasks.workunit.client.1.smithi173.stdout:4/154: link f4 da/dc/d13/f32 0 2024-04-20T11:09:27.386 INFO:tasks.workunit.client.1.smithi173.stdout:1/124: dread d5/fb [0,4194304] 0 2024-04-20T11:09:27.386 INFO:tasks.workunit.client.1.smithi173.stdout:6/146: readlink d4/d18/l1b 0 2024-04-20T11:09:27.388 INFO:tasks.workunit.client.1.smithi173.stdout:1/125: chown d5/d6/dc 1123861 1 2024-04-20T11:09:27.389 INFO:tasks.workunit.client.1.smithi173.stdout:8/202: truncate d4/f10 1492739 0 2024-04-20T11:09:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:27 smithi117 ceph-mon[15659]: pgmap v23: 65 pgs: 65 active+clean; 2.0 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 57 MiB/s rd, 94 MiB/s wr, 286 op/s 2024-04-20T11:09:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:27 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:27 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:27 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:09:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:27 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:09:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:27 smithi117 ceph-mon[15659]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T11:09:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:27 smithi117 ceph-mon[15659]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T11:09:27.393 INFO:tasks.workunit.client.1.smithi173.stdout:3/186: truncate d7/d17/f29 2096180 0 2024-04-20T11:09:27.403 INFO:tasks.workunit.client.1.smithi173.stdout:1/126: creat d5/d7/da/f28 x:0 0 0 2024-04-20T11:09:27.406 INFO:tasks.workunit.client.1.smithi173.stdout:7/146: mkdir d9/d33/d35 0 2024-04-20T11:09:27.409 INFO:tasks.workunit.client.1.smithi173.stdout:9/194: truncate d5/d9/f27 3554318 0 2024-04-20T11:09:27.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:27 smithi173 ceph-mon[23800]: pgmap v23: 65 pgs: 65 active+clean; 2.0 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 57 MiB/s rd, 94 MiB/s wr, 286 op/s 2024-04-20T11:09:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:27 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:27 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:27 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:09:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:27 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:09:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:27 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T11:09:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:27 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T11:09:27.411 INFO:tasks.workunit.client.1.smithi173.stdout:0/151: mkdir d6/d8/df/d2c/d2d 0 2024-04-20T11:09:27.412 INFO:tasks.workunit.client.1.smithi173.stdout:5/120: dwrite f7 [4194304,4194304] 0 2024-04-20T11:09:27.415 INFO:tasks.workunit.client.1.smithi173.stdout:0/152: chown d6/d8/df/d15 0 1 2024-04-20T11:09:27.416 INFO:tasks.workunit.client.1.smithi173.stdout:8/203: mknod d4/c39 0 2024-04-20T11:09:27.417 INFO:tasks.workunit.client.1.smithi173.stdout:4/155: dwrite da/dc/f27 [0,4194304] 0 2024-04-20T11:09:27.420 INFO:tasks.workunit.client.1.smithi173.stdout:3/187: symlink d7/d9/d3b/d45/l4a 0 2024-04-20T11:09:27.436 INFO:tasks.workunit.client.1.smithi173.stdout:1/127: dwrite f0 [0,4194304] 0 2024-04-20T11:09:27.436 INFO:tasks.workunit.client.1.smithi173.stdout:5/121: link da/fc da/de/d19/f2a 0 2024-04-20T11:09:27.441 INFO:tasks.workunit.client.1.smithi173.stdout:9/195: getdents d5/d9/d20/d2b 0 2024-04-20T11:09:27.441 INFO:tasks.workunit.client.1.smithi173.stdout:6/147: dwrite d4/fd [0,4194304] 0 2024-04-20T11:09:27.446 INFO:tasks.workunit.client.1.smithi173.stdout:6/148: dread - d4/d18/f26 zero size 2024-04-20T11:09:27.454 INFO:tasks.workunit.client.1.smithi173.stdout:8/204: dwrite d4/f17 [0,4194304] 0 2024-04-20T11:09:27.457 INFO:tasks.workunit.client.1.smithi173.stdout:5/122: rename da/f1f to da/f2b 0 2024-04-20T11:09:27.458 INFO:tasks.workunit.client.1.smithi173.stdout:3/188: truncate d7/f15 288776 0 2024-04-20T11:09:27.459 INFO:tasks.workunit.client.1.smithi173.stdout:5/123: dread - da/de/f24 zero size 2024-04-20T11:09:27.460 INFO:tasks.workunit.client.1.smithi173.stdout:7/147: dread d9/da/d17/f1a [0,4194304] 0 2024-04-20T11:09:27.460 INFO:tasks.workunit.client.1.smithi173.stdout:8/205: mkdir d4/d18/d3a 0 2024-04-20T11:09:27.460 INFO:tasks.workunit.client.1.smithi173.stdout:3/189: stat d7/d9/f3f 0 2024-04-20T11:09:27.461 INFO:tasks.workunit.client.1.smithi173.stdout:6/149: chown d4/d14/f1a 77481730 1 2024-04-20T11:09:27.462 INFO:tasks.workunit.client.1.smithi173.stdout:9/196: truncate d5/d9/f1d 2671779 0 2024-04-20T11:09:27.465 INFO:tasks.workunit.client.1.smithi173.stdout:7/148: truncate d9/f30 589543 0 2024-04-20T11:09:27.465 INFO:tasks.workunit.client.1.smithi173.stdout:3/190: readlink d7/d2b/l34 0 2024-04-20T11:09:27.466 INFO:tasks.workunit.client.1.smithi173.stdout:5/124: rmdir da/dd 39 2024-04-20T11:09:27.467 INFO:tasks.workunit.client.1.smithi173.stdout:4/156: dread f1 [4194304,4194304] 0 2024-04-20T11:09:27.473 INFO:tasks.workunit.client.1.smithi173.stdout:1/128: fdatasync f3 0 2024-04-20T11:09:27.473 INFO:tasks.workunit.client.1.smithi173.stdout:6/150: mknod d4/d24/c31 0 2024-04-20T11:09:27.473 INFO:tasks.workunit.client.1.smithi173.stdout:9/197: symlink d5/d9/d20/d2b/l3e 0 2024-04-20T11:09:27.473 INFO:tasks.workunit.client.1.smithi173.stdout:8/206: symlink d4/l3b 0 2024-04-20T11:09:27.474 INFO:tasks.workunit.client.1.smithi173.stdout:7/149: rmdir d9/d1f 39 2024-04-20T11:09:27.474 INFO:tasks.workunit.client.1.smithi173.stdout:5/125: mkdir da/de/d19/d2c 0 2024-04-20T11:09:27.474 INFO:tasks.workunit.client.1.smithi173.stdout:9/198: write d5/d9/f1a [707372,81858] 0 2024-04-20T11:09:27.477 INFO:tasks.workunit.client.1.smithi173.stdout:6/151: symlink d4/d1c/l32 0 2024-04-20T11:09:27.478 INFO:tasks.workunit.client.1.smithi173.stdout:6/152: fsync d4/f20 0 2024-04-20T11:09:27.479 INFO:tasks.workunit.client.1.smithi173.stdout:5/126: readlink da/de/d19/l1a 0 2024-04-20T11:09:27.482 INFO:tasks.workunit.client.1.smithi173.stdout:5/127: dread - da/de/f24 zero size 2024-04-20T11:09:27.482 INFO:tasks.workunit.client.1.smithi173.stdout:8/207: rename d4/d2e to d4/d18/d3a/d3c 0 2024-04-20T11:09:27.483 INFO:tasks.workunit.client.1.smithi173.stdout:1/129: link d5/d7/l18 d5/d6/d17/l29 0 2024-04-20T11:09:27.484 INFO:tasks.workunit.client.1.smithi173.stdout:7/150: mknod d9/d33/d35/c36 0 2024-04-20T11:09:27.485 INFO:tasks.workunit.client.1.smithi173.stdout:5/128: fsync da/de/f24 0 2024-04-20T11:09:27.486 INFO:tasks.workunit.client.1.smithi173.stdout:6/153: read d4/f20 [36866,90830] 0 2024-04-20T11:09:27.490 INFO:tasks.workunit.client.1.smithi173.stdout:8/208: symlink d4/l3d 0 2024-04-20T11:09:27.495 INFO:tasks.workunit.client.1.smithi173.stdout:4/157: dread da/dc/d15/f1c [0,4194304] 0 2024-04-20T11:09:27.495 INFO:tasks.workunit.client.1.smithi173.stdout:3/191: dwrite d7/d9/f3a [0,4194304] 0 2024-04-20T11:09:27.496 INFO:tasks.workunit.client.1.smithi173.stdout:6/154: truncate d4/d18/f26 269296 0 2024-04-20T11:09:27.500 INFO:tasks.workunit.client.1.smithi173.stdout:3/192: read d7/d9/f35 [4330956,86978] 0 2024-04-20T11:09:27.502 INFO:tasks.workunit.client.1.smithi173.stdout:7/151: getdents d9/da/d31 0 2024-04-20T11:09:27.505 INFO:tasks.workunit.client.1.smithi173.stdout:7/152: write d9/da/f12 [1674093,63131] 0 2024-04-20T11:09:27.506 INFO:tasks.workunit.client.1.smithi173.stdout:4/158: creat da/dc/f33 x:0 0 0 2024-04-20T11:09:27.507 INFO:tasks.workunit.client.1.smithi173.stdout:1/130: dwrite d5/d7/da/f14 [0,4194304] 0 2024-04-20T11:09:27.512 INFO:tasks.workunit.client.1.smithi173.stdout:3/193: rmdir d7/d9/d3b 39 2024-04-20T11:09:27.512 INFO:tasks.workunit.client.1.smithi173.stdout:4/159: read - da/dc/d13/f2b zero size 2024-04-20T11:09:27.512 INFO:tasks.workunit.client.1.smithi173.stdout:5/129: write da/de/d19/f2a [3725882,33797] 0 2024-04-20T11:09:27.519 INFO:tasks.workunit.client.1.smithi173.stdout:6/155: mknod d4/c33 0 2024-04-20T11:09:27.522 INFO:tasks.workunit.client.1.smithi173.stdout:8/209: dread d4/f8 [0,4194304] 0 2024-04-20T11:09:27.525 INFO:tasks.workunit.client.1.smithi173.stdout:7/153: dwrite d9/da/d17/f16 [0,4194304] 0 2024-04-20T11:09:27.527 INFO:tasks.workunit.client.1.smithi173.stdout:5/130: fdatasync da/dd/f11 0 2024-04-20T11:09:27.527 INFO:tasks.workunit.client.1.smithi173.stdout:6/156: mknod d4/d1c/c34 0 2024-04-20T11:09:27.529 INFO:tasks.workunit.client.1.smithi173.stdout:6/157: dread - d4/d24/f2e zero size 2024-04-20T11:09:27.535 INFO:tasks.workunit.client.1.smithi173.stdout:8/210: fdatasync d4/f10 0 2024-04-20T11:09:27.537 INFO:tasks.workunit.client.1.smithi173.stdout:3/194: dwrite d7/d17/f19 [0,4194304] 0 2024-04-20T11:09:27.540 INFO:tasks.workunit.client.1.smithi173.stdout:2/137: sync 2024-04-20T11:09:27.542 INFO:tasks.workunit.client.1.smithi173.stdout:7/154: stat d9/da/d17/l18 0 2024-04-20T11:09:27.545 INFO:tasks.workunit.client.1.smithi173.stdout:5/131: creat da/de/f2d x:0 0 0 2024-04-20T11:09:27.548 INFO:tasks.workunit.client.1.smithi173.stdout:6/158: unlink d4/fd 0 2024-04-20T11:09:27.551 INFO:tasks.workunit.client.1.smithi173.stdout:1/131: rename d5/d6/dc/c13 to d5/d6/dc/c2a 0 2024-04-20T11:09:27.556 INFO:tasks.workunit.client.1.smithi173.stdout:5/132: fsync da/de/f2d 0 2024-04-20T11:09:27.561 INFO:tasks.workunit.client.1.smithi173.stdout:6/159: creat d4/d24/f35 x:0 0 0 2024-04-20T11:09:27.571 INFO:tasks.workunit.client.1.smithi173.stdout:2/138: fdatasync d2/d13/d28/f15 0 2024-04-20T11:09:27.571 INFO:tasks.workunit.client.1.smithi173.stdout:1/132: dwrite d5/d7/da/f14 [0,4194304] 0 2024-04-20T11:09:27.572 INFO:tasks.workunit.client.1.smithi173.stdout:7/155: rename d9/da/d17/f16 to d9/da/d2e/f37 0 2024-04-20T11:09:27.574 INFO:tasks.workunit.client.1.smithi173.stdout:5/133: dread f7 [4194304,4194304] 0 2024-04-20T11:09:27.576 INFO:tasks.workunit.client.1.smithi173.stdout:1/133: rename d5/d7 to d5/d7/da/d20/d2b 22 2024-04-20T11:09:27.586 INFO:tasks.workunit.client.1.smithi173.stdout:8/211: rename d4/ce to d4/c3e 0 2024-04-20T11:09:27.586 INFO:tasks.workunit.client.1.smithi173.stdout:3/195: creat d7/d9/f4b x:0 0 0 2024-04-20T11:09:27.586 INFO:tasks.workunit.client.1.smithi173.stdout:2/139: creat d2/de/f2b x:0 0 0 2024-04-20T11:09:27.587 INFO:tasks.workunit.client.1.smithi173.stdout:2/140: write d2/de/f24 [1034870,95570] 0 2024-04-20T11:09:27.587 INFO:tasks.workunit.client.1.smithi173.stdout:3/196: symlink d7/d2b/l4c 0 2024-04-20T11:09:27.589 INFO:tasks.workunit.client.1.smithi173.stdout:1/134: creat d5/d6/dc/f2c x:0 0 0 2024-04-20T11:09:27.590 INFO:tasks.workunit.client.1.smithi173.stdout:7/156: dread f1 [0,4194304] 0 2024-04-20T11:09:27.590 INFO:tasks.workunit.client.1.smithi173.stdout:6/160: dread d4/d14/f1a [0,4194304] 0 2024-04-20T11:09:27.591 INFO:tasks.workunit.client.1.smithi173.stdout:1/135: dread - d5/d6/dc/f2c zero size 2024-04-20T11:09:27.593 INFO:tasks.workunit.client.1.smithi173.stdout:7/157: write d9/da/f25 [95217,57806] 0 2024-04-20T11:09:27.598 INFO:tasks.workunit.client.1.smithi173.stdout:5/134: dread da/de/d19/f2a [0,4194304] 0 2024-04-20T11:09:27.603 INFO:tasks.workunit.client.1.smithi173.stdout:5/135: mknod da/de/c2e 0 2024-04-20T11:09:27.606 INFO:tasks.workunit.client.1.smithi173.stdout:5/136: dread - da/f2b zero size 2024-04-20T11:09:27.612 INFO:tasks.workunit.client.1.smithi173.stdout:7/158: dread d9/da/f27 [0,4194304] 0 2024-04-20T11:09:27.613 INFO:tasks.workunit.client.1.smithi173.stdout:3/197: dwrite d7/d9/f28 [0,4194304] 0 2024-04-20T11:09:27.613 INFO:tasks.workunit.client.1.smithi173.stdout:1/136: dwrite d5/d7/d12/f15 [0,4194304] 0 2024-04-20T11:09:27.616 INFO:tasks.workunit.client.1.smithi173.stdout:3/198: truncate d7/d2b/f3d 713457 0 2024-04-20T11:09:27.616 INFO:tasks.workunit.client.1.smithi173.stdout:6/161: dwrite d4/f13 [0,4194304] 0 2024-04-20T11:09:27.618 INFO:tasks.workunit.client.1.smithi173.stdout:3/199: chown d7/d9/d30/f44 28088898 1 2024-04-20T11:09:27.622 INFO:tasks.workunit.client.1.smithi173.stdout:7/159: write d9/da/f27 [460510,15509] 0 2024-04-20T11:09:27.626 INFO:tasks.workunit.client.1.smithi173.stdout:5/137: rmdir da/d12/d25 0 2024-04-20T11:09:27.628 INFO:tasks.workunit.client.1.smithi173.stdout:5/138: chown da/de/l14 3364 1 2024-04-20T11:09:27.630 INFO:tasks.workunit.client.1.smithi173.stdout:7/160: rmdir d9/da/d17 39 2024-04-20T11:09:27.631 INFO:tasks.workunit.client.1.smithi173.stdout:7/161: write d9/f21 [332698,20843] 0 2024-04-20T11:09:27.634 INFO:tasks.workunit.client.1.smithi173.stdout:5/139: rename da/dd/f23 to da/dd/f2f 0 2024-04-20T11:09:27.636 INFO:tasks.workunit.client.1.smithi173.stdout:5/140: dread - da/dd/f2f zero size 2024-04-20T11:09:27.636 INFO:tasks.workunit.client.1.smithi173.stdout:1/137: getdents d5/d7/da 0 2024-04-20T11:09:27.638 INFO:tasks.workunit.client.1.smithi173.stdout:5/141: stat da/de/f2d 0 2024-04-20T11:09:27.641 INFO:tasks.workunit.client.1.smithi173.stdout:7/162: creat d9/da/d31/f38 x:0 0 0 2024-04-20T11:09:27.643 INFO:tasks.workunit.client.1.smithi173.stdout:1/138: dread f4 [0,4194304] 0 2024-04-20T11:09:27.644 INFO:tasks.workunit.client.1.smithi173.stdout:3/200: write d7/d9/f13 [3245463,29717] 0 2024-04-20T11:09:27.647 INFO:tasks.workunit.client.1.smithi173.stdout:3/201: chown d7/d9/d30/c33 103 1 2024-04-20T11:09:27.654 INFO:tasks.workunit.client.1.smithi173.stdout:3/202: rename d7/d9/d3b/c39 to d7/d9/d30/c4d 0 2024-04-20T11:09:27.696 INFO:tasks.workunit.client.1.smithi173.stdout:0/153: sync 2024-04-20T11:09:27.706 INFO:tasks.workunit.client.1.smithi173.stdout:7/163: dwrite d9/f15 [0,4194304] 0 2024-04-20T11:09:27.708 INFO:tasks.workunit.client.1.smithi173.stdout:3/203: fdatasync d7/d9/d30/f44 0 2024-04-20T11:09:27.710 INFO:tasks.workunit.client.1.smithi173.stdout:3/204: dread - d7/f47 zero size 2024-04-20T11:09:27.710 INFO:tasks.workunit.client.1.smithi173.stdout:0/154: creat d6/d8/df/d15/f2e x:0 0 0 2024-04-20T11:09:27.712 INFO:tasks.workunit.client.1.smithi173.stdout:0/155: stat d6/d8/df/d15/f22 0 2024-04-20T11:09:27.720 INFO:tasks.workunit.client.1.smithi173.stdout:7/164: dread d9/da/f14 [0,4194304] 0 2024-04-20T11:09:27.720 INFO:tasks.workunit.client.1.smithi173.stdout:0/156: rename d6/fd to d6/d8/f2f 0 2024-04-20T11:09:27.724 INFO:tasks.workunit.client.1.smithi173.stdout:5/142: dread f8 [0,4194304] 0 2024-04-20T11:09:27.725 INFO:tasks.workunit.client.1.smithi173.stdout:0/157: truncate d6/d8/df/d15/f2e 391406 0 2024-04-20T11:09:27.727 INFO:tasks.workunit.client.1.smithi173.stdout:0/158: chown d6/d8/df/d15/f2e 6698662 1 2024-04-20T11:09:27.727 INFO:tasks.workunit.client.1.smithi173.stdout:5/143: mkdir da/dd/d30 0 2024-04-20T11:09:27.727 INFO:tasks.workunit.client.1.smithi173.stdout:0/159: readlink d6/lc 0 2024-04-20T11:09:27.727 INFO:tasks.workunit.client.1.smithi173.stdout:5/144: chown f9 10343 1 2024-04-20T11:09:27.728 INFO:tasks.workunit.client.1.smithi173.stdout:5/145: dread - da/f29 zero size 2024-04-20T11:09:27.729 INFO:tasks.workunit.client.1.smithi173.stdout:3/205: truncate d7/d9/f3f 3812662 0 2024-04-20T11:09:27.731 INFO:tasks.workunit.client.1.smithi173.stdout:0/160: rename d6/l2a to d6/d8/l30 0 2024-04-20T11:09:27.732 INFO:tasks.workunit.client.1.smithi173.stdout:3/206: write d7/f22 [3138273,124516] 0 2024-04-20T11:09:27.733 INFO:tasks.workunit.client.1.smithi173.stdout:5/146: creat da/dd/f31 x:0 0 0 2024-04-20T11:09:27.741 INFO:tasks.workunit.client.1.smithi173.stdout:5/147: rename da/dd to da/de/d19/d32 0 2024-04-20T11:09:27.747 INFO:tasks.workunit.client.1.smithi173.stdout:5/148: creat da/f33 x:0 0 0 2024-04-20T11:09:27.760 INFO:tasks.workunit.client.1.smithi173.stdout:5/149: dwrite da/f29 [0,4194304] 0 2024-04-20T11:09:27.767 INFO:tasks.workunit.client.1.smithi173.stdout:5/150: mkdir da/d34 0 2024-04-20T11:09:27.768 INFO:tasks.workunit.client.1.smithi173.stdout:5/151: chown da/lb 16592546 1 2024-04-20T11:09:27.770 INFO:tasks.workunit.client.1.smithi173.stdout:5/152: read - da/de/f10 zero size 2024-04-20T11:09:27.773 INFO:tasks.workunit.client.1.smithi173.stdout:5/153: mkdir da/de/d19/d32/d30/d35 0 2024-04-20T11:09:27.774 INFO:tasks.workunit.client.1.smithi173.stdout:5/154: chown da/de/ff 391827988 1 2024-04-20T11:09:27.777 INFO:tasks.workunit.client.1.smithi173.stdout:5/155: creat da/de/d19/d32/f36 x:0 0 0 2024-04-20T11:09:27.778 INFO:tasks.workunit.client.1.smithi173.stdout:5/156: dread - da/de/f2d zero size 2024-04-20T11:09:27.781 INFO:tasks.workunit.client.1.smithi173.stdout:5/157: creat da/de/f37 x:0 0 0 2024-04-20T11:09:27.785 INFO:tasks.workunit.client.1.smithi173.stdout:5/158: rename da/de/f10 to da/de/d19/d32/d30/d35/f38 0 2024-04-20T11:09:27.787 INFO:tasks.workunit.client.1.smithi173.stdout:5/159: write da/de/d19/d32/f31 [577149,78340] 0 2024-04-20T11:09:27.790 INFO:tasks.workunit.client.1.smithi173.stdout:5/160: read - da/de/d19/d32/d30/d35/f38 zero size 2024-04-20T11:09:27.792 INFO:tasks.workunit.client.1.smithi173.stdout:5/161: rmdir da/de/d19/d32/d30/d35 39 2024-04-20T11:09:27.796 INFO:tasks.workunit.client.1.smithi173.stdout:5/162: mknod da/de/d19/d32/d30/c39 0 2024-04-20T11:09:27.806 INFO:tasks.workunit.client.1.smithi173.stdout:5/163: dread da/de/d19/f2a [0,4194304] 0 2024-04-20T11:09:27.807 INFO:tasks.workunit.client.1.smithi173.stdout:5/164: truncate da/d12/f1d 791810 0 2024-04-20T11:09:27.811 INFO:tasks.workunit.client.1.smithi173.stdout:5/165: creat da/de/d19/d32/f3a x:0 0 0 2024-04-20T11:09:27.814 INFO:tasks.workunit.client.1.smithi173.stdout:5/166: getdents da/d34 0 2024-04-20T11:09:27.828 INFO:tasks.workunit.client.1.smithi173.stdout:5/167: dwrite da/de/d19/f2a [0,4194304] 0 2024-04-20T11:09:27.831 INFO:tasks.workunit.client.1.smithi173.stdout:5/168: symlink da/de/d19/d2c/l3b 0 2024-04-20T11:09:27.835 INFO:tasks.workunit.client.1.smithi173.stdout:9/199: sync 2024-04-20T11:09:27.835 INFO:tasks.workunit.client.1.smithi173.stdout:5/169: creat da/de/d19/d32/f3c x:0 0 0 2024-04-20T11:09:27.840 INFO:tasks.workunit.client.1.smithi173.stdout:9/200: creat d5/d9/d20/d29/d2d/f3f x:0 0 0 2024-04-20T11:09:27.844 INFO:tasks.workunit.client.1.smithi173.stdout:9/201: creat d5/d9/d20/f40 x:0 0 0 2024-04-20T11:09:27.845 INFO:tasks.workunit.client.1.smithi173.stdout:5/170: creat da/f3d x:0 0 0 2024-04-20T11:09:27.846 INFO:tasks.workunit.client.1.smithi173.stdout:9/202: chown d5/d9/d20/d29/c24 868333726 1 2024-04-20T11:09:27.846 INFO:tasks.workunit.client.1.smithi173.stdout:5/171: truncate da/f3d 908956 0 2024-04-20T11:09:27.849 INFO:tasks.workunit.client.1.smithi173.stdout:5/172: mkdir da/d12/d3e 0 2024-04-20T11:09:27.850 INFO:tasks.workunit.client.1.smithi173.stdout:9/203: creat d5/d9/d32/f41 x:0 0 0 2024-04-20T11:09:27.850 INFO:tasks.workunit.client.1.smithi173.stdout:5/173: read da/fc [1026856,6958] 0 2024-04-20T11:09:27.852 INFO:tasks.workunit.client.1.smithi173.stdout:5/174: write da/de/f2d [26684,28215] 0 2024-04-20T11:09:27.861 INFO:tasks.workunit.client.1.smithi173.stdout:5/175: mknod da/de/d19/d32/d30/d35/c3f 0 2024-04-20T11:09:27.863 INFO:tasks.workunit.client.1.smithi173.stdout:5/176: write da/de/d19/d32/f3a [321544,28901] 0 2024-04-20T11:09:27.872 INFO:tasks.workunit.client.1.smithi173.stdout:5/177: write da/de/d19/d32/f2f [423454,9075] 0 2024-04-20T11:09:27.873 INFO:tasks.workunit.client.1.smithi173.stdout:5/178: fdatasync da/de/f37 0 2024-04-20T11:09:27.875 INFO:tasks.workunit.client.1.smithi173.stdout:9/204: dread d5/d9/f27 [0,4194304] 0 2024-04-20T11:09:27.877 INFO:tasks.workunit.client.1.smithi173.stdout:5/179: mkdir da/d12/d40 0 2024-04-20T11:09:27.878 INFO:tasks.workunit.client.1.smithi173.stdout:5/180: read - da/f2b zero size 2024-04-20T11:09:27.879 INFO:tasks.workunit.client.1.smithi173.stdout:5/181: chown da/de/l14 137255 1 2024-04-20T11:09:27.880 INFO:tasks.workunit.client.1.smithi173.stdout:9/205: creat d5/d9/d20/d29/d2d/f42 x:0 0 0 2024-04-20T11:09:27.887 INFO:tasks.workunit.client.1.smithi173.stdout:9/206: unlink d5/l12 0 2024-04-20T11:09:27.889 INFO:tasks.workunit.client.1.smithi173.stdout:5/182: creat da/de/d19/d32/d30/f41 x:0 0 0 2024-04-20T11:09:27.889 INFO:tasks.workunit.client.1.smithi173.stdout:5/183: dread - da/f33 zero size 2024-04-20T11:09:27.891 INFO:tasks.workunit.client.1.smithi173.stdout:5/184: dread - da/de/d19/d32/d30/f41 zero size 2024-04-20T11:09:27.891 INFO:tasks.workunit.client.1.smithi173.stdout:9/207: creat d5/d9/d20/d29/d2d/f43 x:0 0 0 2024-04-20T11:09:27.894 INFO:tasks.workunit.client.1.smithi173.stdout:5/185: truncate da/de/d19/d32/f3a 1147663 0 2024-04-20T11:09:27.894 INFO:tasks.workunit.client.1.smithi173.stdout:9/208: mkdir d5/d9/d32/d44 0 2024-04-20T11:09:27.910 INFO:tasks.workunit.client.1.smithi173.stdout:5/186: dwrite da/de/d19/d32/d30/f41 [0,4194304] 0 2024-04-20T11:09:27.911 INFO:tasks.workunit.client.1.smithi173.stdout:5/187: chown da/f2b 764 1 2024-04-20T11:09:27.916 INFO:tasks.workunit.client.1.smithi173.stdout:5/188: creat da/d12/d40/f42 x:0 0 0 2024-04-20T11:09:27.917 INFO:tasks.workunit.client.1.smithi173.stdout:5/189: chown da/d12 529626 1 2024-04-20T11:09:27.943 INFO:tasks.workunit.client.1.smithi173.stdout:8/212: sync 2024-04-20T11:09:27.943 INFO:tasks.workunit.client.1.smithi173.stdout:2/141: sync 2024-04-20T11:09:27.943 INFO:tasks.workunit.client.1.smithi173.stdout:4/160: sync 2024-04-20T11:09:27.946 INFO:tasks.workunit.client.1.smithi173.stdout:4/161: write da/dc/f2c [755379,84965] 0 2024-04-20T11:09:27.946 INFO:tasks.workunit.client.1.smithi173.stdout:8/213: symlink d4/d18/l3f 0 2024-04-20T11:09:27.948 INFO:tasks.workunit.client.1.smithi173.stdout:4/162: fsync da/dc/d15/f18 0 2024-04-20T11:09:27.949 INFO:tasks.workunit.client.1.smithi173.stdout:2/142: mknod d2/d29/c2c 0 2024-04-20T11:09:27.950 INFO:tasks.workunit.client.1.smithi173.stdout:8/214: creat d4/d18/d3a/d3c/f40 x:0 0 0 2024-04-20T11:09:27.951 INFO:tasks.workunit.client.1.smithi173.stdout:5/190: dwrite da/de/d19/d32/d30/f41 [0,4194304] 0 2024-04-20T11:09:27.953 INFO:tasks.workunit.client.1.smithi173.stdout:5/191: chown da/de/l14 80167775 1 2024-04-20T11:09:27.954 INFO:tasks.workunit.client.1.smithi173.stdout:8/215: truncate d4/d18/f30 999114 0 2024-04-20T11:09:27.955 INFO:tasks.workunit.client.1.smithi173.stdout:4/163: rename da/dc/l17 to da/l34 0 2024-04-20T11:09:27.956 INFO:tasks.workunit.client.1.smithi173.stdout:4/164: stat da/dc/d15 0 2024-04-20T11:09:27.962 INFO:tasks.workunit.client.1.smithi173.stdout:2/143: rename d2/de/l17 to d2/d13/l2d 0 2024-04-20T11:09:27.968 INFO:tasks.workunit.client.1.smithi173.stdout:8/216: creat d4/f41 x:0 0 0 2024-04-20T11:09:27.968 INFO:tasks.workunit.client.1.smithi173.stdout:2/144: mknod d2/de/c2e 0 2024-04-20T11:09:27.970 INFO:tasks.workunit.client.1.smithi173.stdout:8/217: read d4/f28 [7736940,69544] 0 2024-04-20T11:09:27.970 INFO:tasks.workunit.client.1.smithi173.stdout:5/192: dwrite da/f33 [0,4194304] 0 2024-04-20T11:09:27.974 INFO:tasks.workunit.client.1.smithi173.stdout:5/193: dread - da/de/d19/d32/d30/d35/f38 zero size 2024-04-20T11:09:27.981 INFO:tasks.workunit.client.1.smithi173.stdout:2/145: unlink d2/de/lf 0 2024-04-20T11:09:27.983 INFO:tasks.workunit.client.1.smithi173.stdout:2/146: fdatasync d2/de/f2b 0 2024-04-20T11:09:27.984 INFO:tasks.workunit.client.1.smithi173.stdout:4/165: dwrite f4 [0,4194304] 0 2024-04-20T11:09:27.987 INFO:tasks.workunit.client.1.smithi173.stdout:8/218: dwrite d4/f36 [0,4194304] 0 2024-04-20T11:09:27.987 INFO:tasks.workunit.client.1.smithi173.stdout:5/194: mkdir da/de/d19/d32/d30/d43 0 2024-04-20T11:09:27.991 INFO:tasks.workunit.client.1.smithi173.stdout:2/147: symlink d2/d29/l2f 0 2024-04-20T11:09:27.992 INFO:tasks.workunit.client.1.smithi173.stdout:5/195: write da/f29 [2301074,130834] 0 2024-04-20T11:09:27.994 INFO:tasks.workunit.client.1.smithi173.stdout:5/196: dread - da/f2b zero size 2024-04-20T11:09:28.001 INFO:tasks.workunit.client.1.smithi173.stdout:2/148: symlink d2/d29/l30 0 2024-04-20T11:09:28.004 INFO:tasks.workunit.client.1.smithi173.stdout:2/149: dread - d2/de/f21 zero size 2024-04-20T11:09:28.007 INFO:tasks.workunit.client.1.smithi173.stdout:2/150: write d2/de/f2b [31091,77853] 0 2024-04-20T11:09:28.011 INFO:tasks.workunit.client.1.smithi173.stdout:5/197: dwrite da/de/f2d [0,4194304] 0 2024-04-20T11:09:28.012 INFO:tasks.workunit.client.1.smithi173.stdout:5/198: stat f8 0 2024-04-20T11:09:28.017 INFO:tasks.workunit.client.1.smithi173.stdout:5/199: creat da/de/d19/d32/f44 x:0 0 0 2024-04-20T11:09:28.017 INFO:tasks.workunit.client.1.smithi173.stdout:6/162: sync 2024-04-20T11:09:28.020 INFO:tasks.workunit.client.1.smithi173.stdout:5/200: chown da/f3d 122343 1 2024-04-20T11:09:28.025 INFO:tasks.workunit.client.1.smithi173.stdout:2/151: dwrite d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:28.025 INFO:tasks.workunit.client.1.smithi173.stdout:5/201: truncate da/de/d19/d32/d30/d35/f38 398734 0 2024-04-20T11:09:28.030 INFO:tasks.workunit.client.1.smithi173.stdout:4/166: truncate da/dc/f2c 131845 0 2024-04-20T11:09:28.034 INFO:tasks.workunit.client.1.smithi173.stdout:5/202: creat da/d12/f45 x:0 0 0 2024-04-20T11:09:28.037 INFO:tasks.workunit.client.1.smithi173.stdout:5/203: write da/de/d19/d32/d30/f41 [3983044,33333] 0 2024-04-20T11:09:28.037 INFO:tasks.workunit.client.1.smithi173.stdout:8/219: truncate f2 377380 0 2024-04-20T11:09:28.039 INFO:tasks.workunit.client.1.smithi173.stdout:2/152: mkdir d2/d13/d28/d2a/d31 0 2024-04-20T11:09:28.044 INFO:tasks.workunit.client.1.smithi173.stdout:6/163: dwrite d4/f13 [0,4194304] 0 2024-04-20T11:09:28.050 INFO:tasks.workunit.client.1.smithi173.stdout:8/220: truncate d4/f19 1528428 0 2024-04-20T11:09:28.050 INFO:tasks.workunit.client.1.smithi173.stdout:2/153: mknod d2/de/c32 0 2024-04-20T11:09:28.052 INFO:tasks.workunit.client.1.smithi173.stdout:4/167: dwrite da/f25 [0,4194304] 0 2024-04-20T11:09:28.055 INFO:tasks.workunit.client.1.smithi173.stdout:2/154: dread - d2/de/f21 zero size 2024-04-20T11:09:28.058 INFO:tasks.workunit.client.1.smithi173.stdout:8/221: symlink d4/d18/d3a/l42 0 2024-04-20T11:09:28.064 INFO:tasks.workunit.client.1.smithi173.stdout:1/139: sync 2024-04-20T11:09:28.064 INFO:tasks.workunit.client.1.smithi173.stdout:6/164: dread d4/d14/f1a [0,4194304] 0 2024-04-20T11:09:28.064 INFO:tasks.workunit.client.1.smithi173.stdout:3/207: sync 2024-04-20T11:09:28.064 INFO:tasks.workunit.client.1.smithi173.stdout:7/165: sync 2024-04-20T11:09:28.065 INFO:tasks.workunit.client.1.smithi173.stdout:9/209: sync 2024-04-20T11:09:28.065 INFO:tasks.workunit.client.1.smithi173.stdout:0/161: sync 2024-04-20T11:09:28.067 INFO:tasks.workunit.client.1.smithi173.stdout:8/222: mknod d4/d18/d3a/d3c/c43 0 2024-04-20T11:09:28.074 INFO:tasks.workunit.client.1.smithi173.stdout:6/165: mknod d4/d18/c36 0 2024-04-20T11:09:28.080 INFO:tasks.workunit.client.1.smithi173.stdout:8/223: write d4/d18/d3a/d3c/f37 [755494,103645] 0 2024-04-20T11:09:28.080 INFO:tasks.workunit.client.1.smithi173.stdout:4/168: dwrite da/dc/d13/f32 [4194304,4194304] 0 2024-04-20T11:09:28.086 INFO:tasks.workunit.client.1.smithi173.stdout:0/162: dread d6/d8/f12 [0,4194304] 0 2024-04-20T11:09:28.086 INFO:tasks.workunit.client.1.smithi173.stdout:9/210: chown d5/d9/d20/d29/d2d/c3d 610137 1 2024-04-20T11:09:28.087 INFO:tasks.workunit.client.1.smithi173.stdout:1/140: creat d5/f2d x:0 0 0 2024-04-20T11:09:28.090 INFO:tasks.workunit.client.1.smithi173.stdout:1/141: chown d5/d6/f21 102912340 1 2024-04-20T11:09:28.091 INFO:tasks.workunit.client.1.smithi173.stdout:6/166: unlink d4/d1c/l2d 0 2024-04-20T11:09:28.091 INFO:tasks.workunit.client.1.smithi173.stdout:2/155: dwrite d2/de/d11/f1d [0,4194304] 0 2024-04-20T11:09:28.094 INFO:tasks.workunit.client.1.smithi173.stdout:6/167: chown d4/d18/f19 88 1 2024-04-20T11:09:28.101 INFO:tasks.workunit.client.1.smithi173.stdout:7/166: dread d9/da/d17/f1b [0,4194304] 0 2024-04-20T11:09:28.104 INFO:tasks.workunit.client.1.smithi173.stdout:7/167: chown d9/f21 5756 1 2024-04-20T11:09:28.106 INFO:tasks.workunit.client.1.smithi173.stdout:4/169: dread da/dc/d13/f32 [0,4194304] 0 2024-04-20T11:09:28.107 INFO:tasks.workunit.client.1.smithi173.stdout:0/163: write d6/d8/f12 [1213155,38332] 0 2024-04-20T11:09:28.112 INFO:tasks.workunit.client.1.smithi173.stdout:1/142: unlink d5/f2d 0 2024-04-20T11:09:28.114 INFO:tasks.workunit.client.1.smithi173.stdout:3/208: getdents d7/d2b 0 2024-04-20T11:09:28.114 INFO:tasks.workunit.client.1.smithi173.stdout:0/164: chown d6/d8/f19 0 1 2024-04-20T11:09:28.115 INFO:tasks.workunit.client.1.smithi173.stdout:2/156: creat d2/de/d11/f33 x:0 0 0 2024-04-20T11:09:28.117 INFO:tasks.workunit.client.1.smithi173.stdout:5/204: dwrite da/fc [0,4194304] 0 2024-04-20T11:09:28.117 INFO:tasks.workunit.client.1.smithi173.stdout:7/168: mknod d9/da/d31/c39 0 2024-04-20T11:09:28.119 INFO:tasks.workunit.client.1.smithi173.stdout:0/165: chown d6/d8/f19 2007791 1 2024-04-20T11:09:28.120 INFO:tasks.workunit.client.1.smithi173.stdout:5/205: stat da 0 2024-04-20T11:09:28.120 INFO:tasks.workunit.client.1.smithi173.stdout:4/170: creat da/dc/d15/f35 x:0 0 0 2024-04-20T11:09:28.125 INFO:tasks.workunit.client.1.smithi173.stdout:1/143: creat d5/d7/d12/f2e x:0 0 0 2024-04-20T11:09:28.126 INFO:tasks.workunit.client.1.smithi173.stdout:6/168: dwrite d4/f16 [4194304,4194304] 0 2024-04-20T11:09:28.129 INFO:tasks.workunit.client.1.smithi173.stdout:8/224: link d4/d18/d3a/d3c/f40 d4/f44 0 2024-04-20T11:09:28.130 INFO:tasks.workunit.client.1.smithi173.stdout:6/169: stat d4/d18/c36 0 2024-04-20T11:09:28.133 INFO:tasks.workunit.client.1.smithi173.stdout:6/170: dread - d4/d14/f2f zero size 2024-04-20T11:09:28.134 INFO:tasks.workunit.client.1.smithi173.stdout:6/171: stat d4/f27 0 2024-04-20T11:09:28.136 INFO:tasks.workunit.client.1.smithi173.stdout:7/169: creat d9/d33/f3a x:0 0 0 2024-04-20T11:09:28.136 INFO:tasks.workunit.client.1.smithi173.stdout:8/225: symlink d4/d18/d3a/d3c/l45 0 2024-04-20T11:09:28.137 INFO:tasks.workunit.client.1.smithi173.stdout:1/144: creat d5/d7/d12/d1f/f2f x:0 0 0 2024-04-20T11:09:28.140 INFO:tasks.workunit.client.1.smithi173.stdout:0/166: dread d6/d8/f2f [0,4194304] 0 2024-04-20T11:09:28.147 INFO:tasks.workunit.client.1.smithi173.stdout:0/167: fsync d6/d8/f12 0 2024-04-20T11:09:28.149 INFO:tasks.workunit.client.1.smithi173.stdout:1/145: creat d5/f30 x:0 0 0 2024-04-20T11:09:28.150 INFO:tasks.workunit.client.1.smithi173.stdout:3/209: dread d7/d9/f35 [4194304,4194304] 0 2024-04-20T11:09:28.155 INFO:tasks.workunit.client.1.smithi173.stdout:9/211: dwrite d5/d9/f27 [0,4194304] 0 2024-04-20T11:09:28.156 INFO:tasks.workunit.client.1.smithi173.stdout:3/210: mkdir d7/d9/d3b/d4e 0 2024-04-20T11:09:28.158 INFO:tasks.workunit.client.1.smithi173.stdout:0/168: mknod d6/d24/c31 0 2024-04-20T11:09:28.159 INFO:tasks.workunit.client.1.smithi173.stdout:8/226: truncate d4/d18/d3a/d3c/f34 971179 0 2024-04-20T11:09:28.161 INFO:tasks.workunit.client.1.smithi173.stdout:5/206: dwrite f7 [4194304,4194304] 0 2024-04-20T11:09:28.162 INFO:tasks.workunit.client.1.smithi173.stdout:1/146: creat d5/d7/d12/f31 x:0 0 0 2024-04-20T11:09:28.163 INFO:tasks.workunit.client.1.smithi173.stdout:4/171: dwrite da/dc/d15/f1c [0,4194304] 0 2024-04-20T11:09:28.164 INFO:tasks.workunit.client.1.smithi173.stdout:6/172: dread d4/fe [0,4194304] 0 2024-04-20T11:09:28.165 INFO:tasks.workunit.client.1.smithi173.stdout:5/207: write da/f3d [1764906,106993] 0 2024-04-20T11:09:28.166 INFO:tasks.workunit.client.1.smithi173.stdout:1/147: read - d5/f30 zero size 2024-04-20T11:09:28.167 INFO:tasks.workunit.client.1.smithi173.stdout:8/227: write d4/d18/f21 [1423844,12025] 0 2024-04-20T11:09:28.167 INFO:tasks.workunit.client.1.smithi173.stdout:7/170: dread d9/da/f10 [0,4194304] 0 2024-04-20T11:09:28.173 INFO:tasks.workunit.client.1.smithi173.stdout:6/173: dread d4/f25 [0,4194304] 0 2024-04-20T11:09:28.174 INFO:tasks.workunit.client.1.smithi173.stdout:4/172: write f4 [2078761,93523] 0 2024-04-20T11:09:28.175 INFO:tasks.workunit.client.1.smithi173.stdout:0/169: write d6/d8/f1f [3584926,66520] 0 2024-04-20T11:09:28.177 INFO:tasks.workunit.client.1.smithi173.stdout:4/173: chown da/dc/d13/f32 1027096131 1 2024-04-20T11:09:28.182 INFO:tasks.workunit.client.1.smithi173.stdout:1/148: read - d5/d7/d12/f2e zero size 2024-04-20T11:09:28.183 INFO:tasks.workunit.client.1.smithi173.stdout:7/171: mknod d9/da/d31/c3b 0 2024-04-20T11:09:28.190 INFO:tasks.workunit.client.1.smithi173.stdout:7/172: mknod d9/d33/c3c 0 2024-04-20T11:09:28.192 INFO:tasks.workunit.client.1.smithi173.stdout:1/149: truncate d5/d7/d12/f15 4480787 0 2024-04-20T11:09:28.192 INFO:tasks.workunit.client.1.smithi173.stdout:6/174: rename d4/f25 to d4/f37 0 2024-04-20T11:09:28.193 INFO:tasks.workunit.client.1.smithi173.stdout:7/173: write d9/f30 [131867,119075] 0 2024-04-20T11:09:28.202 INFO:tasks.workunit.client.1.smithi173.stdout:8/228: dread d4/d18/d3a/d3c/f34 [0,4194304] 0 2024-04-20T11:09:28.203 INFO:tasks.workunit.client.1.smithi173.stdout:0/170: dwrite d6/d8/df/f11 [0,4194304] 0 2024-04-20T11:09:28.203 INFO:tasks.workunit.client.1.smithi173.stdout:5/208: link da/de/ff da/de/d19/d2c/f46 0 2024-04-20T11:09:28.206 INFO:tasks.workunit.client.1.smithi173.stdout:1/150: creat d5/d6/d17/f32 x:0 0 0 2024-04-20T11:09:28.209 INFO:tasks.workunit.client.1.smithi173.stdout:7/174: creat d9/da/d17/f3d x:0 0 0 2024-04-20T11:09:28.209 INFO:tasks.workunit.client.1.smithi173.stdout:5/209: chown da/d12/c17 60205 1 2024-04-20T11:09:28.213 INFO:tasks.workunit.client.1.smithi173.stdout:3/211: dwrite d7/f8 [4194304,4194304] 0 2024-04-20T11:09:28.215 INFO:tasks.workunit.client.1.smithi173.stdout:4/174: dwrite da/dc/f33 [0,4194304] 0 2024-04-20T11:09:28.215 INFO:tasks.workunit.client.1.smithi173.stdout:5/210: stat da/d12/d3e 0 2024-04-20T11:09:28.218 INFO:tasks.workunit.client.1.smithi173.stdout:6/175: creat d4/f38 x:0 0 0 2024-04-20T11:09:28.220 INFO:tasks.workunit.client.1.smithi173.stdout:4/175: mknod da/d1e/c36 0 2024-04-20T11:09:28.230 INFO:tasks.workunit.client.1.smithi173.stdout:4/176: stat da/l34 0 2024-04-20T11:09:28.234 INFO:tasks.workunit.client.1.smithi173.stdout:9/212: dwrite d5/d9/f1d [0,4194304] 0 2024-04-20T11:09:28.235 INFO:tasks.workunit.client.1.smithi173.stdout:2/157: sync 2024-04-20T11:09:28.243 INFO:tasks.workunit.client.1.smithi173.stdout:6/176: truncate d4/f20 38579 0 2024-04-20T11:09:28.244 INFO:tasks.workunit.client.1.smithi173.stdout:0/171: dwrite d6/d8/f2f [0,4194304] 0 2024-04-20T11:09:28.244 INFO:tasks.workunit.client.1.smithi173.stdout:1/151: dwrite f0 [0,4194304] 0 2024-04-20T11:09:28.246 INFO:tasks.workunit.client.1.smithi173.stdout:4/177: write da/dc/d15/f1d [2122224,121686] 0 2024-04-20T11:09:28.247 INFO:tasks.workunit.client.1.smithi173.stdout:7/175: truncate d9/f15 2688828 0 2024-04-20T11:09:28.254 INFO:tasks.workunit.client.1.smithi173.stdout:6/177: rmdir d4/d24 39 2024-04-20T11:09:28.256 INFO:tasks.workunit.client.1.smithi173.stdout:8/229: dread d4/f19 [0,4194304] 0 2024-04-20T11:09:28.257 INFO:tasks.workunit.client.1.smithi173.stdout:6/178: write d4/f11 [143707,62444] 0 2024-04-20T11:09:28.262 INFO:tasks.workunit.client.1.smithi173.stdout:0/172: unlink d6/d8/df/l1b 0 2024-04-20T11:09:28.266 INFO:tasks.workunit.client.1.smithi173.stdout:9/213: mknod d5/c45 0 2024-04-20T11:09:28.268 INFO:tasks.workunit.client.1.smithi173.stdout:0/173: write d6/d8/f1f [2485366,69807] 0 2024-04-20T11:09:28.268 INFO:tasks.workunit.client.1.smithi173.stdout:2/158: dwrite d2/de/d11/f33 [0,4194304] 0 2024-04-20T11:09:28.268 INFO:tasks.workunit.client.1.smithi173.stdout:5/211: link da/de/d19/d32/d30/d35/f38 da/d34/f47 0 2024-04-20T11:09:28.272 INFO:tasks.workunit.client.1.smithi173.stdout:8/230: rmdir d4/d18/d3a/d3c 39 2024-04-20T11:09:28.272 INFO:tasks.workunit.client.1.smithi173.stdout:2/159: fsync d2/de/d11/f1d 0 2024-04-20T11:09:28.273 INFO:tasks.workunit.client.1.smithi173.stdout:5/212: readlink da/de/d19/d2c/l3b 0 2024-04-20T11:09:28.276 INFO:tasks.workunit.client.1.smithi173.stdout:7/176: symlink d9/l3e 0 2024-04-20T11:09:28.277 INFO:tasks.workunit.client.1.smithi173.stdout:1/152: creat d5/d7/da/d20/f33 x:0 0 0 2024-04-20T11:09:28.278 INFO:tasks.workunit.client.1.smithi173.stdout:0/174: fsync d6/d8/df/f16 0 2024-04-20T11:09:28.280 INFO:tasks.workunit.client.1.smithi173.stdout:3/212: dread d7/f22 [0,4194304] 0 2024-04-20T11:09:28.282 INFO:tasks.workunit.client.1.smithi173.stdout:8/231: rmdir d4 39 2024-04-20T11:09:28.283 INFO:tasks.workunit.client.1.smithi173.stdout:1/153: truncate d5/d7/da/f28 132504 0 2024-04-20T11:09:28.286 INFO:tasks.workunit.client.1.smithi173.stdout:1/154: stat d5/d7/da/df/l1a 0 2024-04-20T11:09:28.289 INFO:tasks.workunit.client.1.smithi173.stdout:0/175: rename d6/l25 to d6/d8/df/d15/l32 0 2024-04-20T11:09:28.292 INFO:tasks.workunit.client.1.smithi173.stdout:6/179: creat d4/d14/f39 x:0 0 0 2024-04-20T11:09:28.294 INFO:tasks.workunit.client.1.smithi173.stdout:6/180: readlink d4/l9 0 2024-04-20T11:09:28.301 INFO:tasks.workunit.client.1.smithi173.stdout:1/155: symlink d5/d7/da/df/l34 0 2024-04-20T11:09:28.301 INFO:tasks.workunit.client.1.smithi173.stdout:4/178: dread f8 [0,4194304] 0 2024-04-20T11:09:28.304 INFO:tasks.workunit.client.1.smithi173.stdout:5/213: dwrite da/d12/f1d [0,4194304] 0 2024-04-20T11:09:28.314 INFO:tasks.workunit.client.1.smithi173.stdout:4/179: rmdir da/dc 39 2024-04-20T11:09:28.315 INFO:tasks.workunit.client.1.smithi173.stdout:4/180: readlink da/l1b 0 2024-04-20T11:09:28.319 INFO:tasks.workunit.client.1.smithi173.stdout:9/214: write d5/f2f [519938,58399] 0 2024-04-20T11:09:28.321 INFO:tasks.workunit.client.1.smithi173.stdout:3/213: rename d7/d9/f13 to d7/d9/f4f 0 2024-04-20T11:09:28.324 INFO:tasks.workunit.client.1.smithi173.stdout:2/160: truncate d2/de/d11/f23 2090823 0 2024-04-20T11:09:28.325 INFO:tasks.workunit.client.1.smithi173.stdout:5/214: mkdir da/de/d19/d32/d30/d35/d48 0 2024-04-20T11:09:28.325 INFO:tasks.workunit.client.1.smithi173.stdout:2/161: chown d2/de/c2e 7 1 2024-04-20T11:09:28.327 INFO:tasks.workunit.client.1.smithi173.stdout:4/181: rmdir da 39 2024-04-20T11:09:28.330 INFO:tasks.workunit.client.1.smithi173.stdout:8/232: creat d4/d18/d3a/f46 x:0 0 0 2024-04-20T11:09:28.335 INFO:tasks.workunit.client.1.smithi173.stdout:3/214: unlink d7/d9/f43 0 2024-04-20T11:09:28.343 INFO:tasks.workunit.client.1.smithi173.stdout:5/215: symlink da/de/d19/l49 0 2024-04-20T11:09:28.344 INFO:tasks.workunit.client.1.smithi173.stdout:5/216: readlink da/d12/l13 0 2024-04-20T11:09:28.349 INFO:tasks.workunit.client.1.smithi173.stdout:9/215: mkdir d5/d9/d32/d44/d46 0 2024-04-20T11:09:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:28 smithi117 ceph-mon[15659]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:09:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:28 smithi117 ceph-mon[15659]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:09:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:28 smithi117 ceph-mon[15659]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:09:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:28 smithi117 ceph-mon[15659]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:09:28.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:28 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:09:28.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:28 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:09:28.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:28 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:09:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:28 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:09:28.446 INFO:tasks.workunit.client.1.smithi173.stdout:3/215: fdatasync d7/d9/f3f 0 2024-04-20T11:09:28.448 INFO:tasks.workunit.client.1.smithi173.stdout:3/216: stat d7/d9/d3b 0 2024-04-20T11:09:28.452 INFO:tasks.workunit.client.1.smithi173.stdout:8/233: dwrite d4/f44 [0,4194304] 0 2024-04-20T11:09:28.452 INFO:tasks.workunit.client.1.smithi173.stdout:2/162: rename d2/d13/d28/d2a/d31 to d2/d34 0 2024-04-20T11:09:28.454 INFO:tasks.workunit.client.1.smithi173.stdout:2/163: write d2/de/f2b [551634,63463] 0 2024-04-20T11:09:28.455 INFO:tasks.workunit.client.1.smithi173.stdout:4/182: creat da/d1a/d30/f37 x:0 0 0 2024-04-20T11:09:28.457 INFO:tasks.workunit.client.1.smithi173.stdout:2/164: write d2/de/f21 [227375,4990] 0 2024-04-20T11:09:28.460 INFO:tasks.workunit.client.1.smithi173.stdout:2/165: truncate d2/de/f21 772418 0 2024-04-20T11:09:28.482 INFO:tasks.workunit.client.1.smithi173.stdout:9/216: creat d5/d9/f47 x:0 0 0 2024-04-20T11:09:28.492 INFO:tasks.workunit.client.1.smithi173.stdout:2/166: mknod d2/c35 0 2024-04-20T11:09:28.492 INFO:tasks.workunit.client.1.smithi173.stdout:9/217: mkdir d5/d9/d32/d48 0 2024-04-20T11:09:28.497 INFO:tasks.workunit.client.1.smithi173.stdout:2/167: dread d2/de/f2b [0,4194304] 0 2024-04-20T11:09:28.500 INFO:tasks.workunit.client.1.smithi173.stdout:9/218: creat d5/d9/d20/d2b/f49 x:0 0 0 2024-04-20T11:09:28.501 INFO:tasks.workunit.client.1.smithi173.stdout:3/217: getdents d7/d9/d3b/d45 0 2024-04-20T11:09:28.502 INFO:tasks.workunit.client.1.smithi173.stdout:9/219: chown d5/d9/f26 181273840 1 2024-04-20T11:09:28.505 INFO:tasks.workunit.client.1.smithi173.stdout:2/168: rename d2/de/d11/l26 to d2/de/d11/l36 0 2024-04-20T11:09:28.513 INFO:tasks.workunit.client.1.smithi173.stdout:2/169: getdents d2/d34 0 2024-04-20T11:09:28.517 INFO:tasks.workunit.client.1.smithi173.stdout:3/218: dwrite d7/f47 [0,4194304] 0 2024-04-20T11:09:28.520 INFO:tasks.workunit.client.1.smithi173.stdout:9/220: dwrite d5/d9/d20/d29/d2d/f3f [0,4194304] 0 2024-04-20T11:09:28.527 INFO:tasks.workunit.client.1.smithi173.stdout:9/221: creat d5/d9/d20/d2b/f4a x:0 0 0 2024-04-20T11:09:28.535 INFO:tasks.workunit.client.1.smithi173.stdout:9/222: rmdir d5/d9/d20/d29/d18 39 2024-04-20T11:09:28.538 INFO:tasks.workunit.client.1.smithi173.stdout:3/219: dwrite d7/d9/f4b [0,4194304] 0 2024-04-20T11:09:28.538 INFO:tasks.workunit.client.1.smithi173.stdout:9/223: readlink d5/d9/d20/d2b/l3e 0 2024-04-20T11:09:28.558 INFO:tasks.workunit.client.1.smithi173.stdout:3/220: dwrite d7/d17/f19 [0,4194304] 0 2024-04-20T11:09:28.560 INFO:tasks.workunit.client.1.smithi173.stdout:9/224: dwrite d5/d9/f27 [4194304,4194304] 0 2024-04-20T11:09:28.563 INFO:tasks.workunit.client.1.smithi173.stdout:3/221: creat d7/f50 x:0 0 0 2024-04-20T11:09:28.566 INFO:tasks.workunit.client.1.smithi173.stdout:3/222: rename d7/f50 to d7/f51 0 2024-04-20T11:09:28.574 INFO:tasks.workunit.client.1.smithi173.stdout:7/177: sync 2024-04-20T11:09:28.580 INFO:tasks.workunit.client.1.smithi173.stdout:7/178: rename d9/da/c34 to d9/da/d17/d2b/c3f 0 2024-04-20T11:09:28.582 INFO:tasks.workunit.client.1.smithi173.stdout:7/179: chown d9/da/f27 96 1 2024-04-20T11:09:28.590 INFO:tasks.workunit.client.1.smithi173.stdout:7/180: getdents d9/d1f 0 2024-04-20T11:09:28.593 INFO:tasks.workunit.client.1.smithi173.stdout:7/181: mkdir d9/d33/d40 0 2024-04-20T11:09:28.595 INFO:tasks.workunit.client.1.smithi173.stdout:7/182: rmdir d9/da/d2e 39 2024-04-20T11:09:28.597 INFO:tasks.workunit.client.1.smithi173.stdout:8/234: truncate d4/d18/f30 480455 0 2024-04-20T11:09:28.599 INFO:tasks.workunit.client.1.smithi173.stdout:7/183: symlink d9/da/l41 0 2024-04-20T11:09:28.602 INFO:tasks.workunit.client.1.smithi173.stdout:7/184: mknod d9/da/d31/c42 0 2024-04-20T11:09:28.622 INFO:tasks.workunit.client.1.smithi173.stdout:2/170: dwrite d2/d13/d28/f15 [0,4194304] 0 2024-04-20T11:09:28.623 INFO:tasks.workunit.client.1.smithi173.stdout:2/171: read d2/de/d11/f1d [1552298,5522] 0 2024-04-20T11:09:28.671 INFO:tasks.workunit.client.1.smithi173.stdout:7/185: truncate d9/da/f27 859442 0 2024-04-20T11:09:28.674 INFO:tasks.workunit.client.1.smithi173.stdout:3/223: dwrite d7/d9/f3f [0,4194304] 0 2024-04-20T11:09:28.680 INFO:tasks.workunit.client.1.smithi173.stdout:3/224: symlink d7/d2b/l52 0 2024-04-20T11:09:28.683 INFO:tasks.workunit.client.1.smithi173.stdout:7/186: mkdir d9/da/d43 0 2024-04-20T11:09:28.687 INFO:tasks.workunit.client.1.smithi173.stdout:2/172: rmdir d2/d13/d28/d2a 0 2024-04-20T11:09:28.689 INFO:tasks.workunit.client.1.smithi173.stdout:7/187: creat d9/d33/d35/f44 x:0 0 0 2024-04-20T11:09:28.689 INFO:tasks.workunit.client.1.smithi173.stdout:2/173: chown d2/de/d11/l25 59 1 2024-04-20T11:09:28.691 INFO:tasks.workunit.client.1.smithi173.stdout:7/188: chown d9/da/d17/c2d 0 1 2024-04-20T11:09:28.692 INFO:tasks.workunit.client.1.smithi173.stdout:2/174: rmdir d2/d29 39 2024-04-20T11:09:28.695 INFO:tasks.workunit.client.1.smithi173.stdout:3/225: getdents d7/d9/d3b 0 2024-04-20T11:09:28.696 INFO:tasks.workunit.client.1.smithi173.stdout:2/175: mknod d2/d13/c37 0 2024-04-20T11:09:28.698 INFO:tasks.workunit.client.1.smithi173.stdout:3/226: fsync d7/d9/f28 0 2024-04-20T11:09:28.698 INFO:tasks.workunit.client.1.smithi173.stdout:7/189: symlink d9/da/l45 0 2024-04-20T11:09:28.706 INFO:tasks.workunit.client.1.smithi173.stdout:3/227: rename d7/d9/d3b/d4e to d7/d53 0 2024-04-20T11:09:28.711 INFO:tasks.workunit.client.1.smithi173.stdout:7/190: unlink d9/da/d2e/f2a 0 2024-04-20T11:09:28.718 INFO:tasks.workunit.client.1.smithi173.stdout:0/176: sync 2024-04-20T11:09:28.719 INFO:tasks.workunit.client.1.smithi173.stdout:2/176: dwrite d2/de/f2b [0,4194304] 0 2024-04-20T11:09:28.725 INFO:tasks.workunit.client.1.smithi173.stdout:3/228: dwrite d7/d2b/f36 [0,4194304] 0 2024-04-20T11:09:28.728 INFO:tasks.workunit.client.1.smithi173.stdout:2/177: creat d2/d13/f38 x:0 0 0 2024-04-20T11:09:28.729 INFO:tasks.workunit.client.1.smithi173.stdout:3/229: truncate d7/f51 890890 0 2024-04-20T11:09:28.737 INFO:tasks.workunit.client.1.smithi173.stdout:7/191: truncate d9/da/d17/f1a 1448974 0 2024-04-20T11:09:28.738 INFO:tasks.workunit.client.1.smithi173.stdout:0/177: dread d6/d8/df/f1d [0,4194304] 0 2024-04-20T11:09:28.743 INFO:tasks.workunit.client.1.smithi173.stdout:2/178: rename d2/l4 to d2/d29/l39 0 2024-04-20T11:09:28.745 INFO:tasks.workunit.client.1.smithi173.stdout:2/179: readlink d2/de/d11/l9 0 2024-04-20T11:09:28.747 INFO:tasks.workunit.client.1.smithi173.stdout:3/230: dwrite d7/d2b/f36 [0,4194304] 0 2024-04-20T11:09:28.749 INFO:tasks.workunit.client.1.smithi173.stdout:3/231: dread - d7/d9/f46 zero size 2024-04-20T11:09:28.753 INFO:tasks.workunit.client.1.smithi173.stdout:7/192: rename d9/f15 to d9/da/f46 0 2024-04-20T11:09:28.754 INFO:tasks.workunit.client.1.smithi173.stdout:6/181: sync 2024-04-20T11:09:28.756 INFO:tasks.workunit.client.1.smithi173.stdout:3/232: symlink d7/d2b/l54 0 2024-04-20T11:09:28.757 INFO:tasks.workunit.client.1.smithi173.stdout:7/193: creat d9/d33/f47 x:0 0 0 2024-04-20T11:09:28.761 INFO:tasks.workunit.client.1.smithi173.stdout:0/178: dread d6/d8/f19 [0,4194304] 0 2024-04-20T11:09:28.763 INFO:tasks.workunit.client.1.smithi173.stdout:6/182: mknod d4/d1c/d28/c3a 0 2024-04-20T11:09:28.764 INFO:tasks.workunit.client.1.smithi173.stdout:3/233: rmdir d7/d9 39 2024-04-20T11:09:28.765 INFO:tasks.workunit.client.1.smithi173.stdout:6/183: read d4/f27 [249966,77052] 0 2024-04-20T11:09:28.766 INFO:tasks.workunit.client.1.smithi173.stdout:2/180: readlink d2/d29/l39 0 2024-04-20T11:09:28.767 INFO:tasks.workunit.client.1.smithi173.stdout:2/181: read d2/de/d11/f1d [3703695,16569] 0 2024-04-20T11:09:28.770 INFO:tasks.workunit.client.1.smithi173.stdout:7/194: rename d9/d1f/f28 to d9/d33/f48 0 2024-04-20T11:09:28.771 INFO:tasks.workunit.client.1.smithi173.stdout:0/179: symlink d6/d24/l33 0 2024-04-20T11:09:28.776 INFO:tasks.workunit.client.1.smithi173.stdout:2/182: creat d2/d34/f3a x:0 0 0 2024-04-20T11:09:28.810 INFO:tasks.workunit.client.1.smithi173.stdout:2/183: creat d2/de/f3b x:0 0 0 2024-04-20T11:09:28.813 INFO:tasks.workunit.client.1.smithi173.stdout:3/234: dwrite d7/f22 [4194304,4194304] 0 2024-04-20T11:09:28.814 INFO:tasks.workunit.client.1.smithi173.stdout:2/184: symlink d2/de/l3c 0 2024-04-20T11:09:28.819 INFO:tasks.workunit.client.1.smithi173.stdout:2/185: creat d2/d13/f3d x:0 0 0 2024-04-20T11:09:28.824 INFO:tasks.workunit.client.1.smithi173.stdout:3/235: mknod d7/d9/c55 0 2024-04-20T11:09:28.825 INFO:tasks.workunit.client.1.smithi173.stdout:0/180: dread d6/d8/f1f [0,4194304] 0 2024-04-20T11:09:28.829 INFO:tasks.workunit.client.1.smithi173.stdout:0/181: mknod d6/d8/c34 0 2024-04-20T11:09:28.830 INFO:tasks.workunit.client.1.smithi173.stdout:3/236: truncate d7/d9/d30/f44 1136837 0 2024-04-20T11:09:28.831 INFO:tasks.workunit.client.1.smithi173.stdout:3/237: chown d7/d9/c1d 356850751 1 2024-04-20T11:09:28.831 INFO:tasks.workunit.client.1.smithi173.stdout:7/195: dread d9/da/d17/f1a [0,4194304] 0 2024-04-20T11:09:28.834 INFO:tasks.workunit.client.1.smithi173.stdout:7/196: write d9/da/f25 [788097,61707] 0 2024-04-20T11:09:28.834 INFO:tasks.workunit.client.1.smithi173.stdout:0/182: unlink d6/d8/df/l23 0 2024-04-20T11:09:28.836 INFO:tasks.workunit.client.1.smithi173.stdout:7/197: chown d9/da/f14 157418 1 2024-04-20T11:09:28.836 INFO:tasks.workunit.client.1.smithi173.stdout:3/238: creat d7/d9/d3b/d45/f56 x:0 0 0 2024-04-20T11:09:28.840 INFO:tasks.workunit.client.1.smithi173.stdout:2/186: write d2/de/d11/fd [1155503,9838] 0 2024-04-20T11:09:28.841 INFO:tasks.workunit.client.1.smithi173.stdout:3/239: creat d7/d17/f57 x:0 0 0 2024-04-20T11:09:28.842 INFO:tasks.workunit.client.1.smithi173.stdout:0/183: creat d6/d8/df/d2c/f35 x:0 0 0 2024-04-20T11:09:28.843 INFO:tasks.workunit.client.1.smithi173.stdout:7/198: creat d9/d33/d40/f49 x:0 0 0 2024-04-20T11:09:28.843 INFO:tasks.workunit.client.1.smithi173.stdout:3/240: write d7/d2b/f36 [84670,10362] 0 2024-04-20T11:09:28.847 INFO:tasks.workunit.client.1.smithi173.stdout:7/199: read d9/da/d2e/f37 [1765887,22275] 0 2024-04-20T11:09:28.851 INFO:tasks.workunit.client.1.smithi173.stdout:3/241: fsync d7/d17/f29 0 2024-04-20T11:09:28.854 INFO:tasks.workunit.client.1.smithi173.stdout:2/187: dread d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:28.859 INFO:tasks.workunit.client.1.smithi173.stdout:0/184: write d6/d8/df/f16 [113230,128914] 0 2024-04-20T11:09:28.861 INFO:tasks.workunit.client.1.smithi173.stdout:7/200: dread d9/da/f12 [0,4194304] 0 2024-04-20T11:09:28.862 INFO:tasks.workunit.client.1.smithi173.stdout:2/188: unlink d2/d29/l30 0 2024-04-20T11:09:28.862 INFO:tasks.workunit.client.1.smithi173.stdout:7/201: stat d9/da/d17/f1b 0 2024-04-20T11:09:28.863 INFO:tasks.workunit.client.1.smithi173.stdout:0/185: chown d6/d8/df/d15/c2b 0 1 2024-04-20T11:09:28.866 INFO:tasks.workunit.client.1.smithi173.stdout:3/242: truncate d7/d17/f29 1497445 0 2024-04-20T11:09:28.868 INFO:tasks.workunit.client.1.smithi173.stdout:7/202: mkdir d9/d33/d35/d4a 0 2024-04-20T11:09:28.869 INFO:tasks.workunit.client.1.smithi173.stdout:7/203: dread - d9/d33/d40/f49 zero size 2024-04-20T11:09:28.870 INFO:tasks.workunit.client.1.smithi173.stdout:0/186: rename d6/d8/df/d2c/f35 to d6/d8/df/d2c/d2d/f36 0 2024-04-20T11:09:28.875 INFO:tasks.workunit.client.1.smithi173.stdout:7/204: dread d9/da/f10 [0,4194304] 0 2024-04-20T11:09:28.876 INFO:tasks.workunit.client.1.smithi173.stdout:4/183: sync 2024-04-20T11:09:28.876 INFO:tasks.workunit.client.1.smithi173.stdout:9/225: sync 2024-04-20T11:09:28.876 INFO:tasks.workunit.client.1.smithi173.stdout:5/217: sync 2024-04-20T11:09:28.876 INFO:tasks.workunit.client.1.smithi173.stdout:8/235: sync 2024-04-20T11:09:28.877 INFO:tasks.workunit.client.1.smithi173.stdout:1/156: sync 2024-04-20T11:09:28.877 INFO:tasks.workunit.client.1.smithi173.stdout:0/187: readlink d6/d8/df/l21 0 2024-04-20T11:09:28.878 INFO:tasks.workunit.client.1.smithi173.stdout:2/189: getdents d2/d34 0 2024-04-20T11:09:28.880 INFO:tasks.workunit.client.1.smithi173.stdout:8/236: read d4/fd [5581019,88250] 0 2024-04-20T11:09:28.881 INFO:tasks.workunit.client.1.smithi173.stdout:1/157: write d5/d7/d12/f31 [107232,95284] 0 2024-04-20T11:09:28.894 INFO:tasks.workunit.client.1.smithi173.stdout:8/237: fdatasync d4/d18/d3a/d3c/f37 0 2024-04-20T11:09:28.894 INFO:tasks.workunit.client.1.smithi173.stdout:4/184: dwrite da/f25 [0,4194304] 0 2024-04-20T11:09:28.894 INFO:tasks.workunit.client.1.smithi173.stdout:3/243: dread d7/f25 [0,4194304] 0 2024-04-20T11:09:28.895 INFO:tasks.workunit.client.1.smithi173.stdout:1/158: write f4 [2208248,42666] 0 2024-04-20T11:09:28.895 INFO:tasks.workunit.client.1.smithi173.stdout:3/244: chown d7/c1c 1767 1 2024-04-20T11:09:28.897 INFO:tasks.workunit.client.1.smithi173.stdout:3/245: dread - d7/d9/f46 zero size 2024-04-20T11:09:28.908 INFO:tasks.workunit.client.1.smithi173.stdout:4/185: write da/dc/d13/f32 [7032909,44748] 0 2024-04-20T11:09:28.908 INFO:tasks.workunit.client.1.smithi173.stdout:1/159: symlink d5/d6/d17/l35 0 2024-04-20T11:09:28.910 INFO:tasks.workunit.client.1.smithi173.stdout:1/160: chown d5/d6/dc 45467 1 2024-04-20T11:09:28.917 INFO:tasks.workunit.client.1.smithi173.stdout:3/246: mkdir d7/d17/d58 0 2024-04-20T11:09:28.918 INFO:tasks.workunit.client.1.smithi173.stdout:2/190: creat d2/f3e x:0 0 0 2024-04-20T11:09:28.918 INFO:tasks.workunit.client.1.smithi173.stdout:4/186: truncate da/d1a/d30/f37 161076 0 2024-04-20T11:09:28.919 INFO:tasks.workunit.client.1.smithi173.stdout:9/226: rename d5/d9/f27 to d5/f4b 0 2024-04-20T11:09:28.919 INFO:tasks.workunit.client.1.smithi173.stdout:5/218: link da/de/d19/d32/d30/c39 da/d12/c4a 0 2024-04-20T11:09:28.920 INFO:tasks.workunit.client.1.smithi173.stdout:0/188: link d6/c10 d6/c37 0 2024-04-20T11:09:28.920 INFO:tasks.workunit.client.1.smithi173.stdout:7/205: getdents d9/d33/d35 0 2024-04-20T11:09:28.921 INFO:tasks.workunit.client.1.smithi173.stdout:1/161: symlink d5/d7/da/d20/l36 0 2024-04-20T11:09:28.922 INFO:tasks.workunit.client.1.smithi173.stdout:4/187: fsync da/dc/d15/f1d 0 2024-04-20T11:09:28.922 INFO:tasks.workunit.client.1.smithi173.stdout:2/191: dread - d2/de/f3b zero size 2024-04-20T11:09:28.925 INFO:tasks.workunit.client.1.smithi173.stdout:1/162: readlink d5/d7/da/d20/l36 0 2024-04-20T11:09:28.926 INFO:tasks.workunit.client.1.smithi173.stdout:7/206: creat d9/d33/d40/f4b x:0 0 0 2024-04-20T11:09:28.927 INFO:tasks.workunit.client.1.smithi173.stdout:0/189: dread d6/d8/df/f1d [0,4194304] 0 2024-04-20T11:09:28.928 INFO:tasks.workunit.client.1.smithi173.stdout:5/219: rename da/de/d19/l1a to da/de/d19/d32/d30/d43/l4b 0 2024-04-20T11:09:28.928 INFO:tasks.workunit.client.1.smithi173.stdout:1/163: write f0 [713734,105369] 0 2024-04-20T11:09:28.929 INFO:tasks.workunit.client.1.smithi173.stdout:7/207: chown d9/d33/d40/f4b 8716459 1 2024-04-20T11:09:28.929 INFO:tasks.workunit.client.1.smithi173.stdout:5/220: stat da/de 0 2024-04-20T11:09:28.932 INFO:tasks.workunit.client.1.smithi173.stdout:1/164: write d5/d7/d12/d1f/f26 [129778,91236] 0 2024-04-20T11:09:28.934 INFO:tasks.workunit.client.1.smithi173.stdout:1/165: fdatasync d5/d7/f19 0 2024-04-20T11:09:28.936 INFO:tasks.workunit.client.1.smithi173.stdout:3/247: dread d7/d2b/f36 [0,4194304] 0 2024-04-20T11:09:28.936 INFO:tasks.workunit.client.1.smithi173.stdout:1/166: write f4 [2127388,7495] 0 2024-04-20T11:09:28.937 INFO:tasks.workunit.client.1.smithi173.stdout:3/248: fsync d7/f47 0 2024-04-20T11:09:28.937 INFO:tasks.workunit.client.1.smithi173.stdout:2/192: dread d2/de/d11/fd [0,4194304] 0 2024-04-20T11:09:28.938 INFO:tasks.workunit.client.1.smithi173.stdout:3/249: readlink d7/d2b/l34 0 2024-04-20T11:09:28.939 INFO:tasks.workunit.client.1.smithi173.stdout:2/193: read - d2/de/f3b zero size 2024-04-20T11:09:28.940 INFO:tasks.workunit.client.1.smithi173.stdout:3/250: write d7/d17/f24 [494720,27441] 0 2024-04-20T11:09:28.941 INFO:tasks.workunit.client.1.smithi173.stdout:3/251: dread - d7/d9/f14 zero size 2024-04-20T11:09:28.944 INFO:tasks.workunit.client.1.smithi173.stdout:3/252: write d7/d17/f2c [925453,79558] 0 2024-04-20T11:09:28.947 INFO:tasks.workunit.client.1.smithi173.stdout:4/188: rename da/dc/f2c to da/d1e/f38 0 2024-04-20T11:09:28.956 INFO:tasks.workunit.client.1.smithi173.stdout:3/253: symlink d7/d53/l59 0 2024-04-20T11:09:28.957 INFO:tasks.workunit.client.1.smithi173.stdout:3/254: stat d7/d9/d3b/f38 0 2024-04-20T11:09:28.962 INFO:tasks.workunit.client.1.smithi173.stdout:0/190: rename d6/d8/f19 to d6/d8/df/d2c/d2d/f38 0 2024-04-20T11:09:28.964 INFO:tasks.workunit.client.1.smithi173.stdout:7/208: dwrite d9/da/ff [0,4194304] 0 2024-04-20T11:09:28.966 INFO:tasks.workunit.client.1.smithi173.stdout:1/167: link f0 d5/d6/f37 0 2024-04-20T11:09:28.967 INFO:tasks.workunit.client.1.smithi173.stdout:7/209: truncate d9/d33/d35/f44 51810 0 2024-04-20T11:09:28.968 INFO:tasks.workunit.client.1.smithi173.stdout:1/168: chown d5/d6/dc/c1d 484 1 2024-04-20T11:09:28.968 INFO:tasks.workunit.client.1.smithi173.stdout:4/189: write f1 [4283651,50700] 0 2024-04-20T11:09:28.969 INFO:tasks.workunit.client.1.smithi173.stdout:2/194: creat d2/f3f x:0 0 0 2024-04-20T11:09:28.972 INFO:tasks.workunit.client.1.smithi173.stdout:6/184: sync 2024-04-20T11:09:28.972 INFO:tasks.workunit.client.1.smithi173.stdout:2/195: stat d2/de/f2b 0 2024-04-20T11:09:28.974 INFO:tasks.workunit.client.1.smithi173.stdout:7/210: dread d9/f21 [0,4194304] 0 2024-04-20T11:09:28.974 INFO:tasks.workunit.client.1.smithi173.stdout:2/196: dread - d2/f3e zero size 2024-04-20T11:09:28.974 INFO:tasks.workunit.client.1.smithi173.stdout:2/197: chown d2/d29 246832 1 2024-04-20T11:09:28.980 INFO:tasks.workunit.client.1.smithi173.stdout:4/190: unlink da/l1b 0 2024-04-20T11:09:28.981 INFO:tasks.workunit.client.1.smithi173.stdout:1/169: rmdir d5/d7/d12/d1f 39 2024-04-20T11:09:28.985 INFO:tasks.workunit.client.1.smithi173.stdout:0/191: dread d6/d8/df/d2c/d2d/f38 [0,4194304] 0 2024-04-20T11:09:28.991 INFO:tasks.workunit.client.1.smithi173.stdout:2/198: symlink d2/de/d11/l40 0 2024-04-20T11:09:28.992 INFO:tasks.workunit.client.1.smithi173.stdout:4/191: dread da/dc/d13/f32 [0,4194304] 0 2024-04-20T11:09:28.992 INFO:tasks.workunit.client.1.smithi173.stdout:2/199: chown d2/d13/d28/f15 5602 1 2024-04-20T11:09:28.997 INFO:tasks.workunit.client.1.smithi173.stdout:3/255: link f4 d7/d9/f5a 0 2024-04-20T11:09:28.997 INFO:tasks.workunit.client.1.smithi173.stdout:6/185: write d4/d14/f1a [2016237,52824] 0 2024-04-20T11:09:28.999 INFO:tasks.workunit.client.1.smithi173.stdout:1/170: mkdir d5/d7/da/d20/d38 0 2024-04-20T11:09:29.000 INFO:tasks.workunit.client.1.smithi173.stdout:1/171: chown d5/d7/da/d20/d38 1151 1 2024-04-20T11:09:29.001 INFO:tasks.workunit.client.1.smithi173.stdout:6/186: read d4/f11 [238173,117978] 0 2024-04-20T11:09:29.002 INFO:tasks.workunit.client.1.smithi173.stdout:0/192: readlink d6/d8/df/d15/l28 0 2024-04-20T11:09:29.003 INFO:tasks.workunit.client.1.smithi173.stdout:6/187: write d4/d18/f2b [5114909,37171] 0 2024-04-20T11:09:29.003 INFO:tasks.workunit.client.1.smithi173.stdout:5/221: rename da/de/d19/d32/f3a to da/de/d19/d32/d30/f4c 0 2024-04-20T11:09:29.005 INFO:tasks.workunit.client.1.smithi173.stdout:7/211: creat d9/d33/d35/d4a/f4c x:0 0 0 2024-04-20T11:09:29.005 INFO:tasks.workunit.client.1.smithi173.stdout:5/222: fdatasync da/de/d19/f2a 0 2024-04-20T11:09:29.006 INFO:tasks.workunit.client.1.smithi173.stdout:4/192: mkdir da/d1a/d30/d39 0 2024-04-20T11:09:29.007 INFO:tasks.workunit.client.1.smithi173.stdout:2/200: symlink d2/d34/l41 0 2024-04-20T11:09:29.007 INFO:tasks.workunit.client.1.smithi173.stdout:5/223: truncate da/d34/f47 1161472 0 2024-04-20T11:09:29.010 INFO:tasks.workunit.client.1.smithi173.stdout:3/256: write f4 [1168113,13996] 0 2024-04-20T11:09:29.011 INFO:tasks.workunit.client.1.smithi173.stdout:5/224: truncate da/de/d19/d32/f15 492392 0 2024-04-20T11:09:29.012 INFO:tasks.workunit.client.1.smithi173.stdout:1/172: creat d5/d7/f39 x:0 0 0 2024-04-20T11:09:29.018 INFO:tasks.workunit.client.1.smithi173.stdout:6/188: chown d4/f20 0 1 2024-04-20T11:09:29.019 INFO:tasks.workunit.client.1.smithi173.stdout:0/193: creat d6/d8/df/d15/f39 x:0 0 0 2024-04-20T11:09:29.020 INFO:tasks.workunit.client.1.smithi173.stdout:6/189: chown d4/d18/l1b 427796714 1 2024-04-20T11:09:29.022 INFO:tasks.workunit.client.1.smithi173.stdout:6/190: fsync d4/d14/f2f 0 2024-04-20T11:09:29.022 INFO:tasks.workunit.client.1.smithi173.stdout:7/212: creat d9/da/d31/f4d x:0 0 0 2024-04-20T11:09:29.024 INFO:tasks.workunit.client.1.smithi173.stdout:2/201: readlink d2/de/l1e 0 2024-04-20T11:09:29.025 INFO:tasks.workunit.client.1.smithi173.stdout:3/257: dwrite d7/d9/f3a [4194304,4194304] 0 2024-04-20T11:09:29.026 INFO:tasks.workunit.client.1.smithi173.stdout:2/202: write d2/d13/d28/f15 [4470901,97535] 0 2024-04-20T11:09:29.026 INFO:tasks.workunit.client.1.smithi173.stdout:3/258: write d7/d9/f3a [6690777,33546] 0 2024-04-20T11:09:29.040 INFO:tasks.workunit.client.1.smithi173.stdout:4/193: truncate da/dc/d15/f18 3844804 0 2024-04-20T11:09:29.040 INFO:tasks.workunit.client.1.smithi173.stdout:4/194: stat f8 0 2024-04-20T11:09:29.043 INFO:tasks.workunit.client.1.smithi173.stdout:4/195: write da/dc/d15/f35 [709929,123170] 0 2024-04-20T11:09:29.046 INFO:tasks.workunit.client.1.smithi173.stdout:3/259: symlink d7/d9/d3b/l5b 0 2024-04-20T11:09:29.047 INFO:tasks.workunit.client.1.smithi173.stdout:3/260: fdatasync d7/f47 0 2024-04-20T11:09:29.047 INFO:tasks.workunit.client.1.smithi173.stdout:2/203: write d2/de/d11/f10 [2621226,58795] 0 2024-04-20T11:09:29.049 INFO:tasks.workunit.client.1.smithi173.stdout:3/261: chown d7/d9/f35 2 1 2024-04-20T11:09:29.050 INFO:tasks.workunit.client.1.smithi173.stdout:5/225: mknod da/d34/c4d 0 2024-04-20T11:09:29.052 INFO:tasks.workunit.client.1.smithi173.stdout:5/226: read da/de/d19/f2a [1017138,120055] 0 2024-04-20T11:09:29.053 INFO:tasks.workunit.client.1.smithi173.stdout:0/194: symlink d6/d8/df/l3a 0 2024-04-20T11:09:29.055 INFO:tasks.workunit.client.1.smithi173.stdout:8/238: sync 2024-04-20T11:09:29.055 INFO:tasks.workunit.client.1.smithi173.stdout:9/227: sync 2024-04-20T11:09:29.055 INFO:tasks.workunit.client.1.smithi173.stdout:5/227: dread - da/de/d19/d32/f3c zero size 2024-04-20T11:09:29.055 INFO:tasks.workunit.client.1.smithi173.stdout:0/195: fdatasync d6/f7 0 2024-04-20T11:09:29.057 INFO:tasks.workunit.client.1.smithi173.stdout:9/228: dread - d5/d9/d32/f41 zero size 2024-04-20T11:09:29.058 INFO:tasks.workunit.client.1.smithi173.stdout:4/196: symlink da/dc/d13/l3a 0 2024-04-20T11:09:29.063 INFO:tasks.workunit.client.1.smithi173.stdout:7/213: creat d9/f4e x:0 0 0 2024-04-20T11:09:29.065 INFO:tasks.workunit.client.1.smithi173.stdout:2/204: dwrite d2/de/f2b [0,4194304] 0 2024-04-20T11:09:29.067 INFO:tasks.workunit.client.1.smithi173.stdout:8/239: fsync d4/f8 0 2024-04-20T11:09:29.067 INFO:tasks.workunit.client.1.smithi173.stdout:2/205: readlink d2/d29/l2f 0 2024-04-20T11:09:29.070 INFO:tasks.workunit.client.1.smithi173.stdout:1/173: truncate d5/d6/dc/fe 3056949 0 2024-04-20T11:09:29.071 INFO:tasks.workunit.client.1.smithi173.stdout:8/240: write d4/d18/d3a/d3c/f37 [1728444,53378] 0 2024-04-20T11:09:29.073 INFO:tasks.workunit.client.1.smithi173.stdout:3/262: mkdir d7/d9/d5c 0 2024-04-20T11:09:29.073 INFO:tasks.workunit.client.1.smithi173.stdout:4/197: creat da/d1a/d30/f3b x:0 0 0 2024-04-20T11:09:29.076 INFO:tasks.workunit.client.1.smithi173.stdout:7/214: unlink l2 0 2024-04-20T11:09:29.079 INFO:tasks.workunit.client.1.smithi173.stdout:0/196: mknod d6/d8/df/c3b 0 2024-04-20T11:09:29.079 INFO:tasks.workunit.client.1.smithi173.stdout:7/215: write f6 [5707798,8057] 0 2024-04-20T11:09:29.080 INFO:tasks.workunit.client.1.smithi173.stdout:5/228: dwrite da/de/d19/d32/f44 [0,4194304] 0 2024-04-20T11:09:29.083 INFO:tasks.workunit.client.1.smithi173.stdout:6/191: getdents d4/d14 0 2024-04-20T11:09:29.089 INFO:tasks.workunit.client.1.smithi173.stdout:7/216: chown d9/da/d17/c2d 423174179 1 2024-04-20T11:09:29.090 INFO:tasks.workunit.client.1.smithi173.stdout:8/241: creat d4/d18/d3a/d3c/f47 x:0 0 0 2024-04-20T11:09:29.093 INFO:tasks.workunit.client.1.smithi173.stdout:9/229: rename d5/d9/d32/d44/d46 to d5/d4c 0 2024-04-20T11:09:29.108 INFO:tasks.workunit.client.1.smithi173.stdout:0/197: truncate d6/d8/df/d2c/d2d/f38 1252607 0 2024-04-20T11:09:29.109 INFO:tasks.workunit.client.1.smithi173.stdout:9/230: dread d5/f2f [0,4194304] 0 2024-04-20T11:09:29.109 INFO:tasks.workunit.client.1.smithi173.stdout:2/206: getdents d2/d34 0 2024-04-20T11:09:29.110 INFO:tasks.workunit.client.1.smithi173.stdout:3/263: dread d7/d9/f4b [0,4194304] 0 2024-04-20T11:09:29.115 INFO:tasks.workunit.client.1.smithi173.stdout:4/198: link l9 da/d1a/l3c 0 2024-04-20T11:09:29.115 INFO:tasks.workunit.client.1.smithi173.stdout:5/229: dread da/de/f2d [0,4194304] 0 2024-04-20T11:09:29.116 INFO:tasks.workunit.client.1.smithi173.stdout:2/207: write d2/de/f3b [859371,85066] 0 2024-04-20T11:09:29.116 INFO:tasks.workunit.client.1.smithi173.stdout:1/174: dwrite d5/d6/d17/f27 [0,4194304] 0 2024-04-20T11:09:29.116 INFO:tasks.workunit.client.1.smithi173.stdout:6/192: write d4/f11 [2236117,118683] 0 2024-04-20T11:09:29.119 INFO:tasks.workunit.client.1.smithi173.stdout:4/199: chown da/dc/d13/c21 1388127 1 2024-04-20T11:09:29.120 INFO:tasks.workunit.client.1.smithi173.stdout:2/208: chown d2/de/f3b 0 1 2024-04-20T11:09:29.123 INFO:tasks.workunit.client.1.smithi173.stdout:1/175: write d5/d6/dc/f2c [59354,4946] 0 2024-04-20T11:09:29.124 INFO:tasks.workunit.client.1.smithi173.stdout:7/217: getdents d9/da 0 2024-04-20T11:09:29.124 INFO:tasks.workunit.client.1.smithi173.stdout:0/198: dread d6/f1e [0,4194304] 0 2024-04-20T11:09:29.125 INFO:tasks.workunit.client.1.smithi173.stdout:9/231: creat d5/d9/d20/d2b/d31/f4d x:0 0 0 2024-04-20T11:09:29.128 INFO:tasks.workunit.client.1.smithi173.stdout:7/218: stat d9/da/d17/d2b 0 2024-04-20T11:09:29.128 INFO:tasks.workunit.client.1.smithi173.stdout:5/230: symlink da/d34/l4e 0 2024-04-20T11:09:29.129 INFO:tasks.workunit.client.1.smithi173.stdout:6/193: mkdir d4/d1c/d28/d3b 0 2024-04-20T11:09:29.130 INFO:tasks.workunit.client.1.smithi173.stdout:4/200: dread da/d1a/d30/f37 [0,4194304] 0 2024-04-20T11:09:29.133 INFO:tasks.workunit.client.1.smithi173.stdout:9/232: chown d5/d9/d20/d29/d2d/c3d 61 1 2024-04-20T11:09:29.133 INFO:tasks.workunit.client.1.smithi173.stdout:7/219: symlink d9/d1f/l4f 0 2024-04-20T11:09:29.136 INFO:tasks.workunit.client.1.smithi173.stdout:0/199: chown d6/d8/df/l17 111285645 1 2024-04-20T11:09:29.136 INFO:tasks.workunit.client.1.smithi173.stdout:8/242: dwrite d4/f19 [0,4194304] 0 2024-04-20T11:09:29.142 INFO:tasks.workunit.client.1.smithi173.stdout:8/243: write d4/d18/d3a/f46 [60329,130149] 0 2024-04-20T11:09:29.143 INFO:tasks.workunit.client.1.smithi173.stdout:5/231: mkdir da/d4f 0 2024-04-20T11:09:29.145 INFO:tasks.workunit.client.1.smithi173.stdout:5/232: chown da/de/d19/c20 30042461 1 2024-04-20T11:09:29.147 INFO:tasks.workunit.client.1.smithi173.stdout:1/176: rmdir d5/d6/d17/d1b 0 2024-04-20T11:09:29.148 INFO:tasks.workunit.client.1.smithi173.stdout:1/177: dread - d5/d7/f39 zero size 2024-04-20T11:09:29.149 INFO:tasks.workunit.client.1.smithi173.stdout:0/200: mkdir d6/d8/d3c 0 2024-04-20T11:09:29.157 INFO:tasks.workunit.client.1.smithi173.stdout:2/209: dwrite d2/d13/f3d [0,4194304] 0 2024-04-20T11:09:29.160 INFO:tasks.workunit.client.1.smithi173.stdout:5/233: fsync da/de/f2d 0 2024-04-20T11:09:29.162 INFO:tasks.workunit.client.1.smithi173.stdout:0/201: mknod d6/d8/df/d2c/d2d/c3d 0 2024-04-20T11:09:29.162 INFO:tasks.workunit.client.1.smithi173.stdout:8/244: read d4/d18/d3a/f46 [22886,62831] 0 2024-04-20T11:09:29.170 INFO:tasks.workunit.client.1.smithi173.stdout:4/201: dwrite da/d1a/d30/f3b [0,4194304] 0 2024-04-20T11:09:29.175 INFO:tasks.workunit.client.1.smithi173.stdout:4/202: read f8 [832743,101100] 0 2024-04-20T11:09:29.180 INFO:tasks.workunit.client.1.smithi173.stdout:2/210: link d2/de/d11/f10 d2/d29/f42 0 2024-04-20T11:09:29.181 INFO:tasks.workunit.client.1.smithi173.stdout:0/202: rename d6/lc to d6/d8/df/d15/l3e 0 2024-04-20T11:09:29.183 INFO:tasks.workunit.client.1.smithi173.stdout:6/194: dread d4/f11 [0,4194304] 0 2024-04-20T11:09:29.185 INFO:tasks.workunit.client.1.smithi173.stdout:8/245: rmdir d4/d18/d3a/d3c 39 2024-04-20T11:09:29.188 INFO:tasks.workunit.client.1.smithi173.stdout:9/233: dwrite d5/d9/fa [4194304,4194304] 0 2024-04-20T11:09:29.190 INFO:tasks.workunit.client.1.smithi173.stdout:2/211: creat d2/de/f43 x:0 0 0 2024-04-20T11:09:29.190 INFO:tasks.workunit.client.1.smithi173.stdout:7/220: dwrite d9/d33/d35/d4a/f4c [0,4194304] 0 2024-04-20T11:09:29.191 INFO:tasks.workunit.client.1.smithi173.stdout:4/203: mknod da/d1a/d30/d39/c3d 0 2024-04-20T11:09:29.192 INFO:tasks.workunit.client.1.smithi173.stdout:6/195: creat d4/d14/f3c x:0 0 0 2024-04-20T11:09:29.193 INFO:tasks.workunit.client.1.smithi173.stdout:7/221: fsync d9/d33/f3a 0 2024-04-20T11:09:29.200 INFO:tasks.workunit.client.1.smithi173.stdout:4/204: stat da/dc/d13 0 2024-04-20T11:09:29.200 INFO:tasks.workunit.client.1.smithi173.stdout:7/222: fdatasync d9/da/d31/f38 0 2024-04-20T11:09:29.200 INFO:tasks.workunit.client.1.smithi173.stdout:1/178: dwrite d5/d6/d17/f32 [0,4194304] 0 2024-04-20T11:09:29.200 INFO:tasks.workunit.client.1.smithi173.stdout:8/246: link d4/f19 d4/d18/f48 0 2024-04-20T11:09:29.201 INFO:tasks.workunit.client.1.smithi173.stdout:2/212: rename d2/d29/f42 to d2/d13/f44 0 2024-04-20T11:09:29.204 INFO:tasks.workunit.client.1.smithi173.stdout:1/179: chown d5/d7/d12/d1c 4835 1 2024-04-20T11:09:29.208 INFO:tasks.workunit.client.1.smithi173.stdout:7/223: fsync f1 0 2024-04-20T11:09:29.212 INFO:tasks.workunit.client.1.smithi173.stdout:2/213: write d2/d13/f3d [4776509,24182] 0 2024-04-20T11:09:29.217 INFO:tasks.workunit.client.1.smithi173.stdout:0/203: dwrite d6/d8/df/d15/f2e [0,4194304] 0 2024-04-20T11:09:29.224 INFO:tasks.workunit.client.1.smithi173.stdout:8/247: dread d4/d18/f21 [0,4194304] 0 2024-04-20T11:09:29.228 INFO:tasks.workunit.client.1.smithi173.stdout:6/196: dwrite d4/d24/f35 [0,4194304] 0 2024-04-20T11:09:29.231 INFO:tasks.workunit.client.1.smithi173.stdout:7/224: readlink d9/da/le 0 2024-04-20T11:09:29.237 INFO:tasks.workunit.client.1.smithi173.stdout:9/234: getdents d5/d9 0 2024-04-20T11:09:29.242 INFO:tasks.workunit.client.1.smithi173.stdout:0/204: dwrite d6/d8/df/d15/f39 [0,4194304] 0 2024-04-20T11:09:29.244 INFO:tasks.workunit.client.1.smithi173.stdout:6/197: rename d4/l9 to d4/d1c/l3d 0 2024-04-20T11:09:29.246 INFO:tasks.workunit.client.1.smithi173.stdout:0/205: dread d6/d8/df/f1d [0,4194304] 0 2024-04-20T11:09:29.248 INFO:tasks.workunit.client.1.smithi173.stdout:7/225: mkdir d9/d33/d50 0 2024-04-20T11:09:29.249 INFO:tasks.workunit.client.1.smithi173.stdout:0/206: fdatasync d6/d8/df/f16 0 2024-04-20T11:09:29.250 INFO:tasks.workunit.client.1.smithi173.stdout:4/205: link da/dc/d15/c23 da/d1a/d30/c3e 0 2024-04-20T11:09:29.259 INFO:tasks.workunit.client.1.smithi173.stdout:1/180: dread d5/fb [0,4194304] 0 2024-04-20T11:09:29.261 INFO:tasks.workunit.client.1.smithi173.stdout:7/226: rename d9/d1f/c29 to d9/d1f/c51 0 2024-04-20T11:09:29.264 INFO:tasks.workunit.client.1.smithi173.stdout:6/198: chown d4/d14/c1f 55 1 2024-04-20T11:09:29.264 INFO:tasks.workunit.client.1.smithi173.stdout:8/248: dwrite d4/d18/d3a/d3c/f37 [0,4194304] 0 2024-04-20T11:09:29.266 INFO:tasks.workunit.client.1.smithi173.stdout:0/207: symlink d6/l3f 0 2024-04-20T11:09:29.270 INFO:tasks.workunit.client.1.smithi173.stdout:7/227: mkdir d9/d33/d35/d4a/d52 0 2024-04-20T11:09:29.273 INFO:tasks.workunit.client.1.smithi173.stdout:8/249: mknod d4/d18/d3a/d3c/c49 0 2024-04-20T11:09:29.273 INFO:tasks.workunit.client.1.smithi173.stdout:9/235: dwrite d5/f4b [4194304,4194304] 0 2024-04-20T11:09:29.276 INFO:tasks.workunit.client.1.smithi173.stdout:3/264: sync 2024-04-20T11:09:29.277 INFO:tasks.workunit.client.1.smithi173.stdout:7/228: symlink d9/d1f/l53 0 2024-04-20T11:09:29.277 INFO:tasks.workunit.client.1.smithi173.stdout:4/206: dwrite da/dc/d13/f29 [0,4194304] 0 2024-04-20T11:09:29.277 INFO:tasks.workunit.client.1.smithi173.stdout:1/181: creat d5/d6/dc/f3a x:0 0 0 2024-04-20T11:09:29.278 INFO:tasks.workunit.client.1.smithi173.stdout:9/236: readlink d5/d9/d20/l37 0 2024-04-20T11:09:29.281 INFO:tasks.workunit.client.1.smithi173.stdout:4/207: chown da/c2f 5177 1 2024-04-20T11:09:29.283 INFO:tasks.workunit.client.1.smithi173.stdout:3/265: chown d7/d2b/f41 170489637 1 2024-04-20T11:09:29.283 INFO:tasks.workunit.client.1.smithi173.stdout:3/266: chown d7/d53 1 1 2024-04-20T11:09:29.288 INFO:tasks.workunit.client.1.smithi173.stdout:3/267: dread - d7/d9/d3b/d45/f56 zero size 2024-04-20T11:09:29.289 INFO:tasks.workunit.client.1.smithi173.stdout:7/229: truncate f1 3563565 0 2024-04-20T11:09:29.290 INFO:tasks.workunit.client.1.smithi173.stdout:6/199: dread d4/fe [0,4194304] 0 2024-04-20T11:09:29.291 INFO:tasks.workunit.client.1.smithi173.stdout:9/237: creat d5/d9/d32/f4e x:0 0 0 2024-04-20T11:09:29.291 INFO:tasks.workunit.client.1.smithi173.stdout:4/208: symlink da/dc/d15/l3f 0 2024-04-20T11:09:29.296 INFO:tasks.workunit.client.1.smithi173.stdout:3/268: mknod d7/d2b/c5d 0 2024-04-20T11:09:29.297 INFO:tasks.workunit.client.1.smithi173.stdout:7/230: mkdir d9/d33/d35/d54 0 2024-04-20T11:09:29.298 INFO:tasks.workunit.client.1.smithi173.stdout:9/238: truncate d5/d9/d32/f4e 710350 0 2024-04-20T11:09:29.300 INFO:tasks.workunit.client.1.smithi173.stdout:0/208: link d6/d8/df/d15/l3e d6/d8/l40 0 2024-04-20T11:09:29.303 INFO:tasks.workunit.client.1.smithi173.stdout:4/209: truncate da/d1e/f38 809873 0 2024-04-20T11:09:29.304 INFO:tasks.workunit.client.1.smithi173.stdout:1/182: dwrite d5/d6/f9 [0,4194304] 0 2024-04-20T11:09:29.305 INFO:tasks.workunit.client.1.smithi173.stdout:0/209: truncate d6/f29 602316 0 2024-04-20T11:09:29.351 INFO:tasks.workunit.client.1.smithi173.stdout:4/210: write da/dc/d13/f2b [406697,123900] 0 2024-04-20T11:09:29.351 INFO:tasks.workunit.client.1.smithi173.stdout:0/210: write d6/d8/fe [2057263,5513] 0 2024-04-20T11:09:29.355 INFO:tasks.workunit.client.1.smithi173.stdout:8/250: dwrite d4/f32 [0,4194304] 0 2024-04-20T11:09:29.358 INFO:tasks.workunit.client.1.smithi173.stdout:6/200: dwrite d4/f38 [0,4194304] 0 2024-04-20T11:09:29.362 INFO:tasks.workunit.client.1.smithi173.stdout:8/251: fdatasync d4/d18/f48 0 2024-04-20T11:09:29.363 INFO:tasks.workunit.client.1.smithi173.stdout:3/269: dwrite d7/d9/f4f [0,4194304] 0 2024-04-20T11:09:29.366 INFO:tasks.workunit.client.1.smithi173.stdout:0/211: creat d6/d8/d3c/f41 x:0 0 0 2024-04-20T11:09:29.369 INFO:tasks.workunit.client.1.smithi173.stdout:8/252: chown d4/d18/d3a/d3c/f34 41 1 2024-04-20T11:09:29.373 INFO:tasks.workunit.client.1.smithi173.stdout:1/183: dwrite d5/d7/d12/f31 [0,4194304] 0 2024-04-20T11:09:29.373 INFO:tasks.workunit.client.1.smithi173.stdout:1/184: stat d5/d7/da 0 2024-04-20T11:09:29.382 INFO:tasks.workunit.client.1.smithi173.stdout:0/212: mkdir d6/d8/df/d2c/d42 0 2024-04-20T11:09:29.386 INFO:tasks.workunit.client.1.smithi173.stdout:1/185: truncate d5/d6/d17/f32 4438674 0 2024-04-20T11:09:29.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:29 smithi117 ceph-mon[15659]: pgmap v24: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 71 MiB/s rd, 164 MiB/s wr, 422 op/s 2024-04-20T11:09:29.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:29 smithi117 ceph-mon[15659]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:09:29.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:29 smithi117 ceph-mon[15659]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:09:29.390 INFO:tasks.workunit.client.1.smithi173.stdout:0/213: write d6/d8/df/d2c/d2d/f36 [855812,109773] 0 2024-04-20T11:09:29.393 INFO:tasks.workunit.client.1.smithi173.stdout:3/270: dwrite d7/d9/f46 [0,4194304] 0 2024-04-20T11:09:29.396 INFO:tasks.workunit.client.1.smithi173.stdout:8/253: dread d4/f44 [0,4194304] 0 2024-04-20T11:09:29.396 INFO:tasks.workunit.client.1.smithi173.stdout:1/186: dread - d5/d7/d12/d1f/f2f zero size 2024-04-20T11:09:29.396 INFO:tasks.workunit.client.1.smithi173.stdout:5/234: sync 2024-04-20T11:09:29.399 INFO:tasks.workunit.client.1.smithi173.stdout:6/201: dwrite d4/d14/f3c [0,4194304] 0 2024-04-20T11:09:29.403 INFO:tasks.workunit.client.1.smithi173.stdout:5/235: fdatasync da/de/ff 0 2024-04-20T11:09:29.405 INFO:tasks.workunit.client.1.smithi173.stdout:8/254: fdatasync d4/d18/f21 0 2024-04-20T11:09:29.406 INFO:tasks.workunit.client.1.smithi173.stdout:1/187: truncate d5/f30 414104 0 2024-04-20T11:09:29.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:29 smithi173 ceph-mon[23800]: pgmap v24: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 71 MiB/s rd, 164 MiB/s wr, 422 op/s 2024-04-20T11:09:29.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:29 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:09:29.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:29 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:09:29.414 INFO:tasks.workunit.client.1.smithi173.stdout:6/202: mkdir d4/d24/d3e 0 2024-04-20T11:09:29.415 INFO:tasks.workunit.client.1.smithi173.stdout:4/211: dread f1 [0,4194304] 0 2024-04-20T11:09:29.418 INFO:tasks.workunit.client.1.smithi173.stdout:5/236: symlink da/de/d19/d32/l50 0 2024-04-20T11:09:29.420 INFO:tasks.workunit.client.1.smithi173.stdout:1/188: mkdir d5/d6/d3b 0 2024-04-20T11:09:29.420 INFO:tasks.workunit.client.1.smithi173.stdout:4/212: creat da/d1a/f40 x:0 0 0 2024-04-20T11:09:29.421 INFO:tasks.workunit.client.1.smithi173.stdout:0/214: dwrite d6/d8/df/d15/f39 [0,4194304] 0 2024-04-20T11:09:29.422 INFO:tasks.workunit.client.1.smithi173.stdout:3/271: dwrite d7/f18 [0,4194304] 0 2024-04-20T11:09:29.422 INFO:tasks.workunit.client.1.smithi173.stdout:8/255: write d4/fd [7910493,43334] 0 2024-04-20T11:09:29.427 INFO:tasks.workunit.client.1.smithi173.stdout:5/237: creat da/d12/f51 x:0 0 0 2024-04-20T11:09:29.429 INFO:tasks.workunit.client.1.smithi173.stdout:3/272: rmdir d7/d9/d30 39 2024-04-20T11:09:29.430 INFO:tasks.workunit.client.1.smithi173.stdout:0/215: dread d6/f29 [0,4194304] 0 2024-04-20T11:09:29.430 INFO:tasks.workunit.client.1.smithi173.stdout:8/256: rmdir d4 39 2024-04-20T11:09:29.433 INFO:tasks.workunit.client.1.smithi173.stdout:6/203: dread d4/f11 [0,4194304] 0 2024-04-20T11:09:29.436 INFO:tasks.workunit.client.1.smithi173.stdout:3/273: symlink d7/d9/l5e 0 2024-04-20T11:09:29.438 INFO:tasks.workunit.client.1.smithi173.stdout:3/274: write d7/d9/f5a [572191,128625] 0 2024-04-20T11:09:29.440 INFO:tasks.workunit.client.1.smithi173.stdout:1/189: rename d5/d7/d12/d1c/l1e to d5/d7/d12/l3c 0 2024-04-20T11:09:29.442 INFO:tasks.workunit.client.1.smithi173.stdout:6/204: creat d4/d18/f3f x:0 0 0 2024-04-20T11:09:29.445 INFO:tasks.workunit.client.1.smithi173.stdout:1/190: write d5/d6/f9 [3140650,55519] 0 2024-04-20T11:09:29.445 INFO:tasks.workunit.client.1.smithi173.stdout:5/238: truncate da/de/d19/d32/f11 3461615 0 2024-04-20T11:09:29.447 INFO:tasks.workunit.client.1.smithi173.stdout:6/205: dread d4/f20 [0,4194304] 0 2024-04-20T11:09:29.448 INFO:tasks.workunit.client.1.smithi173.stdout:5/239: fdatasync da/de/d19/d32/f15 0 2024-04-20T11:09:29.449 INFO:tasks.workunit.client.1.smithi173.stdout:8/257: rename d4/f17 to d4/d18/f4a 0 2024-04-20T11:09:29.451 INFO:tasks.workunit.client.1.smithi173.stdout:6/206: chown d4/c33 9275418 1 2024-04-20T11:09:29.451 INFO:tasks.workunit.client.1.smithi173.stdout:1/191: symlink d5/d6/d17/l3d 0 2024-04-20T11:09:29.452 INFO:tasks.workunit.client.1.smithi173.stdout:0/216: getdents d6/d8/df/d15 0 2024-04-20T11:09:29.453 INFO:tasks.workunit.client.1.smithi173.stdout:1/192: readlink d5/d6/dc/l23 0 2024-04-20T11:09:29.454 INFO:tasks.workunit.client.1.smithi173.stdout:5/240: creat da/de/d19/d32/f52 x:0 0 0 2024-04-20T11:09:29.455 INFO:tasks.workunit.client.1.smithi173.stdout:0/217: chown d6/d8/df/f16 52 1 2024-04-20T11:09:29.456 INFO:tasks.workunit.client.1.smithi173.stdout:1/193: dread - d5/d7/d12/f2e zero size 2024-04-20T11:09:29.456 INFO:tasks.workunit.client.1.smithi173.stdout:6/207: dread d4/d18/f26 [0,4194304] 0 2024-04-20T11:09:29.460 INFO:tasks.workunit.client.1.smithi173.stdout:4/213: truncate f1 869695 0 2024-04-20T11:09:29.466 INFO:tasks.workunit.client.1.smithi173.stdout:8/258: dwrite d4/d18/d3a/d3c/f40 [0,4194304] 0 2024-04-20T11:09:29.467 INFO:tasks.workunit.client.1.smithi173.stdout:6/208: getdents d4/d24/d3e 0 2024-04-20T11:09:29.469 INFO:tasks.workunit.client.1.smithi173.stdout:6/209: dread d4/f20 [0,4194304] 0 2024-04-20T11:09:29.472 INFO:tasks.workunit.client.1.smithi173.stdout:8/259: chown d4/c3e 231734 1 2024-04-20T11:09:29.473 INFO:tasks.workunit.client.1.smithi173.stdout:6/210: read d4/f16 [2325252,96492] 0 2024-04-20T11:09:29.479 INFO:tasks.workunit.client.1.smithi173.stdout:1/194: getdents d5/d7/d12 0 2024-04-20T11:09:29.479 INFO:tasks.workunit.client.1.smithi173.stdout:8/260: mknod d4/d18/d3a/d3c/c4b 0 2024-04-20T11:09:29.480 INFO:tasks.workunit.client.1.smithi173.stdout:6/211: symlink d4/d1c/l40 0 2024-04-20T11:09:29.481 INFO:tasks.workunit.client.1.smithi173.stdout:4/214: dwrite da/f25 [0,4194304] 0 2024-04-20T11:09:29.481 INFO:tasks.workunit.client.1.smithi173.stdout:1/195: fsync d5/d6/f9 0 2024-04-20T11:09:29.483 INFO:tasks.workunit.client.1.smithi173.stdout:1/196: fdatasync d5/d7/da/f28 0 2024-04-20T11:09:29.485 INFO:tasks.workunit.client.1.smithi173.stdout:5/241: write da/de/d19/d32/f11 [3604801,39694] 0 2024-04-20T11:09:29.486 INFO:tasks.workunit.client.1.smithi173.stdout:8/261: write d4/d18/d3a/f46 [568006,91988] 0 2024-04-20T11:09:29.489 INFO:tasks.workunit.client.1.smithi173.stdout:6/212: write d4/d14/f2f [868254,66195] 0 2024-04-20T11:09:29.492 INFO:tasks.workunit.client.1.smithi173.stdout:1/197: rmdir d5/d7/d12/d1c 39 2024-04-20T11:09:29.498 INFO:tasks.workunit.client.1.smithi173.stdout:1/198: chown d5/d6/d17/l29 192 1 2024-04-20T11:09:29.498 INFO:tasks.workunit.client.1.smithi173.stdout:6/213: creat d4/d1c/d28/d3b/f41 x:0 0 0 2024-04-20T11:09:29.500 INFO:tasks.workunit.client.1.smithi173.stdout:4/215: getdents da 0 2024-04-20T11:09:29.501 INFO:tasks.workunit.client.1.smithi173.stdout:6/214: dread - d4/d1c/d28/d3b/f41 zero size 2024-04-20T11:09:29.506 INFO:tasks.workunit.client.1.smithi173.stdout:8/262: dwrite d4/f8 [0,4194304] 0 2024-04-20T11:09:29.512 INFO:tasks.workunit.client.1.smithi173.stdout:5/242: write da/d34/f47 [647455,127335] 0 2024-04-20T11:09:29.513 INFO:tasks.workunit.client.1.smithi173.stdout:7/231: sync 2024-04-20T11:09:29.513 INFO:tasks.workunit.client.1.smithi173.stdout:2/214: sync 2024-04-20T11:09:29.520 INFO:tasks.workunit.client.1.smithi173.stdout:5/243: creat da/de/d19/d2c/f53 x:0 0 0 2024-04-20T11:09:29.522 INFO:tasks.workunit.client.1.smithi173.stdout:2/215: getdents d2/de 0 2024-04-20T11:09:29.523 INFO:tasks.workunit.client.1.smithi173.stdout:2/216: read - d2/f3f zero size 2024-04-20T11:09:29.524 INFO:tasks.workunit.client.1.smithi173.stdout:8/263: write f2 [1149705,19129] 0 2024-04-20T11:09:29.524 INFO:tasks.workunit.client.1.smithi173.stdout:4/216: dread da/dc/d15/f1d [0,4194304] 0 2024-04-20T11:09:29.525 INFO:tasks.workunit.client.1.smithi173.stdout:6/215: dwrite d4/f13 [0,4194304] 0 2024-04-20T11:09:29.528 INFO:tasks.workunit.client.1.smithi173.stdout:8/264: rmdir d4/d18/d3a 39 2024-04-20T11:09:29.530 INFO:tasks.workunit.client.1.smithi173.stdout:7/232: truncate d9/da/d17/f1b 332693 0 2024-04-20T11:09:29.531 INFO:tasks.workunit.client.1.smithi173.stdout:5/244: rename da/de/d19/d32/f36 to da/de/d19/d32/d30/d35/f54 0 2024-04-20T11:09:29.532 INFO:tasks.workunit.client.1.smithi173.stdout:4/217: symlink da/d1a/d30/d39/l41 0 2024-04-20T11:09:29.532 INFO:tasks.workunit.client.1.smithi173.stdout:5/245: chown da/de/d19/d2c 86 1 2024-04-20T11:09:29.533 INFO:tasks.workunit.client.1.smithi173.stdout:4/218: stat da/dc/d13/c21 0 2024-04-20T11:09:29.536 INFO:tasks.workunit.client.1.smithi173.stdout:6/216: unlink d4/d18/f26 0 2024-04-20T11:09:29.538 INFO:tasks.workunit.client.1.smithi173.stdout:7/233: truncate d9/da/f10 250400 0 2024-04-20T11:09:29.540 INFO:tasks.workunit.client.1.smithi173.stdout:4/219: chown f1 6862859 1 2024-04-20T11:09:29.540 INFO:tasks.workunit.client.1.smithi173.stdout:8/265: rmdir d4/d18/d3a/d3c 39 2024-04-20T11:09:29.540 INFO:tasks.workunit.client.1.smithi173.stdout:5/246: creat da/d34/f55 x:0 0 0 2024-04-20T11:09:29.542 INFO:tasks.workunit.client.1.smithi173.stdout:7/234: mkdir d9/da/d17/d55 0 2024-04-20T11:09:29.543 INFO:tasks.workunit.client.1.smithi173.stdout:6/217: write d4/d14/f39 [839064,42281] 0 2024-04-20T11:09:29.546 INFO:tasks.workunit.client.1.smithi173.stdout:6/218: fsync d4/d14/f2f 0 2024-04-20T11:09:29.547 INFO:tasks.workunit.client.1.smithi173.stdout:7/235: unlink d9/da/le 0 2024-04-20T11:09:29.549 INFO:tasks.workunit.client.1.smithi173.stdout:5/247: rename da/d34/l4e to da/de/d19/l56 0 2024-04-20T11:09:29.550 INFO:tasks.workunit.client.1.smithi173.stdout:6/219: mkdir d4/d24/d42 0 2024-04-20T11:09:29.553 INFO:tasks.workunit.client.1.smithi173.stdout:5/248: mknod da/d4f/c57 0 2024-04-20T11:09:29.564 INFO:tasks.workunit.client.1.smithi173.stdout:6/220: creat d4/d14/f43 x:0 0 0 2024-04-20T11:09:29.565 INFO:tasks.workunit.client.1.smithi173.stdout:7/236: dwrite d9/d33/d35/f44 [0,4194304] 0 2024-04-20T11:09:29.567 INFO:tasks.workunit.client.1.smithi173.stdout:4/220: write f4 [3392725,128492] 0 2024-04-20T11:09:29.570 INFO:tasks.workunit.client.1.smithi173.stdout:6/221: fdatasync d4/f37 0 2024-04-20T11:09:29.570 INFO:tasks.workunit.client.1.smithi173.stdout:8/266: write d4/d18/f30 [1323156,104670] 0 2024-04-20T11:09:29.571 INFO:tasks.workunit.client.1.smithi173.stdout:6/222: dread - d4/d14/f43 zero size 2024-04-20T11:09:29.572 INFO:tasks.workunit.client.1.smithi173.stdout:4/221: mknod da/dc/c42 0 2024-04-20T11:09:29.576 INFO:tasks.workunit.client.1.smithi173.stdout:7/237: rename d9/d33/d50 to d9/d33/d35/d56 0 2024-04-20T11:09:29.580 INFO:tasks.workunit.client.1.smithi173.stdout:8/267: symlink d4/d18/l4c 0 2024-04-20T11:09:29.584 INFO:tasks.workunit.client.1.smithi173.stdout:6/223: dread d4/d14/f2f [0,4194304] 0 2024-04-20T11:09:29.585 INFO:tasks.workunit.client.1.smithi173.stdout:7/238: creat d9/da/d43/f57 x:0 0 0 2024-04-20T11:09:29.588 INFO:tasks.workunit.client.1.smithi173.stdout:8/268: symlink d4/d18/d3a/d3c/l4d 0 2024-04-20T11:09:29.588 INFO:tasks.workunit.client.1.smithi173.stdout:8/269: chown d4/f41 30 1 2024-04-20T11:09:29.590 INFO:tasks.workunit.client.1.smithi173.stdout:7/239: rename d9/da/d17/d55 to d9/d33/d35/d58 0 2024-04-20T11:09:29.590 INFO:tasks.workunit.client.1.smithi173.stdout:6/224: mknod d4/d24/d3e/c44 0 2024-04-20T11:09:29.592 INFO:tasks.workunit.client.1.smithi173.stdout:8/270: symlink d4/d18/l4e 0 2024-04-20T11:09:29.593 INFO:tasks.workunit.client.1.smithi173.stdout:5/249: dread da/f29 [0,4194304] 0 2024-04-20T11:09:29.595 INFO:tasks.workunit.client.1.smithi173.stdout:5/250: readlink da/de/d19/d32/l50 0 2024-04-20T11:09:29.631 INFO:tasks.workunit.client.1.smithi173.stdout:9/239: sync 2024-04-20T11:09:29.637 INFO:tasks.workunit.client.1.smithi173.stdout:7/240: rename d9/da/d17/f3d to d9/da/d2e/f59 0 2024-04-20T11:09:29.641 INFO:tasks.workunit.client.1.smithi173.stdout:9/240: mkdir d5/d9/d20/d4f 0 2024-04-20T11:09:29.650 INFO:tasks.workunit.client.1.smithi173.stdout:7/241: dread d9/da/f25 [0,4194304] 0 2024-04-20T11:09:29.652 INFO:tasks.workunit.client.1.smithi173.stdout:6/225: dwrite d4/d18/f2b [0,4194304] 0 2024-04-20T11:09:29.656 INFO:tasks.workunit.client.1.smithi173.stdout:8/271: dwrite d4/d18/d3a/d3c/f47 [0,4194304] 0 2024-04-20T11:09:29.656 INFO:tasks.workunit.client.1.smithi173.stdout:7/242: read d9/d33/d35/f44 [3522730,59884] 0 2024-04-20T11:09:29.656 INFO:tasks.workunit.client.1.smithi173.stdout:6/226: write d4/d14/f2f [922882,39817] 0 2024-04-20T11:09:29.657 INFO:tasks.workunit.client.1.smithi173.stdout:7/243: chown d9/d1f 83411327 1 2024-04-20T11:09:29.666 INFO:tasks.workunit.client.1.smithi173.stdout:5/251: dread f7 [0,4194304] 0 2024-04-20T11:09:29.668 INFO:tasks.workunit.client.1.smithi173.stdout:6/227: link d4/d1c/l32 d4/d1c/l45 0 2024-04-20T11:09:29.668 INFO:tasks.workunit.client.1.smithi173.stdout:5/252: readlink da/de/l14 0 2024-04-20T11:09:29.675 INFO:tasks.workunit.client.1.smithi173.stdout:6/228: creat d4/f46 x:0 0 0 2024-04-20T11:09:29.676 INFO:tasks.workunit.client.1.smithi173.stdout:6/229: stat d4/d14/f2f 0 2024-04-20T11:09:29.676 INFO:tasks.workunit.client.1.smithi173.stdout:9/241: dread d5/f6 [0,4194304] 0 2024-04-20T11:09:29.677 INFO:tasks.workunit.client.1.smithi173.stdout:5/253: dread da/de/d19/d32/f15 [0,4194304] 0 2024-04-20T11:09:29.677 INFO:tasks.workunit.client.1.smithi173.stdout:7/244: write f1 [3300077,5859] 0 2024-04-20T11:09:29.679 INFO:tasks.workunit.client.1.smithi173.stdout:6/230: rmdir d4/d14 39 2024-04-20T11:09:29.681 INFO:tasks.workunit.client.1.smithi173.stdout:9/242: write d5/d9/d32/f4e [1589845,88038] 0 2024-04-20T11:09:29.685 INFO:tasks.workunit.client.1.smithi173.stdout:5/254: creat da/de/d19/d32/d30/d43/f58 x:0 0 0 2024-04-20T11:09:29.687 INFO:tasks.workunit.client.1.smithi173.stdout:8/272: dwrite d4/f19 [0,4194304] 0 2024-04-20T11:09:29.687 INFO:tasks.workunit.client.1.smithi173.stdout:9/243: symlink d5/d9/d20/d29/d2d/l50 0 2024-04-20T11:09:29.689 INFO:tasks.workunit.client.1.smithi173.stdout:9/244: dread - d5/d9/d20/f40 zero size 2024-04-20T11:09:29.690 INFO:tasks.workunit.client.1.smithi173.stdout:9/245: dread - d5/d9/d20/d2b/d31/f4d zero size 2024-04-20T11:09:29.691 INFO:tasks.workunit.client.1.smithi173.stdout:6/231: rename d4/d14/c23 to d4/d1c/d28/c47 0 2024-04-20T11:09:29.693 INFO:tasks.workunit.client.1.smithi173.stdout:7/245: getdents d9/d33/d40 0 2024-04-20T11:09:29.694 INFO:tasks.workunit.client.1.smithi173.stdout:6/232: read d4/f37 [21918,109135] 0 2024-04-20T11:09:29.699 INFO:tasks.workunit.client.1.smithi173.stdout:7/246: unlink d9/da/ff 0 2024-04-20T11:09:29.702 INFO:tasks.workunit.client.1.smithi173.stdout:8/273: mkdir d4/d18/d4f 0 2024-04-20T11:09:29.704 INFO:tasks.workunit.client.1.smithi173.stdout:5/255: truncate f6 791454 0 2024-04-20T11:09:29.709 INFO:tasks.workunit.client.1.smithi173.stdout:5/256: mkdir da/d12/d40/d59 0 2024-04-20T11:09:29.711 INFO:tasks.workunit.client.1.smithi173.stdout:3/275: sync 2024-04-20T11:09:29.712 INFO:tasks.workunit.client.1.smithi173.stdout:6/233: link d4/d24/c31 d4/d14/c48 0 2024-04-20T11:09:29.713 INFO:tasks.workunit.client.1.smithi173.stdout:5/257: dread da/de/d19/d32/f15 [0,4194304] 0 2024-04-20T11:09:29.713 INFO:tasks.workunit.client.1.smithi173.stdout:7/247: dwrite d9/f1d [0,4194304] 0 2024-04-20T11:09:29.715 INFO:tasks.workunit.client.1.smithi173.stdout:8/274: rename d4/d18/d3a/d3c/c33 to d4/d18/c50 0 2024-04-20T11:09:29.715 INFO:tasks.workunit.client.1.smithi173.stdout:6/234: fsync d4/d1c/d28/d3b/f41 0 2024-04-20T11:09:29.716 INFO:tasks.workunit.client.1.smithi173.stdout:3/276: fdatasync d7/d9/f4b 0 2024-04-20T11:09:29.722 INFO:tasks.workunit.client.1.smithi173.stdout:7/248: mkdir d9/d33/d35/d56/d5a 0 2024-04-20T11:09:29.730 INFO:tasks.workunit.client.1.smithi173.stdout:5/258: symlink da/de/d19/d32/d30/l5a 0 2024-04-20T11:09:29.730 INFO:tasks.workunit.client.1.smithi173.stdout:8/275: mknod d4/d18/d4f/c51 0 2024-04-20T11:09:29.732 INFO:tasks.workunit.client.1.smithi173.stdout:5/259: readlink da/de/d19/d32/d30/l5a 0 2024-04-20T11:09:29.733 INFO:tasks.workunit.client.1.smithi173.stdout:6/235: symlink d4/l49 0 2024-04-20T11:09:29.733 INFO:tasks.workunit.client.1.smithi173.stdout:9/246: dread d5/d9/f1d [0,4194304] 0 2024-04-20T11:09:29.735 INFO:tasks.workunit.client.1.smithi173.stdout:7/249: mkdir d9/d33/d35/d4a/d52/d5b 0 2024-04-20T11:09:29.735 INFO:tasks.workunit.client.1.smithi173.stdout:3/277: dwrite d7/f18 [4194304,4194304] 0 2024-04-20T11:09:29.738 INFO:tasks.workunit.client.1.smithi173.stdout:9/247: chown d5/d9/d20/d2b/f49 1746741312 1 2024-04-20T11:09:29.741 INFO:tasks.workunit.client.1.smithi173.stdout:2/217: sync 2024-04-20T11:09:29.741 INFO:tasks.workunit.client.1.smithi173.stdout:0/218: sync 2024-04-20T11:09:29.741 INFO:tasks.workunit.client.1.smithi173.stdout:4/222: sync 2024-04-20T11:09:29.741 INFO:tasks.workunit.client.1.smithi173.stdout:1/199: sync 2024-04-20T11:09:29.743 INFO:tasks.workunit.client.1.smithi173.stdout:6/236: rename d4/d18/f19 to d4/d1c/f4a 0 2024-04-20T11:09:29.744 INFO:tasks.workunit.client.1.smithi173.stdout:5/260: getdents da/d12/d40 0 2024-04-20T11:09:29.746 INFO:tasks.workunit.client.1.smithi173.stdout:8/276: dread d4/d18/f24 [0,4194304] 0 2024-04-20T11:09:29.747 INFO:tasks.workunit.client.1.smithi173.stdout:9/248: symlink d5/d9/d20/l51 0 2024-04-20T11:09:29.749 INFO:tasks.workunit.client.1.smithi173.stdout:1/200: readlink d5/d7/da/df/l34 0 2024-04-20T11:09:29.750 INFO:tasks.workunit.client.1.smithi173.stdout:6/237: fsync d4/f20 0 2024-04-20T11:09:29.750 INFO:tasks.workunit.client.1.smithi173.stdout:4/223: write da/f25 [1574938,118833] 0 2024-04-20T11:09:29.751 INFO:tasks.workunit.client.1.smithi173.stdout:2/218: write d2/de/f3b [1609635,16556] 0 2024-04-20T11:09:29.752 INFO:tasks.workunit.client.1.smithi173.stdout:5/261: fsync da/de/d19/d32/d30/f41 0 2024-04-20T11:09:29.757 INFO:tasks.workunit.client.1.smithi173.stdout:0/219: read d6/d8/df/d15/f22 [862706,72806] 0 2024-04-20T11:09:29.758 INFO:tasks.workunit.client.1.smithi173.stdout:9/249: write d5/d9/d20/d2b/f4a [438227,100132] 0 2024-04-20T11:09:29.762 INFO:tasks.workunit.client.1.smithi173.stdout:6/238: chown d4/d14/c1f 337 1 2024-04-20T11:09:29.762 INFO:tasks.workunit.client.1.smithi173.stdout:7/250: dwrite f1 [0,4194304] 0 2024-04-20T11:09:29.767 INFO:tasks.workunit.client.1.smithi173.stdout:7/251: truncate d9/d33/d40/f49 942433 0 2024-04-20T11:09:29.769 INFO:tasks.workunit.client.1.smithi173.stdout:4/224: rename da/d1e/l20 to da/l43 0 2024-04-20T11:09:29.772 INFO:tasks.workunit.client.1.smithi173.stdout:9/250: creat d5/d4c/f52 x:0 0 0 2024-04-20T11:09:29.772 INFO:tasks.workunit.client.1.smithi173.stdout:2/219: creat d2/d34/f45 x:0 0 0 2024-04-20T11:09:29.776 INFO:tasks.workunit.client.1.smithi173.stdout:8/277: dread d4/d18/d3a/d3c/f40 [0,4194304] 0 2024-04-20T11:09:29.776 INFO:tasks.workunit.client.1.smithi173.stdout:6/239: symlink d4/d1c/d28/l4b 0 2024-04-20T11:09:29.777 INFO:tasks.workunit.client.1.smithi173.stdout:7/252: unlink d9/f21 0 2024-04-20T11:09:29.782 INFO:tasks.workunit.client.1.smithi173.stdout:2/220: unlink d2/de/d11/fd 0 2024-04-20T11:09:29.787 INFO:tasks.workunit.client.1.smithi173.stdout:6/240: creat d4/d24/d3e/f4c x:0 0 0 2024-04-20T11:09:29.787 INFO:tasks.workunit.client.1.smithi173.stdout:8/278: unlink d4/d18/d3a/d3c/c43 0 2024-04-20T11:09:29.788 INFO:tasks.workunit.client.1.smithi173.stdout:2/221: mkdir d2/d13/d46 0 2024-04-20T11:09:29.788 INFO:tasks.workunit.client.1.smithi173.stdout:3/278: dread d7/f25 [0,4194304] 0 2024-04-20T11:09:29.788 INFO:tasks.workunit.client.1.smithi173.stdout:6/241: chown d4/d14/f2f 927191406 1 2024-04-20T11:09:29.789 INFO:tasks.workunit.client.1.smithi173.stdout:6/242: truncate d4/d14/f3c 4954712 0 2024-04-20T11:09:29.792 INFO:tasks.workunit.client.1.smithi173.stdout:2/222: symlink d2/l47 0 2024-04-20T11:09:29.792 INFO:tasks.workunit.client.1.smithi173.stdout:8/279: creat d4/d18/d3a/d3c/f52 x:0 0 0 2024-04-20T11:09:29.794 INFO:tasks.workunit.client.1.smithi173.stdout:2/223: write d2/d34/f45 [340844,83291] 0 2024-04-20T11:09:29.796 INFO:tasks.workunit.client.1.smithi173.stdout:1/201: dread d5/d7/da/f14 [0,4194304] 0 2024-04-20T11:09:29.806 INFO:tasks.workunit.client.1.smithi173.stdout:0/220: truncate d6/d8/f1f 4089389 0 2024-04-20T11:09:29.810 INFO:tasks.workunit.client.1.smithi173.stdout:4/225: read da/d1a/d30/f3b [1189542,39296] 0 2024-04-20T11:09:29.811 INFO:tasks.workunit.client.1.smithi173.stdout:2/224: rename d2/l47 to d2/l48 0 2024-04-20T11:09:29.820 INFO:tasks.workunit.client.1.smithi173.stdout:2/225: rename c1 to d2/de/d11/c49 0 2024-04-20T11:09:29.820 INFO:tasks.workunit.client.1.smithi173.stdout:7/253: dwrite d9/da/f46 [0,4194304] 0 2024-04-20T11:09:29.820 INFO:tasks.workunit.client.1.smithi173.stdout:9/251: dwrite d5/d9/f26 [0,4194304] 0 2024-04-20T11:09:29.823 INFO:tasks.workunit.client.1.smithi173.stdout:7/254: dread - d9/da/d31/f38 zero size 2024-04-20T11:09:29.823 INFO:tasks.workunit.client.1.smithi173.stdout:4/226: rmdir da/d1e 39 2024-04-20T11:09:29.825 INFO:tasks.workunit.client.1.smithi173.stdout:1/202: dread f3 [0,4194304] 0 2024-04-20T11:09:29.826 INFO:tasks.workunit.client.1.smithi173.stdout:6/243: truncate d4/d14/f1a 467516 0 2024-04-20T11:09:29.841 INFO:tasks.workunit.client.1.smithi173.stdout:2/226: truncate d2/de/d11/f23 1751524 0 2024-04-20T11:09:29.842 INFO:tasks.workunit.client.1.smithi173.stdout:4/227: dread f8 [0,4194304] 0 2024-04-20T11:09:29.843 INFO:tasks.workunit.client.1.smithi173.stdout:2/227: write d2/d34/f45 [1407794,43789] 0 2024-04-20T11:09:29.845 INFO:tasks.workunit.client.1.smithi173.stdout:2/228: chown d2/de/d11/l40 6319206 1 2024-04-20T11:09:29.846 INFO:tasks.workunit.client.1.smithi173.stdout:1/203: symlink d5/d6/d3b/l3e 0 2024-04-20T11:09:29.858 INFO:tasks.workunit.client.1.smithi173.stdout:2/229: mkdir d2/d13/d28/d4a 0 2024-04-20T11:09:29.861 INFO:tasks.workunit.client.1.smithi173.stdout:9/252: getdents d5/d9/d20/d29 0 2024-04-20T11:09:29.863 INFO:tasks.workunit.client.1.smithi173.stdout:7/255: getdents d9/d1f 0 2024-04-20T11:09:29.863 INFO:tasks.workunit.client.1.smithi173.stdout:6/244: getdents d4/d1c 0 2024-04-20T11:09:29.864 INFO:tasks.workunit.client.1.smithi173.stdout:7/256: fdatasync d9/f1d 0 2024-04-20T11:09:29.866 INFO:tasks.workunit.client.1.smithi173.stdout:4/228: symlink da/d1a/d30/l44 0 2024-04-20T11:09:29.867 INFO:tasks.workunit.client.1.smithi173.stdout:2/230: mkdir d2/d13/d46/d4b 0 2024-04-20T11:09:29.868 INFO:tasks.workunit.client.1.smithi173.stdout:7/257: rmdir d9/da/d17/d2b 39 2024-04-20T11:09:29.869 INFO:tasks.workunit.client.1.smithi173.stdout:1/204: rename d5/d7/d12/f2e to d5/d7/f3f 0 2024-04-20T11:09:29.870 INFO:tasks.workunit.client.1.smithi173.stdout:7/258: chown d9/d33/d40/f49 5 1 2024-04-20T11:09:29.873 INFO:tasks.workunit.client.1.smithi173.stdout:9/253: creat d5/f53 x:0 0 0 2024-04-20T11:09:29.934 INFO:tasks.workunit.client.1.smithi173.stdout:5/262: sync 2024-04-20T11:09:29.949 INFO:tasks.workunit.client.1.smithi173.stdout:4/229: symlink da/dc/d13/l45 0 2024-04-20T11:09:29.957 INFO:tasks.workunit.client.1.smithi173.stdout:9/254: write d5/f38 [184794,57829] 0 2024-04-20T11:09:29.959 INFO:tasks.workunit.client.1.smithi173.stdout:4/230: mknod da/dc/d13/c46 0 2024-04-20T11:09:29.961 INFO:tasks.workunit.client.1.smithi173.stdout:7/259: dread d9/da/f12 [0,4194304] 0 2024-04-20T11:09:29.961 INFO:tasks.workunit.client.1.smithi173.stdout:2/231: dwrite d2/de/f21 [0,4194304] 0 2024-04-20T11:09:29.964 INFO:tasks.workunit.client.1.smithi173.stdout:7/260: chown d9/da/lb 245567 1 2024-04-20T11:09:29.968 INFO:tasks.workunit.client.1.smithi173.stdout:7/261: write f1 [1740085,81556] 0 2024-04-20T11:09:29.970 INFO:tasks.workunit.client.1.smithi173.stdout:4/231: creat da/dc/d13/f47 x:0 0 0 2024-04-20T11:09:29.971 INFO:tasks.workunit.client.1.smithi173.stdout:2/232: symlink d2/de/d11/l4c 0 2024-04-20T11:09:29.977 INFO:tasks.workunit.client.1.smithi173.stdout:5/263: dread f8 [4194304,4194304] 0 2024-04-20T11:09:29.977 INFO:tasks.workunit.client.1.smithi173.stdout:9/255: dread d5/d9/d32/f4e [0,4194304] 0 2024-04-20T11:09:29.979 INFO:tasks.workunit.client.1.smithi173.stdout:4/232: mkdir da/d48 0 2024-04-20T11:09:29.979 INFO:tasks.workunit.client.1.smithi173.stdout:5/264: readlink da/de/d19/d32/l18 0 2024-04-20T11:09:29.981 INFO:tasks.workunit.client.1.smithi173.stdout:5/265: fdatasync da/d34/f55 0 2024-04-20T11:09:29.983 INFO:tasks.workunit.client.1.smithi173.stdout:1/205: dread d5/d7/d12/f15 [0,4194304] 0 2024-04-20T11:09:29.983 INFO:tasks.workunit.client.1.smithi173.stdout:5/266: chown da/de/d19/d2c 756 1 2024-04-20T11:09:29.984 INFO:tasks.workunit.client.1.smithi173.stdout:2/233: creat d2/d13/d28/d4a/f4d x:0 0 0 2024-04-20T11:09:29.985 INFO:tasks.workunit.client.1.smithi173.stdout:7/262: dread f6 [4194304,4194304] 0 2024-04-20T11:09:29.987 INFO:tasks.workunit.client.1.smithi173.stdout:7/263: read d9/da/f12 [561948,116840] 0 2024-04-20T11:09:29.990 INFO:tasks.workunit.client.1.smithi173.stdout:2/234: chown d2/de/d11/f23 7450326 1 2024-04-20T11:09:29.991 INFO:tasks.workunit.client.1.smithi173.stdout:1/206: mkdir d5/d7/da/df/d40 0 2024-04-20T11:09:29.996 INFO:tasks.workunit.client.1.smithi173.stdout:9/256: dwrite d5/d9/d20/d29/d18/f1f [4194304,4194304] 0 2024-04-20T11:09:30.004 INFO:tasks.workunit.client.1.smithi173.stdout:5/267: rename f6 to da/de/f5b 0 2024-04-20T11:09:30.005 INFO:tasks.workunit.client.1.smithi173.stdout:4/233: write da/d1a/d30/f3b [5079676,9984] 0 2024-04-20T11:09:30.009 INFO:tasks.workunit.client.1.smithi173.stdout:9/257: write d5/f2f [371443,63492] 0 2024-04-20T11:09:30.010 INFO:tasks.workunit.client.1.smithi173.stdout:5/268: fsync da/de/d19/d32/d30/d43/f58 0 2024-04-20T11:09:30.010 INFO:tasks.workunit.client.1.smithi173.stdout:1/207: dread d5/d6/d17/f27 [0,4194304] 0 2024-04-20T11:09:30.010 INFO:tasks.workunit.client.1.smithi173.stdout:9/258: fsync d5/d9/f1a 0 2024-04-20T11:09:30.012 INFO:tasks.workunit.client.1.smithi173.stdout:5/269: chown da/d12/c26 446572 1 2024-04-20T11:09:30.012 INFO:tasks.workunit.client.1.smithi173.stdout:2/235: dwrite d2/d34/f45 [0,4194304] 0 2024-04-20T11:09:30.013 INFO:tasks.workunit.client.1.smithi173.stdout:4/234: mknod da/d1a/d30/d39/c49 0 2024-04-20T11:09:30.013 INFO:tasks.workunit.client.1.smithi173.stdout:9/259: write d5/d4c/f52 [316367,655] 0 2024-04-20T11:09:30.014 INFO:tasks.workunit.client.1.smithi173.stdout:4/235: readlink da/d1a/l1f 0 2024-04-20T11:09:30.017 INFO:tasks.workunit.client.1.smithi173.stdout:5/270: truncate da/de/d19/d32/d30/d43/f58 348285 0 2024-04-20T11:09:30.020 INFO:tasks.workunit.client.1.smithi173.stdout:1/208: creat d5/d7/f41 x:0 0 0 2024-04-20T11:09:30.020 INFO:tasks.workunit.client.1.smithi173.stdout:7/264: truncate d9/da/d2e/f37 1321643 0 2024-04-20T11:09:30.020 INFO:tasks.workunit.client.1.smithi173.stdout:2/236: write d2/de/f2b [4757251,106824] 0 2024-04-20T11:09:30.023 INFO:tasks.workunit.client.1.smithi173.stdout:3/279: sync 2024-04-20T11:09:30.024 INFO:tasks.workunit.client.1.smithi173.stdout:5/271: creat da/de/d19/f5c x:0 0 0 2024-04-20T11:09:30.025 INFO:tasks.workunit.client.1.smithi173.stdout:4/236: rmdir da/d1e 39 2024-04-20T11:09:30.027 INFO:tasks.workunit.client.1.smithi173.stdout:2/237: mkdir d2/d29/d4e 0 2024-04-20T11:09:30.028 INFO:tasks.workunit.client.1.smithi173.stdout:2/238: chown d2/de/d11/f10 2182 1 2024-04-20T11:09:30.028 INFO:tasks.workunit.client.1.smithi173.stdout:9/260: rename d5/d9/d20/d29/l14 to d5/d9/d20/d29/d2d/l54 0 2024-04-20T11:09:30.030 INFO:tasks.workunit.client.1.smithi173.stdout:5/272: mknod da/de/d19/d32/d30/d35/c5d 0 2024-04-20T11:09:30.031 INFO:tasks.workunit.client.1.smithi173.stdout:2/239: write d2/de/f43 [623209,122610] 0 2024-04-20T11:09:30.033 INFO:tasks.workunit.client.1.smithi173.stdout:7/265: link d9/da/d2e/f59 d9/da/f5c 0 2024-04-20T11:09:30.033 INFO:tasks.workunit.client.1.smithi173.stdout:5/273: fdatasync da/de/d19/d32/f2f 0 2024-04-20T11:09:30.034 INFO:tasks.workunit.client.1.smithi173.stdout:9/261: dread d5/d4c/f52 [0,4194304] 0 2024-04-20T11:09:30.034 INFO:tasks.workunit.client.1.smithi173.stdout:7/266: stat d9/d33 0 2024-04-20T11:09:30.035 INFO:tasks.workunit.client.1.smithi173.stdout:5/274: chown da/de/c2e 0 1 2024-04-20T11:09:30.037 INFO:tasks.workunit.client.1.smithi173.stdout:2/240: symlink d2/d13/d46/l4f 0 2024-04-20T11:09:30.045 INFO:tasks.workunit.client.1.smithi173.stdout:1/209: rename d5/d7 to d5/d6/d17/d42 0 2024-04-20T11:09:30.049 INFO:tasks.workunit.client.1.smithi173.stdout:3/280: rename d7/l3e to d7/d17/l5f 0 2024-04-20T11:09:30.050 INFO:tasks.workunit.client.1.smithi173.stdout:1/210: mknod d5/d6/d3b/c43 0 2024-04-20T11:09:30.054 INFO:tasks.workunit.client.1.smithi173.stdout:7/267: rename d9/da/l41 to d9/d33/d35/l5d 0 2024-04-20T11:09:30.055 INFO:tasks.workunit.client.1.smithi173.stdout:9/262: dwrite d5/d9/f1a [0,4194304] 0 2024-04-20T11:09:30.055 INFO:tasks.workunit.client.1.smithi173.stdout:3/281: unlink d7/d9/f3f 0 2024-04-20T11:09:30.057 INFO:tasks.workunit.client.1.smithi173.stdout:2/241: dwrite d2/d13/f3d [4194304,4194304] 0 2024-04-20T11:09:30.057 INFO:tasks.workunit.client.1.smithi173.stdout:1/211: chown d5/d6/d17/d42/d12/d1c 1720 1 2024-04-20T11:09:30.058 INFO:tasks.workunit.client.1.smithi173.stdout:4/237: dread da/dc/d15/f1c [0,4194304] 0 2024-04-20T11:09:30.059 INFO:tasks.workunit.client.1.smithi173.stdout:3/282: write d7/d17/f2c [1301314,46652] 0 2024-04-20T11:09:30.061 INFO:tasks.workunit.client.1.smithi173.stdout:3/283: readlink d7/d9/d3b/l5b 0 2024-04-20T11:09:30.063 INFO:tasks.workunit.client.1.smithi173.stdout:7/268: rename d9/da/d17/l1c to d9/d33/d35/d58/l5e 0 2024-04-20T11:09:30.064 INFO:tasks.workunit.client.1.smithi173.stdout:7/269: dread - d9/da/d31/f38 zero size 2024-04-20T11:09:30.066 INFO:tasks.workunit.client.1.smithi173.stdout:2/242: dread d2/de/f43 [0,4194304] 0 2024-04-20T11:09:30.078 INFO:tasks.workunit.client.1.smithi173.stdout:7/270: unlink d9/l3e 0 2024-04-20T11:09:30.079 INFO:tasks.workunit.client.1.smithi173.stdout:7/271: stat d9/da/d31/c3b 0 2024-04-20T11:09:30.080 INFO:tasks.workunit.client.1.smithi173.stdout:7/272: write f1 [4035610,56455] 0 2024-04-20T11:09:30.083 INFO:tasks.workunit.client.1.smithi173.stdout:3/284: creat d7/d9/d30/f60 x:0 0 0 2024-04-20T11:09:30.087 INFO:tasks.workunit.client.1.smithi173.stdout:9/263: rename d5/l1c to d5/d9/d20/d29/d18/l55 0 2024-04-20T11:09:30.088 INFO:tasks.workunit.client.1.smithi173.stdout:9/264: write d5/d9/f26 [2555369,130286] 0 2024-04-20T11:09:30.090 INFO:tasks.workunit.client.1.smithi173.stdout:1/212: link f4 d5/d6/d17/d42/d12/f44 0 2024-04-20T11:09:30.095 INFO:tasks.workunit.client.1.smithi173.stdout:3/285: mkdir d7/d17/d61 0 2024-04-20T11:09:30.096 INFO:tasks.workunit.client.1.smithi173.stdout:3/286: fsync d7/f18 0 2024-04-20T11:09:30.097 INFO:tasks.workunit.client.1.smithi173.stdout:3/287: stat d7/d17/f2c 0 2024-04-20T11:09:30.098 INFO:tasks.workunit.client.1.smithi173.stdout:3/288: fdatasync d7/d9/d30/f60 0 2024-04-20T11:09:30.099 INFO:tasks.workunit.client.1.smithi173.stdout:1/213: symlink d5/d6/d17/d42/d12/d1c/l45 0 2024-04-20T11:09:30.100 INFO:tasks.workunit.client.1.smithi173.stdout:4/238: dread da/dc/d13/f32 [4194304,4194304] 0 2024-04-20T11:09:30.102 INFO:tasks.workunit.client.1.smithi173.stdout:4/239: fsync da/d1a/f40 0 2024-04-20T11:09:30.103 INFO:tasks.workunit.client.1.smithi173.stdout:9/265: dread d5/d9/d20/d29/d18/f1f [4194304,4194304] 0 2024-04-20T11:09:30.111 INFO:tasks.workunit.client.1.smithi173.stdout:3/289: truncate d7/f2d 5144902 0 2024-04-20T11:09:30.112 INFO:tasks.workunit.client.1.smithi173.stdout:3/290: read - d7/d17/f2f zero size 2024-04-20T11:09:30.112 INFO:tasks.workunit.client.1.smithi173.stdout:2/243: getdents d2/d13/d28/d4a 0 2024-04-20T11:09:30.114 INFO:tasks.workunit.client.1.smithi173.stdout:3/291: write d7/d9/f46 [2645388,74997] 0 2024-04-20T11:09:30.123 INFO:tasks.workunit.client.1.smithi173.stdout:1/214: symlink d5/d6/d17/d42/da/df/d40/l46 0 2024-04-20T11:09:30.124 INFO:tasks.workunit.client.1.smithi173.stdout:4/240: dread da/dc/f33 [0,4194304] 0 2024-04-20T11:09:30.125 INFO:tasks.workunit.client.1.smithi173.stdout:2/244: creat d2/d13/f50 x:0 0 0 2024-04-20T11:09:30.126 INFO:tasks.workunit.client.1.smithi173.stdout:1/215: write d5/d6/dc/f3a [686480,45860] 0 2024-04-20T11:09:30.127 INFO:tasks.workunit.client.1.smithi173.stdout:2/245: truncate d2/d13/f38 911771 0 2024-04-20T11:09:30.130 INFO:tasks.workunit.client.1.smithi173.stdout:7/273: write d9/da/d17/f1b [578078,13711] 0 2024-04-20T11:09:30.131 INFO:tasks.workunit.client.1.smithi173.stdout:2/246: read d2/de/f2b [2327269,108981] 0 2024-04-20T11:09:30.133 INFO:tasks.workunit.client.1.smithi173.stdout:0/221: sync 2024-04-20T11:09:30.134 INFO:tasks.workunit.client.1.smithi173.stdout:8/280: sync 2024-04-20T11:09:30.134 INFO:tasks.workunit.client.1.smithi173.stdout:7/274: write d9/d33/d35/d4a/f4c [2791113,50682] 0 2024-04-20T11:09:30.146 INFO:tasks.workunit.client.1.smithi173.stdout:1/216: creat d5/d6/d17/d42/d12/f47 x:0 0 0 2024-04-20T11:09:30.151 INFO:tasks.workunit.client.1.smithi173.stdout:3/292: dread d7/d9/f35 [0,4194304] 0 2024-04-20T11:09:30.152 INFO:tasks.workunit.client.1.smithi173.stdout:8/281: creat d4/d18/d3a/d3c/f53 x:0 0 0 2024-04-20T11:09:30.152 INFO:tasks.workunit.client.1.smithi173.stdout:9/266: dwrite d5/d4c/f52 [0,4194304] 0 2024-04-20T11:09:30.155 INFO:tasks.workunit.client.1.smithi173.stdout:0/222: mkdir d6/d8/df/d43 0 2024-04-20T11:09:30.159 INFO:tasks.workunit.client.1.smithi173.stdout:1/217: write d5/d6/d17/d42/d12/d1f/f2f [526073,83345] 0 2024-04-20T11:09:30.159 INFO:tasks.workunit.client.1.smithi173.stdout:0/223: write d6/d8/d3c/f41 [603708,8495] 0 2024-04-20T11:09:30.161 INFO:tasks.workunit.client.1.smithi173.stdout:1/218: write d5/d6/d17/d42/d12/f47 [588212,17926] 0 2024-04-20T11:09:30.161 INFO:tasks.workunit.client.1.smithi173.stdout:8/282: fdatasync d4/f10 0 2024-04-20T11:09:30.163 INFO:tasks.workunit.client.1.smithi173.stdout:9/267: write d5/d9/f1d [3111669,85140] 0 2024-04-20T11:09:30.164 INFO:tasks.workunit.client.1.smithi173.stdout:2/247: getdents d2/d13/d46 0 2024-04-20T11:09:30.171 INFO:tasks.workunit.client.1.smithi173.stdout:9/268: creat d5/d9/d32/d44/f56 x:0 0 0 2024-04-20T11:09:30.171 INFO:tasks.workunit.client.1.smithi173.stdout:9/269: fsync d5/f4b 0 2024-04-20T11:09:30.172 INFO:tasks.workunit.client.1.smithi173.stdout:2/248: creat d2/d34/f51 x:0 0 0 2024-04-20T11:09:30.174 INFO:tasks.workunit.client.1.smithi173.stdout:9/270: chown d5/d9/d20/d29/d2d/c3d 294593 1 2024-04-20T11:09:30.175 INFO:tasks.workunit.client.1.smithi173.stdout:7/275: link d9/da/d2e/f37 d9/d33/d35/d58/f5f 0 2024-04-20T11:09:30.176 INFO:tasks.workunit.client.1.smithi173.stdout:0/224: write d6/f29 [1266713,103056] 0 2024-04-20T11:09:30.177 INFO:tasks.workunit.client.1.smithi173.stdout:8/283: write d4/f28 [6300732,129193] 0 2024-04-20T11:09:30.177 INFO:tasks.workunit.client.1.smithi173.stdout:0/225: chown d6/d8/df 13 1 2024-04-20T11:09:30.179 INFO:tasks.workunit.client.1.smithi173.stdout:1/219: write d5/d6/d17/d42/d12/f44 [1097459,121765] 0 2024-04-20T11:09:30.180 INFO:tasks.workunit.client.1.smithi173.stdout:8/284: fdatasync d4/d18/d3a/f46 0 2024-04-20T11:09:30.181 INFO:tasks.workunit.client.1.smithi173.stdout:8/285: chown d4 0 1 2024-04-20T11:09:30.182 INFO:tasks.workunit.client.1.smithi173.stdout:3/293: creat d7/d2b/f62 x:0 0 0 2024-04-20T11:09:30.182 INFO:tasks.workunit.client.1.smithi173.stdout:8/286: chown d4/c3e 43 1 2024-04-20T11:09:30.183 INFO:tasks.workunit.client.1.smithi173.stdout:3/294: stat d7/d53 0 2024-04-20T11:09:30.184 INFO:tasks.workunit.client.1.smithi173.stdout:0/226: mkdir d6/d8/d3c/d44 0 2024-04-20T11:09:30.185 INFO:tasks.workunit.client.1.smithi173.stdout:2/249: rmdir d2 39 2024-04-20T11:09:30.187 INFO:tasks.workunit.client.1.smithi173.stdout:1/220: mkdir d5/d6/d17/d42/da/d20/d48 0 2024-04-20T11:09:30.188 INFO:tasks.workunit.client.1.smithi173.stdout:8/287: creat d4/d18/d3a/d3c/f54 x:0 0 0 2024-04-20T11:09:30.189 INFO:tasks.workunit.client.1.smithi173.stdout:0/227: mknod d6/d8/c45 0 2024-04-20T11:09:30.192 INFO:tasks.workunit.client.1.smithi173.stdout:2/250: chown d2/d13/d28/c19 107 1 2024-04-20T11:09:30.193 INFO:tasks.workunit.client.1.smithi173.stdout:9/271: dwrite d5/d9/d20/d2b/f49 [0,4194304] 0 2024-04-20T11:09:30.193 INFO:tasks.workunit.client.1.smithi173.stdout:0/228: write d6/d8/f2f [1513167,131026] 0 2024-04-20T11:09:30.194 INFO:tasks.workunit.client.1.smithi173.stdout:3/295: link d7/d9/f35 d7/d9/d3b/d45/f63 0 2024-04-20T11:09:30.195 INFO:tasks.workunit.client.1.smithi173.stdout:2/251: chown d2/d34/l41 11 1 2024-04-20T11:09:30.196 INFO:tasks.workunit.client.1.smithi173.stdout:3/296: chown d7/d9/f4b 188 1 2024-04-20T11:09:30.196 INFO:tasks.workunit.client.1.smithi173.stdout:9/272: fdatasync d5/d9/d20/d2b/f4a 0 2024-04-20T11:09:30.197 INFO:tasks.workunit.client.1.smithi173.stdout:0/229: write d6/f7 [1455206,128203] 0 2024-04-20T11:09:30.197 INFO:tasks.workunit.client.1.smithi173.stdout:9/273: dread - d5/f53 zero size 2024-04-20T11:09:30.198 INFO:tasks.workunit.client.1.smithi173.stdout:8/288: rename d4/d18/d3a/d3c/f47 to d4/d18/f55 0 2024-04-20T11:09:30.198 INFO:tasks.workunit.client.1.smithi173.stdout:0/230: fsync d6/d8/d3c/f41 0 2024-04-20T11:09:30.199 INFO:tasks.workunit.client.1.smithi173.stdout:8/289: readlink l3 0 2024-04-20T11:09:30.227 INFO:tasks.workunit.client.1.smithi173.stdout:6/245: sync 2024-04-20T11:09:30.228 INFO:tasks.workunit.client.1.smithi173.stdout:9/274: write d5/d9/d32/d44/f56 [477614,71252] 0 2024-04-20T11:09:30.229 INFO:tasks.workunit.client.1.smithi173.stdout:6/246: stat d4/f46 0 2024-04-20T11:09:30.231 INFO:tasks.workunit.client.1.smithi173.stdout:9/275: fsync d5/d9/d20/d2b/d31/f4d 0 2024-04-20T11:09:30.235 INFO:tasks.workunit.client.1.smithi173.stdout:6/247: dread d4/fe [0,4194304] 0 2024-04-20T11:09:30.236 INFO:tasks.workunit.client.1.smithi173.stdout:3/297: truncate d7/d17/f19 1387792 0 2024-04-20T11:09:30.238 INFO:tasks.workunit.client.1.smithi173.stdout:2/252: dwrite d2/d13/d28/f15 [0,4194304] 0 2024-04-20T11:09:30.238 INFO:tasks.workunit.client.1.smithi173.stdout:6/248: write d4/f13 [886675,63742] 0 2024-04-20T11:09:30.239 INFO:tasks.workunit.client.1.smithi173.stdout:3/298: read - d7/d17/f2f zero size 2024-04-20T11:09:30.248 INFO:tasks.workunit.client.1.smithi173.stdout:8/290: dwrite d4/d18/f21 [0,4194304] 0 2024-04-20T11:09:30.250 INFO:tasks.workunit.client.1.smithi173.stdout:2/253: truncate d2/d13/f44 16261 0 2024-04-20T11:09:30.250 INFO:tasks.workunit.client.1.smithi173.stdout:8/291: chown d4/d18/l3f 3444036 1 2024-04-20T11:09:30.251 INFO:tasks.workunit.client.1.smithi173.stdout:3/299: symlink d7/d9/d5c/l64 0 2024-04-20T11:09:30.253 INFO:tasks.workunit.client.1.smithi173.stdout:9/276: dwrite d5/d9/d20/d29/d18/f1f [4194304,4194304] 0 2024-04-20T11:09:30.255 INFO:tasks.workunit.client.1.smithi173.stdout:2/254: truncate d2/de/f43 943052 0 2024-04-20T11:09:30.256 INFO:tasks.workunit.client.1.smithi173.stdout:3/300: write d7/d17/f57 [687603,72707] 0 2024-04-20T11:09:30.265 INFO:tasks.workunit.client.1.smithi173.stdout:9/277: fsync d5/d9/d32/f4e 0 2024-04-20T11:09:30.267 INFO:tasks.workunit.client.1.smithi173.stdout:2/255: chown d2/l48 4457588 1 2024-04-20T11:09:30.269 INFO:tasks.workunit.client.1.smithi173.stdout:6/249: dwrite d4/fe [0,4194304] 0 2024-04-20T11:09:30.270 INFO:tasks.workunit.client.1.smithi173.stdout:3/301: chown d7/d2b/l34 6 1 2024-04-20T11:09:30.338 INFO:tasks.workunit.client.1.smithi173.stdout:3/302: chown d7/d17/f24 111 1 2024-04-20T11:09:30.338 INFO:tasks.workunit.client.1.smithi173.stdout:8/292: dwrite d4/d18/f21 [0,4194304] 0 2024-04-20T11:09:30.338 INFO:tasks.workunit.client.1.smithi173.stdout:6/250: readlink d4/d18/l1d 0 2024-04-20T11:09:30.338 INFO:tasks.workunit.client.1.smithi173.stdout:3/303: readlink d7/d9/d3b/d45/l4a 0 2024-04-20T11:09:30.338 INFO:tasks.workunit.client.1.smithi173.stdout:2/256: symlink d2/d13/l52 0 2024-04-20T11:09:30.338 INFO:tasks.workunit.client.1.smithi173.stdout:9/278: symlink d5/d9/d32/d48/l57 0 2024-04-20T11:09:30.338 INFO:tasks.workunit.client.1.smithi173.stdout:6/251: truncate d4/fe 4951785 0 2024-04-20T11:09:30.338 INFO:tasks.workunit.client.1.smithi173.stdout:8/293: write d4/d18/d3a/d3c/f54 [757136,70675] 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:3/304: unlink d7/d9/l40 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:6/252: chown d4/d24/c31 1 1 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:2/257: rename d2/d29 to d2/d13/d46/d53 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:2/258: fdatasync d2/de/d11/f1d 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:9/279: dwrite d5/d9/d20/d29/d2d/f42 [0,4194304] 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:8/294: write d4/d18/f55 [1173628,22159] 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:9/280: chown d5/d9/d20/d29/d2d/d33/c36 1380725 1 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:6/253: creat d4/d24/d3e/f4d x:0 0 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:6/254: readlink d4/l49 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:6/255: write d4/f46 [735977,74756] 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:6/256: mknod d4/c4e 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:2/259: rename d2/d13/c27 to d2/c54 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:6/257: symlink d4/d1c/d28/l4f 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:8/295: dwrite d4/d18/d3a/d3c/f52 [0,4194304] 0 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:5/275: sync 2024-04-20T11:09:30.339 INFO:tasks.workunit.client.1.smithi173.stdout:4/241: sync 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:2/260: symlink d2/d13/d46/d4b/l55 0 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:2/261: creat d2/d34/f56 x:0 0 0 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:4/242: mkdir da/d4a 0 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:6/258: write d4/f16 [786536,72091] 0 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:4/243: mknod da/dc/d13/c4b 0 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:5/276: dwrite da/de/f37 [0,4194304] 0 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:2/262: creat d2/d13/d46/d53/d4e/f57 x:0 0 0 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:9/281: dread f0 [0,4194304] 0 2024-04-20T11:09:30.340 INFO:tasks.workunit.client.1.smithi173.stdout:5/277: write da/d12/f45 [58258,127986] 0 2024-04-20T11:09:30.341 INFO:tasks.workunit.client.1.smithi173.stdout:4/244: fsync da/dc/d13/f29 0 2024-04-20T11:09:30.345 INFO:tasks.workunit.client.1.smithi173.stdout:2/263: mkdir d2/d34/d58 0 2024-04-20T11:09:30.348 INFO:tasks.workunit.client.1.smithi173.stdout:9/282: creat d5/d9/f58 x:0 0 0 2024-04-20T11:09:30.349 INFO:tasks.workunit.client.1.smithi173.stdout:9/283: chown d5/f4b 0 1 2024-04-20T11:09:30.349 INFO:tasks.workunit.client.1.smithi173.stdout:5/278: mkdir da/d4f/d5e 0 2024-04-20T11:09:30.350 INFO:tasks.workunit.client.1.smithi173.stdout:4/245: mknod da/dc/d15/c4c 0 2024-04-20T11:09:30.351 INFO:tasks.workunit.client.1.smithi173.stdout:6/259: link d4/l1e d4/d24/d42/l50 0 2024-04-20T11:09:30.352 INFO:tasks.workunit.client.1.smithi173.stdout:9/284: chown d5/c16 15570195 1 2024-04-20T11:09:30.352 INFO:tasks.workunit.client.1.smithi173.stdout:4/246: write da/dc/d15/f35 [1386826,65156] 0 2024-04-20T11:09:30.357 INFO:tasks.workunit.client.1.smithi173.stdout:5/279: unlink da/de/l21 0 2024-04-20T11:09:30.359 INFO:tasks.workunit.client.1.smithi173.stdout:6/260: mknod d4/d14/c51 0 2024-04-20T11:09:30.361 INFO:tasks.workunit.client.1.smithi173.stdout:2/264: dwrite d2/d34/f3a [0,4194304] 0 2024-04-20T11:09:30.362 INFO:tasks.workunit.client.1.smithi173.stdout:2/265: write d2/f3e [204806,25226] 0 2024-04-20T11:09:30.364 INFO:tasks.workunit.client.1.smithi173.stdout:5/280: creat da/d4f/f5f x:0 0 0 2024-04-20T11:09:30.369 INFO:tasks.workunit.client.1.smithi173.stdout:6/261: creat d4/d1c/d28/d3b/f52 x:0 0 0 2024-04-20T11:09:30.376 INFO:tasks.workunit.client.1.smithi173.stdout:4/247: link da/dc/c22 da/d1e/c4d 0 2024-04-20T11:09:30.380 INFO:tasks.workunit.client.1.smithi173.stdout:4/248: chown da/dc/c2d 6 1 2024-04-20T11:09:30.383 INFO:tasks.workunit.client.1.smithi173.stdout:5/281: dwrite da/de/f37 [0,4194304] 0 2024-04-20T11:09:30.384 INFO:tasks.workunit.client.1.smithi173.stdout:9/285: dwrite d5/d9/d20/d29/f11 [0,4194304] 0 2024-04-20T11:09:30.384 INFO:tasks.workunit.client.1.smithi173.stdout:5/282: chown da/fc 1 1 2024-04-20T11:09:30.386 INFO:tasks.workunit.client.1.smithi173.stdout:6/262: rename d4/d14/f39 to d4/d1c/d28/f53 0 2024-04-20T11:09:30.386 INFO:tasks.workunit.client.1.smithi173.stdout:2/266: getdents d2/d13/d46/d53/d4e 0 2024-04-20T11:09:30.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:30 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:30 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:30 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:30 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:30 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:30 smithi117 ceph-mon[15659]: Reconfiguring prometheus.smithi117 (dependencies changed)... 2024-04-20T11:09:30.398 INFO:tasks.workunit.client.1.smithi173.stdout:5/283: link da/d12/f51 da/d12/d3e/f60 0 2024-04-20T11:09:30.400 INFO:tasks.workunit.client.1.smithi173.stdout:2/267: fsync d2/d13/f44 0 2024-04-20T11:09:30.402 INFO:tasks.workunit.client.1.smithi173.stdout:9/286: getdents d5/d9/d20/d2b 0 2024-04-20T11:09:30.403 INFO:tasks.workunit.client.1.smithi173.stdout:4/249: dwrite da/d1a/f40 [0,4194304] 0 2024-04-20T11:09:30.404 INFO:tasks.workunit.client.1.smithi173.stdout:6/263: dread d4/fe [0,4194304] 0 2024-04-20T11:09:30.410 INFO:tasks.workunit.client.1.smithi173.stdout:5/284: rmdir da/de/d19/d32/d30/d43 39 2024-04-20T11:09:30.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:30 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:30 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:30 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:30 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:30 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:30.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:30 smithi173 ceph-mon[23800]: Reconfiguring prometheus.smithi117 (dependencies changed)... 2024-04-20T11:09:30.414 INFO:tasks.workunit.client.1.smithi173.stdout:2/268: creat d2/d13/d46/d53/d4e/f59 x:0 0 0 2024-04-20T11:09:30.419 INFO:tasks.workunit.client.1.smithi173.stdout:6/264: creat d4/d24/f54 x:0 0 0 2024-04-20T11:09:30.420 INFO:tasks.workunit.client.1.smithi173.stdout:5/285: symlink da/de/d19/l61 0 2024-04-20T11:09:30.420 INFO:tasks.workunit.client.1.smithi173.stdout:6/265: read d4/f13 [1809496,79640] 0 2024-04-20T11:09:30.423 INFO:tasks.workunit.client.1.smithi173.stdout:6/266: write d4/d1c/d28/d3b/f41 [643018,93629] 0 2024-04-20T11:09:30.426 INFO:tasks.workunit.client.1.smithi173.stdout:1/221: sync 2024-04-20T11:09:30.426 INFO:tasks.workunit.client.1.smithi173.stdout:0/231: sync 2024-04-20T11:09:30.426 INFO:tasks.workunit.client.1.smithi173.stdout:7/276: sync 2024-04-20T11:09:30.428 INFO:tasks.workunit.client.1.smithi173.stdout:0/232: write d6/f29 [821883,92037] 0 2024-04-20T11:09:30.430 INFO:tasks.workunit.client.1.smithi173.stdout:2/269: creat d2/d34/d58/f5a x:0 0 0 2024-04-20T11:09:30.431 INFO:tasks.workunit.client.1.smithi173.stdout:6/267: write d4/f11 [1563748,5790] 0 2024-04-20T11:09:30.435 INFO:tasks.workunit.client.1.smithi173.stdout:4/250: dread da/dc/d13/f29 [0,4194304] 0 2024-04-20T11:09:30.435 INFO:tasks.workunit.client.1.smithi173.stdout:2/270: unlink d2/de/f24 0 2024-04-20T11:09:30.437 INFO:tasks.workunit.client.1.smithi173.stdout:6/268: creat d4/d1c/d28/d3b/f55 x:0 0 0 2024-04-20T11:09:30.439 INFO:tasks.workunit.client.1.smithi173.stdout:1/222: link d5/fb d5/d6/d3b/f49 0 2024-04-20T11:09:30.448 INFO:tasks.workunit.client.1.smithi173.stdout:1/223: dread d5/d6/d17/d42/d12/f47 [0,4194304] 0 2024-04-20T11:09:30.448 INFO:tasks.workunit.client.1.smithi173.stdout:0/233: link d6/d8/f1f d6/d8/d3c/d44/f46 0 2024-04-20T11:09:30.450 INFO:tasks.workunit.client.1.smithi173.stdout:7/277: dwrite d9/da/f14 [0,4194304] 0 2024-04-20T11:09:30.452 INFO:tasks.workunit.client.1.smithi173.stdout:4/251: dread da/d1a/d30/f3b [0,4194304] 0 2024-04-20T11:09:30.452 INFO:tasks.workunit.client.1.smithi173.stdout:1/224: dread d5/d6/d17/d42/da/f28 [0,4194304] 0 2024-04-20T11:09:30.456 INFO:tasks.workunit.client.1.smithi173.stdout:2/271: write d2/d13/f44 [14334,91354] 0 2024-04-20T11:09:30.457 INFO:tasks.workunit.client.1.smithi173.stdout:1/225: mkdir d5/d6/dc/d4a 0 2024-04-20T11:09:30.458 INFO:tasks.workunit.client.1.smithi173.stdout:6/269: write d4/f27 [202669,73903] 0 2024-04-20T11:09:30.459 INFO:tasks.workunit.client.1.smithi173.stdout:2/272: fdatasync d2/de/d11/f33 0 2024-04-20T11:09:30.463 INFO:tasks.workunit.client.1.smithi173.stdout:5/286: dread da/de/d19/d32/f11 [0,4194304] 0 2024-04-20T11:09:30.464 INFO:tasks.workunit.client.1.smithi173.stdout:6/270: write d4/d14/f2f [281841,101955] 0 2024-04-20T11:09:30.467 INFO:tasks.workunit.client.1.smithi173.stdout:2/273: write d2/d13/d28/f15 [2412316,114028] 0 2024-04-20T11:09:30.470 INFO:tasks.workunit.client.1.smithi173.stdout:1/226: link d5/d6/d17/d42/f41 d5/d6/d17/f4b 0 2024-04-20T11:09:30.470 INFO:tasks.workunit.client.1.smithi173.stdout:5/287: fsync da/de/d19/d32/f15 0 2024-04-20T11:09:30.472 INFO:tasks.workunit.client.1.smithi173.stdout:7/278: dwrite d9/da/d31/f4d [0,4194304] 0 2024-04-20T11:09:30.474 INFO:tasks.workunit.client.1.smithi173.stdout:4/252: dwrite da/dc/d13/f32 [4194304,4194304] 0 2024-04-20T11:09:30.475 INFO:tasks.workunit.client.1.smithi173.stdout:7/279: stat d9/ld 0 2024-04-20T11:09:30.484 INFO:tasks.workunit.client.1.smithi173.stdout:1/227: creat d5/d6/d17/d42/da/df/d40/f4c x:0 0 0 2024-04-20T11:09:30.485 INFO:tasks.workunit.client.1.smithi173.stdout:4/253: fsync da/dc/d15/f1c 0 2024-04-20T11:09:30.486 INFO:tasks.workunit.client.1.smithi173.stdout:5/288: link da/f2b da/de/d19/d2c/f62 0 2024-04-20T11:09:30.487 INFO:tasks.workunit.client.1.smithi173.stdout:7/280: dread d9/f1d [0,4194304] 0 2024-04-20T11:09:30.488 INFO:tasks.workunit.client.1.smithi173.stdout:1/228: symlink d5/d6/d17/d42/da/d20/l4d 0 2024-04-20T11:09:30.492 INFO:tasks.workunit.client.1.smithi173.stdout:1/229: symlink d5/d6/d17/d42/da/df/l4e 0 2024-04-20T11:09:30.496 INFO:tasks.workunit.client.1.smithi173.stdout:1/230: getdents d5/d6/d17/d42/da/d20/d38 0 2024-04-20T11:09:30.502 INFO:tasks.workunit.client.1.smithi173.stdout:1/231: creat d5/d6/d17/d42/d12/d1f/f4f x:0 0 0 2024-04-20T11:09:30.540 INFO:tasks.workunit.client.1.smithi173.stdout:3/305: sync 2024-04-20T11:09:30.541 INFO:tasks.workunit.client.1.smithi173.stdout:7/281: dwrite d9/d33/d40/f4b [0,4194304] 0 2024-04-20T11:09:30.542 INFO:tasks.workunit.client.1.smithi173.stdout:6/271: truncate d4/d18/f2b 372110 0 2024-04-20T11:09:30.544 INFO:tasks.workunit.client.1.smithi173.stdout:2/274: truncate d2/f3e 67512 0 2024-04-20T11:09:30.545 INFO:tasks.workunit.client.1.smithi173.stdout:5/289: dwrite da/de/d19/d2c/f53 [0,4194304] 0 2024-04-20T11:09:30.548 INFO:tasks.workunit.client.1.smithi173.stdout:6/272: chown d4/d1c/d28/d3b/f55 20044454 1 2024-04-20T11:09:30.548 INFO:tasks.workunit.client.1.smithi173.stdout:1/232: write d5/d6/d17/d42/f3f [241204,124301] 0 2024-04-20T11:09:30.549 INFO:tasks.workunit.client.1.smithi173.stdout:6/273: dread - d4/d24/d3e/f4d zero size 2024-04-20T11:09:30.554 INFO:tasks.workunit.client.1.smithi173.stdout:7/282: fsync d9/d33/d35/d58/f5f 0 2024-04-20T11:09:30.554 INFO:tasks.workunit.client.1.smithi173.stdout:2/275: creat d2/d34/f5b x:0 0 0 2024-04-20T11:09:30.556 INFO:tasks.workunit.client.1.smithi173.stdout:2/276: dread - d2/d13/f50 zero size 2024-04-20T11:09:30.560 INFO:tasks.workunit.client.1.smithi173.stdout:7/283: symlink d9/d33/d35/d56/l60 0 2024-04-20T11:09:30.565 INFO:tasks.workunit.client.1.smithi173.stdout:7/284: unlink d9/da/f5c 0 2024-04-20T11:09:30.568 INFO:tasks.workunit.client.1.smithi173.stdout:2/277: link d2/de/f21 d2/f5c 0 2024-04-20T11:09:30.570 INFO:tasks.workunit.client.1.smithi173.stdout:3/306: dread d7/d17/f29 [0,4194304] 0 2024-04-20T11:09:30.572 INFO:tasks.workunit.client.1.smithi173.stdout:1/233: dwrite d5/d6/d17/d42/d12/f31 [0,4194304] 0 2024-04-20T11:09:30.575 INFO:tasks.workunit.client.1.smithi173.stdout:2/278: unlink d2/de/d11/l40 0 2024-04-20T11:09:30.576 INFO:tasks.workunit.client.1.smithi173.stdout:2/279: dread - d2/f3f zero size 2024-04-20T11:09:30.578 INFO:tasks.workunit.client.1.smithi173.stdout:2/280: chown d2/d13/l52 108452 1 2024-04-20T11:09:30.581 INFO:tasks.workunit.client.1.smithi173.stdout:1/234: write d5/d6/f37 [2305438,111890] 0 2024-04-20T11:09:30.581 INFO:tasks.workunit.client.1.smithi173.stdout:2/281: dread - d2/d13/d46/d53/d4e/f57 zero size 2024-04-20T11:09:30.583 INFO:tasks.workunit.client.1.smithi173.stdout:1/235: chown d5/d6/d17/l29 9737 1 2024-04-20T11:09:30.584 INFO:tasks.workunit.client.1.smithi173.stdout:3/307: fsync d7/d9/f35 0 2024-04-20T11:09:30.585 INFO:tasks.workunit.client.1.smithi173.stdout:1/236: truncate d5/d6/f21 902234 0 2024-04-20T11:09:30.586 INFO:tasks.workunit.client.1.smithi173.stdout:1/237: fsync d5/d6/d17/d42/da/f28 0 2024-04-20T11:09:30.587 INFO:tasks.workunit.client.1.smithi173.stdout:2/282: symlink d2/d13/d46/l5d 0 2024-04-20T11:09:30.588 INFO:tasks.workunit.client.1.smithi173.stdout:5/290: dread da/de/f2d [0,4194304] 0 2024-04-20T11:09:30.590 INFO:tasks.workunit.client.1.smithi173.stdout:2/283: truncate d2/d34/f5b 845258 0 2024-04-20T11:09:30.590 INFO:tasks.workunit.client.1.smithi173.stdout:1/238: symlink d5/d6/d17/d42/l50 0 2024-04-20T11:09:30.591 INFO:tasks.workunit.client.1.smithi173.stdout:5/291: symlink da/d12/l63 0 2024-04-20T11:09:30.593 INFO:tasks.workunit.client.1.smithi173.stdout:5/292: write da/de/f37 [4573126,95840] 0 2024-04-20T11:09:30.593 INFO:tasks.workunit.client.1.smithi173.stdout:1/239: symlink d5/d6/d17/d42/da/l51 0 2024-04-20T11:09:30.597 INFO:tasks.workunit.client.1.smithi173.stdout:7/285: truncate d9/da/d31/f4d 1604131 0 2024-04-20T11:09:30.599 INFO:tasks.workunit.client.1.smithi173.stdout:5/293: write f7 [8551927,23394] 0 2024-04-20T11:09:30.600 INFO:tasks.workunit.client.1.smithi173.stdout:1/240: fsync d5/d6/d17/d42/d12/f15 0 2024-04-20T11:09:30.603 INFO:tasks.workunit.client.1.smithi173.stdout:1/241: dread d5/d6/d17/d42/da/f28 [0,4194304] 0 2024-04-20T11:09:30.603 INFO:tasks.workunit.client.1.smithi173.stdout:2/284: link d2/de/d11/f1d d2/de/f5e 0 2024-04-20T11:09:30.604 INFO:tasks.workunit.client.1.smithi173.stdout:7/286: symlink d9/da/d2e/l61 0 2024-04-20T11:09:30.607 INFO:tasks.workunit.client.1.smithi173.stdout:5/294: creat da/d12/f64 x:0 0 0 2024-04-20T11:09:30.609 INFO:tasks.workunit.client.1.smithi173.stdout:5/295: read da/d12/f1d [619589,38492] 0 2024-04-20T11:09:30.614 INFO:tasks.workunit.client.1.smithi173.stdout:1/242: mkdir d5/d6/d17/d42/da/d20/d38/d52 0 2024-04-20T11:09:30.618 INFO:tasks.workunit.client.1.smithi173.stdout:2/285: dwrite d2/d13/f50 [0,4194304] 0 2024-04-20T11:09:30.619 INFO:tasks.workunit.client.1.smithi173.stdout:5/296: truncate da/de/ff 2010318 0 2024-04-20T11:09:30.692 INFO:tasks.workunit.client.1.smithi173.stdout:8/296: sync 2024-04-20T11:09:30.693 INFO:tasks.workunit.client.1.smithi173.stdout:9/287: sync 2024-04-20T11:09:30.693 INFO:tasks.workunit.client.1.smithi173.stdout:2/286: symlink d2/d13/d28/l5f 0 2024-04-20T11:09:30.694 INFO:tasks.workunit.client.1.smithi173.stdout:5/297: write da/de/f2d [3760221,28384] 0 2024-04-20T11:09:30.694 INFO:tasks.workunit.client.1.smithi173.stdout:7/287: truncate d9/d33/d40/f49 52875 0 2024-04-20T11:09:30.697 INFO:tasks.workunit.client.1.smithi173.stdout:1/243: mknod d5/d6/d17/d42/da/d20/d38/d52/c53 0 2024-04-20T11:09:30.699 INFO:tasks.workunit.client.1.smithi173.stdout:5/298: mkdir da/d4f/d65 0 2024-04-20T11:09:30.704 INFO:tasks.workunit.client.1.smithi173.stdout:9/288: rename d5/d9/d32 to d5/d9/d20/d2b/d59 0 2024-04-20T11:09:30.705 INFO:tasks.workunit.client.1.smithi173.stdout:2/287: dread d2/d13/d28/f15 [0,4194304] 0 2024-04-20T11:09:30.705 INFO:tasks.workunit.client.1.smithi173.stdout:5/299: mkdir da/de/d19/d32/d66 0 2024-04-20T11:09:30.707 INFO:tasks.workunit.client.1.smithi173.stdout:9/289: stat d5/l7 0 2024-04-20T11:09:30.712 INFO:tasks.workunit.client.1.smithi173.stdout:1/244: rmdir d5/d6/d17 39 2024-04-20T11:09:30.713 INFO:tasks.workunit.client.1.smithi173.stdout:7/288: dwrite d9/d33/d35/f44 [0,4194304] 0 2024-04-20T11:09:30.714 INFO:tasks.workunit.client.1.smithi173.stdout:7/289: chown d9/da/d31 21 1 2024-04-20T11:09:30.714 INFO:tasks.workunit.client.1.smithi173.stdout:5/300: creat da/de/d19/d32/d30/d35/d48/f67 x:0 0 0 2024-04-20T11:09:30.716 INFO:tasks.workunit.client.1.smithi173.stdout:5/301: stat da/de/d19/d32/d30/l5a 0 2024-04-20T11:09:30.717 INFO:tasks.workunit.client.1.smithi173.stdout:5/302: truncate da/f2b 667926 0 2024-04-20T11:09:30.722 INFO:tasks.workunit.client.1.smithi173.stdout:1/245: rename d5/d6/f37 to d5/d6/d17/d42/f54 0 2024-04-20T11:09:30.724 INFO:tasks.workunit.client.1.smithi173.stdout:2/288: write d2/de/f5e [1220321,19094] 0 2024-04-20T11:09:30.725 INFO:tasks.workunit.client.1.smithi173.stdout:5/303: creat da/de/d19/d2c/f68 x:0 0 0 2024-04-20T11:09:30.728 INFO:tasks.workunit.client.1.smithi173.stdout:9/290: mknod d5/d9/d20/d2b/c5a 0 2024-04-20T11:09:30.730 INFO:tasks.workunit.client.1.smithi173.stdout:7/290: dwrite f1 [0,4194304] 0 2024-04-20T11:09:30.733 INFO:tasks.workunit.client.1.smithi173.stdout:1/246: rename d5/d6/d17/d42/f3f to d5/d6/dc/d4a/f55 0 2024-04-20T11:09:30.734 INFO:tasks.workunit.client.1.smithi173.stdout:7/291: read d9/da/d17/f1a [96006,121186] 0 2024-04-20T11:09:30.737 INFO:tasks.workunit.client.1.smithi173.stdout:7/292: truncate d9/d33/f47 420520 0 2024-04-20T11:09:30.737 INFO:tasks.workunit.client.1.smithi173.stdout:9/291: rename d5/d9/d20/d2b/f4a to d5/d9/d20/d2b/d31/f5b 0 2024-04-20T11:09:30.739 INFO:tasks.workunit.client.1.smithi173.stdout:7/293: symlink d9/da/l62 0 2024-04-20T11:09:30.751 INFO:tasks.workunit.client.1.smithi173.stdout:7/294: link f6 d9/d33/d35/d54/f63 0 2024-04-20T11:09:30.756 INFO:tasks.workunit.client.1.smithi173.stdout:7/295: dread f6 [4194304,4194304] 0 2024-04-20T11:09:30.757 INFO:tasks.workunit.client.1.smithi173.stdout:1/247: dread f4 [0,4194304] 0 2024-04-20T11:09:30.760 INFO:tasks.workunit.client.1.smithi173.stdout:9/292: truncate d5/d9/f1a 1211153 0 2024-04-20T11:09:30.762 INFO:tasks.workunit.client.1.smithi173.stdout:9/293: write d5/d9/d20/d2b/d31/f4d [919237,120883] 0 2024-04-20T11:09:30.763 INFO:tasks.workunit.client.1.smithi173.stdout:2/289: dread d2/de/f5e [0,4194304] 0 2024-04-20T11:09:30.764 INFO:tasks.workunit.client.1.smithi173.stdout:1/248: symlink d5/d6/d17/d42/da/d20/d48/l56 0 2024-04-20T11:09:30.768 INFO:tasks.workunit.client.1.smithi173.stdout:2/290: creat d2/de/f60 x:0 0 0 2024-04-20T11:09:30.769 INFO:tasks.workunit.client.1.smithi173.stdout:7/296: rename d9/da/d2e/f59 to d9/da/d17/f64 0 2024-04-20T11:09:30.770 INFO:tasks.workunit.client.1.smithi173.stdout:2/291: dread - d2/f3f zero size 2024-04-20T11:09:30.771 INFO:tasks.workunit.client.1.smithi173.stdout:1/249: rename d5 to d5/d57 22 2024-04-20T11:09:30.771 INFO:tasks.workunit.client.1.smithi173.stdout:2/292: dread - d2/f3f zero size 2024-04-20T11:09:30.772 INFO:tasks.workunit.client.1.smithi173.stdout:7/297: creat d9/d1f/f65 x:0 0 0 2024-04-20T11:09:30.773 INFO:tasks.workunit.client.1.smithi173.stdout:9/294: link d5/d9/f1d d5/d9/d20/d29/f5c 0 2024-04-20T11:09:30.775 INFO:tasks.workunit.client.1.smithi173.stdout:2/293: unlink d2/d13/d28/c19 0 2024-04-20T11:09:30.779 INFO:tasks.workunit.client.1.smithi173.stdout:2/294: symlink d2/d13/l61 0 2024-04-20T11:09:30.781 INFO:tasks.workunit.client.1.smithi173.stdout:9/295: link d5/d9/d20/l37 d5/d9/d20/d4f/l5d 0 2024-04-20T11:09:30.829 INFO:tasks.workunit.client.1.smithi173.stdout:0/234: sync 2024-04-20T11:09:30.829 INFO:tasks.workunit.client.1.smithi173.stdout:4/254: sync 2024-04-20T11:09:30.830 INFO:tasks.workunit.client.1.smithi173.stdout:9/296: rmdir d5/d9/d20/d29/d2d 39 2024-04-20T11:09:30.832 INFO:tasks.workunit.client.1.smithi173.stdout:7/298: write d9/d33/d40/f49 [73401,50547] 0 2024-04-20T11:09:30.837 INFO:tasks.workunit.client.1.smithi173.stdout:2/295: link d2/de/l3c d2/d13/d46/d53/d4e/l62 0 2024-04-20T11:09:30.840 INFO:tasks.workunit.client.1.smithi173.stdout:1/250: dwrite d5/d6/d17/d42/d12/d1f/f2f [0,4194304] 0 2024-04-20T11:09:30.840 INFO:tasks.workunit.client.1.smithi173.stdout:7/299: creat d9/d33/d35/f66 x:0 0 0 2024-04-20T11:09:30.842 INFO:tasks.workunit.client.1.smithi173.stdout:9/297: readlink d5/d9/d20/d29/d2d/l54 0 2024-04-20T11:09:30.844 INFO:tasks.workunit.client.1.smithi173.stdout:2/296: creat d2/d34/d58/f63 x:0 0 0 2024-04-20T11:09:30.846 INFO:tasks.workunit.client.1.smithi173.stdout:7/300: write d9/f2c [989301,125489] 0 2024-04-20T11:09:30.846 INFO:tasks.workunit.client.1.smithi173.stdout:1/251: fdatasync d5/d6/d17/f32 0 2024-04-20T11:09:30.847 INFO:tasks.workunit.client.1.smithi173.stdout:0/235: mknod d6/d8/df/d2c/d42/c47 0 2024-04-20T11:09:30.852 INFO:tasks.workunit.client.1.smithi173.stdout:9/298: mkdir d5/d9/d20/d4f/d5e 0 2024-04-20T11:09:30.853 INFO:tasks.workunit.client.1.smithi173.stdout:0/236: chown d6/d8/c45 197065 1 2024-04-20T11:09:30.855 INFO:tasks.workunit.client.1.smithi173.stdout:2/297: creat d2/f64 x:0 0 0 2024-04-20T11:09:30.856 INFO:tasks.workunit.client.1.smithi173.stdout:0/237: write d6/f13 [1662881,1399] 0 2024-04-20T11:09:30.856 INFO:tasks.workunit.client.1.smithi173.stdout:7/301: dread d9/d33/f47 [0,4194304] 0 2024-04-20T11:09:30.862 INFO:tasks.workunit.client.1.smithi173.stdout:1/252: truncate d5/d6/d17/d42/da/df/d40/f4c 726608 0 2024-04-20T11:09:30.866 INFO:tasks.workunit.client.1.smithi173.stdout:2/298: link d2/d13/d46/d53/l39 d2/l65 0 2024-04-20T11:09:30.871 INFO:tasks.workunit.client.1.smithi173.stdout:9/299: rmdir d5 39 2024-04-20T11:09:30.878 INFO:tasks.workunit.client.1.smithi173.stdout:9/300: mknod d5/d9/d20/d2b/c5f 0 2024-04-20T11:09:30.881 INFO:tasks.workunit.client.1.smithi173.stdout:0/238: dwrite d6/f7 [4194304,4194304] 0 2024-04-20T11:09:30.882 INFO:tasks.workunit.client.1.smithi173.stdout:7/302: dwrite d9/f30 [0,4194304] 0 2024-04-20T11:09:30.891 INFO:tasks.workunit.client.1.smithi173.stdout:9/301: mknod d5/d9/d20/c60 0 2024-04-20T11:09:30.913 INFO:tasks.workunit.client.1.smithi173.stdout:3/308: sync 2024-04-20T11:09:30.913 INFO:tasks.workunit.client.1.smithi173.stdout:6/274: sync 2024-04-20T11:09:30.915 INFO:tasks.workunit.client.1.smithi173.stdout:2/299: dwrite d2/d34/d58/f5a [0,4194304] 0 2024-04-20T11:09:30.915 INFO:tasks.workunit.client.1.smithi173.stdout:0/239: symlink d6/l48 0 2024-04-20T11:09:30.919 INFO:tasks.workunit.client.1.smithi173.stdout:7/303: mknod d9/c67 0 2024-04-20T11:09:30.921 INFO:tasks.workunit.client.1.smithi173.stdout:7/304: write d9/da/d17/f1b [1145062,71052] 0 2024-04-20T11:09:30.923 INFO:tasks.workunit.client.1.smithi173.stdout:9/302: mknod d5/d9/d20/d29/d2d/c61 0 2024-04-20T11:09:30.927 INFO:tasks.workunit.client.1.smithi173.stdout:2/300: mknod d2/d13/d28/d4a/c66 0 2024-04-20T11:09:30.929 INFO:tasks.workunit.client.1.smithi173.stdout:7/305: readlink d9/d33/d35/l5d 0 2024-04-20T11:09:30.933 INFO:tasks.workunit.client.1.smithi173.stdout:0/240: mknod d6/d8/df/d43/c49 0 2024-04-20T11:09:30.935 INFO:tasks.workunit.client.1.smithi173.stdout:6/275: dread d4/d14/f1a [0,4194304] 0 2024-04-20T11:09:30.936 INFO:tasks.workunit.client.1.smithi173.stdout:3/309: link d7/c1c d7/d9/d30/c65 0 2024-04-20T11:09:30.937 INFO:tasks.workunit.client.1.smithi173.stdout:6/276: truncate d4/d14/f3c 5714973 0 2024-04-20T11:09:30.938 INFO:tasks.workunit.client.1.smithi173.stdout:0/241: unlink c0 0 2024-04-20T11:09:30.939 INFO:tasks.workunit.client.1.smithi173.stdout:2/301: truncate d2/de/f21 2963826 0 2024-04-20T11:09:30.941 INFO:tasks.workunit.client.1.smithi173.stdout:7/306: symlink d9/d33/d35/d56/d5a/l68 0 2024-04-20T11:09:30.944 INFO:tasks.workunit.client.1.smithi173.stdout:2/302: mknod d2/d13/d46/d53/c67 0 2024-04-20T11:09:30.948 INFO:tasks.workunit.client.1.smithi173.stdout:2/303: dread - d2/d34/d58/f63 zero size 2024-04-20T11:09:30.948 INFO:tasks.workunit.client.1.smithi173.stdout:0/242: truncate d6/d8/df/d2c/d2d/f38 2040243 0 2024-04-20T11:09:30.949 INFO:tasks.workunit.client.1.smithi173.stdout:3/310: creat d7/d9/f66 x:0 0 0 2024-04-20T11:09:30.950 INFO:tasks.workunit.client.1.smithi173.stdout:0/243: chown d6/f27 353 1 2024-04-20T11:09:30.951 INFO:tasks.workunit.client.1.smithi173.stdout:9/303: dwrite d5/d9/d20/d29/f5c [0,4194304] 0 2024-04-20T11:09:30.951 INFO:tasks.workunit.client.1.smithi173.stdout:7/307: link d9/d33/c3c d9/d33/d35/d4a/c69 0 2024-04-20T11:09:30.952 INFO:tasks.workunit.client.1.smithi173.stdout:3/311: read d7/f18 [7457713,70642] 0 2024-04-20T11:09:30.955 INFO:tasks.workunit.client.1.smithi173.stdout:3/312: fsync d7/d9/f4f 0 2024-04-20T11:09:30.955 INFO:tasks.workunit.client.1.smithi173.stdout:0/244: creat d6/d8/d3c/f4a x:0 0 0 2024-04-20T11:09:30.955 INFO:tasks.workunit.client.1.smithi173.stdout:2/304: truncate d2/d13/d46/d53/d4e/f57 1046850 0 2024-04-20T11:09:30.959 INFO:tasks.workunit.client.1.smithi173.stdout:3/313: mknod d7/d9/d3b/c67 0 2024-04-20T11:09:30.962 INFO:tasks.workunit.client.1.smithi173.stdout:9/304: truncate d5/d9/f1b 723554 0 2024-04-20T11:09:30.963 INFO:tasks.workunit.client.1.smithi173.stdout:9/305: fdatasync d5/f6 0 2024-04-20T11:09:30.969 INFO:tasks.workunit.client.1.smithi173.stdout:9/306: rename d5/c16 to d5/d9/d20/d4f/c62 0 2024-04-20T11:09:30.972 INFO:tasks.workunit.client.1.smithi173.stdout:9/307: truncate d5/f2f 1274921 0 2024-04-20T11:09:30.974 INFO:tasks.workunit.client.1.smithi173.stdout:2/305: dread d2/d13/f3d [4194304,4194304] 0 2024-04-20T11:09:30.974 INFO:tasks.workunit.client.1.smithi173.stdout:3/314: dread d7/d9/f35 [0,4194304] 0 2024-04-20T11:09:30.978 INFO:tasks.workunit.client.1.smithi173.stdout:9/308: mknod d5/d9/d20/c63 0 2024-04-20T11:09:30.990 INFO:tasks.workunit.client.1.smithi173.stdout:2/306: write d2/f3e [590708,20461] 0 2024-04-20T11:09:30.995 INFO:tasks.workunit.client.1.smithi173.stdout:9/309: dwrite d5/d9/d20/d29/f11 [4194304,4194304] 0 2024-04-20T11:09:30.997 INFO:tasks.workunit.client.1.smithi173.stdout:2/307: creat d2/d13/d46/d53/d4e/f68 x:0 0 0 2024-04-20T11:09:30.999 INFO:tasks.workunit.client.1.smithi173.stdout:2/308: chown d2/d13/c18 0 1 2024-04-20T11:09:31.020 INFO:tasks.workunit.client.1.smithi173.stdout:2/309: dread d2/de/d11/f23 [0,4194304] 0 2024-04-20T11:09:31.023 INFO:tasks.workunit.client.1.smithi173.stdout:2/310: write d2/d34/f51 [329098,59363] 0 2024-04-20T11:09:31.025 INFO:tasks.workunit.client.1.smithi173.stdout:2/311: chown d2/d13/c18 662229 1 2024-04-20T11:09:31.028 INFO:tasks.workunit.client.1.smithi173.stdout:2/312: write d2/d13/f50 [2244650,37722] 0 2024-04-20T11:09:31.042 INFO:tasks.workunit.client.1.smithi173.stdout:8/297: sync 2024-04-20T11:09:31.062 INFO:tasks.workunit.client.1.smithi173.stdout:8/298: dwrite d4/d18/f55 [0,4194304] 0 2024-04-20T11:09:31.095 INFO:tasks.workunit.client.1.smithi173.stdout:8/299: dwrite d4/d18/f24 [0,4194304] 0 2024-04-20T11:09:31.100 INFO:tasks.workunit.client.1.smithi173.stdout:8/300: truncate d4/d18/d3a/d3c/f37 4659786 0 2024-04-20T11:09:31.110 INFO:tasks.workunit.client.1.smithi173.stdout:8/301: truncate d4/f8 968328 0 2024-04-20T11:09:31.116 INFO:tasks.workunit.client.1.smithi173.stdout:4/255: sync 2024-04-20T11:09:31.116 INFO:tasks.workunit.client.1.smithi173.stdout:5/304: sync 2024-04-20T11:09:31.116 INFO:tasks.workunit.client.1.smithi173.stdout:1/253: sync 2024-04-20T11:09:31.116 INFO:tasks.workunit.client.1.smithi173.stdout:6/277: sync 2024-04-20T11:09:31.120 INFO:tasks.workunit.client.1.smithi173.stdout:4/256: fsync da/dc/d15/f1d 0 2024-04-20T11:09:31.120 INFO:tasks.workunit.client.1.smithi173.stdout:6/278: chown d4/f16 138084 1 2024-04-20T11:09:31.122 INFO:tasks.workunit.client.1.smithi173.stdout:4/257: chown f1 271696 1 2024-04-20T11:09:31.128 INFO:tasks.workunit.client.1.smithi173.stdout:5/305: unlink da/de/d19/d32/d30/d43/f58 0 2024-04-20T11:09:31.133 INFO:tasks.workunit.client.1.smithi173.stdout:6/279: rename d4/d18/l1b to d4/d14/l56 0 2024-04-20T11:09:31.134 INFO:tasks.workunit.client.1.smithi173.stdout:8/302: dread d4/d18/f24 [0,4194304] 0 2024-04-20T11:09:31.136 INFO:tasks.workunit.client.1.smithi173.stdout:6/280: chown d4/f13 377878 1 2024-04-20T11:09:31.140 INFO:tasks.workunit.client.1.smithi173.stdout:8/303: symlink d4/l56 0 2024-04-20T11:09:31.143 INFO:tasks.workunit.client.1.smithi173.stdout:8/304: write f2 [1146657,89658] 0 2024-04-20T11:09:31.147 INFO:tasks.workunit.client.1.smithi173.stdout:4/258: dread da/dc/f33 [0,4194304] 0 2024-04-20T11:09:31.149 INFO:tasks.workunit.client.1.smithi173.stdout:4/259: read da/dc/f33 [1631836,108113] 0 2024-04-20T11:09:31.154 INFO:tasks.workunit.client.1.smithi173.stdout:4/260: rmdir da/d1a 39 2024-04-20T11:09:31.161 INFO:tasks.workunit.client.1.smithi173.stdout:4/261: creat da/dc/d15/f4e x:0 0 0 2024-04-20T11:09:31.162 INFO:tasks.workunit.client.1.smithi173.stdout:6/281: dwrite d4/d24/d3e/f4c [0,4194304] 0 2024-04-20T11:09:31.166 INFO:tasks.workunit.client.1.smithi173.stdout:6/282: rmdir d4/d18 39 2024-04-20T11:09:31.166 INFO:tasks.workunit.client.1.smithi173.stdout:8/305: dwrite d4/f44 [4194304,4194304] 0 2024-04-20T11:09:31.166 INFO:tasks.workunit.client.1.smithi173.stdout:5/306: dread da/fc [0,4194304] 0 2024-04-20T11:09:31.169 INFO:tasks.workunit.client.1.smithi173.stdout:6/283: write d4/d1c/d28/d3b/f41 [1222572,73286] 0 2024-04-20T11:09:31.174 INFO:tasks.workunit.client.1.smithi173.stdout:4/262: rename da/d1a/l28 to da/d1a/d30/l4f 0 2024-04-20T11:09:31.174 INFO:tasks.workunit.client.1.smithi173.stdout:5/307: symlink da/de/d19/l69 0 2024-04-20T11:09:31.176 INFO:tasks.workunit.client.1.smithi173.stdout:6/284: creat d4/d24/d3e/f57 x:0 0 0 2024-04-20T11:09:31.177 INFO:tasks.workunit.client.1.smithi173.stdout:5/308: fdatasync da/de/f37 0 2024-04-20T11:09:31.177 INFO:tasks.workunit.client.1.smithi173.stdout:4/263: rename da/d1a/d30/d39 to da/d1a/d30/d39/d50 22 2024-04-20T11:09:31.178 INFO:tasks.workunit.client.1.smithi173.stdout:8/306: link d4/f10 d4/d18/d4f/f57 0 2024-04-20T11:09:31.191 INFO:tasks.workunit.client.1.smithi173.stdout:5/309: dread da/de/d19/d2c/f53 [0,4194304] 0 2024-04-20T11:09:31.194 INFO:tasks.workunit.client.1.smithi173.stdout:4/264: rmdir da/d4a 0 2024-04-20T11:09:31.195 INFO:tasks.workunit.client.1.smithi173.stdout:4/265: chown da/dc/d15/l3f 0 1 2024-04-20T11:09:31.197 INFO:tasks.workunit.client.1.smithi173.stdout:5/310: symlink da/de/l6a 0 2024-04-20T11:09:31.197 INFO:tasks.workunit.client.1.smithi173.stdout:4/266: stat da/dc/d15/f1c 0 2024-04-20T11:09:31.199 INFO:tasks.workunit.client.1.smithi173.stdout:8/307: rename d4/d18/l4c to d4/d18/d3a/l58 0 2024-04-20T11:09:31.201 INFO:tasks.workunit.client.1.smithi173.stdout:7/308: sync 2024-04-20T11:09:31.203 INFO:tasks.workunit.client.1.smithi173.stdout:5/311: dread da/de/d19/d32/d30/f4c [0,4194304] 0 2024-04-20T11:09:31.204 INFO:tasks.workunit.client.1.smithi173.stdout:8/308: mkdir d4/d18/d59 0 2024-04-20T11:09:31.205 INFO:tasks.workunit.client.1.smithi173.stdout:6/285: dread d4/d24/f35 [0,4194304] 0 2024-04-20T11:09:31.206 INFO:tasks.workunit.client.1.smithi173.stdout:4/267: getdents da/d1a 0 2024-04-20T11:09:31.207 INFO:tasks.workunit.client.1.smithi173.stdout:5/312: truncate da/de/d19/d32/f3c 983082 0 2024-04-20T11:09:31.208 INFO:tasks.workunit.client.1.smithi173.stdout:4/268: read da/dc/d13/f29 [2688427,91829] 0 2024-04-20T11:09:31.210 INFO:tasks.workunit.client.1.smithi173.stdout:8/309: mkdir d4/d18/d59/d5a 0 2024-04-20T11:09:31.215 INFO:tasks.workunit.client.1.smithi173.stdout:8/310: mkdir d4/d18/d4f/d5b 0 2024-04-20T11:09:31.215 INFO:tasks.workunit.client.1.smithi173.stdout:6/286: dread d4/d18/f2b [0,4194304] 0 2024-04-20T11:09:31.217 INFO:tasks.workunit.client.1.smithi173.stdout:4/269: link da/dc/f33 da/dc/d13/f51 0 2024-04-20T11:09:31.219 INFO:tasks.workunit.client.1.smithi173.stdout:7/309: dwrite d9/da/d31/f38 [0,4194304] 0 2024-04-20T11:09:31.219 INFO:tasks.workunit.client.1.smithi173.stdout:8/311: read d4/f19 [1557016,86126] 0 2024-04-20T11:09:31.226 INFO:tasks.workunit.client.1.smithi173.stdout:4/270: mkdir da/d1e/d52 0 2024-04-20T11:09:31.234 INFO:tasks.workunit.client.1.smithi173.stdout:4/271: truncate da/d1a/d30/f37 732667 0 2024-04-20T11:09:31.235 INFO:tasks.workunit.client.1.smithi173.stdout:4/272: stat da/d1e/c36 0 2024-04-20T11:09:31.236 INFO:tasks.workunit.client.1.smithi173.stdout:5/313: dwrite da/de/d19/d32/d30/d35/f54 [0,4194304] 0 2024-04-20T11:09:31.239 INFO:tasks.workunit.client.1.smithi173.stdout:5/314: write da/de/d19/d32/f44 [4813971,17004] 0 2024-04-20T11:09:31.245 INFO:tasks.workunit.client.1.smithi173.stdout:8/312: dwrite d4/d18/d3a/d3c/f34 [0,4194304] 0 2024-04-20T11:09:31.251 INFO:tasks.workunit.client.1.smithi173.stdout:3/315: sync 2024-04-20T11:09:31.251 INFO:tasks.workunit.client.1.smithi173.stdout:0/245: sync 2024-04-20T11:09:31.252 INFO:tasks.workunit.client.1.smithi173.stdout:2/313: sync 2024-04-20T11:09:31.252 INFO:tasks.workunit.client.1.smithi173.stdout:6/287: dread d4/f16 [0,4194304] 0 2024-04-20T11:09:31.252 INFO:tasks.workunit.client.1.smithi173.stdout:9/310: sync 2024-04-20T11:09:31.254 INFO:tasks.workunit.client.1.smithi173.stdout:5/315: symlink da/d12/d40/d59/l6b 0 2024-04-20T11:09:31.255 INFO:tasks.workunit.client.1.smithi173.stdout:5/316: chown da/de 81 1 2024-04-20T11:09:31.257 INFO:tasks.workunit.client.1.smithi173.stdout:9/311: chown d5/d9/d20/c60 0 1 2024-04-20T11:09:31.263 INFO:tasks.workunit.client.1.smithi173.stdout:6/288: creat d4/d18/f58 x:0 0 0 2024-04-20T11:09:31.266 INFO:tasks.workunit.client.1.smithi173.stdout:5/317: rmdir da/de/d19 39 2024-04-20T11:09:31.268 INFO:tasks.workunit.client.1.smithi173.stdout:2/314: link d2/f64 d2/d34/d58/f69 0 2024-04-20T11:09:31.268 INFO:tasks.workunit.client.1.smithi173.stdout:5/318: chown f8 237 1 2024-04-20T11:09:31.268 INFO:tasks.workunit.client.1.smithi173.stdout:6/289: stat d4/d1c/d28/d3b/f55 0 2024-04-20T11:09:31.273 INFO:tasks.workunit.client.1.smithi173.stdout:8/313: dread d4/f8 [0,4194304] 0 2024-04-20T11:09:31.276 INFO:tasks.workunit.client.1.smithi173.stdout:5/319: mkdir da/d34/d6c 0 2024-04-20T11:09:31.276 INFO:tasks.workunit.client.1.smithi173.stdout:8/314: chown d4/d18/d3a/d3c 1098812595 1 2024-04-20T11:09:31.276 INFO:tasks.workunit.client.1.smithi173.stdout:2/315: fsync d2/d13/d46/d53/d4e/f57 0 2024-04-20T11:09:31.277 INFO:tasks.workunit.client.1.smithi173.stdout:6/290: write d4/d1c/d28/d3b/f55 [808468,5420] 0 2024-04-20T11:09:31.277 INFO:tasks.workunit.client.1.smithi173.stdout:3/316: write d7/f2d [4250249,42561] 0 2024-04-20T11:09:31.281 INFO:tasks.workunit.client.1.smithi173.stdout:3/317: fsync d7/f22 0 2024-04-20T11:09:31.283 INFO:tasks.workunit.client.1.smithi173.stdout:0/246: dread d6/d8/df/d2c/d2d/f38 [0,4194304] 0 2024-04-20T11:09:31.284 INFO:tasks.workunit.client.1.smithi173.stdout:6/291: readlink d4/d1c/l32 0 2024-04-20T11:09:31.285 INFO:tasks.workunit.client.1.smithi173.stdout:9/312: dwrite d5/d9/d20/d29/d2d/f43 [0,4194304] 0 2024-04-20T11:09:31.287 INFO:tasks.workunit.client.1.smithi173.stdout:5/320: chown da/de/d19/l56 852 1 2024-04-20T11:09:31.290 INFO:tasks.workunit.client.1.smithi173.stdout:4/273: dread da/dc/d15/f18 [0,4194304] 0 2024-04-20T11:09:31.291 INFO:tasks.workunit.client.1.smithi173.stdout:3/318: chown d7/d9/d30/f44 1754143 1 2024-04-20T11:09:31.300 INFO:tasks.workunit.client.1.smithi173.stdout:2/316: dwrite d2/de/d11/f33 [0,4194304] 0 2024-04-20T11:09:31.300 INFO:tasks.workunit.client.1.smithi173.stdout:4/274: rename da/d1a/d30 to da/d48/d53 0 2024-04-20T11:09:31.306 INFO:tasks.workunit.client.1.smithi173.stdout:2/317: fdatasync d2/f64 0 2024-04-20T11:09:31.308 INFO:tasks.workunit.client.1.smithi173.stdout:3/319: rename d7/d9/d30/c33 to d7/d17/d58/c68 0 2024-04-20T11:09:31.311 INFO:tasks.workunit.client.1.smithi173.stdout:9/313: creat d5/d9/d20/d29/f64 x:0 0 0 2024-04-20T11:09:31.312 INFO:tasks.workunit.client.1.smithi173.stdout:5/321: dwrite da/de/d19/d32/d30/f41 [0,4194304] 0 2024-04-20T11:09:31.321 INFO:tasks.workunit.client.1.smithi173.stdout:9/314: write d5/d9/d20/d29/d2d/f43 [5032188,101490] 0 2024-04-20T11:09:31.321 INFO:tasks.workunit.client.1.smithi173.stdout:2/318: mkdir d2/d34/d58/d6a 0 2024-04-20T11:09:31.321 INFO:tasks.workunit.client.1.smithi173.stdout:6/292: dwrite d4/f37 [0,4194304] 0 2024-04-20T11:09:31.324 INFO:tasks.workunit.client.1.smithi173.stdout:1/254: sync 2024-04-20T11:09:31.327 INFO:tasks.workunit.client.1.smithi173.stdout:2/319: readlink d2/l65 0 2024-04-20T11:09:31.327 INFO:tasks.workunit.client.1.smithi173.stdout:3/320: symlink d7/d2b/l69 0 2024-04-20T11:09:31.332 INFO:tasks.workunit.client.1.smithi173.stdout:2/320: write d2/d34/f5b [568279,129691] 0 2024-04-20T11:09:31.333 INFO:tasks.workunit.client.1.smithi173.stdout:8/315: dread d4/f28 [4194304,4194304] 0 2024-04-20T11:09:31.335 INFO:tasks.workunit.client.1.smithi173.stdout:4/275: write da/d1e/f38 [822515,127753] 0 2024-04-20T11:09:31.335 INFO:tasks.workunit.client.1.smithi173.stdout:3/321: mknod d7/c6a 0 2024-04-20T11:09:31.339 INFO:tasks.workunit.client.1.smithi173.stdout:2/321: creat d2/d34/d58/f6b x:0 0 0 2024-04-20T11:09:31.340 INFO:tasks.workunit.client.1.smithi173.stdout:1/255: link d5/d6/d17/d42/l50 d5/d6/d17/l58 0 2024-04-20T11:09:31.340 INFO:tasks.workunit.client.1.smithi173.stdout:3/322: dread - d7/d9/f14 zero size 2024-04-20T11:09:31.342 INFO:tasks.workunit.client.1.smithi173.stdout:8/316: unlink d4/d18/f4a 0 2024-04-20T11:09:31.342 INFO:tasks.workunit.client.1.smithi173.stdout:5/322: dread da/fc [0,4194304] 0 2024-04-20T11:09:31.346 INFO:tasks.workunit.client.1.smithi173.stdout:4/276: read da/d1e/f38 [70078,83165] 0 2024-04-20T11:09:31.346 INFO:tasks.workunit.client.1.smithi173.stdout:8/317: read d4/f36 [2345391,29722] 0 2024-04-20T11:09:31.347 INFO:tasks.workunit.client.1.smithi173.stdout:2/322: write d2/d34/d58/f63 [331571,92390] 0 2024-04-20T11:09:31.352 INFO:tasks.workunit.client.1.smithi173.stdout:1/256: stat d5/d6/d17/d42/f54 0 2024-04-20T11:09:31.353 INFO:tasks.workunit.client.1.smithi173.stdout:3/323: truncate d7/d9/d30/f60 924239 0 2024-04-20T11:09:31.359 INFO:tasks.workunit.client.1.smithi173.stdout:9/315: dwrite d5/f53 [0,4194304] 0 2024-04-20T11:09:31.359 INFO:tasks.workunit.client.1.smithi173.stdout:3/324: mknod d7/d53/c6b 0 2024-04-20T11:09:31.364 INFO:tasks.workunit.client.1.smithi173.stdout:8/318: rename d4/f36 to d4/d18/f5c 0 2024-04-20T11:09:31.365 INFO:tasks.workunit.client.1.smithi173.stdout:3/325: symlink d7/d53/l6c 0 2024-04-20T11:09:31.368 INFO:tasks.workunit.client.1.smithi173.stdout:9/316: rmdir d5/d9/d20/d29/d2d/d33 39 2024-04-20T11:09:31.373 INFO:tasks.workunit.client.1.smithi173.stdout:9/317: fdatasync d5/d9/d20/d29/f5c 0 2024-04-20T11:09:31.375 INFO:tasks.workunit.client.1.smithi173.stdout:5/323: dwrite da/de/d19/d32/f11 [0,4194304] 0 2024-04-20T11:09:31.375 INFO:tasks.workunit.client.1.smithi173.stdout:6/293: dwrite d4/d18/f2b [0,4194304] 0 2024-04-20T11:09:31.378 INFO:tasks.workunit.client.1.smithi173.stdout:9/318: write d5/f4b [9417242,43587] 0 2024-04-20T11:09:31.381 INFO:tasks.workunit.client.1.smithi173.stdout:8/319: write d4/f8 [1309694,103522] 0 2024-04-20T11:09:31.382 INFO:tasks.workunit.client.1.smithi173.stdout:9/319: truncate d5/f38 4335264 0 2024-04-20T11:09:31.383 INFO:tasks.workunit.client.1.smithi173.stdout:6/294: fdatasync d4/fe 0 2024-04-20T11:09:31.384 INFO:tasks.workunit.client.1.smithi173.stdout:5/324: creat da/d4f/d5e/f6d x:0 0 0 2024-04-20T11:09:31.389 INFO:tasks.workunit.client.1.smithi173.stdout:6/295: chown d4/f37 1 1 2024-04-20T11:09:31.391 INFO:tasks.workunit.client.1.smithi173.stdout:3/326: write d7/d17/f26 [686505,39613] 0 2024-04-20T11:09:31.393 INFO:tasks.workunit.client.1.smithi173.stdout:5/325: creat da/de/d19/d32/d30/d35/f6e x:0 0 0 2024-04-20T11:09:31.394 INFO:tasks.workunit.client.1.smithi173.stdout:8/320: creat d4/d18/d59/d5a/f5d x:0 0 0 2024-04-20T11:09:31.396 INFO:tasks.workunit.client.1.smithi173.stdout:9/320: unlink d5/d9/f13 0 2024-04-20T11:09:31.397 INFO:tasks.workunit.client.1.smithi173.stdout:3/327: write f4 [3478631,107765] 0 2024-04-20T11:09:31.400 INFO:tasks.workunit.client.1.smithi173.stdout:6/296: fsync d4/f20 0 2024-04-20T11:09:31.403 INFO:tasks.workunit.client.1.smithi173.stdout:2/323: dread d2/d34/f3a [0,4194304] 0 2024-04-20T11:09:31.414 INFO:tasks.workunit.client.1.smithi173.stdout:4/277: dread da/f25 [0,4194304] 0 2024-04-20T11:09:31.415 INFO:tasks.workunit.client.1.smithi173.stdout:2/324: creat d2/d13/d28/d4a/f6c x:0 0 0 2024-04-20T11:09:31.421 INFO:tasks.workunit.client.1.smithi173.stdout:4/278: fsync da/dc/d13/f47 0 2024-04-20T11:09:31.429 INFO:tasks.workunit.client.1.smithi173.stdout:5/326: dwrite da/de/d19/d32/d30/f4c [0,4194304] 0 2024-04-20T11:09:31.429 INFO:tasks.workunit.client.1.smithi173.stdout:4/279: rmdir da/d1a 39 2024-04-20T11:09:31.433 INFO:tasks.workunit.client.1.smithi173.stdout:9/321: dwrite d5/d9/f1d [0,4194304] 0 2024-04-20T11:09:31.438 INFO:tasks.workunit.client.1.smithi173.stdout:5/327: creat da/d34/f6f x:0 0 0 2024-04-20T11:09:31.443 INFO:tasks.workunit.client.1.smithi173.stdout:6/297: dwrite d4/d14/f2f [0,4194304] 0 2024-04-20T11:09:31.450 INFO:tasks.workunit.client.1.smithi173.stdout:3/328: dread d7/d17/f19 [0,4194304] 0 2024-04-20T11:09:31.454 INFO:tasks.workunit.client.1.smithi173.stdout:5/328: creat da/de/d19/d32/d30/f70 x:0 0 0 2024-04-20T11:09:31.459 INFO:tasks.workunit.client.1.smithi173.stdout:3/329: symlink d7/d9/d3b/l6d 0 2024-04-20T11:09:31.462 INFO:tasks.workunit.client.1.smithi173.stdout:6/298: getdents d4/d24 0 2024-04-20T11:09:31.463 INFO:tasks.workunit.client.1.smithi173.stdout:3/330: chown d7/f51 40844983 1 2024-04-20T11:09:31.466 INFO:tasks.workunit.client.1.smithi173.stdout:5/329: unlink da/de/d19/d32/d30/d35/f38 0 2024-04-20T11:09:31.467 INFO:tasks.workunit.client.1.smithi173.stdout:6/299: fdatasync d4/d24/f54 0 2024-04-20T11:09:31.468 INFO:tasks.workunit.client.1.smithi173.stdout:7/310: sync 2024-04-20T11:09:31.470 INFO:tasks.workunit.client.1.smithi173.stdout:3/331: mknod d7/d17/c6e 0 2024-04-20T11:09:31.470 INFO:tasks.workunit.client.1.smithi173.stdout:4/280: dwrite da/dc/d13/f29 [0,4194304] 0 2024-04-20T11:09:31.473 INFO:tasks.workunit.client.1.smithi173.stdout:6/300: write d4/d14/f3c [1047907,119211] 0 2024-04-20T11:09:31.475 INFO:tasks.workunit.client.1.smithi173.stdout:5/330: symlink da/d34/l71 0 2024-04-20T11:09:31.477 INFO:tasks.workunit.client.1.smithi173.stdout:7/311: write f1 [3176460,101618] 0 2024-04-20T11:09:31.477 INFO:tasks.workunit.client.1.smithi173.stdout:4/281: write da/dc/d13/f2b [1347448,91961] 0 2024-04-20T11:09:31.478 INFO:tasks.workunit.client.1.smithi173.stdout:6/301: fdatasync d4/d18/f3f 0 2024-04-20T11:09:31.478 INFO:tasks.workunit.client.1.smithi173.stdout:5/331: read da/de/d19/f2a [3422873,51969] 0 2024-04-20T11:09:31.482 INFO:tasks.workunit.client.1.smithi173.stdout:9/322: dwrite d5/d9/d20/d2b/d31/f5b [0,4194304] 0 2024-04-20T11:09:31.483 INFO:tasks.workunit.client.1.smithi173.stdout:6/302: fdatasync d4/d14/f43 0 2024-04-20T11:09:31.486 INFO:tasks.workunit.client.1.smithi173.stdout:9/323: fsync d5/f2f 0 2024-04-20T11:09:31.489 INFO:tasks.workunit.client.1.smithi173.stdout:9/324: chown d5/d9/d20/d2b/d59/d48 0 1 2024-04-20T11:09:31.489 INFO:tasks.workunit.client.1.smithi173.stdout:5/332: dread - da/de/d19/d32/d30/d35/d48/f67 zero size 2024-04-20T11:09:31.492 INFO:tasks.workunit.client.1.smithi173.stdout:6/303: mkdir d4/d59 0 2024-04-20T11:09:31.496 INFO:tasks.workunit.client.1.smithi173.stdout:9/325: read d5/d9/d20/d2b/d59/d44/f56 [79551,98216] 0 2024-04-20T11:09:31.497 INFO:tasks.workunit.client.1.smithi173.stdout:5/333: mkdir da/de/d19/d32/d72 0 2024-04-20T11:09:31.504 INFO:tasks.workunit.client.1.smithi173.stdout:5/334: fsync da/de/d19/d32/f52 0 2024-04-20T11:09:31.509 INFO:tasks.workunit.client.1.smithi173.stdout:6/304: dread d4/f46 [0,4194304] 0 2024-04-20T11:09:31.512 INFO:tasks.workunit.client.1.smithi173.stdout:3/332: dwrite d7/d9/f28 [0,4194304] 0 2024-04-20T11:09:31.513 INFO:tasks.workunit.client.1.smithi173.stdout:9/326: getdents d5/d9/d20/d29/d2d 0 2024-04-20T11:09:31.521 INFO:tasks.workunit.client.1.smithi173.stdout:6/305: write d4/f46 [1642184,117086] 0 2024-04-20T11:09:31.523 INFO:tasks.workunit.client.1.smithi173.stdout:4/282: dread da/dc/d13/f2b [0,4194304] 0 2024-04-20T11:09:31.528 INFO:tasks.workunit.client.1.smithi173.stdout:5/335: dread da/d12/f45 [0,4194304] 0 2024-04-20T11:09:31.528 INFO:tasks.workunit.client.1.smithi173.stdout:7/312: dread d9/d33/d35/d54/f63 [0,4194304] 0 2024-04-20T11:09:31.529 INFO:tasks.workunit.client.1.smithi173.stdout:4/283: mknod da/d1e/d52/c54 0 2024-04-20T11:09:31.532 INFO:tasks.workunit.client.1.smithi173.stdout:7/313: mknod d9/d33/d35/c6a 0 2024-04-20T11:09:31.533 INFO:tasks.workunit.client.1.smithi173.stdout:4/284: fdatasync da/dc/f33 0 2024-04-20T11:09:31.535 INFO:tasks.workunit.client.1.smithi173.stdout:7/314: truncate d9/f4e 50858 0 2024-04-20T11:09:31.535 INFO:tasks.workunit.client.1.smithi173.stdout:6/306: rename d4/fe to d4/f5a 0 2024-04-20T11:09:31.538 INFO:tasks.workunit.client.1.smithi173.stdout:4/285: creat da/d48/d53/f55 x:0 0 0 2024-04-20T11:09:31.539 INFO:tasks.workunit.client.1.smithi173.stdout:7/315: creat d9/d33/f6b x:0 0 0 2024-04-20T11:09:31.542 INFO:tasks.workunit.client.1.smithi173.stdout:4/286: rmdir da/d48/d53/d39 39 2024-04-20T11:09:31.543 INFO:tasks.workunit.client.1.smithi173.stdout:7/316: mknod d9/d33/d35/d56/c6c 0 2024-04-20T11:09:31.546 INFO:tasks.workunit.client.1.smithi173.stdout:6/307: dread d4/d1c/f4a [0,4194304] 0 2024-04-20T11:09:31.547 INFO:tasks.workunit.client.1.smithi173.stdout:9/327: truncate d5/d9/f26 2441755 0 2024-04-20T11:09:31.548 INFO:tasks.workunit.client.1.smithi173.stdout:4/287: getdents da/dc 0 2024-04-20T11:09:31.551 INFO:tasks.workunit.client.1.smithi173.stdout:6/308: unlink d4/c33 0 2024-04-20T11:09:31.554 INFO:tasks.workunit.client.1.smithi173.stdout:3/333: dwrite d7/f15 [0,4194304] 0 2024-04-20T11:09:31.557 INFO:tasks.workunit.client.1.smithi173.stdout:3/334: write d7/f15 [3521451,14340] 0 2024-04-20T11:09:31.562 INFO:tasks.workunit.client.1.smithi173.stdout:9/328: rename d5/l7 to d5/d9/d20/d2b/l65 0 2024-04-20T11:09:31.565 INFO:tasks.workunit.client.1.smithi173.stdout:7/317: getdents d9/da/d31 0 2024-04-20T11:09:31.567 INFO:tasks.workunit.client.1.smithi173.stdout:4/288: creat da/d48/d53/d39/f56 x:0 0 0 2024-04-20T11:09:31.568 INFO:tasks.workunit.client.1.smithi173.stdout:9/329: symlink d5/d9/d20/d4f/l66 0 2024-04-20T11:09:31.568 INFO:tasks.workunit.client.1.smithi173.stdout:6/309: link d4/d1c/d28/c2a d4/d24/c5b 0 2024-04-20T11:09:31.573 INFO:tasks.workunit.client.1.smithi173.stdout:7/318: creat d9/d33/d35/d4a/d52/d5b/f6d x:0 0 0 2024-04-20T11:09:31.573 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:31 smithi117 ceph-mon[15659]: Reconfiguring daemon prometheus.smithi117 on smithi117 2024-04-20T11:09:31.573 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:31 smithi117 ceph-mon[15659]: pgmap v25: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 42 MiB/s rd, 113 MiB/s wr, 275 op/s 2024-04-20T11:09:31.574 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:09:31.576 INFO:tasks.workunit.client.1.smithi173.stdout:6/310: mknod d4/d1c/d28/c5c 0 2024-04-20T11:09:31.577 INFO:tasks.workunit.client.1.smithi173.stdout:6/311: fdatasync d4/f46 0 2024-04-20T11:09:31.578 INFO:tasks.workunit.client.1.smithi173.stdout:6/312: chown d4/d24/d42 4822 1 2024-04-20T11:09:31.579 INFO:tasks.workunit.client.1.smithi173.stdout:0/247: sync 2024-04-20T11:09:31.579 INFO:tasks.workunit.client.1.smithi173.stdout:1/257: sync 2024-04-20T11:09:31.580 INFO:tasks.workunit.client.1.smithi173.stdout:7/319: rename d9/d1f/c51 to d9/d33/d35/d56/d5a/c6e 0 2024-04-20T11:09:31.581 INFO:tasks.workunit.client.1.smithi173.stdout:3/335: dwrite d7/f25 [0,4194304] 0 2024-04-20T11:09:31.581 INFO:tasks.workunit.client.1.smithi173.stdout:0/248: rename d6 to d6/d8/df/d2c/d2d/d4b 22 2024-04-20T11:09:31.583 INFO:tasks.workunit.client.1.smithi173.stdout:0/249: fsync d6/d8/d3c/f41 0 2024-04-20T11:09:31.591 INFO:tasks.workunit.client.1.smithi173.stdout:1/258: creat d5/d6/dc/d4a/f59 x:0 0 0 2024-04-20T11:09:31.593 INFO:tasks.workunit.client.1.smithi173.stdout:4/289: link da/d1a/l1f da/d48/l57 0 2024-04-20T11:09:31.600 INFO:tasks.workunit.client.1.smithi173.stdout:0/250: rename d6/d8/df/d2c/d2d/f38 to d6/d8/df/d43/f4c 0 2024-04-20T11:09:31.603 INFO:tasks.workunit.client.1.smithi173.stdout:0/251: write d6/d8/df/d15/f39 [1572792,54444] 0 2024-04-20T11:09:31.603 INFO:tasks.workunit.client.1.smithi173.stdout:9/330: getdents d5/d9/d20/d4f 0 2024-04-20T11:09:31.604 INFO:tasks.workunit.client.1.smithi173.stdout:6/313: symlink d4/d24/d42/l5d 0 2024-04-20T11:09:31.606 INFO:tasks.workunit.client.1.smithi173.stdout:7/320: mkdir d9/da/d17/d2b/d6f 0 2024-04-20T11:09:31.607 INFO:tasks.workunit.client.1.smithi173.stdout:7/321: readlink d9/da/l62 0 2024-04-20T11:09:31.609 INFO:tasks.workunit.client.1.smithi173.stdout:7/322: dread - d9/da/d43/f57 zero size 2024-04-20T11:09:31.611 INFO:tasks.workunit.client.1.smithi173.stdout:1/259: rename d5/d6/dc/l23 to d5/d6/d3b/l5a 0 2024-04-20T11:09:31.615 INFO:tasks.workunit.client.1.smithi173.stdout:0/252: write d6/d8/df/f1d [697879,10785] 0 2024-04-20T11:09:31.615 INFO:tasks.workunit.client.1.smithi173.stdout:6/314: truncate d4/d1c/f4a 624313 0 2024-04-20T11:09:31.617 INFO:tasks.workunit.client.1.smithi173.stdout:4/290: mkdir da/d48/d53/d58 0 2024-04-20T11:09:31.621 INFO:tasks.workunit.client.1.smithi173.stdout:3/336: creat d7/d9/f6f x:0 0 0 2024-04-20T11:09:31.622 INFO:tasks.workunit.client.1.smithi173.stdout:3/337: stat d7/d9/f6f 0 2024-04-20T11:09:31.622 INFO:tasks.workunit.client.1.smithi173.stdout:7/323: creat d9/d33/d35/d58/f70 x:0 0 0 2024-04-20T11:09:31.625 INFO:tasks.workunit.client.1.smithi173.stdout:9/331: rename d5/d9/d20/d29/d2d/l39 to d5/d9/d20/d2b/d31/l67 0 2024-04-20T11:09:31.626 INFO:tasks.workunit.client.1.smithi173.stdout:1/260: creat d5/d6/d17/d42/d12/d1c/f5b x:0 0 0 2024-04-20T11:09:31.628 INFO:tasks.workunit.client.1.smithi173.stdout:0/253: mknod d6/d8/c4d 0 2024-04-20T11:09:31.632 INFO:tasks.workunit.client.1.smithi173.stdout:4/291: symlink da/dc/d13/l59 0 2024-04-20T11:09:31.634 INFO:tasks.workunit.client.1.smithi173.stdout:4/292: chown da/dc/f33 211 1 2024-04-20T11:09:31.637 INFO:tasks.workunit.client.1.smithi173.stdout:9/332: creat d5/d9/d20/d4f/f68 x:0 0 0 2024-04-20T11:09:31.637 INFO:tasks.workunit.client.1.smithi173.stdout:1/261: fdatasync d5/d6/d17/d42/d12/f15 0 2024-04-20T11:09:31.640 INFO:tasks.workunit.client.1.smithi173.stdout:0/254: mknod d6/d8/c4e 0 2024-04-20T11:09:31.643 INFO:tasks.workunit.client.1.smithi173.stdout:6/315: truncate d4/f46 1118470 0 2024-04-20T11:09:31.644 INFO:tasks.workunit.client.1.smithi173.stdout:3/338: symlink d7/d9/l70 0 2024-04-20T11:09:31.646 INFO:tasks.workunit.client.1.smithi173.stdout:6/316: write d4/d1c/d28/d3b/f55 [1741954,35176] 0 2024-04-20T11:09:31.649 INFO:tasks.workunit.client.1.smithi173.stdout:1/262: mknod d5/d6/d17/d42/c5c 0 2024-04-20T11:09:31.650 INFO:tasks.workunit.client.1.smithi173.stdout:4/293: dwrite da/dc/d13/f47 [0,4194304] 0 2024-04-20T11:09:31.651 INFO:tasks.workunit.client.1.smithi173.stdout:4/294: stat da/dc/d15/f35 0 2024-04-20T11:09:31.653 INFO:tasks.workunit.client.1.smithi173.stdout:4/295: write da/dc/d15/f35 [726937,38603] 0 2024-04-20T11:09:31.653 INFO:tasks.workunit.client.1.smithi173.stdout:0/255: creat d6/d24/f4f x:0 0 0 2024-04-20T11:09:31.659 INFO:tasks.workunit.client.1.smithi173.stdout:3/339: creat d7/d53/f71 x:0 0 0 2024-04-20T11:09:31.659 INFO:tasks.workunit.client.1.smithi173.stdout:0/256: dread d6/d8/df/f16 [0,4194304] 0 2024-04-20T11:09:31.660 INFO:tasks.workunit.client.1.smithi173.stdout:6/317: truncate d4/f20 446525 0 2024-04-20T11:09:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:31 smithi173 ceph-mon[23800]: Reconfiguring daemon prometheus.smithi117 on smithi117 2024-04-20T11:09:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:31 smithi173 ceph-mon[23800]: pgmap v25: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 42 MiB/s rd, 113 MiB/s wr, 275 op/s 2024-04-20T11:09:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:09:31.662 INFO:tasks.workunit.client.1.smithi173.stdout:2/325: sync 2024-04-20T11:09:31.663 INFO:tasks.workunit.client.1.smithi173.stdout:8/321: sync 2024-04-20T11:09:31.664 INFO:tasks.workunit.client.1.smithi173.stdout:2/326: chown d2/d13/d46 7522 1 2024-04-20T11:09:31.665 INFO:tasks.workunit.client.1.smithi173.stdout:1/263: rename f3 to d5/d6/d17/d42/da/df/d40/f5d 0 2024-04-20T11:09:31.672 INFO:tasks.workunit.client.1.smithi173.stdout:3/340: creat d7/d9/d5c/f72 x:0 0 0 2024-04-20T11:09:31.674 INFO:tasks.workunit.client.1.smithi173.stdout:3/341: dread - d7/d9/f2e zero size 2024-04-20T11:09:31.674 INFO:tasks.workunit.client.1.smithi173.stdout:0/257: mkdir d6/d8/df/d2c/d42/d50 0 2024-04-20T11:09:31.677 INFO:tasks.workunit.client.1.smithi173.stdout:9/333: link d5/d9/d20/d2b/l3e d5/d9/d20/d2b/d59/d44/l69 0 2024-04-20T11:09:31.680 INFO:tasks.workunit.client.1.smithi173.stdout:1/264: symlink d5/d6/d17/d42/da/l5e 0 2024-04-20T11:09:31.682 INFO:tasks.workunit.client.1.smithi173.stdout:6/318: creat d4/d59/f5e x:0 0 0 2024-04-20T11:09:31.683 INFO:tasks.workunit.client.1.smithi173.stdout:1/265: fdatasync d5/d6/d17/d42/f39 0 2024-04-20T11:09:31.683 INFO:tasks.workunit.client.1.smithi173.stdout:3/342: mknod d7/d9/d3b/d45/c73 0 2024-04-20T11:09:31.684 INFO:tasks.workunit.client.1.smithi173.stdout:8/322: mknod d4/c5e 0 2024-04-20T11:09:31.689 INFO:tasks.workunit.client.1.smithi173.stdout:3/343: write d7/d9/f5a [4772571,24584] 0 2024-04-20T11:09:31.698 INFO:tasks.workunit.client.1.smithi173.stdout:0/258: dwrite d6/d24/f4f [0,4194304] 0 2024-04-20T11:09:31.699 INFO:tasks.workunit.client.1.smithi173.stdout:9/334: link d5/f6 d5/d9/d20/d2b/d59/d44/f6a 0 2024-04-20T11:09:31.700 INFO:tasks.workunit.client.1.smithi173.stdout:1/266: mknod d5/d6/d3b/c5f 0 2024-04-20T11:09:31.708 INFO:tasks.workunit.client.1.smithi173.stdout:8/323: mkdir d4/d18/d3a/d3c/d5f 0 2024-04-20T11:09:31.709 INFO:tasks.workunit.client.1.smithi173.stdout:1/267: read d5/d6/d17/d42/d12/d1f/f2f [1339283,101407] 0 2024-04-20T11:09:31.709 INFO:tasks.workunit.client.1.smithi173.stdout:7/324: dwrite d9/d33/d35/d58/f5f [0,4194304] 0 2024-04-20T11:09:31.712 INFO:tasks.workunit.client.1.smithi173.stdout:8/324: fsync d4/d18/d3a/d3c/f53 0 2024-04-20T11:09:31.714 INFO:tasks.workunit.client.1.smithi173.stdout:5/336: sync 2024-04-20T11:09:31.720 INFO:tasks.workunit.client.1.smithi173.stdout:3/344: link d7/d9/f66 d7/d2b/f74 0 2024-04-20T11:09:31.722 INFO:tasks.workunit.client.1.smithi173.stdout:3/345: chown d7/d17/f2c 123042953 1 2024-04-20T11:09:31.723 INFO:tasks.workunit.client.1.smithi173.stdout:1/268: rmdir d5/d6/dc 39 2024-04-20T11:09:31.726 INFO:tasks.workunit.client.1.smithi173.stdout:0/259: dread d6/f29 [0,4194304] 0 2024-04-20T11:09:31.727 INFO:tasks.workunit.client.1.smithi173.stdout:3/346: chown d7/d9/d3b/d45/f63 0 1 2024-04-20T11:09:31.732 INFO:tasks.workunit.client.1.smithi173.stdout:9/335: creat d5/d9/d20/d29/d2d/d33/f6b x:0 0 0 2024-04-20T11:09:31.734 INFO:tasks.workunit.client.1.smithi173.stdout:5/337: mknod da/c73 0 2024-04-20T11:09:31.745 INFO:tasks.workunit.client.1.smithi173.stdout:0/260: unlink d6/d8/f2f 0 2024-04-20T11:09:31.756 INFO:tasks.workunit.client.1.smithi173.stdout:5/338: rename da/de/d19/d32/f3c to da/d4f/d65/f74 0 2024-04-20T11:09:31.761 INFO:tasks.workunit.client.1.smithi173.stdout:0/261: mkdir d6/d8/d51 0 2024-04-20T11:09:31.762 INFO:tasks.workunit.client.1.smithi173.stdout:1/269: link d5/d6/d17/d42/da/l51 d5/d6/d17/d42/l60 0 2024-04-20T11:09:31.764 INFO:tasks.workunit.client.1.smithi173.stdout:4/296: dwrite da/dc/d13/f51 [0,4194304] 0 2024-04-20T11:09:31.770 INFO:tasks.workunit.client.1.smithi173.stdout:3/347: dwrite d7/d17/f26 [0,4194304] 0 2024-04-20T11:09:31.779 INFO:tasks.workunit.client.1.smithi173.stdout:5/339: fsync f9 0 2024-04-20T11:09:31.786 INFO:tasks.workunit.client.1.smithi173.stdout:8/325: write d4/ff [1041218,59547] 0 2024-04-20T11:09:31.792 INFO:tasks.workunit.client.1.smithi173.stdout:2/327: dwrite d2/d34/d58/f69 [0,4194304] 0 2024-04-20T11:09:31.792 INFO:tasks.workunit.client.1.smithi173.stdout:1/270: read d5/d6/d17/d42/d12/d1f/f2f [3366145,38992] 0 2024-04-20T11:09:31.792 INFO:tasks.workunit.client.1.smithi173.stdout:4/297: symlink da/d1e/d52/l5a 0 2024-04-20T11:09:31.796 INFO:tasks.workunit.client.1.smithi173.stdout:3/348: unlink d7/d9/f3a 0 2024-04-20T11:09:31.801 INFO:tasks.workunit.client.1.smithi173.stdout:8/326: stat d4/d18/d59/d5a/f5d 0 2024-04-20T11:09:31.802 INFO:tasks.workunit.client.1.smithi173.stdout:9/336: dread d5/d9/d20/d29/d2d/f43 [0,4194304] 0 2024-04-20T11:09:31.805 INFO:tasks.workunit.client.1.smithi173.stdout:0/262: getdents d6/d8/df/d2c/d42 0 2024-04-20T11:09:31.807 INFO:tasks.workunit.client.1.smithi173.stdout:2/328: creat d2/d13/f6d x:0 0 0 2024-04-20T11:09:31.810 INFO:tasks.workunit.client.1.smithi173.stdout:3/349: chown d7/d9/f5a 387116 1 2024-04-20T11:09:31.811 INFO:tasks.workunit.client.1.smithi173.stdout:9/337: fdatasync d5/d9/d20/d29/f11 0 2024-04-20T11:09:31.811 INFO:tasks.workunit.client.1.smithi173.stdout:6/319: dwrite d4/f46 [0,4194304] 0 2024-04-20T11:09:31.818 INFO:tasks.workunit.client.1.smithi173.stdout:5/340: rename da/d12 to da/de/d75 0 2024-04-20T11:09:31.820 INFO:tasks.workunit.client.1.smithi173.stdout:3/350: read d7/d9/f5a [1666002,47086] 0 2024-04-20T11:09:31.822 INFO:tasks.workunit.client.1.smithi173.stdout:3/351: write d7/d9/f4f [2488976,116502] 0 2024-04-20T11:09:31.825 INFO:tasks.workunit.client.1.smithi173.stdout:2/329: symlink d2/de/l6e 0 2024-04-20T11:09:31.825 INFO:tasks.workunit.client.1.smithi173.stdout:7/325: sync 2024-04-20T11:09:31.827 INFO:tasks.workunit.client.1.smithi173.stdout:9/338: rename d5/d9/d20/d2b/d31/f5b to d5/d9/d20/d2b/d31/f6c 0 2024-04-20T11:09:31.828 INFO:tasks.workunit.client.1.smithi173.stdout:6/320: truncate d4/f13 2943703 0 2024-04-20T11:09:31.829 INFO:tasks.workunit.client.1.smithi173.stdout:4/298: rename da/d48/d53/f55 to da/d48/d53/f5b 0 2024-04-20T11:09:31.830 INFO:tasks.workunit.client.1.smithi173.stdout:3/352: fdatasync d7/d9/f4f 0 2024-04-20T11:09:31.831 INFO:tasks.workunit.client.1.smithi173.stdout:5/341: creat da/de/d19/f76 x:0 0 0 2024-04-20T11:09:31.833 INFO:tasks.workunit.client.1.smithi173.stdout:3/353: readlink d7/d9/d3b/l6d 0 2024-04-20T11:09:31.833 INFO:tasks.workunit.client.1.smithi173.stdout:8/327: creat d4/d18/f60 x:0 0 0 2024-04-20T11:09:31.838 INFO:tasks.workunit.client.1.smithi173.stdout:8/328: write d4/d18/f21 [4737709,74169] 0 2024-04-20T11:09:31.844 INFO:tasks.workunit.client.1.smithi173.stdout:5/342: symlink da/d4f/l77 0 2024-04-20T11:09:31.846 INFO:tasks.workunit.client.1.smithi173.stdout:6/321: stat d4/d24/d42/l50 0 2024-04-20T11:09:31.846 INFO:tasks.workunit.client.1.smithi173.stdout:3/354: rename d7/d9/d3b/d45/l4a to d7/d9/d3b/l75 0 2024-04-20T11:09:31.847 INFO:tasks.workunit.client.1.smithi173.stdout:0/263: getdents d6/d8/df/d2c/d42 0 2024-04-20T11:09:31.850 INFO:tasks.workunit.client.1.smithi173.stdout:8/329: unlink d4/d18/d4f/c51 0 2024-04-20T11:09:31.852 INFO:tasks.workunit.client.1.smithi173.stdout:7/326: rmdir d9/da/d17/d2b/d6f 0 2024-04-20T11:09:31.853 INFO:tasks.workunit.client.1.smithi173.stdout:3/355: read d7/f25 [2714643,17549] 0 2024-04-20T11:09:31.857 INFO:tasks.workunit.client.1.smithi173.stdout:1/271: dwrite d5/d6/d17/d42/f41 [0,4194304] 0 2024-04-20T11:09:31.858 INFO:tasks.workunit.client.1.smithi173.stdout:7/327: fsync d9/d33/f6b 0 2024-04-20T11:09:31.861 INFO:tasks.workunit.client.1.smithi173.stdout:9/339: dwrite d5/f4b [0,4194304] 0 2024-04-20T11:09:31.862 INFO:tasks.workunit.client.1.smithi173.stdout:6/322: mkdir d4/d14/d5f 0 2024-04-20T11:09:31.863 INFO:tasks.workunit.client.1.smithi173.stdout:8/330: read d4/d18/d3a/d3c/f37 [3987045,50472] 0 2024-04-20T11:09:31.866 INFO:tasks.workunit.client.1.smithi173.stdout:0/264: creat d6/d8/df/d2c/d2d/f52 x:0 0 0 2024-04-20T11:09:31.866 INFO:tasks.workunit.client.1.smithi173.stdout:7/328: write d9/d33/d35/d4a/f4c [3381717,88416] 0 2024-04-20T11:09:31.867 INFO:tasks.workunit.client.1.smithi173.stdout:1/272: fsync f0 0 2024-04-20T11:09:31.878 INFO:tasks.workunit.client.1.smithi173.stdout:0/265: creat d6/d24/f53 x:0 0 0 2024-04-20T11:09:31.879 INFO:tasks.workunit.client.1.smithi173.stdout:2/330: dwrite d2/de/d11/f1d [0,4194304] 0 2024-04-20T11:09:31.884 INFO:tasks.workunit.client.1.smithi173.stdout:0/266: write d6/f13 [685421,51916] 0 2024-04-20T11:09:31.885 INFO:tasks.workunit.client.1.smithi173.stdout:8/331: link d4/d18/d3a/d3c/f40 d4/d18/d3a/d3c/d5f/f61 0 2024-04-20T11:09:31.893 INFO:tasks.workunit.client.1.smithi173.stdout:4/299: dread da/d1a/f40 [0,4194304] 0 2024-04-20T11:09:31.893 INFO:tasks.workunit.client.1.smithi173.stdout:7/329: getdents d9/da/d2e 0 2024-04-20T11:09:31.897 INFO:tasks.workunit.client.1.smithi173.stdout:3/356: dwrite d7/d9/f4f [0,4194304] 0 2024-04-20T11:09:31.899 INFO:tasks.workunit.client.1.smithi173.stdout:4/300: chown da/d1e 1 1 2024-04-20T11:09:31.903 INFO:tasks.workunit.client.1.smithi173.stdout:4/301: stat da/dc/ld 0 2024-04-20T11:09:31.905 INFO:tasks.workunit.client.1.smithi173.stdout:1/273: write d5/d6/d3b/f49 [3078479,99328] 0 2024-04-20T11:09:31.909 INFO:tasks.workunit.client.1.smithi173.stdout:8/332: getdents d4/d18/d4f 0 2024-04-20T11:09:31.913 INFO:tasks.workunit.client.1.smithi173.stdout:6/323: dwrite d4/d18/f58 [0,4194304] 0 2024-04-20T11:09:31.914 INFO:tasks.workunit.client.1.smithi173.stdout:9/340: dwrite d5/d9/d20/d29/f64 [0,4194304] 0 2024-04-20T11:09:31.917 INFO:tasks.workunit.client.1.smithi173.stdout:7/330: rename d9/d33/d35/d54/f63 to d9/da/d17/d2b/f71 0 2024-04-20T11:09:31.920 INFO:tasks.workunit.client.1.smithi173.stdout:4/302: getdents da/d48/d53/d58 0 2024-04-20T11:09:31.925 INFO:tasks.workunit.client.1.smithi173.stdout:0/267: dread d6/d8/df/d43/f4c [0,4194304] 0 2024-04-20T11:09:31.935 INFO:tasks.workunit.client.1.smithi173.stdout:0/268: mkdir d6/d54 0 2024-04-20T11:09:31.945 INFO:tasks.workunit.client.1.smithi173.stdout:9/341: dread d5/d9/d20/d29/f11 [4194304,4194304] 0 2024-04-20T11:09:31.945 INFO:tasks.workunit.client.1.smithi173.stdout:0/269: unlink d6/d8/df/d15/f22 0 2024-04-20T11:09:31.947 INFO:tasks.workunit.client.1.smithi173.stdout:7/331: dread d9/da/d17/f1a [0,4194304] 0 2024-04-20T11:09:31.950 INFO:tasks.workunit.client.1.smithi173.stdout:8/333: truncate d4/d18/f21 2420420 0 2024-04-20T11:09:31.959 INFO:tasks.workunit.client.1.smithi173.stdout:6/324: dwrite d4/d1c/d28/d3b/f41 [0,4194304] 0 2024-04-20T11:09:31.960 INFO:tasks.workunit.client.1.smithi173.stdout:0/270: mknod d6/d8/df/d43/c55 0 2024-04-20T11:09:31.962 INFO:tasks.workunit.client.1.smithi173.stdout:1/274: dwrite d5/d6/d17/d42/d12/f15 [0,4194304] 0 2024-04-20T11:09:31.963 INFO:tasks.workunit.client.1.smithi173.stdout:2/331: dread d2/de/f2b [0,4194304] 0 2024-04-20T11:09:31.967 INFO:tasks.workunit.client.1.smithi173.stdout:4/303: dwrite da/dc/d15/f2a [8388608,4194304] 0 2024-04-20T11:09:31.970 INFO:tasks.workunit.client.1.smithi173.stdout:2/332: write d2/d34/d58/f69 [3631932,38604] 0 2024-04-20T11:09:31.970 INFO:tasks.workunit.client.1.smithi173.stdout:8/334: symlink d4/d18/l62 0 2024-04-20T11:09:31.973 INFO:tasks.workunit.client.1.smithi173.stdout:1/275: unlink d5/d6/d3b/f49 0 2024-04-20T11:09:31.977 INFO:tasks.workunit.client.1.smithi173.stdout:4/304: chown da/l19 31319 1 2024-04-20T11:09:31.979 INFO:tasks.workunit.client.1.smithi173.stdout:4/305: chown da/dc/d13/c14 9986539 1 2024-04-20T11:09:31.987 INFO:tasks.workunit.client.1.smithi173.stdout:0/271: truncate d6/d8/f12 1211506 0 2024-04-20T11:09:31.987 INFO:tasks.workunit.client.1.smithi173.stdout:4/306: chown da/dc/d13/c21 8483 1 2024-04-20T11:09:31.989 INFO:tasks.workunit.client.1.smithi173.stdout:9/342: dwrite d5/d9/d20/d29/d2d/f42 [0,4194304] 0 2024-04-20T11:09:31.991 INFO:tasks.workunit.client.1.smithi173.stdout:0/272: chown d6/d8/d51 194086 1 2024-04-20T11:09:31.991 INFO:tasks.workunit.client.1.smithi173.stdout:8/335: symlink d4/d18/d3a/l63 0 2024-04-20T11:09:31.993 INFO:tasks.workunit.client.1.smithi173.stdout:1/276: creat d5/d6/d17/d42/da/d20/d48/f61 x:0 0 0 2024-04-20T11:09:31.996 INFO:tasks.workunit.client.1.smithi173.stdout:2/333: link d2/d13/c37 d2/d13/d28/d4a/c6f 0 2024-04-20T11:09:31.999 INFO:tasks.workunit.client.1.smithi173.stdout:4/307: creat da/d1e/d52/f5c x:0 0 0 2024-04-20T11:09:32.001 INFO:tasks.workunit.client.1.smithi173.stdout:5/343: sync 2024-04-20T11:09:32.003 INFO:tasks.workunit.client.1.smithi173.stdout:2/334: fsync d2/de/f5e 0 2024-04-20T11:09:32.005 INFO:tasks.workunit.client.1.smithi173.stdout:7/332: dwrite d9/d33/d35/d4a/d52/d5b/f6d [0,4194304] 0 2024-04-20T11:09:32.012 INFO:tasks.workunit.client.1.smithi173.stdout:0/273: creat d6/d8/df/d2c/d42/f56 x:0 0 0 2024-04-20T11:09:32.013 INFO:tasks.workunit.client.1.smithi173.stdout:6/325: dwrite d4/d24/d3e/f4d [0,4194304] 0 2024-04-20T11:09:32.014 INFO:tasks.workunit.client.1.smithi173.stdout:7/333: chown d9/d33/d35/d56 863 1 2024-04-20T11:09:32.014 INFO:tasks.workunit.client.1.smithi173.stdout:8/336: dread d4/f8 [0,4194304] 0 2024-04-20T11:09:32.019 INFO:tasks.workunit.client.1.smithi173.stdout:4/308: symlink da/dc/d13/l5d 0 2024-04-20T11:09:32.019 INFO:tasks.workunit.client.1.smithi173.stdout:1/277: mknod d5/d6/dc/c62 0 2024-04-20T11:09:32.022 INFO:tasks.workunit.client.1.smithi173.stdout:5/344: mknod da/de/d19/d2c/c78 0 2024-04-20T11:09:32.023 INFO:tasks.workunit.client.1.smithi173.stdout:8/337: creat d4/d18/d59/f64 x:0 0 0 2024-04-20T11:09:32.023 INFO:tasks.workunit.client.1.smithi173.stdout:6/326: fsync d4/d1c/d28/d3b/f55 0 2024-04-20T11:09:32.025 INFO:tasks.workunit.client.1.smithi173.stdout:7/334: mkdir d9/d1f/d72 0 2024-04-20T11:09:32.026 INFO:tasks.workunit.client.1.smithi173.stdout:2/335: rename d2/de/d11/l6 to d2/d13/d46/d53/l70 0 2024-04-20T11:09:32.029 INFO:tasks.workunit.client.1.smithi173.stdout:5/345: stat da/de/f5b 0 2024-04-20T11:09:32.031 INFO:tasks.workunit.client.1.smithi173.stdout:8/338: symlink d4/d18/d4f/l65 0 2024-04-20T11:09:32.032 INFO:tasks.workunit.client.1.smithi173.stdout:0/274: symlink d6/d8/df/d2c/d42/d50/l57 0 2024-04-20T11:09:32.032 INFO:tasks.workunit.client.1.smithi173.stdout:6/327: read d4/d24/d3e/f4c [2376727,119114] 0 2024-04-20T11:09:32.034 INFO:tasks.workunit.client.1.smithi173.stdout:7/335: mkdir d9/da/d17/d2b/d73 0 2024-04-20T11:09:32.034 INFO:tasks.workunit.client.1.smithi173.stdout:6/328: fdatasync d4/d1c/d28/d3b/f52 0 2024-04-20T11:09:32.035 INFO:tasks.workunit.client.1.smithi173.stdout:0/275: write d6/d8/df/f16 [591152,13953] 0 2024-04-20T11:09:32.035 INFO:tasks.workunit.client.1.smithi173.stdout:2/336: rename d2/d13/l61 to d2/d13/d46/d53/d4e/l71 0 2024-04-20T11:09:32.037 INFO:tasks.workunit.client.1.smithi173.stdout:6/329: dread - d4/d24/f54 zero size 2024-04-20T11:09:32.038 INFO:tasks.workunit.client.1.smithi173.stdout:5/346: write da/f29 [1483473,36345] 0 2024-04-20T11:09:32.042 INFO:tasks.workunit.client.1.smithi173.stdout:2/337: mknod d2/de/c72 0 2024-04-20T11:09:32.044 INFO:tasks.workunit.client.1.smithi173.stdout:6/330: fsync d4/d1c/d28/d3b/f52 0 2024-04-20T11:09:32.045 INFO:tasks.workunit.client.1.smithi173.stdout:2/338: dread - d2/d34/f56 zero size 2024-04-20T11:09:32.049 INFO:tasks.workunit.client.1.smithi173.stdout:5/347: creat da/de/d19/d2c/f79 x:0 0 0 2024-04-20T11:09:32.050 INFO:tasks.workunit.client.1.smithi173.stdout:0/276: rename d6/d8/l30 to d6/d8/df/d2c/d2d/l58 0 2024-04-20T11:09:32.054 INFO:tasks.workunit.client.1.smithi173.stdout:5/348: readlink da/de/d19/l49 0 2024-04-20T11:09:32.055 INFO:tasks.workunit.client.1.smithi173.stdout:1/278: dwrite d5/d6/d17/d42/d12/f44 [0,4194304] 0 2024-04-20T11:09:32.056 INFO:tasks.workunit.client.1.smithi173.stdout:2/339: rename d2/de/d11/f1d to d2/d13/d46/f73 0 2024-04-20T11:09:32.074 INFO:tasks.workunit.client.1.smithi173.stdout:8/339: dwrite f2 [0,4194304] 0 2024-04-20T11:09:32.074 INFO:tasks.workunit.client.1.smithi173.stdout:7/336: dwrite d9/f30 [4194304,4194304] 0 2024-04-20T11:09:32.076 INFO:tasks.workunit.client.1.smithi173.stdout:5/349: symlink da/de/d75/d40/d59/l7a 0 2024-04-20T11:09:32.077 INFO:tasks.workunit.client.1.smithi173.stdout:4/309: dwrite da/d48/d53/f37 [0,4194304] 0 2024-04-20T11:09:32.081 INFO:tasks.workunit.client.1.smithi173.stdout:1/279: creat d5/d6/d17/d42/d12/d1f/f63 x:0 0 0 2024-04-20T11:09:32.087 INFO:tasks.workunit.client.1.smithi173.stdout:5/350: creat da/de/d75/f7b x:0 0 0 2024-04-20T11:09:32.087 INFO:tasks.workunit.client.1.smithi173.stdout:6/331: dwrite d4/d14/f3c [0,4194304] 0 2024-04-20T11:09:32.093 INFO:tasks.workunit.client.1.smithi173.stdout:8/340: creat d4/d18/d3a/f66 x:0 0 0 2024-04-20T11:09:32.095 INFO:tasks.workunit.client.1.smithi173.stdout:0/277: chown d6/d8/df/d2c/d2d/l58 99446689 1 2024-04-20T11:09:32.099 INFO:tasks.workunit.client.1.smithi173.stdout:5/351: symlink da/de/d19/d32/l7c 0 2024-04-20T11:09:32.099 INFO:tasks.workunit.client.1.smithi173.stdout:3/357: sync 2024-04-20T11:09:32.113 INFO:tasks.workunit.client.1.smithi173.stdout:1/280: mknod d5/d6/d17/d42/da/df/c64 0 2024-04-20T11:09:32.114 INFO:tasks.workunit.client.1.smithi173.stdout:7/337: getdents d9 0 2024-04-20T11:09:32.117 INFO:tasks.workunit.client.1.smithi173.stdout:2/340: dread d2/de/f3b [0,4194304] 0 2024-04-20T11:09:32.118 INFO:tasks.workunit.client.1.smithi173.stdout:0/278: creat d6/d8/df/d43/f59 x:0 0 0 2024-04-20T11:09:32.120 INFO:tasks.workunit.client.1.smithi173.stdout:6/332: dwrite d4/f37 [0,4194304] 0 2024-04-20T11:09:32.125 INFO:tasks.workunit.client.1.smithi173.stdout:4/310: truncate f8 2478921 0 2024-04-20T11:09:32.132 INFO:tasks.workunit.client.1.smithi173.stdout:0/279: dwrite d6/d8/df/d2c/d42/f56 [0,4194304] 0 2024-04-20T11:09:32.170 INFO:tasks.workunit.client.1.smithi173.stdout:1/281: symlink d5/d6/d17/d42/da/df/l65 0 2024-04-20T11:09:32.170 INFO:tasks.workunit.client.1.smithi173.stdout:7/338: creat d9/da/d17/d2b/f74 x:0 0 0 2024-04-20T11:09:32.172 INFO:tasks.workunit.client.1.smithi173.stdout:5/352: creat da/de/d19/d32/d66/f7d x:0 0 0 2024-04-20T11:09:32.177 INFO:tasks.workunit.client.1.smithi173.stdout:1/282: dread d5/f30 [0,4194304] 0 2024-04-20T11:09:32.178 INFO:tasks.workunit.client.1.smithi173.stdout:9/343: sync 2024-04-20T11:09:32.179 INFO:tasks.workunit.client.1.smithi173.stdout:4/311: mkdir da/d1e/d52/d5e 0 2024-04-20T11:09:32.186 INFO:tasks.workunit.client.1.smithi173.stdout:3/358: getdents d7/d9 0 2024-04-20T11:09:32.187 INFO:tasks.workunit.client.1.smithi173.stdout:9/344: fdatasync d5/d9/d20/d2b/d31/f4d 0 2024-04-20T11:09:32.189 INFO:tasks.workunit.client.1.smithi173.stdout:4/312: truncate da/d48/d53/f5b 547748 0 2024-04-20T11:09:32.189 INFO:tasks.workunit.client.1.smithi173.stdout:6/333: link d4/f37 d4/f60 0 2024-04-20T11:09:32.190 INFO:tasks.workunit.client.1.smithi173.stdout:7/339: rename d9/d33/d40/f4b to d9/d1f/d72/f75 0 2024-04-20T11:09:32.195 INFO:tasks.workunit.client.1.smithi173.stdout:5/353: dwrite da/fc [0,4194304] 0 2024-04-20T11:09:32.197 INFO:tasks.workunit.client.1.smithi173.stdout:5/354: readlink da/de/d75/l63 0 2024-04-20T11:09:32.198 INFO:tasks.workunit.client.1.smithi173.stdout:6/334: mkdir d4/d18/d61 0 2024-04-20T11:09:32.199 INFO:tasks.workunit.client.1.smithi173.stdout:0/280: link d6/d8/df/l21 d6/d8/df/l5a 0 2024-04-20T11:09:32.203 INFO:tasks.workunit.client.1.smithi173.stdout:7/340: unlink d9/d1f/l53 0 2024-04-20T11:09:32.203 INFO:tasks.workunit.client.1.smithi173.stdout:2/341: write d2/de/d11/f23 [2375756,104030] 0 2024-04-20T11:09:32.204 INFO:tasks.workunit.client.1.smithi173.stdout:8/341: truncate d4/f32 1346860 0 2024-04-20T11:09:32.206 INFO:tasks.workunit.client.1.smithi173.stdout:7/341: fsync d9/da/f14 0 2024-04-20T11:09:32.213 INFO:tasks.workunit.client.1.smithi173.stdout:4/313: link da/dc/d15/f4e da/dc/d15/f5f 0 2024-04-20T11:09:32.215 INFO:tasks.workunit.client.1.smithi173.stdout:9/345: creat d5/f6d x:0 0 0 2024-04-20T11:09:32.217 INFO:tasks.workunit.client.1.smithi173.stdout:9/346: write d5/d4c/f52 [1456771,121449] 0 2024-04-20T11:09:32.219 INFO:tasks.workunit.client.1.smithi173.stdout:3/359: write d7/d9/ff [727828,71361] 0 2024-04-20T11:09:32.219 INFO:tasks.workunit.client.1.smithi173.stdout:2/342: fdatasync d2/d13/d46/f73 0 2024-04-20T11:09:32.221 INFO:tasks.workunit.client.1.smithi173.stdout:8/342: creat d4/d18/d59/d5a/f67 x:0 0 0 2024-04-20T11:09:32.226 INFO:tasks.workunit.client.1.smithi173.stdout:0/281: link d6/d8/df/d15/f2e d6/d8/d51/f5b 0 2024-04-20T11:09:32.231 INFO:tasks.workunit.client.1.smithi173.stdout:8/343: write d4/d18/d3a/d3c/f54 [49988,80844] 0 2024-04-20T11:09:32.232 INFO:tasks.workunit.client.1.smithi173.stdout:7/342: dwrite d9/f30 [0,4194304] 0 2024-04-20T11:09:32.236 INFO:tasks.workunit.client.1.smithi173.stdout:4/314: dread da/dc/d13/f29 [0,4194304] 0 2024-04-20T11:09:32.239 INFO:tasks.workunit.client.1.smithi173.stdout:4/315: mkdir da/d1a/d60 0 2024-04-20T11:09:32.239 INFO:tasks.workunit.client.1.smithi173.stdout:6/335: write d4/f20 [521062,28362] 0 2024-04-20T11:09:32.241 INFO:tasks.workunit.client.1.smithi173.stdout:7/343: symlink d9/l76 0 2024-04-20T11:09:32.243 INFO:tasks.workunit.client.1.smithi173.stdout:4/316: write da/d48/d53/f37 [1225294,66198] 0 2024-04-20T11:09:32.243 INFO:tasks.workunit.client.1.smithi173.stdout:5/355: dread da/f33 [0,4194304] 0 2024-04-20T11:09:32.246 INFO:tasks.workunit.client.1.smithi173.stdout:5/356: chown da/de/d19/c20 16261222 1 2024-04-20T11:09:32.256 INFO:tasks.workunit.client.1.smithi173.stdout:9/347: dread d5/d9/f1b [0,4194304] 0 2024-04-20T11:09:32.261 INFO:tasks.workunit.client.1.smithi173.stdout:8/344: link d4/l11 d4/d18/d59/d5a/l68 0 2024-04-20T11:09:32.262 INFO:tasks.workunit.client.1.smithi173.stdout:0/282: dwrite d6/d8/df/d2c/d2d/f52 [0,4194304] 0 2024-04-20T11:09:32.262 INFO:tasks.workunit.client.1.smithi173.stdout:2/343: dwrite d2/d13/d46/d53/d4e/f68 [0,4194304] 0 2024-04-20T11:09:32.266 INFO:tasks.workunit.client.1.smithi173.stdout:2/344: chown d2/d34/d58/f69 5746 1 2024-04-20T11:09:32.266 INFO:tasks.workunit.client.1.smithi173.stdout:0/283: stat d6/d8/c34 0 2024-04-20T11:09:32.268 INFO:tasks.workunit.client.1.smithi173.stdout:2/345: chown d2/de/c2e 452 1 2024-04-20T11:09:32.272 INFO:tasks.workunit.client.1.smithi173.stdout:6/336: write d4/f5a [2938749,46052] 0 2024-04-20T11:09:32.274 INFO:tasks.workunit.client.1.smithi173.stdout:3/360: dread d7/d9/ff [0,4194304] 0 2024-04-20T11:09:32.276 INFO:tasks.workunit.client.1.smithi173.stdout:9/348: unlink d5/d9/d20/d2b/d31/f6c 0 2024-04-20T11:09:32.283 INFO:tasks.workunit.client.1.smithi173.stdout:5/357: creat da/de/f7e x:0 0 0 2024-04-20T11:09:32.283 INFO:tasks.workunit.client.1.smithi173.stdout:6/337: creat d4/d18/f62 x:0 0 0 2024-04-20T11:09:32.285 INFO:tasks.workunit.client.1.smithi173.stdout:7/344: symlink d9/d33/d35/d58/l77 0 2024-04-20T11:09:32.285 INFO:tasks.workunit.client.1.smithi173.stdout:4/317: link da/l34 da/d1a/l61 0 2024-04-20T11:09:32.287 INFO:tasks.workunit.client.1.smithi173.stdout:0/284: symlink d6/l5c 0 2024-04-20T11:09:32.290 INFO:tasks.workunit.client.1.smithi173.stdout:3/361: unlink d7/d2b/l52 0 2024-04-20T11:09:32.290 INFO:tasks.workunit.client.1.smithi173.stdout:4/318: read f1 [285797,12385] 0 2024-04-20T11:09:32.292 INFO:tasks.workunit.client.1.smithi173.stdout:8/345: rename d4/d18/l4e to d4/d18/l69 0 2024-04-20T11:09:32.292 INFO:tasks.workunit.client.1.smithi173.stdout:1/283: sync 2024-04-20T11:09:32.292 INFO:tasks.workunit.client.1.smithi173.stdout:3/362: write d7/d9/f4b [3775755,65769] 0 2024-04-20T11:09:32.293 INFO:tasks.workunit.client.1.smithi173.stdout:8/346: read - d4/f41 zero size 2024-04-20T11:09:32.294 INFO:tasks.workunit.client.1.smithi173.stdout:1/284: write f0 [242167,97102] 0 2024-04-20T11:09:32.305 INFO:tasks.workunit.client.1.smithi173.stdout:9/349: creat d5/d9/f6e x:0 0 0 2024-04-20T11:09:32.305 INFO:tasks.workunit.client.1.smithi173.stdout:2/346: getdents d2/d13/d46/d53/d4e 0 2024-04-20T11:09:32.308 INFO:tasks.workunit.client.1.smithi173.stdout:6/338: rename d4/l1e to d4/l63 0 2024-04-20T11:09:32.310 INFO:tasks.workunit.client.1.smithi173.stdout:8/347: write d4/d18/d4f/f57 [889362,57878] 0 2024-04-20T11:09:32.310 INFO:tasks.workunit.client.1.smithi173.stdout:6/339: chown d4/d18/f58 600523500 1 2024-04-20T11:09:32.310 INFO:tasks.workunit.client.1.smithi173.stdout:3/363: mknod d7/d9/d3b/d45/c76 0 2024-04-20T11:09:32.311 INFO:tasks.workunit.client.1.smithi173.stdout:7/345: dread d9/d33/d35/d58/f5f [0,4194304] 0 2024-04-20T11:09:32.311 INFO:tasks.workunit.client.1.smithi173.stdout:3/364: write d7/d9/f4b [2536825,205] 0 2024-04-20T11:09:32.314 INFO:tasks.workunit.client.1.smithi173.stdout:1/285: mknod d5/d6/d17/d42/c66 0 2024-04-20T11:09:32.315 INFO:tasks.workunit.client.1.smithi173.stdout:9/350: symlink d5/d9/d20/d29/d18/l6f 0 2024-04-20T11:09:32.315 INFO:tasks.workunit.client.1.smithi173.stdout:1/286: chown d5/d6/d3b 54301 1 2024-04-20T11:09:32.317 INFO:tasks.workunit.client.1.smithi173.stdout:2/347: truncate d2/f5c 1351357 0 2024-04-20T11:09:32.319 INFO:tasks.workunit.client.1.smithi173.stdout:3/365: unlink d7/d9/c21 0 2024-04-20T11:09:32.320 INFO:tasks.workunit.client.1.smithi173.stdout:6/340: mknod d4/d14/d5f/c64 0 2024-04-20T11:09:32.320 INFO:tasks.workunit.client.1.smithi173.stdout:4/319: dwrite da/f25 [4194304,4194304] 0 2024-04-20T11:09:32.321 INFO:tasks.workunit.client.1.smithi173.stdout:1/287: creat d5/d6/d3b/f67 x:0 0 0 2024-04-20T11:09:32.322 INFO:tasks.workunit.client.1.smithi173.stdout:6/341: chown d4/d24/d3e 42431224 1 2024-04-20T11:09:32.323 INFO:tasks.workunit.client.1.smithi173.stdout:2/348: creat d2/d13/f74 x:0 0 0 2024-04-20T11:09:32.325 INFO:tasks.workunit.client.1.smithi173.stdout:1/288: chown d5/d6/d3b/f67 19175 1 2024-04-20T11:09:32.325 INFO:tasks.workunit.client.1.smithi173.stdout:8/348: creat d4/f6a x:0 0 0 2024-04-20T11:09:32.328 INFO:tasks.workunit.client.1.smithi173.stdout:4/320: mkdir da/dc/d62 0 2024-04-20T11:09:32.335 INFO:tasks.workunit.client.1.smithi173.stdout:9/351: dwrite d5/f4b [0,4194304] 0 2024-04-20T11:09:32.341 INFO:tasks.workunit.client.1.smithi173.stdout:4/321: chown da/d1e/c4d 1011633 1 2024-04-20T11:09:32.342 INFO:tasks.workunit.client.1.smithi173.stdout:1/289: write d5/d6/d17/d42/da/f14 [473105,54090] 0 2024-04-20T11:09:32.347 INFO:tasks.workunit.client.1.smithi173.stdout:7/346: dwrite d9/d1f/d72/f75 [0,4194304] 0 2024-04-20T11:09:32.348 INFO:tasks.workunit.client.1.smithi173.stdout:9/352: chown d5/d9/d20/d29/d2d/f43 2208428 1 2024-04-20T11:09:32.352 INFO:tasks.workunit.client.1.smithi173.stdout:2/349: dwrite d2/f64 [4194304,4194304] 0 2024-04-20T11:09:32.356 INFO:tasks.workunit.client.1.smithi173.stdout:9/353: chown d5/d9/d20/d2b/c5f 10251892 1 2024-04-20T11:09:32.356 INFO:tasks.workunit.client.1.smithi173.stdout:3/366: chown d7/d9/d3b 480328 1 2024-04-20T11:09:32.358 INFO:tasks.workunit.client.1.smithi173.stdout:1/290: symlink d5/d6/d17/d42/d12/d1f/l68 0 2024-04-20T11:09:32.359 INFO:tasks.workunit.client.1.smithi173.stdout:2/350: creat d2/d13/d46/d4b/f75 x:0 0 0 2024-04-20T11:09:32.361 INFO:tasks.workunit.client.1.smithi173.stdout:9/354: write d5/f2f [1066822,102382] 0 2024-04-20T11:09:32.361 INFO:tasks.workunit.client.1.smithi173.stdout:6/342: dread d4/f11 [0,4194304] 0 2024-04-20T11:09:32.363 INFO:tasks.workunit.client.1.smithi173.stdout:3/367: rmdir d7 39 2024-04-20T11:09:32.364 INFO:tasks.workunit.client.1.smithi173.stdout:2/351: dread - d2/f3f zero size 2024-04-20T11:09:32.368 INFO:tasks.workunit.client.1.smithi173.stdout:1/291: symlink d5/d6/d17/d42/l69 0 2024-04-20T11:09:32.372 INFO:tasks.workunit.client.1.smithi173.stdout:6/343: symlink d4/d24/l65 0 2024-04-20T11:09:32.375 INFO:tasks.workunit.client.1.smithi173.stdout:4/322: dread da/dc/d15/f1d [0,4194304] 0 2024-04-20T11:09:32.375 INFO:tasks.workunit.client.1.smithi173.stdout:9/355: creat d5/d9/d20/d29/d18/f70 x:0 0 0 2024-04-20T11:09:32.378 INFO:tasks.workunit.client.1.smithi173.stdout:6/344: creat d4/d18/f66 x:0 0 0 2024-04-20T11:09:32.379 INFO:tasks.workunit.client.1.smithi173.stdout:8/349: dwrite d4/f32 [0,4194304] 0 2024-04-20T11:09:32.380 INFO:tasks.workunit.client.1.smithi173.stdout:4/323: chown da/d1e/d52/l5a 47186 1 2024-04-20T11:09:32.383 INFO:tasks.workunit.client.1.smithi173.stdout:9/356: mknod d5/d4c/c71 0 2024-04-20T11:09:32.386 INFO:tasks.workunit.client.1.smithi173.stdout:7/347: dwrite d9/da/d2e/f37 [0,4194304] 0 2024-04-20T11:09:32.387 INFO:tasks.workunit.client.1.smithi173.stdout:1/292: link d5/d6/d17/d42/d12/d1f/f63 d5/d6/d17/d42/da/f6a 0 2024-04-20T11:09:32.390 INFO:tasks.workunit.client.1.smithi173.stdout:5/358: sync 2024-04-20T11:09:32.392 INFO:tasks.workunit.client.1.smithi173.stdout:9/357: creat d5/d4c/f72 x:0 0 0 2024-04-20T11:09:32.394 INFO:tasks.workunit.client.1.smithi173.stdout:6/345: rename d4/d1c/d28/d3b/f55 to d4/d24/d42/f67 0 2024-04-20T11:09:32.395 INFO:tasks.workunit.client.1.smithi173.stdout:9/358: truncate d5/d9/d20/d29/d18/f70 767623 0 2024-04-20T11:09:32.398 INFO:tasks.workunit.client.1.smithi173.stdout:1/293: fdatasync d5/f30 0 2024-04-20T11:09:32.398 INFO:tasks.workunit.client.1.smithi173.stdout:2/352: write d2/de/f2b [2840545,78351] 0 2024-04-20T11:09:32.399 INFO:tasks.workunit.client.1.smithi173.stdout:4/324: getdents da/d48/d53/d58 0 2024-04-20T11:09:32.402 INFO:tasks.workunit.client.1.smithi173.stdout:7/348: rmdir d9/da/d17 39 2024-04-20T11:09:32.405 INFO:tasks.workunit.client.1.smithi173.stdout:1/294: mknod d5/d6/dc/d4a/c6b 0 2024-04-20T11:09:32.407 INFO:tasks.workunit.client.1.smithi173.stdout:2/353: creat d2/d13/d46/d53/f76 x:0 0 0 2024-04-20T11:09:32.411 INFO:tasks.workunit.client.1.smithi173.stdout:3/368: dwrite d7/d2b/f74 [0,4194304] 0 2024-04-20T11:09:32.412 INFO:tasks.workunit.client.1.smithi173.stdout:4/325: creat da/d1a/d60/f63 x:0 0 0 2024-04-20T11:09:32.416 INFO:tasks.workunit.client.1.smithi173.stdout:9/359: rename d5/d9/l17 to d5/d9/d20/d2b/d59/l73 0 2024-04-20T11:09:32.417 INFO:tasks.workunit.client.1.smithi173.stdout:5/359: mkdir da/d34/d7f 0 2024-04-20T11:09:32.418 INFO:tasks.workunit.client.1.smithi173.stdout:5/360: readlink da/l28 0 2024-04-20T11:09:32.420 INFO:tasks.workunit.client.1.smithi173.stdout:1/295: mknod d5/d6/d3b/c6c 0 2024-04-20T11:09:32.421 INFO:tasks.workunit.client.1.smithi173.stdout:8/350: truncate d4/f28 4398080 0 2024-04-20T11:09:32.423 INFO:tasks.workunit.client.1.smithi173.stdout:2/354: mknod d2/d13/d28/d4a/c77 0 2024-04-20T11:09:32.425 INFO:tasks.workunit.client.1.smithi173.stdout:2/355: write d2/d34/d58/f6b [873564,53637] 0 2024-04-20T11:09:32.429 INFO:tasks.workunit.client.1.smithi173.stdout:4/326: dwrite da/dc/d15/f1d [0,4194304] 0 2024-04-20T11:09:32.431 INFO:tasks.workunit.client.1.smithi173.stdout:3/369: symlink d7/d53/l77 0 2024-04-20T11:09:32.432 INFO:tasks.workunit.client.1.smithi173.stdout:1/296: mknod d5/d6/d17/d42/da/df/c6d 0 2024-04-20T11:09:32.432 INFO:tasks.workunit.client.1.smithi173.stdout:9/360: chown d5/ce 115866120 1 2024-04-20T11:09:32.433 INFO:tasks.workunit.client.1.smithi173.stdout:6/346: link d4/d14/c1f d4/c68 0 2024-04-20T11:09:32.433 INFO:tasks.workunit.client.1.smithi173.stdout:2/356: symlink d2/d13/d46/d53/l78 0 2024-04-20T11:09:32.433 INFO:tasks.workunit.client.1.smithi173.stdout:3/370: chown d7/d2b/l69 63853 1 2024-04-20T11:09:32.434 INFO:tasks.workunit.client.1.smithi173.stdout:6/347: readlink d4/d24/l65 0 2024-04-20T11:09:32.434 INFO:tasks.workunit.client.1.smithi173.stdout:7/349: creat d9/da/d17/f78 x:0 0 0 2024-04-20T11:09:32.437 INFO:tasks.workunit.client.1.smithi173.stdout:1/297: dread - d5/d6/d3b/f67 zero size 2024-04-20T11:09:32.437 INFO:tasks.workunit.client.1.smithi173.stdout:3/371: read - d7/d9/d3b/d45/f56 zero size 2024-04-20T11:09:32.439 INFO:tasks.workunit.client.1.smithi173.stdout:6/348: chown d4/d59/f5e 1 1 2024-04-20T11:09:32.441 INFO:tasks.workunit.client.1.smithi173.stdout:2/357: mknod d2/d13/c79 0 2024-04-20T11:09:32.441 INFO:tasks.workunit.client.1.smithi173.stdout:1/298: truncate d5/d6/d3b/f67 847542 0 2024-04-20T11:09:32.443 INFO:tasks.workunit.client.1.smithi173.stdout:9/361: chown d5/d9/d20/d29/d2d/l2e 403748 1 2024-04-20T11:09:32.445 INFO:tasks.workunit.client.1.smithi173.stdout:9/362: chown d5/d9/d20/d2b/c5a 302621 1 2024-04-20T11:09:32.446 INFO:tasks.workunit.client.1.smithi173.stdout:8/351: creat d4/d18/f6b x:0 0 0 2024-04-20T11:09:32.447 INFO:tasks.workunit.client.1.smithi173.stdout:9/363: dread - d5/d9/f6e zero size 2024-04-20T11:09:32.448 INFO:tasks.workunit.client.1.smithi173.stdout:0/285: sync 2024-04-20T11:09:32.449 INFO:tasks.workunit.client.1.smithi173.stdout:7/350: truncate d9/d33/f47 584559 0 2024-04-20T11:09:32.450 INFO:tasks.workunit.client.1.smithi173.stdout:2/358: creat d2/de/f7a x:0 0 0 2024-04-20T11:09:32.452 INFO:tasks.workunit.client.1.smithi173.stdout:5/361: symlink da/d34/d7f/l80 0 2024-04-20T11:09:32.452 INFO:tasks.workunit.client.1.smithi173.stdout:7/351: write d9/d33/d40/f49 [131977,531] 0 2024-04-20T11:09:32.453 INFO:tasks.workunit.client.1.smithi173.stdout:0/286: write d6/d24/f4f [2631803,51760] 0 2024-04-20T11:09:32.454 INFO:tasks.workunit.client.1.smithi173.stdout:4/327: creat da/f64 x:0 0 0 2024-04-20T11:09:32.455 INFO:tasks.workunit.client.1.smithi173.stdout:3/372: creat d7/d17/d61/f78 x:0 0 0 2024-04-20T11:09:32.459 INFO:tasks.workunit.client.1.smithi173.stdout:0/287: readlink d6/d8/df/d15/l28 0 2024-04-20T11:09:32.461 INFO:tasks.workunit.client.1.smithi173.stdout:2/359: creat d2/d13/d28/f7b x:0 0 0 2024-04-20T11:09:32.463 INFO:tasks.workunit.client.1.smithi173.stdout:2/360: readlink d2/de/l22 0 2024-04-20T11:09:32.464 INFO:tasks.workunit.client.1.smithi173.stdout:6/349: dwrite d4/d14/f3c [0,4194304] 0 2024-04-20T11:09:32.464 INFO:tasks.workunit.client.1.smithi173.stdout:5/362: chown da/d34/d6c 225 1 2024-04-20T11:09:32.464 INFO:tasks.workunit.client.1.smithi173.stdout:9/364: symlink d5/d9/d20/d4f/d5e/l74 0 2024-04-20T11:09:32.465 INFO:tasks.workunit.client.1.smithi173.stdout:4/328: mkdir da/dc/d13/d65 0 2024-04-20T11:09:32.471 INFO:tasks.workunit.client.1.smithi173.stdout:0/288: rename d6/d8/c34 to d6/d8/df/d2c/d42/d50/c5d 0 2024-04-20T11:09:32.473 INFO:tasks.workunit.client.1.smithi173.stdout:5/363: mkdir da/de/d19/d81 0 2024-04-20T11:09:32.476 INFO:tasks.workunit.client.1.smithi173.stdout:9/365: chown d5/d9/d20/d2b/d59/l73 6 1 2024-04-20T11:09:32.477 INFO:tasks.workunit.client.1.smithi173.stdout:8/352: dwrite d4/d18/f5c [0,4194304] 0 2024-04-20T11:09:32.478 INFO:tasks.workunit.client.1.smithi173.stdout:0/289: fsync d6/d8/df/d2c/d42/f56 0 2024-04-20T11:09:32.480 INFO:tasks.workunit.client.1.smithi173.stdout:1/299: truncate d5/d6/d17/d42/f19 1445340 0 2024-04-20T11:09:32.481 INFO:tasks.workunit.client.1.smithi173.stdout:7/352: dwrite d9/d33/f6b [0,4194304] 0 2024-04-20T11:09:32.481 INFO:tasks.workunit.client.1.smithi173.stdout:6/350: mknod d4/d14/c69 0 2024-04-20T11:09:32.481 INFO:tasks.workunit.client.1.smithi173.stdout:4/329: rename da/d1e/d52/d5e to da/d1a/d60/d66 0 2024-04-20T11:09:32.483 INFO:tasks.workunit.client.1.smithi173.stdout:2/361: link d2/d34/f45 d2/d13/d28/f7c 0 2024-04-20T11:09:32.485 INFO:tasks.workunit.client.1.smithi173.stdout:6/351: read d4/f60 [2083975,118850] 0 2024-04-20T11:09:32.488 INFO:tasks.workunit.client.1.smithi173.stdout:8/353: rmdir d4/d18/d59/d5a 39 2024-04-20T11:09:32.489 INFO:tasks.workunit.client.1.smithi173.stdout:2/362: rmdir d2/d13/d46/d53/d4e 39 2024-04-20T11:09:32.497 INFO:tasks.workunit.client.1.smithi173.stdout:7/353: creat d9/da/d17/f79 x:0 0 0 2024-04-20T11:09:32.497 INFO:tasks.workunit.client.1.smithi173.stdout:2/363: write d2/d13/f38 [828198,13615] 0 2024-04-20T11:09:32.497 INFO:tasks.workunit.client.1.smithi173.stdout:3/373: dwrite d7/d9/f4f [0,4194304] 0 2024-04-20T11:09:32.498 INFO:tasks.workunit.client.1.smithi173.stdout:7/354: stat d9/da/d2e/l61 0 2024-04-20T11:09:32.505 INFO:tasks.workunit.client.1.smithi173.stdout:4/330: creat da/f67 x:0 0 0 2024-04-20T11:09:32.507 INFO:tasks.workunit.client.1.smithi173.stdout:8/354: mknod d4/d18/d3a/c6c 0 2024-04-20T11:09:32.513 INFO:tasks.workunit.client.1.smithi173.stdout:8/355: fdatasync d4/d18/f5c 0 2024-04-20T11:09:32.515 INFO:tasks.workunit.client.1.smithi173.stdout:0/290: getdents d6/d8/df/d2c/d42 0 2024-04-20T11:09:32.517 INFO:tasks.workunit.client.1.smithi173.stdout:7/355: mkdir d9/d33/d40/d7a 0 2024-04-20T11:09:32.518 INFO:tasks.workunit.client.1.smithi173.stdout:5/364: dwrite da/de/d19/d32/d30/d35/f6e [0,4194304] 0 2024-04-20T11:09:32.525 INFO:tasks.workunit.client.1.smithi173.stdout:1/300: getdents d5/d6/d17/d42/da/df 0 2024-04-20T11:09:32.525 INFO:tasks.workunit.client.1.smithi173.stdout:6/352: dwrite d4/f37 [0,4194304] 0 2024-04-20T11:09:32.525 INFO:tasks.workunit.client.1.smithi173.stdout:5/365: symlink da/l82 0 2024-04-20T11:09:32.526 INFO:tasks.workunit.client.1.smithi173.stdout:3/374: truncate d7/d9/d30/f60 1043173 0 2024-04-20T11:09:32.527 INFO:tasks.workunit.client.1.smithi173.stdout:5/366: fsync f8 0 2024-04-20T11:09:32.528 INFO:tasks.workunit.client.1.smithi173.stdout:7/356: creat d9/d1f/f7b x:0 0 0 2024-04-20T11:09:32.531 INFO:tasks.workunit.client.1.smithi173.stdout:4/331: dread f1 [0,4194304] 0 2024-04-20T11:09:32.533 INFO:tasks.workunit.client.1.smithi173.stdout:3/375: fdatasync d7/d17/f19 0 2024-04-20T11:09:32.534 INFO:tasks.workunit.client.1.smithi173.stdout:1/301: mknod d5/d6/d17/c6e 0 2024-04-20T11:09:32.538 INFO:tasks.workunit.client.1.smithi173.stdout:0/291: dread d6/d8/df/d43/f4c [0,4194304] 0 2024-04-20T11:09:32.542 INFO:tasks.workunit.client.1.smithi173.stdout:2/364: dwrite d2/d13/f74 [0,4194304] 0 2024-04-20T11:09:32.547 INFO:tasks.workunit.client.1.smithi173.stdout:8/356: dwrite d4/d18/d3a/d3c/f53 [0,4194304] 0 2024-04-20T11:09:32.551 INFO:tasks.workunit.client.1.smithi173.stdout:3/376: write d7/f51 [1796477,54752] 0 2024-04-20T11:09:32.552 INFO:tasks.workunit.client.1.smithi173.stdout:5/367: rename da/de/d75/c4a to da/de/d75/c83 0 2024-04-20T11:09:32.553 INFO:tasks.workunit.client.1.smithi173.stdout:1/302: creat d5/d6/d17/d42/d12/d1c/f6f x:0 0 0 2024-04-20T11:09:32.553 INFO:tasks.workunit.client.1.smithi173.stdout:3/377: chown d7/d53 4581 1 2024-04-20T11:09:32.555 INFO:tasks.workunit.client.1.smithi173.stdout:2/365: mkdir d2/d13/d46/d7d 0 2024-04-20T11:09:32.558 INFO:tasks.workunit.client.1.smithi173.stdout:3/378: dread - d7/d2b/f62 zero size 2024-04-20T11:09:32.558 INFO:tasks.workunit.client.1.smithi173.stdout:5/368: mkdir da/de/d19/d84 0 2024-04-20T11:09:32.559 INFO:tasks.workunit.client.1.smithi173.stdout:2/366: fsync d2/d13/d46/d53/f76 0 2024-04-20T11:09:32.560 INFO:tasks.workunit.client.1.smithi173.stdout:5/369: chown da/de/ff 101 1 2024-04-20T11:09:32.565 INFO:tasks.workunit.client.1.smithi173.stdout:4/332: dread da/dc/f33 [0,4194304] 0 2024-04-20T11:09:32.565 INFO:tasks.workunit.client.1.smithi173.stdout:8/357: mkdir d4/d18/d59/d6d 0 2024-04-20T11:09:32.565 INFO:tasks.workunit.client.1.smithi173.stdout:7/357: write d9/d33/f47 [18723,32285] 0 2024-04-20T11:09:32.566 INFO:tasks.workunit.client.1.smithi173.stdout:9/366: sync 2024-04-20T11:09:32.569 INFO:tasks.workunit.client.1.smithi173.stdout:2/367: stat d2/f5c 0 2024-04-20T11:09:32.570 INFO:tasks.workunit.client.1.smithi173.stdout:6/353: truncate d4/d14/f1a 3385134 0 2024-04-20T11:09:32.572 INFO:tasks.workunit.client.1.smithi173.stdout:5/370: rename da/d4f/l77 to da/de/d19/l85 0 2024-04-20T11:09:32.573 INFO:tasks.workunit.client.1.smithi173.stdout:6/354: chown d4/d1c/c34 1047410 1 2024-04-20T11:09:32.574 INFO:tasks.workunit.client.1.smithi173.stdout:4/333: creat da/d1e/d52/f68 x:0 0 0 2024-04-20T11:09:32.577 INFO:tasks.workunit.client.1.smithi173.stdout:7/358: creat d9/d33/d35/f7c x:0 0 0 2024-04-20T11:09:32.577 INFO:tasks.workunit.client.1.smithi173.stdout:3/379: unlink d7/d9/d5c/f72 0 2024-04-20T11:09:32.579 INFO:tasks.workunit.client.1.smithi173.stdout:0/292: getdents d6/d8/df/d2c/d42/d50 0 2024-04-20T11:09:32.581 INFO:tasks.workunit.client.1.smithi173.stdout:4/334: symlink da/dc/d15/l69 0 2024-04-20T11:09:32.586 INFO:tasks.workunit.client.1.smithi173.stdout:5/371: creat da/d4f/d65/f86 x:0 0 0 2024-04-20T11:09:32.587 INFO:tasks.workunit.client.1.smithi173.stdout:1/303: write d5/d6/d17/d42/da/f6a [585544,109132] 0 2024-04-20T11:09:32.587 INFO:tasks.workunit.client.1.smithi173.stdout:3/380: dread - d7/d9/f6f zero size 2024-04-20T11:09:32.588 INFO:tasks.workunit.client.1.smithi173.stdout:2/368: rename d2/c54 to d2/de/c7e 0 2024-04-20T11:09:32.590 INFO:tasks.workunit.client.1.smithi173.stdout:7/359: creat d9/d1f/f7d x:0 0 0 2024-04-20T11:09:32.591 INFO:tasks.workunit.client.1.smithi173.stdout:3/381: chown d7/d9/d3b/d45/f56 84950 1 2024-04-20T11:09:32.592 INFO:tasks.workunit.client.1.smithi173.stdout:6/355: dwrite d4/f37 [0,4194304] 0 2024-04-20T11:09:32.592 INFO:tasks.workunit.client.1.smithi173.stdout:0/293: mkdir d6/d8/d5e 0 2024-04-20T11:09:32.595 INFO:tasks.workunit.client.1.smithi173.stdout:0/294: chown l5 566166 1 2024-04-20T11:09:32.600 INFO:tasks.workunit.client.1.smithi173.stdout:2/369: creat d2/d13/d28/d4a/f7f x:0 0 0 2024-04-20T11:09:32.600 INFO:tasks.workunit.client.1.smithi173.stdout:1/304: chown d5/fb 304307 1 2024-04-20T11:09:32.602 INFO:tasks.workunit.client.1.smithi173.stdout:7/360: symlink d9/d33/d40/l7e 0 2024-04-20T11:09:32.604 INFO:tasks.workunit.client.1.smithi173.stdout:8/358: getdents d4/d18/d3a/d3c/d5f 0 2024-04-20T11:09:32.606 INFO:tasks.workunit.client.1.smithi173.stdout:8/359: chown d4 1 1 2024-04-20T11:09:32.606 INFO:tasks.workunit.client.1.smithi173.stdout:2/370: mknod d2/d34/c80 0 2024-04-20T11:09:32.612 INFO:tasks.workunit.client.1.smithi173.stdout:6/356: dwrite d4/d18/f58 [0,4194304] 0 2024-04-20T11:09:32.616 INFO:tasks.workunit.client.1.smithi173.stdout:0/295: rename d6/d8/d5e to d6/d5f 0 2024-04-20T11:09:32.618 INFO:tasks.workunit.client.1.smithi173.stdout:3/382: getdents d7/d17/d61 0 2024-04-20T11:09:32.620 INFO:tasks.workunit.client.1.smithi173.stdout:0/296: stat d6/d24/f4f 0 2024-04-20T11:09:32.622 INFO:tasks.workunit.client.1.smithi173.stdout:8/360: rename d4/c3e to d4/d18/d3a/d3c/d5f/c6e 0 2024-04-20T11:09:32.625 INFO:tasks.workunit.client.1.smithi173.stdout:1/305: dread d5/d6/d17/d42/f41 [0,4194304] 0 2024-04-20T11:09:32.626 INFO:tasks.workunit.client.1.smithi173.stdout:5/372: dwrite da/d4f/d65/f74 [0,4194304] 0 2024-04-20T11:09:32.628 INFO:tasks.workunit.client.1.smithi173.stdout:3/383: creat d7/d9/d3b/f79 x:0 0 0 2024-04-20T11:09:32.630 INFO:tasks.workunit.client.1.smithi173.stdout:0/297: chown d6/c37 513380026 1 2024-04-20T11:09:32.638 INFO:tasks.workunit.client.1.smithi173.stdout:5/373: mknod da/de/d75/d40/d59/c87 0 2024-04-20T11:09:32.644 INFO:tasks.workunit.client.1.smithi173.stdout:8/361: rename d4/d18/f55 to d4/d18/d3a/f6f 0 2024-04-20T11:09:32.645 INFO:tasks.workunit.client.1.smithi173.stdout:6/357: dwrite d4/d14/f2f [0,4194304] 0 2024-04-20T11:09:32.645 INFO:tasks.workunit.client.1.smithi173.stdout:0/298: mkdir d6/d8/df/d60 0 2024-04-20T11:09:32.646 INFO:tasks.workunit.client.1.smithi173.stdout:8/362: truncate d4/f6a 9961 0 2024-04-20T11:09:32.647 INFO:tasks.workunit.client.1.smithi173.stdout:6/358: fsync d4/d24/f54 0 2024-04-20T11:09:32.653 INFO:tasks.workunit.client.1.smithi173.stdout:4/335: truncate da/f25 1153663 0 2024-04-20T11:09:32.656 INFO:tasks.workunit.client.1.smithi173.stdout:1/306: dwrite d5/d6/dc/fe [0,4194304] 0 2024-04-20T11:09:32.659 INFO:tasks.workunit.client.1.smithi173.stdout:4/336: mknod da/d48/c6a 0 2024-04-20T11:09:32.659 INFO:tasks.workunit.client.1.smithi173.stdout:7/361: write d9/da/d31/f4d [581051,29884] 0 2024-04-20T11:09:32.663 INFO:tasks.workunit.client.1.smithi173.stdout:8/363: readlink d4/d18/d59/d5a/l68 0 2024-04-20T11:09:32.664 INFO:tasks.workunit.client.1.smithi173.stdout:4/337: write f4 [10662125,78828] 0 2024-04-20T11:09:32.667 INFO:tasks.workunit.client.1.smithi173.stdout:9/367: sync 2024-04-20T11:09:32.667 INFO:tasks.workunit.client.1.smithi173.stdout:1/307: fsync d5/d6/dc/d4a/f59 0 2024-04-20T11:09:32.669 INFO:tasks.workunit.client.1.smithi173.stdout:6/359: creat d4/d14/f6a x:0 0 0 2024-04-20T11:09:32.672 INFO:tasks.workunit.client.1.smithi173.stdout:7/362: mkdir d9/d1f/d7f 0 2024-04-20T11:09:32.675 INFO:tasks.workunit.client.1.smithi173.stdout:1/308: write f0 [2757204,54676] 0 2024-04-20T11:09:32.675 INFO:tasks.workunit.client.1.smithi173.stdout:5/374: getdents da/de/d19/d32/d30 0 2024-04-20T11:09:32.676 INFO:tasks.workunit.client.1.smithi173.stdout:4/338: creat da/d1a/d60/f6b x:0 0 0 2024-04-20T11:09:32.677 INFO:tasks.workunit.client.1.smithi173.stdout:8/364: symlink d4/d18/d4f/l70 0 2024-04-20T11:09:32.679 INFO:tasks.workunit.client.1.smithi173.stdout:2/371: dwrite d2/d13/d28/f7c [0,4194304] 0 2024-04-20T11:09:32.688 INFO:tasks.workunit.client.1.smithi173.stdout:3/384: truncate d7/f8 5489824 0 2024-04-20T11:09:32.691 INFO:tasks.workunit.client.1.smithi173.stdout:7/363: getdents d9/da/d31 0 2024-04-20T11:09:32.691 INFO:tasks.workunit.client.1.smithi173.stdout:9/368: chown d5/d9/d20/d29/d18/f70 481893 1 2024-04-20T11:09:32.692 INFO:tasks.workunit.client.1.smithi173.stdout:4/339: read da/dc/d15/f35 [1254548,68217] 0 2024-04-20T11:09:32.694 INFO:tasks.workunit.client.1.smithi173.stdout:4/340: dread - da/f64 zero size 2024-04-20T11:09:32.695 INFO:tasks.workunit.client.1.smithi173.stdout:5/375: rmdir da/d34/d6c 0 2024-04-20T11:09:32.695 INFO:tasks.workunit.client.1.smithi173.stdout:2/372: write d2/d34/f56 [602996,41480] 0 2024-04-20T11:09:32.702 INFO:tasks.workunit.client.1.smithi173.stdout:0/299: dwrite d6/d8/df/f1d [0,4194304] 0 2024-04-20T11:09:32.706 INFO:tasks.workunit.client.1.smithi173.stdout:8/365: mknod d4/d18/d3a/c71 0 2024-04-20T11:09:32.709 INFO:tasks.workunit.client.1.smithi173.stdout:9/369: symlink d5/d4c/l75 0 2024-04-20T11:09:32.709 INFO:tasks.workunit.client.1.smithi173.stdout:3/385: symlink d7/d9/l7a 0 2024-04-20T11:09:32.711 INFO:tasks.workunit.client.1.smithi173.stdout:0/300: unlink d6/l3f 0 2024-04-20T11:09:32.713 INFO:tasks.workunit.client.1.smithi173.stdout:4/341: symlink da/l6c 0 2024-04-20T11:09:32.716 INFO:tasks.workunit.client.1.smithi173.stdout:0/301: write d6/d24/f53 [849878,74810] 0 2024-04-20T11:09:32.722 INFO:tasks.workunit.client.1.smithi173.stdout:0/302: mknod d6/d8/c61 0 2024-04-20T11:09:32.723 INFO:tasks.workunit.client.1.smithi173.stdout:2/373: dwrite d2/d13/d28/d4a/f7f [0,4194304] 0 2024-04-20T11:09:32.725 INFO:tasks.workunit.client.1.smithi173.stdout:2/374: mkdir d2/d34/d81 0 2024-04-20T11:09:32.728 INFO:tasks.workunit.client.1.smithi173.stdout:0/303: creat d6/d8/df/d2c/d42/f62 x:0 0 0 2024-04-20T11:09:32.731 INFO:tasks.workunit.client.1.smithi173.stdout:2/375: truncate d2/d13/d28/d4a/f6c 445698 0 2024-04-20T11:09:32.734 INFO:tasks.workunit.client.1.smithi173.stdout:7/364: dread d9/da/d31/f4d [0,4194304] 0 2024-04-20T11:09:32.734 INFO:tasks.workunit.client.1.smithi173.stdout:0/304: truncate d6/d8/df/d2c/d42/f56 4859821 0 2024-04-20T11:09:32.737 INFO:tasks.workunit.client.1.smithi173.stdout:0/305: fsync d6/f27 0 2024-04-20T11:09:32.738 INFO:tasks.workunit.client.1.smithi173.stdout:7/365: write d9/da/d17/f79 [902759,29772] 0 2024-04-20T11:09:32.739 INFO:tasks.workunit.client.1.smithi173.stdout:2/376: creat d2/d13/d28/f82 x:0 0 0 2024-04-20T11:09:32.745 INFO:tasks.workunit.client.1.smithi173.stdout:7/366: read d9/d33/d35/d4a/f4c [2058105,96666] 0 2024-04-20T11:09:32.748 INFO:tasks.workunit.client.1.smithi173.stdout:6/360: truncate d4/d1c/d28/f53 703724 0 2024-04-20T11:09:32.751 INFO:tasks.workunit.client.1.smithi173.stdout:2/377: unlink d2/de/l22 0 2024-04-20T11:09:32.758 INFO:tasks.workunit.client.1.smithi173.stdout:2/378: chown d2/d13/c79 2573 1 2024-04-20T11:09:32.761 INFO:tasks.workunit.client.1.smithi173.stdout:1/309: truncate d5/d6/d17/d42/d12/f47 491121 0 2024-04-20T11:09:32.765 INFO:tasks.workunit.client.1.smithi173.stdout:5/376: write f9 [3025670,83524] 0 2024-04-20T11:09:32.768 INFO:tasks.workunit.client.1.smithi173.stdout:6/361: rename d4/d18/f2b to d4/d24/f6b 0 2024-04-20T11:09:32.769 INFO:tasks.workunit.client.1.smithi173.stdout:9/370: dread f0 [0,4194304] 0 2024-04-20T11:09:32.769 INFO:tasks.workunit.client.1.smithi173.stdout:7/367: creat d9/da/d17/d2b/f80 x:0 0 0 2024-04-20T11:09:32.775 INFO:tasks.workunit.client.1.smithi173.stdout:0/306: dwrite d6/d8/df/d2c/d2d/f36 [0,4194304] 0 2024-04-20T11:09:32.778 INFO:tasks.workunit.client.1.smithi173.stdout:0/307: chown d6/d8/df/d43/c49 750192007 1 2024-04-20T11:09:32.780 INFO:tasks.workunit.client.1.smithi173.stdout:3/386: truncate d7/d17/f3c 1245209 0 2024-04-20T11:09:32.784 INFO:tasks.workunit.client.1.smithi173.stdout:2/379: dwrite d2/de/f7a [0,4194304] 0 2024-04-20T11:09:32.796 INFO:tasks.workunit.client.1.smithi173.stdout:1/310: dread d5/d6/d17/d42/d12/f15 [0,4194304] 0 2024-04-20T11:09:32.800 INFO:tasks.workunit.client.1.smithi173.stdout:8/366: sync 2024-04-20T11:09:32.807 INFO:tasks.workunit.client.1.smithi173.stdout:7/368: fsync d9/da/d17/f1a 0 2024-04-20T11:09:32.807 INFO:tasks.workunit.client.1.smithi173.stdout:3/387: rmdir d7/d53 39 2024-04-20T11:09:32.808 INFO:tasks.workunit.client.1.smithi173.stdout:2/380: creat d2/d13/d46/d4b/f83 x:0 0 0 2024-04-20T11:09:32.810 INFO:tasks.workunit.client.1.smithi173.stdout:5/377: dread da/de/d19/d2c/f53 [0,4194304] 0 2024-04-20T11:09:32.811 INFO:tasks.workunit.client.1.smithi173.stdout:1/311: write d5/d6/d17/f27 [1359466,56699] 0 2024-04-20T11:09:32.813 INFO:tasks.workunit.client.1.smithi173.stdout:5/378: write da/f29 [2874030,43336] 0 2024-04-20T11:09:32.814 INFO:tasks.workunit.client.1.smithi173.stdout:9/371: mknod d5/c76 0 2024-04-20T11:09:32.815 INFO:tasks.workunit.client.1.smithi173.stdout:5/379: readlink da/de/l14 0 2024-04-20T11:09:32.819 INFO:tasks.workunit.client.1.smithi173.stdout:0/308: symlink d6/l63 0 2024-04-20T11:09:32.821 INFO:tasks.workunit.client.1.smithi173.stdout:2/381: creat d2/de/f84 x:0 0 0 2024-04-20T11:09:32.824 INFO:tasks.workunit.client.1.smithi173.stdout:1/312: rmdir d5/d6/d17/d42/d12/d1c 39 2024-04-20T11:09:32.824 INFO:tasks.workunit.client.1.smithi173.stdout:7/369: dwrite d9/d33/d35/d58/f70 [0,4194304] 0 2024-04-20T11:09:32.826 INFO:tasks.workunit.client.1.smithi173.stdout:7/370: chown d9/d33/f6b 3984135 1 2024-04-20T11:09:32.826 INFO:tasks.workunit.client.1.smithi173.stdout:1/313: fsync d5/d6/d17/d42/da/f14 0 2024-04-20T11:09:32.827 INFO:tasks.workunit.client.1.smithi173.stdout:7/371: chown f1 7691 1 2024-04-20T11:09:32.827 INFO:tasks.workunit.client.1.smithi173.stdout:9/372: creat d5/d9/d20/d29/d18/f77 x:0 0 0 2024-04-20T11:09:32.828 INFO:tasks.workunit.client.1.smithi173.stdout:5/380: symlink da/de/d19/l88 0 2024-04-20T11:09:32.833 INFO:tasks.workunit.client.1.smithi173.stdout:7/372: chown d9/d33/d35/d58/l77 749 1 2024-04-20T11:09:32.833 INFO:tasks.workunit.client.1.smithi173.stdout:5/381: chown da/de/d19/d32/l7c 1003257151 1 2024-04-20T11:09:32.834 INFO:tasks.workunit.client.1.smithi173.stdout:6/362: write d4/f5a [3363709,36975] 0 2024-04-20T11:09:32.836 INFO:tasks.workunit.client.1.smithi173.stdout:8/367: rename f2 to d4/d18/f72 0 2024-04-20T11:09:32.836 INFO:tasks.workunit.client.1.smithi173.stdout:2/382: write d2/d13/d46/d53/d4e/f59 [371789,107589] 0 2024-04-20T11:09:32.837 INFO:tasks.workunit.client.1.smithi173.stdout:1/314: mknod d5/d6/d17/d42/da/df/c70 0 2024-04-20T11:09:32.837 INFO:tasks.workunit.client.1.smithi173.stdout:9/373: write d5/d9/d20/d29/d18/f1f [2429222,56938] 0 2024-04-20T11:09:32.840 INFO:tasks.workunit.client.1.smithi173.stdout:5/382: stat da/de/d19/d32/d30/f41 0 2024-04-20T11:09:32.843 INFO:tasks.workunit.client.1.smithi173.stdout:4/342: dwrite da/f25 [0,4194304] 0 2024-04-20T11:09:32.843 INFO:tasks.workunit.client.1.smithi173.stdout:2/383: write d2/de/f5e [3858857,36303] 0 2024-04-20T11:09:32.847 INFO:tasks.workunit.client.1.smithi173.stdout:2/384: write d2/de/d11/f33 [1158774,107989] 0 2024-04-20T11:09:32.850 INFO:tasks.workunit.client.1.smithi173.stdout:7/373: rmdir d9/d33/d35/d4a 39 2024-04-20T11:09:32.857 INFO:tasks.workunit.client.1.smithi173.stdout:3/388: rename d7/d53/l77 to d7/d9/d5c/l7b 0 2024-04-20T11:09:32.859 INFO:tasks.workunit.client.1.smithi173.stdout:0/309: link d6/d8/df/d43/c55 d6/d5f/c64 0 2024-04-20T11:09:32.862 INFO:tasks.workunit.client.1.smithi173.stdout:9/374: dwrite d5/d4c/f52 [0,4194304] 0 2024-04-20T11:09:32.871 INFO:tasks.workunit.client.1.smithi173.stdout:1/315: readlink d5/d6/d17/d42/da/l51 0 2024-04-20T11:09:32.871 INFO:tasks.workunit.client.1.smithi173.stdout:4/343: mknod da/d1a/c6d 0 2024-04-20T11:09:32.871 INFO:tasks.workunit.client.1.smithi173.stdout:5/383: dwrite da/de/d75/f64 [0,4194304] 0 2024-04-20T11:09:32.876 INFO:tasks.workunit.client.1.smithi173.stdout:2/385: rename d2/d13/d28/f82 to d2/d13/d46/f85 0 2024-04-20T11:09:32.877 INFO:tasks.workunit.client.1.smithi173.stdout:9/375: mkdir d5/d9/d20/d2b/d59/d44/d78 0 2024-04-20T11:09:32.879 INFO:tasks.workunit.client.1.smithi173.stdout:2/386: fsync d2/d13/d28/d4a/f4d 0 2024-04-20T11:09:32.879 INFO:tasks.workunit.client.1.smithi173.stdout:5/384: creat da/de/d75/f89 x:0 0 0 2024-04-20T11:09:32.880 INFO:tasks.workunit.client.1.smithi173.stdout:0/310: mkdir d6/d54/d65 0 2024-04-20T11:09:32.881 INFO:tasks.workunit.client.1.smithi173.stdout:5/385: stat da/de/d75/f45 0 2024-04-20T11:09:32.882 INFO:tasks.workunit.client.1.smithi173.stdout:5/386: dread - da/de/f7e zero size 2024-04-20T11:09:32.885 INFO:tasks.workunit.client.1.smithi173.stdout:4/344: mkdir da/d1a/d6e 0 2024-04-20T11:09:32.887 INFO:tasks.workunit.client.1.smithi173.stdout:9/376: mkdir d5/d4c/d79 0 2024-04-20T11:09:32.889 INFO:tasks.workunit.client.1.smithi173.stdout:0/311: read d6/d8/d3c/f41 [11349,110543] 0 2024-04-20T11:09:32.890 INFO:tasks.workunit.client.1.smithi173.stdout:5/387: creat da/d4f/d5e/f8a x:0 0 0 2024-04-20T11:09:32.891 INFO:tasks.workunit.client.1.smithi173.stdout:6/363: dread d4/d14/f1a [0,4194304] 0 2024-04-20T11:09:32.894 INFO:tasks.workunit.client.1.smithi173.stdout:6/364: truncate d4/d14/f3c 6703394 0 2024-04-20T11:09:32.895 INFO:tasks.workunit.client.1.smithi173.stdout:9/377: write d5/f2f [609959,103266] 0 2024-04-20T11:09:32.899 INFO:tasks.workunit.client.1.smithi173.stdout:6/365: write d4/d18/f58 [4093537,3903] 0 2024-04-20T11:09:32.900 INFO:tasks.workunit.client.1.smithi173.stdout:3/389: write d7/d17/f3c [1987210,30094] 0 2024-04-20T11:09:32.901 INFO:tasks.workunit.client.1.smithi173.stdout:5/388: symlink da/de/d75/d40/d59/l8b 0 2024-04-20T11:09:32.903 INFO:tasks.workunit.client.1.smithi173.stdout:4/345: getdents da/dc/d13 0 2024-04-20T11:09:32.905 INFO:tasks.workunit.client.1.smithi173.stdout:0/312: unlink d6/d8/df/l3a 0 2024-04-20T11:09:32.907 INFO:tasks.workunit.client.1.smithi173.stdout:6/366: readlink d4/d1c/d28/l30 0 2024-04-20T11:09:32.908 INFO:tasks.workunit.client.1.smithi173.stdout:1/316: dwrite d5/d6/d17/d42/f19 [0,4194304] 0 2024-04-20T11:09:32.910 INFO:tasks.workunit.client.1.smithi173.stdout:4/346: unlink da/dc/d15/f18 0 2024-04-20T11:09:32.912 INFO:tasks.workunit.client.1.smithi173.stdout:0/313: symlink d6/d8/df/d15/l66 0 2024-04-20T11:09:32.913 INFO:tasks.workunit.client.1.smithi173.stdout:3/390: mknod d7/d2b/c7c 0 2024-04-20T11:09:32.914 INFO:tasks.workunit.client.1.smithi173.stdout:5/389: rename da/de/d19/d32/f15 to da/de/d19/d32/d30/d35/f8c 0 2024-04-20T11:09:32.916 INFO:tasks.workunit.client.1.smithi173.stdout:4/347: write da/d1a/d60/f63 [321412,108964] 0 2024-04-20T11:09:32.917 INFO:tasks.workunit.client.1.smithi173.stdout:3/391: chown d7/d2b/c7c 5738348 1 2024-04-20T11:09:32.921 INFO:tasks.workunit.client.1.smithi173.stdout:8/368: dread d4/d18/d3a/d3c/f37 [0,4194304] 0 2024-04-20T11:09:32.921 INFO:tasks.workunit.client.1.smithi173.stdout:4/348: dread - da/dc/d15/f5f zero size 2024-04-20T11:09:32.922 INFO:tasks.workunit.client.1.smithi173.stdout:3/392: truncate d7/d17/f3c 2563071 0 2024-04-20T11:09:32.925 INFO:tasks.workunit.client.1.smithi173.stdout:3/393: chown d7/d53/l6c 2 1 2024-04-20T11:09:32.926 INFO:tasks.workunit.client.1.smithi173.stdout:8/369: write d4/d18/d59/d5a/f67 [777101,96138] 0 2024-04-20T11:09:32.930 INFO:tasks.workunit.client.1.smithi173.stdout:1/317: write d5/fb [5133692,130232] 0 2024-04-20T11:09:32.936 INFO:tasks.workunit.client.1.smithi173.stdout:4/349: creat da/d1a/d60/f6f x:0 0 0 2024-04-20T11:09:32.938 INFO:tasks.workunit.client.1.smithi173.stdout:9/378: dread d5/d9/f1a [0,4194304] 0 2024-04-20T11:09:32.939 INFO:tasks.workunit.client.1.smithi173.stdout:8/370: rename d4/f44 to d4/d18/d3a/d3c/f73 0 2024-04-20T11:09:32.940 INFO:tasks.workunit.client.1.smithi173.stdout:5/390: dwrite da/f3d [0,4194304] 0 2024-04-20T11:09:32.943 INFO:tasks.workunit.client.1.smithi173.stdout:0/314: write d6/d8/d3c/d44/f46 [3418277,51723] 0 2024-04-20T11:09:32.947 INFO:tasks.workunit.client.1.smithi173.stdout:9/379: chown d5/l8 5779185 1 2024-04-20T11:09:32.947 INFO:tasks.workunit.client.1.smithi173.stdout:5/391: rmdir da/de/d75/d3e 39 2024-04-20T11:09:32.948 INFO:tasks.workunit.client.1.smithi173.stdout:6/367: dread d4/d24/f35 [0,4194304] 0 2024-04-20T11:09:32.949 INFO:tasks.workunit.client.1.smithi173.stdout:3/394: dwrite d7/d17/f24 [0,4194304] 0 2024-04-20T11:09:32.953 INFO:tasks.workunit.client.1.smithi173.stdout:8/371: write d4/d18/d59/d5a/f5d [140296,129365] 0 2024-04-20T11:09:32.958 INFO:tasks.workunit.client.1.smithi173.stdout:5/392: unlink da/de/d75/l13 0 2024-04-20T11:09:32.959 INFO:tasks.workunit.client.1.smithi173.stdout:8/372: chown d4/d18/d3a/d3c/c4b 2514 1 2024-04-20T11:09:32.962 INFO:tasks.workunit.client.1.smithi173.stdout:4/350: truncate da/dc/d15/f1d 2715787 0 2024-04-20T11:09:32.963 INFO:tasks.workunit.client.1.smithi173.stdout:9/380: rename d5/d9/d20/d29/d2d/f42 to d5/d4c/d79/f7a 0 2024-04-20T11:09:32.967 INFO:tasks.workunit.client.1.smithi173.stdout:4/351: chown da/dc/d13/d65 254 1 2024-04-20T11:09:32.968 INFO:tasks.workunit.client.1.smithi173.stdout:1/318: dwrite d5/d6/d17/f4b [0,4194304] 0 2024-04-20T11:09:32.971 INFO:tasks.workunit.client.1.smithi173.stdout:3/395: creat d7/d9/d30/f7d x:0 0 0 2024-04-20T11:09:32.972 INFO:tasks.workunit.client.1.smithi173.stdout:0/315: dwrite d6/d24/f53 [0,4194304] 0 2024-04-20T11:09:32.974 INFO:tasks.workunit.client.1.smithi173.stdout:4/352: readlink da/d1a/l1f 0 2024-04-20T11:09:32.978 INFO:tasks.workunit.client.1.smithi173.stdout:1/319: chown d5/d6/d17/d42/da/df/c64 1254953001 1 2024-04-20T11:09:32.979 INFO:tasks.workunit.client.1.smithi173.stdout:5/393: creat da/de/d19/d84/f8d x:0 0 0 2024-04-20T11:09:32.983 INFO:tasks.workunit.client.1.smithi173.stdout:3/396: write d7/d9/f6f [813656,104650] 0 2024-04-20T11:09:32.986 INFO:tasks.workunit.client.1.smithi173.stdout:1/320: write d5/d6/d17/d42/d12/f31 [1626491,85994] 0 2024-04-20T11:09:32.990 INFO:tasks.workunit.client.1.smithi173.stdout:0/316: dread d6/d8/d3c/f41 [0,4194304] 0 2024-04-20T11:09:32.993 INFO:tasks.workunit.client.1.smithi173.stdout:5/394: rename da/de/d19/d84 to da/d4f/d65/d8e 0 2024-04-20T11:09:32.994 INFO:tasks.workunit.client.1.smithi173.stdout:6/368: dwrite d4/d24/f35 [0,4194304] 0 2024-04-20T11:09:32.997 INFO:tasks.workunit.client.1.smithi173.stdout:5/395: read - da/d4f/f5f zero size 2024-04-20T11:09:32.999 INFO:tasks.workunit.client.1.smithi173.stdout:4/353: getdents da/d48 0 2024-04-20T11:09:32.999 INFO:tasks.workunit.client.1.smithi173.stdout:0/317: unlink d6/d8/df/d15/l66 0 2024-04-20T11:09:33.000 INFO:tasks.workunit.client.1.smithi173.stdout:6/369: write d4/d14/f43 [116241,106401] 0 2024-04-20T11:09:33.004 INFO:tasks.workunit.client.1.smithi173.stdout:1/321: truncate d5/d6/d17/d42/da/d20/d48/f61 1022002 0 2024-04-20T11:09:33.006 INFO:tasks.workunit.client.1.smithi173.stdout:0/318: unlink d6/d24/l33 0 2024-04-20T11:09:33.006 INFO:tasks.workunit.client.1.smithi173.stdout:4/354: mknod da/d1a/d60/c70 0 2024-04-20T11:09:33.008 INFO:tasks.workunit.client.1.smithi173.stdout:9/381: dwrite d5/d9/fa [4194304,4194304] 0 2024-04-20T11:09:33.010 INFO:tasks.workunit.client.1.smithi173.stdout:0/319: readlink d6/d8/df/d2c/d42/d50/l57 0 2024-04-20T11:09:33.011 INFO:tasks.workunit.client.1.smithi173.stdout:1/322: dread d5/d6/d17/d42/da/f28 [0,4194304] 0 2024-04-20T11:09:33.011 INFO:tasks.workunit.client.1.smithi173.stdout:6/370: creat d4/f6c x:0 0 0 2024-04-20T11:09:33.013 INFO:tasks.workunit.client.1.smithi173.stdout:3/397: truncate d7/f51 1390388 0 2024-04-20T11:09:33.019 INFO:tasks.workunit.client.1.smithi173.stdout:5/396: write da/de/d75/f51 [665161,81838] 0 2024-04-20T11:09:33.022 INFO:tasks.workunit.client.1.smithi173.stdout:0/320: rename d6/d8/c4e to d6/d8/df/d43/c67 0 2024-04-20T11:09:33.026 INFO:tasks.workunit.client.1.smithi173.stdout:1/323: symlink d5/d6/l71 0 2024-04-20T11:09:33.026 INFO:tasks.workunit.client.1.smithi173.stdout:4/355: rmdir da/d48/d53/d58 0 2024-04-20T11:09:33.029 INFO:tasks.workunit.client.1.smithi173.stdout:1/324: stat d5/d6/d17/f4b 0 2024-04-20T11:09:33.031 INFO:tasks.workunit.client.1.smithi173.stdout:5/397: link da/f2b da/de/d75/d3e/f8f 0 2024-04-20T11:09:33.035 INFO:tasks.workunit.client.1.smithi173.stdout:5/398: readlink da/l82 0 2024-04-20T11:09:33.036 INFO:tasks.workunit.client.1.smithi173.stdout:3/398: dwrite d7/d2b/f74 [0,4194304] 0 2024-04-20T11:09:33.038 INFO:tasks.workunit.client.1.smithi173.stdout:3/399: readlink d7/d9/l7a 0 2024-04-20T11:09:33.038 INFO:tasks.workunit.client.1.smithi173.stdout:6/371: dwrite d4/f5a [0,4194304] 0 2024-04-20T11:09:33.042 INFO:tasks.workunit.client.1.smithi173.stdout:1/325: creat d5/d6/dc/d4a/f72 x:0 0 0 2024-04-20T11:09:33.042 INFO:tasks.workunit.client.1.smithi173.stdout:4/356: getdents da/d1a/d60/d66 0 2024-04-20T11:09:33.047 INFO:tasks.workunit.client.1.smithi173.stdout:6/372: unlink d4/d18/l1d 0 2024-04-20T11:09:33.051 INFO:tasks.workunit.client.1.smithi173.stdout:0/321: truncate d6/d8/df/f1d 754212 0 2024-04-20T11:09:33.057 INFO:tasks.workunit.client.1.smithi173.stdout:3/400: unlink d7/d2b/f36 0 2024-04-20T11:09:33.058 INFO:tasks.workunit.client.1.smithi173.stdout:5/399: write da/de/d19/d2c/f62 [246288,59664] 0 2024-04-20T11:09:33.060 INFO:tasks.workunit.client.1.smithi173.stdout:4/357: truncate da/dc/d15/f1d 2787705 0 2024-04-20T11:09:33.063 INFO:tasks.workunit.client.1.smithi173.stdout:0/322: rename d6/d8/df/d2c/d42/c47 to d6/d8/d3c/d44/c68 0 2024-04-20T11:09:33.065 INFO:tasks.workunit.client.1.smithi173.stdout:5/400: dread - da/d34/f55 zero size 2024-04-20T11:09:33.066 INFO:tasks.workunit.client.1.smithi173.stdout:4/358: creat da/d1e/d52/f71 x:0 0 0 2024-04-20T11:09:33.066 INFO:tasks.workunit.client.1.smithi173.stdout:6/373: link d4/d24/d3e/c44 d4/d14/d5f/c6d 0 2024-04-20T11:09:33.068 INFO:tasks.workunit.client.1.smithi173.stdout:5/401: readlink da/de/d75/d40/d59/l7a 0 2024-04-20T11:09:33.069 INFO:tasks.workunit.client.1.smithi173.stdout:6/374: creat d4/d24/f6e x:0 0 0 2024-04-20T11:09:33.071 INFO:tasks.workunit.client.1.smithi173.stdout:5/402: fsync da/de/d19/f76 0 2024-04-20T11:09:33.072 INFO:tasks.workunit.client.1.smithi173.stdout:3/401: rename d7/d17/l5f to d7/d9/l7e 0 2024-04-20T11:09:33.073 INFO:tasks.workunit.client.1.smithi173.stdout:6/375: fsync d4/d24/d42/f67 0 2024-04-20T11:09:33.073 INFO:tasks.workunit.client.1.smithi173.stdout:5/403: fdatasync da/de/f37 0 2024-04-20T11:09:33.080 INFO:tasks.workunit.client.1.smithi173.stdout:4/359: rename da/l43 to da/d48/l72 0 2024-04-20T11:09:33.082 INFO:tasks.workunit.client.1.smithi173.stdout:3/402: dread d7/d17/f57 [0,4194304] 0 2024-04-20T11:09:33.083 INFO:tasks.workunit.client.1.smithi173.stdout:1/326: dread d5/d6/d17/d42/f54 [0,4194304] 0 2024-04-20T11:09:33.083 INFO:tasks.workunit.client.1.smithi173.stdout:0/323: dwrite d6/d8/d3c/f4a [0,4194304] 0 2024-04-20T11:09:33.086 INFO:tasks.workunit.client.1.smithi173.stdout:1/327: write d5/d6/d17/d42/da/f6a [868822,115072] 0 2024-04-20T11:09:33.090 INFO:tasks.workunit.client.1.smithi173.stdout:3/403: mkdir d7/d17/d58/d7f 0 2024-04-20T11:09:33.093 INFO:tasks.workunit.client.1.smithi173.stdout:1/328: chown d5/d6/d17/d42/da/d20/l4d 49225 1 2024-04-20T11:09:33.096 INFO:tasks.workunit.client.1.smithi173.stdout:0/324: unlink d6/d8/df/d15/f2e 0 2024-04-20T11:09:33.096 INFO:tasks.workunit.client.1.smithi173.stdout:5/404: dwrite da/de/d19/d32/d30/f4c [4194304,4194304] 0 2024-04-20T11:09:33.098 INFO:tasks.workunit.client.1.smithi173.stdout:6/376: link d4/d1c/d28/c47 d4/d14/c6f 0 2024-04-20T11:09:33.101 INFO:tasks.workunit.client.1.smithi173.stdout:5/405: truncate da/de/d19/d32/f11 4341914 0 2024-04-20T11:09:33.102 INFO:tasks.workunit.client.1.smithi173.stdout:4/360: rename da/d1e/l26 to da/d48/d53/d39/l73 0 2024-04-20T11:09:33.107 INFO:tasks.workunit.client.1.smithi173.stdout:3/404: creat d7/d9/d30/f80 x:0 0 0 2024-04-20T11:09:33.109 INFO:tasks.workunit.client.1.smithi173.stdout:0/325: mkdir d6/d8/d3c/d44/d69 0 2024-04-20T11:09:33.113 INFO:tasks.workunit.client.1.smithi173.stdout:4/361: write da/dc/d15/f35 [2073479,123132] 0 2024-04-20T11:09:33.117 INFO:tasks.workunit.client.1.smithi173.stdout:8/373: sync 2024-04-20T11:09:33.118 INFO:tasks.workunit.client.1.smithi173.stdout:7/374: sync 2024-04-20T11:09:33.118 INFO:tasks.workunit.client.1.smithi173.stdout:2/387: sync 2024-04-20T11:09:33.119 INFO:tasks.workunit.client.1.smithi173.stdout:8/374: chown d4/l3d 91259 1 2024-04-20T11:09:33.124 INFO:tasks.workunit.client.1.smithi173.stdout:3/405: fdatasync d7/d17/f57 0 2024-04-20T11:09:33.127 INFO:tasks.workunit.client.1.smithi173.stdout:0/326: dwrite d6/d8/d3c/f4a [0,4194304] 0 2024-04-20T11:09:33.129 INFO:tasks.workunit.client.1.smithi173.stdout:0/327: chown d6/c10 394857181 1 2024-04-20T11:09:33.131 INFO:tasks.workunit.client.1.smithi173.stdout:4/362: creat da/d48/d53/d39/f74 x:0 0 0 2024-04-20T11:09:33.133 INFO:tasks.workunit.client.1.smithi173.stdout:1/329: link d5/d6/d17/d42/da/d20/d48/f61 d5/d6/f73 0 2024-04-20T11:09:33.135 INFO:tasks.workunit.client.1.smithi173.stdout:6/377: dread d4/f46 [0,4194304] 0 2024-04-20T11:09:33.142 INFO:tasks.workunit.client.1.smithi173.stdout:7/375: mkdir d9/d33/d81 0 2024-04-20T11:09:33.150 INFO:tasks.workunit.client.1.smithi173.stdout:5/406: link da/c73 da/d4f/d65/d8e/c90 0 2024-04-20T11:09:33.151 INFO:tasks.workunit.client.1.smithi173.stdout:4/363: chown da/l34 8 1 2024-04-20T11:09:33.153 INFO:tasks.workunit.client.1.smithi173.stdout:1/330: write d5/d6/d17/d42/f54 [823619,65736] 0 2024-04-20T11:09:33.157 INFO:tasks.workunit.client.1.smithi173.stdout:6/378: creat d4/d24/d42/f70 x:0 0 0 2024-04-20T11:09:33.159 INFO:tasks.workunit.client.1.smithi173.stdout:7/376: symlink d9/d1f/d72/l82 0 2024-04-20T11:09:33.161 INFO:tasks.workunit.client.1.smithi173.stdout:4/364: mknod da/d48/d53/d39/c75 0 2024-04-20T11:09:33.162 INFO:tasks.workunit.client.1.smithi173.stdout:0/328: dread d6/f27 [0,4194304] 0 2024-04-20T11:09:33.166 INFO:tasks.workunit.client.1.smithi173.stdout:9/382: sync 2024-04-20T11:09:33.167 INFO:tasks.workunit.client.1.smithi173.stdout:1/331: creat d5/d6/d17/d42/da/d20/d38/d52/f74 x:0 0 0 2024-04-20T11:09:33.168 INFO:tasks.workunit.client.1.smithi173.stdout:9/383: fdatasync d5/d4c/f52 0 2024-04-20T11:09:33.169 INFO:tasks.workunit.client.1.smithi173.stdout:5/407: dwrite da/de/d19/d32/d66/f7d [0,4194304] 0 2024-04-20T11:09:33.169 INFO:tasks.workunit.client.1.smithi173.stdout:8/375: rename d4/d18/l69 to d4/d18/l74 0 2024-04-20T11:09:33.170 INFO:tasks.workunit.client.1.smithi173.stdout:2/388: dread d2/d34/d58/f5a [0,4194304] 0 2024-04-20T11:09:33.172 INFO:tasks.workunit.client.1.smithi173.stdout:5/408: chown da/de/d19/c20 6132768 1 2024-04-20T11:09:33.173 INFO:tasks.workunit.client.1.smithi173.stdout:8/376: read - d4/d18/f6b zero size 2024-04-20T11:09:33.174 INFO:tasks.workunit.client.1.smithi173.stdout:3/406: getdents d7/d9 0 2024-04-20T11:09:33.176 INFO:tasks.workunit.client.1.smithi173.stdout:4/365: stat da/d48/d53/d39/l73 0 2024-04-20T11:09:33.178 INFO:tasks.workunit.client.1.smithi173.stdout:9/384: truncate d5/d9/d20/d2b/d59/f4e 1791077 0 2024-04-20T11:09:33.179 INFO:tasks.workunit.client.1.smithi173.stdout:6/379: chown d4/d1c/d28/l4b 4 1 2024-04-20T11:09:33.180 INFO:tasks.workunit.client.1.smithi173.stdout:0/329: creat d6/d8/df/d2c/d2d/f6a x:0 0 0 2024-04-20T11:09:33.183 INFO:tasks.workunit.client.1.smithi173.stdout:6/380: write d4/d24/f6e [976341,97559] 0 2024-04-20T11:09:33.183 INFO:tasks.workunit.client.1.smithi173.stdout:7/377: dwrite d9/d33/d35/f66 [0,4194304] 0 2024-04-20T11:09:33.184 INFO:tasks.workunit.client.1.smithi173.stdout:0/330: write d6/f27 [4258669,30329] 0 2024-04-20T11:09:33.184 INFO:tasks.workunit.client.1.smithi173.stdout:1/332: mkdir d5/d6/d17/d42/da/d20/d38/d52/d75 0 2024-04-20T11:09:33.188 INFO:tasks.workunit.client.1.smithi173.stdout:8/377: fdatasync d4/d18/d3a/d3c/f40 0 2024-04-20T11:09:33.189 INFO:tasks.workunit.client.1.smithi173.stdout:5/409: creat da/de/d19/d32/d72/f91 x:0 0 0 2024-04-20T11:09:33.190 INFO:tasks.workunit.client.1.smithi173.stdout:9/385: read d5/d9/d20/d29/f11 [8121525,11879] 0 2024-04-20T11:09:33.192 INFO:tasks.workunit.client.1.smithi173.stdout:3/407: mknod d7/d2b/c81 0 2024-04-20T11:09:33.195 INFO:tasks.workunit.client.1.smithi173.stdout:5/410: chown da/de/ff 463676 1 2024-04-20T11:09:33.195 INFO:tasks.workunit.client.1.smithi173.stdout:5/411: chown f9 11211815 1 2024-04-20T11:09:33.198 INFO:tasks.workunit.client.1.smithi173.stdout:2/389: link d2/d13/l2d d2/d13/d46/d53/d4e/l86 0 2024-04-20T11:09:33.198 INFO:tasks.workunit.client.1.smithi173.stdout:7/378: unlink d9/da/l62 0 2024-04-20T11:09:33.199 INFO:tasks.workunit.client.1.smithi173.stdout:4/366: dread da/d1e/f38 [0,4194304] 0 2024-04-20T11:09:33.205 INFO:tasks.workunit.client.1.smithi173.stdout:7/379: creat d9/da/d17/d2b/f83 x:0 0 0 2024-04-20T11:09:33.206 INFO:tasks.workunit.client.1.smithi173.stdout:8/378: mknod d4/d18/d59/d6d/c75 0 2024-04-20T11:09:33.207 INFO:tasks.workunit.client.1.smithi173.stdout:8/379: stat d4/l11 0 2024-04-20T11:09:33.209 INFO:tasks.workunit.client.1.smithi173.stdout:5/412: mkdir da/de/d19/d81/d92 0 2024-04-20T11:09:33.209 INFO:tasks.workunit.client.1.smithi173.stdout:7/380: fdatasync d9/d1f/d72/f75 0 2024-04-20T11:09:33.211 INFO:tasks.workunit.client.1.smithi173.stdout:2/390: mkdir d2/d34/d58/d6a/d87 0 2024-04-20T11:09:33.215 INFO:tasks.workunit.client.1.smithi173.stdout:5/413: fdatasync da/de/d19/d32/f31 0 2024-04-20T11:09:33.219 INFO:tasks.workunit.client.1.smithi173.stdout:4/367: creat da/dc/d62/f76 x:0 0 0 2024-04-20T11:09:33.220 INFO:tasks.workunit.client.1.smithi173.stdout:1/333: dwrite d5/d6/f21 [0,4194304] 0 2024-04-20T11:09:33.222 INFO:tasks.workunit.client.1.smithi173.stdout:2/391: symlink d2/d13/d28/l88 0 2024-04-20T11:09:33.224 INFO:tasks.workunit.client.1.smithi173.stdout:4/368: mkdir da/d1e/d77 0 2024-04-20T11:09:33.225 INFO:tasks.workunit.client.1.smithi173.stdout:7/381: write d9/d33/d35/f44 [4687535,76071] 0 2024-04-20T11:09:33.225 INFO:tasks.workunit.client.1.smithi173.stdout:5/414: dread - da/d4f/d5e/f6d zero size 2024-04-20T11:09:33.233 INFO:tasks.workunit.client.1.smithi173.stdout:6/381: dread d4/d24/f6b [0,4194304] 0 2024-04-20T11:09:33.239 INFO:tasks.workunit.client.1.smithi173.stdout:6/382: chown d4/d24 0 1 2024-04-20T11:09:33.239 INFO:tasks.workunit.client.1.smithi173.stdout:1/334: symlink d5/d6/d17/d42/d12/l76 0 2024-04-20T11:09:33.240 INFO:tasks.workunit.client.1.smithi173.stdout:2/392: creat d2/d34/d81/f89 x:0 0 0 2024-04-20T11:09:33.241 INFO:tasks.workunit.client.1.smithi173.stdout:0/331: dwrite d6/d8/df/d43/f4c [0,4194304] 0 2024-04-20T11:09:33.241 INFO:tasks.workunit.client.1.smithi173.stdout:7/382: read d9/f2c [645579,111948] 0 2024-04-20T11:09:33.243 INFO:tasks.workunit.client.1.smithi173.stdout:2/393: fsync d2/de/d11/f33 0 2024-04-20T11:09:33.247 INFO:tasks.workunit.client.1.smithi173.stdout:6/383: symlink d4/d24/d3e/l71 0 2024-04-20T11:09:33.247 INFO:tasks.workunit.client.1.smithi173.stdout:9/386: dwrite d5/d9/d20/d2b/d59/d44/f56 [0,4194304] 0 2024-04-20T11:09:33.249 INFO:tasks.workunit.client.1.smithi173.stdout:1/335: write d5/d6/d17/d42/da/df/d40/f5d [2200707,17970] 0 2024-04-20T11:09:33.250 INFO:tasks.workunit.client.1.smithi173.stdout:8/380: truncate d4/d18/d3a/d3c/f53 3499380 0 2024-04-20T11:09:33.254 INFO:tasks.workunit.client.1.smithi173.stdout:9/387: stat d5/d4c/l75 0 2024-04-20T11:09:33.257 INFO:tasks.workunit.client.1.smithi173.stdout:6/384: write d4/d24/d42/f70 [570106,67245] 0 2024-04-20T11:09:33.261 INFO:tasks.workunit.client.1.smithi173.stdout:5/415: dwrite da/de/f2d [4194304,4194304] 0 2024-04-20T11:09:33.261 INFO:tasks.workunit.client.1.smithi173.stdout:0/332: creat d6/d8/d3c/f6b x:0 0 0 2024-04-20T11:09:33.262 INFO:tasks.workunit.client.1.smithi173.stdout:4/369: getdents da/d1a 0 2024-04-20T11:09:33.265 INFO:tasks.workunit.client.1.smithi173.stdout:8/381: mknod d4/d18/d59/d6d/c76 0 2024-04-20T11:09:33.265 INFO:tasks.workunit.client.1.smithi173.stdout:1/336: unlink d5/d6/d17/d42/d12/l76 0 2024-04-20T11:09:33.266 INFO:tasks.workunit.client.1.smithi173.stdout:4/370: chown da/dc/d13/f47 24005 1 2024-04-20T11:09:33.267 INFO:tasks.workunit.client.1.smithi173.stdout:6/385: creat d4/d24/d42/f72 x:0 0 0 2024-04-20T11:09:33.269 INFO:tasks.workunit.client.1.smithi173.stdout:0/333: creat d6/d8/df/d2c/d42/f6c x:0 0 0 2024-04-20T11:09:33.274 INFO:tasks.workunit.client.1.smithi173.stdout:0/334: stat d6/d8/df/d2c/d42/d50/c5d 0 2024-04-20T11:09:33.278 INFO:tasks.workunit.client.1.smithi173.stdout:2/394: dwrite d2/d13/d46/d53/d4e/f59 [0,4194304] 0 2024-04-20T11:09:33.278 INFO:tasks.workunit.client.1.smithi173.stdout:4/371: write da/d1a/f40 [2702955,26268] 0 2024-04-20T11:09:33.283 INFO:tasks.workunit.client.1.smithi173.stdout:0/335: creat d6/d8/df/d15/f6d x:0 0 0 2024-04-20T11:09:33.284 INFO:tasks.workunit.client.1.smithi173.stdout:2/395: stat d2/d13/d46/d53/f76 0 2024-04-20T11:09:33.291 INFO:tasks.workunit.client.1.smithi173.stdout:1/337: rmdir d5/d6 39 2024-04-20T11:09:33.291 INFO:tasks.workunit.client.1.smithi173.stdout:4/372: rmdir da 39 2024-04-20T11:09:33.292 INFO:tasks.workunit.client.1.smithi173.stdout:9/388: write d5/f6 [3180927,38188] 0 2024-04-20T11:09:33.292 INFO:tasks.workunit.client.1.smithi173.stdout:2/396: write d2/d13/d28/d4a/f4d [953187,9150] 0 2024-04-20T11:09:33.293 INFO:tasks.workunit.client.1.smithi173.stdout:3/408: sync 2024-04-20T11:09:33.297 INFO:tasks.workunit.client.1.smithi173.stdout:5/416: dwrite da/d34/f55 [0,4194304] 0 2024-04-20T11:09:33.300 INFO:tasks.workunit.client.1.smithi173.stdout:9/389: truncate d5/d9/f58 602616 0 2024-04-20T11:09:33.301 INFO:tasks.workunit.client.1.smithi173.stdout:3/409: creat d7/d9/d3b/f82 x:0 0 0 2024-04-20T11:09:33.301 INFO:tasks.workunit.client.1.smithi173.stdout:0/336: symlink d6/d54/d65/l6e 0 2024-04-20T11:09:33.302 INFO:tasks.workunit.client.1.smithi173.stdout:6/386: truncate d4/f5a 1083142 0 2024-04-20T11:09:33.305 INFO:tasks.workunit.client.1.smithi173.stdout:0/337: fdatasync d6/f7 0 2024-04-20T11:09:33.306 INFO:tasks.workunit.client.1.smithi173.stdout:1/338: mknod d5/c77 0 2024-04-20T11:09:33.308 INFO:tasks.workunit.client.1.smithi173.stdout:2/397: rename d2/de/c2e to d2/d13/d28/c8a 0 2024-04-20T11:09:33.308 INFO:tasks.workunit.client.1.smithi173.stdout:4/373: mknod da/d1e/d52/c78 0 2024-04-20T11:09:33.310 INFO:tasks.workunit.client.1.smithi173.stdout:9/390: chown d5/d9/d20/d2b/d59/f4e 18902914 1 2024-04-20T11:09:33.311 INFO:tasks.workunit.client.1.smithi173.stdout:5/417: symlink da/de/d19/d32/d30/d35/d48/l93 0 2024-04-20T11:09:33.313 INFO:tasks.workunit.client.1.smithi173.stdout:2/398: chown d2/d34/d58/f63 1 1 2024-04-20T11:09:33.317 INFO:tasks.workunit.client.1.smithi173.stdout:8/382: dwrite d4/d18/d3a/d3c/d5f/f61 [4194304,4194304] 0 2024-04-20T11:09:33.317 INFO:tasks.workunit.client.1.smithi173.stdout:7/383: dread d9/d33/d35/d4a/d52/d5b/f6d [0,4194304] 0 2024-04-20T11:09:33.319 INFO:tasks.workunit.client.1.smithi173.stdout:9/391: creat d5/d9/d20/f7b x:0 0 0 2024-04-20T11:09:33.319 INFO:tasks.workunit.client.1.smithi173.stdout:5/418: chown da/de/c2e 156550 1 2024-04-20T11:09:33.321 INFO:tasks.workunit.client.1.smithi173.stdout:3/410: rename d7/d9/d3b to d7/d17/d61/d83 0 2024-04-20T11:09:33.322 INFO:tasks.workunit.client.1.smithi173.stdout:7/384: fdatasync d9/da/d17/d2b/f74 0 2024-04-20T11:09:33.329 INFO:tasks.workunit.client.1.smithi173.stdout:9/392: chown d5/d9/cc 21 1 2024-04-20T11:09:33.329 INFO:tasks.workunit.client.1.smithi173.stdout:2/399: mknod d2/d34/d58/d6a/c8b 0 2024-04-20T11:09:33.334 INFO:tasks.workunit.client.1.smithi173.stdout:8/383: symlink d4/d18/l77 0 2024-04-20T11:09:33.342 INFO:tasks.workunit.client.1.smithi173.stdout:7/385: write d9/d33/d35/d4a/f4c [4713573,10244] 0 2024-04-20T11:09:33.344 INFO:tasks.workunit.client.1.smithi173.stdout:6/387: dwrite d4/f16 [0,4194304] 0 2024-04-20T11:09:33.348 INFO:tasks.workunit.client.1.smithi173.stdout:0/338: dwrite d6/d8/df/d43/f4c [0,4194304] 0 2024-04-20T11:09:33.350 INFO:tasks.workunit.client.1.smithi173.stdout:6/388: fsync d4/d24/d42/f67 0 2024-04-20T11:09:33.351 INFO:tasks.workunit.client.1.smithi173.stdout:1/339: dwrite d5/d6/dc/d4a/f59 [0,4194304] 0 2024-04-20T11:09:33.352 INFO:tasks.workunit.client.1.smithi173.stdout:4/374: truncate da/d48/d53/f3b 1390861 0 2024-04-20T11:09:33.354 INFO:tasks.workunit.client.1.smithi173.stdout:0/339: chown d6/f13 215 1 2024-04-20T11:09:33.360 INFO:tasks.workunit.client.1.smithi173.stdout:2/400: mkdir d2/d8c 0 2024-04-20T11:09:33.369 INFO:tasks.workunit.client.1.smithi173.stdout:8/384: dwrite d4/f6a [0,4194304] 0 2024-04-20T11:09:33.371 INFO:tasks.workunit.client.1.smithi173.stdout:8/385: readlink d4/l3b 0 2024-04-20T11:09:33.378 INFO:tasks.workunit.client.1.smithi173.stdout:9/393: dwrite d5/f53 [0,4194304] 0 2024-04-20T11:09:33.384 INFO:tasks.workunit.client.1.smithi173.stdout:2/401: dread d2/d13/d28/d4a/f7f [0,4194304] 0 2024-04-20T11:09:33.384 INFO:tasks.workunit.client.1.smithi173.stdout:1/340: dread d5/d6/d17/f4b [0,4194304] 0 2024-04-20T11:09:33.393 INFO:tasks.workunit.client.1.smithi173.stdout:1/341: fdatasync d5/d6/d17/d42/da/f14 0 2024-04-20T11:09:33.401 INFO:tasks.workunit.client.1.smithi173.stdout:8/386: dread d4/d18/d3a/f46 [0,4194304] 0 2024-04-20T11:09:33.413 INFO:tasks.workunit.client.1.smithi173.stdout:5/419: getdents da/d34/d7f 0 2024-04-20T11:09:33.425 INFO:tasks.workunit.client.1.smithi173.stdout:0/340: symlink d6/d8/df/d2c/d42/l6f 0 2024-04-20T11:09:33.427 INFO:tasks.workunit.client.1.smithi173.stdout:0/341: read d6/d8/df/d43/f4c [3636462,102076] 0 2024-04-20T11:09:33.449 INFO:tasks.workunit.client.1.smithi173.stdout:2/402: dwrite d2/d13/d28/f15 [0,4194304] 0 2024-04-20T11:09:33.451 INFO:tasks.workunit.client.1.smithi173.stdout:8/387: fdatasync d4/d18/f24 0 2024-04-20T11:09:33.458 INFO:tasks.workunit.client.1.smithi173.stdout:4/375: symlink da/dc/d13/d65/l79 0 2024-04-20T11:09:33.461 INFO:tasks.workunit.client.1.smithi173.stdout:4/376: write da/dc/d15/f1c [901182,20835] 0 2024-04-20T11:09:33.463 INFO:tasks.workunit.client.1.smithi173.stdout:4/377: stat da/d48/d53/f5b 0 2024-04-20T11:09:33.467 INFO:tasks.workunit.client.1.smithi173.stdout:3/411: link d7/d9/f28 d7/d17/d61/f84 0 2024-04-20T11:09:33.469 INFO:tasks.workunit.client.1.smithi173.stdout:3/412: fsync d7/f15 0 2024-04-20T11:09:33.473 INFO:tasks.workunit.client.1.smithi173.stdout:0/342: creat d6/d8/df/d15/f70 x:0 0 0 2024-04-20T11:09:33.474 INFO:tasks.workunit.client.1.smithi173.stdout:0/343: stat d6/d8/c4d 0 2024-04-20T11:09:33.478 INFO:tasks.workunit.client.1.smithi173.stdout:2/403: dread d2/d13/f74 [0,4194304] 0 2024-04-20T11:09:33.486 INFO:tasks.workunit.client.1.smithi173.stdout:4/378: unlink da/dc/d15/c23 0 2024-04-20T11:09:33.495 INFO:tasks.workunit.client.1.smithi173.stdout:2/404: link d2/f64 d2/d13/d46/d53/d4e/f8d 0 2024-04-20T11:09:33.498 INFO:tasks.workunit.client.1.smithi173.stdout:0/344: creat d6/d8/d51/f71 x:0 0 0 2024-04-20T11:09:33.498 INFO:tasks.workunit.client.1.smithi173.stdout:2/405: truncate d2/d13/d28/d4a/f6c 1494102 0 2024-04-20T11:09:33.510 INFO:tasks.workunit.client.1.smithi173.stdout:0/345: getdents d6 0 2024-04-20T11:09:33.513 INFO:tasks.workunit.client.1.smithi173.stdout:0/346: fdatasync d6/d8/f12 0 2024-04-20T11:09:33.544 INFO:tasks.workunit.client.1.smithi173.stdout:3/413: mkdir d7/d85 0 2024-04-20T11:09:33.547 INFO:tasks.workunit.client.1.smithi173.stdout:3/414: write d7/d9/f28 [870315,5857] 0 2024-04-20T11:09:33.549 INFO:tasks.workunit.client.1.smithi173.stdout:3/415: stat d7/d2b/l34 0 2024-04-20T11:09:33.550 INFO:tasks.workunit.client.1.smithi173.stdout:7/386: truncate d9/da/d17/f1b 817202 0 2024-04-20T11:09:33.552 INFO:tasks.workunit.client.1.smithi173.stdout:7/387: chown d9/da/f25 2 1 2024-04-20T11:09:33.555 INFO:tasks.workunit.client.1.smithi173.stdout:3/416: truncate d7/f47 1485002 0 2024-04-20T11:09:33.559 INFO:tasks.workunit.client.1.smithi173.stdout:3/417: stat d7/d17/d61/d83/l75 0 2024-04-20T11:09:33.565 INFO:tasks.workunit.client.1.smithi173.stdout:9/394: truncate d5/d9/d20/d2b/d59/d44/f56 3372647 0 2024-04-20T11:09:33.565 INFO:tasks.workunit.client.1.smithi173.stdout:1/342: symlink d5/d6/d17/d42/l78 0 2024-04-20T11:09:33.568 INFO:tasks.workunit.client.1.smithi173.stdout:9/395: write d5/d9/d20/d29/f64 [4870030,74915] 0 2024-04-20T11:09:33.570 INFO:tasks.workunit.client.1.smithi173.stdout:4/379: rmdir da/dc 39 2024-04-20T11:09:33.577 INFO:tasks.workunit.client.1.smithi173.stdout:4/380: mkdir da/d1e/d52/d7a 0 2024-04-20T11:09:33.579 INFO:tasks.workunit.client.1.smithi173.stdout:7/388: dread d9/da/d31/f38 [0,4194304] 0 2024-04-20T11:09:33.582 INFO:tasks.workunit.client.1.smithi173.stdout:7/389: truncate d9/da/d17/f78 445645 0 2024-04-20T11:09:33.582 INFO:tasks.workunit.client.1.smithi173.stdout:9/396: creat d5/d9/d20/d2b/d59/d44/d78/f7c x:0 0 0 2024-04-20T11:09:33.584 INFO:tasks.workunit.client.1.smithi173.stdout:2/406: write d2/d13/d46/d53/d4e/f59 [4283983,33965] 0 2024-04-20T11:09:33.584 INFO:tasks.workunit.client.1.smithi173.stdout:4/381: symlink da/d1a/d6e/l7b 0 2024-04-20T11:09:33.587 INFO:tasks.workunit.client.1.smithi173.stdout:9/397: mkdir d5/d9/d20/d4f/d5e/d7d 0 2024-04-20T11:09:33.590 INFO:tasks.workunit.client.1.smithi173.stdout:4/382: symlink da/d1a/l7c 0 2024-04-20T11:09:33.594 INFO:tasks.workunit.client.1.smithi173.stdout:5/420: rmdir da 39 2024-04-20T11:09:33.597 INFO:tasks.workunit.client.1.smithi173.stdout:7/390: link d9/d33/d35/d4a/c69 d9/d1f/c84 0 2024-04-20T11:09:33.601 INFO:tasks.workunit.client.1.smithi173.stdout:7/391: mkdir d9/d33/d85 0 2024-04-20T11:09:33.602 INFO:tasks.workunit.client.1.smithi173.stdout:0/347: rmdir d6/d8/df/d15 39 2024-04-20T11:09:33.604 INFO:tasks.workunit.client.1.smithi173.stdout:5/421: truncate da/d34/f6f 593364 0 2024-04-20T11:09:33.604 INFO:tasks.workunit.client.1.smithi173.stdout:7/392: mknod d9/d33/d35/d56/c86 0 2024-04-20T11:09:33.606 INFO:tasks.workunit.client.1.smithi173.stdout:5/422: fdatasync da/d4f/d65/f86 0 2024-04-20T11:09:33.606 INFO:tasks.workunit.client.1.smithi173.stdout:4/383: dwrite da/d1a/d60/f6b [0,4194304] 0 2024-04-20T11:09:33.607 INFO:tasks.workunit.client.1.smithi173.stdout:8/388: mknod d4/c78 0 2024-04-20T11:09:33.609 INFO:tasks.workunit.client.1.smithi173.stdout:6/389: rename d4/c4e to d4/d14/d5f/c73 0 2024-04-20T11:09:33.609 INFO:tasks.workunit.client.1.smithi173.stdout:8/389: fdatasync d4/d18/d3a/d3c/f34 0 2024-04-20T11:09:33.609 INFO:tasks.workunit.client.1.smithi173.stdout:3/418: rename d7/d17 to d7/d17/d86 22 2024-04-20T11:09:33.612 INFO:tasks.workunit.client.1.smithi173.stdout:2/407: rmdir d2 39 2024-04-20T11:09:33.613 INFO:tasks.workunit.client.1.smithi173.stdout:4/384: chown f8 7119719 1 2024-04-20T11:09:33.614 INFO:tasks.workunit.client.1.smithi173.stdout:7/393: creat d9/d1f/d7f/f87 x:0 0 0 2024-04-20T11:09:33.615 INFO:tasks.workunit.client.1.smithi173.stdout:5/423: rename da/de/d19/d32/f52 to da/de/d19/d2c/f94 0 2024-04-20T11:09:33.618 INFO:tasks.workunit.client.1.smithi173.stdout:3/419: creat d7/d53/f87 x:0 0 0 2024-04-20T11:09:33.623 INFO:tasks.workunit.client.1.smithi173.stdout:0/348: dwrite d6/d8/d3c/f41 [0,4194304] 0 2024-04-20T11:09:33.624 INFO:tasks.workunit.client.1.smithi173.stdout:4/385: mkdir da/d1a/d60/d7d 0 2024-04-20T11:09:33.626 INFO:tasks.workunit.client.1.smithi173.stdout:6/390: dread d4/d14/f2f [0,4194304] 0 2024-04-20T11:09:33.627 INFO:tasks.workunit.client.1.smithi173.stdout:0/349: write d6/d8/df/f16 [849825,92553] 0 2024-04-20T11:09:33.628 INFO:tasks.workunit.client.1.smithi173.stdout:5/424: rename da/d34/f55 to da/de/d75/d40/d59/f95 0 2024-04-20T11:09:33.632 INFO:tasks.workunit.client.1.smithi173.stdout:3/420: creat d7/d17/d61/d83/f88 x:0 0 0 2024-04-20T11:09:33.635 INFO:tasks.workunit.client.1.smithi173.stdout:2/408: dread d2/d34/d58/f6b [0,4194304] 0 2024-04-20T11:09:33.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:33 smithi117 ceph-mon[15659]: pgmap v26: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 45 MiB/s rd, 122 MiB/s wr, 292 op/s 2024-04-20T11:09:33.641 INFO:tasks.workunit.client.1.smithi173.stdout:1/343: write d5/d6/d17/d42/d12/d1f/f2f [4473239,99398] 0 2024-04-20T11:09:33.642 INFO:tasks.workunit.client.1.smithi173.stdout:0/350: rmdir d6/d8/df/d2c/d42 39 2024-04-20T11:09:33.643 INFO:tasks.workunit.client.1.smithi173.stdout:5/425: creat da/de/d75/d40/d59/f96 x:0 0 0 2024-04-20T11:09:33.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:33 smithi173 ceph-mon[23800]: pgmap v26: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 45 MiB/s rd, 122 MiB/s wr, 292 op/s 2024-04-20T11:09:33.646 INFO:tasks.workunit.client.1.smithi173.stdout:8/390: dwrite d4/d18/f5c [4194304,4194304] 0 2024-04-20T11:09:33.647 INFO:tasks.workunit.client.1.smithi173.stdout:2/409: write d2/de/f3b [902740,42454] 0 2024-04-20T11:09:33.648 INFO:tasks.workunit.client.1.smithi173.stdout:3/421: truncate d7/d9/f6f 1085754 0 2024-04-20T11:09:33.651 INFO:tasks.workunit.client.1.smithi173.stdout:8/391: chown d4/d18/l77 9987 1 2024-04-20T11:09:33.651 INFO:tasks.workunit.client.1.smithi173.stdout:5/426: mknod da/de/d75/d40/d59/c97 0 2024-04-20T11:09:33.657 INFO:tasks.workunit.client.1.smithi173.stdout:2/410: rename d2/de/f43 to d2/d34/d58/d6a/f8e 0 2024-04-20T11:09:33.657 INFO:tasks.workunit.client.1.smithi173.stdout:2/411: stat d2/de/f7a 0 2024-04-20T11:09:33.662 INFO:tasks.workunit.client.1.smithi173.stdout:4/386: symlink da/d48/d53/d39/l7e 0 2024-04-20T11:09:33.663 INFO:tasks.workunit.client.1.smithi173.stdout:8/392: mknod d4/d18/d3a/c79 0 2024-04-20T11:09:33.664 INFO:tasks.workunit.client.1.smithi173.stdout:2/412: write d2/d34/d58/f6b [1744280,83791] 0 2024-04-20T11:09:33.665 INFO:tasks.workunit.client.1.smithi173.stdout:5/427: link da/d4f/d5e/f8a da/d4f/d5e/f98 0 2024-04-20T11:09:33.666 INFO:tasks.workunit.client.1.smithi173.stdout:2/413: write d2/d34/f51 [1092040,48604] 0 2024-04-20T11:09:33.667 INFO:tasks.workunit.client.1.smithi173.stdout:0/351: link d6/d8/df/d15/f70 d6/d8/df/d2c/d42/f72 0 2024-04-20T11:09:33.669 INFO:tasks.workunit.client.1.smithi173.stdout:5/428: write da/de/d19/d32/d30/f70 [155057,99150] 0 2024-04-20T11:09:33.669 INFO:tasks.workunit.client.1.smithi173.stdout:0/352: chown d6/d8/df/d2c/d42 1348386 1 2024-04-20T11:09:33.680 INFO:tasks.workunit.client.1.smithi173.stdout:8/393: creat d4/d18/d3a/f7a x:0 0 0 2024-04-20T11:09:33.682 INFO:tasks.workunit.client.1.smithi173.stdout:9/398: truncate d5/d4c/d79/f7a 604835 0 2024-04-20T11:09:33.684 INFO:tasks.workunit.client.1.smithi173.stdout:0/353: write d6/d8/df/d2c/d42/f72 [26366,61189] 0 2024-04-20T11:09:33.684 INFO:tasks.workunit.client.1.smithi173.stdout:8/394: write d4/f32 [1389944,86522] 0 2024-04-20T11:09:33.690 INFO:tasks.workunit.client.1.smithi173.stdout:2/414: dwrite d2/d34/f56 [0,4194304] 0 2024-04-20T11:09:33.692 INFO:tasks.workunit.client.1.smithi173.stdout:0/354: creat d6/d54/d65/f73 x:0 0 0 2024-04-20T11:09:33.701 INFO:tasks.workunit.client.1.smithi173.stdout:0/355: write d6/d8/df/d2c/d42/f56 [2953167,37077] 0 2024-04-20T11:09:33.702 INFO:tasks.workunit.client.1.smithi173.stdout:2/415: symlink d2/l8f 0 2024-04-20T11:09:33.704 INFO:tasks.workunit.client.1.smithi173.stdout:5/429: dwrite da/d4f/d5e/f98 [0,4194304] 0 2024-04-20T11:09:33.704 INFO:tasks.workunit.client.1.smithi173.stdout:9/399: link d5/d9/d20/c63 d5/d9/d20/d29/d2d/d33/c7e 0 2024-04-20T11:09:33.707 INFO:tasks.workunit.client.1.smithi173.stdout:9/400: readlink d5/d9/d20/d2b/l65 0 2024-04-20T11:09:33.708 INFO:tasks.workunit.client.1.smithi173.stdout:5/430: chown da/de/d19/d32/d30/d35 3 1 2024-04-20T11:09:33.708 INFO:tasks.workunit.client.1.smithi173.stdout:0/356: rename d6/d8/df/d15/f6d to d6/d24/f74 0 2024-04-20T11:09:33.716 INFO:tasks.workunit.client.1.smithi173.stdout:9/401: read d5/f38 [1301184,70654] 0 2024-04-20T11:09:33.719 INFO:tasks.workunit.client.1.smithi173.stdout:2/416: dread d2/f3e [0,4194304] 0 2024-04-20T11:09:33.727 INFO:tasks.workunit.client.1.smithi173.stdout:5/431: rename da/d34/c4d to da/d4f/d65/d8e/c99 0 2024-04-20T11:09:33.728 INFO:tasks.workunit.client.1.smithi173.stdout:9/402: mkdir d5/d7f 0 2024-04-20T11:09:33.732 INFO:tasks.workunit.client.1.smithi173.stdout:6/391: truncate d4/f11 2301881 0 2024-04-20T11:09:33.734 INFO:tasks.workunit.client.1.smithi173.stdout:5/432: write da/de/d19/d2c/f62 [95048,33860] 0 2024-04-20T11:09:33.738 INFO:tasks.workunit.client.1.smithi173.stdout:9/403: mkdir d5/d4c/d79/d80 0 2024-04-20T11:09:33.738 INFO:tasks.workunit.client.1.smithi173.stdout:1/344: write d5/d6/f73 [1465281,114758] 0 2024-04-20T11:09:33.739 INFO:tasks.workunit.client.1.smithi173.stdout:0/357: dwrite d6/d8/df/d2c/d42/f72 [0,4194304] 0 2024-04-20T11:09:33.739 INFO:tasks.workunit.client.1.smithi173.stdout:3/422: write d7/f51 [252101,124414] 0 2024-04-20T11:09:33.746 INFO:tasks.workunit.client.1.smithi173.stdout:9/404: stat d5/d9/d20/d29/d2d/l2e 0 2024-04-20T11:09:33.748 INFO:tasks.workunit.client.1.smithi173.stdout:3/423: stat d7/d17/d58/c68 0 2024-04-20T11:09:33.749 INFO:tasks.workunit.client.1.smithi173.stdout:1/345: stat d5/d6/d17/d42/d12/d1f/f63 0 2024-04-20T11:09:33.750 INFO:tasks.workunit.client.1.smithi173.stdout:0/358: mkdir d6/d8/d3c/d44/d75 0 2024-04-20T11:09:33.752 INFO:tasks.workunit.client.1.smithi173.stdout:5/433: mkdir da/de/d19/d32/d30/d43/d9a 0 2024-04-20T11:09:33.753 INFO:tasks.workunit.client.1.smithi173.stdout:3/424: stat d7/d53/c6b 0 2024-04-20T11:09:33.753 INFO:tasks.workunit.client.1.smithi173.stdout:2/417: dread d2/de/d11/f23 [0,4194304] 0 2024-04-20T11:09:33.755 INFO:tasks.workunit.client.1.smithi173.stdout:7/394: dwrite d9/da/f10 [0,4194304] 0 2024-04-20T11:09:33.757 INFO:tasks.workunit.client.1.smithi173.stdout:9/405: fsync d5/d9/f26 0 2024-04-20T11:09:33.771 INFO:tasks.workunit.client.1.smithi173.stdout:1/346: chown d5/d6/d17/d42/da/d20/d38/d52 0 1 2024-04-20T11:09:33.775 INFO:tasks.workunit.client.1.smithi173.stdout:2/418: fsync d2/d34/d58/f63 0 2024-04-20T11:09:33.776 INFO:tasks.workunit.client.1.smithi173.stdout:4/387: dwrite da/dc/d15/f1d [0,4194304] 0 2024-04-20T11:09:33.779 INFO:tasks.workunit.client.1.smithi173.stdout:4/388: fdatasync da/f64 0 2024-04-20T11:09:33.783 INFO:tasks.workunit.client.1.smithi173.stdout:4/389: write da/d1a/d60/f6f [986773,50783] 0 2024-04-20T11:09:33.783 INFO:tasks.workunit.client.1.smithi173.stdout:6/392: truncate d4/f16 1940921 0 2024-04-20T11:09:33.785 INFO:tasks.workunit.client.1.smithi173.stdout:5/434: rmdir da/de/d19/d32/d30/d43 39 2024-04-20T11:09:33.793 INFO:tasks.workunit.client.1.smithi173.stdout:3/425: dwrite d7/d17/f57 [0,4194304] 0 2024-04-20T11:09:33.805 INFO:tasks.workunit.client.1.smithi173.stdout:4/390: fdatasync da/dc/d13/f51 0 2024-04-20T11:09:33.808 INFO:tasks.workunit.client.1.smithi173.stdout:6/393: rename d4/d1c/d28 to d4/d24/d42/d74 0 2024-04-20T11:09:33.809 INFO:tasks.workunit.client.1.smithi173.stdout:2/419: dwrite d2/de/d11/f33 [0,4194304] 0 2024-04-20T11:09:33.813 INFO:tasks.workunit.client.1.smithi173.stdout:2/420: chown d2/d34/f51 812635 1 2024-04-20T11:09:33.819 INFO:tasks.workunit.client.1.smithi173.stdout:0/359: truncate d6/d8/fe 3536192 0 2024-04-20T11:09:33.820 INFO:tasks.workunit.client.1.smithi173.stdout:4/391: creat da/d1e/d52/d7a/f7f x:0 0 0 2024-04-20T11:09:33.821 INFO:tasks.workunit.client.1.smithi173.stdout:6/394: creat d4/d18/d61/f75 x:0 0 0 2024-04-20T11:09:33.825 INFO:tasks.workunit.client.1.smithi173.stdout:6/395: readlink d4/d14/l56 0 2024-04-20T11:09:33.826 INFO:tasks.workunit.client.1.smithi173.stdout:7/395: dread d9/d33/d35/d58/f5f [0,4194304] 0 2024-04-20T11:09:33.827 INFO:tasks.workunit.client.1.smithi173.stdout:2/421: rename d2/de/d11/l1c to d2/l90 0 2024-04-20T11:09:33.828 INFO:tasks.workunit.client.1.smithi173.stdout:1/347: truncate d5/d6/d17/d42/da/df/d40/f5d 3478582 0 2024-04-20T11:09:33.830 INFO:tasks.workunit.client.1.smithi173.stdout:3/426: dwrite d7/d2b/f62 [0,4194304] 0 2024-04-20T11:09:33.831 INFO:tasks.workunit.client.1.smithi173.stdout:4/392: mknod da/dc/d62/c80 0 2024-04-20T11:09:33.832 INFO:tasks.workunit.client.1.smithi173.stdout:7/396: write d9/da/d17/d2b/f80 [546373,118484] 0 2024-04-20T11:09:33.834 INFO:tasks.workunit.client.1.smithi173.stdout:2/422: rmdir d2/d13 39 2024-04-20T11:09:33.837 INFO:tasks.workunit.client.1.smithi173.stdout:9/406: dread d5/d9/d20/d29/f5c [0,4194304] 0 2024-04-20T11:09:33.844 INFO:tasks.workunit.client.1.smithi173.stdout:5/435: dwrite da/de/f24 [0,4194304] 0 2024-04-20T11:09:33.845 INFO:tasks.workunit.client.1.smithi173.stdout:7/397: mknod d9/d33/d35/d56/c88 0 2024-04-20T11:09:33.845 INFO:tasks.workunit.client.1.smithi173.stdout:1/348: symlink d5/d6/d17/d42/da/d20/d48/l79 0 2024-04-20T11:09:33.849 INFO:tasks.workunit.client.1.smithi173.stdout:7/398: truncate d9/f2c 1963499 0 2024-04-20T11:09:33.849 INFO:tasks.workunit.client.1.smithi173.stdout:7/399: chown d9/da/lb 5 1 2024-04-20T11:09:33.855 INFO:tasks.workunit.client.1.smithi173.stdout:9/407: read d5/d9/d20/d29/d2d/f43 [3347874,63697] 0 2024-04-20T11:09:33.859 INFO:tasks.workunit.client.1.smithi173.stdout:7/400: mkdir d9/d33/d40/d89 0 2024-04-20T11:09:33.863 INFO:tasks.workunit.client.1.smithi173.stdout:3/427: link d7/d9/lc d7/d53/l89 0 2024-04-20T11:09:33.865 INFO:tasks.workunit.client.1.smithi173.stdout:6/396: dread d4/d18/f58 [0,4194304] 0 2024-04-20T11:09:33.865 INFO:tasks.workunit.client.1.smithi173.stdout:1/349: dwrite d5/d6/f73 [0,4194304] 0 2024-04-20T11:09:33.865 INFO:tasks.workunit.client.1.smithi173.stdout:5/436: mknod da/de/d19/d32/d30/c9b 0 2024-04-20T11:09:33.867 INFO:tasks.workunit.client.1.smithi173.stdout:2/423: getdents d2/d34/d58/d6a 0 2024-04-20T11:09:33.868 INFO:tasks.workunit.client.1.smithi173.stdout:7/401: creat d9/d33/d40/d89/f8a x:0 0 0 2024-04-20T11:09:33.872 INFO:tasks.workunit.client.1.smithi173.stdout:4/393: dread da/d48/d53/f37 [0,4194304] 0 2024-04-20T11:09:33.873 INFO:tasks.workunit.client.1.smithi173.stdout:6/397: creat d4/d14/d5f/f76 x:0 0 0 2024-04-20T11:09:33.876 INFO:tasks.workunit.client.1.smithi173.stdout:2/424: write d2/d13/d46/d53/f76 [221830,6723] 0 2024-04-20T11:09:33.877 INFO:tasks.workunit.client.1.smithi173.stdout:9/408: getdents d5/d9/d20/d29/d18 0 2024-04-20T11:09:33.881 INFO:tasks.workunit.client.1.smithi173.stdout:9/409: dread d5/d9/d20/d29/d18/f70 [0,4194304] 0 2024-04-20T11:09:33.882 INFO:tasks.workunit.client.1.smithi173.stdout:7/402: getdents d9/d33/d40/d7a 0 2024-04-20T11:09:33.884 INFO:tasks.workunit.client.1.smithi173.stdout:7/403: dread - d9/d33/f3a zero size 2024-04-20T11:09:33.884 INFO:tasks.workunit.client.1.smithi173.stdout:2/425: creat d2/d13/d46/d7d/f91 x:0 0 0 2024-04-20T11:09:33.888 INFO:tasks.workunit.client.1.smithi173.stdout:9/410: creat d5/d4c/f81 x:0 0 0 2024-04-20T11:09:33.890 INFO:tasks.workunit.client.1.smithi173.stdout:9/411: read d5/d9/f58 [146038,15862] 0 2024-04-20T11:09:33.891 INFO:tasks.workunit.client.1.smithi173.stdout:9/412: chown d5/d9/d20/d2b/d59 226559 1 2024-04-20T11:09:33.893 INFO:tasks.workunit.client.1.smithi173.stdout:1/350: fsync d5/d6/d17/d42/da/d20/d48/f61 0 2024-04-20T11:09:33.896 INFO:tasks.workunit.client.1.smithi173.stdout:4/394: getdents da/dc/d15 0 2024-04-20T11:09:33.897 INFO:tasks.workunit.client.1.smithi173.stdout:3/428: dread d7/f47 [0,4194304] 0 2024-04-20T11:09:33.898 INFO:tasks.workunit.client.1.smithi173.stdout:4/395: stat da/d48/d53/d39/c49 0 2024-04-20T11:09:33.898 INFO:tasks.workunit.client.1.smithi173.stdout:9/413: mkdir d5/d9/d20/d2b/d59/d44/d78/d82 0 2024-04-20T11:09:33.898 INFO:tasks.workunit.client.1.smithi173.stdout:5/437: dread da/de/d19/d32/d30/d35/f8c [0,4194304] 0 2024-04-20T11:09:33.901 INFO:tasks.workunit.client.1.smithi173.stdout:4/396: readlink da/dc/d15/l69 0 2024-04-20T11:09:33.901 INFO:tasks.workunit.client.1.smithi173.stdout:7/404: dwrite d9/da/f46 [0,4194304] 0 2024-04-20T11:09:33.902 INFO:tasks.workunit.client.1.smithi173.stdout:1/351: rmdir d5/d6/d17/d42/d12/d1c 39 2024-04-20T11:09:33.903 INFO:tasks.workunit.client.1.smithi173.stdout:3/429: write d7/d9/f4b [138466,100155] 0 2024-04-20T11:09:33.904 INFO:tasks.workunit.client.1.smithi173.stdout:9/414: mknod d5/d9/d20/d29/d18/c83 0 2024-04-20T11:09:33.905 INFO:tasks.workunit.client.1.smithi173.stdout:7/405: dread - d9/da/d17/f64 zero size 2024-04-20T11:09:33.908 INFO:tasks.workunit.client.1.smithi173.stdout:5/438: truncate da/de/d19/d32/d30/d35/f8c 537747 0 2024-04-20T11:09:33.911 INFO:tasks.workunit.client.1.smithi173.stdout:4/397: dread f1 [0,4194304] 0 2024-04-20T11:09:33.912 INFO:tasks.workunit.client.1.smithi173.stdout:7/406: creat d9/d1f/d72/f8b x:0 0 0 2024-04-20T11:09:33.915 INFO:tasks.workunit.client.1.smithi173.stdout:3/430: dread d7/d2b/f3d [0,4194304] 0 2024-04-20T11:09:33.916 INFO:tasks.workunit.client.1.smithi173.stdout:2/426: dwrite d2/d13/d28/d4a/f7f [4194304,4194304] 0 2024-04-20T11:09:33.920 INFO:tasks.workunit.client.1.smithi173.stdout:1/352: write d5/d6/d17/f4b [1743782,26364] 0 2024-04-20T11:09:33.922 INFO:tasks.workunit.client.1.smithi173.stdout:5/439: rename da/d4f/d65/d8e/c90 to da/de/d19/d32/c9c 0 2024-04-20T11:09:33.924 INFO:tasks.workunit.client.1.smithi173.stdout:3/431: mkdir d7/d17/d58/d8a 0 2024-04-20T11:09:33.927 INFO:tasks.workunit.client.1.smithi173.stdout:2/427: rename d2/de/d11/l9 to d2/d13/d46/d4b/l92 0 2024-04-20T11:09:33.929 INFO:tasks.workunit.client.1.smithi173.stdout:3/432: chown d7/d53/l59 140841 1 2024-04-20T11:09:33.929 INFO:tasks.workunit.client.1.smithi173.stdout:9/415: write d5/d9/d20/d29/d18/f70 [954539,66430] 0 2024-04-20T11:09:33.931 INFO:tasks.workunit.client.1.smithi173.stdout:2/428: rmdir d2/de/d11 39 2024-04-20T11:09:33.932 INFO:tasks.workunit.client.1.smithi173.stdout:9/416: chown d5/d9/d20/d4f 14245 1 2024-04-20T11:09:33.933 INFO:tasks.workunit.client.1.smithi173.stdout:2/429: chown d2/d34/d58/d6a 3896 1 2024-04-20T11:09:33.938 INFO:tasks.workunit.client.1.smithi173.stdout:2/430: symlink d2/d8c/l93 0 2024-04-20T11:09:33.940 INFO:tasks.workunit.client.1.smithi173.stdout:4/398: dwrite da/d1a/f40 [0,4194304] 0 2024-04-20T11:09:33.940 INFO:tasks.workunit.client.1.smithi173.stdout:7/407: dread d9/da/d2e/f37 [0,4194304] 0 2024-04-20T11:09:33.941 INFO:tasks.workunit.client.1.smithi173.stdout:2/431: fsync d2/d13/d28/f7b 0 2024-04-20T11:09:33.951 INFO:tasks.workunit.client.1.smithi173.stdout:4/399: rename f1 to da/d1a/f81 0 2024-04-20T11:09:33.953 INFO:tasks.workunit.client.1.smithi173.stdout:2/432: mkdir d2/d13/d28/d4a/d94 0 2024-04-20T11:09:33.953 INFO:tasks.workunit.client.1.smithi173.stdout:5/440: dwrite da/de/d19/d32/d30/d35/d48/f67 [0,4194304] 0 2024-04-20T11:09:33.958 INFO:tasks.workunit.client.1.smithi173.stdout:7/408: dread d9/d33/d35/f44 [4194304,4194304] 0 2024-04-20T11:09:33.958 INFO:tasks.workunit.client.1.smithi173.stdout:5/441: fdatasync da/de/d19/d32/d30/d35/f54 0 2024-04-20T11:09:33.960 INFO:tasks.workunit.client.1.smithi173.stdout:4/400: symlink da/d48/d53/d39/l82 0 2024-04-20T11:09:33.961 INFO:tasks.workunit.client.1.smithi173.stdout:1/353: dwrite d5/f30 [0,4194304] 0 2024-04-20T11:09:33.965 INFO:tasks.workunit.client.1.smithi173.stdout:4/401: truncate da/f64 486376 0 2024-04-20T11:09:33.966 INFO:tasks.workunit.client.1.smithi173.stdout:9/417: dwrite d5/d9/d20/d2b/d59/d44/f56 [0,4194304] 0 2024-04-20T11:09:33.967 INFO:tasks.workunit.client.1.smithi173.stdout:2/433: rename d2/d13/l2d to d2/l95 0 2024-04-20T11:09:33.968 INFO:tasks.workunit.client.1.smithi173.stdout:1/354: write d5/d6/d17/d42/da/df/d40/f4c [146620,74947] 0 2024-04-20T11:09:33.969 INFO:tasks.workunit.client.1.smithi173.stdout:8/395: sync 2024-04-20T11:09:33.975 INFO:tasks.workunit.client.1.smithi173.stdout:8/396: fdatasync d4/d18/d3a/f46 0 2024-04-20T11:09:33.976 INFO:tasks.workunit.client.1.smithi173.stdout:1/355: chown d5/d6/d17/l58 74 1 2024-04-20T11:09:33.977 INFO:tasks.workunit.client.1.smithi173.stdout:2/434: mknod d2/de/d11/c96 0 2024-04-20T11:09:33.978 INFO:tasks.workunit.client.1.smithi173.stdout:8/397: fsync d4/d18/d3a/d3c/f54 0 2024-04-20T11:09:33.981 INFO:tasks.workunit.client.1.smithi173.stdout:3/433: dread d7/f2d [0,4194304] 0 2024-04-20T11:09:33.983 INFO:tasks.workunit.client.1.smithi173.stdout:1/356: rmdir d5/d6/dc 39 2024-04-20T11:09:33.988 INFO:tasks.workunit.client.1.smithi173.stdout:2/435: creat d2/d34/d58/d6a/d87/f97 x:0 0 0 2024-04-20T11:09:33.989 INFO:tasks.workunit.client.1.smithi173.stdout:1/357: symlink d5/d6/d17/d42/da/d20/d38/l7a 0 2024-04-20T11:09:33.993 INFO:tasks.workunit.client.1.smithi173.stdout:4/402: write f8 [1507182,3139] 0 2024-04-20T11:09:33.994 INFO:tasks.workunit.client.1.smithi173.stdout:7/409: dread d9/da/f12 [0,4194304] 0 2024-04-20T11:09:33.994 INFO:tasks.workunit.client.1.smithi173.stdout:5/442: truncate da/f2b 22405 0 2024-04-20T11:09:33.996 INFO:tasks.workunit.client.1.smithi173.stdout:2/436: symlink d2/d13/d28/d4a/l98 0 2024-04-20T11:09:33.997 INFO:tasks.workunit.client.1.smithi173.stdout:7/410: write d9/d1f/f7d [228091,37689] 0 2024-04-20T11:09:33.999 INFO:tasks.workunit.client.1.smithi173.stdout:3/434: unlink d7/d9/d30/c65 0 2024-04-20T11:09:34.000 INFO:tasks.workunit.client.1.smithi173.stdout:4/403: creat da/d1a/d60/f83 x:0 0 0 2024-04-20T11:09:34.002 INFO:tasks.workunit.client.1.smithi173.stdout:5/443: creat da/d4f/d65/d8e/f9d x:0 0 0 2024-04-20T11:09:34.003 INFO:tasks.workunit.client.1.smithi173.stdout:2/437: symlink d2/d13/l99 0 2024-04-20T11:09:34.005 INFO:tasks.workunit.client.1.smithi173.stdout:1/358: creat d5/d6/d17/d42/da/d20/d38/d52/d75/f7b x:0 0 0 2024-04-20T11:09:34.008 INFO:tasks.workunit.client.1.smithi173.stdout:3/435: creat d7/d17/d61/f8b x:0 0 0 2024-04-20T11:09:34.009 INFO:tasks.workunit.client.1.smithi173.stdout:3/436: stat d7/f18 0 2024-04-20T11:09:34.013 INFO:tasks.workunit.client.1.smithi173.stdout:5/444: truncate da/de/f37 2639611 0 2024-04-20T11:09:34.014 INFO:tasks.workunit.client.1.smithi173.stdout:3/437: mknod d7/d53/c8c 0 2024-04-20T11:09:34.015 INFO:tasks.workunit.client.1.smithi173.stdout:4/404: dwrite da/d1a/f40 [0,4194304] 0 2024-04-20T11:09:34.016 INFO:tasks.workunit.client.1.smithi173.stdout:5/445: chown da/de/d75/d40/d59 456806 1 2024-04-20T11:09:34.020 INFO:tasks.workunit.client.1.smithi173.stdout:4/405: mkdir da/dc/d62/d84 0 2024-04-20T11:09:34.022 INFO:tasks.workunit.client.1.smithi173.stdout:3/438: write d7/d9/f46 [2547338,64164] 0 2024-04-20T11:09:34.022 INFO:tasks.workunit.client.1.smithi173.stdout:1/359: getdents d5/d6 0 2024-04-20T11:09:34.027 INFO:tasks.workunit.client.1.smithi173.stdout:4/406: creat da/d1e/d52/f85 x:0 0 0 2024-04-20T11:09:34.028 INFO:tasks.workunit.client.1.smithi173.stdout:3/439: rename d7/d9/f6f to d7/d9/d30/f8d 0 2024-04-20T11:09:34.030 INFO:tasks.workunit.client.1.smithi173.stdout:1/360: chown d5/d6/d17/d42/d12/c16 3861169 1 2024-04-20T11:09:34.030 INFO:tasks.workunit.client.1.smithi173.stdout:2/438: dread d2/f64 [4194304,4194304] 0 2024-04-20T11:09:34.034 INFO:tasks.workunit.client.1.smithi173.stdout:5/446: dwrite da/de/d19/d32/d30/d35/f54 [0,4194304] 0 2024-04-20T11:09:34.035 INFO:tasks.workunit.client.1.smithi173.stdout:5/447: chown da/d4f 4401840 1 2024-04-20T11:09:34.035 INFO:tasks.workunit.client.1.smithi173.stdout:5/448: stat f7 0 2024-04-20T11:09:34.037 INFO:tasks.workunit.client.1.smithi173.stdout:5/449: stat da/de/d19/d32/d30/d35/d48 0 2024-04-20T11:09:34.043 INFO:tasks.workunit.client.1.smithi173.stdout:8/398: truncate d4/d18/f5c 6952357 0 2024-04-20T11:09:34.046 INFO:tasks.workunit.client.1.smithi173.stdout:4/407: dread da/dc/d15/f35 [0,4194304] 0 2024-04-20T11:09:34.048 INFO:tasks.workunit.client.1.smithi173.stdout:1/361: mknod d5/d6/dc/c7c 0 2024-04-20T11:09:34.053 INFO:tasks.workunit.client.1.smithi173.stdout:2/439: write d2/f64 [6152321,32309] 0 2024-04-20T11:09:34.060 INFO:tasks.workunit.client.1.smithi173.stdout:2/440: truncate d2/d34/f5b 1631944 0 2024-04-20T11:09:34.061 INFO:tasks.workunit.client.1.smithi173.stdout:4/408: fsync da/dc/d15/f5f 0 2024-04-20T11:09:34.061 INFO:tasks.workunit.client.1.smithi173.stdout:7/411: dwrite d9/da/d17/f1a [0,4194304] 0 2024-04-20T11:09:34.066 INFO:tasks.workunit.client.1.smithi173.stdout:1/362: dread d5/f30 [0,4194304] 0 2024-04-20T11:09:34.067 INFO:tasks.workunit.client.1.smithi173.stdout:8/399: dwrite d4/d18/d3a/d3c/d5f/f61 [0,4194304] 0 2024-04-20T11:09:34.070 INFO:tasks.workunit.client.1.smithi173.stdout:7/412: write d9/d33/d35/f66 [2872344,123745] 0 2024-04-20T11:09:34.072 INFO:tasks.workunit.client.1.smithi173.stdout:3/440: chown d7/d9/lc 51 1 2024-04-20T11:09:34.073 INFO:tasks.workunit.client.1.smithi173.stdout:4/409: mknod da/d48/c86 0 2024-04-20T11:09:34.075 INFO:tasks.workunit.client.1.smithi173.stdout:1/363: readlink d5/d6/d17/d42/da/d20/d48/l79 0 2024-04-20T11:09:34.076 INFO:tasks.workunit.client.1.smithi173.stdout:4/410: read - da/dc/d62/f76 zero size 2024-04-20T11:09:34.084 INFO:tasks.workunit.client.1.smithi173.stdout:4/411: symlink da/d1a/d60/l87 0 2024-04-20T11:09:34.088 INFO:tasks.workunit.client.1.smithi173.stdout:1/364: creat d5/d6/d17/d42/d12/d1c/f7d x:0 0 0 2024-04-20T11:09:34.090 INFO:tasks.workunit.client.1.smithi173.stdout:2/441: dwrite d2/d34/f45 [4194304,4194304] 0 2024-04-20T11:09:34.090 INFO:tasks.workunit.client.1.smithi173.stdout:4/412: fsync da/dc/d13/f32 0 2024-04-20T11:09:34.092 INFO:tasks.workunit.client.1.smithi173.stdout:1/365: fsync f4 0 2024-04-20T11:09:34.094 INFO:tasks.workunit.client.1.smithi173.stdout:8/400: write d4/d18/d3a/d3c/f37 [3321413,5520] 0 2024-04-20T11:09:34.095 INFO:tasks.workunit.client.1.smithi173.stdout:2/442: chown d2/de/f84 23326 1 2024-04-20T11:09:34.095 INFO:tasks.workunit.client.1.smithi173.stdout:7/413: dwrite d9/da/f46 [4194304,4194304] 0 2024-04-20T11:09:34.099 INFO:tasks.workunit.client.1.smithi173.stdout:4/413: symlink da/dc/l88 0 2024-04-20T11:09:34.100 INFO:tasks.workunit.client.1.smithi173.stdout:3/441: truncate d7/d9/f66 3461400 0 2024-04-20T11:09:34.101 INFO:tasks.workunit.client.1.smithi173.stdout:8/401: readlink d4/d18/d3a/d3c/l38 0 2024-04-20T11:09:34.103 INFO:tasks.workunit.client.1.smithi173.stdout:4/414: fdatasync da/f64 0 2024-04-20T11:09:34.105 INFO:tasks.workunit.client.1.smithi173.stdout:2/443: chown d2/d34/d58/f5a 6987494 1 2024-04-20T11:09:34.105 INFO:tasks.workunit.client.1.smithi173.stdout:5/450: dread da/de/d19/d32/d30/f4c [0,4194304] 0 2024-04-20T11:09:34.108 INFO:tasks.workunit.client.1.smithi173.stdout:2/444: dread - d2/d13/d46/d4b/f83 zero size 2024-04-20T11:09:34.110 INFO:tasks.workunit.client.1.smithi173.stdout:3/442: truncate d7/d9/f14 440485 0 2024-04-20T11:09:34.111 INFO:tasks.workunit.client.1.smithi173.stdout:1/366: creat d5/d6/d17/d42/d12/f7e x:0 0 0 2024-04-20T11:09:34.111 INFO:tasks.workunit.client.1.smithi173.stdout:5/451: chown da/de/d19/d32/d30/d35/d48/l93 100 1 2024-04-20T11:09:34.113 INFO:tasks.workunit.client.1.smithi173.stdout:0/360: sync 2024-04-20T11:09:34.117 INFO:tasks.workunit.client.1.smithi173.stdout:2/445: mkdir d2/d13/d9a 0 2024-04-20T11:09:34.117 INFO:tasks.workunit.client.1.smithi173.stdout:4/415: creat da/d48/d53/d39/f89 x:0 0 0 2024-04-20T11:09:34.126 INFO:tasks.workunit.client.1.smithi173.stdout:5/452: write da/d4f/d65/d8e/f9d [570460,40889] 0 2024-04-20T11:09:34.129 INFO:tasks.workunit.client.1.smithi173.stdout:4/416: readlink da/d48/d53/d39/l7e 0 2024-04-20T11:09:34.131 INFO:tasks.workunit.client.1.smithi173.stdout:5/453: mkdir da/d9e 0 2024-04-20T11:09:34.132 INFO:tasks.workunit.client.1.smithi173.stdout:1/367: truncate d5/d6/dc/d4a/f72 639731 0 2024-04-20T11:09:34.133 INFO:tasks.workunit.client.1.smithi173.stdout:0/361: creat d6/d8/df/d2c/d42/d50/f76 x:0 0 0 2024-04-20T11:09:34.134 INFO:tasks.workunit.client.1.smithi173.stdout:2/446: rename d2/l95 to d2/d13/d46/l9b 0 2024-04-20T11:09:34.134 INFO:tasks.workunit.client.1.smithi173.stdout:7/414: dwrite d9/da/f10 [0,4194304] 0 2024-04-20T11:09:34.137 INFO:tasks.workunit.client.1.smithi173.stdout:0/362: mkdir d6/d8/d3c/d77 0 2024-04-20T11:09:34.138 INFO:tasks.workunit.client.1.smithi173.stdout:7/415: chown d9/da/d17/d2b/f80 7562793 1 2024-04-20T11:09:34.140 INFO:tasks.workunit.client.1.smithi173.stdout:0/363: fsync d6/d24/f4f 0 2024-04-20T11:09:34.144 INFO:tasks.workunit.client.1.smithi173.stdout:7/416: creat d9/d33/d40/d7a/f8c x:0 0 0 2024-04-20T11:09:34.148 INFO:tasks.workunit.client.1.smithi173.stdout:7/417: dread - d9/da/d43/f57 zero size 2024-04-20T11:09:34.152 INFO:tasks.workunit.client.1.smithi173.stdout:7/418: write d9/da/d17/d2b/f80 [795822,42143] 0 2024-04-20T11:09:34.154 INFO:tasks.workunit.client.1.smithi173.stdout:5/454: dwrite da/de/d19/d32/d30/f4c [0,4194304] 0 2024-04-20T11:09:34.158 INFO:tasks.workunit.client.1.smithi173.stdout:5/455: creat da/d4f/d65/d8e/f9f x:0 0 0 2024-04-20T11:09:34.160 INFO:tasks.workunit.client.1.smithi173.stdout:7/419: read d9/d33/d40/f49 [122519,91948] 0 2024-04-20T11:09:34.161 INFO:tasks.workunit.client.1.smithi173.stdout:2/447: dread d2/d34/d58/f5a [0,4194304] 0 2024-04-20T11:09:34.165 INFO:tasks.workunit.client.1.smithi173.stdout:7/420: mkdir d9/d1f/d7f/d8d 0 2024-04-20T11:09:34.168 INFO:tasks.workunit.client.1.smithi173.stdout:2/448: creat d2/d13/d46/f9c x:0 0 0 2024-04-20T11:09:34.168 INFO:tasks.workunit.client.1.smithi173.stdout:1/368: dwrite f0 [0,4194304] 0 2024-04-20T11:09:34.168 INFO:tasks.workunit.client.1.smithi173.stdout:7/421: stat d9/da/d17/c1e 0 2024-04-20T11:09:34.169 INFO:tasks.workunit.client.1.smithi173.stdout:4/417: dwrite da/d48/d53/f3b [0,4194304] 0 2024-04-20T11:09:34.171 INFO:tasks.workunit.client.1.smithi173.stdout:5/456: rename da/de/d75/d3e/f8f to da/d9e/fa0 0 2024-04-20T11:09:34.172 INFO:tasks.workunit.client.1.smithi173.stdout:0/364: truncate d6/d8/df/d15/f70 1952050 0 2024-04-20T11:09:34.172 INFO:tasks.workunit.client.1.smithi173.stdout:1/369: read d5/d6/d17/f4b [2700335,97774] 0 2024-04-20T11:09:34.172 INFO:tasks.workunit.client.1.smithi173.stdout:2/449: chown d2/d13/d46/f73 11489779 1 2024-04-20T11:09:34.174 INFO:tasks.workunit.client.1.smithi173.stdout:2/450: fsync d2/de/f84 0 2024-04-20T11:09:34.180 INFO:tasks.workunit.client.1.smithi173.stdout:7/422: mknod d9/d33/d35/d4a/c8e 0 2024-04-20T11:09:34.181 INFO:tasks.workunit.client.1.smithi173.stdout:0/365: symlink d6/d8/df/d43/l78 0 2024-04-20T11:09:34.182 INFO:tasks.workunit.client.1.smithi173.stdout:5/457: creat da/de/d75/d40/fa1 x:0 0 0 2024-04-20T11:09:34.182 INFO:tasks.workunit.client.1.smithi173.stdout:4/418: mknod da/d48/d53/c8a 0 2024-04-20T11:09:34.183 INFO:tasks.workunit.client.1.smithi173.stdout:1/370: mkdir d5/d6/dc/d7f 0 2024-04-20T11:09:34.184 INFO:tasks.workunit.client.1.smithi173.stdout:5/458: read - da/de/d75/f7b zero size 2024-04-20T11:09:34.187 INFO:tasks.workunit.client.1.smithi173.stdout:5/459: stat da/de/d19/d2c/f79 0 2024-04-20T11:09:34.188 INFO:tasks.workunit.client.1.smithi173.stdout:1/371: stat d5/d6/d17/d42/d12/d1c/f6f 0 2024-04-20T11:09:34.189 INFO:tasks.workunit.client.1.smithi173.stdout:1/372: chown d5/d6/d17 26211377 1 2024-04-20T11:09:34.193 INFO:tasks.workunit.client.1.smithi173.stdout:5/460: mkdir da/de/d19/d32/d30/d35/d48/da2 0 2024-04-20T11:09:34.196 INFO:tasks.workunit.client.1.smithi173.stdout:0/366: fsync d6/d8/df/d15/f70 0 2024-04-20T11:09:34.198 INFO:tasks.workunit.client.1.smithi173.stdout:5/461: symlink da/de/d75/d40/d59/la3 0 2024-04-20T11:09:34.201 INFO:tasks.workunit.client.1.smithi173.stdout:4/419: dwrite da/dc/d13/f32 [4194304,4194304] 0 2024-04-20T11:09:34.202 INFO:tasks.workunit.client.1.smithi173.stdout:2/451: truncate d2/d34/d58/d6a/f8e 807519 0 2024-04-20T11:09:34.202 INFO:tasks.workunit.client.1.smithi173.stdout:0/367: rename d6/d8/c45 to d6/d5f/c79 0 2024-04-20T11:09:34.203 INFO:tasks.workunit.client.1.smithi173.stdout:7/423: dwrite d9/f30 [4194304,4194304] 0 2024-04-20T11:09:34.203 INFO:tasks.workunit.client.1.smithi173.stdout:5/462: mkdir da/d34/d7f/da4 0 2024-04-20T11:09:34.205 INFO:tasks.workunit.client.1.smithi173.stdout:0/368: rmdir d6/d5f 39 2024-04-20T11:09:34.208 INFO:tasks.workunit.client.1.smithi173.stdout:4/420: mkdir da/dc/d15/d8b 0 2024-04-20T11:09:34.210 INFO:tasks.workunit.client.1.smithi173.stdout:7/424: mkdir d9/d1f/d72/d8f 0 2024-04-20T11:09:34.217 INFO:tasks.workunit.client.1.smithi173.stdout:4/421: unlink da/d1a/d60/f6f 0 2024-04-20T11:09:34.217 INFO:tasks.workunit.client.1.smithi173.stdout:7/425: dread d9/da/f12 [0,4194304] 0 2024-04-20T11:09:34.220 INFO:tasks.workunit.client.1.smithi173.stdout:2/452: dread d2/d13/d28/f7c [4194304,4194304] 0 2024-04-20T11:09:34.223 INFO:tasks.workunit.client.1.smithi173.stdout:0/369: creat d6/d8/df/d2c/d42/f7a x:0 0 0 2024-04-20T11:09:34.223 INFO:tasks.workunit.client.1.smithi173.stdout:2/453: chown d2/d13/l99 211561519 1 2024-04-20T11:09:34.223 INFO:tasks.workunit.client.1.smithi173.stdout:7/426: fdatasync d9/da/d17/f1b 0 2024-04-20T11:09:34.224 INFO:tasks.workunit.client.1.smithi173.stdout:2/454: stat d2 0 2024-04-20T11:09:34.225 INFO:tasks.workunit.client.1.smithi173.stdout:5/463: dwrite da/de/d19/d32/d30/d35/f6e [0,4194304] 0 2024-04-20T11:09:34.226 INFO:tasks.workunit.client.1.smithi173.stdout:0/370: write d6/d24/f4f [446367,87886] 0 2024-04-20T11:09:34.228 INFO:tasks.workunit.client.1.smithi173.stdout:9/418: sync 2024-04-20T11:09:34.228 INFO:tasks.workunit.client.1.smithi173.stdout:6/398: sync 2024-04-20T11:09:34.230 INFO:tasks.workunit.client.1.smithi173.stdout:7/427: rename d9/da/d2e/l61 to d9/da/d43/l90 0 2024-04-20T11:09:34.233 INFO:tasks.workunit.client.1.smithi173.stdout:7/428: dread - d9/d33/f3a zero size 2024-04-20T11:09:34.237 INFO:tasks.workunit.client.1.smithi173.stdout:6/399: unlink d4/f6c 0 2024-04-20T11:09:34.237 INFO:tasks.workunit.client.1.smithi173.stdout:6/400: stat d4/d14/c69 0 2024-04-20T11:09:34.237 INFO:tasks.workunit.client.1.smithi173.stdout:0/371: write d6/f29 [1997285,118958] 0 2024-04-20T11:09:34.239 INFO:tasks.workunit.client.1.smithi173.stdout:9/419: fdatasync d5/d9/f47 0 2024-04-20T11:09:34.240 INFO:tasks.workunit.client.1.smithi173.stdout:4/422: dwrite da/f25 [4194304,4194304] 0 2024-04-20T11:09:34.243 INFO:tasks.workunit.client.1.smithi173.stdout:4/423: chown da/dc/d13 445 1 2024-04-20T11:09:34.247 INFO:tasks.workunit.client.1.smithi173.stdout:5/464: truncate da/de/d19/d32/f2f 91958 0 2024-04-20T11:09:34.249 INFO:tasks.workunit.client.1.smithi173.stdout:2/455: dwrite d2/d13/f6d [0,4194304] 0 2024-04-20T11:09:34.249 INFO:tasks.workunit.client.1.smithi173.stdout:0/372: truncate d6/d8/df/d2c/d42/f6c 396404 0 2024-04-20T11:09:34.252 INFO:tasks.workunit.client.1.smithi173.stdout:6/401: symlink d4/d14/d5f/l77 0 2024-04-20T11:09:34.258 INFO:tasks.workunit.client.1.smithi173.stdout:9/420: mknod d5/d9/d20/d29/d2d/d33/c84 0 2024-04-20T11:09:34.266 INFO:tasks.workunit.client.1.smithi173.stdout:7/429: write d9/da/d17/f1b [1209470,14636] 0 2024-04-20T11:09:34.266 INFO:tasks.workunit.client.1.smithi173.stdout:6/402: readlink d4/d1c/l3d 0 2024-04-20T11:09:34.270 INFO:tasks.workunit.client.1.smithi173.stdout:9/421: creat d5/d9/d20/f85 x:0 0 0 2024-04-20T11:09:34.275 INFO:tasks.workunit.client.1.smithi173.stdout:5/465: rename da/de/d19/d32/d30/d43 to da/d34/d7f/da5 0 2024-04-20T11:09:34.275 INFO:tasks.workunit.client.1.smithi173.stdout:0/373: creat d6/d8/d3c/d44/d75/f7b x:0 0 0 2024-04-20T11:09:34.279 INFO:tasks.workunit.client.1.smithi173.stdout:9/422: dread d5/d9/f58 [0,4194304] 0 2024-04-20T11:09:34.286 INFO:tasks.workunit.client.1.smithi173.stdout:0/374: creat d6/d54/d65/f7c x:0 0 0 2024-04-20T11:09:34.287 INFO:tasks.workunit.client.1.smithi173.stdout:2/456: getdents d2/d13/d46/d7d 0 2024-04-20T11:09:34.289 INFO:tasks.workunit.client.1.smithi173.stdout:2/457: truncate d2/d13/f38 1076314 0 2024-04-20T11:09:34.290 INFO:tasks.workunit.client.1.smithi173.stdout:9/423: rename d5/f4b to d5/d9/d20/d4f/d5e/d7d/f86 0 2024-04-20T11:09:34.293 INFO:tasks.workunit.client.1.smithi173.stdout:6/403: getdents d4/d24/d3e 0 2024-04-20T11:09:34.294 INFO:tasks.workunit.client.1.smithi173.stdout:4/424: truncate da/dc/d13/f47 2021991 0 2024-04-20T11:09:34.295 INFO:tasks.workunit.client.1.smithi173.stdout:6/404: truncate d4/f20 673150 0 2024-04-20T11:09:34.297 INFO:tasks.workunit.client.1.smithi173.stdout:6/405: truncate d4/d18/f58 4273009 0 2024-04-20T11:09:34.298 INFO:tasks.workunit.client.1.smithi173.stdout:2/458: rename d2/d34/d58/d6a to d2/de/d11/d9d 0 2024-04-20T11:09:34.299 INFO:tasks.workunit.client.1.smithi173.stdout:2/459: chown d2/la 1 1 2024-04-20T11:09:34.300 INFO:tasks.workunit.client.1.smithi173.stdout:4/425: rmdir da/d1a/d6e 39 2024-04-20T11:09:34.302 INFO:tasks.workunit.client.1.smithi173.stdout:6/406: mknod d4/d18/d61/c78 0 2024-04-20T11:09:34.303 INFO:tasks.workunit.client.1.smithi173.stdout:2/460: rmdir d2/d8c 39 2024-04-20T11:09:34.304 INFO:tasks.workunit.client.1.smithi173.stdout:4/426: fsync da/dc/d13/f2b 0 2024-04-20T11:09:34.305 INFO:tasks.workunit.client.1.smithi173.stdout:4/427: chown da/dc/d15 885 1 2024-04-20T11:09:34.306 INFO:tasks.workunit.client.1.smithi173.stdout:0/375: dwrite d6/f27 [0,4194304] 0 2024-04-20T11:09:34.309 INFO:tasks.workunit.client.1.smithi173.stdout:9/424: creat d5/d9/d20/d29/d2d/f87 x:0 0 0 2024-04-20T11:09:34.312 INFO:tasks.workunit.client.1.smithi173.stdout:2/461: creat d2/d34/d81/f9e x:0 0 0 2024-04-20T11:09:34.314 INFO:tasks.workunit.client.1.smithi173.stdout:2/462: chown d2/d34/d81/f9e 51 1 2024-04-20T11:09:34.314 INFO:tasks.workunit.client.1.smithi173.stdout:5/466: dwrite da/d34/f47 [0,4194304] 0 2024-04-20T11:09:34.318 INFO:tasks.workunit.client.1.smithi173.stdout:0/376: creat d6/d54/f7d x:0 0 0 2024-04-20T11:09:34.329 INFO:tasks.workunit.client.1.smithi173.stdout:4/428: write da/d48/d53/f5b [1072336,65319] 0 2024-04-20T11:09:34.331 INFO:tasks.workunit.client.1.smithi173.stdout:6/407: dread d4/f11 [0,4194304] 0 2024-04-20T11:09:34.332 INFO:tasks.workunit.client.1.smithi173.stdout:5/467: creat da/de/d19/d2c/fa6 x:0 0 0 2024-04-20T11:09:34.333 INFO:tasks.workunit.client.1.smithi173.stdout:2/463: dwrite d2/d13/f44 [0,4194304] 0 2024-04-20T11:09:34.334 INFO:tasks.workunit.client.1.smithi173.stdout:0/377: rename d6/d8/d3c/f6b to d6/d8/df/d15/f7e 0 2024-04-20T11:09:34.337 INFO:tasks.workunit.client.1.smithi173.stdout:9/425: creat d5/d9/d20/d2b/d59/d44/d78/d82/f88 x:0 0 0 2024-04-20T11:09:34.338 INFO:tasks.workunit.client.1.smithi173.stdout:4/429: readlink da/d48/l72 0 2024-04-20T11:09:34.343 INFO:tasks.workunit.client.1.smithi173.stdout:0/378: mkdir d6/d8/df/d43/d7f 0 2024-04-20T11:09:34.351 INFO:tasks.workunit.client.1.smithi173.stdout:2/464: mkdir d2/d13/d46/d53/d9f 0 2024-04-20T11:09:34.353 INFO:tasks.workunit.client.1.smithi173.stdout:9/426: truncate d5/d9/d20/d29/d2d/f3f 3524070 0 2024-04-20T11:09:34.354 INFO:tasks.workunit.client.1.smithi173.stdout:8/402: sync 2024-04-20T11:09:34.354 INFO:tasks.workunit.client.1.smithi173.stdout:0/379: mknod d6/d8/df/d43/d7f/c80 0 2024-04-20T11:09:34.356 INFO:tasks.workunit.client.1.smithi173.stdout:8/403: chown d4/d18 218058 1 2024-04-20T11:09:34.357 INFO:tasks.workunit.client.1.smithi173.stdout:2/465: rename d2/d13/c79 to d2/de/d11/d9d/ca0 0 2024-04-20T11:09:34.357 INFO:tasks.workunit.client.1.smithi173.stdout:9/427: stat d5/d9/d20/l51 0 2024-04-20T11:09:34.359 INFO:tasks.workunit.client.1.smithi173.stdout:0/380: fsync d6/d8/df/d2c/d42/d50/f76 0 2024-04-20T11:09:34.361 INFO:tasks.workunit.client.1.smithi173.stdout:5/468: dwrite da/f33 [0,4194304] 0 2024-04-20T11:09:34.361 INFO:tasks.workunit.client.1.smithi173.stdout:8/404: mkdir d4/d18/d59/d5a/d7b 0 2024-04-20T11:09:34.363 INFO:tasks.workunit.client.1.smithi173.stdout:6/408: dread d4/d24/d42/d74/d3b/f41 [0,4194304] 0 2024-04-20T11:09:34.364 INFO:tasks.workunit.client.1.smithi173.stdout:2/466: creat d2/d13/d46/d53/d4e/fa1 x:0 0 0 2024-04-20T11:09:34.369 INFO:tasks.workunit.client.1.smithi173.stdout:5/469: mkdir da/d34/da7 0 2024-04-20T11:09:34.369 INFO:tasks.workunit.client.1.smithi173.stdout:0/381: chown d6/d5f/c79 1315274089 1 2024-04-20T11:09:34.369 INFO:tasks.workunit.client.1.smithi173.stdout:4/430: dwrite da/dc/d15/f4e [0,4194304] 0 2024-04-20T11:09:34.371 INFO:tasks.workunit.client.1.smithi173.stdout:4/431: dread - da/d1e/d52/f68 zero size 2024-04-20T11:09:34.374 INFO:tasks.workunit.client.1.smithi173.stdout:6/409: mkdir d4/d14/d79 0 2024-04-20T11:09:34.374 INFO:tasks.workunit.client.1.smithi173.stdout:4/432: chown da/d1e/d52/l5a 31357 1 2024-04-20T11:09:34.375 INFO:tasks.workunit.client.1.smithi173.stdout:6/410: write d4/d24/f54 [250276,92955] 0 2024-04-20T11:09:34.379 INFO:tasks.workunit.client.1.smithi173.stdout:0/382: creat d6/d54/f81 x:0 0 0 2024-04-20T11:09:34.379 INFO:tasks.workunit.client.1.smithi173.stdout:6/411: fsync d4/d24/d42/f72 0 2024-04-20T11:09:34.386 INFO:tasks.workunit.client.1.smithi173.stdout:2/467: creat d2/d8c/fa2 x:0 0 0 2024-04-20T11:09:34.387 INFO:tasks.workunit.client.1.smithi173.stdout:4/433: fsync da/dc/d15/f35 0 2024-04-20T11:09:34.392 INFO:tasks.workunit.client.1.smithi173.stdout:2/468: fdatasync d2/d13/f3d 0 2024-04-20T11:09:34.393 INFO:tasks.workunit.client.1.smithi173.stdout:2/469: chown d2/de/f84 1 1 2024-04-20T11:09:34.393 INFO:tasks.workunit.client.1.smithi173.stdout:0/383: mknod d6/d8/d3c/d77/c82 0 2024-04-20T11:09:34.395 INFO:tasks.workunit.client.1.smithi173.stdout:6/412: dwrite d4/d14/d5f/f76 [0,4194304] 0 2024-04-20T11:09:34.399 INFO:tasks.workunit.client.1.smithi173.stdout:5/470: getdents da/de/d19/d32/d30 0 2024-04-20T11:09:34.400 INFO:tasks.workunit.client.1.smithi173.stdout:6/413: stat d4/d24/c5b 0 2024-04-20T11:09:34.402 INFO:tasks.workunit.client.1.smithi173.stdout:8/405: truncate d4/d18/d4f/f57 1133306 0 2024-04-20T11:09:34.406 INFO:tasks.workunit.client.1.smithi173.stdout:5/471: creat da/de/d19/d2c/fa8 x:0 0 0 2024-04-20T11:09:34.408 INFO:tasks.workunit.client.1.smithi173.stdout:8/406: write d4/f8 [279731,32560] 0 2024-04-20T11:09:34.410 INFO:tasks.workunit.client.1.smithi173.stdout:0/384: getdents d6/d8/d3c/d77 0 2024-04-20T11:09:34.412 INFO:tasks.workunit.client.1.smithi173.stdout:5/472: symlink da/d34/d7f/da5/la9 0 2024-04-20T11:09:34.415 INFO:tasks.workunit.client.1.smithi173.stdout:8/407: fsync d4/d18/d59/f64 0 2024-04-20T11:09:34.415 INFO:tasks.workunit.client.1.smithi173.stdout:4/434: dwrite da/dc/fe [0,4194304] 0 2024-04-20T11:09:34.415 INFO:tasks.workunit.client.1.smithi173.stdout:0/385: write d6/f27 [1132116,57827] 0 2024-04-20T11:09:34.417 INFO:tasks.workunit.client.1.smithi173.stdout:5/473: read f8 [1728420,42557] 0 2024-04-20T11:09:34.417 INFO:tasks.workunit.client.1.smithi173.stdout:8/408: fdatasync d4/d18/f24 0 2024-04-20T11:09:34.423 INFO:tasks.workunit.client.1.smithi173.stdout:6/414: dwrite d4/d18/f62 [0,4194304] 0 2024-04-20T11:09:34.427 INFO:tasks.workunit.client.1.smithi173.stdout:8/409: symlink d4/d18/d59/d6d/l7c 0 2024-04-20T11:09:34.429 INFO:tasks.workunit.client.1.smithi173.stdout:1/373: sync 2024-04-20T11:09:34.429 INFO:tasks.workunit.client.1.smithi173.stdout:3/443: sync 2024-04-20T11:09:34.431 INFO:tasks.workunit.client.1.smithi173.stdout:5/474: rename da/d4f/d65/f74 to da/de/d75/d3e/faa 0 2024-04-20T11:09:34.437 INFO:tasks.workunit.client.1.smithi173.stdout:6/415: symlink d4/d1c/l7a 0 2024-04-20T11:09:34.437 INFO:tasks.workunit.client.1.smithi173.stdout:6/416: fsync d4/d24/d42/f67 0 2024-04-20T11:09:34.438 INFO:tasks.workunit.client.1.smithi173.stdout:1/374: fdatasync d5/d6/dc/fe 0 2024-04-20T11:09:34.440 INFO:tasks.workunit.client.1.smithi173.stdout:4/435: dread da/dc/fe [0,4194304] 0 2024-04-20T11:09:34.440 INFO:tasks.workunit.client.1.smithi173.stdout:8/410: rename d4/d18/d3a/d3c/d5f/f61 to d4/d18/d4f/f7d 0 2024-04-20T11:09:34.441 INFO:tasks.workunit.client.1.smithi173.stdout:3/444: mkdir d7/d2b/d8e 0 2024-04-20T11:09:34.442 INFO:tasks.workunit.client.1.smithi173.stdout:0/386: dwrite d6/d24/f4f [0,4194304] 0 2024-04-20T11:09:34.443 INFO:tasks.workunit.client.1.smithi173.stdout:2/470: dread d2/d13/d28/f7c [0,4194304] 0 2024-04-20T11:09:34.445 INFO:tasks.workunit.client.1.smithi173.stdout:7/430: sync 2024-04-20T11:09:34.445 INFO:tasks.workunit.client.1.smithi173.stdout:9/428: sync 2024-04-20T11:09:34.446 INFO:tasks.workunit.client.1.smithi173.stdout:6/417: mknod d4/d59/c7b 0 2024-04-20T11:09:34.446 INFO:tasks.workunit.client.1.smithi173.stdout:0/387: dread - d6/d8/df/d2c/d42/f62 zero size 2024-04-20T11:09:34.446 INFO:tasks.workunit.client.1.smithi173.stdout:7/431: readlink l7 0 2024-04-20T11:09:34.448 INFO:tasks.workunit.client.1.smithi173.stdout:8/411: readlink d4/d18/l74 0 2024-04-20T11:09:34.451 INFO:tasks.workunit.client.1.smithi173.stdout:9/429: chown d5/f2f 321 1 2024-04-20T11:09:34.451 INFO:tasks.workunit.client.1.smithi173.stdout:2/471: symlink d2/de/d11/la3 0 2024-04-20T11:09:34.452 INFO:tasks.workunit.client.1.smithi173.stdout:6/418: fdatasync d4/d18/f3f 0 2024-04-20T11:09:34.453 INFO:tasks.workunit.client.1.smithi173.stdout:3/445: write d7/d9/f28 [1062846,91540] 0 2024-04-20T11:09:34.454 INFO:tasks.workunit.client.1.smithi173.stdout:0/388: write d6/d8/df/d2c/d42/f7a [987596,102315] 0 2024-04-20T11:09:34.457 INFO:tasks.workunit.client.1.smithi173.stdout:3/446: fsync d7/d53/f71 0 2024-04-20T11:09:34.458 INFO:tasks.workunit.client.1.smithi173.stdout:6/419: creat d4/d24/d42/f7c x:0 0 0 2024-04-20T11:09:34.460 INFO:tasks.workunit.client.1.smithi173.stdout:0/389: truncate d6/f29 2835611 0 2024-04-20T11:09:34.462 INFO:tasks.workunit.client.1.smithi173.stdout:0/390: chown d6/d8/df/d43/d7f/c80 554 1 2024-04-20T11:09:34.465 INFO:tasks.workunit.client.1.smithi173.stdout:4/436: dread da/dc/d15/f4e [0,4194304] 0 2024-04-20T11:09:34.466 INFO:tasks.workunit.client.1.smithi173.stdout:6/420: readlink d4/d1c/l32 0 2024-04-20T11:09:34.467 INFO:tasks.workunit.client.1.smithi173.stdout:5/475: truncate da/de/f24 2337993 0 2024-04-20T11:09:34.470 INFO:tasks.workunit.client.1.smithi173.stdout:2/472: dread d2/d34/d58/f5a [0,4194304] 0 2024-04-20T11:09:34.471 INFO:tasks.workunit.client.1.smithi173.stdout:1/375: truncate d5/d6/d17/d42/d12/d1f/f2f 2744570 0 2024-04-20T11:09:34.472 INFO:tasks.workunit.client.1.smithi173.stdout:4/437: creat da/d1a/d60/f8c x:0 0 0 2024-04-20T11:09:34.473 INFO:tasks.workunit.client.1.smithi173.stdout:7/432: getdents d9/d33/d35/d4a/d52 0 2024-04-20T11:09:34.474 INFO:tasks.workunit.client.1.smithi173.stdout:4/438: chown da/d1e/d52/c54 0 1 2024-04-20T11:09:34.477 INFO:tasks.workunit.client.1.smithi173.stdout:3/447: write d7/d17/d61/d83/f38 [932690,95851] 0 2024-04-20T11:09:34.479 INFO:tasks.workunit.client.1.smithi173.stdout:9/430: getdents d5/d9/d20/d2b/d31 0 2024-04-20T11:09:34.484 INFO:tasks.workunit.client.1.smithi173.stdout:2/473: rmdir d2/d13/d46/d4b 39 2024-04-20T11:09:34.486 INFO:tasks.workunit.client.1.smithi173.stdout:1/376: rmdir d5/d6/d17/d42/d12/d1c 39 2024-04-20T11:09:34.487 INFO:tasks.workunit.client.1.smithi173.stdout:7/433: mkdir d9/d33/d91 0 2024-04-20T11:09:34.487 INFO:tasks.workunit.client.1.smithi173.stdout:1/377: fsync d5/d6/d17/d42/da/f6a 0 2024-04-20T11:09:34.489 INFO:tasks.workunit.client.1.smithi173.stdout:7/434: read d9/da/f12 [294614,114778] 0 2024-04-20T11:09:34.489 INFO:tasks.workunit.client.1.smithi173.stdout:4/439: symlink da/dc/l8d 0 2024-04-20T11:09:34.492 INFO:tasks.workunit.client.1.smithi173.stdout:1/378: truncate d5/d6/d3b/f67 875596 0 2024-04-20T11:09:34.492 INFO:tasks.workunit.client.1.smithi173.stdout:6/421: creat d4/d1c/f7d x:0 0 0 2024-04-20T11:09:34.494 INFO:tasks.workunit.client.1.smithi173.stdout:8/412: dread d4/f19 [0,4194304] 0 2024-04-20T11:09:34.495 INFO:tasks.workunit.client.1.smithi173.stdout:5/476: write da/de/d19/d32/f2f [96624,71935] 0 2024-04-20T11:09:34.496 INFO:tasks.workunit.client.1.smithi173.stdout:0/391: dread d6/d8/f1f [0,4194304] 0 2024-04-20T11:09:34.497 INFO:tasks.workunit.client.1.smithi173.stdout:7/435: symlink d9/d33/l92 0 2024-04-20T11:09:34.499 INFO:tasks.workunit.client.1.smithi173.stdout:4/440: mkdir da/dc/d15/d8e 0 2024-04-20T11:09:34.501 INFO:tasks.workunit.client.1.smithi173.stdout:9/431: symlink d5/d9/d20/d29/d2d/l89 0 2024-04-20T11:09:34.502 INFO:tasks.workunit.client.1.smithi173.stdout:6/422: mkdir d4/d18/d7e 0 2024-04-20T11:09:34.502 INFO:tasks.workunit.client.1.smithi173.stdout:2/474: dwrite d2/de/d11/d9d/d87/f97 [0,4194304] 0 2024-04-20T11:09:34.502 INFO:tasks.workunit.client.1.smithi173.stdout:4/441: dread - da/d1e/d52/f71 zero size 2024-04-20T11:09:34.503 INFO:tasks.workunit.client.1.smithi173.stdout:4/442: readlink da/l34 0 2024-04-20T11:09:34.506 INFO:tasks.workunit.client.1.smithi173.stdout:4/443: fsync da/d1a/f40 0 2024-04-20T11:09:34.509 INFO:tasks.workunit.client.1.smithi173.stdout:3/448: link d7/f25 d7/d53/f8f 0 2024-04-20T11:09:34.509 INFO:tasks.workunit.client.1.smithi173.stdout:4/444: stat da/dc/d13 0 2024-04-20T11:09:34.510 INFO:tasks.workunit.client.1.smithi173.stdout:6/423: mknod d4/d18/d61/c7f 0 2024-04-20T11:09:34.511 INFO:tasks.workunit.client.1.smithi173.stdout:8/413: chown d4/d18/d3a/d3c/f53 819 1 2024-04-20T11:09:34.513 INFO:tasks.workunit.client.1.smithi173.stdout:9/432: unlink d5/d9/d20/d2b/l65 0 2024-04-20T11:09:34.513 INFO:tasks.workunit.client.1.smithi173.stdout:6/424: chown d4/d18/d61/c78 3646974 1 2024-04-20T11:09:34.515 INFO:tasks.workunit.client.1.smithi173.stdout:1/379: dwrite d5/d6/d17/d42/d12/d1f/f63 [0,4194304] 0 2024-04-20T11:09:34.515 INFO:tasks.workunit.client.1.smithi173.stdout:6/425: chown d4/l49 239901 1 2024-04-20T11:09:34.517 INFO:tasks.workunit.client.1.smithi173.stdout:3/449: mkdir d7/d17/d61/d90 0 2024-04-20T11:09:34.517 INFO:tasks.workunit.client.1.smithi173.stdout:0/392: write d6/d8/df/d2c/d42/f72 [39682,42683] 0 2024-04-20T11:09:34.520 INFO:tasks.workunit.client.1.smithi173.stdout:5/477: creat da/de/d75/fab x:0 0 0 2024-04-20T11:09:34.526 INFO:tasks.workunit.client.1.smithi173.stdout:7/436: dwrite d9/d33/d40/f49 [0,4194304] 0 2024-04-20T11:09:34.527 INFO:tasks.workunit.client.1.smithi173.stdout:3/450: unlink d7/d9/f28 0 2024-04-20T11:09:34.566 INFO:tasks.workunit.client.1.smithi173.stdout:0/393: unlink d6/d8/df/d2c/d42/f62 0 2024-04-20T11:09:34.571 INFO:tasks.workunit.client.1.smithi173.stdout:2/475: truncate d2/d34/f56 1238705 0 2024-04-20T11:09:34.576 INFO:tasks.workunit.client.1.smithi173.stdout:3/451: mkdir d7/d2b/d8e/d91 0 2024-04-20T11:09:34.580 INFO:tasks.workunit.client.1.smithi173.stdout:2/476: write d2/d13/f50 [3910785,51625] 0 2024-04-20T11:09:34.581 INFO:tasks.workunit.client.1.smithi173.stdout:9/433: dwrite d5/d9/f1d [0,4194304] 0 2024-04-20T11:09:34.581 INFO:tasks.workunit.client.1.smithi173.stdout:5/478: dwrite da/fc [0,4194304] 0 2024-04-20T11:09:34.585 INFO:tasks.workunit.client.1.smithi173.stdout:4/445: getdents da 0 2024-04-20T11:09:34.586 INFO:tasks.workunit.client.1.smithi173.stdout:1/380: dwrite d5/f30 [0,4194304] 0 2024-04-20T11:09:34.591 INFO:tasks.workunit.client.1.smithi173.stdout:2/477: write d2/de/d11/f33 [263778,120426] 0 2024-04-20T11:09:34.591 INFO:tasks.workunit.client.1.smithi173.stdout:1/381: readlink d5/d6/d17/l3d 0 2024-04-20T11:09:34.593 INFO:tasks.workunit.client.1.smithi173.stdout:7/437: dread d9/d33/d35/d58/f70 [0,4194304] 0 2024-04-20T11:09:34.596 INFO:tasks.workunit.client.1.smithi173.stdout:7/438: chown d9/d33/d35/d58 0 1 2024-04-20T11:09:34.597 INFO:tasks.workunit.client.1.smithi173.stdout:8/414: dread d4/d18/f5c [4194304,4194304] 0 2024-04-20T11:09:34.597 INFO:tasks.workunit.client.1.smithi173.stdout:3/452: rename d7/d9/d5c/l64 to d7/d9/d5c/l92 0 2024-04-20T11:09:34.600 INFO:tasks.workunit.client.1.smithi173.stdout:0/394: truncate d6/f7 4340858 0 2024-04-20T11:09:34.602 INFO:tasks.workunit.client.1.smithi173.stdout:0/395: chown d6/d24/c31 4990582 1 2024-04-20T11:09:34.604 INFO:tasks.workunit.client.1.smithi173.stdout:2/478: rmdir d2/d13/d46/d53 39 2024-04-20T11:09:34.606 INFO:tasks.workunit.client.1.smithi173.stdout:0/396: write d6/d54/d65/f73 [85430,2865] 0 2024-04-20T11:09:34.609 INFO:tasks.workunit.client.1.smithi173.stdout:2/479: write d2/d34/d81/f9e [902631,13052] 0 2024-04-20T11:09:34.611 INFO:tasks.workunit.client.1.smithi173.stdout:1/382: dwrite d5/d6/dc/f3a [0,4194304] 0 2024-04-20T11:09:34.612 INFO:tasks.workunit.client.1.smithi173.stdout:3/453: creat d7/d17/f93 x:0 0 0 2024-04-20T11:09:34.614 INFO:tasks.workunit.client.1.smithi173.stdout:8/415: creat d4/d18/d4f/f7e x:0 0 0 2024-04-20T11:09:34.617 INFO:tasks.workunit.client.1.smithi173.stdout:7/439: dwrite d9/da/f14 [0,4194304] 0 2024-04-20T11:09:34.617 INFO:tasks.workunit.client.1.smithi173.stdout:8/416: fdatasync d4/d18/d3a/d3c/f52 0 2024-04-20T11:09:34.620 INFO:tasks.workunit.client.1.smithi173.stdout:5/479: creat da/de/fac x:0 0 0 2024-04-20T11:09:34.620 INFO:tasks.workunit.client.1.smithi173.stdout:7/440: chown d9/d33/d35/d56/c6c 6854 1 2024-04-20T11:09:34.620 INFO:tasks.workunit.client.1.smithi173.stdout:5/480: readlink da/l82 0 2024-04-20T11:09:34.620 INFO:tasks.workunit.client.1.smithi173.stdout:6/426: sync 2024-04-20T11:09:34.621 INFO:tasks.workunit.client.1.smithi173.stdout:0/397: creat d6/d8/df/d15/f83 x:0 0 0 2024-04-20T11:09:34.623 INFO:tasks.workunit.client.1.smithi173.stdout:2/480: mknod d2/d34/d58/ca4 0 2024-04-20T11:09:34.624 INFO:tasks.workunit.client.1.smithi173.stdout:6/427: write d4/d14/f43 [618582,58098] 0 2024-04-20T11:09:34.627 INFO:tasks.workunit.client.1.smithi173.stdout:9/434: mkdir d5/d9/d20/d29/d18/d8a 0 2024-04-20T11:09:34.633 INFO:tasks.workunit.client.1.smithi173.stdout:1/383: read - d5/d6/d17/d42/d12/d1c/f5b zero size 2024-04-20T11:09:34.635 INFO:tasks.workunit.client.1.smithi173.stdout:7/441: unlink d9/da/d31/c39 0 2024-04-20T11:09:34.637 INFO:tasks.workunit.client.1.smithi173.stdout:7/442: dread - d9/d33/d40/d7a/f8c zero size 2024-04-20T11:09:34.638 INFO:tasks.workunit.client.1.smithi173.stdout:2/481: mkdir d2/d13/da5 0 2024-04-20T11:09:34.638 INFO:tasks.workunit.client.1.smithi173.stdout:5/481: dwrite da/de/fac [0,4194304] 0 2024-04-20T11:09:34.652 INFO:tasks.workunit.client.1.smithi173.stdout:1/384: dwrite f4 [0,4194304] 0 2024-04-20T11:09:34.653 INFO:tasks.workunit.client.1.smithi173.stdout:7/443: rename d9/d33/d35/d4a/f4c to d9/d33/d35/d4a/f93 0 2024-04-20T11:09:34.654 INFO:tasks.workunit.client.1.smithi173.stdout:4/446: getdents da 0 2024-04-20T11:09:34.657 INFO:tasks.workunit.client.1.smithi173.stdout:7/444: fdatasync d9/da/d17/f78 0 2024-04-20T11:09:34.659 INFO:tasks.workunit.client.1.smithi173.stdout:3/454: getdents d7/d17/d58 0 2024-04-20T11:09:34.660 INFO:tasks.workunit.client.1.smithi173.stdout:7/445: readlink d9/d33/l92 0 2024-04-20T11:09:34.663 INFO:tasks.workunit.client.1.smithi173.stdout:5/482: dwrite da/de/d19/d2c/fa8 [0,4194304] 0 2024-04-20T11:09:34.666 INFO:tasks.workunit.client.1.smithi173.stdout:1/385: symlink d5/d6/d17/d42/da/d20/d38/l80 0 2024-04-20T11:09:34.666 INFO:tasks.workunit.client.1.smithi173.stdout:5/483: fdatasync da/de/d19/d32/d72/f91 0 2024-04-20T11:09:34.668 INFO:tasks.workunit.client.1.smithi173.stdout:4/447: creat da/dc/d13/d65/f8f x:0 0 0 2024-04-20T11:09:34.669 INFO:tasks.workunit.client.1.smithi173.stdout:8/417: dread d4/d18/d3a/d3c/f37 [0,4194304] 0 2024-04-20T11:09:34.669 INFO:tasks.workunit.client.1.smithi173.stdout:9/435: link d5/d9/d20/d4f/l66 d5/d9/l8b 0 2024-04-20T11:09:34.672 INFO:tasks.workunit.client.1.smithi173.stdout:4/448: chown da/d1e/d52/f68 59 1 2024-04-20T11:09:34.678 INFO:tasks.workunit.client.1.smithi173.stdout:4/449: readlink da/d48/d53/l4f 0 2024-04-20T11:09:34.682 INFO:tasks.workunit.client.1.smithi173.stdout:7/446: getdents d9/d33/d35/d56/d5a 0 2024-04-20T11:09:34.693 INFO:tasks.workunit.client.1.smithi173.stdout:8/418: dread d4/d18/d3a/f46 [0,4194304] 0 2024-04-20T11:09:34.693 INFO:tasks.workunit.client.1.smithi173.stdout:2/482: dread d2/de/d11/f33 [0,4194304] 0 2024-04-20T11:09:34.696 INFO:tasks.workunit.client.1.smithi173.stdout:4/450: creat da/d1a/d60/f90 x:0 0 0 2024-04-20T11:09:34.697 INFO:tasks.workunit.client.1.smithi173.stdout:7/447: rmdir d9/d1f/d7f 39 2024-04-20T11:09:34.706 INFO:tasks.workunit.client.1.smithi173.stdout:0/398: truncate d6/d8/df/d2c/d42/f72 1767322 0 2024-04-20T11:09:34.713 INFO:tasks.workunit.client.1.smithi173.stdout:4/451: fsync da/dc/d13/f29 0 2024-04-20T11:09:34.716 INFO:tasks.workunit.client.1.smithi173.stdout:0/399: fdatasync d6/d24/f4f 0 2024-04-20T11:09:34.718 INFO:tasks.workunit.client.1.smithi173.stdout:2/483: unlink d2/d13/d28/d4a/f4d 0 2024-04-20T11:09:34.718 INFO:tasks.workunit.client.1.smithi173.stdout:1/386: dwrite d5/d6/f9 [4194304,4194304] 0 2024-04-20T11:09:34.721 INFO:tasks.workunit.client.1.smithi173.stdout:8/419: mknod d4/d18/d3a/c7f 0 2024-04-20T11:09:34.721 INFO:tasks.workunit.client.1.smithi173.stdout:5/484: dwrite da/de/d75/d40/d59/f95 [4194304,4194304] 0 2024-04-20T11:09:34.728 INFO:tasks.workunit.client.1.smithi173.stdout:3/455: dwrite d7/d17/d61/f8b [0,4194304] 0 2024-04-20T11:09:34.728 INFO:tasks.workunit.client.1.smithi173.stdout:7/448: write d9/da/f25 [985314,31583] 0 2024-04-20T11:09:34.733 INFO:tasks.workunit.client.1.smithi173.stdout:9/436: dwrite d5/d9/d20/f85 [0,4194304] 0 2024-04-20T11:09:34.739 INFO:tasks.workunit.client.1.smithi173.stdout:0/400: creat d6/d8/df/d2c/d42/f84 x:0 0 0 2024-04-20T11:09:34.739 INFO:tasks.workunit.client.1.smithi173.stdout:4/452: unlink da/dc/d13/l59 0 2024-04-20T11:09:34.740 INFO:tasks.workunit.client.1.smithi173.stdout:3/456: write d7/d17/f2c [1392687,62020] 0 2024-04-20T11:09:34.741 INFO:tasks.workunit.client.1.smithi173.stdout:1/387: dread - d5/d6/d17/d42/d12/d1c/f6f zero size 2024-04-20T11:09:34.746 INFO:tasks.workunit.client.1.smithi173.stdout:6/428: dwrite d4/f16 [0,4194304] 0 2024-04-20T11:09:34.748 INFO:tasks.workunit.client.1.smithi173.stdout:6/429: stat d4/f20 0 2024-04-20T11:09:34.750 INFO:tasks.workunit.client.1.smithi173.stdout:2/484: symlink d2/de/d11/d9d/d87/la6 0 2024-04-20T11:09:34.751 INFO:tasks.workunit.client.1.smithi173.stdout:1/388: stat d5/d6/d17/d42/da/df/c70 0 2024-04-20T11:09:34.753 INFO:tasks.workunit.client.1.smithi173.stdout:9/437: symlink d5/d9/d20/l8c 0 2024-04-20T11:09:34.760 INFO:tasks.workunit.client.1.smithi173.stdout:4/453: creat da/d48/f91 x:0 0 0 2024-04-20T11:09:34.761 INFO:tasks.workunit.client.1.smithi173.stdout:0/401: symlink d6/d8/d3c/d44/d75/l85 0 2024-04-20T11:09:34.763 INFO:tasks.workunit.client.1.smithi173.stdout:3/457: readlink d7/d2b/l4c 0 2024-04-20T11:09:34.769 INFO:tasks.workunit.client.1.smithi173.stdout:1/389: mkdir d5/d6/d17/d42/da/d20/d48/d81 0 2024-04-20T11:09:34.772 INFO:tasks.workunit.client.1.smithi173.stdout:0/402: chown d6/d8/df/c3b 123417 1 2024-04-20T11:09:34.772 INFO:tasks.workunit.client.1.smithi173.stdout:2/485: fsync d2/d13/d46/d53/d4e/f68 0 2024-04-20T11:09:34.773 INFO:tasks.workunit.client.1.smithi173.stdout:6/430: creat d4/d18/d7e/f80 x:0 0 0 2024-04-20T11:09:34.779 INFO:tasks.workunit.client.1.smithi173.stdout:5/485: dwrite da/de/d19/d2c/fa8 [4194304,4194304] 0 2024-04-20T11:09:34.781 INFO:tasks.workunit.client.1.smithi173.stdout:9/438: dread d5/d9/d20/d29/d18/f70 [0,4194304] 0 2024-04-20T11:09:34.783 INFO:tasks.workunit.client.1.smithi173.stdout:2/486: write d2/d8c/fa2 [683765,76963] 0 2024-04-20T11:09:34.784 INFO:tasks.workunit.client.1.smithi173.stdout:8/420: dwrite d4/f32 [4194304,4194304] 0 2024-04-20T11:09:34.789 INFO:tasks.workunit.client.1.smithi173.stdout:9/439: fdatasync d5/d9/d20/d29/d2d/d33/f6b 0 2024-04-20T11:09:34.789 INFO:tasks.workunit.client.1.smithi173.stdout:5/486: stat da/de/d75/f1d 0 2024-04-20T11:09:34.794 INFO:tasks.workunit.client.1.smithi173.stdout:6/431: truncate d4/d24/d42/d74/d3b/f52 649107 0 2024-04-20T11:09:34.794 INFO:tasks.workunit.client.1.smithi173.stdout:5/487: fsync da/d4f/d65/d8e/f9d 0 2024-04-20T11:09:34.798 INFO:tasks.workunit.client.1.smithi173.stdout:6/432: rmdir d4/d24/d42/d74 39 2024-04-20T11:09:34.802 INFO:tasks.workunit.client.1.smithi173.stdout:3/458: creat d7/f94 x:0 0 0 2024-04-20T11:09:34.806 INFO:tasks.workunit.client.1.smithi173.stdout:6/433: chown d4/d24/d42/d74/c5c 7 1 2024-04-20T11:09:34.806 INFO:tasks.workunit.client.1.smithi173.stdout:3/459: chown d7/f22 0 1 2024-04-20T11:09:34.808 INFO:tasks.workunit.client.1.smithi173.stdout:0/403: dwrite d6/d8/d3c/d44/d75/f7b [0,4194304] 0 2024-04-20T11:09:34.808 INFO:tasks.workunit.client.1.smithi173.stdout:1/390: dwrite d5/d6/d17/d42/da/df/d40/f5d [0,4194304] 0 2024-04-20T11:09:34.812 INFO:tasks.workunit.client.1.smithi173.stdout:2/487: creat d2/de/fa7 x:0 0 0 2024-04-20T11:09:34.812 INFO:tasks.workunit.client.1.smithi173.stdout:0/404: dread - d6/d54/d65/f7c zero size 2024-04-20T11:09:34.816 INFO:tasks.workunit.client.1.smithi173.stdout:8/421: getdents d4/d18 0 2024-04-20T11:09:34.821 INFO:tasks.workunit.client.1.smithi173.stdout:8/422: stat d4/d18/f24 0 2024-04-20T11:09:34.822 INFO:tasks.workunit.client.1.smithi173.stdout:1/391: creat d5/d6/d17/d42/d12/d1c/f82 x:0 0 0 2024-04-20T11:09:34.825 INFO:tasks.workunit.client.1.smithi173.stdout:9/440: dwrite d5/d4c/f72 [0,4194304] 0 2024-04-20T11:09:34.828 INFO:tasks.workunit.client.1.smithi173.stdout:9/441: chown d5/d9/d20/d29/d2d/f87 457041626 1 2024-04-20T11:09:34.829 INFO:tasks.workunit.client.1.smithi173.stdout:0/405: mkdir d6/d8/df/d2c/d42/d50/d86 0 2024-04-20T11:09:34.830 INFO:tasks.workunit.client.1.smithi173.stdout:6/434: mkdir d4/d24/d42/d81 0 2024-04-20T11:09:34.830 INFO:tasks.workunit.client.1.smithi173.stdout:3/460: creat d7/d17/d58/d7f/f95 x:0 0 0 2024-04-20T11:09:34.834 INFO:tasks.workunit.client.1.smithi173.stdout:2/488: unlink d2/d13/c37 0 2024-04-20T11:09:34.841 INFO:tasks.workunit.client.1.smithi173.stdout:6/435: readlink d4/l15 0 2024-04-20T11:09:34.843 INFO:tasks.workunit.client.1.smithi173.stdout:3/461: mknod d7/d53/c96 0 2024-04-20T11:09:34.847 INFO:tasks.workunit.client.1.smithi173.stdout:1/392: creat d5/d6/dc/f83 x:0 0 0 2024-04-20T11:09:34.848 INFO:tasks.workunit.client.1.smithi173.stdout:4/454: truncate da/d48/d53/f37 145332 0 2024-04-20T11:09:34.850 INFO:tasks.workunit.client.1.smithi173.stdout:4/455: write da/dc/f33 [420770,124367] 0 2024-04-20T11:09:34.850 INFO:tasks.workunit.client.1.smithi173.stdout:2/489: dwrite d2/de/d11/d9d/d87/f97 [0,4194304] 0 2024-04-20T11:09:34.851 INFO:tasks.workunit.client.1.smithi173.stdout:6/436: mkdir d4/d18/d7e/d82 0 2024-04-20T11:09:34.856 INFO:tasks.workunit.client.1.smithi173.stdout:3/462: readlink d7/d9/l12 0 2024-04-20T11:09:34.858 INFO:tasks.workunit.client.1.smithi173.stdout:3/463: dread - d7/d17/f93 zero size 2024-04-20T11:09:34.864 INFO:tasks.workunit.client.1.smithi173.stdout:7/449: sync 2024-04-20T11:09:34.865 INFO:tasks.workunit.client.1.smithi173.stdout:6/437: mkdir d4/d59/d83 0 2024-04-20T11:09:34.866 INFO:tasks.workunit.client.1.smithi173.stdout:9/442: write d5/f38 [2340384,97634] 0 2024-04-20T11:09:34.867 INFO:tasks.workunit.client.1.smithi173.stdout:6/438: readlink d4/d1c/l7a 0 2024-04-20T11:09:34.868 INFO:tasks.workunit.client.1.smithi173.stdout:1/393: rename d5/d6/d17/d42/da/c24 to d5/d6/d17/d42/da/d20/d38/d52/d75/c84 0 2024-04-20T11:09:34.872 INFO:tasks.workunit.client.1.smithi173.stdout:2/490: dwrite d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:34.873 INFO:tasks.workunit.client.1.smithi173.stdout:4/456: symlink da/d48/d53/d39/l92 0 2024-04-20T11:09:34.874 INFO:tasks.workunit.client.1.smithi173.stdout:0/406: truncate d6/d8/df/d2c/d42/f56 4331780 0 2024-04-20T11:09:34.874 INFO:tasks.workunit.client.1.smithi173.stdout:1/394: stat d5/d6/d17/d42/da/df/c70 0 2024-04-20T11:09:34.877 INFO:tasks.workunit.client.1.smithi173.stdout:8/423: dwrite d4/f10 [0,4194304] 0 2024-04-20T11:09:34.877 INFO:tasks.workunit.client.1.smithi173.stdout:4/457: chown da/d1e/d52/c78 2712 1 2024-04-20T11:09:34.877 INFO:tasks.workunit.client.1.smithi173.stdout:1/395: chown d5/d6/d17/d42/f39 52960 1 2024-04-20T11:09:34.903 INFO:tasks.workunit.client.1.smithi173.stdout:1/396: write d5/d6/d17/d42/da/f6a [1545575,15757] 0 2024-04-20T11:09:34.903 INFO:tasks.workunit.client.1.smithi173.stdout:2/491: read d2/d13/d46/d53/d4e/f68 [1754487,70446] 0 2024-04-20T11:09:34.905 INFO:tasks.workunit.client.1.smithi173.stdout:3/464: creat d7/d17/d61/f97 x:0 0 0 2024-04-20T11:09:34.905 INFO:tasks.workunit.client.1.smithi173.stdout:7/450: symlink d9/d1f/l94 0 2024-04-20T11:09:34.907 INFO:tasks.workunit.client.1.smithi173.stdout:4/458: mknod da/c93 0 2024-04-20T11:09:34.909 INFO:tasks.workunit.client.1.smithi173.stdout:9/443: creat d5/d4c/d79/d80/f8d x:0 0 0 2024-04-20T11:09:34.912 INFO:tasks.workunit.client.1.smithi173.stdout:0/407: mknod d6/d8/df/d2c/c87 0 2024-04-20T11:09:34.913 INFO:tasks.workunit.client.1.smithi173.stdout:2/492: symlink d2/d13/la8 0 2024-04-20T11:09:34.916 INFO:tasks.workunit.client.1.smithi173.stdout:2/493: fdatasync d2/d34/d58/f6b 0 2024-04-20T11:09:34.916 INFO:tasks.workunit.client.1.smithi173.stdout:8/424: dwrite d4/d18/d3a/d3c/f34 [0,4194304] 0 2024-04-20T11:09:34.918 INFO:tasks.workunit.client.1.smithi173.stdout:8/425: chown d4/d18/d3a/d3c/l4d 8243135 1 2024-04-20T11:09:34.923 INFO:tasks.workunit.client.1.smithi173.stdout:7/451: creat d9/d33/d81/f95 x:0 0 0 2024-04-20T11:09:34.931 INFO:tasks.workunit.client.1.smithi173.stdout:1/397: truncate d5/d6/d17/d42/da/f14 1215852 0 2024-04-20T11:09:34.932 INFO:tasks.workunit.client.1.smithi173.stdout:9/444: symlink d5/d9/d20/d29/l8e 0 2024-04-20T11:09:34.933 INFO:tasks.workunit.client.1.smithi173.stdout:2/494: dread d2/de/d11/d9d/f8e [0,4194304] 0 2024-04-20T11:09:34.933 INFO:tasks.workunit.client.1.smithi173.stdout:9/445: dread - d5/d4c/f81 zero size 2024-04-20T11:09:34.935 INFO:tasks.workunit.client.1.smithi173.stdout:2/495: fdatasync d2/d13/d46/d53/d4e/f68 0 2024-04-20T11:09:34.935 INFO:tasks.workunit.client.1.smithi173.stdout:0/408: symlink d6/d8/df/l88 0 2024-04-20T11:09:34.935 INFO:tasks.workunit.client.1.smithi173.stdout:9/446: dread - d5/d9/d20/d4f/f68 zero size 2024-04-20T11:09:34.937 INFO:tasks.workunit.client.1.smithi173.stdout:2/496: chown d2/de/d11/l25 4 1 2024-04-20T11:09:34.937 INFO:tasks.workunit.client.1.smithi173.stdout:9/447: stat d5/d9/d20/d29/d18/f1f 0 2024-04-20T11:09:34.941 INFO:tasks.workunit.client.1.smithi173.stdout:3/465: truncate d7/d17/f20 619147 0 2024-04-20T11:09:34.943 INFO:tasks.workunit.client.1.smithi173.stdout:4/459: write da/dc/d13/f47 [284513,127875] 0 2024-04-20T11:09:34.947 INFO:tasks.workunit.client.1.smithi173.stdout:7/452: truncate f6 4713917 0 2024-04-20T11:09:34.948 INFO:tasks.workunit.client.1.smithi173.stdout:4/460: truncate da/d1e/d52/f5c 253945 0 2024-04-20T11:09:34.949 INFO:tasks.workunit.client.1.smithi173.stdout:8/426: link d4/d18/c50 d4/d18/c80 0 2024-04-20T11:09:34.949 INFO:tasks.workunit.client.1.smithi173.stdout:5/488: sync 2024-04-20T11:09:34.953 INFO:tasks.workunit.client.1.smithi173.stdout:4/461: write da/d48/d53/f5b [2425,56476] 0 2024-04-20T11:09:34.958 INFO:tasks.workunit.client.1.smithi173.stdout:2/497: dwrite d2/de/d11/d9d/d87/f97 [4194304,4194304] 0 2024-04-20T11:09:34.958 INFO:tasks.workunit.client.1.smithi173.stdout:9/448: symlink d5/d7f/l8f 0 2024-04-20T11:09:34.961 INFO:tasks.workunit.client.1.smithi173.stdout:2/498: stat d2/d34/f3a 0 2024-04-20T11:09:34.962 INFO:tasks.workunit.client.1.smithi173.stdout:0/409: truncate d6/f27 3372064 0 2024-04-20T11:09:34.966 INFO:tasks.workunit.client.1.smithi173.stdout:3/466: dread d7/d17/f3c [0,4194304] 0 2024-04-20T11:09:34.970 INFO:tasks.workunit.client.1.smithi173.stdout:1/398: dwrite d5/d6/d17/d42/d12/f15 [0,4194304] 0 2024-04-20T11:09:34.974 INFO:tasks.workunit.client.1.smithi173.stdout:0/410: creat d6/d8/d3c/f89 x:0 0 0 2024-04-20T11:09:34.976 INFO:tasks.workunit.client.1.smithi173.stdout:4/462: mknod da/d1a/d6e/c94 0 2024-04-20T11:09:34.976 INFO:tasks.workunit.client.1.smithi173.stdout:1/399: readlink d5/d6/d17/d42/d12/d1f/l68 0 2024-04-20T11:09:34.978 INFO:tasks.workunit.client.1.smithi173.stdout:4/463: readlink da/d48/d53/l4f 0 2024-04-20T11:09:34.980 INFO:tasks.workunit.client.1.smithi173.stdout:3/467: dread - d7/d17/d61/d83/d45/f56 zero size 2024-04-20T11:09:34.981 INFO:tasks.workunit.client.1.smithi173.stdout:8/427: getdents d4/d18/d4f/d5b 0 2024-04-20T11:09:34.982 INFO:tasks.workunit.client.1.smithi173.stdout:6/439: sync 2024-04-20T11:09:34.985 INFO:tasks.workunit.client.1.smithi173.stdout:4/464: truncate da/dc/d13/d65/f8f 676996 0 2024-04-20T11:09:34.985 INFO:tasks.workunit.client.1.smithi173.stdout:0/411: creat d6/d8/df/d2c/d2d/f8a x:0 0 0 2024-04-20T11:09:34.985 INFO:tasks.workunit.client.1.smithi173.stdout:5/489: truncate da/f3d 2364987 0 2024-04-20T11:09:34.985 INFO:tasks.workunit.client.1.smithi173.stdout:7/453: dread d9/d33/d35/d4a/d52/d5b/f6d [0,4194304] 0 2024-04-20T11:09:34.986 INFO:tasks.workunit.client.1.smithi173.stdout:9/449: dwrite d5/d9/d20/d2b/d59/d44/f6a [0,4194304] 0 2024-04-20T11:09:34.989 INFO:tasks.workunit.client.1.smithi173.stdout:2/499: link d2/d13/d46/d53/f76 d2/fa9 0 2024-04-20T11:09:34.990 INFO:tasks.workunit.client.1.smithi173.stdout:5/490: dread - da/de/f7e zero size 2024-04-20T11:09:34.994 INFO:tasks.workunit.client.1.smithi173.stdout:1/400: dread d5/d6/dc/f2c [0,4194304] 0 2024-04-20T11:09:34.997 INFO:tasks.workunit.client.1.smithi173.stdout:3/468: unlink d7/d17/d61/f8b 0 2024-04-20T11:09:34.999 INFO:tasks.workunit.client.1.smithi173.stdout:6/440: write d4/d14/f2f [1295417,93048] 0 2024-04-20T11:09:35.001 INFO:tasks.workunit.client.1.smithi173.stdout:2/500: symlink d2/d8c/laa 0 2024-04-20T11:09:35.001 INFO:tasks.workunit.client.1.smithi173.stdout:9/450: readlink d5/d9/d20/d29/d2d/l50 0 2024-04-20T11:09:35.002 INFO:tasks.workunit.client.1.smithi173.stdout:5/491: mkdir da/de/d19/d32/d30/dad 0 2024-04-20T11:09:35.003 INFO:tasks.workunit.client.1.smithi173.stdout:4/465: dread da/dc/d15/f5f [0,4194304] 0 2024-04-20T11:09:35.003 INFO:tasks.workunit.client.1.smithi173.stdout:8/428: unlink d4/d18/d3a/d3c/c4b 0 2024-04-20T11:09:35.004 INFO:tasks.workunit.client.1.smithi173.stdout:1/401: mkdir d5/d6/d17/d42/d12/d1c/d85 0 2024-04-20T11:09:35.004 INFO:tasks.workunit.client.1.smithi173.stdout:9/451: fdatasync d5/d9/d20/d2b/f49 0 2024-04-20T11:09:35.005 INFO:tasks.workunit.client.1.smithi173.stdout:0/412: mkdir d6/d8/df/d2c/d42/d50/d86/d8b 0 2024-04-20T11:09:35.007 INFO:tasks.workunit.client.1.smithi173.stdout:7/454: creat d9/d33/d85/f96 x:0 0 0 2024-04-20T11:09:35.008 INFO:tasks.workunit.client.1.smithi173.stdout:6/441: rmdir d4/d24/d42/d74/d3b 39 2024-04-20T11:09:35.009 INFO:tasks.workunit.client.1.smithi173.stdout:1/402: write d5/d6/d3b/f67 [583421,106038] 0 2024-04-20T11:09:35.011 INFO:tasks.workunit.client.1.smithi173.stdout:2/501: unlink d2/d13/d46/d53/l78 0 2024-04-20T11:09:35.015 INFO:tasks.workunit.client.1.smithi173.stdout:9/452: write d5/f6d [621852,66245] 0 2024-04-20T11:09:35.017 INFO:tasks.workunit.client.1.smithi173.stdout:8/429: creat d4/d18/d4f/f81 x:0 0 0 2024-04-20T11:09:35.018 INFO:tasks.workunit.client.1.smithi173.stdout:9/453: chown d5/d9/d20/d29/d2d/l50 38 1 2024-04-20T11:09:35.020 INFO:tasks.workunit.client.1.smithi173.stdout:9/454: chown d5/d9/cc 2000776305 1 2024-04-20T11:09:35.021 INFO:tasks.workunit.client.1.smithi173.stdout:4/466: dwrite da/d1a/d60/f63 [0,4194304] 0 2024-04-20T11:09:35.023 INFO:tasks.workunit.client.1.smithi173.stdout:6/442: rmdir d4/d59 39 2024-04-20T11:09:35.026 INFO:tasks.workunit.client.1.smithi173.stdout:7/455: mkdir d9/d33/d85/d97 0 2024-04-20T11:09:35.030 INFO:tasks.workunit.client.1.smithi173.stdout:2/502: mknod d2/d34/d58/cab 0 2024-04-20T11:09:35.031 INFO:tasks.workunit.client.1.smithi173.stdout:2/503: dread - d2/de/f60 zero size 2024-04-20T11:09:35.034 INFO:tasks.workunit.client.1.smithi173.stdout:8/430: mknod d4/d18/d3a/d3c/c82 0 2024-04-20T11:09:35.035 INFO:tasks.workunit.client.1.smithi173.stdout:5/492: chown da/f3d 602234283 1 2024-04-20T11:09:35.038 INFO:tasks.workunit.client.1.smithi173.stdout:9/455: mknod d5/d4c/d79/d80/c90 0 2024-04-20T11:09:35.039 INFO:tasks.workunit.client.1.smithi173.stdout:6/443: truncate d4/f38 456673 0 2024-04-20T11:09:35.040 INFO:tasks.workunit.client.1.smithi173.stdout:0/413: creat d6/d8/df/d2c/d42/d50/d86/d8b/f8c x:0 0 0 2024-04-20T11:09:35.041 INFO:tasks.workunit.client.1.smithi173.stdout:3/469: getdents d7/d17/d58 0 2024-04-20T11:09:35.042 INFO:tasks.workunit.client.1.smithi173.stdout:1/403: link d5/d6/d17/d42/d12/d1c/f82 d5/d6/d17/d42/da/d20/d38/d52/d75/f86 0 2024-04-20T11:09:35.045 INFO:tasks.workunit.client.1.smithi173.stdout:8/431: write d4/d18/d59/d5a/f67 [307069,50690] 0 2024-04-20T11:09:35.046 INFO:tasks.workunit.client.1.smithi173.stdout:5/493: creat da/de/d75/d40/fae x:0 0 0 2024-04-20T11:09:35.050 INFO:tasks.workunit.client.1.smithi173.stdout:2/504: unlink d2/d13/d46/d4b/f75 0 2024-04-20T11:09:35.051 INFO:tasks.workunit.client.1.smithi173.stdout:1/404: creat d5/d6/d17/d42/da/d20/f87 x:0 0 0 2024-04-20T11:09:35.054 INFO:tasks.workunit.client.1.smithi173.stdout:7/456: dread d9/d33/d35/d4a/f93 [0,4194304] 0 2024-04-20T11:09:35.055 INFO:tasks.workunit.client.1.smithi173.stdout:6/444: creat d4/d59/f84 x:0 0 0 2024-04-20T11:09:35.056 INFO:tasks.workunit.client.1.smithi173.stdout:3/470: getdents d7/d2b/d8e/d91 0 2024-04-20T11:09:35.056 INFO:tasks.workunit.client.1.smithi173.stdout:5/494: mknod da/de/d75/d40/d59/caf 0 2024-04-20T11:09:35.058 INFO:tasks.workunit.client.1.smithi173.stdout:0/414: fsync d6/d8/df/f11 0 2024-04-20T11:09:35.058 INFO:tasks.workunit.client.1.smithi173.stdout:8/432: dread d4/f32 [4194304,4194304] 0 2024-04-20T11:09:35.059 INFO:tasks.workunit.client.1.smithi173.stdout:7/457: dread - d9/d33/d40/d7a/f8c zero size 2024-04-20T11:09:35.059 INFO:tasks.workunit.client.1.smithi173.stdout:1/405: dread d5/d6/d17/d42/da/df/d40/f4c [0,4194304] 0 2024-04-20T11:09:35.059 INFO:tasks.workunit.client.1.smithi173.stdout:4/467: link da/dc/d13/c4b da/d1a/c95 0 2024-04-20T11:09:35.060 INFO:tasks.workunit.client.1.smithi173.stdout:0/415: stat d6/d8/f1f 0 2024-04-20T11:09:35.061 INFO:tasks.workunit.client.1.smithi173.stdout:8/433: chown d4/c9 164539567 1 2024-04-20T11:09:35.062 INFO:tasks.workunit.client.1.smithi173.stdout:1/406: write d5/d6/dc/f83 [867975,100154] 0 2024-04-20T11:09:35.062 INFO:tasks.workunit.client.1.smithi173.stdout:4/468: chown da/d48/d53/l4f 205473381 1 2024-04-20T11:09:35.065 INFO:tasks.workunit.client.1.smithi173.stdout:1/407: write d5/d6/d17/d42/da/f28 [1080351,56281] 0 2024-04-20T11:09:35.071 INFO:tasks.workunit.client.1.smithi173.stdout:9/456: getdents d5/d4c/d79 0 2024-04-20T11:09:35.075 INFO:tasks.workunit.client.1.smithi173.stdout:7/458: mknod d9/da/d43/c98 0 2024-04-20T11:09:35.079 INFO:tasks.workunit.client.1.smithi173.stdout:0/416: rmdir d6/d54 39 2024-04-20T11:09:35.082 INFO:tasks.workunit.client.1.smithi173.stdout:8/434: mknod d4/d18/d59/d5a/c83 0 2024-04-20T11:09:35.085 INFO:tasks.workunit.client.1.smithi173.stdout:3/471: rename d7/d9/d30/f7d to d7/d17/f98 0 2024-04-20T11:09:35.086 INFO:tasks.workunit.client.1.smithi173.stdout:4/469: creat da/d48/d53/d39/f96 x:0 0 0 2024-04-20T11:09:35.087 INFO:tasks.workunit.client.1.smithi173.stdout:2/505: truncate d2/f64 4403668 0 2024-04-20T11:09:35.089 INFO:tasks.workunit.client.1.smithi173.stdout:1/408: unlink d5/d6/d17/d42/f41 0 2024-04-20T11:09:35.092 INFO:tasks.workunit.client.1.smithi173.stdout:9/457: unlink d5/f6d 0 2024-04-20T11:09:35.094 INFO:tasks.workunit.client.1.smithi173.stdout:6/445: rmdir d4/d59/d83 0 2024-04-20T11:09:35.096 INFO:tasks.workunit.client.1.smithi173.stdout:5/495: creat da/d34/d7f/da4/fb0 x:0 0 0 2024-04-20T11:09:35.098 INFO:tasks.workunit.client.1.smithi173.stdout:5/496: write da/d4f/f5f [773428,103506] 0 2024-04-20T11:09:35.101 INFO:tasks.workunit.client.1.smithi173.stdout:8/435: read d4/d18/f24 [276244,103451] 0 2024-04-20T11:09:35.102 INFO:tasks.workunit.client.1.smithi173.stdout:4/470: creat da/d1e/d52/f97 x:0 0 0 2024-04-20T11:09:35.109 INFO:tasks.workunit.client.1.smithi173.stdout:7/459: mkdir d9/d33/d91/d99 0 2024-04-20T11:09:35.112 INFO:tasks.workunit.client.1.smithi173.stdout:7/460: rename d9/d33 to d9/d33/d35/d4a/d9a 22 2024-04-20T11:09:35.116 INFO:tasks.workunit.client.1.smithi173.stdout:4/471: creat da/dc/d13/d65/f98 x:0 0 0 2024-04-20T11:09:35.120 INFO:tasks.workunit.client.1.smithi173.stdout:6/446: mknod d4/d24/d42/d74/c85 0 2024-04-20T11:09:35.122 INFO:tasks.workunit.client.1.smithi173.stdout:6/447: chown d4/d24/d42/d74/c5c 0 1 2024-04-20T11:09:35.124 INFO:tasks.workunit.client.1.smithi173.stdout:6/448: dread - d4/d24/d42/f7c zero size 2024-04-20T11:09:35.125 INFO:tasks.workunit.client.1.smithi173.stdout:7/461: creat d9/d33/d81/f9b x:0 0 0 2024-04-20T11:09:35.137 INFO:tasks.workunit.client.1.smithi173.stdout:0/417: truncate d6/d24/f53 1887794 0 2024-04-20T11:09:35.139 INFO:tasks.workunit.client.1.smithi173.stdout:4/472: getdents da/d1a/d60/d7d 0 2024-04-20T11:09:35.142 INFO:tasks.workunit.client.1.smithi173.stdout:1/409: truncate d5/d6/f9 3515057 0 2024-04-20T11:09:35.142 INFO:tasks.workunit.client.1.smithi173.stdout:5/497: getdents da 0 2024-04-20T11:09:35.144 INFO:tasks.workunit.client.1.smithi173.stdout:7/462: mknod d9/c9c 0 2024-04-20T11:09:35.145 INFO:tasks.workunit.client.1.smithi173.stdout:1/410: write d5/d6/f21 [105690,58103] 0 2024-04-20T11:09:35.148 INFO:tasks.workunit.client.1.smithi173.stdout:5/498: truncate da/de/d19/d2c/f94 444009 0 2024-04-20T11:09:35.149 INFO:tasks.workunit.client.1.smithi173.stdout:6/449: rename d4/d14/l2c to d4/d14/d79/l86 0 2024-04-20T11:09:35.151 INFO:tasks.workunit.client.1.smithi173.stdout:9/458: truncate d5/d9/d20/d29/d2d/f43 1189372 0 2024-04-20T11:09:35.151 INFO:tasks.workunit.client.1.smithi173.stdout:3/472: dwrite d7/d9/f5a [0,4194304] 0 2024-04-20T11:09:35.156 INFO:tasks.workunit.client.1.smithi173.stdout:3/473: chown d7/d17/f2f 433 1 2024-04-20T11:09:35.164 INFO:tasks.workunit.client.1.smithi173.stdout:8/436: dwrite d4/d18/f5c [4194304,4194304] 0 2024-04-20T11:09:35.164 INFO:tasks.workunit.client.1.smithi173.stdout:4/473: dwrite da/d1e/d52/f5c [0,4194304] 0 2024-04-20T11:09:35.169 INFO:tasks.workunit.client.1.smithi173.stdout:4/474: write da/dc/d13/f51 [2752321,62928] 0 2024-04-20T11:09:35.171 INFO:tasks.workunit.client.1.smithi173.stdout:4/475: chown da/d48/d53/d39/l82 1 1 2024-04-20T11:09:35.173 INFO:tasks.workunit.client.1.smithi173.stdout:4/476: dread - da/d1e/d52/f71 zero size 2024-04-20T11:09:35.178 INFO:tasks.workunit.client.1.smithi173.stdout:0/418: dread d6/d8/df/d15/f39 [0,4194304] 0 2024-04-20T11:09:35.180 INFO:tasks.workunit.client.1.smithi173.stdout:7/463: write d9/d1f/d7f/f87 [92029,28661] 0 2024-04-20T11:09:35.184 INFO:tasks.workunit.client.1.smithi173.stdout:8/437: creat d4/d18/d59/d6d/f84 x:0 0 0 2024-04-20T11:09:35.188 INFO:tasks.workunit.client.1.smithi173.stdout:2/506: sync 2024-04-20T11:09:35.190 INFO:tasks.workunit.client.1.smithi173.stdout:5/499: dwrite da/f3d [0,4194304] 0 2024-04-20T11:09:35.194 INFO:tasks.workunit.client.1.smithi173.stdout:6/450: creat d4/d14/f87 x:0 0 0 2024-04-20T11:09:35.195 INFO:tasks.workunit.client.1.smithi173.stdout:4/477: symlink da/dc/d62/d84/l99 0 2024-04-20T11:09:35.196 INFO:tasks.workunit.client.1.smithi173.stdout:6/451: dread - d4/d18/f66 zero size 2024-04-20T11:09:35.199 INFO:tasks.workunit.client.1.smithi173.stdout:1/411: symlink d5/d6/d17/d42/d12/l88 0 2024-04-20T11:09:35.199 INFO:tasks.workunit.client.1.smithi173.stdout:0/419: dwrite d6/d8/df/d2c/d2d/f8a [0,4194304] 0 2024-04-20T11:09:35.200 INFO:tasks.workunit.client.1.smithi173.stdout:7/464: symlink d9/da/d2e/l9d 0 2024-04-20T11:09:35.202 INFO:tasks.workunit.client.1.smithi173.stdout:0/420: readlink d6/d8/df/l17 0 2024-04-20T11:09:35.204 INFO:tasks.workunit.client.1.smithi173.stdout:5/500: mkdir da/d9e/db1 0 2024-04-20T11:09:35.205 INFO:tasks.workunit.client.1.smithi173.stdout:8/438: symlink d4/d18/d3a/l85 0 2024-04-20T11:09:35.205 INFO:tasks.workunit.client.1.smithi173.stdout:6/452: symlink d4/d24/l88 0 2024-04-20T11:09:35.207 INFO:tasks.workunit.client.1.smithi173.stdout:4/478: mkdir da/d1e/d52/d9a 0 2024-04-20T11:09:35.208 INFO:tasks.workunit.client.1.smithi173.stdout:9/459: dread d5/d9/d20/d2b/f49 [0,4194304] 0 2024-04-20T11:09:35.208 INFO:tasks.workunit.client.1.smithi173.stdout:7/465: write d9/d33/d40/f49 [919543,46334] 0 2024-04-20T11:09:35.212 INFO:tasks.workunit.client.1.smithi173.stdout:1/412: unlink d5/d6/d17/d42/d12/d1f/f4f 0 2024-04-20T11:09:35.217 INFO:tasks.workunit.client.1.smithi173.stdout:8/439: fdatasync d4/d18/d3a/d3c/f40 0 2024-04-20T11:09:35.225 INFO:tasks.workunit.client.1.smithi173.stdout:2/507: link d2/d13/d46/d53/l39 d2/d13/d46/d53/lac 0 2024-04-20T11:09:35.226 INFO:tasks.workunit.client.1.smithi173.stdout:1/413: write d5/d6/d17/d42/da/d20/d48/f61 [1056430,65168] 0 2024-04-20T11:09:35.230 INFO:tasks.workunit.client.1.smithi173.stdout:0/421: dread d6/d8/d3c/d44/f46 [0,4194304] 0 2024-04-20T11:09:35.230 INFO:tasks.workunit.client.1.smithi173.stdout:2/508: stat d2/de/d11/d9d 0 2024-04-20T11:09:35.235 INFO:tasks.workunit.client.1.smithi173.stdout:8/440: truncate d4/d18/d3a/d3c/f54 758031 0 2024-04-20T11:09:35.238 INFO:tasks.workunit.client.1.smithi173.stdout:6/453: dwrite d4/d24/d42/f7c [0,4194304] 0 2024-04-20T11:09:35.239 INFO:tasks.workunit.client.1.smithi173.stdout:3/474: dwrite d7/d2b/f41 [0,4194304] 0 2024-04-20T11:09:35.245 INFO:tasks.workunit.client.1.smithi173.stdout:8/441: mkdir d4/d18/d3a/d3c/d86 0 2024-04-20T11:09:35.246 INFO:tasks.workunit.client.1.smithi173.stdout:9/460: dwrite d5/d4c/f52 [0,4194304] 0 2024-04-20T11:09:35.247 INFO:tasks.workunit.client.1.smithi173.stdout:2/509: mknod d2/de/d11/cad 0 2024-04-20T11:09:35.253 INFO:tasks.workunit.client.1.smithi173.stdout:3/475: creat d7/d17/f99 x:0 0 0 2024-04-20T11:09:35.253 INFO:tasks.workunit.client.1.smithi173.stdout:8/442: rmdir d4/d18/d3a/d3c/d5f 39 2024-04-20T11:09:35.253 INFO:tasks.workunit.client.1.smithi173.stdout:2/510: mkdir d2/d13/d46/d4b/dae 0 2024-04-20T11:09:35.254 INFO:tasks.workunit.client.1.smithi173.stdout:1/414: dread d5/d6/d17/d42/d12/f44 [0,4194304] 0 2024-04-20T11:09:35.258 INFO:tasks.workunit.client.1.smithi173.stdout:6/454: link d4/d14/d5f/c64 d4/d1c/c89 0 2024-04-20T11:09:35.259 INFO:tasks.workunit.client.1.smithi173.stdout:1/415: fsync d5/d6/d17/d42/d12/d1c/f6f 0 2024-04-20T11:09:35.260 INFO:tasks.workunit.client.1.smithi173.stdout:2/511: mknod d2/d13/d46/d53/caf 0 2024-04-20T11:09:35.262 INFO:tasks.workunit.client.1.smithi173.stdout:6/455: creat d4/d18/d7e/f8a x:0 0 0 2024-04-20T11:09:35.265 INFO:tasks.workunit.client.1.smithi173.stdout:1/416: mkdir d5/d6/d89 0 2024-04-20T11:09:35.268 INFO:tasks.workunit.client.1.smithi173.stdout:1/417: stat d5/d6/d17/d42/da/d20/d38/l80 0 2024-04-20T11:09:35.270 INFO:tasks.workunit.client.1.smithi173.stdout:5/501: dread da/de/d19/d32/d30/f4c [0,4194304] 0 2024-04-20T11:09:35.271 INFO:tasks.workunit.client.1.smithi173.stdout:8/443: dread d4/f10 [0,4194304] 0 2024-04-20T11:09:35.272 INFO:tasks.workunit.client.1.smithi173.stdout:1/418: write d5/d6/d17/d42/d12/d1c/f6f [212749,10461] 0 2024-04-20T11:09:35.277 INFO:tasks.workunit.client.1.smithi173.stdout:1/419: fdatasync d5/d6/d17/d42/da/f6a 0 2024-04-20T11:09:35.278 INFO:tasks.workunit.client.1.smithi173.stdout:5/502: rename da/de/d19/d2c/l3b to da/lb2 0 2024-04-20T11:09:35.279 INFO:tasks.workunit.client.1.smithi173.stdout:4/479: dread da/dc/d13/f29 [0,4194304] 0 2024-04-20T11:09:35.280 INFO:tasks.workunit.client.1.smithi173.stdout:6/456: dread d4/f27 [0,4194304] 0 2024-04-20T11:09:35.284 INFO:tasks.workunit.client.1.smithi173.stdout:4/480: rmdir da/d48/d53/d39 39 2024-04-20T11:09:35.284 INFO:tasks.workunit.client.1.smithi173.stdout:0/422: dwrite d6/d8/df/d2c/d42/f72 [0,4194304] 0 2024-04-20T11:09:35.287 INFO:tasks.workunit.client.1.smithi173.stdout:4/481: write da/d1e/d52/f85 [315384,4105] 0 2024-04-20T11:09:35.302 INFO:tasks.workunit.client.1.smithi173.stdout:3/476: dread d7/d53/f8f [0,4194304] 0 2024-04-20T11:09:35.303 INFO:tasks.workunit.client.1.smithi173.stdout:0/423: rename d6/d8/l40 to d6/d8/df/d2c/d42/d50/d86/l8d 0 2024-04-20T11:09:35.304 INFO:tasks.workunit.client.1.smithi173.stdout:1/420: getdents d5/d6/d17/d42/da 0 2024-04-20T11:09:35.308 INFO:tasks.workunit.client.1.smithi173.stdout:4/482: dwrite da/f67 [0,4194304] 0 2024-04-20T11:09:35.309 INFO:tasks.workunit.client.1.smithi173.stdout:0/424: fdatasync d6/d8/df/d2c/d2d/f8a 0 2024-04-20T11:09:35.309 INFO:tasks.workunit.client.1.smithi173.stdout:6/457: truncate d4/f16 670287 0 2024-04-20T11:09:35.314 INFO:tasks.workunit.client.1.smithi173.stdout:1/421: chown d5/d6/d17/l3d 639462276 1 2024-04-20T11:09:35.315 INFO:tasks.workunit.client.1.smithi173.stdout:0/425: stat d6/d8/d3c/d44/f46 0 2024-04-20T11:09:35.316 INFO:tasks.workunit.client.1.smithi173.stdout:3/477: rename d7/d2b/f41 to d7/d17/d58/d8a/f9a 0 2024-04-20T11:09:35.317 INFO:tasks.workunit.client.1.smithi173.stdout:8/444: dwrite d4/f28 [4194304,4194304] 0 2024-04-20T11:09:35.317 INFO:tasks.workunit.client.1.smithi173.stdout:4/483: mkdir da/d1a/d60/d66/d9b 0 2024-04-20T11:09:35.320 INFO:tasks.workunit.client.1.smithi173.stdout:3/478: truncate d7/d17/f93 146694 0 2024-04-20T11:09:35.323 INFO:tasks.workunit.client.1.smithi173.stdout:6/458: rename d4/d24/d42/f70 to d4/d18/d7e/d82/f8b 0 2024-04-20T11:09:35.324 INFO:tasks.workunit.client.1.smithi173.stdout:0/426: creat d6/d8/df/d2c/d42/f8e x:0 0 0 2024-04-20T11:09:35.327 INFO:tasks.workunit.client.1.smithi173.stdout:6/459: write d4/d14/f6a [68013,30550] 0 2024-04-20T11:09:35.330 INFO:tasks.workunit.client.1.smithi173.stdout:6/460: chown d4/f20 521 1 2024-04-20T11:09:35.332 INFO:tasks.workunit.client.1.smithi173.stdout:1/422: symlink d5/d6/d17/d42/d12/d1c/d85/l8a 0 2024-04-20T11:09:35.334 INFO:tasks.workunit.client.1.smithi173.stdout:5/503: getdents da/d4f/d65/d8e 0 2024-04-20T11:09:35.335 INFO:tasks.workunit.client.1.smithi173.stdout:1/423: chown d5/d6/d17/d42/da/d20/d38/d52/d75/f7b 15 1 2024-04-20T11:09:35.336 INFO:tasks.workunit.client.1.smithi173.stdout:0/427: write d6/d8/d3c/d44/f46 [2989162,128031] 0 2024-04-20T11:09:35.336 INFO:tasks.workunit.client.1.smithi173.stdout:4/484: symlink da/d1a/l9c 0 2024-04-20T11:09:35.337 INFO:tasks.workunit.client.1.smithi173.stdout:3/479: unlink d7/d9/f4b 0 2024-04-20T11:09:35.342 INFO:tasks.workunit.client.1.smithi173.stdout:0/428: rmdir d6/d8 39 2024-04-20T11:09:35.343 INFO:tasks.workunit.client.1.smithi173.stdout:1/424: creat d5/d6/d17/d42/d12/d1c/f8b x:0 0 0 2024-04-20T11:09:35.344 INFO:tasks.workunit.client.1.smithi173.stdout:3/480: chown d7/d9/l5e 47 1 2024-04-20T11:09:35.345 INFO:tasks.workunit.client.1.smithi173.stdout:5/504: rmdir da/de/d19/d81 39 2024-04-20T11:09:35.345 INFO:tasks.workunit.client.1.smithi173.stdout:4/485: symlink da/d48/d53/l9d 0 2024-04-20T11:09:35.348 INFO:tasks.workunit.client.1.smithi173.stdout:6/461: rename d4/d18/d61/f75 to d4/d14/f8c 0 2024-04-20T11:09:35.348 INFO:tasks.workunit.client.1.smithi173.stdout:5/505: symlink da/d4f/lb3 0 2024-04-20T11:09:35.349 INFO:tasks.workunit.client.1.smithi173.stdout:8/445: dread d4/d18/d3a/d3c/f73 [4194304,4194304] 0 2024-04-20T11:09:35.374 INFO:tasks.workunit.client.1.smithi173.stdout:7/466: sync 2024-04-20T11:09:35.383 INFO:tasks.workunit.client.1.smithi173.stdout:1/425: rename d5/d6/d17/d42/da/l51 to d5/d6/d17/d42/da/d20/d48/d81/l8c 0 2024-04-20T11:09:35.384 INFO:tasks.workunit.client.1.smithi173.stdout:5/506: symlink da/d4f/d65/d8e/lb4 0 2024-04-20T11:09:35.385 INFO:tasks.workunit.client.1.smithi173.stdout:4/486: mknod da/d48/d53/d39/c9e 0 2024-04-20T11:09:35.386 INFO:tasks.workunit.client.1.smithi173.stdout:8/446: write d4/f10 [3434639,65418] 0 2024-04-20T11:09:35.387 INFO:tasks.workunit.client.1.smithi173.stdout:6/462: link d4/d24/f6e d4/d24/d3e/f8d 0 2024-04-20T11:09:35.387 INFO:tasks.workunit.client.1.smithi173.stdout:4/487: readlink da/d48/l57 0 2024-04-20T11:09:35.388 INFO:tasks.workunit.client.1.smithi173.stdout:7/467: mkdir d9/d33/d35/d54/d9e 0 2024-04-20T11:09:35.389 INFO:tasks.workunit.client.1.smithi173.stdout:7/468: chown d9/d33/l92 9 1 2024-04-20T11:09:35.390 INFO:tasks.workunit.client.1.smithi173.stdout:8/447: readlink d4/d18/l74 0 2024-04-20T11:09:35.391 INFO:tasks.workunit.client.1.smithi173.stdout:7/469: dread - d9/d33/f3a zero size 2024-04-20T11:09:35.391 INFO:tasks.workunit.client.1.smithi173.stdout:6/463: readlink d4/d14/l17 0 2024-04-20T11:09:35.392 INFO:tasks.workunit.client.1.smithi173.stdout:5/507: write da/de/d19/d32/d30/d35/f6e [2886630,71499] 0 2024-04-20T11:09:35.394 INFO:tasks.workunit.client.1.smithi173.stdout:1/426: creat d5/d6/d17/d42/da/d20/d38/d52/f8d x:0 0 0 2024-04-20T11:09:35.394 INFO:tasks.workunit.client.1.smithi173.stdout:8/448: fdatasync d4/d18/d3a/f66 0 2024-04-20T11:09:35.394 INFO:tasks.workunit.client.1.smithi173.stdout:7/470: chown d9/d33/d85 7648 1 2024-04-20T11:09:35.404 INFO:tasks.workunit.client.1.smithi173.stdout:5/508: mkdir da/d34/d7f/da5/db5 0 2024-04-20T11:09:35.405 INFO:tasks.workunit.client.1.smithi173.stdout:1/427: unlink d5/d6/d17/l29 0 2024-04-20T11:09:35.407 INFO:tasks.workunit.client.1.smithi173.stdout:8/449: fdatasync d4/d18/f48 0 2024-04-20T11:09:35.411 INFO:tasks.workunit.client.1.smithi173.stdout:7/471: link d9/da/d17/d2b/c3f d9/da/d17/d2b/c9f 0 2024-04-20T11:09:35.413 INFO:tasks.workunit.client.1.smithi173.stdout:6/464: dwrite d4/d24/d42/d74/d3b/f52 [0,4194304] 0 2024-04-20T11:09:35.414 INFO:tasks.workunit.client.1.smithi173.stdout:1/428: stat d5/d6/d17/d42/d12/d1f/f2f 0 2024-04-20T11:09:35.415 INFO:tasks.workunit.client.1.smithi173.stdout:7/472: chown d9/da/d17/d2b/f83 55 1 2024-04-20T11:09:35.417 INFO:tasks.workunit.client.1.smithi173.stdout:8/450: write d4/d18/f48 [2912969,119217] 0 2024-04-20T11:09:35.420 INFO:tasks.workunit.client.1.smithi173.stdout:4/488: dread da/dc/d15/f1d [0,4194304] 0 2024-04-20T11:09:35.421 INFO:tasks.workunit.client.1.smithi173.stdout:7/473: dread d9/f4e [0,4194304] 0 2024-04-20T11:09:35.423 INFO:tasks.workunit.client.1.smithi173.stdout:6/465: mkdir d4/d1c/d8e 0 2024-04-20T11:09:35.427 INFO:tasks.workunit.client.1.smithi173.stdout:5/509: dwrite da/d34/f6f [0,4194304] 0 2024-04-20T11:09:35.429 INFO:tasks.workunit.client.1.smithi173.stdout:6/466: mkdir d4/d59/d8f 0 2024-04-20T11:09:35.432 INFO:tasks.workunit.client.1.smithi173.stdout:7/474: creat d9/fa0 x:0 0 0 2024-04-20T11:09:35.434 INFO:tasks.workunit.client.1.smithi173.stdout:1/429: dwrite d5/d6/d17/d42/da/d20/d38/d52/d75/f7b [0,4194304] 0 2024-04-20T11:09:35.443 INFO:tasks.workunit.client.1.smithi173.stdout:5/510: mknod da/de/d75/cb6 0 2024-04-20T11:09:35.444 INFO:tasks.workunit.client.1.smithi173.stdout:8/451: getdents d4/d18/d3a/d3c/d5f 0 2024-04-20T11:09:35.446 INFO:tasks.workunit.client.1.smithi173.stdout:4/489: truncate da/d1e/f38 927791 0 2024-04-20T11:09:35.447 INFO:tasks.workunit.client.1.smithi173.stdout:4/490: fsync da/dc/d13/f29 0 2024-04-20T11:09:35.447 INFO:tasks.workunit.client.1.smithi173.stdout:7/475: creat d9/d1f/fa1 x:0 0 0 2024-04-20T11:09:35.449 INFO:tasks.workunit.client.1.smithi173.stdout:4/491: dread - da/d1e/d52/f97 zero size 2024-04-20T11:09:35.449 INFO:tasks.workunit.client.1.smithi173.stdout:8/452: creat d4/d18/d4f/d5b/f87 x:0 0 0 2024-04-20T11:09:35.454 INFO:tasks.workunit.client.1.smithi173.stdout:7/476: symlink d9/d33/d35/la2 0 2024-04-20T11:09:35.456 INFO:tasks.workunit.client.1.smithi173.stdout:4/492: getdents da/dc/d15/d8e 0 2024-04-20T11:09:35.462 INFO:tasks.workunit.client.1.smithi173.stdout:6/467: dwrite d4/f16 [0,4194304] 0 2024-04-20T11:09:35.463 INFO:tasks.workunit.client.1.smithi173.stdout:8/453: link d4/c31 d4/d18/c88 0 2024-04-20T11:09:35.463 INFO:tasks.workunit.client.1.smithi173.stdout:6/468: chown d4/d24/d3e/l71 42232 1 2024-04-20T11:09:35.466 INFO:tasks.workunit.client.1.smithi173.stdout:7/477: write f6 [5204603,103131] 0 2024-04-20T11:09:35.470 INFO:tasks.workunit.client.1.smithi173.stdout:6/469: rmdir d4/d24 39 2024-04-20T11:09:35.471 INFO:tasks.workunit.client.1.smithi173.stdout:4/493: dread da/d1a/d60/f63 [0,4194304] 0 2024-04-20T11:09:35.475 INFO:tasks.workunit.client.1.smithi173.stdout:7/478: rename d9/da/l45 to d9/d1f/d7f/d8d/la3 0 2024-04-20T11:09:35.477 INFO:tasks.workunit.client.1.smithi173.stdout:6/470: mkdir d4/d1c/d8e/d90 0 2024-04-20T11:09:35.482 INFO:tasks.workunit.client.1.smithi173.stdout:7/479: rename d9/d33/d35/d58 to d9/d33/d35/d56/d5a/da4 0 2024-04-20T11:09:35.483 INFO:tasks.workunit.client.1.smithi173.stdout:4/494: creat da/d48/d53/f9f x:0 0 0 2024-04-20T11:09:35.484 INFO:tasks.workunit.client.1.smithi173.stdout:6/471: symlink d4/d24/l91 0 2024-04-20T11:09:35.485 INFO:tasks.workunit.client.1.smithi173.stdout:7/480: truncate d9/da/d17/d2b/f83 394125 0 2024-04-20T11:09:35.488 INFO:tasks.workunit.client.1.smithi173.stdout:4/495: creat da/d1a/fa0 x:0 0 0 2024-04-20T11:09:35.490 INFO:tasks.workunit.client.1.smithi173.stdout:7/481: creat d9/d1f/fa5 x:0 0 0 2024-04-20T11:09:35.491 INFO:tasks.workunit.client.1.smithi173.stdout:4/496: creat da/d1e/d52/fa1 x:0 0 0 2024-04-20T11:09:35.492 INFO:tasks.workunit.client.1.smithi173.stdout:7/482: chown d9/f4e 25872 1 2024-04-20T11:09:35.494 INFO:tasks.workunit.client.1.smithi173.stdout:4/497: unlink da/f67 0 2024-04-20T11:09:35.496 INFO:tasks.workunit.client.1.smithi173.stdout:7/483: creat d9/d33/d85/fa6 x:0 0 0 2024-04-20T11:09:35.497 INFO:tasks.workunit.client.1.smithi173.stdout:6/472: truncate d4/d14/f3c 1863219 0 2024-04-20T11:09:35.497 INFO:tasks.workunit.client.1.smithi173.stdout:7/484: chown d9/d33/d91/d99 139998671 1 2024-04-20T11:09:35.498 INFO:tasks.workunit.client.1.smithi173.stdout:4/498: creat da/d48/fa2 x:0 0 0 2024-04-20T11:09:35.500 INFO:tasks.workunit.client.1.smithi173.stdout:4/499: dread - da/d1e/d52/fa1 zero size 2024-04-20T11:09:35.500 INFO:tasks.workunit.client.1.smithi173.stdout:7/485: mknod d9/d33/d85/ca7 0 2024-04-20T11:09:35.505 INFO:tasks.workunit.client.1.smithi173.stdout:7/486: symlink d9/d33/d40/d89/la8 0 2024-04-20T11:09:35.508 INFO:tasks.workunit.client.1.smithi173.stdout:6/473: dread d4/d24/d42/f67 [0,4194304] 0 2024-04-20T11:09:35.508 INFO:tasks.workunit.client.1.smithi173.stdout:7/487: symlink d9/d1f/d7f/la9 0 2024-04-20T11:09:35.510 INFO:tasks.workunit.client.1.smithi173.stdout:7/488: readlink d9/d33/d35/d56/d5a/l68 0 2024-04-20T11:09:35.511 INFO:tasks.workunit.client.1.smithi173.stdout:9/461: sync 2024-04-20T11:09:35.514 INFO:tasks.workunit.client.1.smithi173.stdout:7/489: dread d9/da/d17/f78 [0,4194304] 0 2024-04-20T11:09:35.520 INFO:tasks.workunit.client.1.smithi173.stdout:7/490: rmdir d9/da/d17/d2b/d73 0 2024-04-20T11:09:35.521 INFO:tasks.workunit.client.1.smithi173.stdout:4/500: dread da/dc/d13/f51 [0,4194304] 0 2024-04-20T11:09:35.522 INFO:tasks.workunit.client.1.smithi173.stdout:9/462: write d5/d9/d20/d29/d2d/f43 [1533553,58546] 0 2024-04-20T11:09:35.526 INFO:tasks.workunit.client.1.smithi173.stdout:6/474: dwrite d4/d18/d7e/d82/f8b [0,4194304] 0 2024-04-20T11:09:35.527 INFO:tasks.workunit.client.1.smithi173.stdout:7/491: rename d9/d33/d35/d4a/d52/d5b to d9/da/d17/d2b/daa 0 2024-04-20T11:09:35.528 INFO:tasks.workunit.client.1.smithi173.stdout:4/501: unlink da/d1e/d52/fa1 0 2024-04-20T11:09:35.534 INFO:tasks.workunit.client.1.smithi173.stdout:6/475: mknod d4/d1c/d8e/c92 0 2024-04-20T11:09:35.538 INFO:tasks.workunit.client.1.smithi173.stdout:6/476: chown d4/d24/f54 22478919 1 2024-04-20T11:09:35.539 INFO:tasks.workunit.client.1.smithi173.stdout:6/477: fsync d4/d24/d3e/f4d 0 2024-04-20T11:09:35.540 INFO:tasks.workunit.client.1.smithi173.stdout:9/463: write d5/d9/f1a [1761568,67831] 0 2024-04-20T11:09:35.540 INFO:tasks.workunit.client.1.smithi173.stdout:6/478: dread - d4/d14/f87 zero size 2024-04-20T11:09:35.543 INFO:tasks.workunit.client.1.smithi173.stdout:6/479: write d4/d18/d7e/d82/f8b [1841073,5506] 0 2024-04-20T11:09:35.544 INFO:tasks.workunit.client.1.smithi173.stdout:9/464: getdents d5/d9/d20/d2b/d31 0 2024-04-20T11:09:35.546 INFO:tasks.workunit.client.1.smithi173.stdout:9/465: readlink d5/d9/d20/d29/l8e 0 2024-04-20T11:09:35.548 INFO:tasks.workunit.client.1.smithi173.stdout:7/492: dread f6 [0,4194304] 0 2024-04-20T11:09:35.548 INFO:tasks.workunit.client.1.smithi173.stdout:6/480: write d4/d24/d42/d74/d3b/f41 [4399384,18901] 0 2024-04-20T11:09:35.550 INFO:tasks.workunit.client.1.smithi173.stdout:9/466: chown d5/d9/d20/d29/d18/l55 0 1 2024-04-20T11:09:35.552 INFO:tasks.workunit.client.1.smithi173.stdout:9/467: chown d5/d9 1195508 1 2024-04-20T11:09:35.557 INFO:tasks.workunit.client.1.smithi173.stdout:6/481: rename d4/cb to d4/d24/d42/d74/d3b/c93 0 2024-04-20T11:09:35.559 INFO:tasks.workunit.client.1.smithi173.stdout:6/482: truncate d4/d18/f58 4716824 0 2024-04-20T11:09:35.561 INFO:tasks.workunit.client.1.smithi173.stdout:4/502: dread da/dc/d15/f2a [4194304,4194304] 0 2024-04-20T11:09:35.565 INFO:tasks.workunit.client.1.smithi173.stdout:4/503: write da/d1a/d60/f63 [3810547,60502] 0 2024-04-20T11:09:35.567 INFO:tasks.workunit.client.1.smithi173.stdout:4/504: write f8 [2813852,87925] 0 2024-04-20T11:09:35.568 INFO:tasks.workunit.client.1.smithi173.stdout:7/493: dwrite d9/da/d17/f79 [0,4194304] 0 2024-04-20T11:09:35.572 INFO:tasks.workunit.client.1.smithi173.stdout:7/494: mknod d9/da/cab 0 2024-04-20T11:09:35.574 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:35 smithi117 ceph-mon[15659]: pgmap v27: 65 pgs: 65 active+clean; 2.6 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 169 MiB/s wr, 387 op/s 2024-04-20T11:09:35.578 INFO:tasks.workunit.client.1.smithi173.stdout:6/483: dwrite d4/d14/f2f [0,4194304] 0 2024-04-20T11:09:35.579 INFO:tasks.workunit.client.1.smithi173.stdout:9/468: dread d5/d9/fa [4194304,4194304] 0 2024-04-20T11:09:35.582 INFO:tasks.workunit.client.1.smithi173.stdout:7/495: symlink d9/d1f/lac 0 2024-04-20T11:09:35.586 INFO:tasks.workunit.client.1.smithi173.stdout:9/469: creat d5/d4c/f91 x:0 0 0 2024-04-20T11:09:35.590 INFO:tasks.workunit.client.1.smithi173.stdout:7/496: mkdir d9/d33/d35/d56/d5a/da4/dad 0 2024-04-20T11:09:35.596 INFO:tasks.workunit.client.1.smithi173.stdout:7/497: rename d9/f2c to d9/d33/d35/d56/d5a/da4/fae 0 2024-04-20T11:09:35.599 INFO:tasks.workunit.client.1.smithi173.stdout:6/484: dread d4/f38 [0,4194304] 0 2024-04-20T11:09:35.600 INFO:tasks.workunit.client.1.smithi173.stdout:9/470: dread d5/d9/d20/d2b/d59/d44/f6a [0,4194304] 0 2024-04-20T11:09:35.601 INFO:tasks.workunit.client.1.smithi173.stdout:6/485: write d4/d18/f66 [332514,102330] 0 2024-04-20T11:09:35.601 INFO:tasks.workunit.client.1.smithi173.stdout:7/498: creat d9/d1f/d7f/d8d/faf x:0 0 0 2024-04-20T11:09:35.606 INFO:tasks.workunit.client.1.smithi173.stdout:9/471: creat d5/d9/d20/d2b/d31/f92 x:0 0 0 2024-04-20T11:09:35.607 INFO:tasks.workunit.client.1.smithi173.stdout:6/486: mkdir d4/d1c/d8e/d94 0 2024-04-20T11:09:35.609 INFO:tasks.workunit.client.1.smithi173.stdout:9/472: write d5/d9/d20/d4f/f68 [885823,50434] 0 2024-04-20T11:09:35.612 INFO:tasks.workunit.client.1.smithi173.stdout:9/473: write d5/d9/f1a [690800,77968] 0 2024-04-20T11:09:35.612 INFO:tasks.workunit.client.1.smithi173.stdout:6/487: mknod d4/d24/d3e/c95 0 2024-04-20T11:09:35.617 INFO:tasks.workunit.client.1.smithi173.stdout:7/499: rename d9/d33/c3c to d9/d33/d40/d89/cb0 0 2024-04-20T11:09:35.619 INFO:tasks.workunit.client.1.smithi173.stdout:9/474: symlink d5/d4c/d79/d80/l93 0 2024-04-20T11:09:35.625 INFO:tasks.workunit.client.1.smithi173.stdout:7/500: mknod d9/d33/d35/cb1 0 2024-04-20T11:09:35.627 INFO:tasks.workunit.client.1.smithi173.stdout:7/501: read d9/d33/f47 [305760,4253] 0 2024-04-20T11:09:35.627 INFO:tasks.workunit.client.1.smithi173.stdout:6/488: link d4/d24/d42/d74/l30 d4/d1c/d8e/d90/l96 0 2024-04-20T11:09:35.629 INFO:tasks.workunit.client.1.smithi173.stdout:7/502: write d9/da/f46 [4255012,29284] 0 2024-04-20T11:09:35.633 INFO:tasks.workunit.client.1.smithi173.stdout:7/503: mkdir d9/d33/d35/d56/d5a/da4/db2 0 2024-04-20T11:09:35.640 INFO:tasks.workunit.client.1.smithi173.stdout:7/504: mkdir d9/d33/d35/d54/db3 0 2024-04-20T11:09:35.652 INFO:tasks.workunit.client.1.smithi173.stdout:2/512: sync 2024-04-20T11:09:35.658 INFO:tasks.workunit.client.1.smithi173.stdout:2/513: dread d2/d34/d81/f9e [0,4194304] 0 2024-04-20T11:09:35.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:35 smithi173 ceph-mon[23800]: pgmap v27: 65 pgs: 65 active+clean; 2.6 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 169 MiB/s wr, 387 op/s 2024-04-20T11:09:35.661 INFO:tasks.workunit.client.1.smithi173.stdout:7/505: truncate d9/d33/f6b 2540226 0 2024-04-20T11:09:35.670 INFO:tasks.workunit.client.1.smithi173.stdout:2/514: dwrite d2/de/f3b [0,4194304] 0 2024-04-20T11:09:35.672 INFO:tasks.workunit.client.1.smithi173.stdout:7/506: link d9/d33/f47 d9/d33/d85/fb4 0 2024-04-20T11:09:35.678 INFO:tasks.workunit.client.1.smithi173.stdout:7/507: creat d9/d33/d35/d56/d5a/da4/fb5 x:0 0 0 2024-04-20T11:09:35.679 INFO:tasks.workunit.client.1.smithi173.stdout:2/515: getdents d2 0 2024-04-20T11:09:35.681 INFO:tasks.workunit.client.1.smithi173.stdout:2/516: mkdir d2/de/d11/d9d/d87/db0 0 2024-04-20T11:09:35.701 INFO:tasks.workunit.client.1.smithi173.stdout:2/517: dread d2/d13/f74 [0,4194304] 0 2024-04-20T11:09:35.715 INFO:tasks.workunit.client.1.smithi173.stdout:2/518: dwrite d2/d34/f5b [0,4194304] 0 2024-04-20T11:09:35.719 INFO:tasks.workunit.client.1.smithi173.stdout:2/519: write d2/d13/d46/d53/d4e/fa1 [694853,103585] 0 2024-04-20T11:09:35.737 INFO:tasks.workunit.client.1.smithi173.stdout:2/520: dwrite d2/d34/f51 [0,4194304] 0 2024-04-20T11:09:35.758 INFO:tasks.workunit.client.1.smithi173.stdout:2/521: dwrite d2/d13/d46/f85 [0,4194304] 0 2024-04-20T11:09:35.763 INFO:tasks.workunit.client.1.smithi173.stdout:2/522: write d2/d34/d81/f9e [706663,111640] 0 2024-04-20T11:09:35.777 INFO:tasks.workunit.client.1.smithi173.stdout:2/523: rename d2/de/l1e to d2/d13/d46/d53/lb1 0 2024-04-20T11:09:35.783 INFO:tasks.workunit.client.1.smithi173.stdout:2/524: dread d2/d13/d46/d53/d4e/f57 [0,4194304] 0 2024-04-20T11:09:35.784 INFO:tasks.workunit.client.1.smithi173.stdout:2/525: readlink d2/d13/l52 0 2024-04-20T11:09:35.786 INFO:tasks.workunit.client.1.smithi173.stdout:0/429: sync 2024-04-20T11:09:35.786 INFO:tasks.workunit.client.1.smithi173.stdout:3/481: sync 2024-04-20T11:09:35.787 INFO:tasks.workunit.client.1.smithi173.stdout:2/526: write d2/de/fa7 [457090,129682] 0 2024-04-20T11:09:35.790 INFO:tasks.workunit.client.1.smithi173.stdout:3/482: dread - d7/d17/d61/f78 zero size 2024-04-20T11:09:35.790 INFO:tasks.workunit.client.1.smithi173.stdout:0/430: mkdir d6/d5f/d8f 0 2024-04-20T11:09:35.792 INFO:tasks.workunit.client.1.smithi173.stdout:3/483: dread - d7/d9/d30/f80 zero size 2024-04-20T11:09:35.792 INFO:tasks.workunit.client.1.smithi173.stdout:2/527: mkdir d2/d13/da5/db2 0 2024-04-20T11:09:35.794 INFO:tasks.workunit.client.1.smithi173.stdout:2/528: fsync d2/d34/d81/f89 0 2024-04-20T11:09:35.794 INFO:tasks.workunit.client.1.smithi173.stdout:3/484: read - d7/d17/d61/d83/f79 zero size 2024-04-20T11:09:35.796 INFO:tasks.workunit.client.1.smithi173.stdout:0/431: creat d6/d8/df/d43/f90 x:0 0 0 2024-04-20T11:09:35.796 INFO:tasks.workunit.client.1.smithi173.stdout:2/529: stat d2/d13/d28/f15 0 2024-04-20T11:09:35.798 INFO:tasks.workunit.client.1.smithi173.stdout:3/485: creat d7/d17/d61/f9b x:0 0 0 2024-04-20T11:09:35.827 INFO:tasks.workunit.client.1.smithi173.stdout:0/432: readlink d6/d8/df/d2c/d42/d50/d86/l8d 0 2024-04-20T11:09:35.830 INFO:tasks.workunit.client.1.smithi173.stdout:3/486: symlink d7/d85/l9c 0 2024-04-20T11:09:35.833 INFO:tasks.workunit.client.1.smithi173.stdout:0/433: mkdir d6/d54/d91 0 2024-04-20T11:09:35.840 INFO:tasks.workunit.client.1.smithi173.stdout:3/487: rename d7/d2b/d8e/d91 to d7/d2b/d9d 0 2024-04-20T11:09:35.843 INFO:tasks.workunit.client.1.smithi173.stdout:3/488: fdatasync d7/f25 0 2024-04-20T11:09:35.849 INFO:tasks.workunit.client.1.smithi173.stdout:0/434: rename d6/d5f/c64 to d6/d8/df/d15/c92 0 2024-04-20T11:09:35.850 INFO:tasks.workunit.client.1.smithi173.stdout:2/530: truncate d2/d34/f51 2686871 0 2024-04-20T11:09:35.851 INFO:tasks.workunit.client.1.smithi173.stdout:0/435: dread - d6/d8/df/d43/f90 zero size 2024-04-20T11:09:35.853 INFO:tasks.workunit.client.1.smithi173.stdout:1/430: sync 2024-04-20T11:09:35.854 INFO:tasks.workunit.client.1.smithi173.stdout:3/489: rename d7/d17/c42 to d7/d17/d61/d83/d45/c9e 0 2024-04-20T11:09:35.856 INFO:tasks.workunit.client.1.smithi173.stdout:0/436: readlink d6/l48 0 2024-04-20T11:09:35.863 INFO:tasks.workunit.client.1.smithi173.stdout:1/431: symlink d5/d6/d17/d42/d12/d1f/l8e 0 2024-04-20T11:09:35.868 INFO:tasks.workunit.client.1.smithi173.stdout:3/490: link d7/d17/d61/d83/d45/f63 d7/d17/f9f 0 2024-04-20T11:09:35.880 INFO:tasks.workunit.client.1.smithi173.stdout:3/491: rename d7/d53/l89 to d7/d17/d58/d8a/la0 0 2024-04-20T11:09:35.883 INFO:tasks.workunit.client.1.smithi173.stdout:5/511: sync 2024-04-20T11:09:35.883 INFO:tasks.workunit.client.1.smithi173.stdout:4/505: sync 2024-04-20T11:09:35.883 INFO:tasks.workunit.client.1.smithi173.stdout:8/454: sync 2024-04-20T11:09:35.884 INFO:tasks.workunit.client.1.smithi173.stdout:3/492: fsync d7/d9/f5a 0 2024-04-20T11:09:35.887 INFO:tasks.workunit.client.1.smithi173.stdout:1/432: rename d5/d6/d17/d42/da/d20/d38/l7a to d5/d6/dc/d7f/l8f 0 2024-04-20T11:09:35.888 INFO:tasks.workunit.client.1.smithi173.stdout:4/506: read - da/d48/fa2 zero size 2024-04-20T11:09:35.891 INFO:tasks.workunit.client.1.smithi173.stdout:5/512: read da/f3d [4179135,93156] 0 2024-04-20T11:09:35.892 INFO:tasks.workunit.client.1.smithi173.stdout:2/531: dread d2/d34/d58/f5a [0,4194304] 0 2024-04-20T11:09:35.896 INFO:tasks.workunit.client.1.smithi173.stdout:5/513: stat da/de/d75/d40/d59 0 2024-04-20T11:09:35.896 INFO:tasks.workunit.client.1.smithi173.stdout:4/507: write da/dc/d13/f32 [6608739,39847] 0 2024-04-20T11:09:35.900 INFO:tasks.workunit.client.1.smithi173.stdout:2/532: mknod d2/d13/d46/cb3 0 2024-04-20T11:09:35.903 INFO:tasks.workunit.client.1.smithi173.stdout:0/437: dread d6/f7 [0,4194304] 0 2024-04-20T11:09:35.905 INFO:tasks.workunit.client.1.smithi173.stdout:4/508: mkdir da/d1e/d52/da3 0 2024-04-20T11:09:35.908 INFO:tasks.workunit.client.1.smithi173.stdout:1/433: dread d5/d6/d17/d42/da/f14 [0,4194304] 0 2024-04-20T11:09:35.908 INFO:tasks.workunit.client.1.smithi173.stdout:4/509: truncate da/f25 8624602 0 2024-04-20T11:09:35.909 INFO:tasks.workunit.client.1.smithi173.stdout:0/438: chown d6/d24/f53 8148 1 2024-04-20T11:09:35.911 INFO:tasks.workunit.client.1.smithi173.stdout:8/455: dwrite d4/d18/d4f/f81 [0,4194304] 0 2024-04-20T11:09:35.912 INFO:tasks.workunit.client.1.smithi173.stdout:4/510: unlink da/dc/d13/l31 0 2024-04-20T11:09:35.914 INFO:tasks.workunit.client.1.smithi173.stdout:0/439: write d6/d8/df/d2c/d2d/f36 [2134417,65871] 0 2024-04-20T11:09:35.917 INFO:tasks.workunit.client.1.smithi173.stdout:0/440: chown d6/d5f 4708 1 2024-04-20T11:09:35.917 INFO:tasks.workunit.client.1.smithi173.stdout:3/493: dread d7/d9/ff [0,4194304] 0 2024-04-20T11:09:35.917 INFO:tasks.workunit.client.1.smithi173.stdout:1/434: write d5/d6/d17/d42/f19 [4606952,96531] 0 2024-04-20T11:09:35.923 INFO:tasks.workunit.client.1.smithi173.stdout:4/511: creat da/d1a/d60/fa4 x:0 0 0 2024-04-20T11:09:35.924 INFO:tasks.workunit.client.1.smithi173.stdout:8/456: symlink d4/d18/d3a/l89 0 2024-04-20T11:09:35.927 INFO:tasks.workunit.client.1.smithi173.stdout:8/457: chown d4/c31 990334490 1 2024-04-20T11:09:35.927 INFO:tasks.workunit.client.1.smithi173.stdout:4/512: creat da/dc/fa5 x:0 0 0 2024-04-20T11:09:35.928 INFO:tasks.workunit.client.1.smithi173.stdout:3/494: truncate d7/d17/f20 345242 0 2024-04-20T11:09:35.931 INFO:tasks.workunit.client.1.smithi173.stdout:4/513: mkdir da/d1e/da6 0 2024-04-20T11:09:35.931 INFO:tasks.workunit.client.1.smithi173.stdout:8/458: read d4/d18/d3a/d3c/f53 [595218,39916] 0 2024-04-20T11:09:35.932 INFO:tasks.workunit.client.1.smithi173.stdout:5/514: dread da/de/d19/f2a [0,4194304] 0 2024-04-20T11:09:35.932 INFO:tasks.workunit.client.1.smithi173.stdout:3/495: symlink d7/d17/d58/la1 0 2024-04-20T11:09:35.937 INFO:tasks.workunit.client.1.smithi173.stdout:8/459: mknod d4/d18/d59/d5a/c8a 0 2024-04-20T11:09:35.942 INFO:tasks.workunit.client.1.smithi173.stdout:4/514: creat da/dc/d15/d8e/fa7 x:0 0 0 2024-04-20T11:09:35.944 INFO:tasks.workunit.client.1.smithi173.stdout:4/515: chown da/d1e/da6 141308 1 2024-04-20T11:09:35.945 INFO:tasks.workunit.client.1.smithi173.stdout:0/441: dwrite d6/d8/fe [0,4194304] 0 2024-04-20T11:09:35.946 INFO:tasks.workunit.client.1.smithi173.stdout:8/460: mknod d4/d18/d4f/d5b/c8b 0 2024-04-20T11:09:35.950 INFO:tasks.workunit.client.1.smithi173.stdout:8/461: chown d4/d18/d4f/d5b/c8b 371 1 2024-04-20T11:09:35.952 INFO:tasks.workunit.client.1.smithi173.stdout:8/462: chown d4/d18/d3a/c79 61991 1 2024-04-20T11:09:35.955 INFO:tasks.workunit.client.1.smithi173.stdout:5/515: truncate da/de/d75/f64 3660818 0 2024-04-20T11:09:35.958 INFO:tasks.workunit.client.1.smithi173.stdout:5/516: chown da/lb 0 1 2024-04-20T11:09:35.958 INFO:tasks.workunit.client.1.smithi173.stdout:8/463: mkdir d4/d18/d59/d5a/d7b/d8c 0 2024-04-20T11:09:35.965 INFO:tasks.workunit.client.1.smithi173.stdout:8/464: creat d4/d18/d59/d5a/d7b/d8c/f8d x:0 0 0 2024-04-20T11:09:35.966 INFO:tasks.workunit.client.1.smithi173.stdout:0/442: dwrite d6/d54/f7d [0,4194304] 0 2024-04-20T11:09:35.966 INFO:tasks.workunit.client.1.smithi173.stdout:4/516: dwrite da/dc/f33 [4194304,4194304] 0 2024-04-20T11:09:35.968 INFO:tasks.workunit.client.1.smithi173.stdout:4/517: fsync da/d48/d53/f9f 0 2024-04-20T11:09:35.971 INFO:tasks.workunit.client.1.smithi173.stdout:4/518: chown da/dc 488580832 1 2024-04-20T11:09:35.973 INFO:tasks.workunit.client.1.smithi173.stdout:4/519: dread - da/d1a/d60/fa4 zero size 2024-04-20T11:09:35.978 INFO:tasks.workunit.client.1.smithi173.stdout:8/465: creat d4/d18/d59/d5a/d7b/d8c/f8e x:0 0 0 2024-04-20T11:09:35.981 INFO:tasks.workunit.client.1.smithi173.stdout:8/466: write d4/d18/d4f/f81 [4236204,7819] 0 2024-04-20T11:09:35.984 INFO:tasks.workunit.client.1.smithi173.stdout:5/517: dwrite da/de/d75/fab [0,4194304] 0 2024-04-20T11:09:35.986 INFO:tasks.workunit.client.1.smithi173.stdout:3/496: dread d7/d17/d61/d83/d45/f63 [0,4194304] 0 2024-04-20T11:09:35.989 INFO:tasks.workunit.client.1.smithi173.stdout:8/467: truncate d4/d18/d59/d5a/f67 1657066 0 2024-04-20T11:09:35.990 INFO:tasks.workunit.client.1.smithi173.stdout:5/518: mkdir da/db7 0 2024-04-20T11:09:35.991 INFO:tasks.workunit.client.1.smithi173.stdout:9/475: sync 2024-04-20T11:09:35.991 INFO:tasks.workunit.client.1.smithi173.stdout:7/508: sync 2024-04-20T11:09:35.991 INFO:tasks.workunit.client.1.smithi173.stdout:6/489: sync 2024-04-20T11:09:35.993 INFO:tasks.workunit.client.1.smithi173.stdout:4/520: dwrite da/f25 [4194304,4194304] 0 2024-04-20T11:09:35.995 INFO:tasks.workunit.client.1.smithi173.stdout:8/468: write d4/d18/d3a/d3c/f52 [3912596,39680] 0 2024-04-20T11:09:35.997 INFO:tasks.workunit.client.1.smithi173.stdout:4/521: write da/d48/d53/d39/f56 [501744,4364] 0 2024-04-20T11:09:36.001 INFO:tasks.workunit.client.1.smithi173.stdout:6/490: mkdir d4/d59/d97 0 2024-04-20T11:09:36.002 INFO:tasks.workunit.client.1.smithi173.stdout:0/443: truncate d6/f29 1227684 0 2024-04-20T11:09:36.007 INFO:tasks.workunit.client.1.smithi173.stdout:3/497: dwrite d7/d2b/f62 [0,4194304] 0 2024-04-20T11:09:36.010 INFO:tasks.workunit.client.1.smithi173.stdout:8/469: mknod d4/d18/d59/d6d/c8f 0 2024-04-20T11:09:36.010 INFO:tasks.workunit.client.1.smithi173.stdout:0/444: mknod d6/d8/d3c/d44/c93 0 2024-04-20T11:09:36.013 INFO:tasks.workunit.client.1.smithi173.stdout:5/519: dwrite da/f33 [0,4194304] 0 2024-04-20T11:09:36.014 INFO:tasks.workunit.client.1.smithi173.stdout:0/445: stat d6/d8/df/d2c/d42/d50/f76 0 2024-04-20T11:09:36.018 INFO:tasks.workunit.client.1.smithi173.stdout:7/509: mknod d9/d33/d85/d97/cb6 0 2024-04-20T11:09:36.018 INFO:tasks.workunit.client.1.smithi173.stdout:9/476: truncate d5/d9/d20/d29/d2d/f3f 2109082 0 2024-04-20T11:09:36.019 INFO:tasks.workunit.client.1.smithi173.stdout:4/522: creat da/d1e/d52/d9a/fa8 x:0 0 0 2024-04-20T11:09:36.020 INFO:tasks.workunit.client.1.smithi173.stdout:6/491: truncate d4/d24/f6e 886883 0 2024-04-20T11:09:36.022 INFO:tasks.workunit.client.1.smithi173.stdout:8/470: symlink d4/d18/d59/d5a/d7b/d8c/l90 0 2024-04-20T11:09:36.038 INFO:tasks.workunit.client.1.smithi173.stdout:6/492: write d4/d14/f6a [1084811,61910] 0 2024-04-20T11:09:36.041 INFO:tasks.workunit.client.1.smithi173.stdout:5/520: chown da/de/d19/d32/d30/d35/c3f 186243126 1 2024-04-20T11:09:36.042 INFO:tasks.workunit.client.1.smithi173.stdout:9/477: write d5/f6 [800007,48265] 0 2024-04-20T11:09:36.042 INFO:tasks.workunit.client.1.smithi173.stdout:4/523: truncate da/dc/d15/d8e/fa7 594522 0 2024-04-20T11:09:36.042 INFO:tasks.workunit.client.1.smithi173.stdout:8/471: rmdir d4/d18/d4f/d5b 39 2024-04-20T11:09:36.042 INFO:tasks.workunit.client.1.smithi173.stdout:6/493: dread - d4/d18/d7e/f8a zero size 2024-04-20T11:09:36.044 INFO:tasks.workunit.client.1.smithi173.stdout:7/510: creat d9/fb7 x:0 0 0 2024-04-20T11:09:36.046 INFO:tasks.workunit.client.1.smithi173.stdout:9/478: rmdir d5/d9/d20/d4f/d5e 39 2024-04-20T11:09:36.047 INFO:tasks.workunit.client.1.smithi173.stdout:9/479: fdatasync d5/d9/d20/d29/f5c 0 2024-04-20T11:09:36.047 INFO:tasks.workunit.client.1.smithi173.stdout:4/524: symlink da/d1a/la9 0 2024-04-20T11:09:36.048 INFO:tasks.workunit.client.1.smithi173.stdout:7/511: symlink d9/d33/d91/lb8 0 2024-04-20T11:09:36.049 INFO:tasks.workunit.client.1.smithi173.stdout:4/525: truncate da/f64 1192353 0 2024-04-20T11:09:36.053 INFO:tasks.workunit.client.1.smithi173.stdout:4/526: write da/d1a/d60/f63 [4341157,121167] 0 2024-04-20T11:09:36.057 INFO:tasks.workunit.client.1.smithi173.stdout:7/512: rename d9/da/f10 to d9/d33/d35/d56/d5a/da4/dad/fb9 0 2024-04-20T11:09:36.058 INFO:tasks.workunit.client.1.smithi173.stdout:9/480: mkdir d5/d7f/d94 0 2024-04-20T11:09:36.058 INFO:tasks.workunit.client.1.smithi173.stdout:8/472: dwrite d4/f41 [0,4194304] 0 2024-04-20T11:09:36.060 INFO:tasks.workunit.client.1.smithi173.stdout:6/494: link d4/d14/d5f/l77 d4/d14/d79/l98 0 2024-04-20T11:09:36.075 INFO:tasks.workunit.client.1.smithi173.stdout:4/527: symlink da/d48/d53/laa 0 2024-04-20T11:09:36.078 INFO:tasks.workunit.client.1.smithi173.stdout:6/495: creat d4/d24/d42/d81/f99 x:0 0 0 2024-04-20T11:09:36.079 INFO:tasks.workunit.client.1.smithi173.stdout:2/533: sync 2024-04-20T11:09:36.080 INFO:tasks.workunit.client.1.smithi173.stdout:9/481: write d5/d9/f26 [1406244,83373] 0 2024-04-20T11:09:36.081 INFO:tasks.workunit.client.1.smithi173.stdout:4/528: creat da/d1e/da6/fab x:0 0 0 2024-04-20T11:09:36.083 INFO:tasks.workunit.client.1.smithi173.stdout:8/473: dwrite d4/f32 [4194304,4194304] 0 2024-04-20T11:09:36.087 INFO:tasks.workunit.client.1.smithi173.stdout:2/534: rename d2/d13/d28/f7c to d2/de/d11/d9d/d87/fb4 0 2024-04-20T11:09:36.089 INFO:tasks.workunit.client.1.smithi173.stdout:4/529: rmdir da/d1e/d52 39 2024-04-20T11:09:36.089 INFO:tasks.workunit.client.1.smithi173.stdout:2/535: chown d2/de/d11/cad 2 1 2024-04-20T11:09:36.092 INFO:tasks.workunit.client.1.smithi173.stdout:6/496: creat d4/f9a x:0 0 0 2024-04-20T11:09:36.092 INFO:tasks.workunit.client.1.smithi173.stdout:8/474: mknod d4/d18/d3a/d3c/d86/c91 0 2024-04-20T11:09:36.093 INFO:tasks.workunit.client.1.smithi173.stdout:2/536: write d2/d13/d28/f15 [4488287,12168] 0 2024-04-20T11:09:36.098 INFO:tasks.workunit.client.1.smithi173.stdout:4/530: rename da/dc/c2d to da/d1a/d60/cac 0 2024-04-20T11:09:36.101 INFO:tasks.workunit.client.1.smithi173.stdout:8/475: symlink d4/d18/d59/d6d/l92 0 2024-04-20T11:09:36.101 INFO:tasks.workunit.client.1.smithi173.stdout:2/537: dread d2/d13/d46/d53/d4e/fa1 [0,4194304] 0 2024-04-20T11:09:36.107 INFO:tasks.workunit.client.1.smithi173.stdout:2/538: mknod d2/d34/cb5 0 2024-04-20T11:09:36.109 INFO:tasks.workunit.client.1.smithi173.stdout:6/497: write d4/d1c/f4a [412314,97059] 0 2024-04-20T11:09:36.112 INFO:tasks.workunit.client.1.smithi173.stdout:9/482: dread d5/d9/d20/d29/f11 [0,4194304] 0 2024-04-20T11:09:36.112 INFO:tasks.workunit.client.1.smithi173.stdout:4/531: mknod da/d1e/d52/cad 0 2024-04-20T11:09:36.115 INFO:tasks.workunit.client.1.smithi173.stdout:4/532: write da/dc/d13/f29 [4737159,71001] 0 2024-04-20T11:09:36.118 INFO:tasks.workunit.client.1.smithi173.stdout:8/476: dwrite d4/d18/d59/d5a/d7b/d8c/f8e [0,4194304] 0 2024-04-20T11:09:36.123 INFO:tasks.workunit.client.1.smithi173.stdout:6/498: mknod d4/c9b 0 2024-04-20T11:09:36.125 INFO:tasks.workunit.client.1.smithi173.stdout:4/533: mkdir da/d1a/d60/d66/dae 0 2024-04-20T11:09:36.130 INFO:tasks.workunit.client.1.smithi173.stdout:9/483: mkdir d5/d9/d20/d4f/d5e/d7d/d95 0 2024-04-20T11:09:36.132 INFO:tasks.workunit.client.1.smithi173.stdout:4/534: symlink da/d1a/d60/laf 0 2024-04-20T11:09:36.136 INFO:tasks.workunit.client.1.smithi173.stdout:8/477: creat d4/d18/f93 x:0 0 0 2024-04-20T11:09:36.137 INFO:tasks.workunit.client.1.smithi173.stdout:8/478: write d4/f6a [363622,28437] 0 2024-04-20T11:09:36.139 INFO:tasks.workunit.client.1.smithi173.stdout:6/499: getdents d4/d18 0 2024-04-20T11:09:36.140 INFO:tasks.workunit.client.1.smithi173.stdout:2/539: dread d2/d13/f50 [0,4194304] 0 2024-04-20T11:09:36.145 INFO:tasks.workunit.client.1.smithi173.stdout:8/479: mknod d4/d18/d4f/d5b/c94 0 2024-04-20T11:09:36.147 INFO:tasks.workunit.client.1.smithi173.stdout:6/500: rename d4/d18/d7e/d82 to d4/d24/d3e/d9c 0 2024-04-20T11:09:36.148 INFO:tasks.workunit.client.1.smithi173.stdout:4/535: dwrite da/dc/d62/f76 [0,4194304] 0 2024-04-20T11:09:36.153 INFO:tasks.workunit.client.1.smithi173.stdout:8/480: creat d4/d18/d59/d5a/f95 x:0 0 0 2024-04-20T11:09:36.156 INFO:tasks.workunit.client.1.smithi173.stdout:4/536: mknod da/d1e/d52/cb0 0 2024-04-20T11:09:36.158 INFO:tasks.workunit.client.1.smithi173.stdout:4/537: write da/f64 [954239,127381] 0 2024-04-20T11:09:36.164 INFO:tasks.workunit.client.1.smithi173.stdout:2/540: truncate d2/de/f2b 3274251 0 2024-04-20T11:09:36.166 INFO:tasks.workunit.client.1.smithi173.stdout:4/538: fdatasync da/dc/d15/f5f 0 2024-04-20T11:09:36.168 INFO:tasks.workunit.client.1.smithi173.stdout:4/539: creat da/d1a/d60/fb1 x:0 0 0 2024-04-20T11:09:36.170 INFO:tasks.workunit.client.1.smithi173.stdout:4/540: chown da/d1a/d60/fb1 53072 1 2024-04-20T11:09:36.176 INFO:tasks.workunit.client.1.smithi173.stdout:4/541: creat da/d1e/fb2 x:0 0 0 2024-04-20T11:09:36.178 INFO:tasks.workunit.client.1.smithi173.stdout:8/481: truncate d4/f6a 3334463 0 2024-04-20T11:09:36.181 INFO:tasks.workunit.client.1.smithi173.stdout:2/541: truncate d2/d13/d28/f15 1678671 0 2024-04-20T11:09:36.182 INFO:tasks.workunit.client.1.smithi173.stdout:8/482: unlink d4/d18/d3a/d3c/d5f/c6e 0 2024-04-20T11:09:36.196 INFO:tasks.workunit.client.1.smithi173.stdout:4/542: write da/d48/d53/f37 [23402,45250] 0 2024-04-20T11:09:36.198 INFO:tasks.workunit.client.1.smithi173.stdout:8/483: rmdir d4/d18/d3a/d3c/d5f 0 2024-04-20T11:09:36.201 INFO:tasks.workunit.client.1.smithi173.stdout:3/498: sync 2024-04-20T11:09:36.201 INFO:tasks.workunit.client.1.smithi173.stdout:1/435: sync 2024-04-20T11:09:36.201 INFO:tasks.workunit.client.1.smithi173.stdout:1/436: fdatasync f0 0 2024-04-20T11:09:36.204 INFO:tasks.workunit.client.1.smithi173.stdout:3/499: write d7/d17/d61/d83/f79 [951895,67771] 0 2024-04-20T11:09:36.215 INFO:tasks.workunit.client.1.smithi173.stdout:4/543: dwrite da/d1a/d60/f63 [0,4194304] 0 2024-04-20T11:09:36.218 INFO:tasks.workunit.client.1.smithi173.stdout:1/437: truncate d5/d6/f21 3896454 0 2024-04-20T11:09:36.224 INFO:tasks.workunit.client.1.smithi173.stdout:3/500: dread d7/d9/f35 [0,4194304] 0 2024-04-20T11:09:36.225 INFO:tasks.workunit.client.1.smithi173.stdout:1/438: creat d5/d6/d17/d42/f90 x:0 0 0 2024-04-20T11:09:36.225 INFO:tasks.workunit.client.1.smithi173.stdout:4/544: link da/dc/d13/l5d da/d1a/d60/d66/lb3 0 2024-04-20T11:09:36.228 INFO:tasks.workunit.client.1.smithi173.stdout:4/545: write da/dc/d13/f51 [8988527,84597] 0 2024-04-20T11:09:36.232 INFO:tasks.workunit.client.1.smithi173.stdout:3/501: symlink d7/d53/la2 0 2024-04-20T11:09:36.233 INFO:tasks.workunit.client.1.smithi173.stdout:4/546: mkdir da/d1a/d6e/db4 0 2024-04-20T11:09:36.235 INFO:tasks.workunit.client.1.smithi173.stdout:3/502: chown d7/d17/d61/f78 12057441 1 2024-04-20T11:09:36.236 INFO:tasks.workunit.client.1.smithi173.stdout:1/439: dread d5/d6/dc/d4a/f55 [0,4194304] 0 2024-04-20T11:09:36.238 INFO:tasks.workunit.client.1.smithi173.stdout:1/440: readlink d5/d6/d17/l3d 0 2024-04-20T11:09:36.241 INFO:tasks.workunit.client.1.smithi173.stdout:3/503: rename d7/d2b/c81 to d7/ca3 0 2024-04-20T11:09:36.242 INFO:tasks.workunit.client.1.smithi173.stdout:3/504: fsync d7/d53/f87 0 2024-04-20T11:09:36.244 INFO:tasks.workunit.client.1.smithi173.stdout:3/505: fsync d7/d17/d61/f9b 0 2024-04-20T11:09:36.248 INFO:tasks.workunit.client.1.smithi173.stdout:4/547: dread da/dc/f33 [0,4194304] 0 2024-04-20T11:09:36.261 INFO:tasks.workunit.client.1.smithi173.stdout:1/441: dwrite d5/d6/d17/d42/da/df/d40/f4c [0,4194304] 0 2024-04-20T11:09:36.262 INFO:tasks.workunit.client.1.smithi173.stdout:1/442: readlink d5/d6/l71 0 2024-04-20T11:09:36.265 INFO:tasks.workunit.client.1.smithi173.stdout:4/548: dwrite da/d1a/d60/f8c [0,4194304] 0 2024-04-20T11:09:36.266 INFO:tasks.workunit.client.1.smithi173.stdout:4/549: stat da/d1a/l61 0 2024-04-20T11:09:36.268 INFO:tasks.workunit.client.1.smithi173.stdout:1/443: getdents d5/d6/d3b 0 2024-04-20T11:09:36.268 INFO:tasks.workunit.client.1.smithi173.stdout:4/550: dread - da/d48/d53/f9f zero size 2024-04-20T11:09:36.270 INFO:tasks.workunit.client.1.smithi173.stdout:4/551: write da/d1e/d52/f71 [378728,85129] 0 2024-04-20T11:09:36.273 INFO:tasks.workunit.client.1.smithi173.stdout:4/552: write da/dc/d13/d65/f98 [591702,52031] 0 2024-04-20T11:09:36.274 INFO:tasks.workunit.client.1.smithi173.stdout:1/444: write d5/d6/d17/d42/f54 [3462437,118548] 0 2024-04-20T11:09:36.281 INFO:tasks.workunit.client.1.smithi173.stdout:4/553: mkdir da/d1e/db5 0 2024-04-20T11:09:36.283 INFO:tasks.workunit.client.1.smithi173.stdout:3/506: dread d7/f22 [0,4194304] 0 2024-04-20T11:09:36.283 INFO:tasks.workunit.client.1.smithi173.stdout:1/445: mkdir d5/d6/dc/d91 0 2024-04-20T11:09:36.284 INFO:tasks.workunit.client.1.smithi173.stdout:1/446: chown c2 1907 1 2024-04-20T11:09:36.301 INFO:tasks.workunit.client.1.smithi173.stdout:0/446: sync 2024-04-20T11:09:36.304 INFO:tasks.workunit.client.1.smithi173.stdout:4/554: mknod da/d1e/d52/da3/cb6 0 2024-04-20T11:09:36.304 INFO:tasks.workunit.client.1.smithi173.stdout:3/507: fdatasync d7/d17/f2f 0 2024-04-20T11:09:36.305 INFO:tasks.workunit.client.1.smithi173.stdout:1/447: symlink d5/d6/l92 0 2024-04-20T11:09:36.306 INFO:tasks.workunit.client.1.smithi173.stdout:1/448: chown d5/c77 13311 1 2024-04-20T11:09:36.308 INFO:tasks.workunit.client.1.smithi173.stdout:4/555: write da/dc/f33 [4713312,82255] 0 2024-04-20T11:09:36.311 INFO:tasks.workunit.client.1.smithi173.stdout:3/508: creat d7/d17/d58/d7f/fa4 x:0 0 0 2024-04-20T11:09:36.312 INFO:tasks.workunit.client.1.smithi173.stdout:4/556: creat da/dc/fb7 x:0 0 0 2024-04-20T11:09:36.314 INFO:tasks.workunit.client.1.smithi173.stdout:1/449: rmdir d5/d6/dc/d91 0 2024-04-20T11:09:36.316 INFO:tasks.workunit.client.1.smithi173.stdout:3/509: chown d7/d9/c2a 1622638 1 2024-04-20T11:09:36.316 INFO:tasks.workunit.client.1.smithi173.stdout:4/557: creat da/dc/fb8 x:0 0 0 2024-04-20T11:09:36.318 INFO:tasks.workunit.client.1.smithi173.stdout:1/450: mkdir d5/d6/d17/d93 0 2024-04-20T11:09:36.318 INFO:tasks.workunit.client.1.smithi173.stdout:4/558: write da/f25 [4133791,78304] 0 2024-04-20T11:09:36.320 INFO:tasks.workunit.client.1.smithi173.stdout:3/510: truncate d7/d2b/f3d 460722 0 2024-04-20T11:09:36.321 INFO:tasks.workunit.client.1.smithi173.stdout:0/447: dwrite d6/d8/df/d43/f59 [0,4194304] 0 2024-04-20T11:09:36.325 INFO:tasks.workunit.client.1.smithi173.stdout:4/559: mknod da/d1e/da6/cb9 0 2024-04-20T11:09:36.325 INFO:tasks.workunit.client.1.smithi173.stdout:1/451: mkdir d5/d6/dc/d4a/d94 0 2024-04-20T11:09:36.327 INFO:tasks.workunit.client.1.smithi173.stdout:3/511: rmdir d7/d17/d58/d7f 39 2024-04-20T11:09:36.328 INFO:tasks.workunit.client.1.smithi173.stdout:0/448: creat d6/d8/df/d2c/d42/d50/d86/f94 x:0 0 0 2024-04-20T11:09:36.331 INFO:tasks.workunit.client.1.smithi173.stdout:0/449: write d6/d8/df/d2c/d2d/f6a [391026,50179] 0 2024-04-20T11:09:36.333 INFO:tasks.workunit.client.1.smithi173.stdout:1/452: creat d5/d6/d17/d42/d12/d1f/f95 x:0 0 0 2024-04-20T11:09:36.335 INFO:tasks.workunit.client.1.smithi173.stdout:3/512: symlink d7/d17/d58/d8a/la5 0 2024-04-20T11:09:36.338 INFO:tasks.workunit.client.1.smithi173.stdout:0/450: unlink d6/d8/df/d43/f90 0 2024-04-20T11:09:36.342 INFO:tasks.workunit.client.1.smithi173.stdout:1/453: symlink d5/d6/d17/d42/da/d20/d38/l96 0 2024-04-20T11:09:36.344 INFO:tasks.workunit.client.1.smithi173.stdout:4/560: link da/c2f da/dc/d62/cba 0 2024-04-20T11:09:36.347 INFO:tasks.workunit.client.1.smithi173.stdout:1/454: dread d5/d6/dc/f2c [0,4194304] 0 2024-04-20T11:09:36.354 INFO:tasks.workunit.client.1.smithi173.stdout:4/561: dread da/d1a/d60/f8c [0,4194304] 0 2024-04-20T11:09:36.364 INFO:tasks.workunit.client.1.smithi173.stdout:4/562: rmdir da/d1e/d77 0 2024-04-20T11:09:36.368 INFO:tasks.workunit.client.1.smithi173.stdout:0/451: dwrite d6/d8/d51/f5b [0,4194304] 0 2024-04-20T11:09:36.378 INFO:tasks.workunit.client.1.smithi173.stdout:1/455: dread d5/d6/dc/f3a [0,4194304] 0 2024-04-20T11:09:36.378 INFO:tasks.workunit.client.1.smithi173.stdout:3/513: dwrite d7/d2b/f3d [0,4194304] 0 2024-04-20T11:09:36.383 INFO:tasks.workunit.client.1.smithi173.stdout:1/456: dread - d5/d6/d17/d42/d12/d1c/f8b zero size 2024-04-20T11:09:36.391 INFO:tasks.workunit.client.1.smithi173.stdout:0/452: dwrite d6/d8/df/d15/f39 [4194304,4194304] 0 2024-04-20T11:09:36.391 INFO:tasks.workunit.client.1.smithi173.stdout:4/563: dread da/dc/fe [0,4194304] 0 2024-04-20T11:09:36.392 INFO:tasks.workunit.client.1.smithi173.stdout:5/521: sync 2024-04-20T11:09:36.393 INFO:tasks.workunit.client.1.smithi173.stdout:1/457: rename d5/d6/d17/f32 to d5/d6/d17/d93/f97 0 2024-04-20T11:09:36.394 INFO:tasks.workunit.client.1.smithi173.stdout:5/522: stat da/de/d75/f89 0 2024-04-20T11:09:36.402 INFO:tasks.workunit.client.1.smithi173.stdout:4/564: chown da/d48/d53/c3e 640608797 1 2024-04-20T11:09:36.402 INFO:tasks.workunit.client.1.smithi173.stdout:0/453: creat d6/d8/df/d15/f95 x:0 0 0 2024-04-20T11:09:36.405 INFO:tasks.workunit.client.1.smithi173.stdout:0/454: fsync d6/d8/df/d2c/d2d/f36 0 2024-04-20T11:09:36.406 INFO:tasks.workunit.client.1.smithi173.stdout:1/458: rename d5/f30 to d5/d6/d3b/f98 0 2024-04-20T11:09:36.411 INFO:tasks.workunit.client.1.smithi173.stdout:3/514: dwrite d7/d9/f66 [4194304,4194304] 0 2024-04-20T11:09:36.420 INFO:tasks.workunit.client.1.smithi173.stdout:0/455: rmdir d6/d8/d3c 39 2024-04-20T11:09:36.423 INFO:tasks.workunit.client.1.smithi173.stdout:0/456: read d6/d8/df/d15/f70 [3953504,41779] 0 2024-04-20T11:09:36.424 INFO:tasks.workunit.client.1.smithi173.stdout:7/513: sync 2024-04-20T11:09:36.424 INFO:tasks.workunit.client.1.smithi173.stdout:9/484: sync 2024-04-20T11:09:36.424 INFO:tasks.workunit.client.1.smithi173.stdout:2/542: sync 2024-04-20T11:09:36.424 INFO:tasks.workunit.client.1.smithi173.stdout:6/501: sync 2024-04-20T11:09:36.425 INFO:tasks.workunit.client.1.smithi173.stdout:8/484: sync 2024-04-20T11:09:36.427 INFO:tasks.workunit.client.1.smithi173.stdout:7/514: stat d9/da/d17/c24 0 2024-04-20T11:09:36.427 INFO:tasks.workunit.client.1.smithi173.stdout:4/565: creat da/d48/d53/fbb x:0 0 0 2024-04-20T11:09:36.440 INFO:tasks.workunit.client.1.smithi173.stdout:6/502: chown d4/d1c/d8e/d94 21 1 2024-04-20T11:09:36.443 INFO:tasks.workunit.client.1.smithi173.stdout:4/566: write da/dc/d15/f5f [2599101,27571] 0 2024-04-20T11:09:36.446 INFO:tasks.workunit.client.1.smithi173.stdout:5/523: dread da/de/d19/d32/d30/f41 [0,4194304] 0 2024-04-20T11:09:36.446 INFO:tasks.workunit.client.1.smithi173.stdout:7/515: mkdir d9/d33/d40/d89/dba 0 2024-04-20T11:09:36.446 INFO:tasks.workunit.client.1.smithi173.stdout:6/503: symlink d4/d18/d7e/l9d 0 2024-04-20T11:09:36.447 INFO:tasks.workunit.client.1.smithi173.stdout:8/485: creat d4/d18/d4f/f96 x:0 0 0 2024-04-20T11:09:36.448 INFO:tasks.workunit.client.1.smithi173.stdout:6/504: fsync d4/d18/f3f 0 2024-04-20T11:09:36.450 INFO:tasks.workunit.client.1.smithi173.stdout:4/567: mknod da/d1a/d60/d66/d9b/cbc 0 2024-04-20T11:09:36.452 INFO:tasks.workunit.client.1.smithi173.stdout:6/505: rmdir d4/d24/d42 39 2024-04-20T11:09:36.459 INFO:tasks.workunit.client.1.smithi173.stdout:8/486: creat d4/d18/d3a/f97 x:0 0 0 2024-04-20T11:09:36.459 INFO:tasks.workunit.client.1.smithi173.stdout:0/457: dwrite d6/d54/d65/f73 [0,4194304] 0 2024-04-20T11:09:36.460 INFO:tasks.workunit.client.1.smithi173.stdout:2/543: dread d2/de/d11/f23 [0,4194304] 0 2024-04-20T11:09:36.463 INFO:tasks.workunit.client.1.smithi173.stdout:7/516: write d9/d33/d85/fb4 [673014,82335] 0 2024-04-20T11:09:36.467 INFO:tasks.workunit.client.1.smithi173.stdout:8/487: rmdir d4/d18/d59/d5a/d7b 39 2024-04-20T11:09:36.468 INFO:tasks.workunit.client.1.smithi173.stdout:2/544: rmdir d2/de/d11/d9d 39 2024-04-20T11:09:36.468 INFO:tasks.workunit.client.1.smithi173.stdout:5/524: dwrite da/de/d75/f89 [0,4194304] 0 2024-04-20T11:09:36.471 INFO:tasks.workunit.client.1.smithi173.stdout:0/458: mknod d6/d5f/d8f/c96 0 2024-04-20T11:09:36.472 INFO:tasks.workunit.client.1.smithi173.stdout:4/568: dwrite da/dc/d13/d65/f98 [0,4194304] 0 2024-04-20T11:09:36.476 INFO:tasks.workunit.client.1.smithi173.stdout:8/488: symlink d4/d18/d4f/l98 0 2024-04-20T11:09:36.479 INFO:tasks.workunit.client.1.smithi173.stdout:8/489: fdatasync d4/d18/d59/d5a/f67 0 2024-04-20T11:09:36.479 INFO:tasks.workunit.client.1.smithi173.stdout:5/525: unlink da/f2b 0 2024-04-20T11:09:36.481 INFO:tasks.workunit.client.1.smithi173.stdout:2/545: mkdir d2/d13/d46/d4b/dae/db6 0 2024-04-20T11:09:36.481 INFO:tasks.workunit.client.1.smithi173.stdout:0/459: truncate d6/f13 2242804 0 2024-04-20T11:09:36.483 INFO:tasks.workunit.client.1.smithi173.stdout:7/517: link d9/d33/d35/d56/d5a/da4/fb5 d9/da/d17/fbb 0 2024-04-20T11:09:36.487 INFO:tasks.workunit.client.1.smithi173.stdout:5/526: chown da/d34/d7f/da5/db5 15655 1 2024-04-20T11:09:36.489 INFO:tasks.workunit.client.1.smithi173.stdout:0/460: creat d6/d8/d3c/d44/d75/f97 x:0 0 0 2024-04-20T11:09:36.493 INFO:tasks.workunit.client.1.smithi173.stdout:2/546: read d2/d13/d28/d4a/f6c [1184687,38636] 0 2024-04-20T11:09:36.494 INFO:tasks.workunit.client.1.smithi173.stdout:2/547: stat d2/d34/d58/f63 0 2024-04-20T11:09:36.499 INFO:tasks.workunit.client.1.smithi173.stdout:6/506: dread d4/d18/f62 [0,4194304] 0 2024-04-20T11:09:36.501 INFO:tasks.workunit.client.1.smithi173.stdout:8/490: dwrite d4/d18/d59/d6d/f84 [0,4194304] 0 2024-04-20T11:09:36.504 INFO:tasks.workunit.client.1.smithi173.stdout:8/491: write d4/d18/d59/d5a/f67 [904594,29501] 0 2024-04-20T11:09:36.505 INFO:tasks.workunit.client.1.smithi173.stdout:5/527: rmdir da/db7 0 2024-04-20T11:09:36.506 INFO:tasks.workunit.client.1.smithi173.stdout:6/507: mknod d4/d14/d5f/c9e 0 2024-04-20T11:09:36.507 INFO:tasks.workunit.client.1.smithi173.stdout:2/548: truncate d2/de/f2b 2783335 0 2024-04-20T11:09:36.509 INFO:tasks.workunit.client.1.smithi173.stdout:4/569: dwrite da/d1e/f38 [0,4194304] 0 2024-04-20T11:09:36.512 INFO:tasks.workunit.client.1.smithi173.stdout:0/461: getdents d6/d8/df/d2c/d42 0 2024-04-20T11:09:36.516 INFO:tasks.workunit.client.1.smithi173.stdout:7/518: rename d9/d33/d35/d56/d5a to d9/dbc 0 2024-04-20T11:09:36.518 INFO:tasks.workunit.client.1.smithi173.stdout:2/549: mknod d2/d34/cb7 0 2024-04-20T11:09:36.528 INFO:tasks.workunit.client.1.smithi173.stdout:0/462: creat d6/d54/d65/f98 x:0 0 0 2024-04-20T11:09:36.529 INFO:tasks.workunit.client.1.smithi173.stdout:7/519: symlink d9/d33/d35/d4a/lbd 0 2024-04-20T11:09:36.529 INFO:tasks.workunit.client.1.smithi173.stdout:4/570: dwrite da/dc/d13/f29 [4194304,4194304] 0 2024-04-20T11:09:36.530 INFO:tasks.workunit.client.1.smithi173.stdout:6/508: dwrite d4/d24/d42/f72 [0,4194304] 0 2024-04-20T11:09:36.532 INFO:tasks.workunit.client.1.smithi173.stdout:4/571: fsync da/f64 0 2024-04-20T11:09:36.532 INFO:tasks.workunit.client.1.smithi173.stdout:2/550: write d2/de/d11/f23 [1687342,114172] 0 2024-04-20T11:09:36.534 INFO:tasks.workunit.client.1.smithi173.stdout:4/572: fdatasync da/d1e/fb2 0 2024-04-20T11:09:36.537 INFO:tasks.workunit.client.1.smithi173.stdout:1/459: sync 2024-04-20T11:09:36.537 INFO:tasks.workunit.client.1.smithi173.stdout:7/520: rmdir d9/d33/d91 39 2024-04-20T11:09:36.542 INFO:tasks.workunit.client.1.smithi173.stdout:4/573: symlink da/d1a/d60/d66/lbd 0 2024-04-20T11:09:36.543 INFO:tasks.workunit.client.1.smithi173.stdout:1/460: stat d5/d6/dc/c2a 0 2024-04-20T11:09:36.544 INFO:tasks.workunit.client.1.smithi173.stdout:7/521: rename d9/d33/d81/f95 to d9/dbc/fbe 0 2024-04-20T11:09:36.548 INFO:tasks.workunit.client.1.smithi173.stdout:8/492: dread d4/f10 [0,4194304] 0 2024-04-20T11:09:36.549 INFO:tasks.workunit.client.1.smithi173.stdout:4/574: mknod da/dc/d62/cbe 0 2024-04-20T11:09:36.558 INFO:tasks.workunit.client.1.smithi173.stdout:5/528: dread da/de/d19/d2c/fa8 [4194304,4194304] 0 2024-04-20T11:09:36.558 INFO:tasks.workunit.client.1.smithi173.stdout:4/575: creat da/d1a/d60/d66/d9b/fbf x:0 0 0 2024-04-20T11:09:36.559 INFO:tasks.workunit.client.1.smithi173.stdout:2/551: write d2/f5c [722835,70813] 0 2024-04-20T11:09:36.566 INFO:tasks.workunit.client.1.smithi173.stdout:7/522: creat d9/fbf x:0 0 0 2024-04-20T11:09:36.567 INFO:tasks.workunit.client.1.smithi173.stdout:0/463: dwrite d6/d8/df/d2c/d42/f56 [4194304,4194304] 0 2024-04-20T11:09:36.567 INFO:tasks.workunit.client.1.smithi173.stdout:4/576: mknod da/d1e/d52/cc0 0 2024-04-20T11:09:36.573 INFO:tasks.workunit.client.1.smithi173.stdout:2/552: mkdir d2/de/d11/db8 0 2024-04-20T11:09:36.574 INFO:tasks.workunit.client.1.smithi173.stdout:0/464: truncate d6/d8/df/d2c/d42/f6c 1199848 0 2024-04-20T11:09:36.574 INFO:tasks.workunit.client.1.smithi173.stdout:4/577: fdatasync da/dc/f27 0 2024-04-20T11:09:36.575 INFO:tasks.workunit.client.1.smithi173.stdout:0/465: chown d6/d8/df/d2c/d42/f84 6843 1 2024-04-20T11:09:36.576 INFO:tasks.workunit.client.1.smithi173.stdout:7/523: rename d9/da/d17/d2b/f83 to d9/d33/d91/d99/fc0 0 2024-04-20T11:09:36.589 INFO:tasks.workunit.client.1.smithi173.stdout:0/466: dread d6/d8/fe [0,4194304] 0 2024-04-20T11:09:36.590 INFO:tasks.workunit.client.1.smithi173.stdout:5/529: getdents da/d4f 0 2024-04-20T11:09:36.590 INFO:tasks.workunit.client.1.smithi173.stdout:4/578: dwrite f8 [0,4194304] 0 2024-04-20T11:09:36.592 INFO:tasks.workunit.client.1.smithi173.stdout:4/579: stat da/d1e/c36 0 2024-04-20T11:09:36.592 INFO:tasks.workunit.client.1.smithi173.stdout:7/524: dwrite d9/fa0 [0,4194304] 0 2024-04-20T11:09:36.592 INFO:tasks.workunit.client.1.smithi173.stdout:1/461: truncate d5/d6/dc/f3a 1947734 0 2024-04-20T11:09:36.596 INFO:tasks.workunit.client.1.smithi173.stdout:4/580: fsync da/dc/d62/f76 0 2024-04-20T11:09:36.596 INFO:tasks.workunit.client.1.smithi173.stdout:8/493: write d4/d18/d3a/d3c/f54 [1439352,86215] 0 2024-04-20T11:09:36.599 INFO:tasks.workunit.client.1.smithi173.stdout:1/462: fsync d5/d6/d17/d42/d12/d1f/f63 0 2024-04-20T11:09:36.599 INFO:tasks.workunit.client.1.smithi173.stdout:2/553: mknod d2/d13/d46/d4b/dae/db6/cb9 0 2024-04-20T11:09:36.601 INFO:tasks.workunit.client.1.smithi173.stdout:7/525: symlink d9/d33/d35/d54/lc1 0 2024-04-20T11:09:36.603 INFO:tasks.workunit.client.1.smithi173.stdout:4/581: write da/d1e/d52/f5c [2633669,94347] 0 2024-04-20T11:09:36.604 INFO:tasks.workunit.client.1.smithi173.stdout:0/467: write d6/d8/df/d2c/d42/f7a [1374596,10298] 0 2024-04-20T11:09:36.607 INFO:tasks.workunit.client.1.smithi173.stdout:5/530: symlink da/de/d19/d32/d30/d35/d48/lb8 0 2024-04-20T11:09:36.608 INFO:tasks.workunit.client.1.smithi173.stdout:0/468: write d6/d8/d51/f71 [243416,30405] 0 2024-04-20T11:09:36.618 INFO:tasks.workunit.client.1.smithi173.stdout:4/582: symlink da/dc/d15/d8e/lc1 0 2024-04-20T11:09:36.619 INFO:tasks.workunit.client.1.smithi173.stdout:1/463: creat d5/d6/d17/d42/da/df/d40/f99 x:0 0 0 2024-04-20T11:09:36.621 INFO:tasks.workunit.client.1.smithi173.stdout:9/485: sync 2024-04-20T11:09:36.622 INFO:tasks.workunit.client.1.smithi173.stdout:3/515: sync 2024-04-20T11:09:36.622 INFO:tasks.workunit.client.1.smithi173.stdout:5/531: unlink da/de/d19/d32/l18 0 2024-04-20T11:09:36.626 INFO:tasks.workunit.client.1.smithi173.stdout:0/469: dwrite d6/d54/f81 [0,4194304] 0 2024-04-20T11:09:36.627 INFO:tasks.workunit.client.1.smithi173.stdout:7/526: mknod d9/d33/d35/d54/d9e/cc2 0 2024-04-20T11:09:36.629 INFO:tasks.workunit.client.1.smithi173.stdout:4/583: mknod da/d1a/d60/d66/cc2 0 2024-04-20T11:09:36.631 INFO:tasks.workunit.client.1.smithi173.stdout:4/584: chown da/f64 256225 1 2024-04-20T11:09:36.636 INFO:tasks.workunit.client.1.smithi173.stdout:9/486: dwrite d5/d9/d20/d29/f5c [0,4194304] 0 2024-04-20T11:09:36.636 INFO:tasks.workunit.client.1.smithi173.stdout:5/532: mkdir da/de/d19/d32/db9 0 2024-04-20T11:09:36.641 INFO:tasks.workunit.client.1.smithi173.stdout:8/494: rename d4/d18/d3a/d3c/d86/c91 to d4/d18/d3a/c99 0 2024-04-20T11:09:36.641 INFO:tasks.workunit.client.1.smithi173.stdout:4/585: mknod da/dc/d15/cc3 0 2024-04-20T11:09:36.642 INFO:tasks.workunit.client.1.smithi173.stdout:8/495: stat d4/d18/d4f 0 2024-04-20T11:09:36.643 INFO:tasks.workunit.client.1.smithi173.stdout:8/496: fsync d4/f8 0 2024-04-20T11:09:36.644 INFO:tasks.workunit.client.1.smithi173.stdout:2/554: getdents d2/d13/d46/d4b/dae/db6 0 2024-04-20T11:09:36.647 INFO:tasks.workunit.client.1.smithi173.stdout:7/527: mkdir d9/da/dc3 0 2024-04-20T11:09:36.648 INFO:tasks.workunit.client.1.smithi173.stdout:4/586: creat da/fc4 x:0 0 0 2024-04-20T11:09:36.650 INFO:tasks.workunit.client.1.smithi173.stdout:4/587: fdatasync da/dc/d62/f76 0 2024-04-20T11:09:36.652 INFO:tasks.workunit.client.1.smithi173.stdout:0/470: dread d6/d8/fe [0,4194304] 0 2024-04-20T11:09:36.654 INFO:tasks.workunit.client.1.smithi173.stdout:0/471: read d6/f13 [2137072,77763] 0 2024-04-20T11:09:36.656 INFO:tasks.workunit.client.1.smithi173.stdout:3/516: getdents d7 0 2024-04-20T11:09:36.658 INFO:tasks.workunit.client.1.smithi173.stdout:7/528: creat d9/dbc/da4/dad/fc4 x:0 0 0 2024-04-20T11:09:36.662 INFO:tasks.workunit.client.1.smithi173.stdout:2/555: truncate d2/d34/f51 1072824 0 2024-04-20T11:09:36.665 INFO:tasks.workunit.client.1.smithi173.stdout:1/464: rename d5/d6/d17/d42/da/f14 to d5/d6/d17/f9a 0 2024-04-20T11:09:36.666 INFO:tasks.workunit.client.1.smithi173.stdout:2/556: read d2/de/f7a [2382340,106865] 0 2024-04-20T11:09:36.667 INFO:tasks.workunit.client.1.smithi173.stdout:1/465: write d5/d6/f73 [2532239,108028] 0 2024-04-20T11:09:36.667 INFO:tasks.workunit.client.1.smithi173.stdout:4/588: creat da/d1a/d60/d7d/fc5 x:0 0 0 2024-04-20T11:09:36.670 INFO:tasks.workunit.client.1.smithi173.stdout:4/589: chown da/dc/d13/d65/l79 212797095 1 2024-04-20T11:09:36.671 INFO:tasks.workunit.client.1.smithi173.stdout:7/529: unlink d9/d33/d85/f96 0 2024-04-20T11:09:36.672 INFO:tasks.workunit.client.1.smithi173.stdout:7/530: truncate d9/d33/f47 1152864 0 2024-04-20T11:09:36.673 INFO:tasks.workunit.client.1.smithi173.stdout:9/487: getdents d5 0 2024-04-20T11:09:36.675 INFO:tasks.workunit.client.1.smithi173.stdout:7/531: write d9/d33/d81/f9b [358014,109737] 0 2024-04-20T11:09:36.675 INFO:tasks.workunit.client.1.smithi173.stdout:9/488: write d5/d9/f26 [91202,5831] 0 2024-04-20T11:09:36.680 INFO:tasks.workunit.client.1.smithi173.stdout:0/472: truncate d6/d24/f4f 902451 0 2024-04-20T11:09:36.682 INFO:tasks.workunit.client.1.smithi173.stdout:2/557: mknod d2/d13/d46/d4b/dae/cba 0 2024-04-20T11:09:36.682 INFO:tasks.workunit.client.1.smithi173.stdout:4/590: dread da/d1a/d60/f63 [0,4194304] 0 2024-04-20T11:09:36.682 INFO:tasks.workunit.client.1.smithi173.stdout:2/558: fdatasync d2/d13/f38 0 2024-04-20T11:09:36.692 INFO:tasks.workunit.client.1.smithi173.stdout:0/473: dread d6/f7 [0,4194304] 0 2024-04-20T11:09:36.693 INFO:tasks.workunit.client.1.smithi173.stdout:7/532: stat d9/d1f/c84 0 2024-04-20T11:09:36.695 INFO:tasks.workunit.client.1.smithi173.stdout:8/497: write d4/d18/d3a/d3c/f53 [1514333,2762] 0 2024-04-20T11:09:36.699 INFO:tasks.workunit.client.1.smithi173.stdout:4/591: rmdir da/d1a/d6e 39 2024-04-20T11:09:36.704 INFO:tasks.workunit.client.1.smithi173.stdout:2/559: creat d2/d13/d46/d4b/dae/fbb x:0 0 0 2024-04-20T11:09:36.710 INFO:tasks.workunit.client.1.smithi173.stdout:5/533: rename da/de/d19/d32/d30/d35/d48/l93 to da/de/d75/lba 0 2024-04-20T11:09:36.714 INFO:tasks.workunit.client.1.smithi173.stdout:0/474: symlink d6/d24/l99 0 2024-04-20T11:09:36.716 INFO:tasks.workunit.client.1.smithi173.stdout:7/533: rmdir d9/d33/d91/d99 39 2024-04-20T11:09:36.717 INFO:tasks.workunit.client.1.smithi173.stdout:7/534: write d9/d33/d85/fa6 [1037713,10092] 0 2024-04-20T11:09:36.718 INFO:tasks.workunit.client.1.smithi173.stdout:9/489: fsync d5/d9/d20/d29/d2d/f3f 0 2024-04-20T11:09:36.719 INFO:tasks.workunit.client.1.smithi173.stdout:7/535: dread - d9/da/d43/f57 zero size 2024-04-20T11:09:36.723 INFO:tasks.workunit.client.1.smithi173.stdout:9/490: read d5/d9/d20/d29/f64 [3976956,98692] 0 2024-04-20T11:09:36.725 INFO:tasks.workunit.client.1.smithi173.stdout:8/498: read d4/d18/d59/d5a/d7b/d8c/f8e [754278,103403] 0 2024-04-20T11:09:36.730 INFO:tasks.workunit.client.1.smithi173.stdout:6/509: sync 2024-04-20T11:09:36.730 INFO:tasks.workunit.client.1.smithi173.stdout:0/475: creat d6/d8/df/d43/d7f/f9a x:0 0 0 2024-04-20T11:09:36.731 INFO:tasks.workunit.client.1.smithi173.stdout:6/510: write d4/f60 [2125997,3270] 0 2024-04-20T11:09:36.731 INFO:tasks.workunit.client.1.smithi173.stdout:0/476: readlink d6/d8/df/d15/l3e 0 2024-04-20T11:09:36.737 INFO:tasks.workunit.client.1.smithi173.stdout:7/536: creat d9/d33/d85/fc5 x:0 0 0 2024-04-20T11:09:36.739 INFO:tasks.workunit.client.1.smithi173.stdout:7/537: chown d9/d33/d35 1 1 2024-04-20T11:09:36.742 INFO:tasks.workunit.client.1.smithi173.stdout:7/538: chown d9/da/d17/d2b/f80 193 1 2024-04-20T11:09:36.744 INFO:tasks.workunit.client.1.smithi173.stdout:7/539: stat d9/d33/d40/d89/cb0 0 2024-04-20T11:09:36.746 INFO:tasks.workunit.client.1.smithi173.stdout:7/540: fdatasync d9/d1f/fa5 0 2024-04-20T11:09:36.748 INFO:tasks.workunit.client.1.smithi173.stdout:2/560: mkdir d2/d13/d9a/dbc 0 2024-04-20T11:09:36.752 INFO:tasks.workunit.client.1.smithi173.stdout:1/466: rename d5/d6/d17/l35 to d5/d6/dc/d7f/l9b 0 2024-04-20T11:09:36.754 INFO:tasks.workunit.client.1.smithi173.stdout:1/467: read - d5/d6/d17/d42/d12/d1c/f8b zero size 2024-04-20T11:09:36.756 INFO:tasks.workunit.client.1.smithi173.stdout:6/511: mkdir d4/d14/d5f/d9f 0 2024-04-20T11:09:36.756 INFO:tasks.workunit.client.1.smithi173.stdout:1/468: fdatasync d5/d6/dc/d4a/f72 0 2024-04-20T11:09:36.771 INFO:tasks.workunit.client.1.smithi173.stdout:5/534: rename da/de/d19/d32/d30/dad to da/de/d19/d32/d30/dbb 0 2024-04-20T11:09:36.771 INFO:tasks.workunit.client.1.smithi173.stdout:5/535: stat f7 0 2024-04-20T11:09:36.772 INFO:tasks.workunit.client.1.smithi173.stdout:7/541: write d9/da/d17/d2b/daa/f6d [4876844,128539] 0 2024-04-20T11:09:36.777 INFO:tasks.workunit.client.1.smithi173.stdout:0/477: creat d6/d54/d91/f9b x:0 0 0 2024-04-20T11:09:36.777 INFO:tasks.workunit.client.1.smithi173.stdout:7/542: stat d9/da/d17/l18 0 2024-04-20T11:09:36.778 INFO:tasks.workunit.client.1.smithi173.stdout:0/478: dread - d6/d24/f74 zero size 2024-04-20T11:09:36.778 INFO:tasks.workunit.client.1.smithi173.stdout:1/469: creat d5/d6/d17/d42/da/d20/d48/f9c x:0 0 0 2024-04-20T11:09:36.780 INFO:tasks.workunit.client.1.smithi173.stdout:0/479: write d6/d8/d3c/d44/d75/f97 [572573,128044] 0 2024-04-20T11:09:36.780 INFO:tasks.workunit.client.1.smithi173.stdout:9/491: rmdir d5/d7f/d94 0 2024-04-20T11:09:36.782 INFO:tasks.workunit.client.1.smithi173.stdout:8/499: creat d4/d18/d4f/f9a x:0 0 0 2024-04-20T11:09:36.784 INFO:tasks.workunit.client.1.smithi173.stdout:4/592: getdents da/dc/d62/d84 0 2024-04-20T11:09:36.785 INFO:tasks.workunit.client.1.smithi173.stdout:3/517: sync 2024-04-20T11:09:36.806 INFO:tasks.workunit.client.1.smithi173.stdout:7/543: mkdir d9/da/d31/dc6 0 2024-04-20T11:09:36.813 INFO:tasks.workunit.client.1.smithi173.stdout:4/593: creat da/dc/d13/fc6 x:0 0 0 2024-04-20T11:09:36.816 INFO:tasks.workunit.client.1.smithi173.stdout:4/594: write da/dc/d15/d8e/fa7 [771681,54340] 0 2024-04-20T11:09:36.817 INFO:tasks.workunit.client.1.smithi173.stdout:2/561: creat d2/d13/d46/d53/fbd x:0 0 0 2024-04-20T11:09:36.818 INFO:tasks.workunit.client.1.smithi173.stdout:6/512: creat d4/d1c/fa0 x:0 0 0 2024-04-20T11:09:36.820 INFO:tasks.workunit.client.1.smithi173.stdout:2/562: dread - d2/de/f84 zero size 2024-04-20T11:09:36.820 INFO:tasks.workunit.client.1.smithi173.stdout:1/470: symlink d5/l9d 0 2024-04-20T11:09:36.820 INFO:tasks.workunit.client.1.smithi173.stdout:6/513: chown d4/d1c/d8e/d90 1137692 1 2024-04-20T11:09:36.821 INFO:tasks.workunit.client.1.smithi173.stdout:8/500: mkdir d4/d18/d3a/d9b 0 2024-04-20T11:09:36.823 INFO:tasks.workunit.client.1.smithi173.stdout:6/514: write d4/d24/d42/f67 [846501,23189] 0 2024-04-20T11:09:36.824 INFO:tasks.workunit.client.1.smithi173.stdout:0/480: rename d6/d8/df/l5a to d6/d8/d3c/d44/l9c 0 2024-04-20T11:09:36.826 INFO:tasks.workunit.client.1.smithi173.stdout:3/518: symlink d7/d17/d58/d7f/la6 0 2024-04-20T11:09:36.833 INFO:tasks.workunit.client.1.smithi173.stdout:4/595: dread da/dc/fe [0,4194304] 0 2024-04-20T11:09:36.836 INFO:tasks.workunit.client.1.smithi173.stdout:3/519: creat d7/d2b/d8e/fa7 x:0 0 0 2024-04-20T11:09:36.838 INFO:tasks.workunit.client.1.smithi173.stdout:8/501: dread d4/d18/d3a/d3c/f34 [4194304,4194304] 0 2024-04-20T11:09:36.844 INFO:tasks.workunit.client.1.smithi173.stdout:4/596: mknod da/dc/cc7 0 2024-04-20T11:09:36.845 INFO:tasks.workunit.client.1.smithi173.stdout:1/471: dwrite d5/d6/d17/d42/da/d20/d38/d52/d75/f7b [0,4194304] 0 2024-04-20T11:09:36.846 INFO:tasks.workunit.client.1.smithi173.stdout:5/536: rmdir da/d9e/db1 0 2024-04-20T11:09:36.847 INFO:tasks.workunit.client.1.smithi173.stdout:6/515: dwrite d4/d18/d7e/f80 [0,4194304] 0 2024-04-20T11:09:36.853 INFO:tasks.workunit.client.1.smithi173.stdout:9/492: rename d5/d9/d20/f85 to d5/d9/d20/d29/f96 0 2024-04-20T11:09:36.855 INFO:tasks.workunit.client.1.smithi173.stdout:3/520: dwrite d7/d17/f2c [0,4194304] 0 2024-04-20T11:09:36.864 INFO:tasks.workunit.client.1.smithi173.stdout:3/521: dread d7/d9/d30/f60 [0,4194304] 0 2024-04-20T11:09:36.867 INFO:tasks.workunit.client.1.smithi173.stdout:6/516: dread d4/d18/f62 [0,4194304] 0 2024-04-20T11:09:36.871 INFO:tasks.workunit.client.1.smithi173.stdout:0/481: getdents d6/d8/df/d43 0 2024-04-20T11:09:36.873 INFO:tasks.workunit.client.1.smithi173.stdout:9/493: dread d5/d9/d20/d2b/d59/f4e [0,4194304] 0 2024-04-20T11:09:36.874 INFO:tasks.workunit.client.1.smithi173.stdout:9/494: stat d5/d9/d20/l51 0 2024-04-20T11:09:36.880 INFO:tasks.workunit.client.1.smithi173.stdout:3/522: truncate d7/d17/f2f 288949 0 2024-04-20T11:09:36.887 INFO:tasks.workunit.client.1.smithi173.stdout:7/544: rename d9/d33/d85/d97/cb6 to d9/d33/d91/cc7 0 2024-04-20T11:09:36.888 INFO:tasks.workunit.client.1.smithi173.stdout:9/495: dwrite d5/d9/d20/d29/d18/f1f [0,4194304] 0 2024-04-20T11:09:36.890 INFO:tasks.workunit.client.1.smithi173.stdout:5/537: dread da/de/d19/d32/d30/d35/f54 [0,4194304] 0 2024-04-20T11:09:36.893 INFO:tasks.workunit.client.1.smithi173.stdout:6/517: creat d4/d1c/fa1 x:0 0 0 2024-04-20T11:09:36.893 INFO:tasks.workunit.client.1.smithi173.stdout:2/563: write d2/de/d11/d9d/f8e [841522,97012] 0 2024-04-20T11:09:36.895 INFO:tasks.workunit.client.1.smithi173.stdout:7/545: creat d9/d33/d35/d54/fc8 x:0 0 0 2024-04-20T11:09:36.897 INFO:tasks.workunit.client.1.smithi173.stdout:5/538: mkdir da/d4f/d5e/dbc 0 2024-04-20T11:09:36.899 INFO:tasks.workunit.client.1.smithi173.stdout:6/518: truncate d4/f27 1136355 0 2024-04-20T11:09:36.900 INFO:tasks.workunit.client.1.smithi173.stdout:2/564: dread - d2/d13/d46/d4b/dae/fbb zero size 2024-04-20T11:09:36.902 INFO:tasks.workunit.client.1.smithi173.stdout:8/502: rename d4/d18/d59/d5a/f5d to d4/d18/f9c 0 2024-04-20T11:09:36.903 INFO:tasks.workunit.client.1.smithi173.stdout:2/565: truncate d2/d13/d46/f85 4375016 0 2024-04-20T11:09:36.905 INFO:tasks.workunit.client.1.smithi173.stdout:9/496: dread d5/d9/d20/d29/f11 [4194304,4194304] 0 2024-04-20T11:09:36.906 INFO:tasks.workunit.client.1.smithi173.stdout:1/472: write d5/d6/dc/f3a [501461,65660] 0 2024-04-20T11:09:36.909 INFO:tasks.workunit.client.1.smithi173.stdout:9/497: truncate d5/d9/f6e 982029 0 2024-04-20T11:09:36.912 INFO:tasks.workunit.client.1.smithi173.stdout:1/473: write d5/d6/d17/d42/d12/d1c/f6f [652393,96215] 0 2024-04-20T11:09:36.917 INFO:tasks.workunit.client.1.smithi173.stdout:8/503: creat d4/d18/d3a/f9d x:0 0 0 2024-04-20T11:09:36.918 INFO:tasks.workunit.client.1.smithi173.stdout:5/539: dread da/de/f5b [0,4194304] 0 2024-04-20T11:09:36.921 INFO:tasks.workunit.client.1.smithi173.stdout:8/504: stat d4/l11 0 2024-04-20T11:09:36.922 INFO:tasks.workunit.client.1.smithi173.stdout:7/546: dwrite d9/fb7 [0,4194304] 0 2024-04-20T11:09:36.930 INFO:tasks.workunit.client.1.smithi173.stdout:5/540: mkdir da/de/d19/d32/d66/dbd 0 2024-04-20T11:09:36.933 INFO:tasks.workunit.client.1.smithi173.stdout:3/523: dwrite d7/f25 [4194304,4194304] 0 2024-04-20T11:09:36.933 INFO:tasks.workunit.client.1.smithi173.stdout:7/547: chown d9/d33/d40/l7e 25761 1 2024-04-20T11:09:36.933 INFO:tasks.workunit.client.1.smithi173.stdout:0/482: dwrite d6/d24/f4f [0,4194304] 0 2024-04-20T11:09:36.934 INFO:tasks.workunit.client.1.smithi173.stdout:8/505: write d4/d18/d3a/d3c/f52 [5004165,6051] 0 2024-04-20T11:09:36.934 INFO:tasks.workunit.client.1.smithi173.stdout:6/519: dwrite d4/d14/d5f/f76 [0,4194304] 0 2024-04-20T11:09:36.936 INFO:tasks.workunit.client.1.smithi173.stdout:8/506: write d4/f8 [546682,81853] 0 2024-04-20T11:09:36.942 INFO:tasks.workunit.client.1.smithi173.stdout:3/524: creat d7/d17/fa8 x:0 0 0 2024-04-20T11:09:36.942 INFO:tasks.workunit.client.1.smithi173.stdout:7/548: mkdir d9/d33/d40/dc9 0 2024-04-20T11:09:36.943 INFO:tasks.workunit.client.1.smithi173.stdout:0/483: mknod d6/d54/c9d 0 2024-04-20T11:09:36.946 INFO:tasks.workunit.client.1.smithi173.stdout:6/520: write d4/d18/f58 [3465888,15209] 0 2024-04-20T11:09:36.946 INFO:tasks.workunit.client.1.smithi173.stdout:5/541: symlink da/d34/lbe 0 2024-04-20T11:09:36.947 INFO:tasks.workunit.client.1.smithi173.stdout:8/507: mknod d4/d18/d3a/d9b/c9e 0 2024-04-20T11:09:36.950 INFO:tasks.workunit.client.1.smithi173.stdout:0/484: mknod d6/d8/d3c/d77/c9e 0 2024-04-20T11:09:36.952 INFO:tasks.workunit.client.1.smithi173.stdout:7/549: symlink d9/d33/d85/d97/lca 0 2024-04-20T11:09:36.954 INFO:tasks.workunit.client.1.smithi173.stdout:5/542: readlink da/de/d75/lba 0 2024-04-20T11:09:36.954 INFO:tasks.workunit.client.1.smithi173.stdout:3/525: link d7/d85/l9c d7/d17/la9 0 2024-04-20T11:09:36.955 INFO:tasks.workunit.client.1.smithi173.stdout:8/508: write d4/f10 [5061344,52161] 0 2024-04-20T11:09:36.956 INFO:tasks.workunit.client.1.smithi173.stdout:3/526: chown d7/c6a 11 1 2024-04-20T11:09:36.958 INFO:tasks.workunit.client.1.smithi173.stdout:5/543: write da/de/f2d [6713427,101023] 0 2024-04-20T11:09:36.961 INFO:tasks.workunit.client.1.smithi173.stdout:5/544: readlink da/de/d75/d40/d59/l8b 0 2024-04-20T11:09:36.963 INFO:tasks.workunit.client.1.smithi173.stdout:3/527: truncate f4 2457758 0 2024-04-20T11:09:36.963 INFO:tasks.workunit.client.1.smithi173.stdout:5/545: write f9 [1664670,88833] 0 2024-04-20T11:09:36.966 INFO:tasks.workunit.client.1.smithi173.stdout:8/509: rename d4/d18/d3a/f6f to d4/f9f 0 2024-04-20T11:09:36.967 INFO:tasks.workunit.client.1.smithi173.stdout:8/510: readlink d4/l11 0 2024-04-20T11:09:36.968 INFO:tasks.workunit.client.1.smithi173.stdout:9/498: dread d5/d9/d20/d2b/d59/d44/f6a [0,4194304] 0 2024-04-20T11:09:36.969 INFO:tasks.workunit.client.1.smithi173.stdout:3/528: fsync d7/d9/f2e 0 2024-04-20T11:09:37.033 INFO:tasks.workunit.client.1.smithi173.stdout:3/529: stat d7/d2b/c7c 0 2024-04-20T11:09:37.036 INFO:tasks.workunit.client.1.smithi173.stdout:0/485: dwrite d6/d8/df/d2c/d42/d50/d86/f94 [0,4194304] 0 2024-04-20T11:09:37.037 INFO:tasks.workunit.client.1.smithi173.stdout:3/530: stat d7/d17/d61/f97 0 2024-04-20T11:09:37.041 INFO:tasks.workunit.client.1.smithi173.stdout:9/499: rmdir d5/d9/d20/d2b/d59/d44 39 2024-04-20T11:09:37.042 INFO:tasks.workunit.client.1.smithi173.stdout:3/531: chown d7/d53 12 1 2024-04-20T11:09:37.047 INFO:tasks.workunit.client.1.smithi173.stdout:0/486: chown d6/d8/df/d43 86 1 2024-04-20T11:09:37.050 INFO:tasks.workunit.client.1.smithi173.stdout:3/532: write d7/d2b/f3d [1999158,32400] 0 2024-04-20T11:09:37.053 INFO:tasks.workunit.client.1.smithi173.stdout:1/474: dread d5/d6/d17/d42/da/d20/d48/f61 [0,4194304] 0 2024-04-20T11:09:37.055 INFO:tasks.workunit.client.1.smithi173.stdout:5/546: dwrite da/de/ff [0,4194304] 0 2024-04-20T11:09:37.057 INFO:tasks.workunit.client.1.smithi173.stdout:8/511: dwrite d4/f9f [0,4194304] 0 2024-04-20T11:09:37.058 INFO:tasks.workunit.client.1.smithi173.stdout:6/521: dread d4/d24/d3e/d9c/f8b [0,4194304] 0 2024-04-20T11:09:37.058 INFO:tasks.workunit.client.1.smithi173.stdout:1/475: chown d5/d6/d3b/f67 0 1 2024-04-20T11:09:37.059 INFO:tasks.workunit.client.1.smithi173.stdout:5/547: write da/de/d19/d2c/fa6 [888211,54694] 0 2024-04-20T11:09:37.060 INFO:tasks.workunit.client.1.smithi173.stdout:3/533: rename d7/f25 to d7/d17/d61/d83/faa 0 2024-04-20T11:09:37.064 INFO:tasks.workunit.client.1.smithi173.stdout:6/522: rmdir d4/d14/d79 39 2024-04-20T11:09:37.068 INFO:tasks.workunit.client.1.smithi173.stdout:9/500: dread d5/d9/d20/d29/d2d/f43 [0,4194304] 0 2024-04-20T11:09:37.071 INFO:tasks.workunit.client.1.smithi173.stdout:8/512: unlink d4/d18/d3a/d3c/f40 0 2024-04-20T11:09:37.072 INFO:tasks.workunit.client.1.smithi173.stdout:0/487: link d6/d8/df/d15/l3e d6/d8/df/d15/l9f 0 2024-04-20T11:09:37.074 INFO:tasks.workunit.client.1.smithi173.stdout:8/513: write d4/d18/d3a/f9d [394660,108996] 0 2024-04-20T11:09:37.074 INFO:tasks.workunit.client.1.smithi173.stdout:1/476: creat d5/d6/d17/d42/d12/d1c/d85/f9e x:0 0 0 2024-04-20T11:09:37.075 INFO:tasks.workunit.client.1.smithi173.stdout:0/488: fsync d6/d8/d3c/d44/d75/f97 0 2024-04-20T11:09:37.077 INFO:tasks.workunit.client.1.smithi173.stdout:8/514: dread - d4/d18/f93 zero size 2024-04-20T11:09:37.077 INFO:tasks.workunit.client.1.smithi173.stdout:0/489: stat d6/d8/df/d2c/d2d/f36 0 2024-04-20T11:09:37.078 INFO:tasks.workunit.client.1.smithi173.stdout:9/501: creat d5/d9/d20/d2b/f97 x:0 0 0 2024-04-20T11:09:37.079 INFO:tasks.workunit.client.1.smithi173.stdout:1/477: write d5/d6/d17/d42/d12/f15 [5054974,3553] 0 2024-04-20T11:09:37.081 INFO:tasks.workunit.client.1.smithi173.stdout:9/502: dread - d5/d9/d20/d2b/f97 zero size 2024-04-20T11:09:37.083 INFO:tasks.workunit.client.1.smithi173.stdout:6/523: creat d4/d24/fa2 x:0 0 0 2024-04-20T11:09:37.086 INFO:tasks.workunit.client.1.smithi173.stdout:0/490: mkdir d6/d8/df/da0 0 2024-04-20T11:09:37.087 INFO:tasks.workunit.client.1.smithi173.stdout:1/478: read d5/d6/d17/d42/d12/f44 [1004258,78303] 0 2024-04-20T11:09:37.087 INFO:tasks.workunit.client.1.smithi173.stdout:8/515: creat d4/d18/d3a/d3c/fa0 x:0 0 0 2024-04-20T11:09:37.090 INFO:tasks.workunit.client.1.smithi173.stdout:6/524: creat d4/d59/d8f/fa3 x:0 0 0 2024-04-20T11:09:37.093 INFO:tasks.workunit.client.1.smithi173.stdout:3/534: mkdir d7/d53/dab 0 2024-04-20T11:09:37.096 INFO:tasks.workunit.client.1.smithi173.stdout:1/479: symlink d5/d6/d17/l9f 0 2024-04-20T11:09:37.097 INFO:tasks.workunit.client.1.smithi173.stdout:9/503: dwrite d5/d9/f26 [0,4194304] 0 2024-04-20T11:09:37.098 INFO:tasks.workunit.client.1.smithi173.stdout:1/480: fdatasync d5/d6/d17/d42/d12/d1c/f5b 0 2024-04-20T11:09:37.100 INFO:tasks.workunit.client.1.smithi173.stdout:4/597: sync 2024-04-20T11:09:37.100 INFO:tasks.workunit.client.1.smithi173.stdout:2/566: sync 2024-04-20T11:09:37.101 INFO:tasks.workunit.client.1.smithi173.stdout:6/525: rename d4/d59/d97 to d4/d1c/d8e/d90/da4 0 2024-04-20T11:09:37.104 INFO:tasks.workunit.client.1.smithi173.stdout:6/526: read d4/d24/f35 [1037454,111643] 0 2024-04-20T11:09:37.106 INFO:tasks.workunit.client.1.smithi173.stdout:5/548: dread da/d34/f6f [0,4194304] 0 2024-04-20T11:09:37.109 INFO:tasks.workunit.client.1.smithi173.stdout:2/567: symlink d2/d13/d46/d7d/lbe 0 2024-04-20T11:09:37.111 INFO:tasks.workunit.client.1.smithi173.stdout:8/516: getdents d4/d18/d3a 0 2024-04-20T11:09:37.114 INFO:tasks.workunit.client.1.smithi173.stdout:4/598: write da/dc/f27 [3344904,18105] 0 2024-04-20T11:09:37.155 INFO:tasks.workunit.client.1.smithi173.stdout:8/517: mknod d4/d18/d3a/ca1 0 2024-04-20T11:09:37.158 INFO:tasks.workunit.client.1.smithi173.stdout:5/549: creat da/d4f/d5e/dbc/fbf x:0 0 0 2024-04-20T11:09:37.160 INFO:tasks.workunit.client.1.smithi173.stdout:8/518: chown d4/d18/d59/d5a 72 1 2024-04-20T11:09:37.160 INFO:tasks.workunit.client.1.smithi173.stdout:2/568: getdents d2/d13/da5/db2 0 2024-04-20T11:09:37.160 INFO:tasks.workunit.client.1.smithi173.stdout:4/599: chown da/d1a/d60/f8c 1146 1 2024-04-20T11:09:37.164 INFO:tasks.workunit.client.1.smithi173.stdout:5/550: chown da/d34/l71 8092 1 2024-04-20T11:09:37.166 INFO:tasks.workunit.client.1.smithi173.stdout:4/600: creat da/dc/d62/fc8 x:0 0 0 2024-04-20T11:09:37.167 INFO:tasks.workunit.client.1.smithi173.stdout:2/569: fsync d2/d13/d46/d53/d4e/fa1 0 2024-04-20T11:09:37.168 INFO:tasks.workunit.client.1.smithi173.stdout:6/527: dwrite d4/d18/f66 [0,4194304] 0 2024-04-20T11:09:37.169 INFO:tasks.workunit.client.1.smithi173.stdout:4/601: dread - da/d48/d53/d39/f96 zero size 2024-04-20T11:09:37.171 INFO:tasks.workunit.client.1.smithi173.stdout:8/519: chown d4/d18/d3a/d3c/fa0 837760 1 2024-04-20T11:09:37.174 INFO:tasks.workunit.client.1.smithi173.stdout:1/481: dread d5/d6/d17/d42/da/f6a [0,4194304] 0 2024-04-20T11:09:37.175 INFO:tasks.workunit.client.1.smithi173.stdout:2/570: symlink d2/d13/d46/d4b/lbf 0 2024-04-20T11:09:37.176 INFO:tasks.workunit.client.1.smithi173.stdout:7/550: sync 2024-04-20T11:09:37.176 INFO:tasks.workunit.client.1.smithi173.stdout:6/528: creat d4/d24/d42/d74/d3b/fa5 x:0 0 0 2024-04-20T11:09:37.179 INFO:tasks.workunit.client.1.smithi173.stdout:4/602: symlink da/d1a/d60/d66/lc9 0 2024-04-20T11:09:37.179 INFO:tasks.workunit.client.1.smithi173.stdout:7/551: chown d9/d33/d35/d4a/d52 22 1 2024-04-20T11:09:37.180 INFO:tasks.workunit.client.1.smithi173.stdout:2/571: read d2/de/d11/d9d/f8e [200543,14347] 0 2024-04-20T11:09:37.184 INFO:tasks.workunit.client.1.smithi173.stdout:1/482: mknod d5/d6/d17/d42/da/d20/d38/ca0 0 2024-04-20T11:09:37.187 INFO:tasks.workunit.client.1.smithi173.stdout:8/520: rmdir d4/d18/d4f/d5b 39 2024-04-20T11:09:37.189 INFO:tasks.workunit.client.1.smithi173.stdout:5/551: dwrite da/de/d75/f1d [4194304,4194304] 0 2024-04-20T11:09:37.195 INFO:tasks.workunit.client.1.smithi173.stdout:1/483: creat d5/d6/dc/d4a/d94/fa1 x:0 0 0 2024-04-20T11:09:37.195 INFO:tasks.workunit.client.1.smithi173.stdout:6/529: dwrite d4/d18/f58 [0,4194304] 0 2024-04-20T11:09:37.199 INFO:tasks.workunit.client.1.smithi173.stdout:2/572: getdents d2/d13/d9a/dbc 0 2024-04-20T11:09:37.199 INFO:tasks.workunit.client.1.smithi173.stdout:5/552: creat da/d4f/d5e/dbc/fc0 x:0 0 0 2024-04-20T11:09:37.199 INFO:tasks.workunit.client.1.smithi173.stdout:4/603: rmdir da/d1e/db5 0 2024-04-20T11:09:37.200 INFO:tasks.workunit.client.1.smithi173.stdout:8/521: creat d4/d18/fa2 x:0 0 0 2024-04-20T11:09:37.200 INFO:tasks.workunit.client.1.smithi173.stdout:1/484: chown d5/d6/d17/l3d 796582 1 2024-04-20T11:09:37.204 INFO:tasks.workunit.client.1.smithi173.stdout:2/573: truncate d2/de/f3b 4960909 0 2024-04-20T11:09:37.210 INFO:tasks.workunit.client.1.smithi173.stdout:5/553: symlink da/de/d19/d32/d30/d35/d48/da2/lc1 0 2024-04-20T11:09:37.211 INFO:tasks.workunit.client.1.smithi173.stdout:7/552: dwrite d9/d1f/f7b [0,4194304] 0 2024-04-20T11:09:37.216 INFO:tasks.workunit.client.1.smithi173.stdout:2/574: mknod d2/d13/d46/d4b/dae/db6/cc0 0 2024-04-20T11:09:37.222 INFO:tasks.workunit.client.1.smithi173.stdout:1/485: rename d5/d6/l71 to d5/d6/d17/d42/da/d20/d38/d52/la2 0 2024-04-20T11:09:37.223 INFO:tasks.workunit.client.1.smithi173.stdout:7/553: mknod d9/da/d43/ccb 0 2024-04-20T11:09:37.224 INFO:tasks.workunit.client.1.smithi173.stdout:7/554: chown d9/d33/d85/ca7 449 1 2024-04-20T11:09:37.227 INFO:tasks.workunit.client.1.smithi173.stdout:0/491: sync 2024-04-20T11:09:37.230 INFO:tasks.workunit.client.1.smithi173.stdout:4/604: getdents da/d1a/d60/d66/d9b 0 2024-04-20T11:09:37.230 INFO:tasks.workunit.client.1.smithi173.stdout:2/575: mknod d2/d13/d46/d53/d9f/cc1 0 2024-04-20T11:09:37.237 INFO:tasks.workunit.client.1.smithi173.stdout:2/576: readlink d2/d13/d46/d53/lac 0 2024-04-20T11:09:37.238 INFO:tasks.workunit.client.1.smithi173.stdout:5/554: truncate da/f3d 1948694 0 2024-04-20T11:09:37.238 INFO:tasks.workunit.client.1.smithi173.stdout:8/522: dwrite d4/d18/d59/d5a/d7b/d8c/f8e [0,4194304] 0 2024-04-20T11:09:37.239 INFO:tasks.workunit.client.1.smithi173.stdout:7/555: creat d9/d33/fcc x:0 0 0 2024-04-20T11:09:37.240 INFO:tasks.workunit.client.1.smithi173.stdout:4/605: creat da/d48/d53/d39/fca x:0 0 0 2024-04-20T11:09:37.243 INFO:tasks.workunit.client.1.smithi173.stdout:1/486: dwrite d5/d6/d17/d42/da/d20/f87 [0,4194304] 0 2024-04-20T11:09:37.243 INFO:tasks.workunit.client.1.smithi173.stdout:2/577: write d2/d13/f74 [4924244,116799] 0 2024-04-20T11:09:37.250 INFO:tasks.workunit.client.1.smithi173.stdout:6/530: dread d4/f16 [0,4194304] 0 2024-04-20T11:09:37.250 INFO:tasks.workunit.client.1.smithi173.stdout:0/492: dread d6/f29 [0,4194304] 0 2024-04-20T11:09:37.253 INFO:tasks.workunit.client.1.smithi173.stdout:3/535: sync 2024-04-20T11:09:37.253 INFO:tasks.workunit.client.1.smithi173.stdout:9/504: sync 2024-04-20T11:09:37.253 INFO:tasks.workunit.client.1.smithi173.stdout:6/531: truncate d4/d18/f66 5147028 0 2024-04-20T11:09:37.254 INFO:tasks.workunit.client.1.smithi173.stdout:4/606: creat da/d1a/d60/d7d/fcb x:0 0 0 2024-04-20T11:09:37.257 INFO:tasks.workunit.client.1.smithi173.stdout:4/607: readlink da/d1a/l7c 0 2024-04-20T11:09:37.257 INFO:tasks.workunit.client.1.smithi173.stdout:7/556: truncate d9/d33/d91/d99/fc0 1281914 0 2024-04-20T11:09:37.258 INFO:tasks.workunit.client.1.smithi173.stdout:0/493: chown d6/c10 26643012 1 2024-04-20T11:09:37.259 INFO:tasks.workunit.client.1.smithi173.stdout:9/505: dread - d5/d9/d20/d29/d18/f77 zero size 2024-04-20T11:09:37.260 INFO:tasks.workunit.client.1.smithi173.stdout:2/578: mknod d2/d13/da5/db2/cc2 0 2024-04-20T11:09:37.261 INFO:tasks.workunit.client.1.smithi173.stdout:6/532: creat d4/d1c/d8e/d90/fa6 x:0 0 0 2024-04-20T11:09:37.265 INFO:tasks.workunit.client.1.smithi173.stdout:3/536: truncate d7/f94 236384 0 2024-04-20T11:09:37.268 INFO:tasks.workunit.client.1.smithi173.stdout:9/506: write d5/d9/d20/d29/d18/f77 [127643,54599] 0 2024-04-20T11:09:37.269 INFO:tasks.workunit.client.1.smithi173.stdout:6/533: chown d4/d14/f2f 33335694 1 2024-04-20T11:09:37.270 INFO:tasks.workunit.client.1.smithi173.stdout:2/579: write d2/de/f5e [3473721,123405] 0 2024-04-20T11:09:37.273 INFO:tasks.workunit.client.1.smithi173.stdout:4/608: dread da/d1e/d52/f71 [0,4194304] 0 2024-04-20T11:09:37.274 INFO:tasks.workunit.client.1.smithi173.stdout:7/557: rename d9/dbc/da4/fae to d9/da/fcd 0 2024-04-20T11:09:37.278 INFO:tasks.workunit.client.1.smithi173.stdout:3/537: symlink d7/d17/d61/d83/d45/lac 0 2024-04-20T11:09:37.280 INFO:tasks.workunit.client.1.smithi173.stdout:0/494: link d6/d8/d3c/d44/d75/f97 d6/d8/d3c/d44/d75/fa1 0 2024-04-20T11:09:37.281 INFO:tasks.workunit.client.1.smithi173.stdout:5/555: dwrite da/de/d19/d2c/f79 [0,4194304] 0 2024-04-20T11:09:37.281 INFO:tasks.workunit.client.1.smithi173.stdout:1/487: dwrite d5/d6/d17/d42/f90 [0,4194304] 0 2024-04-20T11:09:37.284 INFO:tasks.workunit.client.1.smithi173.stdout:8/523: truncate d4/d18/d3a/d3c/f53 1458667 0 2024-04-20T11:09:37.285 INFO:tasks.workunit.client.1.smithi173.stdout:9/507: dread - d5/d9/d20/d2b/d59/d44/d78/f7c zero size 2024-04-20T11:09:37.286 INFO:tasks.workunit.client.1.smithi173.stdout:2/580: mkdir d2/d13/d46/d4b/dc3 0 2024-04-20T11:09:37.286 INFO:tasks.workunit.client.1.smithi173.stdout:7/558: unlink d9/da/d31/f4d 0 2024-04-20T11:09:37.289 INFO:tasks.workunit.client.1.smithi173.stdout:7/559: fdatasync d9/d33/d85/fc5 0 2024-04-20T11:09:37.290 INFO:tasks.workunit.client.1.smithi173.stdout:0/495: mkdir d6/d8/df/d2c/d42/d50/d86/d8b/da2 0 2024-04-20T11:09:37.294 INFO:tasks.workunit.client.1.smithi173.stdout:1/488: fsync d5/d6/d17/d42/d12/d1c/f82 0 2024-04-20T11:09:37.295 INFO:tasks.workunit.client.1.smithi173.stdout:4/609: dread da/d1e/f38 [0,4194304] 0 2024-04-20T11:09:37.298 INFO:tasks.workunit.client.1.smithi173.stdout:2/581: symlink d2/d34/lc4 0 2024-04-20T11:09:37.300 INFO:tasks.workunit.client.1.smithi173.stdout:8/524: chown d4/d18/d3a/f9d 103 1 2024-04-20T11:09:37.300 INFO:tasks.workunit.client.1.smithi173.stdout:3/538: fdatasync d7/f8 0 2024-04-20T11:09:37.302 INFO:tasks.workunit.client.1.smithi173.stdout:5/556: mkdir da/d34/d7f/da5/d9a/dc2 0 2024-04-20T11:09:37.305 INFO:tasks.workunit.client.1.smithi173.stdout:0/496: creat d6/d8/df/d2c/d42/fa3 x:0 0 0 2024-04-20T11:09:37.307 INFO:tasks.workunit.client.1.smithi173.stdout:6/534: rename d4/d1c/l40 to d4/d14/la7 0 2024-04-20T11:09:37.309 INFO:tasks.workunit.client.1.smithi173.stdout:0/497: write d6/d8/d51/f5b [3925767,52956] 0 2024-04-20T11:09:37.310 INFO:tasks.workunit.client.1.smithi173.stdout:9/508: dwrite d5/d9/d20/d29/d2d/d33/f6b [0,4194304] 0 2024-04-20T11:09:37.316 INFO:tasks.workunit.client.1.smithi173.stdout:2/582: creat d2/fc5 x:0 0 0 2024-04-20T11:09:37.330 INFO:tasks.workunit.client.1.smithi173.stdout:0/498: read d6/d8/f1f [3475497,6773] 0 2024-04-20T11:09:37.332 INFO:tasks.workunit.client.1.smithi173.stdout:7/560: rename d9/dbc/da4 to d9/d1f/d7f/dce 0 2024-04-20T11:09:37.332 INFO:tasks.workunit.client.1.smithi173.stdout:4/610: dread da/dc/d15/f2a [0,4194304] 0 2024-04-20T11:09:37.334 INFO:tasks.workunit.client.1.smithi173.stdout:9/509: dwrite d5/d9/d20/d29/d2d/d33/f6b [0,4194304] 0 2024-04-20T11:09:37.336 INFO:tasks.workunit.client.1.smithi173.stdout:6/535: creat d4/d24/d42/d74/d3b/fa8 x:0 0 0 2024-04-20T11:09:37.337 INFO:tasks.workunit.client.1.smithi173.stdout:6/536: dread - d4/f9a zero size 2024-04-20T11:09:37.346 INFO:tasks.workunit.client.1.smithi173.stdout:3/539: fdatasync d7/d9/f35 0 2024-04-20T11:09:37.348 INFO:tasks.workunit.client.1.smithi173.stdout:8/525: dread d4/f32 [0,4194304] 0 2024-04-20T11:09:37.349 INFO:tasks.workunit.client.1.smithi173.stdout:1/489: link d5/d6/d17/d42/c10 d5/d6/d17/d42/d12/d1c/ca3 0 2024-04-20T11:09:37.352 INFO:tasks.workunit.client.1.smithi173.stdout:0/499: readlink d6/d8/df/d2c/d42/d50/d86/l8d 0 2024-04-20T11:09:37.353 INFO:tasks.workunit.client.1.smithi173.stdout:4/611: fsync da/d1a/d60/f8c 0 2024-04-20T11:09:37.355 INFO:tasks.workunit.client.1.smithi173.stdout:7/561: rename d9/d1f/d72/l82 to d9/d33/d35/d54/lcf 0 2024-04-20T11:09:37.357 INFO:tasks.workunit.client.1.smithi173.stdout:9/510: mkdir d5/d7f/d98 0 2024-04-20T11:09:37.363 INFO:tasks.workunit.client.1.smithi173.stdout:3/540: unlink d7/d17/d61/d83/f38 0 2024-04-20T11:09:37.364 INFO:tasks.workunit.client.1.smithi173.stdout:0/500: unlink d6/d8/df/d15/l9f 0 2024-04-20T11:09:37.366 INFO:tasks.workunit.client.1.smithi173.stdout:4/612: rmdir da/d1a/d60/d66/d9b 39 2024-04-20T11:09:37.367 INFO:tasks.workunit.client.1.smithi173.stdout:1/490: rename d5/d6/d17/d42/d12/d1f/f26 to d5/d6/d17/d42/da/d20/d38/fa4 0 2024-04-20T11:09:37.368 INFO:tasks.workunit.client.1.smithi173.stdout:7/562: creat d9/d33/d35/d56/fd0 x:0 0 0 2024-04-20T11:09:37.380 INFO:tasks.workunit.client.1.smithi173.stdout:6/537: creat d4/d24/fa9 x:0 0 0 2024-04-20T11:09:37.382 INFO:tasks.workunit.client.1.smithi173.stdout:7/563: dwrite d9/d33/f3a [0,4194304] 0 2024-04-20T11:09:37.383 INFO:tasks.workunit.client.1.smithi173.stdout:5/557: getdents da/de/d19/d81 0 2024-04-20T11:09:37.385 INFO:tasks.workunit.client.1.smithi173.stdout:5/558: readlink da/de/d19/d32/l50 0 2024-04-20T11:09:37.385 INFO:tasks.workunit.client.1.smithi173.stdout:2/583: truncate d2/d13/d46/d53/d4e/f8d 50703 0 2024-04-20T11:09:37.389 INFO:tasks.workunit.client.1.smithi173.stdout:8/526: creat d4/d18/fa3 x:0 0 0 2024-04-20T11:09:37.389 INFO:tasks.workunit.client.1.smithi173.stdout:4/613: dread da/d1a/d60/f63 [0,4194304] 0 2024-04-20T11:09:37.392 INFO:tasks.workunit.client.1.smithi173.stdout:4/614: dread - da/dc/fa5 zero size 2024-04-20T11:09:37.394 INFO:tasks.workunit.client.1.smithi173.stdout:9/511: dwrite d5/d9/d20/d29/f64 [0,4194304] 0 2024-04-20T11:09:37.395 INFO:tasks.workunit.client.1.smithi173.stdout:6/538: symlink d4/d24/d3e/d9c/laa 0 2024-04-20T11:09:37.398 INFO:tasks.workunit.client.1.smithi173.stdout:7/564: unlink d9/da/f14 0 2024-04-20T11:09:37.398 INFO:tasks.workunit.client.1.smithi173.stdout:6/539: chown d4/d24/d42/d74/c2a 415906223 1 2024-04-20T11:09:37.398 INFO:tasks.workunit.client.1.smithi173.stdout:3/541: dwrite d7/d9/f35 [0,4194304] 0 2024-04-20T11:09:37.400 INFO:tasks.workunit.client.1.smithi173.stdout:5/559: mkdir da/de/d19/d32/d72/dc3 0 2024-04-20T11:09:37.401 INFO:tasks.workunit.client.1.smithi173.stdout:3/542: dread d7/f94 [0,4194304] 0 2024-04-20T11:09:37.402 INFO:tasks.workunit.client.1.smithi173.stdout:2/584: creat d2/d13/d46/d4b/fc6 x:0 0 0 2024-04-20T11:09:37.403 INFO:tasks.workunit.client.1.smithi173.stdout:3/543: truncate d7/f94 247144 0 2024-04-20T11:09:37.405 INFO:tasks.workunit.client.1.smithi173.stdout:7/565: dread d9/d33/d85/fa6 [0,4194304] 0 2024-04-20T11:09:37.405 INFO:tasks.workunit.client.1.smithi173.stdout:0/501: creat d6/fa4 x:0 0 0 2024-04-20T11:09:37.407 INFO:tasks.workunit.client.1.smithi173.stdout:0/502: readlink d6/d8/df/d15/l28 0 2024-04-20T11:09:37.407 INFO:tasks.workunit.client.1.smithi173.stdout:7/566: dread - d9/d1f/d72/f8b zero size 2024-04-20T11:09:37.415 INFO:tasks.workunit.client.1.smithi173.stdout:8/527: fsync d4/d18/f9c 0 2024-04-20T11:09:37.417 INFO:tasks.workunit.client.1.smithi173.stdout:2/585: dread d2/d13/d46/d53/d4e/fa1 [0,4194304] 0 2024-04-20T11:09:37.417 INFO:tasks.workunit.client.1.smithi173.stdout:4/615: symlink da/d1e/d52/d7a/lcc 0 2024-04-20T11:09:37.428 INFO:tasks.workunit.client.1.smithi173.stdout:5/560: mknod da/d34/d7f/cc4 0 2024-04-20T11:09:37.431 INFO:tasks.workunit.client.1.smithi173.stdout:4/616: creat da/dc/d15/fcd x:0 0 0 2024-04-20T11:09:37.432 INFO:tasks.workunit.client.1.smithi173.stdout:2/586: mknod d2/d13/d9a/cc7 0 2024-04-20T11:09:37.433 INFO:tasks.workunit.client.1.smithi173.stdout:9/512: creat d5/d9/d20/d29/d18/d8a/f99 x:0 0 0 2024-04-20T11:09:37.437 INFO:tasks.workunit.client.1.smithi173.stdout:7/567: creat d9/d1f/d72/d8f/fd1 x:0 0 0 2024-04-20T11:09:37.439 INFO:tasks.workunit.client.1.smithi173.stdout:5/561: write da/de/d19/d32/d30/f70 [532675,107914] 0 2024-04-20T11:09:37.439 INFO:tasks.workunit.client.1.smithi173.stdout:8/528: mknod d4/d18/d3a/d3c/ca4 0 2024-04-20T11:09:37.440 INFO:tasks.workunit.client.1.smithi173.stdout:4/617: fdatasync f4 0 2024-04-20T11:09:37.443 INFO:tasks.workunit.client.1.smithi173.stdout:4/618: write da/dc/f27 [3196675,67481] 0 2024-04-20T11:09:37.443 INFO:tasks.workunit.client.1.smithi173.stdout:5/562: read da/de/d75/d3e/f60 [605066,15622] 0 2024-04-20T11:09:37.444 INFO:tasks.workunit.client.1.smithi173.stdout:4/619: fsync da/f64 0 2024-04-20T11:09:37.444 INFO:tasks.workunit.client.1.smithi173.stdout:7/568: rename f1 to d9/da/d2e/fd2 0 2024-04-20T11:09:37.447 INFO:tasks.workunit.client.1.smithi173.stdout:9/513: truncate d5/d9/d20/d29/f11 3892219 0 2024-04-20T11:09:37.447 INFO:tasks.workunit.client.1.smithi173.stdout:4/620: chown da/d1a/d60/d7d/fc5 73 1 2024-04-20T11:09:37.448 INFO:tasks.workunit.client.1.smithi173.stdout:5/563: chown da/d34/l71 66052341 1 2024-04-20T11:09:37.448 INFO:tasks.workunit.client.1.smithi173.stdout:0/503: getdents d6/d8/d3c/d77 0 2024-04-20T11:09:37.450 INFO:tasks.workunit.client.1.smithi173.stdout:1/491: truncate d5/fb 1621132 0 2024-04-20T11:09:37.451 INFO:tasks.workunit.client.1.smithi173.stdout:8/529: symlink d4/d18/d59/la5 0 2024-04-20T11:09:37.451 INFO:tasks.workunit.client.1.smithi173.stdout:5/564: write da/de/d19/d32/f31 [149398,68318] 0 2024-04-20T11:09:37.452 INFO:tasks.workunit.client.1.smithi173.stdout:7/569: mknod d9/d33/d91/d99/cd3 0 2024-04-20T11:09:37.453 INFO:tasks.workunit.client.1.smithi173.stdout:9/514: chown d5/d9/d20/d2b/d59/d44/d78/d82 95429263 1 2024-04-20T11:09:37.453 INFO:tasks.workunit.client.1.smithi173.stdout:4/621: truncate da/d1e/d52/d9a/fa8 77527 0 2024-04-20T11:09:37.457 INFO:tasks.workunit.client.1.smithi173.stdout:3/544: dread d7/d2b/f62 [0,4194304] 0 2024-04-20T11:09:37.466 INFO:tasks.workunit.client.1.smithi173.stdout:5/565: dwrite da/d34/f47 [4194304,4194304] 0 2024-04-20T11:09:37.484 INFO:tasks.workunit.client.1.smithi173.stdout:8/530: mkdir d4/d18/da6 0 2024-04-20T11:09:37.489 INFO:tasks.workunit.client.1.smithi173.stdout:4/622: mkdir da/d1e/dce 0 2024-04-20T11:09:37.496 INFO:tasks.workunit.client.1.smithi173.stdout:3/545: symlink d7/d2b/lad 0 2024-04-20T11:09:37.497 INFO:tasks.workunit.client.1.smithi173.stdout:6/540: dwrite d4/d24/d42/d74/f53 [0,4194304] 0 2024-04-20T11:09:37.502 INFO:tasks.workunit.client.1.smithi173.stdout:8/531: dread - d4/d18/d3a/f66 zero size 2024-04-20T11:09:37.502 INFO:tasks.workunit.client.1.smithi173.stdout:1/492: dread d5/d6/f21 [0,4194304] 0 2024-04-20T11:09:37.511 INFO:tasks.workunit.client.1.smithi173.stdout:5/566: mknod da/de/d19/d32/d66/dbd/cc5 0 2024-04-20T11:09:37.515 INFO:tasks.workunit.client.1.smithi173.stdout:6/541: symlink d4/d24/d3e/lab 0 2024-04-20T11:09:37.516 INFO:tasks.workunit.client.1.smithi173.stdout:4/623: creat da/dc/d15/d8b/fcf x:0 0 0 2024-04-20T11:09:37.521 INFO:tasks.workunit.client.1.smithi173.stdout:0/504: dwrite d6/d8/f1f [0,4194304] 0 2024-04-20T11:09:37.521 INFO:tasks.workunit.client.1.smithi173.stdout:9/515: rename d5/d9/d20/d2b/d59/d44/l69 to d5/d9/d20/d29/d2d/l9a 0 2024-04-20T11:09:37.523 INFO:tasks.workunit.client.1.smithi173.stdout:1/493: rename d5/d6/d17/d42/l69 to d5/d6/d17/d42/la5 0 2024-04-20T11:09:37.530 INFO:tasks.workunit.client.1.smithi173.stdout:5/567: dread da/d4f/f5f [0,4194304] 0 2024-04-20T11:09:37.531 INFO:tasks.workunit.client.1.smithi173.stdout:9/516: write d5/d9/d20/d29/d18/f1f [1159648,120374] 0 2024-04-20T11:09:37.531 INFO:tasks.workunit.client.1.smithi173.stdout:0/505: chown d6/d54/d65/f73 15 1 2024-04-20T11:09:37.532 INFO:tasks.workunit.client.1.smithi173.stdout:8/532: dwrite d4/f9f [0,4194304] 0 2024-04-20T11:09:37.533 INFO:tasks.workunit.client.1.smithi173.stdout:2/587: dwrite d2/d34/d58/f69 [0,4194304] 0 2024-04-20T11:09:37.533 INFO:tasks.workunit.client.1.smithi173.stdout:6/542: getdents d4/d59/d8f 0 2024-04-20T11:09:37.538 INFO:tasks.workunit.client.1.smithi173.stdout:4/624: link da/dc/d15/d8e/fa7 da/d1a/d60/fd0 0 2024-04-20T11:09:37.538 INFO:tasks.workunit.client.1.smithi173.stdout:1/494: truncate d5/d6/dc/d4a/f59 5082075 0 2024-04-20T11:09:37.539 INFO:tasks.workunit.client.1.smithi173.stdout:3/546: dread d7/f15 [0,4194304] 0 2024-04-20T11:09:37.541 INFO:tasks.workunit.client.1.smithi173.stdout:4/625: stat da/d48/d53/d39/fca 0 2024-04-20T11:09:37.543 INFO:tasks.workunit.client.1.smithi173.stdout:2/588: creat d2/d34/fc8 x:0 0 0 2024-04-20T11:09:37.543 INFO:tasks.workunit.client.1.smithi173.stdout:8/533: dread - d4/d18/d4f/f7e zero size 2024-04-20T11:09:37.544 INFO:tasks.workunit.client.1.smithi173.stdout:4/626: stat da/d48/c86 0 2024-04-20T11:09:37.545 INFO:tasks.workunit.client.1.smithi173.stdout:1/495: read - d5/d6/d17/d42/da/d20/d38/d52/f8d zero size 2024-04-20T11:09:37.550 INFO:tasks.workunit.client.1.smithi173.stdout:9/517: rename d5/d7f to d5/d9/d20/d29/d18/d9b 0 2024-04-20T11:09:37.554 INFO:tasks.workunit.client.1.smithi173.stdout:2/589: rmdir d2 39 2024-04-20T11:09:37.558 INFO:tasks.workunit.client.1.smithi173.stdout:6/543: getdents d4/d14/d5f/d9f 0 2024-04-20T11:09:37.558 INFO:tasks.workunit.client.1.smithi173.stdout:4/627: mkdir da/dd1 0 2024-04-20T11:09:37.559 INFO:tasks.workunit.client.1.smithi173.stdout:1/496: mknod d5/d6/d17/d42/da/d20/d38/d52/ca6 0 2024-04-20T11:09:37.564 INFO:tasks.workunit.client.1.smithi173.stdout:8/534: dwrite d4/f8 [0,4194304] 0 2024-04-20T11:09:37.569 INFO:tasks.workunit.client.1.smithi173.stdout:3/547: dwrite d7/d9/f4f [0,4194304] 0 2024-04-20T11:09:37.569 INFO:tasks.workunit.client.1.smithi173.stdout:6/544: rmdir d4/d59/d8f 39 2024-04-20T11:09:37.572 INFO:tasks.workunit.client.1.smithi173.stdout:5/568: getdents da/d4f/d65 0 2024-04-20T11:09:37.572 INFO:tasks.workunit.client.1.smithi173.stdout:7/570: sync 2024-04-20T11:09:37.576 INFO:tasks.workunit.client.1.smithi173.stdout:6/545: truncate d4/d24/d42/d74/d3b/f52 4955536 0 2024-04-20T11:09:37.576 INFO:tasks.workunit.client.1.smithi173.stdout:8/535: read d4/d18/d3a/f46 [490760,55031] 0 2024-04-20T11:09:37.577 INFO:tasks.workunit.client.1.smithi173.stdout:1/497: unlink d5/d6/d17/d42/da/d20/d48/d81/l8c 0 2024-04-20T11:09:37.579 INFO:tasks.workunit.client.1.smithi173.stdout:7/571: write d9/da/f12 [1083450,60617] 0 2024-04-20T11:09:37.580 INFO:tasks.workunit.client.1.smithi173.stdout:1/498: chown d5/d6/d17/d42/da/df/c6d 0 1 2024-04-20T11:09:37.589 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:37 smithi117 ceph-mon[15659]: pgmap v28: 65 pgs: 65 active+clean; 2.6 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 33 MiB/s rd, 131 MiB/s wr, 282 op/s 2024-04-20T11:09:37.593 INFO:tasks.workunit.client.1.smithi173.stdout:2/590: link d2/d13/d46/d4b/fc6 d2/d13/d46/d53/d9f/fc9 0 2024-04-20T11:09:37.594 INFO:tasks.workunit.client.1.smithi173.stdout:4/628: dread da/dc/d13/f51 [0,4194304] 0 2024-04-20T11:09:37.595 INFO:tasks.workunit.client.1.smithi173.stdout:6/546: write d4/d24/f6b [2603725,28070] 0 2024-04-20T11:09:37.595 INFO:tasks.workunit.client.1.smithi173.stdout:4/629: truncate da/f64 1213892 0 2024-04-20T11:09:37.596 INFO:tasks.workunit.client.1.smithi173.stdout:6/547: read - d4/d1c/d8e/d90/fa6 zero size 2024-04-20T11:09:37.600 INFO:tasks.workunit.client.1.smithi173.stdout:5/569: mknod da/de/cc6 0 2024-04-20T11:09:37.603 INFO:tasks.workunit.client.1.smithi173.stdout:3/548: link d7/d17/d61/d83/faa d7/d2b/d9d/fae 0 2024-04-20T11:09:37.609 INFO:tasks.workunit.client.1.smithi173.stdout:0/506: truncate d6/d8/df/d2c/d42/d50/d86/f94 3882321 0 2024-04-20T11:09:37.610 INFO:tasks.workunit.client.1.smithi173.stdout:0/507: chown d6/d8/df/d43/d7f/c80 16780332 1 2024-04-20T11:09:37.612 INFO:tasks.workunit.client.1.smithi173.stdout:0/508: fdatasync d6/d8/df/d43/d7f/f9a 0 2024-04-20T11:09:37.615 INFO:tasks.workunit.client.1.smithi173.stdout:9/518: write d5/d9/f1d [443394,106747] 0 2024-04-20T11:09:37.625 INFO:tasks.workunit.client.1.smithi173.stdout:1/499: mknod d5/d6/d17/ca7 0 2024-04-20T11:09:37.625 INFO:tasks.workunit.client.1.smithi173.stdout:5/570: dread da/de/d75/f51 [0,4194304] 0 2024-04-20T11:09:37.626 INFO:tasks.workunit.client.1.smithi173.stdout:2/591: fsync d2/de/d11/d9d/f8e 0 2024-04-20T11:09:37.627 INFO:tasks.workunit.client.1.smithi173.stdout:3/549: rmdir d7/d17 39 2024-04-20T11:09:37.628 INFO:tasks.workunit.client.1.smithi173.stdout:6/548: symlink d4/d59/d8f/lac 0 2024-04-20T11:09:37.630 INFO:tasks.workunit.client.1.smithi173.stdout:0/509: fdatasync d6/d8/df/d43/f4c 0 2024-04-20T11:09:37.633 INFO:tasks.workunit.client.1.smithi173.stdout:9/519: readlink d5/d9/l8b 0 2024-04-20T11:09:37.634 INFO:tasks.workunit.client.1.smithi173.stdout:8/536: creat d4/d18/fa7 x:0 0 0 2024-04-20T11:09:37.637 INFO:tasks.workunit.client.1.smithi173.stdout:7/572: dwrite d9/da/d17/f64 [0,4194304] 0 2024-04-20T11:09:37.640 INFO:tasks.workunit.client.1.smithi173.stdout:7/573: write d9/da/d17/f1b [139863,88778] 0 2024-04-20T11:09:37.642 INFO:tasks.workunit.client.1.smithi173.stdout:5/571: mknod da/de/d19/d32/d30/dbb/cc7 0 2024-04-20T11:09:37.643 INFO:tasks.workunit.client.1.smithi173.stdout:5/572: readlink da/de/l14 0 2024-04-20T11:09:37.655 INFO:tasks.workunit.client.1.smithi173.stdout:6/549: mkdir d4/d24/d42/d74/d3b/dad 0 2024-04-20T11:09:37.659 INFO:tasks.workunit.client.1.smithi173.stdout:7/574: symlink d9/d1f/d72/d8f/ld4 0 2024-04-20T11:09:37.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:37 smithi173 ceph-mon[23800]: pgmap v28: 65 pgs: 65 active+clean; 2.6 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 33 MiB/s rd, 131 MiB/s wr, 282 op/s 2024-04-20T11:09:37.662 INFO:tasks.workunit.client.1.smithi173.stdout:4/630: getdents da/d1e 0 2024-04-20T11:09:37.663 INFO:tasks.workunit.client.1.smithi173.stdout:3/550: chown d7/d17/d61/d83/d45/f56 194 1 2024-04-20T11:09:37.663 INFO:tasks.workunit.client.1.smithi173.stdout:6/550: fdatasync d4/d24/d3e/f57 0 2024-04-20T11:09:37.666 INFO:tasks.workunit.client.1.smithi173.stdout:0/510: mknod d6/d8/df/d2c/ca5 0 2024-04-20T11:09:37.666 INFO:tasks.workunit.client.1.smithi173.stdout:1/500: dread f4 [0,4194304] 0 2024-04-20T11:09:37.666 INFO:tasks.workunit.client.1.smithi173.stdout:8/537: mkdir d4/d18/d59/d5a/da8 0 2024-04-20T11:09:37.667 INFO:tasks.workunit.client.1.smithi173.stdout:9/520: dread d5/d4c/d79/f7a [0,4194304] 0 2024-04-20T11:09:37.668 INFO:tasks.workunit.client.1.smithi173.stdout:0/511: chown d6/d8/df/d2c/d42 35 1 2024-04-20T11:09:37.672 INFO:tasks.workunit.client.1.smithi173.stdout:6/551: mknod d4/d18/d7e/cae 0 2024-04-20T11:09:37.674 INFO:tasks.workunit.client.1.smithi173.stdout:3/551: creat d7/d17/d61/d83/d45/faf x:0 0 0 2024-04-20T11:09:37.676 INFO:tasks.workunit.client.1.smithi173.stdout:3/552: fsync d7/d17/fa8 0 2024-04-20T11:09:37.680 INFO:tasks.workunit.client.1.smithi173.stdout:7/575: truncate d9/d1f/d72/f75 406127 0 2024-04-20T11:09:37.681 INFO:tasks.workunit.client.1.smithi173.stdout:2/592: write d2/d13/d46/d53/f76 [204631,77942] 0 2024-04-20T11:09:37.681 INFO:tasks.workunit.client.1.smithi173.stdout:9/521: fsync d5/d9/d20/d2b/d59/d44/f6a 0 2024-04-20T11:09:37.686 INFO:tasks.workunit.client.1.smithi173.stdout:4/631: dwrite da/d48/d53/f5b [0,4194304] 0 2024-04-20T11:09:37.688 INFO:tasks.workunit.client.1.smithi173.stdout:5/573: getdents da/de/d19/d32/d30/d35/d48 0 2024-04-20T11:09:37.691 INFO:tasks.workunit.client.1.smithi173.stdout:3/553: symlink d7/d17/d61/d83/d45/lb0 0 2024-04-20T11:09:37.693 INFO:tasks.workunit.client.1.smithi173.stdout:5/574: readlink da/d34/l71 0 2024-04-20T11:09:37.693 INFO:tasks.workunit.client.1.smithi173.stdout:2/593: mkdir d2/d34/d81/dca 0 2024-04-20T11:09:37.696 INFO:tasks.workunit.client.1.smithi173.stdout:0/512: dwrite d6/d8/df/d2c/d42/f84 [0,4194304] 0 2024-04-20T11:09:37.699 INFO:tasks.workunit.client.1.smithi173.stdout:6/552: dwrite d4/d24/fa9 [0,4194304] 0 2024-04-20T11:09:37.704 INFO:tasks.workunit.client.1.smithi173.stdout:5/575: write da/d4f/d5e/f98 [743372,74574] 0 2024-04-20T11:09:37.704 INFO:tasks.workunit.client.1.smithi173.stdout:6/553: dread - d4/d14/f87 zero size 2024-04-20T11:09:37.704 INFO:tasks.workunit.client.1.smithi173.stdout:1/501: dread d5/d6/d17/d42/d12/d1c/f6f [0,4194304] 0 2024-04-20T11:09:37.704 INFO:tasks.workunit.client.1.smithi173.stdout:0/513: stat d6/d8/df/d15/f7e 0 2024-04-20T11:09:37.705 INFO:tasks.workunit.client.1.smithi173.stdout:7/576: creat d9/d33/d40/d89/dba/fd5 x:0 0 0 2024-04-20T11:09:37.709 INFO:tasks.workunit.client.1.smithi173.stdout:9/522: link d5/d9/d20/d29/f64 d5/d9/d20/d29/d18/f9c 0 2024-04-20T11:09:37.715 INFO:tasks.workunit.client.1.smithi173.stdout:4/632: rename da/dc/d13/d65/l79 to da/d1e/d52/d9a/ld2 0 2024-04-20T11:09:37.717 INFO:tasks.workunit.client.1.smithi173.stdout:6/554: unlink d4/d59/d8f/lac 0 2024-04-20T11:09:37.719 INFO:tasks.workunit.client.1.smithi173.stdout:6/555: chown d4/d24/fa9 202842778 1 2024-04-20T11:09:37.722 INFO:tasks.workunit.client.1.smithi173.stdout:0/514: dread d6/f1e [0,4194304] 0 2024-04-20T11:09:37.724 INFO:tasks.workunit.client.1.smithi173.stdout:1/502: write d5/d6/d17/d42/f54 [3556829,108528] 0 2024-04-20T11:09:37.726 INFO:tasks.workunit.client.1.smithi173.stdout:9/523: mkdir d5/d9/d20/d2b/d59/d44/d78/d9d 0 2024-04-20T11:09:37.727 INFO:tasks.workunit.client.1.smithi173.stdout:5/576: write da/de/d19/d32/d30/f4c [5775469,89498] 0 2024-04-20T11:09:37.731 INFO:tasks.workunit.client.1.smithi173.stdout:0/515: mkdir d6/d8/d3c/d44/da6 0 2024-04-20T11:09:37.735 INFO:tasks.workunit.client.1.smithi173.stdout:6/556: dread d4/f38 [0,4194304] 0 2024-04-20T11:09:37.736 INFO:tasks.workunit.client.1.smithi173.stdout:3/554: link d7/d17/d58/d8a/la0 d7/d17/d58/d7f/lb1 0 2024-04-20T11:09:37.740 INFO:tasks.workunit.client.1.smithi173.stdout:0/516: truncate d6/d24/f74 830256 0 2024-04-20T11:09:37.744 INFO:tasks.workunit.client.1.smithi173.stdout:0/517: dread - d6/fa4 zero size 2024-04-20T11:09:37.744 INFO:tasks.workunit.client.1.smithi173.stdout:7/577: link d9/d33/d35/d56/c88 d9/d1f/d7f/cd6 0 2024-04-20T11:09:37.748 INFO:tasks.workunit.client.1.smithi173.stdout:6/557: mkdir d4/d14/d5f/daf 0 2024-04-20T11:09:37.756 INFO:tasks.workunit.client.1.smithi173.stdout:7/578: chown d9/d33 10993 1 2024-04-20T11:09:37.759 INFO:tasks.workunit.client.1.smithi173.stdout:5/577: truncate da/de/d19/d2c/f62 798864 0 2024-04-20T11:09:37.765 INFO:tasks.workunit.client.1.smithi173.stdout:7/579: read - d9/d33/d40/d89/dba/fd5 zero size 2024-04-20T11:09:37.766 INFO:tasks.workunit.client.1.smithi173.stdout:8/538: dwrite d4/d18/f21 [0,4194304] 0 2024-04-20T11:09:37.769 INFO:tasks.workunit.client.1.smithi173.stdout:1/503: rename d5/d6/dc/d4a/c6b to d5/d6/d17/d42/da/d20/d38/d52/ca8 0 2024-04-20T11:09:37.772 INFO:tasks.workunit.client.1.smithi173.stdout:0/518: creat d6/d5f/d8f/fa7 x:0 0 0 2024-04-20T11:09:37.774 INFO:tasks.workunit.client.1.smithi173.stdout:9/524: dwrite d5/d9/d20/d29/d18/d8a/f99 [0,4194304] 0 2024-04-20T11:09:37.776 INFO:tasks.workunit.client.1.smithi173.stdout:3/555: creat d7/d17/d58/fb2 x:0 0 0 2024-04-20T11:09:37.777 INFO:tasks.workunit.client.1.smithi173.stdout:6/558: symlink d4/d24/d3e/lb0 0 2024-04-20T11:09:37.789 INFO:tasks.workunit.client.1.smithi173.stdout:5/578: mknod da/d34/d7f/da5/d9a/cc8 0 2024-04-20T11:09:37.793 INFO:tasks.workunit.client.1.smithi173.stdout:6/559: chown d4/d14/d5f 44253 1 2024-04-20T11:09:37.796 INFO:tasks.workunit.client.1.smithi173.stdout:1/504: write d5/d6/d17/d42/da/d20/d38/d52/f8d [84787,71146] 0 2024-04-20T11:09:37.796 INFO:tasks.workunit.client.1.smithi173.stdout:0/519: write d6/d8/d3c/d44/f46 [1075068,1765] 0 2024-04-20T11:09:37.800 INFO:tasks.workunit.client.1.smithi173.stdout:3/556: write d7/d17/d61/d83/f82 [1017914,23192] 0 2024-04-20T11:09:37.800 INFO:tasks.workunit.client.1.smithi173.stdout:2/594: dwrite d2/d13/d46/d53/d9f/fc9 [0,4194304] 0 2024-04-20T11:09:37.805 INFO:tasks.workunit.client.1.smithi173.stdout:1/505: fsync d5/d6/d17/d42/d12/d1c/d85/f9e 0 2024-04-20T11:09:37.808 INFO:tasks.workunit.client.1.smithi173.stdout:3/557: write d7/d17/d61/d83/f79 [289265,80561] 0 2024-04-20T11:09:37.813 INFO:tasks.workunit.client.1.smithi173.stdout:9/525: mkdir d5/d9/d20/d4f/d5e/d9e 0 2024-04-20T11:09:37.821 INFO:tasks.workunit.client.1.smithi173.stdout:4/633: sync 2024-04-20T11:09:37.821 INFO:tasks.workunit.client.1.smithi173.stdout:8/539: symlink d4/d18/da6/la9 0 2024-04-20T11:09:37.828 INFO:tasks.workunit.client.1.smithi173.stdout:4/634: dread da/d1e/d52/f85 [0,4194304] 0 2024-04-20T11:09:37.831 INFO:tasks.workunit.client.1.smithi173.stdout:4/635: read - da/d1a/d60/d7d/fcb zero size 2024-04-20T11:09:37.831 INFO:tasks.workunit.client.1.smithi173.stdout:7/580: truncate d9/da/d17/f64 2787320 0 2024-04-20T11:09:37.834 INFO:tasks.workunit.client.1.smithi173.stdout:5/579: symlink da/de/d19/d32/d30/d35/lc9 0 2024-04-20T11:09:37.834 INFO:tasks.workunit.client.1.smithi173.stdout:0/520: dread d6/d54/f7d [0,4194304] 0 2024-04-20T11:09:37.837 INFO:tasks.workunit.client.1.smithi173.stdout:1/506: symlink d5/d6/d17/d42/da/d20/d38/la9 0 2024-04-20T11:09:37.838 INFO:tasks.workunit.client.1.smithi173.stdout:9/526: mknod d5/d9/d20/d2b/d59/d48/c9f 0 2024-04-20T11:09:37.848 INFO:tasks.workunit.client.1.smithi173.stdout:7/581: rename d9/d33/d40/d89/dba to d9/d33/d81/dd7 0 2024-04-20T11:09:37.850 INFO:tasks.workunit.client.1.smithi173.stdout:6/560: write d4/f5a [1933627,116814] 0 2024-04-20T11:09:37.852 INFO:tasks.workunit.client.1.smithi173.stdout:2/595: dwrite d2/d8c/fa2 [0,4194304] 0 2024-04-20T11:09:37.858 INFO:tasks.workunit.client.1.smithi173.stdout:8/540: symlink d4/d18/d3a/d3c/laa 0 2024-04-20T11:09:37.859 INFO:tasks.workunit.client.1.smithi173.stdout:7/582: write d9/d1f/d72/d8f/fd1 [926646,61333] 0 2024-04-20T11:09:37.862 INFO:tasks.workunit.client.1.smithi173.stdout:2/596: mknod d2/d13/d28/ccb 0 2024-04-20T11:09:37.868 INFO:tasks.workunit.client.1.smithi173.stdout:0/521: dwrite d6/d24/f4f [0,4194304] 0 2024-04-20T11:09:37.870 INFO:tasks.workunit.client.1.smithi173.stdout:4/636: dread da/dc/d13/f32 [8388608,4194304] 0 2024-04-20T11:09:37.872 INFO:tasks.workunit.client.1.smithi173.stdout:5/580: rename da/de/d19/d32/d30/d35 to da/de/d19/dca 0 2024-04-20T11:09:37.877 INFO:tasks.workunit.client.1.smithi173.stdout:0/522: rmdir d6/d8/df 39 2024-04-20T11:09:37.884 INFO:tasks.workunit.client.1.smithi173.stdout:9/527: dwrite d5/d9/d20/d2b/d31/f92 [0,4194304] 0 2024-04-20T11:09:37.886 INFO:tasks.workunit.client.1.smithi173.stdout:5/581: read da/de/d19/d2c/fa6 [885065,32288] 0 2024-04-20T11:09:37.891 INFO:tasks.workunit.client.1.smithi173.stdout:0/523: write d6/d5f/d8f/fa7 [227547,83046] 0 2024-04-20T11:09:37.892 INFO:tasks.workunit.client.1.smithi173.stdout:6/561: truncate d4/d24/d42/f7c 2671321 0 2024-04-20T11:09:37.902 INFO:tasks.workunit.client.1.smithi173.stdout:3/558: dwrite d7/d9/d30/f8d [0,4194304] 0 2024-04-20T11:09:37.908 INFO:tasks.workunit.client.1.smithi173.stdout:7/583: dwrite d9/d33/fcc [0,4194304] 0 2024-04-20T11:09:37.908 INFO:tasks.workunit.client.1.smithi173.stdout:8/541: dwrite d4/d18/d3a/d3c/f37 [4194304,4194304] 0 2024-04-20T11:09:37.911 INFO:tasks.workunit.client.1.smithi173.stdout:9/528: chown d5/d9/d20/d29/f11 3745105 1 2024-04-20T11:09:37.911 INFO:tasks.workunit.client.1.smithi173.stdout:5/582: fdatasync da/de/d75/d40/f42 0 2024-04-20T11:09:37.913 INFO:tasks.workunit.client.1.smithi173.stdout:3/559: chown d7/d17/d61/d83/d45/f56 2124296051 1 2024-04-20T11:09:37.914 INFO:tasks.workunit.client.1.smithi173.stdout:1/507: dwrite d5/d6/d17/d42/d12/f47 [0,4194304] 0 2024-04-20T11:09:37.922 INFO:tasks.workunit.client.1.smithi173.stdout:1/508: chown d5/d6/dc 121 1 2024-04-20T11:09:37.928 INFO:tasks.workunit.client.1.smithi173.stdout:0/524: truncate d6/d8/d3c/d44/d75/fa1 808809 0 2024-04-20T11:09:37.934 INFO:tasks.workunit.client.1.smithi173.stdout:2/597: dwrite d2/d34/f56 [0,4194304] 0 2024-04-20T11:09:37.937 INFO:tasks.workunit.client.1.smithi173.stdout:6/562: dwrite d4/d24/fa9 [0,4194304] 0 2024-04-20T11:09:37.943 INFO:tasks.workunit.client.1.smithi173.stdout:3/560: unlink d7/d17/la9 0 2024-04-20T11:09:37.943 INFO:tasks.workunit.client.1.smithi173.stdout:1/509: write d5/d6/d17/d42/d12/d1f/f63 [2784088,9784] 0 2024-04-20T11:09:37.944 INFO:tasks.workunit.client.1.smithi173.stdout:4/637: dwrite da/d1a/f81 [0,4194304] 0 2024-04-20T11:09:37.951 INFO:tasks.workunit.client.1.smithi173.stdout:6/563: chown d4/d14/d5f 2461289 1 2024-04-20T11:09:37.955 INFO:tasks.workunit.client.1.smithi173.stdout:1/510: chown d5/d6/dc/f3a 246705172 1 2024-04-20T11:09:37.956 INFO:tasks.workunit.client.1.smithi173.stdout:8/542: rename d4/c9 to d4/d18/d59/cab 0 2024-04-20T11:09:37.959 INFO:tasks.workunit.client.1.smithi173.stdout:6/564: dread - d4/d1c/fa1 zero size 2024-04-20T11:09:37.963 INFO:tasks.workunit.client.1.smithi173.stdout:9/529: dwrite d5/d9/d20/d29/f5c [0,4194304] 0 2024-04-20T11:09:37.967 INFO:tasks.workunit.client.1.smithi173.stdout:2/598: dwrite d2/d34/d58/f69 [0,4194304] 0 2024-04-20T11:09:37.967 INFO:tasks.workunit.client.1.smithi173.stdout:9/530: write d5/d4c/f72 [2984910,96432] 0 2024-04-20T11:09:37.968 INFO:tasks.workunit.client.1.smithi173.stdout:0/525: symlink d6/d8/df/da0/la8 0 2024-04-20T11:09:37.969 INFO:tasks.workunit.client.1.smithi173.stdout:4/638: mkdir da/d1e/d52/d9a/dd3 0 2024-04-20T11:09:37.971 INFO:tasks.workunit.client.1.smithi173.stdout:4/639: fsync da/d1e/da6/fab 0 2024-04-20T11:09:37.974 INFO:tasks.workunit.client.1.smithi173.stdout:7/584: getdents d9/d33/d91 0 2024-04-20T11:09:37.978 INFO:tasks.workunit.client.1.smithi173.stdout:6/565: creat d4/d24/d3e/d9c/fb1 x:0 0 0 2024-04-20T11:09:37.989 INFO:tasks.workunit.client.1.smithi173.stdout:1/511: dwrite d5/d6/d17/d93/f97 [0,4194304] 0 2024-04-20T11:09:37.992 INFO:tasks.workunit.client.1.smithi173.stdout:8/543: symlink d4/d18/lac 0 2024-04-20T11:09:37.992 INFO:tasks.workunit.client.1.smithi173.stdout:0/526: unlink d6/d8/df/d2c/d2d/f36 0 2024-04-20T11:09:37.994 INFO:tasks.workunit.client.1.smithi173.stdout:9/531: dwrite d5/d9/d20/d2b/f97 [0,4194304] 0 2024-04-20T11:09:37.997 INFO:tasks.workunit.client.1.smithi173.stdout:8/544: chown d4/d18/d3a/d3c/c82 2529 1 2024-04-20T11:09:38.001 INFO:tasks.workunit.client.1.smithi173.stdout:0/527: write d6/d8/df/d15/f95 [166102,72129] 0 2024-04-20T11:09:38.002 INFO:tasks.workunit.client.1.smithi173.stdout:5/583: dread da/de/d75/f89 [0,4194304] 0 2024-04-20T11:09:38.003 INFO:tasks.workunit.client.1.smithi173.stdout:7/585: read d9/da/d17/f64 [1393103,2375] 0 2024-04-20T11:09:38.004 INFO:tasks.workunit.client.1.smithi173.stdout:4/640: mknod da/d1a/d60/d66/d9b/cd4 0 2024-04-20T11:09:38.010 INFO:tasks.workunit.client.1.smithi173.stdout:8/545: read d4/f19 [3917333,38714] 0 2024-04-20T11:09:38.010 INFO:tasks.workunit.client.1.smithi173.stdout:1/512: mknod d5/d6/dc/caa 0 2024-04-20T11:09:38.015 INFO:tasks.workunit.client.1.smithi173.stdout:7/586: write d9/da/d43/f57 [273429,59752] 0 2024-04-20T11:09:38.019 INFO:tasks.workunit.client.1.smithi173.stdout:1/513: chown d5/d6/d17/d42/d12/d1c/l45 15 1 2024-04-20T11:09:38.019 INFO:tasks.workunit.client.1.smithi173.stdout:6/566: getdents d4 0 2024-04-20T11:09:38.023 INFO:tasks.workunit.client.1.smithi173.stdout:0/528: symlink d6/d8/df/d2c/d2d/la9 0 2024-04-20T11:09:38.023 INFO:tasks.workunit.client.1.smithi173.stdout:9/532: link d5/d9/d20/d29/d2d/d33/c84 d5/d9/d20/d2b/d59/d44/ca0 0 2024-04-20T11:09:38.025 INFO:tasks.workunit.client.1.smithi173.stdout:8/546: symlink d4/d18/d59/d5a/d7b/d8c/lad 0 2024-04-20T11:09:38.028 INFO:tasks.workunit.client.1.smithi173.stdout:3/561: dwrite d7/f8 [4194304,4194304] 0 2024-04-20T11:09:38.029 INFO:tasks.workunit.client.1.smithi173.stdout:2/599: dread d2/d34/f5b [0,4194304] 0 2024-04-20T11:09:38.030 INFO:tasks.workunit.client.1.smithi173.stdout:4/641: mknod da/d1a/d6e/db4/cd5 0 2024-04-20T11:09:38.036 INFO:tasks.workunit.client.1.smithi173.stdout:4/642: read - da/d1a/d60/fb1 zero size 2024-04-20T11:09:38.036 INFO:tasks.workunit.client.1.smithi173.stdout:2/600: read - d2/d13/d28/f7b zero size 2024-04-20T11:09:38.039 INFO:tasks.workunit.client.1.smithi173.stdout:3/562: chown d7/d9/lc 114 1 2024-04-20T11:09:38.040 INFO:tasks.workunit.client.1.smithi173.stdout:9/533: write d5/d9/d20/d29/d18/f70 [353197,130101] 0 2024-04-20T11:09:38.045 INFO:tasks.workunit.client.1.smithi173.stdout:1/514: getdents d5/d6/dc/d4a 0 2024-04-20T11:09:38.050 INFO:tasks.workunit.client.1.smithi173.stdout:0/529: write d6/d8/df/f1d [747081,123758] 0 2024-04-20T11:09:38.050 INFO:tasks.workunit.client.1.smithi173.stdout:4/643: write da/d1e/d52/d9a/fa8 [781418,37419] 0 2024-04-20T11:09:38.053 INFO:tasks.workunit.client.1.smithi173.stdout:3/563: unlink d7/d17/d61/d83/c67 0 2024-04-20T11:09:38.054 INFO:tasks.workunit.client.1.smithi173.stdout:7/587: dwrite d9/da/f46 [8388608,4194304] 0 2024-04-20T11:09:38.055 INFO:tasks.workunit.client.1.smithi173.stdout:5/584: dwrite da/de/d19/d2c/f94 [0,4194304] 0 2024-04-20T11:09:38.062 INFO:tasks.workunit.client.1.smithi173.stdout:1/515: symlink d5/d6/d17/d42/da/d20/d38/d52/d75/lab 0 2024-04-20T11:09:38.064 INFO:tasks.workunit.client.1.smithi173.stdout:4/644: symlink da/d1e/d52/d7a/ld6 0 2024-04-20T11:09:38.066 INFO:tasks.workunit.client.1.smithi173.stdout:3/564: creat d7/d2b/d8e/fb3 x:0 0 0 2024-04-20T11:09:38.067 INFO:tasks.workunit.client.1.smithi173.stdout:0/530: chown d6/d8/df/d43/c49 98739 1 2024-04-20T11:09:38.067 INFO:tasks.workunit.client.1.smithi173.stdout:5/585: fsync da/d4f/d5e/dbc/fc0 0 2024-04-20T11:09:38.070 INFO:tasks.workunit.client.1.smithi173.stdout:6/567: dread d4/d18/f62 [0,4194304] 0 2024-04-20T11:09:38.072 INFO:tasks.workunit.client.1.smithi173.stdout:4/645: dread - da/d1a/d60/fb1 zero size 2024-04-20T11:09:38.075 INFO:tasks.workunit.client.1.smithi173.stdout:2/601: dwrite d2/de/f21 [0,4194304] 0 2024-04-20T11:09:38.079 INFO:tasks.workunit.client.1.smithi173.stdout:8/547: dwrite d4/f6a [0,4194304] 0 2024-04-20T11:09:38.081 INFO:tasks.workunit.client.1.smithi173.stdout:6/568: unlink d4/d24/d3e/d9c/f8b 0 2024-04-20T11:09:38.081 INFO:tasks.workunit.client.1.smithi173.stdout:3/565: mknod d7/d17/d61/cb4 0 2024-04-20T11:09:38.085 INFO:tasks.workunit.client.1.smithi173.stdout:8/548: dread - d4/d18/fa2 zero size 2024-04-20T11:09:38.085 INFO:tasks.workunit.client.1.smithi173.stdout:3/566: write d7/f8 [1070951,57308] 0 2024-04-20T11:09:38.087 INFO:tasks.workunit.client.1.smithi173.stdout:8/549: chown d4/d18/d3a/d3c/l45 5788 1 2024-04-20T11:09:38.090 INFO:tasks.workunit.client.1.smithi173.stdout:0/531: symlink d6/d8/df/laa 0 2024-04-20T11:09:38.090 INFO:tasks.workunit.client.1.smithi173.stdout:5/586: rename da/de/d19/l69 to da/de/d19/d32/db9/lcb 0 2024-04-20T11:09:38.093 INFO:tasks.workunit.client.1.smithi173.stdout:1/516: write d5/d6/f21 [1031905,17195] 0 2024-04-20T11:09:38.095 INFO:tasks.workunit.client.1.smithi173.stdout:7/588: link d9/d1f/d7f/f87 d9/d1f/d7f/dce/db2/fd8 0 2024-04-20T11:09:38.096 INFO:tasks.workunit.client.1.smithi173.stdout:4/646: mknod da/d1e/d52/d7a/cd7 0 2024-04-20T11:09:38.099 INFO:tasks.workunit.client.1.smithi173.stdout:2/602: mkdir d2/de/d11/d9d/d87/db0/dcc 0 2024-04-20T11:09:38.099 INFO:tasks.workunit.client.1.smithi173.stdout:4/647: stat c3 0 2024-04-20T11:09:38.102 INFO:tasks.workunit.client.1.smithi173.stdout:3/567: mknod d7/d17/d61/cb5 0 2024-04-20T11:09:38.103 INFO:tasks.workunit.client.1.smithi173.stdout:2/603: symlink d2/d8c/lcd 0 2024-04-20T11:09:38.103 INFO:tasks.workunit.client.1.smithi173.stdout:9/534: dread d5/d9/d20/d2b/f49 [0,4194304] 0 2024-04-20T11:09:38.107 INFO:tasks.workunit.client.1.smithi173.stdout:7/589: write f6 [2354501,123061] 0 2024-04-20T11:09:38.108 INFO:tasks.workunit.client.1.smithi173.stdout:0/532: creat d6/d8/df/d43/d7f/fab x:0 0 0 2024-04-20T11:09:38.109 INFO:tasks.workunit.client.1.smithi173.stdout:4/648: creat da/dc/d13/fd8 x:0 0 0 2024-04-20T11:09:38.109 INFO:tasks.workunit.client.1.smithi173.stdout:9/535: dread - d5/d9/d20/f7b zero size 2024-04-20T11:09:38.111 INFO:tasks.workunit.client.1.smithi173.stdout:6/569: dwrite d4/f20 [0,4194304] 0 2024-04-20T11:09:38.114 INFO:tasks.workunit.client.1.smithi173.stdout:5/587: truncate da/de/d19/f5c 716282 0 2024-04-20T11:09:38.117 INFO:tasks.workunit.client.1.smithi173.stdout:0/533: write d6/d8/d3c/f89 [435030,105233] 0 2024-04-20T11:09:38.120 INFO:tasks.workunit.client.1.smithi173.stdout:7/590: mkdir d9/da/d17/d2b/daa/dd9 0 2024-04-20T11:09:38.126 INFO:tasks.workunit.client.1.smithi173.stdout:8/550: truncate d4/d18/f5c 5468025 0 2024-04-20T11:09:38.130 INFO:tasks.workunit.client.1.smithi173.stdout:2/604: fdatasync d2/d34/f51 0 2024-04-20T11:09:38.132 INFO:tasks.workunit.client.1.smithi173.stdout:9/536: unlink d5/d9/d20/d2b/d59/d44/f56 0 2024-04-20T11:09:38.134 INFO:tasks.workunit.client.1.smithi173.stdout:7/591: rmdir d9/d33/d91 39 2024-04-20T11:09:38.135 INFO:tasks.workunit.client.1.smithi173.stdout:6/570: mknod d4/d24/d42/cb2 0 2024-04-20T11:09:38.136 INFO:tasks.workunit.client.1.smithi173.stdout:3/568: getdents d7/d17/d61/d90 0 2024-04-20T11:09:38.137 INFO:tasks.workunit.client.1.smithi173.stdout:5/588: fsync da/de/d75/f64 0 2024-04-20T11:09:38.138 INFO:tasks.workunit.client.1.smithi173.stdout:4/649: dwrite f8 [0,4194304] 0 2024-04-20T11:09:38.140 INFO:tasks.workunit.client.1.smithi173.stdout:1/517: dread d5/d6/f73 [0,4194304] 0 2024-04-20T11:09:38.142 INFO:tasks.workunit.client.1.smithi173.stdout:8/551: creat d4/d18/d59/d5a/da8/fae x:0 0 0 2024-04-20T11:09:38.144 INFO:tasks.workunit.client.1.smithi173.stdout:3/569: write d7/d2b/f74 [5485391,82369] 0 2024-04-20T11:09:38.146 INFO:tasks.workunit.client.1.smithi173.stdout:1/518: read - d5/d6/d17/d42/d12/d1c/f82 zero size 2024-04-20T11:09:38.147 INFO:tasks.workunit.client.1.smithi173.stdout:8/552: chown d4/l3d 6 1 2024-04-20T11:09:38.152 INFO:tasks.workunit.client.1.smithi173.stdout:2/605: creat d2/d13/d46/d53/fce x:0 0 0 2024-04-20T11:09:38.154 INFO:tasks.workunit.client.1.smithi173.stdout:1/519: chown d5/d6/d17/d42/d12/d1f/f2f 51 1 2024-04-20T11:09:38.155 INFO:tasks.workunit.client.1.smithi173.stdout:6/571: symlink d4/d1c/lb3 0 2024-04-20T11:09:38.155 INFO:tasks.workunit.client.1.smithi173.stdout:2/606: fdatasync d2/d34/d58/f63 0 2024-04-20T11:09:38.157 INFO:tasks.workunit.client.1.smithi173.stdout:5/589: rename da/de/d75/d40/d59/f95 to da/d4f/d65/d8e/fcc 0 2024-04-20T11:09:38.158 INFO:tasks.workunit.client.1.smithi173.stdout:0/534: dwrite d6/d8/df/d43/d7f/f9a [0,4194304] 0 2024-04-20T11:09:38.160 INFO:tasks.workunit.client.1.smithi173.stdout:8/553: fsync d4/d18/d4f/f9a 0 2024-04-20T11:09:38.161 INFO:tasks.workunit.client.1.smithi173.stdout:2/607: write d2/d13/f38 [340206,10750] 0 2024-04-20T11:09:38.164 INFO:tasks.workunit.client.1.smithi173.stdout:6/572: creat d4/d14/d5f/fb4 x:0 0 0 2024-04-20T11:09:38.168 INFO:tasks.workunit.client.1.smithi173.stdout:5/590: write da/d4f/d5e/f6d [341251,17643] 0 2024-04-20T11:09:38.168 INFO:tasks.workunit.client.1.smithi173.stdout:8/554: chown d4/d18/fa7 62204744 1 2024-04-20T11:09:38.172 INFO:tasks.workunit.client.1.smithi173.stdout:4/650: dwrite da/dc/d13/f2b [0,4194304] 0 2024-04-20T11:09:38.176 INFO:tasks.workunit.client.1.smithi173.stdout:3/570: rename d7/d17/d61/cb5 to d7/cb6 0 2024-04-20T11:09:38.177 INFO:tasks.workunit.client.1.smithi173.stdout:4/651: dread - da/d48/d53/d39/f74 zero size 2024-04-20T11:09:38.178 INFO:tasks.workunit.client.1.smithi173.stdout:9/537: truncate d5/d9/d20/d29/f96 1277309 0 2024-04-20T11:09:38.179 INFO:tasks.workunit.client.1.smithi173.stdout:7/592: truncate d9/d33/d35/d4a/f93 3509463 0 2024-04-20T11:09:38.182 INFO:tasks.workunit.client.1.smithi173.stdout:5/591: rmdir da/de/d19/d81 39 2024-04-20T11:09:38.185 INFO:tasks.workunit.client.1.smithi173.stdout:3/571: mkdir d7/d2b/d9d/db7 0 2024-04-20T11:09:38.188 INFO:tasks.workunit.client.1.smithi173.stdout:6/573: link d4/f16 d4/d24/d42/d74/d3b/fb5 0 2024-04-20T11:09:38.189 INFO:tasks.workunit.client.1.smithi173.stdout:3/572: write d7/d17/f9f [4365747,70073] 0 2024-04-20T11:09:38.191 INFO:tasks.workunit.client.1.smithi173.stdout:1/520: dwrite d5/d6/d17/d42/d12/d1c/f8b [0,4194304] 0 2024-04-20T11:09:38.192 INFO:tasks.workunit.client.1.smithi173.stdout:5/592: symlink da/de/d19/d32/lcd 0 2024-04-20T11:09:38.195 INFO:tasks.workunit.client.1.smithi173.stdout:9/538: truncate d5/d4c/d79/f7a 1193563 0 2024-04-20T11:09:38.197 INFO:tasks.workunit.client.1.smithi173.stdout:1/521: chown d5/d6/dc/c2a 94860998 1 2024-04-20T11:09:38.207 INFO:tasks.workunit.client.1.smithi173.stdout:2/608: dwrite d2/f3e [0,4194304] 0 2024-04-20T11:09:38.208 INFO:tasks.workunit.client.1.smithi173.stdout:5/593: rmdir da/de/d75 39 2024-04-20T11:09:38.208 INFO:tasks.workunit.client.1.smithi173.stdout:8/555: truncate d4/d18/d3a/d3c/f54 1126396 0 2024-04-20T11:09:38.215 INFO:tasks.workunit.client.1.smithi173.stdout:2/609: mkdir d2/d13/d9a/dcf 0 2024-04-20T11:09:38.215 INFO:tasks.workunit.client.1.smithi173.stdout:1/522: mkdir d5/d6/d17/d42/da/df/d40/dac 0 2024-04-20T11:09:38.220 INFO:tasks.workunit.client.1.smithi173.stdout:7/593: truncate d9/fa0 4033720 0 2024-04-20T11:09:38.221 INFO:tasks.workunit.client.1.smithi173.stdout:6/574: dwrite d4/d14/f43 [0,4194304] 0 2024-04-20T11:09:38.221 INFO:tasks.workunit.client.1.smithi173.stdout:9/539: truncate d5/d9/d20/d29/d18/f9c 1264457 0 2024-04-20T11:09:38.223 INFO:tasks.workunit.client.1.smithi173.stdout:2/610: dread - d2/d13/d46/d53/fbd zero size 2024-04-20T11:09:38.225 INFO:tasks.workunit.client.1.smithi173.stdout:8/556: chown d4/d18/d59/d5a/c83 4271 1 2024-04-20T11:09:38.227 INFO:tasks.workunit.client.1.smithi173.stdout:8/557: fsync d4/d18/d4f/f9a 0 2024-04-20T11:09:38.237 INFO:tasks.workunit.client.1.smithi173.stdout:4/652: dread da/dc/fe [0,4194304] 0 2024-04-20T11:09:38.241 INFO:tasks.workunit.client.1.smithi173.stdout:1/523: mkdir d5/d6/d17/d42/da/d20/dad 0 2024-04-20T11:09:38.244 INFO:tasks.workunit.client.1.smithi173.stdout:6/575: dread d4/d14/f6a [0,4194304] 0 2024-04-20T11:09:38.245 INFO:tasks.workunit.client.1.smithi173.stdout:3/573: truncate d7/f51 469628 0 2024-04-20T11:09:38.252 INFO:tasks.workunit.client.1.smithi173.stdout:4/653: creat da/d48/fd9 x:0 0 0 2024-04-20T11:09:38.253 INFO:tasks.workunit.client.1.smithi173.stdout:7/594: creat d9/d33/d40/dc9/fda x:0 0 0 2024-04-20T11:09:38.254 INFO:tasks.workunit.client.1.smithi173.stdout:9/540: mkdir d5/d9/d20/d2b/d59/da1 0 2024-04-20T11:09:38.256 INFO:tasks.workunit.client.1.smithi173.stdout:1/524: symlink d5/d6/dc/d7f/lae 0 2024-04-20T11:09:38.257 INFO:tasks.workunit.client.1.smithi173.stdout:3/574: write d7/d17/fa8 [495235,91018] 0 2024-04-20T11:09:38.263 INFO:tasks.workunit.client.1.smithi173.stdout:3/575: fsync d7/d9/f2e 0 2024-04-20T11:09:38.264 INFO:tasks.workunit.client.1.smithi173.stdout:2/611: dread d2/de/f2b [0,4194304] 0 2024-04-20T11:09:38.265 INFO:tasks.workunit.client.1.smithi173.stdout:5/594: dread f9 [0,4194304] 0 2024-04-20T11:09:38.266 INFO:tasks.workunit.client.1.smithi173.stdout:0/535: dread d6/d8/df/d2c/d42/d50/d86/f94 [0,4194304] 0 2024-04-20T11:09:38.267 INFO:tasks.workunit.client.1.smithi173.stdout:7/595: fdatasync d9/da/d2e/f37 0 2024-04-20T11:09:38.269 INFO:tasks.workunit.client.1.smithi173.stdout:6/576: dwrite d4/d18/f66 [0,4194304] 0 2024-04-20T11:09:38.272 INFO:tasks.workunit.client.1.smithi173.stdout:2/612: read d2/d13/f74 [4626057,10368] 0 2024-04-20T11:09:38.273 INFO:tasks.workunit.client.1.smithi173.stdout:1/525: rename d5/d6/dc/c1d to d5/d6/d3b/caf 0 2024-04-20T11:09:38.278 INFO:tasks.workunit.client.1.smithi173.stdout:8/558: dwrite d4/d18/d3a/f7a [0,4194304] 0 2024-04-20T11:09:38.280 INFO:tasks.workunit.client.1.smithi173.stdout:5/595: symlink da/d4f/d5e/lce 0 2024-04-20T11:09:38.281 INFO:tasks.workunit.client.1.smithi173.stdout:7/596: write d9/dbc/fbe [256609,103209] 0 2024-04-20T11:09:38.283 INFO:tasks.workunit.client.1.smithi173.stdout:3/576: unlink d7/d53/l59 0 2024-04-20T11:09:38.289 INFO:tasks.workunit.client.1.smithi173.stdout:2/613: dread - d2/d13/d46/d4b/f83 zero size 2024-04-20T11:09:38.291 INFO:tasks.workunit.client.1.smithi173.stdout:4/654: dread da/d1a/f81 [0,4194304] 0 2024-04-20T11:09:38.294 INFO:tasks.workunit.client.1.smithi173.stdout:2/614: readlink d2/l90 0 2024-04-20T11:09:38.297 INFO:tasks.workunit.client.1.smithi173.stdout:5/596: chown da/de/d19/f76 12473 1 2024-04-20T11:09:38.297 INFO:tasks.workunit.client.1.smithi173.stdout:9/541: dwrite d5/d9/d20/d29/d18/f77 [0,4194304] 0 2024-04-20T11:09:38.299 INFO:tasks.workunit.client.1.smithi173.stdout:7/597: rmdir d9/da/d17/d2b/daa 39 2024-04-20T11:09:38.301 INFO:tasks.workunit.client.1.smithi173.stdout:4/655: dread - da/dc/d15/d8b/fcf zero size 2024-04-20T11:09:38.302 INFO:tasks.workunit.client.1.smithi173.stdout:2/615: mkdir d2/d13/d46/dd0 0 2024-04-20T11:09:38.307 INFO:tasks.workunit.client.1.smithi173.stdout:9/542: dread - d5/d4c/d79/d80/f8d zero size 2024-04-20T11:09:38.313 INFO:tasks.workunit.client.1.smithi173.stdout:1/526: write d5/d6/d17/d42/d12/d1c/f82 [231926,83809] 0 2024-04-20T11:09:38.314 INFO:tasks.workunit.client.1.smithi173.stdout:2/616: write d2/de/f84 [112514,36837] 0 2024-04-20T11:09:38.316 INFO:tasks.workunit.client.1.smithi173.stdout:4/656: chown da/d1a/l3c 2 1 2024-04-20T11:09:38.318 INFO:tasks.workunit.client.1.smithi173.stdout:0/536: dwrite d6/d24/f74 [0,4194304] 0 2024-04-20T11:09:38.319 INFO:tasks.workunit.client.1.smithi173.stdout:5/597: rename da/de/d19/d32/db9 to da/de/d19/d2c/dcf 0 2024-04-20T11:09:38.324 INFO:tasks.workunit.client.1.smithi173.stdout:7/598: mknod d9/da/d17/d2b/daa/dd9/cdb 0 2024-04-20T11:09:38.326 INFO:tasks.workunit.client.1.smithi173.stdout:1/527: rmdir d5/d6/d17/d42/da 39 2024-04-20T11:09:38.327 INFO:tasks.workunit.client.1.smithi173.stdout:9/543: chown d5/c30 104 1 2024-04-20T11:09:38.327 INFO:tasks.workunit.client.1.smithi173.stdout:8/559: dwrite d4/d18/d59/f64 [0,4194304] 0 2024-04-20T11:09:38.328 INFO:tasks.workunit.client.1.smithi173.stdout:6/577: dwrite d4/d24/d42/d74/d3b/fb5 [0,4194304] 0 2024-04-20T11:09:38.332 INFO:tasks.workunit.client.1.smithi173.stdout:4/657: unlink da/dc/d62/fc8 0 2024-04-20T11:09:38.332 INFO:tasks.workunit.client.1.smithi173.stdout:5/598: symlink da/d4f/d5e/ld0 0 2024-04-20T11:09:38.335 INFO:tasks.workunit.client.1.smithi173.stdout:1/528: read - d5/d6/d17/d42/d12/d1c/f5b zero size 2024-04-20T11:09:38.337 INFO:tasks.workunit.client.1.smithi173.stdout:2/617: rename d2/d13/l99 to d2/d34/d81/dca/ld1 0 2024-04-20T11:09:38.342 INFO:tasks.workunit.client.1.smithi173.stdout:8/560: creat d4/d18/da6/faf x:0 0 0 2024-04-20T11:09:38.342 INFO:tasks.workunit.client.1.smithi173.stdout:5/599: truncate da/d4f/d65/f86 173193 0 2024-04-20T11:09:38.343 INFO:tasks.workunit.client.1.smithi173.stdout:2/618: write d2/d13/d46/f85 [4031588,14775] 0 2024-04-20T11:09:38.343 INFO:tasks.workunit.client.1.smithi173.stdout:1/529: write d5/d6/d17/d42/d12/d1c/f5b [818639,82210] 0 2024-04-20T11:09:38.347 INFO:tasks.workunit.client.1.smithi173.stdout:7/599: mkdir d9/ddc 0 2024-04-20T11:09:38.348 INFO:tasks.workunit.client.1.smithi173.stdout:4/658: rmdir da/d1e/d52/d9a 39 2024-04-20T11:09:38.351 INFO:tasks.workunit.client.1.smithi173.stdout:1/530: rename d5/d6/d3b to d5/d6/d3b/db0 22 2024-04-20T11:09:38.351 INFO:tasks.workunit.client.1.smithi173.stdout:2/619: readlink d2/d13/d46/d53/d4e/l71 0 2024-04-20T11:09:38.354 INFO:tasks.workunit.client.1.smithi173.stdout:2/620: write d2/d13/d46/d53/fce [820626,61172] 0 2024-04-20T11:09:38.354 INFO:tasks.workunit.client.1.smithi173.stdout:1/531: stat d5/d6/d17/d42/d12/d1c/d85/f9e 0 2024-04-20T11:09:38.355 INFO:tasks.workunit.client.1.smithi173.stdout:8/561: symlink d4/d18/lb0 0 2024-04-20T11:09:38.358 INFO:tasks.workunit.client.1.smithi173.stdout:8/562: write d4/d18/f21 [1958657,811] 0 2024-04-20T11:09:38.360 INFO:tasks.workunit.client.1.smithi173.stdout:9/544: dread d5/d9/d20/d29/f5c [0,4194304] 0 2024-04-20T11:09:38.380 INFO:tasks.workunit.client.1.smithi173.stdout:6/578: dread d4/f37 [0,4194304] 0 2024-04-20T11:09:38.382 INFO:tasks.workunit.client.1.smithi173.stdout:6/579: write d4/d24/f6b [810546,95561] 0 2024-04-20T11:09:38.406 INFO:tasks.workunit.client.1.smithi173.stdout:0/537: write d6/f13 [2798360,27546] 0 2024-04-20T11:09:38.412 INFO:tasks.workunit.client.1.smithi173.stdout:8/563: mkdir d4/d18/d59/d5a/d7b/db1 0 2024-04-20T11:09:38.413 INFO:tasks.workunit.client.1.smithi173.stdout:9/545: fdatasync d5/d9/d20/f40 0 2024-04-20T11:09:38.416 INFO:tasks.workunit.client.1.smithi173.stdout:7/600: mknod d9/da/cdd 0 2024-04-20T11:09:38.417 INFO:tasks.workunit.client.1.smithi173.stdout:5/600: getdents da/d34/d7f 0 2024-04-20T11:09:38.419 INFO:tasks.workunit.client.1.smithi173.stdout:4/659: mknod da/d1e/d52/d9a/dd3/cda 0 2024-04-20T11:09:38.427 INFO:tasks.workunit.client.1.smithi173.stdout:9/546: creat d5/d9/d20/d29/d18/fa2 x:0 0 0 2024-04-20T11:09:38.430 INFO:tasks.workunit.client.1.smithi173.stdout:7/601: unlink d9/da/d2e/f37 0 2024-04-20T11:09:38.432 INFO:tasks.workunit.client.1.smithi173.stdout:7/602: dread - d9/d33/d40/dc9/fda zero size 2024-04-20T11:09:38.434 INFO:tasks.workunit.client.1.smithi173.stdout:7/603: fsync d9/da/d17/d2b/f71 0 2024-04-20T11:09:38.438 INFO:tasks.workunit.client.1.smithi173.stdout:4/660: rename da/dc/d15/l69 to da/d1a/d60/d66/dae/ldb 0 2024-04-20T11:09:38.439 INFO:tasks.workunit.client.1.smithi173.stdout:2/621: rmdir d2/d13/d9a/dcf 0 2024-04-20T11:09:38.439 INFO:tasks.workunit.client.1.smithi173.stdout:8/564: mknod d4/d18/d4f/cb2 0 2024-04-20T11:09:38.441 INFO:tasks.workunit.client.1.smithi173.stdout:2/622: fdatasync d2/d13/d46/d53/d4e/f57 0 2024-04-20T11:09:38.446 INFO:tasks.workunit.client.1.smithi173.stdout:0/538: rmdir d6/d8/df/d60 0 2024-04-20T11:09:38.448 INFO:tasks.workunit.client.1.smithi173.stdout:7/604: creat d9/d33/d35/d54/fde x:0 0 0 2024-04-20T11:09:38.449 INFO:tasks.workunit.client.1.smithi173.stdout:4/661: mkdir da/dc/d62/d84/ddc 0 2024-04-20T11:09:38.451 INFO:tasks.workunit.client.1.smithi173.stdout:1/532: getdents d5/d6/d17/d42/da/d20 0 2024-04-20T11:09:38.453 INFO:tasks.workunit.client.1.smithi173.stdout:7/605: stat d9/d33/d91/d99/fc0 0 2024-04-20T11:09:38.453 INFO:tasks.workunit.client.1.smithi173.stdout:4/662: mkdir da/dc/ddd 0 2024-04-20T11:09:38.455 INFO:tasks.workunit.client.1.smithi173.stdout:9/547: dread d5/d9/d20/d29/d2d/d33/f6b [0,4194304] 0 2024-04-20T11:09:38.457 INFO:tasks.workunit.client.1.smithi173.stdout:1/533: mknod d5/d6/d17/d42/da/cb1 0 2024-04-20T11:09:38.458 INFO:tasks.workunit.client.1.smithi173.stdout:0/539: mkdir d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac 0 2024-04-20T11:09:38.460 INFO:tasks.workunit.client.1.smithi173.stdout:0/540: chown d6/d8/f12 64020 1 2024-04-20T11:09:38.460 INFO:tasks.workunit.client.1.smithi173.stdout:7/606: dread d9/da/f25 [0,4194304] 0 2024-04-20T11:09:38.463 INFO:tasks.workunit.client.1.smithi173.stdout:9/548: write d5/d9/d20/d29/f5c [641268,117162] 0 2024-04-20T11:09:38.465 INFO:tasks.workunit.client.1.smithi173.stdout:1/534: symlink d5/d6/d17/d93/lb2 0 2024-04-20T11:09:38.468 INFO:tasks.workunit.client.1.smithi173.stdout:4/663: link da/d1e/d52/f68 da/d1e/d52/d7a/fde 0 2024-04-20T11:09:38.470 INFO:tasks.workunit.client.1.smithi173.stdout:0/541: rename d6/d8/df/d2c/d2d/f52 to d6/d24/fad 0 2024-04-20T11:09:38.470 INFO:tasks.workunit.client.1.smithi173.stdout:8/565: link d4/d18/d3a/c7f d4/d18/d3a/d9b/cb3 0 2024-04-20T11:09:38.472 INFO:tasks.workunit.client.1.smithi173.stdout:1/535: mknod d5/d6/d17/d42/d12/d1c/cb3 0 2024-04-20T11:09:38.474 INFO:tasks.workunit.client.1.smithi173.stdout:8/566: stat d4/d18/d59/cab 0 2024-04-20T11:09:38.475 INFO:tasks.workunit.client.1.smithi173.stdout:9/549: creat d5/d9/d20/d2b/d59/da1/fa3 x:0 0 0 2024-04-20T11:09:38.476 INFO:tasks.workunit.client.1.smithi173.stdout:8/567: dread - d4/d18/fa7 zero size 2024-04-20T11:09:38.476 INFO:tasks.workunit.client.1.smithi173.stdout:0/542: rename d6/d24/c31 to d6/cae 0 2024-04-20T11:09:38.478 INFO:tasks.workunit.client.1.smithi173.stdout:1/536: mkdir d5/d6/d17/d42/da/df/d40/db4 0 2024-04-20T11:09:38.480 INFO:tasks.workunit.client.1.smithi173.stdout:9/550: chown d5/d9/d20/d29/d18/d8a/f99 35819 1 2024-04-20T11:09:38.482 INFO:tasks.workunit.client.1.smithi173.stdout:8/568: write d4/d18/d3a/d3c/f37 [6817171,83276] 0 2024-04-20T11:09:38.484 INFO:tasks.workunit.client.1.smithi173.stdout:0/543: dread d6/d5f/d8f/fa7 [0,4194304] 0 2024-04-20T11:09:38.484 INFO:tasks.workunit.client.1.smithi173.stdout:7/607: dread d9/da/d31/f38 [0,4194304] 0 2024-04-20T11:09:38.487 INFO:tasks.workunit.client.1.smithi173.stdout:1/537: mknod d5/d6/d17/d93/cb5 0 2024-04-20T11:09:38.488 INFO:tasks.workunit.client.1.smithi173.stdout:4/664: dwrite da/dc/d15/f5f [0,4194304] 0 2024-04-20T11:09:38.490 INFO:tasks.workunit.client.1.smithi173.stdout:7/608: dread d9/d1f/f7d [0,4194304] 0 2024-04-20T11:09:38.490 INFO:tasks.workunit.client.1.smithi173.stdout:8/569: fdatasync d4/d18/d3a/f97 0 2024-04-20T11:09:38.493 INFO:tasks.workunit.client.1.smithi173.stdout:7/609: truncate d9/d33/d40/f49 4459403 0 2024-04-20T11:09:38.494 INFO:tasks.workunit.client.1.smithi173.stdout:9/551: rename d5/d4c/l75 to d5/d9/d20/d2b/d59/d44/d78/d9d/la4 0 2024-04-20T11:09:38.495 INFO:tasks.workunit.client.1.smithi173.stdout:1/538: mkdir d5/d6/d17/d93/db6 0 2024-04-20T11:09:38.497 INFO:tasks.workunit.client.1.smithi173.stdout:3/577: sync 2024-04-20T11:09:38.499 INFO:tasks.workunit.client.1.smithi173.stdout:3/578: chown d7/d9/f66 6 1 2024-04-20T11:09:38.499 INFO:tasks.workunit.client.1.smithi173.stdout:4/665: creat da/d1a/d60/d66/dae/fdf x:0 0 0 2024-04-20T11:09:38.503 INFO:tasks.workunit.client.1.smithi173.stdout:8/570: creat d4/d18/d59/d5a/d7b/fb4 x:0 0 0 2024-04-20T11:09:38.504 INFO:tasks.workunit.client.1.smithi173.stdout:7/610: creat d9/da/d43/fdf x:0 0 0 2024-04-20T11:09:38.505 INFO:tasks.workunit.client.1.smithi173.stdout:9/552: mknod d5/d9/d20/ca5 0 2024-04-20T11:09:38.509 INFO:tasks.workunit.client.1.smithi173.stdout:0/544: getdents d6/d8/df/d43 0 2024-04-20T11:09:38.510 INFO:tasks.workunit.client.1.smithi173.stdout:4/666: write da/d1e/d52/f71 [673998,65629] 0 2024-04-20T11:09:38.512 INFO:tasks.workunit.client.1.smithi173.stdout:8/571: stat d4/d18/d3a/c6c 0 2024-04-20T11:09:38.514 INFO:tasks.workunit.client.1.smithi173.stdout:4/667: write da/dc/d13/d65/f8f [241719,40516] 0 2024-04-20T11:09:38.516 INFO:tasks.workunit.client.1.smithi173.stdout:4/668: stat da/dc/l88 0 2024-04-20T11:09:38.517 INFO:tasks.workunit.client.1.smithi173.stdout:1/539: dwrite d5/d6/d17/d42/da/d20/d38/d52/d75/f7b [0,4194304] 0 2024-04-20T11:09:38.519 INFO:tasks.workunit.client.1.smithi173.stdout:9/553: unlink d5/d9/d20/d29/d2d/l89 0 2024-04-20T11:09:38.519 INFO:tasks.workunit.client.1.smithi173.stdout:7/611: rename d9/d33/d35/d4a/c8e to d9/d33/d40/d7a/ce0 0 2024-04-20T11:09:38.520 INFO:tasks.workunit.client.1.smithi173.stdout:1/540: dread - d5/d6/d17/d42/d12/f7e zero size 2024-04-20T11:09:38.522 INFO:tasks.workunit.client.1.smithi173.stdout:5/601: truncate da/d34/f47 4573340 0 2024-04-20T11:09:38.523 INFO:tasks.workunit.client.1.smithi173.stdout:7/612: chown d9/d33/d85/ca7 6291 1 2024-04-20T11:09:38.527 INFO:tasks.workunit.client.1.smithi173.stdout:4/669: mkdir da/d1e/da6/de0 0 2024-04-20T11:09:38.527 INFO:tasks.workunit.client.1.smithi173.stdout:8/572: read - d4/d18/d4f/d5b/f87 zero size 2024-04-20T11:09:38.530 INFO:tasks.workunit.client.1.smithi173.stdout:1/541: rmdir d5/d6/d17/d42/da/d20/d38/d52/d75 39 2024-04-20T11:09:38.535 INFO:tasks.workunit.client.1.smithi173.stdout:0/545: dwrite d6/d8/d51/f5b [0,4194304] 0 2024-04-20T11:09:38.536 INFO:tasks.workunit.client.1.smithi173.stdout:9/554: creat d5/d9/d20/d4f/d5e/d9e/fa6 x:0 0 0 2024-04-20T11:09:38.540 INFO:tasks.workunit.client.1.smithi173.stdout:5/602: creat da/fd1 x:0 0 0 2024-04-20T11:09:38.540 INFO:tasks.workunit.client.1.smithi173.stdout:0/546: chown d6/d8/df/d2c/d42 2 1 2024-04-20T11:09:38.542 INFO:tasks.workunit.client.1.smithi173.stdout:8/573: creat d4/d18/d59/d5a/d7b/d8c/fb5 x:0 0 0 2024-04-20T11:09:38.543 INFO:tasks.workunit.client.1.smithi173.stdout:9/555: mkdir d5/d4c/d79/da7 0 2024-04-20T11:09:38.546 INFO:tasks.workunit.client.1.smithi173.stdout:2/623: link d2/d8c/laa d2/d34/ld2 0 2024-04-20T11:09:38.550 INFO:tasks.workunit.client.1.smithi173.stdout:1/542: creat d5/d6/d17/d42/d12/d1c/d85/fb7 x:0 0 0 2024-04-20T11:09:38.551 INFO:tasks.workunit.client.1.smithi173.stdout:2/624: truncate d2/fc5 556551 0 2024-04-20T11:09:38.553 INFO:tasks.workunit.client.1.smithi173.stdout:7/613: dwrite d9/da/d17/f1a [0,4194304] 0 2024-04-20T11:09:38.557 INFO:tasks.workunit.client.1.smithi173.stdout:4/670: dwrite da/d1a/d60/d66/dae/fdf [0,4194304] 0 2024-04-20T11:09:38.558 INFO:tasks.workunit.client.1.smithi173.stdout:2/625: creat d2/d34/d58/fd3 x:0 0 0 2024-04-20T11:09:38.565 INFO:tasks.workunit.client.1.smithi173.stdout:6/580: unlink d4/d24/l91 0 2024-04-20T11:09:38.565 INFO:tasks.workunit.client.1.smithi173.stdout:3/579: write d7/f51 [1184092,36654] 0 2024-04-20T11:09:38.566 INFO:tasks.workunit.client.1.smithi173.stdout:8/574: creat d4/d18/d59/fb6 x:0 0 0 2024-04-20T11:09:38.568 INFO:tasks.workunit.client.1.smithi173.stdout:2/626: read d2/d13/d46/d53/d9f/fc9 [1073681,108216] 0 2024-04-20T11:09:38.571 INFO:tasks.workunit.client.1.smithi173.stdout:4/671: rmdir da/d1a/d60/d66/dae 39 2024-04-20T11:09:38.578 INFO:tasks.workunit.client.1.smithi173.stdout:3/580: chown d7/f15 148884114 1 2024-04-20T11:09:38.582 INFO:tasks.workunit.client.1.smithi173.stdout:8/575: mkdir d4/d18/d3a/d3c/d86/db7 0 2024-04-20T11:09:38.582 INFO:tasks.workunit.client.1.smithi173.stdout:0/547: write d6/d8/d3c/d44/d75/fa1 [1374821,38989] 0 2024-04-20T11:09:38.582 INFO:tasks.workunit.client.1.smithi173.stdout:1/543: dwrite d5/d6/d17/d42/da/f6a [0,4194304] 0 2024-04-20T11:09:38.584 INFO:tasks.workunit.client.1.smithi173.stdout:2/627: unlink d2/d34/f51 0 2024-04-20T11:09:38.585 INFO:tasks.workunit.client.1.smithi173.stdout:8/576: chown d4/d18/d3a/d3c/l45 2822533 1 2024-04-20T11:09:38.588 INFO:tasks.workunit.client.1.smithi173.stdout:4/672: rmdir da/dc/d15 39 2024-04-20T11:09:38.588 INFO:tasks.workunit.client.1.smithi173.stdout:8/577: stat d4/d18/d3a/d3c/c82 0 2024-04-20T11:09:38.600 INFO:tasks.workunit.client.1.smithi173.stdout:1/544: symlink d5/d6/d3b/lb8 0 2024-04-20T11:09:38.600 INFO:tasks.workunit.client.1.smithi173.stdout:9/556: truncate d5/d9/d20/d29/d18/d8a/f99 3315278 0 2024-04-20T11:09:38.606 INFO:tasks.workunit.client.1.smithi173.stdout:8/578: unlink d4/d18/d3a/d9b/c9e 0 2024-04-20T11:09:38.607 INFO:tasks.workunit.client.1.smithi173.stdout:7/614: getdents d9/da/d17/d2b/daa 0 2024-04-20T11:09:38.608 INFO:tasks.workunit.client.1.smithi173.stdout:3/581: truncate d7/f18 392986 0 2024-04-20T11:09:38.608 INFO:tasks.workunit.client.1.smithi173.stdout:8/579: write d4/f10 [3967625,78585] 0 2024-04-20T11:09:38.609 INFO:tasks.workunit.client.1.smithi173.stdout:3/582: readlink d7/d2b/l4c 0 2024-04-20T11:09:38.609 INFO:tasks.workunit.client.1.smithi173.stdout:7/615: stat d9/d1f/d72/d8f/ld4 0 2024-04-20T11:09:38.609 INFO:tasks.workunit.client.1.smithi173.stdout:6/581: rename d4/d24/d42/l5d to d4/d24/d42/lb6 0 2024-04-20T11:09:38.611 INFO:tasks.workunit.client.1.smithi173.stdout:8/580: write d4/d18/f24 [4352400,66617] 0 2024-04-20T11:09:38.613 INFO:tasks.workunit.client.1.smithi173.stdout:8/581: chown l3 497055938 1 2024-04-20T11:09:38.613 INFO:tasks.workunit.client.1.smithi173.stdout:6/582: dread - d4/d24/d42/d74/d3b/fa5 zero size 2024-04-20T11:09:38.623 INFO:tasks.workunit.client.1.smithi173.stdout:0/548: dwrite d6/d24/f53 [0,4194304] 0 2024-04-20T11:09:38.626 INFO:tasks.workunit.client.1.smithi173.stdout:6/583: dread d4/f27 [0,4194304] 0 2024-04-20T11:09:38.626 INFO:tasks.workunit.client.1.smithi173.stdout:2/628: creat d2/de/fd4 x:0 0 0 2024-04-20T11:09:38.626 INFO:tasks.workunit.client.1.smithi173.stdout:1/545: mknod d5/d6/d17/d42/d12/cb9 0 2024-04-20T11:09:38.627 INFO:tasks.workunit.client.1.smithi173.stdout:9/557: symlink d5/d9/d20/d4f/d5e/d7d/d95/la8 0 2024-04-20T11:09:38.628 INFO:tasks.workunit.client.1.smithi173.stdout:0/549: write d6/d8/d3c/d44/d75/fa1 [1528100,42560] 0 2024-04-20T11:09:38.629 INFO:tasks.workunit.client.1.smithi173.stdout:8/582: rename d4/d18/d3a to d4/d18/d59/d6d/db8 0 2024-04-20T11:09:38.629 INFO:tasks.workunit.client.1.smithi173.stdout:2/629: write d2/de/f84 [476270,107053] 0 2024-04-20T11:09:38.632 INFO:tasks.workunit.client.1.smithi173.stdout:4/673: truncate da/dc/d15/f35 981599 0 2024-04-20T11:09:38.634 INFO:tasks.workunit.client.1.smithi173.stdout:8/583: truncate d4/d18/f9c 398045 0 2024-04-20T11:09:38.638 INFO:tasks.workunit.client.1.smithi173.stdout:5/603: sync 2024-04-20T11:09:38.638 INFO:tasks.workunit.client.1.smithi173.stdout:1/546: read d5/d6/d17/d42/d12/d1c/f6f [739404,12888] 0 2024-04-20T11:09:38.640 INFO:tasks.workunit.client.1.smithi173.stdout:4/674: chown da/d48/d53/f37 971 1 2024-04-20T11:09:38.642 INFO:tasks.workunit.client.1.smithi173.stdout:7/616: dwrite d9/d33/d91/d99/fc0 [0,4194304] 0 2024-04-20T11:09:38.642 INFO:tasks.workunit.client.1.smithi173.stdout:4/675: fsync da/d1a/d60/d7d/fc5 0 2024-04-20T11:09:38.644 INFO:tasks.workunit.client.1.smithi173.stdout:9/558: rmdir d5/d9/d20/d29 39 2024-04-20T11:09:38.644 INFO:tasks.workunit.client.1.smithi173.stdout:4/676: dread - da/dc/fb8 zero size 2024-04-20T11:09:38.646 INFO:tasks.workunit.client.1.smithi173.stdout:2/630: mknod d2/d34/d81/cd5 0 2024-04-20T11:09:38.652 INFO:tasks.workunit.client.1.smithi173.stdout:6/584: unlink d4/d1c/l45 0 2024-04-20T11:09:38.659 INFO:tasks.workunit.client.1.smithi173.stdout:7/617: dread d9/d33/f47 [0,4194304] 0 2024-04-20T11:09:38.663 INFO:tasks.workunit.client.1.smithi173.stdout:8/584: dread d4/d18/f30 [0,4194304] 0 2024-04-20T11:09:38.664 INFO:tasks.workunit.client.1.smithi173.stdout:2/631: write d2/d34/d58/f5a [341437,17395] 0 2024-04-20T11:09:38.669 INFO:tasks.workunit.client.1.smithi173.stdout:5/604: mknod da/de/cd2 0 2024-04-20T11:09:38.671 INFO:tasks.workunit.client.1.smithi173.stdout:9/559: dread d5/f2f [0,4194304] 0 2024-04-20T11:09:38.671 INFO:tasks.workunit.client.1.smithi173.stdout:1/547: mknod d5/d6/dc/cba 0 2024-04-20T11:09:38.676 INFO:tasks.workunit.client.1.smithi173.stdout:8/585: mkdir d4/d18/d59/d5a/da8/db9 0 2024-04-20T11:09:38.678 INFO:tasks.workunit.client.1.smithi173.stdout:8/586: chown d4/d18/d59/d6d/db8/d3c/l45 21679776 1 2024-04-20T11:09:38.681 INFO:tasks.workunit.client.1.smithi173.stdout:3/583: dwrite d7/d17/f20 [0,4194304] 0 2024-04-20T11:09:38.681 INFO:tasks.workunit.client.1.smithi173.stdout:8/587: readlink d4/d18/d59/d6d/db8/d3c/l4d 0 2024-04-20T11:09:38.684 INFO:tasks.workunit.client.1.smithi173.stdout:8/588: fsync d4/d18/d59/d6d/db8/d3c/f37 0 2024-04-20T11:09:38.686 INFO:tasks.workunit.client.1.smithi173.stdout:0/550: getdents d6/d54 0 2024-04-20T11:09:38.688 INFO:tasks.workunit.client.1.smithi173.stdout:2/632: mknod d2/cd6 0 2024-04-20T11:09:38.691 INFO:tasks.workunit.client.1.smithi173.stdout:2/633: fsync d2/d13/f38 0 2024-04-20T11:09:38.697 INFO:tasks.workunit.client.1.smithi173.stdout:3/584: rename d7/d2b/f74 to d7/d17/d61/d83/fb8 0 2024-04-20T11:09:38.704 INFO:tasks.workunit.client.1.smithi173.stdout:4/677: getdents da/dc/d15 0 2024-04-20T11:09:38.706 INFO:tasks.workunit.client.1.smithi173.stdout:1/548: truncate d5/d6/d17/d42/f39 343224 0 2024-04-20T11:09:38.707 INFO:tasks.workunit.client.1.smithi173.stdout:4/678: dread - da/d48/d53/d39/f96 zero size 2024-04-20T11:09:38.711 INFO:tasks.workunit.client.1.smithi173.stdout:0/551: mkdir d6/d8/df/d2c/d42/d50/d86/d8b/da2/daf 0 2024-04-20T11:09:38.713 INFO:tasks.workunit.client.1.smithi173.stdout:2/634: creat d2/d13/d28/fd7 x:0 0 0 2024-04-20T11:09:38.717 INFO:tasks.workunit.client.1.smithi173.stdout:3/585: mknod d7/d53/cb9 0 2024-04-20T11:09:38.722 INFO:tasks.workunit.client.1.smithi173.stdout:4/679: fsync da/d1a/d60/f63 0 2024-04-20T11:09:38.728 INFO:tasks.workunit.client.1.smithi173.stdout:3/586: creat d7/d17/d58/d7f/fba x:0 0 0 2024-04-20T11:09:38.728 INFO:tasks.workunit.client.1.smithi173.stdout:2/635: readlink d2/d13/d46/d53/lb1 0 2024-04-20T11:09:38.730 INFO:tasks.workunit.client.1.smithi173.stdout:3/587: stat d7/d17/d58/d8a/la5 0 2024-04-20T11:09:38.734 INFO:tasks.workunit.client.1.smithi173.stdout:3/588: unlink d7/d9/f35 0 2024-04-20T11:09:38.735 INFO:tasks.workunit.client.1.smithi173.stdout:4/680: link da/d48/d53/l9d da/d48/d53/d39/le1 0 2024-04-20T11:09:38.739 INFO:tasks.workunit.client.1.smithi173.stdout:6/585: truncate d4/d14/d5f/f76 124184 0 2024-04-20T11:09:38.740 INFO:tasks.workunit.client.1.smithi173.stdout:4/681: creat da/d1e/d52/d7a/fe2 x:0 0 0 2024-04-20T11:09:38.745 INFO:tasks.workunit.client.1.smithi173.stdout:2/636: getdents d2/d13/d28/d4a 0 2024-04-20T11:09:38.745 INFO:tasks.workunit.client.1.smithi173.stdout:4/682: read - da/d1e/d52/d7a/f7f zero size 2024-04-20T11:09:38.752 INFO:tasks.workunit.client.1.smithi173.stdout:6/586: mknod d4/d14/cb7 0 2024-04-20T11:09:38.758 INFO:tasks.workunit.client.1.smithi173.stdout:4/683: unlink da/d48/d53/d39/f74 0 2024-04-20T11:09:38.761 INFO:tasks.workunit.client.1.smithi173.stdout:5/605: dwrite da/de/d19/d2c/f62 [0,4194304] 0 2024-04-20T11:09:38.762 INFO:tasks.workunit.client.1.smithi173.stdout:9/560: write d5/d9/d20/d29/d18/f9c [2106455,117414] 0 2024-04-20T11:09:38.764 INFO:tasks.workunit.client.1.smithi173.stdout:3/589: dwrite d7/d17/d58/fb2 [0,4194304] 0 2024-04-20T11:09:38.770 INFO:tasks.workunit.client.1.smithi173.stdout:6/587: rename d4/d24/l88 to d4/d24/d42/d81/lb8 0 2024-04-20T11:09:38.771 INFO:tasks.workunit.client.1.smithi173.stdout:8/589: truncate d4/d18/d4f/f81 3008105 0 2024-04-20T11:09:38.775 INFO:tasks.workunit.client.1.smithi173.stdout:5/606: creat da/d4f/fd3 x:0 0 0 2024-04-20T11:09:38.777 INFO:tasks.workunit.client.1.smithi173.stdout:0/552: dwrite d6/d8/fe [0,4194304] 0 2024-04-20T11:09:38.781 INFO:tasks.workunit.client.1.smithi173.stdout:0/553: readlink d6/d8/df/d2c/d2d/la9 0 2024-04-20T11:09:38.782 INFO:tasks.workunit.client.1.smithi173.stdout:3/590: symlink d7/d17/d61/d83/d45/lbb 0 2024-04-20T11:09:38.785 INFO:tasks.workunit.client.1.smithi173.stdout:9/561: creat d5/d9/d20/d2b/fa9 x:0 0 0 2024-04-20T11:09:38.794 INFO:tasks.workunit.client.1.smithi173.stdout:8/590: creat d4/d18/d59/d6d/fba x:0 0 0 2024-04-20T11:09:38.796 INFO:tasks.workunit.client.1.smithi173.stdout:3/591: symlink d7/d17/d61/d83/lbc 0 2024-04-20T11:09:38.796 INFO:tasks.workunit.client.1.smithi173.stdout:2/637: write d2/de/f2b [1912987,79573] 0 2024-04-20T11:09:38.796 INFO:tasks.workunit.client.1.smithi173.stdout:0/554: creat d6/d8/d3c/d44/d75/fb0 x:0 0 0 2024-04-20T11:09:38.798 INFO:tasks.workunit.client.1.smithi173.stdout:9/562: read d5/f2f [769511,26755] 0 2024-04-20T11:09:38.801 INFO:tasks.workunit.client.1.smithi173.stdout:9/563: stat d5/d9/d20/d4f/d5e/d7d/d95 0 2024-04-20T11:09:38.802 INFO:tasks.workunit.client.1.smithi173.stdout:2/638: chown d2/de/f84 5175 1 2024-04-20T11:09:38.806 INFO:tasks.workunit.client.1.smithi173.stdout:2/639: chown d2/d13/d28/d4a/f6c 14 1 2024-04-20T11:09:38.807 INFO:tasks.workunit.client.1.smithi173.stdout:0/555: truncate d6/d8/df/d2c/d42/fa3 970233 0 2024-04-20T11:09:38.808 INFO:tasks.workunit.client.1.smithi173.stdout:6/588: write d4/f38 [765316,50198] 0 2024-04-20T11:09:38.821 INFO:tasks.workunit.client.1.smithi173.stdout:5/607: creat da/d34/da7/fd4 x:0 0 0 2024-04-20T11:09:38.823 INFO:tasks.workunit.client.1.smithi173.stdout:2/640: unlink d2/f3e 0 2024-04-20T11:09:38.824 INFO:tasks.workunit.client.1.smithi173.stdout:9/564: creat d5/d9/d20/d2b/d59/d44/d78/d9d/faa x:0 0 0 2024-04-20T11:09:38.825 INFO:tasks.workunit.client.1.smithi173.stdout:2/641: stat d2/d13/d28/d4a/f6c 0 2024-04-20T11:09:38.825 INFO:tasks.workunit.client.1.smithi173.stdout:9/565: chown f0 508 1 2024-04-20T11:09:38.827 INFO:tasks.workunit.client.1.smithi173.stdout:6/589: creat d4/d18/fb9 x:0 0 0 2024-04-20T11:09:38.829 INFO:tasks.workunit.client.1.smithi173.stdout:0/556: dread d6/d8/df/d2c/d42/f84 [0,4194304] 0 2024-04-20T11:09:38.831 INFO:tasks.workunit.client.1.smithi173.stdout:5/608: mkdir da/d4f/d65/dd5 0 2024-04-20T11:09:38.835 INFO:tasks.workunit.client.1.smithi173.stdout:6/590: creat d4/d18/fba x:0 0 0 2024-04-20T11:09:38.835 INFO:tasks.workunit.client.1.smithi173.stdout:6/591: chown d4/d24/f2e 7 1 2024-04-20T11:09:38.836 INFO:tasks.workunit.client.1.smithi173.stdout:7/618: sync 2024-04-20T11:09:38.836 INFO:tasks.workunit.client.1.smithi173.stdout:8/591: creat d4/fbb x:0 0 0 2024-04-20T11:09:38.838 INFO:tasks.workunit.client.1.smithi173.stdout:8/592: write d4/d18/d59/d6d/f84 [2720058,118874] 0 2024-04-20T11:09:38.839 INFO:tasks.workunit.client.1.smithi173.stdout:3/592: rename d7/f15 to d7/d2b/fbd 0 2024-04-20T11:09:38.839 INFO:tasks.workunit.client.1.smithi173.stdout:8/593: readlink l3 0 2024-04-20T11:09:38.841 INFO:tasks.workunit.client.1.smithi173.stdout:9/566: symlink d5/d9/d20/d29/lab 0 2024-04-20T11:09:38.847 INFO:tasks.workunit.client.1.smithi173.stdout:2/642: creat d2/fd8 x:0 0 0 2024-04-20T11:09:38.849 INFO:tasks.workunit.client.1.smithi173.stdout:3/593: creat d7/d9/d30/fbe x:0 0 0 2024-04-20T11:09:38.858 INFO:tasks.workunit.client.1.smithi173.stdout:3/594: creat d7/d17/d61/d83/fbf x:0 0 0 2024-04-20T11:09:38.860 INFO:tasks.workunit.client.1.smithi173.stdout:6/592: write d4/d59/f5e [379933,78857] 0 2024-04-20T11:09:38.863 INFO:tasks.workunit.client.1.smithi173.stdout:0/557: getdents d6/d8 0 2024-04-20T11:09:38.863 INFO:tasks.workunit.client.1.smithi173.stdout:5/609: rename f9 to da/d34/fd6 0 2024-04-20T11:09:38.865 INFO:tasks.workunit.client.1.smithi173.stdout:8/594: truncate d4/f10 4188968 0 2024-04-20T11:09:38.866 INFO:tasks.workunit.client.1.smithi173.stdout:0/558: fsync d6/d8/df/d2c/d42/d50/f76 0 2024-04-20T11:09:38.867 INFO:tasks.workunit.client.1.smithi173.stdout:9/567: write d5/d4c/d79/f7a [167093,67743] 0 2024-04-20T11:09:38.869 INFO:tasks.workunit.client.1.smithi173.stdout:0/559: readlink d6/d8/df/d2c/d2d/la9 0 2024-04-20T11:09:38.869 INFO:tasks.workunit.client.1.smithi173.stdout:7/619: rename d9/da/d17/l18 to d9/da/d2e/le1 0 2024-04-20T11:09:38.870 INFO:tasks.workunit.client.1.smithi173.stdout:9/568: stat d5/d9/d20/d2b/d31 0 2024-04-20T11:09:38.870 INFO:tasks.workunit.client.1.smithi173.stdout:2/643: dwrite d2/f64 [0,4194304] 0 2024-04-20T11:09:38.873 INFO:tasks.workunit.client.1.smithi173.stdout:2/644: write d2/d13/d46/f9c [459889,74000] 0 2024-04-20T11:09:38.874 INFO:tasks.workunit.client.1.smithi173.stdout:0/560: truncate d6/d8/d3c/d44/d75/f7b 5220572 0 2024-04-20T11:09:38.875 INFO:tasks.workunit.client.1.smithi173.stdout:6/593: link d4/d24/d42/d74/d3b/f52 d4/d24/d3e/fbb 0 2024-04-20T11:09:38.876 INFO:tasks.workunit.client.1.smithi173.stdout:2/645: chown d2/d13/d46/d53/lac 15 1 2024-04-20T11:09:38.878 INFO:tasks.workunit.client.1.smithi173.stdout:2/646: write d2/de/d11/d9d/d87/f97 [3614254,78715] 0 2024-04-20T11:09:38.882 INFO:tasks.workunit.client.1.smithi173.stdout:2/647: write d2/d34/fc8 [209826,74860] 0 2024-04-20T11:09:38.886 INFO:tasks.workunit.client.1.smithi173.stdout:8/595: dwrite d4/d18/d59/d5a/f67 [0,4194304] 0 2024-04-20T11:09:38.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:38 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:38.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:38 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:38.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:38 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:09:38.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:38 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:09:38.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:38 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:09:38.890 INFO:tasks.workunit.client.1.smithi173.stdout:2/648: dread d2/d13/d46/d53/fce [0,4194304] 0 2024-04-20T11:09:38.890 INFO:tasks.workunit.client.1.smithi173.stdout:5/610: write da/d4f/d65/d8e/f8d [263631,6847] 0 2024-04-20T11:09:38.892 INFO:tasks.workunit.client.1.smithi173.stdout:2/649: stat d2/d13/d9a 0 2024-04-20T11:09:38.894 INFO:tasks.workunit.client.1.smithi173.stdout:7/620: chown d9/d1f/d7f/dce/dad/fc4 3914 1 2024-04-20T11:09:38.896 INFO:tasks.workunit.client.1.smithi173.stdout:3/595: link d7/d9/lc d7/d2b/lc0 0 2024-04-20T11:09:38.897 INFO:tasks.workunit.client.1.smithi173.stdout:9/569: symlink d5/d9/d20/d4f/lac 0 2024-04-20T11:09:38.898 INFO:tasks.workunit.client.1.smithi173.stdout:6/594: creat d4/d14/d79/fbc x:0 0 0 2024-04-20T11:09:38.900 INFO:tasks.workunit.client.1.smithi173.stdout:8/596: rename d4/d18/d59/d6d/db8/f97 to d4/d18/d59/d5a/da8/db9/fbc 0 2024-04-20T11:09:38.902 INFO:tasks.workunit.client.1.smithi173.stdout:5/611: read da/de/d19/dca/f54 [1343121,4158] 0 2024-04-20T11:09:38.903 INFO:tasks.workunit.client.1.smithi173.stdout:8/597: chown d4/d18/d59/d6d/db8/d9b 518 1 2024-04-20T11:09:38.903 INFO:tasks.workunit.client.1.smithi173.stdout:2/650: dread d2/d13/d28/d4a/f6c [0,4194304] 0 2024-04-20T11:09:38.907 INFO:tasks.workunit.client.1.smithi173.stdout:6/595: fdatasync d4/f27 0 2024-04-20T11:09:38.907 INFO:tasks.workunit.client.1.smithi173.stdout:9/570: readlink d5/d9/d20/d4f/l5d 0 2024-04-20T11:09:38.909 INFO:tasks.workunit.client.1.smithi173.stdout:5/612: mknod da/d34/da7/cd7 0 2024-04-20T11:09:38.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:38 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:38.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:38 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:38 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:09:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:38 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:09:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:38 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:09:38.912 INFO:tasks.workunit.client.1.smithi173.stdout:2/651: write d2/d34/f45 [8842318,128692] 0 2024-04-20T11:09:38.912 INFO:tasks.workunit.client.1.smithi173.stdout:0/561: dwrite d6/d8/df/d2c/d42/d50/d86/f94 [0,4194304] 0 2024-04-20T11:09:38.913 INFO:tasks.workunit.client.1.smithi173.stdout:9/571: mknod d5/d9/d20/d2b/d31/cad 0 2024-04-20T11:09:38.913 INFO:tasks.workunit.client.1.smithi173.stdout:8/598: read d4/d18/d4f/f7d [6824764,56007] 0 2024-04-20T11:09:38.917 INFO:tasks.workunit.client.1.smithi173.stdout:6/596: creat d4/d24/d42/fbd x:0 0 0 2024-04-20T11:09:38.918 INFO:tasks.workunit.client.1.smithi173.stdout:8/599: dread - d4/d18/d59/d6d/db8/d3c/fa0 zero size 2024-04-20T11:09:38.919 INFO:tasks.workunit.client.1.smithi173.stdout:7/621: link d9/d33/f6b d9/d33/d35/fe2 0 2024-04-20T11:09:38.921 INFO:tasks.workunit.client.1.smithi173.stdout:9/572: write d5/d9/d20/d2b/d59/d44/d78/d9d/faa [16735,61002] 0 2024-04-20T11:09:38.921 INFO:tasks.workunit.client.1.smithi173.stdout:0/562: creat d6/d8/df/d2c/d42/d50/fb1 x:0 0 0 2024-04-20T11:09:38.921 INFO:tasks.workunit.client.1.smithi173.stdout:2/652: symlink d2/d34/d81/dca/ld9 0 2024-04-20T11:09:38.923 INFO:tasks.workunit.client.1.smithi173.stdout:2/653: read d2/fc5 [223283,3636] 0 2024-04-20T11:09:38.924 INFO:tasks.workunit.client.1.smithi173.stdout:7/622: mknod d9/da/d2e/ce3 0 2024-04-20T11:09:38.927 INFO:tasks.workunit.client.1.smithi173.stdout:0/563: write d6/d8/df/d43/d7f/f9a [845041,8765] 0 2024-04-20T11:09:38.929 INFO:tasks.workunit.client.1.smithi173.stdout:0/564: chown d6/d8/df/d15 1339697 1 2024-04-20T11:09:38.932 INFO:tasks.workunit.client.1.smithi173.stdout:0/565: readlink d6/d8/df/d43/l78 0 2024-04-20T11:09:38.933 INFO:tasks.workunit.client.1.smithi173.stdout:1/549: sync 2024-04-20T11:09:38.933 INFO:tasks.workunit.client.1.smithi173.stdout:2/654: creat d2/d13/d46/d4b/fda x:0 0 0 2024-04-20T11:09:38.933 INFO:tasks.workunit.client.1.smithi173.stdout:3/596: dread d7/d17/d61/d83/fb8 [4194304,4194304] 0 2024-04-20T11:09:38.933 INFO:tasks.workunit.client.1.smithi173.stdout:8/600: symlink d4/d18/d59/d6d/db8/lbd 0 2024-04-20T11:09:38.934 INFO:tasks.workunit.client.1.smithi173.stdout:9/573: mknod d5/d9/d20/d4f/cae 0 2024-04-20T11:09:38.939 INFO:tasks.workunit.client.1.smithi173.stdout:3/597: truncate d7/d2b/f3d 5028567 0 2024-04-20T11:09:38.941 INFO:tasks.workunit.client.1.smithi173.stdout:6/597: dwrite d4/d14/d5f/fb4 [0,4194304] 0 2024-04-20T11:09:38.943 INFO:tasks.workunit.client.1.smithi173.stdout:2/655: creat d2/d13/d46/dd0/fdb x:0 0 0 2024-04-20T11:09:38.944 INFO:tasks.workunit.client.1.smithi173.stdout:0/566: mknod d6/d8/df/d2c/d42/d50/d86/cb2 0 2024-04-20T11:09:38.945 INFO:tasks.workunit.client.1.smithi173.stdout:3/598: write d7/d17/d61/d83/f79 [1182077,52139] 0 2024-04-20T11:09:38.952 INFO:tasks.workunit.client.1.smithi173.stdout:8/601: rename d4/d18/d4f/d5b/c94 to d4/d18/d59/d5a/d7b/d8c/cbe 0 2024-04-20T11:09:38.954 INFO:tasks.workunit.client.1.smithi173.stdout:7/623: dwrite d9/d33/d35/f44 [4194304,4194304] 0 2024-04-20T11:09:38.957 INFO:tasks.workunit.client.1.smithi173.stdout:5/613: truncate da/de/d19/dca/f6e 2615138 0 2024-04-20T11:09:38.957 INFO:tasks.workunit.client.1.smithi173.stdout:8/602: write d4/d18/d4f/f7e [428062,1513] 0 2024-04-20T11:09:38.961 INFO:tasks.workunit.client.1.smithi173.stdout:5/614: write da/d34/da7/fd4 [672656,19164] 0 2024-04-20T11:09:38.961 INFO:tasks.workunit.client.1.smithi173.stdout:9/574: creat d5/faf x:0 0 0 2024-04-20T11:09:38.963 INFO:tasks.workunit.client.1.smithi173.stdout:7/624: write d9/d33/d35/d54/fc8 [702975,115389] 0 2024-04-20T11:09:38.976 INFO:tasks.workunit.client.1.smithi173.stdout:1/550: dread d5/d6/d17/d42/da/d20/f87 [0,4194304] 0 2024-04-20T11:09:38.978 INFO:tasks.workunit.client.1.smithi173.stdout:3/599: rename d7/d17/d61/f84 to d7/d53/fc1 0 2024-04-20T11:09:38.979 INFO:tasks.workunit.client.1.smithi173.stdout:5/615: read da/de/d75/d3e/faa [1104893,12037] 0 2024-04-20T11:09:38.981 INFO:tasks.workunit.client.1.smithi173.stdout:8/603: unlink d4/d18/d59/d5a/d7b/d8c/cbe 0 2024-04-20T11:09:38.981 INFO:tasks.workunit.client.1.smithi173.stdout:2/656: dwrite d2/d34/d58/f69 [0,4194304] 0 2024-04-20T11:09:38.982 INFO:tasks.workunit.client.1.smithi173.stdout:6/598: dread d4/d24/d42/d74/f53 [0,4194304] 0 2024-04-20T11:09:38.985 INFO:tasks.workunit.client.1.smithi173.stdout:2/657: dread - d2/fd8 zero size 2024-04-20T11:09:38.985 INFO:tasks.workunit.client.1.smithi173.stdout:4/684: sync 2024-04-20T11:09:38.986 INFO:tasks.workunit.client.1.smithi173.stdout:3/600: mkdir d7/d17/d61/d83/dc2 0 2024-04-20T11:09:38.986 INFO:tasks.workunit.client.1.smithi173.stdout:1/551: dread - d5/d6/d17/d42/d12/d1c/f7d zero size 2024-04-20T11:09:38.987 INFO:tasks.workunit.client.1.smithi173.stdout:8/604: creat d4/d18/da6/fbf x:0 0 0 2024-04-20T11:09:38.989 INFO:tasks.workunit.client.1.smithi173.stdout:4/685: dread - da/d1e/fb2 zero size 2024-04-20T11:09:38.994 INFO:tasks.workunit.client.1.smithi173.stdout:4/686: write da/dc/d13/f32 [7233294,39212] 0 2024-04-20T11:09:38.994 INFO:tasks.workunit.client.1.smithi173.stdout:5/616: write da/d9e/fa0 [2056081,39427] 0 2024-04-20T11:09:39.000 INFO:tasks.workunit.client.1.smithi173.stdout:1/552: mkdir d5/d6/dc/d4a/d94/dbb 0 2024-04-20T11:09:39.002 INFO:tasks.workunit.client.1.smithi173.stdout:4/687: write da/d1a/d60/fa4 [867330,14547] 0 2024-04-20T11:09:39.005 INFO:tasks.workunit.client.1.smithi173.stdout:0/567: dread d6/d8/df/d43/f59 [0,4194304] 0 2024-04-20T11:09:39.006 INFO:tasks.workunit.client.1.smithi173.stdout:2/658: creat d2/de/d11/db8/fdc x:0 0 0 2024-04-20T11:09:39.007 INFO:tasks.workunit.client.1.smithi173.stdout:3/601: creat d7/d17/d61/fc3 x:0 0 0 2024-04-20T11:09:39.010 INFO:tasks.workunit.client.1.smithi173.stdout:9/575: dwrite d5/f2f [0,4194304] 0 2024-04-20T11:09:39.010 INFO:tasks.workunit.client.1.smithi173.stdout:6/599: write d4/d14/d5f/f76 [1077918,32039] 0 2024-04-20T11:09:39.013 INFO:tasks.workunit.client.1.smithi173.stdout:7/625: dwrite d9/d1f/d7f/dce/f70 [0,4194304] 0 2024-04-20T11:09:39.014 INFO:tasks.workunit.client.1.smithi173.stdout:2/659: stat d2/d13/d28/d4a/c66 0 2024-04-20T11:09:39.020 INFO:tasks.workunit.client.1.smithi173.stdout:6/600: chown d4/d24/d42/d74/f53 1419 1 2024-04-20T11:09:39.020 INFO:tasks.workunit.client.1.smithi173.stdout:5/617: write da/f29 [203279,61574] 0 2024-04-20T11:09:39.021 INFO:tasks.workunit.client.1.smithi173.stdout:1/553: truncate d5/d6/d17/d42/d12/d1c/d85/f9e 88165 0 2024-04-20T11:09:39.021 INFO:tasks.workunit.client.1.smithi173.stdout:8/605: dwrite d4/d18/d4f/d5b/f87 [0,4194304] 0 2024-04-20T11:09:39.023 INFO:tasks.workunit.client.1.smithi173.stdout:5/618: chown da/de/d19/d32/d72 0 1 2024-04-20T11:09:39.024 INFO:tasks.workunit.client.1.smithi173.stdout:0/568: creat d6/d8/df/da0/fb3 x:0 0 0 2024-04-20T11:09:39.024 INFO:tasks.workunit.client.1.smithi173.stdout:4/688: mknod da/dd1/ce3 0 2024-04-20T11:09:39.027 INFO:tasks.workunit.client.1.smithi173.stdout:4/689: chown da/dc/d62/d84/l99 12145411 1 2024-04-20T11:09:39.028 INFO:tasks.workunit.client.1.smithi173.stdout:6/601: truncate d4/d1c/d8e/d90/fa6 164998 0 2024-04-20T11:09:39.030 INFO:tasks.workunit.client.1.smithi173.stdout:9/576: creat d5/d9/d20/d2b/d59/da1/fb0 x:0 0 0 2024-04-20T11:09:39.034 INFO:tasks.workunit.client.1.smithi173.stdout:2/660: fsync d2/d13/d46/d53/d4e/f59 0 2024-04-20T11:09:39.034 INFO:tasks.workunit.client.1.smithi173.stdout:3/602: truncate d7/d53/f8f 1899319 0 2024-04-20T11:09:39.035 INFO:tasks.workunit.client.1.smithi173.stdout:0/569: mkdir d6/d5f/d8f/db4 0 2024-04-20T11:09:39.036 INFO:tasks.workunit.client.1.smithi173.stdout:2/661: fsync d2/d34/d81/f89 0 2024-04-20T11:09:39.042 INFO:tasks.workunit.client.1.smithi173.stdout:6/602: creat d4/d18/d7e/fbe x:0 0 0 2024-04-20T11:09:39.043 INFO:tasks.workunit.client.1.smithi173.stdout:9/577: creat d5/d9/d20/d2b/fb1 x:0 0 0 2024-04-20T11:09:39.047 INFO:tasks.workunit.client.1.smithi173.stdout:4/690: mknod da/d1a/d60/d66/ce4 0 2024-04-20T11:09:39.047 INFO:tasks.workunit.client.1.smithi173.stdout:3/603: fsync d7/d17/f98 0 2024-04-20T11:09:39.051 INFO:tasks.workunit.client.1.smithi173.stdout:6/603: creat d4/d59/d8f/fbf x:0 0 0 2024-04-20T11:09:39.054 INFO:tasks.workunit.client.1.smithi173.stdout:3/604: write d7/d53/f71 [486721,61286] 0 2024-04-20T11:09:39.054 INFO:tasks.workunit.client.1.smithi173.stdout:1/554: dwrite d5/d6/d17/d42/da/df/d40/f4c [4194304,4194304] 0 2024-04-20T11:09:39.055 INFO:tasks.workunit.client.1.smithi173.stdout:8/606: dwrite d4/d18/d59/d6d/db8/d3c/f52 [4194304,4194304] 0 2024-04-20T11:09:39.055 INFO:tasks.workunit.client.1.smithi173.stdout:0/570: link d6/d8/df/d2c/d42/d50/d86/d8b/f8c d6/d8/df/d2c/d42/d50/fb5 0 2024-04-20T11:09:39.056 INFO:tasks.workunit.client.1.smithi173.stdout:6/604: dread - d4/d14/f8c zero size 2024-04-20T11:09:39.061 INFO:tasks.workunit.client.1.smithi173.stdout:0/571: chown d6/d8 463747271 1 2024-04-20T11:09:39.064 INFO:tasks.workunit.client.1.smithi173.stdout:7/626: truncate d9/da/f12 486926 0 2024-04-20T11:09:39.064 INFO:tasks.workunit.client.1.smithi173.stdout:8/607: fdatasync d4/d18/d59/d5a/da8/fae 0 2024-04-20T11:09:39.066 INFO:tasks.workunit.client.1.smithi173.stdout:6/605: stat d4/d59/f5e 0 2024-04-20T11:09:39.068 INFO:tasks.workunit.client.1.smithi173.stdout:7/627: readlink d9/d33/d35/l5d 0 2024-04-20T11:09:39.068 INFO:tasks.workunit.client.1.smithi173.stdout:3/605: mkdir d7/d17/d58/d8a/dc4 0 2024-04-20T11:09:39.070 INFO:tasks.workunit.client.1.smithi173.stdout:2/662: dwrite d2/d13/d46/f85 [0,4194304] 0 2024-04-20T11:09:39.071 INFO:tasks.workunit.client.1.smithi173.stdout:4/691: rename da/d48/d53/d39/l92 to da/d1a/d60/d66/d9b/le5 0 2024-04-20T11:09:39.073 INFO:tasks.workunit.client.1.smithi173.stdout:6/606: write d4/d18/f66 [3859661,124378] 0 2024-04-20T11:09:39.078 INFO:tasks.workunit.client.1.smithi173.stdout:0/572: symlink d6/d8/d3c/d44/d69/lb6 0 2024-04-20T11:09:39.079 INFO:tasks.workunit.client.1.smithi173.stdout:8/608: mknod d4/cc0 0 2024-04-20T11:09:39.082 INFO:tasks.workunit.client.1.smithi173.stdout:2/663: chown d2/d13/d46/d53/fbd 7305 1 2024-04-20T11:09:39.082 INFO:tasks.workunit.client.1.smithi173.stdout:3/606: symlink d7/d85/lc5 0 2024-04-20T11:09:39.091 INFO:tasks.workunit.client.1.smithi173.stdout:9/578: dwrite d5/d9/d20/d29/d2d/f43 [0,4194304] 0 2024-04-20T11:09:39.094 INFO:tasks.workunit.client.1.smithi173.stdout:2/664: creat d2/d34/fdd x:0 0 0 2024-04-20T11:09:39.094 INFO:tasks.workunit.client.1.smithi173.stdout:1/555: dwrite d5/d6/d17/d42/d12/d1c/f7d [0,4194304] 0 2024-04-20T11:09:39.094 INFO:tasks.workunit.client.1.smithi173.stdout:5/619: dwrite da/de/f24 [0,4194304] 0 2024-04-20T11:09:39.098 INFO:tasks.workunit.client.1.smithi173.stdout:3/607: creat d7/d9/d5c/fc6 x:0 0 0 2024-04-20T11:09:39.099 INFO:tasks.workunit.client.1.smithi173.stdout:3/608: dread - d7/d17/d61/f78 zero size 2024-04-20T11:09:39.101 INFO:tasks.workunit.client.1.smithi173.stdout:0/573: dread d6/d8/df/d2c/d2d/f6a [0,4194304] 0 2024-04-20T11:09:39.102 INFO:tasks.workunit.client.1.smithi173.stdout:0/574: dread - d6/fa4 zero size 2024-04-20T11:09:39.104 INFO:tasks.workunit.client.1.smithi173.stdout:2/665: write d2/fa9 [542152,123992] 0 2024-04-20T11:09:39.115 INFO:tasks.workunit.client.1.smithi173.stdout:3/609: creat d7/d9/d30/fc7 x:0 0 0 2024-04-20T11:09:39.118 INFO:tasks.workunit.client.1.smithi173.stdout:2/666: symlink d2/d13/lde 0 2024-04-20T11:09:39.119 INFO:tasks.workunit.client.1.smithi173.stdout:2/667: fsync d2/de/f2b 0 2024-04-20T11:09:39.120 INFO:tasks.workunit.client.1.smithi173.stdout:1/556: link d5/d6/dc/f3a d5/d6/d17/d42/d12/d1f/fbc 0 2024-04-20T11:09:39.121 INFO:tasks.workunit.client.1.smithi173.stdout:2/668: write d2/fa9 [1072289,105959] 0 2024-04-20T11:09:39.122 INFO:tasks.workunit.client.1.smithi173.stdout:8/609: rename d4/d18/d59/d6d/c76 to d4/d18/d59/d6d/db8/d3c/cc1 0 2024-04-20T11:09:39.123 INFO:tasks.workunit.client.1.smithi173.stdout:0/575: symlink d6/d8/df/d2c/lb7 0 2024-04-20T11:09:39.124 INFO:tasks.workunit.client.1.smithi173.stdout:2/669: chown d2/d13/d46/d4b/l55 129143933 1 2024-04-20T11:09:39.124 INFO:tasks.workunit.client.1.smithi173.stdout:7/628: truncate d9/d1f/d7f/dce/f70 896963 0 2024-04-20T11:09:39.126 INFO:tasks.workunit.client.1.smithi173.stdout:4/692: truncate da/dc/d15/f1d 2109796 0 2024-04-20T11:09:39.128 INFO:tasks.workunit.client.1.smithi173.stdout:1/557: write d5/d6/d17/d42/d12/f44 [284055,11442] 0 2024-04-20T11:09:39.131 INFO:tasks.workunit.client.1.smithi173.stdout:9/579: rename d5/d9/d20/d29/d2d/d33/c36 to d5/d9/d20/d4f/d5e/cb2 0 2024-04-20T11:09:39.136 INFO:tasks.workunit.client.1.smithi173.stdout:6/607: dwrite d4/d24/d3e/f8d [0,4194304] 0 2024-04-20T11:09:39.138 INFO:tasks.workunit.client.1.smithi173.stdout:2/670: dread d2/d34/d81/f9e [0,4194304] 0 2024-04-20T11:09:39.138 INFO:tasks.workunit.client.1.smithi173.stdout:5/620: write da/de/d75/d40/d59/f96 [516282,44444] 0 2024-04-20T11:09:39.138 INFO:tasks.workunit.client.1.smithi173.stdout:3/610: rename d7/d17/f93 to d7/d17/d61/d83/fc8 0 2024-04-20T11:09:39.144 INFO:tasks.workunit.client.1.smithi173.stdout:9/580: write d5/d9/d20/d29/d18/f70 [1101683,99624] 0 2024-04-20T11:09:39.147 INFO:tasks.workunit.client.1.smithi173.stdout:6/608: mknod d4/d18/d61/cc0 0 2024-04-20T11:09:39.149 INFO:tasks.workunit.client.1.smithi173.stdout:2/671: write d2/de/f21 [150599,82193] 0 2024-04-20T11:09:39.156 INFO:tasks.workunit.client.1.smithi173.stdout:7/629: getdents d9/d33/d35/d4a/d52 0 2024-04-20T11:09:39.159 INFO:tasks.workunit.client.1.smithi173.stdout:5/621: mknod da/de/d19/d32/cd8 0 2024-04-20T11:09:39.159 INFO:tasks.workunit.client.1.smithi173.stdout:6/609: dread - d4/d59/f84 zero size 2024-04-20T11:09:39.161 INFO:tasks.workunit.client.1.smithi173.stdout:2/672: rename d2/d13/d46/dd0/fdb to d2/d13/d9a/fdf 0 2024-04-20T11:09:39.164 INFO:tasks.workunit.client.1.smithi173.stdout:9/581: chown d5/d9/d20/d2b/d59/d44/d78/d82/f88 0 1 2024-04-20T11:09:39.164 INFO:tasks.workunit.client.1.smithi173.stdout:0/576: dwrite d6/d8/d3c/d44/d75/fb0 [0,4194304] 0 2024-04-20T11:09:39.167 INFO:tasks.workunit.client.1.smithi173.stdout:7/630: symlink d9/d1f/d7f/d8d/le4 0 2024-04-20T11:09:39.168 INFO:tasks.workunit.client.1.smithi173.stdout:4/693: dwrite da/d1a/d60/fb1 [0,4194304] 0 2024-04-20T11:09:39.172 INFO:tasks.workunit.client.1.smithi173.stdout:7/631: readlink d9/d1f/d72/d8f/ld4 0 2024-04-20T11:09:39.176 INFO:tasks.workunit.client.1.smithi173.stdout:8/610: dread d4/d18/d59/d6d/f84 [0,4194304] 0 2024-04-20T11:09:39.180 INFO:tasks.workunit.client.1.smithi173.stdout:0/577: mkdir d6/d5f/db8 0 2024-04-20T11:09:39.186 INFO:tasks.workunit.client.1.smithi173.stdout:5/622: write da/de/f37 [3598594,102692] 0 2024-04-20T11:09:39.186 INFO:tasks.workunit.client.1.smithi173.stdout:4/694: symlink da/d1a/d60/d66/le6 0 2024-04-20T11:09:39.193 INFO:tasks.workunit.client.1.smithi173.stdout:7/632: rename d9/d33/d91/d99/cd3 to d9/d33/d35/d54/ce5 0 2024-04-20T11:09:39.194 INFO:tasks.workunit.client.1.smithi173.stdout:4/695: dread - da/d1e/fb2 zero size 2024-04-20T11:09:39.196 INFO:tasks.workunit.client.1.smithi173.stdout:3/611: dwrite d7/d17/d61/fc3 [0,4194304] 0 2024-04-20T11:09:39.202 INFO:tasks.workunit.client.1.smithi173.stdout:1/558: dread d5/d6/d17/d42/d12/d1c/f8b [0,4194304] 0 2024-04-20T11:09:39.202 INFO:tasks.workunit.client.1.smithi173.stdout:6/610: dwrite d4/d18/d7e/fbe [0,4194304] 0 2024-04-20T11:09:39.209 INFO:tasks.workunit.client.1.smithi173.stdout:7/633: write d9/d1f/d72/d8f/fd1 [1465058,43704] 0 2024-04-20T11:09:39.214 INFO:tasks.workunit.client.1.smithi173.stdout:9/582: dwrite d5/d9/d20/d4f/d5e/d9e/fa6 [0,4194304] 0 2024-04-20T11:09:39.214 INFO:tasks.workunit.client.1.smithi173.stdout:2/673: dread d2/d13/f50 [0,4194304] 0 2024-04-20T11:09:39.217 INFO:tasks.workunit.client.1.smithi173.stdout:5/623: write da/de/d19/d2c/f53 [4722235,14921] 0 2024-04-20T11:09:39.219 INFO:tasks.workunit.client.1.smithi173.stdout:2/674: fdatasync d2/de/fd4 0 2024-04-20T11:09:39.220 INFO:tasks.workunit.client.1.smithi173.stdout:8/611: rename d4/d18/d59/d6d/db8/l89 to d4/d18/d59/d6d/db8/lc2 0 2024-04-20T11:09:39.220 INFO:tasks.workunit.client.1.smithi173.stdout:9/583: stat d5/d9/f6e 0 2024-04-20T11:09:39.222 INFO:tasks.workunit.client.1.smithi173.stdout:0/578: mkdir d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/db9 0 2024-04-20T11:09:39.223 INFO:tasks.workunit.client.1.smithi173.stdout:8/612: write d4/d18/d4f/f9a [206618,26185] 0 2024-04-20T11:09:39.225 INFO:tasks.workunit.client.1.smithi173.stdout:7/634: creat d9/d33/d81/fe6 x:0 0 0 2024-04-20T11:09:39.228 INFO:tasks.workunit.client.1.smithi173.stdout:6/611: dread d4/f60 [0,4194304] 0 2024-04-20T11:09:39.228 INFO:tasks.workunit.client.1.smithi173.stdout:5/624: rmdir da/de/d19/d32 39 2024-04-20T11:09:39.233 INFO:tasks.workunit.client.1.smithi173.stdout:4/696: dwrite da/dc/d13/f51 [4194304,4194304] 0 2024-04-20T11:09:39.233 INFO:tasks.workunit.client.1.smithi173.stdout:3/612: link d7/d17/d61/d83/fbf d7/d9/d5c/fc9 0 2024-04-20T11:09:39.235 INFO:tasks.workunit.client.1.smithi173.stdout:2/675: rename d2/cd6 to d2/d34/d81/ce0 0 2024-04-20T11:09:39.238 INFO:tasks.workunit.client.1.smithi173.stdout:8/613: creat d4/d18/d4f/d5b/fc3 x:0 0 0 2024-04-20T11:09:39.244 INFO:tasks.workunit.client.1.smithi173.stdout:9/584: write d5/d9/d20/f40 [384302,3551] 0 2024-04-20T11:09:39.286 INFO:tasks.workunit.client.1.smithi173.stdout:4/697: creat da/dc/ddd/fe7 x:0 0 0 2024-04-20T11:09:39.286 INFO:tasks.workunit.client.1.smithi173.stdout:6/612: creat d4/d59/fc1 x:0 0 0 2024-04-20T11:09:39.288 INFO:tasks.workunit.client.1.smithi173.stdout:5/625: link da/de/d19/dca/c3f da/d4f/d5e/dbc/cd9 0 2024-04-20T11:09:39.292 INFO:tasks.workunit.client.1.smithi173.stdout:8/614: unlink d4/c35 0 2024-04-20T11:09:39.293 INFO:tasks.workunit.client.1.smithi173.stdout:1/559: dwrite d5/d6/d17/d42/f39 [0,4194304] 0 2024-04-20T11:09:39.297 INFO:tasks.workunit.client.1.smithi173.stdout:8/615: write d4/d18/d4f/f7e [616766,82318] 0 2024-04-20T11:09:39.297 INFO:tasks.workunit.client.1.smithi173.stdout:9/585: creat d5/d9/d20/d29/d18/d9b/fb3 x:0 0 0 2024-04-20T11:09:39.298 INFO:tasks.workunit.client.1.smithi173.stdout:0/579: dread d6/f13 [0,4194304] 0 2024-04-20T11:09:39.301 INFO:tasks.workunit.client.1.smithi173.stdout:5/626: dread - da/de/d19/d32/d72/f91 zero size 2024-04-20T11:09:39.302 INFO:tasks.workunit.client.1.smithi173.stdout:4/698: mknod da/d1a/d60/d66/ce8 0 2024-04-20T11:09:39.305 INFO:tasks.workunit.client.1.smithi173.stdout:0/580: creat d6/d54/fba x:0 0 0 2024-04-20T11:09:39.305 INFO:tasks.workunit.client.1.smithi173.stdout:2/676: dwrite d2/d13/f74 [4194304,4194304] 0 2024-04-20T11:09:39.306 INFO:tasks.workunit.client.1.smithi173.stdout:6/613: link d4/d1c/d8e/d90/l96 d4/d1c/d8e/d90/lc2 0 2024-04-20T11:09:39.309 INFO:tasks.workunit.client.1.smithi173.stdout:1/560: creat d5/d6/d17/d42/da/df/d40/dac/fbd x:0 0 0 2024-04-20T11:09:39.311 INFO:tasks.workunit.client.1.smithi173.stdout:8/616: rename d4/c78 to d4/d18/d59/d6d/db8/d3c/d86/cc4 0 2024-04-20T11:09:39.314 INFO:tasks.workunit.client.1.smithi173.stdout:2/677: chown d2/d34/d81/dca/ld9 3755829 1 2024-04-20T11:09:39.316 INFO:tasks.workunit.client.1.smithi173.stdout:8/617: chown d4/d18/d59/d6d/db8 599303 1 2024-04-20T11:09:39.316 INFO:tasks.workunit.client.1.smithi173.stdout:6/614: write d4/d14/d5f/f76 [25360,49556] 0 2024-04-20T11:09:39.316 INFO:tasks.workunit.client.1.smithi173.stdout:5/627: truncate da/de/d19/dca/f6e 939694 0 2024-04-20T11:09:39.319 INFO:tasks.workunit.client.1.smithi173.stdout:5/628: write da/de/f2d [8339932,26245] 0 2024-04-20T11:09:39.322 INFO:tasks.workunit.client.1.smithi173.stdout:0/581: getdents d6/d8/d51 0 2024-04-20T11:09:39.325 INFO:tasks.workunit.client.1.smithi173.stdout:1/561: fdatasync d5/d6/d17/d42/d12/d1f/f2f 0 2024-04-20T11:09:39.325 INFO:tasks.workunit.client.1.smithi173.stdout:2/678: mknod d2/d34/d81/ce1 0 2024-04-20T11:09:39.325 INFO:tasks.workunit.client.1.smithi173.stdout:6/615: symlink d4/d18/d7e/lc3 0 2024-04-20T11:09:39.326 INFO:tasks.workunit.client.1.smithi173.stdout:5/629: write da/de/f24 [1524531,85550] 0 2024-04-20T11:09:39.328 INFO:tasks.workunit.client.1.smithi173.stdout:4/699: rename da/d1e/da6/fab to da/d48/d53/fe9 0 2024-04-20T11:09:39.331 INFO:tasks.workunit.client.1.smithi173.stdout:9/586: dwrite d5/d9/f1a [0,4194304] 0 2024-04-20T11:09:39.331 INFO:tasks.workunit.client.1.smithi173.stdout:8/618: truncate d4/d18/d59/d6d/db8/d3c/f73 7783345 0 2024-04-20T11:09:39.332 INFO:tasks.workunit.client.1.smithi173.stdout:3/613: dread d7/f22 [0,4194304] 0 2024-04-20T11:09:39.334 INFO:tasks.workunit.client.1.smithi173.stdout:4/700: dread - da/d1e/d52/d7a/f7f zero size 2024-04-20T11:09:39.346 INFO:tasks.workunit.client.1.smithi173.stdout:4/701: chown da/d48/d53/f3b 918 1 2024-04-20T11:09:39.347 INFO:tasks.workunit.client.1.smithi173.stdout:2/679: rmdir d2/d13/d28/d4a/d94 0 2024-04-20T11:09:39.347 INFO:tasks.workunit.client.1.smithi173.stdout:9/587: dread d5/d4c/d79/f7a [0,4194304] 0 2024-04-20T11:09:39.354 INFO:tasks.workunit.client.1.smithi173.stdout:6/616: rmdir d4/d24/d42/d74/d3b/dad 0 2024-04-20T11:09:39.357 INFO:tasks.workunit.client.1.smithi173.stdout:3/614: mkdir d7/d17/d61/d90/dca 0 2024-04-20T11:09:39.362 INFO:tasks.workunit.client.1.smithi173.stdout:1/562: dread d5/d6/d3b/f98 [0,4194304] 0 2024-04-20T11:09:39.364 INFO:tasks.workunit.client.1.smithi173.stdout:1/563: stat d5/d6/d17 0 2024-04-20T11:09:39.366 INFO:tasks.workunit.client.1.smithi173.stdout:2/680: dwrite d2/d34/d58/f5a [4194304,4194304] 0 2024-04-20T11:09:39.367 INFO:tasks.workunit.client.1.smithi173.stdout:4/702: dwrite da/dc/f33 [4194304,4194304] 0 2024-04-20T11:09:39.369 INFO:tasks.workunit.client.1.smithi173.stdout:8/619: dread d4/d18/f48 [0,4194304] 0 2024-04-20T11:09:39.369 INFO:tasks.workunit.client.1.smithi173.stdout:4/703: stat da/d48/d53/d39/l41 0 2024-04-20T11:09:39.371 INFO:tasks.workunit.client.1.smithi173.stdout:5/630: link da/de/d19/dca/lc9 da/d34/d7f/da5/lda 0 2024-04-20T11:09:39.372 INFO:tasks.workunit.client.1.smithi173.stdout:1/564: write d5/d6/d17/d42/d12/d1c/d85/f9e [141054,31192] 0 2024-04-20T11:09:39.373 INFO:tasks.workunit.client.1.smithi173.stdout:9/588: mknod d5/cb4 0 2024-04-20T11:09:39.375 INFO:tasks.workunit.client.1.smithi173.stdout:8/620: rmdir d4/d18/d59/d6d/db8/d3c 39 2024-04-20T11:09:39.377 INFO:tasks.workunit.client.1.smithi173.stdout:9/589: fsync d5/d9/d20/d29/d18/d9b/fb3 0 2024-04-20T11:09:39.379 INFO:tasks.workunit.client.1.smithi173.stdout:3/615: mknod d7/d2b/d9d/db7/ccb 0 2024-04-20T11:09:39.386 INFO:tasks.workunit.client.1.smithi173.stdout:4/704: creat da/d48/d53/fea x:0 0 0 2024-04-20T11:09:39.387 INFO:tasks.workunit.client.1.smithi173.stdout:5/631: symlink da/de/d19/d32/d30/ldb 0 2024-04-20T11:09:39.390 INFO:tasks.workunit.client.1.smithi173.stdout:2/681: getdents d2 0 2024-04-20T11:09:39.391 INFO:tasks.workunit.client.1.smithi173.stdout:1/565: write d5/d6/d17/d42/da/d20/d38/d52/d75/f86 [561697,130447] 0 2024-04-20T11:09:39.392 INFO:tasks.workunit.client.1.smithi173.stdout:3/616: link d7/d9/l5e d7/d17/d61/d90/lcc 0 2024-04-20T11:09:39.396 INFO:tasks.workunit.client.1.smithi173.stdout:5/632: symlink da/de/d75/d40/d59/ldc 0 2024-04-20T11:09:39.396 INFO:tasks.workunit.client.1.smithi173.stdout:2/682: read - d2/d13/d46/d53/fbd zero size 2024-04-20T11:09:39.399 INFO:tasks.workunit.client.1.smithi173.stdout:3/617: write d7/d2b/fbd [3053928,88162] 0 2024-04-20T11:09:39.402 INFO:tasks.workunit.client.1.smithi173.stdout:8/621: dwrite d4/d18/fa2 [0,4194304] 0 2024-04-20T11:09:39.403 INFO:tasks.workunit.client.1.smithi173.stdout:9/590: dwrite d5/f2f [0,4194304] 0 2024-04-20T11:09:39.408 INFO:tasks.workunit.client.1.smithi173.stdout:6/617: dread d4/d24/fa9 [0,4194304] 0 2024-04-20T11:09:39.419 INFO:tasks.workunit.client.1.smithi173.stdout:6/618: dread d4/d59/f5e [0,4194304] 0 2024-04-20T11:09:39.420 INFO:tasks.workunit.client.1.smithi173.stdout:3/618: rename d7/d2b/d8e/fa7 to d7/d2b/fcd 0 2024-04-20T11:09:39.420 INFO:tasks.workunit.client.1.smithi173.stdout:4/705: dwrite da/d48/d53/fea [0,4194304] 0 2024-04-20T11:09:39.516 INFO:tasks.workunit.client.1.smithi173.stdout:3/619: chown d7/d17/d61/d90 3549 1 2024-04-20T11:09:39.518 INFO:tasks.workunit.client.1.smithi173.stdout:8/622: rename d4/c5e to d4/d18/d59/d5a/d7b/cc5 0 2024-04-20T11:09:39.521 INFO:tasks.workunit.client.1.smithi173.stdout:4/706: dread - da/dc/d13/fc6 zero size 2024-04-20T11:09:39.525 INFO:tasks.workunit.client.1.smithi173.stdout:1/566: dwrite d5/d6/d17/d42/d12/f44 [0,4194304] 0 2024-04-20T11:09:39.528 INFO:tasks.workunit.client.1.smithi173.stdout:4/707: stat da/d1e/d52/d7a 0 2024-04-20T11:09:39.529 INFO:tasks.workunit.client.1.smithi173.stdout:9/591: dwrite d5/f6 [0,4194304] 0 2024-04-20T11:09:39.530 INFO:tasks.workunit.client.1.smithi173.stdout:2/683: dwrite d2/d13/d46/d53/fce [0,4194304] 0 2024-04-20T11:09:39.534 INFO:tasks.workunit.client.1.smithi173.stdout:8/623: readlink d4/d18/d59/d6d/db8/d3c/l4d 0 2024-04-20T11:09:39.536 INFO:tasks.workunit.client.1.smithi173.stdout:2/684: dread - d2/d13/d46/d53/fbd zero size 2024-04-20T11:09:39.536 INFO:tasks.workunit.client.1.smithi173.stdout:4/708: rmdir da 39 2024-04-20T11:09:39.540 INFO:tasks.workunit.client.1.smithi173.stdout:8/624: write d4/d18/d59/d6d/db8/f66 [883642,103501] 0 2024-04-20T11:09:39.541 INFO:tasks.workunit.client.1.smithi173.stdout:1/567: fsync d5/d6/d17/d42/d12/d1f/fbc 0 2024-04-20T11:09:39.547 INFO:tasks.workunit.client.1.smithi173.stdout:2/685: mknod d2/d13/d46/d7d/ce2 0 2024-04-20T11:09:39.551 INFO:tasks.workunit.client.1.smithi173.stdout:6/619: dwrite d4/f11 [0,4194304] 0 2024-04-20T11:09:39.557 INFO:tasks.workunit.client.1.smithi173.stdout:2/686: write d2/d13/f44 [3125096,89714] 0 2024-04-20T11:09:39.560 INFO:tasks.workunit.client.1.smithi173.stdout:6/620: rmdir d4/d59 39 2024-04-20T11:09:39.560 INFO:tasks.workunit.client.1.smithi173.stdout:1/568: chown d5/d6/d17/d42/da/df/d40/f4c 307174790 1 2024-04-20T11:09:39.561 INFO:tasks.workunit.client.1.smithi173.stdout:4/709: read da/dc/d15/f1d [1961170,47945] 0 2024-04-20T11:09:39.561 INFO:tasks.workunit.client.1.smithi173.stdout:8/625: creat d4/d18/d4f/fc6 x:0 0 0 2024-04-20T11:09:39.565 INFO:tasks.workunit.client.1.smithi173.stdout:9/592: dwrite d5/d9/f1a [0,4194304] 0 2024-04-20T11:09:39.569 INFO:tasks.workunit.client.1.smithi173.stdout:2/687: creat d2/d13/d46/fe3 x:0 0 0 2024-04-20T11:09:39.570 INFO:tasks.workunit.client.1.smithi173.stdout:4/710: unlink da/d1a/d60/fd0 0 2024-04-20T11:09:39.570 INFO:tasks.workunit.client.1.smithi173.stdout:3/620: dread d7/d17/f3c [0,4194304] 0 2024-04-20T11:09:39.571 INFO:tasks.workunit.client.1.smithi173.stdout:1/569: symlink d5/d6/d17/d42/da/df/d40/lbe 0 2024-04-20T11:09:39.572 INFO:tasks.workunit.client.1.smithi173.stdout:3/621: dread - d7/d2b/d8e/fb3 zero size 2024-04-20T11:09:39.572 INFO:tasks.workunit.client.1.smithi173.stdout:4/711: write da/dc/d13/d65/f98 [167648,81071] 0 2024-04-20T11:09:39.580 INFO:tasks.workunit.client.1.smithi173.stdout:3/622: fdatasync d7/d9/ff 0 2024-04-20T11:09:39.584 INFO:tasks.workunit.client.1.smithi173.stdout:8/626: creat d4/d18/d59/fc7 x:0 0 0 2024-04-20T11:09:39.585 INFO:tasks.workunit.client.1.smithi173.stdout:6/621: dwrite d4/f38 [0,4194304] 0 2024-04-20T11:09:39.585 INFO:tasks.workunit.client.1.smithi173.stdout:9/593: dread d5/d9/d20/d4f/d5e/d9e/fa6 [0,4194304] 0 2024-04-20T11:09:39.587 INFO:tasks.workunit.client.1.smithi173.stdout:8/627: chown d4/d18/d59/d6d/db8/lbd 197721654 1 2024-04-20T11:09:39.594 INFO:tasks.workunit.client.1.smithi173.stdout:2/688: dread d2/d13/d28/d4a/f6c [0,4194304] 0 2024-04-20T11:09:39.594 INFO:tasks.workunit.client.1.smithi173.stdout:8/628: mkdir d4/d18/d59/d6d/db8/d9b/dc8 0 2024-04-20T11:09:39.597 INFO:tasks.workunit.client.1.smithi173.stdout:3/623: getdents d7/d2b/d9d 0 2024-04-20T11:09:39.598 INFO:tasks.workunit.client.1.smithi173.stdout:1/570: write d5/d6/d17/d42/da/d20/d38/fa4 [952444,94085] 0 2024-04-20T11:09:39.598 INFO:tasks.workunit.client.1.smithi173.stdout:4/712: write da/dc/d15/d8e/fa7 [1200012,53255] 0 2024-04-20T11:09:39.609 INFO:tasks.workunit.client.1.smithi173.stdout:3/624: chown d7/d9/l7e 5 1 2024-04-20T11:09:39.610 INFO:tasks.workunit.client.1.smithi173.stdout:7/635: sync 2024-04-20T11:09:39.610 INFO:tasks.workunit.client.1.smithi173.stdout:3/625: fdatasync d7/d17/f98 0 2024-04-20T11:09:39.611 INFO:tasks.workunit.client.1.smithi173.stdout:1/571: unlink d5/d6/d17/d42/da/d20/l36 0 2024-04-20T11:09:39.616 INFO:tasks.workunit.client.1.smithi173.stdout:9/594: truncate d5/d9/d20/d4f/d5e/d7d/f86 8703338 0 2024-04-20T11:09:39.617 INFO:tasks.workunit.client.1.smithi173.stdout:2/689: write d2/d13/d46/d53/d4e/fa1 [661464,57405] 0 2024-04-20T11:09:39.619 INFO:tasks.workunit.client.1.smithi173.stdout:7/636: rename d9/d33/d35/d54/fc8 to d9/da/d2e/fe7 0 2024-04-20T11:09:39.622 INFO:tasks.workunit.client.1.smithi173.stdout:1/572: fdatasync d5/d6/d17/d42/d12/f31 0 2024-04-20T11:09:39.623 INFO:tasks.workunit.client.1.smithi173.stdout:8/629: creat d4/d18/d59/fc9 x:0 0 0 2024-04-20T11:09:39.625 INFO:tasks.workunit.client.1.smithi173.stdout:4/713: link da/d1a/d60/c70 da/d1a/d6e/db4/ceb 0 2024-04-20T11:09:39.626 INFO:tasks.workunit.client.1.smithi173.stdout:9/595: mkdir d5/d9/d20/d2b/d59/d44/d78/d82/db5 0 2024-04-20T11:09:39.626 INFO:tasks.workunit.client.1.smithi173.stdout:3/626: dwrite d7/d9/ff [0,4194304] 0 2024-04-20T11:09:39.628 INFO:tasks.workunit.client.1.smithi173.stdout:6/622: dread d4/d24/d42/d74/d3b/fb5 [0,4194304] 0 2024-04-20T11:09:39.633 INFO:tasks.workunit.client.1.smithi173.stdout:4/714: mkdir da/d1a/d60/d7d/dec 0 2024-04-20T11:09:39.638 INFO:tasks.workunit.client.1.smithi173.stdout:1/573: creat d5/d6/d17/fbf x:0 0 0 2024-04-20T11:09:39.639 INFO:tasks.workunit.client.1.smithi173.stdout:6/623: creat d4/d18/fc4 x:0 0 0 2024-04-20T11:09:39.649 INFO:tasks.workunit.client.1.smithi173.stdout:7/637: dwrite d9/d33/d35/f66 [0,4194304] 0 2024-04-20T11:09:39.649 INFO:tasks.workunit.client.1.smithi173.stdout:6/624: unlink d4/d59/f5e 0 2024-04-20T11:09:39.649 INFO:tasks.workunit.client.1.smithi173.stdout:8/630: dwrite d4/d18/d59/fc9 [0,4194304] 0 2024-04-20T11:09:39.651 INFO:tasks.workunit.client.1.smithi173.stdout:7/638: stat d9/d33/d40/d89 0 2024-04-20T11:09:39.653 INFO:tasks.workunit.client.1.smithi173.stdout:7/639: fdatasync d9/da/d17/d2b/daa/f6d 0 2024-04-20T11:09:39.657 INFO:tasks.workunit.client.1.smithi173.stdout:8/631: unlink d4/d18/d59/d5a/d7b/fb4 0 2024-04-20T11:09:39.658 INFO:tasks.workunit.client.1.smithi173.stdout:0/582: sync 2024-04-20T11:09:39.658 INFO:tasks.workunit.client.1.smithi173.stdout:5/633: sync 2024-04-20T11:09:39.660 INFO:tasks.workunit.client.1.smithi173.stdout:8/632: truncate d4/fbb 724747 0 2024-04-20T11:09:39.663 INFO:tasks.workunit.client.1.smithi173.stdout:7/640: symlink d9/d33/d85/d97/le8 0 2024-04-20T11:09:39.664 INFO:tasks.workunit.client.1.smithi173.stdout:0/583: write d6/d8/df/f1d [1341759,32794] 0 2024-04-20T11:09:39.668 INFO:tasks.workunit.client.1.smithi173.stdout:9/596: write d5/d9/f47 [307269,60948] 0 2024-04-20T11:09:39.668 INFO:tasks.workunit.client.1.smithi173.stdout:2/690: dread d2/d13/d28/d4a/f7f [4194304,4194304] 0 2024-04-20T11:09:39.673 INFO:tasks.workunit.client.1.smithi173.stdout:5/634: symlink da/d34/ldd 0 2024-04-20T11:09:39.678 INFO:tasks.workunit.client.1.smithi173.stdout:0/584: creat d6/d8/fbb x:0 0 0 2024-04-20T11:09:39.680 INFO:tasks.workunit.client.1.smithi173.stdout:3/627: dread d7/d17/d61/d83/faa [0,4194304] 0 2024-04-20T11:09:39.680 INFO:tasks.workunit.client.1.smithi173.stdout:7/641: fsync d9/da/d2e/fd2 0 2024-04-20T11:09:39.681 INFO:tasks.workunit.client.1.smithi173.stdout:5/635: chown da/de/d75/c26 13 1 2024-04-20T11:09:39.681 INFO:tasks.workunit.client.1.smithi173.stdout:6/625: dwrite d4/d24/d42/fbd [0,4194304] 0 2024-04-20T11:09:39.685 INFO:tasks.workunit.client.1.smithi173.stdout:4/715: dread da/d1a/d60/f63 [0,4194304] 0 2024-04-20T11:09:39.685 INFO:tasks.workunit.client.1.smithi173.stdout:5/636: dread - da/de/d75/d40/f42 zero size 2024-04-20T11:09:39.686 INFO:tasks.workunit.client.1.smithi173.stdout:3/628: write d7/d17/f20 [31380,75429] 0 2024-04-20T11:09:39.688 INFO:tasks.workunit.client.1.smithi173.stdout:8/633: rename d4/d18/d59/d6d/db8/d3c/f53 to d4/d18/d4f/fca 0 2024-04-20T11:09:39.690 INFO:tasks.workunit.client.1.smithi173.stdout:5/637: chown da/d34/d7f/l80 29067758 1 2024-04-20T11:09:39.692 INFO:tasks.workunit.client.1.smithi173.stdout:0/585: creat d6/d8/df/d2c/d42/d50/d86/fbc x:0 0 0 2024-04-20T11:09:39.693 INFO:tasks.workunit.client.1.smithi173.stdout:2/691: rename d2/d13/d46/d53/c2c to d2/d13/d46/d53/d9f/ce4 0 2024-04-20T11:09:39.699 INFO:tasks.workunit.client.1.smithi173.stdout:8/634: readlink d4/d18/d59/d6d/db8/l85 0 2024-04-20T11:09:39.699 INFO:tasks.workunit.client.1.smithi173.stdout:4/716: creat da/dc/fed x:0 0 0 2024-04-20T11:09:39.702 INFO:tasks.workunit.client.1.smithi173.stdout:5/638: symlink da/de/d19/d32/lde 0 2024-04-20T11:09:39.705 INFO:tasks.workunit.client.1.smithi173.stdout:2/692: rmdir d2/d13/d46/d4b/dae/db6 39 2024-04-20T11:09:39.709 INFO:tasks.workunit.client.1.smithi173.stdout:1/574: dread d5/d6/dc/fe [0,4194304] 0 2024-04-20T11:09:39.709 INFO:tasks.workunit.client.1.smithi173.stdout:9/597: dwrite d5/d9/d20/d29/d18/f9c [0,4194304] 0 2024-04-20T11:09:39.710 INFO:tasks.workunit.client.1.smithi173.stdout:3/629: getdents d7/d17/d58/d8a 0 2024-04-20T11:09:39.712 INFO:tasks.workunit.client.1.smithi173.stdout:4/717: mknod da/d1a/d60/d66/cee 0 2024-04-20T11:09:39.715 INFO:tasks.workunit.client.1.smithi173.stdout:2/693: symlink d2/d13/le5 0 2024-04-20T11:09:39.715 INFO:tasks.workunit.client.1.smithi173.stdout:8/635: creat d4/d18/d59/d5a/fcb x:0 0 0 2024-04-20T11:09:39.717 INFO:tasks.workunit.client.1.smithi173.stdout:2/694: chown d2/d13/d46/d53/d4e 1200000 1 2024-04-20T11:09:39.717 INFO:tasks.workunit.client.1.smithi173.stdout:9/598: fsync d5/d9/f58 0 2024-04-20T11:09:39.720 INFO:tasks.workunit.client.1.smithi173.stdout:2/695: stat d2/de/d11/db8/fdc 0 2024-04-20T11:09:39.723 INFO:tasks.workunit.client.1.smithi173.stdout:3/630: mknod d7/d17/d61/d90/cce 0 2024-04-20T11:09:39.725 INFO:tasks.workunit.client.1.smithi173.stdout:5/639: dread da/de/d19/d2c/fa6 [0,4194304] 0 2024-04-20T11:09:39.725 INFO:tasks.workunit.client.1.smithi173.stdout:4/718: mkdir da/d1e/d52/d9a/dd3/def 0 2024-04-20T11:09:39.731 INFO:tasks.workunit.client.1.smithi173.stdout:2/696: symlink d2/d34/d58/le6 0 2024-04-20T11:09:39.733 INFO:tasks.workunit.client.1.smithi173.stdout:1/575: creat d5/d6/dc/d4a/d94/dbb/fc0 x:0 0 0 2024-04-20T11:09:39.741 INFO:tasks.workunit.client.1.smithi173.stdout:3/631: rename d7/d9 to d7/d17/d61/d83/dcf 0 2024-04-20T11:09:39.744 INFO:tasks.workunit.client.1.smithi173.stdout:9/599: unlink d5/l8 0 2024-04-20T11:09:39.747 INFO:tasks.workunit.client.1.smithi173.stdout:8/636: creat d4/d18/fcc x:0 0 0 2024-04-20T11:09:39.747 INFO:tasks.workunit.client.1.smithi173.stdout:1/576: creat d5/d6/dc/d4a/d94/fc1 x:0 0 0 2024-04-20T11:09:39.748 INFO:tasks.workunit.client.1.smithi173.stdout:8/637: readlink d4/l12 0 2024-04-20T11:09:39.750 INFO:tasks.workunit.client.1.smithi173.stdout:0/586: dread d6/d8/f1f [0,4194304] 0 2024-04-20T11:09:39.752 INFO:tasks.workunit.client.1.smithi173.stdout:4/719: getdents da/dc/d13/d65 0 2024-04-20T11:09:39.754 INFO:tasks.workunit.client.1.smithi173.stdout:4/720: chown da/dc/f27 112016960 1 2024-04-20T11:09:39.754 INFO:tasks.workunit.client.1.smithi173.stdout:3/632: dread d7/d17/f3c [0,4194304] 0 2024-04-20T11:09:39.763 INFO:tasks.workunit.client.1.smithi173.stdout:0/587: unlink d6/d54/f81 0 2024-04-20T11:09:39.764 INFO:tasks.workunit.client.1.smithi173.stdout:2/697: write d2/de/f7a [2476772,15214] 0 2024-04-20T11:09:39.768 INFO:tasks.workunit.client.1.smithi173.stdout:1/577: rename d5/d6/dc/d4a to d5/d6/d17/d42/da/d20/dad/dc2 0 2024-04-20T11:09:39.770 INFO:tasks.workunit.client.1.smithi173.stdout:5/640: write da/d34/f47 [1083351,67285] 0 2024-04-20T11:09:39.774 INFO:tasks.workunit.client.1.smithi173.stdout:4/721: mkdir da/dc/d13/df0 0 2024-04-20T11:09:39.775 INFO:tasks.workunit.client.1.smithi173.stdout:4/722: readlink da/d48/d53/d39/l73 0 2024-04-20T11:09:39.775 INFO:tasks.workunit.client.1.smithi173.stdout:9/600: link d5/cb4 d5/d9/d20/d2b/d59/d44/d78/d9d/cb6 0 2024-04-20T11:09:39.800 INFO:tasks.workunit.client.1.smithi173.stdout:2/698: unlink d2/f3f 0 2024-04-20T11:09:39.801 INFO:tasks.workunit.client.1.smithi173.stdout:0/588: creat d6/d8/df/da0/fbd x:0 0 0 2024-04-20T11:09:39.801 INFO:tasks.workunit.client.1.smithi173.stdout:1/578: symlink d5/d6/d3b/lc3 0 2024-04-20T11:09:39.806 INFO:tasks.workunit.client.1.smithi173.stdout:8/638: dwrite d4/d18/d4f/f7d [0,4194304] 0 2024-04-20T11:09:39.812 INFO:tasks.workunit.client.1.smithi173.stdout:1/579: dread d5/d6/d3b/f67 [0,4194304] 0 2024-04-20T11:09:39.813 INFO:tasks.workunit.client.1.smithi173.stdout:1/580: dread - d5/d6/d17/fbf zero size 2024-04-20T11:09:39.814 INFO:tasks.workunit.client.1.smithi173.stdout:1/581: chown d5/d6/f73 377 1 2024-04-20T11:09:39.815 INFO:tasks.workunit.client.1.smithi173.stdout:1/582: readlink d5/d6/d3b/l3e 0 2024-04-20T11:09:39.829 INFO:tasks.workunit.client.1.smithi173.stdout:9/601: unlink d5/d9/d20/d29/d2d/l54 0 2024-04-20T11:09:39.833 INFO:tasks.workunit.client.1.smithi173.stdout:2/699: fdatasync d2/de/f60 0 2024-04-20T11:09:39.834 INFO:tasks.workunit.client.1.smithi173.stdout:5/641: symlink da/de/d19/d81/d92/ldf 0 2024-04-20T11:09:39.836 INFO:tasks.workunit.client.1.smithi173.stdout:5/642: read da/d34/f47 [3446917,72636] 0 2024-04-20T11:09:39.836 INFO:tasks.workunit.client.1.smithi173.stdout:0/589: link d6/d8/d3c/d44/d75/f97 d6/d8/df/d2c/d42/d50/d86/d8b/fbe 0 2024-04-20T11:09:39.838 INFO:tasks.workunit.client.1.smithi173.stdout:9/602: creat d5/d9/d20/d2b/d59/d44/d78/d9d/fb7 x:0 0 0 2024-04-20T11:09:39.840 INFO:tasks.workunit.client.1.smithi173.stdout:1/583: unlink d5/d6/dc/d7f/l9b 0 2024-04-20T11:09:39.843 INFO:tasks.workunit.client.1.smithi173.stdout:2/700: truncate d2/d13/d46/d4b/fda 483586 0 2024-04-20T11:09:39.844 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:39 smithi173 ceph-mon[23800]: pgmap v29: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 59 MiB/s rd, 201 MiB/s wr, 449 op/s 2024-04-20T11:09:39.845 INFO:tasks.workunit.client.1.smithi173.stdout:0/590: write d6/d8/df/d43/f59 [1574248,97153] 0 2024-04-20T11:09:39.845 INFO:tasks.workunit.client.1.smithi173.stdout:8/639: creat d4/fcd x:0 0 0 2024-04-20T11:09:39.845 INFO:tasks.workunit.client.1.smithi173.stdout:5/643: rmdir da/de/d19/d32/d66/dbd 39 2024-04-20T11:09:39.847 INFO:tasks.workunit.client.1.smithi173.stdout:1/584: symlink d5/d6/dc/d7f/lc4 0 2024-04-20T11:09:39.849 INFO:tasks.workunit.client.1.smithi173.stdout:8/640: write d4/d18/d4f/f9a [935532,52150] 0 2024-04-20T11:09:39.853 INFO:tasks.workunit.client.1.smithi173.stdout:9/603: dread d5/f2f [0,4194304] 0 2024-04-20T11:09:39.853 INFO:tasks.workunit.client.1.smithi173.stdout:5/644: mkdir da/de/d19/dca/d48/da2/de0 0 2024-04-20T11:09:39.853 INFO:tasks.workunit.client.1.smithi173.stdout:2/701: dread d2/de/f84 [0,4194304] 0 2024-04-20T11:09:39.856 INFO:tasks.workunit.client.1.smithi173.stdout:2/702: stat d2/d34/d58/ca4 0 2024-04-20T11:09:39.860 INFO:tasks.workunit.client.1.smithi173.stdout:9/604: symlink d5/d9/d20/d2b/d59/da1/lb8 0 2024-04-20T11:09:39.862 INFO:tasks.workunit.client.1.smithi173.stdout:9/605: write d5/f6 [283281,70531] 0 2024-04-20T11:09:39.865 INFO:tasks.workunit.client.1.smithi173.stdout:9/606: write d5/d9/d20/d29/d18/f9c [3471855,80543] 0 2024-04-20T11:09:39.867 INFO:tasks.workunit.client.1.smithi173.stdout:2/703: creat d2/fe7 x:0 0 0 2024-04-20T11:09:39.868 INFO:tasks.workunit.client.1.smithi173.stdout:9/607: dread - d5/d9/d20/d2b/d59/da1/fa3 zero size 2024-04-20T11:09:39.872 INFO:tasks.workunit.client.1.smithi173.stdout:0/591: dread d6/d8/df/f11 [0,4194304] 0 2024-04-20T11:09:39.881 INFO:tasks.workunit.client.1.smithi173.stdout:9/608: getdents d5/d9/d20/d29/d2d/d33 0 2024-04-20T11:09:39.882 INFO:tasks.workunit.client.1.smithi173.stdout:8/641: write d4/d18/f6b [359580,130011] 0 2024-04-20T11:09:39.883 INFO:tasks.workunit.client.1.smithi173.stdout:1/585: truncate d5/d6/f73 354267 0 2024-04-20T11:09:39.887 INFO:tasks.workunit.client.1.smithi173.stdout:8/642: unlink d4/c14 0 2024-04-20T11:09:39.889 INFO:tasks.workunit.client.1.smithi173.stdout:2/704: dwrite d2/d34/d58/f69 [0,4194304] 0 2024-04-20T11:09:39.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:39 smithi117 ceph-mon[15659]: pgmap v29: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 59 MiB/s rd, 201 MiB/s wr, 449 op/s 2024-04-20T11:09:39.890 INFO:tasks.workunit.client.1.smithi173.stdout:0/592: dwrite d6/d54/d65/f98 [0,4194304] 0 2024-04-20T11:09:39.891 INFO:tasks.workunit.client.1.smithi173.stdout:1/586: unlink d5/d6/dc/d7f/lae 0 2024-04-20T11:09:39.892 INFO:tasks.workunit.client.1.smithi173.stdout:9/609: link d5/d9/fa d5/d9/fb9 0 2024-04-20T11:09:39.895 INFO:tasks.workunit.client.1.smithi173.stdout:9/610: fsync d5/d9/d20/d29/d2d/f43 0 2024-04-20T11:09:39.896 INFO:tasks.workunit.client.1.smithi173.stdout:0/593: readlink d6/d8/d3c/d44/d75/l85 0 2024-04-20T11:09:39.896 INFO:tasks.workunit.client.1.smithi173.stdout:5/645: dwrite da/de/d75/f51 [0,4194304] 0 2024-04-20T11:09:39.898 INFO:tasks.workunit.client.1.smithi173.stdout:2/705: rmdir d2/d34/d81/dca 39 2024-04-20T11:09:39.898 INFO:tasks.workunit.client.1.smithi173.stdout:7/642: sync 2024-04-20T11:09:39.901 INFO:tasks.workunit.client.1.smithi173.stdout:8/643: mkdir d4/d18/d59/d5a/d7b/db1/dce 0 2024-04-20T11:09:39.903 INFO:tasks.workunit.client.1.smithi173.stdout:9/611: readlink d5/d9/d20/d4f/l5d 0 2024-04-20T11:09:39.905 INFO:tasks.workunit.client.1.smithi173.stdout:0/594: mknod d6/d8/df/d2c/d42/d50/d86/d8b/cbf 0 2024-04-20T11:09:39.917 INFO:tasks.workunit.client.1.smithi173.stdout:5/646: rename da/de/d19/d2c/f68 to da/de/d19/d81/d92/fe1 0 2024-04-20T11:09:39.918 INFO:tasks.workunit.client.1.smithi173.stdout:9/612: mkdir d5/d9/d20/d4f/d5e/d7d/dba 0 2024-04-20T11:09:39.919 INFO:tasks.workunit.client.1.smithi173.stdout:8/644: dwrite d4/f19 [0,4194304] 0 2024-04-20T11:09:39.919 INFO:tasks.workunit.client.1.smithi173.stdout:9/613: stat d5/d9/f1b 0 2024-04-20T11:09:39.923 INFO:tasks.workunit.client.1.smithi173.stdout:2/706: mknod d2/d13/d46/d53/d4e/ce8 0 2024-04-20T11:09:39.924 INFO:tasks.workunit.client.1.smithi173.stdout:1/587: dread d5/d6/d17/d42/da/f28 [0,4194304] 0 2024-04-20T11:09:39.925 INFO:tasks.workunit.client.1.smithi173.stdout:7/643: write d9/f4e [106030,57834] 0 2024-04-20T11:09:39.929 INFO:tasks.workunit.client.1.smithi173.stdout:0/595: link d6/d8/df/d2c/d42/f84 d6/d8/df/d43/fc0 0 2024-04-20T11:09:39.929 INFO:tasks.workunit.client.1.smithi173.stdout:9/614: write d5/f38 [961101,35047] 0 2024-04-20T11:09:39.933 INFO:tasks.workunit.client.1.smithi173.stdout:1/588: dread d5/d6/d17/d42/d12/d1c/d85/f9e [0,4194304] 0 2024-04-20T11:09:39.934 INFO:tasks.workunit.client.1.smithi173.stdout:9/615: write d5/d9/d20/f40 [628054,52865] 0 2024-04-20T11:09:39.936 INFO:tasks.workunit.client.1.smithi173.stdout:7/644: rename d9/d33/d81/dd7 to d9/d33/de9 0 2024-04-20T11:09:39.937 INFO:tasks.workunit.client.1.smithi173.stdout:5/647: dwrite da/d4f/d65/d8e/f8d [0,4194304] 0 2024-04-20T11:09:39.940 INFO:tasks.workunit.client.1.smithi173.stdout:0/596: symlink d6/d8/df/d2c/d42/d50/lc1 0 2024-04-20T11:09:39.946 INFO:tasks.workunit.client.1.smithi173.stdout:1/589: creat d5/d6/d17/d42/da/d20/dad/dc2/fc5 x:0 0 0 2024-04-20T11:09:39.950 INFO:tasks.workunit.client.1.smithi173.stdout:2/707: rename d2/de/d11/f23 to d2/d13/d46/fe9 0 2024-04-20T11:09:39.953 INFO:tasks.workunit.client.1.smithi173.stdout:5/648: creat da/de/fe2 x:0 0 0 2024-04-20T11:09:39.956 INFO:tasks.workunit.client.1.smithi173.stdout:1/590: dread - d5/d6/d17/d42/da/d20/d38/d52/f74 zero size 2024-04-20T11:09:39.959 INFO:tasks.workunit.client.1.smithi173.stdout:2/708: unlink d2/d13/d28/f7b 0 2024-04-20T11:09:39.961 INFO:tasks.workunit.client.1.smithi173.stdout:5/649: rmdir da/de/d19/d2c/dcf 39 2024-04-20T11:09:39.963 INFO:tasks.workunit.client.1.smithi173.stdout:0/597: getdents d6/d8/d3c/d44/d75 0 2024-04-20T11:09:39.966 INFO:tasks.workunit.client.1.smithi173.stdout:1/591: symlink d5/d6/lc6 0 2024-04-20T11:09:39.968 INFO:tasks.workunit.client.1.smithi173.stdout:2/709: creat d2/d13/da5/db2/fea x:0 0 0 2024-04-20T11:09:39.970 INFO:tasks.workunit.client.1.smithi173.stdout:5/650: creat da/d4f/d65/fe3 x:0 0 0 2024-04-20T11:09:39.970 INFO:tasks.workunit.client.1.smithi173.stdout:2/710: write d2/d34/d58/f63 [200942,116344] 0 2024-04-20T11:09:39.971 INFO:tasks.workunit.client.1.smithi173.stdout:0/598: mknod d6/d54/d65/cc2 0 2024-04-20T11:09:40.034 INFO:tasks.workunit.client.1.smithi173.stdout:3/633: sync 2024-04-20T11:09:40.034 INFO:tasks.workunit.client.1.smithi173.stdout:4/723: sync 2024-04-20T11:09:40.034 INFO:tasks.workunit.client.1.smithi173.stdout:6/626: sync 2024-04-20T11:09:40.034 INFO:tasks.workunit.client.1.smithi173.stdout:0/599: fdatasync d6/d54/d65/f98 0 2024-04-20T11:09:40.034 INFO:tasks.workunit.client.1.smithi173.stdout:1/592: rename d5/d6/d17/d42/da/d20/d38/l80 to d5/d6/d17/d93/lc7 0 2024-04-20T11:09:40.037 INFO:tasks.workunit.client.1.smithi173.stdout:2/711: symlink d2/d34/leb 0 2024-04-20T11:09:40.037 INFO:tasks.workunit.client.1.smithi173.stdout:3/634: chown d7/d17 35 1 2024-04-20T11:09:40.041 INFO:tasks.workunit.client.1.smithi173.stdout:5/651: unlink da/fc 0 2024-04-20T11:09:40.042 INFO:tasks.workunit.client.1.smithi173.stdout:1/593: dread - d5/d6/d17/d42/da/d20/f33 zero size 2024-04-20T11:09:40.044 INFO:tasks.workunit.client.1.smithi173.stdout:2/712: mknod d2/de/d11/db8/cec 0 2024-04-20T11:09:40.045 INFO:tasks.workunit.client.1.smithi173.stdout:0/600: mknod d6/d8/df/d2c/d42/d50/d86/d8b/cc3 0 2024-04-20T11:09:40.046 INFO:tasks.workunit.client.1.smithi173.stdout:2/713: fsync d2/d13/d46/fe3 0 2024-04-20T11:09:40.047 INFO:tasks.workunit.client.1.smithi173.stdout:5/652: write da/de/d19/d32/f31 [1703201,84877] 0 2024-04-20T11:09:40.048 INFO:tasks.workunit.client.1.smithi173.stdout:1/594: chown d5/d6/d17/d42/da/f6a 11 1 2024-04-20T11:09:40.049 INFO:tasks.workunit.client.1.smithi173.stdout:6/627: symlink d4/d14/d5f/daf/lc5 0 2024-04-20T11:09:40.052 INFO:tasks.workunit.client.1.smithi173.stdout:0/601: write d6/d8/df/d15/f39 [264715,4049] 0 2024-04-20T11:09:40.052 INFO:tasks.workunit.client.1.smithi173.stdout:5/653: write da/de/f2d [7536557,81061] 0 2024-04-20T11:09:40.055 INFO:tasks.workunit.client.1.smithi173.stdout:3/635: getdents d7 0 2024-04-20T11:09:40.055 INFO:tasks.workunit.client.1.smithi173.stdout:4/724: link da/d1e/da6/cb9 da/d1a/d60/d66/d9b/cf1 0 2024-04-20T11:09:40.056 INFO:tasks.workunit.client.1.smithi173.stdout:1/595: rmdir d5/d6/d17/d42 39 2024-04-20T11:09:40.057 INFO:tasks.workunit.client.1.smithi173.stdout:0/602: chown d6/d8/df/d15 75302 1 2024-04-20T11:09:40.057 INFO:tasks.workunit.client.1.smithi173.stdout:6/628: creat d4/d24/d42/d74/d3b/fc6 x:0 0 0 2024-04-20T11:09:40.058 INFO:tasks.workunit.client.1.smithi173.stdout:2/714: mknod d2/d13/d46/d53/ced 0 2024-04-20T11:09:40.059 INFO:tasks.workunit.client.1.smithi173.stdout:5/654: fdatasync da/de/d19/d2c/f94 0 2024-04-20T11:09:40.059 INFO:tasks.workunit.client.1.smithi173.stdout:3/636: fdatasync d7/d17/d61/d83/dcf/f4f 0 2024-04-20T11:09:40.063 INFO:tasks.workunit.client.1.smithi173.stdout:4/725: rename da/l6c to da/dd1/lf2 0 2024-04-20T11:09:40.064 INFO:tasks.workunit.client.1.smithi173.stdout:0/603: chown d6/d8/d3c/d44/c93 334204 1 2024-04-20T11:09:40.066 INFO:tasks.workunit.client.1.smithi173.stdout:4/726: write da/d1a/d60/fb1 [861231,80920] 0 2024-04-20T11:09:40.071 INFO:tasks.workunit.client.1.smithi173.stdout:2/715: read d2/d34/f3a [133515,39178] 0 2024-04-20T11:09:40.074 INFO:tasks.workunit.client.1.smithi173.stdout:6/629: truncate d4/f46 170295 0 2024-04-20T11:09:40.077 INFO:tasks.workunit.client.1.smithi173.stdout:1/596: mkdir d5/d6/d17/d42/da/df/d40/dc8 0 2024-04-20T11:09:40.083 INFO:tasks.workunit.client.1.smithi173.stdout:5/655: truncate da/de/d19/d2c/fa6 101773 0 2024-04-20T11:09:40.085 INFO:tasks.workunit.client.1.smithi173.stdout:0/604: symlink d6/d8/df/d15/lc4 0 2024-04-20T11:09:40.087 INFO:tasks.workunit.client.1.smithi173.stdout:4/727: symlink da/d48/lf3 0 2024-04-20T11:09:40.088 INFO:tasks.workunit.client.1.smithi173.stdout:3/637: mkdir d7/d17/d58/dd0 0 2024-04-20T11:09:40.088 INFO:tasks.workunit.client.1.smithi173.stdout:0/605: chown d6/d8/df/l88 53484 1 2024-04-20T11:09:40.098 INFO:tasks.workunit.client.1.smithi173.stdout:1/597: mkdir d5/d6/d17/d42/d12/dc9 0 2024-04-20T11:09:40.100 INFO:tasks.workunit.client.1.smithi173.stdout:2/716: creat d2/d13/d46/d53/fee x:0 0 0 2024-04-20T11:09:40.102 INFO:tasks.workunit.client.1.smithi173.stdout:0/606: readlink d6/d8/df/d2c/d42/d50/d86/l8d 0 2024-04-20T11:09:40.103 INFO:tasks.workunit.client.1.smithi173.stdout:3/638: truncate d7/d17/d61/d83/dcf/d30/fbe 1010072 0 2024-04-20T11:09:40.106 INFO:tasks.workunit.client.1.smithi173.stdout:6/630: dwrite d4/d24/d3e/d9c/fb1 [0,4194304] 0 2024-04-20T11:09:40.111 INFO:tasks.workunit.client.1.smithi173.stdout:0/607: creat d6/d8/d51/fc5 x:0 0 0 2024-04-20T11:09:40.114 INFO:tasks.workunit.client.1.smithi173.stdout:2/717: unlink d2/d13/d46/d53/d9f/ce4 0 2024-04-20T11:09:40.115 INFO:tasks.workunit.client.1.smithi173.stdout:4/728: rename da/d48 to da/dc/df4 0 2024-04-20T11:09:40.117 INFO:tasks.workunit.client.1.smithi173.stdout:6/631: mkdir d4/d24/d3e/d9c/dc7 0 2024-04-20T11:09:40.122 INFO:tasks.workunit.client.1.smithi173.stdout:5/656: rename da/de/d19/d32/d30/dbb/cc7 to da/d4f/d65/d8e/ce4 0 2024-04-20T11:09:40.127 INFO:tasks.workunit.client.1.smithi173.stdout:3/639: dwrite d7/d17/d58/d7f/fba [0,4194304] 0 2024-04-20T11:09:40.129 INFO:tasks.workunit.client.1.smithi173.stdout:6/632: rmdir d4/d18/d7e 39 2024-04-20T11:09:40.137 INFO:tasks.workunit.client.1.smithi173.stdout:0/608: dread - d6/d8/d51/fc5 zero size 2024-04-20T11:09:40.137 INFO:tasks.workunit.client.1.smithi173.stdout:6/633: rmdir d4/d1c/d8e 39 2024-04-20T11:09:40.137 INFO:tasks.workunit.client.1.smithi173.stdout:4/729: dread da/d1e/d52/d9a/fa8 [0,4194304] 0 2024-04-20T11:09:40.141 INFO:tasks.workunit.client.1.smithi173.stdout:3/640: mkdir d7/d17/d58/d8a/dc4/dd1 0 2024-04-20T11:09:40.141 INFO:tasks.workunit.client.1.smithi173.stdout:2/718: dwrite d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:40.162 INFO:tasks.workunit.client.1.smithi173.stdout:8/645: sync 2024-04-20T11:09:40.177 INFO:tasks.workunit.client.1.smithi173.stdout:2/719: dread - d2/fd8 zero size 2024-04-20T11:09:40.177 INFO:tasks.workunit.client.1.smithi173.stdout:6/634: mknod d4/d24/cc8 0 2024-04-20T11:09:40.177 INFO:tasks.workunit.client.1.smithi173.stdout:5/657: write da/de/d19/dca/f8c [771698,34784] 0 2024-04-20T11:09:40.178 INFO:tasks.workunit.client.1.smithi173.stdout:3/641: chown d7/d17/d61/d83/dcf/f46 57434047 1 2024-04-20T11:09:40.179 INFO:tasks.workunit.client.1.smithi173.stdout:0/609: rmdir d6/d8/d3c/d44/da6 0 2024-04-20T11:09:40.179 INFO:tasks.workunit.client.1.smithi173.stdout:8/646: dread - d4/d18/d59/d5a/d7b/d8c/fb5 zero size 2024-04-20T11:09:40.183 INFO:tasks.workunit.client.1.smithi173.stdout:6/635: symlink d4/d14/d5f/lc9 0 2024-04-20T11:09:40.193 INFO:tasks.workunit.client.1.smithi173.stdout:2/720: rename d2/l65 to d2/d13/d46/d53/d9f/lef 0 2024-04-20T11:09:40.194 INFO:tasks.workunit.client.1.smithi173.stdout:2/721: chown d2/d13/d28/ccb 192890 1 2024-04-20T11:09:40.195 INFO:tasks.workunit.client.1.smithi173.stdout:5/658: getdents da/de/d75/d3e 0 2024-04-20T11:09:40.201 INFO:tasks.workunit.client.1.smithi173.stdout:2/722: creat d2/ff0 x:0 0 0 2024-04-20T11:09:40.202 INFO:tasks.workunit.client.1.smithi173.stdout:8/647: dwrite d4/d18/d59/d6d/db8/d3c/f37 [4194304,4194304] 0 2024-04-20T11:09:40.203 INFO:tasks.workunit.client.1.smithi173.stdout:5/659: creat da/de/d19/fe5 x:0 0 0 2024-04-20T11:09:40.205 INFO:tasks.workunit.client.1.smithi173.stdout:8/648: dread - d4/d18/f93 zero size 2024-04-20T11:09:40.208 INFO:tasks.workunit.client.1.smithi173.stdout:2/723: symlink d2/d13/d46/dd0/lf1 0 2024-04-20T11:09:40.210 INFO:tasks.workunit.client.1.smithi173.stdout:3/642: dwrite d7/d17/f26 [0,4194304] 0 2024-04-20T11:09:40.211 INFO:tasks.workunit.client.1.smithi173.stdout:2/724: stat d2/d13/d46/d53/d4e 0 2024-04-20T11:09:40.212 INFO:tasks.workunit.client.1.smithi173.stdout:6/636: dwrite d4/d24/f2e [0,4194304] 0 2024-04-20T11:09:40.215 INFO:tasks.workunit.client.1.smithi173.stdout:0/610: dread d6/d24/fad [0,4194304] 0 2024-04-20T11:09:40.217 INFO:tasks.workunit.client.1.smithi173.stdout:6/637: truncate d4/d24/d42/d74/d3b/fa8 30409 0 2024-04-20T11:09:40.220 INFO:tasks.workunit.client.1.smithi173.stdout:5/660: rename da/de/d75/d40/d59/l6b to da/d34/le6 0 2024-04-20T11:09:40.220 INFO:tasks.workunit.client.1.smithi173.stdout:8/649: creat d4/d18/fcf x:0 0 0 2024-04-20T11:09:40.220 INFO:tasks.workunit.client.1.smithi173.stdout:0/611: fdatasync d6/d8/df/d43/fc0 0 2024-04-20T11:09:40.223 INFO:tasks.workunit.client.1.smithi173.stdout:3/643: symlink d7/d17/d58/dd0/ld2 0 2024-04-20T11:09:40.232 INFO:tasks.workunit.client.1.smithi173.stdout:6/638: link d4/d14/f3c d4/d14/d5f/d9f/fca 0 2024-04-20T11:09:40.238 INFO:tasks.workunit.client.1.smithi173.stdout:0/612: dread d6/d54/d65/f98 [0,4194304] 0 2024-04-20T11:09:40.239 INFO:tasks.workunit.client.1.smithi173.stdout:6/639: readlink d4/d18/d7e/lc3 0 2024-04-20T11:09:40.239 INFO:tasks.workunit.client.1.smithi173.stdout:3/644: link d7/c6a d7/d17/d61/d83/dcf/cd3 0 2024-04-20T11:09:40.240 INFO:tasks.workunit.client.1.smithi173.stdout:8/650: link d4/d18/d59/d6d/db8/c99 d4/d18/d59/d5a/da8/cd0 0 2024-04-20T11:09:40.242 INFO:tasks.workunit.client.1.smithi173.stdout:8/651: fdatasync d4/d18/d59/d5a/f95 0 2024-04-20T11:09:40.248 INFO:tasks.workunit.client.1.smithi173.stdout:2/725: truncate d2/d34/d58/f5a 5756629 0 2024-04-20T11:09:40.249 INFO:tasks.workunit.client.1.smithi173.stdout:8/652: mknod d4/d18/d59/cd1 0 2024-04-20T11:09:40.252 INFO:tasks.workunit.client.1.smithi173.stdout:6/640: link d4/f11 d4/d14/d5f/d9f/fcb 0 2024-04-20T11:09:40.254 INFO:tasks.workunit.client.1.smithi173.stdout:2/726: write d2/de/f84 [1178901,121089] 0 2024-04-20T11:09:40.258 INFO:tasks.workunit.client.1.smithi173.stdout:8/653: write d4/d18/d59/d5a/d7b/d8c/fb5 [537237,7163] 0 2024-04-20T11:09:40.262 INFO:tasks.workunit.client.1.smithi173.stdout:8/654: chown d4/cc0 3961 1 2024-04-20T11:09:40.262 INFO:tasks.workunit.client.1.smithi173.stdout:0/613: dwrite d6/d54/d65/f7c [0,4194304] 0 2024-04-20T11:09:40.262 INFO:tasks.workunit.client.1.smithi173.stdout:3/645: dwrite d7/d17/d61/d83/dcf/f4f [0,4194304] 0 2024-04-20T11:09:40.262 INFO:tasks.workunit.client.1.smithi173.stdout:6/641: write d4/f16 [3515243,86926] 0 2024-04-20T11:09:40.265 INFO:tasks.workunit.client.1.smithi173.stdout:0/614: chown d6/d54 13461 1 2024-04-20T11:09:40.267 INFO:tasks.workunit.client.1.smithi173.stdout:2/727: rmdir d2/d13/d46/dd0 39 2024-04-20T11:09:40.268 INFO:tasks.workunit.client.1.smithi173.stdout:9/616: sync 2024-04-20T11:09:40.271 INFO:tasks.workunit.client.1.smithi173.stdout:2/728: read - d2/fe7 zero size 2024-04-20T11:09:40.271 INFO:tasks.workunit.client.1.smithi173.stdout:5/661: dwrite da/de/d19/f5c [0,4194304] 0 2024-04-20T11:09:40.274 INFO:tasks.workunit.client.1.smithi173.stdout:8/655: truncate d4/d18/d4f/d5b/fc3 489531 0 2024-04-20T11:09:40.281 INFO:tasks.workunit.client.1.smithi173.stdout:9/617: read d5/d9/d20/d29/d2d/f43 [1526211,52367] 0 2024-04-20T11:09:40.281 INFO:tasks.workunit.client.1.smithi173.stdout:3/646: mknod d7/d17/d61/d83/dcf/cd4 0 2024-04-20T11:09:40.286 INFO:tasks.workunit.client.1.smithi173.stdout:6/642: link d4/d18/f66 d4/d24/d3e/d9c/dc7/fcc 0 2024-04-20T11:09:40.287 INFO:tasks.workunit.client.1.smithi173.stdout:0/615: dread d6/d24/fad [0,4194304] 0 2024-04-20T11:09:40.288 INFO:tasks.workunit.client.1.smithi173.stdout:0/616: stat d6/d54 0 2024-04-20T11:09:40.295 INFO:tasks.workunit.client.1.smithi173.stdout:8/656: rename d4/d18/f30 to d4/d18/fd2 0 2024-04-20T11:09:40.296 INFO:tasks.workunit.client.1.smithi173.stdout:5/662: getdents da/d34/d7f/da4 0 2024-04-20T11:09:40.298 INFO:tasks.workunit.client.1.smithi173.stdout:2/729: dwrite d2/f64 [0,4194304] 0 2024-04-20T11:09:40.300 INFO:tasks.workunit.client.1.smithi173.stdout:2/730: chown d2/de/d11/cad 123 1 2024-04-20T11:09:40.300 INFO:tasks.workunit.client.1.smithi173.stdout:8/657: rmdir d4/d18/d59/d5a 39 2024-04-20T11:09:40.302 INFO:tasks.workunit.client.1.smithi173.stdout:2/731: readlink d2/de/d11/l25 0 2024-04-20T11:09:40.304 INFO:tasks.workunit.client.1.smithi173.stdout:7/645: sync 2024-04-20T11:09:40.304 INFO:tasks.workunit.client.1.smithi173.stdout:4/730: sync 2024-04-20T11:09:40.305 INFO:tasks.workunit.client.1.smithi173.stdout:1/598: sync 2024-04-20T11:09:40.305 INFO:tasks.workunit.client.1.smithi173.stdout:9/618: write d5/d4c/d79/f7a [676918,2734] 0 2024-04-20T11:09:40.307 INFO:tasks.workunit.client.1.smithi173.stdout:8/658: symlink d4/d18/d59/d6d/db8/d3c/d86/ld3 0 2024-04-20T11:09:40.307 INFO:tasks.workunit.client.1.smithi173.stdout:2/732: chown d2/d13/d46/d53/l2f 2 1 2024-04-20T11:09:40.308 INFO:tasks.workunit.client.1.smithi173.stdout:7/646: stat d9/da/cab 0 2024-04-20T11:09:40.314 INFO:tasks.workunit.client.1.smithi173.stdout:9/619: fsync d5/d9/f47 0 2024-04-20T11:09:40.314 INFO:tasks.workunit.client.1.smithi173.stdout:4/731: dread - da/dc/df4/d53/d39/f96 zero size 2024-04-20T11:09:40.315 INFO:tasks.workunit.client.1.smithi173.stdout:8/659: mknod d4/d18/d59/d6d/cd4 0 2024-04-20T11:09:40.317 INFO:tasks.workunit.client.1.smithi173.stdout:7/647: symlink d9/da/lea 0 2024-04-20T11:09:40.319 INFO:tasks.workunit.client.1.smithi173.stdout:6/643: dwrite d4/d18/f62 [0,4194304] 0 2024-04-20T11:09:40.320 INFO:tasks.workunit.client.1.smithi173.stdout:1/599: creat d5/d6/d17/d42/da/df/d40/dac/fca x:0 0 0 2024-04-20T11:09:40.325 INFO:tasks.workunit.client.1.smithi173.stdout:7/648: creat d9/d33/d40/d89/feb x:0 0 0 2024-04-20T11:09:40.327 INFO:tasks.workunit.client.1.smithi173.stdout:4/732: fdatasync da/d1e/d52/f5c 0 2024-04-20T11:09:40.327 INFO:tasks.workunit.client.1.smithi173.stdout:3/647: dwrite d7/d53/f8f [0,4194304] 0 2024-04-20T11:09:40.334 INFO:tasks.workunit.client.1.smithi173.stdout:6/644: creat d4/d24/d42/d81/fcd x:0 0 0 2024-04-20T11:09:40.337 INFO:tasks.workunit.client.1.smithi173.stdout:4/733: chown da/dc/d15/d8e/lc1 14233 1 2024-04-20T11:09:40.339 INFO:tasks.workunit.client.1.smithi173.stdout:6/645: read d4/d24/d3e/d9c/fb1 [1353027,121066] 0 2024-04-20T11:09:40.340 INFO:tasks.workunit.client.1.smithi173.stdout:9/620: creat d5/d9/d20/d4f/d5e/d7d/fbb x:0 0 0 2024-04-20T11:09:40.341 INFO:tasks.workunit.client.1.smithi173.stdout:7/649: mknod d9/d1f/d7f/dce/dad/cec 0 2024-04-20T11:09:40.343 INFO:tasks.workunit.client.1.smithi173.stdout:5/663: dwrite f7 [4194304,4194304] 0 2024-04-20T11:09:40.343 INFO:tasks.workunit.client.1.smithi173.stdout:1/600: rename d5/d6/d3b/c5f to d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/ccb 0 2024-04-20T11:09:40.344 INFO:tasks.workunit.client.1.smithi173.stdout:4/734: dread - da/fc4 zero size 2024-04-20T11:09:40.345 INFO:tasks.workunit.client.1.smithi173.stdout:0/617: dread d6/d8/df/d15/f70 [0,4194304] 0 2024-04-20T11:09:40.346 INFO:tasks.workunit.client.1.smithi173.stdout:2/733: dwrite d2/f5c [0,4194304] 0 2024-04-20T11:09:40.355 INFO:tasks.workunit.client.1.smithi173.stdout:8/660: write d4/d18/d4f/f81 [1492128,29097] 0 2024-04-20T11:09:40.356 INFO:tasks.workunit.client.1.smithi173.stdout:5/664: write da/de/d19/f5c [168782,96830] 0 2024-04-20T11:09:40.358 INFO:tasks.workunit.client.1.smithi173.stdout:9/621: creat d5/d9/d20/d29/d18/d9b/fbc x:0 0 0 2024-04-20T11:09:40.358 INFO:tasks.workunit.client.1.smithi173.stdout:4/735: read da/dc/d15/f35 [943643,114060] 0 2024-04-20T11:09:40.360 INFO:tasks.workunit.client.1.smithi173.stdout:7/650: unlink d9/d33/d91/d99/fc0 0 2024-04-20T11:09:40.366 INFO:tasks.workunit.client.1.smithi173.stdout:3/648: dwrite d7/d17/d61/d83/f82 [0,4194304] 0 2024-04-20T11:09:40.368 INFO:tasks.workunit.client.1.smithi173.stdout:0/618: dread - d6/d8/df/d15/f83 zero size 2024-04-20T11:09:40.369 INFO:tasks.workunit.client.1.smithi173.stdout:6/646: dread d4/d14/d5f/d9f/fcb [0,4194304] 0 2024-04-20T11:09:40.371 INFO:tasks.workunit.client.1.smithi173.stdout:3/649: write d7/d17/d61/d83/f79 [1406778,46192] 0 2024-04-20T11:09:40.371 INFO:tasks.workunit.client.1.smithi173.stdout:8/661: unlink d4/c39 0 2024-04-20T11:09:40.371 INFO:tasks.workunit.client.1.smithi173.stdout:0/619: fsync d6/d8/d51/fc5 0 2024-04-20T11:09:40.378 INFO:tasks.workunit.client.1.smithi173.stdout:9/622: unlink d5/d9/d20/ca5 0 2024-04-20T11:09:40.379 INFO:tasks.workunit.client.1.smithi173.stdout:9/623: chown d5/d4c/d79 1749850 1 2024-04-20T11:09:40.382 INFO:tasks.workunit.client.1.smithi173.stdout:2/734: symlink d2/lf2 0 2024-04-20T11:09:40.383 INFO:tasks.workunit.client.1.smithi173.stdout:5/665: symlink da/de/le7 0 2024-04-20T11:09:40.386 INFO:tasks.workunit.client.1.smithi173.stdout:0/620: fdatasync d6/d8/df/d2c/d2d/f6a 0 2024-04-20T11:09:40.388 INFO:tasks.workunit.client.1.smithi173.stdout:0/621: chown d6/l48 14950143 1 2024-04-20T11:09:40.389 INFO:tasks.workunit.client.1.smithi173.stdout:7/651: creat d9/fed x:0 0 0 2024-04-20T11:09:40.393 INFO:tasks.workunit.client.1.smithi173.stdout:4/736: getdents da/dc/df4 0 2024-04-20T11:09:40.394 INFO:tasks.workunit.client.1.smithi173.stdout:5/666: rename da/de/d75/d40/d59/c97 to da/de/d19/d81/ce8 0 2024-04-20T11:09:40.395 INFO:tasks.workunit.client.1.smithi173.stdout:7/652: rename d9/d33/d85 to d9/d33/d85/d97/dee 22 2024-04-20T11:09:40.396 INFO:tasks.workunit.client.1.smithi173.stdout:0/622: rmdir d6/d5f/db8 0 2024-04-20T11:09:40.397 INFO:tasks.workunit.client.1.smithi173.stdout:4/737: write da/dc/fb7 [1019099,31079] 0 2024-04-20T11:09:40.401 INFO:tasks.workunit.client.1.smithi173.stdout:1/601: truncate d5/d6/d17/d42/da/d20/dad/dc2/f59 3441302 0 2024-04-20T11:09:40.403 INFO:tasks.workunit.client.1.smithi173.stdout:5/667: fdatasync da/d4f/d65/fe3 0 2024-04-20T11:09:40.407 INFO:tasks.workunit.client.1.smithi173.stdout:6/647: truncate d4/d14/f2f 1116744 0 2024-04-20T11:09:40.411 INFO:tasks.workunit.client.1.smithi173.stdout:4/738: chown da/dc/d13/l5d 2912 1 2024-04-20T11:09:40.414 INFO:tasks.workunit.client.1.smithi173.stdout:6/648: stat d4/d24/d42/d74/c85 0 2024-04-20T11:09:40.415 INFO:tasks.workunit.client.1.smithi173.stdout:0/623: creat d6/d8/df/d15/fc6 x:0 0 0 2024-04-20T11:09:40.416 INFO:tasks.workunit.client.1.smithi173.stdout:5/668: stat da/de/d19/d32/c27 0 2024-04-20T11:09:40.421 INFO:tasks.workunit.client.1.smithi173.stdout:3/650: dwrite d7/d2b/f62 [0,4194304] 0 2024-04-20T11:09:40.423 INFO:tasks.workunit.client.1.smithi173.stdout:8/662: dwrite d4/d18/d59/d5a/d7b/d8c/f8e [0,4194304] 0 2024-04-20T11:09:40.423 INFO:tasks.workunit.client.1.smithi173.stdout:6/649: write d4/d1c/fa1 [985375,54075] 0 2024-04-20T11:09:40.424 INFO:tasks.workunit.client.1.smithi173.stdout:5/669: read da/d9e/fa0 [4180874,94784] 0 2024-04-20T11:09:40.424 INFO:tasks.workunit.client.1.smithi173.stdout:1/602: creat d5/d6/d17/d42/da/df/d40/dc8/fcc x:0 0 0 2024-04-20T11:09:40.427 INFO:tasks.workunit.client.1.smithi173.stdout:9/624: dread d5/d9/d20/d29/d2d/d33/f6b [0,4194304] 0 2024-04-20T11:09:40.427 INFO:tasks.workunit.client.1.smithi173.stdout:4/739: write da/dc/df4/d53/d39/f56 [1001089,84959] 0 2024-04-20T11:09:40.434 INFO:tasks.workunit.client.1.smithi173.stdout:5/670: fsync da/de/d75/f89 0 2024-04-20T11:09:40.437 INFO:tasks.workunit.client.1.smithi173.stdout:3/651: creat d7/d2b/d8e/fd5 x:0 0 0 2024-04-20T11:09:40.438 INFO:tasks.workunit.client.1.smithi173.stdout:2/735: dread d2/d34/d58/f5a [0,4194304] 0 2024-04-20T11:09:40.439 INFO:tasks.workunit.client.1.smithi173.stdout:5/671: dread - da/de/fe2 zero size 2024-04-20T11:09:40.440 INFO:tasks.workunit.client.1.smithi173.stdout:7/653: truncate d9/d33/fcc 1909315 0 2024-04-20T11:09:40.441 INFO:tasks.workunit.client.1.smithi173.stdout:4/740: mkdir da/dd1/df5 0 2024-04-20T11:09:40.442 INFO:tasks.workunit.client.1.smithi173.stdout:9/625: symlink d5/d9/d20/d29/d18/d8a/lbd 0 2024-04-20T11:09:40.446 INFO:tasks.workunit.client.1.smithi173.stdout:8/663: write d4/d18/d4f/f96 [81015,84140] 0 2024-04-20T11:09:40.446 INFO:tasks.workunit.client.1.smithi173.stdout:6/650: rename d4/d24/f6e to d4/d14/fce 0 2024-04-20T11:09:40.450 INFO:tasks.workunit.client.1.smithi173.stdout:6/651: stat d4/d24/d42/d74/f53 0 2024-04-20T11:09:40.455 INFO:tasks.workunit.client.1.smithi173.stdout:0/624: dwrite d6/d54/d65/f73 [0,4194304] 0 2024-04-20T11:09:40.456 INFO:tasks.workunit.client.1.smithi173.stdout:3/652: mkdir d7/d17/d61/dd6 0 2024-04-20T11:09:40.459 INFO:tasks.workunit.client.1.smithi173.stdout:7/654: truncate d9/d1f/d7f/dce/db2/fd8 691771 0 2024-04-20T11:09:40.459 INFO:tasks.workunit.client.1.smithi173.stdout:2/736: mknod d2/d13/d28/d4a/cf3 0 2024-04-20T11:09:40.463 INFO:tasks.workunit.client.1.smithi173.stdout:1/603: dwrite d5/d6/d17/d42/da/df/d40/f5d [0,4194304] 0 2024-04-20T11:09:40.465 INFO:tasks.workunit.client.1.smithi173.stdout:4/741: creat da/d1e/d52/ff6 x:0 0 0 2024-04-20T11:09:40.467 INFO:tasks.workunit.client.1.smithi173.stdout:6/652: mknod d4/d14/ccf 0 2024-04-20T11:09:40.470 INFO:tasks.workunit.client.1.smithi173.stdout:9/626: write f0 [893749,105142] 0 2024-04-20T11:09:40.471 INFO:tasks.workunit.client.1.smithi173.stdout:8/664: creat d4/fd5 x:0 0 0 2024-04-20T11:09:40.477 INFO:tasks.workunit.client.1.smithi173.stdout:0/625: creat d6/d8/df/d2c/d42/d50/d86/d8b/fc7 x:0 0 0 2024-04-20T11:09:40.482 INFO:tasks.workunit.client.1.smithi173.stdout:1/604: write d5/d6/d17/d42/f19 [4067798,49227] 0 2024-04-20T11:09:40.488 INFO:tasks.workunit.client.1.smithi173.stdout:3/653: dwrite d7/d17/d61/f9b [0,4194304] 0 2024-04-20T11:09:40.492 INFO:tasks.workunit.client.1.smithi173.stdout:9/627: rename d5/d9/d20/d2b/d59/d44/d78/d9d/faa to d5/d9/d20/d2b/fbe 0 2024-04-20T11:09:40.493 INFO:tasks.workunit.client.1.smithi173.stdout:0/626: creat d6/d8/d3c/fc8 x:0 0 0 2024-04-20T11:09:40.496 INFO:tasks.workunit.client.1.smithi173.stdout:9/628: chown d5/d9/d20/d2b/fbe 52 1 2024-04-20T11:09:40.499 INFO:tasks.workunit.client.1.smithi173.stdout:5/672: dwrite da/de/d19/dca/f6e [0,4194304] 0 2024-04-20T11:09:40.500 INFO:tasks.workunit.client.1.smithi173.stdout:1/605: dread f0 [0,4194304] 0 2024-04-20T11:09:40.506 INFO:tasks.workunit.client.1.smithi173.stdout:1/606: dread d5/d6/d17/d42/d12/d1c/d85/f9e [0,4194304] 0 2024-04-20T11:09:40.506 INFO:tasks.workunit.client.1.smithi173.stdout:7/655: dwrite d9/f1d [0,4194304] 0 2024-04-20T11:09:40.513 INFO:tasks.workunit.client.1.smithi173.stdout:4/742: dread da/dc/df4/d53/f5b [0,4194304] 0 2024-04-20T11:09:40.513 INFO:tasks.workunit.client.1.smithi173.stdout:0/627: rmdir d6/d54/d91 39 2024-04-20T11:09:40.516 INFO:tasks.workunit.client.1.smithi173.stdout:9/629: fsync d5/d9/d20/d2b/d59/f4e 0 2024-04-20T11:09:40.520 INFO:tasks.workunit.client.1.smithi173.stdout:5/673: creat da/de/d75/d40/fe9 x:0 0 0 2024-04-20T11:09:40.524 INFO:tasks.workunit.client.1.smithi173.stdout:6/653: rename d4/d14/d5f/c6d to d4/d1c/d8e/cd0 0 2024-04-20T11:09:40.524 INFO:tasks.workunit.client.1.smithi173.stdout:3/654: stat d7/d17/d61/d83/d45/lb0 0 2024-04-20T11:09:40.525 INFO:tasks.workunit.client.1.smithi173.stdout:9/630: write d5/d9/d20/d2b/d59/d44/d78/d9d/fb7 [657162,73758] 0 2024-04-20T11:09:40.528 INFO:tasks.workunit.client.1.smithi173.stdout:2/737: link d2/d13/d28/d4a/l98 d2/lf4 0 2024-04-20T11:09:40.528 INFO:tasks.workunit.client.1.smithi173.stdout:4/743: creat da/d1a/d60/d7d/ff7 x:0 0 0 2024-04-20T11:09:40.531 INFO:tasks.workunit.client.1.smithi173.stdout:0/628: creat d6/d5f/d8f/fc9 x:0 0 0 2024-04-20T11:09:40.532 INFO:tasks.workunit.client.1.smithi173.stdout:3/655: dread d7/d17/d61/d83/fc8 [0,4194304] 0 2024-04-20T11:09:40.534 INFO:tasks.workunit.client.1.smithi173.stdout:8/665: dwrite d4/ff [0,4194304] 0 2024-04-20T11:09:40.534 INFO:tasks.workunit.client.1.smithi173.stdout:6/654: write d4/d24/d42/f67 [1606310,74334] 0 2024-04-20T11:09:40.535 INFO:tasks.workunit.client.1.smithi173.stdout:4/744: chown da/dc/d13/d65/f98 422366071 1 2024-04-20T11:09:40.537 INFO:tasks.workunit.client.1.smithi173.stdout:9/631: symlink d5/d9/d20/d2b/d31/lbf 0 2024-04-20T11:09:40.540 INFO:tasks.workunit.client.1.smithi173.stdout:6/655: dread - d4/d18/fc4 zero size 2024-04-20T11:09:40.540 INFO:tasks.workunit.client.1.smithi173.stdout:3/656: dread - d7/d17/d61/f78 zero size 2024-04-20T11:09:40.541 INFO:tasks.workunit.client.1.smithi173.stdout:8/666: chown d4/d18/d4f/f7d 1 1 2024-04-20T11:09:40.541 INFO:tasks.workunit.client.1.smithi173.stdout:0/629: chown d6/d8/d51/fc5 454 1 2024-04-20T11:09:40.544 INFO:tasks.workunit.client.1.smithi173.stdout:7/656: getdents d9/d33/d35 0 2024-04-20T11:09:40.546 INFO:tasks.workunit.client.1.smithi173.stdout:2/738: truncate d2/de/f2b 2777774 0 2024-04-20T11:09:40.547 INFO:tasks.workunit.client.1.smithi173.stdout:3/657: chown d7/d17/d61/d83/dcf/f2e 1805 1 2024-04-20T11:09:40.550 INFO:tasks.workunit.client.1.smithi173.stdout:6/656: write d4/d18/d7e/fbe [774769,60583] 0 2024-04-20T11:09:40.550 INFO:tasks.workunit.client.1.smithi173.stdout:4/745: creat da/dc/d62/ff8 x:0 0 0 2024-04-20T11:09:40.550 INFO:tasks.workunit.client.1.smithi173.stdout:5/674: link da/de/d19/d32/d30/c39 da/d34/d7f/da5/cea 0 2024-04-20T11:09:40.552 INFO:tasks.workunit.client.1.smithi173.stdout:7/657: mkdir d9/d33/d35/d4a/def 0 2024-04-20T11:09:40.554 INFO:tasks.workunit.client.1.smithi173.stdout:0/630: unlink d6/d8/d3c/d44/l9c 0 2024-04-20T11:09:40.555 INFO:tasks.workunit.client.1.smithi173.stdout:6/657: fsync d4/d24/d42/fbd 0 2024-04-20T11:09:40.559 INFO:tasks.workunit.client.1.smithi173.stdout:2/739: symlink d2/d13/da5/lf5 0 2024-04-20T11:09:40.561 INFO:tasks.workunit.client.1.smithi173.stdout:1/607: dwrite d5/fb [0,4194304] 0 2024-04-20T11:09:40.562 INFO:tasks.workunit.client.1.smithi173.stdout:2/740: fdatasync d2/d34/fc8 0 2024-04-20T11:09:40.567 INFO:tasks.workunit.client.1.smithi173.stdout:3/658: dread d7/d17/f3c [0,4194304] 0 2024-04-20T11:09:40.571 INFO:tasks.workunit.client.1.smithi173.stdout:9/632: dwrite d5/d9/d20/d4f/f68 [0,4194304] 0 2024-04-20T11:09:40.571 INFO:tasks.workunit.client.1.smithi173.stdout:0/631: mknod d6/d8/df/da0/cca 0 2024-04-20T11:09:40.579 INFO:tasks.workunit.client.1.smithi173.stdout:9/633: write d5/d9/f1a [1316920,97288] 0 2024-04-20T11:09:40.579 INFO:tasks.workunit.client.1.smithi173.stdout:8/667: rename d4/d18/f60 to d4/d18/d59/d6d/db8/fd6 0 2024-04-20T11:09:40.580 INFO:tasks.workunit.client.1.smithi173.stdout:5/675: dwrite da/d4f/d5e/f6d [0,4194304] 0 2024-04-20T11:09:40.582 INFO:tasks.workunit.client.1.smithi173.stdout:4/746: write da/d1a/d60/f6b [5217308,127522] 0 2024-04-20T11:09:40.592 INFO:tasks.workunit.client.1.smithi173.stdout:5/676: chown da/de/d19/d32/lcd 224 1 2024-04-20T11:09:40.592 INFO:tasks.workunit.client.1.smithi173.stdout:3/659: symlink d7/d17/d58/d8a/ld7 0 2024-04-20T11:09:40.595 INFO:tasks.workunit.client.1.smithi173.stdout:2/741: dwrite d2/fe7 [0,4194304] 0 2024-04-20T11:09:40.595 INFO:tasks.workunit.client.1.smithi173.stdout:6/658: creat d4/d24/d42/d74/fd1 x:0 0 0 2024-04-20T11:09:40.599 INFO:tasks.workunit.client.1.smithi173.stdout:5/677: write da/de/f2d [5482426,28913] 0 2024-04-20T11:09:40.599 INFO:tasks.workunit.client.1.smithi173.stdout:3/660: chown d7/d85 26465834 1 2024-04-20T11:09:40.599 INFO:tasks.workunit.client.1.smithi173.stdout:7/658: rename d9/da/fcd to d9/d33/d40/d7a/ff0 0 2024-04-20T11:09:40.607 INFO:tasks.workunit.client.1.smithi173.stdout:3/661: chown d7/d2b/lc0 2 1 2024-04-20T11:09:40.610 INFO:tasks.workunit.client.1.smithi173.stdout:1/608: rename d5/d6/d17/d42/da/d20/d38/d52/d75/f86 to d5/d6/fcd 0 2024-04-20T11:09:40.615 INFO:tasks.workunit.client.1.smithi173.stdout:2/742: creat d2/d8c/ff6 x:0 0 0 2024-04-20T11:09:40.615 INFO:tasks.workunit.client.1.smithi173.stdout:8/668: dwrite d4/d18/fa7 [0,4194304] 0 2024-04-20T11:09:40.616 INFO:tasks.workunit.client.1.smithi173.stdout:4/747: dread da/d1a/d60/f63 [0,4194304] 0 2024-04-20T11:09:40.621 INFO:tasks.workunit.client.1.smithi173.stdout:2/743: dread d2/d13/d46/d4b/fda [0,4194304] 0 2024-04-20T11:09:40.633 INFO:tasks.workunit.client.1.smithi173.stdout:5/678: creat da/de/d19/d32/d72/dc3/feb x:0 0 0 2024-04-20T11:09:40.633 INFO:tasks.workunit.client.1.smithi173.stdout:9/634: write d5/d9/d20/d2b/d31/f4d [2037083,74609] 0 2024-04-20T11:09:40.638 INFO:tasks.workunit.client.1.smithi173.stdout:0/632: dwrite d6/d8/df/d2c/d42/d50/fb5 [0,4194304] 0 2024-04-20T11:09:40.640 INFO:tasks.workunit.client.1.smithi173.stdout:6/659: link d4/d24/d42/d81/f99 d4/d1c/d8e/d90/da4/fd2 0 2024-04-20T11:09:40.641 INFO:tasks.workunit.client.1.smithi173.stdout:0/633: fsync d6/d54/d65/f73 0 2024-04-20T11:09:40.653 INFO:tasks.workunit.client.1.smithi173.stdout:7/659: symlink d9/d1f/d7f/dce/db2/lf1 0 2024-04-20T11:09:40.656 INFO:tasks.workunit.client.1.smithi173.stdout:9/635: creat d5/d4c/fc0 x:0 0 0 2024-04-20T11:09:40.658 INFO:tasks.workunit.client.1.smithi173.stdout:5/679: creat da/de/d19/d32/d30/dbb/fec x:0 0 0 2024-04-20T11:09:40.659 INFO:tasks.workunit.client.1.smithi173.stdout:4/748: creat da/d1a/d60/d7d/dec/ff9 x:0 0 0 2024-04-20T11:09:40.659 INFO:tasks.workunit.client.1.smithi173.stdout:6/660: dread d4/d18/f62 [0,4194304] 0 2024-04-20T11:09:40.662 INFO:tasks.workunit.client.1.smithi173.stdout:3/662: creat d7/d17/d58/fd8 x:0 0 0 2024-04-20T11:09:40.667 INFO:tasks.workunit.client.1.smithi173.stdout:0/634: mknod d6/d8/df/d2c/d42/d50/d86/d8b/da2/ccb 0 2024-04-20T11:09:40.670 INFO:tasks.workunit.client.1.smithi173.stdout:1/609: dread d5/d6/d17/f4b [0,4194304] 0 2024-04-20T11:09:40.671 INFO:tasks.workunit.client.1.smithi173.stdout:9/636: rmdir d5/d9/d20/d2b/d59/da1 39 2024-04-20T11:09:40.671 INFO:tasks.workunit.client.1.smithi173.stdout:5/680: creat da/de/d75/d3e/fed x:0 0 0 2024-04-20T11:09:40.673 INFO:tasks.workunit.client.1.smithi173.stdout:7/660: dread d9/d1f/f7d [0,4194304] 0 2024-04-20T11:09:40.673 INFO:tasks.workunit.client.1.smithi173.stdout:4/749: mknod da/dc/d62/d84/cfa 0 2024-04-20T11:09:40.675 INFO:tasks.workunit.client.1.smithi173.stdout:1/610: chown d5/d6/d17/d42/da/d20/d48/f9c 103471888 1 2024-04-20T11:09:40.676 INFO:tasks.workunit.client.1.smithi173.stdout:7/661: write d9/d33/d35/f44 [5172011,25179] 0 2024-04-20T11:09:40.677 INFO:tasks.workunit.client.1.smithi173.stdout:1/611: readlink d5/d6/dc/d7f/lc4 0 2024-04-20T11:09:40.677 INFO:tasks.workunit.client.1.smithi173.stdout:6/661: creat d4/d24/d3e/fd3 x:0 0 0 2024-04-20T11:09:40.678 INFO:tasks.workunit.client.1.smithi173.stdout:3/663: readlink d7/d17/d61/d83/d45/lbb 0 2024-04-20T11:09:40.683 INFO:tasks.workunit.client.1.smithi173.stdout:2/744: write d2/d13/d46/d7d/f91 [110729,59384] 0 2024-04-20T11:09:40.692 INFO:tasks.workunit.client.1.smithi173.stdout:4/750: mknod da/d1e/cfb 0 2024-04-20T11:09:40.692 INFO:tasks.workunit.client.1.smithi173.stdout:8/669: getdents d4/d18/d59/d6d/db8 0 2024-04-20T11:09:40.694 INFO:tasks.workunit.client.1.smithi173.stdout:7/662: rename d9/da/d31 to d9/d33/d35/d4a/d52/df2 0 2024-04-20T11:09:40.696 INFO:tasks.workunit.client.1.smithi173.stdout:3/664: unlink d7/d17/d58/fd8 0 2024-04-20T11:09:40.698 INFO:tasks.workunit.client.1.smithi173.stdout:9/637: dread d5/d9/fa [0,4194304] 0 2024-04-20T11:09:40.706 INFO:tasks.workunit.client.1.smithi173.stdout:1/612: rmdir d5/d6/d17/d42/da/df/d40/dc8 39 2024-04-20T11:09:40.707 INFO:tasks.workunit.client.1.smithi173.stdout:2/745: symlink d2/d34/d58/lf7 0 2024-04-20T11:09:40.708 INFO:tasks.workunit.client.1.smithi173.stdout:7/663: fsync d9/d1f/d7f/dce/dad/fb9 0 2024-04-20T11:09:40.712 INFO:tasks.workunit.client.1.smithi173.stdout:8/670: dwrite d4/d18/d59/d6d/db8/d3c/f52 [4194304,4194304] 0 2024-04-20T11:09:40.712 INFO:tasks.workunit.client.1.smithi173.stdout:3/665: creat d7/d17/d61/d83/dcf/d5c/fd9 x:0 0 0 2024-04-20T11:09:40.714 INFO:tasks.workunit.client.1.smithi173.stdout:9/638: mknod d5/d9/d20/d2b/d59/d44/d78/d82/db5/cc1 0 2024-04-20T11:09:40.718 INFO:tasks.workunit.client.1.smithi173.stdout:3/666: mknod d7/d17/d61/dd6/cda 0 2024-04-20T11:09:40.721 INFO:tasks.workunit.client.1.smithi173.stdout:8/671: mknod d4/cd7 0 2024-04-20T11:09:40.721 INFO:tasks.workunit.client.1.smithi173.stdout:1/613: truncate d5/d6/d17/d42/d12/d1c/d85/f9e 509013 0 2024-04-20T11:09:40.726 INFO:tasks.workunit.client.1.smithi173.stdout:8/672: read d4/d18/f48 [1677090,83245] 0 2024-04-20T11:09:40.731 INFO:tasks.workunit.client.1.smithi173.stdout:7/664: dread d9/d33/f47 [0,4194304] 0 2024-04-20T11:09:40.733 INFO:tasks.workunit.client.1.smithi173.stdout:9/639: dread d5/d9/f47 [0,4194304] 0 2024-04-20T11:09:40.736 INFO:tasks.workunit.client.1.smithi173.stdout:8/673: chown d4/d18/d59/d5a/d7b/d8c/l90 1215209 1 2024-04-20T11:09:40.738 INFO:tasks.workunit.client.1.smithi173.stdout:5/681: write da/de/d75/d40/f42 [444171,88618] 0 2024-04-20T11:09:40.740 INFO:tasks.workunit.client.1.smithi173.stdout:2/746: dwrite d2/d13/d9a/fdf [0,4194304] 0 2024-04-20T11:09:40.740 INFO:tasks.workunit.client.1.smithi173.stdout:9/640: readlink d5/d9/d20/d4f/lac 0 2024-04-20T11:09:40.743 INFO:tasks.workunit.client.1.smithi173.stdout:8/674: read - d4/d18/d59/d5a/f95 zero size 2024-04-20T11:09:40.745 INFO:tasks.workunit.client.1.smithi173.stdout:7/665: mkdir d9/d33/d35/d4a/d52/df2/dc6/df3 0 2024-04-20T11:09:40.746 INFO:tasks.workunit.client.1.smithi173.stdout:9/641: mkdir d5/d9/d20/d4f/d5e/d9e/dc2 0 2024-04-20T11:09:40.747 INFO:tasks.workunit.client.1.smithi173.stdout:0/635: dwrite d6/d8/f12 [0,4194304] 0 2024-04-20T11:09:40.747 INFO:tasks.workunit.client.1.smithi173.stdout:2/747: creat d2/d34/ff8 x:0 0 0 2024-04-20T11:09:40.747 INFO:tasks.workunit.client.1.smithi173.stdout:4/751: read - da/d1e/d52/f68 zero size 2024-04-20T11:09:40.752 INFO:tasks.workunit.client.1.smithi173.stdout:9/642: write d5/f6 [3729511,22960] 0 2024-04-20T11:09:40.753 INFO:tasks.workunit.client.1.smithi173.stdout:6/662: truncate d4/d24/f2e 3369916 0 2024-04-20T11:09:40.755 INFO:tasks.workunit.client.1.smithi173.stdout:4/752: write da/d1a/d60/fa4 [960657,27143] 0 2024-04-20T11:09:40.756 INFO:tasks.workunit.client.1.smithi173.stdout:3/667: dread d7/d17/d58/d7f/fba [0,4194304] 0 2024-04-20T11:09:40.756 INFO:tasks.workunit.client.1.smithi173.stdout:2/748: chown d2/d13/d46/f73 0 1 2024-04-20T11:09:40.757 INFO:tasks.workunit.client.1.smithi173.stdout:5/682: rename da/d4f/d65/f86 to da/d4f/d65/fee 0 2024-04-20T11:09:40.762 INFO:tasks.workunit.client.1.smithi173.stdout:4/753: stat da/dc/df4/d53 0 2024-04-20T11:09:40.764 INFO:tasks.workunit.client.1.smithi173.stdout:6/663: mknod d4/d14/d79/cd4 0 2024-04-20T11:09:40.771 INFO:tasks.workunit.client.1.smithi173.stdout:8/675: rename d4/d18/d4f/l65 to d4/d18/d59/d5a/d7b/ld8 0 2024-04-20T11:09:40.773 INFO:tasks.workunit.client.1.smithi173.stdout:5/683: creat da/de/d19/d81/fef x:0 0 0 2024-04-20T11:09:40.777 INFO:tasks.workunit.client.1.smithi173.stdout:2/749: rmdir d2/d34/d58 39 2024-04-20T11:09:40.779 INFO:tasks.workunit.client.1.smithi173.stdout:0/636: dwrite d6/d54/f7d [0,4194304] 0 2024-04-20T11:09:40.787 INFO:tasks.workunit.client.1.smithi173.stdout:3/668: creat d7/d17/d61/d83/dcf/d5c/fdb x:0 0 0 2024-04-20T11:09:40.787 INFO:tasks.workunit.client.1.smithi173.stdout:6/664: stat d4/d14/d5f/c64 0 2024-04-20T11:09:40.788 INFO:tasks.workunit.client.1.smithi173.stdout:4/754: symlink da/d1a/d60/d66/d9b/lfc 0 2024-04-20T11:09:40.789 INFO:tasks.workunit.client.1.smithi173.stdout:0/637: mkdir d6/d8/df/d2c/d42/d50/d86/d8b/dcc 0 2024-04-20T11:09:40.792 INFO:tasks.workunit.client.1.smithi173.stdout:2/750: mkdir d2/d13/d46/d4b/dae/db6/df9 0 2024-04-20T11:09:40.793 INFO:tasks.workunit.client.1.smithi173.stdout:8/676: dwrite d4/d18/d59/f64 [0,4194304] 0 2024-04-20T11:09:40.794 INFO:tasks.workunit.client.1.smithi173.stdout:5/684: creat da/de/ff0 x:0 0 0 2024-04-20T11:09:40.794 INFO:tasks.workunit.client.1.smithi173.stdout:6/665: symlink d4/d24/d42/d74/d3b/ld5 0 2024-04-20T11:09:40.796 INFO:tasks.workunit.client.1.smithi173.stdout:2/751: stat d2/d13/d28/ccb 0 2024-04-20T11:09:40.808 INFO:tasks.workunit.client.1.smithi173.stdout:2/752: truncate d2/d8c/ff6 781329 0 2024-04-20T11:09:40.810 INFO:tasks.workunit.client.1.smithi173.stdout:8/677: creat d4/d18/d59/d5a/d7b/db1/fd9 x:0 0 0 2024-04-20T11:09:40.811 INFO:tasks.workunit.client.1.smithi173.stdout:6/666: creat d4/d18/d7e/fd6 x:0 0 0 2024-04-20T11:09:40.812 INFO:tasks.workunit.client.1.smithi173.stdout:5/685: fsync da/de/f5b 0 2024-04-20T11:09:40.817 INFO:tasks.workunit.client.1.smithi173.stdout:5/686: read da/d4f/d5e/f6d [401591,4730] 0 2024-04-20T11:09:40.820 INFO:tasks.workunit.client.1.smithi173.stdout:9/643: dread d5/d9/f26 [0,4194304] 0 2024-04-20T11:09:40.836 INFO:tasks.workunit.client.1.smithi173.stdout:2/753: truncate d2/d34/f3a 4032776 0 2024-04-20T11:09:40.837 INFO:tasks.workunit.client.1.smithi173.stdout:6/667: readlink d4/d24/d42/lb6 0 2024-04-20T11:09:40.837 INFO:tasks.workunit.client.1.smithi173.stdout:2/754: truncate d2/f64 5011594 0 2024-04-20T11:09:40.838 INFO:tasks.workunit.client.1.smithi173.stdout:8/678: fsync d4/d18/d59/d6d/db8/d3c/f34 0 2024-04-20T11:09:40.839 INFO:tasks.workunit.client.1.smithi173.stdout:5/687: truncate da/d9e/fa0 2149726 0 2024-04-20T11:09:40.841 INFO:tasks.workunit.client.1.smithi173.stdout:5/688: chown da/de/d75/d40/d59/l8b 0 1 2024-04-20T11:09:40.843 INFO:tasks.workunit.client.1.smithi173.stdout:6/668: rmdir d4/d1c 39 2024-04-20T11:09:40.844 INFO:tasks.workunit.client.1.smithi173.stdout:2/755: rmdir d2/d13/d28 39 2024-04-20T11:09:40.845 INFO:tasks.workunit.client.1.smithi173.stdout:6/669: chown d4/d24/d42/d74/d3b/f52 2869298 1 2024-04-20T11:09:40.846 INFO:tasks.workunit.client.1.smithi173.stdout:9/644: mknod d5/d9/d20/d29/d2d/cc3 0 2024-04-20T11:09:40.848 INFO:tasks.workunit.client.1.smithi173.stdout:5/689: getdents da/de/d19/dca/d48/da2/de0 0 2024-04-20T11:09:40.850 INFO:tasks.workunit.client.1.smithi173.stdout:6/670: fsync d4/d24/d42/d74/d3b/f41 0 2024-04-20T11:09:40.853 INFO:tasks.workunit.client.1.smithi173.stdout:5/690: creat da/de/d19/d32/ff1 x:0 0 0 2024-04-20T11:09:40.857 INFO:tasks.workunit.client.1.smithi173.stdout:6/671: truncate d4/d1c/f7d 357185 0 2024-04-20T11:09:40.859 INFO:tasks.workunit.client.1.smithi173.stdout:9/645: dwrite d5/d9/d20/d29/f5c [0,4194304] 0 2024-04-20T11:09:40.871 INFO:tasks.workunit.client.1.smithi173.stdout:5/691: getdents da/de/d19/d32/d66/dbd 0 2024-04-20T11:09:40.876 INFO:tasks.workunit.client.1.smithi173.stdout:9/646: dwrite d5/faf [0,4194304] 0 2024-04-20T11:09:40.876 INFO:tasks.workunit.client.1.smithi173.stdout:2/756: write d2/de/d11/d9d/f8e [177636,79695] 0 2024-04-20T11:09:40.878 INFO:tasks.workunit.client.1.smithi173.stdout:2/757: stat d2/d13/d9a/dbc 0 2024-04-20T11:09:40.880 INFO:tasks.workunit.client.1.smithi173.stdout:6/672: write d4/d1c/d8e/d90/da4/fd2 [535117,63443] 0 2024-04-20T11:09:40.885 INFO:tasks.workunit.client.1.smithi173.stdout:2/758: symlink d2/lfa 0 2024-04-20T11:09:40.887 INFO:tasks.workunit.client.1.smithi173.stdout:6/673: mkdir d4/d59/d8f/dd7 0 2024-04-20T11:09:40.888 INFO:tasks.workunit.client.1.smithi173.stdout:9/647: mkdir d5/d9/d20/d29/d18/d9b/d98/dc4 0 2024-04-20T11:09:40.890 INFO:tasks.workunit.client.1.smithi173.stdout:2/759: fdatasync d2/de/d11/d9d/d87/fb4 0 2024-04-20T11:09:40.891 INFO:tasks.workunit.client.1.smithi173.stdout:6/674: mknod d4/cd8 0 2024-04-20T11:09:40.893 INFO:tasks.workunit.client.1.smithi173.stdout:5/692: dwrite da/de/d19/d32/d72/f91 [0,4194304] 0 2024-04-20T11:09:40.894 INFO:tasks.workunit.client.1.smithi173.stdout:9/648: fsync d5/d9/d20/d29/d2d/f87 0 2024-04-20T11:09:40.897 INFO:tasks.workunit.client.1.smithi173.stdout:6/675: mkdir d4/d59/dd9 0 2024-04-20T11:09:40.897 INFO:tasks.workunit.client.1.smithi173.stdout:5/693: fsync da/de/d75/d40/fe9 0 2024-04-20T11:09:40.897 INFO:tasks.workunit.client.1.smithi173.stdout:9/649: fdatasync d5/d9/d20/d29/d18/f70 0 2024-04-20T11:09:40.907 INFO:tasks.workunit.client.1.smithi173.stdout:5/694: mknod da/d4f/d5e/dbc/cf2 0 2024-04-20T11:09:40.910 INFO:tasks.workunit.client.1.smithi173.stdout:6/676: creat d4/fda x:0 0 0 2024-04-20T11:09:40.912 INFO:tasks.workunit.client.1.smithi173.stdout:9/650: creat d5/d9/fc5 x:0 0 0 2024-04-20T11:09:40.913 INFO:tasks.workunit.client.1.smithi173.stdout:5/695: rmdir da/d34/da7 39 2024-04-20T11:09:40.916 INFO:tasks.workunit.client.1.smithi173.stdout:6/677: mknod d4/d18/cdb 0 2024-04-20T11:09:40.918 INFO:tasks.workunit.client.1.smithi173.stdout:2/760: truncate d2/d13/d46/f85 2915517 0 2024-04-20T11:09:40.920 INFO:tasks.workunit.client.1.smithi173.stdout:2/761: write d2/de/d11/f10 [1170556,72994] 0 2024-04-20T11:09:40.922 INFO:tasks.workunit.client.1.smithi173.stdout:9/651: write d5/d9/d20/d29/f11 [2219388,17975] 0 2024-04-20T11:09:40.923 INFO:tasks.workunit.client.1.smithi173.stdout:9/652: fsync d5/d9/f47 0 2024-04-20T11:09:40.924 INFO:tasks.workunit.client.1.smithi173.stdout:2/762: creat d2/d13/d46/d53/d9f/ffb x:0 0 0 2024-04-20T11:09:40.924 INFO:tasks.workunit.client.1.smithi173.stdout:5/696: write da/d4f/d5e/f6d [2896904,107954] 0 2024-04-20T11:09:40.927 INFO:tasks.workunit.client.1.smithi173.stdout:9/653: rmdir d5/d9/d20/d29/d18 39 2024-04-20T11:09:40.933 INFO:tasks.workunit.client.1.smithi173.stdout:9/654: symlink d5/d4c/lc6 0 2024-04-20T11:09:40.934 INFO:tasks.workunit.client.1.smithi173.stdout:2/763: dread d2/d34/d58/f69 [4194304,4194304] 0 2024-04-20T11:09:40.935 INFO:tasks.workunit.client.1.smithi173.stdout:5/697: write da/de/d19/dca/d48/f67 [3233150,22140] 0 2024-04-20T11:09:40.939 INFO:tasks.workunit.client.1.smithi173.stdout:5/698: truncate da/de/d19/d32/f44 4572050 0 2024-04-20T11:09:40.940 INFO:tasks.workunit.client.1.smithi173.stdout:2/764: dread d2/d13/d28/d4a/f6c [0,4194304] 0 2024-04-20T11:09:40.942 INFO:tasks.workunit.client.1.smithi173.stdout:2/765: symlink d2/d13/d46/d53/d4e/lfc 0 2024-04-20T11:09:40.962 INFO:tasks.workunit.client.1.smithi173.stdout:2/766: dread d2/d13/f74 [4194304,4194304] 0 2024-04-20T11:09:40.976 INFO:tasks.workunit.client.1.smithi173.stdout:2/767: dwrite d2/de/d11/d9d/d87/fb4 [8388608,4194304] 0 2024-04-20T11:09:40.985 INFO:tasks.workunit.client.1.smithi173.stdout:2/768: rename d2/d13/d46 to d2/d13/d9a/dbc/dfd 0 2024-04-20T11:09:40.987 INFO:tasks.workunit.client.1.smithi173.stdout:2/769: write d2/d8c/ff6 [1246511,121888] 0 2024-04-20T11:09:41.005 INFO:tasks.workunit.client.1.smithi173.stdout:2/770: write d2/d13/d9a/dbc/dfd/d53/d4e/f68 [1832975,8969] 0 2024-04-20T11:09:41.009 INFO:tasks.workunit.client.1.smithi173.stdout:2/771: chown d2/de/d11/c49 75357186 1 2024-04-20T11:09:41.013 INFO:tasks.workunit.client.1.smithi173.stdout:2/772: mknod d2/d13/d9a/dbc/dfd/d4b/cfe 0 2024-04-20T11:09:41.016 INFO:tasks.workunit.client.1.smithi173.stdout:2/773: dread - d2/d13/d9a/dbc/dfd/d53/fbd zero size 2024-04-20T11:09:41.020 INFO:tasks.workunit.client.1.smithi173.stdout:2/774: truncate d2/d34/d81/f9e 1292648 0 2024-04-20T11:09:41.027 INFO:tasks.workunit.client.1.smithi173.stdout:2/775: mkdir d2/d13/d28/d4a/dff 0 2024-04-20T11:09:41.042 INFO:tasks.workunit.client.1.smithi173.stdout:2/776: dread d2/d8c/ff6 [0,4194304] 0 2024-04-20T11:09:41.074 INFO:tasks.workunit.client.1.smithi173.stdout:2/777: dwrite d2/d13/f6d [0,4194304] 0 2024-04-20T11:09:41.080 INFO:tasks.workunit.client.1.smithi173.stdout:2/778: write d2/de/d11/d9d/d87/fb4 [10849089,93611] 0 2024-04-20T11:09:41.093 INFO:tasks.workunit.client.1.smithi173.stdout:2/779: unlink d2/d34/d81/dca/ld9 0 2024-04-20T11:09:41.098 INFO:tasks.workunit.client.1.smithi173.stdout:2/780: symlink d2/d13/d9a/dbc/dfd/d7d/l100 0 2024-04-20T11:09:41.101 INFO:tasks.workunit.client.1.smithi173.stdout:2/781: stat d2/de/d11/db8 0 2024-04-20T11:09:41.105 INFO:tasks.workunit.client.1.smithi173.stdout:2/782: stat d2/d13/d9a/dbc/dfd/d53/fbd 0 2024-04-20T11:09:41.113 INFO:tasks.workunit.client.1.smithi173.stdout:2/783: mkdir d2/d13/da5/d101 0 2024-04-20T11:09:41.117 INFO:tasks.workunit.client.1.smithi173.stdout:2/784: write d2/de/d11/d9d/d87/f97 [67469,507] 0 2024-04-20T11:09:41.124 INFO:tasks.workunit.client.1.smithi173.stdout:2/785: fsync d2/d34/d58/f6b 0 2024-04-20T11:09:41.137 INFO:tasks.workunit.client.1.smithi173.stdout:2/786: link d2/d34/d58/fd3 d2/d13/d9a/dbc/f102 0 2024-04-20T11:09:41.147 INFO:tasks.workunit.client.1.smithi173.stdout:2/787: mknod d2/d13/d9a/dbc/dfd/d4b/dc3/c103 0 2024-04-20T11:09:41.161 INFO:tasks.workunit.client.1.smithi173.stdout:2/788: write d2/d13/d9a/dbc/f102 [879435,91963] 0 2024-04-20T11:09:41.166 INFO:tasks.workunit.client.1.smithi173.stdout:1/614: sync 2024-04-20T11:09:41.171 INFO:tasks.workunit.client.1.smithi173.stdout:2/789: unlink d2/d13/d9a/dbc/dfd/d53/l39 0 2024-04-20T11:09:41.173 INFO:tasks.workunit.client.1.smithi173.stdout:1/615: rmdir d5/d6/d17/d42/da/d20/d38 39 2024-04-20T11:09:41.176 INFO:tasks.workunit.client.1.smithi173.stdout:1/616: fdatasync f0 0 2024-04-20T11:09:41.182 INFO:tasks.workunit.client.1.smithi173.stdout:1/617: fsync d5/d6/d17/d42/d12/d1c/f7d 0 2024-04-20T11:09:41.189 INFO:tasks.workunit.client.1.smithi173.stdout:1/618: chown d5/d6/d17/d42/da/d20/dad/dc2/fc5 110 1 2024-04-20T11:09:41.200 INFO:tasks.workunit.client.1.smithi173.stdout:2/790: write d2/d13/d9a/dbc/dfd/d53/d9f/fc9 [3486712,59623] 0 2024-04-20T11:09:41.202 INFO:tasks.workunit.client.1.smithi173.stdout:1/619: creat d5/d6/d17/d42/da/df/fce x:0 0 0 2024-04-20T11:09:41.206 INFO:tasks.workunit.client.1.smithi173.stdout:2/791: chown d2/de/c72 393 1 2024-04-20T11:09:41.213 INFO:tasks.workunit.client.1.smithi173.stdout:2/792: creat d2/f104 x:0 0 0 2024-04-20T11:09:41.213 INFO:tasks.workunit.client.1.smithi173.stdout:1/620: read d5/d6/d17/d42/da/d20/dad/dc2/f59 [553451,83771] 0 2024-04-20T11:09:41.218 INFO:tasks.workunit.client.1.smithi173.stdout:2/793: symlink d2/l105 0 2024-04-20T11:09:41.220 INFO:tasks.workunit.client.1.smithi173.stdout:1/621: stat d5/d6/dc/d7f/l8f 0 2024-04-20T11:09:41.223 INFO:tasks.workunit.client.1.smithi173.stdout:2/794: write d2/de/f21 [2548569,125809] 0 2024-04-20T11:09:41.232 INFO:tasks.workunit.client.1.smithi173.stdout:2/795: creat d2/de/f106 x:0 0 0 2024-04-20T11:09:41.236 INFO:tasks.workunit.client.1.smithi173.stdout:1/622: dread d5/d6/d17/d42/da/d20/dad/dc2/f55 [0,4194304] 0 2024-04-20T11:09:41.244 INFO:tasks.workunit.client.1.smithi173.stdout:1/623: rmdir d5/d6/d89 0 2024-04-20T11:09:41.252 INFO:tasks.workunit.client.1.smithi173.stdout:1/624: creat d5/d6/d17/d93/db6/fcf x:0 0 0 2024-04-20T11:09:41.255 INFO:tasks.workunit.client.1.smithi173.stdout:4/755: sync 2024-04-20T11:09:41.255 INFO:tasks.workunit.client.1.smithi173.stdout:7/666: sync 2024-04-20T11:09:41.255 INFO:tasks.workunit.client.1.smithi173.stdout:0/638: sync 2024-04-20T11:09:41.255 INFO:tasks.workunit.client.1.smithi173.stdout:3/669: sync 2024-04-20T11:09:41.261 INFO:tasks.workunit.client.1.smithi173.stdout:1/625: creat d5/d6/dc/d7f/fd0 x:0 0 0 2024-04-20T11:09:41.263 INFO:tasks.workunit.client.1.smithi173.stdout:7/667: chown d9/d1f/d7f/dce/dad/fb9 4597 1 2024-04-20T11:09:41.270 INFO:tasks.workunit.client.1.smithi173.stdout:0/639: dread d6/d8/d51/f71 [0,4194304] 0 2024-04-20T11:09:41.272 INFO:tasks.workunit.client.1.smithi173.stdout:2/796: dread d2/d13/d9a/dbc/dfd/d53/d4e/f8d [0,4194304] 0 2024-04-20T11:09:41.275 INFO:tasks.workunit.client.1.smithi173.stdout:4/756: creat da/d1e/ffd x:0 0 0 2024-04-20T11:09:41.281 INFO:tasks.workunit.client.1.smithi173.stdout:3/670: write d7/d17/d61/d83/dcf/f2e [170955,16318] 0 2024-04-20T11:09:41.286 INFO:tasks.workunit.client.1.smithi173.stdout:7/668: write d9/d33/d85/fa6 [1537636,38599] 0 2024-04-20T11:09:41.290 INFO:tasks.workunit.client.1.smithi173.stdout:2/797: mknod d2/d13/d28/d4a/c107 0 2024-04-20T11:09:41.290 INFO:tasks.workunit.client.1.smithi173.stdout:1/626: truncate d5/fb 191453 0 2024-04-20T11:09:41.290 INFO:tasks.workunit.client.1.smithi173.stdout:0/640: rename d6/d8/df/d2c/d42/d50/fb1 to d6/d54/d91/fcd 0 2024-04-20T11:09:41.292 INFO:tasks.workunit.client.1.smithi173.stdout:2/798: write d2/de/f84 [214132,79861] 0 2024-04-20T11:09:41.296 INFO:tasks.workunit.client.1.smithi173.stdout:0/641: write d6/d8/f1f [5183799,95707] 0 2024-04-20T11:09:41.297 INFO:tasks.workunit.client.1.smithi173.stdout:5/699: sync 2024-04-20T11:09:41.298 INFO:tasks.workunit.client.1.smithi173.stdout:6/678: sync 2024-04-20T11:09:41.299 INFO:tasks.workunit.client.1.smithi173.stdout:7/669: mkdir d9/da/d17/d2b/daa/dd9/df4 0 2024-04-20T11:09:41.299 INFO:tasks.workunit.client.1.smithi173.stdout:9/655: sync 2024-04-20T11:09:41.299 INFO:tasks.workunit.client.1.smithi173.stdout:8/679: sync 2024-04-20T11:09:41.301 INFO:tasks.workunit.client.1.smithi173.stdout:1/627: rmdir d5/d6/d17/d42/da/d20 39 2024-04-20T11:09:41.306 INFO:tasks.workunit.client.1.smithi173.stdout:2/799: mkdir d2/d13/da5/d108 0 2024-04-20T11:09:41.310 INFO:tasks.workunit.client.1.smithi173.stdout:7/670: creat d9/da/d17/d2b/daa/dd9/ff5 x:0 0 0 2024-04-20T11:09:41.313 INFO:tasks.workunit.client.1.smithi173.stdout:9/656: mkdir d5/d9/d20/d2b/d59/d44/d78/d82/dc7 0 2024-04-20T11:09:41.315 INFO:tasks.workunit.client.1.smithi173.stdout:6/679: rename d4/d59/f84 to d4/d1c/d8e/d90/da4/fdc 0 2024-04-20T11:09:41.315 INFO:tasks.workunit.client.1.smithi173.stdout:7/671: write d9/d1f/d72/f8b [965355,12925] 0 2024-04-20T11:09:41.316 INFO:tasks.workunit.client.1.smithi173.stdout:5/700: symlink da/d4f/d65/dd5/lf3 0 2024-04-20T11:09:41.318 INFO:tasks.workunit.client.1.smithi173.stdout:9/657: fsync d5/d9/d20/d29/d2d/f43 0 2024-04-20T11:09:41.319 INFO:tasks.workunit.client.1.smithi173.stdout:8/680: mknod d4/d18/d59/d5a/d7b/db1/cda 0 2024-04-20T11:09:41.322 INFO:tasks.workunit.client.1.smithi173.stdout:6/680: rmdir d4/d1c/d8e 39 2024-04-20T11:09:41.322 INFO:tasks.workunit.client.1.smithi173.stdout:7/672: fsync d9/d1f/fa1 0 2024-04-20T11:09:41.325 INFO:tasks.workunit.client.1.smithi173.stdout:2/800: mknod d2/d13/d9a/dbc/dfd/d53/d9f/c109 0 2024-04-20T11:09:41.325 INFO:tasks.workunit.client.1.smithi173.stdout:4/757: dwrite da/dc/df4/d53/f3b [0,4194304] 0 2024-04-20T11:09:41.329 INFO:tasks.workunit.client.1.smithi173.stdout:3/671: dread d7/d17/d61/d83/dcf/f66 [4194304,4194304] 0 2024-04-20T11:09:41.330 INFO:tasks.workunit.client.1.smithi173.stdout:7/673: readlink d9/d33/d35/d54/lc1 0 2024-04-20T11:09:41.332 INFO:tasks.workunit.client.1.smithi173.stdout:8/681: write d4/d18/d59/d5a/da8/fae [138310,17893] 0 2024-04-20T11:09:41.333 INFO:tasks.workunit.client.1.smithi173.stdout:6/681: truncate d4/d18/f3f 756213 0 2024-04-20T11:09:41.334 INFO:tasks.workunit.client.1.smithi173.stdout:5/701: creat da/d34/d7f/da5/d9a/dc2/ff4 x:0 0 0 2024-04-20T11:09:41.336 INFO:tasks.workunit.client.1.smithi173.stdout:7/674: creat d9/d33/d91/ff6 x:0 0 0 2024-04-20T11:09:41.337 INFO:tasks.workunit.client.1.smithi173.stdout:0/642: dwrite d6/d8/df/f16 [0,4194304] 0 2024-04-20T11:09:41.339 INFO:tasks.workunit.client.1.smithi173.stdout:9/658: truncate d5/d9/d20/d4f/d5e/d7d/fbb 861489 0 2024-04-20T11:09:41.342 INFO:tasks.workunit.client.1.smithi173.stdout:2/801: mknod d2/d13/d9a/dbc/c10a 0 2024-04-20T11:09:41.344 INFO:tasks.workunit.client.1.smithi173.stdout:6/682: readlink d4/d1c/l7a 0 2024-04-20T11:09:41.351 INFO:tasks.workunit.client.1.smithi173.stdout:4/758: creat da/d1e/d52/d9a/dd3/def/ffe x:0 0 0 2024-04-20T11:09:41.351 INFO:tasks.workunit.client.1.smithi173.stdout:1/628: dread d5/d6/d17/d42/d12/d1c/f8b [0,4194304] 0 2024-04-20T11:09:41.353 INFO:tasks.workunit.client.1.smithi173.stdout:3/672: truncate d7/d17/d61/d83/dcf/d30/f80 710611 0 2024-04-20T11:09:41.365 INFO:tasks.workunit.client.1.smithi173.stdout:7/675: dwrite d9/d1f/d72/d8f/fd1 [0,4194304] 0 2024-04-20T11:09:41.366 INFO:tasks.workunit.client.1.smithi173.stdout:1/629: chown d5/d6/d17/f9a 2764131 1 2024-04-20T11:09:41.368 INFO:tasks.workunit.client.1.smithi173.stdout:4/759: unlink da/dc/d13/c14 0 2024-04-20T11:09:41.368 INFO:tasks.workunit.client.1.smithi173.stdout:8/682: dwrite d4/d18/d59/d6d/db8/d3c/f52 [0,4194304] 0 2024-04-20T11:09:41.369 INFO:tasks.workunit.client.1.smithi173.stdout:5/702: dwrite da/d4f/f5f [0,4194304] 0 2024-04-20T11:09:41.373 INFO:tasks.workunit.client.1.smithi173.stdout:3/673: dread - d7/d17/d61/d83/f88 zero size 2024-04-20T11:09:41.384 INFO:tasks.workunit.client.1.smithi173.stdout:2/802: getdents d2/d13 0 2024-04-20T11:09:41.385 INFO:tasks.workunit.client.1.smithi173.stdout:5/703: write da/de/d19/d32/d72/dc3/feb [363918,112219] 0 2024-04-20T11:09:41.391 INFO:tasks.workunit.client.1.smithi173.stdout:7/676: dread d9/da/f27 [0,4194304] 0 2024-04-20T11:09:41.396 INFO:tasks.workunit.client.1.smithi173.stdout:8/683: creat d4/d18/da6/fdb x:0 0 0 2024-04-20T11:09:41.398 INFO:tasks.workunit.client.1.smithi173.stdout:1/630: symlink d5/d6/d17/d42/da/df/d40/db4/ld1 0 2024-04-20T11:09:41.400 INFO:tasks.workunit.client.1.smithi173.stdout:7/677: write d9/fed [1031010,88737] 0 2024-04-20T11:09:41.402 INFO:tasks.workunit.client.1.smithi173.stdout:8/684: dread - d4/d18/d59/d6d/fba zero size 2024-04-20T11:09:41.403 INFO:tasks.workunit.client.1.smithi173.stdout:4/760: dwrite da/dc/df4/d53/f37 [0,4194304] 0 2024-04-20T11:09:41.406 INFO:tasks.workunit.client.1.smithi173.stdout:2/803: dread d2/d13/d9a/dbc/f102 [0,4194304] 0 2024-04-20T11:09:41.406 INFO:tasks.workunit.client.1.smithi173.stdout:7/678: unlink d9/da/d17/c24 0 2024-04-20T11:09:41.408 INFO:tasks.workunit.client.1.smithi173.stdout:8/685: mkdir d4/d18/d59/d5a/d7b/ddc 0 2024-04-20T11:09:41.411 INFO:tasks.workunit.client.1.smithi173.stdout:2/804: truncate d2/d13/d28/fd7 271287 0 2024-04-20T11:09:41.412 INFO:tasks.workunit.client.1.smithi173.stdout:3/674: dwrite d7/d17/d61/d83/dcf/ff [0,4194304] 0 2024-04-20T11:09:41.417 INFO:tasks.workunit.client.1.smithi173.stdout:1/631: link d5/d6/d17/d42/d12/f47 d5/d6/d17/d42/da/df/d40/db4/fd2 0 2024-04-20T11:09:41.423 INFO:tasks.workunit.client.1.smithi173.stdout:0/643: truncate d6/d8/df/d2c/d42/f7a 1371800 0 2024-04-20T11:09:41.424 INFO:tasks.workunit.client.1.smithi173.stdout:7/679: write d9/d33/f48 [2545278,43904] 0 2024-04-20T11:09:41.425 INFO:tasks.workunit.client.1.smithi173.stdout:1/632: readlink d5/d6/d17/d42/d12/d1c/l45 0 2024-04-20T11:09:41.435 INFO:tasks.workunit.client.1.smithi173.stdout:8/686: truncate d4/d18/fd2 5452 0 2024-04-20T11:09:41.435 INFO:tasks.workunit.client.1.smithi173.stdout:5/704: dread da/d4f/d65/d8e/f8d [0,4194304] 0 2024-04-20T11:09:41.438 INFO:tasks.workunit.client.1.smithi173.stdout:8/687: fdatasync d4/f9f 0 2024-04-20T11:09:41.439 INFO:tasks.workunit.client.1.smithi173.stdout:2/805: symlink d2/d13/d9a/dbc/dfd/l10b 0 2024-04-20T11:09:41.441 INFO:tasks.workunit.client.1.smithi173.stdout:7/680: mknod d9/d33/d35/d4a/d52/df2/cf7 0 2024-04-20T11:09:41.441 INFO:tasks.workunit.client.1.smithi173.stdout:4/761: dwrite da/d1a/d60/d7d/dec/ff9 [0,4194304] 0 2024-04-20T11:09:41.444 INFO:tasks.workunit.client.1.smithi173.stdout:6/683: dwrite d4/f46 [0,4194304] 0 2024-04-20T11:09:41.445 INFO:tasks.workunit.client.1.smithi173.stdout:3/675: rename d7/d2b/d9d/fae to d7/d17/d58/fdc 0 2024-04-20T11:09:41.446 INFO:tasks.workunit.client.1.smithi173.stdout:1/633: symlink d5/d6/d17/ld3 0 2024-04-20T11:09:41.446 INFO:tasks.workunit.client.1.smithi173.stdout:2/806: fsync d2/d34/d58/f63 0 2024-04-20T11:09:41.447 INFO:tasks.workunit.client.1.smithi173.stdout:5/705: creat da/de/d19/d81/d92/ff5 x:0 0 0 2024-04-20T11:09:41.449 INFO:tasks.workunit.client.1.smithi173.stdout:4/762: stat da/dc/d13/f32 0 2024-04-20T11:09:41.451 INFO:tasks.workunit.client.1.smithi173.stdout:7/681: symlink d9/da/d2e/lf8 0 2024-04-20T11:09:41.452 INFO:tasks.workunit.client.1.smithi173.stdout:3/676: read d7/d17/d61/d83/fb8 [6025562,77066] 0 2024-04-20T11:09:41.465 INFO:tasks.workunit.client.1.smithi173.stdout:6/684: unlink d4/d14/cb7 0 2024-04-20T11:09:41.466 INFO:tasks.workunit.client.1.smithi173.stdout:4/763: mkdir da/dc/df4/dff 0 2024-04-20T11:09:41.470 INFO:tasks.workunit.client.1.smithi173.stdout:3/677: dwrite d7/d17/d61/f9b [0,4194304] 0 2024-04-20T11:09:41.470 INFO:tasks.workunit.client.1.smithi173.stdout:7/682: dwrite d9/da/f46 [4194304,4194304] 0 2024-04-20T11:09:41.470 INFO:tasks.workunit.client.1.smithi173.stdout:5/706: creat da/de/d19/dca/d48/da2/ff6 x:0 0 0 2024-04-20T11:09:41.472 INFO:tasks.workunit.client.1.smithi173.stdout:4/764: truncate da/d1e/d52/f71 1739503 0 2024-04-20T11:09:41.474 INFO:tasks.workunit.client.1.smithi173.stdout:9/659: sync 2024-04-20T11:09:41.474 INFO:tasks.workunit.client.1.smithi173.stdout:8/688: mkdir d4/d18/d59/d5a/d7b/db1/dce/ddd 0 2024-04-20T11:09:41.477 INFO:tasks.workunit.client.1.smithi173.stdout:6/685: creat d4/d24/d42/d81/fdd x:0 0 0 2024-04-20T11:09:41.480 INFO:tasks.workunit.client.1.smithi173.stdout:0/644: rename d6/d8/fe to d6/d8/d51/fce 0 2024-04-20T11:09:41.481 INFO:tasks.workunit.client.1.smithi173.stdout:3/678: fsync d7/d17/d61/d83/dcf/d30/f60 0 2024-04-20T11:09:41.483 INFO:tasks.workunit.client.1.smithi173.stdout:5/707: mknod da/de/d19/dca/d48/da2/cf7 0 2024-04-20T11:09:41.485 INFO:tasks.workunit.client.1.smithi173.stdout:4/765: symlink da/d1a/l100 0 2024-04-20T11:09:41.485 INFO:tasks.workunit.client.1.smithi173.stdout:2/807: dwrite d2/d13/f74 [0,4194304] 0 2024-04-20T11:09:41.494 INFO:tasks.workunit.client.1.smithi173.stdout:3/679: write d7/d2b/fbd [180416,32629] 0 2024-04-20T11:09:41.495 INFO:tasks.workunit.client.1.smithi173.stdout:4/766: mkdir da/d1a/d60/d7d/d101 0 2024-04-20T11:09:41.496 INFO:tasks.workunit.client.1.smithi173.stdout:5/708: creat da/de/d19/d81/d92/ff8 x:0 0 0 2024-04-20T11:09:41.498 INFO:tasks.workunit.client.1.smithi173.stdout:6/686: symlink d4/d1c/lde 0 2024-04-20T11:09:41.500 INFO:tasks.workunit.client.1.smithi173.stdout:8/689: mknod d4/d18/d59/d6d/db8/d9b/dc8/cde 0 2024-04-20T11:09:41.500 INFO:tasks.workunit.client.1.smithi173.stdout:0/645: creat d6/d8/df/d2c/d42/d50/d86/d8b/dcc/fcf x:0 0 0 2024-04-20T11:09:41.502 INFO:tasks.workunit.client.1.smithi173.stdout:0/646: chown d6/d8/df 219 1 2024-04-20T11:09:41.504 INFO:tasks.workunit.client.1.smithi173.stdout:0/647: chown l5 0 1 2024-04-20T11:09:41.506 INFO:tasks.workunit.client.1.smithi173.stdout:5/709: mknod da/de/d19/dca/d48/cf9 0 2024-04-20T11:09:41.507 INFO:tasks.workunit.client.1.smithi173.stdout:1/634: write d5/d6/d17/f27 [2864727,70924] 0 2024-04-20T11:09:41.509 INFO:tasks.workunit.client.1.smithi173.stdout:9/660: getdents d5/d9/d20/d4f/d5e/d7d 0 2024-04-20T11:09:41.510 INFO:tasks.workunit.client.1.smithi173.stdout:5/710: fsync da/de/d19/d32/d30/f70 0 2024-04-20T11:09:41.512 INFO:tasks.workunit.client.1.smithi173.stdout:8/690: mknod d4/d18/d4f/cdf 0 2024-04-20T11:09:41.519 INFO:tasks.workunit.client.1.smithi173.stdout:2/808: dwrite d2/ff0 [0,4194304] 0 2024-04-20T11:09:41.521 INFO:tasks.workunit.client.1.smithi173.stdout:5/711: dread - da/de/d75/d40/fae zero size 2024-04-20T11:09:41.524 INFO:tasks.workunit.client.1.smithi173.stdout:5/712: fsync da/fd1 0 2024-04-20T11:09:41.525 INFO:tasks.workunit.client.1.smithi173.stdout:8/691: fdatasync d4/d18/d4f/f57 0 2024-04-20T11:09:41.531 INFO:tasks.workunit.client.1.smithi173.stdout:9/661: symlink d5/d9/d20/d29/d18/d9b/d98/lc8 0 2024-04-20T11:09:41.535 INFO:tasks.workunit.client.1.smithi173.stdout:8/692: truncate d4/d18/f24 5115976 0 2024-04-20T11:09:41.541 INFO:tasks.workunit.client.1.smithi173.stdout:0/648: dwrite d6/d54/d65/f7c [4194304,4194304] 0 2024-04-20T11:09:41.541 INFO:tasks.workunit.client.1.smithi173.stdout:1/635: link d5/d6/d17/d42/d12/f47 d5/d6/d17/d42/da/d20/dad/dc2/fd4 0 2024-04-20T11:09:41.543 INFO:tasks.workunit.client.1.smithi173.stdout:8/693: truncate d4/fbb 1091508 0 2024-04-20T11:09:41.543 INFO:tasks.workunit.client.1.smithi173.stdout:7/683: dwrite d9/d33/f6b [0,4194304] 0 2024-04-20T11:09:41.546 INFO:tasks.workunit.client.1.smithi173.stdout:8/694: readlink d4/l3d 0 2024-04-20T11:09:41.551 INFO:tasks.workunit.client.1.smithi173.stdout:5/713: link da/de/d19/dca/d48/da2/cf7 da/d4f/d65/dd5/cfa 0 2024-04-20T11:09:41.554 INFO:tasks.workunit.client.1.smithi173.stdout:8/695: rmdir d4/d18/d4f/d5b 39 2024-04-20T11:09:41.555 INFO:tasks.workunit.client.1.smithi173.stdout:2/809: dread d2/d13/f6d [0,4194304] 0 2024-04-20T11:09:41.559 INFO:tasks.workunit.client.1.smithi173.stdout:7/684: truncate d9/d33/d40/d89/feb 816442 0 2024-04-20T11:09:41.562 INFO:tasks.workunit.client.1.smithi173.stdout:8/696: chown d4/d18/d59/d6d/db8/l42 2221288 1 2024-04-20T11:09:41.563 INFO:tasks.workunit.client.1.smithi173.stdout:4/767: dwrite da/dc/d15/f1d [0,4194304] 0 2024-04-20T11:09:41.570 INFO:tasks.workunit.client.1.smithi173.stdout:6/687: dwrite d4/d24/d3e/d9c/dc7/fcc [0,4194304] 0 2024-04-20T11:09:41.570 INFO:tasks.workunit.client.1.smithi173.stdout:8/697: truncate d4/d18/d59/d6d/fba 154049 0 2024-04-20T11:09:41.573 INFO:tasks.workunit.client.1.smithi173.stdout:4/768: fsync da/dc/fa5 0 2024-04-20T11:09:41.573 INFO:tasks.workunit.client.1.smithi173.stdout:8/698: chown d4/f10 8142562 1 2024-04-20T11:09:41.573 INFO:tasks.workunit.client.1.smithi173.stdout:2/810: mknod d2/d13/d9a/dbc/dfd/d53/c10c 0 2024-04-20T11:09:41.574 INFO:tasks.workunit.client.1.smithi173.stdout:1/636: creat d5/d6/d17/d42/da/d20/d38/fd5 x:0 0 0 2024-04-20T11:09:41.576 INFO:tasks.workunit.client.1.smithi173.stdout:5/714: link da/de/d75/d40/d59/f96 da/de/d19/d32/d72/dc3/ffb 0 2024-04-20T11:09:41.577 INFO:tasks.workunit.client.1.smithi173.stdout:5/715: chown da/d4f/d5e/ld0 1883481841 1 2024-04-20T11:09:41.579 INFO:tasks.workunit.client.1.smithi173.stdout:5/716: chown da/f33 1 1 2024-04-20T11:09:41.586 INFO:tasks.workunit.client.1.smithi173.stdout:9/662: truncate d5/d9/d20/d29/d18/f9c 731966 0 2024-04-20T11:09:41.586 INFO:tasks.workunit.client.1.smithi173.stdout:2/811: rmdir d2/d13/d9a/dbc/dfd/d4b 39 2024-04-20T11:09:41.586 INFO:tasks.workunit.client.1.smithi173.stdout:5/717: unlink da/de/cc6 0 2024-04-20T11:09:41.586 INFO:tasks.workunit.client.1.smithi173.stdout:7/685: getdents d9/d33/d91 0 2024-04-20T11:09:41.594 INFO:tasks.workunit.client.1.smithi173.stdout:0/649: dwrite d6/d8/df/d2c/d42/d50/d86/d8b/fbe [0,4194304] 0 2024-04-20T11:09:41.596 INFO:tasks.workunit.client.1.smithi173.stdout:8/699: write d4/fd [1207953,123657] 0 2024-04-20T11:09:41.600 INFO:tasks.workunit.client.1.smithi173.stdout:2/812: creat d2/de/d11/d9d/f10d x:0 0 0 2024-04-20T11:09:41.601 INFO:tasks.workunit.client.1.smithi173.stdout:8/700: dread - d4/d18/d59/fb6 zero size 2024-04-20T11:09:41.602 INFO:tasks.workunit.client.1.smithi173.stdout:7/686: fsync d9/da/d17/fbb 0 2024-04-20T11:09:41.603 INFO:tasks.workunit.client.1.smithi173.stdout:4/769: truncate da/d1e/f38 3940868 0 2024-04-20T11:09:41.604 INFO:tasks.workunit.client.1.smithi173.stdout:7/687: chown d9/d1f/d72/d8f 1943 1 2024-04-20T11:09:41.605 INFO:tasks.workunit.client.1.smithi173.stdout:0/650: rmdir d6/d8/d3c/d44/d75 39 2024-04-20T11:09:41.607 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:41 smithi117 ceph-mon[15659]: pgmap v30: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 132 MiB/s wr, 313 op/s 2024-04-20T11:09:41.611 INFO:tasks.workunit.client.1.smithi173.stdout:6/688: dwrite d4/d14/f3c [0,4194304] 0 2024-04-20T11:09:41.611 INFO:tasks.workunit.client.1.smithi173.stdout:5/718: write da/d34/d7f/da4/fb0 [843002,90238] 0 2024-04-20T11:09:41.611 INFO:tasks.workunit.client.1.smithi173.stdout:7/688: chown d9/d1f/d7f/dce/dad/cec 588 1 2024-04-20T11:09:41.620 INFO:tasks.workunit.client.1.smithi173.stdout:6/689: chown d4/d18/d7e/fbe 4 1 2024-04-20T11:09:41.624 INFO:tasks.workunit.client.1.smithi173.stdout:0/651: truncate d6/f29 2166381 0 2024-04-20T11:09:41.625 INFO:tasks.workunit.client.1.smithi173.stdout:7/689: mknod d9/d33/d35/cf9 0 2024-04-20T11:09:41.627 INFO:tasks.workunit.client.1.smithi173.stdout:9/663: dwrite d5/d9/d20/d4f/d5e/d7d/f86 [4194304,4194304] 0 2024-04-20T11:09:41.628 INFO:tasks.workunit.client.1.smithi173.stdout:5/719: mknod da/d4f/d65/d8e/cfc 0 2024-04-20T11:09:41.633 INFO:tasks.workunit.client.1.smithi173.stdout:4/770: dread da/dc/df4/d53/d39/f56 [0,4194304] 0 2024-04-20T11:09:41.634 INFO:tasks.workunit.client.1.smithi173.stdout:7/690: creat d9/d1f/d7f/d8d/ffa x:0 0 0 2024-04-20T11:09:41.637 INFO:tasks.workunit.client.1.smithi173.stdout:5/720: write da/de/d19/dca/f8c [879107,82229] 0 2024-04-20T11:09:41.640 INFO:tasks.workunit.client.1.smithi173.stdout:5/721: write da/de/d19/fe5 [99361,26480] 0 2024-04-20T11:09:41.644 INFO:tasks.workunit.client.1.smithi173.stdout:5/722: dread - da/d4f/fd3 zero size 2024-04-20T11:09:41.654 INFO:tasks.workunit.client.1.smithi173.stdout:7/691: symlink d9/lfb 0 2024-04-20T11:09:41.657 INFO:tasks.workunit.client.1.smithi173.stdout:2/813: dwrite d2/d34/d58/fd3 [0,4194304] 0 2024-04-20T11:09:41.662 INFO:tasks.workunit.client.1.smithi173.stdout:8/701: dread d4/d18/f72 [0,4194304] 0 2024-04-20T11:09:41.662 INFO:tasks.workunit.client.1.smithi173.stdout:2/814: mkdir d2/d34/d10e 0 2024-04-20T11:09:41.662 INFO:tasks.workunit.client.1.smithi173.stdout:9/664: getdents d5/d9/d20/d2b 0 2024-04-20T11:09:41.665 INFO:tasks.workunit.client.1.smithi173.stdout:3/680: sync 2024-04-20T11:09:41.666 INFO:tasks.workunit.client.1.smithi173.stdout:6/690: dwrite d4/d14/d5f/fb4 [0,4194304] 0 2024-04-20T11:09:41.669 INFO:tasks.workunit.client.1.smithi173.stdout:8/702: unlink d4/d18/d59/fc9 0 2024-04-20T11:09:41.673 INFO:tasks.workunit.client.1.smithi173.stdout:8/703: read - d4/d18/d59/fc7 zero size 2024-04-20T11:09:41.680 INFO:tasks.workunit.client.1.smithi173.stdout:7/692: getdents d9/d33/d85/d97 0 2024-04-20T11:09:41.680 INFO:tasks.workunit.client.1.smithi173.stdout:9/665: symlink d5/d9/d20/d2b/d59/d44/d78/lc9 0 2024-04-20T11:09:41.682 INFO:tasks.workunit.client.1.smithi173.stdout:4/771: dwrite da/dc/df4/d53/f5b [0,4194304] 0 2024-04-20T11:09:41.682 INFO:tasks.workunit.client.1.smithi173.stdout:6/691: read d4/d14/f6a [262920,67295] 0 2024-04-20T11:09:41.683 INFO:tasks.workunit.client.1.smithi173.stdout:5/723: dwrite da/de/d19/dca/d48/da2/ff6 [0,4194304] 0 2024-04-20T11:09:41.687 INFO:tasks.workunit.client.1.smithi173.stdout:3/681: mknod d7/d17/d61/d90/dca/cdd 0 2024-04-20T11:09:41.687 INFO:tasks.workunit.client.1.smithi173.stdout:5/724: readlink da/d34/d7f/da5/lda 0 2024-04-20T11:09:41.688 INFO:tasks.workunit.client.1.smithi173.stdout:0/652: dwrite d6/d8/df/d2c/d2d/f8a [0,4194304] 0 2024-04-20T11:09:41.691 INFO:tasks.workunit.client.1.smithi173.stdout:4/772: chown da/d1a/f40 425188 1 2024-04-20T11:09:41.691 INFO:tasks.workunit.client.1.smithi173.stdout:0/653: write d6/d5f/d8f/fc9 [387390,36515] 0 2024-04-20T11:09:41.693 INFO:tasks.workunit.client.1.smithi173.stdout:3/682: stat d7/d17/d58/d8a/dc4 0 2024-04-20T11:09:41.693 INFO:tasks.workunit.client.1.smithi173.stdout:2/815: symlink d2/d13/d9a/dbc/dfd/d4b/l10f 0 2024-04-20T11:09:41.693 INFO:tasks.workunit.client.1.smithi173.stdout:8/704: rmdir d4/d18/d59/d6d/db8/d3c/d86 39 2024-04-20T11:09:41.698 INFO:tasks.workunit.client.1.smithi173.stdout:9/666: mknod d5/d9/d20/d2b/d59/d44/cca 0 2024-04-20T11:09:41.698 INFO:tasks.workunit.client.1.smithi173.stdout:5/725: write da/de/d19/dca/f8c [952163,130701] 0 2024-04-20T11:09:41.700 INFO:tasks.workunit.client.1.smithi173.stdout:1/637: sync 2024-04-20T11:09:41.701 INFO:tasks.workunit.client.1.smithi173.stdout:7/693: creat d9/da/dc3/ffc x:0 0 0 2024-04-20T11:09:41.702 INFO:tasks.workunit.client.1.smithi173.stdout:4/773: mknod da/d1a/d6e/db4/c102 0 2024-04-20T11:09:41.708 INFO:tasks.workunit.client.1.smithi173.stdout:4/774: readlink da/d1a/d60/d66/lbd 0 2024-04-20T11:09:41.708 INFO:tasks.workunit.client.1.smithi173.stdout:8/705: rename d4/d18/d59/d6d/db8 to d4/d18/d59/d5a/da8/db9/de0 0 2024-04-20T11:09:41.709 INFO:tasks.workunit.client.1.smithi173.stdout:0/654: mkdir d6/d8/df/dd0 0 2024-04-20T11:09:41.710 INFO:tasks.workunit.client.1.smithi173.stdout:2/816: symlink d2/d13/d9a/dbc/dfd/l110 0 2024-04-20T11:09:41.721 INFO:tasks.workunit.client.1.smithi173.stdout:5/726: chown da/de/d75/c83 24 1 2024-04-20T11:09:41.721 INFO:tasks.workunit.client.1.smithi173.stdout:1/638: creat d5/d6/d17/d42/da/d20/fd6 x:0 0 0 2024-04-20T11:09:41.721 INFO:tasks.workunit.client.1.smithi173.stdout:7/694: mkdir d9/d33/d35/d54/db3/dfd 0 2024-04-20T11:09:41.722 INFO:tasks.workunit.client.1.smithi173.stdout:0/655: write d6/d8/df/d2c/d42/d50/d86/d8b/dcc/fcf [39449,37040] 0 2024-04-20T11:09:41.723 INFO:tasks.workunit.client.1.smithi173.stdout:5/727: dread - da/de/f7e zero size 2024-04-20T11:09:41.723 INFO:tasks.workunit.client.1.smithi173.stdout:8/706: link d4/l2b d4/d18/d59/d5a/d7b/ddc/le1 0 2024-04-20T11:09:41.726 INFO:tasks.workunit.client.1.smithi173.stdout:4/775: chown da/d1a/d60/d66/ce8 4153228 1 2024-04-20T11:09:41.734 INFO:tasks.workunit.client.1.smithi173.stdout:7/695: chown d9/d33/d35/d4a/d52/df2 423477371 1 2024-04-20T11:09:41.740 INFO:tasks.workunit.client.1.smithi173.stdout:9/667: dread d5/d9/d20/d29/f64 [0,4194304] 0 2024-04-20T11:09:41.742 INFO:tasks.workunit.client.1.smithi173.stdout:1/639: write d5/d6/d17/d42/da/df/d40/dc8/fcc [783014,102385] 0 2024-04-20T11:09:41.748 INFO:tasks.workunit.client.1.smithi173.stdout:2/817: dwrite d2/d13/f50 [4194304,4194304] 0 2024-04-20T11:09:41.748 INFO:tasks.workunit.client.1.smithi173.stdout:9/668: write d5/d9/d20/d29/f11 [1619457,37093] 0 2024-04-20T11:09:41.748 INFO:tasks.workunit.client.1.smithi173.stdout:0/656: symlink d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/db9/ld1 0 2024-04-20T11:09:41.752 INFO:tasks.workunit.client.1.smithi173.stdout:4/776: rename da/d1a/d60/d7d/dec to da/dc/d103 0 2024-04-20T11:09:41.755 INFO:tasks.workunit.client.1.smithi173.stdout:7/696: mknod d9/da/d17/cfe 0 2024-04-20T11:09:41.763 INFO:tasks.workunit.client.1.smithi173.stdout:5/728: rename da/de/f2d to da/d34/d7f/ffd 0 2024-04-20T11:09:41.763 INFO:tasks.workunit.client.1.smithi173.stdout:0/657: truncate d6/d8/df/d15/fc6 540329 0 2024-04-20T11:09:41.765 INFO:tasks.workunit.client.1.smithi173.stdout:9/669: mkdir d5/d9/d20/d29/d2d/dcb 0 2024-04-20T11:09:41.775 INFO:tasks.workunit.client.1.smithi173.stdout:7/697: write d9/d33/d40/dc9/fda [445545,116258] 0 2024-04-20T11:09:41.775 INFO:tasks.workunit.client.1.smithi173.stdout:1/640: creat d5/d6/d17/d42/d12/fd7 x:0 0 0 2024-04-20T11:09:41.775 INFO:tasks.workunit.client.1.smithi173.stdout:8/707: dwrite d4/d18/d4f/d5b/f87 [0,4194304] 0 2024-04-20T11:09:41.775 INFO:tasks.workunit.client.1.smithi173.stdout:5/729: fsync da/de/d19/d81/d92/ff8 0 2024-04-20T11:09:41.775 INFO:tasks.workunit.client.1.smithi173.stdout:3/683: dwrite d7/d17/d61/d83/dcf/f5a [0,4194304] 0 2024-04-20T11:09:41.779 INFO:tasks.workunit.client.1.smithi173.stdout:4/777: creat da/d1e/da6/f104 x:0 0 0 2024-04-20T11:09:41.779 INFO:tasks.workunit.client.1.smithi173.stdout:7/698: dread - d9/da/d43/fdf zero size 2024-04-20T11:09:41.780 INFO:tasks.workunit.client.1.smithi173.stdout:2/818: rename d2/d13/da5/d108 to d2/d13/d9a/dbc/dfd/d4b/dae/db6/df9/d111 0 2024-04-20T11:09:41.784 INFO:tasks.workunit.client.1.smithi173.stdout:4/778: write da/dc/d13/fc6 [282685,124295] 0 2024-04-20T11:09:41.788 INFO:tasks.workunit.client.1.smithi173.stdout:9/670: mkdir d5/dcc 0 2024-04-20T11:09:41.788 INFO:tasks.workunit.client.1.smithi173.stdout:5/730: symlink da/d34/d7f/da5/lfe 0 2024-04-20T11:09:41.789 INFO:tasks.workunit.client.1.smithi173.stdout:1/641: fsync d5/d6/d17/d42/d12/f15 0 2024-04-20T11:09:41.789 INFO:tasks.workunit.client.1.smithi173.stdout:7/699: rmdir d9/da/d17/d2b/daa 39 2024-04-20T11:09:41.794 INFO:tasks.workunit.client.1.smithi173.stdout:9/671: fsync d5/d9/d20/d2b/d31/f92 0 2024-04-20T11:09:41.797 INFO:tasks.workunit.client.1.smithi173.stdout:5/731: creat da/d34/d7f/fff x:0 0 0 2024-04-20T11:09:41.801 INFO:tasks.workunit.client.1.smithi173.stdout:2/819: creat d2/d34/d81/f112 x:0 0 0 2024-04-20T11:09:41.821 INFO:tasks.workunit.client.1.smithi173.stdout:4/779: link da/dc/d13/c4b da/dc/d103/c105 0 2024-04-20T11:09:41.825 INFO:tasks.workunit.client.1.smithi173.stdout:0/658: write d6/d8/d3c/f4a [66164,68397] 0 2024-04-20T11:09:41.827 INFO:tasks.workunit.client.1.smithi173.stdout:8/708: dwrite d4/d18/d59/d6d/fba [0,4194304] 0 2024-04-20T11:09:41.827 INFO:tasks.workunit.client.1.smithi173.stdout:3/684: write d7/d17/f2f [1271005,23929] 0 2024-04-20T11:09:41.829 INFO:tasks.workunit.client.1.smithi173.stdout:4/780: read da/dc/d15/d8e/fa7 [892708,77591] 0 2024-04-20T11:09:41.831 INFO:tasks.workunit.client.1.smithi173.stdout:8/709: dread - d4/d18/da6/fbf zero size 2024-04-20T11:09:41.842 INFO:tasks.workunit.client.1.smithi173.stdout:0/659: readlink d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/db9/ld1 0 2024-04-20T11:09:41.845 INFO:tasks.workunit.client.1.smithi173.stdout:5/732: dwrite da/de/d19/d32/f31 [0,4194304] 0 2024-04-20T11:09:41.847 INFO:tasks.workunit.client.1.smithi173.stdout:7/700: dwrite d9/d33/d40/d7a/ff0 [0,4194304] 0 2024-04-20T11:09:41.848 INFO:tasks.workunit.client.1.smithi173.stdout:8/710: truncate d4/d18/d59/d5a/da8/db9/de0/f9d 1069365 0 2024-04-20T11:09:41.849 INFO:tasks.workunit.client.1.smithi173.stdout:2/820: mknod d2/de/d11/d9d/d87/db0/c113 0 2024-04-20T11:09:41.852 INFO:tasks.workunit.client.1.smithi173.stdout:2/821: write d2/ff0 [2897850,66561] 0 2024-04-20T11:09:41.852 INFO:tasks.workunit.client.1.smithi173.stdout:6/692: sync 2024-04-20T11:09:41.856 INFO:tasks.workunit.client.1.smithi173.stdout:1/642: dread d5/d6/d17/d42/f54 [0,4194304] 0 2024-04-20T11:09:41.864 INFO:tasks.workunit.client.1.smithi173.stdout:8/711: write d4/d18/d59/d5a/da8/fae [191547,29619] 0 2024-04-20T11:09:41.868 INFO:tasks.workunit.client.1.smithi173.stdout:6/693: fdatasync d4/d24/f35 0 2024-04-20T11:09:41.871 INFO:tasks.workunit.client.1.smithi173.stdout:2/822: truncate d2/d13/da5/db2/fea 246043 0 2024-04-20T11:09:41.872 INFO:tasks.workunit.client.1.smithi173.stdout:5/733: link da/de/d19/d32/l7c da/de/d19/d81/d92/l100 0 2024-04-20T11:09:41.873 INFO:tasks.workunit.client.1.smithi173.stdout:9/672: dread d5/d9/d20/d4f/d5e/d7d/f86 [0,4194304] 0 2024-04-20T11:09:41.876 INFO:tasks.workunit.client.1.smithi173.stdout:0/660: dwrite d6/d54/f7d [0,4194304] 0 2024-04-20T11:09:41.880 INFO:tasks.workunit.client.1.smithi173.stdout:2/823: mknod d2/d13/d9a/dbc/dfd/d4b/c114 0 2024-04-20T11:09:41.885 INFO:tasks.workunit.client.1.smithi173.stdout:9/673: creat d5/d9/d20/d29/d2d/dcb/fcd x:0 0 0 2024-04-20T11:09:41.888 INFO:tasks.workunit.client.1.smithi173.stdout:3/685: truncate d7/d17/d61/d83/faa 3314448 0 2024-04-20T11:09:41.892 INFO:tasks.workunit.client.1.smithi173.stdout:7/701: dread d9/da/d17/f64 [0,4194304] 0 2024-04-20T11:09:41.896 INFO:tasks.workunit.client.1.smithi173.stdout:2/824: symlink d2/d13/d28/d4a/dff/l115 0 2024-04-20T11:09:41.903 INFO:tasks.workunit.client.1.smithi173.stdout:0/661: dwrite d6/d54/f7d [0,4194304] 0 2024-04-20T11:09:41.905 INFO:tasks.workunit.client.1.smithi173.stdout:8/712: dwrite d4/d18/d59/d5a/da8/db9/fbc [0,4194304] 0 2024-04-20T11:09:41.908 INFO:tasks.workunit.client.1.smithi173.stdout:2/825: creat d2/de/d11/d9d/f116 x:0 0 0 2024-04-20T11:09:41.909 INFO:tasks.workunit.client.1.smithi173.stdout:4/781: dread da/dc/d13/f29 [4194304,4194304] 0 2024-04-20T11:09:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:41 smithi173 ceph-mon[23800]: pgmap v30: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 132 MiB/s wr, 313 op/s 2024-04-20T11:09:41.916 INFO:tasks.workunit.client.1.smithi173.stdout:1/643: dread d5/d6/d17/d42/da/d20/f87 [0,4194304] 0 2024-04-20T11:09:41.916 INFO:tasks.workunit.client.1.smithi173.stdout:6/694: truncate d4/d24/d42/d74/d3b/fb5 3566749 0 2024-04-20T11:09:41.916 INFO:tasks.workunit.client.1.smithi173.stdout:5/734: truncate da/d34/d7f/ffd 4204681 0 2024-04-20T11:09:41.923 INFO:tasks.workunit.client.1.smithi173.stdout:4/782: mkdir da/dc/d13/d106 0 2024-04-20T11:09:41.928 INFO:tasks.workunit.client.1.smithi173.stdout:0/662: write d6/f13 [1180992,125372] 0 2024-04-20T11:09:41.928 INFO:tasks.workunit.client.1.smithi173.stdout:8/713: unlink d4/d18/c2f 0 2024-04-20T11:09:41.933 INFO:tasks.workunit.client.1.smithi173.stdout:0/663: stat d6/d8/d3c/f4a 0 2024-04-20T11:09:41.937 INFO:tasks.workunit.client.1.smithi173.stdout:7/702: dwrite d9/da/d17/f78 [0,4194304] 0 2024-04-20T11:09:41.944 INFO:tasks.workunit.client.1.smithi173.stdout:2/826: dwrite d2/de/d11/d9d/f8e [0,4194304] 0 2024-04-20T11:09:41.945 INFO:tasks.workunit.client.1.smithi173.stdout:9/674: dwrite d5/d9/d20/d29/f96 [0,4194304] 0 2024-04-20T11:09:41.950 INFO:tasks.workunit.client.1.smithi173.stdout:3/686: dwrite d7/d53/f8f [0,4194304] 0 2024-04-20T11:09:41.950 INFO:tasks.workunit.client.1.smithi173.stdout:7/703: chown d9/d33/d35/d4a/d52/df2/c3b 21 1 2024-04-20T11:09:41.952 INFO:tasks.workunit.client.1.smithi173.stdout:6/695: mkdir d4/d24/d3e/d9c/dc7/ddf 0 2024-04-20T11:09:41.955 INFO:tasks.workunit.client.1.smithi173.stdout:8/714: mknod d4/d18/da6/ce2 0 2024-04-20T11:09:41.959 INFO:tasks.workunit.client.1.smithi173.stdout:2/827: fdatasync d2/d13/d9a/dbc/dfd/d53/d9f/ffb 0 2024-04-20T11:09:41.960 INFO:tasks.workunit.client.1.smithi173.stdout:9/675: truncate d5/f53 132724 0 2024-04-20T11:09:41.963 INFO:tasks.workunit.client.1.smithi173.stdout:7/704: link d9/fb7 d9/da/dc3/fff 0 2024-04-20T11:09:41.967 INFO:tasks.workunit.client.1.smithi173.stdout:3/687: creat d7/d17/d61/d83/fde x:0 0 0 2024-04-20T11:09:41.967 INFO:tasks.workunit.client.1.smithi173.stdout:9/676: chown d5/c45 8 1 2024-04-20T11:09:41.970 INFO:tasks.workunit.client.1.smithi173.stdout:4/783: write da/dc/df4/d53/f9f [886946,13523] 0 2024-04-20T11:09:41.972 INFO:tasks.workunit.client.1.smithi173.stdout:2/828: creat d2/de/d11/db8/f117 x:0 0 0 2024-04-20T11:09:41.972 INFO:tasks.workunit.client.1.smithi173.stdout:9/677: fsync d5/d9/f6e 0 2024-04-20T11:09:41.973 INFO:tasks.workunit.client.1.smithi173.stdout:3/688: chown d7/d17/d61/d83/fde 948289 1 2024-04-20T11:09:41.974 INFO:tasks.workunit.client.1.smithi173.stdout:0/664: dwrite d6/d8/df/d2c/d42/d50/f76 [0,4194304] 0 2024-04-20T11:09:41.974 INFO:tasks.workunit.client.1.smithi173.stdout:1/644: write d5/d6/d17/d42/da/d20/dad/dc2/fd4 [1104797,125855] 0 2024-04-20T11:09:41.981 INFO:tasks.workunit.client.1.smithi173.stdout:3/689: unlink d7/d17/f20 0 2024-04-20T11:09:41.982 INFO:tasks.workunit.client.1.smithi173.stdout:4/784: symlink da/dc/ddd/l107 0 2024-04-20T11:09:41.983 INFO:tasks.workunit.client.1.smithi173.stdout:8/715: creat d4/d18/d59/d5a/da8/db9/de0/d3c/fe3 x:0 0 0 2024-04-20T11:09:41.983 INFO:tasks.workunit.client.1.smithi173.stdout:7/705: getdents d9/d33/d85/d97 0 2024-04-20T11:09:41.983 INFO:tasks.workunit.client.1.smithi173.stdout:3/690: fdatasync d7/d2b/f3d 0 2024-04-20T11:09:41.989 INFO:tasks.workunit.client.1.smithi173.stdout:0/665: mknod d6/cd2 0 2024-04-20T11:09:41.994 INFO:tasks.workunit.client.1.smithi173.stdout:7/706: creat d9/d1f/d72/d8f/f100 x:0 0 0 2024-04-20T11:09:41.995 INFO:tasks.workunit.client.1.smithi173.stdout:6/696: truncate d4/f5a 1168370 0 2024-04-20T11:09:41.997 INFO:tasks.workunit.client.1.smithi173.stdout:0/666: read d6/d8/df/d2c/d42/d50/d86/f94 [2405425,22873] 0 2024-04-20T11:09:42.000 INFO:tasks.workunit.client.1.smithi173.stdout:8/716: mkdir d4/d18/d4f/d5b/de4 0 2024-04-20T11:09:42.000 INFO:tasks.workunit.client.1.smithi173.stdout:1/645: dwrite d5/d6/d17/d42/da/d20/d38/d52/f8d [0,4194304] 0 2024-04-20T11:09:42.000 INFO:tasks.workunit.client.1.smithi173.stdout:4/785: creat da/d1e/d52/d9a/dd3/def/f108 x:0 0 0 2024-04-20T11:09:42.002 INFO:tasks.workunit.client.1.smithi173.stdout:1/646: chown d5/d6/dc/f2c 724 1 2024-04-20T11:09:42.003 INFO:tasks.workunit.client.1.smithi173.stdout:9/678: write d5/f53 [1101410,68003] 0 2024-04-20T11:09:42.005 INFO:tasks.workunit.client.1.smithi173.stdout:3/691: dread d7/d53/f8f [0,4194304] 0 2024-04-20T11:09:42.006 INFO:tasks.workunit.client.1.smithi173.stdout:6/697: creat d4/d14/d5f/daf/fe0 x:0 0 0 2024-04-20T11:09:42.009 INFO:tasks.workunit.client.1.smithi173.stdout:3/692: truncate d7/d17/d61/d83/dcf/d5c/fc6 114960 0 2024-04-20T11:09:42.013 INFO:tasks.workunit.client.1.smithi173.stdout:1/647: read d5/d6/d17/d42/da/d20/dad/dc2/f55 [172920,99080] 0 2024-04-20T11:09:42.014 INFO:tasks.workunit.client.1.smithi173.stdout:8/717: symlink d4/d18/d59/d5a/da8/le5 0 2024-04-20T11:09:42.018 INFO:tasks.workunit.client.1.smithi173.stdout:4/786: unlink da/dc/fb7 0 2024-04-20T11:09:42.020 INFO:tasks.workunit.client.1.smithi173.stdout:7/707: link d9/d33/f48 d9/da/dc3/f101 0 2024-04-20T11:09:42.021 INFO:tasks.workunit.client.1.smithi173.stdout:9/679: fsync d5/d9/d20/d29/d2d/f3f 0 2024-04-20T11:09:42.022 INFO:tasks.workunit.client.1.smithi173.stdout:6/698: mknod d4/d14/d5f/daf/ce1 0 2024-04-20T11:09:42.025 INFO:tasks.workunit.client.1.smithi173.stdout:9/680: stat d5/d9/d20/d4f/d5e/d7d/dba 0 2024-04-20T11:09:42.026 INFO:tasks.workunit.client.1.smithi173.stdout:6/699: chown d4/d14/d5f/d9f/fca 181 1 2024-04-20T11:09:42.032 INFO:tasks.workunit.client.1.smithi173.stdout:4/787: creat da/dc/d62/f109 x:0 0 0 2024-04-20T11:09:42.036 INFO:tasks.workunit.client.1.smithi173.stdout:3/693: write d7/d17/d61/d83/dcf/d30/f60 [4744,94001] 0 2024-04-20T11:09:42.037 INFO:tasks.workunit.client.1.smithi173.stdout:1/648: mkdir d5/dd8 0 2024-04-20T11:09:42.038 INFO:tasks.workunit.client.1.smithi173.stdout:0/667: dread d6/d8/d51/f5b [0,4194304] 0 2024-04-20T11:09:42.040 INFO:tasks.workunit.client.1.smithi173.stdout:6/700: mknod d4/d24/d42/d81/ce2 0 2024-04-20T11:09:42.041 INFO:tasks.workunit.client.1.smithi173.stdout:8/718: mkdir d4/d18/d4f/d5b/de4/de6 0 2024-04-20T11:09:42.042 INFO:tasks.workunit.client.1.smithi173.stdout:6/701: fdatasync d4/d14/d5f/f76 0 2024-04-20T11:09:42.042 INFO:tasks.workunit.client.1.smithi173.stdout:7/708: unlink d9/d33/d35/d54/ce5 0 2024-04-20T11:09:42.044 INFO:tasks.workunit.client.1.smithi173.stdout:8/719: write d4/d18/fa2 [3659431,336] 0 2024-04-20T11:09:42.046 INFO:tasks.workunit.client.1.smithi173.stdout:8/720: fdatasync d4/d18/d59/f64 0 2024-04-20T11:09:42.048 INFO:tasks.workunit.client.1.smithi173.stdout:8/721: stat d4/d18/d59/d5a/da8/db9/de0/l85 0 2024-04-20T11:09:42.049 INFO:tasks.workunit.client.1.smithi173.stdout:9/681: dread d5/d4c/f72 [0,4194304] 0 2024-04-20T11:09:42.049 INFO:tasks.workunit.client.1.smithi173.stdout:1/649: creat d5/d6/d17/d42/da/d20/dad/fd9 x:0 0 0 2024-04-20T11:09:42.054 INFO:tasks.workunit.client.1.smithi173.stdout:4/788: write da/dc/d15/d8e/fa7 [2070840,38912] 0 2024-04-20T11:09:42.058 INFO:tasks.workunit.client.1.smithi173.stdout:6/702: mkdir d4/de3 0 2024-04-20T11:09:42.060 INFO:tasks.workunit.client.1.smithi173.stdout:9/682: rmdir d5/d9/d20/d2b/d59/d44 39 2024-04-20T11:09:42.061 INFO:tasks.workunit.client.1.smithi173.stdout:0/668: dwrite d6/d8/d3c/f89 [0,4194304] 0 2024-04-20T11:09:42.062 INFO:tasks.workunit.client.1.smithi173.stdout:7/709: write d9/d33/d81/f9b [1385035,84054] 0 2024-04-20T11:09:42.066 INFO:tasks.workunit.client.1.smithi173.stdout:3/694: dwrite d7/d2b/fcd [0,4194304] 0 2024-04-20T11:09:42.069 INFO:tasks.workunit.client.1.smithi173.stdout:4/789: creat da/dc/d62/f10a x:0 0 0 2024-04-20T11:09:42.069 INFO:tasks.workunit.client.1.smithi173.stdout:1/650: symlink d5/d6/d17/d42/d12/dc9/lda 0 2024-04-20T11:09:42.069 INFO:tasks.workunit.client.1.smithi173.stdout:9/683: mknod d5/d9/d20/d29/d18/d8a/cce 0 2024-04-20T11:09:42.071 INFO:tasks.workunit.client.1.smithi173.stdout:0/669: mknod d6/d8/df/d2c/d42/d50/cd3 0 2024-04-20T11:09:42.072 INFO:tasks.workunit.client.1.smithi173.stdout:8/722: write d4/d18/fd2 [189449,75947] 0 2024-04-20T11:09:42.078 INFO:tasks.workunit.client.1.smithi173.stdout:6/703: rename d4/d14/d5f/l77 to d4/le4 0 2024-04-20T11:09:42.082 INFO:tasks.workunit.client.1.smithi173.stdout:0/670: mkdir d6/d8/df/dd0/dd4 0 2024-04-20T11:09:42.084 INFO:tasks.workunit.client.1.smithi173.stdout:9/684: write d5/d9/d20/d2b/d59/f4e [1303561,111829] 0 2024-04-20T11:09:42.091 INFO:tasks.workunit.client.1.smithi173.stdout:6/704: creat d4/fe5 x:0 0 0 2024-04-20T11:09:42.091 INFO:tasks.workunit.client.1.smithi173.stdout:0/671: symlink d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/db9/ld5 0 2024-04-20T11:09:42.093 INFO:tasks.workunit.client.1.smithi173.stdout:4/790: rename da/dc/df4/d53/f37 to da/dc/f10b 0 2024-04-20T11:09:42.096 INFO:tasks.workunit.client.1.smithi173.stdout:9/685: read d5/d9/d20/d2b/d59/d44/f6a [2540256,120940] 0 2024-04-20T11:09:42.099 INFO:tasks.workunit.client.1.smithi173.stdout:1/651: rename d5/d6/d17/d42/d12/d1f/f2f to d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fdb 0 2024-04-20T11:09:42.099 INFO:tasks.workunit.client.1.smithi173.stdout:9/686: chown d5/d9/f47 708209 1 2024-04-20T11:09:42.105 INFO:tasks.workunit.client.1.smithi173.stdout:9/687: chown d5/d9/d20/d2b/d59/d44/d78 78 1 2024-04-20T11:09:42.107 INFO:tasks.workunit.client.1.smithi173.stdout:0/672: dread d6/d8/df/d2c/d2d/f8a [0,4194304] 0 2024-04-20T11:09:42.108 INFO:tasks.workunit.client.1.smithi173.stdout:1/652: truncate d5/d6/d3b/f67 288360 0 2024-04-20T11:09:42.113 INFO:tasks.workunit.client.1.smithi173.stdout:8/723: dread d4/d18/d59/f64 [0,4194304] 0 2024-04-20T11:09:42.114 INFO:tasks.workunit.client.1.smithi173.stdout:1/653: read d5/d6/d3b/f98 [3062214,55977] 0 2024-04-20T11:09:42.114 INFO:tasks.workunit.client.1.smithi173.stdout:9/688: rename d5/d9/d20/d29/c24 to d5/d9/d20/d29/d2d/dcb/ccf 0 2024-04-20T11:09:42.114 INFO:tasks.workunit.client.1.smithi173.stdout:7/710: dread d9/d33/f3a [0,4194304] 0 2024-04-20T11:09:42.114 INFO:tasks.workunit.client.1.smithi173.stdout:8/724: stat d4/d18/d4f/cdf 0 2024-04-20T11:09:42.114 INFO:tasks.workunit.client.1.smithi173.stdout:0/673: mkdir d6/d5f/d8f/dd6 0 2024-04-20T11:09:42.115 INFO:tasks.workunit.client.1.smithi173.stdout:7/711: chown d9/da/dc3 12 1 2024-04-20T11:09:42.116 INFO:tasks.workunit.client.1.smithi173.stdout:0/674: stat d6/d8/df/d43/f4c 0 2024-04-20T11:09:42.116 INFO:tasks.workunit.client.1.smithi173.stdout:8/725: chown d4/d18/d59/d5a/da8/db9/de0/d9b 1976389918 1 2024-04-20T11:09:42.117 INFO:tasks.workunit.client.1.smithi173.stdout:1/654: mkdir d5/d6/d17/d42/d12/d1f/ddc 0 2024-04-20T11:09:42.117 INFO:tasks.workunit.client.1.smithi173.stdout:6/705: dwrite d4/f37 [0,4194304] 0 2024-04-20T11:09:42.119 INFO:tasks.workunit.client.1.smithi173.stdout:9/689: truncate d5/d9/d20/d29/d2d/f43 4834745 0 2024-04-20T11:09:42.120 INFO:tasks.workunit.client.1.smithi173.stdout:8/726: chown d4/d18/fa2 124 1 2024-04-20T11:09:42.121 INFO:tasks.workunit.client.1.smithi173.stdout:6/706: dread - d4/d24/d42/d74/fd1 zero size 2024-04-20T11:09:42.123 INFO:tasks.workunit.client.1.smithi173.stdout:0/675: unlink d6/d8/df/d2c/d42/d50/d86/d8b/fbe 0 2024-04-20T11:09:42.124 INFO:tasks.workunit.client.1.smithi173.stdout:4/791: write da/dc/d62/f76 [3417548,98672] 0 2024-04-20T11:09:42.124 INFO:tasks.workunit.client.1.smithi173.stdout:8/727: dread - d4/d18/d4f/fc6 zero size 2024-04-20T11:09:42.125 INFO:tasks.workunit.client.1.smithi173.stdout:8/728: write d4/f6a [2532689,8766] 0 2024-04-20T11:09:42.127 INFO:tasks.workunit.client.1.smithi173.stdout:8/729: dread - d4/d18/d4f/fc6 zero size 2024-04-20T11:09:42.131 INFO:tasks.workunit.client.1.smithi173.stdout:9/690: mkdir d5/d9/d20/d2b/d59/d44/d78/dd0 0 2024-04-20T11:09:42.132 INFO:tasks.workunit.client.1.smithi173.stdout:1/655: write d5/d6/d17/f4b [1382152,70640] 0 2024-04-20T11:09:42.139 INFO:tasks.workunit.client.1.smithi173.stdout:1/656: readlink d5/d6/d17/d42/d12/d1c/d85/l8a 0 2024-04-20T11:09:42.150 INFO:tasks.workunit.client.1.smithi173.stdout:4/792: link da/dc/df4/d53/l9d da/d1a/d6e/l10c 0 2024-04-20T11:09:42.150 INFO:tasks.workunit.client.1.smithi173.stdout:8/730: dwrite d4/f6a [0,4194304] 0 2024-04-20T11:09:42.154 INFO:tasks.workunit.client.1.smithi173.stdout:9/691: truncate d5/d9/d20/d2b/f97 4162271 0 2024-04-20T11:09:42.158 INFO:tasks.workunit.client.1.smithi173.stdout:9/692: chown d5/d9/d20/l51 1575484272 1 2024-04-20T11:09:42.162 INFO:tasks.workunit.client.1.smithi173.stdout:4/793: readlink da/d1a/d60/d66/dae/ldb 0 2024-04-20T11:09:42.162 INFO:tasks.workunit.client.1.smithi173.stdout:7/712: dread d9/da/d17/d2b/f71 [0,4194304] 0 2024-04-20T11:09:42.167 INFO:tasks.workunit.client.1.smithi173.stdout:4/794: fdatasync da/d1e/d52/d7a/f7f 0 2024-04-20T11:09:42.168 INFO:tasks.workunit.client.1.smithi173.stdout:6/707: dwrite d4/f5a [0,4194304] 0 2024-04-20T11:09:42.168 INFO:tasks.workunit.client.1.smithi173.stdout:7/713: chown d9/d33/d40/dc9/fda 362 1 2024-04-20T11:09:42.169 INFO:tasks.workunit.client.1.smithi173.stdout:1/657: dwrite d5/d6/d17/d42/da/d20/dad/dc2/fc5 [0,4194304] 0 2024-04-20T11:09:42.172 INFO:tasks.workunit.client.1.smithi173.stdout:6/708: fsync d4/d24/d42/d81/f99 0 2024-04-20T11:09:42.173 INFO:tasks.workunit.client.1.smithi173.stdout:1/658: chown d5/d6/d3b/f98 180986 1 2024-04-20T11:09:42.187 INFO:tasks.workunit.client.1.smithi173.stdout:9/693: dwrite d5/d9/d20/d2b/d59/d44/f6a [4194304,4194304] 0 2024-04-20T11:09:42.193 INFO:tasks.workunit.client.1.smithi173.stdout:7/714: mkdir d9/d33/d35/d54/db3/d102 0 2024-04-20T11:09:42.194 INFO:tasks.workunit.client.1.smithi173.stdout:8/731: write d4/d18/d59/d5a/da8/db9/de0/d3c/f54 [1190414,35451] 0 2024-04-20T11:09:42.195 INFO:tasks.workunit.client.1.smithi173.stdout:7/715: chown d9/d33/d35/d54/fde 476 1 2024-04-20T11:09:42.195 INFO:tasks.workunit.client.1.smithi173.stdout:6/709: rmdir d4/d59 39 2024-04-20T11:09:42.198 INFO:tasks.workunit.client.1.smithi173.stdout:1/659: mknod d5/d6/d17/d42/da/d20/d38/cdd 0 2024-04-20T11:09:42.198 INFO:tasks.workunit.client.1.smithi173.stdout:1/660: chown d5 9473 1 2024-04-20T11:09:42.202 INFO:tasks.workunit.client.1.smithi173.stdout:4/795: mkdir da/d1e/d52/d10d 0 2024-04-20T11:09:42.204 INFO:tasks.workunit.client.1.smithi173.stdout:4/796: chown da/dc/d13/d106 3270 1 2024-04-20T11:09:42.204 INFO:tasks.workunit.client.1.smithi173.stdout:7/716: mknod d9/d33/d40/d89/c103 0 2024-04-20T11:09:42.206 INFO:tasks.workunit.client.1.smithi173.stdout:1/661: mkdir d5/d6/d17/d42/da/df/d40/dc8/dde 0 2024-04-20T11:09:42.208 INFO:tasks.workunit.client.1.smithi173.stdout:9/694: mkdir d5/d9/d20/d4f/d5e/d7d/dba/dd1 0 2024-04-20T11:09:42.208 INFO:tasks.workunit.client.1.smithi173.stdout:8/732: symlink d4/le7 0 2024-04-20T11:09:42.209 INFO:tasks.workunit.client.1.smithi173.stdout:8/733: fdatasync d4/d18/da6/fdb 0 2024-04-20T11:09:42.213 INFO:tasks.workunit.client.1.smithi173.stdout:7/717: dread - d9/d33/d40/d89/f8a zero size 2024-04-20T11:09:42.214 INFO:tasks.workunit.client.1.smithi173.stdout:4/797: mkdir da/dc/d103/d10e 0 2024-04-20T11:09:42.214 INFO:tasks.workunit.client.1.smithi173.stdout:9/695: creat d5/d9/d20/d29/d18/d9b/d98/dc4/fd2 x:0 0 0 2024-04-20T11:09:42.216 INFO:tasks.workunit.client.1.smithi173.stdout:4/798: write da/dc/d103/ff9 [3823030,101268] 0 2024-04-20T11:09:42.218 INFO:tasks.workunit.client.1.smithi173.stdout:8/734: mknod d4/d18/d59/ce8 0 2024-04-20T11:09:42.219 INFO:tasks.workunit.client.1.smithi173.stdout:8/735: chown d4/d18/d59/cab 1209126 1 2024-04-20T11:09:42.221 INFO:tasks.workunit.client.1.smithi173.stdout:7/718: creat d9/d33/d35/d4a/d52/df2/f104 x:0 0 0 2024-04-20T11:09:42.222 INFO:tasks.workunit.client.1.smithi173.stdout:7/719: dread - d9/d33/d81/fe6 zero size 2024-04-20T11:09:42.224 INFO:tasks.workunit.client.1.smithi173.stdout:7/720: write d9/d1f/f7b [4364353,8252] 0 2024-04-20T11:09:42.227 INFO:tasks.workunit.client.1.smithi173.stdout:9/696: chown d5/d9/d20/d2b/f97 3 1 2024-04-20T11:09:42.228 INFO:tasks.workunit.client.1.smithi173.stdout:1/662: rename d5/d6/d3b/lc3 to d5/d6/d17/d42/da/d20/ldf 0 2024-04-20T11:09:42.231 INFO:tasks.workunit.client.1.smithi173.stdout:1/663: write d5/d6/d17/d42/da/df/d40/f5d [2880267,97253] 0 2024-04-20T11:09:42.234 INFO:tasks.workunit.client.1.smithi173.stdout:6/710: dwrite d4/d24/f2e [0,4194304] 0 2024-04-20T11:09:42.236 INFO:tasks.workunit.client.1.smithi173.stdout:6/711: write d4/f38 [136331,24352] 0 2024-04-20T11:09:42.239 INFO:tasks.workunit.client.1.smithi173.stdout:4/799: rename da/d1e/d52/da3/cb6 to da/dc/d15/c10f 0 2024-04-20T11:09:42.244 INFO:tasks.workunit.client.1.smithi173.stdout:9/697: mknod d5/d9/d20/d29/d2d/cd3 0 2024-04-20T11:09:42.246 INFO:tasks.workunit.client.1.smithi173.stdout:8/736: rename d4/c2c to d4/d18/d59/d5a/d7b/ce9 0 2024-04-20T11:09:42.251 INFO:tasks.workunit.client.1.smithi173.stdout:9/698: symlink d5/d9/d20/d4f/d5e/d7d/ld4 0 2024-04-20T11:09:42.252 INFO:tasks.workunit.client.1.smithi173.stdout:8/737: mknod d4/d18/da6/cea 0 2024-04-20T11:09:42.253 INFO:tasks.workunit.client.1.smithi173.stdout:7/721: dwrite d9/d33/f48 [4194304,4194304] 0 2024-04-20T11:09:42.255 INFO:tasks.workunit.client.1.smithi173.stdout:1/664: creat d5/d6/dc/fe0 x:0 0 0 2024-04-20T11:09:42.256 INFO:tasks.workunit.client.1.smithi173.stdout:9/699: truncate d5/d9/fc5 455543 0 2024-04-20T11:09:42.257 INFO:tasks.workunit.client.1.smithi173.stdout:6/712: rmdir d4/d59/dd9 0 2024-04-20T11:09:42.258 INFO:tasks.workunit.client.1.smithi173.stdout:4/800: dwrite da/d1e/d52/d7a/f7f [0,4194304] 0 2024-04-20T11:09:42.260 INFO:tasks.workunit.client.1.smithi173.stdout:1/665: write d5/d6/d17/d42/d12/f47 [4325232,63161] 0 2024-04-20T11:09:42.261 INFO:tasks.workunit.client.1.smithi173.stdout:6/713: write d4/f5a [318794,66489] 0 2024-04-20T11:09:42.267 INFO:tasks.workunit.client.1.smithi173.stdout:9/700: creat d5/d9/d20/d29/d2d/dcb/fd5 x:0 0 0 2024-04-20T11:09:42.270 INFO:tasks.workunit.client.1.smithi173.stdout:1/666: chown d5/d6/d17/d42/l78 8925 1 2024-04-20T11:09:42.270 INFO:tasks.workunit.client.1.smithi173.stdout:6/714: truncate d4/d1c/fa0 540499 0 2024-04-20T11:09:42.278 INFO:tasks.workunit.client.1.smithi173.stdout:5/735: sync 2024-04-20T11:09:42.278 INFO:tasks.workunit.client.1.smithi173.stdout:3/695: sync 2024-04-20T11:09:42.278 INFO:tasks.workunit.client.1.smithi173.stdout:8/738: read d4/f41 [555130,130396] 0 2024-04-20T11:09:42.279 INFO:tasks.workunit.client.1.smithi173.stdout:1/667: chown d5/d6/d17/d42/da/d20/f33 121126473 1 2024-04-20T11:09:42.282 INFO:tasks.workunit.client.1.smithi173.stdout:7/722: write d9/da/d17/fbb [150382,101027] 0 2024-04-20T11:09:42.285 INFO:tasks.workunit.client.1.smithi173.stdout:5/736: write da/d34/f6f [4549422,92618] 0 2024-04-20T11:09:42.286 INFO:tasks.workunit.client.1.smithi173.stdout:3/696: fdatasync d7/d17/d61/f97 0 2024-04-20T11:09:42.286 INFO:tasks.workunit.client.1.smithi173.stdout:4/801: dwrite da/dc/d15/f1d [4194304,4194304] 0 2024-04-20T11:09:42.287 INFO:tasks.workunit.client.1.smithi173.stdout:8/739: rmdir d4/d18/d59/d5a/da8/db9/de0/d9b 39 2024-04-20T11:09:42.290 INFO:tasks.workunit.client.1.smithi173.stdout:1/668: unlink d5/d6/dc/f2c 0 2024-04-20T11:09:42.293 INFO:tasks.workunit.client.1.smithi173.stdout:6/715: dwrite d4/d14/d5f/d9f/fca [4194304,4194304] 0 2024-04-20T11:09:42.293 INFO:tasks.workunit.client.1.smithi173.stdout:3/697: dread d7/d17/d61/d83/dcf/f2e [0,4194304] 0 2024-04-20T11:09:42.296 INFO:tasks.workunit.client.1.smithi173.stdout:3/698: write d7/d17/f2f [1976234,112212] 0 2024-04-20T11:09:42.304 INFO:tasks.workunit.client.1.smithi173.stdout:9/701: dwrite d5/d9/d20/d29/f64 [0,4194304] 0 2024-04-20T11:09:42.305 INFO:tasks.workunit.client.1.smithi173.stdout:1/669: mkdir d5/d6/d17/d42/d12/d1c/d85/de1 0 2024-04-20T11:09:42.320 INFO:tasks.workunit.client.1.smithi173.stdout:8/740: mkdir d4/d18/d4f/d5b/de4/de6/deb 0 2024-04-20T11:09:42.356 INFO:tasks.workunit.client.1.smithi173.stdout:9/702: mknod d5/d9/d20/d29/d18/d9b/d98/dc4/cd6 0 2024-04-20T11:09:42.356 INFO:tasks.workunit.client.1.smithi173.stdout:4/802: symlink da/d1e/dce/l110 0 2024-04-20T11:09:42.357 INFO:tasks.workunit.client.1.smithi173.stdout:5/737: readlink da/de/d19/d2c/dcf/lcb 0 2024-04-20T11:09:42.357 INFO:tasks.workunit.client.1.smithi173.stdout:1/670: symlink d5/d6/dc/le2 0 2024-04-20T11:09:42.359 INFO:tasks.workunit.client.1.smithi173.stdout:6/716: rmdir d4/d24/d3e/d9c/dc7/ddf 0 2024-04-20T11:09:42.360 INFO:tasks.workunit.client.1.smithi173.stdout:1/671: stat d5/d6/d17/d42/f54 0 2024-04-20T11:09:42.360 INFO:tasks.workunit.client.1.smithi173.stdout:7/723: getdents d9/d33/d35/d4a 0 2024-04-20T11:09:42.362 INFO:tasks.workunit.client.1.smithi173.stdout:1/672: chown d5/d6/d17/d42/d12/d1f 8594667 1 2024-04-20T11:09:42.363 INFO:tasks.workunit.client.1.smithi173.stdout:9/703: mkdir d5/dd7 0 2024-04-20T11:09:42.369 INFO:tasks.workunit.client.1.smithi173.stdout:9/704: truncate d5/d9/d20/d29/d2d/dcb/fcd 710173 0 2024-04-20T11:09:42.369 INFO:tasks.workunit.client.1.smithi173.stdout:1/673: dread d5/d6/d17/d42/da/d20/dad/dc2/f55 [0,4194304] 0 2024-04-20T11:09:42.369 INFO:tasks.workunit.client.1.smithi173.stdout:5/738: mkdir da/de/d19/d32/d30/d101 0 2024-04-20T11:09:42.369 INFO:tasks.workunit.client.1.smithi173.stdout:6/717: dread d4/d14/d5f/f76 [0,4194304] 0 2024-04-20T11:09:42.372 INFO:tasks.workunit.client.1.smithi173.stdout:8/741: mkdir d4/d18/d59/d5a/da8/db9/de0/d3c/d86/dec 0 2024-04-20T11:09:42.374 INFO:tasks.workunit.client.1.smithi173.stdout:7/724: dread d9/d33/f48 [4194304,4194304] 0 2024-04-20T11:09:42.376 INFO:tasks.workunit.client.1.smithi173.stdout:1/674: dread - d5/d6/d17/d42/d12/d1f/f95 zero size 2024-04-20T11:09:42.382 INFO:tasks.workunit.client.1.smithi173.stdout:5/739: rmdir da/de/d19/d2c 39 2024-04-20T11:09:42.385 INFO:tasks.workunit.client.1.smithi173.stdout:4/803: truncate da/d1a/d60/f6b 2954429 0 2024-04-20T11:09:42.394 INFO:tasks.workunit.client.1.smithi173.stdout:8/742: dread d4/d18/d59/d5a/da8/db9/de0/f46 [0,4194304] 0 2024-04-20T11:09:42.396 INFO:tasks.workunit.client.1.smithi173.stdout:6/718: creat d4/d1c/fe6 x:0 0 0 2024-04-20T11:09:42.397 INFO:tasks.workunit.client.1.smithi173.stdout:5/740: write da/d34/fd6 [2452231,29705] 0 2024-04-20T11:09:42.397 INFO:tasks.workunit.client.1.smithi173.stdout:7/725: mknod d9/da/d17/d2b/daa/dd9/c105 0 2024-04-20T11:09:42.399 INFO:tasks.workunit.client.1.smithi173.stdout:4/804: symlink da/d1e/l111 0 2024-04-20T11:09:42.400 INFO:tasks.workunit.client.1.smithi173.stdout:7/726: fdatasync d9/d33/d85/fc5 0 2024-04-20T11:09:42.404 INFO:tasks.workunit.client.1.smithi173.stdout:6/719: fdatasync d4/d18/f62 0 2024-04-20T11:09:42.405 INFO:tasks.workunit.client.1.smithi173.stdout:4/805: rmdir da/dd1 39 2024-04-20T11:09:42.405 INFO:tasks.workunit.client.1.smithi173.stdout:9/705: write d5/d9/d20/d29/d18/d8a/f99 [1355183,116089] 0 2024-04-20T11:09:42.406 INFO:tasks.workunit.client.1.smithi173.stdout:1/675: write d5/d6/d17/d42/da/d20/f33 [757988,82453] 0 2024-04-20T11:09:42.407 INFO:tasks.workunit.client.1.smithi173.stdout:5/741: mkdir da/de/d19/d32/d72/d102 0 2024-04-20T11:09:42.409 INFO:tasks.workunit.client.1.smithi173.stdout:4/806: chown da/dc/df4/d53/d39/f89 2036 1 2024-04-20T11:09:42.411 INFO:tasks.workunit.client.1.smithi173.stdout:7/727: rename d9/d33/d40/l7e to d9/da/d17/d2b/daa/dd9/l106 0 2024-04-20T11:09:42.413 INFO:tasks.workunit.client.1.smithi173.stdout:7/728: write d9/da/d17/d2b/daa/f6d [1480170,78543] 0 2024-04-20T11:09:42.417 INFO:tasks.workunit.client.1.smithi173.stdout:9/706: chown d5/d9/d20/d2b/d59/da1 7400 1 2024-04-20T11:09:42.419 INFO:tasks.workunit.client.1.smithi173.stdout:8/743: link d4/d18/d59/d5a/f95 d4/d18/d59/d5a/da8/db9/fed 0 2024-04-20T11:09:42.421 INFO:tasks.workunit.client.1.smithi173.stdout:6/720: dwrite d4/d18/f3f [0,4194304] 0 2024-04-20T11:09:42.422 INFO:tasks.workunit.client.1.smithi173.stdout:1/676: write d5/d6/d17/d42/d12/d1c/f6f [1477311,84605] 0 2024-04-20T11:09:42.424 INFO:tasks.workunit.client.1.smithi173.stdout:2/829: sync 2024-04-20T11:09:42.424 INFO:tasks.workunit.client.1.smithi173.stdout:0/676: sync 2024-04-20T11:09:42.425 INFO:tasks.workunit.client.1.smithi173.stdout:5/742: dwrite da/d34/fd6 [0,4194304] 0 2024-04-20T11:09:42.427 INFO:tasks.workunit.client.1.smithi173.stdout:4/807: unlink da/d1a/d6e/db4/ceb 0 2024-04-20T11:09:42.428 INFO:tasks.workunit.client.1.smithi173.stdout:0/677: chown d6/d8/d3c/fc8 394 1 2024-04-20T11:09:42.428 INFO:tasks.workunit.client.1.smithi173.stdout:5/743: fsync da/de/d19/d32/d72/f91 0 2024-04-20T11:09:42.429 INFO:tasks.workunit.client.1.smithi173.stdout:6/721: mknod d4/d24/d42/d81/ce7 0 2024-04-20T11:09:42.432 INFO:tasks.workunit.client.1.smithi173.stdout:4/808: truncate da/dc/df4/d53/fe9 588901 0 2024-04-20T11:09:42.435 INFO:tasks.workunit.client.1.smithi173.stdout:9/707: creat d5/d9/d20/d2b/d59/fd8 x:0 0 0 2024-04-20T11:09:42.437 INFO:tasks.workunit.client.1.smithi173.stdout:0/678: dread - d6/d54/d91/f9b zero size 2024-04-20T11:09:42.438 INFO:tasks.workunit.client.1.smithi173.stdout:8/744: dread d4/d18/d59/d6d/fba [0,4194304] 0 2024-04-20T11:09:42.438 INFO:tasks.workunit.client.1.smithi173.stdout:4/809: truncate da/dc/d13/fd8 334296 0 2024-04-20T11:09:42.440 INFO:tasks.workunit.client.1.smithi173.stdout:4/810: chown da/d1e/d52/d7a/lcc 137 1 2024-04-20T11:09:42.442 INFO:tasks.workunit.client.1.smithi173.stdout:6/722: rmdir d4/d1c 39 2024-04-20T11:09:42.446 INFO:tasks.workunit.client.1.smithi173.stdout:1/677: creat d5/d6/d17/d42/da/fe3 x:0 0 0 2024-04-20T11:09:42.451 INFO:tasks.workunit.client.1.smithi173.stdout:5/744: dwrite da/de/f37 [0,4194304] 0 2024-04-20T11:09:42.453 INFO:tasks.workunit.client.1.smithi173.stdout:2/830: write d2/d34/d58/f5a [6209556,60060] 0 2024-04-20T11:09:42.453 INFO:tasks.workunit.client.1.smithi173.stdout:4/811: rename da/dc/df4/d53/d39/f89 to da/d1e/d52/d9a/dd3/f112 0 2024-04-20T11:09:42.454 INFO:tasks.workunit.client.1.smithi173.stdout:0/679: unlink d6/f27 0 2024-04-20T11:09:42.455 INFO:tasks.workunit.client.1.smithi173.stdout:8/745: dread d4/d18/d59/d5a/d7b/d8c/fb5 [0,4194304] 0 2024-04-20T11:09:42.456 INFO:tasks.workunit.client.1.smithi173.stdout:0/680: stat d6/f1e 0 2024-04-20T11:09:42.459 INFO:tasks.workunit.client.1.smithi173.stdout:5/745: truncate da/de/d19/d32/f31 4706330 0 2024-04-20T11:09:42.459 INFO:tasks.workunit.client.1.smithi173.stdout:0/681: chown d6/d8/df/d43/d7f 460711 1 2024-04-20T11:09:42.459 INFO:tasks.workunit.client.1.smithi173.stdout:2/831: dread - d2/de/d11/d9d/f116 zero size 2024-04-20T11:09:42.460 INFO:tasks.workunit.client.1.smithi173.stdout:5/746: chown da/fd1 0 1 2024-04-20T11:09:42.460 INFO:tasks.workunit.client.1.smithi173.stdout:1/678: rename d5/d6/d17/fbf to d5/d6/fe4 0 2024-04-20T11:09:42.461 INFO:tasks.workunit.client.1.smithi173.stdout:4/812: dread - da/d1e/fb2 zero size 2024-04-20T11:09:42.462 INFO:tasks.workunit.client.1.smithi173.stdout:0/682: creat d6/d5f/fd7 x:0 0 0 2024-04-20T11:09:42.462 INFO:tasks.workunit.client.1.smithi173.stdout:2/832: fsync d2/d34/d58/fd3 0 2024-04-20T11:09:42.462 INFO:tasks.workunit.client.1.smithi173.stdout:8/746: symlink d4/d18/d59/d5a/d7b/d8c/lee 0 2024-04-20T11:09:42.465 INFO:tasks.workunit.client.1.smithi173.stdout:4/813: chown da/c93 200545 1 2024-04-20T11:09:42.467 INFO:tasks.workunit.client.1.smithi173.stdout:2/833: mkdir d2/d13/d118 0 2024-04-20T11:09:42.467 INFO:tasks.workunit.client.1.smithi173.stdout:1/679: fsync d5/d6/d17/d42/da/df/d40/db4/fd2 0 2024-04-20T11:09:42.470 INFO:tasks.workunit.client.1.smithi173.stdout:0/683: mknod d6/d8/df/d43/d7f/cd8 0 2024-04-20T11:09:42.471 INFO:tasks.workunit.client.1.smithi173.stdout:4/814: symlink da/d1a/d6e/db4/l113 0 2024-04-20T11:09:42.473 INFO:tasks.workunit.client.1.smithi173.stdout:4/815: stat da/dc/df4/fa2 0 2024-04-20T11:09:42.475 INFO:tasks.workunit.client.1.smithi173.stdout:0/684: chown d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/db9/ld5 611 1 2024-04-20T11:09:42.476 INFO:tasks.workunit.client.1.smithi173.stdout:8/747: rename d4/l2b to d4/d18/d59/lef 0 2024-04-20T11:09:42.477 INFO:tasks.workunit.client.1.smithi173.stdout:0/685: chown d6/d8/fbb 1456879 1 2024-04-20T11:09:42.482 INFO:tasks.workunit.client.1.smithi173.stdout:0/686: read d6/f7 [1633537,130724] 0 2024-04-20T11:09:42.482 INFO:tasks.workunit.client.1.smithi173.stdout:1/680: rename d5/d6/d17/d42/da/d20/d38/d52/f8d to d5/d6/d17/d42/da/df/d40/dac/fe5 0 2024-04-20T11:09:42.484 INFO:tasks.workunit.client.1.smithi173.stdout:2/834: getdents d2/de/d11 0 2024-04-20T11:09:42.485 INFO:tasks.workunit.client.1.smithi173.stdout:8/748: mknod d4/d18/d59/d5a/d7b/cf0 0 2024-04-20T11:09:42.493 INFO:tasks.workunit.client.1.smithi173.stdout:1/681: stat d5/d6/d17/d42/da/d20/d38/d52/d75/c84 0 2024-04-20T11:09:42.494 INFO:tasks.workunit.client.1.smithi173.stdout:8/749: dread d4/d18/d4f/f9a [0,4194304] 0 2024-04-20T11:09:42.494 INFO:tasks.workunit.client.1.smithi173.stdout:4/816: write da/dc/d13/f47 [637644,75004] 0 2024-04-20T11:09:42.497 INFO:tasks.workunit.client.1.smithi173.stdout:1/682: chown d5/d6/d17/d42/da/d20/d38/fd5 15966710 1 2024-04-20T11:09:42.498 INFO:tasks.workunit.client.1.smithi173.stdout:1/683: readlink d5/l9d 0 2024-04-20T11:09:42.498 INFO:tasks.workunit.client.1.smithi173.stdout:5/747: dread da/de/d75/d3e/f60 [0,4194304] 0 2024-04-20T11:09:42.499 INFO:tasks.workunit.client.1.smithi173.stdout:0/687: dwrite d6/d8/f12 [0,4194304] 0 2024-04-20T11:09:42.499 INFO:tasks.workunit.client.1.smithi173.stdout:2/835: chown d2/d13/da5 58897925 1 2024-04-20T11:09:42.501 INFO:tasks.workunit.client.1.smithi173.stdout:2/836: fdatasync d2/de/f60 0 2024-04-20T11:09:42.502 INFO:tasks.workunit.client.1.smithi173.stdout:0/688: chown d6/d8/df/d2c/d42/l6f 173774 1 2024-04-20T11:09:42.504 INFO:tasks.workunit.client.1.smithi173.stdout:0/689: chown d6/l48 1 1 2024-04-20T11:09:42.507 INFO:tasks.workunit.client.1.smithi173.stdout:5/748: unlink da/d4f/d5e/f98 0 2024-04-20T11:09:42.512 INFO:tasks.workunit.client.1.smithi173.stdout:4/817: write da/d1e/d52/f97 [48634,20385] 0 2024-04-20T11:09:42.514 INFO:tasks.workunit.client.1.smithi173.stdout:1/684: dwrite d5/d6/d17/d42/d12/f44 [0,4194304] 0 2024-04-20T11:09:42.515 INFO:tasks.workunit.client.1.smithi173.stdout:4/818: chown da/dc/f10b 8 1 2024-04-20T11:09:42.517 INFO:tasks.workunit.client.1.smithi173.stdout:5/749: read da/de/d19/d32/d30/f4c [6020924,110371] 0 2024-04-20T11:09:42.519 INFO:tasks.workunit.client.1.smithi173.stdout:0/690: symlink d6/d8/d3c/d77/ld9 0 2024-04-20T11:09:42.524 INFO:tasks.workunit.client.1.smithi173.stdout:2/837: symlink d2/d13/d9a/dbc/dfd/d53/d4e/l119 0 2024-04-20T11:09:42.533 INFO:tasks.workunit.client.1.smithi173.stdout:8/750: getdents d4/d18/d59/d5a/d7b/db1 0 2024-04-20T11:09:42.536 INFO:tasks.workunit.client.1.smithi173.stdout:4/819: getdents da/dc/d13/d106 0 2024-04-20T11:09:42.537 INFO:tasks.workunit.client.1.smithi173.stdout:1/685: truncate d5/d6/f9 1969407 0 2024-04-20T11:09:42.540 INFO:tasks.workunit.client.1.smithi173.stdout:8/751: rmdir d4/d18/d4f/d5b/de4 39 2024-04-20T11:09:42.540 INFO:tasks.workunit.client.1.smithi173.stdout:5/750: link da/de/d75/c17 da/d4f/d65/c103 0 2024-04-20T11:09:42.541 INFO:tasks.workunit.client.1.smithi173.stdout:1/686: dread - d5/d6/d17/d42/d12/d1f/f95 zero size 2024-04-20T11:09:42.543 INFO:tasks.workunit.client.1.smithi173.stdout:5/751: chown da/de/f37 52700 1 2024-04-20T11:09:42.543 INFO:tasks.workunit.client.1.smithi173.stdout:0/691: dwrite d6/d8/d3c/d44/d75/f7b [0,4194304] 0 2024-04-20T11:09:42.547 INFO:tasks.workunit.client.1.smithi173.stdout:5/752: fdatasync da/de/d75/fab 0 2024-04-20T11:09:42.548 INFO:tasks.workunit.client.1.smithi173.stdout:3/699: sync 2024-04-20T11:09:42.549 INFO:tasks.workunit.client.1.smithi173.stdout:2/838: getdents d2/d13/d28/d4a/dff 0 2024-04-20T11:09:42.550 INFO:tasks.workunit.client.1.smithi173.stdout:1/687: symlink d5/d6/d17/d42/da/d20/dad/le6 0 2024-04-20T11:09:42.552 INFO:tasks.workunit.client.1.smithi173.stdout:0/692: unlink d6/d8/df/d15/f7e 0 2024-04-20T11:09:42.552 INFO:tasks.workunit.client.1.smithi173.stdout:3/700: write d7/d17/d61/fc3 [4586370,100627] 0 2024-04-20T11:09:42.555 INFO:tasks.workunit.client.1.smithi173.stdout:8/752: truncate d4/d18/d59/d5a/d7b/d8c/f8d 608006 0 2024-04-20T11:09:42.556 INFO:tasks.workunit.client.1.smithi173.stdout:3/701: readlink d7/d17/d58/dd0/ld2 0 2024-04-20T11:09:42.561 INFO:tasks.workunit.client.1.smithi173.stdout:3/702: mknod d7/d2b/d9d/cdf 0 2024-04-20T11:09:42.563 INFO:tasks.workunit.client.1.smithi173.stdout:5/753: symlink da/de/d19/d32/d72/l104 0 2024-04-20T11:09:42.565 INFO:tasks.workunit.client.1.smithi173.stdout:8/753: creat d4/d18/d4f/d5b/ff1 x:0 0 0 2024-04-20T11:09:42.565 INFO:tasks.workunit.client.1.smithi173.stdout:3/703: read - d7/d17/d61/d83/fbf zero size 2024-04-20T11:09:42.571 INFO:tasks.workunit.client.1.smithi173.stdout:8/754: fsync d4/d18/d59/d5a/da8/db9/de0/d3c/fa0 0 2024-04-20T11:09:42.573 INFO:tasks.workunit.client.1.smithi173.stdout:3/704: read d7/d17/d61/d83/dcf/f46 [2788813,69332] 0 2024-04-20T11:09:42.573 INFO:tasks.workunit.client.1.smithi173.stdout:5/754: creat da/d9e/f105 x:0 0 0 2024-04-20T11:09:42.575 INFO:tasks.workunit.client.1.smithi173.stdout:4/820: dread da/dc/f27 [0,4194304] 0 2024-04-20T11:09:42.575 INFO:tasks.workunit.client.1.smithi173.stdout:8/755: fsync d4/d18/d4f/f96 0 2024-04-20T11:09:42.578 INFO:tasks.workunit.client.1.smithi173.stdout:1/688: dwrite d5/d6/d17/d42/da/d20/fd6 [0,4194304] 0 2024-04-20T11:09:42.581 INFO:tasks.workunit.client.1.smithi173.stdout:8/756: truncate d4/f10 4975820 0 2024-04-20T11:09:42.587 INFO:tasks.workunit.client.1.smithi173.stdout:1/689: creat d5/d6/d17/d42/da/df/fe7 x:0 0 0 2024-04-20T11:09:42.589 INFO:tasks.workunit.client.1.smithi173.stdout:1/690: fdatasync f0 0 2024-04-20T11:09:42.590 INFO:tasks.workunit.client.1.smithi173.stdout:8/757: creat d4/d18/d59/d5a/d7b/ddc/ff2 x:0 0 0 2024-04-20T11:09:42.593 INFO:tasks.workunit.client.1.smithi173.stdout:3/705: dread f4 [0,4194304] 0 2024-04-20T11:09:42.594 INFO:tasks.workunit.client.1.smithi173.stdout:2/839: dread d2/d13/d9a/dbc/dfd/d53/d4e/f59 [0,4194304] 0 2024-04-20T11:09:42.594 INFO:tasks.workunit.client.1.smithi173.stdout:8/758: chown d4/d18/d59/cd1 110 1 2024-04-20T11:09:42.595 INFO:tasks.workunit.client.1.smithi173.stdout:4/821: link da/dc/d13/l5d da/d1a/d60/d66/dae/l114 0 2024-04-20T11:09:42.596 INFO:tasks.workunit.client.1.smithi173.stdout:5/755: dwrite da/de/d19/f5c [0,4194304] 0 2024-04-20T11:09:42.602 INFO:tasks.workunit.client.1.smithi173.stdout:4/822: creat da/d1e/d52/d9a/dd3/f115 x:0 0 0 2024-04-20T11:09:42.604 INFO:tasks.workunit.client.1.smithi173.stdout:0/693: dread d6/d8/df/d2c/d42/d50/fb5 [0,4194304] 0 2024-04-20T11:09:42.604 INFO:tasks.workunit.client.1.smithi173.stdout:5/756: dread - da/d4f/d65/d8e/f9f zero size 2024-04-20T11:09:42.607 INFO:tasks.workunit.client.1.smithi173.stdout:8/759: creat d4/d18/ff3 x:0 0 0 2024-04-20T11:09:42.610 INFO:tasks.workunit.client.1.smithi173.stdout:4/823: read da/d1a/d60/f6b [1067130,57113] 0 2024-04-20T11:09:42.611 INFO:tasks.workunit.client.1.smithi173.stdout:4/824: chown da/dc/d13/d106 704745 1 2024-04-20T11:09:42.613 INFO:tasks.workunit.client.1.smithi173.stdout:2/840: getdents d2/d13 0 2024-04-20T11:09:42.614 INFO:tasks.workunit.client.1.smithi173.stdout:0/694: fdatasync d6/d8/df/d15/f70 0 2024-04-20T11:09:42.615 INFO:tasks.workunit.client.1.smithi173.stdout:2/841: write d2/f5c [1312102,4484] 0 2024-04-20T11:09:42.617 INFO:tasks.workunit.client.1.smithi173.stdout:0/695: readlink d6/d8/d3c/d44/d75/l85 0 2024-04-20T11:09:42.618 INFO:tasks.workunit.client.1.smithi173.stdout:2/842: write d2/fd8 [330306,37725] 0 2024-04-20T11:09:42.626 INFO:tasks.workunit.client.1.smithi173.stdout:1/691: dwrite d5/d6/d17/d42/d12/d1f/fbc [0,4194304] 0 2024-04-20T11:09:42.628 INFO:tasks.workunit.client.1.smithi173.stdout:8/760: symlink d4/d18/d59/d5a/d7b/ddc/lf4 0 2024-04-20T11:09:42.630 INFO:tasks.workunit.client.1.smithi173.stdout:4/825: mknod da/d1a/d60/d7d/c116 0 2024-04-20T11:09:42.632 INFO:tasks.workunit.client.1.smithi173.stdout:8/761: write d4/d18/fcc [460242,74375] 0 2024-04-20T11:09:42.633 INFO:tasks.workunit.client.1.smithi173.stdout:0/696: mknod d6/d8/d3c/d44/d75/cda 0 2024-04-20T11:09:42.637 INFO:tasks.workunit.client.1.smithi173.stdout:3/706: dread d7/d2b/f62 [0,4194304] 0 2024-04-20T11:09:42.637 INFO:tasks.workunit.client.1.smithi173.stdout:5/757: mkdir da/de/d19/d2c/d106 0 2024-04-20T11:09:42.641 INFO:tasks.workunit.client.1.smithi173.stdout:2/843: dwrite d2/fd8 [0,4194304] 0 2024-04-20T11:09:42.642 INFO:tasks.workunit.client.1.smithi173.stdout:3/707: creat d7/fe0 x:0 0 0 2024-04-20T11:09:42.643 INFO:tasks.workunit.client.1.smithi173.stdout:2/844: fdatasync d2/de/f106 0 2024-04-20T11:09:42.645 INFO:tasks.workunit.client.1.smithi173.stdout:5/758: mkdir da/d4f/d65/dd5/d107 0 2024-04-20T11:09:42.648 INFO:tasks.workunit.client.1.smithi173.stdout:4/826: dread da/dc/f27 [0,4194304] 0 2024-04-20T11:09:42.648 INFO:tasks.workunit.client.1.smithi173.stdout:0/697: dread d6/d5f/d8f/fa7 [0,4194304] 0 2024-04-20T11:09:42.649 INFO:tasks.workunit.client.1.smithi173.stdout:5/759: chown da/de/d75/d40/d59/f96 0 1 2024-04-20T11:09:42.651 INFO:tasks.workunit.client.1.smithi173.stdout:8/762: dread d4/d18/d4f/f7e [0,4194304] 0 2024-04-20T11:09:42.653 INFO:tasks.workunit.client.1.smithi173.stdout:5/760: chown da/de/d19/d32/d30/dbb 0 1 2024-04-20T11:09:42.653 INFO:tasks.workunit.client.1.smithi173.stdout:4/827: dread da/dc/d13/fd8 [0,4194304] 0 2024-04-20T11:09:42.654 INFO:tasks.workunit.client.1.smithi173.stdout:0/698: unlink d6/d8/d3c/d77/ld9 0 2024-04-20T11:09:42.658 INFO:tasks.workunit.client.1.smithi173.stdout:4/828: creat da/dc/df4/d53/d39/f117 x:0 0 0 2024-04-20T11:09:42.660 INFO:tasks.workunit.client.1.smithi173.stdout:7/729: sync 2024-04-20T11:09:42.663 INFO:tasks.workunit.client.1.smithi173.stdout:0/699: mkdir d6/d8/d3c/d44/ddb 0 2024-04-20T11:09:42.665 INFO:tasks.workunit.client.1.smithi173.stdout:7/730: creat d9/d1f/d7f/dce/db2/f107 x:0 0 0 2024-04-20T11:09:42.666 INFO:tasks.workunit.client.1.smithi173.stdout:5/761: dread da/de/d19/d2c/f46 [0,4194304] 0 2024-04-20T11:09:42.667 INFO:tasks.workunit.client.1.smithi173.stdout:7/731: write d9/dbc/fbe [881364,47276] 0 2024-04-20T11:09:42.668 INFO:tasks.workunit.client.1.smithi173.stdout:7/732: stat d9/d1f/d72/d8f 0 2024-04-20T11:09:42.669 INFO:tasks.workunit.client.1.smithi173.stdout:1/692: truncate d5/d6/d17/d42/da/d20/f33 531290 0 2024-04-20T11:09:42.673 INFO:tasks.workunit.client.1.smithi173.stdout:8/763: link d4/d18/d59/d5a/da8/db9/de0/lbd d4/d18/d59/d5a/da8/db9/de0/d3c/d86/dec/lf5 0 2024-04-20T11:09:42.675 INFO:tasks.workunit.client.1.smithi173.stdout:3/708: fsync d7/d17/d61/d83/dcf/d30/f60 0 2024-04-20T11:09:42.677 INFO:tasks.workunit.client.1.smithi173.stdout:1/693: mknod d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/ce8 0 2024-04-20T11:09:42.678 INFO:tasks.workunit.client.1.smithi173.stdout:2/845: write d2/d34/d58/f69 [1977386,40887] 0 2024-04-20T11:09:42.678 INFO:tasks.workunit.client.1.smithi173.stdout:8/764: dread d4/fbb [0,4194304] 0 2024-04-20T11:09:42.681 INFO:tasks.workunit.client.1.smithi173.stdout:4/829: getdents da/d1e/d52/da3 0 2024-04-20T11:09:42.683 INFO:tasks.workunit.client.1.smithi173.stdout:7/733: mkdir d9/d33/d35/d4a/d52/df2/dc6/df3/d108 0 2024-04-20T11:09:42.684 INFO:tasks.workunit.client.1.smithi173.stdout:4/830: write da/d1e/d52/ff6 [492367,20350] 0 2024-04-20T11:09:42.684 INFO:tasks.workunit.client.1.smithi173.stdout:2/846: mknod d2/d13/da5/db2/c11a 0 2024-04-20T11:09:42.688 INFO:tasks.workunit.client.1.smithi173.stdout:3/709: creat d7/d17/d58/d8a/dc4/dd1/fe1 x:0 0 0 2024-04-20T11:09:42.693 INFO:tasks.workunit.client.1.smithi173.stdout:6/723: sync 2024-04-20T11:09:42.693 INFO:tasks.workunit.client.1.smithi173.stdout:9/708: sync 2024-04-20T11:09:42.694 INFO:tasks.workunit.client.1.smithi173.stdout:8/765: rename d4/d18/da6/faf to d4/d18/ff6 0 2024-04-20T11:09:42.694 INFO:tasks.workunit.client.1.smithi173.stdout:0/700: dread d6/d8/d3c/d44/d75/f97 [0,4194304] 0 2024-04-20T11:09:42.699 INFO:tasks.workunit.client.1.smithi173.stdout:3/710: read d7/d53/fc1 [2321345,3437] 0 2024-04-20T11:09:42.699 INFO:tasks.workunit.client.1.smithi173.stdout:9/709: read d5/d9/d20/d4f/d5e/d9e/fa6 [2464735,49295] 0 2024-04-20T11:09:42.701 INFO:tasks.workunit.client.1.smithi173.stdout:4/831: rename da/d1e/d52/f85 to da/dd1/df5/f118 0 2024-04-20T11:09:42.705 INFO:tasks.workunit.client.1.smithi173.stdout:3/711: creat d7/d17/fe2 x:0 0 0 2024-04-20T11:09:42.706 INFO:tasks.workunit.client.1.smithi173.stdout:0/701: dread d6/d8/df/d2c/d42/d50/fb5 [0,4194304] 0 2024-04-20T11:09:42.708 INFO:tasks.workunit.client.1.smithi173.stdout:0/702: write d6/d54/f7d [3177710,19961] 0 2024-04-20T11:09:42.711 INFO:tasks.workunit.client.1.smithi173.stdout:6/724: readlink d4/d1c/d8e/d90/lc2 0 2024-04-20T11:09:42.712 INFO:tasks.workunit.client.1.smithi173.stdout:9/710: symlink d5/dd7/ld9 0 2024-04-20T11:09:42.712 INFO:tasks.workunit.client.1.smithi173.stdout:6/725: chown d4/d1c/l3d 5 1 2024-04-20T11:09:42.716 INFO:tasks.workunit.client.1.smithi173.stdout:4/832: symlink da/d1a/l119 0 2024-04-20T11:09:42.721 INFO:tasks.workunit.client.1.smithi173.stdout:6/726: write d4/f60 [595017,26539] 0 2024-04-20T11:09:42.731 INFO:tasks.workunit.client.1.smithi173.stdout:3/712: rename d7/d17/d61/d83/d45/f63 to d7/d17/d61/d83/dcf/fe3 0 2024-04-20T11:09:42.732 INFO:tasks.workunit.client.1.smithi173.stdout:9/711: dread d5/d4c/f72 [0,4194304] 0 2024-04-20T11:09:42.733 INFO:tasks.workunit.client.1.smithi173.stdout:0/703: dwrite d6/d8/df/f16 [4194304,4194304] 0 2024-04-20T11:09:42.735 INFO:tasks.workunit.client.1.smithi173.stdout:5/762: write da/de/d19/d2c/f62 [2606019,73268] 0 2024-04-20T11:09:42.741 INFO:tasks.workunit.client.1.smithi173.stdout:1/694: truncate d5/d6/d17/d42/d12/d1f/f63 512256 0 2024-04-20T11:09:42.743 INFO:tasks.workunit.client.1.smithi173.stdout:2/847: write d2/d13/d9a/dbc/dfd/f85 [3933472,9063] 0 2024-04-20T11:09:42.744 INFO:tasks.workunit.client.1.smithi173.stdout:7/734: truncate d9/d33/d40/d7a/ff0 213779 0 2024-04-20T11:09:42.754 INFO:tasks.workunit.client.1.smithi173.stdout:6/727: rename d4/d24/d3e/fbb to d4/d14/fe8 0 2024-04-20T11:09:42.757 INFO:tasks.workunit.client.1.smithi173.stdout:3/713: dwrite d7/d17/d61/d83/dcf/d30/fc7 [0,4194304] 0 2024-04-20T11:09:42.758 INFO:tasks.workunit.client.1.smithi173.stdout:7/735: creat d9/d33/d35/d54/db3/f109 x:0 0 0 2024-04-20T11:09:42.759 INFO:tasks.workunit.client.1.smithi173.stdout:1/695: mknod d5/d6/d17/d42/da/d20/dad/dc2/ce9 0 2024-04-20T11:09:42.762 INFO:tasks.workunit.client.1.smithi173.stdout:6/728: write d4/d24/d42/d74/d3b/f41 [2830073,128387] 0 2024-04-20T11:09:42.765 INFO:tasks.workunit.client.1.smithi173.stdout:8/766: truncate d4/d18/d4f/f57 3531665 0 2024-04-20T11:09:42.774 INFO:tasks.workunit.client.1.smithi173.stdout:9/712: dwrite d5/d9/d20/d29/d2d/f43 [0,4194304] 0 2024-04-20T11:09:42.774 INFO:tasks.workunit.client.1.smithi173.stdout:6/729: read d4/d18/f62 [2842705,119721] 0 2024-04-20T11:09:42.777 INFO:tasks.workunit.client.1.smithi173.stdout:7/736: fdatasync d9/d33/fcc 0 2024-04-20T11:09:42.781 INFO:tasks.workunit.client.1.smithi173.stdout:3/714: chown d7/d17/d61/d83/d45/f56 65 1 2024-04-20T11:09:42.793 INFO:tasks.workunit.client.1.smithi173.stdout:5/763: dwrite da/fd1 [0,4194304] 0 2024-04-20T11:09:42.794 INFO:tasks.workunit.client.1.smithi173.stdout:3/715: symlink d7/d17/d61/le4 0 2024-04-20T11:09:42.794 INFO:tasks.workunit.client.1.smithi173.stdout:9/713: fsync d5/d9/d20/d29/d2d/d33/f6b 0 2024-04-20T11:09:42.798 INFO:tasks.workunit.client.1.smithi173.stdout:3/716: chown d7/d53/fc1 2146338 1 2024-04-20T11:09:42.798 INFO:tasks.workunit.client.1.smithi173.stdout:6/730: rename d4/d14/d5f/daf/fe0 to d4/d14/d5f/daf/fe9 0 2024-04-20T11:09:42.802 INFO:tasks.workunit.client.1.smithi173.stdout:6/731: chown d4/d18/d7e/lc3 1539 1 2024-04-20T11:09:42.804 INFO:tasks.workunit.client.1.smithi173.stdout:6/732: write d4/d14/f1a [4639888,9027] 0 2024-04-20T11:09:42.806 INFO:tasks.workunit.client.1.smithi173.stdout:7/737: creat d9/d33/d35/d4a/def/f10a x:0 0 0 2024-04-20T11:09:42.815 INFO:tasks.workunit.client.1.smithi173.stdout:9/714: creat d5/d9/d20/d29/d2d/d33/fda x:0 0 0 2024-04-20T11:09:42.815 INFO:tasks.workunit.client.1.smithi173.stdout:2/848: dread d2/d34/f45 [4194304,4194304] 0 2024-04-20T11:09:42.815 INFO:tasks.workunit.client.1.smithi173.stdout:5/764: dread da/de/d19/d2c/f46 [0,4194304] 0 2024-04-20T11:09:42.818 INFO:tasks.workunit.client.1.smithi173.stdout:2/849: chown d2/d13/d9a/dbc/dfd/l110 447218294 1 2024-04-20T11:09:42.825 INFO:tasks.workunit.client.1.smithi173.stdout:3/717: symlink d7/d17/d58/d8a/dc4/le5 0 2024-04-20T11:09:42.826 INFO:tasks.workunit.client.1.smithi173.stdout:9/715: write d5/d9/d20/d29/d18/f70 [764541,72696] 0 2024-04-20T11:09:42.832 INFO:tasks.workunit.client.1.smithi173.stdout:7/738: write d9/d33/fcc [2600966,121362] 0 2024-04-20T11:09:42.832 INFO:tasks.workunit.client.1.smithi173.stdout:6/733: rename d4/d14/l17 to d4/d24/d3e/d9c/dc7/lea 0 2024-04-20T11:09:42.834 INFO:tasks.workunit.client.1.smithi173.stdout:3/718: read d7/d17/d61/d83/dcf/d30/f60 [392485,92521] 0 2024-04-20T11:09:42.835 INFO:tasks.workunit.client.1.smithi173.stdout:9/716: chown d5/d9/d20/d4f/d5e/d9e/dc2 12 1 2024-04-20T11:09:42.836 INFO:tasks.workunit.client.1.smithi173.stdout:6/734: fdatasync d4/d14/d5f/d9f/fca 0 2024-04-20T11:09:42.837 INFO:tasks.workunit.client.1.smithi173.stdout:8/767: getdents d4/d18/d59/d5a/d7b 0 2024-04-20T11:09:42.842 INFO:tasks.workunit.client.1.smithi173.stdout:9/717: read d5/d9/d20/d2b/f49 [2846624,94794] 0 2024-04-20T11:09:42.842 INFO:tasks.workunit.client.1.smithi173.stdout:2/850: dread d2/d34/d58/f63 [0,4194304] 0 2024-04-20T11:09:42.855 INFO:tasks.workunit.client.1.smithi173.stdout:7/739: dwrite d9/d33/d35/d54/db3/f109 [0,4194304] 0 2024-04-20T11:09:42.876 INFO:tasks.workunit.client.1.smithi173.stdout:5/765: symlink da/de/d19/d32/d30/dbb/l108 0 2024-04-20T11:09:42.876 INFO:tasks.workunit.client.1.smithi173.stdout:1/696: link d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fdb d5/d6/d17/d42/da/d20/dad/dc2/d94/fea 0 2024-04-20T11:09:42.881 INFO:tasks.workunit.client.1.smithi173.stdout:6/735: mknod d4/d1c/d8e/ceb 0 2024-04-20T11:09:42.883 INFO:tasks.workunit.client.1.smithi173.stdout:9/718: unlink d5/d9/d20/d2b/fa9 0 2024-04-20T11:09:42.889 INFO:tasks.workunit.client.1.smithi173.stdout:7/740: dread - d9/fbf zero size 2024-04-20T11:09:42.892 INFO:tasks.workunit.client.1.smithi173.stdout:3/719: mknod d7/d17/d61/d83/dcf/ce6 0 2024-04-20T11:09:42.895 INFO:tasks.workunit.client.1.smithi173.stdout:1/697: dread d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fdb [0,4194304] 0 2024-04-20T11:09:42.897 INFO:tasks.workunit.client.1.smithi173.stdout:1/698: chown d5/d6/c25 183346314 1 2024-04-20T11:09:42.898 INFO:tasks.workunit.client.1.smithi173.stdout:9/719: dwrite d5/d9/d20/d2b/d31/f4d [0,4194304] 0 2024-04-20T11:09:42.900 INFO:tasks.workunit.client.1.smithi173.stdout:5/766: mkdir da/de/d19/d32/d30/d109 0 2024-04-20T11:09:42.902 INFO:tasks.workunit.client.1.smithi173.stdout:5/767: fsync da/de/d19/fe5 0 2024-04-20T11:09:42.902 INFO:tasks.workunit.client.1.smithi173.stdout:7/741: mkdir d9/da/d17/d2b/daa/d10b 0 2024-04-20T11:09:42.904 INFO:tasks.workunit.client.1.smithi173.stdout:3/720: truncate d7/d17/d61/d83/fbf 1002458 0 2024-04-20T11:09:42.905 INFO:tasks.workunit.client.1.smithi173.stdout:5/768: fsync da/de/d19/d32/d30/f70 0 2024-04-20T11:09:42.910 INFO:tasks.workunit.client.1.smithi173.stdout:1/699: mknod d5/d6/dc/d7f/ceb 0 2024-04-20T11:09:42.911 INFO:tasks.workunit.client.1.smithi173.stdout:9/720: symlink d5/d9/d20/d4f/d5e/d7d/dba/ldb 0 2024-04-20T11:09:42.914 INFO:tasks.workunit.client.1.smithi173.stdout:7/742: mkdir d9/d33/d35/d54/d10c 0 2024-04-20T11:09:42.914 INFO:tasks.workunit.client.1.smithi173.stdout:4/833: sync 2024-04-20T11:09:42.914 INFO:tasks.workunit.client.1.smithi173.stdout:0/704: sync 2024-04-20T11:09:42.918 INFO:tasks.workunit.client.1.smithi173.stdout:5/769: rename da/de/d19/d2c/f62 to da/d4f/d65/f10a 0 2024-04-20T11:09:42.919 INFO:tasks.workunit.client.1.smithi173.stdout:8/768: dwrite d4/d18/d4f/f57 [0,4194304] 0 2024-04-20T11:09:42.919 INFO:tasks.workunit.client.1.smithi173.stdout:0/705: truncate d6/d8/d51/fce 4939590 0 2024-04-20T11:09:42.920 INFO:tasks.workunit.client.1.smithi173.stdout:2/851: getdents d2/d13/d9a/dbc/dfd/d53/d4e 0 2024-04-20T11:09:42.921 INFO:tasks.workunit.client.1.smithi173.stdout:8/769: readlink d4/l12 0 2024-04-20T11:09:42.923 INFO:tasks.workunit.client.1.smithi173.stdout:8/770: write d4/fd5 [606739,50659] 0 2024-04-20T11:09:42.924 INFO:tasks.workunit.client.1.smithi173.stdout:6/736: getdents d4/d24/d42/d74 0 2024-04-20T11:09:42.925 INFO:tasks.workunit.client.1.smithi173.stdout:0/706: write d6/d8/df/da0/fb3 [1006034,50502] 0 2024-04-20T11:09:42.931 INFO:tasks.workunit.client.1.smithi173.stdout:1/700: rename d5/d6/d17/d42/d12/d1c/l45 to d5/d6/d17/d42/da/d20/dad/lec 0 2024-04-20T11:09:42.934 INFO:tasks.workunit.client.1.smithi173.stdout:4/834: dwrite da/dc/fb8 [0,4194304] 0 2024-04-20T11:09:42.940 INFO:tasks.workunit.client.1.smithi173.stdout:1/701: chown d5/d6/d17/d42/da/d20/dad/dc2/d94/fc1 228533 1 2024-04-20T11:09:42.940 INFO:tasks.workunit.client.1.smithi173.stdout:4/835: write da/dc/d13/f2b [2155290,30055] 0 2024-04-20T11:09:42.940 INFO:tasks.workunit.client.1.smithi173.stdout:2/852: chown d2/d13/d9a/dbc/dfd/d53/lb1 63749 1 2024-04-20T11:09:42.943 INFO:tasks.workunit.client.1.smithi173.stdout:0/707: read d6/d8/df/d2c/d42/f7a [2749,80409] 0 2024-04-20T11:09:42.951 INFO:tasks.workunit.client.1.smithi173.stdout:9/721: dread d5/faf [0,4194304] 0 2024-04-20T11:09:42.952 INFO:tasks.workunit.client.1.smithi173.stdout:4/836: write da/dc/df4/f91 [956458,93560] 0 2024-04-20T11:09:42.954 INFO:tasks.workunit.client.1.smithi173.stdout:7/743: rename d9/d1f/d7f/dce/f70 to d9/d33/d35/d54/db3/d102/f10d 0 2024-04-20T11:09:42.955 INFO:tasks.workunit.client.1.smithi173.stdout:0/708: mknod d6/d8/cdc 0 2024-04-20T11:09:42.957 INFO:tasks.workunit.client.1.smithi173.stdout:3/721: getdents d7/d2b 0 2024-04-20T11:09:42.958 INFO:tasks.workunit.client.1.smithi173.stdout:3/722: fsync d7/d17/fa8 0 2024-04-20T11:09:42.960 INFO:tasks.workunit.client.1.smithi173.stdout:5/770: rename da/de/l14 to da/d4f/l10b 0 2024-04-20T11:09:42.962 INFO:tasks.workunit.client.1.smithi173.stdout:7/744: write d9/da/d2e/fe7 [1585936,31625] 0 2024-04-20T11:09:42.965 INFO:tasks.workunit.client.1.smithi173.stdout:5/771: chown da/de/d75/d40/d59/c87 23218282 1 2024-04-20T11:09:42.966 INFO:tasks.workunit.client.1.smithi173.stdout:6/737: rename d4/d24/d42/d74/f53 to d4/d24/d3e/d9c/fec 0 2024-04-20T11:09:42.967 INFO:tasks.workunit.client.1.smithi173.stdout:3/723: truncate d7/d53/fc1 1733045 0 2024-04-20T11:09:42.967 INFO:tasks.workunit.client.1.smithi173.stdout:6/738: chown d4/d59/d8f 4 1 2024-04-20T11:09:42.967 INFO:tasks.workunit.client.1.smithi173.stdout:2/853: dwrite d2/de/d11/d9d/d87/fb4 [4194304,4194304] 0 2024-04-20T11:09:42.970 INFO:tasks.workunit.client.1.smithi173.stdout:4/837: link da/d1a/d60/d66/d9b/cbc da/d1a/d60/d7d/d101/c11a 0 2024-04-20T11:09:42.971 INFO:tasks.workunit.client.1.smithi173.stdout:9/722: truncate d5/f2f 1618663 0 2024-04-20T11:09:42.972 INFO:tasks.workunit.client.1.smithi173.stdout:3/724: rmdir d7/d17/d61/d83/d45 39 2024-04-20T11:09:42.975 INFO:tasks.workunit.client.1.smithi173.stdout:6/739: mknod d4/d1c/d8e/ced 0 2024-04-20T11:09:42.978 INFO:tasks.workunit.client.1.smithi173.stdout:4/838: creat da/dc/df4/f11b x:0 0 0 2024-04-20T11:09:42.979 INFO:tasks.workunit.client.1.smithi173.stdout:3/725: creat d7/d17/d61/d83/dcf/d5c/fe7 x:0 0 0 2024-04-20T11:09:42.983 INFO:tasks.workunit.client.1.smithi173.stdout:5/772: rmdir da/de/d19/dca/d48/da2/de0 0 2024-04-20T11:09:42.985 INFO:tasks.workunit.client.1.smithi173.stdout:8/771: dread d4/d18/d59/d5a/da8/db9/de0/f7a [0,4194304] 0 2024-04-20T11:09:42.986 INFO:tasks.workunit.client.1.smithi173.stdout:7/745: getdents d9/da/dc3 0 2024-04-20T11:09:42.993 INFO:tasks.workunit.client.1.smithi173.stdout:2/854: getdents d2/de/d11/d9d 0 2024-04-20T11:09:42.993 INFO:tasks.workunit.client.1.smithi173.stdout:9/723: dwrite d5/d9/d20/d29/d2d/d33/f6b [4194304,4194304] 0 2024-04-20T11:09:42.997 INFO:tasks.workunit.client.1.smithi173.stdout:7/746: write d9/d33/d40/dc9/fda [43521,15747] 0 2024-04-20T11:09:42.999 INFO:tasks.workunit.client.1.smithi173.stdout:2/855: rmdir d2/d13/d9a/dbc/dfd/d4b/dae/db6/df9 39 2024-04-20T11:09:42.999 INFO:tasks.workunit.client.1.smithi173.stdout:9/724: write d5/d9/d20/d29/d2d/d33/f6b [2911142,2705] 0 2024-04-20T11:09:43.002 INFO:tasks.workunit.client.1.smithi173.stdout:7/747: dread - d9/da/dc3/ffc zero size 2024-04-20T11:09:43.010 INFO:tasks.workunit.client.1.smithi173.stdout:9/725: creat d5/dcc/fdc x:0 0 0 2024-04-20T11:09:43.010 INFO:tasks.workunit.client.1.smithi173.stdout:3/726: dwrite d7/d17/d61/d83/dcf/f4f [0,4194304] 0 2024-04-20T11:09:43.011 INFO:tasks.workunit.client.1.smithi173.stdout:2/856: symlink d2/d34/l11b 0 2024-04-20T11:09:43.017 INFO:tasks.workunit.client.1.smithi173.stdout:8/772: dwrite d4/d18/fa2 [0,4194304] 0 2024-04-20T11:09:43.017 INFO:tasks.workunit.client.1.smithi173.stdout:6/740: dwrite d4/d24/d3e/d9c/fb1 [0,4194304] 0 2024-04-20T11:09:43.019 INFO:tasks.workunit.client.1.smithi173.stdout:8/773: chown d4/d18/d59/d5a/d7b/d8c/f8e 1036460 1 2024-04-20T11:09:43.022 INFO:tasks.workunit.client.1.smithi173.stdout:2/857: symlink d2/d13/d9a/dbc/dfd/dd0/l11c 0 2024-04-20T11:09:43.025 INFO:tasks.workunit.client.1.smithi173.stdout:6/741: mknod d4/de3/cee 0 2024-04-20T11:09:43.025 INFO:tasks.workunit.client.1.smithi173.stdout:3/727: truncate d7/d17/d61/d83/dcf/f14 280885 0 2024-04-20T11:09:43.030 INFO:tasks.workunit.client.1.smithi173.stdout:5/773: dread da/d4f/d65/d8e/f8d [0,4194304] 0 2024-04-20T11:09:43.032 INFO:tasks.workunit.client.1.smithi173.stdout:9/726: link d5/c30 d5/d9/d20/d29/d18/cdd 0 2024-04-20T11:09:43.040 INFO:tasks.workunit.client.1.smithi173.stdout:5/774: creat da/de/d19/f10c x:0 0 0 2024-04-20T11:09:43.041 INFO:tasks.workunit.client.1.smithi173.stdout:2/858: getdents d2/d13/d9a/dbc/dfd 0 2024-04-20T11:09:43.120 INFO:tasks.workunit.client.1.smithi173.stdout:6/742: truncate d4/f46 2038770 0 2024-04-20T11:09:43.134 INFO:tasks.workunit.client.1.smithi173.stdout:5/775: dwrite da/de/d19/d81/fef [0,4194304] 0 2024-04-20T11:09:43.138 INFO:tasks.workunit.client.1.smithi173.stdout:5/776: read - da/de/d75/d40/fe9 zero size 2024-04-20T11:09:43.146 INFO:tasks.workunit.client.1.smithi173.stdout:5/777: getdents da/de/d75 0 2024-04-20T11:09:43.146 INFO:tasks.workunit.client.1.smithi173.stdout:9/727: dread d5/d9/d20/d4f/d5e/d9e/fa6 [0,4194304] 0 2024-04-20T11:09:43.151 INFO:tasks.workunit.client.1.smithi173.stdout:5/778: fdatasync da/de/d19/d32/d30/f41 0 2024-04-20T11:09:43.154 INFO:tasks.workunit.client.1.smithi173.stdout:9/728: readlink d5/d9/d20/d29/d2d/l9a 0 2024-04-20T11:09:43.158 INFO:tasks.workunit.client.1.smithi173.stdout:9/729: write d5/d9/f1a [2767051,74527] 0 2024-04-20T11:09:43.159 INFO:tasks.workunit.client.1.smithi173.stdout:5/779: write da/de/d19/d81/d92/ff8 [115560,124849] 0 2024-04-20T11:09:43.162 INFO:tasks.workunit.client.1.smithi173.stdout:6/743: dread d4/d14/fe8 [0,4194304] 0 2024-04-20T11:09:43.163 INFO:tasks.workunit.client.1.smithi173.stdout:5/780: rename da/de/d75/d3e to da/de/d75/d3e/d10d 22 2024-04-20T11:09:43.174 INFO:tasks.workunit.client.1.smithi173.stdout:9/730: getdents d5/d9/d20/d2b/d59 0 2024-04-20T11:09:43.187 INFO:tasks.workunit.client.1.smithi173.stdout:5/781: dwrite da/d34/d7f/ffd [0,4194304] 0 2024-04-20T11:09:43.191 INFO:tasks.workunit.client.1.smithi173.stdout:5/782: write da/d34/d7f/fff [856371,8413] 0 2024-04-20T11:09:43.192 INFO:tasks.workunit.client.1.smithi173.stdout:9/731: dwrite d5/d4c/d79/f7a [0,4194304] 0 2024-04-20T11:09:43.194 INFO:tasks.workunit.client.1.smithi173.stdout:5/783: write da/de/d19/fe5 [205291,44380] 0 2024-04-20T11:09:43.195 INFO:tasks.workunit.client.1.smithi173.stdout:6/744: dread d4/d18/d7e/fbe [0,4194304] 0 2024-04-20T11:09:43.198 INFO:tasks.workunit.client.1.smithi173.stdout:5/784: write da/de/d19/d32/f31 [1313834,27956] 0 2024-04-20T11:09:43.202 INFO:tasks.workunit.client.1.smithi173.stdout:6/745: creat d4/d14/d5f/fef x:0 0 0 2024-04-20T11:09:43.214 INFO:tasks.workunit.client.1.smithi173.stdout:9/732: dread d5/d9/d20/d29/f64 [0,4194304] 0 2024-04-20T11:09:43.215 INFO:tasks.workunit.client.1.smithi173.stdout:6/746: getdents d4/d18/d7e 0 2024-04-20T11:09:43.218 INFO:tasks.workunit.client.1.smithi173.stdout:6/747: readlink d4/d24/d42/l50 0 2024-04-20T11:09:43.220 INFO:tasks.workunit.client.1.smithi173.stdout:6/748: truncate d4/d18/d7e/fd6 398707 0 2024-04-20T11:09:43.221 INFO:tasks.workunit.client.1.smithi173.stdout:9/733: link d5/d9/d20/d4f/d5e/d7d/ld4 d5/d9/d20/d4f/d5e/lde 0 2024-04-20T11:09:43.226 INFO:tasks.workunit.client.1.smithi173.stdout:9/734: creat d5/d9/d20/d2b/d59/d44/fdf x:0 0 0 2024-04-20T11:09:43.227 INFO:tasks.workunit.client.1.smithi173.stdout:6/749: getdents d4/d18/d61 0 2024-04-20T11:09:43.230 INFO:tasks.workunit.client.1.smithi173.stdout:6/750: mknod d4/d14/d5f/d9f/cf0 0 2024-04-20T11:09:43.232 INFO:tasks.workunit.client.1.smithi173.stdout:6/751: dread - d4/d18/fb9 zero size 2024-04-20T11:09:43.233 INFO:tasks.workunit.client.1.smithi173.stdout:9/735: symlink d5/d9/d20/d2b/d59/d44/d78/dd0/le0 0 2024-04-20T11:09:43.234 INFO:tasks.workunit.client.1.smithi173.stdout:6/752: chown d4/cd8 661140 1 2024-04-20T11:09:43.236 INFO:tasks.workunit.client.1.smithi173.stdout:9/736: rmdir d5/d9/d20/d4f/d5e/d7d/dba 39 2024-04-20T11:09:43.236 INFO:tasks.workunit.client.1.smithi173.stdout:5/785: dread da/de/d19/d32/d30/f41 [0,4194304] 0 2024-04-20T11:09:43.238 INFO:tasks.workunit.client.1.smithi173.stdout:5/786: fdatasync da/de/d75/d40/fe9 0 2024-04-20T11:09:43.238 INFO:tasks.workunit.client.1.smithi173.stdout:5/787: readlink da/d4f/d5e/lce 0 2024-04-20T11:09:43.240 INFO:tasks.workunit.client.1.smithi173.stdout:9/737: fdatasync d5/d9/d20/d2b/f97 0 2024-04-20T11:09:43.240 INFO:tasks.workunit.client.1.smithi173.stdout:5/788: fsync da/de/d75/d40/f42 0 2024-04-20T11:09:43.243 INFO:tasks.workunit.client.1.smithi173.stdout:9/738: truncate d5/d9/d20/d2b/fb1 225135 0 2024-04-20T11:09:43.247 INFO:tasks.workunit.client.1.smithi173.stdout:9/739: truncate d5/d9/d20/d2b/d59/da1/fb0 428408 0 2024-04-20T11:09:43.247 INFO:tasks.workunit.client.1.smithi173.stdout:8/774: sync 2024-04-20T11:09:43.247 INFO:tasks.workunit.client.1.smithi173.stdout:3/728: sync 2024-04-20T11:09:43.247 INFO:tasks.workunit.client.1.smithi173.stdout:0/709: sync 2024-04-20T11:09:43.247 INFO:tasks.workunit.client.1.smithi173.stdout:4/839: sync 2024-04-20T11:09:43.249 INFO:tasks.workunit.client.1.smithi173.stdout:6/753: getdents d4/d14/d5f/daf 0 2024-04-20T11:09:43.249 INFO:tasks.workunit.client.1.smithi173.stdout:3/729: write d7/d17/f3c [3397871,97503] 0 2024-04-20T11:09:43.251 INFO:tasks.workunit.client.1.smithi173.stdout:7/748: sync 2024-04-20T11:09:43.251 INFO:tasks.workunit.client.1.smithi173.stdout:1/702: sync 2024-04-20T11:09:43.251 INFO:tasks.workunit.client.1.smithi173.stdout:2/859: sync 2024-04-20T11:09:43.252 INFO:tasks.workunit.client.1.smithi173.stdout:3/730: stat d7/d17/d58/d8a/la5 0 2024-04-20T11:09:43.253 INFO:tasks.workunit.client.1.smithi173.stdout:0/710: rmdir d6/d8/d51 39 2024-04-20T11:09:43.256 INFO:tasks.workunit.client.1.smithi173.stdout:4/840: creat da/d1a/d60/f11c x:0 0 0 2024-04-20T11:09:43.257 INFO:tasks.workunit.client.1.smithi173.stdout:9/740: creat d5/d9/d20/d4f/d5e/d7d/d95/fe1 x:0 0 0 2024-04-20T11:09:43.261 INFO:tasks.workunit.client.1.smithi173.stdout:1/703: dread d5/d6/fcd [0,4194304] 0 2024-04-20T11:09:43.263 INFO:tasks.workunit.client.1.smithi173.stdout:8/775: mknod d4/d18/d4f/d5b/de4/de6/cf7 0 2024-04-20T11:09:43.264 INFO:tasks.workunit.client.1.smithi173.stdout:7/749: dread d9/dbc/fbe [0,4194304] 0 2024-04-20T11:09:43.264 INFO:tasks.workunit.client.1.smithi173.stdout:3/731: rename d7/d17/d61 to d7/d17/d61/d83/dcf/de8 22 2024-04-20T11:09:43.264 INFO:tasks.workunit.client.1.smithi173.stdout:5/789: dwrite da/d34/da7/fd4 [0,4194304] 0 2024-04-20T11:09:43.267 INFO:tasks.workunit.client.1.smithi173.stdout:6/754: mkdir d4/d59/df1 0 2024-04-20T11:09:43.269 INFO:tasks.workunit.client.1.smithi173.stdout:1/704: symlink d5/d6/d17/d42/da/df/d40/db4/led 0 2024-04-20T11:09:43.278 INFO:tasks.workunit.client.1.smithi173.stdout:9/741: unlink d5/c30 0 2024-04-20T11:09:43.278 INFO:tasks.workunit.client.1.smithi173.stdout:4/841: dwrite da/d1e/d52/d9a/dd3/def/f108 [0,4194304] 0 2024-04-20T11:09:43.278 INFO:tasks.workunit.client.1.smithi173.stdout:6/755: truncate d4/d14/f87 347320 0 2024-04-20T11:09:43.285 INFO:tasks.workunit.client.1.smithi173.stdout:0/711: rename d6/d8/c61 to d6/d54/cdd 0 2024-04-20T11:09:43.288 INFO:tasks.workunit.client.1.smithi173.stdout:2/860: dread d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:43.290 INFO:tasks.workunit.client.1.smithi173.stdout:2/861: fdatasync d2/ff0 0 2024-04-20T11:09:43.290 INFO:tasks.workunit.client.1.smithi173.stdout:7/750: dread d9/d33/d85/fa6 [0,4194304] 0 2024-04-20T11:09:43.295 INFO:tasks.workunit.client.1.smithi173.stdout:8/776: rename d4/f9f to d4/d18/d59/d6d/ff8 0 2024-04-20T11:09:43.298 INFO:tasks.workunit.client.1.smithi173.stdout:4/842: symlink da/dc/d13/d106/l11d 0 2024-04-20T11:09:43.299 INFO:tasks.workunit.client.1.smithi173.stdout:2/862: read d2/d34/d58/f5a [3009310,130310] 0 2024-04-20T11:09:43.306 INFO:tasks.workunit.client.1.smithi173.stdout:7/751: unlink d9/da/d17/d2b/daa/dd9/cdb 0 2024-04-20T11:09:43.307 INFO:tasks.workunit.client.1.smithi173.stdout:9/742: dwrite d5/d9/d20/d2b/fbe [0,4194304] 0 2024-04-20T11:09:43.311 INFO:tasks.workunit.client.1.smithi173.stdout:0/712: mknod d6/cde 0 2024-04-20T11:09:43.312 INFO:tasks.workunit.client.1.smithi173.stdout:1/705: write d5/d6/d17/d42/d12/f31 [430513,62763] 0 2024-04-20T11:09:43.316 INFO:tasks.workunit.client.1.smithi173.stdout:5/790: dwrite da/de/d75/f64 [0,4194304] 0 2024-04-20T11:09:43.316 INFO:tasks.workunit.client.1.smithi173.stdout:4/843: mkdir da/dc/d103/d11e 0 2024-04-20T11:09:43.318 INFO:tasks.workunit.client.1.smithi173.stdout:2/863: dwrite d2/d13/d9a/dbc/f102 [0,4194304] 0 2024-04-20T11:09:43.323 INFO:tasks.workunit.client.1.smithi173.stdout:7/752: read - d9/d1f/fa5 zero size 2024-04-20T11:09:43.329 INFO:tasks.workunit.client.1.smithi173.stdout:6/756: dwrite d4/d18/d7e/f8a [0,4194304] 0 2024-04-20T11:09:43.332 INFO:tasks.workunit.client.1.smithi173.stdout:8/777: creat d4/d18/d59/d5a/da8/db9/de0/d9b/dc8/ff9 x:0 0 0 2024-04-20T11:09:43.333 INFO:tasks.workunit.client.1.smithi173.stdout:0/713: truncate d6/d8/df/f11 167364 0 2024-04-20T11:09:43.335 INFO:tasks.workunit.client.1.smithi173.stdout:2/864: dread d2/de/d11/f10 [0,4194304] 0 2024-04-20T11:09:43.338 INFO:tasks.workunit.client.1.smithi173.stdout:4/844: creat da/d1a/d60/d7d/f11f x:0 0 0 2024-04-20T11:09:43.339 INFO:tasks.workunit.client.1.smithi173.stdout:5/791: dwrite da/de/d75/d40/f42 [0,4194304] 0 2024-04-20T11:09:43.341 INFO:tasks.workunit.client.1.smithi173.stdout:1/706: fdatasync d5/fb 0 2024-04-20T11:09:43.341 INFO:tasks.workunit.client.1.smithi173.stdout:5/792: stat da/de/d19/d32/f11 0 2024-04-20T11:09:43.343 INFO:tasks.workunit.client.1.smithi173.stdout:4/845: dread da/dc/df4/d53/fe9 [0,4194304] 0 2024-04-20T11:09:43.348 INFO:tasks.workunit.client.1.smithi173.stdout:2/865: write d2/d8c/ff6 [135997,130748] 0 2024-04-20T11:09:43.358 INFO:tasks.workunit.client.1.smithi173.stdout:9/743: dwrite d5/d9/d20/d29/d2d/f3f [0,4194304] 0 2024-04-20T11:09:43.367 INFO:tasks.workunit.client.1.smithi173.stdout:6/757: dwrite d4/d24/d42/d74/fd1 [0,4194304] 0 2024-04-20T11:09:43.368 INFO:tasks.workunit.client.1.smithi173.stdout:0/714: dwrite d6/d8/d51/f5b [4194304,4194304] 0 2024-04-20T11:09:43.417 INFO:tasks.workunit.client.1.smithi173.stdout:4/846: creat da/d1a/d60/d66/dae/f120 x:0 0 0 2024-04-20T11:09:43.423 INFO:tasks.workunit.client.1.smithi173.stdout:9/744: link d5/d9/d20/f40 d5/d9/d20/d2b/d59/d48/fe2 0 2024-04-20T11:09:43.423 INFO:tasks.workunit.client.1.smithi173.stdout:4/847: mknod da/dc/d13/d106/c121 0 2024-04-20T11:09:43.425 INFO:tasks.workunit.client.1.smithi173.stdout:2/866: getdents d2/de/d11/db8 0 2024-04-20T11:09:43.426 INFO:tasks.workunit.client.1.smithi173.stdout:6/758: getdents d4/d24/d42/d81 0 2024-04-20T11:09:43.427 INFO:tasks.workunit.client.1.smithi173.stdout:0/715: getdents d6/d8/df/dd0 0 2024-04-20T11:09:43.429 INFO:tasks.workunit.client.1.smithi173.stdout:9/745: chown d5/d9/d20/d4f/d5e/d7d/ld4 1131 1 2024-04-20T11:09:43.434 INFO:tasks.workunit.client.1.smithi173.stdout:9/746: write d5/d9/f1a [980160,121906] 0 2024-04-20T11:09:43.434 INFO:tasks.workunit.client.1.smithi173.stdout:0/716: creat d6/d8/df/d2c/d42/d50/d86/d8b/da2/fdf x:0 0 0 2024-04-20T11:09:43.453 INFO:tasks.workunit.client.1.smithi173.stdout:6/759: dwrite d4/f11 [0,4194304] 0 2024-04-20T11:09:43.463 INFO:tasks.workunit.client.1.smithi173.stdout:9/747: dwrite d5/d9/d20/d29/d2d/dcb/fcd [0,4194304] 0 2024-04-20T11:09:43.463 INFO:tasks.workunit.client.1.smithi173.stdout:2/867: dwrite d2/d13/d9a/dbc/dfd/d7d/f91 [0,4194304] 0 2024-04-20T11:09:43.464 INFO:tasks.workunit.client.1.smithi173.stdout:0/717: dwrite d6/d8/df/d15/f39 [0,4194304] 0 2024-04-20T11:09:43.468 INFO:tasks.workunit.client.1.smithi173.stdout:2/868: read d2/d13/f44 [1808111,71352] 0 2024-04-20T11:09:43.489 INFO:tasks.workunit.client.1.smithi173.stdout:6/760: fsync d4/d14/d5f/d9f/fcb 0 2024-04-20T11:09:43.495 INFO:tasks.workunit.client.1.smithi173.stdout:2/869: dread d2/d34/f45 [4194304,4194304] 0 2024-04-20T11:09:43.497 INFO:tasks.workunit.client.1.smithi173.stdout:9/748: dwrite d5/d9/d20/d29/f64 [0,4194304] 0 2024-04-20T11:09:43.498 INFO:tasks.workunit.client.1.smithi173.stdout:0/718: dwrite d6/d8/df/d2c/d2d/f6a [0,4194304] 0 2024-04-20T11:09:43.501 INFO:tasks.workunit.client.1.smithi173.stdout:6/761: getdents d4/d24/d3e/d9c 0 2024-04-20T11:09:43.503 INFO:tasks.workunit.client.1.smithi173.stdout:2/870: creat d2/d13/da5/db2/f11d x:0 0 0 2024-04-20T11:09:43.526 INFO:tasks.workunit.client.1.smithi173.stdout:2/871: dwrite d2/d13/d9a/dbc/dfd/d53/d4e/f57 [0,4194304] 0 2024-04-20T11:09:43.528 INFO:tasks.workunit.client.1.smithi173.stdout:2/872: stat d2/d13/f3d 0 2024-04-20T11:09:43.530 INFO:tasks.workunit.client.1.smithi173.stdout:9/749: dread d5/d9/f26 [0,4194304] 0 2024-04-20T11:09:43.532 INFO:tasks.workunit.client.1.smithi173.stdout:3/732: sync 2024-04-20T11:09:43.534 INFO:tasks.workunit.client.1.smithi173.stdout:2/873: mknod d2/de/d11/d9d/d87/db0/c11e 0 2024-04-20T11:09:43.536 INFO:tasks.workunit.client.1.smithi173.stdout:9/750: dread d5/d9/f58 [0,4194304] 0 2024-04-20T11:09:43.538 INFO:tasks.workunit.client.1.smithi173.stdout:5/793: truncate da/de/d19/d32/d72/dc3/feb 50849 0 2024-04-20T11:09:43.543 INFO:tasks.workunit.client.1.smithi173.stdout:5/794: creat da/de/d19/dca/f10e x:0 0 0 2024-04-20T11:09:43.545 INFO:tasks.workunit.client.1.smithi173.stdout:2/874: truncate d2/d13/d9a/dbc/dfd/d53/d9f/ffb 280199 0 2024-04-20T11:09:43.545 INFO:tasks.workunit.client.1.smithi173.stdout:3/733: getdents d7/d17/d58/d7f 0 2024-04-20T11:09:43.548 INFO:tasks.workunit.client.1.smithi173.stdout:9/751: creat d5/d9/d20/d4f/fe3 x:0 0 0 2024-04-20T11:09:43.555 INFO:tasks.workunit.client.1.smithi173.stdout:4/848: dwrite da/dc/df4/d53/d39/fca [0,4194304] 0 2024-04-20T11:09:43.559 INFO:tasks.workunit.client.1.smithi173.stdout:3/734: truncate d7/d17/d61/d83/dcf/f14 985312 0 2024-04-20T11:09:43.566 INFO:tasks.workunit.client.1.smithi173.stdout:3/735: chown d7/d17/d61/d83/dcf/f2e 6670 1 2024-04-20T11:09:43.571 INFO:tasks.workunit.client.1.smithi173.stdout:6/762: unlink d4/d1c/l3d 0 2024-04-20T11:09:43.571 INFO:tasks.workunit.client.1.smithi173.stdout:9/752: dread d5/d9/f1b [0,4194304] 0 2024-04-20T11:09:43.576 INFO:tasks.workunit.client.1.smithi173.stdout:3/736: creat d7/d2b/d9d/db7/fe9 x:0 0 0 2024-04-20T11:09:43.577 INFO:tasks.workunit.client.1.smithi173.stdout:6/763: read d4/d18/d7e/fd6 [131367,61062] 0 2024-04-20T11:09:43.578 INFO:tasks.workunit.client.1.smithi173.stdout:2/875: dwrite d2/de/d11/d9d/d87/f97 [4194304,4194304] 0 2024-04-20T11:09:43.579 INFO:tasks.workunit.client.1.smithi173.stdout:4/849: dread da/f64 [0,4194304] 0 2024-04-20T11:09:43.579 INFO:tasks.workunit.client.1.smithi173.stdout:3/737: write d7/d17/d61/f78 [111715,90307] 0 2024-04-20T11:09:43.580 INFO:tasks.workunit.client.1.smithi173.stdout:6/764: write d4/d24/d3e/d9c/fb1 [1932152,89659] 0 2024-04-20T11:09:43.583 INFO:tasks.workunit.client.1.smithi173.stdout:3/738: chown d7/d17/d61/d83 1 1 2024-04-20T11:09:43.584 INFO:tasks.workunit.client.1.smithi173.stdout:2/876: mknod d2/d13/d9a/c11f 0 2024-04-20T11:09:43.667 INFO:tasks.workunit.client.1.smithi173.stdout:3/739: rmdir d7/d53 39 2024-04-20T11:09:43.667 INFO:tasks.workunit.client.1.smithi173.stdout:6/765: read d4/f37 [2090491,64016] 0 2024-04-20T11:09:43.668 INFO:tasks.workunit.client.1.smithi173.stdout:0/719: rename d6/d8/df/d2c/d42/f8e to d6/d8/df/fe0 0 2024-04-20T11:09:43.670 INFO:tasks.workunit.client.1.smithi173.stdout:8/778: symlink d4/d18/d59/d5a/lfa 0 2024-04-20T11:09:43.670 INFO:tasks.workunit.client.1.smithi173.stdout:0/720: chown d6/d8/df/d2c 44019 1 2024-04-20T11:09:43.671 INFO:tasks.workunit.client.1.smithi173.stdout:8/779: fdatasync d4/f10 0 2024-04-20T11:09:43.673 INFO:tasks.workunit.client.1.smithi173.stdout:2/877: creat d2/d13/d28/f120 x:0 0 0 2024-04-20T11:09:43.676 INFO:tasks.workunit.client.1.smithi173.stdout:6/766: chown d4/d24/d3e/f8d 1291829 1 2024-04-20T11:09:43.676 INFO:tasks.workunit.client.1.smithi173.stdout:8/780: chown d4/d18/d59/d5a/d7b/cc5 490007 1 2024-04-20T11:09:43.679 INFO:tasks.workunit.client.1.smithi173.stdout:5/795: rename da/de/d19/dca/c3f to da/de/d75/d3e/c10f 0 2024-04-20T11:09:43.680 INFO:tasks.workunit.client.1.smithi173.stdout:0/721: getdents d6/d8/d3c/d44/ddb 0 2024-04-20T11:09:43.682 INFO:tasks.workunit.client.1.smithi173.stdout:6/767: write d4/f11 [1762913,24399] 0 2024-04-20T11:09:43.683 INFO:tasks.workunit.client.1.smithi173.stdout:8/781: write d4/d18/d59/d5a/fcb [237800,125147] 0 2024-04-20T11:09:43.683 INFO:tasks.workunit.client.1.smithi173.stdout:3/740: read d7/d17/d61/fc3 [4503240,94712] 0 2024-04-20T11:09:43.684 INFO:tasks.workunit.client.1.smithi173.stdout:2/878: rename d2/d34/d58/fd3 to d2/d13/d9a/dbc/dfd/f121 0 2024-04-20T11:09:43.689 INFO:tasks.workunit.client.1.smithi173.stdout:5/796: creat da/de/d19/dca/d48/da2/f110 x:0 0 0 2024-04-20T11:09:43.689 INFO:tasks.workunit.client.1.smithi173.stdout:4/850: truncate da/dc/d13/f2b 1217083 0 2024-04-20T11:09:43.692 INFO:tasks.workunit.client.1.smithi173.stdout:8/782: unlink d4/d18/d59/d6d/l92 0 2024-04-20T11:09:43.696 INFO:tasks.workunit.client.1.smithi173.stdout:0/722: symlink d6/d8/df/d2c/d42/d50/d86/d8b/dcc/le1 0 2024-04-20T11:09:43.696 INFO:tasks.workunit.client.1.smithi173.stdout:0/723: chown d6/d54/d65 1 1 2024-04-20T11:09:43.698 INFO:tasks.workunit.client.1.smithi173.stdout:4/851: mknod da/d1a/d60/c122 0 2024-04-20T11:09:43.700 INFO:tasks.workunit.client.1.smithi173.stdout:5/797: mknod da/de/d75/d40/d59/c111 0 2024-04-20T11:09:43.701 INFO:tasks.workunit.client.1.smithi173.stdout:3/741: rename d7/d17/d61/dd6/cda to d7/d2b/cea 0 2024-04-20T11:09:43.703 INFO:tasks.workunit.client.1.smithi173.stdout:0/724: truncate d6/f1e 1017835 0 2024-04-20T11:09:43.706 INFO:tasks.workunit.client.1.smithi173.stdout:8/783: symlink d4/d18/lfb 0 2024-04-20T11:09:43.707 INFO:tasks.workunit.client.1.smithi173.stdout:1/707: sync 2024-04-20T11:09:43.707 INFO:tasks.workunit.client.1.smithi173.stdout:7/753: sync 2024-04-20T11:09:43.711 INFO:tasks.workunit.client.1.smithi173.stdout:5/798: symlink da/d34/d7f/da5/d9a/l112 0 2024-04-20T11:09:43.713 INFO:tasks.workunit.client.1.smithi173.stdout:0/725: readlink d6/l63 0 2024-04-20T11:09:43.716 INFO:tasks.workunit.client.1.smithi173.stdout:6/768: dread d4/d24/d42/fbd [0,4194304] 0 2024-04-20T11:09:43.718 INFO:tasks.workunit.client.1.smithi173.stdout:4/852: mkdir da/d1a/d60/d7d/d101/d123 0 2024-04-20T11:09:43.721 INFO:tasks.workunit.client.1.smithi173.stdout:4/853: chown da/dc/fa5 2083670 1 2024-04-20T11:09:43.728 INFO:tasks.workunit.client.1.smithi173.stdout:8/784: dwrite d4/d18/d4f/f96 [0,4194304] 0 2024-04-20T11:09:43.729 INFO:tasks.workunit.client.1.smithi173.stdout:0/726: fsync d6/d8/d3c/d44/d75/f97 0 2024-04-20T11:09:43.731 INFO:tasks.workunit.client.1.smithi173.stdout:1/708: dwrite d5/d6/f21 [0,4194304] 0 2024-04-20T11:09:43.732 INFO:tasks.workunit.client.1.smithi173.stdout:4/854: dread - da/dc/df4/fa2 zero size 2024-04-20T11:09:43.732 INFO:tasks.workunit.client.1.smithi173.stdout:8/785: dread - d4/fcd zero size 2024-04-20T11:09:43.736 INFO:tasks.workunit.client.1.smithi173.stdout:5/799: dwrite da/d34/f6f [0,4194304] 0 2024-04-20T11:09:43.737 INFO:tasks.workunit.client.1.smithi173.stdout:2/879: rename d2/d34/l11b to d2/d13/d9a/dbc/dfd/d53/l122 0 2024-04-20T11:09:43.741 INFO:tasks.workunit.client.1.smithi173.stdout:5/800: readlink da/de/d75/d40/d59/la3 0 2024-04-20T11:09:43.742 INFO:tasks.workunit.client.1.smithi173.stdout:4/855: read - da/dc/fed zero size 2024-04-20T11:09:43.749 INFO:tasks.workunit.client.1.smithi173.stdout:5/801: chown da/lb 12 1 2024-04-20T11:09:43.750 INFO:tasks.workunit.client.1.smithi173.stdout:8/786: dread d4/d18/d59/d5a/da8/db9/de0/f46 [0,4194304] 0 2024-04-20T11:09:43.754 INFO:tasks.workunit.client.1.smithi173.stdout:2/880: rename d2/de/d11/d9d/f10d to d2/d13/d9a/dbc/dfd/d7d/f123 0 2024-04-20T11:09:43.755 INFO:tasks.workunit.client.1.smithi173.stdout:7/754: write d9/d1f/d7f/dce/f5f [1848486,28388] 0 2024-04-20T11:09:43.763 INFO:tasks.workunit.client.1.smithi173.stdout:8/787: mkdir d4/d18/d59/d5a/da8/db9/de0/d3c/d86/dfc 0 2024-04-20T11:09:43.771 INFO:tasks.workunit.client.1.smithi173.stdout:0/727: dwrite d6/d8/df/d2c/d42/d50/d86/fbc [0,4194304] 0 2024-04-20T11:09:43.771 INFO:tasks.workunit.client.1.smithi173.stdout:7/755: truncate d9/d1f/fa1 760946 0 2024-04-20T11:09:43.772 INFO:tasks.workunit.client.1.smithi173.stdout:2/881: rename d2/d13/d9a/dbc/dfd/d7d/ce2 to d2/de/d11/d9d/d87/db0/dcc/c124 0 2024-04-20T11:09:43.774 INFO:tasks.workunit.client.1.smithi173.stdout:8/788: mkdir d4/d18/d4f/d5b/de4/dfd 0 2024-04-20T11:09:43.776 INFO:tasks.workunit.client.1.smithi173.stdout:2/882: stat d2/d34/d81/f9e 0 2024-04-20T11:09:43.778 INFO:tasks.workunit.client.1.smithi173.stdout:7/756: symlink d9/d33/d85/l10e 0 2024-04-20T11:09:43.784 INFO:tasks.workunit.client.1.smithi173.stdout:0/728: rename d6/d8/df/da0 to d6/d8/d3c/d44/d69/de2 0 2024-04-20T11:09:43.785 INFO:tasks.workunit.client.1.smithi173.stdout:4/856: dwrite da/dc/d15/f2a [12582912,4194304] 0 2024-04-20T11:09:43.790 INFO:tasks.workunit.client.1.smithi173.stdout:5/802: write da/de/d19/d32/d72/dc3/ffb [1211225,36863] 0 2024-04-20T11:09:43.792 INFO:tasks.workunit.client.1.smithi173.stdout:7/757: mknod d9/d33/d35/d54/d10c/c10f 0 2024-04-20T11:09:43.792 INFO:tasks.workunit.client.1.smithi173.stdout:8/789: write d4/d18/d59/d5a/da8/db9/de0/f9d [1723495,68503] 0 2024-04-20T11:09:43.796 INFO:tasks.workunit.client.1.smithi173.stdout:1/709: dread d5/d6/d17/d42/da/df/d40/f5d [0,4194304] 0 2024-04-20T11:09:43.797 INFO:tasks.workunit.client.1.smithi173.stdout:4/857: mkdir da/d1a/d60/d7d/d101/d123/d124 0 2024-04-20T11:09:43.798 INFO:tasks.workunit.client.1.smithi173.stdout:5/803: write da/d4f/d5e/f6d [791597,54680] 0 2024-04-20T11:09:43.799 INFO:tasks.workunit.client.1.smithi173.stdout:7/758: write d9/d33/d35/f66 [1492729,48872] 0 2024-04-20T11:09:43.801 INFO:tasks.workunit.client.1.smithi173.stdout:5/804: stat da/d34/ldd 0 2024-04-20T11:09:43.802 INFO:tasks.workunit.client.1.smithi173.stdout:1/710: readlink d5/d6/d17/d42/da/d20/d38/d52/d75/lab 0 2024-04-20T11:09:43.807 INFO:tasks.workunit.client.1.smithi173.stdout:8/790: rename d4/d18/d59/d5a/lfa to d4/d18/d59/lfe 0 2024-04-20T11:09:43.809 INFO:tasks.workunit.client.1.smithi173.stdout:4/858: creat da/d1a/d60/d66/dae/f125 x:0 0 0 2024-04-20T11:09:43.813 INFO:tasks.workunit.client.1.smithi173.stdout:8/791: write d4/d18/d4f/d5b/fc3 [202232,62890] 0 2024-04-20T11:09:43.814 INFO:tasks.workunit.client.1.smithi173.stdout:5/805: creat da/d34/d7f/f113 x:0 0 0 2024-04-20T11:09:43.815 INFO:tasks.workunit.client.1.smithi173.stdout:1/711: rename d5/d6/d17/d42/d12/d1c/d85/fb7 to d5/d6/dc/d7f/fee 0 2024-04-20T11:09:43.816 INFO:tasks.workunit.client.1.smithi173.stdout:5/806: chown da/de/ff0 934994 1 2024-04-20T11:09:43.818 INFO:tasks.workunit.client.1.smithi173.stdout:1/712: fsync d5/d6/f21 0 2024-04-20T11:09:43.818 INFO:tasks.workunit.client.1.smithi173.stdout:4/859: unlink da/d1e/d52/f71 0 2024-04-20T11:09:43.820 INFO:tasks.workunit.client.1.smithi173.stdout:7/759: dwrite d9/d33/d40/d89/feb [0,4194304] 0 2024-04-20T11:09:43.831 INFO:tasks.workunit.client.1.smithi173.stdout:4/860: stat da/d1a/d60/d66/d9b/cf1 0 2024-04-20T11:09:43.831 INFO:tasks.workunit.client.1.smithi173.stdout:5/807: symlink da/d34/d7f/da5/db5/l114 0 2024-04-20T11:09:43.832 INFO:tasks.workunit.client.1.smithi173.stdout:3/742: sync 2024-04-20T11:09:43.832 INFO:tasks.workunit.client.1.smithi173.stdout:9/753: sync 2024-04-20T11:09:43.832 INFO:tasks.workunit.client.1.smithi173.stdout:6/769: sync 2024-04-20T11:09:43.835 INFO:tasks.workunit.client.1.smithi173.stdout:4/861: chown da/dc/d62/c80 127557 1 2024-04-20T11:09:43.837 INFO:tasks.workunit.client.1.smithi173.stdout:6/770: chown d4/d1c/d8e/ced 33964205 1 2024-04-20T11:09:43.838 INFO:tasks.workunit.client.1.smithi173.stdout:0/729: dread d6/d8/f12 [0,4194304] 0 2024-04-20T11:09:43.838 INFO:tasks.workunit.client.1.smithi173.stdout:8/792: dwrite d4/d18/ff3 [0,4194304] 0 2024-04-20T11:09:43.839 INFO:tasks.workunit.client.1.smithi173.stdout:2/883: dwrite d2/d34/d58/f5a [4194304,4194304] 0 2024-04-20T11:09:43.856 INFO:tasks.workunit.client.1.smithi173.stdout:8/793: dwrite d4/d18/f93 [0,4194304] 0 2024-04-20T11:09:43.860 INFO:tasks.workunit.client.1.smithi173.stdout:8/794: read d4/d18/d59/d5a/da8/db9/fbc [4097300,33973] 0 2024-04-20T11:09:43.863 INFO:tasks.workunit.client.1.smithi173.stdout:8/795: dread - d4/d18/fcf zero size 2024-04-20T11:09:43.875 INFO:tasks.workunit.client.1.smithi173.stdout:1/713: symlink d5/d6/dc/lef 0 2024-04-20T11:09:43.876 INFO:tasks.workunit.client.1.smithi173.stdout:7/760: mkdir d9/d33/d35/d4a/d52/df2/d110 0 2024-04-20T11:09:43.877 INFO:tasks.workunit.client.1.smithi173.stdout:1/714: chown d5/d6/d17/d42/da/d20/dad 3586975 1 2024-04-20T11:09:43.880 INFO:tasks.workunit.client.1.smithi173.stdout:9/754: rmdir d5/d9/d20/d2b/d59/d44/d78/d9d 39 2024-04-20T11:09:43.881 INFO:tasks.workunit.client.1.smithi173.stdout:4/862: symlink da/d1e/da6/l126 0 2024-04-20T11:09:43.881 INFO:tasks.workunit.client.1.smithi173.stdout:8/796: dwrite d4/fd5 [0,4194304] 0 2024-04-20T11:09:43.882 INFO:tasks.workunit.client.1.smithi173.stdout:5/808: dread da/de/d19/d32/d72/dc3/feb [0,4194304] 0 2024-04-20T11:09:43.883 INFO:tasks.workunit.client.1.smithi173.stdout:6/771: rename d4/d1c/d8e/d94 to d4/d24/d3e/d9c/df2 0 2024-04-20T11:09:43.885 INFO:tasks.workunit.client.1.smithi173.stdout:0/730: mkdir d6/d8/df/d2c/d42/d50/de3 0 2024-04-20T11:09:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:43 smithi117 ceph-mon[15659]: pgmap v31: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 47 MiB/s rd, 145 MiB/s wr, 330 op/s 2024-04-20T11:09:43.893 INFO:tasks.workunit.client.1.smithi173.stdout:7/761: stat d9/da/d17/d2b/c3f 0 2024-04-20T11:09:43.898 INFO:tasks.workunit.client.1.smithi173.stdout:5/809: dwrite da/d4f/f5f [0,4194304] 0 2024-04-20T11:09:43.901 INFO:tasks.workunit.client.1.smithi173.stdout:4/863: dread - da/dc/d15/d8b/fcf zero size 2024-04-20T11:09:43.904 INFO:tasks.workunit.client.1.smithi173.stdout:2/884: symlink d2/d13/da5/d101/l125 0 2024-04-20T11:09:43.905 INFO:tasks.workunit.client.1.smithi173.stdout:6/772: creat d4/d1c/d8e/d90/ff3 x:0 0 0 2024-04-20T11:09:43.906 INFO:tasks.workunit.client.1.smithi173.stdout:0/731: truncate d6/f7 681625 0 2024-04-20T11:09:43.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:43 smithi173 ceph-mon[23800]: pgmap v31: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 47 MiB/s rd, 145 MiB/s wr, 330 op/s 2024-04-20T11:09:43.911 INFO:tasks.workunit.client.1.smithi173.stdout:7/762: unlink d9/d1f/l4f 0 2024-04-20T11:09:43.912 INFO:tasks.workunit.client.1.smithi173.stdout:1/715: symlink d5/d6/d17/d42/da/lf0 0 2024-04-20T11:09:43.912 INFO:tasks.workunit.client.1.smithi173.stdout:7/763: chown d9/d1f/d7f/f87 11044 1 2024-04-20T11:09:43.913 INFO:tasks.workunit.client.1.smithi173.stdout:9/755: symlink d5/le4 0 2024-04-20T11:09:43.914 INFO:tasks.workunit.client.1.smithi173.stdout:4/864: mknod da/d1e/d52/d9a/c127 0 2024-04-20T11:09:43.915 INFO:tasks.workunit.client.1.smithi173.stdout:7/764: truncate d9/d33/d81/fe6 178153 0 2024-04-20T11:09:43.915 INFO:tasks.workunit.client.1.smithi173.stdout:8/797: mknod d4/d18/d4f/d5b/de4/dfd/cff 0 2024-04-20T11:09:43.916 INFO:tasks.workunit.client.1.smithi173.stdout:2/885: symlink d2/d13/d9a/dbc/dfd/d53/d9f/l126 0 2024-04-20T11:09:43.919 INFO:tasks.workunit.client.1.smithi173.stdout:0/732: creat d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/db9/fe4 x:0 0 0 2024-04-20T11:09:43.920 INFO:tasks.workunit.client.1.smithi173.stdout:0/733: fdatasync d6/d54/d65/f98 0 2024-04-20T11:09:43.922 INFO:tasks.workunit.client.1.smithi173.stdout:0/734: truncate d6/d5f/d8f/fa7 496797 0 2024-04-20T11:09:43.923 INFO:tasks.workunit.client.1.smithi173.stdout:3/743: write d7/d17/f24 [3219474,94682] 0 2024-04-20T11:09:43.925 INFO:tasks.workunit.client.1.smithi173.stdout:6/773: rename d4/d14/f2f to d4/d24/d42/d81/ff4 0 2024-04-20T11:09:43.927 INFO:tasks.workunit.client.1.smithi173.stdout:6/774: stat d4/d24/d42/d74 0 2024-04-20T11:09:43.927 INFO:tasks.workunit.client.1.smithi173.stdout:3/744: dread d7/d17/d61/d83/fc8 [0,4194304] 0 2024-04-20T11:09:43.928 INFO:tasks.workunit.client.1.smithi173.stdout:7/765: read - d9/d33/d35/d56/fd0 zero size 2024-04-20T11:09:43.929 INFO:tasks.workunit.client.1.smithi173.stdout:0/735: mknod d6/d8/d3c/d44/d69/de2/ce5 0 2024-04-20T11:09:43.931 INFO:tasks.workunit.client.1.smithi173.stdout:0/736: readlink d6/d8/d3c/d44/d69/de2/la8 0 2024-04-20T11:09:43.931 INFO:tasks.workunit.client.1.smithi173.stdout:1/716: creat d5/d6/d17/d42/ff1 x:0 0 0 2024-04-20T11:09:43.932 INFO:tasks.workunit.client.1.smithi173.stdout:1/717: chown d5/fb 0 1 2024-04-20T11:09:43.937 INFO:tasks.workunit.client.1.smithi173.stdout:3/745: truncate d7/d17/d61/f97 34191 0 2024-04-20T11:09:43.939 INFO:tasks.workunit.client.1.smithi173.stdout:0/737: mkdir d6/d54/d91/de6 0 2024-04-20T11:09:43.945 INFO:tasks.workunit.client.1.smithi173.stdout:4/865: dread da/dc/d62/f76 [0,4194304] 0 2024-04-20T11:09:43.946 INFO:tasks.workunit.client.1.smithi173.stdout:4/866: chown da/dc/df4/f11b 40303 1 2024-04-20T11:09:43.948 INFO:tasks.workunit.client.1.smithi173.stdout:7/766: dwrite d9/d33/d35/d4a/d52/df2/f104 [0,4194304] 0 2024-04-20T11:09:43.949 INFO:tasks.workunit.client.1.smithi173.stdout:0/738: mknod d6/d8/d3c/d44/d69/ce7 0 2024-04-20T11:09:43.949 INFO:tasks.workunit.client.1.smithi173.stdout:9/756: write d5/d4c/d79/f7a [4132145,101585] 0 2024-04-20T11:09:43.954 INFO:tasks.workunit.client.1.smithi173.stdout:3/746: chown d7/d2b/c48 11555 1 2024-04-20T11:09:43.954 INFO:tasks.workunit.client.1.smithi173.stdout:4/867: unlink da/d1e/da6/l126 0 2024-04-20T11:09:43.956 INFO:tasks.workunit.client.1.smithi173.stdout:7/767: dread - d9/da/d17/d2b/f74 zero size 2024-04-20T11:09:43.956 INFO:tasks.workunit.client.1.smithi173.stdout:6/775: write d4/d18/d7e/f80 [3033134,39487] 0 2024-04-20T11:09:43.962 INFO:tasks.workunit.client.1.smithi173.stdout:9/757: mkdir d5/d9/d20/d2b/d59/d44/de5 0 2024-04-20T11:09:43.964 INFO:tasks.workunit.client.1.smithi173.stdout:5/810: dwrite da/de/d19/f76 [0,4194304] 0 2024-04-20T11:09:43.964 INFO:tasks.workunit.client.1.smithi173.stdout:7/768: write d9/d33/d35/d4a/def/f10a [474454,102039] 0 2024-04-20T11:09:43.965 INFO:tasks.workunit.client.1.smithi173.stdout:7/769: readlink d9/d33/l92 0 2024-04-20T11:09:43.967 INFO:tasks.workunit.client.1.smithi173.stdout:6/776: creat d4/d59/d8f/ff5 x:0 0 0 2024-04-20T11:09:43.969 INFO:tasks.workunit.client.1.smithi173.stdout:4/868: rename da/d1a/d60/f83 to da/d1e/da6/f128 0 2024-04-20T11:09:43.973 INFO:tasks.workunit.client.1.smithi173.stdout:6/777: dread d4/d24/d42/d81/f99 [0,4194304] 0 2024-04-20T11:09:43.973 INFO:tasks.workunit.client.1.smithi173.stdout:0/739: write d6/d8/df/d15/f95 [436254,85026] 0 2024-04-20T11:09:43.976 INFO:tasks.workunit.client.1.smithi173.stdout:0/740: stat d6/d8/df/d43/l78 0 2024-04-20T11:09:43.979 INFO:tasks.workunit.client.1.smithi173.stdout:9/758: rename d5/ce to d5/d9/d20/d2b/d59/ce6 0 2024-04-20T11:09:43.980 INFO:tasks.workunit.client.1.smithi173.stdout:6/778: mkdir d4/d24/d42/d74/df6 0 2024-04-20T11:09:43.981 INFO:tasks.workunit.client.1.smithi173.stdout:4/869: getdents da/d1a/d60/d7d/d101 0 2024-04-20T11:09:43.984 INFO:tasks.workunit.client.1.smithi173.stdout:0/741: mkdir d6/d8/df/d2c/d42/d50/d86/d8b/da2/de8 0 2024-04-20T11:09:43.985 INFO:tasks.workunit.client.1.smithi173.stdout:9/759: symlink d5/d9/d20/d29/d2d/le7 0 2024-04-20T11:09:43.986 INFO:tasks.workunit.client.1.smithi173.stdout:4/870: truncate da/f25 2411335 0 2024-04-20T11:09:43.988 INFO:tasks.workunit.client.1.smithi173.stdout:6/779: rename d4/d24/d3e/fd3 to d4/d24/d42/d74/df6/ff7 0 2024-04-20T11:09:43.991 INFO:tasks.workunit.client.1.smithi173.stdout:0/742: mkdir d6/d5f/de9 0 2024-04-20T11:09:43.991 INFO:tasks.workunit.client.1.smithi173.stdout:6/780: fsync d4/d24/d42/d81/f99 0 2024-04-20T11:09:43.993 INFO:tasks.workunit.client.1.smithi173.stdout:4/871: mkdir da/dc/d103/d129 0 2024-04-20T11:09:43.996 INFO:tasks.workunit.client.1.smithi173.stdout:5/811: dwrite da/de/d19/d32/f2f [0,4194304] 0 2024-04-20T11:09:43.997 INFO:tasks.workunit.client.1.smithi173.stdout:7/770: dwrite d9/d1f/f7d [0,4194304] 0 2024-04-20T11:09:44.001 INFO:tasks.workunit.client.1.smithi173.stdout:6/781: chown d4/d1c/d8e/d90/lc2 7210 1 2024-04-20T11:09:44.004 INFO:tasks.workunit.client.1.smithi173.stdout:0/743: unlink d6/d8/df/d2c/d42/f6c 0 2024-04-20T11:09:44.007 INFO:tasks.workunit.client.1.smithi173.stdout:0/744: fsync d6/d8/df/d2c/d42/d50/d86/d8b/da2/fdf 0 2024-04-20T11:09:44.009 INFO:tasks.workunit.client.1.smithi173.stdout:5/812: chown da/de/d19/l85 1102007 1 2024-04-20T11:09:44.013 INFO:tasks.workunit.client.1.smithi173.stdout:4/872: rename da/d1a/f81 to da/d1a/d60/d66/d9b/f12a 0 2024-04-20T11:09:44.018 INFO:tasks.workunit.client.1.smithi173.stdout:7/771: creat d9/d33/de9/f111 x:0 0 0 2024-04-20T11:09:44.020 INFO:tasks.workunit.client.1.smithi173.stdout:9/760: mkdir d5/d9/d20/d2b/d59/d44/d78/d82/de8 0 2024-04-20T11:09:44.023 INFO:tasks.workunit.client.1.smithi173.stdout:6/782: rename d4/d24/d42/d81/fcd to d4/de3/ff8 0 2024-04-20T11:09:44.024 INFO:tasks.workunit.client.1.smithi173.stdout:6/783: chown d4/d1c/c89 67 1 2024-04-20T11:09:44.026 INFO:tasks.workunit.client.1.smithi173.stdout:6/784: write d4/d14/f3c [4114748,81260] 0 2024-04-20T11:09:44.032 INFO:tasks.workunit.client.1.smithi173.stdout:6/785: rmdir d4/d18 39 2024-04-20T11:09:44.034 INFO:tasks.workunit.client.1.smithi173.stdout:6/786: chown d4/d14/d5f/daf/ce1 238673 1 2024-04-20T11:09:44.035 INFO:tasks.workunit.client.1.smithi173.stdout:6/787: fsync d4/d59/fc1 0 2024-04-20T11:09:44.038 INFO:tasks.workunit.client.1.smithi173.stdout:5/813: rename da/d4f/d65/c103 to da/de/d19/d32/d30/d109/c115 0 2024-04-20T11:09:44.041 INFO:tasks.workunit.client.1.smithi173.stdout:6/788: write d4/d24/d42/d74/d3b/fa8 [1011168,75109] 0 2024-04-20T11:09:44.043 INFO:tasks.workunit.client.1.smithi173.stdout:5/814: creat da/d34/d7f/da4/f116 x:0 0 0 2024-04-20T11:09:44.043 INFO:tasks.workunit.client.1.smithi173.stdout:9/761: truncate d5/d4c/d79/f7a 880987 0 2024-04-20T11:09:44.045 INFO:tasks.workunit.client.1.smithi173.stdout:7/772: rename d9/d1f/d7f/dce/l77 to d9/d33/d91/l112 0 2024-04-20T11:09:44.048 INFO:tasks.workunit.client.1.smithi173.stdout:6/789: mknod d4/d24/d3e/cf9 0 2024-04-20T11:09:44.049 INFO:tasks.workunit.client.1.smithi173.stdout:4/873: dwrite da/dc/d13/fd8 [0,4194304] 0 2024-04-20T11:09:44.052 INFO:tasks.workunit.client.1.smithi173.stdout:5/815: mknod da/d4f/d65/dd5/c117 0 2024-04-20T11:09:44.052 INFO:tasks.workunit.client.1.smithi173.stdout:0/745: dwrite d6/d8/df/fe0 [0,4194304] 0 2024-04-20T11:09:44.055 INFO:tasks.workunit.client.1.smithi173.stdout:9/762: rmdir d5/d9/d20/d4f 39 2024-04-20T11:09:44.062 INFO:tasks.workunit.client.1.smithi173.stdout:7/773: rmdir d9/d33/d40/d89 39 2024-04-20T11:09:44.064 INFO:tasks.workunit.client.1.smithi173.stdout:7/774: truncate d9/f4e 1164446 0 2024-04-20T11:09:44.071 INFO:tasks.workunit.client.1.smithi173.stdout:9/763: creat d5/d4c/fe9 x:0 0 0 2024-04-20T11:09:44.071 INFO:tasks.workunit.client.1.smithi173.stdout:7/775: mkdir d9/d33/d35/d54/d10c/d113 0 2024-04-20T11:09:44.074 INFO:tasks.workunit.client.1.smithi173.stdout:6/790: unlink d4/l15 0 2024-04-20T11:09:44.074 INFO:tasks.workunit.client.1.smithi173.stdout:4/874: mknod da/d1a/c12b 0 2024-04-20T11:09:44.077 INFO:tasks.workunit.client.1.smithi173.stdout:7/776: truncate d9/da/d17/f64 3717167 0 2024-04-20T11:09:44.078 INFO:tasks.workunit.client.1.smithi173.stdout:6/791: creat d4/d14/d5f/d9f/ffa x:0 0 0 2024-04-20T11:09:44.079 INFO:tasks.workunit.client.1.smithi173.stdout:4/875: creat da/dc/d103/f12c x:0 0 0 2024-04-20T11:09:44.082 INFO:tasks.workunit.client.1.smithi173.stdout:9/764: mknod d5/d9/d20/d4f/d5e/d7d/dba/cea 0 2024-04-20T11:09:44.083 INFO:tasks.workunit.client.1.smithi173.stdout:4/876: dread - da/d1a/d60/d7d/fcb zero size 2024-04-20T11:09:44.087 INFO:tasks.workunit.client.1.smithi173.stdout:4/877: truncate da/dc/d103/ff9 5203123 0 2024-04-20T11:09:44.091 INFO:tasks.workunit.client.1.smithi173.stdout:4/878: fdatasync da/dc/df4/d53/d39/f56 0 2024-04-20T11:09:44.091 INFO:tasks.workunit.client.1.smithi173.stdout:0/746: dread d6/d8/df/d2c/d42/d50/fb5 [0,4194304] 0 2024-04-20T11:09:44.092 INFO:tasks.workunit.client.1.smithi173.stdout:4/879: fdatasync da/d1e/da6/f104 0 2024-04-20T11:09:44.092 INFO:tasks.workunit.client.1.smithi173.stdout:0/747: chown d6/d8/df/d2c/d2d 541 1 2024-04-20T11:09:44.096 INFO:tasks.workunit.client.1.smithi173.stdout:7/777: fsync d9/da/d17/f64 0 2024-04-20T11:09:44.099 INFO:tasks.workunit.client.1.smithi173.stdout:6/792: write d4/d18/d7e/fd6 [1172005,33779] 0 2024-04-20T11:09:44.100 INFO:tasks.workunit.client.1.smithi173.stdout:4/880: mknod da/d1a/d60/d66/c12d 0 2024-04-20T11:09:44.102 INFO:tasks.workunit.client.1.smithi173.stdout:7/778: stat d9/d33/d40/d89/c103 0 2024-04-20T11:09:44.102 INFO:tasks.workunit.client.1.smithi173.stdout:0/748: creat d6/d8/d51/fea x:0 0 0 2024-04-20T11:09:44.104 INFO:tasks.workunit.client.1.smithi173.stdout:0/749: chown d6/d8/d3c/d44/d69/de2/fb3 348 1 2024-04-20T11:09:44.107 INFO:tasks.workunit.client.1.smithi173.stdout:7/779: unlink d9/d1f/d7f/dce/l5e 0 2024-04-20T11:09:44.107 INFO:tasks.workunit.client.1.smithi173.stdout:0/750: write d6/fa4 [508194,130232] 0 2024-04-20T11:09:44.111 INFO:tasks.workunit.client.1.smithi173.stdout:7/780: creat d9/d33/d35/d4a/d52/f114 x:0 0 0 2024-04-20T11:09:44.113 INFO:tasks.workunit.client.1.smithi173.stdout:0/751: rename d6/d54 to d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb 0 2024-04-20T11:09:44.117 INFO:tasks.workunit.client.1.smithi173.stdout:6/793: dwrite d4/d18/d7e/fbe [0,4194304] 0 2024-04-20T11:09:44.124 INFO:tasks.workunit.client.1.smithi173.stdout:0/752: dread d6/f1e [0,4194304] 0 2024-04-20T11:09:44.126 INFO:tasks.workunit.client.1.smithi173.stdout:0/753: fsync d6/d8/df/f1d 0 2024-04-20T11:09:44.129 INFO:tasks.workunit.client.1.smithi173.stdout:4/881: dread da/dc/d15/f4e [0,4194304] 0 2024-04-20T11:09:44.136 INFO:tasks.workunit.client.1.smithi173.stdout:7/781: write d9/da/dc3/fff [1866365,82446] 0 2024-04-20T11:09:44.138 INFO:tasks.workunit.client.1.smithi173.stdout:9/765: dwrite d5/d4c/f52 [0,4194304] 0 2024-04-20T11:09:44.139 INFO:tasks.workunit.client.1.smithi173.stdout:6/794: dwrite d4/d24/d42/d81/f99 [0,4194304] 0 2024-04-20T11:09:44.140 INFO:tasks.workunit.client.1.smithi173.stdout:7/782: stat d9/da/d17/f1b 0 2024-04-20T11:09:44.140 INFO:tasks.workunit.client.1.smithi173.stdout:0/754: truncate d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/fba 117998 0 2024-04-20T11:09:44.151 INFO:tasks.workunit.client.1.smithi173.stdout:0/755: unlink d6/f1e 0 2024-04-20T11:09:44.153 INFO:tasks.workunit.client.1.smithi173.stdout:0/756: chown d6/l63 861268808 1 2024-04-20T11:09:44.156 INFO:tasks.workunit.client.1.smithi173.stdout:0/757: write d6/d8/df/d2c/d42/d50/f76 [4878229,117421] 0 2024-04-20T11:09:44.163 INFO:tasks.workunit.client.1.smithi173.stdout:0/758: rmdir d6/d8/d3c/d44/ddb 0 2024-04-20T11:09:44.167 INFO:tasks.workunit.client.1.smithi173.stdout:0/759: fsync d6/d8/d3c/d44/d69/de2/fb3 0 2024-04-20T11:09:44.167 INFO:tasks.workunit.client.1.smithi173.stdout:9/766: dwrite d5/d9/d20/d2b/d59/fd8 [0,4194304] 0 2024-04-20T11:09:44.171 INFO:tasks.workunit.client.1.smithi173.stdout:9/767: dread - d5/d4c/f81 zero size 2024-04-20T11:09:44.172 INFO:tasks.workunit.client.1.smithi173.stdout:4/882: dread da/dc/f33 [0,4194304] 0 2024-04-20T11:09:44.173 INFO:tasks.workunit.client.1.smithi173.stdout:7/783: write d9/f30 [6557598,47347] 0 2024-04-20T11:09:44.174 INFO:tasks.workunit.client.1.smithi173.stdout:0/760: symlink d6/d8/df/d2c/d42/d50/d86/d8b/lec 0 2024-04-20T11:09:44.175 INFO:tasks.workunit.client.1.smithi173.stdout:7/784: readlink d9/da/lb 0 2024-04-20T11:09:44.175 INFO:tasks.workunit.client.1.smithi173.stdout:0/761: readlink d6/l63 0 2024-04-20T11:09:44.181 INFO:tasks.workunit.client.1.smithi173.stdout:4/883: unlink da/dc/fb8 0 2024-04-20T11:09:44.188 INFO:tasks.workunit.client.1.smithi173.stdout:9/768: mkdir d5/d9/d20/d2b/d59/da1/deb 0 2024-04-20T11:09:44.191 INFO:tasks.workunit.client.1.smithi173.stdout:9/769: chown d5/d4c/lc6 74 1 2024-04-20T11:09:44.192 INFO:tasks.workunit.client.1.smithi173.stdout:7/785: dwrite d9/d33/d85/fc5 [0,4194304] 0 2024-04-20T11:09:44.196 INFO:tasks.workunit.client.1.smithi173.stdout:7/786: truncate d9/d33/d85/fc5 4245119 0 2024-04-20T11:09:44.198 INFO:tasks.workunit.client.1.smithi173.stdout:9/770: unlink d5/d9/fb9 0 2024-04-20T11:09:44.207 INFO:tasks.workunit.client.1.smithi173.stdout:9/771: link d5/faf d5/d9/fec 0 2024-04-20T11:09:44.210 INFO:tasks.workunit.client.1.smithi173.stdout:0/762: dwrite d6/d8/d3c/d44/d75/fa1 [4194304,4194304] 0 2024-04-20T11:09:44.211 INFO:tasks.workunit.client.1.smithi173.stdout:0/763: readlink d6/d24/l99 0 2024-04-20T11:09:44.213 INFO:tasks.workunit.client.1.smithi173.stdout:4/884: dread da/d1e/d52/d9a/dd3/def/f108 [0,4194304] 0 2024-04-20T11:09:44.216 INFO:tasks.workunit.client.1.smithi173.stdout:0/764: truncate d6/d8/df/d2c/d2d/f8a 5186011 0 2024-04-20T11:09:44.227 INFO:tasks.workunit.client.1.smithi173.stdout:7/787: dread d9/d33/d35/d4a/f93 [0,4194304] 0 2024-04-20T11:09:44.229 INFO:tasks.workunit.client.1.smithi173.stdout:9/772: truncate d5/d9/f1d 1038876 0 2024-04-20T11:09:44.230 INFO:tasks.workunit.client.1.smithi173.stdout:7/788: truncate d9/d33/f6b 5029168 0 2024-04-20T11:09:44.230 INFO:tasks.workunit.client.1.smithi173.stdout:4/885: link da/dc/d62/d84/l99 da/dc/df4/l12e 0 2024-04-20T11:09:44.231 INFO:tasks.workunit.client.1.smithi173.stdout:4/886: chown da/f64 5583 1 2024-04-20T11:09:44.232 INFO:tasks.workunit.client.1.smithi173.stdout:7/789: dread - d9/d33/d35/d54/fde zero size 2024-04-20T11:09:44.232 INFO:tasks.workunit.client.1.smithi173.stdout:9/773: stat d5/d9/d20/d29/d18/d9b/l8f 0 2024-04-20T11:09:44.237 INFO:tasks.workunit.client.1.smithi173.stdout:9/774: creat d5/d4c/d79/fed x:0 0 0 2024-04-20T11:09:44.240 INFO:tasks.workunit.client.1.smithi173.stdout:0/765: dwrite d6/d8/df/f11 [0,4194304] 0 2024-04-20T11:09:44.263 INFO:tasks.workunit.client.1.smithi173.stdout:8/798: sync 2024-04-20T11:09:44.266 INFO:tasks.workunit.client.1.smithi173.stdout:7/790: write d9/d1f/d7f/d8d/faf [986665,80637] 0 2024-04-20T11:09:44.270 INFO:tasks.workunit.client.1.smithi173.stdout:0/766: rmdir d6/d8/df 39 2024-04-20T11:09:44.273 INFO:tasks.workunit.client.1.smithi173.stdout:9/775: rename d5/d9/d20/d2b/d59/d44/d78/d82/db5 to d5/d9/d20/d2b/d59/da1/dee 0 2024-04-20T11:09:44.273 INFO:tasks.workunit.client.1.smithi173.stdout:0/767: fdatasync d6/d8/d51/fc5 0 2024-04-20T11:09:44.275 INFO:tasks.workunit.client.1.smithi173.stdout:7/791: truncate d9/d1f/d7f/dce/dad/fc4 656218 0 2024-04-20T11:09:44.280 INFO:tasks.workunit.client.1.smithi173.stdout:9/776: fsync d5/d9/d20/d2b/d59/d44/d78/d9d/fb7 0 2024-04-20T11:09:44.285 INFO:tasks.workunit.client.1.smithi173.stdout:0/768: creat d6/d8/df/d2c/d42/d50/d86/d8b/fed x:0 0 0 2024-04-20T11:09:44.287 INFO:tasks.workunit.client.1.smithi173.stdout:0/769: stat d6/d8/df/d2c/d42/f84 0 2024-04-20T11:09:44.292 INFO:tasks.workunit.client.1.smithi173.stdout:0/770: truncate d6/d8/df/d43/fc0 1077828 0 2024-04-20T11:09:44.293 INFO:tasks.workunit.client.1.smithi173.stdout:7/792: write d9/d33/d35/d54/fde [820589,122701] 0 2024-04-20T11:09:44.296 INFO:tasks.workunit.client.1.smithi173.stdout:8/799: truncate d4/d18/d59/d5a/da8/db9/de0/d3c/f54 1055833 0 2024-04-20T11:09:44.297 INFO:tasks.workunit.client.1.smithi173.stdout:7/793: chown d9/d33/d35/d4a/d52/df2/dc6 116694716 1 2024-04-20T11:09:44.299 INFO:tasks.workunit.client.1.smithi173.stdout:7/794: readlink d9/d1f/d7f/d8d/le4 0 2024-04-20T11:09:44.299 INFO:tasks.workunit.client.1.smithi173.stdout:9/777: dwrite d5/d9/d20/d29/d18/d9b/fbc [0,4194304] 0 2024-04-20T11:09:44.307 INFO:tasks.workunit.client.1.smithi173.stdout:7/795: creat d9/d1f/d7f/f115 x:0 0 0 2024-04-20T11:09:44.313 INFO:tasks.workunit.client.1.smithi173.stdout:7/796: truncate d9/d1f/d72/d8f/f100 115849 0 2024-04-20T11:09:44.314 INFO:tasks.workunit.client.1.smithi173.stdout:8/800: dwrite d4/d18/f6b [0,4194304] 0 2024-04-20T11:09:44.322 INFO:tasks.workunit.client.1.smithi173.stdout:0/771: getdents d6/d5f/d8f 0 2024-04-20T11:09:44.323 INFO:tasks.workunit.client.1.smithi173.stdout:7/797: creat d9/d1f/d72/f116 x:0 0 0 2024-04-20T11:09:44.326 INFO:tasks.workunit.client.1.smithi173.stdout:7/798: dread - d9/da/d17/d2b/daa/dd9/ff5 zero size 2024-04-20T11:09:44.327 INFO:tasks.workunit.client.1.smithi173.stdout:0/772: dread d6/d8/d3c/d44/d75/f7b [4194304,4194304] 0 2024-04-20T11:09:44.330 INFO:tasks.workunit.client.1.smithi173.stdout:0/773: mkdir d6/d8/d3c/dee 0 2024-04-20T11:09:44.333 INFO:tasks.workunit.client.1.smithi173.stdout:8/801: creat d4/d18/d59/f100 x:0 0 0 2024-04-20T11:09:44.333 INFO:tasks.workunit.client.1.smithi173.stdout:0/774: readlink d6/d8/df/l88 0 2024-04-20T11:09:44.334 INFO:tasks.workunit.client.1.smithi173.stdout:8/802: readlink d4/d18/d59/d5a/d7b/ld8 0 2024-04-20T11:09:44.338 INFO:tasks.workunit.client.1.smithi173.stdout:0/775: mkdir d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/def 0 2024-04-20T11:09:44.338 INFO:tasks.workunit.client.1.smithi173.stdout:8/803: write d4/d18/d59/d5a/da8/db9/de0/d9b/dc8/ff9 [927136,72965] 0 2024-04-20T11:09:44.340 INFO:tasks.workunit.client.1.smithi173.stdout:8/804: rmdir d4 39 2024-04-20T11:09:44.341 INFO:tasks.workunit.client.1.smithi173.stdout:0/776: write d6/d8/d3c/d44/d69/de2/fb3 [664966,33863] 0 2024-04-20T11:09:44.344 INFO:tasks.workunit.client.1.smithi173.stdout:9/778: dread d5/d9/d20/d2b/d59/d44/f6a [4194304,4194304] 0 2024-04-20T11:09:44.344 INFO:tasks.workunit.client.1.smithi173.stdout:8/805: chown d4/d18/d59/d5a/f95 7 1 2024-04-20T11:09:44.345 INFO:tasks.workunit.client.1.smithi173.stdout:2/886: sync 2024-04-20T11:09:44.347 INFO:tasks.workunit.client.1.smithi173.stdout:8/806: unlink d4/c31 0 2024-04-20T11:09:44.349 INFO:tasks.workunit.client.1.smithi173.stdout:7/799: dread d9/da/d17/f64 [0,4194304] 0 2024-04-20T11:09:44.352 INFO:tasks.workunit.client.1.smithi173.stdout:0/777: rename d6/d8/df/d43/d7f/f9a to d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/ff0 0 2024-04-20T11:09:44.361 INFO:tasks.workunit.client.1.smithi173.stdout:2/887: dwrite d2/d13/d9a/fdf [0,4194304] 0 2024-04-20T11:09:44.364 INFO:tasks.workunit.client.1.smithi173.stdout:9/779: creat d5/d9/d20/d2b/d59/d44/d78/fef x:0 0 0 2024-04-20T11:09:44.364 INFO:tasks.workunit.client.1.smithi173.stdout:2/888: fdatasync d2/de/d11/d9d/d87/f97 0 2024-04-20T11:09:44.365 INFO:tasks.workunit.client.1.smithi173.stdout:7/800: dwrite d9/d1f/f7d [4194304,4194304] 0 2024-04-20T11:09:44.366 INFO:tasks.workunit.client.1.smithi173.stdout:8/807: creat d4/d18/d4f/d5b/f101 x:0 0 0 2024-04-20T11:09:44.367 INFO:tasks.workunit.client.1.smithi173.stdout:0/778: symlink d6/d8/df/d2c/d42/d50/de3/lf1 0 2024-04-20T11:09:44.370 INFO:tasks.workunit.client.1.smithi173.stdout:2/889: fsync d2/d13/f44 0 2024-04-20T11:09:44.370 INFO:tasks.workunit.client.1.smithi173.stdout:7/801: mknod d9/d33/d35/d4a/c117 0 2024-04-20T11:09:44.377 INFO:tasks.workunit.client.1.smithi173.stdout:2/890: stat d2/d13/d9a/dbc/dfd/d53/d9f/lef 0 2024-04-20T11:09:44.377 INFO:tasks.workunit.client.1.smithi173.stdout:0/779: creat d6/d8/d3c/dee/ff2 x:0 0 0 2024-04-20T11:09:44.379 INFO:tasks.workunit.client.1.smithi173.stdout:0/780: chown d6/d8/df/f11 203 1 2024-04-20T11:09:44.380 INFO:tasks.workunit.client.1.smithi173.stdout:2/891: write d2/d13/da5/db2/f11d [1045475,128856] 0 2024-04-20T11:09:44.384 INFO:tasks.workunit.client.1.smithi173.stdout:2/892: creat d2/d13/da5/d101/f127 x:0 0 0 2024-04-20T11:09:44.384 INFO:tasks.workunit.client.1.smithi173.stdout:7/802: dread d9/da/d17/d2b/f71 [4194304,4194304] 0 2024-04-20T11:09:44.386 INFO:tasks.workunit.client.1.smithi173.stdout:7/803: dread - d9/d1f/d72/f116 zero size 2024-04-20T11:09:44.399 INFO:tasks.workunit.client.1.smithi173.stdout:7/804: dwrite d9/fb7 [0,4194304] 0 2024-04-20T11:09:44.404 INFO:tasks.workunit.client.1.smithi173.stdout:2/893: creat d2/d13/d9a/dbc/dfd/d4b/dae/f128 x:0 0 0 2024-04-20T11:09:44.405 INFO:tasks.workunit.client.1.smithi173.stdout:0/781: truncate d6/f13 945286 0 2024-04-20T11:09:44.407 INFO:tasks.workunit.client.1.smithi173.stdout:4/887: sync 2024-04-20T11:09:44.407 INFO:tasks.workunit.client.1.smithi173.stdout:3/747: sync 2024-04-20T11:09:44.407 INFO:tasks.workunit.client.1.smithi173.stdout:1/718: sync 2024-04-20T11:09:44.407 INFO:tasks.workunit.client.1.smithi173.stdout:6/795: sync 2024-04-20T11:09:44.407 INFO:tasks.workunit.client.1.smithi173.stdout:5/816: sync 2024-04-20T11:09:44.408 INFO:tasks.workunit.client.1.smithi173.stdout:7/805: truncate d9/d33/d35/d54/db3/f109 4902329 0 2024-04-20T11:09:44.413 INFO:tasks.workunit.client.1.smithi173.stdout:4/888: readlink da/d1e/d52/l5a 0 2024-04-20T11:09:44.413 INFO:tasks.workunit.client.1.smithi173.stdout:2/894: rmdir d2/d13/d118 0 2024-04-20T11:09:44.415 INFO:tasks.workunit.client.1.smithi173.stdout:8/808: dwrite d4/d18/fa2 [4194304,4194304] 0 2024-04-20T11:09:44.416 INFO:tasks.workunit.client.1.smithi173.stdout:4/889: readlink da/d1a/d60/laf 0 2024-04-20T11:09:44.417 INFO:tasks.workunit.client.1.smithi173.stdout:2/895: chown d2/de/d11/d9d/d87 68171683 1 2024-04-20T11:09:44.417 INFO:tasks.workunit.client.1.smithi173.stdout:9/780: dwrite d5/d4c/f91 [0,4194304] 0 2024-04-20T11:09:44.419 INFO:tasks.workunit.client.1.smithi173.stdout:1/719: write d5/d6/d17/d42/f19 [2976006,47750] 0 2024-04-20T11:09:44.420 INFO:tasks.workunit.client.1.smithi173.stdout:7/806: creat d9/d33/d35/d4a/d52/df2/dc6/df3/f118 x:0 0 0 2024-04-20T11:09:44.420 INFO:tasks.workunit.client.1.smithi173.stdout:3/748: truncate d7/d17/d61/d83/dcf/f66 8624461 0 2024-04-20T11:09:44.437 INFO:tasks.workunit.client.1.smithi173.stdout:6/796: dwrite d4/d1c/d8e/d90/ff3 [0,4194304] 0 2024-04-20T11:09:44.437 INFO:tasks.workunit.client.1.smithi173.stdout:7/807: symlink d9/d1f/d7f/dce/dad/l119 0 2024-04-20T11:09:44.437 INFO:tasks.workunit.client.1.smithi173.stdout:8/809: write d4/d18/f72 [3797602,42069] 0 2024-04-20T11:09:44.445 INFO:tasks.workunit.client.1.smithi173.stdout:8/810: dread - d4/d18/da6/fbf zero size 2024-04-20T11:09:44.447 INFO:tasks.workunit.client.1.smithi173.stdout:7/808: chown d9/d33/d35/d56/c88 16247 1 2024-04-20T11:09:44.448 INFO:tasks.workunit.client.1.smithi173.stdout:1/720: creat d5/d6/d17/d42/ff2 x:0 0 0 2024-04-20T11:09:44.449 INFO:tasks.workunit.client.1.smithi173.stdout:4/890: getdents da/dd1/df5 0 2024-04-20T11:09:44.451 INFO:tasks.workunit.client.1.smithi173.stdout:9/781: link d5/d9/d20/d29/d2d/l2e d5/d9/lf0 0 2024-04-20T11:09:44.452 INFO:tasks.workunit.client.1.smithi173.stdout:4/891: chown da/dc/d13/df0 6532871 1 2024-04-20T11:09:44.454 INFO:tasks.workunit.client.1.smithi173.stdout:4/892: stat da/d1a/l7c 0 2024-04-20T11:09:44.457 INFO:tasks.workunit.client.1.smithi173.stdout:2/896: rename d2/d13/la8 to d2/d13/d9a/dbc/dfd/d53/l129 0 2024-04-20T11:09:44.458 INFO:tasks.workunit.client.1.smithi173.stdout:8/811: symlink d4/d18/d59/d5a/d7b/db1/l102 0 2024-04-20T11:09:44.458 INFO:tasks.workunit.client.1.smithi173.stdout:5/817: dread da/de/d19/d2c/f79 [0,4194304] 0 2024-04-20T11:09:44.460 INFO:tasks.workunit.client.1.smithi173.stdout:7/809: creat d9/da/f11a x:0 0 0 2024-04-20T11:09:44.464 INFO:tasks.workunit.client.1.smithi173.stdout:9/782: mkdir d5/d9/d20/d29/d2d/df1 0 2024-04-20T11:09:44.464 INFO:tasks.workunit.client.1.smithi173.stdout:7/810: write d9/d33/de9/f111 [811974,14355] 0 2024-04-20T11:09:44.467 INFO:tasks.workunit.client.1.smithi173.stdout:4/893: fsync da/d1e/f38 0 2024-04-20T11:09:44.467 INFO:tasks.workunit.client.1.smithi173.stdout:3/749: dwrite d7/f18 [0,4194304] 0 2024-04-20T11:09:44.470 INFO:tasks.workunit.client.1.smithi173.stdout:7/811: write d9/d1f/d72/d8f/f100 [1163257,110672] 0 2024-04-20T11:09:44.472 INFO:tasks.workunit.client.1.smithi173.stdout:2/897: link d2/de/fd4 d2/d34/d81/dca/f12a 0 2024-04-20T11:09:44.473 INFO:tasks.workunit.client.1.smithi173.stdout:5/818: rmdir da/de/d19/d32/d30/d101 0 2024-04-20T11:09:44.475 INFO:tasks.workunit.client.1.smithi173.stdout:2/898: read d2/fc5 [4970,68620] 0 2024-04-20T11:09:44.479 INFO:tasks.workunit.client.1.smithi173.stdout:7/812: write d9/d33/f3a [2281672,114513] 0 2024-04-20T11:09:44.482 INFO:tasks.workunit.client.1.smithi173.stdout:5/819: truncate da/de/d75/d3e/fed 956060 0 2024-04-20T11:09:44.485 INFO:tasks.workunit.client.1.smithi173.stdout:8/812: dwrite d4/d18/d59/d5a/da8/db9/de0/f46 [0,4194304] 0 2024-04-20T11:09:44.487 INFO:tasks.workunit.client.1.smithi173.stdout:9/783: link d5/d9/d20/c23 d5/d9/d20/d29/d18/d9b/d98/dc4/cf2 0 2024-04-20T11:09:44.488 INFO:tasks.workunit.client.1.smithi173.stdout:7/813: read d9/da/dc3/f101 [8339509,129082] 0 2024-04-20T11:09:44.491 INFO:tasks.workunit.client.1.smithi173.stdout:1/721: dread d5/d6/d17/d42/da/df/d40/db4/fd2 [0,4194304] 0 2024-04-20T11:09:44.492 INFO:tasks.workunit.client.1.smithi173.stdout:3/750: symlink d7/d17/d61/d83/d45/leb 0 2024-04-20T11:09:44.496 INFO:tasks.workunit.client.1.smithi173.stdout:8/813: mknod d4/d18/d59/d5a/da8/db9/de0/d9b/c103 0 2024-04-20T11:09:44.496 INFO:tasks.workunit.client.1.smithi173.stdout:9/784: symlink d5/d9/d20/d2b/d31/lf3 0 2024-04-20T11:09:44.498 INFO:tasks.workunit.client.1.smithi173.stdout:5/820: mkdir da/d9e/d118 0 2024-04-20T11:09:44.499 INFO:tasks.workunit.client.1.smithi173.stdout:4/894: write da/dc/d13/f2b [1451051,25531] 0 2024-04-20T11:09:44.501 INFO:tasks.workunit.client.1.smithi173.stdout:1/722: mkdir d5/d6/d17/d42/da/d20/d48/d81/df3 0 2024-04-20T11:09:44.502 INFO:tasks.workunit.client.1.smithi173.stdout:4/895: write da/d1e/ffd [951223,43952] 0 2024-04-20T11:09:44.503 INFO:tasks.workunit.client.1.smithi173.stdout:1/723: fdatasync d5/d6/dc/d7f/fd0 0 2024-04-20T11:09:44.504 INFO:tasks.workunit.client.1.smithi173.stdout:9/785: mkdir d5/dd7/df4 0 2024-04-20T11:09:44.507 INFO:tasks.workunit.client.1.smithi173.stdout:2/899: truncate d2/d13/d9a/dbc/dfd/f85 4941966 0 2024-04-20T11:09:44.507 INFO:tasks.workunit.client.1.smithi173.stdout:8/814: write d4/d18/d59/d5a/da8/db9/de0/f66 [1370935,101477] 0 2024-04-20T11:09:44.508 INFO:tasks.workunit.client.1.smithi173.stdout:3/751: creat d7/d17/d58/d8a/dc4/dd1/fec x:0 0 0 2024-04-20T11:09:44.512 INFO:tasks.workunit.client.1.smithi173.stdout:1/724: mkdir d5/d6/d17/d93/db6/df4 0 2024-04-20T11:09:44.512 INFO:tasks.workunit.client.1.smithi173.stdout:9/786: creat d5/d4c/d79/d80/ff5 x:0 0 0 2024-04-20T11:09:44.512 INFO:tasks.workunit.client.1.smithi173.stdout:4/896: symlink da/d1e/d52/l12f 0 2024-04-20T11:09:44.514 INFO:tasks.workunit.client.1.smithi173.stdout:4/897: chown da/d1e/ffd 3786 1 2024-04-20T11:09:44.514 INFO:tasks.workunit.client.1.smithi173.stdout:8/815: write d4/d18/d59/d5a/d7b/db1/fd9 [215086,101640] 0 2024-04-20T11:09:44.521 INFO:tasks.workunit.client.1.smithi173.stdout:9/787: mkdir d5/d9/d20/d4f/d5e/df6 0 2024-04-20T11:09:44.523 INFO:tasks.workunit.client.1.smithi173.stdout:4/898: mknod da/dc/df4/d53/d39/c130 0 2024-04-20T11:09:44.528 INFO:tasks.workunit.client.1.smithi173.stdout:2/900: truncate d2/fd8 3345982 0 2024-04-20T11:09:44.529 INFO:tasks.workunit.client.1.smithi173.stdout:8/816: rename d4/d18/d59/d6d to d4/d104 0 2024-04-20T11:09:44.540 INFO:tasks.workunit.client.1.smithi173.stdout:7/814: dwrite d9/d1f/d72/f75 [0,4194304] 0 2024-04-20T11:09:44.542 INFO:tasks.workunit.client.1.smithi173.stdout:4/899: symlink da/dc/df4/d53/l131 0 2024-04-20T11:09:44.542 INFO:tasks.workunit.client.1.smithi173.stdout:3/752: dwrite d7/d17/d61/d83/dcf/d5c/fc6 [0,4194304] 0 2024-04-20T11:09:44.545 INFO:tasks.workunit.client.1.smithi173.stdout:7/815: stat d9/d33/f47 0 2024-04-20T11:09:44.546 INFO:tasks.workunit.client.1.smithi173.stdout:2/901: rmdir d2/d13/da5 39 2024-04-20T11:09:44.548 INFO:tasks.workunit.client.1.smithi173.stdout:5/821: dwrite da/de/d75/f45 [0,4194304] 0 2024-04-20T11:09:44.550 INFO:tasks.workunit.client.1.smithi173.stdout:3/753: fdatasync d7/d17/d61/d83/dcf/f4f 0 2024-04-20T11:09:44.553 INFO:tasks.workunit.client.1.smithi173.stdout:7/816: chown d9/ld 4562 1 2024-04-20T11:09:44.553 INFO:tasks.workunit.client.1.smithi173.stdout:4/900: readlink da/d1e/d52/d7a/ld6 0 2024-04-20T11:09:44.554 INFO:tasks.workunit.client.1.smithi173.stdout:8/817: creat d4/d18/d59/d5a/da8/db9/de0/d3c/f105 x:0 0 0 2024-04-20T11:09:44.554 INFO:tasks.workunit.client.1.smithi173.stdout:5/822: stat da/de/d75/lba 0 2024-04-20T11:09:44.558 INFO:tasks.workunit.client.1.smithi173.stdout:5/823: stat da/d4f/d65/dd5/c117 0 2024-04-20T11:09:44.560 INFO:tasks.workunit.client.1.smithi173.stdout:2/902: creat d2/d34/d81/dca/f12b x:0 0 0 2024-04-20T11:09:44.564 INFO:tasks.workunit.client.1.smithi173.stdout:9/788: dwrite d5/d9/d20/d2b/d59/f4e [0,4194304] 0 2024-04-20T11:09:44.564 INFO:tasks.workunit.client.1.smithi173.stdout:8/818: fdatasync d4/d18/fcc 0 2024-04-20T11:09:44.566 INFO:tasks.workunit.client.1.smithi173.stdout:3/754: symlink d7/d17/d61/dd6/led 0 2024-04-20T11:09:44.569 INFO:tasks.workunit.client.1.smithi173.stdout:4/901: rename da/d1a/d60/d66/dae to da/d1a/d6e/db4/d132 0 2024-04-20T11:09:44.570 INFO:tasks.workunit.client.1.smithi173.stdout:4/902: fdatasync da/dc/d15/f2a 0 2024-04-20T11:09:44.575 INFO:tasks.workunit.client.1.smithi173.stdout:0/782: sync 2024-04-20T11:09:44.575 INFO:tasks.workunit.client.1.smithi173.stdout:6/797: sync 2024-04-20T11:09:44.577 INFO:tasks.workunit.client.1.smithi173.stdout:6/798: stat d4/d24/d42/d81/ce2 0 2024-04-20T11:09:44.578 INFO:tasks.workunit.client.1.smithi173.stdout:8/819: mknod d4/d104/c106 0 2024-04-20T11:09:44.580 INFO:tasks.workunit.client.1.smithi173.stdout:6/799: chown d4/d59/c7b 110 1 2024-04-20T11:09:44.582 INFO:tasks.workunit.client.1.smithi173.stdout:7/817: symlink d9/da/d17/d2b/daa/d10b/l11b 0 2024-04-20T11:09:44.585 INFO:tasks.workunit.client.1.smithi173.stdout:3/755: chown d7/d2b/d8e/fd5 448947576 1 2024-04-20T11:09:44.587 INFO:tasks.workunit.client.1.smithi173.stdout:4/903: mkdir da/dd1/d133 0 2024-04-20T11:09:44.588 INFO:tasks.workunit.client.1.smithi173.stdout:3/756: write d7/fe0 [687587,126300] 0 2024-04-20T11:09:44.597 INFO:tasks.workunit.client.1.smithi173.stdout:8/820: truncate d4/ff 3301800 0 2024-04-20T11:09:44.599 INFO:tasks.workunit.client.1.smithi173.stdout:5/824: dread da/d4f/d5e/f6d [0,4194304] 0 2024-04-20T11:09:44.601 INFO:tasks.workunit.client.1.smithi173.stdout:4/904: mknod da/dd1/c134 0 2024-04-20T11:09:44.602 INFO:tasks.workunit.client.1.smithi173.stdout:7/818: mkdir d9/d33/d40/d11c 0 2024-04-20T11:09:44.605 INFO:tasks.workunit.client.1.smithi173.stdout:3/757: symlink d7/d17/d61/d83/dc2/lee 0 2024-04-20T11:09:44.606 INFO:tasks.workunit.client.1.smithi173.stdout:4/905: read da/dc/d15/f1d [6167248,62276] 0 2024-04-20T11:09:44.607 INFO:tasks.workunit.client.1.smithi173.stdout:2/903: dread d2/d13/d9a/dbc/dfd/fe9 [0,4194304] 0 2024-04-20T11:09:44.608 INFO:tasks.workunit.client.1.smithi173.stdout:4/906: fdatasync da/dc/d13/fc6 0 2024-04-20T11:09:44.608 INFO:tasks.workunit.client.1.smithi173.stdout:6/800: creat d4/d1c/ffb x:0 0 0 2024-04-20T11:09:44.609 INFO:tasks.workunit.client.1.smithi173.stdout:0/783: dread d6/d8/df/d2c/d2d/f8a [0,4194304] 0 2024-04-20T11:09:44.609 INFO:tasks.workunit.client.1.smithi173.stdout:8/821: truncate d4/f8 818851 0 2024-04-20T11:09:44.611 INFO:tasks.workunit.client.1.smithi173.stdout:9/789: getdents d5/d9/d20/d2b/d59/da1 0 2024-04-20T11:09:44.613 INFO:tasks.workunit.client.1.smithi173.stdout:3/758: truncate d7/d17/f57 4191453 0 2024-04-20T11:09:44.620 INFO:tasks.workunit.client.1.smithi173.stdout:3/759: dread d7/d17/d61/d83/dcf/d30/fbe [0,4194304] 0 2024-04-20T11:09:44.620 INFO:tasks.workunit.client.1.smithi173.stdout:7/819: dwrite d9/d33/d40/d89/feb [0,4194304] 0 2024-04-20T11:09:44.620 INFO:tasks.workunit.client.1.smithi173.stdout:4/907: symlink da/d1e/d52/d7a/l135 0 2024-04-20T11:09:44.623 INFO:tasks.workunit.client.1.smithi173.stdout:2/904: rmdir d2/d13/d9a/dbc/dfd/d4b/dae/db6 39 2024-04-20T11:09:44.628 INFO:tasks.workunit.client.1.smithi173.stdout:0/784: dwrite d6/d8/df/d15/f95 [0,4194304] 0 2024-04-20T11:09:44.636 INFO:tasks.workunit.client.1.smithi173.stdout:2/905: dwrite d2/d8c/ff6 [0,4194304] 0 2024-04-20T11:09:44.644 INFO:tasks.workunit.client.1.smithi173.stdout:8/822: rmdir d4/d18/d4f/d5b 39 2024-04-20T11:09:44.651 INFO:tasks.workunit.client.1.smithi173.stdout:3/760: symlink d7/d17/d58/d8a/dc4/lef 0 2024-04-20T11:09:44.656 INFO:tasks.workunit.client.1.smithi173.stdout:6/801: creat d4/d24/ffc x:0 0 0 2024-04-20T11:09:44.663 INFO:tasks.workunit.client.1.smithi173.stdout:2/906: rename d2/d13/d9a/dbc/dfd/d4b/dae/fbb to d2/d13/d9a/dbc/dfd/d53/d9f/f12c 0 2024-04-20T11:09:44.666 INFO:tasks.workunit.client.1.smithi173.stdout:8/823: readlink d4/d18/d59/d5a/da8/db9/de0/lc2 0 2024-04-20T11:09:44.666 INFO:tasks.workunit.client.1.smithi173.stdout:1/725: sync 2024-04-20T11:09:44.667 INFO:tasks.workunit.client.1.smithi173.stdout:4/908: creat da/dc/df4/dff/f136 x:0 0 0 2024-04-20T11:09:44.670 INFO:tasks.workunit.client.1.smithi173.stdout:9/790: truncate d5/d4c/f91 2032513 0 2024-04-20T11:09:44.678 INFO:tasks.workunit.client.1.smithi173.stdout:7/820: mkdir d9/d33/d91/d11d 0 2024-04-20T11:09:44.679 INFO:tasks.workunit.client.1.smithi173.stdout:8/824: mkdir d4/d18/d59/d5a/d7b/db1/d107 0 2024-04-20T11:09:44.683 INFO:tasks.workunit.client.1.smithi173.stdout:2/907: dread d2/d13/da5/db2/f11d [0,4194304] 0 2024-04-20T11:09:44.689 INFO:tasks.workunit.client.1.smithi173.stdout:8/825: mkdir d4/d18/d59/d5a/da8/db9/de0/d9b/dc8/d108 0 2024-04-20T11:09:44.691 INFO:tasks.workunit.client.1.smithi173.stdout:4/909: creat da/d1a/d60/d7d/d101/d123/d124/f137 x:0 0 0 2024-04-20T11:09:44.693 INFO:tasks.workunit.client.1.smithi173.stdout:4/910: chown da/d1e/d52/f97 17 1 2024-04-20T11:09:44.695 INFO:tasks.workunit.client.1.smithi173.stdout:9/791: mkdir d5/d9/d20/d4f/d5e/d9e/dc2/df7 0 2024-04-20T11:09:44.695 INFO:tasks.workunit.client.1.smithi173.stdout:7/821: dwrite d9/fed [0,4194304] 0 2024-04-20T11:09:44.697 INFO:tasks.workunit.client.1.smithi173.stdout:9/792: readlink d5/d9/d20/d2b/d31/lbf 0 2024-04-20T11:09:44.698 INFO:tasks.workunit.client.1.smithi173.stdout:7/822: write d9/da/d17/fbb [1161138,112142] 0 2024-04-20T11:09:44.702 INFO:tasks.workunit.client.1.smithi173.stdout:4/911: creat da/dc/d103/d11e/f138 x:0 0 0 2024-04-20T11:09:44.705 INFO:tasks.workunit.client.1.smithi173.stdout:3/761: dwrite d7/d17/d58/fb2 [0,4194304] 0 2024-04-20T11:09:44.711 INFO:tasks.workunit.client.1.smithi173.stdout:4/912: dread - da/d1e/da6/f128 zero size 2024-04-20T11:09:44.717 INFO:tasks.workunit.client.1.smithi173.stdout:3/762: rename d7/d17/f98 to d7/d17/d58/d8a/dc4/dd1/ff0 0 2024-04-20T11:09:44.722 INFO:tasks.workunit.client.1.smithi173.stdout:2/908: dread d2/d13/d9a/dbc/f102 [0,4194304] 0 2024-04-20T11:09:44.724 INFO:tasks.workunit.client.1.smithi173.stdout:4/913: mknod da/d1a/d6e/c139 0 2024-04-20T11:09:44.724 INFO:tasks.workunit.client.1.smithi173.stdout:7/823: getdents d9/d33/d35/d4a/def 0 2024-04-20T11:09:44.726 INFO:tasks.workunit.client.1.smithi173.stdout:3/763: chown d7/d2b/d9d/cdf 1672 1 2024-04-20T11:09:44.729 INFO:tasks.workunit.client.1.smithi173.stdout:4/914: symlink da/d1a/d6e/db4/l13a 0 2024-04-20T11:09:44.730 INFO:tasks.workunit.client.1.smithi173.stdout:2/909: truncate d2/d13/d9a/dbc/dfd/d53/f76 1438174 0 2024-04-20T11:09:44.730 INFO:tasks.workunit.client.1.smithi173.stdout:3/764: rmdir d7/d17/d58/d7f 39 2024-04-20T11:09:44.733 INFO:tasks.workunit.client.1.smithi173.stdout:2/910: write d2/d34/d58/f69 [501861,62880] 0 2024-04-20T11:09:44.733 INFO:tasks.workunit.client.1.smithi173.stdout:1/726: dwrite d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fdb [0,4194304] 0 2024-04-20T11:09:44.733 INFO:tasks.workunit.client.1.smithi173.stdout:3/765: dread - d7/d17/d58/d8a/dc4/dd1/fe1 zero size 2024-04-20T11:09:44.735 INFO:tasks.workunit.client.1.smithi173.stdout:7/824: creat d9/d33/d40/f11e x:0 0 0 2024-04-20T11:09:44.738 INFO:tasks.workunit.client.1.smithi173.stdout:3/766: fdatasync f4 0 2024-04-20T11:09:44.740 INFO:tasks.workunit.client.1.smithi173.stdout:3/767: dread - d7/d17/d61/d83/fde zero size 2024-04-20T11:09:44.743 INFO:tasks.workunit.client.1.smithi173.stdout:3/768: fdatasync d7/d17/fa8 0 2024-04-20T11:09:44.746 INFO:tasks.workunit.client.1.smithi173.stdout:7/825: unlink d9/d1f/d7f/dce/fb5 0 2024-04-20T11:09:44.747 INFO:tasks.workunit.client.1.smithi173.stdout:1/727: creat d5/d6/d17/ff5 x:0 0 0 2024-04-20T11:09:44.748 INFO:tasks.workunit.client.1.smithi173.stdout:1/728: fsync f0 0 2024-04-20T11:09:44.750 INFO:tasks.workunit.client.1.smithi173.stdout:7/826: chown d9/d33/d35/d4a/c69 7868720 1 2024-04-20T11:09:44.752 INFO:tasks.workunit.client.1.smithi173.stdout:4/915: dwrite da/d1a/d60/fa4 [0,4194304] 0 2024-04-20T11:09:44.759 INFO:tasks.workunit.client.1.smithi173.stdout:1/729: symlink d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/lf6 0 2024-04-20T11:09:44.761 INFO:tasks.workunit.client.1.smithi173.stdout:7/827: symlink d9/d33/d35/d4a/d52/l11f 0 2024-04-20T11:09:44.781 INFO:tasks.workunit.client.1.smithi173.stdout:9/793: dwrite d5/d9/d20/d29/f5c [0,4194304] 0 2024-04-20T11:09:44.781 INFO:tasks.workunit.client.1.smithi173.stdout:3/769: dwrite d7/d17/d58/d8a/dc4/dd1/fe1 [0,4194304] 0 2024-04-20T11:09:44.786 INFO:tasks.workunit.client.1.smithi173.stdout:1/730: link d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/ce8 d5/d6/cf7 0 2024-04-20T11:09:44.787 INFO:tasks.workunit.client.1.smithi173.stdout:9/794: truncate d5/d4c/d79/d80/ff5 581747 0 2024-04-20T11:09:44.787 INFO:tasks.workunit.client.1.smithi173.stdout:4/916: write da/d1e/d52/f68 [145347,107841] 0 2024-04-20T11:09:44.798 INFO:tasks.workunit.client.1.smithi173.stdout:1/731: symlink d5/d6/d17/lf8 0 2024-04-20T11:09:44.805 INFO:tasks.workunit.client.1.smithi173.stdout:1/732: write d5/d6/d17/d42/d12/f31 [265469,19040] 0 2024-04-20T11:09:44.808 INFO:tasks.workunit.client.1.smithi173.stdout:7/828: dwrite d9/d33/d35/f7c [0,4194304] 0 2024-04-20T11:09:44.811 INFO:tasks.workunit.client.1.smithi173.stdout:9/795: getdents d5/d4c 0 2024-04-20T11:09:44.811 INFO:tasks.workunit.client.1.smithi173.stdout:5/825: sync 2024-04-20T11:09:44.816 INFO:tasks.workunit.client.1.smithi173.stdout:1/733: mkdir d5/dd8/df9 0 2024-04-20T11:09:44.817 INFO:tasks.workunit.client.1.smithi173.stdout:7/829: mknod d9/d1f/d7f/d8d/c120 0 2024-04-20T11:09:44.817 INFO:tasks.workunit.client.1.smithi173.stdout:4/917: dread da/d1e/d52/d9a/dd3/def/f108 [0,4194304] 0 2024-04-20T11:09:44.821 INFO:tasks.workunit.client.1.smithi173.stdout:3/770: dread d7/d53/f71 [0,4194304] 0 2024-04-20T11:09:44.823 INFO:tasks.workunit.client.1.smithi173.stdout:4/918: truncate da/dc/d62/f10a 755249 0 2024-04-20T11:09:44.826 INFO:tasks.workunit.client.1.smithi173.stdout:7/830: write d9/d33/d35/d4a/d52/df2/dc6/df3/f118 [999660,23483] 0 2024-04-20T11:09:44.829 INFO:tasks.workunit.client.1.smithi173.stdout:5/826: write da/de/d19/d2c/f46 [325316,89302] 0 2024-04-20T11:09:44.830 INFO:tasks.workunit.client.1.smithi173.stdout:7/831: chown d9/d33/d85/d97 16 1 2024-04-20T11:09:44.836 INFO:tasks.workunit.client.1.smithi173.stdout:1/734: unlink d5/d6/d17/d42/da/df/l65 0 2024-04-20T11:09:44.838 INFO:tasks.workunit.client.1.smithi173.stdout:7/832: fdatasync d9/d1f/d7f/dce/db2/f107 0 2024-04-20T11:09:44.848 INFO:tasks.workunit.client.1.smithi173.stdout:3/771: dwrite d7/d17/d61/d83/faa [4194304,4194304] 0 2024-04-20T11:09:44.851 INFO:tasks.workunit.client.1.smithi173.stdout:5/827: read da/de/d19/d32/f11 [631112,73542] 0 2024-04-20T11:09:44.854 INFO:tasks.workunit.client.1.smithi173.stdout:1/735: rename d5/d6/dc/d7f/ceb to d5/d6/d17/d42/da/df/d40/dc8/dde/cfa 0 2024-04-20T11:09:44.855 INFO:tasks.workunit.client.1.smithi173.stdout:3/772: rmdir d7/d17/d61/d83 39 2024-04-20T11:09:44.856 INFO:tasks.workunit.client.1.smithi173.stdout:1/736: readlink d5/d6/d17/d42/da/df/d40/db4/led 0 2024-04-20T11:09:44.864 INFO:tasks.workunit.client.1.smithi173.stdout:1/737: creat d5/d6/dc/ffb x:0 0 0 2024-04-20T11:09:44.868 INFO:tasks.workunit.client.1.smithi173.stdout:9/796: write d5/d9/d20/d2b/d59/d44/d78/f7c [169576,69085] 0 2024-04-20T11:09:44.869 INFO:tasks.workunit.client.1.smithi173.stdout:7/833: dread d9/d1f/d72/d8f/f100 [0,4194304] 0 2024-04-20T11:09:44.874 INFO:tasks.workunit.client.1.smithi173.stdout:1/738: mkdir d5/d6/d17/d42/da/df/d40/dc8/dfc 0 2024-04-20T11:09:44.885 INFO:tasks.workunit.client.1.smithi173.stdout:9/797: creat d5/d9/ff8 x:0 0 0 2024-04-20T11:09:44.886 INFO:tasks.workunit.client.1.smithi173.stdout:7/834: creat d9/d33/d40/f121 x:0 0 0 2024-04-20T11:09:44.889 INFO:tasks.workunit.client.1.smithi173.stdout:7/835: write d9/d33/d35/f7c [766147,41213] 0 2024-04-20T11:09:44.889 INFO:tasks.workunit.client.1.smithi173.stdout:9/798: readlink d5/d9/d20/d2b/d59/d44/d78/d9d/la4 0 2024-04-20T11:09:44.895 INFO:tasks.workunit.client.1.smithi173.stdout:7/836: fdatasync d9/d33/d35/d54/db3/d102/f10d 0 2024-04-20T11:09:44.896 INFO:tasks.workunit.client.1.smithi173.stdout:3/773: dread d7/d17/d61/d83/dcf/f66 [0,4194304] 0 2024-04-20T11:09:44.897 INFO:tasks.workunit.client.1.smithi173.stdout:9/799: rename d5/d9/d20/d2b/d59/d44/fdf to d5/d9/d20/d2b/d59/d44/ff9 0 2024-04-20T11:09:44.900 INFO:tasks.workunit.client.1.smithi173.stdout:9/800: dread d5/d9/f47 [0,4194304] 0 2024-04-20T11:09:44.900 INFO:tasks.workunit.client.1.smithi173.stdout:5/828: write da/de/d19/d2c/f79 [4383165,67179] 0 2024-04-20T11:09:44.901 INFO:tasks.workunit.client.1.smithi173.stdout:7/837: mknod d9/d33/d40/c122 0 2024-04-20T11:09:44.902 INFO:tasks.workunit.client.1.smithi173.stdout:3/774: symlink d7/d17/d58/lf1 0 2024-04-20T11:09:44.903 INFO:tasks.workunit.client.1.smithi173.stdout:9/801: chown d5/d9/d20/d29/d2d/f87 15303 1 2024-04-20T11:09:44.905 INFO:tasks.workunit.client.1.smithi173.stdout:7/838: stat d9/d33/d35/fe2 0 2024-04-20T11:09:44.905 INFO:tasks.workunit.client.1.smithi173.stdout:1/739: write d5/d6/d17/d42/da/d20/dad/dc2/f72 [466355,38049] 0 2024-04-20T11:09:44.906 INFO:tasks.workunit.client.1.smithi173.stdout:9/802: creat d5/dcc/ffa x:0 0 0 2024-04-20T11:09:44.908 INFO:tasks.workunit.client.1.smithi173.stdout:7/839: write d9/da/f46 [13400244,78208] 0 2024-04-20T11:09:44.908 INFO:tasks.workunit.client.1.smithi173.stdout:9/803: chown d5/d9/d20/d29 1076 1 2024-04-20T11:09:44.917 INFO:tasks.workunit.client.1.smithi173.stdout:7/840: dread d9/d33/de9/f111 [0,4194304] 0 2024-04-20T11:09:44.917 INFO:tasks.workunit.client.1.smithi173.stdout:1/740: dread d5/d6/d17/d42/d12/d1c/f82 [0,4194304] 0 2024-04-20T11:09:44.919 INFO:tasks.workunit.client.1.smithi173.stdout:5/829: link da/de/d19/d32/d30/ldb da/d4f/d65/d8e/l119 0 2024-04-20T11:09:44.921 INFO:tasks.workunit.client.1.smithi173.stdout:3/775: dread d7/d17/d61/d83/dcf/d5c/fc6 [0,4194304] 0 2024-04-20T11:09:44.921 INFO:tasks.workunit.client.1.smithi173.stdout:6/802: sync 2024-04-20T11:09:44.921 INFO:tasks.workunit.client.1.smithi173.stdout:0/785: sync 2024-04-20T11:09:44.921 INFO:tasks.workunit.client.1.smithi173.stdout:8/826: sync 2024-04-20T11:09:44.921 INFO:tasks.workunit.client.1.smithi173.stdout:1/741: write d5/d6/d17/ff5 [970100,70244] 0 2024-04-20T11:09:44.924 INFO:tasks.workunit.client.1.smithi173.stdout:5/830: write da/d4f/fd3 [292589,92181] 0 2024-04-20T11:09:44.929 INFO:tasks.workunit.client.1.smithi173.stdout:3/776: truncate f4 972448 0 2024-04-20T11:09:44.932 INFO:tasks.workunit.client.1.smithi173.stdout:5/831: rename da/d34/d7f/l80 to da/de/d19/d32/d72/l11a 0 2024-04-20T11:09:44.934 INFO:tasks.workunit.client.1.smithi173.stdout:7/841: write d9/d33/d35/d54/db3/d102/f10d [152272,20740] 0 2024-04-20T11:09:44.935 INFO:tasks.workunit.client.1.smithi173.stdout:3/777: unlink d7/d17/c27 0 2024-04-20T11:09:44.936 INFO:tasks.workunit.client.1.smithi173.stdout:9/804: dread d5/d9/d20/d29/f11 [0,4194304] 0 2024-04-20T11:09:44.941 INFO:tasks.workunit.client.1.smithi173.stdout:7/842: write d9/d1f/f7d [6078194,105436] 0 2024-04-20T11:09:44.941 INFO:tasks.workunit.client.1.smithi173.stdout:0/786: symlink d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/lf3 0 2024-04-20T11:09:44.948 INFO:tasks.workunit.client.1.smithi173.stdout:9/805: symlink d5/d9/d20/d29/lfb 0 2024-04-20T11:09:44.951 INFO:tasks.workunit.client.1.smithi173.stdout:7/843: chown d9/d33/d40/d7a/ce0 6 1 2024-04-20T11:09:44.953 INFO:tasks.workunit.client.1.smithi173.stdout:0/787: creat d6/d8/df/d15/ff4 x:0 0 0 2024-04-20T11:09:44.953 INFO:tasks.workunit.client.1.smithi173.stdout:8/827: write d4/d18/d59/d5a/da8/db9/de0/fd6 [305848,102552] 0 2024-04-20T11:09:44.954 INFO:tasks.workunit.client.1.smithi173.stdout:6/803: dread d4/d24/d42/f7c [0,4194304] 0 2024-04-20T11:09:44.958 INFO:tasks.workunit.client.1.smithi173.stdout:9/806: symlink d5/d9/d20/d2b/d59/da1/dee/lfc 0 2024-04-20T11:09:44.962 INFO:tasks.workunit.client.1.smithi173.stdout:6/804: symlink d4/d18/d7e/lfd 0 2024-04-20T11:09:44.962 INFO:tasks.workunit.client.1.smithi173.stdout:5/832: write da/f3d [499287,23367] 0 2024-04-20T11:09:44.969 INFO:tasks.workunit.client.1.smithi173.stdout:2/911: sync 2024-04-20T11:09:44.970 INFO:tasks.workunit.client.1.smithi173.stdout:9/807: creat d5/d9/d20/d2b/d59/d44/de5/ffd x:0 0 0 2024-04-20T11:09:44.971 INFO:tasks.workunit.client.1.smithi173.stdout:5/833: mknod da/c11b 0 2024-04-20T11:09:44.975 INFO:tasks.workunit.client.1.smithi173.stdout:3/778: dwrite d7/d17/f29 [0,4194304] 0 2024-04-20T11:09:44.975 INFO:tasks.workunit.client.1.smithi173.stdout:8/828: dread d4/d18/d59/d5a/da8/db9/de0/d9b/dc8/ff9 [0,4194304] 0 2024-04-20T11:09:44.977 INFO:tasks.workunit.client.1.smithi173.stdout:1/742: dread d5/d6/d17/d42/d12/d1c/f8b [0,4194304] 0 2024-04-20T11:09:44.979 INFO:tasks.workunit.client.1.smithi173.stdout:7/844: dwrite d9/fed [0,4194304] 0 2024-04-20T11:09:44.980 INFO:tasks.workunit.client.1.smithi173.stdout:2/912: fsync d2/de/f7a 0 2024-04-20T11:09:44.981 INFO:tasks.workunit.client.1.smithi173.stdout:0/788: link d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d65/l6e d6/d8/df/d43/d7f/lf5 0 2024-04-20T11:09:44.983 INFO:tasks.workunit.client.1.smithi173.stdout:5/834: mknod da/c11c 0 2024-04-20T11:09:44.991 INFO:tasks.workunit.client.1.smithi173.stdout:6/805: getdents d4/d24/d3e 0 2024-04-20T11:09:44.994 INFO:tasks.workunit.client.1.smithi173.stdout:3/779: mkdir d7/d17/d58/d7f/df2 0 2024-04-20T11:09:44.996 INFO:tasks.workunit.client.1.smithi173.stdout:3/780: dread - d7/d17/d58/d7f/f95 zero size 2024-04-20T11:09:44.997 INFO:tasks.workunit.client.1.smithi173.stdout:2/913: write d2/de/fd4 [263847,114753] 0 2024-04-20T11:09:45.004 INFO:tasks.workunit.client.1.smithi173.stdout:6/806: creat d4/d18/d61/ffe x:0 0 0 2024-04-20T11:09:45.004 INFO:tasks.workunit.client.1.smithi173.stdout:1/743: dwrite d5/d6/d17/d42/da/df/d40/f4c [0,4194304] 0 2024-04-20T11:09:45.008 INFO:tasks.workunit.client.1.smithi173.stdout:0/789: creat d6/d8/df/d2c/d42/d50/d86/d8b/da2/daf/ff6 x:0 0 0 2024-04-20T11:09:45.016 INFO:tasks.workunit.client.1.smithi173.stdout:7/845: unlink d9/da/d17/d2b/c3f 0 2024-04-20T11:09:45.016 INFO:tasks.workunit.client.1.smithi173.stdout:2/914: rename d2/d13/d9a/dbc/dfd/d4b/l10f to d2/d34/d58/l12d 0 2024-04-20T11:09:45.016 INFO:tasks.workunit.client.1.smithi173.stdout:8/829: dread d4/d18/d4f/d5b/fc3 [0,4194304] 0 2024-04-20T11:09:45.017 INFO:tasks.workunit.client.1.smithi173.stdout:9/808: getdents d5/d9/d20/d29/d18/d9b 0 2024-04-20T11:09:45.020 INFO:tasks.workunit.client.1.smithi173.stdout:0/790: stat d6/d8/d51/fea 0 2024-04-20T11:09:45.020 INFO:tasks.workunit.client.1.smithi173.stdout:3/781: mkdir d7/d17/d58/df3 0 2024-04-20T11:09:45.020 INFO:tasks.workunit.client.1.smithi173.stdout:1/744: truncate d5/d6/d17/d93/f97 5287004 0 2024-04-20T11:09:45.020 INFO:tasks.workunit.client.1.smithi173.stdout:4/919: sync 2024-04-20T11:09:45.026 INFO:tasks.workunit.client.1.smithi173.stdout:2/915: read - d2/d13/d9a/dbc/dfd/d53/fbd zero size 2024-04-20T11:09:45.026 INFO:tasks.workunit.client.1.smithi173.stdout:3/782: write d7/d17/d61/f9b [3148368,4815] 0 2024-04-20T11:09:45.030 INFO:tasks.workunit.client.1.smithi173.stdout:5/835: truncate da/de/d75/d3e/f60 730872 0 2024-04-20T11:09:45.035 INFO:tasks.workunit.client.1.smithi173.stdout:1/745: symlink d5/d6/d17/d42/da/d20/dad/dc2/lfd 0 2024-04-20T11:09:45.038 INFO:tasks.workunit.client.1.smithi173.stdout:2/916: fsync d2/de/fa7 0 2024-04-20T11:09:45.039 INFO:tasks.workunit.client.1.smithi173.stdout:9/809: creat d5/d9/d20/d2b/d59/da1/deb/ffe x:0 0 0 2024-04-20T11:09:45.040 INFO:tasks.workunit.client.1.smithi173.stdout:1/746: dread - d5/d6/d17/d42/da/df/d40/dac/fca zero size 2024-04-20T11:09:45.042 INFO:tasks.workunit.client.1.smithi173.stdout:7/846: getdents d9/d33/d40/dc9 0 2024-04-20T11:09:45.045 INFO:tasks.workunit.client.1.smithi173.stdout:8/830: symlink d4/d18/d59/d5a/da8/db9/de0/d9b/dc8/d108/l109 0 2024-04-20T11:09:45.045 INFO:tasks.workunit.client.1.smithi173.stdout:0/791: getdents d6/d8/df/d2c/d42/d50/d86/d8b/da2/de8 0 2024-04-20T11:09:45.045 INFO:tasks.workunit.client.1.smithi173.stdout:6/807: dread d4/f11 [0,4194304] 0 2024-04-20T11:09:45.048 INFO:tasks.workunit.client.1.smithi173.stdout:4/920: link da/dc/d15/c2e da/d1e/d52/da3/c13b 0 2024-04-20T11:09:45.048 INFO:tasks.workunit.client.1.smithi173.stdout:8/831: fdatasync d4/d18/d59/d5a/da8/db9/de0/d3c/fe3 0 2024-04-20T11:09:45.048 INFO:tasks.workunit.client.1.smithi173.stdout:0/792: write d6/d8/d3c/d44/d75/f97 [426884,80675] 0 2024-04-20T11:09:45.049 INFO:tasks.workunit.client.1.smithi173.stdout:2/917: creat d2/d13/da5/db2/f12e x:0 0 0 2024-04-20T11:09:45.053 INFO:tasks.workunit.client.1.smithi173.stdout:5/836: rename da/d4f/d65/d8e/ce4 to da/de/d19/d81/c11d 0 2024-04-20T11:09:45.055 INFO:tasks.workunit.client.1.smithi173.stdout:5/837: chown da/de/d75/cb6 62 1 2024-04-20T11:09:45.061 INFO:tasks.workunit.client.1.smithi173.stdout:9/810: creat d5/d9/d20/d4f/d5e/d9e/fff x:0 0 0 2024-04-20T11:09:45.065 INFO:tasks.workunit.client.1.smithi173.stdout:6/808: symlink d4/d14/d5f/daf/lff 0 2024-04-20T11:09:45.066 INFO:tasks.workunit.client.1.smithi173.stdout:3/783: creat d7/d17/d61/d83/ff4 x:0 0 0 2024-04-20T11:09:45.070 INFO:tasks.workunit.client.1.smithi173.stdout:0/793: mknod d6/d8/df/dd0/cf7 0 2024-04-20T11:09:45.073 INFO:tasks.workunit.client.1.smithi173.stdout:4/921: rename da/d1e/d52/c78 to da/d1a/d6e/db4/c13c 0 2024-04-20T11:09:45.075 INFO:tasks.workunit.client.1.smithi173.stdout:9/811: creat d5/d9/d20/d29/d18/d9b/f100 x:0 0 0 2024-04-20T11:09:45.079 INFO:tasks.workunit.client.1.smithi173.stdout:7/847: dread d9/da/d17/f1a [0,4194304] 0 2024-04-20T11:09:45.081 INFO:tasks.workunit.client.1.smithi173.stdout:0/794: creat d6/d8/df/d2c/d2d/ff8 x:0 0 0 2024-04-20T11:09:45.083 INFO:tasks.workunit.client.1.smithi173.stdout:1/747: truncate d5/d6/d17/d93/f97 6242036 0 2024-04-20T11:09:45.084 INFO:tasks.workunit.client.1.smithi173.stdout:6/809: mknod d4/d24/c100 0 2024-04-20T11:09:45.087 INFO:tasks.workunit.client.1.smithi173.stdout:6/810: fsync d4/d1c/d8e/d90/da4/fd2 0 2024-04-20T11:09:45.090 INFO:tasks.workunit.client.1.smithi173.stdout:4/922: dwrite da/dc/d15/f2a [4194304,4194304] 0 2024-04-20T11:09:45.097 INFO:tasks.workunit.client.1.smithi173.stdout:5/838: dwrite da/de/d19/d32/f11 [0,4194304] 0 2024-04-20T11:09:45.097 INFO:tasks.workunit.client.1.smithi173.stdout:9/812: dwrite d5/d9/d20/d2b/fbe [0,4194304] 0 2024-04-20T11:09:45.099 INFO:tasks.workunit.client.1.smithi173.stdout:5/839: dread da/d4f/d65/fee [0,4194304] 0 2024-04-20T11:09:45.103 INFO:tasks.workunit.client.1.smithi173.stdout:3/784: write d7/f2d [3617706,55568] 0 2024-04-20T11:09:45.103 INFO:tasks.workunit.client.1.smithi173.stdout:6/811: fdatasync d4/d24/d3e/d9c/fec 0 2024-04-20T11:09:45.109 INFO:tasks.workunit.client.1.smithi173.stdout:6/812: chown d4/d14/d79/cd4 88028 1 2024-04-20T11:09:45.109 INFO:tasks.workunit.client.1.smithi173.stdout:8/832: getdents d4/d18/d4f/d5b/de4 0 2024-04-20T11:09:45.110 INFO:tasks.workunit.client.1.smithi173.stdout:0/795: write d6/d8/df/d43/f4c [2896754,120187] 0 2024-04-20T11:09:45.114 INFO:tasks.workunit.client.1.smithi173.stdout:0/796: stat d6/d8/df/d2c/d42/d50 0 2024-04-20T11:09:45.117 INFO:tasks.workunit.client.1.smithi173.stdout:1/748: fdatasync d5/d6/d17/d42/f90 0 2024-04-20T11:09:45.119 INFO:tasks.workunit.client.1.smithi173.stdout:9/813: symlink d5/d9/d20/d4f/d5e/d9e/l101 0 2024-04-20T11:09:45.119 INFO:tasks.workunit.client.1.smithi173.stdout:5/840: symlink da/d4f/d65/l11e 0 2024-04-20T11:09:45.124 INFO:tasks.workunit.client.1.smithi173.stdout:4/923: fdatasync f4 0 2024-04-20T11:09:45.125 INFO:tasks.workunit.client.1.smithi173.stdout:2/918: rename d2/d13/d28/d4a/c66 to d2/d13/d9a/dbc/dfd/d53/c12f 0 2024-04-20T11:09:45.128 INFO:tasks.workunit.client.1.smithi173.stdout:2/919: readlink d2/d8c/lcd 0 2024-04-20T11:09:45.129 INFO:tasks.workunit.client.1.smithi173.stdout:0/797: truncate d6/d8/df/d43/d7f/fab 725949 0 2024-04-20T11:09:45.132 INFO:tasks.workunit.client.1.smithi173.stdout:7/848: rename d9/d33/d40/dc9 to d9/da/d17/d2b/daa/d10b/d123 0 2024-04-20T11:09:45.134 INFO:tasks.workunit.client.1.smithi173.stdout:5/841: creat da/d34/d7f/da5/d9a/dc2/f11f x:0 0 0 2024-04-20T11:09:45.134 INFO:tasks.workunit.client.1.smithi173.stdout:2/920: chown d2/de/d11/l36 144763 1 2024-04-20T11:09:45.137 INFO:tasks.workunit.client.1.smithi173.stdout:6/813: dwrite d4/d18/fc4 [0,4194304] 0 2024-04-20T11:09:45.137 INFO:tasks.workunit.client.1.smithi173.stdout:1/749: fsync d5/d6/d17/d42/da/d20/dad/dc2/d94/fc1 0 2024-04-20T11:09:45.138 INFO:tasks.workunit.client.1.smithi173.stdout:4/924: rename da/d1a/fa0 to da/dd1/f13d 0 2024-04-20T11:09:45.142 INFO:tasks.workunit.client.1.smithi173.stdout:8/833: dwrite d4/d18/d59/d5a/fcb [0,4194304] 0 2024-04-20T11:09:45.142 INFO:tasks.workunit.client.1.smithi173.stdout:1/750: dread - d5/d6/dc/ffb zero size 2024-04-20T11:09:45.146 INFO:tasks.workunit.client.1.smithi173.stdout:1/751: write d5/d6/dc/fe0 [956181,122494] 0 2024-04-20T11:09:45.152 INFO:tasks.workunit.client.1.smithi173.stdout:3/785: write d7/d53/fc1 [1234269,4239] 0 2024-04-20T11:09:45.160 INFO:tasks.workunit.client.1.smithi173.stdout:7/849: dwrite d9/da/dc3/fff [4194304,4194304] 0 2024-04-20T11:09:45.164 INFO:tasks.workunit.client.1.smithi173.stdout:1/752: dread - d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fc0 zero size 2024-04-20T11:09:45.164 INFO:tasks.workunit.client.1.smithi173.stdout:3/786: rmdir d7/d17/d58/d8a 39 2024-04-20T11:09:45.164 INFO:tasks.workunit.client.1.smithi173.stdout:1/753: fdatasync d5/d6/d17/d42/d12/f47 0 2024-04-20T11:09:45.167 INFO:tasks.workunit.client.1.smithi173.stdout:8/834: mkdir d4/d18/d4f/d5b/d10a 0 2024-04-20T11:09:45.167 INFO:tasks.workunit.client.1.smithi173.stdout:5/842: creat da/de/d19/f120 x:0 0 0 2024-04-20T11:09:45.168 INFO:tasks.workunit.client.1.smithi173.stdout:3/787: dread d7/d17/d61/d83/fc8 [0,4194304] 0 2024-04-20T11:09:45.171 INFO:tasks.workunit.client.1.smithi173.stdout:9/814: dread d5/f2f [0,4194304] 0 2024-04-20T11:09:45.171 INFO:tasks.workunit.client.1.smithi173.stdout:7/850: rename d9/da/d17/d2b/daa/dd9 to d9/d33/d81/d124 0 2024-04-20T11:09:45.173 INFO:tasks.workunit.client.1.smithi173.stdout:9/815: dread - d5/d4c/f81 zero size 2024-04-20T11:09:45.175 INFO:tasks.workunit.client.1.smithi173.stdout:2/921: symlink d2/d13/d9a/dbc/dfd/d4b/dae/db6/l130 0 2024-04-20T11:09:45.177 INFO:tasks.workunit.client.1.smithi173.stdout:9/816: read d5/faf [1226648,53852] 0 2024-04-20T11:09:45.181 INFO:tasks.workunit.client.1.smithi173.stdout:6/814: write d4/d24/d42/d81/ff4 [1484828,80347] 0 2024-04-20T11:09:45.184 INFO:tasks.workunit.client.1.smithi173.stdout:7/851: fdatasync d9/d1f/d7f/dce/db2/fd8 0 2024-04-20T11:09:45.188 INFO:tasks.workunit.client.1.smithi173.stdout:9/817: chown d5/d9/d20/d29/d2d/le7 24128 1 2024-04-20T11:09:45.188 INFO:tasks.workunit.client.1.smithi173.stdout:0/798: dwrite d6/f13 [0,4194304] 0 2024-04-20T11:09:45.190 INFO:tasks.workunit.client.1.smithi173.stdout:7/852: chown d9/d33/d35/d4a/def 220 1 2024-04-20T11:09:45.196 INFO:tasks.workunit.client.1.smithi173.stdout:0/799: fsync d6/d8/df/d15/ff4 0 2024-04-20T11:09:45.197 INFO:tasks.workunit.client.1.smithi173.stdout:4/925: dwrite da/d1e/d52/d7a/fe2 [0,4194304] 0 2024-04-20T11:09:45.198 INFO:tasks.workunit.client.1.smithi173.stdout:3/788: mkdir d7/d85/df5 0 2024-04-20T11:09:45.200 INFO:tasks.workunit.client.1.smithi173.stdout:1/754: dwrite d5/d6/dc/f3a [4194304,4194304] 0 2024-04-20T11:09:45.202 INFO:tasks.workunit.client.1.smithi173.stdout:6/815: symlink d4/d24/d3e/l101 0 2024-04-20T11:09:45.205 INFO:tasks.workunit.client.1.smithi173.stdout:5/843: creat da/de/f121 x:0 0 0 2024-04-20T11:09:45.208 INFO:tasks.workunit.client.1.smithi173.stdout:3/789: write d7/d17/d58/fb2 [2249737,94602] 0 2024-04-20T11:09:45.209 INFO:tasks.workunit.client.1.smithi173.stdout:6/816: rmdir d4/d18/d7e 39 2024-04-20T11:09:45.211 INFO:tasks.workunit.client.1.smithi173.stdout:0/800: write d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/db9/fe4 [8308,75110] 0 2024-04-20T11:09:45.212 INFO:tasks.workunit.client.1.smithi173.stdout:2/922: link d2/d13/f44 d2/de/d11/d9d/f131 0 2024-04-20T11:09:45.212 INFO:tasks.workunit.client.1.smithi173.stdout:8/835: truncate d4/d18/d59/d5a/da8/db9/de0/f7a 2664470 0 2024-04-20T11:09:45.213 INFO:tasks.workunit.client.1.smithi173.stdout:3/790: chown d7/d17/d61/d90/dca/cdd 0 1 2024-04-20T11:09:45.214 INFO:tasks.workunit.client.1.smithi173.stdout:2/923: read d2/d13/d28/d4a/f6c [78925,116072] 0 2024-04-20T11:09:45.217 INFO:tasks.workunit.client.1.smithi173.stdout:0/801: truncate d6/d8/df/d15/f95 4539931 0 2024-04-20T11:09:45.218 INFO:tasks.workunit.client.1.smithi173.stdout:1/755: creat d5/d6/d17/d42/d12/d1c/d85/ffe x:0 0 0 2024-04-20T11:09:45.219 INFO:tasks.workunit.client.1.smithi173.stdout:5/844: rename da/de/d19/d32/d30/c39 to da/de/d19/dca/d48/c122 0 2024-04-20T11:09:45.219 INFO:tasks.workunit.client.1.smithi173.stdout:0/802: fsync d6/d8/df/f16 0 2024-04-20T11:09:45.220 INFO:tasks.workunit.client.1.smithi173.stdout:6/817: write d4/d14/f6a [1764300,32191] 0 2024-04-20T11:09:45.226 INFO:tasks.workunit.client.1.smithi173.stdout:9/818: truncate d5/d9/d20/d2b/d59/d44/d78/d9d/fb7 373759 0 2024-04-20T11:09:45.226 INFO:tasks.workunit.client.1.smithi173.stdout:8/836: dread d4/fbb [0,4194304] 0 2024-04-20T11:09:45.227 INFO:tasks.workunit.client.1.smithi173.stdout:4/926: getdents da/d1e/d52/d10d 0 2024-04-20T11:09:45.230 INFO:tasks.workunit.client.1.smithi173.stdout:7/853: write d9/d33/d40/d7a/ff0 [113832,127938] 0 2024-04-20T11:09:45.231 INFO:tasks.workunit.client.1.smithi173.stdout:5/845: fdatasync da/de/d75/fab 0 2024-04-20T11:09:45.234 INFO:tasks.workunit.client.1.smithi173.stdout:3/791: getdents d7/d17/d58/df3 0 2024-04-20T11:09:45.235 INFO:tasks.workunit.client.1.smithi173.stdout:6/818: stat d4/d18/d7e/fbe 0 2024-04-20T11:09:45.240 INFO:tasks.workunit.client.1.smithi173.stdout:0/803: symlink d6/d8/df/d2c/d42/d50/de3/lf9 0 2024-04-20T11:09:45.242 INFO:tasks.workunit.client.1.smithi173.stdout:1/756: mknod d5/d6/d17/d42/da/d20/d38/d52/d75/cff 0 2024-04-20T11:09:45.249 INFO:tasks.workunit.client.1.smithi173.stdout:8/837: dwrite d4/d18/d59/d5a/d7b/ddc/ff2 [0,4194304] 0 2024-04-20T11:09:45.250 INFO:tasks.workunit.client.1.smithi173.stdout:8/838: write d4/d18/d59/fc7 [1013601,100460] 0 2024-04-20T11:09:45.256 INFO:tasks.workunit.client.1.smithi173.stdout:6/819: creat d4/d1c/d8e/f102 x:0 0 0 2024-04-20T11:09:45.258 INFO:tasks.workunit.client.1.smithi173.stdout:0/804: unlink d6/d8/d3c/d44/f46 0 2024-04-20T11:09:45.261 INFO:tasks.workunit.client.1.smithi173.stdout:9/819: rename d5/d9/d20/d29/d18/fa2 to d5/dd7/f102 0 2024-04-20T11:09:45.262 INFO:tasks.workunit.client.1.smithi173.stdout:2/924: dread d2/fd8 [0,4194304] 0 2024-04-20T11:09:45.262 INFO:tasks.workunit.client.1.smithi173.stdout:0/805: readlink d6/d8/df/laa 0 2024-04-20T11:09:45.263 INFO:tasks.workunit.client.1.smithi173.stdout:9/820: stat d5/d9/d20/d4f/l66 0 2024-04-20T11:09:45.264 INFO:tasks.workunit.client.1.smithi173.stdout:8/839: creat d4/d18/d59/d5a/d7b/db1/f10b x:0 0 0 2024-04-20T11:09:45.264 INFO:tasks.workunit.client.1.smithi173.stdout:3/792: symlink d7/d17/d61/d83/lf6 0 2024-04-20T11:09:45.265 INFO:tasks.workunit.client.1.smithi173.stdout:5/846: truncate da/de/d75/d3e/f60 1222890 0 2024-04-20T11:09:45.265 INFO:tasks.workunit.client.1.smithi173.stdout:6/820: readlink d4/d24/d42/d74/l4f 0 2024-04-20T11:09:45.269 INFO:tasks.workunit.client.1.smithi173.stdout:1/757: getdents d5/d6/d17/d93 0 2024-04-20T11:09:45.273 INFO:tasks.workunit.client.1.smithi173.stdout:5/847: write da/d34/d7f/da4/f116 [777397,10899] 0 2024-04-20T11:09:45.274 INFO:tasks.workunit.client.1.smithi173.stdout:3/793: mknod d7/d85/df5/cf7 0 2024-04-20T11:09:45.276 INFO:tasks.workunit.client.1.smithi173.stdout:4/927: dwrite da/d1a/d60/d7d/fc5 [0,4194304] 0 2024-04-20T11:09:45.278 INFO:tasks.workunit.client.1.smithi173.stdout:6/821: creat d4/d24/d3e/d9c/f103 x:0 0 0 2024-04-20T11:09:45.278 INFO:tasks.workunit.client.1.smithi173.stdout:7/854: dread d9/d1f/f7b [0,4194304] 0 2024-04-20T11:09:45.279 INFO:tasks.workunit.client.1.smithi173.stdout:5/848: stat da/de/d75/d40/d59/f96 0 2024-04-20T11:09:45.282 INFO:tasks.workunit.client.1.smithi173.stdout:8/840: truncate d4/d18/d59/d5a/da8/db9/de0/d3c/fe3 257304 0 2024-04-20T11:09:45.282 INFO:tasks.workunit.client.1.smithi173.stdout:6/822: stat d4/d24/d42/d81/ff4 0 2024-04-20T11:09:45.285 INFO:tasks.workunit.client.1.smithi173.stdout:0/806: truncate d6/d8/d3c/f89 4264068 0 2024-04-20T11:09:45.285 INFO:tasks.workunit.client.1.smithi173.stdout:2/925: write d2/d34/fdd [46148,78783] 0 2024-04-20T11:09:45.289 INFO:tasks.workunit.client.1.smithi173.stdout:8/841: fdatasync d4/d18/d59/d5a/da8/db9/de0/fd6 0 2024-04-20T11:09:45.294 INFO:tasks.workunit.client.1.smithi173.stdout:0/807: dread - d6/d8/d51/fea zero size 2024-04-20T11:09:45.294 INFO:tasks.workunit.client.1.smithi173.stdout:3/794: creat d7/d17/d61/ff8 x:0 0 0 2024-04-20T11:09:45.295 INFO:tasks.workunit.client.1.smithi173.stdout:8/842: fdatasync d4/f41 0 2024-04-20T11:09:45.297 INFO:tasks.workunit.client.1.smithi173.stdout:1/758: write d5/d6/d17/d42/da/d20/d48/f9c [275540,80360] 0 2024-04-20T11:09:45.300 INFO:tasks.workunit.client.1.smithi173.stdout:7/855: write d9/d33/d35/d4a/f93 [384722,47844] 0 2024-04-20T11:09:45.301 INFO:tasks.workunit.client.1.smithi173.stdout:6/823: write d4/d24/fa9 [776911,99023] 0 2024-04-20T11:09:45.303 INFO:tasks.workunit.client.1.smithi173.stdout:9/821: dwrite d5/d9/d20/d29/d18/d9b/fb3 [0,4194304] 0 2024-04-20T11:09:45.304 INFO:tasks.workunit.client.1.smithi173.stdout:5/849: rename da/lb to da/d9e/l123 0 2024-04-20T11:09:45.308 INFO:tasks.workunit.client.1.smithi173.stdout:9/822: dread - d5/d9/d20/d29/d2d/d33/fda zero size 2024-04-20T11:09:45.310 INFO:tasks.workunit.client.1.smithi173.stdout:6/824: symlink d4/de3/l104 0 2024-04-20T11:09:45.311 INFO:tasks.workunit.client.1.smithi173.stdout:5/850: unlink da/d4f/d65/f10a 0 2024-04-20T11:09:45.313 INFO:tasks.workunit.client.1.smithi173.stdout:3/795: mknod d7/d17/d61/d83/d45/cf9 0 2024-04-20T11:09:45.314 INFO:tasks.workunit.client.1.smithi173.stdout:9/823: chown d5/d9/d20/d29/d2d/f43 11 1 2024-04-20T11:09:45.315 INFO:tasks.workunit.client.1.smithi173.stdout:6/825: rmdir d4/d59/d8f 39 2024-04-20T11:09:45.316 INFO:tasks.workunit.client.1.smithi173.stdout:1/759: read d5/d6/d17/d42/d12/d1f/f63 [378749,109791] 0 2024-04-20T11:09:45.317 INFO:tasks.workunit.client.1.smithi173.stdout:0/808: creat d6/d8/d3c/d44/ffa x:0 0 0 2024-04-20T11:09:45.318 INFO:tasks.workunit.client.1.smithi173.stdout:4/928: dread da/f25 [0,4194304] 0 2024-04-20T11:09:45.321 INFO:tasks.workunit.client.1.smithi173.stdout:0/809: chown d6/fa4 42570903 1 2024-04-20T11:09:45.321 INFO:tasks.workunit.client.1.smithi173.stdout:1/760: write d5/d6/dc/f3a [4956912,104760] 0 2024-04-20T11:09:45.328 INFO:tasks.workunit.client.1.smithi173.stdout:7/856: getdents d9/d33/d85 0 2024-04-20T11:09:45.328 INFO:tasks.workunit.client.1.smithi173.stdout:8/843: truncate d4/d18/f24 3419006 0 2024-04-20T11:09:45.329 INFO:tasks.workunit.client.1.smithi173.stdout:4/929: dread - da/dc/d15/fcd zero size 2024-04-20T11:09:45.330 INFO:tasks.workunit.client.1.smithi173.stdout:8/844: chown d4/cc0 115 1 2024-04-20T11:09:45.331 INFO:tasks.workunit.client.1.smithi173.stdout:5/851: mkdir da/de/d19/d2c/d106/d124 0 2024-04-20T11:09:45.331 INFO:tasks.workunit.client.1.smithi173.stdout:0/810: mknod d6/d5f/cfb 0 2024-04-20T11:09:45.332 INFO:tasks.workunit.client.1.smithi173.stdout:8/845: chown d4/f19 300607 1 2024-04-20T11:09:45.332 INFO:tasks.workunit.client.1.smithi173.stdout:6/826: unlink d4/d24/d3e/d9c/dc7/lea 0 2024-04-20T11:09:45.337 INFO:tasks.workunit.client.1.smithi173.stdout:3/796: dwrite d7/d2b/d8e/fb3 [0,4194304] 0 2024-04-20T11:09:45.337 INFO:tasks.workunit.client.1.smithi173.stdout:5/852: write da/d34/d7f/da4/fb0 [556127,125595] 0 2024-04-20T11:09:45.344 INFO:tasks.workunit.client.1.smithi173.stdout:9/824: getdents d5/d9/d20/d2b 0 2024-04-20T11:09:45.350 INFO:tasks.workunit.client.1.smithi173.stdout:8/846: mknod d4/d18/d59/d5a/da8/db9/de0/d3c/c10c 0 2024-04-20T11:09:45.351 INFO:tasks.workunit.client.1.smithi173.stdout:3/797: symlink d7/lfa 0 2024-04-20T11:09:45.354 INFO:tasks.workunit.client.1.smithi173.stdout:6/827: creat d4/d59/f105 x:0 0 0 2024-04-20T11:09:45.355 INFO:tasks.workunit.client.1.smithi173.stdout:8/847: truncate d4/d18/da6/fbf 505896 0 2024-04-20T11:09:45.356 INFO:tasks.workunit.client.1.smithi173.stdout:0/811: dread d6/d8/df/d43/fc0 [0,4194304] 0 2024-04-20T11:09:45.357 INFO:tasks.workunit.client.1.smithi173.stdout:4/930: write da/dd1/df5/f118 [1282137,63739] 0 2024-04-20T11:09:45.361 INFO:tasks.workunit.client.1.smithi173.stdout:4/931: chown da/d1e/d52/d9a/dd3/def/ffe 4 1 2024-04-20T11:09:45.362 INFO:tasks.workunit.client.1.smithi173.stdout:7/857: dread d9/d33/f3a [0,4194304] 0 2024-04-20T11:09:45.363 INFO:tasks.workunit.client.1.smithi173.stdout:4/932: dread - da/dc/df4/fa2 zero size 2024-04-20T11:09:45.364 INFO:tasks.workunit.client.1.smithi173.stdout:6/828: creat d4/d1c/d8e/d90/da4/f106 x:0 0 0 2024-04-20T11:09:45.365 INFO:tasks.workunit.client.1.smithi173.stdout:8/848: symlink d4/d18/d59/d5a/l10d 0 2024-04-20T11:09:45.368 INFO:tasks.workunit.client.1.smithi173.stdout:5/853: truncate da/de/d19/f5c 463982 0 2024-04-20T11:09:45.369 INFO:tasks.workunit.client.1.smithi173.stdout:9/825: link d5/d9/d20/d2b/d31/l67 d5/d9/d20/d4f/d5e/d7d/d95/l103 0 2024-04-20T11:09:45.430 INFO:tasks.workunit.client.1.smithi173.stdout:7/858: unlink d9/c67 0 2024-04-20T11:09:45.431 INFO:tasks.workunit.client.1.smithi173.stdout:8/849: write d4/d18/fcf [962581,102598] 0 2024-04-20T11:09:45.432 INFO:tasks.workunit.client.1.smithi173.stdout:6/829: mkdir d4/d59/d8f/d107 0 2024-04-20T11:09:45.434 INFO:tasks.workunit.client.1.smithi173.stdout:5/854: unlink da/de/d19/d2c/f94 0 2024-04-20T11:09:45.434 INFO:tasks.workunit.client.1.smithi173.stdout:3/798: dwrite d7/d17/d61/d83/dcf/d5c/fe7 [0,4194304] 0 2024-04-20T11:09:45.439 INFO:tasks.workunit.client.1.smithi173.stdout:0/812: dwrite d6/d8/df/d15/ff4 [0,4194304] 0 2024-04-20T11:09:45.439 INFO:tasks.workunit.client.1.smithi173.stdout:6/830: readlink d4/d14/d5f/daf/lc5 0 2024-04-20T11:09:45.446 INFO:tasks.workunit.client.1.smithi173.stdout:7/859: creat d9/d33/d35/d4a/def/f125 x:0 0 0 2024-04-20T11:09:45.446 INFO:tasks.workunit.client.1.smithi173.stdout:3/799: write d7/d53/f8f [8982563,64927] 0 2024-04-20T11:09:45.450 INFO:tasks.workunit.client.1.smithi173.stdout:5/855: readlink da/de/d19/d32/d30/l5a 0 2024-04-20T11:09:45.456 INFO:tasks.workunit.client.1.smithi173.stdout:5/856: creat da/de/d19/d81/d92/f125 x:0 0 0 2024-04-20T11:09:45.464 INFO:tasks.workunit.client.1.smithi173.stdout:3/800: link d7/d2b/fbd d7/d53/dab/ffb 0 2024-04-20T11:09:45.466 INFO:tasks.workunit.client.1.smithi173.stdout:9/826: dread d5/d4c/f72 [0,4194304] 0 2024-04-20T11:09:45.468 INFO:tasks.workunit.client.1.smithi173.stdout:5/857: dread da/de/d19/d81/d92/ff8 [0,4194304] 0 2024-04-20T11:09:45.468 INFO:tasks.workunit.client.1.smithi173.stdout:8/850: dwrite d4/d18/d59/d5a/da8/db9/de0/fd6 [0,4194304] 0 2024-04-20T11:09:45.469 INFO:tasks.workunit.client.1.smithi173.stdout:0/813: dwrite d6/d8/f12 [0,4194304] 0 2024-04-20T11:09:45.473 INFO:tasks.workunit.client.1.smithi173.stdout:3/801: creat d7/d17/d61/dd6/ffc x:0 0 0 2024-04-20T11:09:45.475 INFO:tasks.workunit.client.1.smithi173.stdout:8/851: fsync d4/d18/d59/d5a/f67 0 2024-04-20T11:09:45.475 INFO:tasks.workunit.client.1.smithi173.stdout:9/827: truncate d5/d9/d20/d2b/d59/fd8 5220915 0 2024-04-20T11:09:45.478 INFO:tasks.workunit.client.1.smithi173.stdout:3/802: fsync d7/d17/d61/d83/dcf/d30/fc7 0 2024-04-20T11:09:45.482 INFO:tasks.workunit.client.1.smithi173.stdout:2/926: sync 2024-04-20T11:09:45.484 INFO:tasks.workunit.client.1.smithi173.stdout:7/860: dread f6 [0,4194304] 0 2024-04-20T11:09:45.485 INFO:tasks.workunit.client.1.smithi173.stdout:8/852: stat d4/ff 0 2024-04-20T11:09:45.486 INFO:tasks.workunit.client.1.smithi173.stdout:2/927: chown d2/d34/fc8 467 1 2024-04-20T11:09:45.490 INFO:tasks.workunit.client.1.smithi173.stdout:3/803: unlink d7/f22 0 2024-04-20T11:09:45.494 INFO:tasks.workunit.client.1.smithi173.stdout:3/804: chown d7/d17/d58/d7f/df2 6546 1 2024-04-20T11:09:45.501 INFO:tasks.workunit.client.1.smithi173.stdout:0/814: getdents d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb 0 2024-04-20T11:09:45.505 INFO:tasks.workunit.client.1.smithi173.stdout:2/928: link d2/f5c d2/d8c/f132 0 2024-04-20T11:09:45.506 INFO:tasks.workunit.client.1.smithi173.stdout:6/831: dwrite d4/d59/d8f/fa3 [0,4194304] 0 2024-04-20T11:09:45.510 INFO:tasks.workunit.client.1.smithi173.stdout:9/828: dwrite d5/d9/d20/d2b/f49 [4194304,4194304] 0 2024-04-20T11:09:45.515 INFO:tasks.workunit.client.1.smithi173.stdout:3/805: creat d7/d2b/ffd x:0 0 0 2024-04-20T11:09:45.516 INFO:tasks.workunit.client.1.smithi173.stdout:8/853: dwrite d4/d18/d59/fc7 [0,4194304] 0 2024-04-20T11:09:45.518 INFO:tasks.workunit.client.1.smithi173.stdout:0/815: unlink d6/d8/df/d2c/lb7 0 2024-04-20T11:09:45.520 INFO:tasks.workunit.client.1.smithi173.stdout:6/832: unlink d4/d14/f8c 0 2024-04-20T11:09:45.521 INFO:tasks.workunit.client.1.smithi173.stdout:5/858: dwrite da/de/fac [0,4194304] 0 2024-04-20T11:09:45.523 INFO:tasks.workunit.client.1.smithi173.stdout:0/816: chown d6/d8/d51/f71 530558 1 2024-04-20T11:09:45.528 INFO:tasks.workunit.client.1.smithi173.stdout:0/817: stat d6/d8/df/d43/f59 0 2024-04-20T11:09:45.532 INFO:tasks.workunit.client.1.smithi173.stdout:2/929: creat d2/de/f133 x:0 0 0 2024-04-20T11:09:45.532 INFO:tasks.workunit.client.1.smithi173.stdout:9/829: mknod d5/d4c/d79/da7/c104 0 2024-04-20T11:09:45.535 INFO:tasks.workunit.client.1.smithi173.stdout:3/806: creat d7/d17/d58/d8a/dc4/ffe x:0 0 0 2024-04-20T11:09:45.537 INFO:tasks.workunit.client.1.smithi173.stdout:5/859: rmdir da/de/d19/d32/d30/dbb 39 2024-04-20T11:09:45.538 INFO:tasks.workunit.client.1.smithi173.stdout:0/818: creat d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/ffc x:0 0 0 2024-04-20T11:09:45.539 INFO:tasks.workunit.client.1.smithi173.stdout:7/861: dread d9/d1f/f7d [4194304,4194304] 0 2024-04-20T11:09:45.542 INFO:tasks.workunit.client.1.smithi173.stdout:2/930: symlink d2/d13/d9a/dbc/dfd/d53/d9f/l134 0 2024-04-20T11:09:45.543 INFO:tasks.workunit.client.1.smithi173.stdout:8/854: getdents d4 0 2024-04-20T11:09:45.543 INFO:tasks.workunit.client.1.smithi173.stdout:9/830: creat d5/d9/d20/d4f/d5e/d7d/d95/f105 x:0 0 0 2024-04-20T11:09:45.544 INFO:tasks.workunit.client.1.smithi173.stdout:5/860: creat da/de/d19/d32/f126 x:0 0 0 2024-04-20T11:09:45.546 INFO:tasks.workunit.client.1.smithi173.stdout:7/862: dread - d9/d1f/f65 zero size 2024-04-20T11:09:45.546 INFO:tasks.workunit.client.1.smithi173.stdout:3/807: mkdir d7/d17/d58/dff 0 2024-04-20T11:09:45.547 INFO:tasks.workunit.client.1.smithi173.stdout:5/861: stat da/d4f/d65 0 2024-04-20T11:09:45.547 INFO:tasks.workunit.client.1.smithi173.stdout:2/931: fdatasync d2/d34/d81/f9e 0 2024-04-20T11:09:45.550 INFO:tasks.workunit.client.1.smithi173.stdout:0/819: dread d6/d8/df/d2c/d42/fa3 [0,4194304] 0 2024-04-20T11:09:45.552 INFO:tasks.workunit.client.1.smithi173.stdout:5/862: symlink da/de/d19/d81/l127 0 2024-04-20T11:09:45.555 INFO:tasks.workunit.client.1.smithi173.stdout:8/855: creat d4/d18/d4f/f10e x:0 0 0 2024-04-20T11:09:45.556 INFO:tasks.workunit.client.1.smithi173.stdout:1/761: sync 2024-04-20T11:09:45.556 INFO:tasks.workunit.client.1.smithi173.stdout:2/932: rmdir d2/d13/da5/db2 39 2024-04-20T11:09:45.559 INFO:tasks.workunit.client.1.smithi173.stdout:2/933: dread - d2/d34/ff8 zero size 2024-04-20T11:09:45.560 INFO:tasks.workunit.client.1.smithi173.stdout:1/762: dread - d5/d6/d17/d42/ff2 zero size 2024-04-20T11:09:45.562 INFO:tasks.workunit.client.1.smithi173.stdout:5/863: mkdir da/de/d75/d128 0 2024-04-20T11:09:45.563 INFO:tasks.workunit.client.1.smithi173.stdout:6/833: write d4/d24/d42/d74/d3b/fb5 [1240397,101841] 0 2024-04-20T11:09:45.566 INFO:tasks.workunit.client.1.smithi173.stdout:2/934: rmdir d2/de/d11/d9d 39 2024-04-20T11:09:45.568 INFO:tasks.workunit.client.1.smithi173.stdout:9/831: write d5/d4c/f81 [721914,26586] 0 2024-04-20T11:09:45.568 INFO:tasks.workunit.client.1.smithi173.stdout:7/863: dwrite d9/d33/d85/fc5 [4194304,4194304] 0 2024-04-20T11:09:45.570 INFO:tasks.workunit.client.1.smithi173.stdout:3/808: write d7/d17/d61/f97 [122762,109427] 0 2024-04-20T11:09:45.571 INFO:tasks.workunit.client.1.smithi173.stdout:1/763: truncate d5/d6/d17/d42/f39 2023273 0 2024-04-20T11:09:45.572 INFO:tasks.workunit.client.1.smithi173.stdout:3/809: chown d7/d2b/f62 1060 1 2024-04-20T11:09:45.574 INFO:tasks.workunit.client.1.smithi173.stdout:8/856: link d4/d18/d4f/d5b/f87 d4/d18/d59/d5a/da8/db9/de0/d3c/d86/dfc/f10f 0 2024-04-20T11:09:45.580 INFO:tasks.workunit.client.1.smithi173.stdout:2/935: read - d2/d13/da5/db2/f12e zero size 2024-04-20T11:09:45.581 INFO:tasks.workunit.client.1.smithi173.stdout:9/832: write d5/d9/d20/d29/f11 [3086205,68591] 0 2024-04-20T11:09:45.587 INFO:tasks.workunit.client.1.smithi173.stdout:8/857: chown d4/d18/d59/d5a/da8/db9/de0/lbd 560414 1 2024-04-20T11:09:45.587 INFO:tasks.workunit.client.1.smithi173.stdout:1/764: fdatasync d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fdb 0 2024-04-20T11:09:45.588 INFO:tasks.workunit.client.1.smithi173.stdout:0/820: dwrite d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/fcd [0,4194304] 0 2024-04-20T11:09:45.590 INFO:tasks.workunit.client.1.smithi173.stdout:7/864: symlink d9/d33/d40/l126 0 2024-04-20T11:09:45.592 INFO:tasks.workunit.client.1.smithi173.stdout:9/833: fsync d5/f2f 0 2024-04-20T11:09:45.595 INFO:tasks.workunit.client.1.smithi173.stdout:4/933: sync 2024-04-20T11:09:45.598 INFO:tasks.workunit.client.1.smithi173.stdout:5/864: link da/de/le7 da/de/d19/d2c/l129 0 2024-04-20T11:09:45.603 INFO:tasks.workunit.client.1.smithi173.stdout:0/821: creat d6/d8/df/d2c/d42/d50/d86/d8b/da2/daf/ffd x:0 0 0 2024-04-20T11:09:45.604 INFO:tasks.workunit.client.1.smithi173.stdout:7/865: write d9/d33/f48 [3270746,87700] 0 2024-04-20T11:09:45.606 INFO:tasks.workunit.client.1.smithi173.stdout:4/934: creat da/dc/d15/d8b/f13e x:0 0 0 2024-04-20T11:09:45.609 INFO:tasks.workunit.client.1.smithi173.stdout:0/822: unlink d6/d8/df/d15/f95 0 2024-04-20T11:09:45.611 INFO:tasks.workunit.client.1.smithi173.stdout:4/935: fsync da/d1a/d6e/db4/d132/f125 0 2024-04-20T11:09:45.613 INFO:tasks.workunit.client.1.smithi173.stdout:5/865: rename da/de/d19/d2c/f79 to da/d34/f12a 0 2024-04-20T11:09:45.614 INFO:tasks.workunit.client.1.smithi173.stdout:9/834: truncate d5/d9/d20/d4f/d5e/d7d/d95/f105 334147 0 2024-04-20T11:09:45.618 INFO:tasks.workunit.client.1.smithi173.stdout:1/765: write d5/d6/d17/d42/f39 [2456186,103096] 0 2024-04-20T11:09:45.619 INFO:tasks.workunit.client.1.smithi173.stdout:3/810: dwrite d7/d17/d58/d8a/f9a [4194304,4194304] 0 2024-04-20T11:09:45.620 INFO:tasks.workunit.client.1.smithi173.stdout:5/866: chown da/d34/d7f/da5/db5/l114 43 1 2024-04-20T11:09:45.621 INFO:tasks.workunit.client.1.smithi173.stdout:2/936: write d2/d13/d9a/dbc/dfd/f9c [1078370,84769] 0 2024-04-20T11:09:45.628 INFO:tasks.workunit.client.1.smithi173.stdout:3/811: dread - d7/d17/d61/d83/dcf/d5c/fdb zero size 2024-04-20T11:09:45.628 INFO:tasks.workunit.client.1.smithi173.stdout:0/823: symlink d6/d8/df/d2c/d42/d50/d86/d8b/da2/lfe 0 2024-04-20T11:09:45.629 INFO:tasks.workunit.client.1.smithi173.stdout:9/835: unlink d5/d9/d20/d4f/d5e/d7d/dba/cea 0 2024-04-20T11:09:45.630 INFO:tasks.workunit.client.1.smithi173.stdout:1/766: mknod d5/d6/d17/d42/da/d20/dad/dc2/d94/c100 0 2024-04-20T11:09:45.632 INFO:tasks.workunit.client.1.smithi173.stdout:2/937: symlink d2/d13/d9a/dbc/dfd/d53/d4e/l135 0 2024-04-20T11:09:45.633 INFO:tasks.workunit.client.1.smithi173.stdout:7/866: dread d9/d1f/f7b [0,4194304] 0 2024-04-20T11:09:45.637 INFO:tasks.workunit.client.1.smithi173.stdout:3/812: stat d7/d17/d61/d83/dcf/cd4 0 2024-04-20T11:09:45.639 INFO:tasks.workunit.client.1.smithi173.stdout:9/836: chown d5/d9/d20/d29/d2d/d33/l3a 691778 1 2024-04-20T11:09:45.641 INFO:tasks.workunit.client.1.smithi173.stdout:8/858: dread d4/d104/fba [0,4194304] 0 2024-04-20T11:09:45.644 INFO:tasks.workunit.client.1.smithi173.stdout:9/837: write d5/d9/d20/d2b/d31/f92 [2658419,51149] 0 2024-04-20T11:09:45.646 INFO:tasks.workunit.client.1.smithi173.stdout:7/867: unlink d9/d33/d35/d56/c88 0 2024-04-20T11:09:45.649 INFO:tasks.workunit.client.1.smithi173.stdout:4/936: dwrite da/dc/df4/d53/fea [0,4194304] 0 2024-04-20T11:09:45.650 INFO:tasks.workunit.client.1.smithi173.stdout:8/859: rmdir d4/d18/d59/d5a/da8/db9/de0/d3c/d86/dfc 39 2024-04-20T11:09:45.652 INFO:tasks.workunit.client.1.smithi173.stdout:9/838: truncate d5/dcc/ffa 298365 0 2024-04-20T11:09:45.658 INFO:tasks.workunit.client.1.smithi173.stdout:9/839: chown d5/d9/d20/d29 86340440 1 2024-04-20T11:09:45.661 INFO:tasks.workunit.client.1.smithi173.stdout:3/813: read d7/d17/f9f [3389560,53218] 0 2024-04-20T11:09:45.663 INFO:tasks.workunit.client.1.smithi173.stdout:8/860: chown d4/d18/d59/d5a/d7b/ld8 16 1 2024-04-20T11:09:45.666 INFO:tasks.workunit.client.1.smithi173.stdout:0/824: write d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/ff0 [4429914,26982] 0 2024-04-20T11:09:45.666 INFO:tasks.workunit.client.1.smithi173.stdout:2/938: dwrite d2/d13/d9a/dbc/dfd/f9c [0,4194304] 0 2024-04-20T11:09:45.671 INFO:tasks.workunit.client.1.smithi173.stdout:3/814: write d7/d17/f26 [2525117,62144] 0 2024-04-20T11:09:45.674 INFO:tasks.workunit.client.1.smithi173.stdout:6/834: sync 2024-04-20T11:09:45.674 INFO:tasks.workunit.client.1.smithi173.stdout:9/840: rename d5/d4c to d5/d9/d20/d2b/d106 0 2024-04-20T11:09:45.677 INFO:tasks.workunit.client.1.smithi173.stdout:2/939: fsync d2/de/f21 0 2024-04-20T11:09:45.681 INFO:tasks.workunit.client.1.smithi173.stdout:3/815: rmdir d7/d53/dab 39 2024-04-20T11:09:45.686 INFO:tasks.workunit.client.1.smithi173.stdout:8/861: fdatasync d4/d18/d59/d5a/da8/db9/de0/f7a 0 2024-04-20T11:09:45.690 INFO:tasks.workunit.client.1.smithi173.stdout:2/940: creat d2/d8c/f136 x:0 0 0 2024-04-20T11:09:45.694 INFO:tasks.workunit.client.1.smithi173.stdout:9/841: unlink d5/d9/d20/d2b/d31/cad 0 2024-04-20T11:09:45.701 INFO:tasks.workunit.client.1.smithi173.stdout:5/867: dread da/de/d19/dca/d48/da2/ff6 [0,4194304] 0 2024-04-20T11:09:45.709 INFO:tasks.workunit.client.1.smithi173.stdout:1/767: dread d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fdb [0,4194304] 0 2024-04-20T11:09:45.711 INFO:tasks.workunit.client.1.smithi173.stdout:8/862: mkdir d4/d18/d59/d5a/da8/db9/de0/d3c/d86/db7/d110 0 2024-04-20T11:09:45.713 INFO:tasks.workunit.client.1.smithi173.stdout:7/868: write d9/da/d17/d2b/f74 [178837,96710] 0 2024-04-20T11:09:45.720 INFO:tasks.workunit.client.1.smithi173.stdout:2/941: rename d2/de/d11/d9d/c8b to d2/d13/d9a/dbc/dfd/d7d/c137 0 2024-04-20T11:09:45.721 INFO:tasks.workunit.client.1.smithi173.stdout:4/937: dwrite da/dc/d15/fcd [0,4194304] 0 2024-04-20T11:09:45.725 INFO:tasks.workunit.client.1.smithi173.stdout:8/863: read - d4/d18/d59/d5a/da8/db9/de0/d3c/fa0 zero size 2024-04-20T11:09:45.732 INFO:tasks.workunit.client.1.smithi173.stdout:6/835: dwrite d4/d24/d42/d74/d3b/fa8 [0,4194304] 0 2024-04-20T11:09:45.732 INFO:tasks.workunit.client.1.smithi173.stdout:1/768: rmdir d5/d6/d17/d42/da/d20 39 2024-04-20T11:09:45.737 INFO:tasks.workunit.client.1.smithi173.stdout:2/942: symlink d2/de/d11/db8/l138 0 2024-04-20T11:09:45.741 INFO:tasks.workunit.client.1.smithi173.stdout:4/938: fsync da/f25 0 2024-04-20T11:09:45.742 INFO:tasks.workunit.client.1.smithi173.stdout:7/869: mkdir d9/da/d127 0 2024-04-20T11:09:45.746 INFO:tasks.workunit.client.1.smithi173.stdout:9/842: getdents d5/d9/d20/d2b/d59/d44/d78/dd0 0 2024-04-20T11:09:45.749 INFO:tasks.workunit.client.1.smithi173.stdout:0/825: dread d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d65/f7c [0,4194304] 0 2024-04-20T11:09:45.749 INFO:tasks.workunit.client.1.smithi173.stdout:2/943: creat d2/de/d11/db8/f139 x:0 0 0 2024-04-20T11:09:45.753 INFO:tasks.workunit.client.1.smithi173.stdout:3/816: dwrite d7/d17/d61/d83/dcf/f14 [0,4194304] 0 2024-04-20T11:09:45.753 INFO:tasks.workunit.client.1.smithi173.stdout:6/836: mkdir d4/d59/d8f/dd7/d108 0 2024-04-20T11:09:45.756 INFO:tasks.workunit.client.1.smithi173.stdout:4/939: rename da/d1a/d60/d66/cc2 to da/d1a/d6e/db4/d132/c13f 0 2024-04-20T11:09:45.758 INFO:tasks.workunit.client.1.smithi173.stdout:6/837: dread - d4/d24/ffc zero size 2024-04-20T11:09:45.762 INFO:tasks.workunit.client.1.smithi173.stdout:5/868: link da/d4f/d65/d8e/l119 da/de/d19/d32/d66/l12b 0 2024-04-20T11:09:45.766 INFO:tasks.workunit.client.1.smithi173.stdout:3/817: rename d7/d53/f8f to d7/d85/df5/f100 0 2024-04-20T11:09:45.770 INFO:tasks.workunit.client.1.smithi173.stdout:4/940: mkdir da/dc/ddd/d140 0 2024-04-20T11:09:45.770 INFO:tasks.workunit.client.1.smithi173.stdout:2/944: truncate d2/d13/d28/d4a/f7f 1771107 0 2024-04-20T11:09:45.773 INFO:tasks.workunit.client.1.smithi173.stdout:2/945: stat d2/d13/d9a/dbc/dfd/f85 0 2024-04-20T11:09:45.773 INFO:tasks.workunit.client.1.smithi173.stdout:5/869: rmdir da/d34/d7f/da5/d9a 39 2024-04-20T11:09:45.774 INFO:tasks.workunit.client.1.smithi173.stdout:1/769: getdents d5/d6/d17/d42/da/d20/d38/d52 0 2024-04-20T11:09:45.780 INFO:tasks.workunit.client.1.smithi173.stdout:2/946: write d2/de/d11/db8/f117 [796998,47293] 0 2024-04-20T11:09:45.783 INFO:tasks.workunit.client.1.smithi173.stdout:3/818: symlink d7/d17/d61/d83/d45/l101 0 2024-04-20T11:09:45.785 INFO:tasks.workunit.client.1.smithi173.stdout:2/947: chown d2/de/l1a 11 1 2024-04-20T11:09:45.786 INFO:tasks.workunit.client.1.smithi173.stdout:1/770: mknod d5/d6/d17/d42/d12/d1c/c101 0 2024-04-20T11:09:45.787 INFO:tasks.workunit.client.1.smithi173.stdout:6/838: write d4/d24/d3e/f4d [3902527,38284] 0 2024-04-20T11:09:45.787 INFO:tasks.workunit.client.1.smithi173.stdout:0/826: dwrite d6/d8/df/d2c/d42/f72 [0,4194304] 0 2024-04-20T11:09:45.791 INFO:tasks.workunit.client.1.smithi173.stdout:6/839: chown d4/d14/d5f/daf 14834799 1 2024-04-20T11:09:45.792 INFO:tasks.workunit.client.1.smithi173.stdout:3/819: mknod d7/d53/dab/c102 0 2024-04-20T11:09:45.795 INFO:tasks.workunit.client.1.smithi173.stdout:5/870: write da/de/d19/d32/d30/f41 [4592490,121148] 0 2024-04-20T11:09:45.801 INFO:tasks.workunit.client.1.smithi173.stdout:2/948: rename d2/d13/d9a/dbc/dfd/d53/c67 to d2/d13/d9a/dbc/dfd/d4b/c13a 0 2024-04-20T11:09:45.803 INFO:tasks.workunit.client.1.smithi173.stdout:2/949: truncate d2/d13/d28/f120 301866 0 2024-04-20T11:09:45.805 INFO:tasks.workunit.client.1.smithi173.stdout:5/871: creat da/de/d19/d32/d30/d109/f12c x:0 0 0 2024-04-20T11:09:45.806 INFO:tasks.workunit.client.1.smithi173.stdout:1/771: write d5/d6/d17/d42/da/d20/d38/d52/d75/f7b [141194,52406] 0 2024-04-20T11:09:45.807 INFO:tasks.workunit.client.1.smithi173.stdout:6/840: write d4/d24/d42/f72 [4016306,100988] 0 2024-04-20T11:09:45.810 INFO:tasks.workunit.client.1.smithi173.stdout:3/820: creat d7/d17/d61/d83/f103 x:0 0 0 2024-04-20T11:09:45.812 INFO:tasks.workunit.client.1.smithi173.stdout:0/827: link d6/f29 d6/d24/fff 0 2024-04-20T11:09:45.815 INFO:tasks.workunit.client.1.smithi173.stdout:1/772: unlink d5/d6/d17/d42/l50 0 2024-04-20T11:09:45.818 INFO:tasks.workunit.client.1.smithi173.stdout:3/821: read d7/d17/d61/d83/dcf/d30/f8d [419242,62432] 0 2024-04-20T11:09:45.819 INFO:tasks.workunit.client.1.smithi173.stdout:2/950: dwrite d2/d13/da5/db2/f12e [0,4194304] 0 2024-04-20T11:09:45.823 INFO:tasks.workunit.client.1.smithi173.stdout:1/773: write d5/d6/d17/d42/d12/d1c/f8b [557608,95958] 0 2024-04-20T11:09:45.823 INFO:tasks.workunit.client.1.smithi173.stdout:6/841: rename d4/d24/d3e/l101 to d4/d18/l109 0 2024-04-20T11:09:45.825 INFO:tasks.workunit.client.1.smithi173.stdout:5/872: getdents da/d4f/d5e 0 2024-04-20T11:09:45.826 INFO:tasks.workunit.client.1.smithi173.stdout:0/828: getdents d6/d5f/d8f/db4 0 2024-04-20T11:09:45.827 INFO:tasks.workunit.client.1.smithi173.stdout:5/873: chown da/d4f/d5e 369 1 2024-04-20T11:09:45.828 INFO:tasks.workunit.client.1.smithi173.stdout:2/951: rmdir d2/de/d11/d9d/d87 39 2024-04-20T11:09:45.830 INFO:tasks.workunit.client.1.smithi173.stdout:1/774: truncate d5/d6/d17/d42/da/df/d40/dc8/fcc 1778118 0 2024-04-20T11:09:45.834 INFO:tasks.workunit.client.1.smithi173.stdout:0/829: readlink d6/d8/df/l21 0 2024-04-20T11:09:45.841 INFO:tasks.workunit.client.1.smithi173.stdout:2/952: creat d2/d13/d28/f13b x:0 0 0 2024-04-20T11:09:45.843 INFO:tasks.workunit.client.1.smithi173.stdout:3/822: write d7/d17/d61/d83/dcf/d5c/fc9 [724912,76629] 0 2024-04-20T11:09:45.848 INFO:tasks.workunit.client.1.smithi173.stdout:3/823: dread - d7/d17/d58/d7f/f95 zero size 2024-04-20T11:09:45.848 INFO:tasks.workunit.client.1.smithi173.stdout:6/842: dwrite d4/d24/d42/f67 [0,4194304] 0 2024-04-20T11:09:45.852 INFO:tasks.workunit.client.1.smithi173.stdout:6/843: truncate d4/d14/d5f/f76 1515183 0 2024-04-20T11:09:45.859 INFO:tasks.workunit.client.1.smithi173.stdout:1/775: dread d5/d6/d17/d42/da/d20/dad/dc2/f59 [0,4194304] 0 2024-04-20T11:09:45.868 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:45 smithi173 ceph-mon[23800]: pgmap v32: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 65 MiB/s rd, 191 MiB/s wr, 454 op/s 2024-04-20T11:09:45.868 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:45 smithi173 ceph-mon[23800]: Upgrade: Updating mgr.smithi173.gnrjof 2024-04-20T11:09:45.868 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:45 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:09:45.868 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:45 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:09:45.868 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:45 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:09:45.868 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:45 smithi173 ceph-mon[23800]: Deploying daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T11:09:45.879 INFO:tasks.workunit.client.1.smithi173.stdout:2/953: creat d2/d34/d58/f13c x:0 0 0 2024-04-20T11:09:45.881 INFO:tasks.workunit.client.1.smithi173.stdout:2/954: stat d2/d13/d28/d4a/dff/l115 0 2024-04-20T11:09:45.881 INFO:tasks.workunit.client.1.smithi173.stdout:0/830: creat d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/f100 x:0 0 0 2024-04-20T11:09:45.882 INFO:tasks.workunit.client.1.smithi173.stdout:6/844: chown d4/d24/d42/d74/c85 257038 1 2024-04-20T11:09:45.884 INFO:tasks.workunit.client.1.smithi173.stdout:6/845: chown d4/d18/fb9 36969 1 2024-04-20T11:09:45.886 INFO:tasks.workunit.client.1.smithi173.stdout:1/776: write d5/d6/d17/f4b [4740235,109642] 0 2024-04-20T11:09:45.886 INFO:tasks.workunit.client.1.smithi173.stdout:8/864: sync 2024-04-20T11:09:45.888 INFO:tasks.workunit.client.1.smithi173.stdout:3/824: link d7/d17/d61/d83/d45/l101 d7/d17/d58/df3/l104 0 2024-04-20T11:09:45.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:45 smithi117 ceph-mon[15659]: pgmap v32: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 65 MiB/s rd, 191 MiB/s wr, 454 op/s 2024-04-20T11:09:45.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:45 smithi117 ceph-mon[15659]: Upgrade: Updating mgr.smithi173.gnrjof 2024-04-20T11:09:45.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:45 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:09:45.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:45 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:09:45.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:45 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:09:45.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:45 smithi117 ceph-mon[15659]: Deploying daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T11:09:45.890 INFO:tasks.workunit.client.1.smithi173.stdout:0/831: getdents d6/d5f/d8f/db4 0 2024-04-20T11:09:45.893 INFO:tasks.workunit.client.1.smithi173.stdout:2/955: rename d2/de/fa7 to d2/d13/d9a/dbc/dfd/d53/f13d 0 2024-04-20T11:09:45.893 INFO:tasks.workunit.client.1.smithi173.stdout:0/832: read d6/d8/df/d2c/d42/f7a [229984,127797] 0 2024-04-20T11:09:45.894 INFO:tasks.workunit.client.1.smithi173.stdout:3/825: mkdir d7/d17/d61/d83/dc2/d105 0 2024-04-20T11:09:45.895 INFO:tasks.workunit.client.1.smithi173.stdout:1/777: symlink d5/l102 0 2024-04-20T11:09:45.895 INFO:tasks.workunit.client.1.smithi173.stdout:5/874: link da/de/d75/d3e/c10f da/de/d75/d40/c12d 0 2024-04-20T11:09:45.898 INFO:tasks.workunit.client.1.smithi173.stdout:0/833: write d6/d8/d3c/dee/ff2 [61520,62699] 0 2024-04-20T11:09:45.898 INFO:tasks.workunit.client.1.smithi173.stdout:3/826: chown d7/d2b/d8e/fb3 14225886 1 2024-04-20T11:09:45.901 INFO:tasks.workunit.client.1.smithi173.stdout:0/834: stat d6/d8/df/d43/d7f/cd8 0 2024-04-20T11:09:45.902 INFO:tasks.workunit.client.1.smithi173.stdout:5/875: mkdir da/d9e/d118/d12e 0 2024-04-20T11:09:45.905 INFO:tasks.workunit.client.1.smithi173.stdout:6/846: dwrite d4/f60 [0,4194304] 0 2024-04-20T11:09:45.910 INFO:tasks.workunit.client.1.smithi173.stdout:5/876: creat da/d9e/d118/d12e/f12f x:0 0 0 2024-04-20T11:09:45.911 INFO:tasks.workunit.client.1.smithi173.stdout:8/865: dread d4/d18/d59/d5a/f67 [0,4194304] 0 2024-04-20T11:09:45.911 INFO:tasks.workunit.client.1.smithi173.stdout:2/956: write d2/fc5 [849332,110984] 0 2024-04-20T11:09:45.915 INFO:tasks.workunit.client.1.smithi173.stdout:0/835: mknod d6/d8/df/dd0/dd4/c101 0 2024-04-20T11:09:45.916 INFO:tasks.workunit.client.1.smithi173.stdout:1/778: dwrite f0 [4194304,4194304] 0 2024-04-20T11:09:45.922 INFO:tasks.workunit.client.1.smithi173.stdout:2/957: chown d2/d13/d9a/dbc/dfd/d4b/fda 3372 1 2024-04-20T11:09:45.922 INFO:tasks.workunit.client.1.smithi173.stdout:8/866: mkdir d4/d18/d4f/d5b/de4/de6/d111 0 2024-04-20T11:09:45.925 INFO:tasks.workunit.client.1.smithi173.stdout:8/867: fdatasync d4/d18/d4f/fc6 0 2024-04-20T11:09:45.929 INFO:tasks.workunit.client.1.smithi173.stdout:3/827: link d7/d17/d61/d83/dcf/l1e d7/d85/df5/l106 0 2024-04-20T11:09:45.931 INFO:tasks.workunit.client.1.smithi173.stdout:2/958: creat d2/f13e x:0 0 0 2024-04-20T11:09:45.937 INFO:tasks.workunit.client.1.smithi173.stdout:6/847: dwrite d4/d24/d3e/f4c [0,4194304] 0 2024-04-20T11:09:45.940 INFO:tasks.workunit.client.1.smithi173.stdout:0/836: dwrite d6/d8/d51/fea [0,4194304] 0 2024-04-20T11:09:45.943 INFO:tasks.workunit.client.1.smithi173.stdout:2/959: dread d2/d13/d28/d4a/f6c [0,4194304] 0 2024-04-20T11:09:45.949 INFO:tasks.workunit.client.1.smithi173.stdout:8/868: dwrite d4/d18/d59/d5a/da8/db9/de0/f46 [0,4194304] 0 2024-04-20T11:09:45.951 INFO:tasks.workunit.client.1.smithi173.stdout:7/870: sync 2024-04-20T11:09:45.951 INFO:tasks.workunit.client.1.smithi173.stdout:4/941: sync 2024-04-20T11:09:45.951 INFO:tasks.workunit.client.1.smithi173.stdout:9/843: sync 2024-04-20T11:09:45.952 INFO:tasks.workunit.client.1.smithi173.stdout:0/837: symlink d6/d8/d3c/d44/d75/l102 0 2024-04-20T11:09:45.952 INFO:tasks.workunit.client.1.smithi173.stdout:8/869: fdatasync d4/d18/fa2 0 2024-04-20T11:09:45.965 INFO:tasks.workunit.client.1.smithi173.stdout:5/877: dread da/de/d75/d40/f42 [0,4194304] 0 2024-04-20T11:09:45.965 INFO:tasks.workunit.client.1.smithi173.stdout:9/844: rmdir d5/d9/d20/d2b/d106/d79 39 2024-04-20T11:09:45.965 INFO:tasks.workunit.client.1.smithi173.stdout:4/942: rename da/dc/d13/l3a to da/d1e/d52/d7a/l141 0 2024-04-20T11:09:45.967 INFO:tasks.workunit.client.1.smithi173.stdout:7/871: read d9/da/d17/d2b/daa/d10b/d123/fda [524301,102461] 0 2024-04-20T11:09:45.970 INFO:tasks.workunit.client.1.smithi173.stdout:2/960: creat d2/d13/d9a/dbc/dfd/d4b/dae/f13f x:0 0 0 2024-04-20T11:09:45.970 INFO:tasks.workunit.client.1.smithi173.stdout:1/779: dread d5/d6/d3b/f98 [0,4194304] 0 2024-04-20T11:09:45.971 INFO:tasks.workunit.client.1.smithi173.stdout:6/848: dwrite d4/d14/fce [0,4194304] 0 2024-04-20T11:09:45.974 INFO:tasks.workunit.client.1.smithi173.stdout:9/845: mkdir d5/dcc/d107 0 2024-04-20T11:09:45.976 INFO:tasks.workunit.client.1.smithi173.stdout:5/878: creat da/de/d19/d32/d72/f130 x:0 0 0 2024-04-20T11:09:45.976 INFO:tasks.workunit.client.1.smithi173.stdout:4/943: mknod da/d1a/d60/d7d/d101/c142 0 2024-04-20T11:09:45.979 INFO:tasks.workunit.client.1.smithi173.stdout:7/872: rename d9/d33/d35/d4a/def to d9/d33/de9/d128 0 2024-04-20T11:09:45.979 INFO:tasks.workunit.client.1.smithi173.stdout:8/870: dread d4/d18/d4f/f9a [0,4194304] 0 2024-04-20T11:09:45.988 INFO:tasks.workunit.client.1.smithi173.stdout:4/944: unlink da/d1a/d60/f8c 0 2024-04-20T11:09:45.989 INFO:tasks.workunit.client.1.smithi173.stdout:5/879: write da/de/d19/d2c/f46 [1496327,111632] 0 2024-04-20T11:09:45.995 INFO:tasks.workunit.client.1.smithi173.stdout:7/873: rename d9/da/d43 to d9/da/d17/d2b/daa/d10b/d123/d129 0 2024-04-20T11:09:45.998 INFO:tasks.workunit.client.1.smithi173.stdout:3/828: dread d7/d2b/fcd [0,4194304] 0 2024-04-20T11:09:45.998 INFO:tasks.workunit.client.1.smithi173.stdout:8/871: mknod d4/d18/d59/d5a/da8/db9/de0/d9b/c112 0 2024-04-20T11:09:45.999 INFO:tasks.workunit.client.1.smithi173.stdout:6/849: write d4/f20 [1991607,10275] 0 2024-04-20T11:09:46.000 INFO:tasks.workunit.client.1.smithi173.stdout:0/838: dwrite d6/d8/df/d15/ff4 [0,4194304] 0 2024-04-20T11:09:46.002 INFO:tasks.workunit.client.1.smithi173.stdout:2/961: truncate d2/d13/d9a/fdf 1846160 0 2024-04-20T11:09:46.005 INFO:tasks.workunit.client.1.smithi173.stdout:6/850: stat d4/f46 0 2024-04-20T11:09:46.005 INFO:tasks.workunit.client.1.smithi173.stdout:7/874: write d9/da/d17/d2b/f71 [4280150,30983] 0 2024-04-20T11:09:46.006 INFO:tasks.workunit.client.1.smithi173.stdout:1/780: dwrite d5/d6/d17/d42/d12/f15 [0,4194304] 0 2024-04-20T11:09:46.007 INFO:tasks.workunit.client.1.smithi173.stdout:6/851: fdatasync d4/d1c/fa0 0 2024-04-20T11:09:46.008 INFO:tasks.workunit.client.1.smithi173.stdout:5/880: getdents da/de/d75/d40 0 2024-04-20T11:09:46.012 INFO:tasks.workunit.client.1.smithi173.stdout:2/962: mknod d2/d34/d81/dca/c140 0 2024-04-20T11:09:46.012 INFO:tasks.workunit.client.1.smithi173.stdout:4/945: dread da/d1e/ffd [0,4194304] 0 2024-04-20T11:09:46.013 INFO:tasks.workunit.client.1.smithi173.stdout:4/946: chown da/dc/d13/d106 2322677 1 2024-04-20T11:09:46.017 INFO:tasks.workunit.client.1.smithi173.stdout:4/947: readlink da/dc/d13/d106/l11d 0 2024-04-20T11:09:46.018 INFO:tasks.workunit.client.1.smithi173.stdout:8/872: dread d4/d18/d59/d5a/da8/db9/de0/f66 [0,4194304] 0 2024-04-20T11:09:46.021 INFO:tasks.workunit.client.1.smithi173.stdout:9/846: dread d5/d9/d20/d29/d18/f1f [4194304,4194304] 0 2024-04-20T11:09:46.023 INFO:tasks.workunit.client.1.smithi173.stdout:0/839: creat d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/f103 x:0 0 0 2024-04-20T11:09:46.026 INFO:tasks.workunit.client.1.smithi173.stdout:0/840: read d6/d8/d3c/d44/d75/f7b [4237341,63656] 0 2024-04-20T11:09:46.027 INFO:tasks.workunit.client.1.smithi173.stdout:5/881: dwrite da/de/fac [0,4194304] 0 2024-04-20T11:09:46.029 INFO:tasks.workunit.client.1.smithi173.stdout:6/852: rename d4/d24/l65 to d4/d18/d7e/l10a 0 2024-04-20T11:09:46.031 INFO:tasks.workunit.client.1.smithi173.stdout:7/875: mknod d9/da/d17/c12a 0 2024-04-20T11:09:46.032 INFO:tasks.workunit.client.1.smithi173.stdout:1/781: rmdir d5/d6/d17/d93/db6 39 2024-04-20T11:09:46.033 INFO:tasks.workunit.client.1.smithi173.stdout:7/876: write d9/da/d17/d2b/f71 [992348,36373] 0 2024-04-20T11:09:46.037 INFO:tasks.workunit.client.1.smithi173.stdout:1/782: truncate d5/d6/dc/d7f/fd0 757014 0 2024-04-20T11:09:46.038 INFO:tasks.workunit.client.1.smithi173.stdout:0/841: creat d6/d8/d3c/d44/d69/f104 x:0 0 0 2024-04-20T11:09:46.040 INFO:tasks.workunit.client.1.smithi173.stdout:5/882: creat da/d9e/d118/d12e/f131 x:0 0 0 2024-04-20T11:09:46.041 INFO:tasks.workunit.client.1.smithi173.stdout:4/948: mknod da/d1a/d60/d66/c143 0 2024-04-20T11:09:46.049 INFO:tasks.workunit.client.1.smithi173.stdout:2/963: write d2/de/f2b [116775,69308] 0 2024-04-20T11:09:46.051 INFO:tasks.workunit.client.1.smithi173.stdout:9/847: unlink d5/d9/d20/d2b/d106/d79/da7/c104 0 2024-04-20T11:09:46.058 INFO:tasks.workunit.client.1.smithi173.stdout:8/873: write d4/d18/f21 [3952865,118154] 0 2024-04-20T11:09:46.058 INFO:tasks.workunit.client.1.smithi173.stdout:0/842: symlink d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac/l105 0 2024-04-20T11:09:46.058 INFO:tasks.workunit.client.1.smithi173.stdout:8/874: dread - d4/d18/d59/d5a/da8/db9/fed zero size 2024-04-20T11:09:46.060 INFO:tasks.workunit.client.1.smithi173.stdout:7/877: creat d9/d33/d35/d54/d10c/d113/f12b x:0 0 0 2024-04-20T11:09:46.062 INFO:tasks.workunit.client.1.smithi173.stdout:9/848: chown d5/d9/d20/d29/d2d/dcb/ccf 477051 1 2024-04-20T11:09:46.066 INFO:tasks.workunit.client.1.smithi173.stdout:6/853: rename d4/d24/c5b to d4/c10b 0 2024-04-20T11:09:46.071 INFO:tasks.workunit.client.1.smithi173.stdout:7/878: mkdir d9/da/d2e/d12c 0 2024-04-20T11:09:46.073 INFO:tasks.workunit.client.1.smithi173.stdout:1/783: link d5/d6/lc6 d5/d6/d17/d42/da/df/d40/dc8/dfc/l103 0 2024-04-20T11:09:46.075 INFO:tasks.workunit.client.1.smithi173.stdout:0/843: mknod d6/d8/df/d2c/c106 0 2024-04-20T11:09:46.077 INFO:tasks.workunit.client.1.smithi173.stdout:4/949: rename da/dc/df4/f11b to da/dc/d103/d10e/f144 0 2024-04-20T11:09:46.078 INFO:tasks.workunit.client.1.smithi173.stdout:5/883: write da/de/d75/f51 [2130958,73798] 0 2024-04-20T11:09:46.079 INFO:tasks.workunit.client.1.smithi173.stdout:6/854: write d4/de3/ff8 [453760,20154] 0 2024-04-20T11:09:46.080 INFO:tasks.workunit.client.1.smithi173.stdout:7/879: creat d9/da/dc3/f12d x:0 0 0 2024-04-20T11:09:46.081 INFO:tasks.workunit.client.1.smithi173.stdout:6/855: chown d4/d18/d7e/fbe 29816933 1 2024-04-20T11:09:46.082 INFO:tasks.workunit.client.1.smithi173.stdout:1/784: unlink d5/d6/d17/d93/lb2 0 2024-04-20T11:09:46.083 INFO:tasks.workunit.client.1.smithi173.stdout:1/785: stat d5/d6/d3b/l5a 0 2024-04-20T11:09:46.087 INFO:tasks.workunit.client.1.smithi173.stdout:2/964: rename d2/d13/d28 to d2/de/d11/d9d/d87/db0/d141 0 2024-04-20T11:09:46.091 INFO:tasks.workunit.client.1.smithi173.stdout:6/856: creat d4/d24/d3e/d9c/df2/f10c x:0 0 0 2024-04-20T11:09:46.093 INFO:tasks.workunit.client.1.smithi173.stdout:1/786: mkdir d5/d6/d104 0 2024-04-20T11:09:46.095 INFO:tasks.workunit.client.1.smithi173.stdout:5/884: mknod da/de/d19/d2c/c132 0 2024-04-20T11:09:46.097 INFO:tasks.workunit.client.1.smithi173.stdout:8/875: rename d4/d18/d4f/d5b/fc3 to d4/d18/d59/d5a/d7b/f113 0 2024-04-20T11:09:46.181 INFO:tasks.workunit.client.1.smithi173.stdout:4/950: rename da/dc to da/dc/d15/d145 22 2024-04-20T11:09:46.183 INFO:tasks.workunit.client.1.smithi173.stdout:1/787: creat d5/d6/d17/d42/da/df/d40/dc8/dde/f105 x:0 0 0 2024-04-20T11:09:46.185 INFO:tasks.workunit.client.1.smithi173.stdout:5/885: creat da/d34/da7/f133 x:0 0 0 2024-04-20T11:09:46.186 INFO:tasks.workunit.client.1.smithi173.stdout:7/880: write d9/da/d17/f1b [1193133,23566] 0 2024-04-20T11:09:46.189 INFO:tasks.workunit.client.1.smithi173.stdout:8/876: chown d4/d18/d59/d5a/l10d 13023599 1 2024-04-20T11:09:46.189 INFO:tasks.workunit.client.1.smithi173.stdout:2/965: rename d2/de/d11/d9d to d2/de/d11/db8/d142 0 2024-04-20T11:09:46.194 INFO:tasks.workunit.client.1.smithi173.stdout:6/857: creat d4/d24/d42/d74/f10d x:0 0 0 2024-04-20T11:09:46.194 INFO:tasks.workunit.client.1.smithi173.stdout:0/844: dread d6/d8/df/d2c/d42/d50/d86/fbc [0,4194304] 0 2024-04-20T11:09:46.197 INFO:tasks.workunit.client.1.smithi173.stdout:7/881: fdatasync d9/d33/d40/d7a/f8c 0 2024-04-20T11:09:46.200 INFO:tasks.workunit.client.1.smithi173.stdout:4/951: rename da/d1a/d60/fb1 to da/dc/df4/dff/f146 0 2024-04-20T11:09:46.206 INFO:tasks.workunit.client.1.smithi173.stdout:7/882: fsync d9/fbf 0 2024-04-20T11:09:46.208 INFO:tasks.workunit.client.1.smithi173.stdout:2/966: rename d2/d13/d9a/dbc/dfd/d53/d4e/f8d to d2/d13/d9a/dbc/f143 0 2024-04-20T11:09:46.209 INFO:tasks.workunit.client.1.smithi173.stdout:6/858: mknod d4/d24/c10e 0 2024-04-20T11:09:46.213 INFO:tasks.workunit.client.1.smithi173.stdout:6/859: chown d4/d14/c69 293503071 1 2024-04-20T11:09:46.217 INFO:tasks.workunit.client.1.smithi173.stdout:1/788: rename d5/d6/dc/d7f/fd0 to d5/d6/d17/d93/db6/df4/f106 0 2024-04-20T11:09:46.221 INFO:tasks.workunit.client.1.smithi173.stdout:2/967: truncate d2/de/d11/f33 4287644 0 2024-04-20T11:09:46.222 INFO:tasks.workunit.client.1.smithi173.stdout:7/883: creat d9/d33/d40/d11c/f12e x:0 0 0 2024-04-20T11:09:46.223 INFO:tasks.workunit.client.1.smithi173.stdout:8/877: getdents d4/d18/d59/d5a/da8/db9/de0/d3c/d86/dec 0 2024-04-20T11:09:46.224 INFO:tasks.workunit.client.1.smithi173.stdout:1/789: chown d5/d6/dc/c2a 207 1 2024-04-20T11:09:46.225 INFO:tasks.workunit.client.1.smithi173.stdout:0/845: rename d6/d8/df/d2c/d42/d50/d86/d8b/da2/dac to d6/d8/d3c/dee/d107 0 2024-04-20T11:09:46.230 INFO:tasks.workunit.client.1.smithi173.stdout:4/952: getdents da/d1e/d52/d9a 0 2024-04-20T11:09:46.230 INFO:tasks.workunit.client.1.smithi173.stdout:7/884: dread - d9/da/d17/d2b/daa/d10b/d123/d129/fdf zero size 2024-04-20T11:09:46.234 INFO:tasks.workunit.client.1.smithi173.stdout:7/885: chown d9/dbc/fbe 236 1 2024-04-20T11:09:46.237 INFO:tasks.workunit.client.1.smithi173.stdout:4/953: write da/d1a/d60/d7d/fc5 [2879452,58875] 0 2024-04-20T11:09:46.240 INFO:tasks.workunit.client.1.smithi173.stdout:2/968: creat d2/d13/d9a/dbc/dfd/d4b/dae/db6/df9/d111/f144 x:0 0 0 2024-04-20T11:09:46.243 INFO:tasks.workunit.client.1.smithi173.stdout:1/790: creat d5/d6/d17/d42/da/d20/d38/d52/d75/f107 x:0 0 0 2024-04-20T11:09:46.244 INFO:tasks.workunit.client.1.smithi173.stdout:2/969: chown d2/de/f3b 787737 1 2024-04-20T11:09:46.248 INFO:tasks.workunit.client.1.smithi173.stdout:2/970: stat d2/d13/d9a/dbc/dfd/d4b/dc3 0 2024-04-20T11:09:46.249 INFO:tasks.workunit.client.1.smithi173.stdout:1/791: read d5/d6/d17/d42/d12/d1f/fbc [2981897,44015] 0 2024-04-20T11:09:46.250 INFO:tasks.workunit.client.1.smithi173.stdout:4/954: fdatasync da/dc/df4/d53/fbb 0 2024-04-20T11:09:46.253 INFO:tasks.workunit.client.1.smithi173.stdout:1/792: chown d5/d6/d3b/f98 1990720 1 2024-04-20T11:09:46.254 INFO:tasks.workunit.client.1.smithi173.stdout:6/860: dread d4/d24/f2e [0,4194304] 0 2024-04-20T11:09:46.255 INFO:tasks.workunit.client.1.smithi173.stdout:1/793: fsync d5/d6/d17/d42/da/df/fe7 0 2024-04-20T11:09:46.258 INFO:tasks.workunit.client.1.smithi173.stdout:7/886: rmdir d9/da/d2e/d12c 0 2024-04-20T11:09:46.259 INFO:tasks.workunit.client.1.smithi173.stdout:4/955: creat da/d1a/d6e/db4/d132/f147 x:0 0 0 2024-04-20T11:09:46.262 INFO:tasks.workunit.client.1.smithi173.stdout:6/861: rmdir d4/d24/d42/d74/df6 39 2024-04-20T11:09:46.262 INFO:tasks.workunit.client.1.smithi173.stdout:0/846: link d6/d8/df/d43/d7f/fab d6/d8/df/d2c/d42/f108 0 2024-04-20T11:09:46.262 INFO:tasks.workunit.client.1.smithi173.stdout:4/956: chown da/d1a/la9 520 1 2024-04-20T11:09:46.263 INFO:tasks.workunit.client.1.smithi173.stdout:2/971: mknod d2/d13/d9a/dbc/dfd/d4b/dae/c145 0 2024-04-20T11:09:46.266 INFO:tasks.workunit.client.1.smithi173.stdout:1/794: creat d5/f108 x:0 0 0 2024-04-20T11:09:46.267 INFO:tasks.workunit.client.1.smithi173.stdout:7/887: fsync d9/da/d17/fbb 0 2024-04-20T11:09:46.268 INFO:tasks.workunit.client.1.smithi173.stdout:6/862: unlink d4/d14/d5f/d9f/ffa 0 2024-04-20T11:09:46.271 INFO:tasks.workunit.client.1.smithi173.stdout:1/795: chown d5/d6/d17/d42/d12/f31 122636 1 2024-04-20T11:09:46.271 INFO:tasks.workunit.client.1.smithi173.stdout:2/972: symlink d2/d8c/l146 0 2024-04-20T11:09:46.271 INFO:tasks.workunit.client.1.smithi173.stdout:8/878: dread d4/d18/fa7 [0,4194304] 0 2024-04-20T11:09:46.271 INFO:tasks.workunit.client.1.smithi173.stdout:6/863: readlink d4/d24/d3e/l71 0 2024-04-20T11:09:46.274 INFO:tasks.workunit.client.1.smithi173.stdout:2/973: fsync d2/d34/d81/f9e 0 2024-04-20T11:09:46.277 INFO:tasks.workunit.client.1.smithi173.stdout:2/974: readlink d2/d13/d9a/dbc/dfd/d53/lb1 0 2024-04-20T11:09:46.280 INFO:tasks.workunit.client.1.smithi173.stdout:6/864: stat d4/d24/d42/d74/df6/ff7 0 2024-04-20T11:09:46.280 INFO:tasks.workunit.client.1.smithi173.stdout:7/888: symlink d9/d33/d35/d54/db3/dfd/l12f 0 2024-04-20T11:09:46.281 INFO:tasks.workunit.client.1.smithi173.stdout:7/889: chown d9/da/dc3 47 1 2024-04-20T11:09:46.283 INFO:tasks.workunit.client.1.smithi173.stdout:8/879: creat d4/d18/d59/d5a/d7b/d8c/f114 x:0 0 0 2024-04-20T11:09:46.287 INFO:tasks.workunit.client.1.smithi173.stdout:8/880: dread - d4/d18/d59/d5a/da8/db9/fed zero size 2024-04-20T11:09:46.290 INFO:tasks.workunit.client.1.smithi173.stdout:1/796: getdents d5/d6/d17/d93/db6/df4 0 2024-04-20T11:09:46.297 INFO:tasks.workunit.client.1.smithi173.stdout:7/890: getdents d9/d33/d91 0 2024-04-20T11:09:46.299 INFO:tasks.workunit.client.1.smithi173.stdout:8/881: creat d4/d18/d4f/d5b/de4/de6/deb/f115 x:0 0 0 2024-04-20T11:09:46.300 INFO:tasks.workunit.client.1.smithi173.stdout:0/847: dread d6/d8/df/d2c/d2d/f6a [0,4194304] 0 2024-04-20T11:09:46.301 INFO:tasks.workunit.client.1.smithi173.stdout:1/797: creat d5/d6/d17/d42/d12/dc9/f109 x:0 0 0 2024-04-20T11:09:46.304 INFO:tasks.workunit.client.1.smithi173.stdout:4/957: dwrite da/dc/fe [0,4194304] 0 2024-04-20T11:09:46.305 INFO:tasks.workunit.client.1.smithi173.stdout:7/891: chown d9/d1f/d7f/dce/db2 246328601 1 2024-04-20T11:09:46.307 INFO:tasks.workunit.client.1.smithi173.stdout:1/798: symlink d5/dd8/l10a 0 2024-04-20T11:09:46.308 INFO:tasks.workunit.client.1.smithi173.stdout:4/958: chown da/dc/df4/dff 1 1 2024-04-20T11:09:46.308 INFO:tasks.workunit.client.1.smithi173.stdout:6/865: dwrite d4/d14/f1a [0,4194304] 0 2024-04-20T11:09:46.308 INFO:tasks.workunit.client.1.smithi173.stdout:0/848: rmdir d6/d8/df/d2c/d42/d50/d86 39 2024-04-20T11:09:46.311 INFO:tasks.workunit.client.1.smithi173.stdout:0/849: chown d6/d24/f53 47 1 2024-04-20T11:09:46.313 INFO:tasks.workunit.client.1.smithi173.stdout:7/892: fdatasync d9/da/d17/fbb 0 2024-04-20T11:09:46.313 INFO:tasks.workunit.client.1.smithi173.stdout:6/866: write d4/d24/d3e/d9c/fb1 [856687,126329] 0 2024-04-20T11:09:46.313 INFO:tasks.workunit.client.1.smithi173.stdout:8/882: mkdir d4/d18/d59/d5a/d7b/db1/dce/ddd/d116 0 2024-04-20T11:09:46.318 INFO:tasks.workunit.client.1.smithi173.stdout:6/867: readlink d4/d18/d7e/lfd 0 2024-04-20T11:09:46.325 INFO:tasks.workunit.client.1.smithi173.stdout:6/868: fsync d4/d24/d42/f7c 0 2024-04-20T11:09:46.326 INFO:tasks.workunit.client.1.smithi173.stdout:0/850: creat d6/d8/df/d2c/d42/d50/d86/d8b/dcc/f109 x:0 0 0 2024-04-20T11:09:46.331 INFO:tasks.workunit.client.1.smithi173.stdout:8/883: rename d4/d18/da6/la9 to d4/d18/d59/d5a/l117 0 2024-04-20T11:09:46.332 INFO:tasks.workunit.client.1.smithi173.stdout:4/959: dread da/dc/d13/f47 [0,4194304] 0 2024-04-20T11:09:46.334 INFO:tasks.workunit.client.1.smithi173.stdout:6/869: getdents d4/d59/d8f/d107 0 2024-04-20T11:09:46.334 INFO:tasks.workunit.client.1.smithi173.stdout:1/799: dwrite d5/d6/d3b/f98 [4194304,4194304] 0 2024-04-20T11:09:46.351 INFO:tasks.workunit.client.1.smithi173.stdout:8/884: getdents d4 0 2024-04-20T11:09:46.358 INFO:tasks.workunit.client.1.smithi173.stdout:6/870: read d4/d1c/d8e/d90/ff3 [140710,100866] 0 2024-04-20T11:09:46.362 INFO:tasks.workunit.client.1.smithi173.stdout:8/885: mknod d4/d18/d59/d5a/da8/db9/de0/d3c/d86/db7/d110/c118 0 2024-04-20T11:09:46.365 INFO:tasks.workunit.client.1.smithi173.stdout:4/960: write da/f25 [1130973,60341] 0 2024-04-20T11:09:46.367 INFO:tasks.workunit.client.1.smithi173.stdout:6/871: creat d4/d24/d42/f10f x:0 0 0 2024-04-20T11:09:46.368 INFO:tasks.workunit.client.1.smithi173.stdout:8/886: write d4/d18/d59/d5a/da8/db9/de0/f9d [1018032,89972] 0 2024-04-20T11:09:46.372 INFO:tasks.workunit.client.1.smithi173.stdout:6/872: read d4/d24/d42/f7c [960965,26997] 0 2024-04-20T11:09:46.375 INFO:tasks.workunit.client.1.smithi173.stdout:6/873: readlink d4/d1c/d8e/d90/lc2 0 2024-04-20T11:09:46.376 INFO:tasks.workunit.client.1.smithi173.stdout:7/893: dread d9/d1f/d7f/dce/dad/fb9 [0,4194304] 0 2024-04-20T11:09:46.376 INFO:tasks.workunit.client.1.smithi173.stdout:8/887: write d4/d18/d4f/f7e [1191390,62801] 0 2024-04-20T11:09:46.377 INFO:tasks.workunit.client.1.smithi173.stdout:6/874: chown d4/d14/c69 2 1 2024-04-20T11:09:46.380 INFO:tasks.workunit.client.1.smithi173.stdout:7/894: symlink d9/d33/d91/l130 0 2024-04-20T11:09:46.380 INFO:tasks.workunit.client.1.smithi173.stdout:0/851: dread d6/d8/d3c/f41 [0,4194304] 0 2024-04-20T11:09:46.381 INFO:tasks.workunit.client.1.smithi173.stdout:6/875: unlink d4/d18/d61/cc0 0 2024-04-20T11:09:46.383 INFO:tasks.workunit.client.1.smithi173.stdout:8/888: creat d4/d18/d59/d5a/da8/db9/de0/d3c/d86/dec/f119 x:0 0 0 2024-04-20T11:09:46.400 INFO:tasks.workunit.client.1.smithi173.stdout:4/961: dread da/dc/d15/f5f [0,4194304] 0 2024-04-20T11:09:46.400 INFO:tasks.workunit.client.1.smithi173.stdout:8/889: dwrite d4/d18/d59/d5a/d7b/db1/fd9 [0,4194304] 0 2024-04-20T11:09:46.402 INFO:tasks.workunit.client.1.smithi173.stdout:4/962: dread - da/d1a/d6e/db4/d132/f147 zero size 2024-04-20T11:09:46.403 INFO:tasks.workunit.client.1.smithi173.stdout:6/876: dwrite d4/d24/d3e/d9c/fec [0,4194304] 0 2024-04-20T11:09:46.407 INFO:tasks.workunit.client.1.smithi173.stdout:8/890: creat d4/d18/d59/d5a/da8/db9/de0/d9b/dc8/d108/f11a x:0 0 0 2024-04-20T11:09:46.408 INFO:tasks.workunit.client.1.smithi173.stdout:0/852: dread d6/d8/df/fe0 [0,4194304] 0 2024-04-20T11:09:46.410 INFO:tasks.workunit.client.1.smithi173.stdout:8/891: readlink d4/lb 0 2024-04-20T11:09:46.411 INFO:tasks.workunit.client.1.smithi173.stdout:4/963: mknod da/dc/d103/d129/c148 0 2024-04-20T11:09:46.414 INFO:tasks.workunit.client.1.smithi173.stdout:0/853: mknod d6/d8/d3c/d44/d69/c10a 0 2024-04-20T11:09:46.414 INFO:tasks.workunit.client.1.smithi173.stdout:6/877: getdents d4/d59 0 2024-04-20T11:09:46.417 INFO:tasks.workunit.client.1.smithi173.stdout:4/964: creat da/dc/d62/d84/f149 x:0 0 0 2024-04-20T11:09:46.417 INFO:tasks.workunit.client.1.smithi173.stdout:6/878: dread - d4/d1c/d8e/d90/da4/fdc zero size 2024-04-20T11:09:46.418 INFO:tasks.workunit.client.1.smithi173.stdout:8/892: symlink d4/d18/d4f/d5b/de4/l11b 0 2024-04-20T11:09:46.421 INFO:tasks.workunit.client.1.smithi173.stdout:4/965: mknod da/d1a/d6e/c14a 0 2024-04-20T11:09:46.422 INFO:tasks.workunit.client.1.smithi173.stdout:6/879: rmdir d4/d1c/d8e/d90/da4 39 2024-04-20T11:09:46.424 INFO:tasks.workunit.client.1.smithi173.stdout:0/854: creat d6/d8/df/d2c/d42/d50/f10b x:0 0 0 2024-04-20T11:09:46.424 INFO:tasks.workunit.client.1.smithi173.stdout:4/966: fdatasync da/dc/d13/f2b 0 2024-04-20T11:09:46.428 INFO:tasks.workunit.client.1.smithi173.stdout:6/880: mknod d4/d24/d42/d74/df6/c110 0 2024-04-20T11:09:46.429 INFO:tasks.workunit.client.1.smithi173.stdout:8/893: creat d4/d18/d59/d5a/da8/db9/de0/d3c/f11c x:0 0 0 2024-04-20T11:09:46.431 INFO:tasks.workunit.client.1.smithi173.stdout:8/894: chown d4/d18/da6/fbf 13 1 2024-04-20T11:09:46.440 INFO:tasks.workunit.client.1.smithi173.stdout:0/855: dwrite d6/d8/df/d15/f70 [0,4194304] 0 2024-04-20T11:09:46.445 INFO:tasks.workunit.client.1.smithi173.stdout:0/856: creat d6/d8/df/d43/f10c x:0 0 0 2024-04-20T11:09:46.449 INFO:tasks.workunit.client.1.smithi173.stdout:6/881: dread d4/d24/d42/d81/ff4 [0,4194304] 0 2024-04-20T11:09:46.450 INFO:tasks.workunit.client.1.smithi173.stdout:3/829: sync 2024-04-20T11:09:46.451 INFO:tasks.workunit.client.1.smithi173.stdout:5/886: sync 2024-04-20T11:09:46.451 INFO:tasks.workunit.client.1.smithi173.stdout:9/849: sync 2024-04-20T11:09:46.451 INFO:tasks.workunit.client.1.smithi173.stdout:0/857: getdents d6/d8/df/d2c/d42/d50/d86/d8b/da2/de8 0 2024-04-20T11:09:46.455 INFO:tasks.workunit.client.1.smithi173.stdout:3/830: symlink d7/d17/d58/d8a/l107 0 2024-04-20T11:09:46.455 INFO:tasks.workunit.client.1.smithi173.stdout:6/882: write d4/d24/d42/d74/d3b/f41 [3147168,49892] 0 2024-04-20T11:09:46.456 INFO:tasks.workunit.client.1.smithi173.stdout:5/887: creat da/de/d19/dca/f134 x:0 0 0 2024-04-20T11:09:46.457 INFO:tasks.workunit.client.1.smithi173.stdout:3/831: fdatasync d7/d2b/d9d/db7/fe9 0 2024-04-20T11:09:46.459 INFO:tasks.workunit.client.1.smithi173.stdout:9/850: truncate d5/d9/d20/d2b/d106/f91 1854576 0 2024-04-20T11:09:46.461 INFO:tasks.workunit.client.1.smithi173.stdout:9/851: dread - d5/d9/d20/d29/d2d/dcb/fd5 zero size 2024-04-20T11:09:46.464 INFO:tasks.workunit.client.1.smithi173.stdout:3/832: mknod d7/d17/d58/d8a/dc4/dd1/c108 0 2024-04-20T11:09:46.465 INFO:tasks.workunit.client.1.smithi173.stdout:0/858: getdents d6/d8/d51 0 2024-04-20T11:09:46.467 INFO:tasks.workunit.client.1.smithi173.stdout:5/888: rename da/de/ff to da/de/d19/d32/d30/dbb/f135 0 2024-04-20T11:09:46.472 INFO:tasks.workunit.client.1.smithi173.stdout:3/833: mkdir d7/d17/d58/d8a/d109 0 2024-04-20T11:09:46.473 INFO:tasks.workunit.client.1.smithi173.stdout:6/883: write d4/d1c/f7d [784128,10217] 0 2024-04-20T11:09:46.475 INFO:tasks.workunit.client.1.smithi173.stdout:9/852: dwrite d5/d9/f1d [0,4194304] 0 2024-04-20T11:09:46.475 INFO:tasks.workunit.client.1.smithi173.stdout:5/889: mkdir da/d4f/d5e/dbc/d136 0 2024-04-20T11:09:46.476 INFO:tasks.workunit.client.1.smithi173.stdout:0/859: dread d6/d8/df/d43/d7f/fab [0,4194304] 0 2024-04-20T11:09:46.477 INFO:tasks.workunit.client.1.smithi173.stdout:9/853: readlink d5/d9/d20/d29/l22 0 2024-04-20T11:09:46.479 INFO:tasks.workunit.client.1.smithi173.stdout:9/854: fsync d5/d9/fc5 0 2024-04-20T11:09:46.480 INFO:tasks.workunit.client.1.smithi173.stdout:0/860: write d6/d8/d3c/d44/d75/f97 [6796844,49787] 0 2024-04-20T11:09:46.480 INFO:tasks.workunit.client.1.smithi173.stdout:5/890: dread da/de/d75/d3e/fed [0,4194304] 0 2024-04-20T11:09:46.482 INFO:tasks.workunit.client.1.smithi173.stdout:9/855: stat d5/d9/d20/d4f/d5e/d9e/l101 0 2024-04-20T11:09:46.483 INFO:tasks.workunit.client.1.smithi173.stdout:5/891: chown da/de/c2e 223010 1 2024-04-20T11:09:46.483 INFO:tasks.workunit.client.1.smithi173.stdout:5/892: stat da/de 0 2024-04-20T11:09:46.485 INFO:tasks.workunit.client.1.smithi173.stdout:6/884: creat d4/d24/f111 x:0 0 0 2024-04-20T11:09:46.486 INFO:tasks.workunit.client.1.smithi173.stdout:5/893: truncate da/de/d19/f120 158075 0 2024-04-20T11:09:46.487 INFO:tasks.workunit.client.1.smithi173.stdout:5/894: chown da/d9e/f105 7571904 1 2024-04-20T11:09:46.491 INFO:tasks.workunit.client.1.smithi173.stdout:0/861: dread d6/d8/df/d15/f70 [0,4194304] 0 2024-04-20T11:09:46.497 INFO:tasks.workunit.client.1.smithi173.stdout:3/834: write d7/d17/d61/d83/f88 [549112,78589] 0 2024-04-20T11:09:46.499 INFO:tasks.workunit.client.1.smithi173.stdout:9/856: dwrite d5/d9/d20/d2b/d59/d44/d78/f7c [0,4194304] 0 2024-04-20T11:09:46.502 INFO:tasks.workunit.client.1.smithi173.stdout:0/862: rmdir d6/d8/d3c/dee/d107 39 2024-04-20T11:09:46.502 INFO:tasks.workunit.client.1.smithi173.stdout:6/885: unlink d4/d1c/d8e/cd0 0 2024-04-20T11:09:46.504 INFO:tasks.workunit.client.1.smithi173.stdout:3/835: creat d7/d17/d61/d83/dc2/f10a x:0 0 0 2024-04-20T11:09:46.505 INFO:tasks.workunit.client.1.smithi173.stdout:0/863: stat d6/d8/d3c/d44/d75/l85 0 2024-04-20T11:09:46.507 INFO:tasks.workunit.client.1.smithi173.stdout:6/886: creat d4/d59/d8f/dd7/f112 x:0 0 0 2024-04-20T11:09:46.509 INFO:tasks.workunit.client.1.smithi173.stdout:3/836: symlink d7/d17/d61/dd6/l10b 0 2024-04-20T11:09:46.511 INFO:tasks.workunit.client.1.smithi173.stdout:3/837: fdatasync d7/d17/fa8 0 2024-04-20T11:09:46.513 INFO:tasks.workunit.client.1.smithi173.stdout:0/864: mknod d6/d8/df/d2c/d42/d50/d86/d8b/da2/de8/c10d 0 2024-04-20T11:09:46.517 INFO:tasks.workunit.client.1.smithi173.stdout:0/865: truncate d6/d8/df/f16 8596588 0 2024-04-20T11:09:46.518 INFO:tasks.workunit.client.1.smithi173.stdout:9/857: dwrite d5/d9/d20/d29/d18/d9b/f100 [0,4194304] 0 2024-04-20T11:09:46.524 INFO:tasks.workunit.client.1.smithi173.stdout:9/858: write d5/d9/d20/d2b/d31/f4d [3445994,123256] 0 2024-04-20T11:09:46.532 INFO:tasks.workunit.client.1.smithi173.stdout:6/887: dwrite d4/d14/d5f/d9f/fcb [0,4194304] 0 2024-04-20T11:09:46.532 INFO:tasks.workunit.client.1.smithi173.stdout:5/895: dwrite da/d4f/d5e/dbc/fc0 [0,4194304] 0 2024-04-20T11:09:46.536 INFO:tasks.workunit.client.1.smithi173.stdout:3/838: dread d7/d53/dab/ffb [0,4194304] 0 2024-04-20T11:09:46.539 INFO:tasks.workunit.client.1.smithi173.stdout:0/866: write d6/d8/df/d2c/d42/fa3 [719753,114945] 0 2024-04-20T11:09:46.544 INFO:tasks.workunit.client.1.smithi173.stdout:9/859: dwrite d5/d9/d20/d2b/d31/f4d [0,4194304] 0 2024-04-20T11:09:46.546 INFO:tasks.workunit.client.1.smithi173.stdout:2/975: sync 2024-04-20T11:09:46.547 INFO:tasks.workunit.client.1.smithi173.stdout:5/896: creat da/d34/d7f/da5/d9a/f137 x:0 0 0 2024-04-20T11:09:46.553 INFO:tasks.workunit.client.1.smithi173.stdout:3/839: link d7/ca3 d7/d17/d58/d8a/c10c 0 2024-04-20T11:09:46.554 INFO:tasks.workunit.client.1.smithi173.stdout:5/897: mkdir da/de/d138 0 2024-04-20T11:09:46.556 INFO:tasks.workunit.client.1.smithi173.stdout:3/840: chown d7/f51 276 1 2024-04-20T11:09:46.558 INFO:tasks.workunit.client.1.smithi173.stdout:5/898: creat da/d34/da7/f139 x:0 0 0 2024-04-20T11:09:46.560 INFO:tasks.workunit.client.1.smithi173.stdout:3/841: fdatasync d7/d53/dab/ffb 0 2024-04-20T11:09:46.564 INFO:tasks.workunit.client.1.smithi173.stdout:3/842: fdatasync d7/d17/d61/d83/dcf/d5c/fd9 0 2024-04-20T11:09:46.565 INFO:tasks.workunit.client.1.smithi173.stdout:0/867: dread d6/d8/d3c/f4a [0,4194304] 0 2024-04-20T11:09:46.566 INFO:tasks.workunit.client.1.smithi173.stdout:3/843: dread - d7/d17/d61/d83/ff4 zero size 2024-04-20T11:09:46.572 INFO:tasks.workunit.client.1.smithi173.stdout:2/976: write d2/de/f21 [2468558,30212] 0 2024-04-20T11:09:46.574 INFO:tasks.workunit.client.1.smithi173.stdout:0/868: symlink d6/d8/df/d2c/l10e 0 2024-04-20T11:09:46.578 INFO:tasks.workunit.client.1.smithi173.stdout:5/899: write da/de/d75/f89 [3155538,126529] 0 2024-04-20T11:09:46.580 INFO:tasks.workunit.client.1.smithi173.stdout:0/869: symlink d6/d8/d3c/d44/d69/de2/l10f 0 2024-04-20T11:09:46.581 INFO:tasks.workunit.client.1.smithi173.stdout:5/900: readlink da/d34/ldd 0 2024-04-20T11:09:46.581 INFO:tasks.workunit.client.1.smithi173.stdout:9/860: dwrite d5/f2f [0,4194304] 0 2024-04-20T11:09:46.582 INFO:tasks.workunit.client.1.smithi173.stdout:0/870: chown d6/d8/d3c/d44 8570284 1 2024-04-20T11:09:46.586 INFO:tasks.workunit.client.1.smithi173.stdout:5/901: rmdir da/d9e 39 2024-04-20T11:09:46.588 INFO:tasks.workunit.client.1.smithi173.stdout:3/844: dwrite d7/d17/d58/fdc [0,4194304] 0 2024-04-20T11:09:46.589 INFO:tasks.workunit.client.1.smithi173.stdout:5/902: dread - da/d34/d7f/f113 zero size 2024-04-20T11:09:46.592 INFO:tasks.workunit.client.1.smithi173.stdout:2/977: dwrite d2/d13/d9a/dbc/dfd/d4b/dae/f13f [0,4194304] 0 2024-04-20T11:09:46.594 INFO:tasks.workunit.client.1.smithi173.stdout:0/871: creat d6/d5f/d8f/db4/f110 x:0 0 0 2024-04-20T11:09:46.598 INFO:tasks.workunit.client.1.smithi173.stdout:9/861: creat d5/d9/d20/d2b/d106/d79/d80/f108 x:0 0 0 2024-04-20T11:09:46.598 INFO:tasks.workunit.client.1.smithi173.stdout:3/845: dread - d7/d2b/d8e/fd5 zero size 2024-04-20T11:09:46.598 INFO:tasks.workunit.client.1.smithi173.stdout:5/903: mknod da/de/d19/d2c/d106/c13a 0 2024-04-20T11:09:46.600 INFO:tasks.workunit.client.1.smithi173.stdout:3/846: chown d7/d17/d58/d8a/dc4/dd1 8591570 1 2024-04-20T11:09:46.610 INFO:tasks.workunit.client.1.smithi173.stdout:3/847: creat d7/d17/d58/d8a/dc4/f10d x:0 0 0 2024-04-20T11:09:46.611 INFO:tasks.workunit.client.1.smithi173.stdout:9/862: mknod d5/d9/d20/d29/d2d/c109 0 2024-04-20T11:09:46.612 INFO:tasks.workunit.client.1.smithi173.stdout:9/863: chown d5/d9/d20/d2b/d59 176 1 2024-04-20T11:09:46.613 INFO:tasks.workunit.client.1.smithi173.stdout:2/978: dread d2/d13/d9a/dbc/dfd/fe9 [0,4194304] 0 2024-04-20T11:09:46.614 INFO:tasks.workunit.client.1.smithi173.stdout:2/979: stat d2/d34/d81/dca 0 2024-04-20T11:09:46.615 INFO:tasks.workunit.client.1.smithi173.stdout:3/848: symlink d7/d2b/d9d/l10e 0 2024-04-20T11:09:46.616 INFO:tasks.workunit.client.1.smithi173.stdout:5/904: rmdir da/d4f/d5e/dbc/d136 0 2024-04-20T11:09:46.618 INFO:tasks.workunit.client.1.smithi173.stdout:9/864: symlink d5/d9/d20/d2b/d106/d79/d80/l10a 0 2024-04-20T11:09:46.619 INFO:tasks.workunit.client.1.smithi173.stdout:5/905: chown da/de/d75/d40/d59/caf 3147 1 2024-04-20T11:09:46.620 INFO:tasks.workunit.client.1.smithi173.stdout:2/980: symlink d2/l147 0 2024-04-20T11:09:46.621 INFO:tasks.workunit.client.1.smithi173.stdout:5/906: dread - da/d4f/d65/d8e/f9f zero size 2024-04-20T11:09:46.622 INFO:tasks.workunit.client.1.smithi173.stdout:2/981: chown d2/d34/cb7 3720761 1 2024-04-20T11:09:46.622 INFO:tasks.workunit.client.1.smithi173.stdout:7/895: sync 2024-04-20T11:09:46.622 INFO:tasks.workunit.client.1.smithi173.stdout:1/800: sync 2024-04-20T11:09:46.623 INFO:tasks.workunit.client.1.smithi173.stdout:3/849: rename d7/d17/d61/d83/fc8 to d7/d17/d61/d83/dcf/f10f 0 2024-04-20T11:09:46.626 INFO:tasks.workunit.client.1.smithi173.stdout:7/896: truncate d9/da/dc3/ffc 827724 0 2024-04-20T11:09:46.627 INFO:tasks.workunit.client.1.smithi173.stdout:2/982: mknod d2/de/d11/db8/d142/d87/db0/d141/c148 0 2024-04-20T11:09:46.628 INFO:tasks.workunit.client.1.smithi173.stdout:7/897: dread - d9/d33/d40/f11e zero size 2024-04-20T11:09:46.633 INFO:tasks.workunit.client.1.smithi173.stdout:2/983: mkdir d2/d13/d9a/dbc/dfd/d4b/dae/db6/df9/d111/d149 0 2024-04-20T11:09:46.634 INFO:tasks.workunit.client.1.smithi173.stdout:7/898: mkdir d9/da/d17/d131 0 2024-04-20T11:09:46.635 INFO:tasks.workunit.client.1.smithi173.stdout:0/872: rmdir d6 39 2024-04-20T11:09:46.638 INFO:tasks.workunit.client.1.smithi173.stdout:2/984: truncate d2/de/f60 916249 0 2024-04-20T11:09:46.641 INFO:tasks.workunit.client.1.smithi173.stdout:7/899: symlink d9/d33/de9/l132 0 2024-04-20T11:09:46.641 INFO:tasks.workunit.client.1.smithi173.stdout:1/801: dwrite d5/d6/d17/d42/d12/d1c/f6f [0,4194304] 0 2024-04-20T11:09:46.642 INFO:tasks.workunit.client.1.smithi173.stdout:0/873: chown d6/d8/df/laa 5 1 2024-04-20T11:09:46.645 INFO:tasks.workunit.client.1.smithi173.stdout:3/850: dwrite d7/d2b/d9d/db7/fe9 [0,4194304] 0 2024-04-20T11:09:46.654 INFO:tasks.workunit.client.1.smithi173.stdout:5/907: dread f7 [4194304,4194304] 0 2024-04-20T11:09:46.655 INFO:tasks.workunit.client.1.smithi173.stdout:2/985: fdatasync d2/d8c/fa2 0 2024-04-20T11:09:46.656 INFO:tasks.workunit.client.1.smithi173.stdout:7/900: rename d9/da/cab to d9/d1f/d7f/dce/db2/c133 0 2024-04-20T11:09:46.659 INFO:tasks.workunit.client.1.smithi173.stdout:2/986: chown d2/de/d11/db8/d142/d87/db0/d141/d4a/c107 13 1 2024-04-20T11:09:46.662 INFO:tasks.workunit.client.1.smithi173.stdout:1/802: rmdir d5/d6/d17/d42/da/df/d40/dc8 39 2024-04-20T11:09:46.663 INFO:tasks.workunit.client.1.smithi173.stdout:9/865: truncate d5/d9/d20/d29/d2d/d33/f6b 7446270 0 2024-04-20T11:09:46.665 INFO:tasks.workunit.client.1.smithi173.stdout:0/874: symlink d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/l111 0 2024-04-20T11:09:46.666 INFO:tasks.workunit.client.1.smithi173.stdout:4/967: sync 2024-04-20T11:09:46.667 INFO:tasks.workunit.client.1.smithi173.stdout:6/888: sync 2024-04-20T11:09:46.667 INFO:tasks.workunit.client.1.smithi173.stdout:8/895: sync 2024-04-20T11:09:46.670 INFO:tasks.workunit.client.1.smithi173.stdout:4/968: write da/dc/d13/fc6 [1161645,58055] 0 2024-04-20T11:09:46.673 INFO:tasks.workunit.client.1.smithi173.stdout:2/987: stat d2/fa9 0 2024-04-20T11:09:46.676 INFO:tasks.workunit.client.1.smithi173.stdout:6/889: read d4/d18/d7e/f8a [3431274,72852] 0 2024-04-20T11:09:46.677 INFO:tasks.workunit.client.1.smithi173.stdout:6/890: write d4/d14/f6a [1188304,21987] 0 2024-04-20T11:09:46.687 INFO:tasks.workunit.client.1.smithi173.stdout:4/969: creat da/d1e/d52/da3/f14b x:0 0 0 2024-04-20T11:09:46.691 INFO:tasks.workunit.client.1.smithi173.stdout:6/891: dwrite d4/f37 [0,4194304] 0 2024-04-20T11:09:46.696 INFO:tasks.workunit.client.1.smithi173.stdout:8/896: mknod d4/d18/d4f/d5b/d10a/c11d 0 2024-04-20T11:09:46.697 INFO:tasks.workunit.client.1.smithi173.stdout:0/875: fdatasync d6/f29 0 2024-04-20T11:09:46.698 INFO:tasks.workunit.client.1.smithi173.stdout:9/866: dread d5/d9/d20/d2b/fbe [0,4194304] 0 2024-04-20T11:09:46.700 INFO:tasks.workunit.client.1.smithi173.stdout:2/988: mknod d2/de/d11/db8/d142/d87/db0/d141/d4a/c14a 0 2024-04-20T11:09:46.701 INFO:tasks.workunit.client.1.smithi173.stdout:4/970: read - da/dc/d62/ff8 zero size 2024-04-20T11:09:46.703 INFO:tasks.workunit.client.1.smithi173.stdout:2/989: write d2/de/d11/db8/f139 [698766,81986] 0 2024-04-20T11:09:46.703 INFO:tasks.workunit.client.1.smithi173.stdout:1/803: creat d5/d6/d17/d42/da/df/d40/f10b x:0 0 0 2024-04-20T11:09:46.707 INFO:tasks.workunit.client.1.smithi173.stdout:3/851: getdents d7/d2b 0 2024-04-20T11:09:46.708 INFO:tasks.workunit.client.1.smithi173.stdout:5/908: creat da/d9e/f13b x:0 0 0 2024-04-20T11:09:46.709 INFO:tasks.workunit.client.1.smithi173.stdout:8/897: fdatasync d4/d18/d4f/f9a 0 2024-04-20T11:09:46.712 INFO:tasks.workunit.client.1.smithi173.stdout:7/901: getdents d9/da/dc3 0 2024-04-20T11:09:46.712 INFO:tasks.workunit.client.1.smithi173.stdout:0/876: fsync d6/d8/d3c/dee/d107/f100 0 2024-04-20T11:09:46.712 INFO:tasks.workunit.client.1.smithi173.stdout:3/852: dread - d7/d17/d61/d83/dcf/d5c/fdb zero size 2024-04-20T11:09:46.714 INFO:tasks.workunit.client.1.smithi173.stdout:3/853: write d7/d17/f26 [4518484,49486] 0 2024-04-20T11:09:46.716 INFO:tasks.workunit.client.1.smithi173.stdout:4/971: creat da/d1a/d60/d7d/d101/f14c x:0 0 0 2024-04-20T11:09:46.716 INFO:tasks.workunit.client.1.smithi173.stdout:7/902: write d9/d33/de9/d128/f125 [147346,50970] 0 2024-04-20T11:09:46.717 INFO:tasks.workunit.client.1.smithi173.stdout:2/990: rename d2/d13/d9a/dbc/dfd/d4b/dae to d2/d13/d9a/dbc/dfd/d4b/dae/d14b 22 2024-04-20T11:09:46.720 INFO:tasks.workunit.client.1.smithi173.stdout:0/877: dread - d6/d8/fbb zero size 2024-04-20T11:09:46.722 INFO:tasks.workunit.client.1.smithi173.stdout:6/892: creat d4/d18/f113 x:0 0 0 2024-04-20T11:09:46.722 INFO:tasks.workunit.client.1.smithi173.stdout:9/867: dwrite d5/d9/d20/d29/d18/d9b/f100 [0,4194304] 0 2024-04-20T11:09:46.723 INFO:tasks.workunit.client.1.smithi173.stdout:1/804: unlink d5/d6/d3b/caf 0 2024-04-20T11:09:46.724 INFO:tasks.workunit.client.1.smithi173.stdout:4/972: dread da/dc/d62/f10a [0,4194304] 0 2024-04-20T11:09:46.726 INFO:tasks.workunit.client.1.smithi173.stdout:9/868: fdatasync d5/d9/d20/d29/d2d/f3f 0 2024-04-20T11:09:46.726 INFO:tasks.workunit.client.1.smithi173.stdout:3/854: mknod d7/d17/d58/d8a/dc4/c110 0 2024-04-20T11:09:46.732 INFO:tasks.workunit.client.1.smithi173.stdout:6/893: rename d4/d24/cc8 to d4/d59/d8f/d107/c114 0 2024-04-20T11:09:46.733 INFO:tasks.workunit.client.1.smithi173.stdout:7/903: read d9/da/d17/d2b/f80 [289587,36029] 0 2024-04-20T11:09:46.734 INFO:tasks.workunit.client.1.smithi173.stdout:5/909: dwrite da/d4f/fd3 [0,4194304] 0 2024-04-20T11:09:46.735 INFO:tasks.workunit.client.1.smithi173.stdout:4/973: mknod da/dc/d15/d8b/c14d 0 2024-04-20T11:09:46.739 INFO:tasks.workunit.client.1.smithi173.stdout:2/991: dwrite d2/ff0 [0,4194304] 0 2024-04-20T11:09:46.743 INFO:tasks.workunit.client.1.smithi173.stdout:9/869: unlink d5/d9/d20/d2b/d59/da1/fa3 0 2024-04-20T11:09:46.744 INFO:tasks.workunit.client.1.smithi173.stdout:1/805: dread d5/d6/d3b/f98 [0,4194304] 0 2024-04-20T11:09:46.747 INFO:tasks.workunit.client.1.smithi173.stdout:9/870: chown d5/d9/d20/d4f/d5e/d9e/fa6 1194251 1 2024-04-20T11:09:46.850 INFO:tasks.workunit.client.1.smithi173.stdout:3/855: rename d7/d53/fc1 to d7/d17/d61/d90/dca/f111 0 2024-04-20T11:09:46.851 INFO:tasks.workunit.client.1.smithi173.stdout:8/898: write d4/ff [2078779,17180] 0 2024-04-20T11:09:46.852 INFO:tasks.workunit.client.1.smithi173.stdout:8/899: stat d4/l3d 0 2024-04-20T11:09:46.852 INFO:tasks.workunit.client.1.smithi173.stdout:3/856: stat d7/d17/d61/d83/d45/lb0 0 2024-04-20T11:09:46.853 INFO:tasks.workunit.client.1.smithi173.stdout:7/904: write d9/d33/d40/d89/f8a [264460,69666] 0 2024-04-20T11:09:46.855 INFO:tasks.workunit.client.1.smithi173.stdout:7/905: chown d9/da/d17/d2b 2668 1 2024-04-20T11:09:46.857 INFO:tasks.workunit.client.1.smithi173.stdout:9/871: truncate d5/d9/d20/d2b/d59/d44/d78/f7c 4677062 0 2024-04-20T11:09:46.861 INFO:tasks.workunit.client.1.smithi173.stdout:6/894: dwrite d4/d14/d79/fbc [0,4194304] 0 2024-04-20T11:09:46.862 INFO:tasks.workunit.client.1.smithi173.stdout:6/895: chown d4/d14/f3c 15903287 1 2024-04-20T11:09:46.872 INFO:tasks.workunit.client.1.smithi173.stdout:7/906: dwrite d9/da/dc3/ffc [0,4194304] 0 2024-04-20T11:09:46.877 INFO:tasks.workunit.client.1.smithi173.stdout:6/896: dwrite d4/d14/f1a [0,4194304] 0 2024-04-20T11:09:46.879 INFO:tasks.workunit.client.1.smithi173.stdout:6/897: readlink d4/d24/d42/d81/lb8 0 2024-04-20T11:09:46.882 INFO:tasks.workunit.client.1.smithi173.stdout:6/898: chown d4/d18/d61/ffe 1 1 2024-04-20T11:09:46.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:09:46.904 INFO:tasks.workunit.client.1.smithi173.stdout:4/974: symlink da/dc/l14e 0 2024-04-20T11:09:46.907 INFO:tasks.workunit.client.1.smithi173.stdout:1/806: fsync d5/d6/f73 0 2024-04-20T11:09:46.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:09:46.921 INFO:tasks.workunit.client.1.smithi173.stdout:5/910: rename da/de/d75/c83 to da/de/d75/d40/d59/c13c 0 2024-04-20T11:09:46.922 INFO:tasks.workunit.client.1.smithi173.stdout:4/975: dwrite da/d1e/d52/f97 [0,4194304] 0 2024-04-20T11:09:46.931 INFO:tasks.workunit.client.1.smithi173.stdout:6/899: symlink d4/d1c/d8e/d90/l115 0 2024-04-20T11:09:46.932 INFO:tasks.workunit.client.1.smithi173.stdout:1/807: dread d5/d6/d17/d42/f39 [0,4194304] 0 2024-04-20T11:09:46.935 INFO:tasks.workunit.client.1.smithi173.stdout:1/808: fsync d5/d6/d17/d42/da/fe3 0 2024-04-20T11:09:46.937 INFO:tasks.workunit.client.1.smithi173.stdout:4/976: creat da/dc/df4/dff/f14f x:0 0 0 2024-04-20T11:09:46.937 INFO:tasks.workunit.client.1.smithi173.stdout:9/872: dwrite d5/faf [0,4194304] 0 2024-04-20T11:09:46.939 INFO:tasks.workunit.client.1.smithi173.stdout:3/857: dread d7/d17/f57 [0,4194304] 0 2024-04-20T11:09:46.941 INFO:tasks.workunit.client.1.smithi173.stdout:5/911: dwrite da/f3d [0,4194304] 0 2024-04-20T11:09:46.941 INFO:tasks.workunit.client.1.smithi173.stdout:3/858: dread - d7/d2b/d8e/fd5 zero size 2024-04-20T11:09:46.941 INFO:tasks.workunit.client.1.smithi173.stdout:7/907: truncate d9/d1f/f7d 3365744 0 2024-04-20T11:09:46.943 INFO:tasks.workunit.client.1.smithi173.stdout:0/878: sync 2024-04-20T11:09:46.945 INFO:tasks.workunit.client.1.smithi173.stdout:9/873: chown d5/d9/d20/d2b/d106/d79/d80/l10a 4487920 1 2024-04-20T11:09:46.945 INFO:tasks.workunit.client.1.smithi173.stdout:6/900: mkdir d4/d24/d42/d74/d3b/d116 0 2024-04-20T11:09:46.946 INFO:tasks.workunit.client.1.smithi173.stdout:2/992: rename d2/d34/f45 to d2/de/d11/db8/d142/f14c 0 2024-04-20T11:09:46.949 INFO:tasks.workunit.client.1.smithi173.stdout:1/809: mkdir d5/d6/d17/d42/d10c 0 2024-04-20T11:09:46.950 INFO:tasks.workunit.client.1.smithi173.stdout:4/977: mkdir da/d1e/d52/d10d/d150 0 2024-04-20T11:09:46.950 INFO:tasks.workunit.client.1.smithi173.stdout:3/859: fsync d7/d17/d61/d83/dcf/f4f 0 2024-04-20T11:09:46.952 INFO:tasks.workunit.client.1.smithi173.stdout:7/908: mknod d9/d33/de9/c134 0 2024-04-20T11:09:46.954 INFO:tasks.workunit.client.1.smithi173.stdout:7/909: chown d9/da 0 1 2024-04-20T11:09:46.954 INFO:tasks.workunit.client.1.smithi173.stdout:8/900: rename d4/d18/d59/d5a/da8/db9/fed to d4/d18/d59/d5a/d7b/db1/f11e 0 2024-04-20T11:09:46.954 INFO:tasks.workunit.client.1.smithi173.stdout:3/860: fdatasync d7/d17/d61/f9b 0 2024-04-20T11:09:46.955 INFO:tasks.workunit.client.1.smithi173.stdout:6/901: symlink d4/d59/d8f/d107/l117 0 2024-04-20T11:09:46.957 INFO:tasks.workunit.client.1.smithi173.stdout:2/993: mknod d2/de/d11/db8/d142/d87/db0/c14d 0 2024-04-20T11:09:46.958 INFO:tasks.workunit.client.1.smithi173.stdout:9/874: mkdir d5/d9/d20/d4f/d5e/d9e/dc2/d10b 0 2024-04-20T11:09:46.963 INFO:tasks.workunit.client.1.smithi173.stdout:6/902: creat d4/d59/d8f/f118 x:0 0 0 2024-04-20T11:09:46.964 INFO:tasks.workunit.client.1.smithi173.stdout:1/810: rename d5/d6/d17/d42/da/d20/dad/dc2/d94/fc1 to d5/d6/d17/d42/d12/dc9/f10d 0 2024-04-20T11:09:46.965 INFO:tasks.workunit.client.1.smithi173.stdout:5/912: write da/de/f121 [848,31599] 0 2024-04-20T11:09:46.966 INFO:tasks.workunit.client.1.smithi173.stdout:8/901: creat d4/d18/d59/d5a/da8/db9/f11f x:0 0 0 2024-04-20T11:09:46.978 INFO:tasks.workunit.client.1.smithi173.stdout:9/875: read d5/d9/d20/d2b/d59/d48/fe2 [292775,123679] 0 2024-04-20T11:09:46.981 INFO:tasks.workunit.client.1.smithi173.stdout:1/811: rmdir d5/d6/d3b 39 2024-04-20T11:09:46.987 INFO:tasks.workunit.client.1.smithi173.stdout:8/902: creat d4/d18/d59/d5a/d7b/d8c/f120 x:0 0 0 2024-04-20T11:09:46.989 INFO:tasks.workunit.client.1.smithi173.stdout:3/861: rename d7/d17/d58/df3/l104 to d7/d17/d58/d8a/d109/l112 0 2024-04-20T11:09:46.992 INFO:tasks.workunit.client.1.smithi173.stdout:7/910: dread d9/d1f/d72/f75 [0,4194304] 0 2024-04-20T11:09:46.992 INFO:tasks.workunit.client.1.smithi173.stdout:4/978: dread da/dc/d62/f76 [0,4194304] 0 2024-04-20T11:09:46.993 INFO:tasks.workunit.client.1.smithi173.stdout:0/879: dread d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/f7d [0,4194304] 0 2024-04-20T11:09:46.997 INFO:tasks.workunit.client.1.smithi173.stdout:3/862: symlink d7/d2b/d9d/l113 0 2024-04-20T11:09:46.999 INFO:tasks.workunit.client.1.smithi173.stdout:8/903: chown d4/d18/d59/d5a/da8/cd0 4183 1 2024-04-20T11:09:47.000 INFO:tasks.workunit.client.1.smithi173.stdout:9/876: dread d5/d9/d20/d2b/d106/d79/f7a [0,4194304] 0 2024-04-20T11:09:47.000 INFO:tasks.workunit.client.1.smithi173.stdout:4/979: unlink da/dc/d62/f109 0 2024-04-20T11:09:47.002 INFO:tasks.workunit.client.1.smithi173.stdout:0/880: chown d6/d8/df/d2c/d2d/f6a 0 1 2024-04-20T11:09:47.004 INFO:tasks.workunit.client.1.smithi173.stdout:3/863: rmdir d7/d85/df5 39 2024-04-20T11:09:47.005 INFO:tasks.workunit.client.1.smithi173.stdout:0/881: dread - d6/d8/d51/fc5 zero size 2024-04-20T11:09:47.006 INFO:tasks.workunit.client.1.smithi173.stdout:6/903: dwrite d4/d18/d7e/f8a [0,4194304] 0 2024-04-20T11:09:47.006 INFO:tasks.workunit.client.1.smithi173.stdout:2/994: dread d2/d13/d9a/dbc/dfd/f73 [0,4194304] 0 2024-04-20T11:09:47.007 INFO:tasks.workunit.client.1.smithi173.stdout:3/864: dread - d7/d2b/ffd zero size 2024-04-20T11:09:47.008 INFO:tasks.workunit.client.1.smithi173.stdout:1/812: rename d5/d6/d3b/l5a to d5/l10e 0 2024-04-20T11:09:47.008 INFO:tasks.workunit.client.1.smithi173.stdout:5/913: dwrite da/d4f/d65/fe3 [0,4194304] 0 2024-04-20T11:09:47.010 INFO:tasks.workunit.client.1.smithi173.stdout:2/995: write d2/d34/d58/f5a [8747152,3597] 0 2024-04-20T11:09:47.014 INFO:tasks.workunit.client.1.smithi173.stdout:9/877: rmdir d5/d9/d20/d4f/d5e/d7d 39 2024-04-20T11:09:47.015 INFO:tasks.workunit.client.1.smithi173.stdout:6/904: write d4/d24/d3e/d9c/df2/f10c [142388,81709] 0 2024-04-20T11:09:47.016 INFO:tasks.workunit.client.1.smithi173.stdout:8/904: chown d4/d18/d59/d5a/da8/db9/de0/c79 0 1 2024-04-20T11:09:47.016 INFO:tasks.workunit.client.1.smithi173.stdout:3/865: chown d7/d17/d61/dd6/ffc 7332 1 2024-04-20T11:09:47.018 INFO:tasks.workunit.client.1.smithi173.stdout:7/911: getdents d9/d1f/d72/d8f 0 2024-04-20T11:09:47.026 INFO:tasks.workunit.client.1.smithi173.stdout:7/912: dread - d9/d33/d40/d11c/f12e zero size 2024-04-20T11:09:47.029 INFO:tasks.workunit.client.1.smithi173.stdout:8/905: mkdir d4/d18/d4f/d121 0 2024-04-20T11:09:47.030 INFO:tasks.workunit.client.1.smithi173.stdout:0/882: dread d6/f29 [0,4194304] 0 2024-04-20T11:09:47.032 INFO:tasks.workunit.client.1.smithi173.stdout:9/878: rmdir d5/dcc/d107 0 2024-04-20T11:09:47.033 INFO:tasks.workunit.client.1.smithi173.stdout:7/913: mkdir d9/d33/d35/d54/d135 0 2024-04-20T11:09:47.040 INFO:tasks.workunit.client.1.smithi173.stdout:0/883: stat d6/d8/df/d15/lc4 0 2024-04-20T11:09:47.041 INFO:tasks.workunit.client.1.smithi173.stdout:3/866: getdents d7/d17/d58/d8a/dc4 0 2024-04-20T11:09:47.042 INFO:tasks.workunit.client.1.smithi173.stdout:8/906: creat d4/d18/d4f/d5b/de4/de6/f122 x:0 0 0 2024-04-20T11:09:47.047 INFO:tasks.workunit.client.1.smithi173.stdout:4/980: dwrite da/d1a/f40 [0,4194304] 0 2024-04-20T11:09:47.049 INFO:tasks.workunit.client.1.smithi173.stdout:2/996: dwrite d2/d34/fdd [0,4194304] 0 2024-04-20T11:09:47.051 INFO:tasks.workunit.client.1.smithi173.stdout:3/867: creat d7/d2b/d8e/f114 x:0 0 0 2024-04-20T11:09:47.051 INFO:tasks.workunit.client.1.smithi173.stdout:1/813: dread d5/d6/d17/d42/da/d20/f87 [0,4194304] 0 2024-04-20T11:09:47.052 INFO:tasks.workunit.client.1.smithi173.stdout:7/914: symlink d9/d33/d35/d54/db3/dfd/l136 0 2024-04-20T11:09:47.057 INFO:tasks.workunit.client.1.smithi173.stdout:3/868: dread - d7/d17/d61/d83/d45/f56 zero size 2024-04-20T11:09:47.062 INFO:tasks.workunit.client.1.smithi173.stdout:4/981: rename da/d1e/d52 to da/dc/d15/d8b/d151 0 2024-04-20T11:09:47.063 INFO:tasks.workunit.client.1.smithi173.stdout:8/907: symlink d4/d18/d59/d5a/da8/db9/de0/d3c/d86/db7/l123 0 2024-04-20T11:09:47.065 INFO:tasks.workunit.client.1.smithi173.stdout:2/997: fsync d2/d34/d58/f6b 0 2024-04-20T11:09:47.070 INFO:tasks.workunit.client.1.smithi173.stdout:1/814: truncate d5/d6/d17/d42/da/f6a 1285021 0 2024-04-20T11:09:47.070 INFO:tasks.workunit.client.1.smithi173.stdout:7/915: symlink d9/d33/d35/d4a/d52/df2/dc6/df3/l137 0 2024-04-20T11:09:47.072 INFO:tasks.workunit.client.1.smithi173.stdout:6/905: dread d4/d24/d3e/d9c/fb1 [0,4194304] 0 2024-04-20T11:09:47.078 INFO:tasks.workunit.client.1.smithi173.stdout:4/982: mkdir da/dc/ddd/d140/d152 0 2024-04-20T11:09:47.079 INFO:tasks.workunit.client.1.smithi173.stdout:2/998: mknod d2/de/d11/db8/d142/d87/db0/c14e 0 2024-04-20T11:09:47.081 INFO:tasks.workunit.client.1.smithi173.stdout:0/884: dwrite d6/d8/d51/fce [0,4194304] 0 2024-04-20T11:09:47.081 INFO:tasks.workunit.client.1.smithi173.stdout:8/908: dread d4/d18/fcc [0,4194304] 0 2024-04-20T11:09:47.083 INFO:tasks.workunit.client.1.smithi173.stdout:6/906: mknod d4/d59/d8f/c119 0 2024-04-20T11:09:47.091 INFO:tasks.workunit.client.1.smithi173.stdout:7/916: fsync d9/fa0 0 2024-04-20T11:09:47.091 INFO:tasks.workunit.client.1.smithi173.stdout:4/983: fsync da/fc4 0 2024-04-20T11:09:47.092 INFO:tasks.workunit.client.1.smithi173.stdout:1/815: truncate d5/d6/d17/d42/da/df/fce 856366 0 2024-04-20T11:09:47.096 INFO:tasks.workunit.client.1.smithi173.stdout:3/869: dwrite d7/d17/f3c [0,4194304] 0 2024-04-20T11:09:47.097 INFO:tasks.workunit.client.1.smithi173.stdout:8/909: creat d4/d18/d4f/d5b/de4/de6/f124 x:0 0 0 2024-04-20T11:09:47.100 INFO:tasks.workunit.client.1.smithi173.stdout:7/917: read d9/da/d17/d2b/f80 [229224,48846] 0 2024-04-20T11:09:47.103 INFO:tasks.workunit.client.1.smithi173.stdout:6/907: read d4/d24/d3e/f8d [856209,67072] 0 2024-04-20T11:09:47.103 INFO:tasks.workunit.client.1.smithi173.stdout:9/879: dread d5/d9/d20/d29/d2d/f43 [0,4194304] 0 2024-04-20T11:09:47.107 INFO:tasks.workunit.client.1.smithi173.stdout:2/999: dwrite d2/f5c [0,4194304] 0 2024-04-20T11:09:47.112 INFO:tasks.workunit.client.1.smithi173.stdout:1/816: fsync d5/d6/d17/d42/d12/dc9/f10d 0 2024-04-20T11:09:47.113 INFO:tasks.workunit.client.1.smithi173.stdout:6/908: dread - d4/d24/d42/d74/d3b/fa5 zero size 2024-04-20T11:09:47.113 INFO:tasks.workunit.client.1.smithi173.stdout:9/880: dread - d5/d9/d20/d2b/d106/d79/d80/f8d zero size 2024-04-20T11:09:47.117 INFO:tasks.workunit.client.1.smithi173.stdout:3/870: creat d7/d17/d61/d83/d45/f115 x:0 0 0 2024-04-20T11:09:47.119 INFO:tasks.workunit.client.1.smithi173.stdout:7/918: dread d9/f4e [0,4194304] 0 2024-04-20T11:09:47.121 INFO:tasks.workunit.client.1.smithi173.stdout:0/885: getdents d6/d8/d3c/dee/d107 0 2024-04-20T11:09:47.122 INFO:tasks.workunit.client.1.smithi173.stdout:1/817: dread - d5/d6/d17/d42/d12/fd7 zero size 2024-04-20T11:09:47.124 INFO:tasks.workunit.client.1.smithi173.stdout:6/909: mknod d4/d24/d3e/d9c/df2/c11a 0 2024-04-20T11:09:47.125 INFO:tasks.workunit.client.1.smithi173.stdout:3/871: rmdir d7/d17/d58/d8a/dc4 39 2024-04-20T11:09:47.129 INFO:tasks.workunit.client.1.smithi173.stdout:7/919: unlink d9/d33/d35/d4a/c69 0 2024-04-20T11:09:47.130 INFO:tasks.workunit.client.1.smithi173.stdout:1/818: readlink d5/d6/d17/l9f 0 2024-04-20T11:09:47.132 INFO:tasks.workunit.client.1.smithi173.stdout:0/886: rename d6/d8/d3c/d44/d69/de2 to d6/d8/d3c/dee/d107/d112 0 2024-04-20T11:09:47.135 INFO:tasks.workunit.client.1.smithi173.stdout:6/910: mkdir d4/d18/d7e/d11b 0 2024-04-20T11:09:47.139 INFO:tasks.workunit.client.1.smithi173.stdout:3/872: creat d7/d17/d61/d83/dc2/f116 x:0 0 0 2024-04-20T11:09:47.140 INFO:tasks.workunit.client.1.smithi173.stdout:4/984: dread da/dc/d15/f1d [4194304,4194304] 0 2024-04-20T11:09:47.142 INFO:tasks.workunit.client.1.smithi173.stdout:7/920: rmdir d9/d33/d81/d124 39 2024-04-20T11:09:47.144 INFO:tasks.workunit.client.1.smithi173.stdout:7/921: fdatasync d9/d33/d81/fe6 0 2024-04-20T11:09:47.144 INFO:tasks.workunit.client.1.smithi173.stdout:9/881: truncate d5/d9/d20/d2b/d106/f52 1398248 0 2024-04-20T11:09:47.146 INFO:tasks.workunit.client.1.smithi173.stdout:0/887: write d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/f7d [1676237,14949] 0 2024-04-20T11:09:47.147 INFO:tasks.workunit.client.1.smithi173.stdout:8/910: dread d4/d18/d59/d5a/da8/db9/de0/d3c/f52 [0,4194304] 0 2024-04-20T11:09:47.148 INFO:tasks.workunit.client.1.smithi173.stdout:6/911: creat d4/d24/d3e/d9c/dc7/f11c x:0 0 0 2024-04-20T11:09:47.149 INFO:tasks.workunit.client.1.smithi173.stdout:3/873: chown d7/d17/d58/d8a/dc4/dd1 435532688 1 2024-04-20T11:09:47.151 INFO:tasks.workunit.client.1.smithi173.stdout:4/985: creat da/dc/d13/f153 x:0 0 0 2024-04-20T11:09:47.152 INFO:tasks.workunit.client.1.smithi173.stdout:8/911: stat d4/d18/d59/d5a/da8/db9/de0/d9b/dc8/ff9 0 2024-04-20T11:09:47.158 INFO:tasks.workunit.client.1.smithi173.stdout:0/888: fsync d6/d8/d3c/dee/d107/d112/fbd 0 2024-04-20T11:09:47.159 INFO:tasks.workunit.client.1.smithi173.stdout:3/874: rmdir d7/d17/d61/d83/dc2 39 2024-04-20T11:09:47.162 INFO:tasks.workunit.client.1.smithi173.stdout:3/875: dread - d7/d2b/d8e/fd5 zero size 2024-04-20T11:09:47.163 INFO:tasks.workunit.client.1.smithi173.stdout:9/882: dwrite d5/d9/d20/d2b/d59/fd8 [0,4194304] 0 2024-04-20T11:09:47.163 INFO:tasks.workunit.client.1.smithi173.stdout:4/986: truncate da/f64 1343344 0 2024-04-20T11:09:47.165 INFO:tasks.workunit.client.1.smithi173.stdout:1/819: creat d5/d6/d17/d42/da/df/d40/f10f x:0 0 0 2024-04-20T11:09:47.166 INFO:tasks.workunit.client.1.smithi173.stdout:6/912: dwrite d4/d24/d42/f67 [0,4194304] 0 2024-04-20T11:09:47.169 INFO:tasks.workunit.client.1.smithi173.stdout:4/987: write da/dc/d103/ff9 [1862506,108710] 0 2024-04-20T11:09:47.171 INFO:tasks.workunit.client.1.smithi173.stdout:0/889: fsync d6/d8/df/d2c/d2d/f8a 0 2024-04-20T11:09:47.174 INFO:tasks.workunit.client.1.smithi173.stdout:6/913: fsync d4/d24/d42/f72 0 2024-04-20T11:09:47.175 INFO:tasks.workunit.client.1.smithi173.stdout:9/883: write d5/f6 [716003,21257] 0 2024-04-20T11:09:47.181 INFO:tasks.workunit.client.1.smithi173.stdout:4/988: mkdir da/dc/d13/d154 0 2024-04-20T11:09:47.182 INFO:tasks.workunit.client.1.smithi173.stdout:1/820: mkdir d5/d6/d17/d42/da/d110 0 2024-04-20T11:09:47.184 INFO:tasks.workunit.client.1.smithi173.stdout:3/876: rename d7/d2b/lc0 to d7/d17/d61/d83/dc2/l117 0 2024-04-20T11:09:47.184 INFO:tasks.workunit.client.1.smithi173.stdout:6/914: truncate d4/d24/d42/d74/df6/ff7 912313 0 2024-04-20T11:09:47.187 INFO:tasks.workunit.client.1.smithi173.stdout:7/922: dwrite d9/d33/f47 [0,4194304] 0 2024-04-20T11:09:47.187 INFO:tasks.workunit.client.1.smithi173.stdout:4/989: dread - da/d1e/fb2 zero size 2024-04-20T11:09:47.191 INFO:tasks.workunit.client.1.smithi173.stdout:1/821: chown d5/d6/d17/d42/c10 12836080 1 2024-04-20T11:09:47.191 INFO:tasks.workunit.client.1.smithi173.stdout:4/990: dread - da/dc/d103/f12c zero size 2024-04-20T11:09:47.196 INFO:tasks.workunit.client.1.smithi173.stdout:0/890: dwrite d6/d8/d3c/dee/d107/ffc [0,4194304] 0 2024-04-20T11:09:47.201 INFO:tasks.workunit.client.1.smithi173.stdout:3/877: mkdir d7/d17/d61/d83/dcf/d118 0 2024-04-20T11:09:47.202 INFO:tasks.workunit.client.1.smithi173.stdout:8/912: dwrite d4/d18/f48 [0,4194304] 0 2024-04-20T11:09:47.205 INFO:tasks.workunit.client.1.smithi173.stdout:0/891: truncate d6/d8/fbb 492119 0 2024-04-20T11:09:47.208 INFO:tasks.workunit.client.1.smithi173.stdout:4/991: symlink da/dc/d15/d8b/d151/l155 0 2024-04-20T11:09:47.211 INFO:tasks.workunit.client.1.smithi173.stdout:0/892: readlink d6/d8/d3c/d44/d75/l85 0 2024-04-20T11:09:47.211 INFO:tasks.workunit.client.1.smithi173.stdout:6/915: getdents d4/d18/d61 0 2024-04-20T11:09:47.213 INFO:tasks.workunit.client.1.smithi173.stdout:3/878: creat d7/d17/d58/d8a/dc4/f119 x:0 0 0 2024-04-20T11:09:47.214 INFO:tasks.workunit.client.1.smithi173.stdout:0/893: chown d6/d8/df/d15/l3e 0 1 2024-04-20T11:09:47.216 INFO:tasks.workunit.client.1.smithi173.stdout:7/923: getdents d9/d33/d40/d7a 0 2024-04-20T11:09:47.217 INFO:tasks.workunit.client.1.smithi173.stdout:4/992: truncate da/dc/df4/fd9 672931 0 2024-04-20T11:09:47.218 INFO:tasks.workunit.client.1.smithi173.stdout:9/884: dread d5/d9/d20/d29/d2d/f3f [0,4194304] 0 2024-04-20T11:09:47.221 INFO:tasks.workunit.client.1.smithi173.stdout:1/822: write d5/d6/d17/d42/da/df/d40/dac/fbd [892342,55976] 0 2024-04-20T11:09:47.225 INFO:tasks.workunit.client.1.smithi173.stdout:3/879: dread d7/d17/d61/d83/fbf [0,4194304] 0 2024-04-20T11:09:47.226 INFO:tasks.workunit.client.1.smithi173.stdout:6/916: rename d4/d24/d42/d74/fd1 to d4/d59/d8f/dd7/d108/f11d 0 2024-04-20T11:09:47.233 INFO:tasks.workunit.client.1.smithi173.stdout:5/914: sync 2024-04-20T11:09:47.235 INFO:tasks.workunit.client.1.smithi173.stdout:9/885: mknod d5/d9/d20/d29/c10c 0 2024-04-20T11:09:47.236 INFO:tasks.workunit.client.1.smithi173.stdout:9/886: fsync d5/d9/f6e 0 2024-04-20T11:09:47.239 INFO:tasks.workunit.client.1.smithi173.stdout:0/894: write d6/d8/df/d2c/d42/f108 [579741,81938] 0 2024-04-20T11:09:47.243 INFO:tasks.workunit.client.1.smithi173.stdout:1/823: write d5/d6/d17/d42/da/df/d40/f99 [569829,118852] 0 2024-04-20T11:09:47.245 INFO:tasks.workunit.client.1.smithi173.stdout:9/887: unlink d5/d9/d20/d29/d2d/le7 0 2024-04-20T11:09:47.246 INFO:tasks.workunit.client.1.smithi173.stdout:8/913: dread d4/d18/f6b [0,4194304] 0 2024-04-20T11:09:47.247 INFO:tasks.workunit.client.1.smithi173.stdout:6/917: write d4/d24/d3e/d9c/dc7/fcc [4798374,30291] 0 2024-04-20T11:09:47.247 INFO:tasks.workunit.client.1.smithi173.stdout:9/888: fdatasync d5/d9/f1a 0 2024-04-20T11:09:47.248 INFO:tasks.workunit.client.1.smithi173.stdout:1/824: write d5/d6/d17/d42/da/d20/dad/dc2/d94/fea [3512311,65814] 0 2024-04-20T11:09:47.248 INFO:tasks.workunit.client.1.smithi173.stdout:3/880: creat d7/f11a x:0 0 0 2024-04-20T11:09:47.250 INFO:tasks.workunit.client.1.smithi173.stdout:6/918: fsync d4/d1c/ffb 0 2024-04-20T11:09:47.253 INFO:tasks.workunit.client.1.smithi173.stdout:4/993: getdents da/dc/d15/d8b/d151 0 2024-04-20T11:09:47.255 INFO:tasks.workunit.client.1.smithi173.stdout:6/919: truncate d4/d24/f111 410489 0 2024-04-20T11:09:47.258 INFO:tasks.workunit.client.1.smithi173.stdout:0/895: dread d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/ff0 [4194304,4194304] 0 2024-04-20T11:09:47.259 INFO:tasks.workunit.client.1.smithi173.stdout:8/914: creat d4/d18/d59/d5a/d7b/db1/dce/ddd/f125 x:0 0 0 2024-04-20T11:09:47.259 INFO:tasks.workunit.client.1.smithi173.stdout:3/881: rmdir d7 39 2024-04-20T11:09:47.259 INFO:tasks.workunit.client.1.smithi173.stdout:7/924: dread d9/d33/d40/f49 [0,4194304] 0 2024-04-20T11:09:47.264 INFO:tasks.workunit.client.1.smithi173.stdout:4/994: creat da/dc/df4/d53/d39/f156 x:0 0 0 2024-04-20T11:09:47.267 INFO:tasks.workunit.client.1.smithi173.stdout:9/889: link d5/faf d5/d9/d20/d29/d2d/d33/f10d 0 2024-04-20T11:09:47.268 INFO:tasks.workunit.client.1.smithi173.stdout:8/915: fsync d4/d18/d59/d5a/da8/db9/de0/f7a 0 2024-04-20T11:09:47.269 INFO:tasks.workunit.client.1.smithi173.stdout:5/915: dwrite da/de/d19/f2a [4194304,4194304] 0 2024-04-20T11:09:47.270 INFO:tasks.workunit.client.1.smithi173.stdout:7/925: creat d9/d33/d35/d54/db3/d102/f138 x:0 0 0 2024-04-20T11:09:47.272 INFO:tasks.workunit.client.1.smithi173.stdout:1/825: dwrite d5/d6/d17/d42/d12/f47 [0,4194304] 0 2024-04-20T11:09:47.273 INFO:tasks.workunit.client.1.smithi173.stdout:9/890: read - d5/d9/d20/d2b/d59/d44/ff9 zero size 2024-04-20T11:09:47.274 INFO:tasks.workunit.client.1.smithi173.stdout:5/916: write da/de/d19/d32/f31 [3018713,63479] 0 2024-04-20T11:09:47.275 INFO:tasks.workunit.client.1.smithi173.stdout:3/882: chown d7/d17/d61/d83/d45/lac 290 1 2024-04-20T11:09:47.278 INFO:tasks.workunit.client.1.smithi173.stdout:1/826: chown d5/d6/d17/d42/da/d20/fd6 24 1 2024-04-20T11:09:47.278 INFO:tasks.workunit.client.1.smithi173.stdout:6/920: rename d4/d24/d3e/d9c/dc7/f11c to d4/f11e 0 2024-04-20T11:09:47.278 INFO:tasks.workunit.client.1.smithi173.stdout:7/926: dread d9/d33/d40/d89/f8a [0,4194304] 0 2024-04-20T11:09:47.281 INFO:tasks.workunit.client.1.smithi173.stdout:7/927: fsync d9/d1f/d72/f116 0 2024-04-20T11:09:47.282 INFO:tasks.workunit.client.1.smithi173.stdout:1/827: chown d5/d6/d17/d42/da/df/d40/db4/fd2 22240 1 2024-04-20T11:09:47.285 INFO:tasks.workunit.client.1.smithi173.stdout:1/828: truncate d5/d6/fe4 528369 0 2024-04-20T11:09:47.285 INFO:tasks.workunit.client.1.smithi173.stdout:5/917: write da/d4f/d65/fee [759732,39865] 0 2024-04-20T11:09:47.287 INFO:tasks.workunit.client.1.smithi173.stdout:1/829: dread - d5/d6/dc/d7f/fee zero size 2024-04-20T11:09:47.287 INFO:tasks.workunit.client.1.smithi173.stdout:4/995: rename da/dc/df4/d53/d39/f156 to da/dc/d15/f157 0 2024-04-20T11:09:47.290 INFO:tasks.workunit.client.1.smithi173.stdout:8/916: mknod d4/d18/d4f/d5b/c126 0 2024-04-20T11:09:47.293 INFO:tasks.workunit.client.1.smithi173.stdout:8/917: write d4/d18/d4f/d5b/de4/de6/f122 [198654,92205] 0 2024-04-20T11:09:47.294 INFO:tasks.workunit.client.1.smithi173.stdout:9/891: write f0 [126738,105337] 0 2024-04-20T11:09:47.297 INFO:tasks.workunit.client.1.smithi173.stdout:0/896: dread d6/d24/fad [0,4194304] 0 2024-04-20T11:09:47.297 INFO:tasks.workunit.client.1.smithi173.stdout:4/996: symlink da/d1e/da6/l158 0 2024-04-20T11:09:47.299 INFO:tasks.workunit.client.1.smithi173.stdout:1/830: write d5/d6/d17/d42/da/d20/f87 [902527,31660] 0 2024-04-20T11:09:47.301 INFO:tasks.workunit.client.1.smithi173.stdout:3/883: creat d7/d17/d61/d83/dcf/f11b x:0 0 0 2024-04-20T11:09:47.302 INFO:tasks.workunit.client.1.smithi173.stdout:9/892: chown d5/d9/d20/c63 13 1 2024-04-20T11:09:47.304 INFO:tasks.workunit.client.1.smithi173.stdout:5/918: mknod da/c13d 0 2024-04-20T11:09:47.307 INFO:tasks.workunit.client.1.smithi173.stdout:7/928: creat d9/d33/d35/f139 x:0 0 0 2024-04-20T11:09:47.308 INFO:tasks.workunit.client.1.smithi173.stdout:8/918: mknod d4/d18/d59/d5a/da8/db9/de0/d3c/d86/c127 0 2024-04-20T11:09:47.310 INFO:tasks.workunit.client.1.smithi173.stdout:6/921: dwrite d4/d24/d42/d81/ff4 [0,4194304] 0 2024-04-20T11:09:47.314 INFO:tasks.workunit.client.1.smithi173.stdout:9/893: creat d5/d9/d20/d29/d18/d9b/d98/dc4/f10e x:0 0 0 2024-04-20T11:09:47.324 INFO:tasks.workunit.client.1.smithi173.stdout:6/922: rename d4/d24/d42/d81/lb8 to d4/d24/d3e/d9c/df2/l11f 0 2024-04-20T11:09:47.326 INFO:tasks.workunit.client.1.smithi173.stdout:7/929: dwrite d9/d33/d35/f139 [0,4194304] 0 2024-04-20T11:09:47.328 INFO:tasks.workunit.client.1.smithi173.stdout:7/930: chown d9/d33/d35/d54/db3/d102/f138 365092 1 2024-04-20T11:09:47.336 INFO:tasks.workunit.client.1.smithi173.stdout:3/884: truncate d7/d17/d61/d83/dcf/f4f 406713 0 2024-04-20T11:09:47.340 INFO:tasks.workunit.client.1.smithi173.stdout:5/919: creat da/d34/f13e x:0 0 0 2024-04-20T11:09:47.341 INFO:tasks.workunit.client.1.smithi173.stdout:0/897: dwrite d6/d8/df/d43/f59 [0,4194304] 0 2024-04-20T11:09:47.344 INFO:tasks.workunit.client.1.smithi173.stdout:1/831: getdents d5/d6/dc/d7f 0 2024-04-20T11:09:47.345 INFO:tasks.workunit.client.1.smithi173.stdout:6/923: mkdir d4/d18/d7e/d120 0 2024-04-20T11:09:47.348 INFO:tasks.workunit.client.1.smithi173.stdout:7/931: rename d9/d33/d40/d7a to d9/d33/d35/d4a/d52/df2/d13a 0 2024-04-20T11:09:47.348 INFO:tasks.workunit.client.1.smithi173.stdout:8/919: dwrite d4/d18/d59/d5a/da8/db9/de0/d3c/f73 [0,4194304] 0 2024-04-20T11:09:47.350 INFO:tasks.workunit.client.1.smithi173.stdout:1/832: stat d5/d6/d17/d42/da/d20/dad/dc2/d94/fea 0 2024-04-20T11:09:47.352 INFO:tasks.workunit.client.1.smithi173.stdout:5/920: write da/d4f/d5e/f6d [4020766,110269] 0 2024-04-20T11:09:47.352 INFO:tasks.workunit.client.1.smithi173.stdout:0/898: symlink d6/d8/d3c/dee/d107/db9/l113 0 2024-04-20T11:09:47.352 INFO:tasks.workunit.client.1.smithi173.stdout:1/833: read d5/d6/fe4 [483341,65995] 0 2024-04-20T11:09:47.356 INFO:tasks.workunit.client.1.smithi173.stdout:1/834: dread - d5/d6/d17/d42/da/fe3 zero size 2024-04-20T11:09:47.359 INFO:tasks.workunit.client.1.smithi173.stdout:9/894: rename d5/d9/d20 to d5/d9/d10f 0 2024-04-20T11:09:47.363 INFO:tasks.workunit.client.1.smithi173.stdout:3/885: symlink d7/d17/d58/d7f/df2/l11c 0 2024-04-20T11:09:47.370 INFO:tasks.workunit.client.1.smithi173.stdout:0/899: symlink d6/d8/df/d2c/d42/d50/d86/d8b/dcc/l114 0 2024-04-20T11:09:47.373 INFO:tasks.workunit.client.1.smithi173.stdout:7/932: creat d9/d33/d35/d54/d135/f13b x:0 0 0 2024-04-20T11:09:47.374 INFO:tasks.workunit.client.1.smithi173.stdout:6/924: creat d4/d24/f121 x:0 0 0 2024-04-20T11:09:47.376 INFO:tasks.workunit.client.1.smithi173.stdout:5/921: link da/de/d19/d32/d72/dc3/ffb da/d4f/d65/f13f 0 2024-04-20T11:09:47.378 INFO:tasks.workunit.client.1.smithi173.stdout:1/835: mknod d5/d6/d17/d42/da/df/d40/c111 0 2024-04-20T11:09:47.380 INFO:tasks.workunit.client.1.smithi173.stdout:3/886: symlink d7/d17/d61/d83/d45/l11d 0 2024-04-20T11:09:47.381 INFO:tasks.workunit.client.1.smithi173.stdout:1/836: read d5/d6/dc/fe0 [201190,71919] 0 2024-04-20T11:09:47.383 INFO:tasks.workunit.client.1.smithi173.stdout:6/925: symlink d4/d59/d8f/dd7/d108/l122 0 2024-04-20T11:09:47.383 INFO:tasks.workunit.client.1.smithi173.stdout:5/922: truncate da/de/ff0 408625 0 2024-04-20T11:09:47.384 INFO:tasks.workunit.client.1.smithi173.stdout:1/837: write d5/d6/d17/d42/d12/d1c/f8b [1787039,119429] 0 2024-04-20T11:09:47.388 INFO:tasks.workunit.client.1.smithi173.stdout:3/887: mkdir d7/d17/d11e 0 2024-04-20T11:09:47.389 INFO:tasks.workunit.client.1.smithi173.stdout:1/838: chown d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fc0 172 1 2024-04-20T11:09:47.395 INFO:tasks.workunit.client.1.smithi173.stdout:5/923: rename da/de/d19/d2c/f53 to da/d34/d7f/da5/db5/f140 0 2024-04-20T11:09:47.398 INFO:tasks.workunit.client.1.smithi173.stdout:7/933: creat d9/d33/d35/f13c x:0 0 0 2024-04-20T11:09:47.399 INFO:tasks.workunit.client.1.smithi173.stdout:0/900: dread d6/d8/df/d43/f4c [0,4194304] 0 2024-04-20T11:09:47.403 INFO:tasks.workunit.client.1.smithi173.stdout:6/926: mkdir d4/d18/d7e/d120/d123 0 2024-04-20T11:09:47.405 INFO:tasks.workunit.client.1.smithi173.stdout:9/895: truncate d5/d9/d10f/d29/d18/f9c 3824894 0 2024-04-20T11:09:47.409 INFO:tasks.workunit.client.1.smithi173.stdout:1/839: creat d5/d6/d17/d42/da/df/d40/dc8/f112 x:0 0 0 2024-04-20T11:09:47.411 INFO:tasks.workunit.client.1.smithi173.stdout:5/924: symlink da/de/d75/l141 0 2024-04-20T11:09:47.412 INFO:tasks.workunit.client.1.smithi173.stdout:7/934: symlink d9/d33/d35/d4a/d52/df2/dc6/df3/d108/l13d 0 2024-04-20T11:09:47.417 INFO:tasks.workunit.client.1.smithi173.stdout:6/927: dwrite d4/f5a [0,4194304] 0 2024-04-20T11:09:47.417 INFO:tasks.workunit.client.1.smithi173.stdout:1/840: symlink d5/d6/d17/d42/da/d20/dad/l113 0 2024-04-20T11:09:47.419 INFO:tasks.workunit.client.1.smithi173.stdout:1/841: stat d5/d6/d17/d42/da/fe3 0 2024-04-20T11:09:47.421 INFO:tasks.workunit.client.1.smithi173.stdout:3/888: getdents d7 0 2024-04-20T11:09:47.421 INFO:tasks.workunit.client.1.smithi173.stdout:9/896: mkdir d5/d9/d10f/d2b/d59/d44/d78/d82/de8/d110 0 2024-04-20T11:09:47.425 INFO:tasks.workunit.client.1.smithi173.stdout:6/928: mkdir d4/d59/d8f/d124 0 2024-04-20T11:09:47.426 INFO:tasks.workunit.client.1.smithi173.stdout:9/897: fsync d5/faf 0 2024-04-20T11:09:47.428 INFO:tasks.workunit.client.1.smithi173.stdout:9/898: write d5/d9/f1d [3739664,20353] 0 2024-04-20T11:09:47.429 INFO:tasks.workunit.client.1.smithi173.stdout:9/899: read - d5/d9/ff8 zero size 2024-04-20T11:09:47.429 INFO:tasks.workunit.client.1.smithi173.stdout:3/889: readlink d7/d17/d61/d83/dcf/l23 0 2024-04-20T11:09:47.431 INFO:tasks.workunit.client.1.smithi173.stdout:3/890: chown d7/d17/d61/d83/dc2 66469772 1 2024-04-20T11:09:47.432 INFO:tasks.workunit.client.1.smithi173.stdout:5/925: creat da/d34/f142 x:0 0 0 2024-04-20T11:09:47.433 INFO:tasks.workunit.client.1.smithi173.stdout:6/929: mkdir d4/d59/d125 0 2024-04-20T11:09:47.434 INFO:tasks.workunit.client.1.smithi173.stdout:9/900: truncate d5/d9/d10f/d2b/d59/d44/d78/fef 811868 0 2024-04-20T11:09:47.436 INFO:tasks.workunit.client.1.smithi173.stdout:0/901: truncate d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/fcd 3147218 0 2024-04-20T11:09:47.438 INFO:tasks.workunit.client.1.smithi173.stdout:0/902: dread - d6/d8/df/d2c/d42/d50/d86/d8b/da2/fdf zero size 2024-04-20T11:09:47.440 INFO:tasks.workunit.client.1.smithi173.stdout:0/903: chown d6/d8/d51/fce 4635212 1 2024-04-20T11:09:47.446 INFO:tasks.workunit.client.1.smithi173.stdout:3/891: dwrite d7/d17/d58/d8a/dc4/f119 [0,4194304] 0 2024-04-20T11:09:47.448 INFO:tasks.workunit.client.1.smithi173.stdout:3/892: write d7/d53/dab/ffb [3683171,72164] 0 2024-04-20T11:09:47.533 INFO:tasks.workunit.client.1.smithi173.stdout:8/920: sync 2024-04-20T11:09:47.533 INFO:tasks.workunit.client.1.smithi173.stdout:4/997: sync 2024-04-20T11:09:47.680 INFO:tasks.workunit.client.1.smithi173.stdout:5/926: rmdir da/d4f/d65 39 2024-04-20T11:09:47.691 INFO:tasks.workunit.client.1.smithi173.stdout:7/935: rename d9/da/d17/cfe to d9/d1f/d7f/dce/c13e 0 2024-04-20T11:09:47.695 INFO:tasks.workunit.client.1.smithi173.stdout:7/936: write d9/d33/d35/d4a/d52/df2/f104 [786149,25071] 0 2024-04-20T11:09:47.698 INFO:tasks.workunit.client.1.smithi173.stdout:3/893: dread d7/d17/d58/fdc [4194304,4194304] 0 2024-04-20T11:09:47.701 INFO:tasks.workunit.client.1.smithi173.stdout:4/998: symlink da/dc/d15/d8b/d151/d7a/l159 0 2024-04-20T11:09:47.704 INFO:tasks.workunit.client.1.smithi173.stdout:4/999: chown da/d1a/d60/d66/le6 213 1 2024-04-20T11:09:47.705 INFO:tasks.workunit.client.1.smithi173.stdout:8/921: creat d4/d18/d59/d5a/da8/db9/de0/d3c/d86/dec/f128 x:0 0 0 2024-04-20T11:09:47.707 INFO:tasks.workunit.client.1.smithi173.stdout:5/927: symlink da/d34/d7f/da5/db5/l143 0 2024-04-20T11:09:47.714 INFO:tasks.workunit.client.1.smithi173.stdout:9/901: mknod d5/d9/d10f/c111 0 2024-04-20T11:09:47.718 INFO:tasks.workunit.client.1.smithi173.stdout:1/842: rename d5/d6/d17/d42/da/d20/d38/d52/d75/c84 to d5/d6/d17/d42/da/df/d40/dc8/c114 0 2024-04-20T11:09:47.718 INFO:tasks.workunit.client.1.smithi173.stdout:7/937: dwrite d9/d1f/d7f/dce/db2/f107 [0,4194304] 0 2024-04-20T11:09:47.734 INFO:tasks.workunit.client.1.smithi173.stdout:8/922: dwrite d4/d18/d4f/d5b/de4/de6/f122 [0,4194304] 0 2024-04-20T11:09:47.737 INFO:tasks.workunit.client.1.smithi173.stdout:0/904: dwrite d6/d24/f4f [0,4194304] 0 2024-04-20T11:09:47.737 INFO:tasks.workunit.client.1.smithi173.stdout:6/930: creat d4/d59/d125/f126 x:0 0 0 2024-04-20T11:09:47.738 INFO:tasks.workunit.client.1.smithi173.stdout:7/938: fdatasync d9/d1f/d7f/f115 0 2024-04-20T11:09:47.740 INFO:tasks.workunit.client.1.smithi173.stdout:5/928: dread da/de/d19/f76 [0,4194304] 0 2024-04-20T11:09:47.741 INFO:tasks.workunit.client.1.smithi173.stdout:7/939: stat d9/d33/d35/d4a 0 2024-04-20T11:09:47.744 INFO:tasks.workunit.client.1.smithi173.stdout:1/843: truncate d5/d6/d17/d42/d12/dc9/f10d 434514 0 2024-04-20T11:09:47.751 INFO:tasks.workunit.client.1.smithi173.stdout:3/894: link d7/d85/df5/f100 d7/d17/d61/d90/f11f 0 2024-04-20T11:09:47.756 INFO:tasks.workunit.client.1.smithi173.stdout:1/844: write d5/d6/d17/d42/da/df/d40/dc8/fcc [1719831,36889] 0 2024-04-20T11:09:47.756 INFO:tasks.workunit.client.1.smithi173.stdout:3/895: dread - d7/d17/d61/d83/fde zero size 2024-04-20T11:09:47.760 INFO:tasks.workunit.client.1.smithi173.stdout:7/940: unlink d9/d33/d35/d4a/d52/df2/d13a/ff0 0 2024-04-20T11:09:47.761 INFO:tasks.workunit.client.1.smithi173.stdout:6/931: dwrite d4/d24/d42/f67 [0,4194304] 0 2024-04-20T11:09:47.766 INFO:tasks.workunit.client.1.smithi173.stdout:8/923: link d4/d18/d4f/f57 d4/d18/d59/d5a/d7b/db1/dce/f129 0 2024-04-20T11:09:47.766 INFO:tasks.workunit.client.1.smithi173.stdout:9/902: getdents d5/d9/d10f/d2b/d59/da1/deb 0 2024-04-20T11:09:47.768 INFO:tasks.workunit.client.1.smithi173.stdout:1/845: dread d5/d6/fcd [0,4194304] 0 2024-04-20T11:09:47.770 INFO:tasks.workunit.client.1.smithi173.stdout:0/905: mkdir d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/de6/d115 0 2024-04-20T11:09:47.771 INFO:tasks.workunit.client.1.smithi173.stdout:3/896: dread - d7/d17/f99 zero size 2024-04-20T11:09:47.774 INFO:tasks.workunit.client.1.smithi173.stdout:1/846: readlink d5/d6/d17/d42/da/d20/d48/l56 0 2024-04-20T11:09:47.777 INFO:tasks.workunit.client.1.smithi173.stdout:1/847: dread - d5/d6/d17/d42/da/d20/dad/fd9 zero size 2024-04-20T11:09:47.778 INFO:tasks.workunit.client.1.smithi173.stdout:0/906: write d6/d8/d3c/dee/d107/db9/fe4 [691017,49294] 0 2024-04-20T11:09:47.781 INFO:tasks.workunit.client.1.smithi173.stdout:7/941: dwrite d9/d33/d35/d54/db3/d102/f10d [0,4194304] 0 2024-04-20T11:09:47.789 INFO:tasks.workunit.client.1.smithi173.stdout:0/907: read d6/d8/df/d15/ff4 [2768339,78082] 0 2024-04-20T11:09:47.793 INFO:tasks.workunit.client.1.smithi173.stdout:1/848: dwrite d5/d6/d17/d42/f54 [4194304,4194304] 0 2024-04-20T11:09:47.799 INFO:tasks.workunit.client.1.smithi173.stdout:7/942: creat d9/d1f/d7f/dce/dad/f13f x:0 0 0 2024-04-20T11:09:47.801 INFO:tasks.workunit.client.1.smithi173.stdout:8/924: creat d4/d18/d59/f12a x:0 0 0 2024-04-20T11:09:47.804 INFO:tasks.workunit.client.1.smithi173.stdout:0/908: rmdir d6/d8/d3c/d77 39 2024-04-20T11:09:47.805 INFO:tasks.workunit.client.1.smithi173.stdout:9/903: link d5/le4 d5/d9/d10f/d4f/d5e/d9e/l112 0 2024-04-20T11:09:47.814 INFO:tasks.workunit.client.1.smithi173.stdout:7/943: rename d9/d33/f3a to d9/da/d17/d131/f140 0 2024-04-20T11:09:47.818 INFO:tasks.workunit.client.1.smithi173.stdout:8/925: dwrite d4/d18/d59/d5a/da8/db9/de0/f9d [0,4194304] 0 2024-04-20T11:09:47.819 INFO:tasks.workunit.client.1.smithi173.stdout:0/909: creat d6/d8/d51/f116 x:0 0 0 2024-04-20T11:09:47.821 INFO:tasks.workunit.client.1.smithi173.stdout:9/904: mknod d5/dcc/c113 0 2024-04-20T11:09:47.823 INFO:tasks.workunit.client.1.smithi173.stdout:0/910: truncate d6/d8/d3c/d44/ffa 245457 0 2024-04-20T11:09:47.825 INFO:tasks.workunit.client.1.smithi173.stdout:1/849: mkdir d5/d115 0 2024-04-20T11:09:47.825 INFO:tasks.workunit.client.1.smithi173.stdout:0/911: dread - d6/d8/d3c/dee/d107/d112/fbd zero size 2024-04-20T11:09:47.829 INFO:tasks.workunit.client.1.smithi173.stdout:8/926: mknod d4/d18/d59/d5a/da8/db9/de0/d3c/d86/db7/d110/c12b 0 2024-04-20T11:09:47.830 INFO:tasks.workunit.client.1.smithi173.stdout:9/905: fsync d5/d9/d10f/d2b/d59/d48/fe2 0 2024-04-20T11:09:47.833 INFO:tasks.workunit.client.1.smithi173.stdout:0/912: creat d6/d8/d3c/dee/d107/db9/f117 x:0 0 0 2024-04-20T11:09:47.836 INFO:tasks.workunit.client.1.smithi173.stdout:5/929: truncate da/de/d19/f120 121777 0 2024-04-20T11:09:47.836 INFO:tasks.workunit.client.1.smithi173.stdout:8/927: symlink d4/d18/d4f/d5b/de4/l12c 0 2024-04-20T11:09:47.858 INFO:tasks.workunit.client.1.smithi173.stdout:7/944: getdents d9 0 2024-04-20T11:09:47.858 INFO:tasks.workunit.client.1.smithi173.stdout:0/913: symlink d6/d8/d3c/dee/l118 0 2024-04-20T11:09:47.859 INFO:tasks.workunit.client.1.smithi173.stdout:1/850: link d5/d6/d17/d42/d12/cb9 d5/d6/d17/d42/d12/d1f/c116 0 2024-04-20T11:09:47.860 INFO:tasks.workunit.client.1.smithi173.stdout:3/897: write d7/d17/d61/d83/dcf/d30/fbe [82474,97977] 0 2024-04-20T11:09:47.866 INFO:tasks.workunit.client.1.smithi173.stdout:3/898: fdatasync d7/d17/d58/d8a/dc4/f10d 0 2024-04-20T11:09:47.868 INFO:tasks.workunit.client.1.smithi173.stdout:6/932: dwrite d4/d24/d42/d74/d3b/f52 [0,4194304] 0 2024-04-20T11:09:47.870 INFO:tasks.workunit.client.1.smithi173.stdout:3/899: chown d7/d17/d61/d90/dca/f111 650 1 2024-04-20T11:09:47.886 INFO:tasks.workunit.client.1.smithi173.stdout:0/914: dwrite d6/d8/df/d15/f39 [4194304,4194304] 0 2024-04-20T11:09:47.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:47 smithi117 ceph-mon[15659]: pgmap v33: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 50 MiB/s rd, 139 MiB/s wr, 324 op/s 2024-04-20T11:09:47.890 INFO:tasks.workunit.client.1.smithi173.stdout:5/930: dread da/d4f/d65/d8e/fcc [0,4194304] 0 2024-04-20T11:09:47.911 INFO:tasks.workunit.client.1.smithi173.stdout:9/906: truncate d5/d9/d10f/d29/f96 1316493 0 2024-04-20T11:09:47.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:47 smithi173 ceph-mon[23800]: pgmap v33: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 50 MiB/s rd, 139 MiB/s wr, 324 op/s 2024-04-20T11:09:47.912 INFO:tasks.workunit.client.1.smithi173.stdout:9/907: write d5/d9/d10f/d29/f5c [4788517,44194] 0 2024-04-20T11:09:47.914 INFO:tasks.workunit.client.1.smithi173.stdout:9/908: chown d5/d9/d10f/d2b/d59/da1/fb0 2233466 1 2024-04-20T11:09:47.915 INFO:tasks.workunit.client.1.smithi173.stdout:9/909: fsync d5/d9/d10f/d29/d2d/dcb/fd5 0 2024-04-20T11:09:47.917 INFO:tasks.workunit.client.1.smithi173.stdout:9/910: readlink d5/d9/d10f/d29/l35 0 2024-04-20T11:09:47.925 INFO:tasks.workunit.client.1.smithi173.stdout:6/933: unlink d4/d1c/d8e/f102 0 2024-04-20T11:09:47.926 INFO:tasks.workunit.client.1.smithi173.stdout:3/900: mknod d7/d17/d61/d90/c120 0 2024-04-20T11:09:47.928 INFO:tasks.workunit.client.1.smithi173.stdout:0/915: symlink d6/d8/df/d2c/d42/d50/d86/d8b/da2/l119 0 2024-04-20T11:09:47.929 INFO:tasks.workunit.client.1.smithi173.stdout:5/931: dread - da/de/d19/d32/ff1 zero size 2024-04-20T11:09:47.934 INFO:tasks.workunit.client.1.smithi173.stdout:7/945: symlink d9/d33/d91/l141 0 2024-04-20T11:09:47.936 INFO:tasks.workunit.client.1.smithi173.stdout:5/932: unlink da/de/d19/d81/d92/f125 0 2024-04-20T11:09:47.938 INFO:tasks.workunit.client.1.smithi173.stdout:5/933: chown da/de/fe2 255103 1 2024-04-20T11:09:47.941 INFO:tasks.workunit.client.1.smithi173.stdout:6/934: link d4/d1c/d8e/c92 d4/d24/d42/d74/df6/c127 0 2024-04-20T11:09:47.943 INFO:tasks.workunit.client.1.smithi173.stdout:6/935: write d4/d59/d125/f126 [244383,111955] 0 2024-04-20T11:09:47.943 INFO:tasks.workunit.client.1.smithi173.stdout:0/916: link d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d65/f98 d6/d24/f11a 0 2024-04-20T11:09:47.945 INFO:tasks.workunit.client.1.smithi173.stdout:6/936: readlink d4/d14/d5f/lc9 0 2024-04-20T11:09:47.946 INFO:tasks.workunit.client.1.smithi173.stdout:0/917: chown d6/d5f/d8f/fc9 32254788 1 2024-04-20T11:09:47.947 INFO:tasks.workunit.client.1.smithi173.stdout:5/934: rename da/de/d19/d32/d72/l11a to da/d34/d7f/da5/d9a/l144 0 2024-04-20T11:09:47.952 INFO:tasks.workunit.client.1.smithi173.stdout:7/946: dwrite d9/d33/f48 [4194304,4194304] 0 2024-04-20T11:09:47.952 INFO:tasks.workunit.client.1.smithi173.stdout:5/935: mkdir da/de/d138/d145 0 2024-04-20T11:09:47.953 INFO:tasks.workunit.client.1.smithi173.stdout:3/901: dread d7/d17/d61/d83/dcf/d5c/fc6 [0,4194304] 0 2024-04-20T11:09:47.956 INFO:tasks.workunit.client.1.smithi173.stdout:5/936: mknod da/d4f/d65/d8e/c146 0 2024-04-20T11:09:47.960 INFO:tasks.workunit.client.1.smithi173.stdout:5/937: write da/de/fac [2336551,52286] 0 2024-04-20T11:09:47.962 INFO:tasks.workunit.client.1.smithi173.stdout:3/902: creat d7/d17/d61/d83/dcf/d118/f121 x:0 0 0 2024-04-20T11:09:47.962 INFO:tasks.workunit.client.1.smithi173.stdout:7/947: getdents d9/d1f 0 2024-04-20T11:09:47.965 INFO:tasks.workunit.client.1.smithi173.stdout:5/938: stat da/lb2 0 2024-04-20T11:09:47.968 INFO:tasks.workunit.client.1.smithi173.stdout:1/851: truncate d5/d6/d3b/f98 1438455 0 2024-04-20T11:09:47.969 INFO:tasks.workunit.client.1.smithi173.stdout:3/903: mkdir d7/d85/df5/d122 0 2024-04-20T11:09:47.970 INFO:tasks.workunit.client.1.smithi173.stdout:8/928: dwrite d4/d18/d59/d5a/f95 [0,4194304] 0 2024-04-20T11:09:47.975 INFO:tasks.workunit.client.1.smithi173.stdout:5/939: write da/de/d19/f76 [4476394,89031] 0 2024-04-20T11:09:47.978 INFO:tasks.workunit.client.1.smithi173.stdout:3/904: truncate d7/d2b/d8e/f114 199432 0 2024-04-20T11:09:47.979 INFO:tasks.workunit.client.1.smithi173.stdout:1/852: mkdir d5/d6/dc/d7f/d117 0 2024-04-20T11:09:47.981 INFO:tasks.workunit.client.1.smithi173.stdout:0/918: dread d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/fcd [0,4194304] 0 2024-04-20T11:09:47.983 INFO:tasks.workunit.client.1.smithi173.stdout:0/919: stat d6/d5f/d8f/dd6 0 2024-04-20T11:09:47.986 INFO:tasks.workunit.client.1.smithi173.stdout:5/940: creat da/d34/d7f/da5/d9a/f147 x:0 0 0 2024-04-20T11:09:47.989 INFO:tasks.workunit.client.1.smithi173.stdout:1/853: creat d5/d6/d17/d42/da/df/d40/db4/f118 x:0 0 0 2024-04-20T11:09:47.993 INFO:tasks.workunit.client.1.smithi173.stdout:5/941: symlink da/d34/d7f/l148 0 2024-04-20T11:09:47.994 INFO:tasks.workunit.client.1.smithi173.stdout:5/942: fsync da/de/f121 0 2024-04-20T11:09:47.996 INFO:tasks.workunit.client.1.smithi173.stdout:3/905: link d7/d17/d61/d83/fbf d7/d17/d61/d83/dcf/f123 0 2024-04-20T11:09:48.001 INFO:tasks.workunit.client.1.smithi173.stdout:1/854: mkdir d5/d6/d17/d42/d12/d1f/ddc/d119 0 2024-04-20T11:09:48.003 INFO:tasks.workunit.client.1.smithi173.stdout:1/855: fdatasync d5/d6/d17/d42/da/d20/dad/fd9 0 2024-04-20T11:09:48.004 INFO:tasks.workunit.client.1.smithi173.stdout:3/906: rename d7/d53/c8c to d7/d17/d58/d8a/d109/c124 0 2024-04-20T11:09:48.005 INFO:tasks.workunit.client.1.smithi173.stdout:5/943: dread da/de/d19/fe5 [0,4194304] 0 2024-04-20T11:09:48.007 INFO:tasks.workunit.client.1.smithi173.stdout:7/948: truncate d9/d1f/d7f/d8d/faf 33093 0 2024-04-20T11:09:48.008 INFO:tasks.workunit.client.1.smithi173.stdout:8/929: dread d4/d18/d59/d5a/d7b/db1/dce/f129 [0,4194304] 0 2024-04-20T11:09:48.010 INFO:tasks.workunit.client.1.smithi173.stdout:7/949: truncate d9/d1f/d72/d8f/f100 1987864 0 2024-04-20T11:09:48.011 INFO:tasks.workunit.client.1.smithi173.stdout:8/930: write d4/d18/fa2 [143948,111694] 0 2024-04-20T11:09:48.014 INFO:tasks.workunit.client.1.smithi173.stdout:5/944: rename da/de/d19/d32/d30 to da/d4f/d65/dd5/d107/d149 0 2024-04-20T11:09:48.014 INFO:tasks.workunit.client.1.smithi173.stdout:0/920: dread d6/d8/df/d2c/d2d/f6a [0,4194304] 0 2024-04-20T11:09:48.015 INFO:tasks.workunit.client.1.smithi173.stdout:0/921: readlink d6/d24/l99 0 2024-04-20T11:09:48.016 INFO:tasks.workunit.client.1.smithi173.stdout:1/856: symlink d5/d6/d17/d42/da/df/d40/dc8/dfc/l11a 0 2024-04-20T11:09:48.017 INFO:tasks.workunit.client.1.smithi173.stdout:5/945: truncate da/d4f/d65/dd5/d107/d149/f70 1078937 0 2024-04-20T11:09:48.018 INFO:tasks.workunit.client.1.smithi173.stdout:7/950: mkdir d9/d1f/d72/d8f/d142 0 2024-04-20T11:09:48.023 INFO:tasks.workunit.client.1.smithi173.stdout:8/931: creat d4/d18/d59/d5a/da8/db9/de0/d9b/f12d x:0 0 0 2024-04-20T11:09:48.026 INFO:tasks.workunit.client.1.smithi173.stdout:3/907: write d7/f94 [899160,82046] 0 2024-04-20T11:09:48.029 INFO:tasks.workunit.client.1.smithi173.stdout:1/857: rename f4 to d5/d6/d17/d42/da/d20/d48/d81/df3/f11b 0 2024-04-20T11:09:48.032 INFO:tasks.workunit.client.1.smithi173.stdout:1/858: readlink d5/d6/d17/d93/lc7 0 2024-04-20T11:09:48.032 INFO:tasks.workunit.client.1.smithi173.stdout:5/946: unlink da/de/d19/d81/c11d 0 2024-04-20T11:09:48.032 INFO:tasks.workunit.client.1.smithi173.stdout:8/932: truncate d4/d18/d59/d5a/da8/db9/de0/d9b/f12d 110493 0 2024-04-20T11:09:48.036 INFO:tasks.workunit.client.1.smithi173.stdout:7/951: dwrite d9/d33/d35/f13c [0,4194304] 0 2024-04-20T11:09:48.036 INFO:tasks.workunit.client.1.smithi173.stdout:1/859: read - d5/d6/d17/d42/da/df/d40/f10f zero size 2024-04-20T11:09:48.038 INFO:tasks.workunit.client.1.smithi173.stdout:3/908: rename d7/d17/d61/d83/l5b to d7/d17/d58/d8a/dc4/l125 0 2024-04-20T11:09:48.039 INFO:tasks.workunit.client.1.smithi173.stdout:8/933: chown d4/d18/d59/d5a/l117 38 1 2024-04-20T11:09:48.041 INFO:tasks.workunit.client.1.smithi173.stdout:1/860: chown d5/d6/d17/c6e 1 1 2024-04-20T11:09:48.042 INFO:tasks.workunit.client.1.smithi173.stdout:0/922: dread d6/d8/df/d2c/d42/f72 [0,4194304] 0 2024-04-20T11:09:48.043 INFO:tasks.workunit.client.1.smithi173.stdout:3/909: creat d7/d17/d11e/f126 x:0 0 0 2024-04-20T11:09:48.045 INFO:tasks.workunit.client.1.smithi173.stdout:1/861: truncate d5/d6/d17/d42/da/fe3 770870 0 2024-04-20T11:09:48.134 INFO:tasks.workunit.client.1.smithi173.stdout:3/910: mknod d7/d53/dab/c127 0 2024-04-20T11:09:48.134 INFO:tasks.workunit.client.1.smithi173.stdout:0/923: creat d6/d8/d3c/dee/d107/d112/f11b x:0 0 0 2024-04-20T11:09:48.137 INFO:tasks.workunit.client.1.smithi173.stdout:5/947: dwrite da/d34/d7f/da4/fb0 [0,4194304] 0 2024-04-20T11:09:48.144 INFO:tasks.workunit.client.1.smithi173.stdout:0/924: mknod d6/d8/df/c11c 0 2024-04-20T11:09:48.152 INFO:tasks.workunit.client.1.smithi173.stdout:3/911: truncate d7/d17/d61/d83/dcf/d30/f80 1553218 0 2024-04-20T11:09:48.159 INFO:tasks.workunit.client.1.smithi173.stdout:0/925: creat d6/f11d x:0 0 0 2024-04-20T11:09:48.166 INFO:tasks.workunit.client.1.smithi173.stdout:7/952: dread d9/da/d17/f64 [0,4194304] 0 2024-04-20T11:09:48.173 INFO:tasks.workunit.client.1.smithi173.stdout:0/926: write d6/d8/df/d2c/d42/d50/d86/fbc [4395614,111148] 0 2024-04-20T11:09:48.173 INFO:tasks.workunit.client.1.smithi173.stdout:1/862: dwrite d5/d6/d17/d42/da/d20/d38/d52/f74 [0,4194304] 0 2024-04-20T11:09:48.178 INFO:tasks.workunit.client.1.smithi173.stdout:7/953: symlink d9/d33/d35/d54/db3/d102/l143 0 2024-04-20T11:09:48.182 INFO:tasks.workunit.client.1.smithi173.stdout:5/948: dwrite da/de/d19/d2c/fa8 [0,4194304] 0 2024-04-20T11:09:48.185 INFO:tasks.workunit.client.1.smithi173.stdout:7/954: truncate d9/da/dc3/f12d 1021306 0 2024-04-20T11:09:48.188 INFO:tasks.workunit.client.1.smithi173.stdout:3/912: dwrite d7/d17/d61/d83/dcf/f46 [0,4194304] 0 2024-04-20T11:09:48.196 INFO:tasks.workunit.client.1.smithi173.stdout:5/949: mkdir da/de/d19/d2c/d106/d124/d14a 0 2024-04-20T11:09:48.199 INFO:tasks.workunit.client.1.smithi173.stdout:5/950: chown da/d4f/d65/dd5/lf3 1203674096 1 2024-04-20T11:09:48.209 INFO:tasks.workunit.client.1.smithi173.stdout:5/951: write da/d9e/f105 [331802,77898] 0 2024-04-20T11:09:48.216 INFO:tasks.workunit.client.1.smithi173.stdout:1/863: dwrite d5/d6/d17/d42/da/d20/dad/dc2/fc5 [0,4194304] 0 2024-04-20T11:09:48.221 INFO:tasks.workunit.client.1.smithi173.stdout:0/927: dwrite d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/ff0 [0,4194304] 0 2024-04-20T11:09:48.233 INFO:tasks.workunit.client.1.smithi173.stdout:5/952: dwrite da/d9e/f13b [0,4194304] 0 2024-04-20T11:09:48.238 INFO:tasks.workunit.client.1.smithi173.stdout:5/953: chown da/d34/l71 1901498906 1 2024-04-20T11:09:48.243 INFO:tasks.workunit.client.1.smithi173.stdout:5/954: symlink da/de/d19/d2c/d106/l14b 0 2024-04-20T11:09:48.251 INFO:tasks.workunit.client.1.smithi173.stdout:0/928: rename d6/d8/d3c/dee/d107/d112/ce5 to d6/d8/d51/c11e 0 2024-04-20T11:09:48.251 INFO:tasks.workunit.client.1.smithi173.stdout:1/864: dwrite d5/d6/d17/d93/db6/fcf [0,4194304] 0 2024-04-20T11:09:48.256 INFO:tasks.workunit.client.1.smithi173.stdout:0/929: mknod d6/d8/df/d15/c11f 0 2024-04-20T11:09:48.258 INFO:tasks.workunit.client.1.smithi173.stdout:1/865: symlink d5/d6/d17/d42/d12/d1c/l11c 0 2024-04-20T11:09:48.259 INFO:tasks.workunit.client.1.smithi173.stdout:0/930: fdatasync d6/d8/df/d2c/d42/f84 0 2024-04-20T11:09:48.263 INFO:tasks.workunit.client.1.smithi173.stdout:0/931: write d6/d8/df/d2c/d42/d50/f10b [962509,126157] 0 2024-04-20T11:09:48.268 INFO:tasks.workunit.client.1.smithi173.stdout:0/932: chown d6/d8/df/d2c/d42/d50/d86/d8b/fc7 1584596 1 2024-04-20T11:09:48.270 INFO:tasks.workunit.client.1.smithi173.stdout:0/933: read - d6/d8/d51/f116 zero size 2024-04-20T11:09:48.274 INFO:tasks.workunit.client.1.smithi173.stdout:1/866: dwrite f0 [0,4194304] 0 2024-04-20T11:09:48.277 INFO:tasks.workunit.client.1.smithi173.stdout:0/934: symlink d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d65/l120 0 2024-04-20T11:09:48.279 INFO:tasks.workunit.client.1.smithi173.stdout:1/867: mknod d5/d6/d17/d42/da/d20/dad/dc2/c11d 0 2024-04-20T11:09:48.281 INFO:tasks.workunit.client.1.smithi173.stdout:0/935: fsync d6/d24/fad 0 2024-04-20T11:09:48.284 INFO:tasks.workunit.client.1.smithi173.stdout:6/937: sync 2024-04-20T11:09:48.284 INFO:tasks.workunit.client.1.smithi173.stdout:9/911: sync 2024-04-20T11:09:48.286 INFO:tasks.workunit.client.1.smithi173.stdout:1/868: truncate d5/d6/d17/d42/d12/d1f/f95 483885 0 2024-04-20T11:09:48.287 INFO:tasks.workunit.client.1.smithi173.stdout:5/955: dwrite f8 [0,4194304] 0 2024-04-20T11:09:48.287 INFO:tasks.workunit.client.1.smithi173.stdout:0/936: write d6/d8/df/d2c/d42/d50/d86/d8b/dcc/f109 [501564,130284] 0 2024-04-20T11:09:48.289 INFO:tasks.workunit.client.1.smithi173.stdout:6/938: write d4/d1c/d8e/d90/da4/fd2 [2591680,88717] 0 2024-04-20T11:09:48.290 INFO:tasks.workunit.client.1.smithi173.stdout:0/937: fdatasync d6/d8/d3c/dee/d107/d112/f11b 0 2024-04-20T11:09:48.295 INFO:tasks.workunit.client.1.smithi173.stdout:5/956: dread da/d4f/d65/fee [0,4194304] 0 2024-04-20T11:09:48.299 INFO:tasks.workunit.client.1.smithi173.stdout:6/939: mkdir d4/d14/d5f/d9f/d128 0 2024-04-20T11:09:48.309 INFO:tasks.workunit.client.1.smithi173.stdout:5/957: creat da/d34/d7f/da5/d9a/dc2/f14c x:0 0 0 2024-04-20T11:09:48.310 INFO:tasks.workunit.client.1.smithi173.stdout:9/912: rmdir d5/d9/d10f/d29/d2d/df1 0 2024-04-20T11:09:48.312 INFO:tasks.workunit.client.1.smithi173.stdout:6/940: mkdir d4/d18/d7e/d11b/d129 0 2024-04-20T11:09:48.313 INFO:tasks.workunit.client.1.smithi173.stdout:6/941: readlink d4/d1c/l7a 0 2024-04-20T11:09:48.314 INFO:tasks.workunit.client.1.smithi173.stdout:5/958: rename da/d4f/d65/dd5/d107/d149/d109/f12c to da/de/d138/f14d 0 2024-04-20T11:09:48.320 INFO:tasks.workunit.client.1.smithi173.stdout:1/869: dread d5/d6/d17/d93/f97 [0,4194304] 0 2024-04-20T11:09:48.321 INFO:tasks.workunit.client.1.smithi173.stdout:0/938: write d6/d8/df/d2c/d42/d50/d86/d8b/dcc/fcf [704318,79253] 0 2024-04-20T11:09:48.325 INFO:tasks.workunit.client.1.smithi173.stdout:6/942: rmdir d4/d18 39 2024-04-20T11:09:48.326 INFO:tasks.workunit.client.1.smithi173.stdout:8/934: sync 2024-04-20T11:09:48.327 INFO:tasks.workunit.client.1.smithi173.stdout:9/913: dwrite d5/d9/d10f/d2b/d59/da1/fb0 [0,4194304] 0 2024-04-20T11:09:48.328 INFO:tasks.workunit.client.1.smithi173.stdout:1/870: symlink d5/d6/d17/d42/da/d20/d38/d52/l11e 0 2024-04-20T11:09:48.329 INFO:tasks.workunit.client.1.smithi173.stdout:0/939: fdatasync d6/d8/d3c/d44/d75/fb0 0 2024-04-20T11:09:48.330 INFO:tasks.workunit.client.1.smithi173.stdout:5/959: write da/d34/f47 [546118,16812] 0 2024-04-20T11:09:48.331 INFO:tasks.workunit.client.1.smithi173.stdout:6/943: symlink d4/d1c/d8e/d90/da4/l12a 0 2024-04-20T11:09:48.331 INFO:tasks.workunit.client.1.smithi173.stdout:1/871: chown d5/d6/d17/d42/da/d20/d38/cdd 41255884 1 2024-04-20T11:09:48.332 INFO:tasks.workunit.client.1.smithi173.stdout:0/940: readlink d6/d8/df/d15/lc4 0 2024-04-20T11:09:48.336 INFO:tasks.workunit.client.1.smithi173.stdout:5/960: mknod da/de/d75/d3e/c14e 0 2024-04-20T11:09:48.337 INFO:tasks.workunit.client.1.smithi173.stdout:5/961: fdatasync da/de/d19/f2a 0 2024-04-20T11:09:48.338 INFO:tasks.workunit.client.1.smithi173.stdout:6/944: unlink d4/de3/l104 0 2024-04-20T11:09:48.339 INFO:tasks.workunit.client.1.smithi173.stdout:1/872: chown d5/d6/d17/ld3 297 1 2024-04-20T11:09:48.342 INFO:tasks.workunit.client.1.smithi173.stdout:0/941: rename d6/d24 to d6/d8/df/d2c/d2d/d121 0 2024-04-20T11:09:48.344 INFO:tasks.workunit.client.1.smithi173.stdout:9/914: creat d5/d9/d10f/d2b/d106/f114 x:0 0 0 2024-04-20T11:09:48.346 INFO:tasks.workunit.client.1.smithi173.stdout:0/942: read d6/d8/d3c/f4a [197784,124545] 0 2024-04-20T11:09:48.347 INFO:tasks.workunit.client.1.smithi173.stdout:5/962: mkdir da/d34/d7f/d14f 0 2024-04-20T11:09:48.347 INFO:tasks.workunit.client.1.smithi173.stdout:6/945: symlink d4/d24/d3e/d9c/df2/l12b 0 2024-04-20T11:09:48.351 INFO:tasks.workunit.client.1.smithi173.stdout:8/935: link d4/d18/d59/cab d4/d18/d59/c12e 0 2024-04-20T11:09:48.353 INFO:tasks.workunit.client.1.smithi173.stdout:1/873: symlink d5/d6/dc/d7f/d117/l11f 0 2024-04-20T11:09:48.353 INFO:tasks.workunit.client.1.smithi173.stdout:8/936: fsync d4/d18/d4f/f10e 0 2024-04-20T11:09:48.355 INFO:tasks.workunit.client.1.smithi173.stdout:0/943: chown d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/fba 847 1 2024-04-20T11:09:48.355 INFO:tasks.workunit.client.1.smithi173.stdout:5/963: mknod da/de/d75/d128/c150 0 2024-04-20T11:09:48.355 INFO:tasks.workunit.client.1.smithi173.stdout:6/946: rmdir d4/d18 39 2024-04-20T11:09:48.356 INFO:tasks.workunit.client.1.smithi173.stdout:9/915: dread d5/d9/d10f/d2b/d59/d44/d78/d9d/fb7 [0,4194304] 0 2024-04-20T11:09:48.358 INFO:tasks.workunit.client.1.smithi173.stdout:6/947: dread - d4/d1c/d8e/d90/da4/f106 zero size 2024-04-20T11:09:48.366 INFO:tasks.workunit.client.1.smithi173.stdout:1/874: dread d5/d6/d3b/f98 [0,4194304] 0 2024-04-20T11:09:48.367 INFO:tasks.workunit.client.1.smithi173.stdout:5/964: dread da/d4f/d65/d8e/fcc [0,4194304] 0 2024-04-20T11:09:48.408 INFO:tasks.workunit.client.1.smithi173.stdout:3/913: sync 2024-04-20T11:09:48.408 INFO:tasks.workunit.client.1.smithi173.stdout:7/955: sync 2024-04-20T11:09:48.417 INFO:tasks.workunit.client.1.smithi173.stdout:9/916: symlink d5/d9/d10f/d2b/d106/d79/l115 0 2024-04-20T11:09:48.417 INFO:tasks.workunit.client.1.smithi173.stdout:6/948: creat d4/d14/d5f/daf/f12c x:0 0 0 2024-04-20T11:09:48.420 INFO:tasks.workunit.client.1.smithi173.stdout:9/917: chown d5/d9/d10f/d29/d2d/cd3 2 1 2024-04-20T11:09:48.420 INFO:tasks.workunit.client.1.smithi173.stdout:6/949: readlink d4/d1c/d8e/d90/l96 0 2024-04-20T11:09:48.422 INFO:tasks.workunit.client.1.smithi173.stdout:9/918: stat d5/d9/d10f/d2b/l3e 0 2024-04-20T11:09:48.428 INFO:tasks.workunit.client.1.smithi173.stdout:3/914: symlink d7/d2b/l128 0 2024-04-20T11:09:48.431 INFO:tasks.workunit.client.1.smithi173.stdout:9/919: creat d5/d9/d10f/d2b/d59/da1/f116 x:0 0 0 2024-04-20T11:09:48.434 INFO:tasks.workunit.client.1.smithi173.stdout:7/956: creat d9/d33/d81/f144 x:0 0 0 2024-04-20T11:09:48.436 INFO:tasks.workunit.client.1.smithi173.stdout:7/957: chown d9/da/lb 38663899 1 2024-04-20T11:09:48.442 INFO:tasks.workunit.client.1.smithi173.stdout:7/958: unlink d9/d1f/d7f/d8d/c120 0 2024-04-20T11:09:48.447 INFO:tasks.workunit.client.1.smithi173.stdout:6/950: dread d4/d24/f35 [0,4194304] 0 2024-04-20T11:09:48.450 INFO:tasks.workunit.client.1.smithi173.stdout:6/951: unlink d4/f11 0 2024-04-20T11:09:48.455 INFO:tasks.workunit.client.1.smithi173.stdout:6/952: read - d4/d59/d8f/fbf zero size 2024-04-20T11:09:48.459 INFO:tasks.workunit.client.1.smithi173.stdout:6/953: write d4/d24/ffc [737955,129572] 0 2024-04-20T11:09:48.462 INFO:tasks.workunit.client.1.smithi173.stdout:7/959: dwrite d9/d33/d35/d54/db3/f109 [0,4194304] 0 2024-04-20T11:09:48.463 INFO:tasks.workunit.client.1.smithi173.stdout:6/954: creat d4/d24/d42/d74/df6/f12d x:0 0 0 2024-04-20T11:09:48.466 INFO:tasks.workunit.client.1.smithi173.stdout:6/955: write d4/d59/d8f/fa3 [844358,102994] 0 2024-04-20T11:09:48.467 INFO:tasks.workunit.client.1.smithi173.stdout:1/875: symlink d5/d6/d17/d42/da/df/d40/l120 0 2024-04-20T11:09:48.470 INFO:tasks.workunit.client.1.smithi173.stdout:1/876: readlink d5/d6/d17/d42/d12/d1f/l68 0 2024-04-20T11:09:48.473 INFO:tasks.workunit.client.1.smithi173.stdout:7/960: symlink d9/d33/d91/d11d/l145 0 2024-04-20T11:09:48.474 INFO:tasks.workunit.client.1.smithi173.stdout:6/956: rmdir d4/d18/d7e/d11b 39 2024-04-20T11:09:48.477 INFO:tasks.workunit.client.1.smithi173.stdout:1/877: creat d5/dd8/f121 x:0 0 0 2024-04-20T11:09:48.479 INFO:tasks.workunit.client.1.smithi173.stdout:8/937: write d4/d18/d59/d5a/da8/db9/de0/f7a [668653,95155] 0 2024-04-20T11:09:48.482 INFO:tasks.workunit.client.1.smithi173.stdout:5/965: mknod da/d34/c151 0 2024-04-20T11:09:48.489 INFO:tasks.workunit.client.1.smithi173.stdout:8/938: mknod d4/d18/d4f/d5b/de4/de6/d111/c12f 0 2024-04-20T11:09:48.491 INFO:tasks.workunit.client.1.smithi173.stdout:0/944: dwrite d6/d8/d3c/d44/d75/fb0 [0,4194304] 0 2024-04-20T11:09:48.493 INFO:tasks.workunit.client.1.smithi173.stdout:1/878: getdents d5/d6/d17/d42/da/d20/d38/d52 0 2024-04-20T11:09:48.494 INFO:tasks.workunit.client.1.smithi173.stdout:3/915: write f4 [1223660,73923] 0 2024-04-20T11:09:48.497 INFO:tasks.workunit.client.1.smithi173.stdout:7/961: dwrite d9/da/d17/d2b/daa/d10b/d123/d129/fdf [0,4194304] 0 2024-04-20T11:09:48.498 INFO:tasks.workunit.client.1.smithi173.stdout:8/939: fsync d4/d18/d59/d5a/f67 0 2024-04-20T11:09:48.505 INFO:tasks.workunit.client.1.smithi173.stdout:5/966: dwrite da/d34/f12a [0,4194304] 0 2024-04-20T11:09:48.509 INFO:tasks.workunit.client.1.smithi173.stdout:9/920: truncate d5/d9/d10f/d2b/d59/d44/d78/f7c 4179817 0 2024-04-20T11:09:48.509 INFO:tasks.workunit.client.1.smithi173.stdout:1/879: creat d5/dd8/df9/f122 x:0 0 0 2024-04-20T11:09:48.513 INFO:tasks.workunit.client.1.smithi173.stdout:3/916: read d7/d17/d58/d7f/fba [1630639,5821] 0 2024-04-20T11:09:48.516 INFO:tasks.workunit.client.1.smithi173.stdout:7/962: unlink d9/d33/d35/d54/db3/d102/f10d 0 2024-04-20T11:09:48.520 INFO:tasks.workunit.client.1.smithi173.stdout:7/963: truncate d9/da/dc3/f12d 1898704 0 2024-04-20T11:09:48.523 INFO:tasks.workunit.client.1.smithi173.stdout:8/940: creat d4/d18/d4f/d121/f130 x:0 0 0 2024-04-20T11:09:48.525 INFO:tasks.workunit.client.1.smithi173.stdout:8/941: chown d4/d18/d4f/d121/f130 291950838 1 2024-04-20T11:09:48.528 INFO:tasks.workunit.client.1.smithi173.stdout:1/880: mkdir d5/d6/d3b/d123 0 2024-04-20T11:09:48.530 INFO:tasks.workunit.client.1.smithi173.stdout:3/917: mkdir d7/d17/d58/d8a/d109/d129 0 2024-04-20T11:09:48.532 INFO:tasks.workunit.client.1.smithi173.stdout:7/964: write d9/dbc/fbe [760868,39725] 0 2024-04-20T11:09:48.534 INFO:tasks.workunit.client.1.smithi173.stdout:5/967: mknod da/de/d138/d145/c152 0 2024-04-20T11:09:48.540 INFO:tasks.workunit.client.1.smithi173.stdout:8/942: creat d4/d104/f131 x:0 0 0 2024-04-20T11:09:48.541 INFO:tasks.workunit.client.1.smithi173.stdout:3/918: mkdir d7/d17/d61/d90/d12a 0 2024-04-20T11:09:48.542 INFO:tasks.workunit.client.1.smithi173.stdout:3/919: dread - d7/d17/d11e/f126 zero size 2024-04-20T11:09:48.542 INFO:tasks.workunit.client.1.smithi173.stdout:7/965: mknod d9/dbc/c146 0 2024-04-20T11:09:48.544 INFO:tasks.workunit.client.1.smithi173.stdout:5/968: readlink da/d4f/d65/d8e/l119 0 2024-04-20T11:09:48.545 INFO:tasks.workunit.client.1.smithi173.stdout:8/943: getdents d4/d18/d59/d5a/d7b/db1/dce/ddd/d116 0 2024-04-20T11:09:48.557 INFO:tasks.workunit.client.1.smithi173.stdout:5/969: rename da/de/f121 to da/d34/f153 0 2024-04-20T11:09:48.558 INFO:tasks.workunit.client.1.smithi173.stdout:0/945: write d6/d8/df/d2c/d2d/d121/f74 [1994441,94941] 0 2024-04-20T11:09:48.573 INFO:tasks.workunit.client.1.smithi173.stdout:0/946: dwrite d6/d8/df/d2c/d2d/d121/f4f [4194304,4194304] 0 2024-04-20T11:09:48.588 INFO:tasks.workunit.client.1.smithi173.stdout:7/966: write d9/d33/d40/f49 [2990852,10209] 0 2024-04-20T11:09:48.589 INFO:tasks.workunit.client.1.smithi173.stdout:3/920: truncate d7/d53/dab/ffb 4190756 0 2024-04-20T11:09:48.594 INFO:tasks.workunit.client.1.smithi173.stdout:9/921: dwrite d5/d9/d10f/d2b/d59/f41 [0,4194304] 0 2024-04-20T11:09:48.596 INFO:tasks.workunit.client.1.smithi173.stdout:3/921: mknod d7/d17/d61/d83/dcf/d5c/c12b 0 2024-04-20T11:09:48.597 INFO:tasks.workunit.client.1.smithi173.stdout:7/967: creat d9/da/d127/f147 x:0 0 0 2024-04-20T11:09:48.602 INFO:tasks.workunit.client.1.smithi173.stdout:8/944: dwrite d4/d18/f9c [0,4194304] 0 2024-04-20T11:09:48.607 INFO:tasks.workunit.client.1.smithi173.stdout:3/922: symlink d7/d17/d58/d8a/d109/l12c 0 2024-04-20T11:09:48.607 INFO:tasks.workunit.client.1.smithi173.stdout:0/947: write d6/d8/df/d2c/d2d/d121/f11a [4568460,123096] 0 2024-04-20T11:09:48.610 INFO:tasks.workunit.client.1.smithi173.stdout:3/923: write d7/d17/d61/f9b [1625101,24610] 0 2024-04-20T11:09:48.613 INFO:tasks.workunit.client.1.smithi173.stdout:5/970: dwrite da/de/f37 [0,4194304] 0 2024-04-20T11:09:48.615 INFO:tasks.workunit.client.1.smithi173.stdout:8/945: dread d4/d18/fd2 [0,4194304] 0 2024-04-20T11:09:48.616 INFO:tasks.workunit.client.1.smithi173.stdout:5/971: chown da/de/d75/d40/d59/caf 48952729 1 2024-04-20T11:09:48.617 INFO:tasks.workunit.client.1.smithi173.stdout:3/924: read d7/d17/d58/fb2 [1888556,30546] 0 2024-04-20T11:09:48.629 INFO:tasks.workunit.client.1.smithi173.stdout:8/946: creat d4/d18/d59/d5a/da8/db9/de0/d9b/dc8/f132 x:0 0 0 2024-04-20T11:09:48.631 INFO:tasks.workunit.client.1.smithi173.stdout:0/948: dwrite d6/d8/d3c/dee/ff2 [0,4194304] 0 2024-04-20T11:09:48.634 INFO:tasks.workunit.client.1.smithi173.stdout:8/947: chown d4/d18/f93 41269363 1 2024-04-20T11:09:48.639 INFO:tasks.workunit.client.1.smithi173.stdout:0/949: creat d6/d5f/de9/f122 x:0 0 0 2024-04-20T11:09:48.639 INFO:tasks.workunit.client.1.smithi173.stdout:8/948: fsync d4/d18/d4f/f57 0 2024-04-20T11:09:48.640 INFO:tasks.workunit.client.1.smithi173.stdout:5/972: write da/f29 [3714923,58507] 0 2024-04-20T11:09:48.641 INFO:tasks.workunit.client.1.smithi173.stdout:7/968: dwrite d9/d33/fcc [0,4194304] 0 2024-04-20T11:09:48.645 INFO:tasks.workunit.client.1.smithi173.stdout:5/973: fdatasync da/d34/da7/fd4 0 2024-04-20T11:09:48.650 INFO:tasks.workunit.client.1.smithi173.stdout:7/969: creat d9/d33/d35/d4a/d52/df2/dc6/f148 x:0 0 0 2024-04-20T11:09:48.654 INFO:tasks.workunit.client.1.smithi173.stdout:8/949: link d4/f28 d4/f133 0 2024-04-20T11:09:48.655 INFO:tasks.workunit.client.1.smithi173.stdout:3/925: dwrite d7/f8 [4194304,4194304] 0 2024-04-20T11:09:48.656 INFO:tasks.workunit.client.1.smithi173.stdout:7/970: chown d9/d33/d35/d4a/d52/f114 500491 1 2024-04-20T11:09:48.658 INFO:tasks.workunit.client.1.smithi173.stdout:7/971: fsync d9/da/dc3/f101 0 2024-04-20T11:09:48.658 INFO:tasks.workunit.client.1.smithi173.stdout:0/950: dread d6/d8/df/d43/f4c [0,4194304] 0 2024-04-20T11:09:48.661 INFO:tasks.workunit.client.1.smithi173.stdout:7/972: truncate d9/d33/d35/d54/d135/f13b 124103 0 2024-04-20T11:09:48.666 INFO:tasks.workunit.client.1.smithi173.stdout:5/974: write da/de/d19/d32/d66/f7d [4523869,64472] 0 2024-04-20T11:09:48.672 INFO:tasks.workunit.client.1.smithi173.stdout:5/975: mknod da/de/d19/d2c/d106/d124/d14a/c154 0 2024-04-20T11:09:48.675 INFO:tasks.workunit.client.1.smithi173.stdout:5/976: mkdir da/d4f/d65/d155 0 2024-04-20T11:09:48.679 INFO:tasks.workunit.client.1.smithi173.stdout:8/950: dwrite d4/d18/d4f/f9a [0,4194304] 0 2024-04-20T11:09:48.684 INFO:tasks.workunit.client.1.smithi173.stdout:5/977: mknod da/d34/d7f/da5/d9a/c156 0 2024-04-20T11:09:48.684 INFO:tasks.workunit.client.1.smithi173.stdout:3/926: dwrite d7/d17/d61/d83/dcf/f10f [0,4194304] 0 2024-04-20T11:09:48.688 INFO:tasks.workunit.client.1.smithi173.stdout:5/978: chown da/de/d19/d32/d72/dc3 106484 1 2024-04-20T11:09:48.692 INFO:tasks.workunit.client.1.smithi173.stdout:3/927: mknod d7/d17/d61/d90/c12d 0 2024-04-20T11:09:48.693 INFO:tasks.workunit.client.1.smithi173.stdout:0/951: dread d6/d8/d3c/f4a [0,4194304] 0 2024-04-20T11:09:48.695 INFO:tasks.workunit.client.1.smithi173.stdout:8/951: link d4/d18/d59/lef d4/d18/d59/d5a/d7b/l134 0 2024-04-20T11:09:48.706 INFO:tasks.workunit.client.1.smithi173.stdout:5/979: dwrite da/d9e/f13b [0,4194304] 0 2024-04-20T11:09:48.706 INFO:tasks.workunit.client.1.smithi173.stdout:0/952: dread d6/d5f/d8f/fa7 [0,4194304] 0 2024-04-20T11:09:48.707 INFO:tasks.workunit.client.1.smithi173.stdout:0/953: fdatasync d6/d8/d51/f116 0 2024-04-20T11:09:48.708 INFO:tasks.workunit.client.1.smithi173.stdout:8/952: rename d4/d18/d59/d5a/d7b/ddc to d4/d18/d59/d5a/da8/db9/de0/d3c/d86/d135 0 2024-04-20T11:09:48.713 INFO:tasks.workunit.client.1.smithi173.stdout:0/954: write d6/d8/df/d2c/d42/f108 [1676214,97202] 0 2024-04-20T11:09:48.716 INFO:tasks.workunit.client.1.smithi173.stdout:0/955: readlink d6/l5c 0 2024-04-20T11:09:48.717 INFO:tasks.workunit.client.1.smithi173.stdout:8/953: mknod d4/d18/d59/d5a/da8/db9/de0/d3c/d86/db7/c136 0 2024-04-20T11:09:48.720 INFO:tasks.workunit.client.1.smithi173.stdout:5/980: rename f8 to da/d9e/f157 0 2024-04-20T11:09:48.724 INFO:tasks.workunit.client.1.smithi173.stdout:5/981: write da/d4f/d65/d8e/fcc [7338752,28887] 0 2024-04-20T11:09:48.725 INFO:tasks.workunit.client.1.smithi173.stdout:8/954: dread d4/ff [0,4194304] 0 2024-04-20T11:09:48.732 INFO:tasks.workunit.client.1.smithi173.stdout:5/982: truncate da/de/d75/d40/fa1 730563 0 2024-04-20T11:09:48.733 INFO:tasks.workunit.client.1.smithi173.stdout:3/928: dwrite d7/d17/f99 [0,4194304] 0 2024-04-20T11:09:48.738 INFO:tasks.workunit.client.1.smithi173.stdout:5/983: fdatasync da/de/f7e 0 2024-04-20T11:09:48.742 INFO:tasks.workunit.client.1.smithi173.stdout:1/881: sync 2024-04-20T11:09:48.742 INFO:tasks.workunit.client.1.smithi173.stdout:6/957: sync 2024-04-20T11:09:48.742 INFO:tasks.workunit.client.1.smithi173.stdout:8/955: rename d4/d18/l74 to d4/d18/d4f/l137 0 2024-04-20T11:09:48.755 INFO:tasks.workunit.client.1.smithi173.stdout:1/882: rename d5/d6/d17/d42/da/fe3 to d5/d6/d17/d42/da/df/d40/dac/f124 0 2024-04-20T11:09:48.760 INFO:tasks.workunit.client.1.smithi173.stdout:5/984: dread da/d34/f12a [0,4194304] 0 2024-04-20T11:09:48.763 INFO:tasks.workunit.client.1.smithi173.stdout:1/883: creat d5/d6/d17/d42/da/df/d40/dc8/dde/f125 x:0 0 0 2024-04-20T11:09:48.764 INFO:tasks.workunit.client.1.smithi173.stdout:0/956: dread d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d65/f73 [0,4194304] 0 2024-04-20T11:09:48.765 INFO:tasks.workunit.client.1.smithi173.stdout:6/958: dwrite d4/d24/d3e/f4d [4194304,4194304] 0 2024-04-20T11:09:48.770 INFO:tasks.workunit.client.1.smithi173.stdout:1/884: dread - d5/d6/d17/d42/da/df/d40/dc8/dde/f125 zero size 2024-04-20T11:09:48.778 INFO:tasks.workunit.client.1.smithi173.stdout:6/959: fdatasync d4/d1c/fe6 0 2024-04-20T11:09:48.787 INFO:tasks.workunit.client.1.smithi173.stdout:9/922: sync 2024-04-20T11:09:48.787 INFO:tasks.workunit.client.1.smithi173.stdout:6/960: chown d4/l49 0 1 2024-04-20T11:09:48.788 INFO:tasks.workunit.client.1.smithi173.stdout:5/985: link da/de/d75/d40/d59/c87 da/d4f/c158 0 2024-04-20T11:09:48.790 INFO:tasks.workunit.client.1.smithi173.stdout:3/929: truncate d7/d2b/fcd 469406 0 2024-04-20T11:09:48.791 INFO:tasks.workunit.client.1.smithi173.stdout:6/961: chown d4/d14/d5f/daf/fe9 25177 1 2024-04-20T11:09:48.794 INFO:tasks.workunit.client.1.smithi173.stdout:0/957: dwrite d6/d8/d3c/d44/d75/f7b [0,4194304] 0 2024-04-20T11:09:48.795 INFO:tasks.workunit.client.1.smithi173.stdout:9/923: creat d5/d9/d10f/d2b/d106/d79/f117 x:0 0 0 2024-04-20T11:09:48.804 INFO:tasks.workunit.client.1.smithi173.stdout:1/885: write d5/d6/d17/d42/da/d20/f33 [364899,27595] 0 2024-04-20T11:09:48.806 INFO:tasks.workunit.client.1.smithi173.stdout:8/956: dread d4/d18/d59/d5a/d7b/db1/dce/f129 [0,4194304] 0 2024-04-20T11:09:48.807 INFO:tasks.workunit.client.1.smithi173.stdout:6/962: creat d4/d24/d3e/f12e x:0 0 0 2024-04-20T11:09:48.812 INFO:tasks.workunit.client.1.smithi173.stdout:9/924: fdatasync d5/f38 0 2024-04-20T11:09:48.812 INFO:tasks.workunit.client.1.smithi173.stdout:8/957: fdatasync d4/d18/d59/d5a/da8/db9/de0/d3c/d86/d135/ff2 0 2024-04-20T11:09:48.813 INFO:tasks.workunit.client.1.smithi173.stdout:6/963: write d4/d18/d7e/f8a [4039479,42756] 0 2024-04-20T11:09:48.814 INFO:tasks.workunit.client.1.smithi173.stdout:1/886: stat d5/d6/d17/d42/d12/d1f/f95 0 2024-04-20T11:09:48.815 INFO:tasks.workunit.client.1.smithi173.stdout:6/964: fdatasync d4/d18/fc4 0 2024-04-20T11:09:48.816 INFO:tasks.workunit.client.1.smithi173.stdout:9/925: write d5/f2f [758589,70032] 0 2024-04-20T11:09:48.822 INFO:tasks.workunit.client.1.smithi173.stdout:9/926: creat d5/d9/d10f/d2b/f118 x:0 0 0 2024-04-20T11:09:48.823 INFO:tasks.workunit.client.1.smithi173.stdout:0/958: getdents d6/d5f/d8f/db4 0 2024-04-20T11:09:48.824 INFO:tasks.workunit.client.1.smithi173.stdout:1/887: fsync d5/d6/d17/d42/da/d20/f33 0 2024-04-20T11:09:48.831 INFO:tasks.workunit.client.1.smithi173.stdout:9/927: chown d5/d9/d10f/d2b/d59/d44/d78/d9d/la4 1535639647 1 2024-04-20T11:09:48.835 INFO:tasks.workunit.client.1.smithi173.stdout:0/959: creat d6/d8/df/d2c/d2d/d121/f123 x:0 0 0 2024-04-20T11:09:48.835 INFO:tasks.workunit.client.1.smithi173.stdout:9/928: write d5/f6 [2676436,107537] 0 2024-04-20T11:09:48.836 INFO:tasks.workunit.client.1.smithi173.stdout:3/930: truncate d7/d17/d61/d90/dca/f111 1193564 0 2024-04-20T11:09:48.841 INFO:tasks.workunit.client.1.smithi173.stdout:8/958: rename d4/d18/d59/d5a/da8/db9/de0/d3c/f73 to d4/d18/d4f/f138 0 2024-04-20T11:09:48.842 INFO:tasks.workunit.client.1.smithi173.stdout:1/888: mkdir d5/d6/d17/d42/da/df/d40/d126 0 2024-04-20T11:09:48.846 INFO:tasks.workunit.client.1.smithi173.stdout:3/931: fsync d7/d17/d58/d8a/dc4/ffe 0 2024-04-20T11:09:48.849 INFO:tasks.workunit.client.1.smithi173.stdout:8/959: write d4/d18/f93 [5176692,128485] 0 2024-04-20T11:09:48.850 INFO:tasks.workunit.client.1.smithi173.stdout:7/973: sync 2024-04-20T11:09:48.850 INFO:tasks.workunit.client.1.smithi173.stdout:0/960: creat d6/d8/df/d2c/d42/d50/d86/d8b/dcc/f124 x:0 0 0 2024-04-20T11:09:48.853 INFO:tasks.workunit.client.1.smithi173.stdout:3/932: fsync d7/f47 0 2024-04-20T11:09:48.853 INFO:tasks.workunit.client.1.smithi173.stdout:6/965: dwrite d4/d24/d42/d74/f10d [0,4194304] 0 2024-04-20T11:09:48.855 INFO:tasks.workunit.client.1.smithi173.stdout:7/974: readlink d9/d33/d35/d4a/lbd 0 2024-04-20T11:09:48.856 INFO:tasks.workunit.client.1.smithi173.stdout:5/986: dread da/f3d [0,4194304] 0 2024-04-20T11:09:48.862 INFO:tasks.workunit.client.1.smithi173.stdout:0/961: fdatasync d6/d8/df/d2c/d2d/d121/fff 0 2024-04-20T11:09:48.863 INFO:tasks.workunit.client.1.smithi173.stdout:8/960: mkdir d4/d18/d139 0 2024-04-20T11:09:48.864 INFO:tasks.workunit.client.1.smithi173.stdout:3/933: readlink d7/d17/d58/d8a/ld7 0 2024-04-20T11:09:48.865 INFO:tasks.workunit.client.1.smithi173.stdout:1/889: truncate d5/d6/d17/d42/da/df/d40/f99 1152774 0 2024-04-20T11:09:48.874 INFO:tasks.workunit.client.1.smithi173.stdout:9/929: dwrite d5/dcc/fdc [0,4194304] 0 2024-04-20T11:09:48.882 INFO:tasks.workunit.client.1.smithi173.stdout:0/962: dread d6/d8/df/d2c/d2d/d121/fff [0,4194304] 0 2024-04-20T11:09:48.916 INFO:tasks.workunit.client.1.smithi173.stdout:3/934: mknod d7/d17/d61/d90/c12e 0 2024-04-20T11:09:48.919 INFO:tasks.workunit.client.1.smithi173.stdout:3/935: dread - d7/d17/d61/d83/dcf/f11b zero size 2024-04-20T11:09:48.919 INFO:tasks.workunit.client.1.smithi173.stdout:5/987: write da/de/d75/d40/f42 [4112091,84518] 0 2024-04-20T11:09:48.921 INFO:tasks.workunit.client.1.smithi173.stdout:9/930: chown d5/d9/d10f/d2b/d59/d44/d78/f7c 0 1 2024-04-20T11:09:48.923 INFO:tasks.workunit.client.1.smithi173.stdout:0/963: rename d6/d8/d3c/dee/d107/db9/fe4 to d6/d8/df/d43/d7f/f125 0 2024-04-20T11:09:48.924 INFO:tasks.workunit.client.1.smithi173.stdout:9/931: read - d5/d9/d10f/d2b/d59/d44/de5/ffd zero size 2024-04-20T11:09:48.926 INFO:tasks.workunit.client.1.smithi173.stdout:8/961: dwrite d4/d18/d59/d5a/da8/db9/de0/f46 [0,4194304] 0 2024-04-20T11:09:48.927 INFO:tasks.workunit.client.1.smithi173.stdout:7/975: creat d9/ddc/f149 x:0 0 0 2024-04-20T11:09:48.929 INFO:tasks.workunit.client.1.smithi173.stdout:8/962: dread - d4/d18/d59/d5a/da8/db9/de0/d3c/fa0 zero size 2024-04-20T11:09:48.931 INFO:tasks.workunit.client.1.smithi173.stdout:5/988: read da/de/d75/d40/fa1 [139106,81378] 0 2024-04-20T11:09:48.934 INFO:tasks.workunit.client.1.smithi173.stdout:6/966: symlink d4/d14/d5f/d9f/d128/l12f 0 2024-04-20T11:09:48.934 INFO:tasks.workunit.client.1.smithi173.stdout:3/936: creat d7/d17/d58/d8a/d109/d129/f12f x:0 0 0 2024-04-20T11:09:48.936 INFO:tasks.workunit.client.1.smithi173.stdout:9/932: getdents d5/d9/d10f/d4f/d5e/d9e/dc2/df7 0 2024-04-20T11:09:48.938 INFO:tasks.workunit.client.1.smithi173.stdout:5/989: write da/d34/d7f/da5/d9a/f147 [105772,6787] 0 2024-04-20T11:09:48.938 INFO:tasks.workunit.client.1.smithi173.stdout:0/964: dread d6/d8/d3c/f4a [0,4194304] 0 2024-04-20T11:09:48.939 INFO:tasks.workunit.client.1.smithi173.stdout:8/963: mkdir d4/d18/d59/d5a/da8/db9/de0/d3c/d86/d135/d13a 0 2024-04-20T11:09:48.942 INFO:tasks.workunit.client.1.smithi173.stdout:7/976: getdents d9/d33/d35/d54/d10c 0 2024-04-20T11:09:48.944 INFO:tasks.workunit.client.1.smithi173.stdout:3/937: symlink d7/d17/d61/d83/d45/l130 0 2024-04-20T11:09:48.946 INFO:tasks.workunit.client.1.smithi173.stdout:1/890: dwrite d5/d6/dc/f3a [4194304,4194304] 0 2024-04-20T11:09:48.946 INFO:tasks.workunit.client.1.smithi173.stdout:0/965: fsync d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d91/f103 0 2024-04-20T11:09:48.950 INFO:tasks.workunit.client.1.smithi173.stdout:5/990: rename da/d34/d7f/da5/cea to da/de/d19/dca/d48/c159 0 2024-04-20T11:09:48.950 INFO:tasks.workunit.client.1.smithi173.stdout:9/933: creat d5/d9/d10f/d4f/d5e/d7d/dba/dd1/f119 x:0 0 0 2024-04-20T11:09:48.952 INFO:tasks.workunit.client.1.smithi173.stdout:1/891: chown d5/l10e 1768670163 1 2024-04-20T11:09:48.953 INFO:tasks.workunit.client.1.smithi173.stdout:3/938: mknod d7/d17/d58/d8a/dc4/c131 0 2024-04-20T11:09:48.955 INFO:tasks.workunit.client.1.smithi173.stdout:3/939: chown d7/d17/d58/d7f/f95 0 1 2024-04-20T11:09:48.955 INFO:tasks.workunit.client.1.smithi173.stdout:7/977: rename d9/da/cdd to d9/d33/d35/d54/d10c/c14a 0 2024-04-20T11:09:48.957 INFO:tasks.workunit.client.1.smithi173.stdout:6/967: dwrite d4/d24/d3e/d9c/f103 [0,4194304] 0 2024-04-20T11:09:48.958 INFO:tasks.workunit.client.1.smithi173.stdout:0/966: symlink d6/d8/df/d43/d7f/l126 0 2024-04-20T11:09:48.961 INFO:tasks.workunit.client.1.smithi173.stdout:5/991: mknod da/d4f/d5e/dbc/c15a 0 2024-04-20T11:09:48.966 INFO:tasks.workunit.client.1.smithi173.stdout:1/892: fsync d5/d6/d17/d42/da/d20/d48/f61 0 2024-04-20T11:09:48.969 INFO:tasks.workunit.client.1.smithi173.stdout:8/964: write d4/d18/da6/fdb [441679,48762] 0 2024-04-20T11:09:48.970 INFO:tasks.workunit.client.1.smithi173.stdout:9/934: mknod d5/d9/d10f/d29/d18/d8a/c11a 0 2024-04-20T11:09:48.980 INFO:tasks.workunit.client.1.smithi173.stdout:6/968: symlink d4/d59/d8f/d124/l130 0 2024-04-20T11:09:48.984 INFO:tasks.workunit.client.1.smithi173.stdout:1/893: dwrite d5/d6/d17/d42/da/d20/dad/dc2/fd4 [0,4194304] 0 2024-04-20T11:09:48.985 INFO:tasks.workunit.client.1.smithi173.stdout:9/935: creat d5/d9/d10f/d29/d2d/dcb/f11b x:0 0 0 2024-04-20T11:09:48.986 INFO:tasks.workunit.client.1.smithi173.stdout:5/992: dwrite da/d4f/d65/dd5/d107/d149/dbb/f135 [0,4194304] 0 2024-04-20T11:09:48.987 INFO:tasks.workunit.client.1.smithi173.stdout:7/978: link d9/d33/d85/d97/le8 d9/da/d17/d131/l14b 0 2024-04-20T11:09:48.988 INFO:tasks.workunit.client.1.smithi173.stdout:5/993: dread - da/de/d75/d40/fae zero size 2024-04-20T11:09:48.989 INFO:tasks.workunit.client.1.smithi173.stdout:6/969: mknod d4/de3/c131 0 2024-04-20T11:09:48.992 INFO:tasks.workunit.client.1.smithi173.stdout:3/940: write d7/d17/d61/d90/dca/f111 [234695,112237] 0 2024-04-20T11:09:48.992 INFO:tasks.workunit.client.1.smithi173.stdout:5/994: readlink da/d4f/d65/dd5/lf3 0 2024-04-20T11:09:48.997 INFO:tasks.workunit.client.1.smithi173.stdout:1/894: creat d5/d6/d17/d42/d12/d1f/ddc/f127 x:0 0 0 2024-04-20T11:09:48.999 INFO:tasks.workunit.client.1.smithi173.stdout:7/979: truncate d9/d33/d81/f9b 1229490 0 2024-04-20T11:09:49.002 INFO:tasks.workunit.client.1.smithi173.stdout:7/980: write d9/dbc/fbe [1893344,121652] 0 2024-04-20T11:09:49.002 INFO:tasks.workunit.client.1.smithi173.stdout:6/970: mknod d4/d14/d5f/c132 0 2024-04-20T11:09:49.014 INFO:tasks.workunit.client.1.smithi173.stdout:5/995: unlink da/de/d19/d32/f44 0 2024-04-20T11:09:49.015 INFO:tasks.workunit.client.1.smithi173.stdout:3/941: symlink d7/d17/d61/d83/l132 0 2024-04-20T11:09:49.015 INFO:tasks.workunit.client.1.smithi173.stdout:1/895: symlink d5/d115/l128 0 2024-04-20T11:09:49.016 INFO:tasks.workunit.client.1.smithi173.stdout:6/971: creat d4/d18/d7e/d11b/f133 x:0 0 0 2024-04-20T11:09:49.020 INFO:tasks.workunit.client.1.smithi173.stdout:6/972: write d4/f37 [4483424,12383] 0 2024-04-20T11:09:49.021 INFO:tasks.workunit.client.1.smithi173.stdout:9/936: dwrite d5/d9/d10f/d2b/d106/fc0 [0,4194304] 0 2024-04-20T11:09:49.023 INFO:tasks.workunit.client.1.smithi173.stdout:5/996: rename da/de/d19/d2c/fa8 to da/de/d19/dca/d48/f15b 0 2024-04-20T11:09:49.024 INFO:tasks.workunit.client.1.smithi173.stdout:9/937: dread - d5/d9/d10f/d2b/f118 zero size 2024-04-20T11:09:49.028 INFO:tasks.workunit.client.1.smithi173.stdout:5/997: chown da/de/d19/dca/d48/da2/f110 216638940 1 2024-04-20T11:09:49.029 INFO:tasks.workunit.client.1.smithi173.stdout:8/965: dread d4/d18/d59/d5a/d7b/d8c/f8e [0,4194304] 0 2024-04-20T11:09:49.031 INFO:tasks.workunit.client.1.smithi173.stdout:9/938: chown d5/c1e 9 1 2024-04-20T11:09:49.033 INFO:tasks.workunit.client.1.smithi173.stdout:3/942: creat d7/d17/d61/d83/dcf/f133 x:0 0 0 2024-04-20T11:09:49.036 INFO:tasks.workunit.client.1.smithi173.stdout:6/973: dwrite d4/f20 [0,4194304] 0 2024-04-20T11:09:49.039 INFO:tasks.workunit.client.1.smithi173.stdout:6/974: truncate d4/d18/d61/ffe 630317 0 2024-04-20T11:09:49.040 INFO:tasks.workunit.client.1.smithi173.stdout:5/998: symlink da/de/d19/d2c/d106/d124/d14a/l15c 0 2024-04-20T11:09:49.042 INFO:tasks.workunit.client.1.smithi173.stdout:1/896: link d5/d6/d17/d42/da/df/d40/c111 d5/d6/dc/d7f/c129 0 2024-04-20T11:09:49.045 INFO:tasks.workunit.client.1.smithi173.stdout:9/939: creat d5/dd7/df4/f11c x:0 0 0 2024-04-20T11:09:49.045 INFO:tasks.workunit.client.1.smithi173.stdout:3/943: mkdir d7/d85/df5/d122/d134 0 2024-04-20T11:09:49.047 INFO:tasks.workunit.client.1.smithi173.stdout:1/897: write d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fdb [2764613,38115] 0 2024-04-20T11:09:49.048 INFO:tasks.workunit.client.1.smithi173.stdout:8/966: mknod d4/d18/d4f/c13b 0 2024-04-20T11:09:49.049 INFO:tasks.workunit.client.1.smithi173.stdout:5/999: symlink da/de/d19/d32/d72/dc3/l15d 0 2024-04-20T11:09:49.052 INFO:tasks.workunit.client.1.smithi173.stdout:3/944: mkdir d7/d135 0 2024-04-20T11:09:49.053 INFO:tasks.workunit.client.1.smithi173.stdout:1/898: symlink d5/d6/d3b/l12a 0 2024-04-20T11:09:49.054 INFO:tasks.workunit.client.1.smithi173.stdout:8/967: rmdir d4/d18 39 2024-04-20T11:09:49.060 INFO:tasks.workunit.client.1.smithi173.stdout:1/899: mknod d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/c12b 0 2024-04-20T11:09:49.062 INFO:tasks.workunit.client.1.smithi173.stdout:8/968: symlink d4/d18/d139/l13c 0 2024-04-20T11:09:49.069 INFO:tasks.workunit.client.1.smithi173.stdout:8/969: creat d4/d18/d59/d5a/da8/db9/de0/d3c/d86/db7/f13d x:0 0 0 2024-04-20T11:09:49.072 INFO:tasks.workunit.client.1.smithi173.stdout:1/900: dread d5/d6/d17/d42/da/df/d40/dc8/fcc [0,4194304] 0 2024-04-20T11:09:49.073 INFO:tasks.workunit.client.1.smithi173.stdout:6/975: dwrite d4/d59/d8f/fbf [0,4194304] 0 2024-04-20T11:09:49.074 INFO:tasks.workunit.client.1.smithi173.stdout:9/940: dread d5/d9/d10f/d29/d18/d9b/fbc [0,4194304] 0 2024-04-20T11:09:49.075 INFO:tasks.workunit.client.1.smithi173.stdout:3/945: unlink d7/d2b/d9d/l10e 0 2024-04-20T11:09:49.077 INFO:tasks.workunit.client.1.smithi173.stdout:9/941: dread - d5/d9/d10f/d2b/d106/d79/d80/f8d zero size 2024-04-20T11:09:49.077 INFO:tasks.workunit.client.1.smithi173.stdout:6/976: fsync d4/d1c/fe6 0 2024-04-20T11:09:49.081 INFO:tasks.workunit.client.1.smithi173.stdout:3/946: write d7/d17/d61/dd6/ffc [309129,37779] 0 2024-04-20T11:09:49.083 INFO:tasks.workunit.client.1.smithi173.stdout:1/901: rename d5/d6/d17/d42/da/df/l4e to d5/d6/d17/d42/da/d20/d38/d52/d75/l12c 0 2024-04-20T11:09:49.084 INFO:tasks.workunit.client.1.smithi173.stdout:1/902: stat d5/d6/d17/d93/lc7 0 2024-04-20T11:09:49.086 INFO:tasks.workunit.client.1.smithi173.stdout:1/903: readlink d5/d6/d17/d42/d12/dc9/lda 0 2024-04-20T11:09:49.087 INFO:tasks.workunit.client.1.smithi173.stdout:1/904: truncate d5/d6/dc/fe 5741595 0 2024-04-20T11:09:49.096 INFO:tasks.workunit.client.1.smithi173.stdout:1/905: rmdir d5/d6 39 2024-04-20T11:09:49.096 INFO:tasks.workunit.client.1.smithi173.stdout:8/970: truncate d4/d18/d59/d5a/da8/db9/de0/f9d 1048136 0 2024-04-20T11:09:49.099 INFO:tasks.workunit.client.1.smithi173.stdout:9/942: unlink d5/d9/d10f/d4f/d5e/d7d/d95/l103 0 2024-04-20T11:09:49.100 INFO:tasks.workunit.client.1.smithi173.stdout:3/947: creat d7/d53/dab/f136 x:0 0 0 2024-04-20T11:09:49.101 INFO:tasks.workunit.client.1.smithi173.stdout:6/977: rename d4/d24/d42/d74/df6/c127 to d4/d24/d3e/d9c/dc7/c134 0 2024-04-20T11:09:49.103 INFO:tasks.workunit.client.1.smithi173.stdout:3/948: creat d7/d53/dab/f137 x:0 0 0 2024-04-20T11:09:49.103 INFO:tasks.workunit.client.1.smithi173.stdout:9/943: truncate d5/d9/d10f/d2b/d59/d48/fe2 1278286 0 2024-04-20T11:09:49.103 INFO:tasks.workunit.client.1.smithi173.stdout:6/978: write d4/d14/f1a [385922,81717] 0 2024-04-20T11:09:49.104 INFO:tasks.workunit.client.1.smithi173.stdout:3/949: stat d7/d17/d11e 0 2024-04-20T11:09:49.104 INFO:tasks.workunit.client.1.smithi173.stdout:8/971: link d4/d18/d4f/fc6 d4/d18/da6/f13e 0 2024-04-20T11:09:49.106 INFO:tasks.workunit.client.1.smithi173.stdout:6/979: dread - d4/d59/d8f/f118 zero size 2024-04-20T11:09:49.107 INFO:tasks.workunit.client.1.smithi173.stdout:3/950: dread d7/d2b/d8e/f114 [0,4194304] 0 2024-04-20T11:09:49.109 INFO:tasks.workunit.client.1.smithi173.stdout:3/951: dread - d7/d17/d58/d8a/dc4/dd1/fec zero size 2024-04-20T11:09:49.109 INFO:tasks.workunit.client.1.smithi173.stdout:8/972: unlink d4/lb 0 2024-04-20T11:09:49.110 INFO:tasks.workunit.client.1.smithi173.stdout:9/944: fsync d5/d9/d10f/d29/d18/f70 0 2024-04-20T11:09:49.111 INFO:tasks.workunit.client.1.smithi173.stdout:3/952: stat d7/d17/d58/d7f/df2 0 2024-04-20T11:09:49.112 INFO:tasks.workunit.client.1.smithi173.stdout:6/980: unlink d4/d24/d3e/f4c 0 2024-04-20T11:09:49.118 INFO:tasks.workunit.client.1.smithi173.stdout:8/973: rename d4/d18/d59/d5a/da8/fae to d4/d18/d59/d5a/da8/db9/f13f 0 2024-04-20T11:09:49.120 INFO:tasks.workunit.client.1.smithi173.stdout:8/974: read - d4/d18/d4f/d5b/de4/de6/deb/f115 zero size 2024-04-20T11:09:49.123 INFO:tasks.workunit.client.1.smithi173.stdout:9/945: link d5/d9/d10f/d2b/d59/d44/f6a d5/d9/d10f/d2b/d59/da1/f11d 0 2024-04-20T11:09:49.126 INFO:tasks.workunit.client.1.smithi173.stdout:3/953: rmdir d7/d17/d61/d83/dc2/d105 0 2024-04-20T11:09:49.130 INFO:tasks.workunit.client.1.smithi173.stdout:9/946: mknod d5/dd7/c11e 0 2024-04-20T11:09:49.169 INFO:tasks.workunit.client.1.smithi173.stdout:3/954: symlink d7/d17/d58/df3/l138 0 2024-04-20T11:09:49.171 INFO:tasks.workunit.client.1.smithi173.stdout:6/981: truncate d4/d14/d79/fbc 823194 0 2024-04-20T11:09:49.171 INFO:tasks.workunit.client.1.smithi173.stdout:8/975: link d4/d18/d59/d5a/da8/db9/de0/d3c/f37 d4/d18/d59/d5a/d7b/f140 0 2024-04-20T11:09:49.172 INFO:tasks.workunit.client.1.smithi173.stdout:9/947: write d5/d9/d10f/d2b/d59/d44/ff9 [207374,89455] 0 2024-04-20T11:09:49.176 INFO:tasks.workunit.client.1.smithi173.stdout:3/955: chown d7/d17/d58/d8a/dc4/dd1/ff0 46295198 1 2024-04-20T11:09:49.176 INFO:tasks.workunit.client.1.smithi173.stdout:8/976: fdatasync d4/d18/d4f/f7d 0 2024-04-20T11:09:49.179 INFO:tasks.workunit.client.1.smithi173.stdout:6/982: rename d4/d24/d3e/d9c/df2/l12b to d4/d59/d125/l135 0 2024-04-20T11:09:49.179 INFO:tasks.workunit.client.1.smithi173.stdout:3/956: stat d7/d17/d61/d83/dcf/f133 0 2024-04-20T11:09:49.183 INFO:tasks.workunit.client.1.smithi173.stdout:6/983: fsync d4/d59/fc1 0 2024-04-20T11:09:49.184 INFO:tasks.workunit.client.1.smithi173.stdout:8/977: mkdir d4/d18/d59/d5a/da8/db9/de0/d3c/d141 0 2024-04-20T11:09:49.187 INFO:tasks.workunit.client.1.smithi173.stdout:6/984: chown d4/d18/fba 82 1 2024-04-20T11:09:49.189 INFO:tasks.workunit.client.1.smithi173.stdout:3/957: getdents d7/d17/d61/d83/d45 0 2024-04-20T11:09:49.193 INFO:tasks.workunit.client.1.smithi173.stdout:6/985: symlink d4/d14/l136 0 2024-04-20T11:09:49.195 INFO:tasks.workunit.client.1.smithi173.stdout:3/958: write d7/d17/d61/d83/ff4 [512295,18148] 0 2024-04-20T11:09:49.197 INFO:tasks.workunit.client.1.smithi173.stdout:6/986: chown d4/d1c/lde 48328119 1 2024-04-20T11:09:49.200 INFO:tasks.workunit.client.1.smithi173.stdout:8/978: rename d4/d18/d4f/d5b/de4/l12c to d4/d18/d59/d5a/da8/db9/de0/d3c/d86/d135/l142 0 2024-04-20T11:09:49.203 INFO:tasks.workunit.client.1.smithi173.stdout:6/987: write d4/d18/d61/ffe [626036,103342] 0 2024-04-20T11:09:49.205 INFO:tasks.workunit.client.1.smithi173.stdout:3/959: creat d7/d17/d58/d8a/f139 x:0 0 0 2024-04-20T11:09:49.206 INFO:tasks.workunit.client.1.smithi173.stdout:9/948: dwrite d5/d9/fc5 [0,4194304] 0 2024-04-20T11:09:49.225 INFO:tasks.workunit.client.1.smithi173.stdout:8/979: creat d4/d18/d59/d5a/f143 x:0 0 0 2024-04-20T11:09:49.228 INFO:tasks.workunit.client.1.smithi173.stdout:3/960: getdents d7/d17/d58/d8a/d109/d129 0 2024-04-20T11:09:49.229 INFO:tasks.workunit.client.1.smithi173.stdout:6/988: dread d4/f46 [0,4194304] 0 2024-04-20T11:09:49.239 INFO:tasks.workunit.client.1.smithi173.stdout:9/949: dread d5/d9/d10f/d29/d18/d9b/fb3 [0,4194304] 0 2024-04-20T11:09:49.247 INFO:tasks.workunit.client.1.smithi173.stdout:8/980: dwrite d4/d18/f72 [0,4194304] 0 2024-04-20T11:09:49.252 INFO:tasks.workunit.client.1.smithi173.stdout:6/989: dwrite d4/d24/ffc [0,4194304] 0 2024-04-20T11:09:49.255 INFO:tasks.workunit.client.1.smithi173.stdout:3/961: dwrite d7/d17/d61/d83/dcf/d30/f80 [0,4194304] 0 2024-04-20T11:09:49.257 INFO:tasks.workunit.client.1.smithi173.stdout:3/962: dread - d7/d17/d11e/f126 zero size 2024-04-20T11:09:49.261 INFO:tasks.workunit.client.1.smithi173.stdout:8/981: creat d4/d18/d59/d5a/f144 x:0 0 0 2024-04-20T11:09:49.262 INFO:tasks.workunit.client.1.smithi173.stdout:9/950: dread d5/d9/d10f/d2b/d106/d79/f7a [0,4194304] 0 2024-04-20T11:09:49.265 INFO:tasks.workunit.client.1.smithi173.stdout:3/963: creat d7/d17/d58/d8a/dc4/dd1/f13a x:0 0 0 2024-04-20T11:09:49.268 INFO:tasks.workunit.client.1.smithi173.stdout:9/951: write d5/d9/d10f/d29/d2d/dcb/f11b [141083,113571] 0 2024-04-20T11:09:49.270 INFO:tasks.workunit.client.1.smithi173.stdout:7/981: sync 2024-04-20T11:09:49.270 INFO:tasks.workunit.client.1.smithi173.stdout:0/967: sync 2024-04-20T11:09:49.271 INFO:tasks.workunit.client.1.smithi173.stdout:1/906: sync 2024-04-20T11:09:49.272 INFO:tasks.workunit.client.1.smithi173.stdout:3/964: mkdir d7/d17/d61/d83/dcf/d30/d13b 0 2024-04-20T11:09:49.273 INFO:tasks.workunit.client.1.smithi173.stdout:9/952: mknod d5/d9/d10f/d2b/d31/c11f 0 2024-04-20T11:09:49.280 INFO:tasks.workunit.client.1.smithi173.stdout:7/982: creat d9/d33/d35/d54/f14c x:0 0 0 2024-04-20T11:09:49.281 INFO:tasks.workunit.client.1.smithi173.stdout:3/965: fdatasync d7/d17/f2c 0 2024-04-20T11:09:49.283 INFO:tasks.workunit.client.1.smithi173.stdout:6/990: dwrite d4/d18/d7e/d11b/f133 [0,4194304] 0 2024-04-20T11:09:49.337 INFO:tasks.workunit.client.1.smithi173.stdout:1/907: getdents d5/d6/dc/d7f/d117 0 2024-04-20T11:09:49.339 INFO:tasks.workunit.client.1.smithi173.stdout:8/982: dwrite d4/d18/f6b [0,4194304] 0 2024-04-20T11:09:49.341 INFO:tasks.workunit.client.1.smithi173.stdout:7/983: mknod d9/d1f/d72/d8f/d142/c14d 0 2024-04-20T11:09:49.345 INFO:tasks.workunit.client.1.smithi173.stdout:3/966: creat d7/d85/df5/d122/f13c x:0 0 0 2024-04-20T11:09:49.347 INFO:tasks.workunit.client.1.smithi173.stdout:8/983: mkdir d4/d104/d145 0 2024-04-20T11:09:49.347 INFO:tasks.workunit.client.1.smithi173.stdout:0/968: dwrite d6/d8/d3c/f4a [0,4194304] 0 2024-04-20T11:09:49.349 INFO:tasks.workunit.client.1.smithi173.stdout:0/969: readlink d6/d8/d3c/d44/d75/l85 0 2024-04-20T11:09:49.353 INFO:tasks.workunit.client.1.smithi173.stdout:7/984: mkdir d9/da/d17/d2b/daa/d10b/d123/d14e 0 2024-04-20T11:09:49.355 INFO:tasks.workunit.client.1.smithi173.stdout:7/985: dread - d9/d33/d35/d56/fd0 zero size 2024-04-20T11:09:49.356 INFO:tasks.workunit.client.1.smithi173.stdout:0/970: truncate d6/d8/d3c/dee/d107/db9/f117 784296 0 2024-04-20T11:09:49.363 INFO:tasks.workunit.client.1.smithi173.stdout:6/991: getdents d4/d18 0 2024-04-20T11:09:49.363 INFO:tasks.workunit.client.1.smithi173.stdout:8/984: rmdir d4/d18/da6 39 2024-04-20T11:09:49.371 INFO:tasks.workunit.client.1.smithi173.stdout:0/971: rename d6/d8/df/f11 to d6/d8/df/d2c/d42/d50/d86/d8b/da2/de8/f127 0 2024-04-20T11:09:49.372 INFO:tasks.workunit.client.1.smithi173.stdout:7/986: dread d9/d33/d81/f9b [0,4194304] 0 2024-04-20T11:09:49.375 INFO:tasks.workunit.client.1.smithi173.stdout:6/992: write d4/d24/f2e [475061,18207] 0 2024-04-20T11:09:49.376 INFO:tasks.workunit.client.1.smithi173.stdout:6/993: stat d4/d18/f113 0 2024-04-20T11:09:49.382 INFO:tasks.workunit.client.1.smithi173.stdout:8/985: rename d4/d18/d59/d5a/d7b/ld8 to d4/d18/d59/d5a/d7b/d8c/l146 0 2024-04-20T11:09:49.387 INFO:tasks.workunit.client.1.smithi173.stdout:7/987: chown d9/d33/d35/l5d 224485035 1 2024-04-20T11:09:49.387 INFO:tasks.workunit.client.1.smithi173.stdout:1/908: dwrite d5/d6/d17/d42/da/d20/fd6 [0,4194304] 0 2024-04-20T11:09:49.389 INFO:tasks.workunit.client.1.smithi173.stdout:7/988: stat d9/d33/d35/d54/db3 0 2024-04-20T11:09:49.390 INFO:tasks.workunit.client.1.smithi173.stdout:8/986: truncate d4/d18/d59/d5a/da8/db9/de0/d9b/f12d 1122990 0 2024-04-20T11:09:49.391 INFO:tasks.workunit.client.1.smithi173.stdout:6/994: fsync d4/d24/d3e/f57 0 2024-04-20T11:09:49.395 INFO:tasks.workunit.client.1.smithi173.stdout:3/967: dwrite d7/d2b/d8e/fd5 [0,4194304] 0 2024-04-20T11:09:49.395 INFO:tasks.workunit.client.1.smithi173.stdout:7/989: dread - d9/d1f/d72/f116 zero size 2024-04-20T11:09:49.396 INFO:tasks.workunit.client.1.smithi173.stdout:3/968: write d7/d17/fa8 [1004171,42180] 0 2024-04-20T11:09:49.399 INFO:tasks.workunit.client.1.smithi173.stdout:1/909: truncate d5/d6/f21 3191611 0 2024-04-20T11:09:49.402 INFO:tasks.workunit.client.1.smithi173.stdout:0/972: rename d6/d8/df/d2c/d42/d50/fb5 to d6/d5f/d8f/dd6/f128 0 2024-04-20T11:09:49.402 INFO:tasks.workunit.client.1.smithi173.stdout:8/987: read d4/d18/d4f/f138 [5214426,84288] 0 2024-04-20T11:09:49.404 INFO:tasks.workunit.client.1.smithi173.stdout:6/995: mknod d4/d1c/c137 0 2024-04-20T11:09:49.404 INFO:tasks.workunit.client.1.smithi173.stdout:7/990: symlink d9/d1f/d7f/l14f 0 2024-04-20T11:09:49.404 INFO:tasks.workunit.client.1.smithi173.stdout:0/973: chown d6/d8/d3c/d44/ffa 83 1 2024-04-20T11:09:49.406 INFO:tasks.workunit.client.1.smithi173.stdout:7/991: stat d9/dbc/fbe 0 2024-04-20T11:09:49.407 INFO:tasks.workunit.client.1.smithi173.stdout:1/910: mknod d5/d6/dc/d7f/c12d 0 2024-04-20T11:09:49.409 INFO:tasks.workunit.client.1.smithi173.stdout:8/988: readlink d4/d18/lfb 0 2024-04-20T11:09:49.409 INFO:tasks.workunit.client.1.smithi173.stdout:1/911: chown d5/d6/d17/d42/da/df/c6d 287979 1 2024-04-20T11:09:49.411 INFO:tasks.workunit.client.1.smithi173.stdout:6/996: creat d4/d24/d3e/f138 x:0 0 0 2024-04-20T11:09:49.412 INFO:tasks.workunit.client.1.smithi173.stdout:8/989: chown d4/d18/d4f/c13b 7070343 1 2024-04-20T11:09:49.415 INFO:tasks.workunit.client.1.smithi173.stdout:0/974: creat d6/d8/d3c/d77/f129 x:0 0 0 2024-04-20T11:09:49.418 INFO:tasks.workunit.client.1.smithi173.stdout:1/912: fdatasync d5/d6/d17/d42/da/d20/dad/dc2/d94/fa1 0 2024-04-20T11:09:49.419 INFO:tasks.workunit.client.1.smithi173.stdout:6/997: rename d4/d24/d3e/lab to d4/d18/d7e/d120/d123/l139 0 2024-04-20T11:09:49.422 INFO:tasks.workunit.client.1.smithi173.stdout:3/969: dwrite d7/f11a [0,4194304] 0 2024-04-20T11:09:49.422 INFO:tasks.workunit.client.1.smithi173.stdout:0/975: rmdir d6/d8/df/d2c/d2d/d121 39 2024-04-20T11:09:49.426 INFO:tasks.workunit.client.1.smithi173.stdout:1/913: creat d5/d115/f12e x:0 0 0 2024-04-20T11:09:49.429 INFO:tasks.workunit.client.1.smithi173.stdout:0/976: fsync d6/d8/df/d2c/d42/d50/d86/d8b/da2/daf/ffd 0 2024-04-20T11:09:49.429 INFO:tasks.workunit.client.1.smithi173.stdout:7/992: dwrite d9/d33/d35/d4a/d52/df2/dc6/f148 [0,4194304] 0 2024-04-20T11:09:49.431 INFO:tasks.workunit.client.1.smithi173.stdout:6/998: symlink d4/d18/l13a 0 2024-04-20T11:09:49.432 INFO:tasks.workunit.client.1.smithi173.stdout:1/914: truncate d5/fb 591034 0 2024-04-20T11:09:49.432 INFO:tasks.workunit.client.1.smithi173.stdout:7/993: read d9/f4e [595714,119341] 0 2024-04-20T11:09:49.437 INFO:tasks.workunit.client.1.smithi173.stdout:9/953: sync 2024-04-20T11:09:49.439 INFO:tasks.workunit.client.1.smithi173.stdout:6/999: creat d4/d14/d5f/daf/f13b x:0 0 0 2024-04-20T11:09:49.440 INFO:tasks.workunit.client.1.smithi173.stdout:8/990: link d4/d18/d59/d5a/da8/db9/de0/l85 d4/d18/d59/d5a/d7b/db1/l147 0 2024-04-20T11:09:49.448 INFO:tasks.workunit.client.1.smithi173.stdout:7/994: mknod d9/da/dc3/c150 0 2024-04-20T11:09:49.454 INFO:tasks.workunit.client.1.smithi173.stdout:3/970: dread d7/d17/f19 [0,4194304] 0 2024-04-20T11:09:49.458 INFO:tasks.workunit.client.1.smithi173.stdout:7/995: unlink d9/d33/d35/d4a/d52/df2/d13a/f8c 0 2024-04-20T11:09:49.461 INFO:tasks.workunit.client.1.smithi173.stdout:7/996: stat d9/d33/de9/c134 0 2024-04-20T11:09:49.463 INFO:tasks.workunit.client.1.smithi173.stdout:0/977: truncate d6/d8/df/d15/f70 566608 0 2024-04-20T11:09:49.469 INFO:tasks.workunit.client.1.smithi173.stdout:8/991: write d4/d18/fa7 [2213049,43983] 0 2024-04-20T11:09:49.469 INFO:tasks.workunit.client.1.smithi173.stdout:9/954: dwrite d5/d9/d10f/d2b/d106/f72 [0,4194304] 0 2024-04-20T11:09:49.471 INFO:tasks.workunit.client.1.smithi173.stdout:0/978: chown d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/lf3 21260 1 2024-04-20T11:09:49.472 INFO:tasks.workunit.client.1.smithi173.stdout:9/955: chown d5/d9/d10f/d29/l35 216952095 1 2024-04-20T11:09:49.473 INFO:tasks.workunit.client.1.smithi173.stdout:1/915: dread d5/d6/d17/d93/db6/fcf [0,4194304] 0 2024-04-20T11:09:49.479 INFO:tasks.workunit.client.1.smithi173.stdout:9/956: mkdir d5/d9/d10f/d4f/d120 0 2024-04-20T11:09:49.481 INFO:tasks.workunit.client.1.smithi173.stdout:1/916: creat d5/d6/d17/d42/da/df/d40/dc8/f12f x:0 0 0 2024-04-20T11:09:49.486 INFO:tasks.workunit.client.1.smithi173.stdout:0/979: getdents d6 0 2024-04-20T11:09:49.487 INFO:tasks.workunit.client.1.smithi173.stdout:7/997: dwrite d9/d1f/d7f/dce/f5f [4194304,4194304] 0 2024-04-20T11:09:49.495 INFO:tasks.workunit.client.1.smithi173.stdout:9/957: rename d5/d9/d10f/d2b/d106/fc0 to d5/d9/d10f/d2b/d59/da1/dee/f121 0 2024-04-20T11:09:49.499 INFO:tasks.workunit.client.1.smithi173.stdout:1/917: dread d5/d6/d17/ff5 [0,4194304] 0 2024-04-20T11:09:49.499 INFO:tasks.workunit.client.1.smithi173.stdout:7/998: symlink d9/d33/l151 0 2024-04-20T11:09:49.502 INFO:tasks.workunit.client.1.smithi173.stdout:1/918: write d5/d6/d17/d42/ff2 [818087,117786] 0 2024-04-20T11:09:49.502 INFO:tasks.workunit.client.1.smithi173.stdout:7/999: stat d9/d33/d35/d4a/d52/df2/cf7 0 2024-04-20T11:09:49.503 INFO:tasks.workunit.client.1.smithi173.stdout:0/980: creat d6/d8/df/d2c/d42/d50/f12a x:0 0 0 2024-04-20T11:09:49.506 INFO:tasks.workunit.client.1.smithi173.stdout:1/919: unlink d5/f108 0 2024-04-20T11:09:49.509 INFO:tasks.workunit.client.1.smithi173.stdout:0/981: mknod d6/d8/df/d2c/d42/c12b 0 2024-04-20T11:09:49.511 INFO:tasks.workunit.client.1.smithi173.stdout:1/920: creat d5/d6/d3b/d123/f130 x:0 0 0 2024-04-20T11:09:49.517 INFO:tasks.workunit.client.1.smithi173.stdout:9/958: dwrite d5/d9/d10f/d29/d2d/dcb/fd5 [0,4194304] 0 2024-04-20T11:09:49.517 INFO:tasks.workunit.client.1.smithi173.stdout:1/921: truncate d5/d6/f73 1386627 0 2024-04-20T11:09:49.522 INFO:tasks.workunit.client.1.smithi173.stdout:0/982: dread d6/f29 [0,4194304] 0 2024-04-20T11:09:49.530 INFO:tasks.workunit.client.1.smithi173.stdout:8/992: dread d4/f6a [0,4194304] 0 2024-04-20T11:09:49.532 INFO:tasks.workunit.client.1.smithi173.stdout:0/983: mknod d6/d5f/d8f/c12c 0 2024-04-20T11:09:49.535 INFO:tasks.workunit.client.1.smithi173.stdout:9/959: dwrite d5/d9/d10f/d4f/d5e/d7d/dba/dd1/f119 [0,4194304] 0 2024-04-20T11:09:49.539 INFO:tasks.workunit.client.1.smithi173.stdout:8/993: fsync d4/d18/d59/f64 0 2024-04-20T11:09:49.540 INFO:tasks.workunit.client.1.smithi173.stdout:0/984: fsync d6/d8/d3c/f41 0 2024-04-20T11:09:49.556 INFO:tasks.workunit.client.1.smithi173.stdout:8/994: chown d4/f133 909 1 2024-04-20T11:09:49.559 INFO:tasks.workunit.client.1.smithi173.stdout:1/922: getdents d5/d6/d17/d42/da/d20/d48 0 2024-04-20T11:09:49.563 INFO:tasks.workunit.client.1.smithi173.stdout:1/923: mknod d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/c131 0 2024-04-20T11:09:49.593 INFO:tasks.workunit.client.1.smithi173.stdout:1/924: dwrite d5/d6/d17/d42/d12/f15 [0,4194304] 0 2024-04-20T11:09:49.600 INFO:tasks.workunit.client.1.smithi173.stdout:0/985: write d6/f29 [2247780,55442] 0 2024-04-20T11:09:49.601 INFO:tasks.workunit.client.1.smithi173.stdout:8/995: dread d4/d18/d4f/fca [0,4194304] 0 2024-04-20T11:09:49.605 INFO:tasks.workunit.client.1.smithi173.stdout:1/925: unlink d5/d6/d17/d42/da/d20/ldf 0 2024-04-20T11:09:49.605 INFO:tasks.workunit.client.1.smithi173.stdout:0/986: fdatasync d6/d8/df/fe0 0 2024-04-20T11:09:49.613 INFO:tasks.workunit.client.1.smithi173.stdout:1/926: rmdir d5/d6/d17/d42/da/d20/d38/d52/d75 39 2024-04-20T11:09:49.615 INFO:tasks.workunit.client.1.smithi173.stdout:0/987: symlink d6/d8/df/d15/l12d 0 2024-04-20T11:09:49.618 INFO:tasks.workunit.client.1.smithi173.stdout:8/996: dread d4/d18/d4f/f7e [0,4194304] 0 2024-04-20T11:09:49.620 INFO:tasks.workunit.client.1.smithi173.stdout:0/988: stat d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/d65/f98 0 2024-04-20T11:09:49.623 INFO:tasks.workunit.client.1.smithi173.stdout:1/927: rename d5/d6/d17/d42/da/df/d40/db4/fd2 to d5/d6/d17/d42/da/d20/d48/f132 0 2024-04-20T11:09:49.627 INFO:tasks.workunit.client.1.smithi173.stdout:8/997: symlink d4/d18/d4f/d5b/de4/de6/l148 0 2024-04-20T11:09:49.632 INFO:tasks.workunit.client.1.smithi173.stdout:0/989: write d6/d8/df/d2c/d42/d50/d86/d8b/da2/de8/f127 [2899823,65187] 0 2024-04-20T11:09:49.635 INFO:tasks.workunit.client.1.smithi173.stdout:8/998: creat d4/d104/d145/f149 x:0 0 0 2024-04-20T11:09:49.637 INFO:tasks.workunit.client.1.smithi173.stdout:0/990: unlink d6/f7 0 2024-04-20T11:09:49.649 INFO:tasks.workunit.client.1.smithi173.stdout:1/928: write d5/d6/d17/f9a [2102695,87268] 0 2024-04-20T11:09:49.653 INFO:tasks.workunit.client.1.smithi173.stdout:3/971: sync 2024-04-20T11:09:49.654 INFO:tasks.workunit.client.1.smithi173.stdout:1/929: dread - d5/d6/d17/d42/d12/d1c/d85/ffe zero size 2024-04-20T11:09:49.655 INFO:tasks.workunit.client.1.smithi173.stdout:3/972: write f4 [793507,6428] 0 2024-04-20T11:09:49.656 INFO:tasks.workunit.client.1.smithi173.stdout:8/999: dwrite d4/d18/d59/d5a/f144 [0,4194304] 0 2024-04-20T11:09:49.656 INFO:tasks.workunit.client.1.smithi173.stdout:0/991: unlink d6/f29 0 2024-04-20T11:09:49.660 INFO:tasks.workunit.client.1.smithi173.stdout:1/930: symlink d5/d6/d17/d42/d12/dc9/l133 0 2024-04-20T11:09:49.661 INFO:tasks.workunit.client.1.smithi173.stdout:3/973: truncate d7/d17/f9f 520224 0 2024-04-20T11:09:49.661 INFO:tasks.workunit.client.1.smithi173.stdout:1/931: readlink d5/d6/l92 0 2024-04-20T11:09:49.663 INFO:tasks.workunit.client.1.smithi173.stdout:3/974: fdatasync d7/d17/d11e/f126 0 2024-04-20T11:09:49.664 INFO:tasks.workunit.client.1.smithi173.stdout:0/992: creat d6/d8/df/dd0/dd4/f12e x:0 0 0 2024-04-20T11:09:49.669 INFO:tasks.workunit.client.1.smithi173.stdout:0/993: creat d6/d8/df/d2c/d42/d50/de3/f12f x:0 0 0 2024-04-20T11:09:49.671 INFO:tasks.workunit.client.1.smithi173.stdout:3/975: mkdir d7/d2b/d13d 0 2024-04-20T11:09:49.675 INFO:tasks.workunit.client.1.smithi173.stdout:1/932: truncate d5/d6/d17/f27 4506317 0 2024-04-20T11:09:49.678 INFO:tasks.workunit.client.1.smithi173.stdout:0/994: mkdir d6/d8/d3c/d77/d130 0 2024-04-20T11:09:49.686 INFO:tasks.workunit.client.1.smithi173.stdout:1/933: write d5/d6/d17/d42/da/df/d40/dc8/fcc [1582833,62806] 0 2024-04-20T11:09:49.691 INFO:tasks.workunit.client.1.smithi173.stdout:1/934: symlink d5/d6/d17/d42/d12/d1c/l134 0 2024-04-20T11:09:49.694 INFO:tasks.workunit.client.1.smithi173.stdout:0/995: symlink d6/d8/df/d2c/d42/d50/d86/d8b/dcc/deb/l131 0 2024-04-20T11:09:49.698 INFO:tasks.workunit.client.1.smithi173.stdout:0/996: write d6/d8/df/d2c/d2d/f8a [2889453,56006] 0 2024-04-20T11:09:49.717 INFO:tasks.workunit.client.1.smithi173.stdout:1/935: dwrite d5/d6/d3b/d123/f130 [0,4194304] 0 2024-04-20T11:09:49.736 INFO:tasks.workunit.client.1.smithi173.stdout:1/936: dwrite d5/d6/d17/d42/da/d20/d48/f132 [0,4194304] 0 2024-04-20T11:09:49.740 INFO:tasks.workunit.client.1.smithi173.stdout:1/937: creat d5/d6/d17/d42/d10c/f135 x:0 0 0 2024-04-20T11:09:49.743 INFO:tasks.workunit.client.1.smithi173.stdout:1/938: symlink d5/d6/d17/d42/da/d110/l136 0 2024-04-20T11:09:49.748 INFO:tasks.workunit.client.1.smithi173.stdout:1/939: symlink d5/d6/d17/d42/da/d20/d38/l137 0 2024-04-20T11:09:49.751 INFO:tasks.workunit.client.1.smithi173.stdout:9/960: sync 2024-04-20T11:09:49.752 INFO:tasks.workunit.client.1.smithi173.stdout:1/940: read d5/d6/d17/d42/da/d20/dad/dc2/f72 [353576,61814] 0 2024-04-20T11:09:49.753 INFO:tasks.workunit.client.1.smithi173.stdout:9/961: write d5/d9/d10f/d2b/d59/d44/d78/fef [553585,42059] 0 2024-04-20T11:09:49.767 INFO:tasks.workunit.client.1.smithi173.stdout:1/941: dwrite d5/d6/d17/d42/da/df/fe7 [0,4194304] 0 2024-04-20T11:09:49.773 INFO:tasks.workunit.client.1.smithi173.stdout:1/942: mknod d5/d6/d17/d42/d12/d1f/ddc/c138 0 2024-04-20T11:09:49.780 INFO:tasks.workunit.client.1.smithi173.stdout:1/943: fsync d5/d6/d17/d42/da/d20/d48/f61 0 2024-04-20T11:09:49.783 INFO:tasks.workunit.client.1.smithi173.stdout:1/944: chown d5/d6/d17/d42/d12/f31 162729 1 2024-04-20T11:09:49.786 INFO:tasks.workunit.client.1.smithi173.stdout:9/962: dread d5/d9/f1a [0,4194304] 0 2024-04-20T11:09:49.793 INFO:tasks.workunit.client.1.smithi173.stdout:9/963: readlink d5/d9/d10f/d4f/d5e/d9e/l112 0 2024-04-20T11:09:49.796 INFO:tasks.workunit.client.1.smithi173.stdout:9/964: fdatasync d5/d9/d10f/d29/d18/f1f 0 2024-04-20T11:09:49.797 INFO:tasks.workunit.client.1.smithi173.stdout:1/945: dwrite d5/d6/d17/d42/f54 [4194304,4194304] 0 2024-04-20T11:09:49.800 INFO:tasks.workunit.client.1.smithi173.stdout:9/965: chown d5/d9/d10f/d29/d2d/c3c 11367343 1 2024-04-20T11:09:49.806 INFO:tasks.workunit.client.1.smithi173.stdout:9/966: truncate d5/d9/d10f/d2b/d106/f114 985959 0 2024-04-20T11:09:49.807 INFO:tasks.workunit.client.1.smithi173.stdout:1/946: mkdir d5/d6/d17/d42/da/d20/d38/d52/d75/d139 0 2024-04-20T11:09:49.811 INFO:tasks.workunit.client.1.smithi173.stdout:1/947: mknod d5/d6/d17/d42/da/df/c13a 0 2024-04-20T11:09:49.817 INFO:tasks.workunit.client.1.smithi173.stdout:1/948: rename d5/d6/d17/f4b to d5/d6/d17/d42/d12/d1f/f13b 0 2024-04-20T11:09:49.819 INFO:tasks.workunit.client.1.smithi173.stdout:1/949: dread - d5/d6/d17/d42/da/df/d40/dac/fca zero size 2024-04-20T11:09:49.820 INFO:tasks.workunit.client.1.smithi173.stdout:9/967: dread d5/d9/d10f/d29/d18/f1f [4194304,4194304] 0 2024-04-20T11:09:49.824 INFO:tasks.workunit.client.1.smithi173.stdout:1/950: mkdir d5/d6/d17/d42/da/df/d40/dc8/dde/d13c 0 2024-04-20T11:09:49.825 INFO:tasks.workunit.client.1.smithi173.stdout:9/968: mknod d5/d9/d10f/d2b/d59/d44/d78/dd0/c122 0 2024-04-20T11:09:49.833 INFO:tasks.workunit.client.1.smithi173.stdout:0/997: sync 2024-04-20T11:09:49.833 INFO:tasks.workunit.client.1.smithi173.stdout:3/976: sync 2024-04-20T11:09:49.842 INFO:tasks.workunit.client.1.smithi173.stdout:1/951: dwrite d5/d6/dc/fe0 [0,4194304] 0 2024-04-20T11:09:49.843 INFO:tasks.workunit.client.1.smithi173.stdout:3/977: link d7/d17/d61/d83/dcf/d5c/c12b d7/c13e 0 2024-04-20T11:09:49.845 INFO:tasks.workunit.client.1.smithi173.stdout:0/998: link d6/d8/d51/f5b d6/d5f/d8f/f132 0 2024-04-20T11:09:49.848 INFO:tasks.workunit.client.1.smithi173.stdout:9/969: dwrite d5/d9/d10f/d2b/d106/fe9 [0,4194304] 0 2024-04-20T11:09:49.849 INFO:tasks.workunit.client.1.smithi173.stdout:1/952: chown d5/d6/d17/d42/da/d20/d38/d52/ca8 2896 1 2024-04-20T11:09:49.851 INFO:tasks.workunit.client.1.smithi173.stdout:9/970: chown d5/d9/d10f/d29/d2d/d33/l3a 758 1 2024-04-20T11:09:49.859 INFO:tasks.workunit.client.1.smithi173.stdout:0/999: fdatasync d6/d8/df/d2c/d2d/d121/fff 0 2024-04-20T11:09:49.862 INFO:tasks.workunit.client.1.smithi173.stdout:1/953: truncate d5/d6/f21 3663873 0 2024-04-20T11:09:49.879 INFO:tasks.workunit.client.1.smithi173.stdout:1/954: dwrite d5/d6/d3b/d123/f130 [0,4194304] 0 2024-04-20T11:09:49.884 INFO:tasks.workunit.client.1.smithi173.stdout:3/978: truncate d7/d17/d61/d83/dcf/f46 3064186 0 2024-04-20T11:09:49.885 INFO:tasks.workunit.client.1.smithi173.stdout:3/979: chown d7/d17/d61/d90/f11f 122 1 2024-04-20T11:09:49.886 INFO:tasks.workunit.client.1.smithi173.stdout:3/980: stat d7/d17/d61/d83/dc2 0 2024-04-20T11:09:49.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:49 smithi173 ceph-mon[23800]: pgmap v34: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 79 MiB/s rd, 201 MiB/s wr, 489 op/s 2024-04-20T11:09:49.917 INFO:tasks.workunit.client.1.smithi173.stdout:9/971: rmdir d5/d9/d10f/d4f/d5e/df6 0 2024-04-20T11:09:49.919 INFO:tasks.workunit.client.1.smithi173.stdout:3/981: mknod d7/d17/d61/d83/dcf/d5c/c13f 0 2024-04-20T11:09:49.922 INFO:tasks.workunit.client.1.smithi173.stdout:9/972: unlink d5/d9/d10f/d2b/d59/d44/cca 0 2024-04-20T11:09:49.925 INFO:tasks.workunit.client.1.smithi173.stdout:3/982: creat d7/d17/d61/d83/dcf/f140 x:0 0 0 2024-04-20T11:09:49.932 INFO:tasks.workunit.client.1.smithi173.stdout:9/973: link d5/d9/d10f/d29/d2d/dcb/ccf d5/d9/d10f/d2b/d59/c123 0 2024-04-20T11:09:49.939 INFO:tasks.workunit.client.1.smithi173.stdout:9/974: mkdir d5/d9/d10f/d2b/d59/da1/dee/d124 0 2024-04-20T11:09:49.940 INFO:tasks.workunit.client.1.smithi173.stdout:9/975: stat d5/d9/d10f/c63 0 2024-04-20T11:09:49.950 INFO:tasks.workunit.client.1.smithi173.stdout:9/976: symlink d5/d9/d10f/d2b/d59/d44/d78/d82/de8/d110/l125 0 2024-04-20T11:09:49.966 INFO:tasks.workunit.client.1.smithi173.stdout:3/983: truncate d7/d17/d61/d83/dcf/f14 3024874 0 2024-04-20T11:09:49.975 INFO:tasks.workunit.client.1.smithi173.stdout:3/984: truncate d7/d17/d58/d8a/dc4/f10d 981475 0 2024-04-20T11:09:49.980 INFO:tasks.workunit.client.1.smithi173.stdout:9/977: dwrite d5/d9/d10f/d2b/d59/d44/ff9 [0,4194304] 0 2024-04-20T11:09:50.000 INFO:tasks.workunit.client.1.smithi173.stdout:9/978: write d5/d9/d10f/d29/d18/d9b/d98/dc4/fd2 [576122,60975] 0 2024-04-20T11:09:50.010 INFO:tasks.workunit.client.1.smithi173.stdout:3/985: dread d7/d17/f26 [0,4194304] 0 2024-04-20T11:09:50.018 INFO:tasks.workunit.client.1.smithi173.stdout:3/986: unlink d7/d17/d61/d83/d45/l101 0 2024-04-20T11:09:50.020 INFO:tasks.workunit.client.1.smithi173.stdout:3/987: stat d7/d85 0 2024-04-20T11:09:50.023 INFO:tasks.workunit.client.1.smithi173.stdout:3/988: fdatasync d7/d17/d61/d83/dcf/d30/f80 0 2024-04-20T11:09:50.028 INFO:tasks.workunit.client.1.smithi173.stdout:9/979: truncate d5/d9/d10f/d2b/d59/fd8 3269334 0 2024-04-20T11:09:50.030 INFO:tasks.workunit.client.1.smithi173.stdout:1/955: sync 2024-04-20T11:09:50.034 INFO:tasks.workunit.client.1.smithi173.stdout:1/956: dread - d5/d6/dc/d7f/fee zero size 2024-04-20T11:09:50.037 INFO:tasks.workunit.client.1.smithi173.stdout:3/989: dread d7/d17/d61/d83/fbf [0,4194304] 0 2024-04-20T11:09:50.038 INFO:tasks.workunit.client.1.smithi173.stdout:9/980: mknod d5/d9/d10f/d2b/d59/d44/d78/dd0/c126 0 2024-04-20T11:09:50.041 INFO:tasks.workunit.client.1.smithi173.stdout:1/957: dread - d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fc0 zero size 2024-04-20T11:09:50.046 INFO:tasks.workunit.client.1.smithi173.stdout:1/958: write d5/d6/d17/d42/da/d20/dad/dc2/fd4 [1974064,106606] 0 2024-04-20T11:09:50.048 INFO:tasks.workunit.client.1.smithi173.stdout:1/959: chown d5/d6/d17/d42/da/df/d40/d126 383684 1 2024-04-20T11:09:50.055 INFO:tasks.workunit.client.1.smithi173.stdout:9/981: write d5/d9/d10f/d2b/d59/da1/f11d [5988910,108934] 0 2024-04-20T11:09:50.056 INFO:tasks.workunit.client.1.smithi173.stdout:1/960: rename d5/d6/d17/d42/d12/d1c/f5b to d5/d6/d17/d42/d12/d1f/ddc/f13d 0 2024-04-20T11:09:50.062 INFO:tasks.workunit.client.1.smithi173.stdout:1/961: fsync d5/d6/d17/d42/f90 0 2024-04-20T11:09:50.065 INFO:tasks.workunit.client.1.smithi173.stdout:1/962: chown d5/d6/d17/d42/da/df/d40 10502014 1 2024-04-20T11:09:50.066 INFO:tasks.workunit.client.1.smithi173.stdout:3/990: dread d7/d17/d61/d83/dcf/d30/f8d [0,4194304] 0 2024-04-20T11:09:50.070 INFO:tasks.workunit.client.1.smithi173.stdout:9/982: dread d5/d9/fc5 [0,4194304] 0 2024-04-20T11:09:50.074 INFO:tasks.workunit.client.1.smithi173.stdout:3/991: creat d7/d85/f141 x:0 0 0 2024-04-20T11:09:50.074 INFO:tasks.workunit.client.1.smithi173.stdout:1/963: dread d5/d6/f73 [0,4194304] 0 2024-04-20T11:09:50.077 INFO:tasks.workunit.client.1.smithi173.stdout:3/992: write d7/d17/d61/d83/f103 [713433,25825] 0 2024-04-20T11:09:50.082 INFO:tasks.workunit.client.1.smithi173.stdout:9/983: write d5/d9/d10f/d2b/d59/fd8 [943218,111921] 0 2024-04-20T11:09:50.087 INFO:tasks.workunit.client.1.smithi173.stdout:3/993: unlink d7/d2b/l69 0 2024-04-20T11:09:50.089 INFO:tasks.workunit.client.1.smithi173.stdout:9/984: mkdir d5/d9/d10f/d4f/d120/d127 0 2024-04-20T11:09:50.090 INFO:tasks.workunit.client.1.smithi173.stdout:1/964: rename d5/d6/d17/d42/da/d20/dad/lec to d5/d6/d17/d42/d12/l13e 0 2024-04-20T11:09:50.093 INFO:tasks.workunit.client.1.smithi173.stdout:3/994: symlink d7/d2b/l142 0 2024-04-20T11:09:50.094 INFO:tasks.workunit.client.1.smithi173.stdout:9/985: mknod d5/d9/d10f/d2b/d59/d44/d78/d82/de8/d110/c128 0 2024-04-20T11:09:50.100 INFO:tasks.workunit.client.1.smithi173.stdout:9/986: read d5/d9/d10f/d2b/d59/d48/fe2 [404649,39731] 0 2024-04-20T11:09:50.103 INFO:tasks.workunit.client.1.smithi173.stdout:9/987: mkdir d5/d9/d10f/d129 0 2024-04-20T11:09:50.104 INFO:tasks.workunit.client.1.smithi173.stdout:3/995: link d7/d17/d61/d90/dca/cdd d7/d17/d61/c143 0 2024-04-20T11:09:50.106 INFO:tasks.workunit.client.1.smithi173.stdout:9/988: symlink d5/l12a 0 2024-04-20T11:09:50.109 INFO:tasks.workunit.client.1.smithi173.stdout:9/989: creat d5/dd7/f12b x:0 0 0 2024-04-20T11:09:50.111 INFO:tasks.workunit.client.1.smithi173.stdout:9/990: dread - d5/d9/ff8 zero size 2024-04-20T11:09:50.112 INFO:tasks.workunit.client.1.smithi173.stdout:1/965: dwrite d5/d6/d17/d42/da/df/d40/dac/fbd [0,4194304] 0 2024-04-20T11:09:50.118 INFO:tasks.workunit.client.1.smithi173.stdout:1/966: mknod d5/d6/d17/d42/da/d20/d48/d81/c13f 0 2024-04-20T11:09:50.120 INFO:tasks.workunit.client.1.smithi173.stdout:9/991: link d5/d9/d10f/d29/d18/d9b/fbc d5/f12c 0 2024-04-20T11:09:50.121 INFO:tasks.workunit.client.1.smithi173.stdout:3/996: dwrite d7/d17/d61/d83/dcf/d30/fc7 [0,4194304] 0 2024-04-20T11:09:50.122 INFO:tasks.workunit.client.1.smithi173.stdout:1/967: creat d5/dd8/f140 x:0 0 0 2024-04-20T11:09:50.126 INFO:tasks.workunit.client.1.smithi173.stdout:3/997: symlink d7/d17/d58/d7f/l144 0 2024-04-20T11:09:50.129 INFO:tasks.workunit.client.1.smithi173.stdout:1/968: dread d5/d6/d17/d42/da/d20/f33 [0,4194304] 0 2024-04-20T11:09:50.133 INFO:tasks.workunit.client.1.smithi173.stdout:1/969: chown d5/d6/d17/d42/da/df/d40/db4 78082 1 2024-04-20T11:09:50.133 INFO:tasks.workunit.client.1.smithi173.stdout:9/992: rename d5/d9/d10f/d29/d2d/l9a to d5/d9/d10f/d29/d2d/l12d 0 2024-04-20T11:09:50.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:49 smithi117 ceph-mon[15659]: pgmap v34: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 79 MiB/s rd, 201 MiB/s wr, 489 op/s 2024-04-20T11:09:50.150 INFO:tasks.workunit.client.1.smithi173.stdout:1/970: fdatasync d5/d6/d17/d42/da/f6a 0 2024-04-20T11:09:50.151 INFO:tasks.workunit.client.1.smithi173.stdout:9/993: creat d5/d9/d10f/d129/f12e x:0 0 0 2024-04-20T11:09:50.156 INFO:tasks.workunit.client.1.smithi173.stdout:1/971: mknod d5/d6/d3b/d123/c141 0 2024-04-20T11:09:50.162 INFO:tasks.workunit.client.1.smithi173.stdout:1/972: chown d5/d6/f21 8781376 1 2024-04-20T11:09:50.163 INFO:tasks.workunit.client.1.smithi173.stdout:9/994: mkdir d5/d9/d10f/d29/d18/d12f 0 2024-04-20T11:09:50.166 INFO:tasks.workunit.client.1.smithi173.stdout:1/973: fsync d5/d6/d17/d42/da/df/d40/dc8/fcc 0 2024-04-20T11:09:50.175 INFO:tasks.workunit.client.1.smithi173.stdout:1/974: mkdir d5/d6/d17/d42/d12/d1c/d85/de1/d142 0 2024-04-20T11:09:50.181 INFO:tasks.workunit.client.1.smithi173.stdout:3/998: dwrite d7/d17/d58/d7f/f95 [0,4194304] 0 2024-04-20T11:09:50.184 INFO:tasks.workunit.client.1.smithi173.stdout:9/995: truncate d5/d9/d10f/d29/d18/d9b/f100 262869 0 2024-04-20T11:09:50.189 INFO:tasks.workunit.client.1.smithi173.stdout:9/996: mkdir d5/dd7/d130 0 2024-04-20T11:09:50.194 INFO:tasks.workunit.client.1.smithi173.stdout:9/997: creat d5/d9/d10f/d2b/d106/d79/d80/f131 x:0 0 0 2024-04-20T11:09:50.195 INFO:tasks.workunit.client.1.smithi173.stdout:1/975: dread d5/d6/f9 [0,4194304] 0 2024-04-20T11:09:50.196 INFO:tasks.workunit.client.1.smithi173.stdout:3/999: getdents d7/d85 0 2024-04-20T11:09:50.197 INFO:tasks.workunit.client.1.smithi173.stdout:9/998: read - d5/d9/d10f/d2b/f118 zero size 2024-04-20T11:09:50.202 INFO:tasks.workunit.client.1.smithi173.stdout:1/976: truncate d5/d6/d17/d42/da/d20/dad/fd9 328458 0 2024-04-20T11:09:50.206 INFO:tasks.workunit.client.1.smithi173.stdout:9/999: write d5/d9/d10f/d2b/d106/d79/d80/ff5 [261851,120159] 0 2024-04-20T11:09:50.215 INFO:tasks.workunit.client.1.smithi173.stdout:1/977: truncate d5/d6/d17/d42/da/df/d40/f5d 7689920 0 2024-04-20T11:09:50.219 INFO:tasks.workunit.client.1.smithi173.stdout:1/978: mknod d5/d6/d17/d42/d12/c143 0 2024-04-20T11:09:50.224 INFO:tasks.workunit.client.1.smithi173.stdout:1/979: unlink d5/d6/d17/d42/d10c/f135 0 2024-04-20T11:09:50.240 INFO:tasks.workunit.client.1.smithi173.stdout:1/980: write d5/d6/d17/d42/da/d20/d38/fa4 [488097,102468] 0 2024-04-20T11:09:50.242 INFO:tasks.workunit.client.1.smithi173.stdout:1/981: chown d5/d6/d17/d42/d12/d1c/d85/ffe 6046514 1 2024-04-20T11:09:50.247 INFO:tasks.workunit.client.1.smithi173.stdout:1/982: symlink d5/d6/d17/d93/db6/df4/l144 0 2024-04-20T11:09:50.271 INFO:tasks.workunit.client.1.smithi173.stdout:1/983: dread d5/d6/d17/d42/d12/f44 [0,4194304] 0 2024-04-20T11:09:50.274 INFO:tasks.workunit.client.1.smithi173.stdout:1/984: stat d5/d6/d17/d42/da/df/d40/dac/fbd 0 2024-04-20T11:09:50.280 INFO:tasks.workunit.client.1.smithi173.stdout:1/985: fdatasync d5/d6/d17/d42/da/df/fce 0 2024-04-20T11:09:50.286 INFO:tasks.workunit.client.1.smithi173.stdout:1/986: truncate d5/d6/d17/d42/da/df/d40/dc8/f12f 607162 0 2024-04-20T11:09:50.288 INFO:tasks.workunit.client.1.smithi173.stdout:1/987: read - d5/d6/dc/d7f/fee zero size 2024-04-20T11:09:50.320 INFO:tasks.workunit.client.1.smithi173.stdout:1/988: dwrite f0 [4194304,4194304] 0 2024-04-20T11:09:50.358 INFO:tasks.workunit.client.1.smithi173.stdout:1/989: dread d5/d6/d17/d42/d12/d1c/f6f [0,4194304] 0 2024-04-20T11:09:50.360 INFO:tasks.workunit.client.1.smithi173.stdout:1/990: stat d5/d6/dc/fe 0 2024-04-20T11:09:50.364 INFO:tasks.workunit.client.1.smithi173.stdout:1/991: unlink d5/d6/d17/d93/lc7 0 2024-04-20T11:09:50.369 INFO:tasks.workunit.client.1.smithi173.stdout:1/992: dread d5/d6/d17/d42/d12/dc9/f10d [0,4194304] 0 2024-04-20T11:09:50.372 INFO:tasks.workunit.client.1.smithi173.stdout:1/993: mkdir d5/d115/d145 0 2024-04-20T11:09:50.376 INFO:tasks.workunit.client.1.smithi173.stdout:1/994: fsync d5/d6/d17/d42/da/d20/dad/dc2/d94/fea 0 2024-04-20T11:09:50.381 INFO:tasks.workunit.client.1.smithi173.stdout:1/995: readlink d5/d6/d17/d42/da/d20/d38/d52/la2 0 2024-04-20T11:09:50.413 INFO:tasks.workunit.client.1.smithi173.stdout:1/996: dwrite d5/d6/f9 [0,4194304] 0 2024-04-20T11:09:50.445 INFO:tasks.workunit.client.1.smithi173.stdout:1/997: dwrite d5/d6/d17/d42/d12/fd7 [0,4194304] 0 2024-04-20T11:09:50.451 INFO:tasks.workunit.client.1.smithi173.stdout:1/998: write d5/d6/d17/d42/da/d20/dad/dc2/d94/dbb/fdb [1867463,14331] 0 2024-04-20T11:09:50.459 INFO:tasks.workunit.client.1.smithi173.stdout:1/999: mknod d5/d6/d17/d42/da/df/d40/dac/c146 0 2024-04-20T11:09:50.470 INFO:tasks.workunit.client.1.smithi173.stderr:+ rm -rf -- ./tmp.8cjNyXBYLx 2024-04-20T11:09:51.836 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:51 smithi173 ceph-mon[23800]: pgmap v35: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 53 MiB/s rd, 131 MiB/s wr, 322 op/s 2024-04-20T11:09:52.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:51 smithi117 ceph-mon[15659]: pgmap v35: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 53 MiB/s rd, 131 MiB/s wr, 322 op/s 2024-04-20T11:09:53.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:53 smithi117 ceph-mon[15659]: pgmap v36: 65 pgs: 65 active+clean; 2.6 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 56 MiB/s rd, 135 MiB/s wr, 347 op/s 2024-04-20T11:09:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:53 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:53 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:53 smithi173 ceph-mon[23800]: pgmap v36: 65 pgs: 65 active+clean; 2.6 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 56 MiB/s rd, 135 MiB/s wr, 347 op/s 2024-04-20T11:09:53.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:53 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:53 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:54.134 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T11:09:54.135 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T11:09:54.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.773+0000 7f8c55400700 1 -- 172.21.15.117:0/1744179366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 msgr2=0x7f8c50106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.773+0000 7f8c55400700 1 --2- 172.21.15.117:0/1744179366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 0x7f8c50106440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009870 tx=0x7f8c3c009b80 comp rx=0 tx=0).stop 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.773+0000 7f8c55400700 1 -- 172.21.15.117:0/1744179366 shutdown_connections 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.773+0000 7f8c55400700 1 --2- 172.21.15.117:0/1744179366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 0x7f8c50106440 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.773+0000 7f8c55400700 1 --2- 172.21.15.117:0/1744179366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c50101720 0x7f8c50103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.773+0000 7f8c55400700 1 -- 172.21.15.117:0/1744179366 >> 172.21.15.117:0/1744179366 conn(0x7f8c500fb0d0 msgr2=0x7f8c500fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.774+0000 7f8c55400700 1 -- 172.21.15.117:0/1744179366 shutdown_connections 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.774+0000 7f8c55400700 1 -- 172.21.15.117:0/1744179366 wait complete. 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.775+0000 7f8c55400700 1 Processor -- start 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.775+0000 7f8c55400700 1 -- start start 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.776+0000 7f8c55400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c50101720 0x7f8c50071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.776+0000 7f8c55400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 0x7f8c50071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.776+0000 7f8c55400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50072280 con 0x7f8c50101720 2024-04-20T11:09:54.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.776+0000 7f8c55400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500723f0 con 0x7f8c50104050 2024-04-20T11:09:54.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.776+0000 7f8c4de00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 0x7f8c50071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:54.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.776+0000 7f8c4de00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 0x7f8c50071cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:33320/0 (socket says 172.21.15.117:33320) 2024-04-20T11:09:54.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.776+0000 7f8c4de00700 1 -- 172.21.15.117:0/2249395534 learned_addr learned my addr 172.21.15.117:0/2249395534 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:54.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.777+0000 7f8c4e800700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c50101720 0x7f8c50071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:54.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.777+0000 7f8c4de00700 1 -- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c50101720 msgr2=0x7f8c50071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:54.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.777+0000 7f8c4de00700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c50101720 0x7f8c50071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:54.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.777+0000 7f8c4de00700 1 -- 172.21.15.117:0/2249395534 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c009520 con 0x7f8c50104050 2024-04-20T11:09:54.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.777+0000 7f8c4e800700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c50101720 0x7f8c50071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:09:54.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.778+0000 7f8c4de00700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 0x7f8c50071cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c004730 tx=0x7f8c3c005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:54.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.778+0000 7f8c47400700 1 -- 172.21.15.117:0/2249395534 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3c01c070 con 0x7f8c50104050 2024-04-20T11:09:54.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.779+0000 7f8c47400700 1 -- 172.21.15.117:0/2249395534 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c3c003d10 con 0x7f8c50104050 2024-04-20T11:09:54.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.779+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c501aab90 con 0x7f8c50104050 2024-04-20T11:09:54.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.779+0000 7f8c47400700 1 -- 172.21.15.117:0/2249395534 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3c022420 con 0x7f8c50104050 2024-04-20T11:09:54.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.780+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c501ab020 con 0x7f8c50104050 2024-04-20T11:09:54.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.782+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c500fcc50 con 0x7f8c50104050 2024-04-20T11:09:54.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.782+0000 7f8c47400700 1 -- 172.21.15.117:0/2249395534 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7f8c3c020070 con 0x7f8c50104050 2024-04-20T11:09:54.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.784+0000 7f8c47400700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8c3406ea80 0x7f8c34070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:54.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.784+0000 7f8c47400700 1 -- 172.21.15.117:0/2249395534 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f8c3c08fb20 con 0x7f8c50104050 2024-04-20T11:09:54.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.784+0000 7f8c4e800700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8c3406ea80 0x7f8c34070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:54.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.785+0000 7f8c4e800700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8c3406ea80 0x7f8c34070f40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c40005b40 tx=0x7f8c40005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:54.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:54.793+0000 7f8c47400700 1 -- 172.21.15.117:0/2249395534 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f8c3c058c00 con 0x7f8c50104050 2024-04-20T11:09:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:54 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:09:55.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:54 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:09:55.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.149+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c5002ceb0 con 0x7f8c3406ea80 2024-04-20T11:09:55.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.151+0000 7f8c47400700 1 -- 172.21.15.117:0/2249395534 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f8c5002ceb0 con 0x7f8c3406ea80 2024-04-20T11:09:55.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.153+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8c3406ea80 msgr2=0x7f8c34070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:55.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.153+0000 7f8c55400700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8c3406ea80 0x7f8c34070f40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c40005b40 tx=0x7f8c40005a90 comp rx=0 tx=0).stop 2024-04-20T11:09:55.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.153+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 msgr2=0x7f8c50071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:55.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.153+0000 7f8c55400700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 0x7f8c50071cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c004730 tx=0x7f8c3c005d90 comp rx=0 tx=0).stop 2024-04-20T11:09:55.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.154+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 shutdown_connections 2024-04-20T11:09:55.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.154+0000 7f8c55400700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8c3406ea80 0x7f8c34070f40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.154+0000 7f8c55400700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c50101720 0x7f8c50071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.154+0000 7f8c55400700 1 --2- 172.21.15.117:0/2249395534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50104050 0x7f8c50071cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.154+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 >> 172.21.15.117:0/2249395534 conn(0x7f8c500fb0d0 msgr2=0x7f8c50104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:55.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.154+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 shutdown_connections 2024-04-20T11:09:55.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.154+0000 7f8c55400700 1 -- 172.21.15.117:0/2249395534 wait complete. 2024-04-20T11:09:55.174 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:09:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.331+0000 7fc10b200700 1 -- 172.21.15.117:0/3448495158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 msgr2=0x7fc104103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.331+0000 7fc10b200700 1 --2- 172.21.15.117:0/3448495158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 0x7fc104103240 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009910 tx=0x7fc0f4009c20 comp rx=0 tx=0).stop 2024-04-20T11:09:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.331+0000 7fc10b200700 1 -- 172.21.15.117:0/3448495158 shutdown_connections 2024-04-20T11:09:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.331+0000 7fc10b200700 1 --2- 172.21.15.117:0/3448495158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc104103f80 0x7fc104104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.331+0000 7fc10b200700 1 --2- 172.21.15.117:0/3448495158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 0x7fc104103240 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.331+0000 7fc10b200700 1 -- 172.21.15.117:0/3448495158 >> 172.21.15.117:0/3448495158 conn(0x7fc1040fe3a0 msgr2=0x7fc104100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:55.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.332+0000 7fc10b200700 1 -- 172.21.15.117:0/3448495158 shutdown_connections 2024-04-20T11:09:55.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.332+0000 7fc10b200700 1 -- 172.21.15.117:0/3448495158 wait complete. 2024-04-20T11:09:55.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.333+0000 7fc10b200700 1 Processor -- start 2024-04-20T11:09:55.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.333+0000 7fc10b200700 1 -- start start 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.334+0000 7fc10b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 0x7fc104198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.334+0000 7fc10b200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc104103f80 0x7fc104198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.334+0000 7fc10b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc104199380 con 0x7fc104102e20 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.334+0000 7fc10b200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041994c0 con 0x7fc104103f80 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.334+0000 7fc103e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 0x7fc104198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.334+0000 7fc103e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 0x7fc104198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54818/0 (socket says 172.21.15.117:54818) 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.334+0000 7fc103e00700 1 -- 172.21.15.117:0/1892659269 learned_addr learned my addr 172.21.15.117:0/1892659269 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.334+0000 7fc103400700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc104103f80 0x7fc104198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.335+0000 7fc103e00700 1 -- 172.21.15.117:0/1892659269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc104103f80 msgr2=0x7fc104198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.335+0000 7fc103e00700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc104103f80 0x7fc104198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.335+0000 7fc103e00700 1 -- 172.21.15.117:0/1892659269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f40095f0 con 0x7fc104102e20 2024-04-20T11:09:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.336+0000 7fc103e00700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 0x7fc104198820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005b40 tx=0x7fc0f400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.336+0000 7fc100c00700 1 -- 172.21.15.117:0/1892659269 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0f401d070 con 0x7fc104102e20 2024-04-20T11:09:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.336+0000 7fc100c00700 1 -- 172.21.15.117:0/1892659269 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0f40050d0 con 0x7fc104102e20 2024-04-20T11:09:55.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.336+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10419de70 con 0x7fc104102e20 2024-04-20T11:09:55.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.336+0000 7fc100c00700 1 -- 172.21.15.117:0/1892659269 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0f4022470 con 0x7fc104102e20 2024-04-20T11:09:55.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.336+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10419e360 con 0x7fc104102e20 2024-04-20T11:09:55.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.338+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc104192920 con 0x7fc104102e20 2024-04-20T11:09:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.339+0000 7fc100c00700 1 -- 172.21.15.117:0/1892659269 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7fc0f40037f0 con 0x7fc104102e20 2024-04-20T11:09:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.340+0000 7fc100c00700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc0e806ea70 0x7fc0e8070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.340+0000 7fc100c00700 1 -- 172.21.15.117:0/1892659269 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fc0f4090090 con 0x7fc104102e20 2024-04-20T11:09:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.341+0000 7fc103400700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc0e806ea70 0x7fc0e8070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.341+0000 7fc103400700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc0e806ea70 0x7fc0e8070f30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009c60 tx=0x7fc0f0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:55.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.348+0000 7fc100c00700 1 -- 172.21.15.117:0/1892659269 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fc0f405cc90 con 0x7fc104102e20 2024-04-20T11:09:55.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.654+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc10402cef0 con 0x7fc0e806ea70 2024-04-20T11:09:55.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.656+0000 7fc100c00700 1 -- 172.21.15.117:0/1892659269 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fc10402cef0 con 0x7fc0e806ea70 2024-04-20T11:09:55.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc0e806ea70 msgr2=0x7fc0e8070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:55.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc0e806ea70 0x7fc0e8070f30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009c60 tx=0x7fc0f0009260 comp rx=0 tx=0).stop 2024-04-20T11:09:55.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 msgr2=0x7fc104198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:55.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 0x7fc104198820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005b40 tx=0x7fc0f400f690 comp rx=0 tx=0).stop 2024-04-20T11:09:55.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 shutdown_connections 2024-04-20T11:09:55.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc0e806ea70 0x7fc0e8070f30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104102e20 0x7fc104198820 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 --2- 172.21.15.117:0/1892659269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc104103f80 0x7fc104198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 >> 172.21.15.117:0/1892659269 conn(0x7fc1040fe3a0 msgr2=0x7fc104107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:55.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 shutdown_connections 2024-04-20T11:09:55.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.659+0000 7fc10b200700 1 -- 172.21.15.117:0/1892659269 wait complete. 2024-04-20T11:09:55.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.844+0000 7fc835000700 1 -- 172.21.15.117:0/2985807748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8300722c0 msgr2=0x7fc8300726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:55.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.844+0000 7fc835000700 1 --2- 172.21.15.117:0/2985807748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8300722c0 0x7fc8300726e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc820009870 tx=0x7fc820009b80 comp rx=0 tx=0).stop 2024-04-20T11:09:55.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.845+0000 7fc835000700 1 -- 172.21.15.117:0/2985807748 shutdown_connections 2024-04-20T11:09:55.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.845+0000 7fc835000700 1 --2- 172.21.15.117:0/2985807748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc830108220 0x7fc8301086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.845+0000 7fc835000700 1 --2- 172.21.15.117:0/2985807748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8300722c0 0x7fc8300726e0 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc820009870 tx=0x7fc820009b80 comp rx=0 tx=0).stop 2024-04-20T11:09:55.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.845+0000 7fc835000700 1 -- 172.21.15.117:0/2985807748 >> 172.21.15.117:0/2985807748 conn(0x7fc83006d970 msgr2=0x7fc83006fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:55.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.846+0000 7fc835000700 1 -- 172.21.15.117:0/2985807748 shutdown_connections 2024-04-20T11:09:55.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.847+0000 7fc835000700 1 -- 172.21.15.117:0/2985807748 wait complete. 2024-04-20T11:09:55.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.848+0000 7fc835000700 1 Processor -- start 2024-04-20T11:09:55.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.849+0000 7fc835000700 1 -- start start 2024-04-20T11:09:55.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.849+0000 7fc835000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc830108220 0x7fc8301a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:55.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.849+0000 7fc835000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8301a5a50 0x7fc8301aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:55.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.849+0000 7fc835000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8301a5f60 con 0x7fc830108220 2024-04-20T11:09:55.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.849+0000 7fc835000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8301a60d0 con 0x7fc8301a5a50 2024-04-20T11:09:55.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.849+0000 7fc82f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc830108220 0x7fc8301a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:55.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.849+0000 7fc82ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8301a5a50 0x7fc8301aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:55.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.850+0000 7fc82ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8301a5a50 0x7fc8301aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:33348/0 (socket says 172.21.15.117:33348) 2024-04-20T11:09:55.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.850+0000 7fc82ea00700 1 -- 172.21.15.117:0/1850040131 learned_addr learned my addr 172.21.15.117:0/1850040131 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:55.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.850+0000 7fc82ea00700 1 -- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc830108220 msgr2=0x7fc8301a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:55.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.850+0000 7fc82ea00700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc830108220 0x7fc8301a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:55.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.850+0000 7fc82ea00700 1 -- 172.21.15.117:0/1850040131 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820009520 con 0x7fc8301a5a50 2024-04-20T11:09:55.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.851+0000 7fc82ea00700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8301a5a50 0x7fc8301aaa70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00e940 tx=0x7fc81c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:55.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.851+0000 7fc827e00700 1 -- 172.21.15.117:0/1850040131 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc81c004a00 con 0x7fc8301a5a50 2024-04-20T11:09:55.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.851+0000 7fc827e00700 1 -- 172.21.15.117:0/1850040131 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc81c00cc80 con 0x7fc8301a5a50 2024-04-20T11:09:55.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.851+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8301ab010 con 0x7fc8301a5a50 2024-04-20T11:09:55.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.851+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8301ab5b0 con 0x7fc8301a5a50 2024-04-20T11:09:55.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.852+0000 7fc827e00700 1 -- 172.21.15.117:0/1850040131 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc81c005590 con 0x7fc8301a5a50 2024-04-20T11:09:55.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.854+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc810005320 con 0x7fc8301a5a50 2024-04-20T11:09:55.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.857+0000 7fc827e00700 1 -- 172.21.15.117:0/1850040131 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7fc81c00c7c0 con 0x7fc8301a5a50 2024-04-20T11:09:55.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.860+0000 7fc827e00700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc81406ea70 0x7fc814070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:55.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.860+0000 7fc827e00700 1 -- 172.21.15.117:0/1850040131 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fc81c014070 con 0x7fc8301a5a50 2024-04-20T11:09:55.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.861+0000 7fc82f400700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc81406ea70 0x7fc814070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:55.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.862+0000 7fc82f400700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc81406ea70 0x7fc814070f30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc820009230 tx=0x7fc820019040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:55.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:55.869+0000 7fc827e00700 1 -- 172.21.15.117:0/1850040131 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fc81c05b8d0 con 0x7fc8301a5a50 2024-04-20T11:09:56.048 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:55 smithi173 ceph-mon[23800]: pgmap v37: 65 pgs: 65 active+clean; 2.4 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 66 MiB/s rd, 152 MiB/s wr, 412 op/s 2024-04-20T11:09:56.078 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:55 smithi117 ceph-mon[15659]: pgmap v37: 65 pgs: 65 active+clean; 2.4 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 66 MiB/s rd, 152 MiB/s wr, 412 op/s 2024-04-20T11:09:56.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.231+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc810000bf0 con 0x7fc81406ea70 2024-04-20T11:09:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.250+0000 7fc827e00700 1 -- 172.21.15.117:0/1850040131 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc810000bf0 con 0x7fc81406ea70 2024-04-20T11:09:56.252 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:09:56.252 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (10m) 46s ago 12m 13.7M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:09:56.252 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (12m) 46s ago 12m 7948k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:09:56.252 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (11m) 65s ago 11m 7750k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:09:56.252 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (12m) 46s ago 12m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (11m) 65s ago 11m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (10m) 46s ago 12m 75.8M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (5m) 46s ago 5m 14.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (5m) 46s ago 5m 172M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (5m) 65s ago 5m 14.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (5m) 65s ago 5m 23.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (92s) 46s ago 14m 477M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 starting - - - - 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (14m) 46s ago 14m 53.3M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (11m) 65s ago 11m 41.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (12m) 46s ago 12m 8162k - 1.5.0 0da6a335fe13 6b69ffbf445a 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (11m) 65s ago 11m 8074k - 1.5.0 0da6a335fe13 81909a8618f8 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (10m) 46s ago 10m 89.9M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:09:56.253 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (9m) 46s ago 9m 93.3M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:09:56.254 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (8m) 46s ago 8m 78.2M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:09:56.254 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (8m) 65s ago 8m 104M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:09:56.254 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (7m) 65s ago 7m 98.1M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:09:56.254 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (7m) 65s ago 7m 95.3M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:09:56.254 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 starting - - - - 2024-04-20T11:09:56.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.253+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc81406ea70 msgr2=0x7fc814070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:56.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.253+0000 7fc835000700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc81406ea70 0x7fc814070f30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc820009230 tx=0x7fc820019040 comp rx=0 tx=0).stop 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.253+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8301a5a50 msgr2=0x7fc8301aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.253+0000 7fc835000700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8301a5a50 0x7fc8301aaa70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00e940 tx=0x7fc81c00ec50 comp rx=0 tx=0).stop 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.253+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 shutdown_connections 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.253+0000 7fc835000700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc81406ea70 0x7fc814070f30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.253+0000 7fc835000700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc830108220 0x7fc8301a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.253+0000 7fc835000700 1 --2- 172.21.15.117:0/1850040131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8301a5a50 0x7fc8301aaa70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.254+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 >> 172.21.15.117:0/1850040131 conn(0x7fc83006d970 msgr2=0x7fc83010ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.254+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 shutdown_connections 2024-04-20T11:09:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.254+0000 7fc835000700 1 -- 172.21.15.117:0/1850040131 wait complete. 2024-04-20T11:09:56.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.424+0000 7f6b8a600700 1 -- 172.21.15.117:0/3248911666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84101560 msgr2=0x7f6b84103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:56.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.424+0000 7f6b8a600700 1 --2- 172.21.15.117:0/3248911666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84101560 0x7f6b84103950 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009910 tx=0x7f6b78009c20 comp rx=0 tx=0).stop 2024-04-20T11:09:56.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.425+0000 7f6b8a600700 1 -- 172.21.15.117:0/3248911666 shutdown_connections 2024-04-20T11:09:56.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.425+0000 7f6b8a600700 1 --2- 172.21.15.117:0/3248911666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b84103e90 0x7f6b84106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.425+0000 7f6b8a600700 1 --2- 172.21.15.117:0/3248911666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84101560 0x7f6b84103950 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.425+0000 7f6b8a600700 1 -- 172.21.15.117:0/3248911666 >> 172.21.15.117:0/3248911666 conn(0x7f6b840faef0 msgr2=0x7f6b840fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:56.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.425+0000 7f6b8a600700 1 -- 172.21.15.117:0/3248911666 shutdown_connections 2024-04-20T11:09:56.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.425+0000 7f6b8a600700 1 -- 172.21.15.117:0/3248911666 wait complete. 2024-04-20T11:09:56.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.426+0000 7f6b8a600700 1 Processor -- start 2024-04-20T11:09:56.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.426+0000 7f6b8a600700 1 -- start start 2024-04-20T11:09:56.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.427+0000 7f6b8a600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b84101560 0x7f6b84198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:56.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.427+0000 7f6b8a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84103e90 0x7f6b84198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:56.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.427+0000 7f6b8a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b84199120 con 0x7f6b84103e90 2024-04-20T11:09:56.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.427+0000 7f6b8a600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b84199260 con 0x7f6b84101560 2024-04-20T11:09:56.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.427+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84103e90 0x7f6b84198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:56.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.427+0000 7f6b83e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84103e90 0x7f6b84198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54852/0 (socket says 172.21.15.117:54852) 2024-04-20T11:09:56.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.427+0000 7f6b83e00700 1 -- 172.21.15.117:0/2322389801 learned_addr learned my addr 172.21.15.117:0/2322389801 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:56.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.427+0000 7f6b89200700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b84101560 0x7f6b84198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:56.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.428+0000 7f6b83e00700 1 -- 172.21.15.117:0/2322389801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b84101560 msgr2=0x7f6b84198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:56.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.428+0000 7f6b83e00700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b84101560 0x7f6b84198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.428+0000 7f6b83e00700 1 -- 172.21.15.117:0/2322389801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780095f0 con 0x7f6b84103e90 2024-04-20T11:09:56.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.428+0000 7f6b83e00700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84103e90 0x7f6b84198b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c930 tx=0x7f6b7000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:56.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.428+0000 7f6b81600700 1 -- 172.21.15.117:0/2322389801 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b70007ad0 con 0x7f6b84103e90 2024-04-20T11:09:56.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.429+0000 7f6b81600700 1 -- 172.21.15.117:0/2322389801 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b70007c30 con 0x7f6b84103e90 2024-04-20T11:09:56.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.429+0000 7f6b81600700 1 -- 172.21.15.117:0/2322389801 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b70018610 con 0x7f6b84103e90 2024-04-20T11:09:56.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.429+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8419dc80 con 0x7f6b84103e90 2024-04-20T11:09:56.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.429+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8419e1d0 con 0x7f6b84103e90 2024-04-20T11:09:56.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.430+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b84062510 con 0x7f6b84103e90 2024-04-20T11:09:56.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.432+0000 7f6b81600700 1 -- 172.21.15.117:0/2322389801 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7f6b7000f450 con 0x7f6b84103e90 2024-04-20T11:09:56.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.434+0000 7f6b81600700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6b6c06ea80 0x7f6b6c070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:56.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.434+0000 7f6b81600700 1 -- 172.21.15.117:0/2322389801 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f6b7008ecf0 con 0x7f6b84103e90 2024-04-20T11:09:56.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.434+0000 7f6b89200700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6b6c06ea80 0x7f6b6c070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:56.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.435+0000 7f6b89200700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6b6c06ea80 0x7f6b6c070f40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6b78006010 tx=0x7f6b7800b580 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:56.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.438+0000 7f6b81600700 1 -- 172.21.15.117:0/2322389801 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f6b7005b8f0 con 0x7f6b84103e90 2024-04-20T11:09:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:56 smithi117 ceph-mon[15659]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:56.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:56 smithi117 ceph-mon[15659]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:56.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.931+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b841095a0 con 0x7f6b84103e90 2024-04-20T11:09:56.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.932+0000 7f6b81600700 1 -- 172.21.15.117:0/2322389801 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f6b7005b480 con 0x7f6b84103e90 2024-04-20T11:09:56.933 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:09:56.933 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:09:56.933 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:09:56.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:09:56.933 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:09:56.934 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6b6c06ea80 msgr2=0x7f6b6c070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6b6c06ea80 0x7f6b6c070f40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6b78006010 tx=0x7f6b7800b580 comp rx=0 tx=0).stop 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84103e90 msgr2=0x7f6b84198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84103e90 0x7f6b84198b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c930 tx=0x7f6b7000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 shutdown_connections 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f6b6c06ea80 0x7f6b6c070f40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b84101560 0x7f6b84198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 --2- 172.21.15.117:0/2322389801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84103e90 0x7f6b84198b90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 >> 172.21.15.117:0/2322389801 conn(0x7f6b840faef0 msgr2=0x7f6b841003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 shutdown_connections 2024-04-20T11:09:56.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:56.936+0000 7f6b8a600700 1 -- 172.21.15.117:0/2322389801 wait complete. 2024-04-20T11:09:56.963 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:56 smithi173 ceph-mon[23800]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:56.963 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:56 smithi173 ceph-mon[23800]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:57.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.129+0000 7fce0e800700 1 -- 172.21.15.117:0/259667680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08101e20 msgr2=0x7fce08102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:57.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.129+0000 7fce0e800700 1 --2- 172.21.15.117:0/259667680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08101e20 0x7fce08102280 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009910 tx=0x7fcdf4009c20 comp rx=0 tx=0).stop 2024-04-20T11:09:57.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.129+0000 7fce0e800700 1 -- 172.21.15.117:0/259667680 shutdown_connections 2024-04-20T11:09:57.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.129+0000 7fce0e800700 1 --2- 172.21.15.117:0/259667680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08101e20 0x7fce08102280 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:57.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.129+0000 7fce0e800700 1 --2- 172.21.15.117:0/259667680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce08100c20 0x7fce08101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:57.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.129+0000 7fce0e800700 1 -- 172.21.15.117:0/259667680 >> 172.21.15.117:0/259667680 conn(0x7fce080fc1c0 msgr2=0x7fce080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:57.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.130+0000 7fce0e800700 1 -- 172.21.15.117:0/259667680 shutdown_connections 2024-04-20T11:09:57.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.130+0000 7fce0e800700 1 -- 172.21.15.117:0/259667680 wait complete. 2024-04-20T11:09:57.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.131+0000 7fce0e800700 1 Processor -- start 2024-04-20T11:09:57.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.131+0000 7fce0e800700 1 -- start start 2024-04-20T11:09:57.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.131+0000 7fce0e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08100c20 0x7fce0810fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:57.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.131+0000 7fce0e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0810ff70 0x7fce08112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:57.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.131+0000 7fce0e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce08110480 con 0x7fce08100c20 2024-04-20T11:09:57.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.131+0000 7fce0e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce081105f0 con 0x7fce0810ff70 2024-04-20T11:09:57.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.132+0000 7fce07400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08100c20 0x7fce0810fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:57.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.132+0000 7fce07400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08100c20 0x7fce0810fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54858/0 (socket says 172.21.15.117:54858) 2024-04-20T11:09:57.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.132+0000 7fce07400700 1 -- 172.21.15.117:0/3122405936 learned_addr learned my addr 172.21.15.117:0/3122405936 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:09:57.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.133+0000 7fce07400700 1 -- 172.21.15.117:0/3122405936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0810ff70 msgr2=0x7fce08112ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T11:09:57.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.133+0000 7fce07400700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0810ff70 0x7fce08112ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:57.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.133+0000 7fce07400700 1 -- 172.21.15.117:0/3122405936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40095f0 con 0x7fce08100c20 2024-04-20T11:09:57.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.134+0000 7fce07400700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08100c20 0x7fce0810fa30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800d900 tx=0x7fcdf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:57.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.135+0000 7fcdffe00700 1 -- 172.21.15.117:0/3122405936 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf8009900 con 0x7fce08100c20 2024-04-20T11:09:57.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.135+0000 7fcdffe00700 1 -- 172.21.15.117:0/3122405936 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdf8010460 con 0x7fce08100c20 2024-04-20T11:09:57.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.135+0000 7fcdffe00700 1 -- 172.21.15.117:0/3122405936 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf800f5d0 con 0x7fce08100c20 2024-04-20T11:09:57.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.135+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce08113480 con 0x7fce08100c20 2024-04-20T11:09:57.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.135+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce081139d0 con 0x7fce08100c20 2024-04-20T11:09:57.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.137+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce08114480 con 0x7fce08100c20 2024-04-20T11:09:57.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.139+0000 7fcdffe00700 1 -- 172.21.15.117:0/3122405936 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 32) v1 ==== 92710+0+0 (secure 0 0 0) 0x7fcdf8009c70 con 0x7fce08100c20 2024-04-20T11:09:57.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.140+0000 7fcdffe00700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fcdec06ea70 0x7fcdec070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:09:57.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.140+0000 7fcdffe00700 1 -- 172.21.15.117:0/3122405936 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fcdf808ea00 con 0x7fce08100c20 2024-04-20T11:09:57.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.140+0000 7fce06a00700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fcdec06ea70 0x7fcdec070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:09:57.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.141+0000 7fce06a00700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fcdec06ea70 0x7fcdec070f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400b5c0 tx=0x7fcdf4005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:09:57.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.146+0000 7fcdffe00700 1 -- 172.21.15.117:0/3122405936 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fcdf805b680 con 0x7fce08100c20 2024-04-20T11:09:57.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.477+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce08113cb0 con 0x7fcdec06ea70 2024-04-20T11:09:57.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.479+0000 7fcdffe00700 1 -- 172.21.15.117:0/3122405936 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fce08113cb0 con 0x7fcdec06ea70 2024-04-20T11:09:57.479 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:09:57.480 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:09:57.480 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:09:57.480 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:09:57.480 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [], 2024-04-20T11:09:57.480 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "", 2024-04-20T11:09:57.480 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mgr daemons", 2024-04-20T11:09:57.480 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:09:57.480 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:09:57.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fcdec06ea70 msgr2=0x7fcdec070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:57.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fcdec06ea70 0x7fcdec070f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400b5c0 tx=0x7fcdf4005fb0 comp rx=0 tx=0).stop 2024-04-20T11:09:57.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08100c20 msgr2=0x7fce0810fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:09:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08100c20 0x7fce0810fa30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800d900 tx=0x7fcdf800dcc0 comp rx=0 tx=0).stop 2024-04-20T11:09:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 shutdown_connections 2024-04-20T11:09:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fcdec06ea70 0x7fcdec070f30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce08100c20 0x7fce0810fa30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 --2- 172.21.15.117:0/3122405936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0810ff70 0x7fce08112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:09:57.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.482+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 >> 172.21.15.117:0/3122405936 conn(0x7fce080fc1c0 msgr2=0x7fce081054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:09:57.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.483+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 shutdown_connections 2024-04-20T11:09:57.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:09:57.483+0000 7fce0e800700 1 -- 172.21.15.117:0/3122405936 wait complete. 2024-04-20T11:09:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:57 smithi173 ceph-mon[23800]: pgmap v38: 65 pgs: 65 active+clean; 2.4 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 44 MiB/s rd, 98 MiB/s wr, 271 op/s 2024-04-20T11:09:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:57 smithi173 ceph-mon[23800]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:57 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2322389801' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:09:58.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:57 smithi117 ceph-mon[15659]: pgmap v38: 65 pgs: 65 active+clean; 2.4 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 44 MiB/s rd, 98 MiB/s wr, 271 op/s 2024-04-20T11:09:58.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:57 smithi117 ceph-mon[15659]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:58.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:57 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2322389801' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:09:58.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:58 smithi117 ceph-mon[15659]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:58.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:58 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:58 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:59.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:58 smithi173 ceph-mon[23800]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:09:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:58 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:09:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:58 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:00.113 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:09:59 smithi173 ceph-mon[23800]: pgmap v39: 65 pgs: 65 active+clean; 1.9 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 44 MiB/s rd, 98 MiB/s wr, 328 op/s 2024-04-20T11:10:00.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:09:59 smithi117 ceph-mon[15659]: pgmap v39: 65 pgs: 65 active+clean; 1.9 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 44 MiB/s rd, 98 MiB/s wr, 328 op/s 2024-04-20T11:10:00.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:00 smithi117 ceph-mon[15659]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:10:00.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:00 smithi117 ceph-mon[15659]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:10:00.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:00 smithi117 ceph-mon[15659]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:10:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:00 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:00 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:00.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:00 smithi173 ceph-mon[23800]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:10:00.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:00 smithi173 ceph-mon[23800]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:10:00.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:00 smithi173 ceph-mon[23800]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:10:00.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:00 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:00.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:00 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:01.863 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:01 smithi117 ceph-mon[15659]: pgmap v40: 65 pgs: 65 active+clean; 1.9 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 16 MiB/s rd, 35 MiB/s wr, 163 op/s 2024-04-20T11:10:01.863 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:01 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:10:01.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:01 smithi173 ceph-mon[23800]: pgmap v40: 65 pgs: 65 active+clean; 1.9 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 16 MiB/s rd, 35 MiB/s wr, 163 op/s 2024-04-20T11:10:01.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:01 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:10:03.827 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:03 smithi117 ceph-mon[15659]: pgmap v41: 65 pgs: 65 active+clean; 1.8 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 16 MiB/s rd, 36 MiB/s wr, 182 op/s 2024-04-20T11:10:03.973 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:03 smithi173 ceph-mon[23800]: pgmap v41: 65 pgs: 65 active+clean; 1.8 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 16 MiB/s rd, 36 MiB/s wr, 182 op/s 2024-04-20T11:10:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:04 smithi173 ceph-mon[23800]: Standby manager daemon smithi173.gnrjof restarted 2024-04-20T11:10:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:04 smithi173 ceph-mon[23800]: Standby manager daemon smithi173.gnrjof started 2024-04-20T11:10:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:04 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/3466545720' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T11:10:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:04 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/3466545720' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:10:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:04 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/3466545720' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T11:10:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:04 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/3466545720' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:10:05.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:04 smithi117 ceph-mon[15659]: Standby manager daemon smithi173.gnrjof restarted 2024-04-20T11:10:05.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:04 smithi117 ceph-mon[15659]: Standby manager daemon smithi173.gnrjof started 2024-04-20T11:10:05.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:04 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/3466545720' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T11:10:05.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:04 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/3466545720' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:10:05.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:04 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/3466545720' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T11:10:05.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:04 smithi117 ceph-mon[15659]: from='mgr.? 172.21.15.173:0/3466545720' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:10:06.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: pgmap v42: 65 pgs: 65 active+clean; 1.5 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 13 MiB/s rd, 31 MiB/s wr, 189 op/s 2024-04-20T11:10:06.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: mgrmap e33: smithi117.ivnklj(active, since 78s), standbys: smithi173.gnrjof 2024-04-20T11:10:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:10:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:10:06.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:05 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: pgmap v42: 65 pgs: 65 active+clean; 1.5 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 13 MiB/s rd, 31 MiB/s wr, 189 op/s 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: mgrmap e33: smithi117.ivnklj(active, since 78s), standbys: smithi173.gnrjof 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:10:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:05 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.971 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.ivnklj"}]: dispatch 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.ivnklj"}]': finished 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi173.gnrjof"}]: dispatch 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi173.gnrjof"}]': finished 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.972 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:06.973 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:06.973 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:06.973 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:07.079 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:10:07.079 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:07.079 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.ivnklj"}]: dispatch 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi117.ivnklj"}]': finished 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi173.gnrjof"}]: dispatch 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi173.gnrjof"}]': finished 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:07.080 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:07.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:07.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:07.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:07.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:07.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:07.081 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:07 smithi117 ceph-mon[15659]: Upgrade: Setting container_image for all mgr 2024-04-20T11:10:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:07 smithi117 ceph-mon[15659]: Upgrade: Setting container_image for all rgw 2024-04-20T11:10:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:07 smithi117 ceph-mon[15659]: Upgrade: Setting container_image for all rbd-mirror 2024-04-20T11:10:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:07 smithi117 ceph-mon[15659]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-20T11:10:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:07 smithi117 ceph-mon[15659]: Upgrade: Setting container_image for all iscsi 2024-04-20T11:10:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:07 smithi117 ceph-mon[15659]: Upgrade: Setting container_image for all nfs 2024-04-20T11:10:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:07 smithi117 ceph-mon[15659]: Upgrade: Setting container_image for all nvmeof 2024-04-20T11:10:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:07 smithi117 ceph-mon[15659]: pgmap v43: 65 pgs: 65 active+clean; 1.5 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 18 KiB/s rd, 991 KiB/s wr, 107 op/s 2024-04-20T11:10:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:07 smithi173 ceph-mon[23800]: Upgrade: Setting container_image for all mgr 2024-04-20T11:10:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:07 smithi173 ceph-mon[23800]: Upgrade: Setting container_image for all rgw 2024-04-20T11:10:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:07 smithi173 ceph-mon[23800]: Upgrade: Setting container_image for all rbd-mirror 2024-04-20T11:10:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:07 smithi173 ceph-mon[23800]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-20T11:10:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:07 smithi173 ceph-mon[23800]: Upgrade: Setting container_image for all iscsi 2024-04-20T11:10:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:07 smithi173 ceph-mon[23800]: Upgrade: Setting container_image for all nfs 2024-04-20T11:10:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:07 smithi173 ceph-mon[23800]: Upgrade: Setting container_image for all nvmeof 2024-04-20T11:10:08.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:07 smithi173 ceph-mon[23800]: pgmap v43: 65 pgs: 65 active+clean; 1.5 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 18 KiB/s rd, 991 KiB/s wr, 107 op/s 2024-04-20T11:10:10.076 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:09 smithi117 ceph-mon[15659]: pgmap v44: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-20T11:10:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:09 smithi173 ceph-mon[23800]: pgmap v44: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-20T11:10:11.772 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:11 smithi117 ceph-mon[15659]: pgmap v45: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 19 KiB/s rd, 1011 KiB/s wr, 90 op/s 2024-04-20T11:10:11.772 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:11 smithi117 ceph-mon[15659]: Upgrade: Updating node-exporter.smithi117 (1/2) 2024-04-20T11:10:11.772 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:11 smithi117 ceph-mon[15659]: Deploying daemon node-exporter.smithi117 on smithi117 2024-04-20T11:10:12.156 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:11 smithi173 ceph-mon[23800]: pgmap v45: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 19 KiB/s rd, 1011 KiB/s wr, 90 op/s 2024-04-20T11:10:12.156 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:11 smithi173 ceph-mon[23800]: Upgrade: Updating node-exporter.smithi117 (1/2) 2024-04-20T11:10:12.156 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:11 smithi173 ceph-mon[23800]: Deploying daemon node-exporter.smithi117 on smithi117 2024-04-20T11:10:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:13 smithi117 ceph-mon[15659]: pgmap v46: 65 pgs: 65 active+clean; 1.1 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 29 KiB/s rd, 1021 KiB/s wr, 95 op/s 2024-04-20T11:10:14.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:13 smithi173 ceph-mon[23800]: pgmap v46: 65 pgs: 65 active+clean; 1.1 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 29 KiB/s rd, 1021 KiB/s wr, 95 op/s 2024-04-20T11:10:16.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:15 smithi117 ceph-mon[15659]: pgmap v47: 65 pgs: 65 active+clean; 836 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 18 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-04-20T11:10:16.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:15 smithi173 ceph-mon[23800]: pgmap v47: 65 pgs: 65 active+clean; 836 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 18 KiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-04-20T11:10:17.063 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:16 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:10:17.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:16 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:10:18.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:17 smithi117 ceph-mon[15659]: pgmap v48: 65 pgs: 65 active+clean; 836 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 18 KiB/s rd, 904 KiB/s wr, 76 op/s 2024-04-20T11:10:18.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:17 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:18.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:17 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:18.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:17 smithi173 ceph-mon[23800]: pgmap v48: 65 pgs: 65 active+clean; 836 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 18 KiB/s rd, 904 KiB/s wr, 76 op/s 2024-04-20T11:10:18.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:17 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:18.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:17 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:18.845 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T11:10:18.845 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-20T11:10:18.946 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:18 smithi173 ceph-mon[23800]: mgrmap e34: smithi117.ivnklj(active, since 91s), standbys: smithi173.gnrjof 2024-04-20T11:10:19.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:18 smithi117 ceph-mon[15659]: mgrmap e34: smithi117.ivnklj(active, since 91s), standbys: smithi173.gnrjof 2024-04-20T11:10:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:19 smithi173 ceph-mon[23800]: pgmap v49: 65 pgs: 65 active+clean; 558 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-04-20T11:10:20.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:19 smithi117 ceph-mon[15659]: pgmap v49: 65 pgs: 65 active+clean; 558 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-04-20T11:10:20.877 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:20 smithi173 ceph-mon[23800]: Upgrade: Updating node-exporter.smithi173 (2/2) 2024-04-20T11:10:20.877 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:20 smithi173 ceph-mon[23800]: Deploying daemon node-exporter.smithi173 on smithi173 2024-04-20T11:10:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:20 smithi117 ceph-mon[15659]: Upgrade: Updating node-exporter.smithi173 (2/2) 2024-04-20T11:10:21.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:20 smithi117 ceph-mon[15659]: Deploying daemon node-exporter.smithi173 on smithi173 2024-04-20T11:10:21.882 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:21 smithi173 ceph-mon[23800]: pgmap v50: 65 pgs: 65 active+clean; 558 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 992 KiB/s wr, 75 op/s 2024-04-20T11:10:22.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:21 smithi117 ceph-mon[15659]: pgmap v50: 65 pgs: 65 active+clean; 558 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 992 KiB/s wr, 75 op/s 2024-04-20T11:10:24.002 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:23 smithi173 ceph-mon[23800]: pgmap v51: 65 pgs: 65 active+clean; 536 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-04-20T11:10:24.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:23 smithi117 ceph-mon[15659]: pgmap v51: 65 pgs: 65 active+clean; 536 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-04-20T11:10:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:25 smithi117 ceph-mon[15659]: pgmap v52: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-20T11:10:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:25 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:25 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:25.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:25 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:10:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:25 smithi173 ceph-mon[23800]: pgmap v52: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-20T11:10:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:25 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:25 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:25 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:10:27.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:27 smithi117 ceph-mon[15659]: pgmap v53: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 994 KiB/s wr, 74 op/s 2024-04-20T11:10:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:27 smithi173 ceph-mon[23800]: pgmap v53: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 994 KiB/s wr, 74 op/s 2024-04-20T11:10:27.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.685+0000 7fc949c00700 1 -- 172.21.15.117:0/3602364864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 msgr2=0x7fc944100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:27.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.685+0000 7fc949c00700 1 --2- 172.21.15.117:0/3602364864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 0x7fc944100e60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc934009910 tx=0x7fc934009c20 comp rx=0 tx=0).stop 2024-04-20T11:10:27.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.686+0000 7fc949c00700 1 -- 172.21.15.117:0/3602364864 shutdown_connections 2024-04-20T11:10:27.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.686+0000 7fc949c00700 1 --2- 172.21.15.117:0/3602364864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc944101c40 0x7fc9441020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:27.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.686+0000 7fc949c00700 1 --2- 172.21.15.117:0/3602364864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 0x7fc944100e60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:27.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.686+0000 7fc949c00700 1 -- 172.21.15.117:0/3602364864 >> 172.21.15.117:0/3602364864 conn(0x7fc9440fc000 msgr2=0x7fc9440fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:27.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.686+0000 7fc949c00700 1 -- 172.21.15.117:0/3602364864 shutdown_connections 2024-04-20T11:10:27.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.686+0000 7fc949c00700 1 -- 172.21.15.117:0/3602364864 wait complete. 2024-04-20T11:10:27.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.687+0000 7fc949c00700 1 Processor -- start 2024-04-20T11:10:27.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.688+0000 7fc949c00700 1 -- start start 2024-04-20T11:10:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.688+0000 7fc949c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 0x7fc944198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.688+0000 7fc949c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc944101c40 0x7fc944198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.688+0000 7fc949c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9441992c0 con 0x7fc944100a40 2024-04-20T11:10:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.688+0000 7fc949c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944199400 con 0x7fc944101c40 2024-04-20T11:10:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.689+0000 7fc943e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 0x7fc944198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.689+0000 7fc943e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 0x7fc944198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60506/0 (socket says 172.21.15.117:60506) 2024-04-20T11:10:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.689+0000 7fc943e00700 1 -- 172.21.15.117:0/3087114690 learned_addr learned my addr 172.21.15.117:0/3087114690 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:10:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.689+0000 7fc943400700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc944101c40 0x7fc944198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.689+0000 7fc943e00700 1 -- 172.21.15.117:0/3087114690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc944101c40 msgr2=0x7fc944198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.689+0000 7fc943e00700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc944101c40 0x7fc944198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.689+0000 7fc943e00700 1 -- 172.21.15.117:0/3087114690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9340095f0 con 0x7fc944100a40 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.690+0000 7fc943e00700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 0x7fc944198760 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc93400bd70 tx=0x7fc93400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.690+0000 7fc940c00700 1 -- 172.21.15.117:0/3087114690 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc93401d070 con 0x7fc944100a40 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.690+0000 7fc940c00700 1 -- 172.21.15.117:0/3087114690 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc93400f460 con 0x7fc944100a40 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.690+0000 7fc940c00700 1 -- 172.21.15.117:0/3087114690 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc934021640 con 0x7fc944100a40 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.690+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94418eea0 con 0x7fc944100a40 2024-04-20T11:10:27.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.690+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94418f3c0 con 0x7fc944100a40 2024-04-20T11:10:27.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.692+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc944106140 con 0x7fc944100a40 2024-04-20T11:10:27.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.692+0000 7fc940c00700 1 -- 172.21.15.117:0/3087114690 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fc93400fad0 con 0x7fc944100a40 2024-04-20T11:10:27.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.693+0000 7fc940c00700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc92806e940 0x7fc928070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:27.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.693+0000 7fc940c00700 1 -- 172.21.15.117:0/3087114690 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fc93408fbd0 con 0x7fc944100a40 2024-04-20T11:10:27.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.694+0000 7fc943400700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc92806e940 0x7fc928070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:27.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.695+0000 7fc943400700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc92806e940 0x7fc928070e00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc930009a10 tx=0x7fc930009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:27.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:27.702+0000 7fc940c00700 1 -- 172.21.15.117:0/3087114690 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fc93405c9f0 con 0x7fc944100a40 2024-04-20T11:10:28.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.119+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc94402cdb0 con 0x7fc92806e940 2024-04-20T11:10:28.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.121+0000 7fc940c00700 1 -- 172.21.15.117:0/3087114690 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fc94402cdb0 con 0x7fc92806e940 2024-04-20T11:10:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.125+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc92806e940 msgr2=0x7fc928070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.125+0000 7fc949c00700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc92806e940 0x7fc928070e00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc930009a10 tx=0x7fc930009190 comp rx=0 tx=0).stop 2024-04-20T11:10:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.125+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 msgr2=0x7fc944198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.125+0000 7fc949c00700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 0x7fc944198760 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc93400bd70 tx=0x7fc93400bda0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.126+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 shutdown_connections 2024-04-20T11:10:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.126+0000 7fc949c00700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc92806e940 0x7fc928070e00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.126+0000 7fc949c00700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc944100a40 0x7fc944198760 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.126+0000 7fc949c00700 1 --2- 172.21.15.117:0/3087114690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc944101c40 0x7fc944198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.126+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 >> 172.21.15.117:0/3087114690 conn(0x7fc9440fc000 msgr2=0x7fc944103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:28.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.126+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 shutdown_connections 2024-04-20T11:10:28.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.126+0000 7fc949c00700 1 -- 172.21.15.117:0/3087114690 wait complete. 2024-04-20T11:10:28.146 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:10:28.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.310+0000 7fee92a00700 1 -- 172.21.15.117:0/1796903637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c101e20 msgr2=0x7fee8c102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:28.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.310+0000 7fee92a00700 1 --2- 172.21.15.117:0/1796903637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c101e20 0x7fee8c102280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fee78009910 tx=0x7fee78009c20 comp rx=0 tx=0).stop 2024-04-20T11:10:28.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.310+0000 7fee92a00700 1 -- 172.21.15.117:0/1796903637 shutdown_connections 2024-04-20T11:10:28.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.310+0000 7fee92a00700 1 --2- 172.21.15.117:0/1796903637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c101e20 0x7fee8c102280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.310+0000 7fee92a00700 1 --2- 172.21.15.117:0/1796903637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c100c20 0x7fee8c101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.310+0000 7fee92a00700 1 -- 172.21.15.117:0/1796903637 >> 172.21.15.117:0/1796903637 conn(0x7fee8c0fc1e0 msgr2=0x7fee8c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:28.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.310+0000 7fee92a00700 1 -- 172.21.15.117:0/1796903637 shutdown_connections 2024-04-20T11:10:28.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.310+0000 7fee92a00700 1 -- 172.21.15.117:0/1796903637 wait complete. 2024-04-20T11:10:28.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.311+0000 7fee92a00700 1 Processor -- start 2024-04-20T11:10:28.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.312+0000 7fee92a00700 1 -- start start 2024-04-20T11:10:28.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.312+0000 7fee92a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c100c20 0x7fee8c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:28.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.312+0000 7fee92a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c101e20 0x7fee8c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:28.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.312+0000 7fee92a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c1992a0 con 0x7fee8c100c20 2024-04-20T11:10:28.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.312+0000 7fee92a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c1993e0 con 0x7fee8c101e20 2024-04-20T11:10:28.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.312+0000 7fee8be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c100c20 0x7fee8c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:28.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.313+0000 7fee8be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c100c20 0x7fee8c1987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60512/0 (socket says 172.21.15.117:60512) 2024-04-20T11:10:28.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.313+0000 7fee8be00700 1 -- 172.21.15.117:0/118498356 learned_addr learned my addr 172.21.15.117:0/118498356 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:10:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.313+0000 7fee8be00700 1 -- 172.21.15.117:0/118498356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c101e20 msgr2=0x7fee8c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.313+0000 7fee8be00700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c101e20 0x7fee8c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.313+0000 7fee8be00700 1 -- 172.21.15.117:0/118498356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780095f0 con 0x7fee8c100c20 2024-04-20T11:10:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.313+0000 7fee8be00700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c100c20 0x7fee8c1987d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c910 tx=0x7fee7c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.314+0000 7fee88c00700 1 -- 172.21.15.117:0/118498356 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee7c0041f0 con 0x7fee8c100c20 2024-04-20T11:10:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.314+0000 7fee88c00700 1 -- 172.21.15.117:0/118498356 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee7c007dc0 con 0x7fee8c100c20 2024-04-20T11:10:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.314+0000 7fee88c00700 1 -- 172.21.15.117:0/118498356 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee7c003e70 con 0x7fee8c100c20 2024-04-20T11:10:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.314+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8c19de50 con 0x7fee8c100c20 2024-04-20T11:10:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.314+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8c19e340 con 0x7fee8c100c20 2024-04-20T11:10:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.315+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8c106320 con 0x7fee8c100c20 2024-04-20T11:10:28.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.316+0000 7fee88c00700 1 -- 172.21.15.117:0/118498356 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fee7c004350 con 0x7fee8c100c20 2024-04-20T11:10:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.317+0000 7fee88c00700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee7006e940 0x7fee70070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.317+0000 7fee88c00700 1 -- 172.21.15.117:0/118498356 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fee7c08e5e0 con 0x7fee8c100c20 2024-04-20T11:10:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.318+0000 7fee8b400700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee7006e940 0x7fee70070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:28.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.319+0000 7fee8b400700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee7006e940 0x7fee70070e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fee780051c0 tx=0x7fee78005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:28.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.324+0000 7fee88c00700 1 -- 172.21.15.117:0/118498356 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fee7c05b400 con 0x7fee8c100c20 2024-04-20T11:10:28.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.686+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee8c02cef0 con 0x7fee7006e940 2024-04-20T11:10:28.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.687+0000 7fee88c00700 1 -- 172.21.15.117:0/118498356 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fee8c02cef0 con 0x7fee7006e940 2024-04-20T11:10:28.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee7006e940 msgr2=0x7fee70070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:28.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee7006e940 0x7fee70070e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fee780051c0 tx=0x7fee78005fd0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c100c20 msgr2=0x7fee8c1987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:28.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c100c20 0x7fee8c1987d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c910 tx=0x7fee7c00ccd0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 shutdown_connections 2024-04-20T11:10:28.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee7006e940 0x7fee70070e00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee8c100c20 0x7fee8c1987d0 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c910 tx=0x7fee7c00ccd0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 --2- 172.21.15.117:0/118498356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c101e20 0x7fee8c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.690+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 >> 172.21.15.117:0/118498356 conn(0x7fee8c0fc1e0 msgr2=0x7fee8c103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:28.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.691+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 shutdown_connections 2024-04-20T11:10:28.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.691+0000 7fee92a00700 1 -- 172.21.15.117:0/118498356 wait complete. 2024-04-20T11:10:28.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.867+0000 7f4391c00700 1 -- 172.21.15.117:0/3882227098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c101e00 msgr2=0x7f438c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:28.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.867+0000 7f4391c00700 1 --2- 172.21.15.117:0/3882227098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c101e00 0x7f438c102260 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4378009910 tx=0x7f4378009c20 comp rx=0 tx=0).stop 2024-04-20T11:10:28.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.867+0000 7f4391c00700 1 -- 172.21.15.117:0/3882227098 shutdown_connections 2024-04-20T11:10:28.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.867+0000 7f4391c00700 1 --2- 172.21.15.117:0/3882227098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c101e00 0x7f438c102260 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.867+0000 7f4391c00700 1 --2- 172.21.15.117:0/3882227098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f438c100c00 0x7f438c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.867+0000 7f4391c00700 1 -- 172.21.15.117:0/3882227098 >> 172.21.15.117:0/3882227098 conn(0x7f438c0fc1c0 msgr2=0x7f438c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:28.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.867+0000 7f4391c00700 1 -- 172.21.15.117:0/3882227098 shutdown_connections 2024-04-20T11:10:28.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.867+0000 7f4391c00700 1 -- 172.21.15.117:0/3882227098 wait complete. 2024-04-20T11:10:28.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.869+0000 7f4391c00700 1 Processor -- start 2024-04-20T11:10:28.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.869+0000 7f4391c00700 1 -- start start 2024-04-20T11:10:28.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.869+0000 7f4391c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c100c00 0x7f438c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:28.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.869+0000 7f4391c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f438c101e00 0x7f438c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:28.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.869+0000 7f4391c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c199320 con 0x7f438c100c00 2024-04-20T11:10:28.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.869+0000 7f4391c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c199460 con 0x7f438c101e00 2024-04-20T11:10:28.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.870+0000 7f438aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c100c00 0x7f438c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:28.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.870+0000 7f438aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c100c00 0x7f438c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60534/0 (socket says 172.21.15.117:60534) 2024-04-20T11:10:28.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.870+0000 7f438aa00700 1 -- 172.21.15.117:0/1519061633 learned_addr learned my addr 172.21.15.117:0/1519061633 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:10:28.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.870+0000 7f438a000700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f438c101e00 0x7f438c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.870+0000 7f438aa00700 1 -- 172.21.15.117:0/1519061633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f438c101e00 msgr2=0x7f438c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.870+0000 7f438aa00700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f438c101e00 0x7f438c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.870+0000 7f438aa00700 1 -- 172.21.15.117:0/1519061633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780095f0 con 0x7f438c100c00 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.870+0000 7f438aa00700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c100c00 0x7f438c1987c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f437c00e8f0 tx=0x7f437c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.871+0000 7f4383400700 1 -- 172.21.15.117:0/1519061633 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f437c009790 con 0x7f438c100c00 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.871+0000 7f4383400700 1 -- 172.21.15.117:0/1519061633 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f437c00c810 con 0x7f438c100c00 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.871+0000 7f4383400700 1 -- 172.21.15.117:0/1519061633 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f437c003d20 con 0x7f438c100c00 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.871+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c19dec0 con 0x7f438c100c00 2024-04-20T11:10:28.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.871+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c19e3b0 con 0x7f438c100c00 2024-04-20T11:10:28.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.873+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c1062b0 con 0x7f438c100c00 2024-04-20T11:10:28.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.874+0000 7f4383400700 1 -- 172.21.15.117:0/1519061633 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f437c0041f0 con 0x7f438c100c00 2024-04-20T11:10:28.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.875+0000 7f4383400700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f437006ea00 0x7f4370070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:28.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.875+0000 7f4383400700 1 -- 172.21.15.117:0/1519061633 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f437c014070 con 0x7f438c100c00 2024-04-20T11:10:28.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.876+0000 7f438a000700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f437006ea00 0x7f4370070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:28.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.877+0000 7f438a000700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f437006ea00 0x7f4370070ec0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f437800b800 tx=0x7f4378009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:28.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:28.881+0000 7f4383400700 1 -- 172.21.15.117:0/1519061633 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f437c05b940 con 0x7f438c100c00 2024-04-20T11:10:29.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.300+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f438c02ceb0 con 0x7f437006ea00 2024-04-20T11:10:29.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.313+0000 7f4383400700 1 -- 172.21.15.117:0/1519061633 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f438c02ceb0 con 0x7f437006ea00 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (11m) 24s ago 12m 13.9M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (13m) 24s ago 13m 8061k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (11m) 0s ago 11m 7880k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (13m) 24s ago 13m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (11m) 0s ago 11m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (10m) 24s ago 12m 76.4M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (5m) 24s ago 5m 14.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:10:29.313 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (5m) 24s ago 5m 240M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (5m) 0s ago 5m 14.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (5m) 0s ago 5m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (2m) 24s ago 14m 494M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (36s) 0s ago 11m 447M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (14m) 24s ago 14m 55.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (11m) 0s ago 11m 44.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 starting - - - - 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (5s) 0s ago 11m 2512k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (10m) 24s ago 10m 185M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (10m) 24s ago 10m 179M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (9m) 24s ago 9m 146M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (8m) 0s ago 8m 205M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:10:29.314 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (8m) 0s ago 8m 182M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:10:29.315 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (7m) 0s ago 7m 171M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:10:29.315 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (52s) 24s ago 12m 50.5M - 2.43.0 a07b618ecd1d ddbcab7472b9 2024-04-20T11:10:29.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f437006ea00 msgr2=0x7f4370070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f437006ea00 0x7f4370070ec0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f437800b800 tx=0x7f4378009f70 comp rx=0 tx=0).stop 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c100c00 msgr2=0x7f438c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c100c00 0x7f438c1987c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f437c00e8f0 tx=0x7f437c00ec00 comp rx=0 tx=0).stop 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 shutdown_connections 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f437006ea00 0x7f4370070ec0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c100c00 0x7f438c1987c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 --2- 172.21.15.117:0/1519061633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f438c101e00 0x7f438c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 >> 172.21.15.117:0/1519061633 conn(0x7f438c0fc1c0 msgr2=0x7f438c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:29.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.316+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 shutdown_connections 2024-04-20T11:10:29.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.317+0000 7f4391c00700 1 -- 172.21.15.117:0/1519061633 wait complete. 2024-04-20T11:10:29.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.499+0000 7fc801c00700 1 -- 172.21.15.117:0/1939664452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc103df0 msgr2=0x7fc7fc1061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:29.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.499+0000 7fc801c00700 1 --2- 172.21.15.117:0/1939664452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc103df0 0x7fc7fc1061e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009910 tx=0x7fc7e8009c20 comp rx=0 tx=0).stop 2024-04-20T11:10:29.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.499+0000 7fc801c00700 1 -- 172.21.15.117:0/1939664452 shutdown_connections 2024-04-20T11:10:29.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.499+0000 7fc801c00700 1 --2- 172.21.15.117:0/1939664452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc103df0 0x7fc7fc1061e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.499+0000 7fc801c00700 1 --2- 172.21.15.117:0/1939664452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc1014c0 0x7fc7fc1038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.499+0000 7fc801c00700 1 -- 172.21.15.117:0/1939664452 >> 172.21.15.117:0/1939664452 conn(0x7fc7fc0faef0 msgr2=0x7fc7fc0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:29.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.499+0000 7fc801c00700 1 -- 172.21.15.117:0/1939664452 shutdown_connections 2024-04-20T11:10:29.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:29 smithi117 ceph-mon[15659]: pgmap v54: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-04-20T11:10:29.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:29 smithi117 ceph-mon[15659]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:10:29.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:29 smithi117 ceph-mon[15659]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:10:29.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:29 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:29.500 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:29 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.500+0000 7fc801c00700 1 -- 172.21.15.117:0/1939664452 wait complete. 2024-04-20T11:10:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.501+0000 7fc801c00700 1 Processor -- start 2024-04-20T11:10:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.501+0000 7fc801c00700 1 -- start start 2024-04-20T11:10:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.501+0000 7fc801c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1014c0 0x7fc7fc071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.501+0000 7fc801c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103df0 0x7fc7fc0724c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.501+0000 7fc801c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc072ae0 con 0x7fc7fc1014c0 2024-04-20T11:10:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.501+0000 7fc801c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc072c20 con 0x7fc7fc103df0 2024-04-20T11:10:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.502+0000 7fc7fbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1014c0 0x7fc7fc071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.502+0000 7fc7fbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1014c0 0x7fc7fc071f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60550/0 (socket says 172.21.15.117:60550) 2024-04-20T11:10:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.502+0000 7fc7fbe00700 1 -- 172.21.15.117:0/703090 learned_addr learned my addr 172.21.15.117:0/703090 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:10:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.502+0000 7fc7fb400700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103df0 0x7fc7fc0724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.502+0000 7fc7fbe00700 1 -- 172.21.15.117:0/703090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103df0 msgr2=0x7fc7fc0724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:29.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.502+0000 7fc7fbe00700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103df0 0x7fc7fc0724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.502+0000 7fc7fbe00700 1 -- 172.21.15.117:0/703090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e80095f0 con 0x7fc7fc1014c0 2024-04-20T11:10:29.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.503+0000 7fc7fbe00700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1014c0 0x7fc7fc071f80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00b4b0 tx=0x7fc7ec00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:29.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.503+0000 7fc7f8c00700 1 -- 172.21.15.117:0/703090 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7ec011800 con 0x7fc7fc1014c0 2024-04-20T11:10:29.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.503+0000 7fc7f8c00700 1 -- 172.21.15.117:0/703090 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7ec011e20 con 0x7fc7fc1014c0 2024-04-20T11:10:29.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.503+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7fc1a4410 con 0x7fc7fc1014c0 2024-04-20T11:10:29.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.503+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc1a49e0 con 0x7fc7fc1014c0 2024-04-20T11:10:29.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.503+0000 7fc7f8c00700 1 -- 172.21.15.117:0/703090 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7ec00f2e0 con 0x7fc7fc1014c0 2024-04-20T11:10:29.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.505+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7dc005320 con 0x7fc7fc1014c0 2024-04-20T11:10:29.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.506+0000 7fc7f8c00700 1 -- 172.21.15.117:0/703090 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fc7ec011960 con 0x7fc7fc1014c0 2024-04-20T11:10:29.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.506+0000 7fc7f8c00700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc7e006e940 0x7fc7e0070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:29.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.507+0000 7fc7f8c00700 1 -- 172.21.15.117:0/703090 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fc7ec08e3d0 con 0x7fc7fc1014c0 2024-04-20T11:10:29.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.507+0000 7fc7fb400700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc7e006e940 0x7fc7e0070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:29.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.508+0000 7fc7fb400700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc7e006e940 0x7fc7e0070e00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80051c0 tx=0x7fc7e800b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:29.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.515+0000 7fc7f8c00700 1 -- 172.21.15.117:0/703090 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fc7ec05b1f0 con 0x7fc7fc1014c0 2024-04-20T11:10:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:29 smithi173 ceph-mon[23800]: pgmap v54: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-04-20T11:10:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:29 smithi173 ceph-mon[23800]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:10:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:29 smithi173 ceph-mon[23800]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:10:29.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:29 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:29.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:29 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:29.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.954+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc7dc005de0 con 0x7fc7fc1014c0 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.955+0000 7fc7f8c00700 1 -- 172.21.15.117:0/703090 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7fc7ec05ad80 con 0x7fc7fc1014c0 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:10:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:10:29.956 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:10:29.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc7e006e940 msgr2=0x7fc7e0070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc7e006e940 0x7fc7e0070e00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80051c0 tx=0x7fc7e800b540 comp rx=0 tx=0).stop 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1014c0 msgr2=0x7fc7fc071f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1014c0 0x7fc7fc071f80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00b4b0 tx=0x7fc7ec00b870 comp rx=0 tx=0).stop 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 shutdown_connections 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc7e006e940 0x7fc7e0070e00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1014c0 0x7fc7fc071f80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 --2- 172.21.15.117:0/703090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103df0 0x7fc7fc0724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 >> 172.21.15.117:0/703090 conn(0x7fc7fc0faef0 msgr2=0x7fc7fc0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 shutdown_connections 2024-04-20T11:10:29.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:29.958+0000 7fc801c00700 1 -- 172.21.15.117:0/703090 wait complete. 2024-04-20T11:10:30.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.174+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3965564991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cd8103f80 msgr2=0x7f1cd8104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:30.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.174+0000 7f1cdcc00700 1 --2- 172.21.15.117:0/3965564991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cd8103f80 0x7f1cd8104400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009910 tx=0x7f1cc4009c20 comp rx=0 tx=0).stop 2024-04-20T11:10:30.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.175+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3965564991 shutdown_connections 2024-04-20T11:10:30.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.175+0000 7f1cdcc00700 1 --2- 172.21.15.117:0/3965564991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cd8103f80 0x7f1cd8104400 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:30.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.175+0000 7f1cdcc00700 1 --2- 172.21.15.117:0/3965564991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd8102e20 0x7f1cd8103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:30.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.175+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3965564991 >> 172.21.15.117:0/3965564991 conn(0x7f1cd80fe3a0 msgr2=0x7f1cd8100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:30.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.175+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3965564991 shutdown_connections 2024-04-20T11:10:30.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.176+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3965564991 wait complete. 2024-04-20T11:10:30.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.178+0000 7f1cdcc00700 1 Processor -- start 2024-04-20T11:10:30.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.178+0000 7f1cdcc00700 1 -- start start 2024-04-20T11:10:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.179+0000 7f1cdcc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd8102e20 0x7f1cd8071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.179+0000 7f1cdcc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cd8103f80 0x7f1cd80724c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.179+0000 7f1cdcc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8072a90 con 0x7f1cd8102e20 2024-04-20T11:10:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.179+0000 7f1cdcc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8072c00 con 0x7f1cd8103f80 2024-04-20T11:10:30.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.179+0000 7f1cd5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd8102e20 0x7f1cd8071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:30.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.180+0000 7f1cd5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd8102e20 0x7f1cd8071f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60560/0 (socket says 172.21.15.117:60560) 2024-04-20T11:10:30.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.180+0000 7f1cd5e00700 1 -- 172.21.15.117:0/3098997311 learned_addr learned my addr 172.21.15.117:0/3098997311 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:10:30.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.180+0000 7f1cd5400700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cd8103f80 0x7f1cd80724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:30.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.181+0000 7f1cd5e00700 1 -- 172.21.15.117:0/3098997311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cd8103f80 msgr2=0x7f1cd80724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:30.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.181+0000 7f1cd5e00700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cd8103f80 0x7f1cd80724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:30.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.181+0000 7f1cd5e00700 1 -- 172.21.15.117:0/3098997311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc40095f0 con 0x7f1cd8102e20 2024-04-20T11:10:30.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.182+0000 7f1cd5e00700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd8102e20 0x7f1cd8071f80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800e950 tx=0x7f1cc800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:30.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.183+0000 7f1ccea00700 1 -- 172.21.15.117:0/3098997311 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc800cba0 con 0x7f1cd8102e20 2024-04-20T11:10:30.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.183+0000 7f1ccea00700 1 -- 172.21.15.117:0/3098997311 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cc800eea0 con 0x7f1cd8102e20 2024-04-20T11:10:30.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.183+0000 7f1ccea00700 1 -- 172.21.15.117:0/3098997311 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc8018650 con 0x7f1cd8102e20 2024-04-20T11:10:30.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.183+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd81aab10 con 0x7f1cd8102e20 2024-04-20T11:10:30.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.183+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd81ab060 con 0x7f1cd8102e20 2024-04-20T11:10:30.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.185+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd806c0a0 con 0x7f1cd8102e20 2024-04-20T11:10:30.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.186+0000 7f1ccea00700 1 -- 172.21.15.117:0/3098997311 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f1cc800cd00 con 0x7f1cd8102e20 2024-04-20T11:10:30.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.187+0000 7f1ccea00700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f1cbc06ea60 0x7f1cbc070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:30.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.189+0000 7f1ccea00700 1 -- 172.21.15.117:0/3098997311 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f1cc8014070 con 0x7f1cd8102e20 2024-04-20T11:10:30.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.189+0000 7f1cd5400700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f1cbc06ea60 0x7f1cbc070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:30.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.191+0000 7f1cd5400700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f1cbc06ea60 0x7f1cbc070f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b6f0 tx=0x7f1cc4005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:30.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.196+0000 7f1ccea00700 1 -- 172.21.15.117:0/3098997311 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f1cc8053f60 con 0x7f1cd8102e20 2024-04-20T11:10:30.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.539+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cd802ced0 con 0x7f1cbc06ea60 2024-04-20T11:10:30.539 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:30 smithi117 ceph-mon[15659]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:10:30.539 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:30 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/703090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:30.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.541+0000 7f1ccea00700 1 -- 172.21.15.117:0/3098997311 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f1cd802ced0 con 0x7f1cbc06ea60 2024-04-20T11:10:30.540 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:10:30.540 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "2/2 daemons upgraded", 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:10:30.541 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:10:30.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.543+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f1cbc06ea60 msgr2=0x7f1cbc070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:30.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.543+0000 7f1cdcc00700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f1cbc06ea60 0x7f1cbc070f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b6f0 tx=0x7f1cc4005fb0 comp rx=0 tx=0).stop 2024-04-20T11:10:30.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd8102e20 msgr2=0x7f1cd8071f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:30.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd8102e20 0x7f1cd8071f80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800e950 tx=0x7f1cc800ec60 comp rx=0 tx=0).stop 2024-04-20T11:10:30.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 shutdown_connections 2024-04-20T11:10:30.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f1cbc06ea60 0x7f1cbc070f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:30.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd8102e20 0x7f1cd8071f80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:30.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 --2- 172.21.15.117:0/3098997311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cd8103f80 0x7f1cd80724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:30.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 >> 172.21.15.117:0/3098997311 conn(0x7f1cd80fe3a0 msgr2=0x7f1cd8107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:30.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 shutdown_connections 2024-04-20T11:10:30.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:10:30.544+0000 7f1cdcc00700 1 -- 172.21.15.117:0/3098997311 wait complete. 2024-04-20T11:10:30.564 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:30 smithi173 ceph-mon[23800]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:10:30.564 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:30 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/703090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:31 smithi173 ceph-mon[23800]: pgmap v55: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 994 KiB/s wr, 74 op/s 2024-04-20T11:10:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:31 smithi173 ceph-mon[23800]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:10:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:10:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:31.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:31 smithi117 ceph-mon[15659]: pgmap v55: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 994 KiB/s wr, 74 op/s 2024-04-20T11:10:31.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:31 smithi117 ceph-mon[15659]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:10:31.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:10:31.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:31.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:33.570 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:33 smithi117 ceph-mon[15659]: pgmap v56: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1009 KiB/s wr, 84 op/s 2024-04-20T11:10:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:33 smithi173 ceph-mon[23800]: pgmap v56: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1009 KiB/s wr, 84 op/s 2024-04-20T11:10:35.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:35 smithi173 ceph-mon[23800]: pgmap v57: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-04-20T11:10:35.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:35 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:35.915 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:35 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:36.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:35 smithi117 ceph-mon[15659]: pgmap v57: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-04-20T11:10:36.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:35 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:36.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:35 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:38.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:37 smithi117 ceph-mon[15659]: pgmap v58: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-04-20T11:10:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:37 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:37 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:37 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:37 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:38.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:37 smithi173 ceph-mon[23800]: pgmap v58: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-04-20T11:10:38.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:37 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:38.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:37 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:38.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:37 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:38.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:37 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:39.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:39 smithi117 ceph-mon[15659]: pgmap v59: 65 pgs: 65 active+clean; 297 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.9 MiB/s wr, 111 op/s 2024-04-20T11:10:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:39 smithi173 ceph-mon[23800]: pgmap v59: 65 pgs: 65 active+clean; 297 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.9 MiB/s wr, 111 op/s 2024-04-20T11:10:42.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:41 smithi117 ceph-mon[15659]: pgmap v60: 65 pgs: 65 active+clean; 297 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 72 op/s 2024-04-20T11:10:42.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:41 smithi173 ceph-mon[23800]: pgmap v60: 65 pgs: 65 active+clean; 297 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 72 op/s 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: pgmap v61: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: pgmap v61: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:10:46.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:45 smithi117 ceph-mon[15659]: pgmap v62: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.9 MiB/s wr, 106 op/s 2024-04-20T11:10:46.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:45 smithi173 ceph-mon[23800]: pgmap v62: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.9 MiB/s wr, 106 op/s 2024-04-20T11:10:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:10:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:10:47.858 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:47 smithi117 ceph-mon[15659]: Upgrade: Updating prometheus.smithi117 2024-04-20T11:10:47.858 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:47 smithi117 ceph-mon[15659]: pgmap v63: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-04-20T11:10:47.858 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:47 smithi117 ceph-mon[15659]: Deploying daemon prometheus.smithi117 on smithi117 2024-04-20T11:10:48.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:47 smithi173 ceph-mon[23800]: Upgrade: Updating prometheus.smithi117 2024-04-20T11:10:48.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:47 smithi173 ceph-mon[23800]: pgmap v63: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-04-20T11:10:48.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:47 smithi173 ceph-mon[23800]: Deploying daemon prometheus.smithi117 on smithi117 2024-04-20T11:10:49.898 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:49 smithi117 ceph-mon[15659]: pgmap v64: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-04-20T11:10:50.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:49 smithi173 ceph-mon[23800]: pgmap v64: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-04-20T11:10:52.124 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:51 smithi117 ceph-mon[15659]: pgmap v65: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 1.1 MiB/s wr, 78 op/s 2024-04-20T11:10:52.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:51 smithi173 ceph-mon[23800]: pgmap v65: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 1.1 MiB/s wr, 78 op/s 2024-04-20T11:10:53.128 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:52 smithi117 ceph-mon[15659]: pgmap v66: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-20T11:10:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:52 smithi173 ceph-mon[23800]: pgmap v66: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-20T11:10:55.272 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:55 smithi117 ceph-mon[15659]: pgmap v67: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2024-04-20T11:10:55.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:55 smithi173 ceph-mon[23800]: pgmap v67: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2024-04-20T11:10:56.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:56 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:56 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:56 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:10:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:56 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:56 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:10:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:56 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:10:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:57 smithi173 ceph-mon[23800]: pgmap v68: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-04-20T11:10:57.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:57 smithi117 ceph-mon[15659]: pgmap v68: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-04-20T11:10:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:10:59 smithi173 ceph-mon[23800]: pgmap v69: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-04-20T11:10:59.737 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:10:59 smithi117 ceph-mon[15659]: pgmap v69: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-04-20T11:11:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.721+0000 7f7b0a200700 1 -- 172.21.15.117:0/2911016193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 msgr2=0x7f7b041038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.721+0000 7f7b0a200700 1 --2- 172.21.15.117:0/2911016193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 0x7f7b041038b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009910 tx=0x7f7af8009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.721+0000 7f7b0a200700 1 -- 172.21.15.117:0/2911016193 shutdown_connections 2024-04-20T11:11:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.721+0000 7f7b0a200700 1 --2- 172.21.15.117:0/2911016193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103df0 0x7f7b041061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.721+0000 7f7b0a200700 1 --2- 172.21.15.117:0/2911016193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 0x7f7b041038b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.721+0000 7f7b0a200700 1 -- 172.21.15.117:0/2911016193 >> 172.21.15.117:0/2911016193 conn(0x7f7b040faef0 msgr2=0x7f7b040fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.721+0000 7f7b0a200700 1 -- 172.21.15.117:0/2911016193 shutdown_connections 2024-04-20T11:11:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.721+0000 7f7b0a200700 1 -- 172.21.15.117:0/2911016193 wait complete. 2024-04-20T11:11:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.722+0000 7f7b0a200700 1 Processor -- start 2024-04-20T11:11:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.723+0000 7f7b0a200700 1 -- start start 2024-04-20T11:11:00.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.723+0000 7f7b0a200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 0x7f7b04100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:00.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.723+0000 7f7b0a200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103df0 0x7f7b040ff470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:00.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.723+0000 7f7b0a200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040ff9b0 con 0x7f7b041014c0 2024-04-20T11:11:00.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.723+0000 7f7b0a200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040ffaf0 con 0x7f7b04103df0 2024-04-20T11:11:00.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.724+0000 7f7b08e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 0x7f7b04100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.724+0000 7f7b08e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 0x7f7b04100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40278/0 (socket says 172.21.15.117:40278) 2024-04-20T11:11:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.724+0000 7f7b08e00700 1 -- 172.21.15.117:0/198951909 learned_addr learned my addr 172.21.15.117:0/198951909 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.724+0000 7f7b03e00700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103df0 0x7f7b040ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.724+0000 7f7b08e00700 1 -- 172.21.15.117:0/198951909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103df0 msgr2=0x7f7b040ff470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.724+0000 7f7b08e00700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103df0 0x7f7b040ff470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.724+0000 7f7b08e00700 1 -- 172.21.15.117:0/198951909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80095f0 con 0x7f7b041014c0 2024-04-20T11:11:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.725+0000 7f7b08e00700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 0x7f7b04100dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7af80059d0 tx=0x7f7af800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.725+0000 7f7b01600700 1 -- 172.21.15.117:0/198951909 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af801d070 con 0x7f7b041014c0 2024-04-20T11:11:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.725+0000 7f7b01600700 1 -- 172.21.15.117:0/198951909 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7af8005dc0 con 0x7f7b041014c0 2024-04-20T11:11:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.725+0000 7f7b01600700 1 -- 172.21.15.117:0/198951909 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af8021410 con 0x7f7b041014c0 2024-04-20T11:11:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.725+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b040ffd70 con 0x7f7b041014c0 2024-04-20T11:11:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.725+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b041002e0 con 0x7f7b041014c0 2024-04-20T11:11:00.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.727+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04062510 con 0x7f7b041014c0 2024-04-20T11:11:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.730+0000 7f7b01600700 1 -- 172.21.15.117:0/198951909 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f7af800f460 con 0x7f7b041014c0 2024-04-20T11:11:00.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.731+0000 7f7b01600700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f7aec06e940 0x7f7aec070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:00.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.732+0000 7f7b01600700 1 -- 172.21.15.117:0/198951909 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f7af808f830 con 0x7f7b041014c0 2024-04-20T11:11:00.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.733+0000 7f7b03e00700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f7aec06e940 0x7f7aec070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:00.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.734+0000 7f7b03e00700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f7aec06e940 0x7f7aec070e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005950 tx=0x7f7af00058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:00.736+0000 7f7b01600700 1 -- 172.21.15.117:0/198951909 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f7af805c8b0 con 0x7f7b041014c0 2024-04-20T11:11:01.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.159+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b041002e0 con 0x7f7aec06e940 2024-04-20T11:11:01.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.161+0000 7f7b01600700 1 -- 172.21.15.117:0/198951909 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f7b041002e0 con 0x7f7aec06e940 2024-04-20T11:11:01.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f7aec06e940 msgr2=0x7f7aec070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:01.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f7aec06e940 0x7f7aec070e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005950 tx=0x7f7af00058e0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 msgr2=0x7f7b04100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:01.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 0x7f7b04100dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7af80059d0 tx=0x7f7af800be20 comp rx=0 tx=0).stop 2024-04-20T11:11:01.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 shutdown_connections 2024-04-20T11:11:01.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f7aec06e940 0x7f7aec070e00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b041014c0 0x7f7b04100dd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 --2- 172.21.15.117:0/198951909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103df0 0x7f7b040ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 >> 172.21.15.117:0/198951909 conn(0x7f7b040faef0 msgr2=0x7f7b040fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:01.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.164+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 shutdown_connections 2024-04-20T11:11:01.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.165+0000 7f7b0a200700 1 -- 172.21.15.117:0/198951909 wait complete. 2024-04-20T11:11:01.183 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:11:01.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.380+0000 7f715c800700 1 -- 172.21.15.117:0/1077862353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7154101e40 msgr2=0x7f71541042d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:01.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.380+0000 7f715c800700 1 --2- 172.21.15.117:0/1077862353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7154101e40 0x7f71541042d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7144009910 tx=0x7f7144009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:01.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.381+0000 7f715c800700 1 -- 172.21.15.117:0/1077862353 shutdown_connections 2024-04-20T11:11:01.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.381+0000 7f715c800700 1 --2- 172.21.15.117:0/1077862353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7154101e40 0x7f71541042d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.381+0000 7f715c800700 1 --2- 172.21.15.117:0/1077862353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71540ff4d0 0x7f7154101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.381+0000 7f715c800700 1 -- 172.21.15.117:0/1077862353 >> 172.21.15.117:0/1077862353 conn(0x7f71540fb0d0 msgr2=0x7f71540fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:01.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.381+0000 7f715c800700 1 -- 172.21.15.117:0/1077862353 shutdown_connections 2024-04-20T11:11:01.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.381+0000 7f715c800700 1 -- 172.21.15.117:0/1077862353 wait complete. 2024-04-20T11:11:01.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.383+0000 7f715c800700 1 Processor -- start 2024-04-20T11:11:01.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.383+0000 7f715c800700 1 -- start start 2024-04-20T11:11:01.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.384+0000 7f715c800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71540ff4d0 0x7f715419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:01.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.384+0000 7f715c800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7154101e40 0x7f715419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:01.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.384+0000 7f715c800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715419d710 con 0x7f7154101e40 2024-04-20T11:11:01.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.384+0000 7f715c800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715419d850 con 0x7f71540ff4d0 2024-04-20T11:11:01.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.384+0000 7f7159e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71540ff4d0 0x7f715419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:01.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.384+0000 7f7159e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71540ff4d0 0x7f715419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.117:60506/0 (socket says 172.21.15.117:60506) 2024-04-20T11:11:01.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.384+0000 7f7159e00700 1 -- 172.21.15.117:0/2982897069 learned_addr learned my addr 172.21.15.117:0/2982897069 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:01.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.385+0000 7f7159400700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7154101e40 0x7f715419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:01.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.385+0000 7f7159e00700 1 -- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7154101e40 msgr2=0x7f715419d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:01.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.385+0000 7f7159e00700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7154101e40 0x7f715419d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.385+0000 7f7159e00700 1 -- 172.21.15.117:0/2982897069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71440095f0 con 0x7f71540ff4d0 2024-04-20T11:11:01.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.385+0000 7f7159400700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7154101e40 0x7f715419d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:11:01.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.386+0000 7f7159e00700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71540ff4d0 0x7f715419cbb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f714800e920 tx=0x7f714800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:01.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.386+0000 7f714ea00700 1 -- 172.21.15.117:0/2982897069 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7148009790 con 0x7f71540ff4d0 2024-04-20T11:11:01.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.386+0000 7f714ea00700 1 -- 172.21.15.117:0/2982897069 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f714800cba0 con 0x7f71540ff4d0 2024-04-20T11:11:01.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.386+0000 7f714ea00700 1 -- 172.21.15.117:0/2982897069 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7148018650 con 0x7f71540ff4d0 2024-04-20T11:11:01.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.386+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7154105c90 con 0x7f71540ff4d0 2024-04-20T11:11:01.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.387+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7154106260 con 0x7f71540ff4d0 2024-04-20T11:11:01.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.389+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7154062510 con 0x7f71540ff4d0 2024-04-20T11:11:01.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.391+0000 7f714ea00700 1 -- 172.21.15.117:0/2982897069 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f714800cd10 con 0x7f71540ff4d0 2024-04-20T11:11:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.393+0000 7f714ea00700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f713c06e940 0x7f713c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:01.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.393+0000 7f714ea00700 1 -- 172.21.15.117:0/2982897069 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f7148014070 con 0x7f71540ff4d0 2024-04-20T11:11:01.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.395+0000 7f7159400700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f713c06e940 0x7f713c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:01.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.397+0000 7f7159400700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f713c06e940 0x7f713c070e00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7144006010 tx=0x7f7144009f90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:01.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.400+0000 7f714ea00700 1 -- 172.21.15.117:0/2982897069 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f7148053f60 con 0x7f71540ff4d0 2024-04-20T11:11:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:01 smithi173 ceph-mon[23800]: pgmap v70: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-04-20T11:11:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:01 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:11:01.727 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:01 smithi117 ceph-mon[15659]: pgmap v70: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-04-20T11:11:01.727 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:01 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:11:01.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.727+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71541028a0 con 0x7f713c06e940 2024-04-20T11:11:01.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.729+0000 7f714ea00700 1 -- 172.21.15.117:0/2982897069 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f71541028a0 con 0x7f713c06e940 2024-04-20T11:11:01.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.732+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f713c06e940 msgr2=0x7f713c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:01.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.732+0000 7f715c800700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f713c06e940 0x7f713c070e00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7144006010 tx=0x7f7144009f90 comp rx=0 tx=0).stop 2024-04-20T11:11:01.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.732+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71540ff4d0 msgr2=0x7f715419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:01.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.732+0000 7f715c800700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71540ff4d0 0x7f715419cbb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f714800e920 tx=0x7f714800ece0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.733+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 shutdown_connections 2024-04-20T11:11:01.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.733+0000 7f715c800700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f713c06e940 0x7f713c070e00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.733+0000 7f715c800700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71540ff4d0 0x7f715419cbb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.733+0000 7f715c800700 1 --2- 172.21.15.117:0/2982897069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7154101e40 0x7f715419d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.733+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 >> 172.21.15.117:0/2982897069 conn(0x7f71540fb0d0 msgr2=0x7f71540fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:01.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.733+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 shutdown_connections 2024-04-20T11:11:01.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.733+0000 7f715c800700 1 -- 172.21.15.117:0/2982897069 wait complete. 2024-04-20T11:11:01.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.898+0000 7f0b59c00700 1 -- 172.21.15.117:0/700649633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 msgr2=0x7f0b54102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:01.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.898+0000 7f0b59c00700 1 --2- 172.21.15.117:0/700649633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 0x7f0b54102280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009910 tx=0x7f0b40009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:01.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.898+0000 7f0b59c00700 1 -- 172.21.15.117:0/700649633 shutdown_connections 2024-04-20T11:11:01.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.898+0000 7f0b59c00700 1 --2- 172.21.15.117:0/700649633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 0x7f0b54102280 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.898+0000 7f0b59c00700 1 --2- 172.21.15.117:0/700649633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b54100c20 0x7f0b54101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.898+0000 7f0b59c00700 1 -- 172.21.15.117:0/700649633 >> 172.21.15.117:0/700649633 conn(0x7f0b540fc1e0 msgr2=0x7f0b540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:01.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.898+0000 7f0b59c00700 1 -- 172.21.15.117:0/700649633 shutdown_connections 2024-04-20T11:11:01.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.898+0000 7f0b59c00700 1 -- 172.21.15.117:0/700649633 wait complete. 2024-04-20T11:11:01.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.899+0000 7f0b59c00700 1 Processor -- start 2024-04-20T11:11:01.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.899+0000 7f0b59c00700 1 -- start start 2024-04-20T11:11:01.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.900+0000 7f0b59c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b54100c20 0x7f0b54198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:01.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.900+0000 7f0b59c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 0x7f0b54198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:01.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.900+0000 7f0b59c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54199470 con 0x7f0b54101e20 2024-04-20T11:11:01.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.900+0000 7f0b59c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b541995b0 con 0x7f0b54100c20 2024-04-20T11:11:01.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.900+0000 7f0b52000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 0x7f0b54198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:01.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.900+0000 7f0b52000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 0x7f0b54198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40318/0 (socket says 172.21.15.117:40318) 2024-04-20T11:11:01.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.900+0000 7f0b52000700 1 -- 172.21.15.117:0/3005531205 learned_addr learned my addr 172.21.15.117:0/3005531205 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:01.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.900+0000 7f0b52a00700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b54100c20 0x7f0b54198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:01.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b52000700 1 -- 172.21.15.117:0/3005531205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b54100c20 msgr2=0x7f0b54198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b52000700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b54100c20 0x7f0b54198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b52000700 1 -- 172.21.15.117:0/3005531205 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400095f0 con 0x7f0b54101e20 2024-04-20T11:11:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b52000700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 0x7f0b54198e50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000bd70 tx=0x7f0b4000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b4b400700 1 -- 172.21.15.117:0/3005531205 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4001d070 con 0x7f0b54101e20 2024-04-20T11:11:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b4b400700 1 -- 172.21.15.117:0/3005531205 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b4000f460 con 0x7f0b54101e20 2024-04-20T11:11:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b4b400700 1 -- 172.21.15.117:0/3005531205 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b40021620 con 0x7f0b54101e20 2024-04-20T11:11:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5419dfb0 con 0x7f0b54101e20 2024-04-20T11:11:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.901+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5419e4f0 con 0x7f0b54101e20 2024-04-20T11:11:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.903+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54106320 con 0x7f0b54101e20 2024-04-20T11:11:01.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.904+0000 7f0b4b400700 1 -- 172.21.15.117:0/3005531205 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f0b4000fa70 con 0x7f0b54101e20 2024-04-20T11:11:01.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.905+0000 7f0b4b400700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f0b3806e940 0x7f0b38070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:01.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.905+0000 7f0b4b400700 1 -- 172.21.15.117:0/3005531205 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f0b4008fbf0 con 0x7f0b54101e20 2024-04-20T11:11:01.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.906+0000 7f0b52a00700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f0b3806e940 0x7f0b38070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:01.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.907+0000 7f0b52a00700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f0b3806e940 0x7f0b38070e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b44006fd0 tx=0x7f0b44008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:01.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:01.915+0000 7f0b4b400700 1 -- 172.21.15.117:0/3005531205 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f0b4005ca10 con 0x7f0b54101e20 2024-04-20T11:11:02.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.217+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b5402cdb0 con 0x7f0b3806e940 2024-04-20T11:11:02.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.231+0000 7f0b4b400700 1 -- 172.21.15.117:0/3005531205 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0b5402cdb0 con 0x7f0b3806e940 2024-04-20T11:11:02.231 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:11:02.231 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (11m) 27s ago 13m 13.9M - 0.25.0 c8568f914cd2 367b18de4954 2024-04-20T11:11:02.231 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (13m) 27s ago 13m 8128k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (12m) 33s ago 12m 7880k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (13m) 27s ago 13m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (12m) 33s ago 12m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (11m) 27s ago 13m 76.5M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (6m) 27s ago 6m 14.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (6m) 27s ago 6m 240M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (6m) 33s ago 6m 14.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (6m) 33s ago 6m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (2m) 27s ago 15m 496M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (69s) 33s ago 12m 447M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (15m) 27s ago 15m 56.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (12m) 33s ago 12m 44.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (47s) 27s ago 13m 6735k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:11:02.232 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (38s) 33s ago 12m 2512k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:11:02.233 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (11m) 27s ago 11m 180M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:11:02.233 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (10m) 27s ago 10m 175M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:11:02.233 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (9m) 27s ago 9m 153M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:11:02.233 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (9m) 33s ago 9m 205M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:11:02.233 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (8m) 33s ago 8m 182M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:11:02.233 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (8m) 33s ago 8m 171M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:11:02.233 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 starting - - - - 2024-04-20T11:11:02.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f0b3806e940 msgr2=0x7f0b38070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f0b3806e940 0x7f0b38070e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b44006fd0 tx=0x7f0b44008040 comp rx=0 tx=0).stop 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 msgr2=0x7f0b54198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 0x7f0b54198e50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000bd70 tx=0x7f0b4000be50 comp rx=0 tx=0).stop 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 shutdown_connections 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f0b3806e940 0x7f0b38070e00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b54100c20 0x7f0b54198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 --2- 172.21.15.117:0/3005531205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b54101e20 0x7f0b54198e50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 >> 172.21.15.117:0/3005531205 conn(0x7f0b540fc1e0 msgr2=0x7f0b54103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:02.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.234+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 shutdown_connections 2024-04-20T11:11:02.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.235+0000 7f0b59c00700 1 -- 172.21.15.117:0/3005531205 wait complete. 2024-04-20T11:11:02.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.450+0000 7f8d7e800700 1 -- 172.21.15.117:0/2227799200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78100c20 msgr2=0x7f8d78101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:02.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.450+0000 7f8d7e800700 1 --2- 172.21.15.117:0/2227799200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78100c20 0x7f8d78101040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009910 tx=0x7f8d68009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:02.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.450+0000 7f8d7e800700 1 -- 172.21.15.117:0/2227799200 shutdown_connections 2024-04-20T11:11:02.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.450+0000 7f8d7e800700 1 --2- 172.21.15.117:0/2227799200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d78101e20 0x7f8d78102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.450+0000 7f8d7e800700 1 --2- 172.21.15.117:0/2227799200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78100c20 0x7f8d78101040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.450+0000 7f8d7e800700 1 -- 172.21.15.117:0/2227799200 >> 172.21.15.117:0/2227799200 conn(0x7f8d780fc1e0 msgr2=0x7f8d780fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:02.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.451+0000 7f8d7e800700 1 -- 172.21.15.117:0/2227799200 shutdown_connections 2024-04-20T11:11:02.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.451+0000 7f8d7e800700 1 -- 172.21.15.117:0/2227799200 wait complete. 2024-04-20T11:11:02.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.453+0000 7f8d7e800700 1 Processor -- start 2024-04-20T11:11:02.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.454+0000 7f8d7e800700 1 -- start start 2024-04-20T11:11:02.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.454+0000 7f8d7e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d78100c20 0x7f8d78196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:02.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.454+0000 7f8d7e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78101e20 0x7f8d78196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:02.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.454+0000 7f8d7e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78197160 con 0x7f8d78101e20 2024-04-20T11:11:02.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.454+0000 7f8d7e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d781972a0 con 0x7f8d78100c20 2024-04-20T11:11:02.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.454+0000 7f8d76a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78101e20 0x7f8d78196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:02.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.455+0000 7f8d76a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78101e20 0x7f8d78196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40334/0 (socket says 172.21.15.117:40334) 2024-04-20T11:11:02.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.455+0000 7f8d76a00700 1 -- 172.21.15.117:0/4273876267 learned_addr learned my addr 172.21.15.117:0/4273876267 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:02.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.455+0000 7f8d77400700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d78100c20 0x7f8d78196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:02.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.455+0000 7f8d76a00700 1 -- 172.21.15.117:0/4273876267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d78100c20 msgr2=0x7f8d78196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:02.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.455+0000 7f8d76a00700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d78100c20 0x7f8d78196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.455+0000 7f8d76a00700 1 -- 172.21.15.117:0/4273876267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680095f0 con 0x7f8d78101e20 2024-04-20T11:11:02.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.456+0000 7f8d76a00700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78101e20 0x7f8d78196b40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d8d0 tx=0x7f8d6400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:02.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.456+0000 7f8d6fe00700 1 -- 172.21.15.117:0/4273876267 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d640098a0 con 0x7f8d78101e20 2024-04-20T11:11:02.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.456+0000 7f8d6fe00700 1 -- 172.21.15.117:0/4273876267 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d64010460 con 0x7f8d78101e20 2024-04-20T11:11:02.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.456+0000 7f8d6fe00700 1 -- 172.21.15.117:0/4273876267 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d6400f5f0 con 0x7f8d78101e20 2024-04-20T11:11:02.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.456+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7819bd00 con 0x7f8d78101e20 2024-04-20T11:11:02.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.456+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7819c2d0 con 0x7f8d78101e20 2024-04-20T11:11:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.459+0000 7f8d6fe00700 1 -- 172.21.15.117:0/4273876267 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f8d6400f750 con 0x7f8d78101e20 2024-04-20T11:11:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.460+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d781907a0 con 0x7f8d78101e20 2024-04-20T11:11:02.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.461+0000 7f8d6fe00700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8d5c06e990 0x7f8d5c070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:02.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.461+0000 7f8d6fe00700 1 -- 172.21.15.117:0/4273876267 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f8d6408e670 con 0x7f8d78101e20 2024-04-20T11:11:02.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.462+0000 7f8d77400700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8d5c06e990 0x7f8d5c070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:02.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.465+0000 7f8d77400700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8d5c06e990 0x7f8d5c070e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005090 tx=0x7f8d68005180 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:02.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.467+0000 7f8d6fe00700 1 -- 172.21.15.117:0/4273876267 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f8d6405b740 con 0x7f8d78101e20 2024-04-20T11:11:02.595 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:02 smithi117 ceph-mon[15659]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:02.596 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:02 smithi117 ceph-mon[15659]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:02 smithi173 ceph-mon[23800]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:02 smithi173 ceph-mon[23800]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:02.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.875+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d7819be90 con 0x7f8d78101e20 2024-04-20T11:11:02.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.877+0000 7f8d6fe00700 1 -- 172.21.15.117:0/4273876267 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f8d7819be90 con 0x7f8d78101e20 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:11:02.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:11:02.878 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:11:02.878 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:11:02.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:11:02.878 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:11:02.878 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:11:02.878 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:11:02.878 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:11:02.878 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:11:02.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8d5c06e990 msgr2=0x7f8d5c070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:02.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8d5c06e990 0x7f8d5c070e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005090 tx=0x7f8d68005180 comp rx=0 tx=0).stop 2024-04-20T11:11:02.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78101e20 msgr2=0x7f8d78196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:02.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78101e20 0x7f8d78196b40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d8d0 tx=0x7f8d6400dbe0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 shutdown_connections 2024-04-20T11:11:02.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f8d5c06e990 0x7f8d5c070e50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d78100c20 0x7f8d78196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 --2- 172.21.15.117:0/4273876267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d78101e20 0x7f8d78196b40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:02.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.880+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 >> 172.21.15.117:0/4273876267 conn(0x7f8d780fc1e0 msgr2=0x7f8d78105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:02.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.881+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 shutdown_connections 2024-04-20T11:11:02.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:02.881+0000 7f8d7e800700 1 -- 172.21.15.117:0/4273876267 wait complete. 2024-04-20T11:11:03.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.062+0000 7f1445800700 1 -- 172.21.15.117:0/2216566586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 msgr2=0x7f1440102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:03.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.062+0000 7f1445800700 1 --2- 172.21.15.117:0/2216566586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 0x7f1440102260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f142c009910 tx=0x7f142c009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:03.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.063+0000 7f1445800700 1 -- 172.21.15.117:0/2216566586 shutdown_connections 2024-04-20T11:11:03.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.063+0000 7f1445800700 1 --2- 172.21.15.117:0/2216566586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 0x7f1440102260 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.063+0000 7f1445800700 1 --2- 172.21.15.117:0/2216566586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1440100c00 0x7f1440101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.063+0000 7f1445800700 1 -- 172.21.15.117:0/2216566586 >> 172.21.15.117:0/2216566586 conn(0x7f14400fc1c0 msgr2=0x7f14400fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:03.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.063+0000 7f1445800700 1 -- 172.21.15.117:0/2216566586 shutdown_connections 2024-04-20T11:11:03.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.063+0000 7f1445800700 1 -- 172.21.15.117:0/2216566586 wait complete. 2024-04-20T11:11:03.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.064+0000 7f1445800700 1 Processor -- start 2024-04-20T11:11:03.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.064+0000 7f1445800700 1 -- start start 2024-04-20T11:11:03.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.065+0000 7f1445800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1440100c00 0x7f14401987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:03.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.065+0000 7f1445800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 0x7f1440198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:03.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.065+0000 7f1445800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440199320 con 0x7f1440101e00 2024-04-20T11:11:03.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.065+0000 7f1445800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440199460 con 0x7f1440100c00 2024-04-20T11:11:03.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.065+0000 7f143e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 0x7f1440198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:03.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.065+0000 7f143e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 0x7f1440198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40360/0 (socket says 172.21.15.117:40360) 2024-04-20T11:11:03.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.065+0000 7f143e000700 1 -- 172.21.15.117:0/3967782902 learned_addr learned my addr 172.21.15.117:0/3967782902 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:03.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.065+0000 7f143ea00700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1440100c00 0x7f14401987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:03.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.066+0000 7f143ea00700 1 -- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 msgr2=0x7f1440198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:03.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.066+0000 7f143ea00700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 0x7f1440198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.066+0000 7f143ea00700 1 -- 172.21.15.117:0/3967782902 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0095f0 con 0x7f1440100c00 2024-04-20T11:11:03.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.067+0000 7f143ea00700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1440100c00 0x7f14401987c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f143000d6c0 tx=0x7f143000d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:03.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.067+0000 7f1437400700 1 -- 172.21.15.117:0/3967782902 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14300041f0 con 0x7f1440100c00 2024-04-20T11:11:03.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.067+0000 7f1437400700 1 -- 172.21.15.117:0/3967782902 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1430004350 con 0x7f1440100c00 2024-04-20T11:11:03.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.067+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144019dec0 con 0x7f1440100c00 2024-04-20T11:11:03.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.067+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144019e400 con 0x7f1440100c00 2024-04-20T11:11:03.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.068+0000 7f1437400700 1 -- 172.21.15.117:0/3967782902 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1430003d20 con 0x7f1440100c00 2024-04-20T11:11:03.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.068+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14401062b0 con 0x7f1440100c00 2024-04-20T11:11:03.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.072+0000 7f1437400700 1 -- 172.21.15.117:0/3967782902 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f143000f690 con 0x7f1440100c00 2024-04-20T11:11:03.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.073+0000 7f1437400700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f142406e940 0x7f1424070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:03.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.073+0000 7f1437400700 1 -- 172.21.15.117:0/3967782902 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f143008e820 con 0x7f1440100c00 2024-04-20T11:11:03.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.074+0000 7f143e000700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f142406e940 0x7f1424070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:03.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.074+0000 7f143e000700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f142406e940 0x7f1424070e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f142c0051c0 tx=0x7f142c00b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:03.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.086+0000 7f1437400700 1 -- 172.21.15.117:0/3967782902 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f143005b640 con 0x7f1440100c00 2024-04-20T11:11:03.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.384+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f144002ce50 con 0x7f142406e940 2024-04-20T11:11:03.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.386+0000 7f1437400700 1 -- 172.21.15.117:0/3967782902 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f144002ce50 con 0x7f142406e940 2024-04-20T11:11:03.385 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:11:03.385 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:11:03.385 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:11:03.386 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:11:03.386 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:11:03.386 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:11:03.386 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:11:03.386 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "2/2 daemons upgraded", 2024-04-20T11:11:03.386 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-20T11:11:03.386 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:11:03.386 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:11:03.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.388+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f142406e940 msgr2=0x7f1424070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:03.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.388+0000 7f1445800700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f142406e940 0x7f1424070e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f142c0051c0 tx=0x7f142c00b540 comp rx=0 tx=0).stop 2024-04-20T11:11:03.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.388+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1440100c00 msgr2=0x7f14401987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:03.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.388+0000 7f1445800700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1440100c00 0x7f14401987c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f143000d6c0 tx=0x7f143000d9d0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.389+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 shutdown_connections 2024-04-20T11:11:03.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.389+0000 7f1445800700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f142406e940 0x7f1424070e00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.389+0000 7f1445800700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1440100c00 0x7f14401987c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.389+0000 7f1445800700 1 --2- 172.21.15.117:0/3967782902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440101e00 0x7f1440198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.389+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 >> 172.21.15.117:0/3967782902 conn(0x7f14400fc1c0 msgr2=0x7f14401033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:03.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.389+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 shutdown_connections 2024-04-20T11:11:03.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:03.389+0000 7f1445800700 1 -- 172.21.15.117:0/3967782902 wait complete. 2024-04-20T11:11:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:03 smithi117 ceph-mon[15659]: pgmap v71: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-04-20T11:11:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:03 smithi117 ceph-mon[15659]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:03 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/4273876267' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:03 smithi173 ceph-mon[23800]: pgmap v71: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-04-20T11:11:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:03 smithi173 ceph-mon[23800]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:03 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/4273876267' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:04.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:04 smithi117 ceph-mon[15659]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:04 smithi173 ceph-mon[23800]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:05.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:05 smithi117 ceph-mon[15659]: pgmap v72: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-20T11:11:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:05 smithi173 ceph-mon[23800]: pgmap v72: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-20T11:11:07.840 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:07 smithi117 ceph-mon[15659]: pgmap v73: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-04-20T11:11:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:07 smithi173 ceph-mon[23800]: pgmap v73: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-04-20T11:11:09.810 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:09 smithi117 ceph-mon[15659]: pgmap v74: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-04-20T11:11:09.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:09 smithi173 ceph-mon[23800]: pgmap v74: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-04-20T11:11:11.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:11 smithi117 ceph-mon[15659]: pgmap v75: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-04-20T11:11:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:11 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:11 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:11 smithi173 ceph-mon[23800]: pgmap v75: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-04-20T11:11:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:11 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:11 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:13 smithi173 ceph-mon[23800]: pgmap v76: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.3 MiB/s wr, 236 op/s 2024-04-20T11:11:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:13 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:13 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:13.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:13 smithi117 ceph-mon[15659]: pgmap v76: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.3 MiB/s wr, 236 op/s 2024-04-20T11:11:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:13 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:13 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:15 smithi117 ceph-mon[15659]: pgmap v77: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 363 op/s 2024-04-20T11:11:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:15 smithi173 ceph-mon[23800]: pgmap v77: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 363 op/s 2024-04-20T11:11:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:16 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:11:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:16 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:11:17.825 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:17 smithi117 ceph-mon[15659]: pgmap v78: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-04-20T11:11:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:17 smithi173 ceph-mon[23800]: pgmap v78: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-04-20T11:11:19.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:19 smithi117 ceph-mon[15659]: pgmap v79: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.8 MiB/s wr, 432 op/s 2024-04-20T11:11:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:19 smithi173 ceph-mon[23800]: pgmap v79: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.8 MiB/s wr, 432 op/s 2024-04-20T11:11:20.953 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-20T11:11:20.953 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: pgmap v80: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 304 op/s 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: pgmap v80: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 304 op/s 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:21.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:22.277 DEBUG:teuthology.parallel:result is None 2024-04-20T11:11:22.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:22 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:11:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:22 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-20T11:11:23.829 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:23 smithi173 ceph-mon[23800]: pgmap v81: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 1.3 MiB/s wr, 332 op/s 2024-04-20T11:11:23.847 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:23 smithi117 ceph-mon[15659]: pgmap v81: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 1.3 MiB/s wr, 332 op/s 2024-04-20T11:11:25.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:25 smithi117 ceph-mon[15659]: pgmap v82: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 1.5 MiB/s wr, 343 op/s 2024-04-20T11:11:25.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:25 smithi117 ceph-mon[15659]: Upgrade: Updating alertmanager.smithi117 2024-04-20T11:11:25.875 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:25 smithi117 ceph-mon[15659]: Deploying daemon alertmanager.smithi117 on smithi117 2024-04-20T11:11:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:25 smithi173 ceph-mon[23800]: pgmap v82: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 1.5 MiB/s wr, 343 op/s 2024-04-20T11:11:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:25 smithi173 ceph-mon[23800]: Upgrade: Updating alertmanager.smithi117 2024-04-20T11:11:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:25 smithi173 ceph-mon[23800]: Deploying daemon alertmanager.smithi117 on smithi117 2024-04-20T11:11:27.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:27 smithi117 ceph-mon[15659]: pgmap v83: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 969 KiB/s wr, 202 op/s 2024-04-20T11:11:27.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:27 smithi173 ceph-mon[23800]: pgmap v83: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 969 KiB/s wr, 202 op/s 2024-04-20T11:11:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:29 smithi117 ceph-mon[15659]: pgmap v84: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 251 op/s 2024-04-20T11:11:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:29 smithi173 ceph-mon[23800]: pgmap v84: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 251 op/s 2024-04-20T11:11:31.870 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:31 smithi117 ceph-mon[15659]: pgmap v85: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 16 KiB/s rd, 680 KiB/s wr, 140 op/s 2024-04-20T11:11:31.871 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:11:31.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:31 smithi173 ceph-mon[23800]: pgmap v85: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 16 KiB/s rd, 680 KiB/s wr, 140 op/s 2024-04-20T11:11:31.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:11:33.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.561+0000 7fa524a00700 1 -- 172.21.15.117:0/3767175953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 msgr2=0x7fa520103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:33.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.561+0000 7fa524a00700 1 --2- 172.21.15.117:0/3767175953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 0x7fa520103050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa510009910 tx=0x7fa510009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.562+0000 7fa524a00700 1 -- 172.21.15.117:0/3767175953 shutdown_connections 2024-04-20T11:11:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.562+0000 7fa524a00700 1 --2- 172.21.15.117:0/3767175953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa520103e30 0x7fa520104290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.562+0000 7fa524a00700 1 --2- 172.21.15.117:0/3767175953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 0x7fa520103050 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.562+0000 7fa524a00700 1 -- 172.21.15.117:0/3767175953 >> 172.21.15.117:0/3767175953 conn(0x7fa5200fe210 msgr2=0x7fa520100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.562+0000 7fa524a00700 1 -- 172.21.15.117:0/3767175953 shutdown_connections 2024-04-20T11:11:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.562+0000 7fa524a00700 1 -- 172.21.15.117:0/3767175953 wait complete. 2024-04-20T11:11:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.563+0000 7fa524a00700 1 Processor -- start 2024-04-20T11:11:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.563+0000 7fa524a00700 1 -- start start 2024-04-20T11:11:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.563+0000 7fa524a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 0x7fa520078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.563+0000 7fa524a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa520103e30 0x7fa520079180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.563+0000 7fa524a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200756f0 con 0x7fa520102c30 2024-04-20T11:11:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.563+0000 7fa524a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa520075830 con 0x7fa520103e30 2024-04-20T11:11:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.564+0000 7fa51e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 0x7fa520078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.564+0000 7fa51e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 0x7fa520078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60948/0 (socket says 172.21.15.117:60948) 2024-04-20T11:11:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.564+0000 7fa51e800700 1 -- 172.21.15.117:0/998859897 learned_addr learned my addr 172.21.15.117:0/998859897 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:33.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.564+0000 7fa51e800700 1 -- 172.21.15.117:0/998859897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa520103e30 msgr2=0x7fa520079180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.564+0000 7fa51e800700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa520103e30 0x7fa520079180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.564+0000 7fa51e800700 1 -- 172.21.15.117:0/998859897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100095f0 con 0x7fa520102c30 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.565+0000 7fa51e800700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 0x7fa520078c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa5100059d0 tx=0x7fa51000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.567+0000 7fa517400700 1 -- 172.21.15.117:0/998859897 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa51001d070 con 0x7fa520102c30 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.567+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa520075ab0 con 0x7fa520102c30 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.568+0000 7fa517400700 1 -- 172.21.15.117:0/998859897 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa51000f460 con 0x7fa520102c30 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.568+0000 7fa517400700 1 -- 172.21.15.117:0/998859897 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa510021640 con 0x7fa520102c30 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.568+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa520076020 con 0x7fa520102c30 2024-04-20T11:11:33.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.569+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa500005320 con 0x7fa520102c30 2024-04-20T11:11:33.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.571+0000 7fa517400700 1 -- 172.21.15.117:0/998859897 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fa510003a60 con 0x7fa520102c30 2024-04-20T11:11:33.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.572+0000 7fa517400700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa50406e940 0x7fa504070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:33.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.572+0000 7fa517400700 1 -- 172.21.15.117:0/998859897 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fa51008fb30 con 0x7fa520102c30 2024-04-20T11:11:33.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.573+0000 7fa51de00700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa50406e940 0x7fa504070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:33.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.573+0000 7fa51de00700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa50406e940 0x7fa504070e00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00a9b0 tx=0x7fa50c005c90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:33.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.579+0000 7fa517400700 1 -- 172.21.15.117:0/998859897 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fa51005c950 con 0x7fa520102c30 2024-04-20T11:11:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:33 smithi117 ceph-mon[15659]: pgmap v86: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 705 KiB/s wr, 148 op/s 2024-04-20T11:11:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:33 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:33 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:33 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:33 smithi173 ceph-mon[23800]: pgmap v86: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 705 KiB/s wr, 148 op/s 2024-04-20T11:11:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:33 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:33 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:33 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:33.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.914+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa500000bf0 con 0x7fa50406e940 2024-04-20T11:11:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.917+0000 7fa517400700 1 -- 172.21.15.117:0/998859897 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fa500000bf0 con 0x7fa50406e940 2024-04-20T11:11:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa50406e940 msgr2=0x7fa504070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa50406e940 0x7fa504070e00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00a9b0 tx=0x7fa50c005c90 comp rx=0 tx=0).stop 2024-04-20T11:11:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 msgr2=0x7fa520078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 0x7fa520078c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa5100059d0 tx=0x7fa51000be20 comp rx=0 tx=0).stop 2024-04-20T11:11:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 shutdown_connections 2024-04-20T11:11:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fa50406e940 0x7fa504070e00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa520102c30 0x7fa520078c40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 --2- 172.21.15.117:0/998859897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa520103e30 0x7fa520079180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 >> 172.21.15.117:0/998859897 conn(0x7fa5200fe210 msgr2=0x7fa520107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 shutdown_connections 2024-04-20T11:11:33.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:33.921+0000 7fa524a00700 1 -- 172.21.15.117:0/998859897 wait complete. 2024-04-20T11:11:33.944 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:11:34.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.116+0000 7f882ee00700 1 -- 172.21.15.117:0/948452249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828101680 msgr2=0x7f8828103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:34.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.116+0000 7f882ee00700 1 --2- 172.21.15.117:0/948452249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828101680 0x7f8828103a70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8818009910 tx=0x7f8818009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:34.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.116+0000 7f882ee00700 1 -- 172.21.15.117:0/948452249 shutdown_connections 2024-04-20T11:11:34.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.116+0000 7f882ee00700 1 --2- 172.21.15.117:0/948452249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8828103fb0 0x7f88281063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.116+0000 7f882ee00700 1 --2- 172.21.15.117:0/948452249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828101680 0x7f8828103a70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.116+0000 7f882ee00700 1 -- 172.21.15.117:0/948452249 >> 172.21.15.117:0/948452249 conn(0x7f88280fb0d0 msgr2=0x7f88280fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:34.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.116+0000 7f882ee00700 1 -- 172.21.15.117:0/948452249 shutdown_connections 2024-04-20T11:11:34.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.116+0000 7f882ee00700 1 -- 172.21.15.117:0/948452249 wait complete. 2024-04-20T11:11:34.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.117+0000 7f882ee00700 1 Processor -- start 2024-04-20T11:11:34.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.118+0000 7f882ee00700 1 -- start start 2024-04-20T11:11:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.118+0000 7f882ee00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8828101680 0x7f8828198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.118+0000 7f882ee00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828103fb0 0x7f8828198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.118+0000 7f882ee00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88281992e0 con 0x7f8828103fb0 2024-04-20T11:11:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.118+0000 7f882ee00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8828199420 con 0x7f8828101680 2024-04-20T11:11:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.118+0000 7f8827400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828103fb0 0x7f8828198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.118+0000 7f8827400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828103fb0 0x7f8828198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58170/0 (socket says 172.21.15.117:58170) 2024-04-20T11:11:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.118+0000 7f8827400700 1 -- 172.21.15.117:0/841386213 learned_addr learned my addr 172.21.15.117:0/841386213 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.119+0000 7f8827e00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8828101680 0x7f8828198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.119+0000 7f8827e00700 1 -- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828103fb0 msgr2=0x7f8828198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.119+0000 7f8827e00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828103fb0 0x7f8828198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.119+0000 7f8827e00700 1 -- 172.21.15.117:0/841386213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180095f0 con 0x7f8828101680 2024-04-20T11:11:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.120+0000 7f8827e00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8828101680 0x7f8828198780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8818000c00 tx=0x7f881800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.120+0000 7f8824c00700 1 -- 172.21.15.117:0/841386213 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f881801d070 con 0x7f8828101680 2024-04-20T11:11:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.120+0000 7f8824c00700 1 -- 172.21.15.117:0/841386213 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8818005dc0 con 0x7f8828101680 2024-04-20T11:11:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.120+0000 7f8824c00700 1 -- 172.21.15.117:0/841386213 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f881800fe80 con 0x7f8828101680 2024-04-20T11:11:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.120+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882819de20 con 0x7f8828101680 2024-04-20T11:11:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.120+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882819e2e0 con 0x7f8828101680 2024-04-20T11:11:34.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.122+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88280fcc50 con 0x7f8828101680 2024-04-20T11:11:34.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.123+0000 7f8824c00700 1 -- 172.21.15.117:0/841386213 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f881800f6f0 con 0x7f8828101680 2024-04-20T11:11:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.124+0000 7f8824c00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f880c06e990 0x7f880c070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.124+0000 7f8824c00700 1 -- 172.21.15.117:0/841386213 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f881808f990 con 0x7f8828101680 2024-04-20T11:11:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.125+0000 7f8827400700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f880c06e990 0x7f880c070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.126+0000 7f8827400700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f880c06e990 0x7f880c070e50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8814005950 tx=0x7f881400b500 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:34.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.144+0000 7f8824c00700 1 -- 172.21.15.117:0/841386213 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f881805c7b0 con 0x7f8828101680 2024-04-20T11:11:34.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.457+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f882802cea0 con 0x7f880c06e990 2024-04-20T11:11:34.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.459+0000 7f8824c00700 1 -- 172.21.15.117:0/841386213 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f882802cea0 con 0x7f880c06e990 2024-04-20T11:11:34.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f880c06e990 msgr2=0x7f880c070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:34.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f880c06e990 0x7f880c070e50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8814005950 tx=0x7f881400b500 comp rx=0 tx=0).stop 2024-04-20T11:11:34.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8828101680 msgr2=0x7f8828198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:34.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8828101680 0x7f8828198780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8818000c00 tx=0x7f881800bd70 comp rx=0 tx=0).stop 2024-04-20T11:11:34.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 shutdown_connections 2024-04-20T11:11:34.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f880c06e990 0x7f880c070e50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8828101680 0x7f8828198780 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 --2- 172.21.15.117:0/841386213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8828103fb0 0x7f8828198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 >> 172.21.15.117:0/841386213 conn(0x7f88280fb0d0 msgr2=0x7f8828100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:34.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 shutdown_connections 2024-04-20T11:11:34.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.462+0000 7f882ee00700 1 -- 172.21.15.117:0/841386213 wait complete. 2024-04-20T11:11:34.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.619+0000 7fee2ca00700 1 -- 172.21.15.117:0/296272856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee280ffb10 msgr2=0x7fee280fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:34.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.619+0000 7fee2ca00700 1 --2- 172.21.15.117:0/296272856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee280ffb10 0x7fee280fff30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fee18009910 tx=0x7fee18009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:34.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.619+0000 7fee2ca00700 1 -- 172.21.15.117:0/296272856 shutdown_connections 2024-04-20T11:11:34.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.619+0000 7fee2ca00700 1 --2- 172.21.15.117:0/296272856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee28100d10 0x7fee28101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.619+0000 7fee2ca00700 1 --2- 172.21.15.117:0/296272856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee280ffb10 0x7fee280fff30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.619+0000 7fee2ca00700 1 -- 172.21.15.117:0/296272856 >> 172.21.15.117:0/296272856 conn(0x7fee280fb0d0 msgr2=0x7fee280fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:34.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.619+0000 7fee2ca00700 1 -- 172.21.15.117:0/296272856 shutdown_connections 2024-04-20T11:11:34.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.619+0000 7fee2ca00700 1 -- 172.21.15.117:0/296272856 wait complete. 2024-04-20T11:11:34.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.620+0000 7fee2ca00700 1 Processor -- start 2024-04-20T11:11:34.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.621+0000 7fee2ca00700 1 -- start start 2024-04-20T11:11:34.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.621+0000 7fee2ca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee280ffb10 0x7fee281966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:34.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.621+0000 7fee2ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28100d10 0x7fee28196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.621+0000 7fee2ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28197200 con 0x7fee28100d10 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.621+0000 7fee2ca00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28197340 con 0x7fee280ffb10 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.621+0000 7fee25400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28100d10 0x7fee28196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.621+0000 7fee25400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28100d10 0x7fee28196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58188/0 (socket says 172.21.15.117:58188) 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.621+0000 7fee25400700 1 -- 172.21.15.117:0/3371692425 learned_addr learned my addr 172.21.15.117:0/3371692425 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.622+0000 7fee25e00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee280ffb10 0x7fee281966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.622+0000 7fee25400700 1 -- 172.21.15.117:0/3371692425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee280ffb10 msgr2=0x7fee281966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.622+0000 7fee25400700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee280ffb10 0x7fee281966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.622+0000 7fee25400700 1 -- 172.21.15.117:0/3371692425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee180095f0 con 0x7fee28100d10 2024-04-20T11:11:34.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.622+0000 7fee25e00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee280ffb10 0x7fee281966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:11:34.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.622+0000 7fee25400700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28100d10 0x7fee28196be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c8a0 tx=0x7fee1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:34.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.623+0000 7fee1ea00700 1 -- 172.21.15.117:0/3371692425 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1400cea0 con 0x7fee28100d10 2024-04-20T11:11:34.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.623+0000 7fee1ea00700 1 -- 172.21.15.117:0/3371692425 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee14004830 con 0x7fee28100d10 2024-04-20T11:11:34.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.623+0000 7fee1ea00700 1 -- 172.21.15.117:0/3371692425 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee140055b0 con 0x7fee28100d10 2024-04-20T11:11:34.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.623+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2819bd50 con 0x7fee28100d10 2024-04-20T11:11:34.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.623+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2819c2d0 con 0x7fee28100d10 2024-04-20T11:11:34.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.624+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee28062510 con 0x7fee28100d10 2024-04-20T11:11:34.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.626+0000 7fee1ea00700 1 -- 172.21.15.117:0/3371692425 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fee140049a0 con 0x7fee28100d10 2024-04-20T11:11:34.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.627+0000 7fee1ea00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee0c06e940 0x7fee0c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:34.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.627+0000 7fee1ea00700 1 -- 172.21.15.117:0/3371692425 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fee1408e570 con 0x7fee28100d10 2024-04-20T11:11:34.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.627+0000 7fee25e00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee0c06e940 0x7fee0c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:34.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.628+0000 7fee25e00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee0c06e940 0x7fee0c070e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee1800b5c0 tx=0x7fee18005160 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:34.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.637+0000 7fee1ea00700 1 -- 172.21.15.117:0/3371692425 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fee1405b310 con 0x7fee28100d10 2024-04-20T11:11:34.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.972+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fee28069580 con 0x7fee0c06e940 2024-04-20T11:11:34.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.983+0000 7fee1ea00700 1 -- 172.21.15.117:0/3371692425 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fee28069580 con 0x7fee0c06e940 2024-04-20T11:11:34.984 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:11:34.984 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 starting - - - - 2024-04-20T11:11:34.984 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (14m) 24s ago 14m 8166k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:11:34.984 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (13m) 65s ago 13m 7880k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (14m) 24s ago 14m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (12m) 65s ago 12m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (11m) 24s ago 13m 76.4M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (6m) 24s ago 6m 15.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (6m) 24s ago 6m 227M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (6m) 65s ago 6m 14.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (6m) 65s ago 6m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (3m) 24s ago 15m 500M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (102s) 65s ago 12m 447M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (15m) 24s ago 16m 57.4M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (12m) 65s ago 12m 44.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (80s) 24s ago 14m 7017k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:11:34.985 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (71s) 65s ago 12m 2512k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:11:34.986 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (11m) 24s ago 11m 187M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:11:34.986 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (11m) 24s ago 11m 188M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:11:34.986 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (10m) 24s ago 10m 161M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:11:34.986 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (9m) 65s ago 9m 205M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:11:34.986 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (9m) 65s ago 9m 182M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:11:34.986 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (8m) 65s ago 8m 171M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:11:34.986 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (40s) 24s ago 13m 41.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:11:34.986 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:34 smithi117 ceph-mon[15659]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:34.986 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:34 smithi117 ceph-mon[15659]: pgmap v87: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 786 KiB/s wr, 143 op/s 2024-04-20T11:11:34.986 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:34 smithi117 ceph-mon[15659]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:34.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee0c06e940 msgr2=0x7fee0c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:34.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee0c06e940 0x7fee0c070e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee1800b5c0 tx=0x7fee18005160 comp rx=0 tx=0).stop 2024-04-20T11:11:34.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28100d10 msgr2=0x7fee28196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:34.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28100d10 0x7fee28196be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c8a0 tx=0x7fee1400cc60 comp rx=0 tx=0).stop 2024-04-20T11:11:34.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 shutdown_connections 2024-04-20T11:11:34.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fee0c06e940 0x7fee0c070e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee280ffb10 0x7fee281966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 --2- 172.21.15.117:0/3371692425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28100d10 0x7fee28196be0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:34.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 >> 172.21.15.117:0/3371692425 conn(0x7fee280fb0d0 msgr2=0x7fee28068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:34.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 shutdown_connections 2024-04-20T11:11:34.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:34.987+0000 7fee2ca00700 1 -- 172.21.15.117:0/3371692425 wait complete. 2024-04-20T11:11:35.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.140+0000 7fd07f200700 1 -- 172.21.15.117:0/416655290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101680 msgr2=0x7fd078103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.140+0000 7fd07f200700 1 --2- 172.21.15.117:0/416655290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101680 0x7fd078103a70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd068009910 tx=0x7fd068009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:35.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.140+0000 7fd07f200700 1 -- 172.21.15.117:0/416655290 shutdown_connections 2024-04-20T11:11:35.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.140+0000 7fd07f200700 1 --2- 172.21.15.117:0/416655290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd078103fb0 0x7fd0781063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.140+0000 7fd07f200700 1 --2- 172.21.15.117:0/416655290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078101680 0x7fd078103a70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.140+0000 7fd07f200700 1 -- 172.21.15.117:0/416655290 >> 172.21.15.117:0/416655290 conn(0x7fd0780fb0d0 msgr2=0x7fd0780fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:35.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.141+0000 7fd07f200700 1 -- 172.21.15.117:0/416655290 shutdown_connections 2024-04-20T11:11:35.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.141+0000 7fd07f200700 1 -- 172.21.15.117:0/416655290 wait complete. 2024-04-20T11:11:35.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.142+0000 7fd07f200700 1 Processor -- start 2024-04-20T11:11:35.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.142+0000 7fd07f200700 1 -- start start 2024-04-20T11:11:35.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.142+0000 7fd07f200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd078101680 0x7fd0781988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.142+0000 7fd07f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078103fb0 0x7fd078198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.142+0000 7fd07f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078199420 con 0x7fd078103fb0 2024-04-20T11:11:35.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.142+0000 7fd07f200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078199560 con 0x7fd078101680 2024-04-20T11:11:35.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078103fb0 0x7fd078198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078103fb0 0x7fd078198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58200/0 (socket says 172.21.15.117:58200) 2024-04-20T11:11:35.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077400700 1 -- 172.21.15.117:0/286698438 learned_addr learned my addr 172.21.15.117:0/286698438 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077e00700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd078101680 0x7fd0781988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077400700 1 -- 172.21.15.117:0/286698438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd078101680 msgr2=0x7fd0781988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077400700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd078101680 0x7fd0781988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077400700 1 -- 172.21.15.117:0/286698438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680095f0 con 0x7fd078103fb0 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077e00700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd078101680 0x7fd0781988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.143+0000 7fd077400700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078103fb0 0x7fd078198e00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b4b0 tx=0x7fd06400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.144+0000 7fd074c00700 1 -- 172.21.15.117:0/286698438 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd064011800 con 0x7fd078103fb0 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.144+0000 7fd074c00700 1 -- 172.21.15.117:0/286698438 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd064011e20 con 0x7fd078103fb0 2024-04-20T11:11:35.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.144+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07819dfc0 con 0x7fd078103fb0 2024-04-20T11:11:35.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.144+0000 7fd074c00700 1 -- 172.21.15.117:0/286698438 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd06400f2e0 con 0x7fd078103fb0 2024-04-20T11:11:35.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.144+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07819e560 con 0x7fd078103fb0 2024-04-20T11:11:35.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.145+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd078062510 con 0x7fd078103fb0 2024-04-20T11:11:35.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.147+0000 7fd074c00700 1 -- 172.21.15.117:0/286698438 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fd064010380 con 0x7fd078103fb0 2024-04-20T11:11:35.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.149+0000 7fd074c00700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd05c06e940 0x7fd05c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.149+0000 7fd074c00700 1 -- 172.21.15.117:0/286698438 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fd06408e500 con 0x7fd078103fb0 2024-04-20T11:11:35.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.149+0000 7fd077e00700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd05c06e940 0x7fd05c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.150+0000 7fd077e00700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd05c06e940 0x7fd05c070e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd06800b5c0 tx=0x7fd068005160 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:35.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.154+0000 7fd074c00700 1 -- 172.21.15.117:0/286698438 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fd06405b2a0 con 0x7fd078103fb0 2024-04-20T11:11:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:34 smithi173 ceph-mon[23800]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:34 smithi173 ceph-mon[23800]: pgmap v87: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 786 KiB/s wr, 143 op/s 2024-04-20T11:11:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:34 smithi173 ceph-mon[23800]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:35.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.505+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd0781097d0 con 0x7fd078103fb0 2024-04-20T11:11:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.507+0000 7fd074c00700 1 -- 172.21.15.117:0/286698438 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7fd06405ae30 con 0x7fd078103fb0 2024-04-20T11:11:35.508 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:11:35.508 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:11:35.508 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:11:35.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:11:35.508 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:11:35.508 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:11:35.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:11:35.508 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:11:35.509 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:11:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd05c06e940 msgr2=0x7fd05c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd05c06e940 0x7fd05c070e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd06800b5c0 tx=0x7fd068005160 comp rx=0 tx=0).stop 2024-04-20T11:11:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078103fb0 msgr2=0x7fd078198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078103fb0 0x7fd078198e00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b4b0 tx=0x7fd06400b870 comp rx=0 tx=0).stop 2024-04-20T11:11:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 shutdown_connections 2024-04-20T11:11:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fd05c06e940 0x7fd05c070e00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd078101680 0x7fd0781988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 --2- 172.21.15.117:0/286698438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd078103fb0 0x7fd078198e00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 >> 172.21.15.117:0/286698438 conn(0x7fd0780fb0d0 msgr2=0x7fd0780ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.511+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 shutdown_connections 2024-04-20T11:11:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.512+0000 7fd07f200700 1 -- 172.21.15.117:0/286698438 wait complete. 2024-04-20T11:11:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.670+0000 7f0891e00700 1 -- 172.21.15.117:0/306523311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 msgr2=0x7f088c102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.670+0000 7f0891e00700 1 --2- 172.21.15.117:0/306523311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 0x7f088c102130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0878009910 tx=0x7f0878009c20 comp rx=0 tx=0).stop 2024-04-20T11:11:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.670+0000 7f0891e00700 1 -- 172.21.15.117:0/306523311 shutdown_connections 2024-04-20T11:11:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.670+0000 7f0891e00700 1 --2- 172.21.15.117:0/306523311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 0x7f088c102130 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.670+0000 7f0891e00700 1 --2- 172.21.15.117:0/306523311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088c0fecf0 0x7f088c0ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.670+0000 7f0891e00700 1 -- 172.21.15.117:0/306523311 >> 172.21.15.117:0/306523311 conn(0x7f088c0fa650 msgr2=0x7f088c0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:35.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.670+0000 7f0891e00700 1 -- 172.21.15.117:0/306523311 shutdown_connections 2024-04-20T11:11:35.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.671+0000 7f0891e00700 1 -- 172.21.15.117:0/306523311 wait complete. 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f0891e00700 1 Processor -- start 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f0891e00700 1 -- start start 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f0891e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088c0fecf0 0x7f088c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f0891e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 0x7f088c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f0891e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c1992c0 con 0x7f088c101cd0 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f0891e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c199400 con 0x7f088c0fecf0 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f088be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 0x7f088c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f088be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 0x7f088c198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58218/0 (socket says 172.21.15.117:58218) 2024-04-20T11:11:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.672+0000 7f088be00700 1 -- 172.21.15.117:0/593196848 learned_addr learned my addr 172.21.15.117:0/593196848 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:11:35.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.673+0000 7f0890a00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088c0fecf0 0x7f088c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.673+0000 7f088be00700 1 -- 172.21.15.117:0/593196848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088c0fecf0 msgr2=0x7f088c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.673+0000 7f088be00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088c0fecf0 0x7f088c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.673+0000 7f088be00700 1 -- 172.21.15.117:0/593196848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780095f0 con 0x7f088c101cd0 2024-04-20T11:11:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.673+0000 7f088be00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 0x7f088c198ca0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f087800bd70 tx=0x7f087800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.673+0000 7f0889600700 1 -- 172.21.15.117:0/593196848 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087801d070 con 0x7f088c101cd0 2024-04-20T11:11:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.673+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c19de00 con 0x7f088c101cd0 2024-04-20T11:11:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.673+0000 7f0889600700 1 -- 172.21.15.117:0/593196848 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f087800f460 con 0x7f088c101cd0 2024-04-20T11:11:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.674+0000 7f0889600700 1 -- 172.21.15.117:0/593196848 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0878021620 con 0x7f088c101cd0 2024-04-20T11:11:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.674+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c19e340 con 0x7f088c101cd0 2024-04-20T11:11:35.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.675+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c1066a0 con 0x7f088c101cd0 2024-04-20T11:11:35.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.676+0000 7f0889600700 1 -- 172.21.15.117:0/593196848 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f087800fa70 con 0x7f088c101cd0 2024-04-20T11:11:35.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.678+0000 7f0889600700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f087406e940 0x7f0874070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.678+0000 7f0889600700 1 -- 172.21.15.117:0/593196848 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f087808fc10 con 0x7f088c101cd0 2024-04-20T11:11:35.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.678+0000 7f0890a00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f087406e940 0x7f0874070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.679+0000 7f0890a00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f087406e940 0x7f0874070e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0880006fd0 tx=0x7f0880008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:35.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:35.683+0000 7f0889600700 1 -- 172.21.15.117:0/593196848 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f087805ca30 con 0x7f088c101cd0 2024-04-20T11:11:36.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.016+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f088c02ceb0 con 0x7f087406e940 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.017+0000 7f0889600700 1 -- 172.21.15.117:0/593196848 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f088c02ceb0 con 0x7f087406e940 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:11:36.018 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "2/2 daemons upgraded", 2024-04-20T11:11:36.019 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-20T11:11:36.019 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:11:36.019 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:11:36.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.020+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f087406e940 msgr2=0x7f0874070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:36.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.020+0000 7f0891e00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f087406e940 0x7f0874070e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0880006fd0 tx=0x7f0880008040 comp rx=0 tx=0).stop 2024-04-20T11:11:36.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.020+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 msgr2=0x7f088c198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:36.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.020+0000 7f0891e00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 0x7f088c198ca0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f087800bd70 tx=0x7f087800be50 comp rx=0 tx=0).stop 2024-04-20T11:11:36.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.021+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 shutdown_connections 2024-04-20T11:11:36.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.021+0000 7f0891e00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f087406e940 0x7f0874070e00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:36.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.021+0000 7f0891e00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088c0fecf0 0x7f088c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:36.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.021+0000 7f0891e00700 1 --2- 172.21.15.117:0/593196848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f088c101cd0 0x7f088c198ca0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:36.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.021+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 >> 172.21.15.117:0/593196848 conn(0x7f088c0fa650 msgr2=0x7f088c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:36.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.021+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 shutdown_connections 2024-04-20T11:11:36.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:11:36.021+0000 7f0891e00700 1 -- 172.21.15.117:0/593196848 wait complete. 2024-04-20T11:11:36.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:35 smithi117 ceph-mon[15659]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:36.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:35 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/286698438' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:35 smithi173 ceph-mon[23800]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:35 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/286698438' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:36 smithi117 ceph-mon[15659]: pgmap v88: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 455 KiB/s wr, 80 op/s 2024-04-20T11:11:37.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:36 smithi117 ceph-mon[15659]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:36 smithi173 ceph-mon[23800]: pgmap v88: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 455 KiB/s wr, 80 op/s 2024-04-20T11:11:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:36 smithi173 ceph-mon[23800]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:39.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:39 smithi117 ceph-mon[15659]: pgmap v89: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 753 KiB/s wr, 106 op/s 2024-04-20T11:11:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:39 smithi173 ceph-mon[23800]: pgmap v89: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 753 KiB/s wr, 106 op/s 2024-04-20T11:11:41.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:41 smithi117 ceph-mon[15659]: pgmap v90: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 523 KiB/s wr, 57 op/s 2024-04-20T11:11:41.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:41 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:41.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:41 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:41 smithi173 ceph-mon[23800]: pgmap v90: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 523 KiB/s wr, 57 op/s 2024-04-20T11:11:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:41 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:41 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:43 smithi173 ceph-mon[23800]: pgmap v91: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 529 KiB/s wr, 64 op/s 2024-04-20T11:11:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:43 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:43.316 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:43 smithi117 ceph-mon[15659]: pgmap v91: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 529 KiB/s wr, 64 op/s 2024-04-20T11:11:43.316 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:43.316 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:43 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:45.354 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:45 smithi117 ceph-mon[15659]: pgmap v92: 65 pgs: 65 active+clean; 274 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 712 KiB/s wr, 82 op/s 2024-04-20T11:11:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:45 smithi173 ceph-mon[23800]: pgmap v92: 65 pgs: 65 active+clean; 274 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 712 KiB/s wr, 82 op/s 2024-04-20T11:11:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:47 smithi117 ceph-mon[15659]: pgmap v93: 65 pgs: 65 active+clean; 274 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 511 KiB/s wr, 58 op/s 2024-04-20T11:11:47.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:47 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:11:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:47 smithi173 ceph-mon[23800]: pgmap v93: 65 pgs: 65 active+clean; 274 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 511 KiB/s wr, 58 op/s 2024-04-20T11:11:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:47 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:11:49.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:49 smithi117 ceph-mon[15659]: pgmap v94: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 16 KiB/s rd, 695 KiB/s wr, 100 op/s 2024-04-20T11:11:49.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:49 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:49.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:49 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:49.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:49 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:11:49.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:49 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:11:49.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:49 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:49 smithi173 ceph-mon[23800]: pgmap v94: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 16 KiB/s rd, 695 KiB/s wr, 100 op/s 2024-04-20T11:11:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:49 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:49 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:49 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:11:49.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:49 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:11:49.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:49 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:11:50.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-20T11:11:50.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-20T11:11:50.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:50.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.361 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:50 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:50.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:50 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:11:51.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:51 smithi117 ceph-mon[15659]: pgmap v95: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 10 KiB/s rd, 397 KiB/s wr, 73 op/s 2024-04-20T11:11:51.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:51 smithi173 ceph-mon[23800]: pgmap v95: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 10 KiB/s rd, 397 KiB/s wr, 73 op/s 2024-04-20T11:11:53.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:53 smithi117 ceph-mon[15659]: pgmap v96: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 592 KiB/s wr, 99 op/s 2024-04-20T11:11:53.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:53 smithi117 ceph-mon[15659]: Upgrade: Updating grafana.smithi117 2024-04-20T11:11:53.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:53 smithi117 ceph-mon[15659]: Deploying daemon grafana.smithi117 on smithi117 2024-04-20T11:11:53.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:53 smithi173 ceph-mon[23800]: pgmap v96: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 592 KiB/s wr, 99 op/s 2024-04-20T11:11:53.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:53 smithi173 ceph-mon[23800]: Upgrade: Updating grafana.smithi117 2024-04-20T11:11:53.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:53 smithi173 ceph-mon[23800]: Deploying daemon grafana.smithi117 on smithi117 2024-04-20T11:11:55.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:55 smithi117 ceph-mon[15659]: pgmap v97: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8.8 KiB/s rd, 587 KiB/s wr, 103 op/s 2024-04-20T11:11:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:55 smithi173 ceph-mon[23800]: pgmap v97: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8.8 KiB/s rd, 587 KiB/s wr, 103 op/s 2024-04-20T11:11:57.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:57 smithi117 ceph-mon[15659]: pgmap v98: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8.8 KiB/s rd, 380 KiB/s wr, 78 op/s 2024-04-20T11:11:57.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:57 smithi173 ceph-mon[23800]: pgmap v98: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8.8 KiB/s rd, 380 KiB/s wr, 78 op/s 2024-04-20T11:11:59.099 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:11:59 smithi117 ceph-mon[15659]: pgmap v99: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 557 KiB/s wr, 114 op/s 2024-04-20T11:11:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:11:59 smithi173 ceph-mon[23800]: pgmap v99: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 557 KiB/s wr, 114 op/s 2024-04-20T11:12:01.103 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-20T11:12:01.103 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-20T11:12:01.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:01 smithi117 ceph-mon[15659]: pgmap v100: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8.7 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-04-20T11:12:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:01 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:12:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:01 smithi173 ceph-mon[23800]: pgmap v100: 65 pgs: 65 active+clean; 266 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8.7 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-04-20T11:12:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:01 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:12:02.156 DEBUG:teuthology.parallel:result is None 2024-04-20T11:12:02.157 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T11:12:02.259 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T11:12:02.259 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-20T11:12:02.341 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-20T11:12:02.341 DEBUG:teuthology.parallel:result is None 2024-04-20T11:12:03.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:03 smithi117 ceph-mon[15659]: pgmap v101: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 378 KiB/s wr, 82 op/s 2024-04-20T11:12:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:03 smithi173 ceph-mon[23800]: pgmap v101: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 378 KiB/s wr, 82 op/s 2024-04-20T11:12:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:05 smithi117 ceph-mon[15659]: pgmap v102: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9.9 KiB/s rd, 370 KiB/s wr, 76 op/s 2024-04-20T11:12:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:05 smithi173 ceph-mon[23800]: pgmap v102: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9.9 KiB/s rd, 370 KiB/s wr, 76 op/s 2024-04-20T11:12:06.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.194+0000 7f092fe00700 1 -- 172.21.15.117:0/1352903833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 msgr2=0x7f09301030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:06.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.194+0000 7f092fe00700 1 --2- 172.21.15.117:0/1352903833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 0x7f09301030c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0920009910 tx=0x7f0920009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:06.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.194+0000 7f092fe00700 1 -- 172.21.15.117:0/1352903833 shutdown_connections 2024-04-20T11:12:06.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.194+0000 7f092fe00700 1 --2- 172.21.15.117:0/1352903833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0930103ea0 0x7f0930104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.194+0000 7f092fe00700 1 --2- 172.21.15.117:0/1352903833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 0x7f09301030c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.194+0000 7f092fe00700 1 -- 172.21.15.117:0/1352903833 >> 172.21.15.117:0/1352903833 conn(0x7f09300fe240 msgr2=0x7f0930100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:06.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.194+0000 7f092fe00700 1 -- 172.21.15.117:0/1352903833 shutdown_connections 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.195+0000 7f092fe00700 1 -- 172.21.15.117:0/1352903833 wait complete. 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.196+0000 7f092fe00700 1 Processor -- start 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.196+0000 7f092fe00700 1 -- start start 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.196+0000 7f092fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 0x7f0930198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.196+0000 7f092fe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0930103ea0 0x7f0930198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.196+0000 7f092fe00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09301991f0 con 0x7f0930102ca0 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.196+0000 7f092fe00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0930199330 con 0x7f0930103ea0 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.196+0000 7f092ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 0x7f0930198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.196+0000 7f092ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 0x7f0930198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45428/0 (socket says 172.21.15.117:45428) 2024-04-20T11:12:06.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.197+0000 7f092ea00700 1 -- 172.21.15.117:0/2837259448 learned_addr learned my addr 172.21.15.117:0/2837259448 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:06.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.197+0000 7f092ea00700 1 -- 172.21.15.117:0/2837259448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0930103ea0 msgr2=0x7f0930198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:06.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.197+0000 7f092e000700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0930103ea0 0x7f0930198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:06.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.197+0000 7f092ea00700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0930103ea0 0x7f0930198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.197+0000 7f092ea00700 1 -- 172.21.15.117:0/2837259448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200095f0 con 0x7f0930102ca0 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.197+0000 7f092ea00700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 0x7f0930198690 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f092000bd70 tx=0x7f092000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.198+0000 7f0927400700 1 -- 172.21.15.117:0/2837259448 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f092001d070 con 0x7f0930102ca0 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.198+0000 7f0927400700 1 -- 172.21.15.117:0/2837259448 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f092000f460 con 0x7f0930102ca0 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.198+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093019dd30 con 0x7f0930102ca0 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.198+0000 7f0927400700 1 -- 172.21.15.117:0/2837259448 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0920021620 con 0x7f0930102ca0 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.198+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093019e240 con 0x7f0930102ca0 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.199+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0930192820 con 0x7f0930102ca0 2024-04-20T11:12:06.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.200+0000 7f0927400700 1 -- 172.21.15.117:0/2837259448 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f092000fa70 con 0x7f0930102ca0 2024-04-20T11:12:06.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.201+0000 7f0927400700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f091406e950 0x7f0914070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:06.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.201+0000 7f0927400700 1 -- 172.21.15.117:0/2837259448 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f092008fb50 con 0x7f0930102ca0 2024-04-20T11:12:06.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.201+0000 7f092e000700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f091406e950 0x7f0914070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:06.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.202+0000 7f092e000700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f091406e950 0x7f0914070e10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f091c005950 tx=0x7f091c005cf0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:06.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.206+0000 7f0927400700 1 -- 172.21.15.117:0/2837259448 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f092005c970 con 0x7f0930102ca0 2024-04-20T11:12:06.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.457+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f093002ced0 con 0x7f091406e950 2024-04-20T11:12:06.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.458+0000 7f0927400700 1 -- 172.21.15.117:0/2837259448 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f093002ced0 con 0x7f091406e950 2024-04-20T11:12:06.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f091406e950 msgr2=0x7f0914070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:06.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f091406e950 0x7f0914070e10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f091c005950 tx=0x7f091c005cf0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 msgr2=0x7f0930198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:06.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 0x7f0930198690 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f092000bd70 tx=0x7f092000be50 comp rx=0 tx=0).stop 2024-04-20T11:12:06.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 shutdown_connections 2024-04-20T11:12:06.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f091406e950 0x7f0914070e10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0930102ca0 0x7f0930198690 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 --2- 172.21.15.117:0/2837259448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0930103ea0 0x7f0930198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.461+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 >> 172.21.15.117:0/2837259448 conn(0x7f09300fe240 msgr2=0x7f09301074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:06.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.462+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 shutdown_connections 2024-04-20T11:12:06.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.462+0000 7f092fe00700 1 -- 172.21.15.117:0/2837259448 wait complete. 2024-04-20T11:12:06.477 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:12:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.619+0000 7fc5c3000700 1 -- 172.21.15.117:0/3272741523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc107260 msgr2=0x7fc5bc107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.619+0000 7fc5c3000700 1 --2- 172.21.15.117:0/3272741523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc107260 0x7fc5bc107680 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009910 tx=0x7fc5ac009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.619+0000 7fc5c3000700 1 -- 172.21.15.117:0/3272741523 shutdown_connections 2024-04-20T11:12:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.619+0000 7fc5c3000700 1 --2- 172.21.15.117:0/3272741523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5bc108430 0x7fc5bc1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.619+0000 7fc5c3000700 1 --2- 172.21.15.117:0/3272741523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc107260 0x7fc5bc107680 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.619+0000 7fc5c3000700 1 -- 172.21.15.117:0/3272741523 >> 172.21.15.117:0/3272741523 conn(0x7fc5bc076400 msgr2=0x7fc5bc078860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.619+0000 7fc5c3000700 1 -- 172.21.15.117:0/3272741523 shutdown_connections 2024-04-20T11:12:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.619+0000 7fc5c3000700 1 -- 172.21.15.117:0/3272741523 wait complete. 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.620+0000 7fc5c3000700 1 Processor -- start 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.620+0000 7fc5c3000700 1 -- start start 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.620+0000 7fc5c3000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5bc107260 0x7fc5bc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.620+0000 7fc5c3000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc108430 0x7fc5bc19d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.620+0000 7fc5c3000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc19d890 con 0x7fc5bc108430 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.620+0000 7fc5c3000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc19da00 con 0x7fc5bc107260 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.621+0000 7fc5bb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc108430 0x7fc5bc19d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.621+0000 7fc5bb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc108430 0x7fc5bc19d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45454/0 (socket says 172.21.15.117:45454) 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.621+0000 7fc5bb400700 1 -- 172.21.15.117:0/1365413494 learned_addr learned my addr 172.21.15.117:0/1365413494 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.621+0000 7fc5bbe00700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5bc107260 0x7fc5bc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.621+0000 7fc5bb400700 1 -- 172.21.15.117:0/1365413494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5bc107260 msgr2=0x7fc5bc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.621+0000 7fc5bb400700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5bc107260 0x7fc5bc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.621+0000 7fc5bb400700 1 -- 172.21.15.117:0/1365413494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0095f0 con 0x7fc5bc108430 2024-04-20T11:12:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.622+0000 7fc5bb400700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc108430 0x7fc5bc19d290 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800b4b0 tx=0x7fc5a800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.622+0000 7fc5b8c00700 1 -- 172.21.15.117:0/1365413494 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a8011800 con 0x7fc5bc108430 2024-04-20T11:12:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.622+0000 7fc5b8c00700 1 -- 172.21.15.117:0/1365413494 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5a8011e20 con 0x7fc5bc108430 2024-04-20T11:12:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.622+0000 7fc5b8c00700 1 -- 172.21.15.117:0/1365413494 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a800f2e0 con 0x7fc5bc108430 2024-04-20T11:12:06.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.622+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc1a23f0 con 0x7fc5bc108430 2024-04-20T11:12:06.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.622+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc1a29c0 con 0x7fc5bc108430 2024-04-20T11:12:06.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.623+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc196e50 con 0x7fc5bc108430 2024-04-20T11:12:06.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.624+0000 7fc5b8c00700 1 -- 172.21.15.117:0/1365413494 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fc5a8010380 con 0x7fc5bc108430 2024-04-20T11:12:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.625+0000 7fc5b8c00700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc5a006e950 0x7fc5a0070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.625+0000 7fc5b8c00700 1 -- 172.21.15.117:0/1365413494 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fc5a808e4a0 con 0x7fc5bc108430 2024-04-20T11:12:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.625+0000 7fc5bbe00700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc5a006e950 0x7fc5a0070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.626+0000 7fc5bbe00700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc5a006e950 0x7fc5a0070e10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac006010 tx=0x7fc5ac00b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.630+0000 7fc5b8c00700 1 -- 172.21.15.117:0/1365413494 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fc5a8057710 con 0x7fc5bc108430 2024-04-20T11:12:06.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.879+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5bc02ce50 con 0x7fc5a006e950 2024-04-20T11:12:06.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.881+0000 7fc5b8c00700 1 -- 172.21.15.117:0/1365413494 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fc5bc02ce50 con 0x7fc5a006e950 2024-04-20T11:12:06.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc5a006e950 msgr2=0x7fc5a0070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:06.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc5a006e950 0x7fc5a0070e10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac006010 tx=0x7fc5ac00b540 comp rx=0 tx=0).stop 2024-04-20T11:12:06.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc108430 msgr2=0x7fc5bc19d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:06.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc108430 0x7fc5bc19d290 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800b4b0 tx=0x7fc5a800b870 comp rx=0 tx=0).stop 2024-04-20T11:12:06.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 shutdown_connections 2024-04-20T11:12:06.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc5a006e950 0x7fc5a0070e10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5bc107260 0x7fc5bc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 --2- 172.21.15.117:0/1365413494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc108430 0x7fc5bc19d290 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:06.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.884+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 >> 172.21.15.117:0/1365413494 conn(0x7fc5bc076400 msgr2=0x7fc5bc10ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:06.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.885+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 shutdown_connections 2024-04-20T11:12:06.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:06.885+0000 7fc5c3000700 1 -- 172.21.15.117:0/1365413494 wait complete. 2024-04-20T11:12:07.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.041+0000 7f05c0600700 1 -- 172.21.15.117:0/2986523768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8103fb0 msgr2=0x7f05b8104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.041+0000 7f05c0600700 1 --2- 172.21.15.117:0/2986523768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8103fb0 0x7f05b8104430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009910 tx=0x7f05a4009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:07.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.041+0000 7f05c0600700 1 -- 172.21.15.117:0/2986523768 shutdown_connections 2024-04-20T11:12:07.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.041+0000 7f05c0600700 1 --2- 172.21.15.117:0/2986523768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8103fb0 0x7f05b8104430 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.041+0000 7f05c0600700 1 --2- 172.21.15.117:0/2986523768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8102e50 0x7f05b8103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.041+0000 7f05c0600700 1 -- 172.21.15.117:0/2986523768 >> 172.21.15.117:0/2986523768 conn(0x7f05b80fe3d0 msgr2=0x7f05b8100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:07.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.041+0000 7f05c0600700 1 -- 172.21.15.117:0/2986523768 shutdown_connections 2024-04-20T11:12:07.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.041+0000 7f05c0600700 1 -- 172.21.15.117:0/2986523768 wait complete. 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.042+0000 7f05c0600700 1 Processor -- start 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.042+0000 7f05c0600700 1 -- start start 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05c0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8102e50 0x7f05b81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05c0600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8103fb0 0x7f05b8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05c0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8199340 con 0x7f05b8102e50 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05c0600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8199480 con 0x7f05b8103fb0 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05bdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8102e50 0x7f05b81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05bdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8102e50 0x7f05b81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45482/0 (socket says 172.21.15.117:45482) 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05bdc00700 1 -- 172.21.15.117:0/1446389108 learned_addr learned my addr 172.21.15.117:0/1446389108 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05bdc00700 1 -- 172.21.15.117:0/1446389108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8103fb0 msgr2=0x7f05b8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05bdc00700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8103fb0 0x7f05b8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.043+0000 7f05bdc00700 1 -- 172.21.15.117:0/1446389108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40095f0 con 0x7f05b8102e50 2024-04-20T11:12:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.044+0000 7f05bdc00700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8102e50 0x7f05b81987e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00d8d0 tx=0x7f05ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.044+0000 7f05b6000700 1 -- 172.21.15.117:0/1446389108 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05ac009940 con 0x7f05b8102e50 2024-04-20T11:12:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.044+0000 7f05b6000700 1 -- 172.21.15.117:0/1446389108 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05ac010460 con 0x7f05b8102e50 2024-04-20T11:12:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.044+0000 7f05b6000700 1 -- 172.21.15.117:0/1446389108 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05ac00f5f0 con 0x7f05b8102e50 2024-04-20T11:12:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.044+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b819dee0 con 0x7f05b8102e50 2024-04-20T11:12:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.044+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b819e400 con 0x7f05b8102e50 2024-04-20T11:12:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.045+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b8192950 con 0x7f05b8102e50 2024-04-20T11:12:07.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.046+0000 7f05b6000700 1 -- 172.21.15.117:0/1446389108 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f05ac0105d0 con 0x7f05b8102e50 2024-04-20T11:12:07.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.048+0000 7f05b6000700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f05a006e950 0x7f05a0070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.048+0000 7f05b6000700 1 -- 172.21.15.117:0/1446389108 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f05ac08e910 con 0x7f05b8102e50 2024-04-20T11:12:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.048+0000 7f05bd200700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f05a006e950 0x7f05a0070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.049+0000 7f05bd200700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f05a006e950 0x7f05a0070e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05a40051c0 tx=0x7f05a4005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:07.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.053+0000 7f05b6000700 1 -- 172.21.15.117:0/1446389108 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f05ac05b6b0 con 0x7f05b8102e50 2024-04-20T11:12:07.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:07 smithi117 ceph-mon[15659]: pgmap v103: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9.9 KiB/s rd, 369 KiB/s wr, 64 op/s 2024-04-20T11:12:07.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:07 smithi117 ceph-mon[15659]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:07.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.284+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f05b802ced0 con 0x7f05a006e950 2024-04-20T11:12:07.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.298+0000 7f05b6000700 1 -- 172.21.15.117:0/1446389108 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f05b802ced0 con 0x7f05a006e950 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (35s) 26s ago 14m 13.5M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (14m) 26s ago 14m 8220k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (13m) 98s ago 13m 7880k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (14m) 26s ago 14m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (13m) 98s ago 13m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (12m) 26s ago 14m 76.4M - 9.4.7 954c08fa6188 6a24f2275b82 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (7m) 26s ago 7m 15.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:12:07.299 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (7m) 26s ago 7m 178M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (7m) 98s ago 7m 14.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (7m) 98s ago 7m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (3m) 26s ago 16m 504M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (2m) 98s ago 13m 447M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (16m) 26s ago 16m 58.4M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (13m) 98s ago 13m 44.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (112s) 26s ago 14m 7616k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (103s) 98s ago 13m 2512k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (12m) 26s ago 12m 206M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (11m) 26s ago 11m 208M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (11m) 26s ago 11m 178M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:12:07.300 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (10m) 98s ago 10m 205M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:12:07.301 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (9m) 98s ago 9m 182M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:12:07.301 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (9m) 98s ago 9m 171M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:12:07.301 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (72s) 26s ago 14m 55.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:12:07.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f05a006e950 msgr2=0x7f05a0070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f05a006e950 0x7f05a0070e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05a40051c0 tx=0x7f05a4005fd0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8102e50 msgr2=0x7f05b81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8102e50 0x7f05b81987e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00d8d0 tx=0x7f05ac00dc90 comp rx=0 tx=0).stop 2024-04-20T11:12:07.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 shutdown_connections 2024-04-20T11:12:07.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f05a006e950 0x7f05a0070e10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05b8102e50 0x7f05b81987e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 --2- 172.21.15.117:0/1446389108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8103fb0 0x7f05b8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.302+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 >> 172.21.15.117:0/1446389108 conn(0x7f05b80fe3d0 msgr2=0x7f05b8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:07.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.303+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 shutdown_connections 2024-04-20T11:12:07.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.303+0000 7f05c0600700 1 -- 172.21.15.117:0/1446389108 wait complete. 2024-04-20T11:12:07.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:07 smithi173 ceph-mon[23800]: pgmap v103: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9.9 KiB/s rd, 369 KiB/s wr, 64 op/s 2024-04-20T11:12:07.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:07 smithi173 ceph-mon[23800]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.456+0000 7f471fa00700 1 -- 172.21.15.117:0/1755983503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718103fb0 msgr2=0x7f47181063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.456+0000 7f471fa00700 1 --2- 172.21.15.117:0/1755983503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718103fb0 0x7f47181063a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4704009910 tx=0x7f4704009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.457+0000 7f471fa00700 1 -- 172.21.15.117:0/1755983503 shutdown_connections 2024-04-20T11:12:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.457+0000 7f471fa00700 1 --2- 172.21.15.117:0/1755983503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718103fb0 0x7f47181063a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.457+0000 7f471fa00700 1 --2- 172.21.15.117:0/1755983503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4718101680 0x7f4718103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.457+0000 7f471fa00700 1 -- 172.21.15.117:0/1755983503 >> 172.21.15.117:0/1755983503 conn(0x7f47180fb0d0 msgr2=0x7f47180fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.457+0000 7f471fa00700 1 -- 172.21.15.117:0/1755983503 shutdown_connections 2024-04-20T11:12:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.457+0000 7f471fa00700 1 -- 172.21.15.117:0/1755983503 wait complete. 2024-04-20T11:12:07.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.458+0000 7f471fa00700 1 Processor -- start 2024-04-20T11:12:07.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.458+0000 7f471fa00700 1 -- start start 2024-04-20T11:12:07.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471fa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718101680 0x7f47181988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471fa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4718103fb0 0x7f4718198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471fa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4718199430 con 0x7f4718101680 2024-04-20T11:12:07.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471fa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4718199570 con 0x7f4718103fb0 2024-04-20T11:12:07.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718101680 0x7f47181988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:07.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718101680 0x7f47181988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45490/0 (socket says 172.21.15.117:45490) 2024-04-20T11:12:07.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471d000700 1 -- 172.21.15.117:0/1370202240 learned_addr learned my addr 172.21.15.117:0/1370202240 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:07.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f4717e00700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4718103fb0 0x7f4718198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:07.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471d000700 1 -- 172.21.15.117:0/1370202240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4718103fb0 msgr2=0x7f4718198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471d000700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4718103fb0 0x7f4718198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.459+0000 7f471d000700 1 -- 172.21.15.117:0/1370202240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040095f0 con 0x7f4718101680 2024-04-20T11:12:07.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.460+0000 7f471d000700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718101680 0x7f47181988d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f470c00b4b0 tx=0x7f470c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:07.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.460+0000 7f4715600700 1 -- 172.21.15.117:0/1370202240 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f470c011800 con 0x7f4718101680 2024-04-20T11:12:07.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.460+0000 7f4715600700 1 -- 172.21.15.117:0/1370202240 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f470c011e20 con 0x7f4718101680 2024-04-20T11:12:07.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.460+0000 7f4715600700 1 -- 172.21.15.117:0/1370202240 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f470c00f2e0 con 0x7f4718101680 2024-04-20T11:12:07.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.460+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471819dfd0 con 0x7f4718101680 2024-04-20T11:12:07.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.460+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471819e510 con 0x7f4718101680 2024-04-20T11:12:07.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.462+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4718062510 con 0x7f4718101680 2024-04-20T11:12:07.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.463+0000 7f4715600700 1 -- 172.21.15.117:0/1370202240 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f470c010380 con 0x7f4718101680 2024-04-20T11:12:07.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.463+0000 7f4715600700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f470006e940 0x7f4700070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.463+0000 7f4715600700 1 -- 172.21.15.117:0/1370202240 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f470c08e350 con 0x7f4718101680 2024-04-20T11:12:07.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.464+0000 7f4717e00700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f470006e940 0x7f4700070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:07.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.464+0000 7f4717e00700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f470006e940 0x7f4700070e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4704000c00 tx=0x7f4704005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.468+0000 7f4715600700 1 -- 172.21.15.117:0/1370202240 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f470c05b210 con 0x7f4718101680 2024-04-20T11:12:07.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.773+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f47181097d0 con 0x7f4718101680 2024-04-20T11:12:07.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.774+0000 7f4715600700 1 -- 172.21.15.117:0/1370202240 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f470c05ada0 con 0x7f4718101680 2024-04-20T11:12:07.773 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:12:07.774 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:12:07.775 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:12:07.775 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:12:07.775 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:12:07.775 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:12:07.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.776+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f470006e940 msgr2=0x7f4700070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.776+0000 7f471fa00700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f470006e940 0x7f4700070e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4704000c00 tx=0x7f4704005fb0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.776+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718101680 msgr2=0x7f47181988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.776+0000 7f471fa00700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718101680 0x7f47181988d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f470c00b4b0 tx=0x7f470c00b870 comp rx=0 tx=0).stop 2024-04-20T11:12:07.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.777+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 shutdown_connections 2024-04-20T11:12:07.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.777+0000 7f471fa00700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f470006e940 0x7f4700070e00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.777+0000 7f471fa00700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718101680 0x7f47181988d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.777+0000 7f471fa00700 1 --2- 172.21.15.117:0/1370202240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4718103fb0 0x7f4718198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.777+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 >> 172.21.15.117:0/1370202240 conn(0x7f47180fb0d0 msgr2=0x7f47181004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:07.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.777+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 shutdown_connections 2024-04-20T11:12:07.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.777+0000 7f471fa00700 1 -- 172.21.15.117:0/1370202240 wait complete. 2024-04-20T11:12:07.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.933+0000 7f3c44c00700 1 -- 172.21.15.117:0/4124717748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40100c20 msgr2=0x7f3c40101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.933+0000 7f3c44c00700 1 --2- 172.21.15.117:0/4124717748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40100c20 0x7f3c40101040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009910 tx=0x7f3c30009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:07.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.933+0000 7f3c44c00700 1 -- 172.21.15.117:0/4124717748 shutdown_connections 2024-04-20T11:12:07.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.933+0000 7f3c44c00700 1 --2- 172.21.15.117:0/4124717748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c40101e20 0x7f3c40102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.933+0000 7f3c44c00700 1 --2- 172.21.15.117:0/4124717748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40100c20 0x7f3c40101040 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.933+0000 7f3c44c00700 1 -- 172.21.15.117:0/4124717748 >> 172.21.15.117:0/4124717748 conn(0x7f3c400fc1c0 msgr2=0x7f3c400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:07.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.934+0000 7f3c44c00700 1 -- 172.21.15.117:0/4124717748 shutdown_connections 2024-04-20T11:12:07.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.934+0000 7f3c44c00700 1 -- 172.21.15.117:0/4124717748 wait complete. 2024-04-20T11:12:07.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.935+0000 7f3c44c00700 1 Processor -- start 2024-04-20T11:12:07.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.935+0000 7f3c44c00700 1 -- start start 2024-04-20T11:12:07.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.935+0000 7f3c44c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c40100c20 0x7f3c40196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.935+0000 7f3c44c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40101e20 0x7f3c40196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.935+0000 7f3c44c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c401972d0 con 0x7f3c40101e20 2024-04-20T11:12:07.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.935+0000 7f3c44c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40197410 con 0x7f3c40100c20 2024-04-20T11:12:07.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.936+0000 7f3c3d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40101e20 0x7f3c40196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:07.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.936+0000 7f3c3d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40101e20 0x7f3c40196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45512/0 (socket says 172.21.15.117:45512) 2024-04-20T11:12:07.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.936+0000 7f3c3d400700 1 -- 172.21.15.117:0/3975908596 learned_addr learned my addr 172.21.15.117:0/3975908596 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:07.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.936+0000 7f3c3de00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c40100c20 0x7f3c40196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:07.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.936+0000 7f3c3d400700 1 -- 172.21.15.117:0/3975908596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c40100c20 msgr2=0x7f3c40196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:07.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.936+0000 7f3c3d400700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c40100c20 0x7f3c40196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:07.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.936+0000 7f3c3d400700 1 -- 172.21.15.117:0/3975908596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300095f0 con 0x7f3c40101e20 2024-04-20T11:12:07.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.937+0000 7f3c3d400700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40101e20 0x7f3c40196cb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00b4b0 tx=0x7f3c2c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:07.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.937+0000 7f3c36a00700 1 -- 172.21.15.117:0/3975908596 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c2c011800 con 0x7f3c40101e20 2024-04-20T11:12:07.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.937+0000 7f3c36a00700 1 -- 172.21.15.117:0/3975908596 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c2c011e20 con 0x7f3c40101e20 2024-04-20T11:12:07.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.937+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4019be20 con 0x7f3c40101e20 2024-04-20T11:12:07.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.937+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4019c3c0 con 0x7f3c40101e20 2024-04-20T11:12:07.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.937+0000 7f3c36a00700 1 -- 172.21.15.117:0/3975908596 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c2c00f2e0 con 0x7f3c40101e20 2024-04-20T11:12:07.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.938+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c20005320 con 0x7f3c40101e20 2024-04-20T11:12:07.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.940+0000 7f3c36a00700 1 -- 172.21.15.117:0/3975908596 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f3c2c011960 con 0x7f3c40101e20 2024-04-20T11:12:07.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.941+0000 7f3c36a00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f3c2406e940 0x7f3c24070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:07.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.941+0000 7f3c36a00700 1 -- 172.21.15.117:0/3975908596 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f3c2c08e320 con 0x7f3c40101e20 2024-04-20T11:12:07.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.941+0000 7f3c3de00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f3c2406e940 0x7f3c24070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:07.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.942+0000 7f3c3de00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f3c2406e940 0x7f3c24070e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c30006010 tx=0x7f3c3000b580 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:07.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:07.947+0000 7f3c36a00700 1 -- 172.21.15.117:0/3975908596 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f3c2c05b160 con 0x7f3c40101e20 2024-04-20T11:12:08.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.199+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c20000bf0 con 0x7f3c2406e940 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.202+0000 7f3c36a00700 1 -- 172.21.15.117:0/3975908596 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f3c20000bf0 con 0x7f3c2406e940 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "2/2 daemons upgraded", 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading grafana daemons", 2024-04-20T11:12:08.202 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:12:08.203 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:12:08.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f3c2406e940 msgr2=0x7f3c24070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:08.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f3c2406e940 0x7f3c24070e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c30006010 tx=0x7f3c3000b580 comp rx=0 tx=0).stop 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40101e20 msgr2=0x7f3c40196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40101e20 0x7f3c40196cb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00b4b0 tx=0x7f3c2c00b870 comp rx=0 tx=0).stop 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 shutdown_connections 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f3c2406e940 0x7f3c24070e00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c40100c20 0x7f3c40196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 --2- 172.21.15.117:0/3975908596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40101e20 0x7f3c40196cb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 >> 172.21.15.117:0/3975908596 conn(0x7f3c400fc1c0 msgr2=0x7f3c40105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 shutdown_connections 2024-04-20T11:12:08.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:08.204+0000 7f3c44c00700 1 -- 172.21.15.117:0/3975908596 wait complete. 2024-04-20T11:12:08.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:08 smithi117 ceph-mon[15659]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:08.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:08 smithi117 ceph-mon[15659]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:08.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:08 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1370202240' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:08.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:08 smithi173 ceph-mon[23800]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:08.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:08 smithi173 ceph-mon[23800]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:08.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:08 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1370202240' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:09.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:09 smithi173 ceph-mon[23800]: pgmap v104: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 388 KiB/s wr, 73 op/s 2024-04-20T11:12:09.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:09 smithi173 ceph-mon[23800]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:09.450 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:09 smithi117 ceph-mon[15659]: pgmap v104: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 388 KiB/s wr, 73 op/s 2024-04-20T11:12:09.451 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:09 smithi117 ceph-mon[15659]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:11 smithi173 ceph-mon[23800]: pgmap v105: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 7.2 KiB/s rd, 210 KiB/s wr, 36 op/s 2024-04-20T11:12:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:11 smithi117 ceph-mon[15659]: pgmap v105: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 7.2 KiB/s rd, 210 KiB/s wr, 36 op/s 2024-04-20T11:12:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:13 smithi117 ceph-mon[15659]: pgmap v106: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 7.2 KiB/s rd, 211 KiB/s wr, 37 op/s 2024-04-20T11:12:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:13 smithi173 ceph-mon[23800]: pgmap v106: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 7.2 KiB/s rd, 211 KiB/s wr, 37 op/s 2024-04-20T11:12:15.221 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:15 smithi117 ceph-mon[15659]: pgmap v107: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 206 KiB/s wr, 28 op/s 2024-04-20T11:12:15.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:15 smithi173 ceph-mon[23800]: pgmap v107: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 206 KiB/s wr, 28 op/s 2024-04-20T11:12:16.541 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:16 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:16.541 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:16 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:16.541 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:16 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:16.541 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:16 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:12:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:16 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:16 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:16 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:16 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:12:17.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:17 smithi117 ceph-mon[15659]: pgmap v108: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-04-20T11:12:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:17 smithi173 ceph-mon[23800]: pgmap v108: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-04-20T11:12:19.530 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:19 smithi117 ceph-mon[15659]: pgmap v109: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-04-20T11:12:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:19 smithi173 ceph-mon[23800]: pgmap v109: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 19 KiB/s wr, 8 op/s 2024-04-20T11:12:21.549 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:21 smithi117 ceph-mon[15659]: pgmap v110: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-20T11:12:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:21 smithi173 ceph-mon[23800]: pgmap v110: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-20T11:12:23.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:23 smithi117 ceph-mon[15659]: pgmap v111: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-20T11:12:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:23 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:23 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:23 smithi173 ceph-mon[23800]: pgmap v111: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-20T11:12:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:23 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:23 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:25 smithi117 ceph-mon[15659]: pgmap v112: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-20T11:12:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:25 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:25 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:25 smithi173 ceph-mon[23800]: pgmap v112: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-20T11:12:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:25 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:25 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:27.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:27 smithi117 ceph-mon[15659]: pgmap v113: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:27.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:27 smithi173 ceph-mon[23800]: pgmap v113: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:29.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:29 smithi117 ceph-mon[15659]: pgmap v114: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:29 smithi173 ceph-mon[23800]: pgmap v114: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: pgmap v115: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-20T11:12:31.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: Upgrade: Finalizing container_image settings 2024-04-20T11:12:31.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-20T11:12:31.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-20T11:12:31.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-20T11:12:31.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-20T11:12:31.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-20T11:12:31.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: Upgrade: Complete! 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:31.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:31.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:31.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:12:31.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:31 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: pgmap v115: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: Upgrade: Finalizing container_image settings 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-20T11:12:31.664 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-20T11:12:31.665 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.665 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-20T11:12:31.665 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-20T11:12:31.665 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-20T11:12:31.665 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-20T11:12:31.665 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-20T11:12:31.665 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.666 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: Upgrade: Complete! 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:12:31.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:31 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:12:33.510 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:33 smithi117 ceph-mon[15659]: pgmap v116: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:33 smithi173 ceph-mon[23800]: pgmap v116: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:35.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:35 smithi117 ceph-mon[15659]: pgmap v117: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:35 smithi173 ceph-mon[23800]: pgmap v117: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:37.536 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:37 smithi117 ceph-mon[15659]: pgmap v118: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:37 smithi173 ceph-mon[23800]: pgmap v118: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:38.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.374+0000 7fc885600700 1 -- 172.21.15.117:0/3152147979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 msgr2=0x7fc880103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:38.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.374+0000 7fc885600700 1 --2- 172.21.15.117:0/3152147979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 0x7fc880103270 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc870009910 tx=0x7fc870009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:38.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.374+0000 7fc885600700 1 -- 172.21.15.117:0/3152147979 shutdown_connections 2024-04-20T11:12:38.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.374+0000 7fc885600700 1 --2- 172.21.15.117:0/3152147979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc880103fb0 0x7fc880104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:38.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.374+0000 7fc885600700 1 --2- 172.21.15.117:0/3152147979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 0x7fc880103270 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:38.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.374+0000 7fc885600700 1 -- 172.21.15.117:0/3152147979 >> 172.21.15.117:0/3152147979 conn(0x7fc8800fe3d0 msgr2=0x7fc880100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:38.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.374+0000 7fc885600700 1 -- 172.21.15.117:0/3152147979 shutdown_connections 2024-04-20T11:12:38.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.374+0000 7fc885600700 1 -- 172.21.15.117:0/3152147979 wait complete. 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.375+0000 7fc885600700 1 Processor -- start 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.375+0000 7fc885600700 1 -- start start 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.375+0000 7fc885600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 0x7fc8801987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.375+0000 7fc885600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc880103fb0 0x7fc880198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.375+0000 7fc885600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880199340 con 0x7fc880102e50 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.375+0000 7fc885600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880199480 con 0x7fc880103fb0 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.376+0000 7fc87ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 0x7fc8801987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.376+0000 7fc87ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 0x7fc8801987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42866/0 (socket says 172.21.15.117:42866) 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.376+0000 7fc87ea00700 1 -- 172.21.15.117:0/1780766113 learned_addr learned my addr 172.21.15.117:0/1780766113 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.376+0000 7fc87e000700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc880103fb0 0x7fc880198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.376+0000 7fc87ea00700 1 -- 172.21.15.117:0/1780766113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc880103fb0 msgr2=0x7fc880198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.376+0000 7fc87ea00700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc880103fb0 0x7fc880198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:38.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.376+0000 7fc87ea00700 1 -- 172.21.15.117:0/1780766113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8700095f0 con 0x7fc880102e50 2024-04-20T11:12:38.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.376+0000 7fc87ea00700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 0x7fc8801987e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc87000bd70 tx=0x7fc87000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:38.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.377+0000 7fc877400700 1 -- 172.21.15.117:0/1780766113 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc87001d070 con 0x7fc880102e50 2024-04-20T11:12:38.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.377+0000 7fc877400700 1 -- 172.21.15.117:0/1780766113 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc870005dc0 con 0x7fc880102e50 2024-04-20T11:12:38.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.377+0000 7fc877400700 1 -- 172.21.15.117:0/1780766113 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc870021410 con 0x7fc880102e50 2024-04-20T11:12:38.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.377+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88019de80 con 0x7fc880102e50 2024-04-20T11:12:38.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.377+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88019e370 con 0x7fc880102e50 2024-04-20T11:12:38.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.378+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc880192950 con 0x7fc880102e50 2024-04-20T11:12:38.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.379+0000 7fc877400700 1 -- 172.21.15.117:0/1780766113 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fc87000f820 con 0x7fc880102e50 2024-04-20T11:12:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.380+0000 7fc877400700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc86406e950 0x7fc864070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:38.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.380+0000 7fc877400700 1 -- 172.21.15.117:0/1780766113 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fc87008fab0 con 0x7fc880102e50 2024-04-20T11:12:38.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.380+0000 7fc87e000700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc86406e950 0x7fc864070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:38.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.381+0000 7fc87e000700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc86406e950 0x7fc864070e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc86c005950 tx=0x7fc86c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:38.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.386+0000 7fc877400700 1 -- 172.21.15.117:0/1780766113 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fc87005c8f0 con 0x7fc880102e50 2024-04-20T11:12:38.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.634+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc88002cef0 con 0x7fc86406e950 2024-04-20T11:12:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.637+0000 7fc877400700 1 -- 172.21.15.117:0/1780766113 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fc88002cef0 con 0x7fc86406e950 2024-04-20T11:12:38.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc86406e950 msgr2=0x7fc864070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc86406e950 0x7fc864070e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc86c005950 tx=0x7fc86c0058e0 comp rx=0 tx=0).stop 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 msgr2=0x7fc8801987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 0x7fc8801987e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc87000bd70 tx=0x7fc87000be50 comp rx=0 tx=0).stop 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 shutdown_connections 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc86406e950 0x7fc864070e10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880102e50 0x7fc8801987e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 --2- 172.21.15.117:0/1780766113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc880103fb0 0x7fc880198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 >> 172.21.15.117:0/1780766113 conn(0x7fc8800fe3d0 msgr2=0x7fc880107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.640+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 shutdown_connections 2024-04-20T11:12:38.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:38.641+0000 7fc885600700 1 -- 172.21.15.117:0/1780766113 wait complete. 2024-04-20T11:12:39.363 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:39 smithi117 ceph-mon[15659]: pgmap v119: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:39.363 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:39 smithi117 ceph-mon[15659]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:39.411 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-20T11:12:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:39 smithi173 ceph-mon[23800]: pgmap v119: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:39 smithi173 ceph-mon[23800]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:39.823 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:12:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:41 smithi173 ceph-mon[23800]: pgmap v120: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:41.522 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:41 smithi117 ceph-mon[15659]: pgmap v120: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:42.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.061+0000 7f00acc00700 1 -- 172.21.15.117:0/2017385876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 msgr2=0x7f00a810cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:42.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.061+0000 7f00acc00700 1 --2- 172.21.15.117:0/2017385876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 0x7f00a810cf70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0094009910 tx=0x7f0094009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:42.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.061+0000 7f00acc00700 1 -- 172.21.15.117:0/2017385876 shutdown_connections 2024-04-20T11:12:42.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.061+0000 7f00acc00700 1 --2- 172.21.15.117:0/2017385876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 0x7f00a810cf70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.061+0000 7f00acc00700 1 --2- 172.21.15.117:0/2017385876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00a80ff900 0x7f00a80ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.061+0000 7f00acc00700 1 -- 172.21.15.117:0/2017385876 >> 172.21.15.117:0/2017385876 conn(0x7f00a80fb5a0 msgr2=0x7f00a80fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:42.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.061+0000 7f00acc00700 1 -- 172.21.15.117:0/2017385876 shutdown_connections 2024-04-20T11:12:42.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.061+0000 7f00acc00700 1 -- 172.21.15.117:0/2017385876 wait complete. 2024-04-20T11:12:42.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.062+0000 7f00acc00700 1 Processor -- start 2024-04-20T11:12:42.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.063+0000 7f00acc00700 1 -- start start 2024-04-20T11:12:42.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.063+0000 7f00acc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00a80ff900 0x7f00a8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:42.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.063+0000 7f00acc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 0x7f00a8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:42.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.063+0000 7f00acc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a8199830 con 0x7f00a81002b0 2024-04-20T11:12:42.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.063+0000 7f00acc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a81999a0 con 0x7f00a80ff900 2024-04-20T11:12:42.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.064+0000 7f00a5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 0x7f00a8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:42.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.064+0000 7f00a5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 0x7f00a8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42880/0 (socket says 172.21.15.117:42880) 2024-04-20T11:12:42.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.064+0000 7f00a5400700 1 -- 172.21.15.117:0/1106020243 learned_addr learned my addr 172.21.15.117:0/1106020243 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:42.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.064+0000 7f00a5e00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00a80ff900 0x7f00a8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.064+0000 7f00a5400700 1 -- 172.21.15.117:0/1106020243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00a80ff900 msgr2=0x7f00a8198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.064+0000 7f00a5400700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00a80ff900 0x7f00a8198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.064+0000 7f00a5400700 1 -- 172.21.15.117:0/1106020243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00940095f0 con 0x7f00a81002b0 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.065+0000 7f00a5400700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 0x7f00a8199230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0094006010 tx=0x7f0094005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.065+0000 7f009ea00700 1 -- 172.21.15.117:0/1106020243 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f009401d070 con 0x7f00a81002b0 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.065+0000 7f009ea00700 1 -- 172.21.15.117:0/1106020243 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00940056c0 con 0x7f00a81002b0 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.065+0000 7f009ea00700 1 -- 172.21.15.117:0/1106020243 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0094021640 con 0x7f00a81002b0 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.065+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a819c620 con 0x7f00a81002b0 2024-04-20T11:12:42.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.065+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a819cb70 con 0x7f00a81002b0 2024-04-20T11:12:42.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.066+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a810adf0 con 0x7f00a81002b0 2024-04-20T11:12:42.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.068+0000 7f009ea00700 1 -- 172.21.15.117:0/1106020243 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f009400f460 con 0x7f00a81002b0 2024-04-20T11:12:42.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.068+0000 7f009ea00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f008c06e950 0x7f008c070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:42.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.069+0000 7f009ea00700 1 -- 172.21.15.117:0/1106020243 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f009408fb10 con 0x7f00a81002b0 2024-04-20T11:12:42.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.069+0000 7f00a5e00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f008c06e950 0x7f008c070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:42.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.069+0000 7f00a5e00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f008c06e950 0x7f008c070e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0098005ea0 tx=0x7f0098005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:42.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.075+0000 7f009ea00700 1 -- 172.21.15.117:0/1106020243 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f0094058e00 con 0x7f00a81002b0 2024-04-20T11:12:42.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.396+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f00a8069a20 con 0x7f00a81002b0 2024-04-20T11:12:42.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.398+0000 7f009ea00700 1 -- 172.21.15.117:0/1106020243 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f009405c4c0 con 0x7f00a81002b0 2024-04-20T11:12:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f008c06e950 msgr2=0x7f008c070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f008c06e950 0x7f008c070e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0098005ea0 tx=0x7f0098005e30 comp rx=0 tx=0).stop 2024-04-20T11:12:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 msgr2=0x7f00a8199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 0x7f00a8199230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0094006010 tx=0x7f0094005470 comp rx=0 tx=0).stop 2024-04-20T11:12:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 shutdown_connections 2024-04-20T11:12:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f008c06e950 0x7f008c070e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00a80ff900 0x7f00a8198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 --2- 172.21.15.117:0/1106020243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a81002b0 0x7f00a8199230 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.400+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 >> 172.21.15.117:0/1106020243 conn(0x7f00a80fb5a0 msgr2=0x7f00a8107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:42.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.401+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 shutdown_connections 2024-04-20T11:12:42.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:42.401+0000 7f00acc00700 1 -- 172.21.15.117:0/1106020243 wait complete. 2024-04-20T11:12:42.415 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:12:43.185 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-20T11:12:43.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:43 smithi117 ceph-mon[15659]: pgmap v121: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:43 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/1106020243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:43.589 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:12:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:43 smithi173 ceph-mon[23800]: pgmap v121: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:43 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/1106020243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:45.547 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:45 smithi117 ceph-mon[15659]: pgmap v122: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:45 smithi173 ceph-mon[23800]: pgmap v122: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.716+0000 7fc4a5c00700 1 -- 172.21.15.117:0/3420461704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 msgr2=0x7fc4a01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.716+0000 7fc4a5c00700 1 --2- 172.21.15.117:0/3420461704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 0x7fc4a01020c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b0a0 tx=0x7fc49000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:12:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.717+0000 7fc4a5c00700 1 -- 172.21.15.117:0/3420461704 shutdown_connections 2024-04-20T11:12:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.717+0000 7fc4a5c00700 1 --2- 172.21.15.117:0/3420461704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a0102690 0x7fc4a010ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.717+0000 7fc4a5c00700 1 --2- 172.21.15.117:0/3420461704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 0x7fc4a01020c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.717+0000 7fc4a5c00700 1 -- 172.21.15.117:0/3420461704 >> 172.21.15.117:0/3420461704 conn(0x7fc4a00fb5c0 msgr2=0x7fc4a00fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.717+0000 7fc4a5c00700 1 -- 172.21.15.117:0/3420461704 shutdown_connections 2024-04-20T11:12:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.717+0000 7fc4a5c00700 1 -- 172.21.15.117:0/3420461704 wait complete. 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.718+0000 7fc4a5c00700 1 Processor -- start 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.718+0000 7fc4a5c00700 1 -- start start 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.718+0000 7fc4a5c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 0x7fc4a0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.718+0000 7fc4a5c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a0102690 0x7fc4a01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.718+0000 7fc4a5c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a0199ac0 con 0x7fc4a0101ce0 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.718+0000 7fc4a5c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a019d720 con 0x7fc4a0102690 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.719+0000 7fc49ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 0x7fc4a0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.719+0000 7fc49ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 0x7fc4a0198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41548/0 (socket says 172.21.15.117:41548) 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.719+0000 7fc49ea00700 1 -- 172.21.15.117:0/2996774202 learned_addr learned my addr 172.21.15.117:0/2996774202 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.719+0000 7fc496000700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a0102690 0x7fc4a01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.719+0000 7fc49ea00700 1 -- 172.21.15.117:0/2996774202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a0102690 msgr2=0x7fc4a01993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.719+0000 7fc49ea00700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a0102690 0x7fc4a01993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.719+0000 7fc49ea00700 1 -- 172.21.15.117:0/2996774202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc490009d00 con 0x7fc4a0101ce0 2024-04-20T11:12:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.719+0000 7fc49ea00700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 0x7fc4a0198ea0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b070 tx=0x7fc490007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.720+0000 7fc497e00700 1 -- 172.21.15.117:0/2996774202 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49000e040 con 0x7fc4a0101ce0 2024-04-20T11:12:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.720+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a019d9a0 con 0x7fc4a0101ce0 2024-04-20T11:12:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.720+0000 7fc497e00700 1 -- 172.21.15.117:0/2996774202 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc490003de0 con 0x7fc4a0101ce0 2024-04-20T11:12:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.720+0000 7fc497e00700 1 -- 172.21.15.117:0/2996774202 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49001d6b0 con 0x7fc4a0101ce0 2024-04-20T11:12:45.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.720+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a019de90 con 0x7fc4a0101ce0 2024-04-20T11:12:45.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.721+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c0052f0 con 0x7fc4a0101ce0 2024-04-20T11:12:45.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.722+0000 7fc497e00700 1 -- 172.21.15.117:0/2996774202 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fc490004040 con 0x7fc4a0101ce0 2024-04-20T11:12:45.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.723+0000 7fc497e00700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc48406e950 0x7fc484070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:45.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.723+0000 7fc497e00700 1 -- 172.21.15.117:0/2996774202 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fc49008ff50 con 0x7fc4a0101ce0 2024-04-20T11:12:45.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.723+0000 7fc496000700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc48406e950 0x7fc484070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:45.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.724+0000 7fc496000700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc48406e950 0x7fc484070e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc488006fd0 tx=0x7fc488008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:45.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:45.728+0000 7fc497e00700 1 -- 172.21.15.117:0/2996774202 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fc490059240 con 0x7fc4a0101ce0 2024-04-20T11:12:46.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.046+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc47c005db0 con 0x7fc4a0101ce0 2024-04-20T11:12:46.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.047+0000 7fc497e00700 1 -- 172.21.15.117:0/2996774202 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7fc4900172f0 con 0x7fc4a0101ce0 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc48406e950 msgr2=0x7fc484070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc48406e950 0x7fc484070e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc488006fd0 tx=0x7fc488008040 comp rx=0 tx=0).stop 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 msgr2=0x7fc4a0198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 0x7fc4a0198ea0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b070 tx=0x7fc490007bd0 comp rx=0 tx=0).stop 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 shutdown_connections 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc48406e950 0x7fc484070e10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4a0101ce0 0x7fc4a0198ea0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 --2- 172.21.15.117:0/2996774202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a0102690 0x7fc4a01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 >> 172.21.15.117:0/2996774202 conn(0x7fc4a00fb5c0 msgr2=0x7fc4a0101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 shutdown_connections 2024-04-20T11:12:46.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:46.050+0000 7fc4a5c00700 1 -- 172.21.15.117:0/2996774202 wait complete. 2024-04-20T11:12:46.066 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)" 2024-04-20T11:12:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:46 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2996774202' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:46 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:12:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:46 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2996774202' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:46 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:12:46.872 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-20T11:12:47.274 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:12:47.597 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:47 smithi117 ceph-mon[15659]: pgmap v123: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:47 smithi173 ceph-mon[23800]: pgmap v123: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:49.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:49 smithi117 ceph-mon[15659]: pgmap v124: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:49.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.596+0000 7fc704000700 1 -- 172.21.15.117:0/3871943982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 msgr2=0x7fc6fc10ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:49.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.596+0000 7fc704000700 1 --2- 172.21.15.117:0/3871943982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 0x7fc6fc10ceb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009910 tx=0x7fc6e8009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:49.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.596+0000 7fc704000700 1 -- 172.21.15.117:0/3871943982 shutdown_connections 2024-04-20T11:12:49.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.596+0000 7fc704000700 1 --2- 172.21.15.117:0/3871943982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 0x7fc6fc10ceb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:49.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.596+0000 7fc704000700 1 --2- 172.21.15.117:0/3871943982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc073310 0x7fc6fc0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:49.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.596+0000 7fc704000700 1 -- 172.21.15.117:0/3871943982 >> 172.21.15.117:0/3871943982 conn(0x7fc6fc0fc6c0 msgr2=0x7fc6fc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:49.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.596+0000 7fc704000700 1 -- 172.21.15.117:0/3871943982 shutdown_connections 2024-04-20T11:12:49.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.596+0000 7fc704000700 1 -- 172.21.15.117:0/3871943982 wait complete. 2024-04-20T11:12:49.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.597+0000 7fc704000700 1 Processor -- start 2024-04-20T11:12:49.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.597+0000 7fc704000700 1 -- start start 2024-04-20T11:12:49.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.598+0000 7fc704000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc073310 0x7fc6fc198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:49.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.598+0000 7fc704000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 0x7fc6fc199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:49.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.598+0000 7fc704000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc199a00 con 0x7fc6fc073c30 2024-04-20T11:12:49.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.598+0000 7fc704000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc19d6d0 con 0x7fc6fc073310 2024-04-20T11:12:49.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.598+0000 7fc700c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 0x7fc6fc199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:49.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.598+0000 7fc700c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 0x7fc6fc199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41572/0 (socket says 172.21.15.117:41572) 2024-04-20T11:12:49.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.598+0000 7fc700c00700 1 -- 172.21.15.117:0/3712078804 learned_addr learned my addr 172.21.15.117:0/3712078804 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:49.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.598+0000 7fc701600700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc073310 0x7fc6fc198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:49.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc700c00700 1 -- 172.21.15.117:0/3712078804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc073310 msgr2=0x7fc6fc198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:49.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc700c00700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc073310 0x7fc6fc198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:49.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc700c00700 1 -- 172.21.15.117:0/3712078804 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80095f0 con 0x7fc6fc073c30 2024-04-20T11:12:49.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc700c00700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 0x7fc6fc199320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80092e0 tx=0x7fc6e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:49.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc6fa000700 1 -- 172.21.15.117:0/3712078804 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e801d070 con 0x7fc6fc073c30 2024-04-20T11:12:49.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc6fa000700 1 -- 172.21.15.117:0/3712078804 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6e800f460 con 0x7fc6fc073c30 2024-04-20T11:12:49.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc19d950 con 0x7fc6fc073c30 2024-04-20T11:12:49.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc6fa000700 1 -- 172.21.15.117:0/3712078804 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6e8017630 con 0x7fc6fc073c30 2024-04-20T11:12:49.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.599+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc19de40 con 0x7fc6fc073c30 2024-04-20T11:12:49.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.601+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc0749c0 con 0x7fc6fc073c30 2024-04-20T11:12:49.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.603+0000 7fc6fa000700 1 -- 172.21.15.117:0/3712078804 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fc6e800f5d0 con 0x7fc6fc073c30 2024-04-20T11:12:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.604+0000 7fc6fa000700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc6e406e940 0x7fc6e4070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.604+0000 7fc6fa000700 1 -- 172.21.15.117:0/3712078804 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fc6e808fff0 con 0x7fc6fc073c30 2024-04-20T11:12:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.604+0000 7fc701600700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc6e406e940 0x7fc6e4070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.605+0000 7fc701600700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc6e406e940 0x7fc6e4070e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.607+0000 7fc6fa000700 1 -- 172.21.15.117:0/3712078804 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fc6e8059340 con 0x7fc6fc073c30 2024-04-20T11:12:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:49 smithi173 ceph-mon[23800]: pgmap v124: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:49.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.946+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc6fc062510 con 0x7fc6fc073c30 2024-04-20T11:12:49.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.946+0000 7fc6fa000700 1 -- 172.21.15.117:0/3712078804 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7fc6e805ca00 con 0x7fc6fc073c30 2024-04-20T11:12:49.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc6e406e940 msgr2=0x7fc6e4070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:49.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc6e406e940 0x7fc6e4070e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0008040 comp rx=0 tx=0).stop 2024-04-20T11:12:49.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 msgr2=0x7fc6fc199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:49.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 0x7fc6fc199320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80092e0 tx=0x7fc6e800be20 comp rx=0 tx=0).stop 2024-04-20T11:12:49.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 shutdown_connections 2024-04-20T11:12:49.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fc6e406e940 0x7fc6e4070e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:49.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc073310 0x7fc6fc198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:49.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 --2- 172.21.15.117:0/3712078804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6fc073c30 0x7fc6fc199320 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:49.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.949+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 >> 172.21.15.117:0/3712078804 conn(0x7fc6fc0fc6c0 msgr2=0x7fc6fc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:49.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.950+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 shutdown_connections 2024-04-20T11:12:49.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:49.950+0000 7fc704000700 1 -- 172.21.15.117:0/3712078804 wait complete. 2024-04-20T11:12:49.966 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:12:50.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:50 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3712078804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:50 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3712078804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:12:50.750 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-20T11:12:51.140 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:12:51.553 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:51 smithi117 ceph-mon[15659]: pgmap v125: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:51 smithi173 ceph-mon[23800]: pgmap v125: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:53.416 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:53 smithi117 ceph-mon[15659]: pgmap v126: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.491+0000 7f5bc7000700 1 -- 172.21.15.117:0/2889955857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 msgr2=0x7f5bc0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.491+0000 7f5bc7000700 1 --2- 172.21.15.117:0/2889955857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 0x7f5bc0104e80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009910 tx=0x7f5ba8009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.492+0000 7f5bc7000700 1 -- 172.21.15.117:0/2889955857 shutdown_connections 2024-04-20T11:12:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.492+0000 7f5bc7000700 1 --2- 172.21.15.117:0/2889955857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 0x7f5bc0104e80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.492+0000 7f5bc7000700 1 --2- 172.21.15.117:0/2889955857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0103110 0x7f5bc01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.492+0000 7f5bc7000700 1 -- 172.21.15.117:0/2889955857 >> 172.21.15.117:0/2889955857 conn(0x7f5bc00fe880 msgr2=0x7f5bc0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.492+0000 7f5bc7000700 1 -- 172.21.15.117:0/2889955857 shutdown_connections 2024-04-20T11:12:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.492+0000 7f5bc7000700 1 -- 172.21.15.117:0/2889955857 wait complete. 2024-04-20T11:12:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.493+0000 7f5bc7000700 1 Processor -- start 2024-04-20T11:12:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.493+0000 7f5bc7000700 1 -- start start 2024-04-20T11:12:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.493+0000 7f5bc7000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0103110 0x7f5bc019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.493+0000 7f5bc7000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 0x7f5bc019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.493+0000 7f5bc7000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc019f520 con 0x7f5bc0104a00 2024-04-20T11:12:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.493+0000 7f5bc7000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0198b90 con 0x7f5bc0103110 2024-04-20T11:12:53.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.494+0000 7f5bbf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 0x7f5bc019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:53.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.494+0000 7f5bbf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 0x7f5bc019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41580/0 (socket says 172.21.15.117:41580) 2024-04-20T11:12:53.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.494+0000 7f5bbf400700 1 -- 172.21.15.117:0/3689949511 learned_addr learned my addr 172.21.15.117:0/3689949511 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:12:53.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.494+0000 7f5bbfe00700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0103110 0x7f5bc019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:53.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.494+0000 7f5bbf400700 1 -- 172.21.15.117:0/3689949511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0103110 msgr2=0x7f5bc019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.494+0000 7f5bbf400700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0103110 0x7f5bc019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.494+0000 7f5bbf400700 1 -- 172.21.15.117:0/3689949511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80095f0 con 0x7f5bc0104a00 2024-04-20T11:12:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.495+0000 7f5bbf400700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 0x7f5bc019ef90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8006010 tx=0x7f5ba800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.495+0000 7f5bbcc00700 1 -- 172.21.15.117:0/3689949511 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ba801d070 con 0x7f5bc0104a00 2024-04-20T11:12:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.495+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc0198e10 con 0x7f5bc0104a00 2024-04-20T11:12:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.495+0000 7f5bbcc00700 1 -- 172.21.15.117:0/3689949511 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ba800f460 con 0x7f5bc0104a00 2024-04-20T11:12:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.495+0000 7f5bbcc00700 1 -- 172.21.15.117:0/3689949511 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ba8017630 con 0x7f5bc0104a00 2024-04-20T11:12:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.495+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc0199380 con 0x7f5bc0104a00 2024-04-20T11:12:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.496+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc010b480 con 0x7f5bc0104a00 2024-04-20T11:12:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.498+0000 7f5bbcc00700 1 -- 172.21.15.117:0/3689949511 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f5ba800f5d0 con 0x7f5bc0104a00 2024-04-20T11:12:53.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.499+0000 7f5bbcc00700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f5ba406e940 0x7f5ba4070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:53.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.499+0000 7f5bbcc00700 1 -- 172.21.15.117:0/3689949511 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f5ba808fcc0 con 0x7f5bc0104a00 2024-04-20T11:12:53.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.499+0000 7f5bbfe00700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f5ba406e940 0x7f5ba4070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:53.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.500+0000 7f5bbfe00700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f5ba406e940 0x7f5ba4070e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005ea0 tx=0x7f5bb0005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.503+0000 7f5bbcc00700 1 -- 172.21.15.117:0/3689949511 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f5ba805cc40 con 0x7f5bc0104a00 2024-04-20T11:12:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:53 smithi173 ceph-mon[23800]: pgmap v126: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:53.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:53.748+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bc0199fe0 con 0x7f5ba406e940 2024-04-20T11:12:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:54 smithi117 ceph-mon[15659]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:54 smithi173 ceph-mon[23800]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:55 smithi117 ceph-mon[15659]: pgmap v127: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:55 smithi173 ceph-mon[23800]: pgmap v127: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:56.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.675+0000 7f5bbcc00700 1 -- 172.21.15.117:0/3689949511 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5756 (secure 0 0 0) 0x7f5bc0199fe0 con 0x7f5ba406e940 2024-04-20T11:12:56.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.678+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f5ba406e940 msgr2=0x7f5ba4070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:56.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f5ba406e940 0x7f5ba4070e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005ea0 tx=0x7f5bb0005e30 comp rx=0 tx=0).stop 2024-04-20T11:12:56.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 msgr2=0x7f5bc019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:56.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 0x7f5bc019ef90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8006010 tx=0x7f5ba800be20 comp rx=0 tx=0).stop 2024-04-20T11:12:56.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 shutdown_connections 2024-04-20T11:12:56.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f5ba406e940 0x7f5ba4070e00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:56.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0103110 0x7f5bc019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:56.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 --2- 172.21.15.117:0/3689949511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc0104a00 0x7f5bc019ef90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:56.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 >> 172.21.15.117:0/3689949511 conn(0x7f5bc00fe880 msgr2=0x7f5bc00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:56.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 shutdown_connections 2024-04-20T11:12:56.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:12:56.679+0000 7f5bc7000700 1 -- 172.21.15.117:0/3689949511 wait complete. 2024-04-20T11:12:56.700 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:12:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:57 smithi117 ceph-mon[15659]: pgmap v128: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:57 smithi173 ceph-mon[23800]: pgmap v128: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:57.741 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph orch ps' 2024-04-20T11:12:58.168 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:12:59.596 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:12:59 smithi117 ceph-mon[15659]: pgmap v129: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:12:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:12:59 smithi173 ceph-mon[23800]: pgmap v129: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:00.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.460+0000 7f4b43000700 1 -- 172.21.15.117:0/1033913950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 msgr2=0x7f4b3c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:00.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.460+0000 7f4b43000700 1 --2- 172.21.15.117:0/1033913950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 0x7f4b3c107900 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009910 tx=0x7f4b2c009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:00.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.460+0000 7f4b43000700 1 -- 172.21.15.117:0/1033913950 shutdown_connections 2024-04-20T11:13:00.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.460+0000 7f4b43000700 1 --2- 172.21.15.117:0/1033913950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c108e10 0x7f4b3c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:00.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.460+0000 7f4b43000700 1 --2- 172.21.15.117:0/1033913950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 0x7f4b3c107900 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:00.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.460+0000 7f4b43000700 1 -- 172.21.15.117:0/1033913950 >> 172.21.15.117:0/1033913950 conn(0x7f4b3c0768f0 msgr2=0x7f4b3c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:00.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.460+0000 7f4b43000700 1 -- 172.21.15.117:0/1033913950 shutdown_connections 2024-04-20T11:13:00.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.460+0000 7f4b43000700 1 -- 172.21.15.117:0/1033913950 wait complete. 2024-04-20T11:13:00.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.461+0000 7f4b43000700 1 Processor -- start 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.461+0000 7f4b43000700 1 -- start start 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.461+0000 7f4b43000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 0x7f4b3c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.461+0000 7f4b43000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c108e10 0x7f4b3c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.461+0000 7f4b43000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c19ddd0 con 0x7f4b3c107520 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.461+0000 7f4b43000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c1a1aa0 con 0x7f4b3c108e10 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.462+0000 7f4b3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 0x7f4b3c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.462+0000 7f4b3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 0x7f4b3c19d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41302/0 (socket says 172.21.15.117:41302) 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.462+0000 7f4b3be00700 1 -- 172.21.15.117:0/1194181637 learned_addr learned my addr 172.21.15.117:0/1194181637 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:00.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.462+0000 7f4b3b400700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c108e10 0x7f4b3c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:00.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.462+0000 7f4b3be00700 1 -- 172.21.15.117:0/1194181637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c108e10 msgr2=0x7f4b3c19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:00.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.462+0000 7f4b3be00700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c108e10 0x7f4b3c19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:00.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.462+0000 7f4b3be00700 1 -- 172.21.15.117:0/1194181637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0095f0 con 0x7f4b3c107520 2024-04-20T11:13:00.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.462+0000 7f4b3be00700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 0x7f4b3c19d1b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00bd70 tx=0x7f4b2c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:00.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.463+0000 7f4b38c00700 1 -- 172.21.15.117:0/1194181637 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2c01d070 con 0x7f4b3c107520 2024-04-20T11:13:00.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.463+0000 7f4b38c00700 1 -- 172.21.15.117:0/1194181637 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b2c0056e0 con 0x7f4b3c107520 2024-04-20T11:13:00.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.463+0000 7f4b38c00700 1 -- 172.21.15.117:0/1194181637 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2c017400 con 0x7f4b3c107520 2024-04-20T11:13:00.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.463+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c1a1d20 con 0x7f4b3c107520 2024-04-20T11:13:00.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.463+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c1a2290 con 0x7f4b3c107520 2024-04-20T11:13:00.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.464+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c10f890 con 0x7f4b3c107520 2024-04-20T11:13:00.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.465+0000 7f4b38c00700 1 -- 172.21.15.117:0/1194181637 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f4b2c003a60 con 0x7f4b3c107520 2024-04-20T11:13:00.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.466+0000 7f4b38c00700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4b2006e940 0x7f4b20070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:00.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.466+0000 7f4b38c00700 1 -- 172.21.15.117:0/1194181637 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f4b2c08fc90 con 0x7f4b3c107520 2024-04-20T11:13:00.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.466+0000 7f4b3b400700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4b2006e940 0x7f4b20070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:00.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.467+0000 7f4b3b400700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4b2006e940 0x7f4b20070e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005950 tx=0x7f4b240058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:00.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.471+0000 7f4b38c00700 1 -- 172.21.15.117:0/1194181637 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f4b2c058f80 con 0x7f4b3c107520 2024-04-20T11:13:00.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.705+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b3c19e530 con 0x7f4b2006e940 2024-04-20T11:13:00.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.736+0000 7f4b38c00700 1 -- 172.21.15.117:0/1194181637 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4b3c19e530 con 0x7f4b2006e940 2024-04-20T11:13:00.736 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:13:00.736 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (88s) 38s ago 15m 13.6M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:13:00.736 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (15m) 38s ago 15m 8266k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (14m) 2m ago 14m 7880k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (15m) 38s ago 15m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (14m) 2m ago 14m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (46s) 38s ago 15m 49.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (8m) 38s ago 8m 15.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (8m) 38s ago 8m 152M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (8m) 2m ago 8m 14.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (8m) 2m ago 8m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (4m) 38s ago 17m 512M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (3m) 2m ago 14m 447M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (17m) 38s ago 17m 58.7M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (14m) 2m ago 14m 44.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:13:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (2m) 38s ago 15m 7638k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:13:00.738 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (2m) 2m ago 14m 2512k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:13:00.738 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (13m) 38s ago 13m 210M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:13:00.738 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (12m) 38s ago 12m 216M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:13:00.738 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (11m) 38s ago 11m 181M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:13:00.738 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (11m) 2m ago 11m 205M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:13:00.738 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (10m) 2m ago 10m 182M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:13:00.738 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (10m) 2m ago 10m 171M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:13:00.738 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (2m) 38s ago 14m 57.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:13:00.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4b2006e940 msgr2=0x7f4b20070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:00.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4b2006e940 0x7f4b20070e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005950 tx=0x7f4b240058e0 comp rx=0 tx=0).stop 2024-04-20T11:13:00.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 msgr2=0x7f4b3c19d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:00.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 0x7f4b3c19d1b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00bd70 tx=0x7f4b2c00be50 comp rx=0 tx=0).stop 2024-04-20T11:13:00.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 shutdown_connections 2024-04-20T11:13:00.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4b2006e940 0x7f4b20070e00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:00.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b3c107520 0x7f4b3c19d1b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:00.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 --2- 172.21.15.117:0/1194181637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b3c108e10 0x7f4b3c19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:00.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 >> 172.21.15.117:0/1194181637 conn(0x7f4b3c0768f0 msgr2=0x7f4b3c077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:00.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 shutdown_connections 2024-04-20T11:13:00.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:00.740+0000 7f4b43000700 1 -- 172.21.15.117:0/1194181637 wait complete. 2024-04-20T11:13:01.554 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:01 smithi117 ceph-mon[15659]: pgmap v130: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:01.554 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:01 smithi117 ceph-mon[15659]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:01.554 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:01 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:01.595 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-20T11:13:01.596 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:13:01.596 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-20T11:13:01.630 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:01 smithi173 ceph-mon[23800]: pgmap v130: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:01.630 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:01 smithi173 ceph-mon[23800]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:01.630 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:01 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:02.016 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:13:03.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:03 smithi117 ceph-mon[15659]: pgmap v131: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:03 smithi173 ceph-mon[23800]: pgmap v131: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:04.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.331+0000 7fdc43c00700 1 -- 172.21.15.117:0/3808138572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c103110 msgr2=0x7fdc3c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:04.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.331+0000 7fdc43c00700 1 --2- 172.21.15.117:0/3808138572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c103110 0x7fdc3c1034f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009910 tx=0x7fdc30009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:04.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.332+0000 7fdc43c00700 1 -- 172.21.15.117:0/3808138572 shutdown_connections 2024-04-20T11:13:04.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.332+0000 7fdc43c00700 1 --2- 172.21.15.117:0/3808138572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc3c104a00 0x7fdc3c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:04.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.332+0000 7fdc43c00700 1 --2- 172.21.15.117:0/3808138572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c103110 0x7fdc3c1034f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:04.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.332+0000 7fdc43c00700 1 -- 172.21.15.117:0/3808138572 >> 172.21.15.117:0/3808138572 conn(0x7fdc3c0fe880 msgr2=0x7fdc3c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:04.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.332+0000 7fdc43c00700 1 -- 172.21.15.117:0/3808138572 shutdown_connections 2024-04-20T11:13:04.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.332+0000 7fdc43c00700 1 -- 172.21.15.117:0/3808138572 wait complete. 2024-04-20T11:13:04.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.333+0000 7fdc43c00700 1 Processor -- start 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.333+0000 7fdc43c00700 1 -- start start 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.333+0000 7fdc43c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc3c103110 0x7fdc3c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.333+0000 7fdc43c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104a00 0x7fdc3c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.333+0000 7fdc43c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c199ac0 con 0x7fdc3c104a00 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.333+0000 7fdc43c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c19d740 con 0x7fdc3c103110 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.334+0000 7fdc3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104a00 0x7fdc3c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.334+0000 7fdc3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104a00 0x7fdc3c1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41612/0 (socket says 172.21.15.117:41612) 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.334+0000 7fdc3be00700 1 -- 172.21.15.117:0/794905001 learned_addr learned my addr 172.21.15.117:0/794905001 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.334+0000 7fdc41200700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc3c103110 0x7fdc3c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.334+0000 7fdc3be00700 1 -- 172.21.15.117:0/794905001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc3c103110 msgr2=0x7fdc3c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.334+0000 7fdc3be00700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc3c103110 0x7fdc3c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:04.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.334+0000 7fdc3be00700 1 -- 172.21.15.117:0/794905001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc300095f0 con 0x7fdc3c104a00 2024-04-20T11:13:04.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.335+0000 7fdc3be00700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104a00 0x7fdc3c1993e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800b4b0 tx=0x7fdc2800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:04.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.335+0000 7fdc39600700 1 -- 172.21.15.117:0/794905001 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc28011800 con 0x7fdc3c104a00 2024-04-20T11:13:04.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.335+0000 7fdc39600700 1 -- 172.21.15.117:0/794905001 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc28011e20 con 0x7fdc3c104a00 2024-04-20T11:13:04.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.335+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c19da20 con 0x7fdc3c104a00 2024-04-20T11:13:04.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.335+0000 7fdc39600700 1 -- 172.21.15.117:0/794905001 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc2800f2e0 con 0x7fdc3c104a00 2024-04-20T11:13:04.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.335+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c19dff0 con 0x7fdc3c104a00 2024-04-20T11:13:04.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.336+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c10b460 con 0x7fdc3c104a00 2024-04-20T11:13:04.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.337+0000 7fdc39600700 1 -- 172.21.15.117:0/794905001 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fdc28010380 con 0x7fdc3c104a00 2024-04-20T11:13:04.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.339+0000 7fdc39600700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc2406e950 0x7fdc24070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:04.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.339+0000 7fdc39600700 1 -- 172.21.15.117:0/794905001 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fdc2808e4a0 con 0x7fdc3c104a00 2024-04-20T11:13:04.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.339+0000 7fdc41200700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc2406e950 0x7fdc24070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:04.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.340+0000 7fdc41200700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc2406e950 0x7fdc24070e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000b5c0 tx=0x7fdc3001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.344+0000 7fdc39600700 1 -- 172.21.15.117:0/794905001 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fdc28057710 con 0x7fdc3c104a00 2024-04-20T11:13:04.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.583+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fdc3c04ec20 con 0x7fdc3c104a00 2024-04-20T11:13:04.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.593+0000 7fdc39600700 1 -- 172.21.15.117:0/794905001 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v48)=0 v48) v1 ==== 125+0+0 (secure 0 0 0) 0x7fdc3c04ec20 con 0x7fdc3c104a00 2024-04-20T11:13:04.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc2406e950 msgr2=0x7fdc24070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:04.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc2406e950 0x7fdc24070e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000b5c0 tx=0x7fdc3001a040 comp rx=0 tx=0).stop 2024-04-20T11:13:04.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104a00 msgr2=0x7fdc3c1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:04.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104a00 0x7fdc3c1993e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800b4b0 tx=0x7fdc2800b870 comp rx=0 tx=0).stop 2024-04-20T11:13:04.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 shutdown_connections 2024-04-20T11:13:04.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdc2406e950 0x7fdc24070e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:04.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc3c103110 0x7fdc3c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:04.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 --2- 172.21.15.117:0/794905001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104a00 0x7fdc3c1993e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:04.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.596+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 >> 172.21.15.117:0/794905001 conn(0x7fdc3c0fe880 msgr2=0x7fdc3c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:04.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.597+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 shutdown_connections 2024-04-20T11:13:04.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:04.597+0000 7fdc43c00700 1 -- 172.21.15.117:0/794905001 wait complete. 2024-04-20T11:13:05.405 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-20T11:13:05.405 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:13:05.405 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-20T11:13:05.817 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:13:05.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:05 smithi117 ceph-mon[15659]: pgmap v132: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:05.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:05 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/794905001' entity='client.admin' 2024-04-20T11:13:05.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:05 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:05 smithi173 ceph-mon[23800]: pgmap v132: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:05 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/794905001' entity='client.admin' 2024-04-20T11:13:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:05 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:06 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:06 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:07.783 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:07 smithi117 ceph-mon[15659]: pgmap v133: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:07 smithi173 ceph-mon[23800]: pgmap v133: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:08.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.352+0000 7f66ac800700 1 -- 172.21.15.117:0/3597105721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 msgr2=0x7f66a4107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.352+0000 7f66ac800700 1 --2- 172.21.15.117:0/3597105721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 0x7f66a4107900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6698009910 tx=0x7f6698009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:08.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.353+0000 7f66ac800700 1 -- 172.21.15.117:0/3597105721 shutdown_connections 2024-04-20T11:13:08.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.353+0000 7f66ac800700 1 --2- 172.21.15.117:0/3597105721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a4108e10 0x7f66a4109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.353+0000 7f66ac800700 1 --2- 172.21.15.117:0/3597105721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 0x7f66a4107900 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.353+0000 7f66ac800700 1 -- 172.21.15.117:0/3597105721 >> 172.21.15.117:0/3597105721 conn(0x7f66a40768f0 msgr2=0x7f66a4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:08.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.353+0000 7f66ac800700 1 -- 172.21.15.117:0/3597105721 shutdown_connections 2024-04-20T11:13:08.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.353+0000 7f66ac800700 1 -- 172.21.15.117:0/3597105721 wait complete. 2024-04-20T11:13:08.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.354+0000 7f66ac800700 1 Processor -- start 2024-04-20T11:13:08.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.354+0000 7f66ac800700 1 -- start start 2024-04-20T11:13:08.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66ac800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 0x7f66a419d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:08.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66ac800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a4108e10 0x7f66a419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66ac800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a419dd20 con 0x7f66a4107520 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66ac800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a419de60 con 0x7f66a4108e10 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66a9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 0x7f66a419d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66a9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 0x7f66a419d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41628/0 (socket says 172.21.15.117:41628) 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66a9e00700 1 -- 172.21.15.117:0/2091957795 learned_addr learned my addr 172.21.15.117:0/2091957795 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66a9400700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a4108e10 0x7f66a419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66a9e00700 1 -- 172.21.15.117:0/2091957795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a4108e10 msgr2=0x7f66a419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66a9e00700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a4108e10 0x7f66a419d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.355+0000 7f66a9e00700 1 -- 172.21.15.117:0/2091957795 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66980095f0 con 0x7f66a4107520 2024-04-20T11:13:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.356+0000 7f66a9e00700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 0x7f66a419d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6698006010 tx=0x7f669800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:08.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.356+0000 7f669ea00700 1 -- 172.21.15.117:0/2091957795 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f669801d070 con 0x7f66a4107520 2024-04-20T11:13:08.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.356+0000 7f669ea00700 1 -- 172.21.15.117:0/2091957795 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f669800f460 con 0x7f66a4107520 2024-04-20T11:13:08.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.356+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a41a1b20 con 0x7f66a4107520 2024-04-20T11:13:08.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.356+0000 7f669ea00700 1 -- 172.21.15.117:0/2091957795 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6698017630 con 0x7f66a4107520 2024-04-20T11:13:08.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.356+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a41a2090 con 0x7f66a4107520 2024-04-20T11:13:08.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.357+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a410f890 con 0x7f66a4107520 2024-04-20T11:13:08.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.358+0000 7f669ea00700 1 -- 172.21.15.117:0/2091957795 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f669800f5d0 con 0x7f66a4107520 2024-04-20T11:13:08.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.359+0000 7f669ea00700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f668c06e940 0x7f668c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:08.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.359+0000 7f669ea00700 1 -- 172.21.15.117:0/2091957795 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f669808fd60 con 0x7f66a4107520 2024-04-20T11:13:08.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.359+0000 7f66a9400700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f668c06e940 0x7f668c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:08.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.360+0000 7f66a9400700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f668c06e940 0x7f668c070e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66a4079130 tx=0x7f6694005d50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:08.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.364+0000 7f669ea00700 1 -- 172.21.15.117:0/2091957795 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f6698094070 con 0x7f66a4107520 2024-04-20T11:13:08.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.588+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f66a4108920 con 0x7f66a4107520 2024-04-20T11:13:08.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.589+0000 7f669ea00700 1 -- 172.21.15.117:0/2091957795 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v48)=0 v48) v1 ==== 155+0+0 (secure 0 0 0) 0x7f66a4108920 con 0x7f66a4107520 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f668c06e940 msgr2=0x7f668c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f668c06e940 0x7f668c070e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66a4079130 tx=0x7f6694005d50 comp rx=0 tx=0).stop 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 msgr2=0x7f66a419d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 0x7f66a419d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6698006010 tx=0x7f669800be20 comp rx=0 tx=0).stop 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 shutdown_connections 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f668c06e940 0x7f668c070e00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66a4107520 0x7f66a419d190 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 --2- 172.21.15.117:0/2091957795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a4108e10 0x7f66a419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.592+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 >> 172.21.15.117:0/2091957795 conn(0x7f66a40768f0 msgr2=0x7f66a4077480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:08.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.593+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 shutdown_connections 2024-04-20T11:13:08.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:08.593+0000 7f66ac800700 1 -- 172.21.15.117:0/2091957795 wait complete. 2024-04-20T11:13:09.549 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-20T11:13:09.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:09 smithi117 ceph-mon[15659]: pgmap v134: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:09.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:09 smithi173 ceph-mon[23800]: pgmap v134: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:09.946 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:13:11.878 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:11 smithi117 ceph-mon[15659]: pgmap v135: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:11 smithi173 ceph-mon[23800]: pgmap v135: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:12.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.224+0000 7fdf4e000700 1 -- 172.21.15.117:0/1981725284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 msgr2=0x7fdf481011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:12.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.224+0000 7fdf4e000700 1 --2- 172.21.15.117:0/1981725284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 0x7fdf481011f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009910 tx=0x7fdf38009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:12.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.224+0000 7fdf4e000700 1 -- 172.21.15.117:0/1981725284 shutdown_connections 2024-04-20T11:13:12.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.224+0000 7fdf4e000700 1 --2- 172.21.15.117:0/1981725284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf481017c0 0x7fdf48105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:12.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.224+0000 7fdf4e000700 1 --2- 172.21.15.117:0/1981725284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 0x7fdf481011f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:12.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.224+0000 7fdf4e000700 1 -- 172.21.15.117:0/1981725284 >> 172.21.15.117:0/1981725284 conn(0x7fdf480fc6c0 msgr2=0x7fdf480feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:12.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.224+0000 7fdf4e000700 1 -- 172.21.15.117:0/1981725284 shutdown_connections 2024-04-20T11:13:12.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.224+0000 7fdf4e000700 1 -- 172.21.15.117:0/1981725284 wait complete. 2024-04-20T11:13:12.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.225+0000 7fdf4e000700 1 Processor -- start 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.226+0000 7fdf4e000700 1 -- start start 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.226+0000 7fdf4e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 0x7fdf48198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.226+0000 7fdf4e000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf481017c0 0x7fdf481994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.226+0000 7fdf4e000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48199b90 con 0x7fdf48100e10 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.226+0000 7fdf4e000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4819d810 con 0x7fdf481017c0 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.226+0000 7fdf47400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 0x7fdf48198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.226+0000 7fdf47400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 0x7fdf48198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41642/0 (socket says 172.21.15.117:41642) 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.226+0000 7fdf47400700 1 -- 172.21.15.117:0/2497116551 learned_addr learned my addr 172.21.15.117:0/2497116551 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.227+0000 7fdf3ea00700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf481017c0 0x7fdf481994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.227+0000 7fdf47400700 1 -- 172.21.15.117:0/2497116551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf481017c0 msgr2=0x7fdf481994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.227+0000 7fdf47400700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf481017c0 0x7fdf481994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:12.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.227+0000 7fdf47400700 1 -- 172.21.15.117:0/2497116551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380095f0 con 0x7fdf48100e10 2024-04-20T11:13:12.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.227+0000 7fdf47400700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 0x7fdf48198f70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdf380092e0 tx=0x7fdf3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:12.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.228+0000 7fdf44c00700 1 -- 172.21.15.117:0/2497116551 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf3801d070 con 0x7fdf48100e10 2024-04-20T11:13:12.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.228+0000 7fdf44c00700 1 -- 172.21.15.117:0/2497116551 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf3800f460 con 0x7fdf48100e10 2024-04-20T11:13:12.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.228+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4819da90 con 0x7fdf48100e10 2024-04-20T11:13:12.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.228+0000 7fdf44c00700 1 -- 172.21.15.117:0/2497116551 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf38017630 con 0x7fdf48100e10 2024-04-20T11:13:12.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.228+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4819df80 con 0x7fdf48100e10 2024-04-20T11:13:12.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.229+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf481022f0 con 0x7fdf48100e10 2024-04-20T11:13:12.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.230+0000 7fdf44c00700 1 -- 172.21.15.117:0/2497116551 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fdf3800f5d0 con 0x7fdf48100e10 2024-04-20T11:13:12.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.231+0000 7fdf44c00700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdf2c06e940 0x7fdf2c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:12.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.231+0000 7fdf44c00700 1 -- 172.21.15.117:0/2497116551 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fdf3808fd70 con 0x7fdf48100e10 2024-04-20T11:13:12.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.231+0000 7fdf3ea00700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdf2c06e940 0x7fdf2c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:12.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.232+0000 7fdf3ea00700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdf2c06e940 0x7fdf2c070e00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf30006fd0 tx=0x7fdf30008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:12.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.236+0000 7fdf44c00700 1 -- 172.21.15.117:0/2497116551 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fdf38059060 con 0x7fdf48100e10 2024-04-20T11:13:12.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.463+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fdf48062510 con 0x7fdf48100e10 2024-04-20T11:13:12.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.465+0000 7fdf44c00700 1 -- 172.21.15.117:0/2497116551 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v48)=0 v48) v1 ==== 163+0+0 (secure 0 0 0) 0x7fdf3805c720 con 0x7fdf48100e10 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdf2c06e940 msgr2=0x7fdf2c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdf2c06e940 0x7fdf2c070e00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf30006fd0 tx=0x7fdf30008040 comp rx=0 tx=0).stop 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 msgr2=0x7fdf48198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 0x7fdf48198f70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdf380092e0 tx=0x7fdf3800be20 comp rx=0 tx=0).stop 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 shutdown_connections 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fdf2c06e940 0x7fdf2c070e00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48100e10 0x7fdf48198f70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 --2- 172.21.15.117:0/2497116551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf481017c0 0x7fdf481994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 >> 172.21.15.117:0/2497116551 conn(0x7fdf480fc6c0 msgr2=0x7fdf48074590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 shutdown_connections 2024-04-20T11:13:12.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:12.468+0000 7fdf4e000700 1 -- 172.21.15.117:0/2497116551 wait complete. 2024-04-20T11:13:13.277 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-20T11:13:13.686 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:13:13.876 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:13 smithi117 ceph-mon[15659]: pgmap v136: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:13 smithi173 ceph-mon[23800]: pgmap v136: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:15.698 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:15 smithi117 ceph-mon[15659]: pgmap v137: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:15 smithi173 ceph-mon[23800]: pgmap v137: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:15.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.996+0000 7fbe5b000700 1 -- 172.21.15.117:0/876170341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073200 msgr2=0x7fbe540735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.996+0000 7fbe5b000700 1 --2- 172.21.15.117:0/876170341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073200 0x7fbe540735e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009910 tx=0x7fbe44009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.996+0000 7fbe5b000700 1 -- 172.21.15.117:0/876170341 shutdown_connections 2024-04-20T11:13:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.996+0000 7fbe5b000700 1 --2- 172.21.15.117:0/876170341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe54073b20 0x7fbe5410cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.996+0000 7fbe5b000700 1 --2- 172.21.15.117:0/876170341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073200 0x7fbe540735e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.997+0000 7fbe5b000700 1 -- 172.21.15.117:0/876170341 >> 172.21.15.117:0/876170341 conn(0x7fbe540fc6a0 msgr2=0x7fbe540feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.997+0000 7fbe5b000700 1 -- 172.21.15.117:0/876170341 shutdown_connections 2024-04-20T11:13:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.997+0000 7fbe5b000700 1 -- 172.21.15.117:0/876170341 wait complete. 2024-04-20T11:13:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.998+0000 7fbe5b000700 1 Processor -- start 2024-04-20T11:13:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.998+0000 7fbe5b000700 1 -- start start 2024-04-20T11:13:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.998+0000 7fbe5b000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe54073200 0x7fbe54198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.998+0000 7fbe5b000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073b20 0x7fbe54199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.998+0000 7fbe5b000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54199840 con 0x7fbe54073b20 2024-04-20T11:13:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.998+0000 7fbe5b000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe541999b0 con 0x7fbe54073200 2024-04-20T11:13:16.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.999+0000 7fbe53400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073b20 0x7fbe54199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:16.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.999+0000 7fbe53400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073b20 0x7fbe54199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46054/0 (socket says 172.21.15.117:46054) 2024-04-20T11:13:16.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.999+0000 7fbe53400700 1 -- 172.21.15.117:0/543278810 learned_addr learned my addr 172.21.15.117:0/543278810 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:16.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.999+0000 7fbe53e00700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe54073200 0x7fbe54198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.999+0000 7fbe53400700 1 -- 172.21.15.117:0/543278810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe54073200 msgr2=0x7fbe54198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.999+0000 7fbe53400700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe54073200 0x7fbe54198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:15.999+0000 7fbe53400700 1 -- 172.21.15.117:0/543278810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440095f0 con 0x7fbe54073b20 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.000+0000 7fbe53400700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073b20 0x7fbe54199240 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c930 tx=0x7fbe4000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.000+0000 7fbe50c00700 1 -- 172.21.15.117:0/543278810 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe40007ad0 con 0x7fbe54073b20 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.000+0000 7fbe50c00700 1 -- 172.21.15.117:0/543278810 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe40007c30 con 0x7fbe54073b20 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.000+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5419c690 con 0x7fbe54073b20 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.000+0000 7fbe50c00700 1 -- 172.21.15.117:0/543278810 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe40018610 con 0x7fbe54073b20 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.000+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5419cbe0 con 0x7fbe54073b20 2024-04-20T11:13:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.001+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe54074940 con 0x7fbe54073b20 2024-04-20T11:13:16.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.003+0000 7fbe50c00700 1 -- 172.21.15.117:0/543278810 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fbe4000f450 con 0x7fbe54073b20 2024-04-20T11:13:16.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.004+0000 7fbe50c00700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbe3806e940 0x7fbe38070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:16.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.004+0000 7fbe50c00700 1 -- 172.21.15.117:0/543278810 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fbe4008ea40 con 0x7fbe54073b20 2024-04-20T11:13:16.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.004+0000 7fbe53e00700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbe3806e940 0x7fbe38070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:16.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.005+0000 7fbe53e00700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbe3806e940 0x7fbe38070e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe44005f50 tx=0x7fbe4401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:16.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.009+0000 7fbe50c00700 1 -- 172.21.15.117:0/543278810 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fbe40057d30 con 0x7fbe54073b20 2024-04-20T11:13:16.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.246+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fbe5404ebb0 con 0x7fbe54073b20 2024-04-20T11:13:16.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.247+0000 7fbe50c00700 1 -- 172.21.15.117:0/543278810 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v48)=0 v48) v1 ==== 135+0+0 (secure 0 0 0) 0x7fbe4005b3f0 con 0x7fbe54073b20 2024-04-20T11:13:16.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbe3806e940 msgr2=0x7fbe38070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:16.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbe3806e940 0x7fbe38070e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe44005f50 tx=0x7fbe4401a040 comp rx=0 tx=0).stop 2024-04-20T11:13:16.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073b20 msgr2=0x7fbe54199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:16.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073b20 0x7fbe54199240 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c930 tx=0x7fbe4000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:13:16.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 shutdown_connections 2024-04-20T11:13:16.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbe3806e940 0x7fbe38070e00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:16.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe54073200 0x7fbe54198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:16.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 --2- 172.21.15.117:0/543278810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe54073b20 0x7fbe54199240 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:16.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.250+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 >> 172.21.15.117:0/543278810 conn(0x7fbe540fc6a0 msgr2=0x7fbe541053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:16.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.251+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 shutdown_connections 2024-04-20T11:13:16.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:16.251+0000 7fbe5b000700 1 -- 172.21.15.117:0/543278810 wait complete. 2024-04-20T11:13:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:16 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:16 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:17.160 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-20T11:13:17.577 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:13:17.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:17 smithi117 ceph-mon[15659]: pgmap v138: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:17 smithi173 ceph-mon[23800]: pgmap v138: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:19.834 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:19 smithi117 ceph-mon[15659]: pgmap v139: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:19 smithi173 ceph-mon[23800]: pgmap v139: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:20.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:19.999+0000 7f6341800700 1 -- 172.21.15.117:0/1969803542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 msgr2=0x7f633c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:20.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:19.999+0000 7f6341800700 1 --2- 172.21.15.117:0/1969803542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 0x7f633c1034f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f632c009910 tx=0x7f632c009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:20.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:19.999+0000 7f6341800700 1 -- 172.21.15.117:0/1969803542 shutdown_connections 2024-04-20T11:13:20.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:19.999+0000 7f6341800700 1 --2- 172.21.15.117:0/1969803542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633c104a00 0x7f633c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:20.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:19.999+0000 7f6341800700 1 --2- 172.21.15.117:0/1969803542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 0x7f633c1034f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:20.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:19.999+0000 7f6341800700 1 -- 172.21.15.117:0/1969803542 >> 172.21.15.117:0/1969803542 conn(0x7f633c0fe880 msgr2=0x7f633c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:20.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.000+0000 7f6341800700 1 -- 172.21.15.117:0/1969803542 shutdown_connections 2024-04-20T11:13:20.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.000+0000 7f6341800700 1 -- 172.21.15.117:0/1969803542 wait complete. 2024-04-20T11:13:20.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.001+0000 7f6341800700 1 Processor -- start 2024-04-20T11:13:20.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.001+0000 7f6341800700 1 -- start start 2024-04-20T11:13:20.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f6341800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 0x7f633c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:20.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f6341800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633c104a00 0x7f633c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:20.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f6341800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c199980 con 0x7f633c103110 2024-04-20T11:13:20.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f6341800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c19d650 con 0x7f633c104a00 2024-04-20T11:13:20.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f633aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 0x7f633c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:20.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f633aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 0x7f633c198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46062/0 (socket says 172.21.15.117:46062) 2024-04-20T11:13:20.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f633aa00700 1 -- 172.21.15.117:0/2403984307 learned_addr learned my addr 172.21.15.117:0/2403984307 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:20.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f633a000700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633c104a00 0x7f633c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:20.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f633aa00700 1 -- 172.21.15.117:0/2403984307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633c104a00 msgr2=0x7f633c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:20.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.002+0000 7f633aa00700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633c104a00 0x7f633c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:20.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.003+0000 7f633aa00700 1 -- 172.21.15.117:0/2403984307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0095f0 con 0x7f633c103110 2024-04-20T11:13:20.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.003+0000 7f633aa00700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 0x7f633c198d60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f632c006010 tx=0x7f632c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:20.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.003+0000 7f6333400700 1 -- 172.21.15.117:0/2403984307 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f632c01d070 con 0x7f633c103110 2024-04-20T11:13:20.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.003+0000 7f6333400700 1 -- 172.21.15.117:0/2403984307 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f632c0056e0 con 0x7f633c103110 2024-04-20T11:13:20.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.003+0000 7f6333400700 1 -- 172.21.15.117:0/2403984307 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f632c017400 con 0x7f633c103110 2024-04-20T11:13:20.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.003+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c19d8d0 con 0x7f633c103110 2024-04-20T11:13:20.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.003+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c19de40 con 0x7f633c103110 2024-04-20T11:13:20.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.004+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c10b480 con 0x7f633c103110 2024-04-20T11:13:20.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.006+0000 7f6333400700 1 -- 172.21.15.117:0/2403984307 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f632c00f6f0 con 0x7f633c103110 2024-04-20T11:13:20.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.006+0000 7f6333400700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f632006e940 0x7f6320070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:20.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.007+0000 7f6333400700 1 -- 172.21.15.117:0/2403984307 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f632c08fc00 con 0x7f633c103110 2024-04-20T11:13:20.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.007+0000 7f633a000700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f632006e940 0x7f6320070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:20.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.007+0000 7f633a000700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f632006e940 0x7f6320070e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6328005950 tx=0x7f63280058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:20.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.011+0000 7f6333400700 1 -- 172.21.15.117:0/2403984307 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f632c058ef0 con 0x7f633c103110 2024-04-20T11:13:20.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.261+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "target": ["mon-mgr", ""]}) v1 -- 0x7f633c19a0e0 con 0x7f632006e940 2024-04-20T11:13:20.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.273+0000 7f6333400700 1 -- 172.21.15.117:0/2403984307 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f633c19a0e0 con 0x7f632006e940 2024-04-20T11:13:20.274 INFO:teuthology.orchestra.run.smithi117.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:13:20.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f632006e940 msgr2=0x7f6320070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:20.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f632006e940 0x7f6320070e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6328005950 tx=0x7f63280058e0 comp rx=0 tx=0).stop 2024-04-20T11:13:20.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 msgr2=0x7f633c198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:20.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 0x7f633c198d60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f632c006010 tx=0x7f632c00be20 comp rx=0 tx=0).stop 2024-04-20T11:13:20.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 shutdown_connections 2024-04-20T11:13:20.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f632006e940 0x7f6320070e00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:20.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f633c103110 0x7f633c198d60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:20.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 --2- 172.21.15.117:0/2403984307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633c104a00 0x7f633c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:20.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 >> 172.21.15.117:0/2403984307 conn(0x7f633c0fe880 msgr2=0x7f633c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:20.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 shutdown_connections 2024-04-20T11:13:20.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:20.276+0000 7f6341800700 1 -- 172.21.15.117:0/2403984307 wait complete. 2024-04-20T11:13:21.152 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-20T11:13:21.153 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:13:21.153 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: pgmap v140: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:21 smithi117 ceph-mon[15659]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:13:21.560 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:13:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: pgmap v140: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:21.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:13:21.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:21.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:21.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:21.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:21.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:21.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:21 smithi173 ceph-mon[23800]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:13:23.529 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:23 smithi117 ceph-mon[15659]: pgmap v141: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:23 smithi173 ceph-mon[23800]: pgmap v141: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:24.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.433+0000 7f013f000700 1 -- 172.21.15.117:0/2732592229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138108e10 msgr2=0x7f0138109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:24.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.433+0000 7f013f000700 1 --2- 172.21.15.117:0/2732592229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138108e10 0x7f0138109290 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0120009910 tx=0x7f0120009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:24.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.433+0000 7f013f000700 1 -- 172.21.15.117:0/2732592229 shutdown_connections 2024-04-20T11:13:24.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.433+0000 7f013f000700 1 --2- 172.21.15.117:0/2732592229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138108e10 0x7f0138109290 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.433+0000 7f013f000700 1 --2- 172.21.15.117:0/2732592229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0138107520 0x7f0138107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.433+0000 7f013f000700 1 -- 172.21.15.117:0/2732592229 >> 172.21.15.117:0/2732592229 conn(0x7f01380768f0 msgr2=0x7f0138078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.433+0000 7f013f000700 1 -- 172.21.15.117:0/2732592229 shutdown_connections 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.433+0000 7f013f000700 1 -- 172.21.15.117:0/2732592229 wait complete. 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.434+0000 7f013f000700 1 Processor -- start 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.434+0000 7f013f000700 1 -- start start 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.434+0000 7f013f000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138107520 0x7f013819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.434+0000 7f013f000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0138108e10 0x7f013819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.434+0000 7f013f000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013819ddf0 con 0x7f0138107520 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.434+0000 7f013f000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381a1a70 con 0x7f0138108e10 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.435+0000 7f0137e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138107520 0x7f013819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.435+0000 7f0137e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138107520 0x7f013819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59188/0 (socket says 172.21.15.117:59188) 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.435+0000 7f0137e00700 1 -- 172.21.15.117:0/2812195295 learned_addr learned my addr 172.21.15.117:0/2812195295 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:24.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.435+0000 7f0137e00700 1 -- 172.21.15.117:0/2812195295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0138108e10 msgr2=0x7f013819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.435+0000 7f0137e00700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0138108e10 0x7f013819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.435+0000 7f0137e00700 1 -- 172.21.15.117:0/2812195295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01200095f0 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.436+0000 7f0137e00700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138107520 0x7f013819d1d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f012800c8a0 tx=0x7f012800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.436+0000 7f0135600700 1 -- 172.21.15.117:0/2812195295 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f012800cea0 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.436+0000 7f0135600700 1 -- 172.21.15.117:0/2812195295 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0128004830 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.436+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01381a1d50 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.436+0000 7f0135600700 1 -- 172.21.15.117:0/2812195295 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01280055b0 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.436+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01381a2320 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.437+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013810f870 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.438+0000 7f0135600700 1 -- 172.21.15.117:0/2812195295 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f01280049a0 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.439+0000 7f0135600700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f011c06e940 0x7f011c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.439+0000 7f0135600700 1 -- 172.21.15.117:0/2812195295 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f012808e4e0 con 0x7f0138107520 2024-04-20T11:13:24.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.439+0000 7f012fe00700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f011c06e940 0x7f011c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.440+0000 7f012fe00700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f011c06e940 0x7f011c070e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f012000b5c0 tx=0x7f0120005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.444+0000 7f0135600700 1 -- 172.21.15.117:0/2812195295 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f01280577d0 con 0x7f0138107520 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.690+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01381144c0 con 0x7f011c06e940 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.693+0000 7f0135600700 1 -- 172.21.15.117:0/2812195295 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f01381144c0 con 0x7f011c06e940 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f011c06e940 msgr2=0x7f011c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f011c06e940 0x7f011c070e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f012000b5c0 tx=0x7f0120005960 comp rx=0 tx=0).stop 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138107520 msgr2=0x7f013819d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138107520 0x7f013819d1d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f012800c8a0 tx=0x7f012800cc60 comp rx=0 tx=0).stop 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 shutdown_connections 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f011c06e940 0x7f011c070e00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138107520 0x7f013819d1d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 --2- 172.21.15.117:0/2812195295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0138108e10 0x7f013819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.696+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 >> 172.21.15.117:0/2812195295 conn(0x7f01380768f0 msgr2=0x7f0138077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.697+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 shutdown_connections 2024-04-20T11:13:24.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.697+0000 7f013f000700 1 -- 172.21.15.117:0/2812195295 wait complete. 2024-04-20T11:13:24.796 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:13:24.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.857+0000 7ffa4b400700 1 -- 172.21.15.117:0/3722444028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073310 msgr2=0x7ffa440736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:24.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.857+0000 7ffa4b400700 1 --2- 172.21.15.117:0/3722444028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073310 0x7ffa440736f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009910 tx=0x7ffa38009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:24.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.857+0000 7ffa4b400700 1 -- 172.21.15.117:0/3722444028 shutdown_connections 2024-04-20T11:13:24.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.857+0000 7ffa4b400700 1 --2- 172.21.15.117:0/3722444028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa44073c30 0x7ffa4410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.857+0000 7ffa4b400700 1 --2- 172.21.15.117:0/3722444028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073310 0x7ffa440736f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.857+0000 7ffa4b400700 1 -- 172.21.15.117:0/3722444028 >> 172.21.15.117:0/3722444028 conn(0x7ffa440fc6c0 msgr2=0x7ffa440feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:24.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.857+0000 7ffa4b400700 1 -- 172.21.15.117:0/3722444028 shutdown_connections 2024-04-20T11:13:24.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.857+0000 7ffa4b400700 1 -- 172.21.15.117:0/3722444028 wait complete. 2024-04-20T11:13:24.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.858+0000 7ffa4b400700 1 Processor -- start 2024-04-20T11:13:24.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.858+0000 7ffa4b400700 1 -- start start 2024-04-20T11:13:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.859+0000 7ffa4b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa44073310 0x7ffa44198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.859+0000 7ffa4b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073c30 0x7ffa44199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.859+0000 7ffa4b400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa441998d0 con 0x7ffa44073c30 2024-04-20T11:13:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.859+0000 7ffa4b400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4419c540 con 0x7ffa44073310 2024-04-20T11:13:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.859+0000 7ffa43e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073c30 0x7ffa44199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.859+0000 7ffa43e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073c30 0x7ffa44199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59214/0 (socket says 172.21.15.117:59214) 2024-04-20T11:13:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.859+0000 7ffa43e00700 1 -- 172.21.15.117:0/1805346624 learned_addr learned my addr 172.21.15.117:0/1805346624 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.859+0000 7ffa48a00700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa44073310 0x7ffa44198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:24.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.860+0000 7ffa43e00700 1 -- 172.21.15.117:0/1805346624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa44073310 msgr2=0x7ffa44198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.860+0000 7ffa43e00700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa44073310 0x7ffa44198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.860+0000 7ffa43e00700 1 -- 172.21.15.117:0/1805346624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa380095f0 con 0x7ffa44073c30 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.860+0000 7ffa48a00700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa44073310 0x7ffa44198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.860+0000 7ffa43e00700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073c30 0x7ffa44199240 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000c8a0 tx=0x7ffa3000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.860+0000 7ffa41600700 1 -- 172.21.15.117:0/1805346624 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3000cea0 con 0x7ffa44073c30 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.861+0000 7ffa41600700 1 -- 172.21.15.117:0/1805346624 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa30004830 con 0x7ffa44073c30 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.861+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4419c820 con 0x7ffa44073c30 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.861+0000 7ffa41600700 1 -- 172.21.15.117:0/1805346624 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa300055b0 con 0x7ffa44073c30 2024-04-20T11:13:24.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.861+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4419cd70 con 0x7ffa44073c30 2024-04-20T11:13:24.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.862+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa440749c0 con 0x7ffa44073c30 2024-04-20T11:13:24.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.863+0000 7ffa41600700 1 -- 172.21.15.117:0/1805346624 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7ffa300049a0 con 0x7ffa44073c30 2024-04-20T11:13:24.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.864+0000 7ffa41600700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7ffa2c06e940 0x7ffa2c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:24.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.864+0000 7ffa41600700 1 -- 172.21.15.117:0/1805346624 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7ffa3008e4e0 con 0x7ffa44073c30 2024-04-20T11:13:24.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.864+0000 7ffa48a00700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7ffa2c06e940 0x7ffa2c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:24.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.865+0000 7ffa48a00700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7ffa2c06e940 0x7ffa2c070e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006010 tx=0x7ffa3801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:24.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:24.869+0000 7ffa41600700 1 -- 172.21.15.117:0/1805346624 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7ffa300577d0 con 0x7ffa44073c30 2024-04-20T11:13:25.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.116+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa44107750 con 0x7ffa2c06e940 2024-04-20T11:13:25.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.121+0000 7ffa41600700 1 -- 172.21.15.117:0/1805346624 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7ffa44107750 con 0x7ffa2c06e940 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7ffa2c06e940 msgr2=0x7ffa2c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7ffa2c06e940 0x7ffa2c070e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006010 tx=0x7ffa3801a040 comp rx=0 tx=0).stop 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073c30 msgr2=0x7ffa44199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073c30 0x7ffa44199240 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000c8a0 tx=0x7ffa3000cc60 comp rx=0 tx=0).stop 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 shutdown_connections 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7ffa2c06e940 0x7ffa2c070e00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa44073310 0x7ffa44198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 --2- 172.21.15.117:0/1805346624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa44073c30 0x7ffa44199240 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 >> 172.21.15.117:0/1805346624 conn(0x7ffa440fc6c0 msgr2=0x7ffa44105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:25.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 shutdown_connections 2024-04-20T11:13:25.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.124+0000 7ffa4b400700 1 -- 172.21.15.117:0/1805346624 wait complete. 2024-04-20T11:13:25.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.279+0000 7fbbc9400700 1 -- 172.21.15.117:0/616619905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc41039d0 msgr2=0x7fbbc4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:25.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.279+0000 7fbbc9400700 1 --2- 172.21.15.117:0/616619905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc41039d0 0x7fbbc4107a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009910 tx=0x7fbbb0009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:25.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.279+0000 7fbbc9400700 1 -- 172.21.15.117:0/616619905 shutdown_connections 2024-04-20T11:13:25.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.279+0000 7fbbc9400700 1 --2- 172.21.15.117:0/616619905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc41039d0 0x7fbbc4107a20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.279+0000 7fbbc9400700 1 --2- 172.21.15.117:0/616619905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbc4103020 0x7fbbc4103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.279+0000 7fbbc9400700 1 -- 172.21.15.117:0/616619905 >> 172.21.15.117:0/616619905 conn(0x7fbbc40fe890 msgr2=0x7fbbc4100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:25.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.279+0000 7fbbc9400700 1 -- 172.21.15.117:0/616619905 shutdown_connections 2024-04-20T11:13:25.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.280+0000 7fbbc9400700 1 -- 172.21.15.117:0/616619905 wait complete. 2024-04-20T11:13:25.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.281+0000 7fbbc9400700 1 Processor -- start 2024-04-20T11:13:25.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.281+0000 7fbbc9400700 1 -- start start 2024-04-20T11:13:25.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.281+0000 7fbbc9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc4103020 0x7fbbc419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:25.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.281+0000 7fbbc9400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbc41039d0 0x7fbbc419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:25.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.281+0000 7fbbc9400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc419f5b0 con 0x7fbbc4103020 2024-04-20T11:13:25.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.281+0000 7fbbc9400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc4198be0 con 0x7fbbc41039d0 2024-04-20T11:13:25.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.282+0000 7fbbc2800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc4103020 0x7fbbc419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:25.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.282+0000 7fbbc2800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc4103020 0x7fbbc419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59234/0 (socket says 172.21.15.117:59234) 2024-04-20T11:13:25.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.282+0000 7fbbc2800700 1 -- 172.21.15.117:0/3838491721 learned_addr learned my addr 172.21.15.117:0/3838491721 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:25.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.282+0000 7fbbc1e00700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbc41039d0 0x7fbbc419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:25.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.282+0000 7fbbc2800700 1 -- 172.21.15.117:0/3838491721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbc41039d0 msgr2=0x7fbbc419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:25.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.282+0000 7fbbc2800700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbc41039d0 0x7fbbc419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.282+0000 7fbbc2800700 1 -- 172.21.15.117:0/3838491721 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00095f0 con 0x7fbbc4103020 2024-04-20T11:13:25.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.283+0000 7fbbc2800700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc4103020 0x7fbbc419ea50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400e940 tx=0x7fbbb400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:25.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.286+0000 7fbbbb400700 1 -- 172.21.15.117:0/3838491721 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbb40097b0 con 0x7fbbc4103020 2024-04-20T11:13:25.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.286+0000 7fbbbb400700 1 -- 172.21.15.117:0/3838491721 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbb4004cf0 con 0x7fbbc4103020 2024-04-20T11:13:25.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.286+0000 7fbbbb400700 1 -- 172.21.15.117:0/3838491721 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbb4005700 con 0x7fbbc4103020 2024-04-20T11:13:25.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.286+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc4198ec0 con 0x7fbbc4103020 2024-04-20T11:13:25.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.286+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc41993c0 con 0x7fbbc4103020 2024-04-20T11:13:25.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.288+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc410b420 con 0x7fbbc4103020 2024-04-20T11:13:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.289+0000 7fbbbb400700 1 -- 172.21.15.117:0/3838491721 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fbbb400cba0 con 0x7fbbc4103020 2024-04-20T11:13:25.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.290+0000 7fbbbb400700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbba806e950 0x7fbba8070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:25.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.290+0000 7fbbbb400700 1 -- 172.21.15.117:0/3838491721 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fbbb4014070 con 0x7fbbc4103020 2024-04-20T11:13:25.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.290+0000 7fbbc1e00700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbba806e950 0x7fbba8070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:25.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.291+0000 7fbbc1e00700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbba806e950 0x7fbba8070e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000b760 tx=0x7fbbb0005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:25.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.299+0000 7fbbbb400700 1 -- 172.21.15.117:0/3838491721 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fbbb405ba30 con 0x7fbbc4103020 2024-04-20T11:13:25.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:25 smithi117 ceph-mon[15659]: pgmap v142: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:25.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:25 smithi117 ceph-mon[15659]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.541+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbc410ffe0 con 0x7fbba806e950 2024-04-20T11:13:25.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.552+0000 7fbbbb400700 1 -- 172.21.15.117:0/3838491721 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fbbc410ffe0 con 0x7fbba806e950 2024-04-20T11:13:25.553 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:13:25.553 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (113s) 63s ago 15m 13.6M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:13:25.553 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (16m) 63s ago 16m 8266k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:13:25.553 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (14m) 2m ago 14m 7880k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:13:25.553 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (16m) 63s ago 16m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (14m) 2m ago 14m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (70s) 63s ago 15m 49.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (8m) 63s ago 8m 15.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (8m) 63s ago 8m 152M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (8m) 2m ago 8m 14.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (8m) 2m ago 8m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (5m) 63s ago 17m 512M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (3m) 2m ago 14m 447M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (17m) 63s ago 17m 58.7M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a3ca404fd50f 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (14m) 2m ago 14m 44.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (3m) 63s ago 16m 7638k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (3m) 2m ago 14m 2512k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (13m) 63s ago 13m 210M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:13:25.554 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (12m) 63s ago 12m 216M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:13:25.555 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (12m) 63s ago 12m 181M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:13:25.555 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (11m) 2m ago 11m 205M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:13:25.555 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (11m) 2m ago 11m 182M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:13:25.555 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (10m) 2m ago 10m 171M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:13:25.555 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (2m) 63s ago 15m 57.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:13:25.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbba806e950 msgr2=0x7fbba8070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:25.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbba806e950 0x7fbba8070e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000b760 tx=0x7fbbb0005fd0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc4103020 msgr2=0x7fbbc419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc4103020 0x7fbbc419ea50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400e940 tx=0x7fbbb400ed00 comp rx=0 tx=0).stop 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 shutdown_connections 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fbba806e950 0x7fbba8070e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbc4103020 0x7fbbc419ea50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 --2- 172.21.15.117:0/3838491721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbc41039d0 0x7fbbc419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.556+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 >> 172.21.15.117:0/3838491721 conn(0x7fbbc40fe890 msgr2=0x7fbbc40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.557+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 shutdown_connections 2024-04-20T11:13:25.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.557+0000 7fbbc9400700 1 -- 172.21.15.117:0/3838491721 wait complete. 2024-04-20T11:13:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:25 smithi173 ceph-mon[23800]: pgmap v142: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:25 smithi173 ceph-mon[23800]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:25.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.709+0000 7f4f24800700 1 -- 172.21.15.117:0/423633786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c1026b0 msgr2=0x7f4f1c10aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:25.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.709+0000 7f4f24800700 1 --2- 172.21.15.117:0/423633786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c1026b0 0x7f4f1c10aba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009910 tx=0x7f4f0c009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:25.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.709+0000 7f4f24800700 1 -- 172.21.15.117:0/423633786 shutdown_connections 2024-04-20T11:13:25.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.709+0000 7f4f24800700 1 --2- 172.21.15.117:0/423633786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c1026b0 0x7f4f1c10aba0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.709+0000 7f4f24800700 1 --2- 172.21.15.117:0/423633786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f1c101d90 0x7f4f1c102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.709+0000 7f4f24800700 1 -- 172.21.15.117:0/423633786 >> 172.21.15.117:0/423633786 conn(0x7f4f1c0fb5c0 msgr2=0x7f4f1c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:25.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.709+0000 7f4f24800700 1 -- 172.21.15.117:0/423633786 shutdown_connections 2024-04-20T11:13:25.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.710+0000 7f4f24800700 1 -- 172.21.15.117:0/423633786 wait complete. 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.711+0000 7f4f24800700 1 Processor -- start 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.711+0000 7f4f24800700 1 -- start start 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.711+0000 7f4f24800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c101d90 0x7f4f1c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.711+0000 7f4f24800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f1c1026b0 0x7f4f1c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.711+0000 7f4f24800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c199c40 con 0x7f4f1c101d90 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.711+0000 7f4f24800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c199d80 con 0x7f4f1c1026b0 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.711+0000 7f4f21e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c101d90 0x7f4f1c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.711+0000 7f4f21e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c101d90 0x7f4f1c1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59252/0 (socket says 172.21.15.117:59252) 2024-04-20T11:13:25.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.712+0000 7f4f21e00700 1 -- 172.21.15.117:0/3307903433 learned_addr learned my addr 172.21.15.117:0/3307903433 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:25.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.712+0000 7f4f21400700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f1c1026b0 0x7f4f1c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:25.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.712+0000 7f4f21e00700 1 -- 172.21.15.117:0/3307903433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f1c1026b0 msgr2=0x7f4f1c1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:25.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.712+0000 7f4f21e00700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f1c1026b0 0x7f4f1c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:25.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.712+0000 7f4f21e00700 1 -- 172.21.15.117:0/3307903433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c0095f0 con 0x7f4f1c101d90 2024-04-20T11:13:25.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.712+0000 7f4f21e00700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c101d90 0x7f4f1c1990b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000c930 tx=0x7f4f1000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:25.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.712+0000 7f4f16a00700 1 -- 172.21.15.117:0/3307903433 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f10007ad0 con 0x7f4f1c101d90 2024-04-20T11:13:25.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.713+0000 7f4f16a00700 1 -- 172.21.15.117:0/3307903433 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f10007c30 con 0x7f4f1c101d90 2024-04-20T11:13:25.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.713+0000 7f4f16a00700 1 -- 172.21.15.117:0/3307903433 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f10018610 con 0x7f4f1c101d90 2024-04-20T11:13:25.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.713+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c19dac0 con 0x7f4f1c101d90 2024-04-20T11:13:25.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.713+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c19e010 con 0x7f4f1c101d90 2024-04-20T11:13:25.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.714+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c109e10 con 0x7f4f1c101d90 2024-04-20T11:13:25.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.715+0000 7f4f16a00700 1 -- 172.21.15.117:0/3307903433 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f4f1000f450 con 0x7f4f1c101d90 2024-04-20T11:13:25.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.716+0000 7f4f16a00700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4f0406e940 0x7f4f04070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:25.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.717+0000 7f4f16a00700 1 -- 172.21.15.117:0/3307903433 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f4f1008e9a0 con 0x7f4f1c101d90 2024-04-20T11:13:25.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.717+0000 7f4f21400700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4f0406e940 0x7f4f04070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:25.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.717+0000 7f4f21400700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4f0406e940 0x7f4f04070e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c006010 tx=0x7f4f0c00b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:25.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:25.722+0000 7f4f16a00700 1 -- 172.21.15.117:0/3307903433 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f4f10057d30 con 0x7f4f1c101d90 2024-04-20T11:13:26.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.045+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4f1c06a5e0 con 0x7f4f1c101d90 2024-04-20T11:13:26.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.045+0000 7f4f16a00700 1 -- 172.21.15.117:0/3307903433 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f4f1005b3f0 con 0x7f4f1c101d90 2024-04-20T11:13:26.046 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:13:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:13:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-20T11:13:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:13:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12, 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:13:26.047 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:13:26.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4f0406e940 msgr2=0x7f4f04070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4f0406e940 0x7f4f04070e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c006010 tx=0x7f4f0c00b540 comp rx=0 tx=0).stop 2024-04-20T11:13:26.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c101d90 msgr2=0x7f4f1c1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c101d90 0x7f4f1c1990b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000c930 tx=0x7f4f1000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 shutdown_connections 2024-04-20T11:13:26.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f4f0406e940 0x7f4f04070e00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f1c101d90 0x7f4f1c1990b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 --2- 172.21.15.117:0/3307903433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f1c1026b0 0x7f4f1c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 >> 172.21.15.117:0/3307903433 conn(0x7f4f1c0fb5c0 msgr2=0x7f4f1c106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:26.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 shutdown_connections 2024-04-20T11:13:26.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.048+0000 7f4f24800700 1 -- 172.21.15.117:0/3307903433 wait complete. 2024-04-20T11:13:26.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.195+0000 7effb5000700 1 -- 172.21.15.117:0/448564889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb0100e20 msgr2=0x7effb0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.195+0000 7effb5000700 1 --2- 172.21.15.117:0/448564889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb0100e20 0x7effb0101200 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7effa0009910 tx=0x7effa0009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:26.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.196+0000 7effb5000700 1 -- 172.21.15.117:0/448564889 shutdown_connections 2024-04-20T11:13:26.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.196+0000 7effb5000700 1 --2- 172.21.15.117:0/448564889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb01017d0 0x7effb0105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.196+0000 7effb5000700 1 --2- 172.21.15.117:0/448564889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb0100e20 0x7effb0101200 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.196+0000 7effb5000700 1 -- 172.21.15.117:0/448564889 >> 172.21.15.117:0/448564889 conn(0x7effb00fc6f0 msgr2=0x7effb00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:26.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.196+0000 7effb5000700 1 -- 172.21.15.117:0/448564889 shutdown_connections 2024-04-20T11:13:26.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.196+0000 7effb5000700 1 -- 172.21.15.117:0/448564889 wait complete. 2024-04-20T11:13:26.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.197+0000 7effb5000700 1 Processor -- start 2024-04-20T11:13:26.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.197+0000 7effb5000700 1 -- start start 2024-04-20T11:13:26.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.197+0000 7effb5000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb0100e20 0x7effb019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:26.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.197+0000 7effb5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb01017d0 0x7effb019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:26.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.197+0000 7effb5000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb019d520 con 0x7effb01017d0 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.197+0000 7effb5000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb0196b00 con 0x7effb0100e20 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.197+0000 7effad400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb01017d0 0x7effb019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effad400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb01017d0 0x7effb019cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59272/0 (socket says 172.21.15.117:59272) 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effad400700 1 -- 172.21.15.117:0/2028496698 learned_addr learned my addr 172.21.15.117:0/2028496698 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effade00700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb0100e20 0x7effb019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effad400700 1 -- 172.21.15.117:0/2028496698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb0100e20 msgr2=0x7effb019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effad400700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb0100e20 0x7effb019c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effad400700 1 -- 172.21.15.117:0/2028496698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00095f0 con 0x7effb01017d0 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effade00700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb0100e20 0x7effb019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effad400700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb01017d0 0x7effb019cf00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00b480 tx=0x7eff9c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.198+0000 7effa6a00700 1 -- 172.21.15.117:0/2028496698 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff9c0118c0 con 0x7effb01017d0 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.199+0000 7effa6a00700 1 -- 172.21.15.117:0/2028496698 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff9c010380 con 0x7effb01017d0 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.199+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb0196de0 con 0x7effb01017d0 2024-04-20T11:13:26.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.199+0000 7effa6a00700 1 -- 172.21.15.117:0/2028496698 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff9c00f4f0 con 0x7effb01017d0 2024-04-20T11:13:26.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.199+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb0197330 con 0x7effb01017d0 2024-04-20T11:13:26.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.200+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb01091c0 con 0x7effb01017d0 2024-04-20T11:13:26.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.201+0000 7effa6a00700 1 -- 172.21.15.117:0/2028496698 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7eff9c0104f0 con 0x7effb01017d0 2024-04-20T11:13:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.202+0000 7effa6a00700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7eff9406e940 0x7eff94070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.202+0000 7effa6a00700 1 -- 172.21.15.117:0/2028496698 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7eff9c08e580 con 0x7effb01017d0 2024-04-20T11:13:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.202+0000 7effade00700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7eff9406e940 0x7eff94070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.203+0000 7effade00700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7eff9406e940 0x7eff94070e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7effa0005f50 tx=0x7effa0005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:26.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.207+0000 7effa6a00700 1 -- 172.21.15.117:0/2028496698 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7eff9c05b320 con 0x7effb01017d0 2024-04-20T11:13:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.494+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7effb0197fc0 con 0x7effb01017d0 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.495+0000 7effa6a00700 1 -- 172.21.15.117:0/2028496698 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1854 (secure 0 0 0) 0x7eff9c05aeb0 con 0x7effb01017d0 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:13:26.496 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:13:26.497 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:13:26.498 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:13:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:13:26.499 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:26.499 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7eff9406e940 msgr2=0x7eff94070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7eff9406e940 0x7eff94070e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7effa0005f50 tx=0x7effa0005dc0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb01017d0 msgr2=0x7effb019cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb01017d0 0x7effb019cf00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00b480 tx=0x7eff9c00b840 comp rx=0 tx=0).stop 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 shutdown_connections 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7eff9406e940 0x7eff94070e00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb0100e20 0x7effb019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 --2- 172.21.15.117:0/2028496698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb01017d0 0x7effb019cf00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 >> 172.21.15.117:0/2028496698 conn(0x7effb00fc6f0 msgr2=0x7effb00fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 shutdown_connections 2024-04-20T11:13:26.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.499+0000 7effb5000700 1 -- 172.21.15.117:0/2028496698 wait complete. 2024-04-20T11:13:26.501 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: Upgrade: Target is version 19.0.0-3133-ga8c84f59 (squid) 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, digests ['quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1'] 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: Upgrade: Setting container_image for all mgr 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:26.604 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-20T11:13:26.605 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:26.605 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:26 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/3307903433' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:13:26.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.650+0000 7fe059a00700 1 -- 172.21.15.117:0/3171822796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 msgr2=0x7fe054105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.650+0000 7fe059a00700 1 --2- 172.21.15.117:0/3171822796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 0x7fe054105810 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe040009910 tx=0x7fe040009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:26.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.650+0000 7fe059a00700 1 -- 172.21.15.117:0/3171822796 shutdown_connections 2024-04-20T11:13:26.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.650+0000 7fe059a00700 1 --2- 172.21.15.117:0/3171822796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 0x7fe054105810 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.650+0000 7fe059a00700 1 --2- 172.21.15.117:0/3171822796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe054100e10 0x7fe0541011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.650+0000 7fe059a00700 1 -- 172.21.15.117:0/3171822796 >> 172.21.15.117:0/3171822796 conn(0x7fe0540fc6c0 msgr2=0x7fe0540feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:26.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.650+0000 7fe059a00700 1 -- 172.21.15.117:0/3171822796 shutdown_connections 2024-04-20T11:13:26.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.650+0000 7fe059a00700 1 -- 172.21.15.117:0/3171822796 wait complete. 2024-04-20T11:13:26.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.651+0000 7fe059a00700 1 Processor -- start 2024-04-20T11:13:26.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.651+0000 7fe059a00700 1 -- start start 2024-04-20T11:13:26.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe059a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe054100e10 0x7fe05419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:26.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe059a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 0x7fe05419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:26.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe059a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419f5c0 con 0x7fe0541017c0 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe059a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe054198b40 con 0x7fe054100e10 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe052000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 0x7fe05419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe052000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 0x7fe05419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59290/0 (socket says 172.21.15.117:59290) 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe052000700 1 -- 172.21.15.117:0/1117395682 learned_addr learned my addr 172.21.15.117:0/1117395682 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe052000700 1 -- 172.21.15.117:0/1117395682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe054100e10 msgr2=0x7fe05419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe052a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe054100e10 0x7fe05419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe052000700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe054100e10 0x7fe05419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe052000700 1 -- 172.21.15.117:0/1117395682 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0400095f0 con 0x7fe0541017c0 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.652+0000 7fe052a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe054100e10 0x7fe05419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:13:26.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.653+0000 7fe052000700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 0x7fe05419eee0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe040005b40 tx=0x7fe04000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:26.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.653+0000 7fe04b400700 1 -- 172.21.15.117:0/1117395682 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04001d070 con 0x7fe0541017c0 2024-04-20T11:13:26.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.653+0000 7fe04b400700 1 -- 172.21.15.117:0/1117395682 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe04000f460 con 0x7fe0541017c0 2024-04-20T11:13:26.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.653+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe054198dc0 con 0x7fe0541017c0 2024-04-20T11:13:26.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.653+0000 7fe04b400700 1 -- 172.21.15.117:0/1117395682 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe040017630 con 0x7fe0541017c0 2024-04-20T11:13:26.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.653+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0541992b0 con 0x7fe0541017c0 2024-04-20T11:13:26.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.654+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe054103f80 con 0x7fe0541017c0 2024-04-20T11:13:26.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.656+0000 7fe04b400700 1 -- 172.21.15.117:0/1117395682 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7fe040003a60 con 0x7fe0541017c0 2024-04-20T11:13:26.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.657+0000 7fe04b400700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fe03806e940 0x7fe038070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:26.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.657+0000 7fe04b400700 1 -- 172.21.15.117:0/1117395682 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7fe04008fd00 con 0x7fe0541017c0 2024-04-20T11:13:26.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.658+0000 7fe052a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fe03806e940 0x7fe038070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.658+0000 7fe052a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fe03806e940 0x7fe038070e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe044005fd0 tx=0x7fe044005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: Upgrade: Target is version 19.0.0-3133-ga8c84f59 (squid) 2024-04-20T11:13:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, digests ['quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1'] 2024-04-20T11:13:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:13:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: Upgrade: Setting container_image for all mgr 2024-04-20T11:13:26.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:26.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-20T11:13:26.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:26.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:26 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3307903433' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:13:26.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.662+0000 7fe04b400700 1 -- 172.21.15.117:0/1117395682 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7fe040093030 con 0x7fe0541017c0 2024-04-20T11:13:26.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.900+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 --> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe05410ddd0 con 0x7fe03806e940 2024-04-20T11:13:26.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.903+0000 7fe04b400700 1 -- 172.21.15.117:0/1117395682 <== mgr.14660 v2:172.21.15.117:6800/3003776987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe05410ddd0 con 0x7fe03806e940 2024-04-20T11:13:26.904 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:13:26.904 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:13:26.904 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:13:26.905 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:13:26.905 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:13:26.905 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:13:26.905 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:13:26.905 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "2/23 daemons upgraded", 2024-04-20T11:13:26.905 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mon daemons", 2024-04-20T11:13:26.905 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:13:26.905 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:13:26.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fe03806e940 msgr2=0x7fe038070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fe03806e940 0x7fe038070e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe044005fd0 tx=0x7fe044005dc0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 msgr2=0x7fe05419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:26.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 0x7fe05419eee0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe040005b40 tx=0x7fe04000be20 comp rx=0 tx=0).stop 2024-04-20T11:13:26.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 shutdown_connections 2024-04-20T11:13:26.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7fe03806e940 0x7fe038070e00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe054100e10 0x7fe05419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 --2- 172.21.15.117:0/1117395682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0541017c0 0x7fe05419eee0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:26.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.906+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 >> 172.21.15.117:0/1117395682 conn(0x7fe0540fc6c0 msgr2=0x7fe0540fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:26.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.907+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 shutdown_connections 2024-04-20T11:13:26.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:26.907+0000 7fe059a00700 1 -- 172.21.15.117:0/1117395682 wait complete. 2024-04-20T11:13:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.053+0000 7f91dae00700 1 -- 172.21.15.117:0/1047001866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 msgr2=0x7f91d410cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.053+0000 7f91dae00700 1 --2- 172.21.15.117:0/1047001866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 0x7f91d410cd00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009910 tx=0x7f91c0009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.053+0000 7f91dae00700 1 -- 172.21.15.117:0/1047001866 shutdown_connections 2024-04-20T11:13:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.053+0000 7f91dae00700 1 --2- 172.21.15.117:0/1047001866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 0x7f91d410cd00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.053+0000 7f91dae00700 1 --2- 172.21.15.117:0/1047001866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91d4073310 0x7f91d40736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.053+0000 7f91dae00700 1 -- 172.21.15.117:0/1047001866 >> 172.21.15.117:0/1047001866 conn(0x7f91d40fc510 msgr2=0x7f91d40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.053+0000 7f91dae00700 1 -- 172.21.15.117:0/1047001866 shutdown_connections 2024-04-20T11:13:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.053+0000 7f91dae00700 1 -- 172.21.15.117:0/1047001866 wait complete. 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.054+0000 7f91dae00700 1 Processor -- start 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.054+0000 7f91dae00700 1 -- start start 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.054+0000 7f91dae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91d4073310 0x7f91d419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.054+0000 7f91dae00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 0x7f91d419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.054+0000 7f91dae00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d419f4c0 con 0x7f91d4073c30 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.054+0000 7f91dae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d4198aa0 con 0x7f91d4073310 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.055+0000 7f91d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 0x7f91d419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.055+0000 7f91d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 0x7f91d419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59306/0 (socket says 172.21.15.117:59306) 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.055+0000 7f91d3400700 1 -- 172.21.15.117:0/66389548 learned_addr learned my addr 172.21.15.117:0/66389548 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.055+0000 7f91d3400700 1 -- 172.21.15.117:0/66389548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91d4073310 msgr2=0x7f91d419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.055+0000 7f91d3400700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91d4073310 0x7f91d419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:27.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.055+0000 7f91d3400700 1 -- 172.21.15.117:0/66389548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00095f0 con 0x7f91d4073c30 2024-04-20T11:13:27.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.055+0000 7f91d3400700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 0x7f91d419eea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f91c000bd70 tx=0x7f91c000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:27.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.056+0000 7f91d0c00700 1 -- 172.21.15.117:0/66389548 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91c001d070 con 0x7f91d4073c30 2024-04-20T11:13:27.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.056+0000 7f91d0c00700 1 -- 172.21.15.117:0/66389548 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91c000f460 con 0x7f91d4073c30 2024-04-20T11:13:27.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.056+0000 7f91d0c00700 1 -- 172.21.15.117:0/66389548 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91c0017630 con 0x7f91d4073c30 2024-04-20T11:13:27.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.056+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d4198d20 con 0x7f91d4073c30 2024-04-20T11:13:27.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.056+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d4199210 con 0x7f91d4073c30 2024-04-20T11:13:27.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.057+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d4107680 con 0x7f91d4073c30 2024-04-20T11:13:27.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.058+0000 7f91d0c00700 1 -- 172.21.15.117:0/66389548 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 34) v1 ==== 92608+0+0 (secure 0 0 0) 0x7f91c000f5d0 con 0x7f91d4073c30 2024-04-20T11:13:27.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.059+0000 7f91d0c00700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f91b806e940 0x7f91b8070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:27.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.059+0000 7f91d0c00700 1 -- 172.21.15.117:0/66389548 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6027+0+0 (secure 0 0 0) 0x7f91c008fd60 con 0x7f91d4073c30 2024-04-20T11:13:27.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.060+0000 7f91d3e00700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f91b806e940 0x7f91b8070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:27.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.060+0000 7f91d3e00700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f91b806e940 0x7f91b8070e00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91c40097b0 tx=0x7f91c4006d20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:27.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.064+0000 7f91d0c00700 1 -- 172.21.15.117:0/66389548 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188226 (secure 0 0 0) 0x7f91c0094070 con 0x7f91d4073c30 2024-04-20T11:13:27.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.375+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f91d4199f60 con 0x7f91d4073c30 2024-04-20T11:13:27.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.376+0000 7f91d0c00700 1 -- 172.21.15.117:0/66389548 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f91c005cb80 con 0x7f91d4073c30 2024-04-20T11:13:27.377 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:13:27.377 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:13:27.377 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:13:27.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.378+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f91b806e940 msgr2=0x7f91b8070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:27.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.378+0000 7f91dae00700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f91b806e940 0x7f91b8070e00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91c40097b0 tx=0x7f91c4006d20 comp rx=0 tx=0).stop 2024-04-20T11:13:27.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 msgr2=0x7f91d419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:27.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 0x7f91d419eea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f91c000bd70 tx=0x7f91c000be50 comp rx=0 tx=0).stop 2024-04-20T11:13:27.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 shutdown_connections 2024-04-20T11:13:27.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:6800/3003776987,v1:172.21.15.117:6801/3003776987] conn(0x7f91b806e940 0x7f91b8070e00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:27.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91d4073310 0x7f91d419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:27.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 --2- 172.21.15.117:0/66389548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4073c30 0x7f91d419eea0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:27.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 >> 172.21.15.117:0/66389548 conn(0x7f91d40fc510 msgr2=0x7f91d41051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:27.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 shutdown_connections 2024-04-20T11:13:27.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:27.379+0000 7f91dae00700 1 -- 172.21.15.117:0/66389548 wait complete. 2024-04-20T11:13:27.529 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:27 smithi117 ceph-mon[15659]: pgmap v143: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:27.529 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:27 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/2028496698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:13:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:27 smithi173 ceph-mon[23800]: pgmap v143: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:27 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2028496698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:13:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:28 smithi117 ceph-mon[15659]: from='client.14884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:28 smithi117 ceph-mon[15659]: from='client.? 172.21.15.117:0/66389548' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:13:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:28 smithi117 ceph-mon[15659]: Upgrade: Updating mon.smithi117 2024-04-20T11:13:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:28 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:28 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:28 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:28 smithi117 ceph-mon[15659]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:28 smithi117 ceph-mon[15659]: Deploying daemon mon.smithi117 on smithi117 2024-04-20T11:13:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:28 smithi173 ceph-mon[23800]: from='client.14884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:28 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/66389548' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:13:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:28 smithi173 ceph-mon[23800]: Upgrade: Updating mon.smithi117 2024-04-20T11:13:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:28 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:28.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:28 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:28.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:28 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:28.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:28 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:28.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:28 smithi173 ceph-mon[23800]: Deploying daemon mon.smithi117 on smithi117 2024-04-20T11:13:29.863 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:29 smithi117 ceph-mon[15659]: pgmap v144: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:29 smithi173 ceph-mon[23800]: pgmap v144: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:30 smithi117 systemd[1]: Stopping Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:13:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:30 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[15655]: 2024-04-20T11:13:30.844+0000 7fcd09200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi117 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:13:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:30 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[15655]: 2024-04-20T11:13:30.844+0000 7fcd09200700 -1 mon.smithi117@0(leader) e2 *** Got Signal Terminated *** 2024-04-20T11:13:31.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:30 smithi117 podman[98862]: 2024-04-20 11:13:30.907631049 +0000 UTC m=+0.247893470 container died a3ca404fd50f2d067d2fd255bb6cccfc043c4d93c9dc6fcbd68ffd06fab9bd75 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=reef-646b8c7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:13:31.625 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:31 smithi117 podman[98862]: 2024-04-20 11:13:31.399217145 +0000 UTC m=+0.739479588 container cleanup a3ca404fd50f2d067d2fd255bb6cccfc043c4d93c9dc6fcbd68ffd06fab9bd75 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-646b8c7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-20T11:13:31.625 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:31 smithi117 bash[98862]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117 2024-04-20T11:13:31.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:31 smithi117 podman[98887]: 2024-04-20 11:13:31.625201224 +0000 UTC m=+0.706382651 container remove a3ca404fd50f2d067d2fd255bb6cccfc043c4d93c9dc6fcbd68ffd06fab9bd75 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=reef-646b8c7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T11:13:32.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:32 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service: Deactivated successfully. 2024-04-20T11:13:32.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:32 smithi117 systemd[1]: Stopped Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:13:32.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:32 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service: Consumed 31.099s CPU time. 2024-04-20T11:13:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:32 smithi117 systemd[1]: Starting Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:13:33.206 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:32 smithi117 podman[99017]: 2024-04-20 11:13:32.813267394 +0000 UTC m=+0.040038134 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:13:33.206 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:32 smithi117 podman[99017]: 2024-04-20 11:13:32.95182001 +0000 UTC m=+0.178590841 container create ec2a5c1b3288fdfd9027bd7a90dadf1878982e5c340a46661df012b1660c7540 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-20T11:13:33.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 podman[99017]: 2024-04-20 11:13:33.206054094 +0000 UTC m=+0.432824871 container init ec2a5c1b3288fdfd9027bd7a90dadf1878982e5c340a46661df012b1660c7540 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-20T11:13:33.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 podman[99017]: 2024-04-20 11:13:33.217880311 +0000 UTC m=+0.444651105 container start ec2a5c1b3288fdfd9027bd7a90dadf1878982e5c340a46661df012b1660c7540 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-20T11:13:33.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T11:13:33.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-20T11:13:33.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: pidfile_write: ignore empty --pid-file 2024-04-20T11:13:33.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: load: jerasure load: lrc 2024-04-20T11:13:33.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: RocksDB version: 7.9.2 2024-04-20T11:13:33.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Git sha 0 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: DB SUMMARY 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: DB Session ID: NB1PLJNM4R2X22SCZLU6 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: CURRENT file: CURRENT 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1497 Bytes 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi117/store.db dir, Total Num: 1, files: 000032.sst 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi117/store.db: 000030.log size: 5484638 ; 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.error_if_exists: 0 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.create_if_missing: 0 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.paranoid_checks: 1 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T11:13:33.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T11:13:33.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T11:13:33.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.env: 0x56083e6d0cc0 2024-04-20T11:13:33.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T11:13:33.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.info_log: 0x560845c1bb00 2024-04-20T11:13:33.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T11:13:33.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.statistics: (nil) 2024-04-20T11:13:33.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.use_fsync: 0 2024-04-20T11:13:33.644 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_log_file_size: 0 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.allow_fallocate: 1 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.use_direct_reads: 0 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T11:13:33.645 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.db_log_dir: 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.wal_dir: 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.write_buffer_manager: 0x560845bea1e0 2024-04-20T11:13:33.646 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.unordered_write: 0 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T11:13:33.647 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T11:13:33.648 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.row_cache: None 2024-04-20T11:13:33.648 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.wal_filter: None 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.two_write_queues: 0 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.wal_compression: 0 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.atomic_flush: 0 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T11:13:33.649 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.log_readahead_size: 0 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T11:13:33.650 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_background_jobs: 2 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_background_compactions: -1 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_subcompactions: 1 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T11:13:33.651 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_open_files: -1 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_background_flushes: -1 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Compression algorithms supported: 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: kZSTD supported: 0 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: kXpressCompression supported: 0 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: kZlibCompression supported: 1 2024-04-20T11:13:33.652 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: kSnappyCompression supported: 1 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: kLZ4Compression supported: 1 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: kBZip2Compression supported: 0 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000015 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.merge_operator: 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_filter: None 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T11:13:33.653 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560845c1bce0) 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_type: 0 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_index_type: 0 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_shortening: 1 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: checksum: 4 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: no_block_cache: 0 2024-04-20T11:13:33.654 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache: 0x560845c1d090 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_options: 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: capacity : 536870912 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_shard_bits : 4 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: strict_capacity_limit : 0 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_cache_compressed: (nil) 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: persistent_cache: (nil) 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size: 4096 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_size_deviation: 10 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_restart_interval: 16 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: index_block_restart_interval: 1 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: metadata_block_size: 4096 2024-04-20T11:13:33.655 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: partition_filters: 0 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: use_delta_encoding: 1 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: filter_policy: bloomfilter 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: whole_key_filtering: 1 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: verify_compression: 0 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: format_version: 5 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_index_compression: 1 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: block_align: 0 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_auto_readahead_size: 262144 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: prepopulate_block_cache: 0 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: initial_auto_readahead_size: 8192 2024-04-20T11:13:33.656 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression: NoCompression 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.num_levels: 7 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T11:13:33.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T11:13:33.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T11:13:33.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T11:13:33.660 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.inplace_update_support: 0 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T11:13:33.661 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.bloom_locality: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.max_successive_merges: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.ttl: 2592000 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.enable_blob_files: false 2024-04-20T11:13:33.662 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.min_blob_size: 0 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi117/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 11784, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-04-20T11:13:33.663 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5ef1c280-e2bc-4bc7-b956-a528835c0edc 2024-04-20T11:13:33.664 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611613272042, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-04-20T11:13:33.664 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-04-20T11:13:33.664 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611613295374, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 4948242, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11785, "largest_seqno": 15645, "table_properties": {"data_size": 4931566, "index_size": 11149, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4485, "raw_key_size": 38690, "raw_average_key_size": 21, "raw_value_size": 4895567, "raw_average_value_size": 2759, "num_data_blocks": 504, "num_entries": 1774, "num_filter_entries": 1774, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713611613, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5ef1c280-e2bc-4bc7-b956-a528835c0edc", "db_session_id": "NB1PLJNM4R2X22SCZLU6", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-04-20T11:13:33.664 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611613295576, "job": 1, "event": "recovery_finished"} 2024-04-20T11:13:33.664 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi117/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560845d0a000 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: DB pointer 0x560845cf4000 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** DB Stats ** 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T11:13:33.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: L0 1/0 4.72 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 565.2 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: L6 1/0 5.47 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Sum 2/0 10.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 565.2 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 565.2 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-20T11:13:33.666 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** Compaction Stats [default] ** 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 565.2 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T11:13:33.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Cumulative compaction: 0.00 GB write, 162.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Interval compaction: 0.00 GB write, 162.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache BinnedLRUCache@0x560845c1d090#2 capacity: 512.00 MB usage: 16.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,12.08 KB,0.00230372%) Misc(1,0.00 KB,0%) 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: starting mon.smithi117 rank 0 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi117 fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117@-1(???) e2 preinit fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117@-1(???).mds e12 new map 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117@-1(???).mds e12 print_map 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: e12 2024-04-20T11:13:33.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: legacy client fscid: 1 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Filesystem 'cephfs' (1) 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: fs_name cephfs 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: epoch 12 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: tableserver 0 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: root 0 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: session_timeout 60 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: session_autoclose 300 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_file_size 1099511627776 2024-04-20T11:13:33.669 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_xattr_size 65536 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: required_client_features {} 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: last_failure 0 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: last_failure_osd_epoch 0 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: snap_rstat 1 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: max_mds 2 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: in 0,1 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: up {0=14460,1=24253} 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: failed 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: damaged 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: stopped 2024-04-20T11:13:33.670 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: data_pools [3] 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: metadata_pool 2 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: inline_data enabled 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: balancer 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: bal_rank_mask -1 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: standby_count_wanted 1 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: qdb_cluster leader: 0 members: 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: Standby daemons: 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:33.671 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout: [mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:33.672 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-20T11:13:33.672 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-20T11:13:33.672 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-20T11:13:33.672 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-20T11:13:33.672 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-04-20T11:13:33.672 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 bash[99017]: ec2a5c1b3288fdfd9027bd7a90dadf1878982e5c340a46661df012b1660c7540 2024-04-20T11:13:33.672 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 systemd[1]: Started Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: pgmap v146: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117 calling monitor election 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mon.smithi117 is new leader, mons smithi117,smithi173 in quorum (ranks 0,1) 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: monmap epoch 2 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: last_changed 2024-04-20T10:58:53.800023+0000 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: created 2024-04-20T10:55:27.473649+0000 2024-04-20T11:13:34.278 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: min_mon_release 18 (reef) 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: election_strategy: 1 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.smithi117 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.smithi173 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: osdmap e49: 6 total, 6 up, 6 in 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mgrmap e34: smithi117.ivnklj(active, since 4m), standbys: smithi173.gnrjof 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: from='mgr.14660 ' entity='' 2024-04-20T11:13:34.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:33 smithi117 ceph-mon[99036]: mgrmap e35: smithi117.ivnklj(active, since 4m), standbys: smithi173.gnrjof 2024-04-20T11:13:34.292 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:33 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:13:34.292 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:33 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:13:34.292 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: from='mgr.14660 172.21.15.117:0/3138879417' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: pgmap v146: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: mon.smithi117 calling monitor election 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: mon.smithi117 is new leader, mons smithi117,smithi173 in quorum (ranks 0,1) 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: monmap epoch 2 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: last_changed 2024-04-20T10:58:53.800023+0000 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: created 2024-04-20T10:55:27.473649+0000 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: min_mon_release 18 (reef) 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: election_strategy: 1 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.smithi117 2024-04-20T11:13:34.293 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.smithi173 2024-04-20T11:13:34.294 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:13:34.294 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: osdmap e49: 6 total, 6 up, 6 in 2024-04-20T11:13:34.294 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: mgrmap e34: smithi117.ivnklj(active, since 4m), standbys: smithi173.gnrjof 2024-04-20T11:13:34.294 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:13:34.294 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:13:34.294 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:13:34.294 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: from='mgr.14660 ' entity='' 2024-04-20T11:13:34.295 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:34 smithi173 ceph-mon[23800]: mgrmap e35: smithi117.ivnklj(active, since 4m), standbys: smithi173.gnrjof 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: Active manager daemon smithi117.ivnklj restarted 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: Activating manager daemon smithi117.ivnklj 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: osdmap e50: 6 total, 6 up, 6 in 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: mgrmap e36: smithi117.ivnklj(active, starting, since 0.0129165s), standbys: smithi173.gnrjof 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: Standby manager daemon smithi173.gnrjof restarted 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: Standby manager daemon smithi173.gnrjof started 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:13:43.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi173.gnrjof", "id": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.? 172.21.15.173:0/2520268859' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.? 172.21.15.173:0/2520268859' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:13:43.142 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.? 172.21.15.173:0/2520268859' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T11:13:43.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: from='mgr.? 172.21.15.173:0/2520268859' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:13:43.143 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:42 smithi117 ceph-mon[99036]: Manager daemon smithi117.ivnklj is now available 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: Active manager daemon smithi117.ivnklj restarted 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: Activating manager daemon smithi117.ivnklj 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: osdmap e50: 6 total, 6 up, 6 in 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: mgrmap e36: smithi117.ivnklj(active, starting, since 0.0129165s), standbys: smithi173.gnrjof 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: Standby manager daemon smithi173.gnrjof restarted 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: Standby manager daemon smithi173.gnrjof started 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi117.ivnklj", "id": "smithi117.ivnklj"}]: dispatch 2024-04-20T11:13:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr metadata", "who": "smithi173.gnrjof", "id": "smithi173.gnrjof"}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/2520268859' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/crt"}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/2520268859' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/2520268859' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi173.gnrjof/key"}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: from='mgr.? 172.21.15.173:0/2520268859' entity='mgr.smithi173.gnrjof' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:13:43.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:42 smithi173 ceph-mon[23800]: Manager daemon smithi117.ivnklj is now available 2024-04-20T11:13:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:13:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/trash_purge_schedule"}]: dispatch 2024-04-20T11:13:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:43 smithi117 ceph-mon[99036]: mgrmap e37: smithi117.ivnklj(active, since 1.03193s), standbys: smithi173.gnrjof 2024-04-20T11:13:44.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:43 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:43 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:43 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:13:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:43 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi117.ivnklj/trash_purge_schedule"}]: dispatch 2024-04-20T11:13:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:43 smithi173 ceph-mon[23800]: mgrmap e37: smithi117.ivnklj(active, since 1.03193s), standbys: smithi173.gnrjof 2024-04-20T11:13:45.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:44 smithi117 ceph-mon[99036]: [20/Apr/2024:11:13:43] ENGINE Bus STARTING 2024-04-20T11:13:45.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:44 smithi117 ceph-mon[99036]: [20/Apr/2024:11:13:43] ENGINE Serving on http://172.21.15.117:8765 2024-04-20T11:13:45.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:44 smithi117 ceph-mon[99036]: [20/Apr/2024:11:13:43] ENGINE Serving on https://172.21.15.117:7150 2024-04-20T11:13:45.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:44 smithi117 ceph-mon[99036]: [20/Apr/2024:11:13:43] ENGINE Bus STARTED 2024-04-20T11:13:45.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:44 smithi117 ceph-mon[99036]: pgmap v3: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:44 smithi173 ceph-mon[23800]: [20/Apr/2024:11:13:43] ENGINE Bus STARTING 2024-04-20T11:13:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:44 smithi173 ceph-mon[23800]: [20/Apr/2024:11:13:43] ENGINE Serving on http://172.21.15.117:8765 2024-04-20T11:13:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:44 smithi173 ceph-mon[23800]: [20/Apr/2024:11:13:43] ENGINE Serving on https://172.21.15.117:7150 2024-04-20T11:13:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:44 smithi173 ceph-mon[23800]: [20/Apr/2024:11:13:43] ENGINE Bus STARTED 2024-04-20T11:13:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:44 smithi173 ceph-mon[23800]: pgmap v3: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:46.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:45 smithi117 ceph-mon[99036]: pgmap v4: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:45 smithi117 ceph-mon[99036]: mgrmap e38: smithi117.ivnklj(active, since 2s), standbys: smithi173.gnrjof 2024-04-20T11:13:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:45 smithi173 ceph-mon[23800]: pgmap v4: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:45 smithi173 ceph-mon[23800]: mgrmap e38: smithi117.ivnklj(active, since 2s), standbys: smithi173.gnrjof 2024-04-20T11:13:48.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:47 smithi117 ceph-mon[99036]: pgmap v5: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:48.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:47 smithi117 ceph-mon[99036]: mgrmap e39: smithi117.ivnklj(active, since 4s), standbys: smithi173.gnrjof 2024-04-20T11:13:48.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:47 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:48.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:47 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:47 smithi173 ceph-mon[23800]: pgmap v5: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:47 smithi173 ceph-mon[23800]: mgrmap e39: smithi117.ivnklj(active, since 4s), standbys: smithi173.gnrjof 2024-04-20T11:13:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:47 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:47 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:50.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:49 smithi117 ceph-mon[99036]: pgmap v6: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:50.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:50.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:49 smithi173 ceph-mon[23800]: pgmap v6: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:13:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:49 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:49 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:51 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:51 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:51 smithi173 ceph-mon[23800]: pgmap v7: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-20T11:13:52.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:51 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:52.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:51 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:52.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:51 smithi117 ceph-mon[99036]: pgmap v7: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-20T11:13:53.901 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:53 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:53.902 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:53 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:53.902 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:53 smithi117 ceph-mon[99036]: pgmap v8: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-20T11:13:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:53 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:53 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:53 smithi173 ceph-mon[23800]: pgmap v8: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-20T11:13:56.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:55 smithi117 ceph-mon[99036]: pgmap v9: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:13:56.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:56.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:56.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:55 smithi173 ceph-mon[23800]: pgmap v9: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:13:56.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:55 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:56.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:55 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:57.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.547+0000 7f0353a00700 1 -- 172.21.15.117:0/1600083687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c1017c0 msgr2=0x7f034c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:57.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.547+0000 7f0353a00700 1 --2- 172.21.15.117:0/1600083687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c1017c0 0x7f034c105810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0338009910 tx=0x7f0338009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:57.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.547+0000 7f0353a00700 1 -- 172.21.15.117:0/1600083687 shutdown_connections 2024-04-20T11:13:57.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.547+0000 7f0353a00700 1 --2- 172.21.15.117:0/1600083687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c1017c0 0x7f034c105810 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.547+0000 7f0353a00700 1 --2- 172.21.15.117:0/1600083687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c100e10 0x7f034c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.547+0000 7f0353a00700 1 -- 172.21.15.117:0/1600083687 >> 172.21.15.117:0/1600083687 conn(0x7f034c0fc6c0 msgr2=0x7f034c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:57.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.548+0000 7f0353a00700 1 -- 172.21.15.117:0/1600083687 shutdown_connections 2024-04-20T11:13:57.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.548+0000 7f0353a00700 1 -- 172.21.15.117:0/1600083687 wait complete. 2024-04-20T11:13:57.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.549+0000 7f0353a00700 1 Processor -- start 2024-04-20T11:13:57.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.549+0000 7f0353a00700 1 -- start start 2024-04-20T11:13:57.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.549+0000 7f0353a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c100e10 0x7f034c0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:57.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.549+0000 7f0353a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c1017c0 0x7f034c073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:57.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.549+0000 7f0353a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c073c60 con 0x7f034c100e10 2024-04-20T11:13:57.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.549+0000 7f0353a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c073da0 con 0x7f034c1017c0 2024-04-20T11:13:57.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.550+0000 7f0351000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c100e10 0x7f034c0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:57.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.550+0000 7f0351000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c100e10 0x7f034c0731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37762/0 (socket says 172.21.15.117:37762) 2024-04-20T11:13:57.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.550+0000 7f0351000700 1 -- 172.21.15.117:0/1262367634 learned_addr learned my addr 172.21.15.117:0/1262367634 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.550+0000 7f034be00700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c1017c0 0x7f034c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.550+0000 7f0351000700 1 -- 172.21.15.117:0/1262367634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c1017c0 msgr2=0x7f034c073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.550+0000 7f0351000700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c1017c0 0x7f034c073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.550+0000 7f0351000700 1 -- 172.21.15.117:0/1262367634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380095f0 con 0x7f034c100e10 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.551+0000 7f0351000700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c100e10 0x7f034c0731e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f034000c8f0 tx=0x7f034000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.551+0000 7f0349600700 1 -- 172.21.15.117:0/1262367634 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0340007a70 con 0x7f034c100e10 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.551+0000 7f0349600700 1 -- 172.21.15.117:0/1262367634 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0340004cf0 con 0x7f034c100e10 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.551+0000 7f0349600700 1 -- 172.21.15.117:0/1262367634 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0340005610 con 0x7f034c100e10 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.551+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c1a2d10 con 0x7f034c100e10 2024-04-20T11:13:57.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.551+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c1a3070 con 0x7f034c100e10 2024-04-20T11:13:57.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.552+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c109210 con 0x7f034c100e10 2024-04-20T11:13:57.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.553+0000 7f0349600700 1 -- 172.21.15.117:0/1262367634 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f0340004750 con 0x7f034c100e10 2024-04-20T11:13:57.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.555+0000 7f0349600700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f033406ee00 0x7f03340712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:57.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.555+0000 7f0349600700 1 -- 172.21.15.117:0/1262367634 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f0340013070 con 0x7f034c100e10 2024-04-20T11:13:57.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.555+0000 7f034be00700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f033406ee00 0x7f03340712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:57.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.556+0000 7f034be00700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f033406ee00 0x7f03340712c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f03380059d0 tx=0x7f0338005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:57.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.562+0000 7f0349600700 1 -- 172.21.15.117:0/1262367634 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0340057b00 con 0x7f034c100e10 2024-04-20T11:13:57.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.809+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f034c10ddd0 con 0x7f033406ee00 2024-04-20T11:13:57.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.812+0000 7f0349600700 1 -- 172.21.15.117:0/1262367634 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f034c10ddd0 con 0x7f033406ee00 2024-04-20T11:13:57.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.816+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f033406ee00 msgr2=0x7f03340712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:57.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.816+0000 7f0353a00700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f033406ee00 0x7f03340712c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f03380059d0 tx=0x7f0338005960 comp rx=0 tx=0).stop 2024-04-20T11:13:57.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.816+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c100e10 msgr2=0x7f034c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:57.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.816+0000 7f0353a00700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c100e10 0x7f034c0731e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f034000c8f0 tx=0x7f034000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.817+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 shutdown_connections 2024-04-20T11:13:57.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.817+0000 7f0353a00700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f033406ee00 0x7f03340712c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.817+0000 7f0353a00700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c100e10 0x7f034c0731e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.817+0000 7f0353a00700 1 --2- 172.21.15.117:0/1262367634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c1017c0 0x7f034c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.817+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 >> 172.21.15.117:0/1262367634 conn(0x7f034c0fc6c0 msgr2=0x7f034c0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:57.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.817+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 shutdown_connections 2024-04-20T11:13:57.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.818+0000 7f0353a00700 1 -- 172.21.15.117:0/1262367634 wait complete. 2024-04-20T11:13:57.834 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:13:57.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.973+0000 7f1f7e200700 1 -- 172.21.15.117:0/3312623990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f780ffa80 msgr2=0x7f1f780fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:57.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.973+0000 7f1f7e200700 1 --2- 172.21.15.117:0/3312623990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f780ffa80 0x7f1f780fff00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009910 tx=0x7f1f64009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:57.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.973+0000 7f1f7e200700 1 -- 172.21.15.117:0/3312623990 shutdown_connections 2024-04-20T11:13:57.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.973+0000 7f1f7e200700 1 --2- 172.21.15.117:0/3312623990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f780ffa80 0x7f1f780fff00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.973+0000 7f1f7e200700 1 --2- 172.21.15.117:0/3312623990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f78105a80 0x7f1f78105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.973+0000 7f1f7e200700 1 -- 172.21.15.117:0/3312623990 >> 172.21.15.117:0/3312623990 conn(0x7f1f78076750 msgr2=0x7f1f780fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:57.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.974+0000 7f1f7e200700 1 -- 172.21.15.117:0/3312623990 shutdown_connections 2024-04-20T11:13:57.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.974+0000 7f1f7e200700 1 -- 172.21.15.117:0/3312623990 wait complete. 2024-04-20T11:13:57.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.975+0000 7f1f7e200700 1 Processor -- start 2024-04-20T11:13:57.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.975+0000 7f1f7e200700 1 -- start start 2024-04-20T11:13:57.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.975+0000 7f1f7e200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f780ffa80 0x7f1f780fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:57.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.975+0000 7f1f7e200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f78105a80 0x7f1f780fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:57.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.975+0000 7f1f7e200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780fc5c0 con 0x7f1f78105a80 2024-04-20T11:13:57.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.975+0000 7f1f7e200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780fc700 con 0x7f1f780ffa80 2024-04-20T11:13:57.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.976+0000 7f1f76a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f78105a80 0x7f1f780fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:57.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.976+0000 7f1f76a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f78105a80 0x7f1f780fc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37782/0 (socket says 172.21.15.117:37782) 2024-04-20T11:13:57.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.976+0000 7f1f76a00700 1 -- 172.21.15.117:0/2020495843 learned_addr learned my addr 172.21.15.117:0/2020495843 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:57.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.976+0000 7f1f77400700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f780ffa80 0x7f1f780fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:57.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.976+0000 7f1f76a00700 1 -- 172.21.15.117:0/2020495843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f780ffa80 msgr2=0x7f1f780fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:57.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.976+0000 7f1f76a00700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f780ffa80 0x7f1f780fbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:57.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.976+0000 7f1f76a00700 1 -- 172.21.15.117:0/2020495843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640095f0 con 0x7f1f78105a80 2024-04-20T11:13:57.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.976+0000 7f1f76a00700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f78105a80 0x7f1f780fc080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f640092e0 tx=0x7f1f6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:57.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.977+0000 7f1f6fe00700 1 -- 172.21.15.117:0/2020495843 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6401d070 con 0x7f1f78105a80 2024-04-20T11:13:57.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.977+0000 7f1f6fe00700 1 -- 172.21.15.117:0/2020495843 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f6400f460 con 0x7f1f78105a80 2024-04-20T11:13:57.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.977+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f780fc980 con 0x7f1f78105a80 2024-04-20T11:13:57.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.977+0000 7f1f6fe00700 1 -- 172.21.15.117:0/2020495843 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f64017630 con 0x7f1f78105a80 2024-04-20T11:13:57.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.977+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f781a2fc0 con 0x7f1f78105a80 2024-04-20T11:13:57.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.978+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f781090b0 con 0x7f1f78105a80 2024-04-20T11:13:57.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.979+0000 7f1f6fe00700 1 -- 172.21.15.117:0/2020495843 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f1f6400f5d0 con 0x7f1f78105a80 2024-04-20T11:13:57.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.980+0000 7f1f6fe00700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f5c06edf0 0x7f1f5c0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:57.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.980+0000 7f1f6fe00700 1 -- 172.21.15.117:0/2020495843 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f1f640905e0 con 0x7f1f78105a80 2024-04-20T11:13:57.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.980+0000 7f1f77400700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f5c06edf0 0x7f1f5c0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:57.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.981+0000 7f1f77400700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f5c06edf0 0x7f1f5c0712b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f68005fd0 tx=0x7f1f6800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:57.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:57.985+0000 7f1f6fe00700 1 -- 172.21.15.117:0/2020495843 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1f640591b0 con 0x7f1f78105a80 2024-04-20T11:13:58.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:57 smithi117 ceph-mon[99036]: pgmap v10: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T11:13:58.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.237+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f780fd090 con 0x7f1f5c06edf0 2024-04-20T11:13:58.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.240+0000 7f1f6fe00700 1 -- 172.21.15.117:0/2020495843 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f1f780fd090 con 0x7f1f5c06edf0 2024-04-20T11:13:58.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.242+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f5c06edf0 msgr2=0x7f1f5c0712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.242+0000 7f1f7e200700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f5c06edf0 0x7f1f5c0712b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f68005fd0 tx=0x7f1f6800a560 comp rx=0 tx=0).stop 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.242+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f78105a80 msgr2=0x7f1f780fc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.242+0000 7f1f7e200700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f78105a80 0x7f1f780fc080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f640092e0 tx=0x7f1f6400be20 comp rx=0 tx=0).stop 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.243+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 shutdown_connections 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.243+0000 7f1f7e200700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f5c06edf0 0x7f1f5c0712b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.243+0000 7f1f7e200700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f780ffa80 0x7f1f780fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.243+0000 7f1f7e200700 1 --2- 172.21.15.117:0/2020495843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f78105a80 0x7f1f780fc080 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.243+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 >> 172.21.15.117:0/2020495843 conn(0x7f1f78076750 msgr2=0x7f1f78068b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.243+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 shutdown_connections 2024-04-20T11:13:58.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.243+0000 7f1f7e200700 1 -- 172.21.15.117:0/2020495843 wait complete. 2024-04-20T11:13:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:57 smithi173 ceph-mon[23800]: pgmap v10: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T11:13:58.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.414+0000 7f92ef600700 1 -- 172.21.15.117:0/1543960054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e81002e0 msgr2=0x7f92e810cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:58.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.414+0000 7f92ef600700 1 --2- 172.21.15.117:0/1543960054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e81002e0 0x7f92e810cfa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009910 tx=0x7f92d4009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:58.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.414+0000 7f92ef600700 1 -- 172.21.15.117:0/1543960054 shutdown_connections 2024-04-20T11:13:58.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.414+0000 7f92ef600700 1 --2- 172.21.15.117:0/1543960054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e81002e0 0x7f92e810cfa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.414+0000 7f92ef600700 1 --2- 172.21.15.117:0/1543960054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92e80ff930 0x7f92e80ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.414+0000 7f92ef600700 1 -- 172.21.15.117:0/1543960054 >> 172.21.15.117:0/1543960054 conn(0x7f92e8076750 msgr2=0x7f92e80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:58.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.415+0000 7f92ef600700 1 -- 172.21.15.117:0/1543960054 shutdown_connections 2024-04-20T11:13:58.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.415+0000 7f92ef600700 1 -- 172.21.15.117:0/1543960054 wait complete. 2024-04-20T11:13:58.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.416+0000 7f92ef600700 1 Processor -- start 2024-04-20T11:13:58.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.416+0000 7f92ef600700 1 -- start start 2024-04-20T11:13:58.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.417+0000 7f92ef600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e80ff930 0x7f92e8101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:58.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.417+0000 7f92ef600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92e81002e0 0x7f92e8102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:58.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.417+0000 7f92ef600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8105d50 con 0x7f92e80ff930 2024-04-20T11:13:58.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.417+0000 7f92ef600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8102690 con 0x7f92e81002e0 2024-04-20T11:13:58.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.417+0000 7f92ecc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e80ff930 0x7f92e8101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:58.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.417+0000 7f92ecc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e80ff930 0x7f92e8101c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37794/0 (socket says 172.21.15.117:37794) 2024-04-20T11:13:58.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.417+0000 7f92ecc00700 1 -- 172.21.15.117:0/3275554590 learned_addr learned my addr 172.21.15.117:0/3275554590 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:58.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.418+0000 7f92e7e00700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92e81002e0 0x7f92e8102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:58.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.418+0000 7f92ecc00700 1 -- 172.21.15.117:0/3275554590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92e81002e0 msgr2=0x7f92e8102150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:58.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.418+0000 7f92ecc00700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92e81002e0 0x7f92e8102150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.418+0000 7f92ecc00700 1 -- 172.21.15.117:0/3275554590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40095f0 con 0x7f92e80ff930 2024-04-20T11:13:58.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.418+0000 7f92ecc00700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e80ff930 0x7f92e8101c10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00d8d0 tx=0x7f92dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:58.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.419+0000 7f92e5600700 1 -- 172.21.15.117:0/3275554590 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92dc009940 con 0x7f92e80ff930 2024-04-20T11:13:58.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.419+0000 7f92e5600700 1 -- 172.21.15.117:0/3275554590 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92dc010460 con 0x7f92e80ff930 2024-04-20T11:13:58.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.419+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e8102970 con 0x7f92e80ff930 2024-04-20T11:13:58.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.419+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e81a7460 con 0x7f92e80ff930 2024-04-20T11:13:58.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.419+0000 7f92e5600700 1 -- 172.21.15.117:0/3275554590 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92dc00f5f0 con 0x7f92e80ff930 2024-04-20T11:13:58.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.420+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c80052f0 con 0x7f92e80ff930 2024-04-20T11:13:58.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.422+0000 7f92e5600700 1 -- 172.21.15.117:0/3275554590 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f92dc009aa0 con 0x7f92e80ff930 2024-04-20T11:13:58.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.423+0000 7f92e5600700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ee00 0x7f92d00712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:58.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.423+0000 7f92e5600700 1 -- 172.21.15.117:0/3275554590 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f92dc08efc0 con 0x7f92e80ff930 2024-04-20T11:13:58.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.424+0000 7f92e7e00700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ee00 0x7f92d00712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:58.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.424+0000 7f92e7e00700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ee00 0x7f92d00712c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f92e8103340 tx=0x7f92d4005ad0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:58.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.432+0000 7f92e5600700 1 -- 172.21.15.117:0/3275554590 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f92dc057c30 con 0x7f92e80ff930 2024-04-20T11:13:58.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.671+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92c8000bc0 con 0x7f92d006ee00 2024-04-20T11:13:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.684+0000 7f92e5600700 1 -- 172.21.15.117:0/3275554590 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f92c8000bc0 con 0x7f92d006ee00 2024-04-20T11:13:58.684 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:13:58.684 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (2m) 8s ago 16m 14.4M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:13:58.684 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (16m) 8s ago 16m 8388k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:13:58.684 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (15m) 11s ago 15m 8144k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:13:58.684 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (16m) 8s ago 16m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:13:58.684 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (15m) 11s ago 15m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (104s) 8s ago 16m 75.3M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (9m) 8s ago 9m 16.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (9m) 8s ago 9m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (9m) 11s ago 9m 16.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (9m) 11s ago 9m 24.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (5m) 8s ago 18m 479M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (4m) 11s ago 15m 448M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (25s) 8s ago 18m 30.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (15m) 11s ago 15m 52.8M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 243eed744f92 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (3m) 8s ago 16m 7402k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (3m) 11s ago 15m 7243k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (14m) 8s ago 14m 211M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (13m) 8s ago 13m 218M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:13:58.685 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (12m) 8s ago 12m 182M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:13:58.686 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (12m) 11s ago 12m 243M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:13:58.686 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (11m) 11s ago 11m 214M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:13:58.686 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (11m) 11s ago 11m 201M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:13:58.686 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (3m) 8s ago 15m 62.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:13:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ee00 msgr2=0x7f92d00712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ee00 0x7f92d00712c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f92e8103340 tx=0x7f92d4005ad0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e80ff930 msgr2=0x7f92e8101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e80ff930 0x7f92e8101c10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00d8d0 tx=0x7f92dc00dc90 comp rx=0 tx=0).stop 2024-04-20T11:13:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 shutdown_connections 2024-04-20T11:13:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ee00 0x7f92d00712c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92e80ff930 0x7f92e8101c10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 --2- 172.21.15.117:0/3275554590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92e81002e0 0x7f92e8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 >> 172.21.15.117:0/3275554590 conn(0x7f92e8076750 msgr2=0x7f92e80fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:58.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.687+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 shutdown_connections 2024-04-20T11:13:58.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.688+0000 7f92ef600700 1 -- 172.21.15.117:0/3275554590 wait complete. 2024-04-20T11:13:58.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.838+0000 7f37f1c00700 1 -- 172.21.15.117:0/4270088699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 msgr2=0x7f37ec1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:58.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.838+0000 7f37f1c00700 1 --2- 172.21.15.117:0/4270088699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 0x7f37ec1011f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009910 tx=0x7f37dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:58.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.838+0000 7f37f1c00700 1 -- 172.21.15.117:0/4270088699 shutdown_connections 2024-04-20T11:13:58.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.838+0000 7f37f1c00700 1 --2- 172.21.15.117:0/4270088699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ec1017c0 0x7f37ec105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.838+0000 7f37f1c00700 1 --2- 172.21.15.117:0/4270088699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 0x7f37ec1011f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.838+0000 7f37f1c00700 1 -- 172.21.15.117:0/4270088699 >> 172.21.15.117:0/4270088699 conn(0x7f37ec0fc6a0 msgr2=0x7f37ec0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:58.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.839+0000 7f37f1c00700 1 -- 172.21.15.117:0/4270088699 shutdown_connections 2024-04-20T11:13:58.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.839+0000 7f37f1c00700 1 -- 172.21.15.117:0/4270088699 wait complete. 2024-04-20T11:13:58.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.840+0000 7f37f1c00700 1 Processor -- start 2024-04-20T11:13:58.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.840+0000 7f37f1c00700 1 -- start start 2024-04-20T11:13:58.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.840+0000 7f37f1c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 0x7f37ec19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:58.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.840+0000 7f37f1c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ec1017c0 0x7f37ec19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:58.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.840+0000 7f37f1c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec19f5c0 con 0x7f37ec100e10 2024-04-20T11:13:58.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.840+0000 7f37f1c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec198bf0 con 0x7f37ec1017c0 2024-04-20T11:13:58.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.841+0000 7f37eaa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 0x7f37ec19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:58.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.841+0000 7f37eaa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 0x7f37ec19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37810/0 (socket says 172.21.15.117:37810) 2024-04-20T11:13:58.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.841+0000 7f37eaa00700 1 -- 172.21.15.117:0/4026930038 learned_addr learned my addr 172.21.15.117:0/4026930038 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:58.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.841+0000 7f37ea000700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ec1017c0 0x7f37ec19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:58.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.841+0000 7f37eaa00700 1 -- 172.21.15.117:0/4026930038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ec1017c0 msgr2=0x7f37ec19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:58.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.841+0000 7f37eaa00700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ec1017c0 0x7f37ec19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:58.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.841+0000 7f37eaa00700 1 -- 172.21.15.117:0/4026930038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc0095f0 con 0x7f37ec100e10 2024-04-20T11:13:58.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.841+0000 7f37eaa00700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 0x7f37ec19ea60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f37dc006010 tx=0x7f37dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:58.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.842+0000 7f37e3400700 1 -- 172.21.15.117:0/4026930038 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37dc01d070 con 0x7f37ec100e10 2024-04-20T11:13:58.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.842+0000 7f37e3400700 1 -- 172.21.15.117:0/4026930038 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37dc0056e0 con 0x7f37ec100e10 2024-04-20T11:13:58.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.842+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec198e70 con 0x7f37ec100e10 2024-04-20T11:13:58.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.842+0000 7f37e3400700 1 -- 172.21.15.117:0/4026930038 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37dc017400 con 0x7f37ec100e10 2024-04-20T11:13:58.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.842+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec199360 con 0x7f37ec100e10 2024-04-20T11:13:58.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.843+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec104f90 con 0x7f37ec100e10 2024-04-20T11:13:58.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.844+0000 7f37e3400700 1 -- 172.21.15.117:0/4026930038 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f37dc00f6f0 con 0x7f37ec100e10 2024-04-20T11:13:58.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.846+0000 7f37e3400700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f37d006ee00 0x7f37d00712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:58.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.846+0000 7f37e3400700 1 -- 172.21.15.117:0/4026930038 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f37dc0904a0 con 0x7f37ec100e10 2024-04-20T11:13:58.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.846+0000 7f37ea000700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f37d006ee00 0x7f37d00712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:58.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.847+0000 7f37ea000700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f37d006ee00 0x7f37d00712c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005950 tx=0x7f37d80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:58.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:58.851+0000 7f37e3400700 1 -- 172.21.15.117:0/4026930038 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f37dc05cba0 con 0x7f37ec100e10 2024-04-20T11:13:58.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:58 smithi117 ceph-mon[99036]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:58.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:58.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:58.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:58.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:58.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.199+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f37ec199ff0 con 0x7f37ec100e10 2024-04-20T11:13:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.200+0000 7f37e3400700 1 -- 172.21.15.117:0/4026930038 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f37dc05c730 con 0x7f37ec100e10 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:13:59.200 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:13:59.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:13:59.201 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:13:59.201 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 11, 2024-04-20T11:13:59.201 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:13:59.201 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:13:59.201 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:13:59.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.202+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f37d006ee00 msgr2=0x7f37d00712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:59.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.202+0000 7f37f1c00700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f37d006ee00 0x7f37d00712c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005950 tx=0x7f37d80058e0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 msgr2=0x7f37ec19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 0x7f37ec19ea60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f37dc006010 tx=0x7f37dc00be20 comp rx=0 tx=0).stop 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 shutdown_connections 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f37d006ee00 0x7f37d00712c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ec100e10 0x7f37ec19ea60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 --2- 172.21.15.117:0/4026930038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ec1017c0 0x7f37ec19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 >> 172.21.15.117:0/4026930038 conn(0x7f37ec0fc6a0 msgr2=0x7f37ec0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 shutdown_connections 2024-04-20T11:13:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.203+0000 7f37f1c00700 1 -- 172.21.15.117:0/4026930038 wait complete. 2024-04-20T11:13:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.357+0000 7f5d5dc00700 1 -- 172.21.15.117:0/1908996863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581017c0 msgr2=0x7f5d58105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.357+0000 7f5d5dc00700 1 --2- 172.21.15.117:0/1908996863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581017c0 0x7f5d58105810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009910 tx=0x7f5d44009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.357+0000 7f5d5dc00700 1 -- 172.21.15.117:0/1908996863 shutdown_connections 2024-04-20T11:13:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.357+0000 7f5d5dc00700 1 --2- 172.21.15.117:0/1908996863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581017c0 0x7f5d58105810 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.357+0000 7f5d5dc00700 1 --2- 172.21.15.117:0/1908996863 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d58100e10 0x7f5d581011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.357+0000 7f5d5dc00700 1 -- 172.21.15.117:0/1908996863 >> 172.21.15.117:0/1908996863 conn(0x7f5d580fc6c0 msgr2=0x7f5d580feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.357+0000 7f5d5dc00700 1 -- 172.21.15.117:0/1908996863 shutdown_connections 2024-04-20T11:13:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.357+0000 7f5d5dc00700 1 -- 172.21.15.117:0/1908996863 wait complete. 2024-04-20T11:13:59.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.358+0000 7f5d5dc00700 1 Processor -- start 2024-04-20T11:13:59.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.359+0000 7f5d5dc00700 1 -- start start 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.359+0000 7f5d5dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d58100e10 0x7f5d5819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.359+0000 7f5d5dc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d581017c0 0x7f5d5819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.359+0000 7f5d5dc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5819f550 con 0x7f5d58100e10 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.359+0000 7f5d5dc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d58198b80 con 0x7f5d581017c0 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.359+0000 7f5d56a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d58100e10 0x7f5d5819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.359+0000 7f5d56a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d58100e10 0x7f5d5819e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37832/0 (socket says 172.21.15.117:37832) 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.359+0000 7f5d56a00700 1 -- 172.21.15.117:0/3011508395 learned_addr learned my addr 172.21.15.117:0/3011508395 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.360+0000 7f5d56000700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d581017c0 0x7f5d5819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.360+0000 7f5d56a00700 1 -- 172.21.15.117:0/3011508395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d581017c0 msgr2=0x7f5d5819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.360+0000 7f5d56a00700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d581017c0 0x7f5d5819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.360+0000 7f5d56a00700 1 -- 172.21.15.117:0/3011508395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440095f0 con 0x7f5d58100e10 2024-04-20T11:13:59.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.361+0000 7f5d56a00700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d58100e10 0x7f5d5819e9f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800e920 tx=0x7f5d4800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:59.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.361+0000 7f5d4f400700 1 -- 172.21.15.117:0/3011508395 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d4800c9d0 con 0x7f5d58100e10 2024-04-20T11:13:59.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.361+0000 7f5d4f400700 1 -- 172.21.15.117:0/3011508395 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d48010430 con 0x7f5d58100e10 2024-04-20T11:13:59.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.361+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d58198e60 con 0x7f5d58100e10 2024-04-20T11:13:59.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.361+0000 7f5d4f400700 1 -- 172.21.15.117:0/3011508395 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d48018630 con 0x7f5d58100e10 2024-04-20T11:13:59.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.361+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d58199380 con 0x7f5d58100e10 2024-04-20T11:13:59.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.363+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d58103f80 con 0x7f5d58100e10 2024-04-20T11:13:59.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.364+0000 7f5d4f400700 1 -- 172.21.15.117:0/3011508395 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f5d480105a0 con 0x7f5d58100e10 2024-04-20T11:13:59.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.365+0000 7f5d4f400700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d3c06edf0 0x7f5d3c0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:59.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.365+0000 7f5d4f400700 1 -- 172.21.15.117:0/3011508395 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f5d48014070 con 0x7f5d58100e10 2024-04-20T11:13:59.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.366+0000 7f5d56000700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d3c06edf0 0x7f5d3c0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:59.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.367+0000 7f5d56000700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d3c06edf0 0x7f5d3c0712b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005b40 tx=0x7f5d44005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:59.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.370+0000 7f5d4f400700 1 -- 172.21.15.117:0/3011508395 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5d48053ff0 con 0x7f5d58100e10 2024-04-20T11:13:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:58 smithi173 ceph-mon[23800]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:58 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:13:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:58 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:58 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:13:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:58 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:58 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:59.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.650+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5d58199ff0 con 0x7f5d58100e10 2024-04-20T11:13:59.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.651+0000 7f5d4f400700 1 -- 172.21.15.117:0/3011508395 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7f5d4805ba30 con 0x7f5d58100e10 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:13:59.651 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:13:59.652 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:13:59.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d3c06edf0 msgr2=0x7f5d3c0712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:59.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d3c06edf0 0x7f5d3c0712b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005b40 tx=0x7f5d44005a90 comp rx=0 tx=0).stop 2024-04-20T11:13:59.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d58100e10 msgr2=0x7f5d5819e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:59.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d58100e10 0x7f5d5819e9f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800e920 tx=0x7f5d4800ec30 comp rx=0 tx=0).stop 2024-04-20T11:13:59.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 shutdown_connections 2024-04-20T11:13:59.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d3c06edf0 0x7f5d3c0712b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d58100e10 0x7f5d5819e9f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 --2- 172.21.15.117:0/3011508395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d581017c0 0x7f5d5819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.654+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 >> 172.21.15.117:0/3011508395 conn(0x7f5d580fc6c0 msgr2=0x7f5d580fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:59.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.655+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 shutdown_connections 2024-04-20T11:13:59.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.655+0000 7f5d5dc00700 1 -- 172.21.15.117:0/3011508395 wait complete. 2024-04-20T11:13:59.655 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:13:59.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.802+0000 7fb5ef600700 1 -- 172.21.15.117:0/1013475833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 msgr2=0x7fb5e8105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:59.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.802+0000 7fb5ef600700 1 --2- 172.21.15.117:0/1013475833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 0x7fb5e8105810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009910 tx=0x7fb5d4009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:59.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.802+0000 7fb5ef600700 1 -- 172.21.15.117:0/1013475833 shutdown_connections 2024-04-20T11:13:59.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.802+0000 7fb5ef600700 1 --2- 172.21.15.117:0/1013475833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 0x7fb5e8105810 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.802+0000 7fb5ef600700 1 --2- 172.21.15.117:0/1013475833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5e8100e10 0x7fb5e81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.802+0000 7fb5ef600700 1 -- 172.21.15.117:0/1013475833 >> 172.21.15.117:0/1013475833 conn(0x7fb5e80fc6c0 msgr2=0x7fb5e80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:59.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.802+0000 7fb5ef600700 1 -- 172.21.15.117:0/1013475833 shutdown_connections 2024-04-20T11:13:59.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.802+0000 7fb5ef600700 1 -- 172.21.15.117:0/1013475833 wait complete. 2024-04-20T11:13:59.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.803+0000 7fb5ef600700 1 Processor -- start 2024-04-20T11:13:59.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.803+0000 7fb5ef600700 1 -- start start 2024-04-20T11:13:59.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5ef600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5e8100e10 0x7fb5e819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:59.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5ef600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 0x7fb5e819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:59.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5ef600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e819f550 con 0x7fb5e81017c0 2024-04-20T11:13:59.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5ef600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8198b80 con 0x7fb5e8100e10 2024-04-20T11:13:59.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5e7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 0x7fb5e819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:59.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5e7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 0x7fb5e819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37856/0 (socket says 172.21.15.117:37856) 2024-04-20T11:13:59.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5e7e00700 1 -- 172.21.15.117:0/3859485008 learned_addr learned my addr 172.21.15.117:0/3859485008 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:13:59.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5ecc00700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5e8100e10 0x7fb5e819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5e7e00700 1 -- 172.21.15.117:0/3859485008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5e8100e10 msgr2=0x7fb5e819e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5e7e00700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5e8100e10 0x7fb5e819e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.804+0000 7fb5e7e00700 1 -- 172.21.15.117:0/3859485008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40095f0 con 0x7fb5e81017c0 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.805+0000 7fb5e7e00700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 0x7fb5e819ef30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4000c00 tx=0x7fb5d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.805+0000 7fb5e5600700 1 -- 172.21.15.117:0/3859485008 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5d401d070 con 0x7fb5e81017c0 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.805+0000 7fb5e5600700 1 -- 172.21.15.117:0/3859485008 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5d4005700 con 0x7fb5e81017c0 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.805+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e8198e00 con 0x7fb5e81017c0 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.805+0000 7fb5e5600700 1 -- 172.21.15.117:0/3859485008 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5d400fda0 con 0x7fb5e81017c0 2024-04-20T11:13:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.805+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e81992f0 con 0x7fb5e81017c0 2024-04-20T11:13:59.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.806+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e8103f80 con 0x7fb5e81017c0 2024-04-20T11:13:59.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.807+0000 7fb5e5600700 1 -- 172.21.15.117:0/3859485008 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fb5d400f5d0 con 0x7fb5e81017c0 2024-04-20T11:13:59.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.808+0000 7fb5e5600700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5d006edf0 0x7fb5d00712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:59.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.808+0000 7fb5e5600700 1 -- 172.21.15.117:0/3859485008 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fb5d4090380 con 0x7fb5e81017c0 2024-04-20T11:13:59.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.809+0000 7fb5ecc00700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5d006edf0 0x7fb5d00712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:59.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.809+0000 7fb5ecc00700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5d006edf0 0x7fb5d00712b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009550 tx=0x7fb5dc0091b0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:59.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:13:59.814+0000 7fb5e5600700 1 -- 172.21.15.117:0/3859485008 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb5d405cb20 con 0x7fb5e81017c0 2024-04-20T11:14:00.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.087+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5e8199ff0 con 0x7fb5d006edf0 2024-04-20T11:14:00.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.088+0000 7fb5e5600700 1 -- 172.21.15.117:0/3859485008 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fb5e8199ff0 con 0x7fb5d006edf0 2024-04-20T11:14:00.089 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:14:00.089 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:14:00.089 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:14:00.089 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:14:00.089 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:14:00.089 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:14:00.089 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:14:00.090 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "3/23 daemons upgraded", 2024-04-20T11:14:00.090 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "", 2024-04-20T11:14:00.090 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:14:00.090 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:14:00.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5d006edf0 msgr2=0x7fb5d00712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5d006edf0 0x7fb5d00712b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009550 tx=0x7fb5dc0091b0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 msgr2=0x7fb5e819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 0x7fb5e819ef30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4000c00 tx=0x7fb5d400bd70 comp rx=0 tx=0).stop 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 shutdown_connections 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5d006edf0 0x7fb5d00712b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5e8100e10 0x7fb5e819e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 --2- 172.21.15.117:0/3859485008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81017c0 0x7fb5e819ef30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 >> 172.21.15.117:0/3859485008 conn(0x7fb5e80fc6c0 msgr2=0x7fb5e80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 shutdown_connections 2024-04-20T11:14:00.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.091+0000 7fb5ef600700 1 -- 172.21.15.117:0/3859485008 wait complete. 2024-04-20T11:14:00.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.257+0000 7fcbabc00700 1 -- 172.21.15.117:0/1453430309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 msgr2=0x7fcba4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:00.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.257+0000 7fcbabc00700 1 --2- 172.21.15.117:0/1453430309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 0x7fcba4109290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009910 tx=0x7fcb90009c20 comp rx=0 tx=0).stop 2024-04-20T11:14:00.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.257+0000 7fcbabc00700 1 -- 172.21.15.117:0/1453430309 shutdown_connections 2024-04-20T11:14:00.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.257+0000 7fcbabc00700 1 --2- 172.21.15.117:0/1453430309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 0x7fcba4109290 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.257+0000 7fcbabc00700 1 --2- 172.21.15.117:0/1453430309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba4107520 0x7fcba4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.257+0000 7fcbabc00700 1 -- 172.21.15.117:0/1453430309 >> 172.21.15.117:0/1453430309 conn(0x7fcba40768f0 msgr2=0x7fcba4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:00.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.258+0000 7fcbabc00700 1 -- 172.21.15.117:0/1453430309 shutdown_connections 2024-04-20T11:14:00.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.258+0000 7fcbabc00700 1 -- 172.21.15.117:0/1453430309 wait complete. 2024-04-20T11:14:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:59 smithi117 ceph-mon[99036]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:59 smithi117 ceph-mon[99036]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T11:14:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:59 smithi117 ceph-mon[99036]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T11:14:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:59 smithi117 ceph-mon[99036]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:00.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:59 smithi117 ceph-mon[99036]: pgmap v11: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T11:14:00.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:59 smithi117 ceph-mon[99036]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:14:00.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:59 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4026930038' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:14:00.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:13:59 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3011508395' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.259+0000 7fcbabc00700 1 Processor -- start 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.259+0000 7fcbabc00700 1 -- start start 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.260+0000 7fcbabc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba4107520 0x7fcba419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.260+0000 7fcbabc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 0x7fcba419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.260+0000 7fcbabc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba419ddf0 con 0x7fcba4108e10 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.260+0000 7fcbabc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba41a1a70 con 0x7fcba4107520 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.260+0000 7fcba3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 0x7fcba419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.260+0000 7fcba3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 0x7fcba419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37868/0 (socket says 172.21.15.117:37868) 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.260+0000 7fcba3e00700 1 -- 172.21.15.117:0/2404159071 learned_addr learned my addr 172.21.15.117:0/2404159071 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:14:00.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.260+0000 7fcba9200700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba4107520 0x7fcba419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:00.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.261+0000 7fcba3e00700 1 -- 172.21.15.117:0/2404159071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba4107520 msgr2=0x7fcba419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:00.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.261+0000 7fcba3e00700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba4107520 0x7fcba419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.261+0000 7fcba3e00700 1 -- 172.21.15.117:0/2404159071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900095f0 con 0x7fcba4108e10 2024-04-20T11:14:00.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.261+0000 7fcba3e00700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 0x7fcba419d710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb90006010 tx=0x7fcb9000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:00.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.261+0000 7fcba1600700 1 -- 172.21.15.117:0/2404159071 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb9001d070 con 0x7fcba4108e10 2024-04-20T11:14:00.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.261+0000 7fcba1600700 1 -- 172.21.15.117:0/2404159071 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb900056e0 con 0x7fcba4108e10 2024-04-20T11:14:00.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.261+0000 7fcba1600700 1 -- 172.21.15.117:0/2404159071 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb9000fdf0 con 0x7fcba4108e10 2024-04-20T11:14:00.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.262+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba41a1cf0 con 0x7fcba4108e10 2024-04-20T11:14:00.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.262+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba41a2210 con 0x7fcba4108e10 2024-04-20T11:14:00.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.263+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba410f870 con 0x7fcba4108e10 2024-04-20T11:14:00.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.264+0000 7fcba1600700 1 -- 172.21.15.117:0/2404159071 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fcb9000f5f0 con 0x7fcba4108e10 2024-04-20T11:14:00.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.265+0000 7fcba1600700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcb8c06ee00 0x7fcb8c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:00.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.265+0000 7fcba1600700 1 -- 172.21.15.117:0/2404159071 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fcb90090430 con 0x7fcba4108e10 2024-04-20T11:14:00.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.265+0000 7fcba9200700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcb8c06ee00 0x7fcb8c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:00.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.266+0000 7fcba9200700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcb8c06ee00 0x7fcb8c0712c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005950 tx=0x7fcb9800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:00.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.270+0000 7fcba1600700 1 -- 172.21.15.117:0/2404159071 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcb90059000 con 0x7fcba4108e10 2024-04-20T11:14:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:59 smithi173 ceph-mon[23800]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:59 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-20T11:14:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:59 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-20T11:14:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:59 smithi173 ceph-mon[23800]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:59 smithi173 ceph-mon[23800]: pgmap v11: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T11:14:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:59 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:14:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:59 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/4026930038' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:14:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:13:59 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/3011508395' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:14:00.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.611+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcba4108920 con 0x7fcba4108e10 2024-04-20T11:14:00.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.611+0000 7fcba1600700 1 -- 172.21.15.117:0/2404159071 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fcb9005c6c0 con 0x7fcba4108e10 2024-04-20T11:14:00.612 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:14:00.612 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:14:00.612 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcb8c06ee00 msgr2=0x7fcb8c0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcb8c06ee00 0x7fcb8c0712c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005950 tx=0x7fcb9800b410 comp rx=0 tx=0).stop 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 msgr2=0x7fcba419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 0x7fcba419d710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb90006010 tx=0x7fcb9000bd70 comp rx=0 tx=0).stop 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 shutdown_connections 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcb8c06ee00 0x7fcb8c0712c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba4107520 0x7fcba419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 --2- 172.21.15.117:0/2404159071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba4108e10 0x7fcba419d710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 >> 172.21.15.117:0/2404159071 conn(0x7fcba40768f0 msgr2=0x7fcba4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 shutdown_connections 2024-04-20T11:14:00.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:00.614+0000 7fcbabc00700 1 -- 172.21.15.117:0/2404159071 wait complete. 2024-04-20T11:14:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:00 smithi117 ceph-mon[99036]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:14:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:00 smithi117 ceph-mon[99036]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:14:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:00 smithi117 ceph-mon[99036]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:14:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:00 smithi117 ceph-mon[99036]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:01.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:00 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2404159071' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:14:01.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:00 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:01.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:00 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:00 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:14:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:00 smithi173 ceph-mon[23800]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:14:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:00 smithi173 ceph-mon[23800]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:14:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:00 smithi173 ceph-mon[23800]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:00 smithi173 ceph-mon[23800]: from='client.? 172.21.15.117:0/2404159071' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:14:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:00 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:00 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: pgmap v12: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:14:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:02 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-20T11:14:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:01 smithi117 ceph-mon[99036]: Updating smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:14:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:01 smithi117 ceph-mon[99036]: Updating smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:14:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:01 smithi117 ceph-mon[99036]: pgmap v12: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T11:14:02.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:02.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:02.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:02.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:02.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:14:02.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-20T11:14:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:04 smithi173 ceph-mon[23800]: pgmap v13: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:04.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:04 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:04.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:04 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:14:04.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:04 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:14:04.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:04 smithi173 ceph-mon[23800]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:04 smithi117 ceph-mon[99036]: pgmap v13: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:14:04.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:14:04.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:05 smithi117 ceph-mon[99036]: Upgrade: Updating mon.smithi173 2024-04-20T11:14:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:05 smithi117 ceph-mon[99036]: Deploying daemon mon.smithi173 on smithi173 2024-04-20T11:14:05.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:05 smithi173 ceph-mon[23800]: Upgrade: Updating mon.smithi173 2024-04-20T11:14:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:05 smithi173 ceph-mon[23800]: Deploying daemon mon.smithi173 on smithi173 2024-04-20T11:14:06.073 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:06 smithi173 ceph-mon[23800]: pgmap v14: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:06 smithi117 ceph-mon[99036]: pgmap v14: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:06.862 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:06 smithi173 systemd[1]: Stopping Ceph mon.smithi173 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:14:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:06 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173[23791]: 2024-04-20T11:14:06.860+0000 7f1a5e400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi173 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:14:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:06 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173[23791]: 2024-04-20T11:14:06.860+0000 7f1a5e400700 -1 mon.smithi173@1(peon) e2 *** Got Signal Terminated *** 2024-04-20T11:14:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:06 smithi173 podman[73999]: 2024-04-20 11:14:06.945437098 +0000 UTC m=+0.211843272 container died 243eed744f9205671aa5b19bc4745a06cf34b478718b166c32e09fecb3e588b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-646b8c7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T11:14:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:07 smithi173 podman[73999]: 2024-04-20 11:14:07.422019706 +0000 UTC m=+0.688425890 container cleanup 243eed744f9205671aa5b19bc4745a06cf34b478718b166c32e09fecb3e588b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=reef-646b8c7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-20T11:14:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:07 smithi173 bash[73999]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173 2024-04-20T11:14:08.000 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:07 smithi173 podman[74013]: 2024-04-20 11:14:07.748186596 +0000 UTC m=+0.785968362 container remove 243eed744f9205671aa5b19bc4745a06cf34b478718b166c32e09fecb3e588b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=reef-646b8c7, ceph=True, org.label-schema.vendor=CentOS) 2024-04-20T11:14:08.000 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:07 smithi173 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi173.service: Deactivated successfully. 2024-04-20T11:14:08.001 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:08 smithi173 systemd[1]: Stopped Ceph mon.smithi173 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:14:08.367 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:08 smithi173 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi173.service: Consumed 20.015s CPU time. 2024-04-20T11:14:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:08 smithi173 systemd[1]: Starting Ceph mon.smithi173 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:14:09.047 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:08 smithi173 podman[74146]: 2024-04-20 11:14:08.633839114 +0000 UTC m=+0.033499578 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:14:09.047 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:08 smithi173 podman[74146]: 2024-04-20 11:14:08.740985987 +0000 UTC m=+0.140646473 container create 32e7ccffcc30e223f211ce00e99726691fa9b1ae90206ff7ae3d46f5414bb4fe (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-20T11:14:09.048 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 podman[74146]: 2024-04-20 11:14:09.045604243 +0000 UTC m=+0.445264772 container init 32e7ccffcc30e223f211ce00e99726691fa9b1ae90206ff7ae3d46f5414bb4fe (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:14:09.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 podman[74146]: 2024-04-20 11:14:09.05841493 +0000 UTC m=+0.458075397 container start 32e7ccffcc30e223f211ce00e99726691fa9b1ae90206ff7ae3d46f5414bb4fe (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-20T11:14:09.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T11:14:09.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-20T11:14:09.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: pidfile_write: ignore empty --pid-file 2024-04-20T11:14:09.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: load: jerasure load: lrc 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: RocksDB version: 7.9.2 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Git sha 0 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: DB SUMMARY 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: DB Session ID: V9Z2S3HWRRRCBY2LDHAH 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: CURRENT file: CURRENT 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1633 Bytes 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi173/store.db dir, Total Num: 1, files: 000030.sst 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi173/store.db: 000028.log size: 18731 ; 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.error_if_exists: 0 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.create_if_missing: 0 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.paranoid_checks: 1 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T11:14:09.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.env: 0x5583f2341cc0 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.info_log: 0x558403631b00 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.statistics: (nil) 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.use_fsync: 0 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_log_file_size: 0 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T11:14:09.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T11:14:09.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T11:14:09.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.allow_fallocate: 1 2024-04-20T11:14:09.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T11:14:09.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T11:14:09.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.use_direct_reads: 0 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.db_log_dir: 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.wal_dir: 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T11:14:09.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.write_buffer_manager: 0x5584036001e0 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.unordered_write: 0 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T11:14:09.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.row_cache: None 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.wal_filter: None 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.two_write_queues: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.wal_compression: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.atomic_flush: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.log_readahead_size: 0 2024-04-20T11:14:09.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_background_jobs: 2 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_background_compactions: -1 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_subcompactions: 1 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T11:14:09.419 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_open_files: -1 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_background_flushes: -1 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Compression algorithms supported: 2024-04-20T11:14:09.420 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: kZSTD supported: 0 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: kXpressCompression supported: 0 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: kZlibCompression supported: 1 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: kSnappyCompression supported: 1 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: kLZ4Compression supported: 1 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: kBZip2Compression supported: 0 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi173/store.db/MANIFEST-000010 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.merge_operator: 2024-04-20T11:14:09.421 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_filter: None 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558403631ce0) 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: cache_index_and_filter_blocks: 1 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: pin_top_level_index_and_filter: 1 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: index_type: 0 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: data_block_index_type: 0 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: index_shortening: 1 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T11:14:09.422 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: checksum: 4 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: no_block_cache: 0 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_cache: 0x558403633090 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_cache_name: BinnedLRUCache 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_cache_options: 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: capacity : 536870912 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: num_shard_bits : 4 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: strict_capacity_limit : 0 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: high_pri_pool_ratio: 0.000 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_cache_compressed: (nil) 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: persistent_cache: (nil) 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_size: 4096 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_size_deviation: 10 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_restart_interval: 16 2024-04-20T11:14:09.423 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: index_block_restart_interval: 1 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: metadata_block_size: 4096 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: partition_filters: 0 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: use_delta_encoding: 1 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: filter_policy: bloomfilter 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: whole_key_filtering: 1 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: verify_compression: 0 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: read_amp_bytes_per_bit: 0 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: format_version: 5 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: enable_index_compression: 1 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: block_align: 0 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: max_auto_readahead_size: 262144 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: prepopulate_block_cache: 0 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: initial_auto_readahead_size: 8192 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T11:14:09.424 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T11:14:09.425 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression: NoCompression 2024-04-20T11:14:09.425 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T11:14:09.425 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T11:14:09.425 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T11:14:09.425 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.num_levels: 7 2024-04-20T11:14:09.425 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T11:14:09.426 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T11:14:09.426 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T11:14:09.426 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T11:14:09.426 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T11:14:09.426 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T11:14:09.426 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T11:14:09.426 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T11:14:09.426 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T11:14:09.427 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T11:14:09.428 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T11:14:09.429 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.inplace_update_support: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.bloom_locality: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.max_successive_merges: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.ttl: 2592000 2024-04-20T11:14:09.430 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.enable_blob_files: false 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.min_blob_size: 0 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T11:14:09.431 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi173/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 32, last_sequence is 16887, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8051f2e2-8a7a-4439-ac38-628c50624f53 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611649115357, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611649116783, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 13445, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16892, "largest_seqno": 16912, "table_properties": {"data_size": 12278, "index_size": 72, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 346, "raw_average_key_size": 23, "raw_value_size": 11916, "raw_average_value_size": 794, "num_data_blocks": 3, "num_entries": 15, "num_filter_entries": 15, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713611649, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8051f2e2-8a7a-4439-ac38-628c50624f53", "db_session_id": "V9Z2S3HWRRRCBY2LDHAH", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611649116940, "job": 1, "event": "recovery_finished"} 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi173/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558403720000 2024-04-20T11:14:09.432 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: DB pointer 0x55840370a000 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ** DB Stats ** 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ** Compaction Stats [default] ** 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T11:14:09.433 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: L0 1/0 13.13 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 11.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: L6 1/0 9.24 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Sum 2/0 9.26 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 11.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 11.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ** Compaction Stats [default] ** 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 11.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T11:14:09.434 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Cumulative compaction: 0.00 GB write, 2.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Interval compaction: 0.00 GB write, 2.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Block cache BinnedLRUCache@0x558403633090#2 capacity: 512.00 MB usage: 0.27 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.16 KB,2.98023e-05%) Misc(1,0.00 KB,0%) 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: starting mon.smithi173 rank 1 at public addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] at bind addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi173 fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:14:09.435 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: mon.smithi173@-1(???) e2 preinit fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: mon.smithi173@-1(???).mds e12 new map 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: mon.smithi173@-1(???).mds e12 print_map 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: e12 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: legacy client fscid: 1 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Filesystem 'cephfs' (1) 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: fs_name cephfs 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: epoch 12 2024-04-20T11:14:09.436 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: tableserver 0 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: root 0 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: session_timeout 60 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: session_autoclose 300 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: max_file_size 1099511627776 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: max_xattr_size 65536 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: required_client_features {} 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: last_failure 0 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: last_failure_osd_epoch 0 2024-04-20T11:14:09.437 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: snap_rstat 1 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: max_mds 2 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: in 0,1 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: up {0=14460,1=24253} 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: failed 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: damaged 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: stopped 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: data_pools [3] 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: metadata_pool 2 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: inline_data enabled 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: balancer 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: bal_rank_mask -1 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: standby_count_wanted 1 2024-04-20T11:14:09.438 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: qdb_cluster leader: 0 members: 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: [mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: [mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: Standby daemons: 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: [mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout: [mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: mon.smithi173@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: mon.smithi173@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: mon.smithi173@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: mon.smithi173@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 ceph-mon[74165]: mon.smithi173@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 bash[74146]: 32e7ccffcc30e223f211ce00e99726691fa9b1ae90206ff7ae3d46f5414bb4fe 2024-04-20T11:14:09.439 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:09 smithi173 systemd[1]: Started Ceph mon.smithi173 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:14:11.344 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:14:11.345 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:14:11.345 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: mon.smithi117 calling monitor election 2024-04-20T11:14:11.345 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: mon.smithi173 calling monitor election 2024-04-20T11:14:11.345 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: mon.smithi117 is new leader, mons smithi117,smithi173 in quorum (ranks 0,1) 2024-04-20T11:14:11.345 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: monmap epoch 3 2024-04-20T11:14:11.345 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:14:11.346 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: last_changed 2024-04-20T11:14:09.957366+0000 2024-04-20T11:14:11.346 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: created 2024-04-20T10:55:27.473649+0000 2024-04-20T11:14:11.346 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: min_mon_release 19 (squid) 2024-04-20T11:14:11.346 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: election_strategy: 1 2024-04-20T11:14:11.346 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.smithi117 2024-04-20T11:14:11.347 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.smithi173 2024-04-20T11:14:11.347 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:14:11.347 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: osdmap e50: 6 total, 6 up, 6 in 2024-04-20T11:14:11.347 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: mgrmap e39: smithi117.ivnklj(active, since 27s), standbys: smithi173.gnrjof 2024-04-20T11:14:11.347 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:14:11.347 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:14:11.348 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:14:11.348 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:11.348 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:11.348 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:11.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi117"}]: dispatch 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mon metadata", "id": "smithi173"}]: dispatch 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: mon.smithi117 calling monitor election 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: mon.smithi173 calling monitor election 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: mon.smithi117 is new leader, mons smithi117,smithi173 in quorum (ranks 0,1) 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: monmap epoch 3 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: last_changed 2024-04-20T11:14:09.957366+0000 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: created 2024-04-20T10:55:27.473649+0000 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: min_mon_release 19 (squid) 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: election_strategy: 1 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.smithi117 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.smithi173 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi173.pybzht=up:active} 2 up:standby 2024-04-20T11:14:11.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: osdmap e50: 6 total, 6 up, 6 in 2024-04-20T11:14:11.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: mgrmap e39: smithi117.ivnklj(active, since 27s), standbys: smithi173.gnrjof 2024-04-20T11:14:11.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:14:11.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:14:11.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:14:11.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:11.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:11.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:12.340 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:12 smithi173 ceph-mon[74165]: pgmap v17: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:12.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:12 smithi117 ceph-mon[99036]: pgmap v17: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:14:13.408 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:14 smithi117 ceph-mon[99036]: pgmap v18: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:14 smithi173 ceph-mon[74165]: pgmap v18: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:15 smithi117 ceph-mon[99036]: pgmap v19: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:15 smithi173 ceph-mon[74165]: pgmap v19: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:17.821 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:17 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:17.821 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:17 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:17.821 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:17 smithi173 ceph-mon[74165]: pgmap v20: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:17 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:17 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:17 smithi117 ceph-mon[99036]: pgmap v20: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:20.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:19 smithi117 ceph-mon[99036]: pgmap v21: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:19 smithi173 ceph-mon[74165]: pgmap v21: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:22.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:21 smithi173 ceph-mon[74165]: pgmap v22: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:22.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:21 smithi117 ceph-mon[99036]: pgmap v22: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:23.564 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: Reconfiguring mon.smithi117 (monmap changed)... 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: Reconfiguring daemon mon.smithi117 on smithi117 2024-04-20T11:14:23.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:23 smithi117 ceph-mon[99036]: pgmap v23: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:23.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:23.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:23.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:23.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:23.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:23.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: Reconfiguring mon.smithi117 (monmap changed)... 2024-04-20T11:14:23.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:14:23.645 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:14:23.646 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:23.646 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: Reconfiguring daemon mon.smithi117 on smithi117 2024-04-20T11:14:23.646 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:23 smithi173 ceph-mon[74165]: pgmap v23: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:26.375 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:25 smithi117 ceph-mon[99036]: pgmap v24: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:26.375 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:25 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:26.375 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:25 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:26.375 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:25 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.ivnklj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:14:26.375 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:25 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:14:26.375 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:25 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:25 smithi173 ceph-mon[74165]: pgmap v24: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:25 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:25 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:26.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:25 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi117.ivnklj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:14:26.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:25 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:14:26.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:25 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:27 smithi117 ceph-mon[99036]: Reconfiguring mgr.smithi117.ivnklj (monmap changed)... 2024-04-20T11:14:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:27 smithi117 ceph-mon[99036]: Reconfiguring daemon mgr.smithi117.ivnklj on smithi117 2024-04-20T11:14:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:27 smithi173 ceph-mon[74165]: Reconfiguring mgr.smithi117.ivnklj (monmap changed)... 2024-04-20T11:14:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:27 smithi173 ceph-mon[74165]: Reconfiguring daemon mgr.smithi117.ivnklj on smithi117 2024-04-20T11:14:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:28 smithi117 ceph-mon[99036]: pgmap v25: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:14:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:14:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-20T11:14:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi117"}]: dispatch 2024-04-20T11:14:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:28 smithi173 ceph-mon[74165]: pgmap v25: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:14:28.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:14:28.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-20T11:14:28.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi117"}]: dispatch 2024-04-20T11:14:28.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:29 smithi117 ceph-mon[99036]: Reconfiguring ceph-exporter.smithi117 (monmap changed)... 2024-04-20T11:14:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:29 smithi117 ceph-mon[99036]: Unable to update caps for client.ceph-exporter.smithi117 2024-04-20T11:14:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:29 smithi117 ceph-mon[99036]: Reconfiguring daemon ceph-exporter.smithi117 on smithi117 2024-04-20T11:14:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:29 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:14:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:29 smithi173 ceph-mon[74165]: Reconfiguring ceph-exporter.smithi117 (monmap changed)... 2024-04-20T11:14:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:29 smithi173 ceph-mon[74165]: Unable to update caps for client.ceph-exporter.smithi117 2024-04-20T11:14:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:29 smithi173 ceph-mon[74165]: Reconfiguring daemon ceph-exporter.smithi117 on smithi117 2024-04-20T11:14:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:29 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:14:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:30 smithi117 ceph-mon[99036]: pgmap v26: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:30 smithi173 ceph-mon[74165]: pgmap v26: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:30.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.784+0000 7f9e81400700 1 -- 172.21.15.117:0/225166831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c1017c0 msgr2=0x7f9e7c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:30.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.784+0000 7f9e81400700 1 --2- 172.21.15.117:0/225166831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c1017c0 0x7f9e7c105810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009910 tx=0x7f9e64009c20 comp rx=0 tx=0).stop 2024-04-20T11:14:30.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.784+0000 7f9e81400700 1 -- 172.21.15.117:0/225166831 shutdown_connections 2024-04-20T11:14:30.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.784+0000 7f9e81400700 1 --2- 172.21.15.117:0/225166831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c1017c0 0x7f9e7c105810 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:30.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.784+0000 7f9e81400700 1 --2- 172.21.15.117:0/225166831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e7c100e10 0x7f9e7c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:30.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.784+0000 7f9e81400700 1 -- 172.21.15.117:0/225166831 >> 172.21.15.117:0/225166831 conn(0x7f9e7c0fc6a0 msgr2=0x7f9e7c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:30.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.784+0000 7f9e81400700 1 -- 172.21.15.117:0/225166831 shutdown_connections 2024-04-20T11:14:30.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.784+0000 7f9e81400700 1 -- 172.21.15.117:0/225166831 wait complete. 2024-04-20T11:14:30.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.785+0000 7f9e81400700 1 Processor -- start 2024-04-20T11:14:30.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.785+0000 7f9e81400700 1 -- start start 2024-04-20T11:14:30.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.786+0000 7f9e81400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c100e10 0x7f9e7c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:30.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.786+0000 7f9e81400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e7c1017c0 0x7f9e7c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:30.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.786+0000 7f9e81400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c19f4b0 con 0x7f9e7c100e10 2024-04-20T11:14:30.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.786+0000 7f9e81400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c198b20 con 0x7f9e7c1017c0 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.786+0000 7f9e7a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c100e10 0x7f9e7c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.786+0000 7f9e7a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c100e10 0x7f9e7c19e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46126/0 (socket says 172.21.15.117:46126) 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.786+0000 7f9e7a800700 1 -- 172.21.15.117:0/3961863806 learned_addr learned my addr 172.21.15.117:0/3961863806 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e73e00700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e7c1017c0 0x7f9e7c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e7a800700 1 -- 172.21.15.117:0/3961863806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e7c1017c0 msgr2=0x7f9e7c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e7a800700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e7c1017c0 0x7f9e7c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e7a800700 1 -- 172.21.15.117:0/3961863806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640095f0 con 0x7f9e7c100e10 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e73e00700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e7c1017c0 0x7f9e7c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:14:30.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e7a800700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c100e10 0x7f9e7c19e9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c930 tx=0x7f9e6c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:30.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e73400700 1 -- 172.21.15.117:0/3961863806 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6c007ad0 con 0x7f9e7c100e10 2024-04-20T11:14:30.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e73400700 1 -- 172.21.15.117:0/3961863806 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e6c007c30 con 0x7f9e7c100e10 2024-04-20T11:14:30.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e73400700 1 -- 172.21.15.117:0/3961863806 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6c018610 con 0x7f9e7c100e10 2024-04-20T11:14:30.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.787+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7c198e00 con 0x7f9e7c100e10 2024-04-20T11:14:30.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.788+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c199300 con 0x7f9e7c100e10 2024-04-20T11:14:30.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.789+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c104f90 con 0x7f9e7c100e10 2024-04-20T11:14:30.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.790+0000 7f9e73400700 1 -- 172.21.15.117:0/3961863806 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f9e6c00f450 con 0x7f9e7c100e10 2024-04-20T11:14:30.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.791+0000 7f9e73400700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e6006edf0 0x7f9e600712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:30.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.791+0000 7f9e73400700 1 -- 172.21.15.117:0/3961863806 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f9e6c08f200 con 0x7f9e7c100e10 2024-04-20T11:14:30.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.791+0000 7f9e73e00700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e6006edf0 0x7f9e600712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:30.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.792+0000 7f9e73e00700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e6006edf0 0x7f9e600712b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006010 tx=0x7f9e6400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:30.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:30.797+0000 7f9e73400700 1 -- 172.21.15.117:0/3961863806 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9e6c05b980 con 0x7f9e7c100e10 2024-04-20T11:14:31.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.060+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e7c19a080 con 0x7f9e6006edf0 2024-04-20T11:14:31.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.062+0000 7f9e73400700 1 -- 172.21.15.117:0/3961863806 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f9e7c19a080 con 0x7f9e6006edf0 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e6006edf0 msgr2=0x7f9e600712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e6006edf0 0x7f9e600712b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006010 tx=0x7f9e6400b540 comp rx=0 tx=0).stop 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c100e10 msgr2=0x7f9e7c19e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c100e10 0x7f9e7c19e9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c930 tx=0x7f9e6c00ccf0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 shutdown_connections 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e6006edf0 0x7f9e600712b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7c100e10 0x7f9e7c19e9e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 --2- 172.21.15.117:0/3961863806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e7c1017c0 0x7f9e7c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 >> 172.21.15.117:0/3961863806 conn(0x7f9e7c0fc6a0 msgr2=0x7f9e7c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 shutdown_connections 2024-04-20T11:14:31.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.065+0000 7f9e81400700 1 -- 172.21.15.117:0/3961863806 wait complete. 2024-04-20T11:14:31.087 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:14:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.227+0000 7f8494000700 1 -- 172.21.15.117:0/4143293010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 msgr2=0x7f848c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.227+0000 7f8494000700 1 --2- 172.21.15.117:0/4143293010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 0x7f848c1020b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f848000b0a0 tx=0x7f848000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.227+0000 7f8494000700 1 -- 172.21.15.117:0/4143293010 shutdown_connections 2024-04-20T11:14:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.227+0000 7f8494000700 1 --2- 172.21.15.117:0/4143293010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f848c1025f0 0x7f848c10aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.227+0000 7f8494000700 1 --2- 172.21.15.117:0/4143293010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 0x7f848c1020b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.227+0000 7f8494000700 1 -- 172.21.15.117:0/4143293010 >> 172.21.15.117:0/4143293010 conn(0x7f848c0766b0 msgr2=0x7f848c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.227+0000 7f8494000700 1 -- 172.21.15.117:0/4143293010 shutdown_connections 2024-04-20T11:14:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.227+0000 7f8494000700 1 -- 172.21.15.117:0/4143293010 wait complete. 2024-04-20T11:14:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.229+0000 7f8494000700 1 Processor -- start 2024-04-20T11:14:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.229+0000 7f8494000700 1 -- start start 2024-04-20T11:14:31.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.229+0000 7f8494000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 0x7f848c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:31.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.229+0000 7f8494000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f848c1025f0 0x7f848c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:31.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.229+0000 7f8494000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c19d350 con 0x7f848c101cd0 2024-04-20T11:14:31.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.229+0000 7f8494000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c196930 con 0x7f848c1025f0 2024-04-20T11:14:31.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.230+0000 7f8491600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 0x7f848c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:31.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.230+0000 7f8491600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 0x7f848c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46154/0 (socket says 172.21.15.117:46154) 2024-04-20T11:14:31.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.230+0000 7f8491600700 1 -- 172.21.15.117:0/3567650555 learned_addr learned my addr 172.21.15.117:0/3567650555 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:14:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.230+0000 7f8490c00700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f848c1025f0 0x7f848c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.230+0000 7f8491600700 1 -- 172.21.15.117:0/3567650555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f848c1025f0 msgr2=0x7f848c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.230+0000 7f8491600700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f848c1025f0 0x7f848c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.230+0000 7f8491600700 1 -- 172.21.15.117:0/3567650555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480009d00 con 0x7f848c101cd0 2024-04-20T11:14:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.231+0000 7f8491600700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 0x7f848c19c7f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f848000b070 tx=0x7f8480003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.232+0000 7f848a000700 1 -- 172.21.15.117:0/3567650555 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f848000e040 con 0x7f848c101cd0 2024-04-20T11:14:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.232+0000 7f848a000700 1 -- 172.21.15.117:0/3567650555 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8480004040 con 0x7f848c101cd0 2024-04-20T11:14:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.232+0000 7f848a000700 1 -- 172.21.15.117:0/3567650555 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f848001d6b0 con 0x7f848c101cd0 2024-04-20T11:14:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.232+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c196bb0 con 0x7f848c101cd0 2024-04-20T11:14:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.233+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c1970a0 con 0x7f848c101cd0 2024-04-20T11:14:31.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.234+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c105460 con 0x7f848c101cd0 2024-04-20T11:14:31.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.235+0000 7f848a000700 1 -- 172.21.15.117:0/3567650555 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f84800046b0 con 0x7f848c101cd0 2024-04-20T11:14:31.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.235+0000 7f848a000700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f847406edf0 0x7f84740712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:31.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.236+0000 7f848a000700 1 -- 172.21.15.117:0/3567650555 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f8480090620 con 0x7f848c101cd0 2024-04-20T11:14:31.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.236+0000 7f8490c00700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f847406edf0 0x7f84740712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:31.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.237+0000 7f8490c00700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f847406edf0 0x7f84740712b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8478005950 tx=0x7f84780058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.241+0000 7f848a000700 1 -- 172.21.15.117:0/3567650555 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f848005cda0 con 0x7f848c101cd0 2024-04-20T11:14:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:31 smithi117 ceph-mon[99036]: Reconfiguring crash.smithi117 (monmap changed)... 2024-04-20T11:14:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T11:14:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:31 smithi117 ceph-mon[99036]: Reconfiguring daemon crash.smithi117 on smithi117 2024-04-20T11:14:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:31 smithi117 ceph-mon[99036]: pgmap v27: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:31 smithi117 ceph-mon[99036]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:31.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.500+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f848c197df0 con 0x7f847406edf0 2024-04-20T11:14:31.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.504+0000 7f848a000700 1 -- 172.21.15.117:0/3567650555 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f848c197df0 con 0x7f847406edf0 2024-04-20T11:14:31.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f847406edf0 msgr2=0x7f84740712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f847406edf0 0x7f84740712b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8478005950 tx=0x7f84780058e0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 msgr2=0x7f848c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 0x7f848c19c7f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f848000b070 tx=0x7f8480003ce0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 shutdown_connections 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f847406edf0 0x7f84740712b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c101cd0 0x7f848c19c7f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 --2- 172.21.15.117:0/3567650555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f848c1025f0 0x7f848c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 >> 172.21.15.117:0/3567650555 conn(0x7f848c0766b0 msgr2=0x7f848c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 shutdown_connections 2024-04-20T11:14:31.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.507+0000 7f8494000700 1 -- 172.21.15.117:0/3567650555 wait complete. 2024-04-20T11:14:31.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.652+0000 7f7fa6600700 1 -- 172.21.15.117:0/2453959107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0108e10 msgr2=0x7f7fa0109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.652+0000 7f7fa6600700 1 --2- 172.21.15.117:0/2453959107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0108e10 0x7f7fa0109290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009910 tx=0x7f7f8c009c20 comp rx=0 tx=0).stop 2024-04-20T11:14:31.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.652+0000 7f7fa6600700 1 -- 172.21.15.117:0/2453959107 shutdown_connections 2024-04-20T11:14:31.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.652+0000 7f7fa6600700 1 --2- 172.21.15.117:0/2453959107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0108e10 0x7f7fa0109290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.652+0000 7f7fa6600700 1 --2- 172.21.15.117:0/2453959107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0107520 0x7f7fa0107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.652+0000 7f7fa6600700 1 -- 172.21.15.117:0/2453959107 >> 172.21.15.117:0/2453959107 conn(0x7f7fa00768f0 msgr2=0x7f7fa0078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:31.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.652+0000 7f7fa6600700 1 -- 172.21.15.117:0/2453959107 shutdown_connections 2024-04-20T11:14:31.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.652+0000 7f7fa6600700 1 -- 172.21.15.117:0/2453959107 wait complete. 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.653+0000 7f7fa6600700 1 Processor -- start 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.654+0000 7f7fa6600700 1 -- start start 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.654+0000 7f7fa6600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0107520 0x7f7fa019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.654+0000 7f7fa6600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108e10 0x7f7fa019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.654+0000 7f7fa6600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa019ddf0 con 0x7f7fa0107520 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.654+0000 7f7fa6600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a1a70 con 0x7f7fa0108e10 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.654+0000 7f7f9f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0107520 0x7f7fa019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.654+0000 7f7f9f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0107520 0x7f7fa019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46178/0 (socket says 172.21.15.117:46178) 2024-04-20T11:14:31.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.654+0000 7f7f9f400700 1 -- 172.21.15.117:0/3073811577 learned_addr learned my addr 172.21.15.117:0/3073811577 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:14:31.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7f9f400700 1 -- 172.21.15.117:0/3073811577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108e10 msgr2=0x7f7fa019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7f9f400700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108e10 0x7f7fa019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7f9f400700 1 -- 172.21.15.117:0/3073811577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0095f0 con 0x7f7fa0107520 2024-04-20T11:14:31.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7f9f400700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0107520 0x7f7fa019d1d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c8a0 tx=0x7f7f9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:31.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7f97e00700 1 -- 172.21.15.117:0/3073811577 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f9000cea0 con 0x7f7fa0107520 2024-04-20T11:14:31.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7f97e00700 1 -- 172.21.15.117:0/3073811577 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f90004830 con 0x7f7fa0107520 2024-04-20T11:14:31.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa01a1d50 con 0x7f7fa0107520 2024-04-20T11:14:31.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7f97e00700 1 -- 172.21.15.117:0/3073811577 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f900055b0 con 0x7f7fa0107520 2024-04-20T11:14:31.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.655+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa01a2320 con 0x7f7fa0107520 2024-04-20T11:14:31.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.657+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f800052f0 con 0x7f7fa0107520 2024-04-20T11:14:31.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.658+0000 7f7f97e00700 1 -- 172.21.15.117:0/3073811577 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f7f90004e10 con 0x7f7fa0107520 2024-04-20T11:14:31.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.658+0000 7f7f97e00700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f8406edf0 0x7f7f840712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:31.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.658+0000 7f7f97e00700 1 -- 172.21.15.117:0/3073811577 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f7f9008ea80 con 0x7f7fa0107520 2024-04-20T11:14:31.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.659+0000 7f7f9ea00700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f8406edf0 0x7f7f840712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:31.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.660+0000 7f7f9ea00700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f8406edf0 0x7f7f840712b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c006010 tx=0x7f7f8c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:31 smithi173 ceph-mon[74165]: Reconfiguring crash.smithi117 (monmap changed)... 2024-04-20T11:14:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T11:14:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:31 smithi173 ceph-mon[74165]: Reconfiguring daemon crash.smithi117 on smithi117 2024-04-20T11:14:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:31 smithi173 ceph-mon[74165]: pgmap v27: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:31.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:31 smithi173 ceph-mon[74165]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:31.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.664+0000 7f7f97e00700 1 -- 172.21.15.117:0/3073811577 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7f90057770 con 0x7f7fa0107520 2024-04-20T11:14:31.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.900+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f80000bc0 con 0x7f7f8406edf0 2024-04-20T11:14:31.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.918+0000 7f7f97e00700 1 -- 172.21.15.117:0/3073811577 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7f80000bc0 con 0x7f7f8406edf0 2024-04-20T11:14:31.918 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:14:31.918 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (2m) 41s ago 17m 14.4M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:14:31.918 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (17m) 41s ago 17m 8388k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:14:31.918 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (16m) 17s ago 16m 8174k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:14:31.918 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (17m) 41s ago 17m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (15m) 17s ago 15m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (2m) 41s ago 16m 75.3M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (9m) 41s ago 9m 16.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (9m) 41s ago 9m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (9m) 17s ago 9m 16.6M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (9m) 17s ago 9m 24.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (6m) 41s ago 18m 479M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (4m) 17s ago 15m 449M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (59s) 41s ago 19m 30.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (23s) 17s ago 15m 16.7M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (4m) 41s ago 17m 7402k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (4m) 17s ago 15m 7260k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (14m) 41s ago 14m 211M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (14m) 41s ago 14m 218M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:14:31.919 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (13m) 41s ago 13m 182M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:14:31.920 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (12m) 17s ago 12m 243M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:14:31.920 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (12m) 17s ago 12m 214M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:14:31.920 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (11m) 17s ago 11m 201M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:14:31.920 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (3m) 41s ago 16m 62.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f8406edf0 msgr2=0x7f7f840712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f8406edf0 0x7f7f840712b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c006010 tx=0x7f7f8c00b540 comp rx=0 tx=0).stop 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0107520 msgr2=0x7f7fa019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0107520 0x7f7fa019d1d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c8a0 tx=0x7f7f9000cc60 comp rx=0 tx=0).stop 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 shutdown_connections 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f8406edf0 0x7f7f840712b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fa0107520 0x7f7fa019d1d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 --2- 172.21.15.117:0/3073811577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108e10 0x7f7fa019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:31.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.922+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 >> 172.21.15.117:0/3073811577 conn(0x7f7fa00768f0 msgr2=0x7f7fa0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:31.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.923+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 shutdown_connections 2024-04-20T11:14:31.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:31.923+0000 7f7fa6600700 1 -- 172.21.15.117:0/3073811577 wait complete. 2024-04-20T11:14:32.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.081+0000 7ff3a8c00700 1 -- 172.21.15.117:0/1896329419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a4100e10 msgr2=0x7ff3a41011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:32.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.081+0000 7ff3a8c00700 1 --2- 172.21.15.117:0/1896329419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a4100e10 0x7ff3a41011f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff394009910 tx=0x7ff394009c20 comp rx=0 tx=0).stop 2024-04-20T11:14:32.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.081+0000 7ff3a8c00700 1 -- 172.21.15.117:0/1896329419 shutdown_connections 2024-04-20T11:14:32.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.081+0000 7ff3a8c00700 1 --2- 172.21.15.117:0/1896329419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3a41017c0 0x7ff3a4105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.081+0000 7ff3a8c00700 1 --2- 172.21.15.117:0/1896329419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a4100e10 0x7ff3a41011f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.081+0000 7ff3a8c00700 1 -- 172.21.15.117:0/1896329419 >> 172.21.15.117:0/1896329419 conn(0x7ff3a40fc6c0 msgr2=0x7ff3a40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:32.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.082+0000 7ff3a8c00700 1 -- 172.21.15.117:0/1896329419 shutdown_connections 2024-04-20T11:14:32.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.082+0000 7ff3a8c00700 1 -- 172.21.15.117:0/1896329419 wait complete. 2024-04-20T11:14:32.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.083+0000 7ff3a8c00700 1 Processor -- start 2024-04-20T11:14:32.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.083+0000 7ff3a8c00700 1 -- start start 2024-04-20T11:14:32.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.083+0000 7ff3a8c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3a4100e10 0x7ff3a419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:32.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.083+0000 7ff3a8c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a41017c0 0x7ff3a419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:32.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.083+0000 7ff3a8c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a419f510 con 0x7ff3a41017c0 2024-04-20T11:14:32.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.083+0000 7ff3a8c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a4198b70 con 0x7ff3a4100e10 2024-04-20T11:14:32.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.084+0000 7ff3a1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a41017c0 0x7ff3a419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:32.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.084+0000 7ff3a1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a41017c0 0x7ff3a419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46196/0 (socket says 172.21.15.117:46196) 2024-04-20T11:14:32.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.084+0000 7ff3a1400700 1 -- 172.21.15.117:0/161838550 learned_addr learned my addr 172.21.15.117:0/161838550 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:14:32.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.084+0000 7ff3a1e00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3a4100e10 0x7ff3a419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:32.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.084+0000 7ff3a1400700 1 -- 172.21.15.117:0/161838550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3a4100e10 msgr2=0x7ff3a419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:32.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.084+0000 7ff3a1400700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3a4100e10 0x7ff3a419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.084+0000 7ff3a1400700 1 -- 172.21.15.117:0/161838550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3940095f0 con 0x7ff3a41017c0 2024-04-20T11:14:32.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.084+0000 7ff3a1400700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a41017c0 0x7ff3a419eec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff39000c960 tx=0x7ff39000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:32.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.085+0000 7ff39aa00700 1 -- 172.21.15.117:0/161838550 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff390007a70 con 0x7ff3a41017c0 2024-04-20T11:14:32.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.085+0000 7ff39aa00700 1 -- 172.21.15.117:0/161838550 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff390007bd0 con 0x7ff3a41017c0 2024-04-20T11:14:32.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.085+0000 7ff39aa00700 1 -- 172.21.15.117:0/161838550 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3900186c0 con 0x7ff3a41017c0 2024-04-20T11:14:32.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.085+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a4198e50 con 0x7ff3a41017c0 2024-04-20T11:14:32.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.085+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a41993a0 con 0x7ff3a41017c0 2024-04-20T11:14:32.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.086+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a4103f80 con 0x7ff3a41017c0 2024-04-20T11:14:32.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.088+0000 7ff39aa00700 1 -- 172.21.15.117:0/161838550 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7ff39000faa0 con 0x7ff3a41017c0 2024-04-20T11:14:32.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.089+0000 7ff39aa00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff38806edf0 0x7ff3880712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:32.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.089+0000 7ff39aa00700 1 -- 172.21.15.117:0/161838550 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7ff39008eff0 con 0x7ff3a41017c0 2024-04-20T11:14:32.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.089+0000 7ff3a1e00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff38806edf0 0x7ff3880712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:32.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.090+0000 7ff3a1e00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff38806edf0 0x7ff3880712b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b5c0 tx=0x7ff39401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:32.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.094+0000 7ff39aa00700 1 -- 172.21.15.117:0/161838550 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff39005b540 con 0x7ff3a41017c0 2024-04-20T11:14:32.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.439+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff3a41022f0 con 0x7ff3a41017c0 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.440+0000 7ff39aa00700 1 -- 172.21.15.117:0/161838550 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7ff39005b360 con 0x7ff3a41017c0 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:14:32.440 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 10, 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:14:32.441 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff38806edf0 msgr2=0x7ff3880712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff38806edf0 0x7ff3880712b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b5c0 tx=0x7ff39401a040 comp rx=0 tx=0).stop 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a41017c0 msgr2=0x7ff3a419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a41017c0 0x7ff3a419eec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff39000c960 tx=0x7ff39000cd20 comp rx=0 tx=0).stop 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 shutdown_connections 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff38806edf0 0x7ff3880712b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3a4100e10 0x7ff3a419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 --2- 172.21.15.117:0/161838550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3a41017c0 0x7ff3a419eec0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 >> 172.21.15.117:0/161838550 conn(0x7ff3a40fc6c0 msgr2=0x7ff3a40fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 shutdown_connections 2024-04-20T11:14:32.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.443+0000 7ff3a8c00700 1 -- 172.21.15.117:0/161838550 wait complete. 2024-04-20T11:14:32.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:32 smithi173 ceph-mon[74165]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:32.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:32 smithi173 ceph-mon[74165]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:32.594 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:32 smithi117 ceph-mon[99036]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:32.594 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:32 smithi117 ceph-mon[99036]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:32.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.598+0000 7f8fdea00700 1 -- 172.21.15.117:0/3046980004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd8105a80 msgr2=0x7f8fd8105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:32.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.598+0000 7f8fdea00700 1 --2- 172.21.15.117:0/3046980004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd8105a80 0x7f8fd8105e60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009910 tx=0x7f8fc8009c20 comp rx=0 tx=0).stop 2024-04-20T11:14:32.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.598+0000 7f8fdea00700 1 -- 172.21.15.117:0/3046980004 shutdown_connections 2024-04-20T11:14:32.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.598+0000 7f8fdea00700 1 --2- 172.21.15.117:0/3046980004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fd80ffa80 0x7f8fd80fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.598+0000 7f8fdea00700 1 --2- 172.21.15.117:0/3046980004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd8105a80 0x7f8fd8105e60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.598+0000 7f8fdea00700 1 -- 172.21.15.117:0/3046980004 >> 172.21.15.117:0/3046980004 conn(0x7f8fd8076750 msgr2=0x7f8fd80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:32.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.598+0000 7f8fdea00700 1 -- 172.21.15.117:0/3046980004 shutdown_connections 2024-04-20T11:14:32.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.598+0000 7f8fdea00700 1 -- 172.21.15.117:0/3046980004 wait complete. 2024-04-20T11:14:32.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.600+0000 7f8fdea00700 1 Processor -- start 2024-04-20T11:14:32.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.600+0000 7f8fdea00700 1 -- start start 2024-04-20T11:14:32.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.600+0000 7f8fdea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd80ffa80 0x7f8fd819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:32.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.600+0000 7f8fdea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fd8105a80 0x7f8fd819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:32.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.600+0000 7f8fdea00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd819d310 con 0x7f8fd80ffa80 2024-04-20T11:14:32.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.600+0000 7f8fdea00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd8196940 con 0x7f8fd8105a80 2024-04-20T11:14:32.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.600+0000 7f8fd7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd80ffa80 0x7f8fd819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:32.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.601+0000 7f8fd7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd80ffa80 0x7f8fd819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46204/0 (socket says 172.21.15.117:46204) 2024-04-20T11:14:32.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.601+0000 7f8fd7e00700 1 -- 172.21.15.117:0/2433169823 learned_addr learned my addr 172.21.15.117:0/2433169823 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:14:32.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.601+0000 7f8fd7e00700 1 -- 172.21.15.117:0/2433169823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fd8105a80 msgr2=0x7f8fd819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:32.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.601+0000 7f8fd7e00700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fd8105a80 0x7f8fd819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.601+0000 7f8fd7e00700 1 -- 172.21.15.117:0/2433169823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc80095f0 con 0x7f8fd80ffa80 2024-04-20T11:14:32.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.601+0000 7f8fd7e00700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd80ffa80 0x7f8fd819c7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8000c00 tx=0x7f8fc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:32.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.602+0000 7f8fd4c00700 1 -- 172.21.15.117:0/2433169823 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc801d070 con 0x7f8fd80ffa80 2024-04-20T11:14:32.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.602+0000 7f8fd4c00700 1 -- 172.21.15.117:0/2433169823 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fc80056e0 con 0x7f8fd80ffa80 2024-04-20T11:14:32.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.602+0000 7f8fd4c00700 1 -- 172.21.15.117:0/2433169823 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc800fe80 con 0x7f8fd80ffa80 2024-04-20T11:14:32.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.602+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd8196bc0 con 0x7f8fd80ffa80 2024-04-20T11:14:32.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.602+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd81970b0 con 0x7f8fd80ffa80 2024-04-20T11:14:32.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.603+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd8108080 con 0x7f8fd80ffa80 2024-04-20T11:14:32.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.605+0000 7f8fd4c00700 1 -- 172.21.15.117:0/2433169823 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f8fc800f6f0 con 0x7f8fd80ffa80 2024-04-20T11:14:32.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.606+0000 7f8fd4c00700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8fbc06edf0 0x7f8fbc0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:32.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.606+0000 7f8fd4c00700 1 -- 172.21.15.117:0/2433169823 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f8fc8090340 con 0x7f8fd80ffa80 2024-04-20T11:14:32.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.606+0000 7f8fd7400700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8fbc06edf0 0x7f8fbc0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:32.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.607+0000 7f8fd7400700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8fbc06edf0 0x7f8fbc0712b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005950 tx=0x7f8fc40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:32.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.611+0000 7f8fd4c00700 1 -- 172.21.15.117:0/2433169823 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8fc805cb60 con 0x7f8fd80ffa80 2024-04-20T11:14:32.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.897+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8fd8068ca0 con 0x7f8fd80ffa80 2024-04-20T11:14:32.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.898+0000 7f8fd4c00700 1 -- 172.21.15.117:0/2433169823 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7f8fc805c6f0 con 0x7f8fd80ffa80 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:14:32.899 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:14:32.900 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:14:32.901 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:14:32.902 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:14:32.902 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:14:32.902 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8fbc06edf0 msgr2=0x7f8fbc0712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8fbc06edf0 0x7f8fbc0712b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005950 tx=0x7f8fc40058e0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd80ffa80 msgr2=0x7f8fd819c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd80ffa80 0x7f8fd819c7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8000c00 tx=0x7f8fc800be20 comp rx=0 tx=0).stop 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 shutdown_connections 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8fbc06edf0 0x7f8fbc0712b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fd80ffa80 0x7f8fd819c7b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 --2- 172.21.15.117:0/2433169823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fd8105a80 0x7f8fd819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:32.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 >> 172.21.15.117:0/2433169823 conn(0x7f8fd8076750 msgr2=0x7f8fd80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:32.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 shutdown_connections 2024-04-20T11:14:32.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:32.901+0000 7f8fdea00700 1 -- 172.21.15.117:0/2433169823 wait complete. 2024-04-20T11:14:32.904 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:14:33.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.052+0000 7fd256a00700 1 -- 172.21.15.117:0/3079313355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073c30 msgr2=0x7fd25010ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:33.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.052+0000 7fd256a00700 1 --2- 172.21.15.117:0/3079313355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073c30 0x7fd25010ceb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009910 tx=0x7fd23c009c20 comp rx=0 tx=0).stop 2024-04-20T11:14:33.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.052+0000 7fd256a00700 1 -- 172.21.15.117:0/3079313355 shutdown_connections 2024-04-20T11:14:33.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.052+0000 7fd256a00700 1 --2- 172.21.15.117:0/3079313355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073c30 0x7fd25010ceb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.052+0000 7fd256a00700 1 --2- 172.21.15.117:0/3079313355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd250073310 0x7fd2500736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.052+0000 7fd256a00700 1 -- 172.21.15.117:0/3079313355 >> 172.21.15.117:0/3079313355 conn(0x7fd2500fc6c0 msgr2=0x7fd2500feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:33.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.052+0000 7fd256a00700 1 -- 172.21.15.117:0/3079313355 shutdown_connections 2024-04-20T11:14:33.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.052+0000 7fd256a00700 1 -- 172.21.15.117:0/3079313355 wait complete. 2024-04-20T11:14:33.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.053+0000 7fd256a00700 1 Processor -- start 2024-04-20T11:14:33.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.053+0000 7fd256a00700 1 -- start start 2024-04-20T11:14:33.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.053+0000 7fd256a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073310 0x7fd250198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:33.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.053+0000 7fd256a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd250073c30 0x7fd250199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:33.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.053+0000 7fd256a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250199a00 con 0x7fd250073310 2024-04-20T11:14:33.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.054+0000 7fd256a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019d6d0 con 0x7fd250073c30 2024-04-20T11:14:33.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.054+0000 7fd24fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073310 0x7fd250198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:33.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.054+0000 7fd24fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073310 0x7fd250198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46228/0 (socket says 172.21.15.117:46228) 2024-04-20T11:14:33.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.054+0000 7fd24fe00700 1 -- 172.21.15.117:0/3697585549 learned_addr learned my addr 172.21.15.117:0/3697585549 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.054+0000 7fd24f400700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd250073c30 0x7fd250199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.054+0000 7fd24fe00700 1 -- 172.21.15.117:0/3697585549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd250073c30 msgr2=0x7fd250199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.054+0000 7fd24fe00700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd250073c30 0x7fd250199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.054+0000 7fd24fe00700 1 -- 172.21.15.117:0/3697585549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0095f0 con 0x7fd250073310 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.055+0000 7fd24fe00700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073310 0x7fd250198de0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd24000b4b0 tx=0x7fd24000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.055+0000 7fd24cc00700 1 -- 172.21.15.117:0/3697585549 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd240011800 con 0x7fd250073310 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.055+0000 7fd24cc00700 1 -- 172.21.15.117:0/3697585549 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd240011e20 con 0x7fd250073310 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.055+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25019d9b0 con 0x7fd250073310 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.055+0000 7fd24cc00700 1 -- 172.21.15.117:0/3697585549 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24000f2e0 con 0x7fd250073310 2024-04-20T11:14:33.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.055+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25019df00 con 0x7fd250073310 2024-04-20T11:14:33.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.056+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2500749c0 con 0x7fd250073310 2024-04-20T11:14:33.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.059+0000 7fd24cc00700 1 -- 172.21.15.117:0/3697585549 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fd240010380 con 0x7fd250073310 2024-04-20T11:14:33.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.060+0000 7fd24cc00700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd23406edf0 0x7fd2340712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:33.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.060+0000 7fd24cc00700 1 -- 172.21.15.117:0/3697585549 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fd24008ede0 con 0x7fd250073310 2024-04-20T11:14:33.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.060+0000 7fd24f400700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd23406edf0 0x7fd2340712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:33.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.061+0000 7fd24f400700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd23406edf0 0x7fd2340712b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005f50 tx=0x7fd23c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:33.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.065+0000 7fd24cc00700 1 -- 172.21.15.117:0/3697585549 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd240057c80 con 0x7fd250073310 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/161838550' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: Reconfiguring osd.0 (monmap changed)... 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: Reconfiguring daemon osd.0 on smithi117 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2433169823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:14:33.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:33 smithi117 ceph-mon[99036]: pgmap v28: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:33.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.311+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2501077c0 con 0x7fd23406edf0 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.312+0000 7fd24cc00700 1 -- 172.21.15.117:0/3697585549 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fd2501077c0 con 0x7fd23406edf0 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "4/23 daemons upgraded", 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mon daemons", 2024-04-20T11:14:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:14:33.313 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd23406edf0 msgr2=0x7fd2340712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd23406edf0 0x7fd2340712b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005f50 tx=0x7fd23c01a040 comp rx=0 tx=0).stop 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073310 msgr2=0x7fd250198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073310 0x7fd250198de0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd24000b4b0 tx=0x7fd24000b870 comp rx=0 tx=0).stop 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 shutdown_connections 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd23406edf0 0x7fd2340712b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250073310 0x7fd250198de0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 --2- 172.21.15.117:0/3697585549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd250073c30 0x7fd250199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 >> 172.21.15.117:0/3697585549 conn(0x7fd2500fc6c0 msgr2=0x7fd250105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.315+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 shutdown_connections 2024-04-20T11:14:33.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.316+0000 7fd256a00700 1 -- 172.21.15.117:0/3697585549 wait complete. 2024-04-20T11:14:33.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.461+0000 7ff0fb600700 1 -- 172.21.15.117:0/758328640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 msgr2=0x7ff0f410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:33.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.461+0000 7ff0fb600700 1 --2- 172.21.15.117:0/758328640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 0x7ff0f410ab70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009910 tx=0x7ff0e0009c20 comp rx=0 tx=0).stop 2024-04-20T11:14:33.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.462+0000 7ff0fb600700 1 -- 172.21.15.117:0/758328640 shutdown_connections 2024-04-20T11:14:33.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.462+0000 7ff0fb600700 1 --2- 172.21.15.117:0/758328640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 0x7ff0f410ab70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.462+0000 7ff0fb600700 1 --2- 172.21.15.117:0/758328640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f4101d60 0x7ff0f4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.462+0000 7ff0fb600700 1 -- 172.21.15.117:0/758328640 >> 172.21.15.117:0/758328640 conn(0x7ff0f40fb5b0 msgr2=0x7ff0f40fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:33.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.462+0000 7ff0fb600700 1 -- 172.21.15.117:0/758328640 shutdown_connections 2024-04-20T11:14:33.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.462+0000 7ff0fb600700 1 -- 172.21.15.117:0/758328640 wait complete. 2024-04-20T11:14:33.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0fb600700 1 Processor -- start 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0fb600700 1 -- start start 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0fb600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f4101d60 0x7ff0f40ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0fb600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 0x7ff0f40fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0fb600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f4100440 con 0x7ff0f4102680 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0fb600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f4100580 con 0x7ff0f4101d60 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 0x7ff0f40fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 0x7ff0f40fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46256/0 (socket says 172.21.15.117:46256) 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.463+0000 7ff0f3e00700 1 -- 172.21.15.117:0/49051815 learned_addr learned my addr 172.21.15.117:0/49051815 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.464+0000 7ff0f3e00700 1 -- 172.21.15.117:0/49051815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f4101d60 msgr2=0x7ff0f40ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.464+0000 7ff0f8c00700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f4101d60 0x7ff0f40ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:33.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.464+0000 7ff0f3e00700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f4101d60 0x7ff0f40ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.464+0000 7ff0f3e00700 1 -- 172.21.15.117:0/49051815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00095f0 con 0x7ff0f4102680 2024-04-20T11:14:33.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.464+0000 7ff0f8c00700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f4101d60 0x7ff0f40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:14:33.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.464+0000 7ff0f3e00700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 0x7ff0f40fff00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000bd70 tx=0x7ff0e000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:33.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.464+0000 7ff0f1600700 1 -- 172.21.15.117:0/49051815 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0e001d070 con 0x7ff0f4102680 2024-04-20T11:14:33.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.465+0000 7ff0f1600700 1 -- 172.21.15.117:0/49051815 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0e000f460 con 0x7ff0f4102680 2024-04-20T11:14:33.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.465+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f4100800 con 0x7ff0f4102680 2024-04-20T11:14:33.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.465+0000 7ff0f1600700 1 -- 172.21.15.117:0/49051815 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0e0017610 con 0x7ff0f4102680 2024-04-20T11:14:33.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.465+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f41a2f50 con 0x7ff0f4102680 2024-04-20T11:14:33.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.466+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f40696e0 con 0x7ff0f4102680 2024-04-20T11:14:33.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.467+0000 7ff0f1600700 1 -- 172.21.15.117:0/49051815 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7ff0e000fa50 con 0x7ff0f4102680 2024-04-20T11:14:33.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.468+0000 7ff0f1600700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0dc06ee00 0x7ff0dc0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:33.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.468+0000 7ff0f1600700 1 -- 172.21.15.117:0/49051815 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7ff0e0090510 con 0x7ff0f4102680 2024-04-20T11:14:33.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.468+0000 7ff0f8c00700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0dc06ee00 0x7ff0dc0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:33.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.469+0000 7ff0f8c00700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0dc06ee00 0x7ff0dc0712c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005fd0 tx=0x7ff0e8005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:33.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.473+0000 7ff0f1600700 1 -- 172.21.15.117:0/49051815 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff0e00591a0 con 0x7ff0f4102680 2024-04-20T11:14:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/161838550' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:14:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:33.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: Reconfiguring osd.0 (monmap changed)... 2024-04-20T11:14:33.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T11:14:33.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:33.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: Reconfiguring daemon osd.0 on smithi117 2024-04-20T11:14:33.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2433169823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:14:33.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:33 smithi173 ceph-mon[74165]: pgmap v28: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:33.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.793+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff0f4100ea0 con 0x7ff0f4102680 2024-04-20T11:14:33.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.794+0000 7ff0f1600700 1 -- 172.21.15.117:0/49051815 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff0e0005740 con 0x7ff0f4102680 2024-04-20T11:14:33.794 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:14:33.794 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:14:33.794 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0dc06ee00 msgr2=0x7ff0dc0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0dc06ee00 0x7ff0dc0712c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005fd0 tx=0x7ff0e8005dc0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 msgr2=0x7ff0f40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 0x7ff0f40fff00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000bd70 tx=0x7ff0e000be50 comp rx=0 tx=0).stop 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 shutdown_connections 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0dc06ee00 0x7ff0dc0712c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f4101d60 0x7ff0f40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 --2- 172.21.15.117:0/49051815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102680 0x7ff0f40fff00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 >> 172.21.15.117:0/49051815 conn(0x7ff0f40fb5b0 msgr2=0x7ff0f4103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 shutdown_connections 2024-04-20T11:14:33.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:14:33.797+0000 7ff0fb600700 1 -- 172.21.15.117:0/49051815 wait complete. 2024-04-20T11:14:34.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:34 smithi117 ceph-mon[99036]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:34.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:34 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/49051815' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:14:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:34 smithi173 ceph-mon[74165]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:34 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/49051815' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:14:36.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:35 smithi117 ceph-mon[99036]: pgmap v29: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:36.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:36.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:36.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:14:36.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:35 smithi173 ceph-mon[74165]: pgmap v29: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:14:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:36.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:36 smithi117 ceph-mon[99036]: Reconfiguring osd.1 (monmap changed)... 2024-04-20T11:14:36.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:36 smithi117 ceph-mon[99036]: Reconfiguring daemon osd.1 on smithi117 2024-04-20T11:14:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:36 smithi173 ceph-mon[74165]: Reconfiguring osd.1 (monmap changed)... 2024-04-20T11:14:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:36 smithi173 ceph-mon[74165]: Reconfiguring daemon osd.1 on smithi117 2024-04-20T11:14:38.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:37 smithi117 ceph-mon[99036]: pgmap v30: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:38.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:37 smithi173 ceph-mon[74165]: pgmap v30: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:38.989 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:38.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:38.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:38 smithi117 ceph-mon[99036]: Reconfiguring osd.2 (monmap changed)... 2024-04-20T11:14:38.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:14:38.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:38.990 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:38 smithi117 ceph-mon[99036]: Reconfiguring daemon osd.2 on smithi117 2024-04-20T11:14:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:38 smithi173 ceph-mon[74165]: Reconfiguring osd.2 (monmap changed)... 2024-04-20T11:14:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:14:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:39.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:38 smithi173 ceph-mon[74165]: Reconfiguring daemon osd.2 on smithi117 2024-04-20T11:14:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:39 smithi117 ceph-mon[99036]: pgmap v31: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:39 smithi173 ceph-mon[74165]: pgmap v31: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:41.667 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:41.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:41.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:41 smithi117 ceph-mon[99036]: Reconfiguring mds.cephfs.smithi117.umapup (monmap changed)... 2024-04-20T11:14:41.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.umapup", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:14:41.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:41.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:41 smithi117 ceph-mon[99036]: Reconfiguring daemon mds.cephfs.smithi117.umapup on smithi117 2024-04-20T11:14:41.668 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:41 smithi117 ceph-mon[99036]: pgmap v32: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:41 smithi173 ceph-mon[74165]: Reconfiguring mds.cephfs.smithi117.umapup (monmap changed)... 2024-04-20T11:14:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.umapup", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:14:41.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:41.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:41 smithi173 ceph-mon[74165]: Reconfiguring daemon mds.cephfs.smithi117.umapup on smithi117 2024-04-20T11:14:41.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:41 smithi173 ceph-mon[74165]: pgmap v32: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:44.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:44.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:44.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:43 smithi117 ceph-mon[99036]: Reconfiguring mds.cephfs.smithi117.miaplu (monmap changed)... 2024-04-20T11:14:44.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.miaplu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:14:44.074 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:44.075 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:43 smithi117 ceph-mon[99036]: Reconfiguring daemon mds.cephfs.smithi117.miaplu on smithi117 2024-04-20T11:14:44.075 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:43 smithi117 ceph-mon[99036]: pgmap v33: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:44.075 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:14:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:43 smithi173 ceph-mon[74165]: Reconfiguring mds.cephfs.smithi117.miaplu (monmap changed)... 2024-04-20T11:14:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.miaplu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:14:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:43 smithi173 ceph-mon[74165]: Reconfiguring daemon mds.cephfs.smithi117.miaplu on smithi117 2024-04-20T11:14:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:43 smithi173 ceph-mon[74165]: pgmap v33: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:14:46.307 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:46 smithi173 ceph-mon[74165]: pgmap v34: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:46.307 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:46.307 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:46.307 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:14:46.307 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:14:46.307 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-20T11:14:46.307 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi173"}]: dispatch 2024-04-20T11:14:46.307 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:46 smithi117 ceph-mon[99036]: pgmap v34: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:14:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:14:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-20T11:14:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi173"}]: dispatch 2024-04-20T11:14:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:47.387 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:47 smithi173 ceph-mon[74165]: Reconfiguring ceph-exporter.smithi173 (monmap changed)... 2024-04-20T11:14:47.387 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:47 smithi173 ceph-mon[74165]: Unable to update caps for client.ceph-exporter.smithi173 2024-04-20T11:14:47.387 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:47 smithi173 ceph-mon[74165]: Reconfiguring daemon ceph-exporter.smithi173 on smithi173 2024-04-20T11:14:47.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:47 smithi117 ceph-mon[99036]: Reconfiguring ceph-exporter.smithi173 (monmap changed)... 2024-04-20T11:14:47.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:47 smithi117 ceph-mon[99036]: Unable to update caps for client.ceph-exporter.smithi173 2024-04-20T11:14:47.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:47 smithi117 ceph-mon[99036]: Reconfiguring daemon ceph-exporter.smithi173 on smithi173 2024-04-20T11:14:48.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:48 smithi117 ceph-mon[99036]: pgmap v35: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:48.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:48 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:48.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:48 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:48.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:48 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T11:14:48.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:48 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:48.409 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:48 smithi173 ceph-mon[74165]: pgmap v35: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:48.409 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:48 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:48.409 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:48 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:48.409 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:48 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T11:14:48.409 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:48 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:49.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:49 smithi117 ceph-mon[99036]: Reconfiguring crash.smithi173 (monmap changed)... 2024-04-20T11:14:49.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:49 smithi117 ceph-mon[99036]: Reconfiguring daemon crash.smithi173 on smithi173 2024-04-20T11:14:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:49 smithi173 ceph-mon[74165]: Reconfiguring crash.smithi173 (monmap changed)... 2024-04-20T11:14:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:49 smithi173 ceph-mon[74165]: Reconfiguring daemon crash.smithi173 on smithi173 2024-04-20T11:14:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:50 smithi117 ceph-mon[99036]: pgmap v36: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:50 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:50 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:50.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:50 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:14:50.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:50 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:14:50.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:50 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:50 smithi173 ceph-mon[74165]: pgmap v36: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:50 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:50 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:50 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi173.gnrjof", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:14:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:50 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:14:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:50 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:51.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:51 smithi117 ceph-mon[99036]: Reconfiguring mgr.smithi173.gnrjof (monmap changed)... 2024-04-20T11:14:51.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:51 smithi117 ceph-mon[99036]: Reconfiguring daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T11:14:51.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:51 smithi173 ceph-mon[74165]: Reconfiguring mgr.smithi173.gnrjof (monmap changed)... 2024-04-20T11:14:51.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:51 smithi173 ceph-mon[74165]: Reconfiguring daemon mgr.smithi173.gnrjof on smithi173 2024-04-20T11:14:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:52 smithi117 ceph-mon[99036]: pgmap v37: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:52 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:52 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:52 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:14:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:52 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:14:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:52 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:52 smithi173 ceph-mon[74165]: pgmap v37: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:52 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:52 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:52 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:14:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:52 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:14:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:52 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:53.298 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:53 smithi173 ceph-mon[74165]: Reconfiguring mon.smithi173 (monmap changed)... 2024-04-20T11:14:53.298 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:53 smithi173 ceph-mon[74165]: Reconfiguring daemon mon.smithi173 on smithi173 2024-04-20T11:14:53.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:53 smithi117 ceph-mon[99036]: Reconfiguring mon.smithi173 (monmap changed)... 2024-04-20T11:14:53.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:53 smithi117 ceph-mon[99036]: Reconfiguring daemon mon.smithi173 on smithi173 2024-04-20T11:14:54.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:54 smithi117 ceph-mon[99036]: pgmap v38: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:54 smithi173 ceph-mon[74165]: pgmap v38: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:55.521 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:55.521 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:55.521 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:55 smithi173 ceph-mon[74165]: Reconfiguring osd.3 (monmap changed)... 2024-04-20T11:14:55.521 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:14:55.522 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:55.522 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:55 smithi173 ceph-mon[74165]: Reconfiguring daemon osd.3 on smithi173 2024-04-20T11:14:55.522 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:55 smithi173 ceph-mon[74165]: pgmap v39: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:55 smithi117 ceph-mon[99036]: Reconfiguring osd.3 (monmap changed)... 2024-04-20T11:14:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:14:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:55 smithi117 ceph-mon[99036]: Reconfiguring daemon osd.3 on smithi173 2024-04-20T11:14:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:55 smithi117 ceph-mon[99036]: pgmap v39: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:57 smithi117 ceph-mon[99036]: Reconfiguring osd.4 (monmap changed)... 2024-04-20T11:14:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:14:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:57 smithi117 ceph-mon[99036]: Reconfiguring daemon osd.4 on smithi173 2024-04-20T11:14:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:57 smithi117 ceph-mon[99036]: pgmap v40: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:57.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:57.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:14:57.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:57 smithi173 ceph-mon[74165]: Reconfiguring osd.4 (monmap changed)... 2024-04-20T11:14:57.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:14:57.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:57.896 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:57 smithi173 ceph-mon[74165]: Reconfiguring daemon osd.4 on smithi173 2024-04-20T11:14:57.897 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:57 smithi173 ceph-mon[74165]: pgmap v40: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:14:58.758 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:14:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:14:59.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:14:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:15:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:00 smithi117 ceph-mon[99036]: pgmap v41: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:00 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:00 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:00 smithi117 ceph-mon[99036]: Reconfiguring osd.5 (monmap changed)... 2024-04-20T11:15:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:00 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:15:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:00 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:00 smithi117 ceph-mon[99036]: Reconfiguring daemon osd.5 on smithi173 2024-04-20T11:15:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:00 smithi173 ceph-mon[74165]: pgmap v41: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:00 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:00 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:00.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:00 smithi173 ceph-mon[74165]: Reconfiguring osd.5 (monmap changed)... 2024-04-20T11:15:00.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:00 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:15:00.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:00 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:00.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:00 smithi173 ceph-mon[74165]: Reconfiguring daemon osd.5 on smithi173 2024-04-20T11:15:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:02 smithi117 ceph-mon[99036]: pgmap v42: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pybzht", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:15:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:02 smithi173 ceph-mon[74165]: pgmap v42: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:02 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:02 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:02 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pybzht", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:15:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:02 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:03.295 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:03 smithi173 ceph-mon[74165]: Reconfiguring mds.cephfs.smithi173.pybzht (monmap changed)... 2024-04-20T11:15:03.295 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:03 smithi173 ceph-mon[74165]: Reconfiguring daemon mds.cephfs.smithi173.pybzht on smithi173 2024-04-20T11:15:03.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:03 smithi117 ceph-mon[99036]: Reconfiguring mds.cephfs.smithi173.pybzht (monmap changed)... 2024-04-20T11:15:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:03 smithi117 ceph-mon[99036]: Reconfiguring daemon mds.cephfs.smithi173.pybzht on smithi173 2024-04-20T11:15:03.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.970+0000 7fb759e00700 1 -- 172.21.15.117:0/3886826881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075410 msgr2=0x7fb7540757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:03.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.970+0000 7fb759e00700 1 --2- 172.21.15.117:0/3886826881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075410 0x7fb7540757f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb744009910 tx=0x7fb744009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:03.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.970+0000 7fb759e00700 1 -- 172.21.15.117:0/3886826881 shutdown_connections 2024-04-20T11:15:03.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.970+0000 7fb759e00700 1 --2- 172.21.15.117:0/3886826881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb754075d30 0x7fb754111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:03.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.970+0000 7fb759e00700 1 --2- 172.21.15.117:0/3886826881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075410 0x7fb7540757f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:03.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.970+0000 7fb759e00700 1 -- 172.21.15.117:0/3886826881 >> 172.21.15.117:0/3886826881 conn(0x7fb7540fe8d0 msgr2=0x7fb754100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:03.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.970+0000 7fb759e00700 1 -- 172.21.15.117:0/3886826881 shutdown_connections 2024-04-20T11:15:03.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.970+0000 7fb759e00700 1 -- 172.21.15.117:0/3886826881 wait complete. 2024-04-20T11:15:03.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.971+0000 7fb759e00700 1 Processor -- start 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.972+0000 7fb759e00700 1 -- start start 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.972+0000 7fb759e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb754075410 0x7fb75419d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.972+0000 7fb759e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075d30 0x7fb75419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.972+0000 7fb759e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75419dd80 con 0x7fb754075d30 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.972+0000 7fb759e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7541a1a00 con 0x7fb754075410 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.973+0000 7fb752000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075d30 0x7fb75419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.973+0000 7fb752000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075d30 0x7fb75419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52310/0 (socket says 172.21.15.117:52310) 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.973+0000 7fb752000700 1 -- 172.21.15.117:0/859511517 learned_addr learned my addr 172.21.15.117:0/859511517 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.973+0000 7fb752a00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb754075410 0x7fb75419d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.973+0000 7fb752000700 1 -- 172.21.15.117:0/859511517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb754075410 msgr2=0x7fb75419d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.973+0000 7fb752000700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb754075410 0x7fb75419d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:03.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.973+0000 7fb752000700 1 -- 172.21.15.117:0/859511517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440095f0 con 0x7fb754075d30 2024-04-20T11:15:03.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.973+0000 7fb752000700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075d30 0x7fb75419d6a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb74000e920 tx=0x7fb74000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:03.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.974+0000 7fb758a00700 1 -- 172.21.15.117:0/859511517 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74000c9d0 con 0x7fb754075d30 2024-04-20T11:15:03.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.974+0000 7fb758a00700 1 -- 172.21.15.117:0/859511517 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb740010430 con 0x7fb754075d30 2024-04-20T11:15:03.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.974+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7541a1ce0 con 0x7fb754075d30 2024-04-20T11:15:03.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.974+0000 7fb758a00700 1 -- 172.21.15.117:0/859511517 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb740018630 con 0x7fb754075d30 2024-04-20T11:15:03.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.974+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7541a2230 con 0x7fb754075d30 2024-04-20T11:15:03.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.975+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb754076b50 con 0x7fb754075d30 2024-04-20T11:15:03.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.976+0000 7fb758a00700 1 -- 172.21.15.117:0/859511517 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fb7400105a0 con 0x7fb754075d30 2024-04-20T11:15:03.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.978+0000 7fb758a00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb73806edf0 0x7fb7380712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:03.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.978+0000 7fb758a00700 1 -- 172.21.15.117:0/859511517 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fb740014070 con 0x7fb754075d30 2024-04-20T11:15:03.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.978+0000 7fb752a00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb73806edf0 0x7fb7380712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:03.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.979+0000 7fb752a00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb73806edf0 0x7fb7380712b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb744009fd0 tx=0x7fb744005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:03.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:03.983+0000 7fb758a00700 1 -- 172.21.15.117:0/859511517 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb740053ff0 con 0x7fb754075d30 2024-04-20T11:15:04.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.256+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb754114440 con 0x7fb73806edf0 2024-04-20T11:15:04.257 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:04 smithi117 ceph-mon[99036]: pgmap v43: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:04.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:04.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:04.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pfrgat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:15:04.258 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:04.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.259+0000 7fb758a00700 1 -- 172.21.15.117:0/859511517 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fb754114440 con 0x7fb73806edf0 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb73806edf0 msgr2=0x7fb7380712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb73806edf0 0x7fb7380712b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb744009fd0 tx=0x7fb744005a90 comp rx=0 tx=0).stop 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075d30 msgr2=0x7fb75419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075d30 0x7fb75419d6a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb74000e920 tx=0x7fb74000ec30 comp rx=0 tx=0).stop 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 shutdown_connections 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb73806edf0 0x7fb7380712b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb754075410 0x7fb75419d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 --2- 172.21.15.117:0/859511517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb754075d30 0x7fb75419d6a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.262+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 >> 172.21.15.117:0/859511517 conn(0x7fb7540fe8d0 msgr2=0x7fb7540ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:04.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.263+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 shutdown_connections 2024-04-20T11:15:04.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.263+0000 7fb759e00700 1 -- 172.21.15.117:0/859511517 wait complete. 2024-04-20T11:15:04.280 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:15:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:04 smithi173 ceph-mon[74165]: pgmap v43: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pfrgat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:15:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:04.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.423+0000 7fcc8f600700 1 -- 172.21.15.117:0/3115413119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 msgr2=0x7fcc88102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.423+0000 7fcc8f600700 1 --2- 172.21.15.117:0/3115413119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 0x7fcc88102230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009910 tx=0x7fcc7c009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.423+0000 7fcc8f600700 1 -- 172.21.15.117:0/3115413119 shutdown_connections 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.423+0000 7fcc8f600700 1 --2- 172.21.15.117:0/3115413119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc88102770 0x7fcc8810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.423+0000 7fcc8f600700 1 --2- 172.21.15.117:0/3115413119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 0x7fcc88102230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.423+0000 7fcc8f600700 1 -- 172.21.15.117:0/3115413119 >> 172.21.15.117:0/3115413119 conn(0x7fcc880fb600 msgr2=0x7fcc880fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.424+0000 7fcc8f600700 1 -- 172.21.15.117:0/3115413119 shutdown_connections 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.424+0000 7fcc8f600700 1 -- 172.21.15.117:0/3115413119 wait complete. 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.425+0000 7fcc8f600700 1 Processor -- start 2024-04-20T11:15:04.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.425+0000 7fcc8f600700 1 -- start start 2024-04-20T11:15:04.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.425+0000 7fcc8f600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 0x7fcc8819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:04.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.425+0000 7fcc8f600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc88102770 0x7fcc8819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:04.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.425+0000 7fcc8f600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8819d370 con 0x7fcc88101e50 2024-04-20T11:15:04.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.425+0000 7fcc8f600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc88196950 con 0x7fcc88102770 2024-04-20T11:15:04.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.426+0000 7fcc8cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 0x7fcc8819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:04.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.426+0000 7fcc8cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 0x7fcc8819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52330/0 (socket says 172.21.15.117:52330) 2024-04-20T11:15:04.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.426+0000 7fcc87e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc88102770 0x7fcc8819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:04.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.426+0000 7fcc8cc00700 1 -- 172.21.15.117:0/88943947 learned_addr learned my addr 172.21.15.117:0/88943947 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.426+0000 7fcc8cc00700 1 -- 172.21.15.117:0/88943947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc88102770 msgr2=0x7fcc8819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.426+0000 7fcc8cc00700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc88102770 0x7fcc8819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.426+0000 7fcc8cc00700 1 -- 172.21.15.117:0/88943947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0095f0 con 0x7fcc88101e50 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.427+0000 7fcc8cc00700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 0x7fcc8819c810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c006010 tx=0x7fcc7c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.427+0000 7fcc85600700 1 -- 172.21.15.117:0/88943947 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7c01d070 con 0x7fcc88101e50 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.427+0000 7fcc85600700 1 -- 172.21.15.117:0/88943947 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc7c0056e0 con 0x7fcc88101e50 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.427+0000 7fcc85600700 1 -- 172.21.15.117:0/88943947 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7c00fdf0 con 0x7fcc88101e50 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.427+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc88196bd0 con 0x7fcc88101e50 2024-04-20T11:15:04.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.427+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc88197070 con 0x7fcc88101e50 2024-04-20T11:15:04.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.429+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc88105510 con 0x7fcc88101e50 2024-04-20T11:15:04.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.429+0000 7fcc85600700 1 -- 172.21.15.117:0/88943947 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fcc7c00f5f0 con 0x7fcc88101e50 2024-04-20T11:15:04.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.430+0000 7fcc85600700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7006ee00 0x7fcc700712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:04.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.430+0000 7fcc85600700 1 -- 172.21.15.117:0/88943947 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fcc7c090390 con 0x7fcc88101e50 2024-04-20T11:15:04.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.431+0000 7fcc87e00700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7006ee00 0x7fcc700712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:04.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.431+0000 7fcc87e00700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7006ee00 0x7fcc700712c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcc88197b10 tx=0x7fcc740073d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:04.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.436+0000 7fcc85600700 1 -- 172.21.15.117:0/88943947 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcc7c05cb10 con 0x7fcc88101e50 2024-04-20T11:15:04.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.684+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc88197e10 con 0x7fcc7006ee00 2024-04-20T11:15:04.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.686+0000 7fcc85600700 1 -- 172.21.15.117:0/88943947 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fcc88197e10 con 0x7fcc7006ee00 2024-04-20T11:15:04.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.689+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7006ee00 msgr2=0x7fcc700712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:04.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.689+0000 7fcc8f600700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7006ee00 0x7fcc700712c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcc88197b10 tx=0x7fcc740073d0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.689+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 msgr2=0x7fcc8819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.689+0000 7fcc8f600700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 0x7fcc8819c810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c006010 tx=0x7fcc7c00bd70 comp rx=0 tx=0).stop 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.690+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 shutdown_connections 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.690+0000 7fcc8f600700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7006ee00 0x7fcc700712c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.690+0000 7fcc8f600700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc88101e50 0x7fcc8819c810 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.690+0000 7fcc8f600700 1 --2- 172.21.15.117:0/88943947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc88102770 0x7fcc8819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.690+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 >> 172.21.15.117:0/88943947 conn(0x7fcc880fb600 msgr2=0x7fcc881030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.690+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 shutdown_connections 2024-04-20T11:15:04.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.690+0000 7fcc8f600700 1 -- 172.21.15.117:0/88943947 wait complete. 2024-04-20T11:15:04.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.848+0000 7f5751200700 1 -- 172.21.15.117:0/1677503048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c102e60 msgr2=0x7f574c103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:04.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.848+0000 7f5751200700 1 --2- 172.21.15.117:0/1677503048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c102e60 0x7f574c103240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f573c009910 tx=0x7f573c009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:04.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.849+0000 7f5751200700 1 -- 172.21.15.117:0/1677503048 shutdown_connections 2024-04-20T11:15:04.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.849+0000 7f5751200700 1 --2- 172.21.15.117:0/1677503048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f574c103810 0x7f574c107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.849+0000 7f5751200700 1 --2- 172.21.15.117:0/1677503048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c102e60 0x7f574c103240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.849+0000 7f5751200700 1 -- 172.21.15.117:0/1677503048 >> 172.21.15.117:0/1677503048 conn(0x7f574c0fe710 msgr2=0x7f574c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:04.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.849+0000 7f5751200700 1 -- 172.21.15.117:0/1677503048 shutdown_connections 2024-04-20T11:15:04.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.849+0000 7f5751200700 1 -- 172.21.15.117:0/1677503048 wait complete. 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.850+0000 7f5751200700 1 Processor -- start 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.850+0000 7f5751200700 1 -- start start 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.850+0000 7f5751200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f574c102e60 0x7f574c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.850+0000 7f5751200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c103810 0x7f574c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.850+0000 7f5751200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c19f380 con 0x7f574c103810 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.850+0000 7f5751200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c1989b0 con 0x7f574c102e60 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.851+0000 7f574aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c103810 0x7f574c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.851+0000 7f574aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c103810 0x7f574c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52350/0 (socket says 172.21.15.117:52350) 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.851+0000 7f574aa00700 1 -- 172.21.15.117:0/1941503124 learned_addr learned my addr 172.21.15.117:0/1941503124 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.851+0000 7f574b400700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f574c102e60 0x7f574c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.851+0000 7f574aa00700 1 -- 172.21.15.117:0/1941503124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f574c102e60 msgr2=0x7f574c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.851+0000 7f574aa00700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f574c102e60 0x7f574c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.851+0000 7f574aa00700 1 -- 172.21.15.117:0/1941503124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0095f0 con 0x7f574c103810 2024-04-20T11:15:04.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.852+0000 7f574aa00700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c103810 0x7f574c19ed60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f573800c8f0 tx=0x7f573800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:04.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.852+0000 7f5743e00700 1 -- 172.21.15.117:0/1941503124 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5738007a50 con 0x7f574c103810 2024-04-20T11:15:04.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.852+0000 7f5743e00700 1 -- 172.21.15.117:0/1941503124 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5738007bb0 con 0x7f574c103810 2024-04-20T11:15:04.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.852+0000 7f5743e00700 1 -- 172.21.15.117:0/1941503124 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5738018630 con 0x7f574c103810 2024-04-20T11:15:04.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.852+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c198c90 con 0x7f574c103810 2024-04-20T11:15:04.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.852+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c1991e0 con 0x7f574c103810 2024-04-20T11:15:04.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.853+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c10b200 con 0x7f574c103810 2024-04-20T11:15:04.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.854+0000 7f5743e00700 1 -- 172.21.15.117:0/1941503124 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f5738007d20 con 0x7f574c103810 2024-04-20T11:15:04.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.855+0000 7f5743e00700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ee00 0x7f57300712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:04.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.856+0000 7f5743e00700 1 -- 172.21.15.117:0/1941503124 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f573808ef10 con 0x7f574c103810 2024-04-20T11:15:04.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.856+0000 7f574b400700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ee00 0x7f57300712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:04.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.856+0000 7f574b400700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ee00 0x7f57300712c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f573c00b5c0 tx=0x7f573c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:04.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:04.861+0000 7f5743e00700 1 -- 172.21.15.117:0/1941503124 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f573805b730 con 0x7f574c103810 2024-04-20T11:15:05.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.093+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f574c199e90 con 0x7f573006ee00 2024-04-20T11:15:05.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.110+0000 7f5743e00700 1 -- 172.21.15.117:0/1941503124 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f574c199e90 con 0x7f573006ee00 2024-04-20T11:15:05.111 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (3m) 74s ago 17m 14.4M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (17m) 74s ago 17m 8388k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (16m) 50s ago 16m 8174k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (17m) 74s ago 17m 6669k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa8976d3b4d0 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (16m) 50s ago 16m 6665k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 1d91d7c73c1b 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (2m) 74s ago 17m 75.3M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (10m) 74s ago 10m 16.8M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (10m) 74s ago 10m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (10m) 50s ago 10m 16.6M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (10m) 50s ago 10m 24.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (6m) 74s ago 19m 479M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (5m) 50s ago 16m 449M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (92s) 74s ago 19m 30.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (56s) 50s ago 16m 16.7M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:15:05.112 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (4m) 74s ago 17m 7402k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:15:05.113 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (4m) 50s ago 16m 7260k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:15:05.113 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (15m) 74s ago 15m 211M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:15:05.113 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (14m) 74s ago 14m 218M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:15:05.113 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (13m) 74s ago 13m 182M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:15:05.113 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (13m) 50s ago 13m 243M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:15:05.113 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (12m) 50s ago 12m 214M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:15:05.113 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (12m) 50s ago 12m 201M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:15:05.113 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (4m) 74s ago 17m 62.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:15:05.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ee00 msgr2=0x7f57300712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:05.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ee00 0x7f57300712c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f573c00b5c0 tx=0x7f573c01a040 comp rx=0 tx=0).stop 2024-04-20T11:15:05.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c103810 msgr2=0x7f574c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:05.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c103810 0x7f574c19ed60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f573800c8f0 tx=0x7f573800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 shutdown_connections 2024-04-20T11:15:05.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ee00 0x7f57300712c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f574c102e60 0x7f574c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 --2- 172.21.15.117:0/1941503124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574c103810 0x7f574c19ed60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.113+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 >> 172.21.15.117:0/1941503124 conn(0x7f574c0fe710 msgr2=0x7f574c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:05.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.114+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 shutdown_connections 2024-04-20T11:15:05.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.114+0000 7f5751200700 1 -- 172.21.15.117:0/1941503124 wait complete. 2024-04-20T11:15:05.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.263+0000 7f3c15000700 1 -- 172.21.15.117:0/1734979363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10108e10 msgr2=0x7f3c10109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:05.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.263+0000 7f3c15000700 1 --2- 172.21.15.117:0/1734979363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10108e10 0x7f3c10109290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009910 tx=0x7f3bfc009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:05.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.263+0000 7f3c15000700 1 -- 172.21.15.117:0/1734979363 shutdown_connections 2024-04-20T11:15:05.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.263+0000 7f3c15000700 1 --2- 172.21.15.117:0/1734979363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10108e10 0x7f3c10109290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.263+0000 7f3c15000700 1 --2- 172.21.15.117:0/1734979363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10107520 0x7f3c10107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.263+0000 7f3c15000700 1 -- 172.21.15.117:0/1734979363 >> 172.21.15.117:0/1734979363 conn(0x7f3c100768f0 msgr2=0x7f3c10078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:05.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.263+0000 7f3c15000700 1 -- 172.21.15.117:0/1734979363 shutdown_connections 2024-04-20T11:15:05.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.263+0000 7f3c15000700 1 -- 172.21.15.117:0/1734979363 wait complete. 2024-04-20T11:15:05.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.264+0000 7f3c15000700 1 Processor -- start 2024-04-20T11:15:05.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.264+0000 7f3c15000700 1 -- start start 2024-04-20T11:15:05.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.265+0000 7f3c15000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10107520 0x7f3c1019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:05.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.265+0000 7f3c15000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10108e10 0x7f3c1019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:05.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.265+0000 7f3c15000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1019ddf0 con 0x7f3c10107520 2024-04-20T11:15:05.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.265+0000 7f3c15000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101a1ac0 con 0x7f3c10108e10 2024-04-20T11:15:05.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.265+0000 7f3c0de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10107520 0x7f3c1019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:05.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.265+0000 7f3c0de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10107520 0x7f3c1019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52364/0 (socket says 172.21.15.117:52364) 2024-04-20T11:15:05.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.265+0000 7f3c0de00700 1 -- 172.21.15.117:0/675453507 learned_addr learned my addr 172.21.15.117:0/675453507 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:05.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.265+0000 7f3c0d400700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10108e10 0x7f3c1019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:05.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.266+0000 7f3c0de00700 1 -- 172.21.15.117:0/675453507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10108e10 msgr2=0x7f3c1019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:05.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.266+0000 7f3c0de00700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10108e10 0x7f3c1019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.266+0000 7f3c0de00700 1 -- 172.21.15.117:0/675453507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0095f0 con 0x7f3c10107520 2024-04-20T11:15:05.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.266+0000 7f3c0de00700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10107520 0x7f3c1019d1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000d8d0 tx=0x7f3c0000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:05.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.266+0000 7f3c06a00700 1 -- 172.21.15.117:0/675453507 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c000098a0 con 0x7f3c10107520 2024-04-20T11:15:05.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.266+0000 7f3c06a00700 1 -- 172.21.15.117:0/675453507 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c00010460 con 0x7f3c10107520 2024-04-20T11:15:05.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.266+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c101a1da0 con 0x7f3c10107520 2024-04-20T11:15:05.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.266+0000 7f3c06a00700 1 -- 172.21.15.117:0/675453507 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0000f5f0 con 0x7f3c10107520 2024-04-20T11:15:05.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.267+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c101a2340 con 0x7f3c10107520 2024-04-20T11:15:05.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.268+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1010f890 con 0x7f3c10107520 2024-04-20T11:15:05.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.269+0000 7f3c06a00700 1 -- 172.21.15.117:0/675453507 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f3c00009a00 con 0x7f3c10107520 2024-04-20T11:15:05.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.270+0000 7f3c06a00700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3bf406edf0 0x7f3bf40712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:05.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.270+0000 7f3c06a00700 1 -- 172.21.15.117:0/675453507 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f3c0008eee0 con 0x7f3c10107520 2024-04-20T11:15:05.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.270+0000 7f3c0d400700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3bf406edf0 0x7f3bf40712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:05.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.271+0000 7f3c0d400700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3bf406edf0 0x7f3bf40712b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006010 tx=0x7f3bfc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:05.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.275+0000 7f3c06a00700 1 -- 172.21.15.117:0/675453507 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3c00057b50 con 0x7f3c10107520 2024-04-20T11:15:05.335 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:05 smithi173 ceph-mon[74165]: Reconfiguring mds.cephfs.smithi173.pfrgat (monmap changed)... 2024-04-20T11:15:05.335 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:05 smithi173 ceph-mon[74165]: Reconfiguring daemon mds.cephfs.smithi173.pfrgat on smithi173 2024-04-20T11:15:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:05 smithi117 ceph-mon[99036]: Reconfiguring mds.cephfs.smithi173.pfrgat (monmap changed)... 2024-04-20T11:15:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:05 smithi117 ceph-mon[99036]: Reconfiguring daemon mds.cephfs.smithi173.pfrgat on smithi173 2024-04-20T11:15:05.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.585+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3c1019e4e0 con 0x7f3c10107520 2024-04-20T11:15:05.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.586+0000 7f3c06a00700 1 -- 172.21.15.117:0/675453507 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f3c0005b210 con 0x7f3c10107520 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:15:05.587 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:15:05.588 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 10, 2024-04-20T11:15:05.588 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:15:05.588 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:15:05.588 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:15:05.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.588+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3bf406edf0 msgr2=0x7f3bf40712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:05.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.588+0000 7f3c15000700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3bf406edf0 0x7f3bf40712b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006010 tx=0x7f3bfc01a040 comp rx=0 tx=0).stop 2024-04-20T11:15:05.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.588+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10107520 msgr2=0x7f3c1019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:05.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.588+0000 7f3c15000700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10107520 0x7f3c1019d1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000d8d0 tx=0x7f3c0000dbe0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.588+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 shutdown_connections 2024-04-20T11:15:05.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.589+0000 7f3c15000700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3bf406edf0 0x7f3bf40712b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.589+0000 7f3c15000700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c10107520 0x7f3c1019d1d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.589+0000 7f3c15000700 1 --2- 172.21.15.117:0/675453507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10108e10 0x7f3c1019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.589+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 >> 172.21.15.117:0/675453507 conn(0x7f3c100768f0 msgr2=0x7f3c10077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:05.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.589+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 shutdown_connections 2024-04-20T11:15:05.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.589+0000 7f3c15000700 1 -- 172.21.15.117:0/675453507 wait complete. 2024-04-20T11:15:05.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.738+0000 7f2ad1200700 1 -- 172.21.15.117:0/3481674428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc103110 msgr2=0x7f2acc1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:05.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.738+0000 7f2ad1200700 1 --2- 172.21.15.117:0/3481674428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc103110 0x7f2acc1034f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009910 tx=0x7f2abc009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:05.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.738+0000 7f2ad1200700 1 -- 172.21.15.117:0/3481674428 shutdown_connections 2024-04-20T11:15:05.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.738+0000 7f2ad1200700 1 --2- 172.21.15.117:0/3481674428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2acc104a00 0x7f2acc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.738+0000 7f2ad1200700 1 --2- 172.21.15.117:0/3481674428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc103110 0x7f2acc1034f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.738+0000 7f2ad1200700 1 -- 172.21.15.117:0/3481674428 >> 172.21.15.117:0/3481674428 conn(0x7f2acc0fe880 msgr2=0x7f2acc100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:05.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.738+0000 7f2ad1200700 1 -- 172.21.15.117:0/3481674428 shutdown_connections 2024-04-20T11:15:05.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.738+0000 7f2ad1200700 1 -- 172.21.15.117:0/3481674428 wait complete. 2024-04-20T11:15:05.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.739+0000 7f2ad1200700 1 Processor -- start 2024-04-20T11:15:05.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.739+0000 7f2ad1200700 1 -- start start 2024-04-20T11:15:05.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.739+0000 7f2ad1200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2acc103110 0x7f2acc198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:05.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ad1200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc104a00 0x7f2acc199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:05.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ad1200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc1999e0 con 0x7f2acc104a00 2024-04-20T11:15:05.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ad1200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc19d660 con 0x7f2acc103110 2024-04-20T11:15:05.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ac1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc104a00 0x7f2acc199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:05.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ac1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc104a00 0x7f2acc199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52376/0 (socket says 172.21.15.117:52376) 2024-04-20T11:15:05.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ac1400700 1 -- 172.21.15.117:0/2536065709 learned_addr learned my addr 172.21.15.117:0/2536065709 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ac9e00700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2acc103110 0x7f2acc198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ac1400700 1 -- 172.21.15.117:0/2536065709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2acc103110 msgr2=0x7f2acc198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ac1400700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2acc103110 0x7f2acc198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.740+0000 7f2ac1400700 1 -- 172.21.15.117:0/2536065709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc0095f0 con 0x7f2acc104a00 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.741+0000 7f2ac1400700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc104a00 0x7f2acc199300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400d8d0 tx=0x7f2ab400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.741+0000 7f2ac3400700 1 -- 172.21.15.117:0/2536065709 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ab4009940 con 0x7f2acc104a00 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.741+0000 7f2ac3400700 1 -- 172.21.15.117:0/2536065709 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ab4010460 con 0x7f2acc104a00 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.741+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc19d940 con 0x7f2acc104a00 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.741+0000 7f2ac3400700 1 -- 172.21.15.117:0/2536065709 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ab400f5f0 con 0x7f2acc104a00 2024-04-20T11:15:05.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.741+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc19df10 con 0x7f2acc104a00 2024-04-20T11:15:05.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.742+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc10b460 con 0x7f2acc104a00 2024-04-20T11:15:05.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.743+0000 7f2ac3400700 1 -- 172.21.15.117:0/2536065709 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f2ab40105d0 con 0x7f2acc104a00 2024-04-20T11:15:05.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.744+0000 7f2ac3400700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2aac06edf0 0x7f2aac0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:05.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.744+0000 7f2ac3400700 1 -- 172.21.15.117:0/2536065709 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f2ab408ef20 con 0x7f2acc104a00 2024-04-20T11:15:05.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.744+0000 7f2ac9e00700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2aac06edf0 0x7f2aac0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:05.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.745+0000 7f2ac9e00700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2aac06edf0 0x7f2aac0712b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2abc006010 tx=0x7f2abc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:05.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:05.749+0000 7f2ac3400700 1 -- 172.21.15.117:0/2536065709 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2ab40c3ff0 con 0x7f2acc104a00 2024-04-20T11:15:06.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.013+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2acc19a080 con 0x7f2acc104a00 2024-04-20T11:15:06.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.014+0000 7f2ac3400700 1 -- 172.21.15.117:0/2536065709 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7f2ab4016780 con 0x7f2acc104a00 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:15:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:15:06.017 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:15:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:15:06.019 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:15:06.019 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:15:06.019 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:15:06.019 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:15:06.019 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:15:06.019 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2aac06edf0 msgr2=0x7f2aac0712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2aac06edf0 0x7f2aac0712b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2abc006010 tx=0x7f2abc00b540 comp rx=0 tx=0).stop 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc104a00 msgr2=0x7f2acc199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc104a00 0x7f2acc199300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400d8d0 tx=0x7f2ab400dc90 comp rx=0 tx=0).stop 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 shutdown_connections 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2aac06edf0 0x7f2aac0712b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2acc103110 0x7f2acc198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 --2- 172.21.15.117:0/2536065709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2acc104a00 0x7f2acc199300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 >> 172.21.15.117:0/2536065709 conn(0x7f2acc0fe880 msgr2=0x7f2acc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:06.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 shutdown_connections 2024-04-20T11:15:06.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.017+0000 7f2ad1200700 1 -- 172.21.15.117:0/2536065709 wait complete. 2024-04-20T11:15:06.021 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:15:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.174+0000 7fe047e00700 1 -- 172.21.15.117:0/3809629969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073ba0 msgr2=0x7fe0481114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.174+0000 7fe047e00700 1 --2- 172.21.15.117:0/3809629969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073ba0 0x7fe0481114a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe030009910 tx=0x7fe030009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.174+0000 7fe047e00700 1 -- 172.21.15.117:0/3809629969 shutdown_connections 2024-04-20T11:15:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.174+0000 7fe047e00700 1 --2- 172.21.15.117:0/3809629969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073ba0 0x7fe0481114a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.174+0000 7fe047e00700 1 --2- 172.21.15.117:0/3809629969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe048073280 0x7fe048073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.174+0000 7fe047e00700 1 -- 172.21.15.117:0/3809629969 >> 172.21.15.117:0/3809629969 conn(0x7fe0480fc500 msgr2=0x7fe0480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.174+0000 7fe047e00700 1 -- 172.21.15.117:0/3809629969 shutdown_connections 2024-04-20T11:15:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.174+0000 7fe047e00700 1 -- 172.21.15.117:0/3809629969 wait complete. 2024-04-20T11:15:06.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.175+0000 7fe047e00700 1 Processor -- start 2024-04-20T11:15:06.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.175+0000 7fe047e00700 1 -- start start 2024-04-20T11:15:06.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.176+0000 7fe047e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073280 0x7fe04819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:06.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.176+0000 7fe047e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe048073ba0 0x7fe04819d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:06.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.176+0000 7fe047e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04819dc20 con 0x7fe048073280 2024-04-20T11:15:06.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.176+0000 7fe047e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481a18f0 con 0x7fe048073ba0 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.176+0000 7fe046a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073280 0x7fe04819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.176+0000 7fe046a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073280 0x7fe04819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52396/0 (socket says 172.21.15.117:52396) 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.176+0000 7fe046a00700 1 -- 172.21.15.117:0/1843487799 learned_addr learned my addr 172.21.15.117:0/1843487799 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.176+0000 7fe03e000700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe048073ba0 0x7fe04819d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe046a00700 1 -- 172.21.15.117:0/1843487799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe048073ba0 msgr2=0x7fe04819d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe046a00700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe048073ba0 0x7fe04819d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe046a00700 1 -- 172.21.15.117:0/1843487799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300095f0 con 0x7fe048073280 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe046a00700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073280 0x7fe04819d000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe03800ba50 tx=0x7fe03800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe03fe00700 1 -- 172.21.15.117:0/1843487799 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe03800ca00 con 0x7fe048073280 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe03fe00700 1 -- 172.21.15.117:0/1843487799 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe03800cb60 con 0x7fe048073280 2024-04-20T11:15:06.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe03fe00700 1 -- 172.21.15.117:0/1843487799 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0380075a0 con 0x7fe048073280 2024-04-20T11:15:06.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0481a1bd0 con 0x7fe048073280 2024-04-20T11:15:06.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.177+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0481a2120 con 0x7fe048073280 2024-04-20T11:15:06.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.178+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0480749c0 con 0x7fe048073280 2024-04-20T11:15:06.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.180+0000 7fe03fe00700 1 -- 172.21.15.117:0/1843487799 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fe03800ccd0 con 0x7fe048073280 2024-04-20T11:15:06.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.180+0000 7fe03fe00700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe02c06edf0 0x7fe02c0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:06.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.180+0000 7fe03fe00700 1 -- 172.21.15.117:0/1843487799 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fe038006e40 con 0x7fe048073280 2024-04-20T11:15:06.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.181+0000 7fe03e000700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe02c06edf0 0x7fe02c0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:06.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.181+0000 7fe03e000700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe02c06edf0 0x7fe02c0712b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe030006010 tx=0x7fe03000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.185+0000 7fe03fe00700 1 -- 172.21.15.117:0/1843487799 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe038057480 con 0x7fe048073280 2024-04-20T11:15:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:06 smithi117 ceph-mon[99036]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:06 smithi117 ceph-mon[99036]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:06 smithi117 ceph-mon[99036]: pgmap v44: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:06 smithi117 ceph-mon[99036]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:06 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/675453507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:06 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2536065709' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:15:06.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:06 smithi173 ceph-mon[74165]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:06.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:06 smithi173 ceph-mon[74165]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:06.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:06 smithi173 ceph-mon[74165]: pgmap v44: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:06.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:06 smithi173 ceph-mon[74165]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:06.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:06 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/675453507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:06 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2536065709' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:15:06.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.427+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0481142d0 con 0x7fe02c06edf0 2024-04-20T11:15:06.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.429+0000 7fe03fe00700 1 -- 172.21.15.117:0/1843487799 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fe0481142d0 con 0x7fe02c06edf0 2024-04-20T11:15:06.429 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:15:06.429 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:15:06.429 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:15:06.429 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:15:06.430 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:15:06.430 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:15:06.430 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:15:06.430 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:15:06.430 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "4/23 daemons upgraded", 2024-04-20T11:15:06.430 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading crash daemons", 2024-04-20T11:15:06.430 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:15:06.430 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:15:06.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe02c06edf0 msgr2=0x7fe02c0712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe02c06edf0 0x7fe02c0712b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe030006010 tx=0x7fe03000b540 comp rx=0 tx=0).stop 2024-04-20T11:15:06.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073280 msgr2=0x7fe04819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073280 0x7fe04819d000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe03800ba50 tx=0x7fe03800be10 comp rx=0 tx=0).stop 2024-04-20T11:15:06.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 shutdown_connections 2024-04-20T11:15:06.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe02c06edf0 0x7fe02c0712b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe048073280 0x7fe04819d000 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 --2- 172.21.15.117:0/1843487799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe048073ba0 0x7fe04819d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.431+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 >> 172.21.15.117:0/1843487799 conn(0x7fe0480fc500 msgr2=0x7fe048100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:06.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.432+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 shutdown_connections 2024-04-20T11:15:06.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.432+0000 7fe047e00700 1 -- 172.21.15.117:0/1843487799 wait complete. 2024-04-20T11:15:06.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.578+0000 7fe7c7c00700 1 -- 172.21.15.117:0/1307471861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0101d40 msgr2=0x7fe7c0102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.578+0000 7fe7c7c00700 1 --2- 172.21.15.117:0/1307471861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0101d40 0x7fe7c0102120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400b0a0 tx=0x7fe7b400b3b0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.578+0000 7fe7c7c00700 1 -- 172.21.15.117:0/1307471861 shutdown_connections 2024-04-20T11:15:06.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.578+0000 7fe7c7c00700 1 --2- 172.21.15.117:0/1307471861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7c0102660 0x7fe7c010ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.578+0000 7fe7c7c00700 1 --2- 172.21.15.117:0/1307471861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0101d40 0x7fe7c0102120 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.578+0000 7fe7c7c00700 1 -- 172.21.15.117:0/1307471861 >> 172.21.15.117:0/1307471861 conn(0x7fe7c0076750 msgr2=0x7fe7c00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:06.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.579+0000 7fe7c7c00700 1 -- 172.21.15.117:0/1307471861 shutdown_connections 2024-04-20T11:15:06.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.579+0000 7fe7c7c00700 1 -- 172.21.15.117:0/1307471861 wait complete. 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.580+0000 7fe7c7c00700 1 Processor -- start 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.580+0000 7fe7c7c00700 1 -- start start 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.580+0000 7fe7c7c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7c0101d40 0x7fe7c019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.580+0000 7fe7c7c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0102660 0x7fe7c019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.580+0000 7fe7c7c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c019d480 con 0x7fe7c0102660 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.580+0000 7fe7c7c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0196a60 con 0x7fe7c0101d40 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.580+0000 7fe7bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0102660 0x7fe7c019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.581+0000 7fe7bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0102660 0x7fe7c019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52406/0 (socket says 172.21.15.117:52406) 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.581+0000 7fe7bfe00700 1 -- 172.21.15.117:0/2750140695 learned_addr learned my addr 172.21.15.117:0/2750140695 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.581+0000 7fe7c5200700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7c0101d40 0x7fe7c019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.581+0000 7fe7bfe00700 1 -- 172.21.15.117:0/2750140695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7c0101d40 msgr2=0x7fe7c019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.581+0000 7fe7bfe00700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7c0101d40 0x7fe7c019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.581+0000 7fe7bfe00700 1 -- 172.21.15.117:0/2750140695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b4009d00 con 0x7fe7c0102660 2024-04-20T11:15:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.582+0000 7fe7bfe00700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0102660 0x7fe7c019ce60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00b3d0 tx=0x7fe7ac00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.582+0000 7fe7bd600700 1 -- 172.21.15.117:0/2750140695 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7ac011800 con 0x7fe7c0102660 2024-04-20T11:15:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.582+0000 7fe7bd600700 1 -- 172.21.15.117:0/2750140695 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7ac011e20 con 0x7fe7c0102660 2024-04-20T11:15:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.582+0000 7fe7bd600700 1 -- 172.21.15.117:0/2750140695 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7ac010ea0 con 0x7fe7c0102660 2024-04-20T11:15:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.582+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c0196d40 con 0x7fe7c0102660 2024-04-20T11:15:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.582+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c0197290 con 0x7fe7c0102660 2024-04-20T11:15:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.583+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c0105530 con 0x7fe7c0102660 2024-04-20T11:15:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.584+0000 7fe7bd600700 1 -- 172.21.15.117:0/2750140695 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 39) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fe7ac010380 con 0x7fe7c0102660 2024-04-20T11:15:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.585+0000 7fe7bd600700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7a806ee00 0x7fe7a80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.585+0000 7fe7bd600700 1 -- 172.21.15.117:0/2750140695 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fe7ac08e930 con 0x7fe7c0102660 2024-04-20T11:15:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.586+0000 7fe7c5200700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7a806ee00 0x7fe7a80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.586+0000 7fe7c5200700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7a806ee00 0x7fe7a80712c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009850 tx=0x7fe7b40097c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:06.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.595+0000 7fe7bd600700 1 -- 172.21.15.117:0/2750140695 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe7ac05b260 con 0x7fe7c0102660 2024-04-20T11:15:06.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.907+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe7c0197f20 con 0x7fe7c0102660 2024-04-20T11:15:06.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.908+0000 7fe7bd600700 1 -- 172.21.15.117:0/2750140695 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe7ac05adf0 con 0x7fe7c0102660 2024-04-20T11:15:06.909 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:15:06.909 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:15:06.909 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7a806ee00 msgr2=0x7fe7a80712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7a806ee00 0x7fe7a80712c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009850 tx=0x7fe7b40097c0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0102660 msgr2=0x7fe7c019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0102660 0x7fe7c019ce60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00b3d0 tx=0x7fe7ac00b790 comp rx=0 tx=0).stop 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 shutdown_connections 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7a806ee00 0x7fe7a80712c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7c0101d40 0x7fe7c019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 --2- 172.21.15.117:0/2750140695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7c0102660 0x7fe7c019ce60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 >> 172.21.15.117:0/2750140695 conn(0x7fe7c0076750 msgr2=0x7fe7c0103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 shutdown_connections 2024-04-20T11:15:06.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:06.911+0000 7fe7c7c00700 1 -- 172.21.15.117:0/2750140695 wait complete. 2024-04-20T11:15:07.485 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:07.485 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:07.485 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:07.485 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:07.485 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:07.485 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all mon 2024-04-20T11:15:07.485 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:07.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]: dispatch 2024-04-20T11:15:07.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]': finished 2024-04-20T11:15:07.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi173"}]: dispatch 2024-04-20T11:15:07.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi173"}]': finished 2024-04-20T11:15:07.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:07.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2750140695' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:15:07.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:07 smithi117 ceph-mon[99036]: pgmap v45: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all mon 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]: dispatch 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi117"}]': finished 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi173"}]: dispatch 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi173"}]': finished 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2750140695' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:15:07.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:07 smithi173 ceph-mon[74165]: pgmap v45: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:10.014 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:09 smithi117 ceph-mon[99036]: Upgrade: Updating crash.smithi117 (1/2) 2024-04-20T11:15:10.014 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:10.014 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T11:15:10.014 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:10.014 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:09 smithi117 ceph-mon[99036]: Deploying daemon crash.smithi117 on smithi117 2024-04-20T11:15:10.014 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:09 smithi117 ceph-mon[99036]: pgmap v46: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:09 smithi173 ceph-mon[74165]: Upgrade: Updating crash.smithi117 (1/2) 2024-04-20T11:15:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi117", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T11:15:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:10.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:09 smithi173 ceph-mon[74165]: Deploying daemon crash.smithi117 on smithi117 2024-04-20T11:15:10.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:09 smithi173 ceph-mon[74165]: pgmap v46: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:12.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:12 smithi117 ceph-mon[99036]: pgmap v47: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:12 smithi173 ceph-mon[74165]: pgmap v47: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:14 smithi117 ceph-mon[99036]: pgmap v48: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:14.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:15:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:14 smithi173 ceph-mon[74165]: pgmap v48: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:15:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:15 smithi117 ceph-mon[99036]: pgmap v49: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:15.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:15.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:15.909 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:15 smithi173 ceph-mon[74165]: pgmap v49: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:16.855 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:16 smithi173 ceph-mon[74165]: mgrmap e40: smithi117.ivnklj(active, since 92s), standbys: smithi173.gnrjof 2024-04-20T11:15:17.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:16 smithi117 ceph-mon[99036]: mgrmap e40: smithi117.ivnklj(active, since 92s), standbys: smithi173.gnrjof 2024-04-20T11:15:18.146 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:17 smithi173 ceph-mon[74165]: pgmap v50: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:18.146 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:17 smithi173 ceph-mon[74165]: Upgrade: Updating crash.smithi173 (2/2) 2024-04-20T11:15:18.146 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:17 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:18.146 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:17 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T11:15:18.146 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:17 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:18.146 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:17 smithi173 ceph-mon[74165]: Deploying daemon crash.smithi173 on smithi173 2024-04-20T11:15:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:17 smithi117 ceph-mon[99036]: pgmap v50: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:17 smithi117 ceph-mon[99036]: Upgrade: Updating crash.smithi173 (2/2) 2024-04-20T11:15:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:17 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:17 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi173", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-20T11:15:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:17 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:17 smithi117 ceph-mon[99036]: Deploying daemon crash.smithi173 on smithi173 2024-04-20T11:15:20.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:19 smithi173 ceph-mon[74165]: pgmap v51: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:19 smithi117 ceph-mon[99036]: pgmap v51: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:22.136 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:21 smithi173 ceph-mon[74165]: pgmap v52: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:22.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:21 smithi117 ceph-mon[99036]: pgmap v52: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:23.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:23.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:23.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:23 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:23.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:23 smithi173 ceph-mon[74165]: pgmap v53: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:23.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:23 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:23 smithi117 ceph-mon[99036]: pgmap v53: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:26.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:26 smithi117 ceph-mon[99036]: pgmap v54: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:26 smithi173 ceph-mon[74165]: pgmap v54: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-20T11:15:28.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:27 smithi117 ceph-mon[99036]: pgmap v55: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:28.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:27 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:28.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:27 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:27 smithi173 ceph-mon[74165]: pgmap v55: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:27 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:27 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:29.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:15:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:15:30.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:30 smithi117 ceph-mon[99036]: pgmap v56: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:30.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:30 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:30.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:30 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:30.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:30 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:30.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:30 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:30.403 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:30 smithi173 ceph-mon[74165]: pgmap v56: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:30.404 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:30 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:30.404 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:30 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:30.404 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:30 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:30.404 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:30 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:32.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:32 smithi117 ceph-mon[99036]: pgmap v57: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:32.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:32.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:32 smithi173 ceph-mon[74165]: pgmap v57: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:34 smithi173 ceph-mon[74165]: pgmap v58: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:34.295 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:34 smithi117 ceph-mon[99036]: pgmap v58: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:35 smithi117 ceph-mon[99036]: pgmap v59: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:35 smithi173 ceph-mon[74165]: pgmap v59: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:37.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.088+0000 7f3d29200700 1 -- 172.21.15.117:0/10932029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 msgr2=0x7f3d24105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.088+0000 7f3d29200700 1 --2- 172.21.15.117:0/10932029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 0x7f3d24105ee0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009910 tx=0x7f3d14009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:37.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.088+0000 7f3d29200700 1 -- 172.21.15.117:0/10932029 shutdown_connections 2024-04-20T11:15:37.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.088+0000 7f3d29200700 1 --2- 172.21.15.117:0/10932029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d240fefd0 0x7f3d240ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.088+0000 7f3d29200700 1 --2- 172.21.15.117:0/10932029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 0x7f3d24105ee0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.088+0000 7f3d29200700 1 -- 172.21.15.117:0/10932029 >> 172.21.15.117:0/10932029 conn(0x7f3d24075eb0 msgr2=0x7f3d240fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.088+0000 7f3d29200700 1 -- 172.21.15.117:0/10932029 shutdown_connections 2024-04-20T11:15:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.088+0000 7f3d29200700 1 -- 172.21.15.117:0/10932029 wait complete. 2024-04-20T11:15:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.090+0000 7f3d29200700 1 Processor -- start 2024-04-20T11:15:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.090+0000 7f3d29200700 1 -- start start 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.090+0000 7f3d29200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d240fefd0 0x7f3d2419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.090+0000 7f3d29200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 0x7f3d2419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.090+0000 7f3d29200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2419dc80 con 0x7f3d24105b00 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.090+0000 7f3d29200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241a1950 con 0x7f3d240fefd0 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.091+0000 7f3d21400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 0x7f3d2419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.091+0000 7f3d21400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 0x7f3d2419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41360/0 (socket says 172.21.15.117:41360) 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.091+0000 7f3d21400700 1 -- 172.21.15.117:0/3072830919 learned_addr learned my addr 172.21.15.117:0/3072830919 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.091+0000 7f3d21e00700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d240fefd0 0x7f3d2419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.091+0000 7f3d21400700 1 -- 172.21.15.117:0/3072830919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d240fefd0 msgr2=0x7f3d2419d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.091+0000 7f3d21400700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d240fefd0 0x7f3d2419d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.091+0000 7f3d21400700 1 -- 172.21.15.117:0/3072830919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d140095f0 con 0x7f3d24105b00 2024-04-20T11:15:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.091+0000 7f3d21400700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 0x7f3d2419d5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000d8d0 tx=0x7f3d1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.092+0000 7f3d1aa00700 1 -- 172.21.15.117:0/3072830919 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d10009940 con 0x7f3d24105b00 2024-04-20T11:15:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.092+0000 7f3d1aa00700 1 -- 172.21.15.117:0/3072830919 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d10010460 con 0x7f3d24105b00 2024-04-20T11:15:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.092+0000 7f3d1aa00700 1 -- 172.21.15.117:0/3072830919 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1000f5f0 con 0x7f3d24105b00 2024-04-20T11:15:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.092+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d241a1c30 con 0x7f3d24105b00 2024-04-20T11:15:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.092+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241a2200 con 0x7f3d24105b00 2024-04-20T11:15:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.093+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d24101f60 con 0x7f3d24105b00 2024-04-20T11:15:37.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.095+0000 7f3d1aa00700 1 -- 172.21.15.117:0/3072830919 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3d100105d0 con 0x7f3d24105b00 2024-04-20T11:15:37.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.096+0000 7f3d1aa00700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0806ecc0 0x7f3d08071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.096+0000 7f3d1aa00700 1 -- 172.21.15.117:0/3072830919 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f3d1008ed60 con 0x7f3d24105b00 2024-04-20T11:15:37.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.096+0000 7f3d21e00700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0806ecc0 0x7f3d08071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.097+0000 7f3d21e00700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0806ecc0 0x7f3d08071180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400b5c0 tx=0x7f3d1401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.104+0000 7f3d1aa00700 1 -- 172.21.15.117:0/3072830919 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3d10093070 con 0x7f3d24105b00 2024-04-20T11:15:37.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.351+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d24114370 con 0x7f3d0806ecc0 2024-04-20T11:15:37.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.353+0000 7f3d1aa00700 1 -- 172.21.15.117:0/3072830919 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f3d24114370 con 0x7f3d0806ecc0 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0806ecc0 msgr2=0x7f3d08071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0806ecc0 0x7f3d08071180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400b5c0 tx=0x7f3d1401a040 comp rx=0 tx=0).stop 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 msgr2=0x7f3d2419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 0x7f3d2419d5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000d8d0 tx=0x7f3d1000dc90 comp rx=0 tx=0).stop 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 shutdown_connections 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0806ecc0 0x7f3d08071180 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d240fefd0 0x7f3d2419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 --2- 172.21.15.117:0/3072830919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24105b00 0x7f3d2419d5a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 >> 172.21.15.117:0/3072830919 conn(0x7f3d24075eb0 msgr2=0x7f3d240fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:37.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 shutdown_connections 2024-04-20T11:15:37.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.356+0000 7f3d29200700 1 -- 172.21.15.117:0/3072830919 wait complete. 2024-04-20T11:15:37.372 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:15:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.519+0000 7ff2a1e00700 1 -- 172.21.15.117:0/2388281387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 msgr2=0x7ff29c10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.519+0000 7ff2a1e00700 1 --2- 172.21.15.117:0/2388281387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 0x7ff29c10cf40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff288009910 tx=0x7ff288009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.519+0000 7ff2a1e00700 1 -- 172.21.15.117:0/2388281387 shutdown_connections 2024-04-20T11:15:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.519+0000 7ff2a1e00700 1 --2- 172.21.15.117:0/2388281387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 0x7ff29c10cf40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.519+0000 7ff2a1e00700 1 --2- 172.21.15.117:0/2388281387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0ff8d0 0x7ff29c0ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.519+0000 7ff2a1e00700 1 -- 172.21.15.117:0/2388281387 >> 172.21.15.117:0/2388281387 conn(0x7ff29c076750 msgr2=0x7ff29c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.520+0000 7ff2a1e00700 1 -- 172.21.15.117:0/2388281387 shutdown_connections 2024-04-20T11:15:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.520+0000 7ff2a1e00700 1 -- 172.21.15.117:0/2388281387 wait complete. 2024-04-20T11:15:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.522+0000 7ff2a1e00700 1 Processor -- start 2024-04-20T11:15:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.522+0000 7ff2a1e00700 1 -- start start 2024-04-20T11:15:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.523+0000 7ff2a1e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0ff8d0 0x7ff29c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.523+0000 7ff2a1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 0x7ff29c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.523+0000 7ff2a1e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c19f550 con 0x7ff29c100280 2024-04-20T11:15:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.523+0000 7ff2a1e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c198b30 con 0x7ff29c0ff8d0 2024-04-20T11:15:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.523+0000 7ff29a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 0x7ff29c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.523+0000 7ff29a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 0x7ff29c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41372/0 (socket says 172.21.15.117:41372) 2024-04-20T11:15:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.523+0000 7ff29a000700 1 -- 172.21.15.117:0/1395262819 learned_addr learned my addr 172.21.15.117:0/1395262819 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.523+0000 7ff29aa00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0ff8d0 0x7ff29c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.524+0000 7ff29a000700 1 -- 172.21.15.117:0/1395262819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0ff8d0 msgr2=0x7ff29c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.524+0000 7ff29a000700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0ff8d0 0x7ff29c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.524+0000 7ff29a000700 1 -- 172.21.15.117:0/1395262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880095f0 con 0x7ff29c100280 2024-04-20T11:15:37.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.524+0000 7ff29a000700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 0x7ff29c19ef30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff288006010 tx=0x7ff2880056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:37.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.524+0000 7ff2a0a00700 1 -- 172.21.15.117:0/1395262819 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff28801d070 con 0x7ff29c100280 2024-04-20T11:15:37.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.524+0000 7ff2a0a00700 1 -- 172.21.15.117:0/1395262819 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff28800bea0 con 0x7ff29c100280 2024-04-20T11:15:37.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.524+0000 7ff2a0a00700 1 -- 172.21.15.117:0/1395262819 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff28800fd70 con 0x7ff29c100280 2024-04-20T11:15:37.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.524+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c198db0 con 0x7ff29c100280 2024-04-20T11:15:37.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.525+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c199300 con 0x7ff29c100280 2024-04-20T11:15:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.526+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29c1078c0 con 0x7ff29c100280 2024-04-20T11:15:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.527+0000 7ff2a0a00700 1 -- 172.21.15.117:0/1395262819 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff28800f560 con 0x7ff29c100280 2024-04-20T11:15:37.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.528+0000 7ff2a0a00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28006ecc0 0x7ff280071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.528+0000 7ff2a0a00700 1 -- 172.21.15.117:0/1395262819 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7ff288090080 con 0x7ff29c100280 2024-04-20T11:15:37.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.529+0000 7ff29aa00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28006ecc0 0x7ff280071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.529+0000 7ff29aa00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28006ecc0 0x7ff280071180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005950 tx=0x7ff28c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:37.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.533+0000 7ff2a0a00700 1 -- 172.21.15.117:0/1395262819 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff28805c9a0 con 0x7ff29c100280 2024-04-20T11:15:37.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.790+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff29c199ff0 con 0x7ff28006ecc0 2024-04-20T11:15:37.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.793+0000 7ff2a0a00700 1 -- 172.21.15.117:0/1395262819 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7ff29c199ff0 con 0x7ff28006ecc0 2024-04-20T11:15:37.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28006ecc0 msgr2=0x7ff280071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28006ecc0 0x7ff280071180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005950 tx=0x7ff28c0058e0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 msgr2=0x7ff29c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 0x7ff29c19ef30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff288006010 tx=0x7ff2880056a0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 shutdown_connections 2024-04-20T11:15:37.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28006ecc0 0x7ff280071180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0ff8d0 0x7ff29c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 --2- 172.21.15.117:0/1395262819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff29c100280 0x7ff29c19ef30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 >> 172.21.15.117:0/1395262819 conn(0x7ff29c076750 msgr2=0x7ff29c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:37.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 shutdown_connections 2024-04-20T11:15:37.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.797+0000 7ff2a1e00700 1 -- 172.21.15.117:0/1395262819 wait complete. 2024-04-20T11:15:37.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.960+0000 7f7341200700 1 -- 172.21.15.117:0/310639596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 msgr2=0x7f733c10ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.960+0000 7f7341200700 1 --2- 172.21.15.117:0/310639596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 0x7f733c10ab50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7328009910 tx=0x7f7328009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:37.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.961+0000 7f7341200700 1 -- 172.21.15.117:0/310639596 shutdown_connections 2024-04-20T11:15:37.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.961+0000 7f7341200700 1 --2- 172.21.15.117:0/310639596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 0x7f733c10ab50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.961+0000 7f7341200700 1 --2- 172.21.15.117:0/310639596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733c101d40 0x7f733c102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.961+0000 7f7341200700 1 -- 172.21.15.117:0/310639596 >> 172.21.15.117:0/310639596 conn(0x7f733c076750 msgr2=0x7f733c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:37.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.961+0000 7f7341200700 1 -- 172.21.15.117:0/310639596 shutdown_connections 2024-04-20T11:15:37.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.961+0000 7f7341200700 1 -- 172.21.15.117:0/310639596 wait complete. 2024-04-20T11:15:37.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.962+0000 7f7341200700 1 Processor -- start 2024-04-20T11:15:37.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.962+0000 7f7341200700 1 -- start start 2024-04-20T11:15:37.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.963+0000 7f7341200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733c101d40 0x7f733c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.963+0000 7f7341200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 0x7f733c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.963+0000 7f7341200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c199a30 con 0x7f733c102660 2024-04-20T11:15:37.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.963+0000 7f7341200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c19d700 con 0x7f733c101d40 2024-04-20T11:15:37.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.963+0000 7f7339400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 0x7f733c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.963+0000 7f7339400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 0x7f733c199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41402/0 (socket says 172.21.15.117:41402) 2024-04-20T11:15:37.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.963+0000 7f7339400700 1 -- 172.21.15.117:0/572325080 learned_addr learned my addr 172.21.15.117:0/572325080 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:37.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.963+0000 7f7339e00700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733c101d40 0x7f733c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.964+0000 7f7339400700 1 -- 172.21.15.117:0/572325080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733c101d40 msgr2=0x7f733c198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:37.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.964+0000 7f7339400700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733c101d40 0x7f733c198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:37.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.964+0000 7f7339400700 1 -- 172.21.15.117:0/572325080 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280095f0 con 0x7f733c102660 2024-04-20T11:15:37.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.964+0000 7f7339400700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 0x7f733c199350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f73280092e0 tx=0x7f732800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:37.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.964+0000 7f7332a00700 1 -- 172.21.15.117:0/572325080 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f732801d070 con 0x7f733c102660 2024-04-20T11:15:37.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.964+0000 7f7332a00700 1 -- 172.21.15.117:0/572325080 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7328003d50 con 0x7f733c102660 2024-04-20T11:15:37.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.964+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c19d980 con 0x7f733c102660 2024-04-20T11:15:37.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.965+0000 7f7332a00700 1 -- 172.21.15.117:0/572325080 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7328017400 con 0x7f733c102660 2024-04-20T11:15:37.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.965+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c19de70 con 0x7f733c102660 2024-04-20T11:15:37.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.966+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733c10a6b0 con 0x7f733c102660 2024-04-20T11:15:37.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.967+0000 7f7332a00700 1 -- 172.21.15.117:0/572325080 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7328003890 con 0x7f733c102660 2024-04-20T11:15:37.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.968+0000 7f7332a00700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f732006ecc0 0x7f7320071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:37.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.968+0000 7f7332a00700 1 -- 172.21.15.117:0/572325080 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f73280901b0 con 0x7f733c102660 2024-04-20T11:15:37.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.968+0000 7f7339e00700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f732006ecc0 0x7f7320071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:37.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.969+0000 7f7339e00700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f732006ecc0 0x7f7320071180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f732c006fd0 tx=0x7f732c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:37.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:37.973+0000 7f7332a00700 1 -- 172.21.15.117:0/572325080 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f73280590f0 con 0x7f733c102660 2024-04-20T11:15:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: pgmap v60: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:38.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]: dispatch 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]': finished 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi173"}]: dispatch 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi173"}]': finished 2024-04-20T11:15:38.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:38 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-20T11:15:38.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.212+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f733c10de20 con 0x7f732006ecc0 2024-04-20T11:15:38.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.244+0000 7f7332a00700 1 -- 172.21.15.117:0/572325080 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f733c10de20 con 0x7f732006ecc0 2024-04-20T11:15:38.244 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:15:38.244 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (4m) 8s ago 18m 14.5M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:15:38.244 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (18m) 8s ago 18m 8514k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:15:38.244 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (17m) 11s ago 17m 8270k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:15:38.244 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (24s) 8s ago 18m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (16s) 11s ago 17m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (3m) 8s ago 17m 75.0M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (10m) 8s ago 10m 17.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (11m) 8s ago 10m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (10m) 11s ago 10m 17.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (10m) 11s ago 10m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (7m) 8s ago 19m 498M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (5m) 11s ago 16m 450M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (2m) 8s ago 20m 46.7M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (89s) 11s ago 16m 33.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (5m) 8s ago 18m 7432k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (5m) 11s ago 16m 7377k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (15m) 8s ago 15m 211M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 8e5aa5a80478 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (15m) 8s ago 15m 219M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:15:38.245 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (14m) 8s ago 14m 183M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:15:38.246 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (13m) 11s ago 13m 244M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:15:38.246 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (13m) 11s ago 13m 215M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:15:38.246 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (12m) 11s ago 12m 202M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:15:38.246 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (4m) 8s ago 17m 67.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:15:38.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f732006ecc0 msgr2=0x7f7320071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:38.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f732006ecc0 0x7f7320071180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f732c006fd0 tx=0x7f732c008040 comp rx=0 tx=0).stop 2024-04-20T11:15:38.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 msgr2=0x7f733c199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 0x7f733c199350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f73280092e0 tx=0x7f732800be20 comp rx=0 tx=0).stop 2024-04-20T11:15:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 shutdown_connections 2024-04-20T11:15:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f732006ecc0 0x7f7320071180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733c101d40 0x7f733c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 --2- 172.21.15.117:0/572325080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c102660 0x7f733c199350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 >> 172.21.15.117:0/572325080 conn(0x7f733c076750 msgr2=0x7f733c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 shutdown_connections 2024-04-20T11:15:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.247+0000 7f7341200700 1 -- 172.21.15.117:0/572325080 wait complete. 2024-04-20T11:15:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.396+0000 7f4121800700 1 -- 172.21.15.117:0/3829415919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 msgr2=0x7f411c068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.396+0000 7f4121800700 1 --2- 172.21.15.117:0/3829415919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 0x7f411c068950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f410c009910 tx=0x7f410c009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.396+0000 7f4121800700 1 -- 172.21.15.117:0/3829415919 shutdown_connections 2024-04-20T11:15:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.396+0000 7f4121800700 1 --2- 172.21.15.117:0/3829415919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f411c068e90 0x7f411c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.396+0000 7f4121800700 1 --2- 172.21.15.117:0/3829415919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 0x7f411c068950 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.396+0000 7f4121800700 1 -- 172.21.15.117:0/3829415919 >> 172.21.15.117:0/3829415919 conn(0x7f411c075480 msgr2=0x7f411c075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.396+0000 7f4121800700 1 -- 172.21.15.117:0/3829415919 shutdown_connections 2024-04-20T11:15:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.396+0000 7f4121800700 1 -- 172.21.15.117:0/3829415919 wait complete. 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.397+0000 7f4121800700 1 Processor -- start 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.397+0000 7f4121800700 1 -- start start 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.397+0000 7f4121800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 0x7f411c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f4121800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f411c068e90 0x7f411c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f4121800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c19f3b0 con 0x7f411c068570 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f4121800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1989e0 con 0x7f411c068e90 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f411be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 0x7f411c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f411be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 0x7f411c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41410/0 (socket says 172.21.15.117:41410) 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f411be00700 1 -- 172.21.15.117:0/584895035 learned_addr learned my addr 172.21.15.117:0/584895035 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f411b400700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f411c068e90 0x7f411c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:38.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f411be00700 1 -- 172.21.15.117:0/584895035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f411c068e90 msgr2=0x7f411c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:38.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f411be00700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f411c068e90 0x7f411c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.398+0000 7f411be00700 1 -- 172.21.15.117:0/584895035 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0095f0 con 0x7f411c068570 2024-04-20T11:15:38.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.399+0000 7f411be00700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 0x7f411c19e850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f410c0092e0 tx=0x7f410c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:38.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.399+0000 7f4118c00700 1 -- 172.21.15.117:0/584895035 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f410c01d070 con 0x7f411c068570 2024-04-20T11:15:38.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.399+0000 7f4118c00700 1 -- 172.21.15.117:0/584895035 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f410c00f460 con 0x7f411c068570 2024-04-20T11:15:38.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.399+0000 7f4118c00700 1 -- 172.21.15.117:0/584895035 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f410c017630 con 0x7f411c068570 2024-04-20T11:15:38.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.399+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c198c60 con 0x7f411c068570 2024-04-20T11:15:38.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.399+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c199150 con 0x7f411c068570 2024-04-20T11:15:38.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.400+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c10aea0 con 0x7f411c068570 2024-04-20T11:15:38.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.401+0000 7f4118c00700 1 -- 172.21.15.117:0/584895035 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f410c00f5d0 con 0x7f411c068570 2024-04-20T11:15:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.402+0000 7f4118c00700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f410006ecc0 0x7f4100071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.403+0000 7f4118c00700 1 -- 172.21.15.117:0/584895035 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f410c090480 con 0x7f411c068570 2024-04-20T11:15:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.403+0000 7f411b400700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f410006ecc0 0x7f4100071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:38.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.404+0000 7f411b400700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f410006ecc0 0x7f4100071180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4108005fd0 tx=0x7f410800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:38.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.407+0000 7f4118c00700 1 -- 172.21.15.117:0/584895035 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f410c05cea0 con 0x7f411c068570 2024-04-20T11:15:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: pgmap v60: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]: dispatch 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi117"}]': finished 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi173"}]: dispatch 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi173"}]': finished 2024-04-20T11:15:38.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:38 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-20T11:15:38.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.730+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f411c199e50 con 0x7f411c068570 2024-04-20T11:15:38.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.731+0000 7f4118c00700 1 -- 172.21.15.117:0/584895035 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f410c05ca30 con 0x7f411c068570 2024-04-20T11:15:38.731 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:15:38.731 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:15:38.731 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:15:38.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:15:38.731 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:15:38.731 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 10, 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:15:38.732 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:15:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.734+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f410006ecc0 msgr2=0x7f4100071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.734+0000 7f4121800700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f410006ecc0 0x7f4100071180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4108005fd0 tx=0x7f410800a560 comp rx=0 tx=0).stop 2024-04-20T11:15:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.734+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 msgr2=0x7f411c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.734+0000 7f4121800700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 0x7f411c19e850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f410c0092e0 tx=0x7f410c00be20 comp rx=0 tx=0).stop 2024-04-20T11:15:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.735+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 shutdown_connections 2024-04-20T11:15:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.735+0000 7f4121800700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f410006ecc0 0x7f4100071180 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.735+0000 7f4121800700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f411c068570 0x7f411c19e850 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.735+0000 7f4121800700 1 --2- 172.21.15.117:0/584895035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f411c068e90 0x7f411c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.735+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 >> 172.21.15.117:0/584895035 conn(0x7f411c075480 msgr2=0x7f411c107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.735+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 shutdown_connections 2024-04-20T11:15:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.735+0000 7f4121800700 1 -- 172.21.15.117:0/584895035 wait complete. 2024-04-20T11:15:38.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.882+0000 7fab90e00700 1 -- 172.21.15.117:0/2214747229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 msgr2=0x7fab8c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:38.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.882+0000 7fab90e00700 1 --2- 172.21.15.117:0/2214747229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 0x7fab8c1034f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009910 tx=0x7fab7c009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:38.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.882+0000 7fab90e00700 1 -- 172.21.15.117:0/2214747229 shutdown_connections 2024-04-20T11:15:38.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.882+0000 7fab90e00700 1 --2- 172.21.15.117:0/2214747229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c104a00 0x7fab8c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.882+0000 7fab90e00700 1 --2- 172.21.15.117:0/2214747229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 0x7fab8c1034f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.882+0000 7fab90e00700 1 -- 172.21.15.117:0/2214747229 >> 172.21.15.117:0/2214747229 conn(0x7fab8c0fe880 msgr2=0x7fab8c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:38.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.882+0000 7fab90e00700 1 -- 172.21.15.117:0/2214747229 shutdown_connections 2024-04-20T11:15:38.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.883+0000 7fab90e00700 1 -- 172.21.15.117:0/2214747229 wait complete. 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.884+0000 7fab90e00700 1 Processor -- start 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.884+0000 7fab90e00700 1 -- start start 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.884+0000 7fab90e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 0x7fab8c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.884+0000 7fab90e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c104a00 0x7fab8c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.884+0000 7fab90e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c1999e0 con 0x7fab8c103110 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.884+0000 7fab90e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c19d660 con 0x7fab8c104a00 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.884+0000 7fab89e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 0x7fab8c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.884+0000 7fab89e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 0x7fab8c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41434/0 (socket says 172.21.15.117:41434) 2024-04-20T11:15:38.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.885+0000 7fab89e00700 1 -- 172.21.15.117:0/2609240327 learned_addr learned my addr 172.21.15.117:0/2609240327 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:38.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.885+0000 7fab89e00700 1 -- 172.21.15.117:0/2609240327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c104a00 msgr2=0x7fab8c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.885+0000 7fab89400700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c104a00 0x7fab8c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.885+0000 7fab89e00700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c104a00 0x7fab8c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.885+0000 7fab89e00700 1 -- 172.21.15.117:0/2609240327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0095f0 con 0x7fab8c103110 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.885+0000 7fab89e00700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 0x7fab8c198dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0092e0 tx=0x7fab7c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.886+0000 7fab82a00700 1 -- 172.21.15.117:0/2609240327 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab7c01d070 con 0x7fab8c103110 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.886+0000 7fab82a00700 1 -- 172.21.15.117:0/2609240327 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab7c00f460 con 0x7fab8c103110 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.886+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8c19d8e0 con 0x7fab8c103110 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.886+0000 7fab82a00700 1 -- 172.21.15.117:0/2609240327 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab7c017630 con 0x7fab8c103110 2024-04-20T11:15:38.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.886+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c19de50 con 0x7fab8c103110 2024-04-20T11:15:38.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.887+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8c10b480 con 0x7fab8c103110 2024-04-20T11:15:38.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.888+0000 7fab82a00700 1 -- 172.21.15.117:0/2609240327 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fab7c00f5d0 con 0x7fab8c103110 2024-04-20T11:15:38.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.889+0000 7fab82a00700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab7006ecd0 0x7fab70071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:38.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.889+0000 7fab82a00700 1 -- 172.21.15.117:0/2609240327 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fab7c090340 con 0x7fab8c103110 2024-04-20T11:15:38.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.889+0000 7fab89400700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab7006ecd0 0x7fab70071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:38.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.890+0000 7fab89400700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab7006ecd0 0x7fab70071190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab8c19a2d0 tx=0x7fab74005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:38.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:38.894+0000 7fab82a00700 1 -- 172.21.15.117:0/2609240327 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fab7c059130 con 0x7fab8c103110 2024-04-20T11:15:39.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.168+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab8c19a080 con 0x7fab8c103110 2024-04-20T11:15:39.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.169+0000 7fab82a00700 1 -- 172.21.15.117:0/2609240327 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7fab7c05c7f0 con 0x7fab8c103110 2024-04-20T11:15:39.169 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:15:39.169 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:15:39.170 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:15:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:15:39.172 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:15:39.173 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:15:39.173 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:15:39.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab7006ecd0 msgr2=0x7fab70071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab7006ecd0 0x7fab70071190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab8c19a2d0 tx=0x7fab74005cf0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 msgr2=0x7fab8c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 0x7fab8c198dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0092e0 tx=0x7fab7c00be20 comp rx=0 tx=0).stop 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 shutdown_connections 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab7006ecd0 0x7fab70071190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c103110 0x7fab8c198dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 --2- 172.21.15.117:0/2609240327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c104a00 0x7fab8c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.172+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 >> 172.21.15.117:0/2609240327 conn(0x7fab8c0fe880 msgr2=0x7fab8c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.173+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 shutdown_connections 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.173+0000 7fab90e00700 1 -- 172.21.15.117:0/2609240327 wait complete. 2024-04-20T11:15:39.174 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:15:39.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:39 smithi117 ceph-mon[99036]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:39.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:39 smithi117 ceph-mon[99036]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:39.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:39 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all crash 2024-04-20T11:15:39.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:39 smithi117 ceph-mon[99036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-20T11:15:39.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:39 smithi117 ceph-mon[99036]: Upgrade: osd.0 is safe to restart 2024-04-20T11:15:39.248 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:39 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/584895035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:39.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.349+0000 7f5c5c200700 1 -- 172.21.15.117:0/3078845518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100e10 msgr2=0x7f5c541011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:39.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.349+0000 7f5c5c200700 1 --2- 172.21.15.117:0/3078845518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100e10 0x7f5c541011f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009910 tx=0x7f5c48009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:39.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.349+0000 7f5c5c200700 1 -- 172.21.15.117:0/3078845518 shutdown_connections 2024-04-20T11:15:39.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.349+0000 7f5c5c200700 1 --2- 172.21.15.117:0/3078845518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c541017c0 0x7f5c54105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.349+0000 7f5c5c200700 1 --2- 172.21.15.117:0/3078845518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c54100e10 0x7f5c541011f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.349+0000 7f5c5c200700 1 -- 172.21.15.117:0/3078845518 >> 172.21.15.117:0/3078845518 conn(0x7f5c540fc6c0 msgr2=0x7f5c540feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:39.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.350+0000 7f5c5c200700 1 -- 172.21.15.117:0/3078845518 shutdown_connections 2024-04-20T11:15:39.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.350+0000 7f5c5c200700 1 -- 172.21.15.117:0/3078845518 wait complete. 2024-04-20T11:15:39.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.351+0000 7f5c5c200700 1 Processor -- start 2024-04-20T11:15:39.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.351+0000 7f5c5c200700 1 -- start start 2024-04-20T11:15:39.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.351+0000 7f5c5c200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c54100e10 0x7f5c5419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:39.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.351+0000 7f5c5c200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c541017c0 0x7f5c5419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:39.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.351+0000 7f5c5c200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419f5b0 con 0x7f5c541017c0 2024-04-20T11:15:39.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.351+0000 7f5c5c200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54198b30 con 0x7f5c54100e10 2024-04-20T11:15:39.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.352+0000 7f5c58e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c541017c0 0x7f5c5419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:39.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.352+0000 7f5c58e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c541017c0 0x7f5c5419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41452/0 (socket says 172.21.15.117:41452) 2024-04-20T11:15:39.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.352+0000 7f5c58e00700 1 -- 172.21.15.117:0/2803465668 learned_addr learned my addr 172.21.15.117:0/2803465668 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.352+0000 7f5c59800700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c54100e10 0x7f5c5419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.352+0000 7f5c58e00700 1 -- 172.21.15.117:0/2803465668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c54100e10 msgr2=0x7f5c5419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.352+0000 7f5c58e00700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c54100e10 0x7f5c5419e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.352+0000 7f5c58e00700 1 -- 172.21.15.117:0/2803465668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c480095f0 con 0x7f5c541017c0 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.352+0000 7f5c58e00700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c541017c0 0x7f5c5419eed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c8f0 tx=0x7f5c4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.353+0000 7f5c52000700 1 -- 172.21.15.117:0/2803465668 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c40007a50 con 0x7f5c541017c0 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.353+0000 7f5c52000700 1 -- 172.21.15.117:0/2803465668 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c40007bb0 con 0x7f5c541017c0 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.353+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c54198e10 con 0x7f5c541017c0 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.353+0000 7f5c52000700 1 -- 172.21.15.117:0/2803465668 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c40018630 con 0x7f5c541017c0 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.353+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c54199360 con 0x7f5c541017c0 2024-04-20T11:15:39.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.354+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c54074920 con 0x7f5c541017c0 2024-04-20T11:15:39.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.355+0000 7f5c52000700 1 -- 172.21.15.117:0/2803465668 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5c40007d20 con 0x7f5c541017c0 2024-04-20T11:15:39.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.356+0000 7f5c52000700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c3c06ecc0 0x7f5c3c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:39.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.356+0000 7f5c52000700 1 -- 172.21.15.117:0/2803465668 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f5c4008ee10 con 0x7f5c541017c0 2024-04-20T11:15:39.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.356+0000 7f5c59800700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c3c06ecc0 0x7f5c3c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:39.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.357+0000 7f5c59800700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c3c06ecc0 0x7f5c3c071180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006010 tx=0x7f5c4800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:39.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.361+0000 7f5c52000700 1 -- 172.21.15.117:0/2803465668 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5c4005b730 con 0x7f5c541017c0 2024-04-20T11:15:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:39 smithi173 ceph-mon[74165]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:39 smithi173 ceph-mon[74165]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:39 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all crash 2024-04-20T11:15:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:39 smithi173 ceph-mon[74165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-20T11:15:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:39 smithi173 ceph-mon[74165]: Upgrade: osd.0 is safe to restart 2024-04-20T11:15:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:39 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/584895035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:15:39.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.612+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c5410dde0 con 0x7f5c3c06ecc0 2024-04-20T11:15:39.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.614+0000 7f5c52000700 1 -- 172.21.15.117:0/2803465668 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5c5410dde0 con 0x7f5c3c06ecc0 2024-04-20T11:15:39.614 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:15:39.614 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:15:39.614 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:15:39.614 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:15:39.614 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:15:39.614 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:15:39.614 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:15:39.615 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:15:39.615 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:15:39.615 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "6/23 daemons upgraded", 2024-04-20T11:15:39.615 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading osd daemons", 2024-04-20T11:15:39.615 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:15:39.615 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:15:39.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.616+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c3c06ecc0 msgr2=0x7f5c3c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c3c06ecc0 0x7f5c3c071180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006010 tx=0x7f5c4800b540 comp rx=0 tx=0).stop 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c541017c0 msgr2=0x7f5c5419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c541017c0 0x7f5c5419eed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c8f0 tx=0x7f5c4000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 shutdown_connections 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c3c06ecc0 0x7f5c3c071180 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c54100e10 0x7f5c5419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 --2- 172.21.15.117:0/2803465668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c541017c0 0x7f5c5419eed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 >> 172.21.15.117:0/2803465668 conn(0x7f5c540fc6c0 msgr2=0x7f5c540fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 shutdown_connections 2024-04-20T11:15:39.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.617+0000 7f5c5c200700 1 -- 172.21.15.117:0/2803465668 wait complete. 2024-04-20T11:15:39.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.778+0000 7f1313c00700 1 -- 172.21.15.117:0/3871140591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 msgr2=0x7f130c111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:39.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.778+0000 7f1313c00700 1 --2- 172.21.15.117:0/3871140591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 0x7f130c111680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009910 tx=0x7f12f8009c20 comp rx=0 tx=0).stop 2024-04-20T11:15:39.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.778+0000 7f1313c00700 1 -- 172.21.15.117:0/3871140591 shutdown_connections 2024-04-20T11:15:39.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.778+0000 7f1313c00700 1 --2- 172.21.15.117:0/3871140591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 0x7f130c111680 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.778+0000 7f1313c00700 1 --2- 172.21.15.117:0/3871140591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c073280 0x7f130c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.778+0000 7f1313c00700 1 -- 172.21.15.117:0/3871140591 >> 172.21.15.117:0/3871140591 conn(0x7f130c0fc6c0 msgr2=0x7f130c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:39.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.778+0000 7f1313c00700 1 -- 172.21.15.117:0/3871140591 shutdown_connections 2024-04-20T11:15:39.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.778+0000 7f1313c00700 1 -- 172.21.15.117:0/3871140591 wait complete. 2024-04-20T11:15:39.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.779+0000 7f1313c00700 1 Processor -- start 2024-04-20T11:15:39.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.779+0000 7f1313c00700 1 -- start start 2024-04-20T11:15:39.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.780+0000 7f1313c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c073280 0x7f130c19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:39.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.780+0000 7f1313c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 0x7f130c19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:39.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.780+0000 7f1313c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c19dcc0 con 0x7f130c073ba0 2024-04-20T11:15:39.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.780+0000 7f1313c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c1a1990 con 0x7f130c073280 2024-04-20T11:15:39.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.780+0000 7f130be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 0x7f130c19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:39.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.780+0000 7f130be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 0x7f130c19d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41470/0 (socket says 172.21.15.117:41470) 2024-04-20T11:15:39.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.780+0000 7f130be00700 1 -- 172.21.15.117:0/3081942241 learned_addr learned my addr 172.21.15.117:0/3081942241 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.780+0000 7f1311200700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c073280 0x7f130c19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f130be00700 1 -- 172.21.15.117:0/3081942241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c073280 msgr2=0x7f130c19d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f130be00700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c073280 0x7f130c19d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f130be00700 1 -- 172.21.15.117:0/3081942241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80095f0 con 0x7f130c073ba0 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f1311200700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c073280 0x7f130c19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f130be00700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 0x7f130c19d5e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12f8000c00 tx=0x7f12f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f1309600700 1 -- 172.21.15.117:0/3081942241 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f801d070 con 0x7f130c073ba0 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f1309600700 1 -- 172.21.15.117:0/3081942241 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12f80056e0 con 0x7f130c073ba0 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f1309600700 1 -- 172.21.15.117:0/3081942241 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f800fe80 con 0x7f130c073ba0 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c1a1c70 con 0x7f130c073ba0 2024-04-20T11:15:39.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.781+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c1a21c0 con 0x7f130c073ba0 2024-04-20T11:15:39.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.782+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c0749c0 con 0x7f130c073ba0 2024-04-20T11:15:39.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.784+0000 7f1309600700 1 -- 172.21.15.117:0/3081942241 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f12f800f6f0 con 0x7f130c073ba0 2024-04-20T11:15:39.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.784+0000 7f1309600700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12f406ec60 0x7f12f4071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:39.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.785+0000 7f1309600700 1 -- 172.21.15.117:0/3081942241 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f12f8090200 con 0x7f130c073ba0 2024-04-20T11:15:39.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.785+0000 7f1311200700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12f406ec60 0x7f12f4071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:39.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.786+0000 7f1311200700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12f406ec60 0x7f12f4071120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1300005fd0 tx=0x7f1300005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:39.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:39.789+0000 7f1309600700 1 -- 172.21.15.117:0/3081942241 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f12f8058ff0 con 0x7f130c073ba0 2024-04-20T11:15:40.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.108+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f130c114370 con 0x7f130c073ba0 2024-04-20T11:15:40.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.109+0000 7f1309600700 1 -- 172.21.15.117:0/3081942241 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f12f805c6b0 con 0x7f130c073ba0 2024-04-20T11:15:40.109 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:15:40.109 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:15:40.109 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12f406ec60 msgr2=0x7f12f4071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12f406ec60 0x7f12f4071120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1300005fd0 tx=0x7f1300005ee0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 msgr2=0x7f130c19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 0x7f130c19d5e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12f8000c00 tx=0x7f12f800be20 comp rx=0 tx=0).stop 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 shutdown_connections 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12f406ec60 0x7f12f4071120 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c073280 0x7f130c19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 --2- 172.21.15.117:0/3081942241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130c073ba0 0x7f130c19d5e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 >> 172.21.15.117:0/3081942241 conn(0x7f130c0fc6c0 msgr2=0x7f130c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:40.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 shutdown_connections 2024-04-20T11:15:40.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:15:40.112+0000 7f1313c00700 1 -- 172.21.15.117:0/3081942241 wait complete. 2024-04-20T11:15:40.162 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:40 smithi117 ceph-mon[99036]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:40.163 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:40 smithi117 ceph-mon[99036]: pgmap v61: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:40.163 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:40 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2609240327' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:15:40.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:40 smithi173 ceph-mon[74165]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:40.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:40 smithi173 ceph-mon[74165]: pgmap v61: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:40.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:40 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2609240327' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:15:41.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:41 smithi117 ceph-mon[99036]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:41.273 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:41 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3081942241' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:15:41.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:41.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T11:15:41.274 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:41 smithi173 ceph-mon[74165]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:41 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3081942241' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:15:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T11:15:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:42.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:42 smithi117 ceph-mon[99036]: Upgrade: Updating osd.0 2024-04-20T11:15:42.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:42 smithi117 ceph-mon[99036]: Deploying daemon osd.0 on smithi117 2024-04-20T11:15:42.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:42 smithi117 ceph-mon[99036]: pgmap v62: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:42.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:42 smithi173 ceph-mon[74165]: Upgrade: Updating osd.0 2024-04-20T11:15:42.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:42 smithi173 ceph-mon[74165]: Deploying daemon osd.0 on smithi117 2024-04-20T11:15:42.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:42 smithi173 ceph-mon[74165]: pgmap v62: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:15:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:15:44.310 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:44 smithi117 ceph-mon[99036]: pgmap v63: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:44 smithi173 ceph-mon[74165]: pgmap v63: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:45.641 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:45 smithi117 systemd[1]: Stopping Ceph osd.0 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:15:46.065 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:45 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[34342]: 2024-04-20T11:15:45.705+0000 7f32b6c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:15:46.065 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:45 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[34342]: 2024-04-20T11:15:45.705+0000 7f32b6c00700 -1 osd.0 50 *** Got signal Terminated *** 2024-04-20T11:15:46.065 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:45 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[34342]: 2024-04-20T11:15:45.705+0000 7f32b6c00700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:15:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:46 smithi117 ceph-mon[99036]: pgmap v64: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:46 smithi117 ceph-mon[99036]: osd.0 marked itself down and dead 2024-04-20T11:15:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:46 smithi173 ceph-mon[74165]: pgmap v64: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:46 smithi173 ceph-mon[74165]: osd.0 marked itself down and dead 2024-04-20T11:15:47.080 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:46 smithi117 podman[109992]: 2024-04-20 11:15:46.715317169 +0000 UTC m=+1.145333795 container stop 8e5aa5a804787c0b86902243bba05c2b041735c24bf73d5408b008ea728d6911 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=reef-646b8c7, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:15:47.080 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:46 smithi117 podman[109992]: 2024-04-20 11:15:46.74164431 +0000 UTC m=+1.171660968 container died 8e5aa5a804787c0b86902243bba05c2b041735c24bf73d5408b008ea728d6911 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-646b8c7, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-20T11:15:47.344 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:47 smithi117 ceph-mon[99036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:15:47.344 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:47 smithi117 ceph-mon[99036]: osdmap e51: 6 total, 5 up, 6 in 2024-04-20T11:15:47.344 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:47 smithi117 ceph-mon[99036]: osdmap e52: 6 total, 5 up, 6 in 2024-04-20T11:15:47.344 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:47 smithi117 podman[109992]: 2024-04-20 11:15:47.314084683 +0000 UTC m=+1.744101349 container cleanup 8e5aa5a804787c0b86902243bba05c2b041735c24bf73d5408b008ea728d6911 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=reef-646b8c7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:15:47.344 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:47 smithi117 bash[109992]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0 2024-04-20T11:15:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:47 smithi173 ceph-mon[74165]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:15:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:47 smithi173 ceph-mon[74165]: osdmap e51: 6 total, 5 up, 6 in 2024-04-20T11:15:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:47 smithi173 ceph-mon[74165]: osdmap e52: 6 total, 5 up, 6 in 2024-04-20T11:15:47.889 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:47 smithi117 podman[110009]: 2024-04-20 11:15:47.591111226 +0000 UTC m=+0.860603316 container remove 8e5aa5a804787c0b86902243bba05c2b041735c24bf73d5408b008ea728d6911 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=reef-646b8c7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-20T11:15:48.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:48 smithi117 ceph-mon[99036]: pgmap v66: 65 pgs: 9 stale+active+clean, 56 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:48 smithi173 ceph-mon[74165]: pgmap v66: 65 pgs: 9 stale+active+clean, 56 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:15:48.655 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:48 smithi117 podman[110101]: 2024-04-20 11:15:48.306584405 +0000 UTC m=+0.035650295 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:15:48.655 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:48 smithi117 podman[110101]: 2024-04-20 11:15:48.458834256 +0000 UTC m=+0.187900183 container create a9111a99c1ff1b0712ab48d7439a9725ec0d80eb86c63e6bf27fd658a1364270 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:15:49.140 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:48 smithi117 podman[110101]: 2024-04-20 11:15:48.804417173 +0000 UTC m=+0.533483107 container init a9111a99c1ff1b0712ab48d7439a9725ec0d80eb86c63e6bf27fd658a1364270 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-20T11:15:49.140 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:48 smithi117 podman[110101]: 2024-04-20 11:15:48.816418513 +0000 UTC m=+0.545484410 container start a9111a99c1ff1b0712ab48d7439a9725ec0d80eb86c63e6bf27fd658a1364270 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-20T11:15:49.140 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:48 smithi117 podman[110101]: 2024-04-20 11:15:48.902957957 +0000 UTC m=+0.632023938 container attach a9111a99c1ff1b0712ab48d7439a9725ec0d80eb86c63e6bf27fd658a1364270 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T11:15:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:49 smithi173 ceph-mon[74165]: pgmap v68: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:49 smithi173 ceph-mon[74165]: Health check failed: Degraded data redundancy: 61/324 objects degraded (18.827%), 20 pgs degraded (PG_DEGRADED) 2024-04-20T11:15:49.448 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:49 smithi117 ceph-mon[99036]: pgmap v68: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:49.448 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:49 smithi117 ceph-mon[99036]: Health check failed: Degraded data redundancy: 61/324 objects degraded (18.827%), 20 pgs degraded (PG_DEGRADED) 2024-04-20T11:15:49.448 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:49 smithi117 podman[110101]: 2024-04-20 11:15:49.28888269 +0000 UTC m=+1.017948624 container died a9111a99c1ff1b0712ab48d7439a9725ec0d80eb86c63e6bf27fd658a1364270 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-20T11:15:50.623 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:50 smithi117 podman[110126]: 2024-04-20 11:15:50.181992412 +0000 UTC m=+0.870619275 container remove a9111a99c1ff1b0712ab48d7439a9725ec0d80eb86c63e6bf27fd658a1364270 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-20T11:15:50.623 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:50 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-20T11:15:50.624 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:50 smithi117 systemd[1]: Stopped Ceph osd.0 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:15:50.624 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:50 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.0.service: Consumed 1min 34.787s CPU time. 2024-04-20T11:15:50.624 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:50 smithi117 systemd[1]: Starting Ceph osd.0 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:15:51.234 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:50 smithi117 podman[110230]: 2024-04-20 11:15:50.875918551 +0000 UTC m=+0.033916401 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:15:51.234 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:51 smithi117 podman[110230]: 2024-04-20 11:15:51.016220079 +0000 UTC m=+0.174217955 container create 49d8105724f2fb5e03a056c4049f038c9226801f6af312a379a27195ed4d86f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T11:15:51.640 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:51 smithi117 podman[110230]: 2024-04-20 11:15:51.354024873 +0000 UTC m=+0.512022778 container init 49d8105724f2fb5e03a056c4049f038c9226801f6af312a379a27195ed4d86f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:15:51.640 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:51 smithi117 podman[110230]: 2024-04-20 11:15:51.365516223 +0000 UTC m=+0.523514065 container start 49d8105724f2fb5e03a056c4049f038c9226801f6af312a379a27195ed4d86f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T11:15:51.640 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:51 smithi117 podman[110230]: 2024-04-20 11:15:51.493897736 +0000 UTC m=+0.651895614 container attach 49d8105724f2fb5e03a056c4049f038c9226801f6af312a379a27195ed4d86f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-20T11:15:52.308 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-mon[99036]: pgmap v69: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:52 smithi173 ceph-mon[74165]: pgmap v69: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:52.642 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:15:52.642 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:15:52.642 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:15:52.642 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:15:52.642 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T11:15:52.642 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 bash[110230]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate[110245]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-20T11:15:52.643 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:52 smithi117 podman[110230]: 2024-04-20 11:15:52.372366094 +0000 UTC m=+1.530363999 container died 49d8105724f2fb5e03a056c4049f038c9226801f6af312a379a27195ed4d86f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T11:15:53.596 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:53 smithi117 podman[110374]: 2024-04-20 11:15:53.239625618 +0000 UTC m=+0.847711482 container remove 49d8105724f2fb5e03a056c4049f038c9226801f6af312a379a27195ed4d86f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-20T11:15:53.890 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:53 smithi117 podman[110436]: 2024-04-20 11:15:53.503021038 +0000 UTC m=+0.035782017 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:15:53.890 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:53 smithi117 podman[110436]: 2024-04-20 11:15:53.62843439 +0000 UTC m=+0.161195393 container create 8eceb629d7d297ad6b5f25559466c6e783dc7babbe296e309a98290518deca11 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-20T11:15:54.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:54 smithi117 ceph-mon[99036]: pgmap v70: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:54.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:53 smithi117 podman[110436]: 2024-04-20 11:15:53.902353015 +0000 UTC m=+0.435114068 container init 8eceb629d7d297ad6b5f25559466c6e783dc7babbe296e309a98290518deca11 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-20T11:15:54.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:53 smithi117 podman[110436]: 2024-04-20 11:15:53.914160087 +0000 UTC m=+0.446921113 container start 8eceb629d7d297ad6b5f25559466c6e783dc7babbe296e309a98290518deca11 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:15:54.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:53 smithi117 ceph-osd[110455]: -- 172.21.15.117:0/3689895265 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e9cabf680 con 0x556e9ca89000 2024-04-20T11:15:54.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:54 smithi117 bash[110436]: 8eceb629d7d297ad6b5f25559466c6e783dc7babbe296e309a98290518deca11 2024-04-20T11:15:54.391 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:54 smithi117 systemd[1]: Started Ceph osd.0 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:15:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:54 smithi173 ceph-mon[74165]: pgmap v70: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:54.889 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:15:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[110451]: 2024-04-20T11:15:54.488+0000 7f5c1d5ca7c0 -1 Falling back to public interface 2024-04-20T11:15:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:55.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:55.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:55.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:56.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:56 smithi173 ceph-mon[74165]: pgmap v71: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:56.456 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:56 smithi117 ceph-mon[99036]: pgmap v71: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:57.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:57 smithi173 ceph-mon[74165]: pgmap v72: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:57.456 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:57 smithi117 ceph-mon[99036]: pgmap v72: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:15:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:15:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:15:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:15:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:15:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:00 smithi117 ceph-mon[99036]: pgmap v73: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:16:00.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:16:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[110451]: 2024-04-20T11:16:00.032+0000 7f5c1d5ca7c0 -1 osd.0 50 log_to_monitors true 2024-04-20T11:16:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:00 smithi173 ceph-mon[74165]: pgmap v73: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:16:01.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:01 smithi117 ceph-mon[99036]: from='osd.0 [v2:172.21.15.117:6802/664509927,v1:172.21.15.117:6803/664509927]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T11:16:01.392 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:16:01 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[110451]: 2024-04-20T11:16:01.069+0000 7f5c0ca00700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:16:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:01 smithi173 ceph-mon[74165]: from='osd.0 [v2:172.21.15.117:6802/664509927,v1:172.21.15.117:6803/664509927]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T11:16:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:02 smithi173 ceph-mon[74165]: pgmap v74: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:16:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:02 smithi173 ceph-mon[74165]: from='osd.0 [v2:172.21.15.117:6802/664509927,v1:172.21.15.117:6803/664509927]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T11:16:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:02 smithi173 ceph-mon[74165]: osdmap e53: 6 total, 5 up, 6 in 2024-04-20T11:16:02.313 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:02 smithi173 ceph-mon[74165]: from='osd.0 [v2:172.21.15.117:6802/664509927,v1:172.21.15.117:6803/664509927]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:16:02.314 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:02 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:02.314 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:02 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:02 smithi117 ceph-mon[99036]: pgmap v74: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:16:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:02 smithi117 ceph-mon[99036]: from='osd.0 [v2:172.21.15.117:6802/664509927,v1:172.21.15.117:6803/664509927]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T11:16:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:02 smithi117 ceph-mon[99036]: osdmap e53: 6 total, 5 up, 6 in 2024-04-20T11:16:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:02 smithi117 ceph-mon[99036]: from='osd.0 [v2:172.21.15.117:6802/664509927,v1:172.21.15.117:6803/664509927]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:16:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:03 smithi117 ceph-mon[99036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:16:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:03 smithi117 ceph-mon[99036]: osd.0 [v2:172.21.15.117:6802/664509927,v1:172.21.15.117:6803/664509927] boot 2024-04-20T11:16:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:03 smithi117 ceph-mon[99036]: osdmap e54: 6 total, 6 up, 6 in 2024-04-20T11:16:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:16:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:03 smithi117 ceph-mon[99036]: osdmap e55: 6 total, 6 up, 6 in 2024-04-20T11:16:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:03 smithi173 ceph-mon[74165]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:16:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:03 smithi173 ceph-mon[74165]: osd.0 [v2:172.21.15.117:6802/664509927,v1:172.21.15.117:6803/664509927] boot 2024-04-20T11:16:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:03 smithi173 ceph-mon[74165]: osdmap e54: 6 total, 6 up, 6 in 2024-04-20T11:16:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:16:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:03 smithi173 ceph-mon[74165]: osdmap e55: 6 total, 6 up, 6 in 2024-04-20T11:16:04.336 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:04 smithi117 ceph-mon[99036]: pgmap v77: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:16:04.337 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:04.337 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:04 smithi173 ceph-mon[74165]: pgmap v77: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/324 objects degraded (18.827%) 2024-04-20T11:16:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:05 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 9/324 objects degraded (2.778%), 3 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:05 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 9/324 objects degraded (2.778%), 3 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:06.178 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:06 smithi117 ceph-mon[99036]: pgmap v79: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9/324 objects degraded (2.778%) 2024-04-20T11:16:06.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:06 smithi173 ceph-mon[74165]: pgmap v79: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9/324 objects degraded (2.778%) 2024-04-20T11:16:07.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:07 smithi117 ceph-mon[99036]: pgmap v80: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:07.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:07 smithi117 ceph-mon[99036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/324 objects degraded (2.778%), 3 pgs degraded) 2024-04-20T11:16:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:07 smithi173 ceph-mon[74165]: pgmap v80: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:07 smithi173 ceph-mon[74165]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/324 objects degraded (2.778%), 3 pgs degraded) 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: pgmap v81: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:10.281 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:10.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.279+0000 7fd290800700 1 -- 172.21.15.117:0/2141549167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 msgr2=0x7fd28810ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.279+0000 7fd290800700 1 --2- 172.21.15.117:0/2141549167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 0x7fd28810ceb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd278009910 tx=0x7fd278009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.280+0000 7fd290800700 1 -- 172.21.15.117:0/2141549167 shutdown_connections 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.280+0000 7fd290800700 1 --2- 172.21.15.117:0/2141549167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 0x7fd28810ceb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.280+0000 7fd290800700 1 --2- 172.21.15.117:0/2141549167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288073310 0x7fd2880736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.280+0000 7fd290800700 1 -- 172.21.15.117:0/2141549167 >> 172.21.15.117:0/2141549167 conn(0x7fd2880fc6c0 msgr2=0x7fd2880feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.280+0000 7fd290800700 1 -- 172.21.15.117:0/2141549167 shutdown_connections 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.280+0000 7fd290800700 1 -- 172.21.15.117:0/2141549167 wait complete. 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.281+0000 7fd290800700 1 Processor -- start 2024-04-20T11:16:10.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.281+0000 7fd290800700 1 -- start start 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.282+0000 7fd290800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288073310 0x7fd288198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.282+0000 7fd290800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 0x7fd2881992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.282+0000 7fd290800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2881999c0 con 0x7fd288073c30 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.282+0000 7fd290800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28819d690 con 0x7fd288073310 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.282+0000 7fd28d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 0x7fd2881992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.282+0000 7fd28d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 0x7fd2881992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57092/0 (socket says 172.21.15.117:57092) 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.282+0000 7fd28d400700 1 -- 172.21.15.117:0/4280923158 learned_addr learned my addr 172.21.15.117:0/4280923158 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.282+0000 7fd28de00700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288073310 0x7fd288198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.283+0000 7fd28d400700 1 -- 172.21.15.117:0/4280923158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288073310 msgr2=0x7fd288198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.283+0000 7fd28d400700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288073310 0x7fd288198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.283+0000 7fd28d400700 1 -- 172.21.15.117:0/4280923158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780095f0 con 0x7fd288073c30 2024-04-20T11:16:10.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.283+0000 7fd28d400700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 0x7fd2881992e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd2780092e0 tx=0x7fd27800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:10.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.283+0000 7fd282a00700 1 -- 172.21.15.117:0/4280923158 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd27801d070 con 0x7fd288073c30 2024-04-20T11:16:10.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.283+0000 7fd282a00700 1 -- 172.21.15.117:0/4280923158 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd27800f460 con 0x7fd288073c30 2024-04-20T11:16:10.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.283+0000 7fd282a00700 1 -- 172.21.15.117:0/4280923158 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd278017630 con 0x7fd288073c30 2024-04-20T11:16:10.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.283+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28819d910 con 0x7fd288073c30 2024-04-20T11:16:10.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.284+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28819de00 con 0x7fd288073c30 2024-04-20T11:16:10.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.285+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2880749c0 con 0x7fd288073c30 2024-04-20T11:16:10.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.286+0000 7fd282a00700 1 -- 172.21.15.117:0/4280923158 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd27800f5d0 con 0x7fd288073c30 2024-04-20T11:16:10.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.287+0000 7fd282a00700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd27006ecc0 0x7fd270071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:10.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.288+0000 7fd282a00700 1 -- 172.21.15.117:0/4280923158 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fd2780902a0 con 0x7fd288073c30 2024-04-20T11:16:10.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.288+0000 7fd28de00700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd27006ecc0 0x7fd270071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:10.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.289+0000 7fd28de00700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd27006ecc0 0x7fd270071180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd27c005fd0 tx=0x7fd27c00a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:10.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.293+0000 7fd282a00700 1 -- 172.21.15.117:0/4280923158 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd278059130 con 0x7fd288073c30 2024-04-20T11:16:10.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: pgmap v81: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:10.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:10.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:10.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:10.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:10.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:10.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:10.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:10.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:10.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:10.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-20T11:16:10.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.547+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd28810ffd0 con 0x7fd27006ecc0 2024-04-20T11:16:10.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.549+0000 7fd282a00700 1 -- 172.21.15.117:0/4280923158 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd28810ffd0 con 0x7fd27006ecc0 2024-04-20T11:16:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.551+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd27006ecc0 msgr2=0x7fd270071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.551+0000 7fd290800700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd27006ecc0 0x7fd270071180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd27c005fd0 tx=0x7fd27c00a560 comp rx=0 tx=0).stop 2024-04-20T11:16:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.551+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 msgr2=0x7fd2881992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.551+0000 7fd290800700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 0x7fd2881992e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd2780092e0 tx=0x7fd27800be20 comp rx=0 tx=0).stop 2024-04-20T11:16:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.551+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 shutdown_connections 2024-04-20T11:16:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.551+0000 7fd290800700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd27006ecc0 0x7fd270071180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.551+0000 7fd290800700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288073310 0x7fd288198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.551+0000 7fd290800700 1 --2- 172.21.15.117:0/4280923158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288073c30 0x7fd2881992e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.552+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 >> 172.21.15.117:0/4280923158 conn(0x7fd2880fc6c0 msgr2=0x7fd288105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:10.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.552+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 shutdown_connections 2024-04-20T11:16:10.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.552+0000 7fd290800700 1 -- 172.21.15.117:0/4280923158 wait complete. 2024-04-20T11:16:10.570 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:16:10.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.716+0000 7fd097a00700 1 -- 172.21.15.117:0/2507683265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 msgr2=0x7fd090101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:10.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.716+0000 7fd097a00700 1 --2- 172.21.15.117:0/2507683265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 0x7fd090101200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd084009910 tx=0x7fd084009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:10.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.716+0000 7fd097a00700 1 -- 172.21.15.117:0/2507683265 shutdown_connections 2024-04-20T11:16:10.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.716+0000 7fd097a00700 1 --2- 172.21.15.117:0/2507683265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0901017d0 0x7fd090105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.716+0000 7fd097a00700 1 --2- 172.21.15.117:0/2507683265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 0x7fd090101200 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.716+0000 7fd097a00700 1 -- 172.21.15.117:0/2507683265 >> 172.21.15.117:0/2507683265 conn(0x7fd0900fc6f0 msgr2=0x7fd0900feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:10.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.716+0000 7fd097a00700 1 -- 172.21.15.117:0/2507683265 shutdown_connections 2024-04-20T11:16:10.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.716+0000 7fd097a00700 1 -- 172.21.15.117:0/2507683265 wait complete. 2024-04-20T11:16:10.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.717+0000 7fd097a00700 1 Processor -- start 2024-04-20T11:16:10.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.718+0000 7fd097a00700 1 -- start start 2024-04-20T11:16:10.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.718+0000 7fd097a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 0x7fd0900731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:10.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.718+0000 7fd097a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0901017d0 0x7fd090073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:10.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.718+0000 7fd097a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090073c60 con 0x7fd090100e20 2024-04-20T11:16:10.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.718+0000 7fd097a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090073da0 con 0x7fd0901017d0 2024-04-20T11:16:10.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.719+0000 7fd095000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 0x7fd0900731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:10.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.719+0000 7fd095000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 0x7fd0900731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57106/0 (socket says 172.21.15.117:57106) 2024-04-20T11:16:10.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.719+0000 7fd095000700 1 -- 172.21.15.117:0/218526533 learned_addr learned my addr 172.21.15.117:0/218526533 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:10.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.719+0000 7fd08fe00700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0901017d0 0x7fd090073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:10.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.719+0000 7fd095000700 1 -- 172.21.15.117:0/218526533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0901017d0 msgr2=0x7fd090073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:10.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.719+0000 7fd095000700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0901017d0 0x7fd090073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:10.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.720+0000 7fd095000700 1 -- 172.21.15.117:0/218526533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840095f0 con 0x7fd090100e20 2024-04-20T11:16:10.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.720+0000 7fd095000700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 0x7fd0900731e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd084006010 tx=0x7fd08400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:10.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.720+0000 7fd08d600700 1 -- 172.21.15.117:0/218526533 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd08401d070 con 0x7fd090100e20 2024-04-20T11:16:10.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.720+0000 7fd08d600700 1 -- 172.21.15.117:0/218526533 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd08400f460 con 0x7fd090100e20 2024-04-20T11:16:10.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.720+0000 7fd08d600700 1 -- 172.21.15.117:0/218526533 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0840175f0 con 0x7fd090100e20 2024-04-20T11:16:10.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.721+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd090074020 con 0x7fd090100e20 2024-04-20T11:16:10.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.721+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0901a30a0 con 0x7fd090100e20 2024-04-20T11:16:10.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.722+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0901091c0 con 0x7fd090100e20 2024-04-20T11:16:10.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.723+0000 7fd08d600700 1 -- 172.21.15.117:0/218526533 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd08400f5d0 con 0x7fd090100e20 2024-04-20T11:16:10.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.724+0000 7fd08d600700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd07806ecc0 0x7fd078071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:10.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.724+0000 7fd08d600700 1 -- 172.21.15.117:0/218526533 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fd084090270 con 0x7fd090100e20 2024-04-20T11:16:10.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.724+0000 7fd08fe00700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd07806ecc0 0x7fd078071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:10.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.725+0000 7fd08fe00700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd07806ecc0 0x7fd078071180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd090074910 tx=0x7fd080009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:10.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.729+0000 7fd08d600700 1 -- 172.21.15.117:0/218526533 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd084059060 con 0x7fd090100e20 2024-04-20T11:16:11.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:10.999+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0900746c0 con 0x7fd07806ecc0 2024-04-20T11:16:11.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.001+0000 7fd08d600700 1 -- 172.21.15.117:0/218526533 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd0900746c0 con 0x7fd07806ecc0 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd07806ecc0 msgr2=0x7fd078071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd07806ecc0 0x7fd078071180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd090074910 tx=0x7fd080009190 comp rx=0 tx=0).stop 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 msgr2=0x7fd0900731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 0x7fd0900731e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd084006010 tx=0x7fd08400bd70 comp rx=0 tx=0).stop 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 shutdown_connections 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd07806ecc0 0x7fd078071180 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd090100e20 0x7fd0900731e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 --2- 172.21.15.117:0/218526533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0901017d0 0x7fd090073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.004+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 >> 172.21.15.117:0/218526533 conn(0x7fd0900fc6f0 msgr2=0x7fd0900fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:11.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.005+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 shutdown_connections 2024-04-20T11:16:11.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.005+0000 7fd097a00700 1 -- 172.21.15.117:0/218526533 wait complete. 2024-04-20T11:16:11.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:11 smithi117 ceph-mon[99036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-20T11:16:11.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:11 smithi117 ceph-mon[99036]: Upgrade: osd.1 is safe to restart 2024-04-20T11:16:11.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.164+0000 7fe38a400700 1 -- 172.21.15.117:0/3628450146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 msgr2=0x7fe3840fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.164+0000 7fe38a400700 1 --2- 172.21.15.117:0/3628450146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 0x7fe3840fff10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe374009910 tx=0x7fe374009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:11.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.164+0000 7fe38a400700 1 -- 172.21.15.117:0/3628450146 shutdown_connections 2024-04-20T11:16:11.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.164+0000 7fe38a400700 1 --2- 172.21.15.117:0/3628450146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe384100450 0x7fe384104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.164+0000 7fe38a400700 1 --2- 172.21.15.117:0/3628450146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 0x7fe3840fff10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.164+0000 7fe38a400700 1 -- 172.21.15.117:0/3628450146 >> 172.21.15.117:0/3628450146 conn(0x7fe384076750 msgr2=0x7fe3840fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:11.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.164+0000 7fe38a400700 1 -- 172.21.15.117:0/3628450146 shutdown_connections 2024-04-20T11:16:11.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.165+0000 7fe38a400700 1 -- 172.21.15.117:0/3628450146 wait complete. 2024-04-20T11:16:11.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.166+0000 7fe38a400700 1 Processor -- start 2024-04-20T11:16:11.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.166+0000 7fe38a400700 1 -- start start 2024-04-20T11:16:11.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.166+0000 7fe38a400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 0x7fe38419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:11.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.166+0000 7fe38a400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe384100450 0x7fe38419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:11.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.166+0000 7fe38a400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38419de10 con 0x7fe3840ffb30 2024-04-20T11:16:11.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.166+0000 7fe38a400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3841a1ae0 con 0x7fe384100450 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.167+0000 7fe383400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 0x7fe38419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.167+0000 7fe383400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 0x7fe38419d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57128/0 (socket says 172.21.15.117:57128) 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.167+0000 7fe383400700 1 -- 172.21.15.117:0/2894181116 learned_addr learned my addr 172.21.15.117:0/2894181116 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.167+0000 7fe37be00700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe384100450 0x7fe38419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.167+0000 7fe383400700 1 -- 172.21.15.117:0/2894181116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe384100450 msgr2=0x7fe38419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.167+0000 7fe383400700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe384100450 0x7fe38419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.167+0000 7fe383400700 1 -- 172.21.15.117:0/2894181116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740095f0 con 0x7fe3840ffb30 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.167+0000 7fe37be00700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe384100450 0x7fe38419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:16:11.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.168+0000 7fe383400700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 0x7fe38419d1f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3740092e0 tx=0x7fe37400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:11.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.168+0000 7fe380c00700 1 -- 172.21.15.117:0/2894181116 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe37401d070 con 0x7fe3840ffb30 2024-04-20T11:16:11.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.168+0000 7fe380c00700 1 -- 172.21.15.117:0/2894181116 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe37400f460 con 0x7fe3840ffb30 2024-04-20T11:16:11.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.168+0000 7fe380c00700 1 -- 172.21.15.117:0/2894181116 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe374017630 con 0x7fe3840ffb30 2024-04-20T11:16:11.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.168+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3841a1d60 con 0x7fe3840ffb30 2024-04-20T11:16:11.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.168+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3841a2250 con 0x7fe3840ffb30 2024-04-20T11:16:11.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.170+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe384068e50 con 0x7fe3840ffb30 2024-04-20T11:16:11.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.171+0000 7fe380c00700 1 -- 172.21.15.117:0/2894181116 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe37400f5d0 con 0x7fe3840ffb30 2024-04-20T11:16:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.172+0000 7fe380c00700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe36806ecc0 0x7fe368071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:11.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.172+0000 7fe380c00700 1 -- 172.21.15.117:0/2894181116 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fe374090450 con 0x7fe3840ffb30 2024-04-20T11:16:11.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.172+0000 7fe37be00700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe36806ecc0 0x7fe368071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:11.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.173+0000 7fe37be00700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe36806ecc0 0x7fe368071180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe38419e750 tx=0x7fe36c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:11.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.180+0000 7fe380c00700 1 -- 172.21.15.117:0/2894181116 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe3740592c0 con 0x7fe3840ffb30 2024-04-20T11:16:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:11 smithi173 ceph-mon[74165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-20T11:16:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:11 smithi173 ceph-mon[74165]: Upgrade: osd.1 is safe to restart 2024-04-20T11:16:11.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.420+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3841144b0 con 0x7fe36806ecc0 2024-04-20T11:16:11.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.433+0000 7fe380c00700 1 -- 172.21.15.117:0/2894181116 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe3841144b0 con 0x7fe36806ecc0 2024-04-20T11:16:11.434 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:16:11.434 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (4m) 10s ago 18m 13.6M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:16:11.434 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (19m) 10s ago 19m 8560k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:16:11.434 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (17m) 44s ago 17m 8270k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:16:11.434 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (57s) 10s ago 18m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:16:11.434 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (49s) 44s ago 17m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:16:11.434 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (3m) 10s ago 18m 75.0M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (11m) 10s ago 11m 17.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (11m) 10s ago 11m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (11m) 44s ago 11m 17.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (11m) 44s ago 11m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (7m) 10s ago 20m 500M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (6m) 44s ago 17m 450M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (2m) 10s ago 20m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (2m) 44s ago 17m 33.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (5m) 10s ago 18m 8212k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (5m) 44s ago 17m 7377k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (17s) 10s ago 16m 12.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (15m) 10s ago 15m 217M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e2ef55dc4eee 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (15m) 10s ago 15m 183M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:16:11.435 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (14m) 44s ago 14m 244M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:16:11.436 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (13m) 44s ago 13m 215M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:16:11.436 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (13m) 44s ago 13m 202M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:16:11.436 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (5m) 10s ago 18m 68.2M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:16:11.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe36806ecc0 msgr2=0x7fe368071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe36806ecc0 0x7fe368071180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe38419e750 tx=0x7fe36c008040 comp rx=0 tx=0).stop 2024-04-20T11:16:11.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 msgr2=0x7fe38419d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 0x7fe38419d1f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3740092e0 tx=0x7fe37400be20 comp rx=0 tx=0).stop 2024-04-20T11:16:11.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 shutdown_connections 2024-04-20T11:16:11.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe36806ecc0 0x7fe368071180 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3840ffb30 0x7fe38419d1f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 --2- 172.21.15.117:0/2894181116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe384100450 0x7fe38419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 >> 172.21.15.117:0/2894181116 conn(0x7fe384076750 msgr2=0x7fe38410e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:11.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 shutdown_connections 2024-04-20T11:16:11.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.436+0000 7fe38a400700 1 -- 172.21.15.117:0/2894181116 wait complete. 2024-04-20T11:16:11.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.591+0000 7f6156400700 1 -- 172.21.15.117:0/3359077808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 msgr2=0x7f6150101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.591+0000 7f6156400700 1 --2- 172.21.15.117:0/3359077808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 0x7f6150101200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6140009910 tx=0x7f6140009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:11.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.591+0000 7f6156400700 1 -- 172.21.15.117:0/3359077808 shutdown_connections 2024-04-20T11:16:11.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.591+0000 7f6156400700 1 --2- 172.21.15.117:0/3359077808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61501017d0 0x7f6150105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.591+0000 7f6156400700 1 --2- 172.21.15.117:0/3359077808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 0x7f6150101200 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.591+0000 7f6156400700 1 -- 172.21.15.117:0/3359077808 >> 172.21.15.117:0/3359077808 conn(0x7f61500fc6f0 msgr2=0x7f61500feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:11.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.591+0000 7f6156400700 1 -- 172.21.15.117:0/3359077808 shutdown_connections 2024-04-20T11:16:11.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.591+0000 7f6156400700 1 -- 172.21.15.117:0/3359077808 wait complete. 2024-04-20T11:16:11.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.593+0000 7f6156400700 1 Processor -- start 2024-04-20T11:16:11.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.593+0000 7f6156400700 1 -- start start 2024-04-20T11:16:11.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.593+0000 7f6156400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6150100e20 0x7f615019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:11.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.593+0000 7f6156400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61501017d0 0x7f615019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:11.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.593+0000 7f6156400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615019d380 con 0x7f61501017d0 2024-04-20T11:16:11.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.594+0000 7f6156400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61501969b0 con 0x7f6150100e20 2024-04-20T11:16:11.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.594+0000 7f614ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61501017d0 0x7f615019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:11.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.594+0000 7f614ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61501017d0 0x7f615019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57152/0 (socket says 172.21.15.117:57152) 2024-04-20T11:16:11.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.594+0000 7f614ea00700 1 -- 172.21.15.117:0/220737079 learned_addr learned my addr 172.21.15.117:0/220737079 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:11.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.594+0000 7f614f400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6150100e20 0x7f615019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:11.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f614ea00700 1 -- 172.21.15.117:0/220737079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6150100e20 msgr2=0x7f615019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f614ea00700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6150100e20 0x7f615019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f614ea00700 1 -- 172.21.15.117:0/220737079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400095f0 con 0x7f61501017d0 2024-04-20T11:16:11.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f614f400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6150100e20 0x7f615019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:16:11.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f614ea00700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61501017d0 0x7f615019cd60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f613c00d900 tx=0x7f613c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:11.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f6147e00700 1 -- 172.21.15.117:0/220737079 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f613c009900 con 0x7f61501017d0 2024-04-20T11:16:11.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f6147e00700 1 -- 172.21.15.117:0/220737079 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f613c010460 con 0x7f61501017d0 2024-04-20T11:16:11.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f6147e00700 1 -- 172.21.15.117:0/220737079 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f613c00f5f0 con 0x7f61501017d0 2024-04-20T11:16:11.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.595+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6150196c90 con 0x7f61501017d0 2024-04-20T11:16:11.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.596+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61501971e0 con 0x7f61501017d0 2024-04-20T11:16:11.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.597+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61501091c0 con 0x7f61501017d0 2024-04-20T11:16:11.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.598+0000 7f6147e00700 1 -- 172.21.15.117:0/220737079 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f613c0105d0 con 0x7f61501017d0 2024-04-20T11:16:11.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.599+0000 7f6147e00700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f613406ecc0 0x7f6134071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:11.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.599+0000 7f6147e00700 1 -- 172.21.15.117:0/220737079 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f613c08eb20 con 0x7f61501017d0 2024-04-20T11:16:11.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.600+0000 7f614f400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f613406ecc0 0x7f6134071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:11.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.601+0000 7f614f400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f613406ecc0 0x7f6134071180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6140005b40 tx=0x7f614001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:11.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.604+0000 7f6147e00700 1 -- 172.21.15.117:0/220737079 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f613c05b560 con 0x7f61501017d0 2024-04-20T11:16:11.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.928+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6150197e80 con 0x7f61501017d0 2024-04-20T11:16:11.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.929+0000 7f6147e00700 1 -- 172.21.15.117:0/220737079 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f613c05b0f0 con 0x7f61501017d0 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 5, 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:16:11.930 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:16:11.931 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:16:11.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:16:11.931 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:16:11.931 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 9, 2024-04-20T11:16:11.931 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 5 2024-04-20T11:16:11.931 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:16:11.931 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:16:11.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.931+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f613406ecc0 msgr2=0x7f6134071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.931+0000 7f6156400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f613406ecc0 0x7f6134071180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6140005b40 tx=0x7f614001a040 comp rx=0 tx=0).stop 2024-04-20T11:16:11.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.931+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61501017d0 msgr2=0x7f615019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:11.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.931+0000 7f6156400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61501017d0 0x7f615019cd60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f613c00d900 tx=0x7f613c00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.932+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 shutdown_connections 2024-04-20T11:16:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.932+0000 7f6156400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f613406ecc0 0x7f6134071180 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.932+0000 7f6156400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6150100e20 0x7f615019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.932+0000 7f6156400700 1 --2- 172.21.15.117:0/220737079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61501017d0 0x7f615019cd60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.932+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 >> 172.21.15.117:0/220737079 conn(0x7f61500fc6f0 msgr2=0x7f61500fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.932+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 shutdown_connections 2024-04-20T11:16:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:11.932+0000 7f6156400700 1 -- 172.21.15.117:0/220737079 wait complete. 2024-04-20T11:16:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.085+0000 7f5be3c00700 1 -- 172.21.15.117:0/1274753174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073310 msgr2=0x7f5bdc0736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.085+0000 7f5be3c00700 1 --2- 172.21.15.117:0/1274753174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073310 0x7f5bdc0736f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009910 tx=0x7f5bd0009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.086+0000 7f5be3c00700 1 -- 172.21.15.117:0/1274753174 shutdown_connections 2024-04-20T11:16:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.086+0000 7f5be3c00700 1 --2- 172.21.15.117:0/1274753174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bdc073c30 0x7f5bdc10cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.086+0000 7f5be3c00700 1 --2- 172.21.15.117:0/1274753174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073310 0x7f5bdc0736f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.086+0000 7f5be3c00700 1 -- 172.21.15.117:0/1274753174 >> 172.21.15.117:0/1274753174 conn(0x7f5bdc0fc6f0 msgr2=0x7f5bdc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.086+0000 7f5be3c00700 1 -- 172.21.15.117:0/1274753174 shutdown_connections 2024-04-20T11:16:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.086+0000 7f5be3c00700 1 -- 172.21.15.117:0/1274753174 wait complete. 2024-04-20T11:16:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.087+0000 7f5be3c00700 1 Processor -- start 2024-04-20T11:16:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.087+0000 7f5be3c00700 1 -- start start 2024-04-20T11:16:12.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.088+0000 7f5be3c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bdc073310 0x7f5bdc198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.088+0000 7f5be3c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073c30 0x7f5bdc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.088+0000 7f5be3c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc199a10 con 0x7f5bdc073c30 2024-04-20T11:16:12.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.088+0000 7f5be3c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc19d6e0 con 0x7f5bdc073310 2024-04-20T11:16:12.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.088+0000 7f5bdbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073c30 0x7f5bdc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.088+0000 7f5bdbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073c30 0x7f5bdc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57170/0 (socket says 172.21.15.117:57170) 2024-04-20T11:16:12.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.088+0000 7f5bdbe00700 1 -- 172.21.15.117:0/2584191700 learned_addr learned my addr 172.21.15.117:0/2584191700 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:12.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.088+0000 7f5be1200700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bdc073310 0x7f5bdc198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5bdbe00700 1 -- 172.21.15.117:0/2584191700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bdc073310 msgr2=0x7f5bdc198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5bdbe00700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bdc073310 0x7f5bdc198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5bdbe00700 1 -- 172.21.15.117:0/2584191700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00095f0 con 0x7f5bdc073c30 2024-04-20T11:16:12.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5bdbe00700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073c30 0x7f5bdc199330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800b4b0 tx=0x7f5bc800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:12.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5bd9600700 1 -- 172.21.15.117:0/2584191700 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc8011800 con 0x7f5bdc073c30 2024-04-20T11:16:12.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5bd9600700 1 -- 172.21.15.117:0/2584191700 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bc8011e20 con 0x7f5bdc073c30 2024-04-20T11:16:12.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc19d9c0 con 0x7f5bdc073c30 2024-04-20T11:16:12.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc19df10 con 0x7f5bdc073c30 2024-04-20T11:16:12.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.089+0000 7f5bd9600700 1 -- 172.21.15.117:0/2584191700 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc800f2e0 con 0x7f5bdc073c30 2024-04-20T11:16:12.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.091+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc0749c0 con 0x7f5bdc073c30 2024-04-20T11:16:12.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.091+0000 7f5bd9600700 1 -- 172.21.15.117:0/2584191700 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5bc8010380 con 0x7f5bdc073c30 2024-04-20T11:16:12.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.093+0000 7f5bd9600700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5bc406ecc0 0x7f5bc4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.093+0000 7f5be1200700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5bc406ecc0 0x7f5bc4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.093+0000 7f5bd9600700 1 -- 172.21.15.117:0/2584191700 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f5bc808e8e0 con 0x7f5bdc073c30 2024-04-20T11:16:12.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.094+0000 7f5be1200700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5bc406ecc0 0x7f5bc4071180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000b5c0 tx=0x7f5bd001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:12.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.097+0000 7f5bd9600700 1 -- 172.21.15.117:0/2584191700 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5bc8057750 con 0x7f5bdc073c30 2024-04-20T11:16:12.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.372+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5bdc062510 con 0x7f5bdc073c30 2024-04-20T11:16:12.373 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:12 smithi117 ceph-mon[99036]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:12.373 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:12 smithi117 ceph-mon[99036]: pgmap v82: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:12.373 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:12 smithi117 ceph-mon[99036]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:12.373 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:12 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/220737079' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.373+0000 7f5bd9600700 1 -- 172.21.15.117:0/2584191700 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7f5bc805ae10 con 0x7f5bdc073c30 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:16:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:16:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:16:12.376 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:16:12.377 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:16:12.377 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:16:12.377 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:16:12.377 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:16:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.377+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5bc406ecc0 msgr2=0x7f5bc4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.377+0000 7f5be3c00700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5bc406ecc0 0x7f5bc4071180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000b5c0 tx=0x7f5bd001a040 comp rx=0 tx=0).stop 2024-04-20T11:16:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.377+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073c30 msgr2=0x7f5bdc199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.377+0000 7f5be3c00700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073c30 0x7f5bdc199330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800b4b0 tx=0x7f5bc800b870 comp rx=0 tx=0).stop 2024-04-20T11:16:12.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.378+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 shutdown_connections 2024-04-20T11:16:12.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.378+0000 7f5be3c00700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5bc406ecc0 0x7f5bc4071180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.378+0000 7f5be3c00700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bdc073310 0x7f5bdc198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.378+0000 7f5be3c00700 1 --2- 172.21.15.117:0/2584191700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bdc073c30 0x7f5bdc199330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.378+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 >> 172.21.15.117:0/2584191700 conn(0x7f5bdc0fc6f0 msgr2=0x7f5bdc1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:12.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.378+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 shutdown_connections 2024-04-20T11:16:12.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.378+0000 7f5be3c00700 1 -- 172.21.15.117:0/2584191700 wait complete. 2024-04-20T11:16:12.380 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:16:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:12 smithi173 ceph-mon[74165]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:12 smithi173 ceph-mon[74165]: pgmap v82: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:12 smithi173 ceph-mon[74165]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:12 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/220737079' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.538+0000 7f031fc00700 1 -- 172.21.15.117:0/646042453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 msgr2=0x7f03181011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.538+0000 7f031fc00700 1 --2- 172.21.15.117:0/646042453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 0x7f03181011f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f030c00b0a0 tx=0x7f030c00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.538+0000 7f031fc00700 1 -- 172.21.15.117:0/646042453 shutdown_connections 2024-04-20T11:16:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.538+0000 7f031fc00700 1 --2- 172.21.15.117:0/646042453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03181017c0 0x7f0318105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.538+0000 7f031fc00700 1 --2- 172.21.15.117:0/646042453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 0x7f03181011f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.538+0000 7f031fc00700 1 -- 172.21.15.117:0/646042453 >> 172.21.15.117:0/646042453 conn(0x7f03180fc6c0 msgr2=0x7f03180feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.538+0000 7f031fc00700 1 -- 172.21.15.117:0/646042453 shutdown_connections 2024-04-20T11:16:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.538+0000 7f031fc00700 1 -- 172.21.15.117:0/646042453 wait complete. 2024-04-20T11:16:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.539+0000 7f031fc00700 1 Processor -- start 2024-04-20T11:16:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.540+0000 7f031fc00700 1 -- start start 2024-04-20T11:16:12.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.540+0000 7f031fc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 0x7f031819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.540+0000 7f031fc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03181017c0 0x7f031819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.540+0000 7f031fc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031819d3b0 con 0x7f0318100e10 2024-04-20T11:16:12.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.540+0000 7f031fc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318196a20 con 0x7f03181017c0 2024-04-20T11:16:12.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.540+0000 7f031d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 0x7f031819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.540+0000 7f031d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 0x7f031819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57194/0 (socket says 172.21.15.117:57194) 2024-04-20T11:16:12.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.540+0000 7f031d200700 1 -- 172.21.15.117:0/1205806397 learned_addr learned my addr 172.21.15.117:0/1205806397 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:12.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.541+0000 7f0317e00700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03181017c0 0x7f031819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.541+0000 7f031d200700 1 -- 172.21.15.117:0/1205806397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03181017c0 msgr2=0x7f031819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.541+0000 7f031d200700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03181017c0 0x7f031819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.541+0000 7f031d200700 1 -- 172.21.15.117:0/1205806397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c009d00 con 0x7f0318100e10 2024-04-20T11:16:12.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.541+0000 7f031d200700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 0x7f031819c8e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f030c00b070 tx=0x7f030c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:12.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.541+0000 7f0315600700 1 -- 172.21.15.117:0/1205806397 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f030c00e040 con 0x7f0318100e10 2024-04-20T11:16:12.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.542+0000 7f0315600700 1 -- 172.21.15.117:0/1205806397 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f030c003de0 con 0x7f0318100e10 2024-04-20T11:16:12.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.542+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0318196ca0 con 0x7f0318100e10 2024-04-20T11:16:12.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.542+0000 7f0315600700 1 -- 172.21.15.117:0/1205806397 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f030c01d6b0 con 0x7f0318100e10 2024-04-20T11:16:12.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.542+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0318197190 con 0x7f0318100e10 2024-04-20T11:16:12.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.543+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03181091b0 con 0x7f0318100e10 2024-04-20T11:16:12.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.544+0000 7f0315600700 1 -- 172.21.15.117:0/1205806397 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f030c004040 con 0x7f0318100e10 2024-04-20T11:16:12.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.545+0000 7f0315600700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f030006ecc0 0x7f0300071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.545+0000 7f0315600700 1 -- 172.21.15.117:0/1205806397 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f030c090520 con 0x7f0318100e10 2024-04-20T11:16:12.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.545+0000 7f0317e00700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f030006ecc0 0x7f0300071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.546+0000 7f0317e00700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f030006ecc0 0x7f0300071180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0318197be0 tx=0x7f0304005d50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:12.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.554+0000 7f0315600700 1 -- 172.21.15.117:0/1205806397 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f030c05cf60 con 0x7f0318100e10 2024-04-20T11:16:12.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.804+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0318197ee0 con 0x7f030006ecc0 2024-04-20T11:16:12.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.805+0000 7f0315600700 1 -- 172.21.15.117:0/1205806397 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0318197ee0 con 0x7f030006ecc0 2024-04-20T11:16:12.806 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:16:12.806 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:16:12.806 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:16:12.806 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:16:12.806 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:16:12.806 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:16:12.807 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:16:12.807 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:16:12.807 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:16:12.807 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "7/23 daemons upgraded", 2024-04-20T11:16:12.807 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading osd daemons", 2024-04-20T11:16:12.807 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:16:12.807 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f030006ecc0 msgr2=0x7f0300071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f030006ecc0 0x7f0300071180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0318197be0 tx=0x7f0304005d50 comp rx=0 tx=0).stop 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 msgr2=0x7f031819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 0x7f031819c8e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f030c00b070 tx=0x7f030c007bd0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 shutdown_connections 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f030006ecc0 0x7f0300071180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0318100e10 0x7f031819c8e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 --2- 172.21.15.117:0/1205806397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03181017c0 0x7f031819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 >> 172.21.15.117:0/1205806397 conn(0x7f03180fc6c0 msgr2=0x7f03180fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 shutdown_connections 2024-04-20T11:16:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.808+0000 7f031fc00700 1 -- 172.21.15.117:0/1205806397 wait complete. 2024-04-20T11:16:12.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.967+0000 7fee2ca00700 1 -- 172.21.15.117:0/1990628750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28107520 msgr2=0x7fee28107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.967+0000 7fee2ca00700 1 --2- 172.21.15.117:0/1990628750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28107520 0x7fee28107900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee18009910 tx=0x7fee18009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:12.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.967+0000 7fee2ca00700 1 -- 172.21.15.117:0/1990628750 shutdown_connections 2024-04-20T11:16:12.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.967+0000 7fee2ca00700 1 --2- 172.21.15.117:0/1990628750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee28108e10 0x7fee28109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.967+0000 7fee2ca00700 1 --2- 172.21.15.117:0/1990628750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28107520 0x7fee28107900 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.967+0000 7fee2ca00700 1 -- 172.21.15.117:0/1990628750 >> 172.21.15.117:0/1990628750 conn(0x7fee280768f0 msgr2=0x7fee28078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:12.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.967+0000 7fee2ca00700 1 -- 172.21.15.117:0/1990628750 shutdown_connections 2024-04-20T11:16:12.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.967+0000 7fee2ca00700 1 -- 172.21.15.117:0/1990628750 wait complete. 2024-04-20T11:16:12.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.969+0000 7fee2ca00700 1 Processor -- start 2024-04-20T11:16:12.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.969+0000 7fee2ca00700 1 -- start start 2024-04-20T11:16:12.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.969+0000 7fee2ca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee28107520 0x7fee2819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.969+0000 7fee2ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28108e10 0x7fee2819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.969+0000 7fee2ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2819ddd0 con 0x7fee28108e10 2024-04-20T11:16:12.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.969+0000 7fee2ca00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281a1aa0 con 0x7fee28107520 2024-04-20T11:16:12.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.969+0000 7fee25400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28108e10 0x7fee2819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.970+0000 7fee25400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28108e10 0x7fee2819d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57214/0 (socket says 172.21.15.117:57214) 2024-04-20T11:16:12.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.970+0000 7fee25400700 1 -- 172.21.15.117:0/984391543 learned_addr learned my addr 172.21.15.117:0/984391543 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:12.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.970+0000 7fee25e00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee28107520 0x7fee2819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.970+0000 7fee25400700 1 -- 172.21.15.117:0/984391543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee28107520 msgr2=0x7fee2819d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.970+0000 7fee25400700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee28107520 0x7fee2819d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.970+0000 7fee25400700 1 -- 172.21.15.117:0/984391543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee180095f0 con 0x7fee28108e10 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.970+0000 7fee25e00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee28107520 0x7fee2819d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.970+0000 7fee25400700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28108e10 0x7fee2819d6f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c8a0 tx=0x7fee1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.971+0000 7fee1ea00700 1 -- 172.21.15.117:0/984391543 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1400cea0 con 0x7fee28108e10 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.971+0000 7fee1ea00700 1 -- 172.21.15.117:0/984391543 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee14004830 con 0x7fee28108e10 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.971+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee281a1d80 con 0x7fee28108e10 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.971+0000 7fee1ea00700 1 -- 172.21.15.117:0/984391543 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee140055b0 con 0x7fee28108e10 2024-04-20T11:16:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.971+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee281a2350 con 0x7fee28108e10 2024-04-20T11:16:12.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.972+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2810f870 con 0x7fee28108e10 2024-04-20T11:16:12.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.974+0000 7fee1ea00700 1 -- 172.21.15.117:0/984391543 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fee140049a0 con 0x7fee28108e10 2024-04-20T11:16:12.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.974+0000 7fee1ea00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fee0c06ecd0 0x7fee0c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:12.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.975+0000 7fee1ea00700 1 -- 172.21.15.117:0/984391543 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fee1408e970 con 0x7fee28108e10 2024-04-20T11:16:12.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.975+0000 7fee25e00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fee0c06ecd0 0x7fee0c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:12.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.975+0000 7fee25e00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fee0c06ecd0 0x7fee0c071190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee18005b40 tx=0x7fee1801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:12.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:12.979+0000 7fee1ea00700 1 -- 172.21.15.117:0/984391543 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fee14057880 con 0x7fee28108e10 2024-04-20T11:16:13.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.315+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fee28108920 con 0x7fee28108e10 2024-04-20T11:16:13.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.315+0000 7fee1ea00700 1 -- 172.21.15.117:0/984391543 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fee1401d090 con 0x7fee28108e10 2024-04-20T11:16:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:16:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:16:13.316 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:16:13.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fee0c06ecd0 msgr2=0x7fee0c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fee0c06ecd0 0x7fee0c071190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee18005b40 tx=0x7fee1801a040 comp rx=0 tx=0).stop 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28108e10 msgr2=0x7fee2819d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28108e10 0x7fee2819d6f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c8a0 tx=0x7fee1400cc60 comp rx=0 tx=0).stop 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 shutdown_connections 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fee0c06ecd0 0x7fee0c071190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee28107520 0x7fee2819d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 --2- 172.21.15.117:0/984391543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee28108e10 0x7fee2819d6f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 >> 172.21.15.117:0/984391543 conn(0x7fee280768f0 msgr2=0x7fee28077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 shutdown_connections 2024-04-20T11:16:13.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:13.318+0000 7fee2ca00700 1 -- 172.21.15.117:0/984391543 wait complete. 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: Upgrade: Updating osd.1 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2584191700' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: Upgrade: Updating osd.1 2024-04-20T11:16:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:16:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2584191700' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:16:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:13.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:13.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:14 smithi173 ceph-mon[74165]: Deploying daemon osd.1 on smithi117 2024-04-20T11:16:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:14 smithi173 ceph-mon[74165]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:14 smithi173 ceph-mon[74165]: pgmap v83: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:14 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/984391543' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:16:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:14 smithi117 ceph-mon[99036]: Deploying daemon osd.1 on smithi117 2024-04-20T11:16:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:14 smithi117 ceph-mon[99036]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:14 smithi117 ceph-mon[99036]: pgmap v83: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:14 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/984391543' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:16:15.318 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:15 smithi117 ceph-mon[99036]: pgmap v84: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:15 smithi173 ceph-mon[74165]: pgmap v84: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:17.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:17 smithi117 systemd[1]: Stopping Ceph osd.1 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:16:17.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:17 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[42123]: 2024-04-20T11:16:17.612+0000 7fab50a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:16:17.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:17 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[42123]: 2024-04-20T11:16:17.612+0000 7fab50a00700 -1 osd.1 55 *** Got signal Terminated *** 2024-04-20T11:16:17.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:17 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[42123]: 2024-04-20T11:16:17.612+0000 7fab50a00700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:16:18.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:18 smithi117 ceph-mon[99036]: pgmap v85: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:18.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:18 smithi117 ceph-mon[99036]: osd.1 marked itself down and dead 2024-04-20T11:16:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:18 smithi173 ceph-mon[74165]: pgmap v85: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:18 smithi173 ceph-mon[74165]: osd.1 marked itself down and dead 2024-04-20T11:16:18.659 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:18 smithi117 podman[115789]: 2024-04-20 11:16:18.557548467 +0000 UTC m=+1.136825158 container died e2ef55dc4eeee3b9e37378bee3e7600a809e2b80ff1e8224d67149e1dae57ad9 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, org.label-schema.vendor=CentOS, RELEASE=reef-646b8c7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:16:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:19 smithi117 ceph-mon[99036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:16:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:19 smithi117 ceph-mon[99036]: osdmap e56: 6 total, 5 up, 6 in 2024-04-20T11:16:19.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:19 smithi117 podman[115789]: 2024-04-20 11:16:19.080023824 +0000 UTC m=+1.659300524 container cleanup e2ef55dc4eeee3b9e37378bee3e7600a809e2b80ff1e8224d67149e1dae57ad9 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=reef-646b8c7, org.label-schema.vendor=CentOS) 2024-04-20T11:16:19.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:19 smithi117 bash[115789]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1 2024-04-20T11:16:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:19 smithi173 ceph-mon[74165]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:16:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:19 smithi173 ceph-mon[74165]: osdmap e56: 6 total, 5 up, 6 in 2024-04-20T11:16:19.786 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:19 smithi117 podman[115809]: 2024-04-20 11:16:19.435879542 +0000 UTC m=+0.862191147 container remove e2ef55dc4eeee3b9e37378bee3e7600a809e2b80ff1e8224d67149e1dae57ad9 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-646b8c7, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:16:20.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:20 smithi117 ceph-mon[99036]: pgmap v87: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:20.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:20 smithi117 ceph-mon[99036]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-20T11:16:20.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:20 smithi117 ceph-mon[99036]: osdmap e57: 6 total, 5 up, 6 in 2024-04-20T11:16:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:20 smithi173 ceph-mon[74165]: pgmap v87: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:20 smithi173 ceph-mon[74165]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-20T11:16:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:20 smithi173 ceph-mon[74165]: osdmap e57: 6 total, 5 up, 6 in 2024-04-20T11:16:20.619 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:20 smithi117 podman[115898]: 2024-04-20 11:16:20.199336853 +0000 UTC m=+0.034932736 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:16:20.619 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:20 smithi117 podman[115898]: 2024-04-20 11:16:20.348787768 +0000 UTC m=+0.184383688 container create 662f8a2015545b99ad615aa12f7d54ce3b990bf9d8b54c7f7a96edf30b6dff0c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:16:20.619 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:20 smithi117 podman[115898]: 2024-04-20 11:16:20.619180641 +0000 UTC m=+0.454776571 container init 662f8a2015545b99ad615aa12f7d54ce3b990bf9d8b54c7f7a96edf30b6dff0c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:16:20.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:20 smithi117 podman[115898]: 2024-04-20 11:16:20.631414047 +0000 UTC m=+0.467009947 container start 662f8a2015545b99ad615aa12f7d54ce3b990bf9d8b54c7f7a96edf30b6dff0c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-20T11:16:20.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:20 smithi117 podman[115898]: 2024-04-20 11:16:20.717891782 +0000 UTC m=+0.553487745 container attach 662f8a2015545b99ad615aa12f7d54ce3b990bf9d8b54c7f7a96edf30b6dff0c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-20T11:16:21.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:21 smithi117 podman[115898]: 2024-04-20 11:16:21.082591878 +0000 UTC m=+0.918187801 container died 662f8a2015545b99ad615aa12f7d54ce3b990bf9d8b54c7f7a96edf30b6dff0c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-20T11:16:22.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:22 smithi173 ceph-mon[74165]: pgmap v89: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:22.349 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:22 smithi117 ceph-mon[99036]: pgmap v89: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:22.349 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:21 smithi117 podman[115924]: 2024-04-20 11:16:21.929826823 +0000 UTC m=+0.828648658 container remove 662f8a2015545b99ad615aa12f7d54ce3b990bf9d8b54c7f7a96edf30b6dff0c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T11:16:22.350 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:21 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-20T11:16:22.350 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:21 smithi117 systemd[1]: Stopped Ceph osd.1 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:16:22.350 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:21 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.1.service: Consumed 1min 40.664s CPU time. 2024-04-20T11:16:22.640 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:22 smithi117 systemd[1]: Starting Ceph osd.1 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:16:23.018 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:22 smithi117 podman[116026]: 2024-04-20 11:16:22.615115955 +0000 UTC m=+0.034565963 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:16:23.018 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:22 smithi117 podman[116026]: 2024-04-20 11:16:22.739189527 +0000 UTC m=+0.158639551 container create 34d521fe114ec465ba39cbe0177cf23d9b3bd5cc050dd690c7d4f64db623552d (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-20T11:16:23.018 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 podman[116026]: 2024-04-20 11:16:23.017973292 +0000 UTC m=+0.437423359 container init 34d521fe114ec465ba39cbe0177cf23d9b3bd5cc050dd690c7d4f64db623552d (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T11:16:23.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-mon[99036]: Health check failed: Degraded data redundancy: 4/324 objects degraded (1.235%), 2 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:23.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 podman[116026]: 2024-04-20 11:16:23.029100997 +0000 UTC m=+0.448551027 container start 34d521fe114ec465ba39cbe0177cf23d9b3bd5cc050dd690c7d4f64db623552d (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T11:16:23.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 podman[116026]: 2024-04-20 11:16:23.133223267 +0000 UTC m=+0.552673334 container attach 34d521fe114ec465ba39cbe0177cf23d9b3bd5cc050dd690c7d4f64db623552d (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-20T11:16:23.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:23 smithi173 ceph-mon[74165]: Health check failed: Degraded data redundancy: 4/324 objects degraded (1.235%), 2 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:24.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:24 smithi117 ceph-mon[99036]: pgmap v90: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 4/324 objects degraded (1.235%) 2024-04-20T11:16:24.393 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 bash[116026]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-20T11:16:24.394 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T11:16:24.395 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-20T11:16:24.395 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-20T11:16:24.395 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-20T11:16:24.395 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-20T11:16:24.395 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T11:16:24.395 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:23 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate[116041]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-20T11:16:24.395 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:24 smithi117 podman[116026]: 2024-04-20 11:16:24.038144334 +0000 UTC m=+1.457594364 container died 34d521fe114ec465ba39cbe0177cf23d9b3bd5cc050dd690c7d4f64db623552d (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:16:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:24 smithi173 ceph-mon[74165]: pgmap v90: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 4/324 objects degraded (1.235%) 2024-04-20T11:16:25.259 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:25 smithi117 ceph-mon[99036]: pgmap v91: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:25.260 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:25 smithi117 ceph-mon[99036]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-20T11:16:25.260 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:24 smithi117 podman[116186]: 2024-04-20 11:16:24.912092363 +0000 UTC m=+0.855943663 container remove 34d521fe114ec465ba39cbe0177cf23d9b3bd5cc050dd690c7d4f64db623552d (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:16:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:25 smithi173 ceph-mon[74165]: pgmap v91: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:25 smithi173 ceph-mon[74165]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-20T11:16:25.566 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:25 smithi117 podman[116251]: 2024-04-20 11:16:25.165152855 +0000 UTC m=+0.033948627 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:16:25.566 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:25 smithi117 podman[116251]: 2024-04-20 11:16:25.279671657 +0000 UTC m=+0.148467450 container create fbba94ccd1b4079fddb27ef691b52a265d8fa7901889476fcf54fc118166cca5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416) 2024-04-20T11:16:25.566 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:25 smithi117 podman[116251]: 2024-04-20 11:16:25.566261919 +0000 UTC m=+0.435057722 container init fbba94ccd1b4079fddb27ef691b52a265d8fa7901889476fcf54fc118166cca5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-20T11:16:25.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:25 smithi117 podman[116251]: 2024-04-20 11:16:25.577014147 +0000 UTC m=+0.445809976 container start fbba94ccd1b4079fddb27ef691b52a265d8fa7901889476fcf54fc118166cca5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-20T11:16:25.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:25 smithi117 ceph-osd[116270]: -- 172.21.15.117:0/1985265076 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626e2657680 con 0x5626e2621000 2024-04-20T11:16:25.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:25 smithi117 bash[116251]: fbba94ccd1b4079fddb27ef691b52a265d8fa7901889476fcf54fc118166cca5 2024-04-20T11:16:25.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:25 smithi117 systemd[1]: Started Ceph osd.1 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:16:26.639 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:26 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[116266]: 2024-04-20T11:16:26.151+0000 7f890ea907c0 -1 Falling back to public interface 2024-04-20T11:16:27.138 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:27.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:27.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:27.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:27.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:27.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:28.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:27 smithi117 ceph-mon[99036]: pgmap v92: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:28.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:27 smithi173 ceph-mon[74165]: pgmap v92: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:29 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:29 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:29 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 46/324 objects degraded (14.198%), 15 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:29 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:29 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:29 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 46/324 objects degraded (14.198%), 15 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:30.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:30 smithi117 ceph-mon[99036]: pgmap v93: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:30 smithi173 ceph-mon[74165]: pgmap v93: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:32.050 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:31 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[116266]: 2024-04-20T11:16:31.759+0000 7f890ea907c0 -1 osd.1 55 log_to_monitors true 2024-04-20T11:16:32.319 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:16:32 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[116266]: 2024-04-20T11:16:32.076+0000 7f88fde00700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:16:32.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:32 smithi117 ceph-mon[99036]: pgmap v94: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:32.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:32 smithi117 ceph-mon[99036]: from='osd.1 [v2:172.21.15.117:6810/2323295100,v1:172.21.15.117:6811/2323295100]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:16:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:32 smithi173 ceph-mon[74165]: pgmap v94: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:32 smithi173 ceph-mon[74165]: from='osd.1 [v2:172.21.15.117:6810/2323295100,v1:172.21.15.117:6811/2323295100]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:16:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:33 smithi117 ceph-mon[99036]: from='osd.1 [v2:172.21.15.117:6810/2323295100,v1:172.21.15.117:6811/2323295100]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T11:16:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:33 smithi117 ceph-mon[99036]: osdmap e58: 6 total, 5 up, 6 in 2024-04-20T11:16:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:33 smithi117 ceph-mon[99036]: from='osd.1 [v2:172.21.15.117:6810/2323295100,v1:172.21.15.117:6811/2323295100]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:16:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:33 smithi173 ceph-mon[74165]: from='osd.1 [v2:172.21.15.117:6810/2323295100,v1:172.21.15.117:6811/2323295100]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T11:16:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:33 smithi173 ceph-mon[74165]: osdmap e58: 6 total, 5 up, 6 in 2024-04-20T11:16:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:33 smithi173 ceph-mon[74165]: from='osd.1 [v2:172.21.15.117:6810/2323295100,v1:172.21.15.117:6811/2323295100]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:16:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:34 smithi117 ceph-mon[99036]: pgmap v96: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:34 smithi117 ceph-mon[99036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:16:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:34 smithi117 ceph-mon[99036]: osd.1 [v2:172.21.15.117:6810/2323295100,v1:172.21.15.117:6811/2323295100] boot 2024-04-20T11:16:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:34 smithi117 ceph-mon[99036]: osdmap e59: 6 total, 6 up, 6 in 2024-04-20T11:16:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:16:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:34 smithi117 ceph-mon[99036]: osdmap e60: 6 total, 6 up, 6 in 2024-04-20T11:16:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:34 smithi173 ceph-mon[74165]: pgmap v96: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:16:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:34 smithi173 ceph-mon[74165]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:16:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:34 smithi173 ceph-mon[74165]: osd.1 [v2:172.21.15.117:6810/2323295100,v1:172.21.15.117:6811/2323295100] boot 2024-04-20T11:16:34.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:34 smithi173 ceph-mon[74165]: osdmap e59: 6 total, 6 up, 6 in 2024-04-20T11:16:34.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:16:34.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:34.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:34.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:34 smithi173 ceph-mon[74165]: osdmap e60: 6 total, 6 up, 6 in 2024-04-20T11:16:35.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:35 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 24/324 objects degraded (7.407%), 7 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:35.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:35 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 24/324 objects degraded (7.407%), 7 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:36 smithi117 ceph-mon[99036]: pgmap v99: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24/324 objects degraded (7.407%) 2024-04-20T11:16:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:36 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:36 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:36 smithi173 ceph-mon[74165]: pgmap v99: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24/324 objects degraded (7.407%) 2024-04-20T11:16:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:36 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:36 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:38 smithi173 ceph-mon[74165]: pgmap v100: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24/324 objects degraded (7.407%) 2024-04-20T11:16:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:38 smithi117 ceph-mon[99036]: pgmap v100: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24/324 objects degraded (7.407%) 2024-04-20T11:16:39.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:39 smithi117 ceph-mon[99036]: pgmap v101: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:39.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:39 smithi117 ceph-mon[99036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/324 objects degraded (7.407%), 7 pgs degraded) 2024-04-20T11:16:39.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:39 smithi173 ceph-mon[74165]: pgmap v101: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:39 smithi173 ceph-mon[74165]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/324 objects degraded (7.407%), 7 pgs degraded) 2024-04-20T11:16:42.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: pgmap v102: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:42.250 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:42.251 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: pgmap v102: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:42.402 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-20T11:16:43.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:43 smithi117 ceph-mon[99036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-20T11:16:43.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:43 smithi117 ceph-mon[99036]: Upgrade: osd.2 is safe to restart 2024-04-20T11:16:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:43 smithi173 ceph-mon[74165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-20T11:16:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:43 smithi173 ceph-mon[74165]: Upgrade: osd.2 is safe to restart 2024-04-20T11:16:43.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.489+0000 7fc97e200700 1 -- 172.21.15.117:0/2259893045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 msgr2=0x7fc97810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.489+0000 7fc97e200700 1 --2- 172.21.15.117:0/2259893045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 0x7fc97810cfd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc964009910 tx=0x7fc964009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.490+0000 7fc97e200700 1 -- 172.21.15.117:0/2259893045 shutdown_connections 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.490+0000 7fc97e200700 1 --2- 172.21.15.117:0/2259893045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 0x7fc97810cfd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.490+0000 7fc97e200700 1 --2- 172.21.15.117:0/2259893045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9780ff960 0x7fc9780ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.490+0000 7fc97e200700 1 -- 172.21.15.117:0/2259893045 >> 172.21.15.117:0/2259893045 conn(0x7fc9780fb5c0 msgr2=0x7fc9780fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.490+0000 7fc97e200700 1 -- 172.21.15.117:0/2259893045 shutdown_connections 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.490+0000 7fc97e200700 1 -- 172.21.15.117:0/2259893045 wait complete. 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.491+0000 7fc97e200700 1 Processor -- start 2024-04-20T11:16:43.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.491+0000 7fc97e200700 1 -- start start 2024-04-20T11:16:43.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.491+0000 7fc97e200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9780ff960 0x7fc978198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:43.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.491+0000 7fc97e200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 0x7fc978199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:43.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc97e200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc978199a40 con 0x7fc978100280 2024-04-20T11:16:43.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc97e200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97819d710 con 0x7fc9780ff960 2024-04-20T11:16:43.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc976a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 0x7fc978199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:43.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc976a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 0x7fc978199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40800/0 (socket says 172.21.15.117:40800) 2024-04-20T11:16:43.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc976a00700 1 -- 172.21.15.117:0/2419620858 learned_addr learned my addr 172.21.15.117:0/2419620858 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:43.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc977400700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9780ff960 0x7fc978198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc976a00700 1 -- 172.21.15.117:0/2419620858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9780ff960 msgr2=0x7fc978198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc976a00700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9780ff960 0x7fc978198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.492+0000 7fc976a00700 1 -- 172.21.15.117:0/2419620858 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640095f0 con 0x7fc978100280 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.493+0000 7fc976a00700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 0x7fc978199360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9640092e0 tx=0x7fc9640056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.493+0000 7fc96fe00700 1 -- 172.21.15.117:0/2419620858 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc96401d070 con 0x7fc978100280 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.493+0000 7fc96fe00700 1 -- 172.21.15.117:0/2419620858 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc96400bed0 con 0x7fc978100280 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.493+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97819d990 con 0x7fc978100280 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.493+0000 7fc96fe00700 1 -- 172.21.15.117:0/2419620858 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc964021410 con 0x7fc978100280 2024-04-20T11:16:43.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.493+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97819dee0 con 0x7fc978100280 2024-04-20T11:16:43.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.495+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97810ae50 con 0x7fc978100280 2024-04-20T11:16:43.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.496+0000 7fc96fe00700 1 -- 172.21.15.117:0/2419620858 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc96400f460 con 0x7fc978100280 2024-04-20T11:16:43.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.497+0000 7fc96fe00700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc95c06ecc0 0x7fc95c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:43.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.497+0000 7fc96fe00700 1 -- 172.21.15.117:0/2419620858 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fc96408fff0 con 0x7fc978100280 2024-04-20T11:16:43.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.497+0000 7fc977400700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc95c06ecc0 0x7fc95c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:43.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.498+0000 7fc977400700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc95c06ecc0 0x7fc95c071180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc968006fd0 tx=0x7fc968008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:43.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.502+0000 7fc96fe00700 1 -- 172.21.15.117:0/2419620858 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc964058ea0 con 0x7fc978100280 2024-04-20T11:16:43.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.751+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc97819a130 con 0x7fc95c06ecc0 2024-04-20T11:16:43.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.754+0000 7fc96fe00700 1 -- 172.21.15.117:0/2419620858 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc97819a130 con 0x7fc95c06ecc0 2024-04-20T11:16:43.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc95c06ecc0 msgr2=0x7fc95c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:43.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc95c06ecc0 0x7fc95c071180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc968006fd0 tx=0x7fc968008040 comp rx=0 tx=0).stop 2024-04-20T11:16:43.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 msgr2=0x7fc978199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:43.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 0x7fc978199360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9640092e0 tx=0x7fc9640056a0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 shutdown_connections 2024-04-20T11:16:43.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc95c06ecc0 0x7fc95c071180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9780ff960 0x7fc978198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 --2- 172.21.15.117:0/2419620858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc978100280 0x7fc978199360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 >> 172.21.15.117:0/2419620858 conn(0x7fc9780fb5c0 msgr2=0x7fc978107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:43.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 shutdown_connections 2024-04-20T11:16:43.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.757+0000 7fc97e200700 1 -- 172.21.15.117:0/2419620858 wait complete. 2024-04-20T11:16:43.773 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:16:43.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.914+0000 7febf8a00700 1 -- 172.21.15.117:0/367030131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 msgr2=0x7febf40736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:43.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.914+0000 7febf8a00700 1 --2- 172.21.15.117:0/367030131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 0x7febf40736f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7febe4009910 tx=0x7febe4009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:43.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.915+0000 7febf8a00700 1 -- 172.21.15.117:0/367030131 shutdown_connections 2024-04-20T11:16:43.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.915+0000 7febf8a00700 1 --2- 172.21.15.117:0/367030131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf4073c30 0x7febf410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.915+0000 7febf8a00700 1 --2- 172.21.15.117:0/367030131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 0x7febf40736f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.915+0000 7febf8a00700 1 -- 172.21.15.117:0/367030131 >> 172.21.15.117:0/367030131 conn(0x7febf40fc6c0 msgr2=0x7febf40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:43.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.915+0000 7febf8a00700 1 -- 172.21.15.117:0/367030131 shutdown_connections 2024-04-20T11:16:43.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.915+0000 7febf8a00700 1 -- 172.21.15.117:0/367030131 wait complete. 2024-04-20T11:16:43.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.916+0000 7febf8a00700 1 Processor -- start 2024-04-20T11:16:43.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.916+0000 7febf8a00700 1 -- start start 2024-04-20T11:16:43.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf8a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 0x7febf4198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:43.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf8a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf4073c30 0x7febf4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:43.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf8a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf41998d0 con 0x7febf4073310 2024-04-20T11:16:43.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf8a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf419c540 con 0x7febf4073c30 2024-04-20T11:16:43.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 0x7febf4198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:43.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 0x7febf4198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41886/0 (socket says 172.21.15.117:41886) 2024-04-20T11:16:43.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf1e00700 1 -- 172.21.15.117:0/1361364907 learned_addr learned my addr 172.21.15.117:0/1361364907 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf1400700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf4073c30 0x7febf4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf1e00700 1 -- 172.21.15.117:0/1361364907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf4073c30 msgr2=0x7febf4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf1e00700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf4073c30 0x7febf4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.917+0000 7febf1e00700 1 -- 172.21.15.117:0/1361364907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe40095f0 con 0x7febf4073310 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.918+0000 7febf1e00700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 0x7febf4198d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7febe40092e0 tx=0x7febe400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.918+0000 7febeaa00700 1 -- 172.21.15.117:0/1361364907 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe401d070 con 0x7febf4073310 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.918+0000 7febeaa00700 1 -- 172.21.15.117:0/1361364907 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febe40056e0 con 0x7febf4073310 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.918+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf419c7c0 con 0x7febf4073310 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.918+0000 7febeaa00700 1 -- 172.21.15.117:0/1361364907 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe400fe80 con 0x7febf4073310 2024-04-20T11:16:43.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.918+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf419ccb0 con 0x7febf4073310 2024-04-20T11:16:43.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.920+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf40749c0 con 0x7febf4073310 2024-04-20T11:16:43.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.920+0000 7febeaa00700 1 -- 172.21.15.117:0/1361364907 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7febe400f6f0 con 0x7febf4073310 2024-04-20T11:16:43.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.921+0000 7febeaa00700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7febd806ecc0 0x7febd8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:43.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.921+0000 7febeaa00700 1 -- 172.21.15.117:0/1361364907 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6285+0+0 (secure 0 0 0) 0x7febe40900e0 con 0x7febf4073310 2024-04-20T11:16:43.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.922+0000 7febf1400700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7febd806ecc0 0x7febd8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:43.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.923+0000 7febf1400700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7febd806ecc0 0x7febd8071180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7febe0005950 tx=0x7febe00058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:43.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:43.926+0000 7febeaa00700 1 -- 172.21.15.117:0/1361364907 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7febe4058ff0 con 0x7febf4073310 2024-04-20T11:16:44.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.189+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febf4107750 con 0x7febd806ecc0 2024-04-20T11:16:44.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.191+0000 7febeaa00700 1 -- 172.21.15.117:0/1361364907 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7febf4107750 con 0x7febd806ecc0 2024-04-20T11:16:44.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7febd806ecc0 msgr2=0x7febd8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7febd806ecc0 0x7febd8071180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7febe0005950 tx=0x7febe00058e0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 msgr2=0x7febf4198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 0x7febf4198d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7febe40092e0 tx=0x7febe400be20 comp rx=0 tx=0).stop 2024-04-20T11:16:44.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 shutdown_connections 2024-04-20T11:16:44.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7febd806ecc0 0x7febd8071180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf4073310 0x7febf4198d00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 --2- 172.21.15.117:0/1361364907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf4073c30 0x7febf4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 >> 172.21.15.117:0/1361364907 conn(0x7febf40fc6c0 msgr2=0x7febf4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:44.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.193+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 shutdown_connections 2024-04-20T11:16:44.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.194+0000 7febf8a00700 1 -- 172.21.15.117:0/1361364907 wait complete. 2024-04-20T11:16:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.351+0000 7fda2f400700 1 -- 172.21.15.117:0/1690451929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28101d40 msgr2=0x7fda28102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.351+0000 7fda2f400700 1 --2- 172.21.15.117:0/1690451929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28101d40 0x7fda28102120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009910 tx=0x7fda1c009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:44.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.351+0000 7fda2f400700 1 -- 172.21.15.117:0/1690451929 shutdown_connections 2024-04-20T11:16:44.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.351+0000 7fda2f400700 1 --2- 172.21.15.117:0/1690451929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda28102660 0x7fda2810ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.351+0000 7fda2f400700 1 --2- 172.21.15.117:0/1690451929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28101d40 0x7fda28102120 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.351+0000 7fda2f400700 1 -- 172.21.15.117:0/1690451929 >> 172.21.15.117:0/1690451929 conn(0x7fda28076750 msgr2=0x7fda280fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:44.352 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:44 smithi117 ceph-mon[99036]: pgmap v103: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:44.352 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:44.353 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:44.353 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:44.353 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:16:44.353 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.351+0000 7fda2f400700 1 -- 172.21.15.117:0/1690451929 shutdown_connections 2024-04-20T11:16:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.351+0000 7fda2f400700 1 -- 172.21.15.117:0/1690451929 wait complete. 2024-04-20T11:16:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.352+0000 7fda2f400700 1 Processor -- start 2024-04-20T11:16:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.352+0000 7fda2f400700 1 -- start start 2024-04-20T11:16:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda2f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda28101d40 0x7fda28198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda2f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28102660 0x7fda281993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda2f400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda28199ac0 con 0x7fda28102660 2024-04-20T11:16:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda2f400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2819d790 con 0x7fda28101d40 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda27e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28102660 0x7fda281993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda27e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28102660 0x7fda281993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41902/0 (socket says 172.21.15.117:41902) 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda27e00700 1 -- 172.21.15.117:0/2529785521 learned_addr learned my addr 172.21.15.117:0/2529785521 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda2ca00700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda28101d40 0x7fda28198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda27e00700 1 -- 172.21.15.117:0/2529785521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda28101d40 msgr2=0x7fda28198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda27e00700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda28101d40 0x7fda28198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.353+0000 7fda27e00700 1 -- 172.21.15.117:0/2529785521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c0095f0 con 0x7fda28102660 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.354+0000 7fda27e00700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28102660 0x7fda281993e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda1400b3d0 tx=0x7fda1400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.354+0000 7fda25600700 1 -- 172.21.15.117:0/2529785521 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda14011800 con 0x7fda28102660 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.354+0000 7fda25600700 1 -- 172.21.15.117:0/2529785521 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda14011e20 con 0x7fda28102660 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.354+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2819da70 con 0x7fda28102660 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.354+0000 7fda25600700 1 -- 172.21.15.117:0/2529785521 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda14010ea0 con 0x7fda28102660 2024-04-20T11:16:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.354+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2819dfc0 con 0x7fda28102660 2024-04-20T11:16:44.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.355+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2810a6b0 con 0x7fda28102660 2024-04-20T11:16:44.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.357+0000 7fda25600700 1 -- 172.21.15.117:0/2529785521 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fda14010380 con 0x7fda28102660 2024-04-20T11:16:44.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.358+0000 7fda25600700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda1006ecc0 0x7fda10071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.358+0000 7fda25600700 1 -- 172.21.15.117:0/2529785521 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fda1408e840 con 0x7fda28102660 2024-04-20T11:16:44.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.358+0000 7fda2ca00700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda1006ecc0 0x7fda10071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.359+0000 7fda2ca00700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda1006ecc0 0x7fda10071180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00b5c0 tx=0x7fda1c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:44.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.362+0000 7fda25600700 1 -- 172.21.15.117:0/2529785521 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fda140576d0 con 0x7fda28102660 2024-04-20T11:16:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:44 smithi173 ceph-mon[74165]: pgmap v103: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:44.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:16:44.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:44.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.604+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda2810de40 con 0x7fda1006ecc0 2024-04-20T11:16:44.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.622+0000 7fda25600700 1 -- 172.21.15.117:0/2529785521 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fda2810de40 con 0x7fda1006ecc0 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (5m) 11s ago 19m 13.9M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (19m) 11s ago 19m 8715k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (18m) 77s ago 18m 8270k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (90s) 11s ago 19m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (82s) 77s ago 18m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (4m) 11s ago 18m 73.6M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (11m) 11s ago 11m 18.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (12m) 11s ago 12m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (11m) 77s ago 11m 17.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (12m) 77s ago 12m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (8m) 11s ago 21m 505M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:16:44.623 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (6m) 77s ago 17m 450M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (3m) 11s ago 21m 49.9M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (2m) 77s ago 17m 33.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (6m) 11s ago 19m 7772k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (6m) 77s ago 18m 7377k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (51s) 11s ago 16m 79.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (19s) 11s ago 16m 11.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (15m) 11s ago 15m 184M 1216M 18.2.2-1237-g646b8c7d 9900ed1a6b81 06f9de7eede3 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (15m) 77s ago 15m 244M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (14m) 77s ago 14m 215M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (13m) 77s ago 13m 202M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:16:44.624 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (5m) 11s ago 18m 68.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:16:44.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda1006ecc0 msgr2=0x7fda10071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda1006ecc0 0x7fda10071180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00b5c0 tx=0x7fda1c01a040 comp rx=0 tx=0).stop 2024-04-20T11:16:44.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28102660 msgr2=0x7fda281993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28102660 0x7fda281993e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda1400b3d0 tx=0x7fda1400b790 comp rx=0 tx=0).stop 2024-04-20T11:16:44.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 shutdown_connections 2024-04-20T11:16:44.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda1006ecc0 0x7fda10071180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda28101d40 0x7fda28198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 --2- 172.21.15.117:0/2529785521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda28102660 0x7fda281993e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.624+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 >> 172.21.15.117:0/2529785521 conn(0x7fda28076750 msgr2=0x7fda28101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:44.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.625+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 shutdown_connections 2024-04-20T11:16:44.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.625+0000 7fda2f400700 1 -- 172.21.15.117:0/2529785521 wait complete. 2024-04-20T11:16:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.771+0000 7ff29bc00700 1 -- 172.21.15.117:0/4251463992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff294100e20 msgr2=0x7ff294101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.771+0000 7ff29bc00700 1 --2- 172.21.15.117:0/4251463992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff294100e20 0x7ff294101200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff288009910 tx=0x7ff288009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.772+0000 7ff29bc00700 1 -- 172.21.15.117:0/4251463992 shutdown_connections 2024-04-20T11:16:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.772+0000 7ff29bc00700 1 --2- 172.21.15.117:0/4251463992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2941017d0 0x7ff294105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.772+0000 7ff29bc00700 1 --2- 172.21.15.117:0/4251463992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff294100e20 0x7ff294101200 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.772+0000 7ff29bc00700 1 -- 172.21.15.117:0/4251463992 >> 172.21.15.117:0/4251463992 conn(0x7ff2940fc6f0 msgr2=0x7ff2940feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.772+0000 7ff29bc00700 1 -- 172.21.15.117:0/4251463992 shutdown_connections 2024-04-20T11:16:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.772+0000 7ff29bc00700 1 -- 172.21.15.117:0/4251463992 wait complete. 2024-04-20T11:16:44.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.773+0000 7ff29bc00700 1 Processor -- start 2024-04-20T11:16:44.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.773+0000 7ff29bc00700 1 -- start start 2024-04-20T11:16:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff29bc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff294100e20 0x7ff294198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff29bc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2941017d0 0x7ff294199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff29bc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff294199a40 con 0x7ff2941017d0 2024-04-20T11:16:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff29bc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29419d710 con 0x7ff294100e20 2024-04-20T11:16:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff293e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2941017d0 0x7ff294199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff293e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2941017d0 0x7ff294199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41932/0 (socket says 172.21.15.117:41932) 2024-04-20T11:16:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff293e00700 1 -- 172.21.15.117:0/2112359717 learned_addr learned my addr 172.21.15.117:0/2112359717 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:44.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff299200700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff294100e20 0x7ff294198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff293e00700 1 -- 172.21.15.117:0/2112359717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff294100e20 msgr2=0x7ff294198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff293e00700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff294100e20 0x7ff294198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.774+0000 7ff293e00700 1 -- 172.21.15.117:0/2112359717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880095f0 con 0x7ff2941017d0 2024-04-20T11:16:44.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.775+0000 7ff293e00700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2941017d0 0x7ff294199360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff28000b480 tx=0x7ff28000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:44.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.775+0000 7ff291600700 1 -- 172.21.15.117:0/2112359717 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2800118c0 con 0x7ff2941017d0 2024-04-20T11:16:44.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.775+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29419d9f0 con 0x7ff2941017d0 2024-04-20T11:16:44.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.775+0000 7ff291600700 1 -- 172.21.15.117:0/2112359717 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff280010380 con 0x7ff2941017d0 2024-04-20T11:16:44.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.775+0000 7ff291600700 1 -- 172.21.15.117:0/2112359717 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff28000f4f0 con 0x7ff2941017d0 2024-04-20T11:16:44.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.775+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29419df40 con 0x7ff2941017d0 2024-04-20T11:16:44.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.777+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff294102300 con 0x7ff2941017d0 2024-04-20T11:16:44.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.777+0000 7ff291600700 1 -- 172.21.15.117:0/2112359717 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff2800104f0 con 0x7ff2941017d0 2024-04-20T11:16:44.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.779+0000 7ff291600700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff27c06ecc0 0x7ff27c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.779+0000 7ff291600700 1 -- 172.21.15.117:0/2112359717 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6285+0+0 (secure 0 0 0) 0x7ff28008e9b0 con 0x7ff2941017d0 2024-04-20T11:16:44.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.779+0000 7ff299200700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff27c06ecc0 0x7ff27c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.780+0000 7ff299200700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff27c06ecc0 0x7ff27c071180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff28800b5c0 tx=0x7ff288005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:44.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:44.784+0000 7ff291600700 1 -- 172.21.15.117:0/2112359717 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff280057840 con 0x7ff2941017d0 2024-04-20T11:16:45.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.102+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff294062510 con 0x7ff2941017d0 2024-04-20T11:16:45.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.104+0000 7ff291600700 1 -- 172.21.15.117:0/2112359717 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7ff28005af00 con 0x7ff2941017d0 2024-04-20T11:16:45.104 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4, 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:16:45.105 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:16:45.106 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 8, 2024-04-20T11:16:45.106 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:16:45.106 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:16:45.106 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:16:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff27c06ecc0 msgr2=0x7ff27c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff27c06ecc0 0x7ff27c071180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff28800b5c0 tx=0x7ff288005960 comp rx=0 tx=0).stop 2024-04-20T11:16:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2941017d0 msgr2=0x7ff294199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2941017d0 0x7ff294199360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff28000b480 tx=0x7ff28000b840 comp rx=0 tx=0).stop 2024-04-20T11:16:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 shutdown_connections 2024-04-20T11:16:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff27c06ecc0 0x7ff27c071180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff294100e20 0x7ff294198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 --2- 172.21.15.117:0/2112359717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2941017d0 0x7ff294199360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.106+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 >> 172.21.15.117:0/2112359717 conn(0x7ff2940fc6f0 msgr2=0x7ff2940fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.107+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 shutdown_connections 2024-04-20T11:16:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.107+0000 7ff29bc00700 1 -- 172.21.15.117:0/2112359717 wait complete. 2024-04-20T11:16:45.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.255+0000 7fc367200700 1 -- 172.21.15.117:0/772310044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36810d0c0 msgr2=0x7fc36810d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.255+0000 7fc367200700 1 --2- 172.21.15.117:0/772310044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36810d0c0 0x7fc36810d4a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc358009910 tx=0x7fc358009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:45.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.256+0000 7fc367200700 1 -- 172.21.15.117:0/772310044 shutdown_connections 2024-04-20T11:16:45.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.256+0000 7fc367200700 1 --2- 172.21.15.117:0/772310044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3681070b0 0x7fc368107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.256+0000 7fc367200700 1 --2- 172.21.15.117:0/772310044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36810d0c0 0x7fc36810d4a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.256+0000 7fc367200700 1 -- 172.21.15.117:0/772310044 >> 172.21.15.117:0/772310044 conn(0x7fc368076830 msgr2=0x7fc368078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:45.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.256+0000 7fc367200700 1 -- 172.21.15.117:0/772310044 shutdown_connections 2024-04-20T11:16:45.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.256+0000 7fc367200700 1 -- 172.21.15.117:0/772310044 wait complete. 2024-04-20T11:16:45.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.257+0000 7fc367200700 1 Processor -- start 2024-04-20T11:16:45.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.257+0000 7fc367200700 1 -- start start 2024-04-20T11:16:45.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.257+0000 7fc367200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3681070b0 0x7fc36819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:45.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.257+0000 7fc367200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc36810d0c0 0x7fc36819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:45.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.257+0000 7fc367200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36819dca0 con 0x7fc3681070b0 2024-04-20T11:16:45.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.257+0000 7fc367200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681a1920 con 0x7fc36810d0c0 2024-04-20T11:16:45.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.258+0000 7fc365e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3681070b0 0x7fc36819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:45.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.258+0000 7fc365e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3681070b0 0x7fc36819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41958/0 (socket says 172.21.15.117:41958) 2024-04-20T11:16:45.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.258+0000 7fc365e00700 1 -- 172.21.15.117:0/3883103775 learned_addr learned my addr 172.21.15.117:0/3883103775 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.258+0000 7fc365e00700 1 -- 172.21.15.117:0/3883103775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc36810d0c0 msgr2=0x7fc36819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.258+0000 7fc365e00700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc36810d0c0 0x7fc36819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.258+0000 7fc365e00700 1 -- 172.21.15.117:0/3883103775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580095f0 con 0x7fc3681070b0 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.258+0000 7fc365e00700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3681070b0 0x7fc36819d080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc358006010 tx=0x7fc35800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.259+0000 7fc35ea00700 1 -- 172.21.15.117:0/3883103775 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc35801d070 con 0x7fc3681070b0 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.259+0000 7fc35ea00700 1 -- 172.21.15.117:0/3883103775 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc35800f460 con 0x7fc3681070b0 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.259+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3681a1ba0 con 0x7fc3681070b0 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.259+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681a2090 con 0x7fc3681070b0 2024-04-20T11:16:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.259+0000 7fc35ea00700 1 -- 172.21.15.117:0/3883103775 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc358017630 con 0x7fc3681070b0 2024-04-20T11:16:45.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.260+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3480052f0 con 0x7fc3681070b0 2024-04-20T11:16:45.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.261+0000 7fc35ea00700 1 -- 172.21.15.117:0/3883103775 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc35800fab0 con 0x7fc3681070b0 2024-04-20T11:16:45.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.262+0000 7fc35ea00700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc34c06ecc0 0x7fc34c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:45.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.262+0000 7fc35ea00700 1 -- 172.21.15.117:0/3883103775 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fc358090250 con 0x7fc3681070b0 2024-04-20T11:16:45.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.262+0000 7fc365400700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc34c06ecc0 0x7fc34c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:45.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.263+0000 7fc365400700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc34c06ecc0 0x7fc34c071180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc36819e590 tx=0x7fc354006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:45.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.266+0000 7fc35ea00700 1 -- 172.21.15.117:0/3883103775 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc358059100 con 0x7fc3681070b0 2024-04-20T11:16:45.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:45 smithi117 ceph-mon[99036]: Upgrade: Updating osd.2 2024-04-20T11:16:45.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:45 smithi117 ceph-mon[99036]: Deploying daemon osd.2 on smithi117 2024-04-20T11:16:45.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:45 smithi117 ceph-mon[99036]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:45 smithi173 ceph-mon[74165]: Upgrade: Updating osd.2 2024-04-20T11:16:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:45 smithi173 ceph-mon[74165]: Deploying daemon osd.2 on smithi117 2024-04-20T11:16:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:45 smithi173 ceph-mon[74165]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.531+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc348005db0 con 0x7fc3681070b0 2024-04-20T11:16:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.532+0000 7fc35ea00700 1 -- 172.21.15.117:0/3883103775 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7fc35805c7c0 con 0x7fc3681070b0 2024-04-20T11:16:45.533 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:16:45.533 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:16:45.533 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:16:45.533 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:16:45.533 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:16:45.534 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:16:45.535 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:16:45.536 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:16:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:16:45.537 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:16:45.537 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc34c06ecc0 msgr2=0x7fc34c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc34c06ecc0 0x7fc34c071180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc36819e590 tx=0x7fc354006d20 comp rx=0 tx=0).stop 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3681070b0 msgr2=0x7fc36819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3681070b0 0x7fc36819d080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc358006010 tx=0x7fc35800be20 comp rx=0 tx=0).stop 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 shutdown_connections 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc34c06ecc0 0x7fc34c071180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3681070b0 0x7fc36819d080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 --2- 172.21.15.117:0/3883103775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc36810d0c0 0x7fc36819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.535+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 >> 172.21.15.117:0/3883103775 conn(0x7fc368076830 msgr2=0x7fc3680789e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:45.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.536+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 shutdown_connections 2024-04-20T11:16:45.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.536+0000 7fc367200700 1 -- 172.21.15.117:0/3883103775 wait complete. 2024-04-20T11:16:45.539 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:16:45.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.687+0000 7fbbc0600700 1 -- 172.21.15.117:0/1017648475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 msgr2=0x7fbbb810ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.687+0000 7fbbc0600700 1 --2- 172.21.15.117:0/1017648475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 0x7fbbb810ab50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009910 tx=0x7fbba4009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:45.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.687+0000 7fbbc0600700 1 -- 172.21.15.117:0/1017648475 shutdown_connections 2024-04-20T11:16:45.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.687+0000 7fbbc0600700 1 --2- 172.21.15.117:0/1017648475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 0x7fbbb810ab50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.687+0000 7fbbc0600700 1 --2- 172.21.15.117:0/1017648475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb8101d40 0x7fbbb8102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.687+0000 7fbbc0600700 1 -- 172.21.15.117:0/1017648475 >> 172.21.15.117:0/1017648475 conn(0x7fbbb8076750 msgr2=0x7fbbb80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:45.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.687+0000 7fbbc0600700 1 -- 172.21.15.117:0/1017648475 shutdown_connections 2024-04-20T11:16:45.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.687+0000 7fbbc0600700 1 -- 172.21.15.117:0/1017648475 wait complete. 2024-04-20T11:16:45.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.688+0000 7fbbc0600700 1 Processor -- start 2024-04-20T11:16:45.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.688+0000 7fbbc0600700 1 -- start start 2024-04-20T11:16:45.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbc0600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb8101d40 0x7fbbb819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:45.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbc0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 0x7fbbb819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:45.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbc0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb819d3a0 con 0x7fbbb8102660 2024-04-20T11:16:45.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbc0600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8196980 con 0x7fbbb8101d40 2024-04-20T11:16:45.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbbd200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 0x7fbbb819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:45.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbbd200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 0x7fbbb819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41976/0 (socket says 172.21.15.117:41976) 2024-04-20T11:16:45.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbbd200700 1 -- 172.21.15.117:0/596154347 learned_addr learned my addr 172.21.15.117:0/596154347 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbbdc00700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb8101d40 0x7fbbb819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbbd200700 1 -- 172.21.15.117:0/596154347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb8101d40 msgr2=0x7fbbb819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbbd200700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb8101d40 0x7fbbb819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbbd200700 1 -- 172.21.15.117:0/596154347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40095f0 con 0x7fbbb8102660 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.689+0000 7fbbbdc00700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb8101d40 0x7fbbb819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.690+0000 7fbbbd200700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 0x7fbbb819cd80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbba40092e0 tx=0x7fbba400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.690+0000 7fbbb6000700 1 -- 172.21.15.117:0/596154347 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbba401d070 con 0x7fbbb8102660 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.690+0000 7fbbb6000700 1 -- 172.21.15.117:0/596154347 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbba4003d50 con 0x7fbbb8102660 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.690+0000 7fbbb6000700 1 -- 172.21.15.117:0/596154347 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbba4017400 con 0x7fbbb8102660 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.690+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb8196c00 con 0x7fbbb8102660 2024-04-20T11:16:45.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.690+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb81970f0 con 0x7fbbb8102660 2024-04-20T11:16:45.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.691+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb81054d0 con 0x7fbbb8102660 2024-04-20T11:16:45.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.692+0000 7fbbb6000700 1 -- 172.21.15.117:0/596154347 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbba4003890 con 0x7fbbb8102660 2024-04-20T11:16:45.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.693+0000 7fbbb6000700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbba006ecc0 0x7fbba0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:45.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.693+0000 7fbbb6000700 1 -- 172.21.15.117:0/596154347 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fbba4090150 con 0x7fbbb8102660 2024-04-20T11:16:45.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.693+0000 7fbbbdc00700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbba006ecc0 0x7fbba0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:45.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.694+0000 7fbbbdc00700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbba006ecc0 0x7fbba0071180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbbac005fd0 tx=0x7fbbac005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:45.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.698+0000 7fbbb6000700 1 -- 172.21.15.117:0/596154347 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbba405caf0 con 0x7fbbb8102660 2024-04-20T11:16:45.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.983+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbb8197eb0 con 0x7fbba006ecc0 2024-04-20T11:16:45.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.985+0000 7fbbb6000700 1 -- 172.21.15.117:0/596154347 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbbb8197eb0 con 0x7fbba006ecc0 2024-04-20T11:16:45.985 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "8/23 daemons upgraded", 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading osd daemons", 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:16:45.986 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbba006ecc0 msgr2=0x7fbba0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbba006ecc0 0x7fbba0071180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbbac005fd0 tx=0x7fbbac005dc0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 msgr2=0x7fbbb819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 0x7fbbb819cd80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbba40092e0 tx=0x7fbba400be20 comp rx=0 tx=0).stop 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 shutdown_connections 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbba006ecc0 0x7fbba0071180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb8101d40 0x7fbbb819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 --2- 172.21.15.117:0/596154347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbb8102660 0x7fbbb819cd80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 >> 172.21.15.117:0/596154347 conn(0x7fbbb8076750 msgr2=0x7fbbb8103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 shutdown_connections 2024-04-20T11:16:45.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:45.988+0000 7fbbc0600700 1 -- 172.21.15.117:0/596154347 wait complete. 2024-04-20T11:16:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.151+0000 7f51a7e00700 1 -- 172.21.15.117:0/1638623580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 msgr2=0x7f51a010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.151+0000 7f51a7e00700 1 --2- 172.21.15.117:0/1638623580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 0x7f51a010cf40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f518c009910 tx=0x7f518c009c20 comp rx=0 tx=0).stop 2024-04-20T11:16:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.151+0000 7f51a7e00700 1 -- 172.21.15.117:0/1638623580 shutdown_connections 2024-04-20T11:16:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.151+0000 7f51a7e00700 1 --2- 172.21.15.117:0/1638623580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 0x7f51a010cf40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.151+0000 7f51a7e00700 1 --2- 172.21.15.117:0/1638623580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a00ff8d0 0x7f51a00ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.151+0000 7f51a7e00700 1 -- 172.21.15.117:0/1638623580 >> 172.21.15.117:0/1638623580 conn(0x7f51a0076750 msgr2=0x7f51a00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.151+0000 7f51a7e00700 1 -- 172.21.15.117:0/1638623580 shutdown_connections 2024-04-20T11:16:46.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.151+0000 7f51a7e00700 1 -- 172.21.15.117:0/1638623580 wait complete. 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.152+0000 7f51a7e00700 1 Processor -- start 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.152+0000 7f51a7e00700 1 -- start start 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a7e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a00ff8d0 0x7f51a019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 0x7f51a019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a7e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a019f4c0 con 0x7f51a0100280 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a7e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0198b30 con 0x7f51a00ff8d0 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 0x7f51a019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 0x7f51a019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41996/0 (socket says 172.21.15.117:41996) 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a4a00700 1 -- 172.21.15.117:0/1167810839 learned_addr learned my addr 172.21.15.117:0/1167810839 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a5400700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a00ff8d0 0x7f51a019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:46.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a4a00700 1 -- 172.21.15.117:0/1167810839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a00ff8d0 msgr2=0x7f51a019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:46.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a4a00700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a00ff8d0 0x7f51a019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:46.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.153+0000 7f51a4a00700 1 -- 172.21.15.117:0/1167810839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0095f0 con 0x7f51a0100280 2024-04-20T11:16:46.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.154+0000 7f51a4a00700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 0x7f51a019ef30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f518c006010 tx=0x7f518c005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:46.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.154+0000 7f519e000700 1 -- 172.21.15.117:0/1167810839 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f518c01d070 con 0x7f51a0100280 2024-04-20T11:16:46.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.154+0000 7f519e000700 1 -- 172.21.15.117:0/1167810839 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f518c0056c0 con 0x7f51a0100280 2024-04-20T11:16:46.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.154+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a0198db0 con 0x7f51a0100280 2024-04-20T11:16:46.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.154+0000 7f519e000700 1 -- 172.21.15.117:0/1167810839 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f518c021640 con 0x7f51a0100280 2024-04-20T11:16:46.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.154+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a0199300 con 0x7f51a0100280 2024-04-20T11:16:46.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.155+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a01078c0 con 0x7f51a0100280 2024-04-20T11:16:46.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.156+0000 7f519e000700 1 -- 172.21.15.117:0/1167810839 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f518c00f460 con 0x7f51a0100280 2024-04-20T11:16:46.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.157+0000 7f519e000700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518806ecc0 0x7f5188071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:46.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.157+0000 7f519e000700 1 -- 172.21.15.117:0/1167810839 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f518c090110 con 0x7f51a0100280 2024-04-20T11:16:46.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.158+0000 7f51a5400700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518806ecc0 0x7f5188071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:46.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.158+0000 7f51a5400700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518806ecc0 0x7f5188071180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5194005ea0 tx=0x7f5194005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:46.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.162+0000 7f519e000700 1 -- 172.21.15.117:0/1167810839 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f518c05ca30 con 0x7f51a0100280 2024-04-20T11:16:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:46 smithi117 ceph-mon[99036]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:46 smithi117 ceph-mon[99036]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:46 smithi117 ceph-mon[99036]: pgmap v104: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:46 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2112359717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:46 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3883103775' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:16:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:46 smithi173 ceph-mon[74165]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:46 smithi173 ceph-mon[74165]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:46 smithi173 ceph-mon[74165]: pgmap v104: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:46 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2112359717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:16:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:46 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3883103775' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:16:46.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.488+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f51a0068e50 con 0x7f51a0100280 2024-04-20T11:16:46.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.489+0000 7f519e000700 1 -- 172.21.15.117:0/1167810839 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f518c05c5c0 con 0x7f51a0100280 2024-04-20T11:16:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:16:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:16:46.490 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518806ecc0 msgr2=0x7f5188071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518806ecc0 0x7f5188071180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5194005ea0 tx=0x7f5194005e30 comp rx=0 tx=0).stop 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 msgr2=0x7f51a019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 0x7f51a019ef30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f518c006010 tx=0x7f518c005470 comp rx=0 tx=0).stop 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 shutdown_connections 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518806ecc0 0x7f5188071180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a00ff8d0 0x7f51a019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 --2- 172.21.15.117:0/1167810839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a0100280 0x7f51a019ef30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 >> 172.21.15.117:0/1167810839 conn(0x7f51a0076750 msgr2=0x7f51a00fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 shutdown_connections 2024-04-20T11:16:46.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:16:46.492+0000 7f51a7e00700 1 -- 172.21.15.117:0/1167810839 wait complete. 2024-04-20T11:16:47.369 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:47 smithi117 ceph-mon[99036]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:47.369 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:47 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1167810839' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:16:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:47 smithi173 ceph-mon[74165]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:47 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1167810839' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:16:48.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:48 smithi117 ceph-mon[99036]: pgmap v105: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:48 smithi173 ceph-mon[74165]: pgmap v105: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:48.889 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:48 smithi117 systemd[1]: Stopping Ceph osd.2 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:16:49.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:49.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:49 smithi117 ceph-mon[99036]: osd.2 marked itself down and dead 2024-04-20T11:16:49.390 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:49 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[49490]: 2024-04-20T11:16:49.051+0000 7f8583000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:16:49.390 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:49 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[49490]: 2024-04-20T11:16:49.051+0000 7f8583000700 -1 osd.2 60 *** Got signal Terminated *** 2024-04-20T11:16:49.390 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:49 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[49490]: 2024-04-20T11:16:49.051+0000 7f8583000700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:16:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:49 smithi173 ceph-mon[74165]: osd.2 marked itself down and dead 2024-04-20T11:16:49.890 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:49 smithi117 podman[121193]: 2024-04-20 11:16:49.567257045 +0000 UTC m=+0.684211360 container died 06f9de7eede3bb7bcefb214e56d9f3447cc08f01a9c71eba22a9ab1e9047bf80 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=reef-646b8c7, GIT_CLEAN=True) 2024-04-20T11:16:50.153 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:50 smithi117 podman[121193]: 2024-04-20 11:16:50.136805806 +0000 UTC m=+1.253760134 container cleanup 06f9de7eede3bb7bcefb214e56d9f3447cc08f01a9c71eba22a9ab1e9047bf80 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, RELEASE=reef-646b8c7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-20T11:16:50.154 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:50 smithi117 bash[121193]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2 2024-04-20T11:16:50.154 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:50 smithi117 ceph-mon[99036]: pgmap v106: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:50.154 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:50 smithi117 ceph-mon[99036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:16:50.154 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:50 smithi117 ceph-mon[99036]: osdmap e61: 6 total, 5 up, 6 in 2024-04-20T11:16:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:50 smithi173 ceph-mon[74165]: pgmap v106: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:50 smithi173 ceph-mon[74165]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:16:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:50 smithi173 ceph-mon[74165]: osdmap e61: 6 total, 5 up, 6 in 2024-04-20T11:16:50.639 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:50 smithi117 podman[121212]: 2024-04-20 11:16:50.371649609 +0000 UTC m=+0.789386016 container remove 06f9de7eede3bb7bcefb214e56d9f3447cc08f01a9c71eba22a9ab1e9047bf80 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=reef-646b8c7, org.label-schema.vendor=CentOS) 2024-04-20T11:16:51.366 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:51 smithi117 ceph-mon[99036]: osdmap e62: 6 total, 5 up, 6 in 2024-04-20T11:16:51.367 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:51 smithi117 podman[121296]: 2024-04-20 11:16:51.055225583 +0000 UTC m=+0.036202731 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:16:51.367 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:51 smithi117 podman[121296]: 2024-04-20 11:16:51.180839463 +0000 UTC m=+0.161816668 container create eb373ece8c1f1fe0c6fc1cf0c5ba74b344ef6076e27a684f064b62a1c8ee3fdb (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-20T11:16:51.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:51 smithi173 ceph-mon[74165]: osdmap e62: 6 total, 5 up, 6 in 2024-04-20T11:16:51.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:51 smithi117 podman[121296]: 2024-04-20 11:16:51.484642426 +0000 UTC m=+0.465619624 container init eb373ece8c1f1fe0c6fc1cf0c5ba74b344ef6076e27a684f064b62a1c8ee3fdb (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-20T11:16:51.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:51 smithi117 podman[121296]: 2024-04-20 11:16:51.496278029 +0000 UTC m=+0.477255220 container start eb373ece8c1f1fe0c6fc1cf0c5ba74b344ef6076e27a684f064b62a1c8ee3fdb (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T11:16:52.129 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:51 smithi117 podman[121296]: 2024-04-20 11:16:51.767128754 +0000 UTC m=+0.748105921 container attach eb373ece8c1f1fe0c6fc1cf0c5ba74b344ef6076e27a684f064b62a1c8ee3fdb (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-20T11:16:52.129 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:51 smithi117 podman[121296]: 2024-04-20 11:16:51.964555927 +0000 UTC m=+0.945533112 container died eb373ece8c1f1fe0c6fc1cf0c5ba74b344ef6076e27a684f064b62a1c8ee3fdb (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:16:52.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:52 smithi117 ceph-mon[99036]: pgmap v109: 65 pgs: 9 stale+active+clean, 56 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:52 smithi173 ceph-mon[74165]: pgmap v109: 65 pgs: 9 stale+active+clean, 56 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:16:53.141 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:52 smithi117 podman[121321]: 2024-04-20 11:16:52.75379068 +0000 UTC m=+0.769672053 container remove eb373ece8c1f1fe0c6fc1cf0c5ba74b344ef6076e27a684f064b62a1c8ee3fdb (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-20T11:16:53.141 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:52 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-20T11:16:53.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:52 smithi117 systemd[1]: Stopped Ceph osd.2 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:16:53.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:52 smithi117 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.2.service: Consumed 1min 26.376s CPU time. 2024-04-20T11:16:53.538 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:53 smithi117 systemd[1]: Starting Ceph osd.2 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:16:53.539 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:53 smithi117 ceph-mon[99036]: pgmap v110: 65 pgs: 7 active+undersized+degraded, 2 stale+active+clean, 8 active+undersized, 48 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 27/324 objects degraded (8.333%) 2024-04-20T11:16:53.539 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:53 smithi117 ceph-mon[99036]: Health check failed: Degraded data redundancy: 27/324 objects degraded (8.333%), 7 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:53 smithi173 ceph-mon[74165]: pgmap v110: 65 pgs: 7 active+undersized+degraded, 2 stale+active+clean, 8 active+undersized, 48 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 27/324 objects degraded (8.333%) 2024-04-20T11:16:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:53 smithi173 ceph-mon[74165]: Health check failed: Degraded data redundancy: 27/324 objects degraded (8.333%), 7 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:53.790 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:53 smithi117 podman[121425]: 2024-04-20 11:16:53.441363872 +0000 UTC m=+0.038584508 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:16:53.790 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:53 smithi117 podman[121425]: 2024-04-20 11:16:53.613289605 +0000 UTC m=+0.210510315 container create 0b3e142e6ec830eb7333f2f61b846a7386523b12cac7f20ba7f29cc7ed0f6f08 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-20T11:16:54.140 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:53 smithi117 podman[121425]: 2024-04-20 11:16:53.917541683 +0000 UTC m=+0.514762356 container init 0b3e142e6ec830eb7333f2f61b846a7386523b12cac7f20ba7f29cc7ed0f6f08 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:16:54.140 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:53 smithi117 podman[121425]: 2024-04-20 11:16:53.929767115 +0000 UTC m=+0.526987812 container start 0b3e142e6ec830eb7333f2f61b846a7386523b12cac7f20ba7f29cc7ed0f6f08 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:16:54.140 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 podman[121425]: 2024-04-20 11:16:54.032795879 +0000 UTC m=+0.630016616 container attach 0b3e142e6ec830eb7333f2f61b846a7386523b12cac7f20ba7f29cc7ed0f6f08 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-20T11:16:55.141 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:16:55.141 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:16:55.141 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 bash[121425]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-20T11:16:55.142 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-20T11:16:55.143 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-20T11:16:55.143 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-20T11:16:55.143 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T11:16:55.143 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate[121440]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-20T11:16:55.143 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:54 smithi117 podman[121425]: 2024-04-20 11:16:54.943064276 +0000 UTC m=+1.540284949 container died 0b3e142e6ec830eb7333f2f61b846a7386523b12cac7f20ba7f29cc7ed0f6f08 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T11:16:56.044 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:55 smithi117 podman[121580]: 2024-04-20 11:16:55.719905364 +0000 UTC m=+0.757999471 container remove 0b3e142e6ec830eb7333f2f61b846a7386523b12cac7f20ba7f29cc7ed0f6f08 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-20T11:16:56.376 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:56 smithi117 ceph-mon[99036]: pgmap v111: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:16:56.376 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:56 smithi117 podman[121644]: 2024-04-20 11:16:56.070714059 +0000 UTC m=+0.121958992 container create 3b78740c3941bdcaeeac8d3981c8a032dac36561d90fb443c6f4716f7cdad656 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-20T11:16:56.376 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:56 smithi117 podman[121644]: 2024-04-20 11:16:55.981575892 +0000 UTC m=+0.032820801 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:16:56.376 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:56 smithi117 podman[121644]: 2024-04-20 11:16:56.376012488 +0000 UTC m=+0.427257457 container init 3b78740c3941bdcaeeac8d3981c8a032dac36561d90fb443c6f4716f7cdad656 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-20T11:16:56.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:56 smithi173 ceph-mon[74165]: pgmap v111: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:16:56.639 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:56 smithi117 podman[121644]: 2024-04-20 11:16:56.387164317 +0000 UTC m=+0.438409229 container start 3b78740c3941bdcaeeac8d3981c8a032dac36561d90fb443c6f4716f7cdad656 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-20T11:16:56.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:56 smithi117 ceph-osd[121662]: -- 172.21.15.117:0/3073403903 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56442321f680 con 0x5644231e9000 2024-04-20T11:16:56.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:56 smithi117 bash[121644]: 3b78740c3941bdcaeeac8d3981c8a032dac36561d90fb443c6f4716f7cdad656 2024-04-20T11:16:56.640 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:56 smithi117 systemd[1]: Started Ceph osd.2 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:16:57.389 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:16:56 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[121658]: 2024-04-20T11:16:56.955+0000 7fb026a487c0 -1 Falling back to public interface 2024-04-20T11:16:57.840 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:57.840 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:57.840 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:57.841 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:57 smithi173 ceph-mon[74165]: pgmap v112: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:16:57.841 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:57.841 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:57 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:57 smithi117 ceph-mon[99036]: pgmap v112: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:16:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:57.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:57 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:59.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:59.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:59.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:16:59 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 43/324 objects degraded (13.272%), 13 pgs degraded (PG_DEGRADED) 2024-04-20T11:16:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:16:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:16:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:16:59 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 43/324 objects degraded (13.272%), 13 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:00.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:00 smithi117 ceph-mon[99036]: pgmap v113: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:17:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:00 smithi173 ceph-mon[74165]: pgmap v113: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:17:02.054 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:17:01 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[121658]: 2024-04-20T11:17:01.872+0000 7fb026a487c0 -1 osd.2 60 log_to_monitors true 2024-04-20T11:17:02.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:02 smithi117 ceph-mon[99036]: pgmap v114: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:17:02.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:02 smithi117 ceph-mon[99036]: from='osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:17:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:02 smithi173 ceph-mon[74165]: pgmap v114: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:17:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:02 smithi173 ceph-mon[74165]: from='osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:17:03.390 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:17:03 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[121658]: 2024-04-20T11:17:03.160+0000 7fb010e00700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:17:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:03 smithi117 ceph-mon[99036]: from='osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T11:17:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:03 smithi117 ceph-mon[99036]: osdmap e63: 6 total, 5 up, 6 in 2024-04-20T11:17:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:03 smithi117 ceph-mon[99036]: from='osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:17:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:03 smithi173 ceph-mon[74165]: from='osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T11:17:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:03 smithi173 ceph-mon[74165]: osdmap e63: 6 total, 5 up, 6 in 2024-04-20T11:17:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:03 smithi173 ceph-mon[74165]: from='osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-20T11:17:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:04 smithi117 ceph-mon[99036]: pgmap v116: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:17:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:04 smithi117 ceph-mon[99036]: from='osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786]' entity='osd.2' 2024-04-20T11:17:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:04 smithi173 ceph-mon[74165]: pgmap v116: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:17:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:04 smithi173 ceph-mon[74165]: from='osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786]' entity='osd.2' 2024-04-20T11:17:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:05 smithi117 ceph-mon[99036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:17:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:05 smithi117 ceph-mon[99036]: osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786] boot 2024-04-20T11:17:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:05 smithi117 ceph-mon[99036]: osdmap e64: 6 total, 6 up, 6 in 2024-04-20T11:17:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:17:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:05 smithi117 ceph-mon[99036]: pgmap v118: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:17:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:05 smithi117 ceph-mon[99036]: osdmap e65: 6 total, 6 up, 6 in 2024-04-20T11:17:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:05 smithi173 ceph-mon[74165]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:17:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:05 smithi173 ceph-mon[74165]: osd.2 [v2:172.21.15.117:6818/155800786,v1:172.21.15.117:6819/155800786] boot 2024-04-20T11:17:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:05 smithi173 ceph-mon[74165]: osdmap e64: 6 total, 6 up, 6 in 2024-04-20T11:17:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:17:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:05 smithi173 ceph-mon[74165]: pgmap v118: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-20T11:17:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:05 smithi173 ceph-mon[74165]: osdmap e65: 6 total, 6 up, 6 in 2024-04-20T11:17:07.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:07.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:07.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:07 smithi117 ceph-mon[99036]: pgmap v120: 65 pgs: 11 active+undersized+degraded, 11 active+undersized, 43 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 37/324 objects degraded (11.420%) 2024-04-20T11:17:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:07 smithi173 ceph-mon[74165]: pgmap v120: 65 pgs: 11 active+undersized+degraded, 11 active+undersized, 43 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 37/324 objects degraded (11.420%) 2024-04-20T11:17:08.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:08 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 37/324 objects degraded (11.420%), 11 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:08 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 37/324 objects degraded (11.420%), 11 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:10.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:09 smithi117 ceph-mon[99036]: pgmap v121: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:09 smithi173 ceph-mon[74165]: pgmap v121: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:11.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:10 smithi117 ceph-mon[99036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 37/324 objects degraded (11.420%), 11 pgs degraded) 2024-04-20T11:17:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:10 smithi173 ceph-mon[74165]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 37/324 objects degraded (11.420%), 11 pgs degraded) 2024-04-20T11:17:11.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:11 smithi117 ceph-mon[99036]: pgmap v122: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:11 smithi173 ceph-mon[74165]: pgmap v122: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: Upgrade: osd.3 is safe to restart 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: pgmap v123: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:13.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:17:13.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:13.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:13.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:13.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:13.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:13.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:13.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:13.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:13.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:13.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-20T11:17:13.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-20T11:17:13.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: Upgrade: osd.3 is safe to restart 2024-04-20T11:17:13.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: pgmap v123: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:13.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:13.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:17:16.298 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:16 smithi173 ceph-mon[74165]: pgmap v124: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:16.298 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:16 smithi173 ceph-mon[74165]: Upgrade: Updating osd.3 2024-04-20T11:17:16.298 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:16.298 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:17:16.298 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:16.298 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:16 smithi173 ceph-mon[74165]: Deploying daemon osd.3 on smithi173 2024-04-20T11:17:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:16 smithi117 ceph-mon[99036]: pgmap v124: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:16 smithi117 ceph-mon[99036]: Upgrade: Updating osd.3 2024-04-20T11:17:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:17:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:16 smithi117 ceph-mon[99036]: Deploying daemon osd.3 on smithi173 2024-04-20T11:17:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.660+0000 7f40a9400700 1 -- 172.21.15.117:0/3011547855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4104a00 msgr2=0x7f40a4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.660+0000 7f40a9400700 1 --2- 172.21.15.117:0/3011547855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4104a00 0x7f40a4104e80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4090009910 tx=0x7f4090009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.661+0000 7f40a9400700 1 -- 172.21.15.117:0/3011547855 shutdown_connections 2024-04-20T11:17:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.661+0000 7f40a9400700 1 --2- 172.21.15.117:0/3011547855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4104a00 0x7f40a4104e80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.661+0000 7f40a9400700 1 --2- 172.21.15.117:0/3011547855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a4103110 0x7f40a41034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.661+0000 7f40a9400700 1 -- 172.21.15.117:0/3011547855 >> 172.21.15.117:0/3011547855 conn(0x7f40a40fe880 msgr2=0x7f40a4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.661+0000 7f40a9400700 1 -- 172.21.15.117:0/3011547855 shutdown_connections 2024-04-20T11:17:16.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.661+0000 7f40a9400700 1 -- 172.21.15.117:0/3011547855 wait complete. 2024-04-20T11:17:16.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.662+0000 7f40a9400700 1 Processor -- start 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.662+0000 7f40a9400700 1 -- start start 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4103110 0x7f40a419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a9400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a4104a00 0x7f40a419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a9400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a419f5b0 con 0x7f40a4103110 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a9400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4198b90 con 0x7f40a4104a00 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a2800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4103110 0x7f40a419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a2800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4103110 0x7f40a419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52842/0 (socket says 172.21.15.117:52842) 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a2800700 1 -- 172.21.15.117:0/954754607 learned_addr learned my addr 172.21.15.117:0/954754607 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a1e00700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a4104a00 0x7f40a419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a2800700 1 -- 172.21.15.117:0/954754607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a4104a00 msgr2=0x7f40a419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a2800700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a4104a00 0x7f40a419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a2800700 1 -- 172.21.15.117:0/954754607 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900095f0 con 0x7f40a4103110 2024-04-20T11:17:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.663+0000 7f40a1e00700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a4104a00 0x7f40a419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:17:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.664+0000 7f40a2800700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4103110 0x7f40a419ea50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f409400e920 tx=0x7f409400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.664+0000 7f409b400700 1 -- 172.21.15.117:0/954754607 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4094009790 con 0x7f40a4103110 2024-04-20T11:17:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.664+0000 7f409b400700 1 -- 172.21.15.117:0/954754607 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f409400cba0 con 0x7f40a4103110 2024-04-20T11:17:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.664+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a4198e70 con 0x7f40a4103110 2024-04-20T11:17:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.664+0000 7f409b400700 1 -- 172.21.15.117:0/954754607 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4094018650 con 0x7f40a4103110 2024-04-20T11:17:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.664+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a4199440 con 0x7f40a4103110 2024-04-20T11:17:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.665+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a410b480 con 0x7f40a4103110 2024-04-20T11:17:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.666+0000 7f409b400700 1 -- 172.21.15.117:0/954754607 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f409400cd10 con 0x7f40a4103110 2024-04-20T11:17:16.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.668+0000 7f409b400700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ecd0 0x7f4088071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:16.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.668+0000 7f409b400700 1 -- 172.21.15.117:0/954754607 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f4094014070 con 0x7f40a4103110 2024-04-20T11:17:16.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.668+0000 7f40a1e00700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ecd0 0x7f4088071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:16.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.669+0000 7f40a1e00700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ecd0 0x7f4088071190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f409000b760 tx=0x7f4090009f70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:16.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.672+0000 7f409b400700 1 -- 172.21.15.117:0/954754607 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4094053ff0 con 0x7f40a4103110 2024-04-20T11:17:16.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.923+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40a419a050 con 0x7f408806ecd0 2024-04-20T11:17:16.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.925+0000 7f409b400700 1 -- 172.21.15.117:0/954754607 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f40a419a050 con 0x7f408806ecd0 2024-04-20T11:17:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ecd0 msgr2=0x7f4088071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ecd0 0x7f4088071190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f409000b760 tx=0x7f4090009f70 comp rx=0 tx=0).stop 2024-04-20T11:17:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4103110 msgr2=0x7f40a419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4103110 0x7f40a419ea50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f409400e920 tx=0x7f409400ece0 comp rx=0 tx=0).stop 2024-04-20T11:17:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 shutdown_connections 2024-04-20T11:17:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ecd0 0x7f4088071190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4103110 0x7f40a419ea50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 --2- 172.21.15.117:0/954754607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a4104a00 0x7f40a419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 >> 172.21.15.117:0/954754607 conn(0x7f40a40fe880 msgr2=0x7f40a40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 shutdown_connections 2024-04-20T11:17:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:16.929+0000 7f40a9400700 1 -- 172.21.15.117:0/954754607 wait complete. 2024-04-20T11:17:16.945 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:17:17.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.093+0000 7f4637c00700 1 -- 172.21.15.117:0/602850794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 msgr2=0x7f4630073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.093+0000 7f4637c00700 1 --2- 172.21.15.117:0/602850794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 0x7f4630073690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4624009910 tx=0x7f4624009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:17.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.093+0000 7f4637c00700 1 -- 172.21.15.117:0/602850794 shutdown_connections 2024-04-20T11:17:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.093+0000 7f4637c00700 1 --2- 172.21.15.117:0/602850794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4630073bd0 0x7f463010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.093+0000 7f4637c00700 1 --2- 172.21.15.117:0/602850794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 0x7f4630073690 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.093+0000 7f4637c00700 1 -- 172.21.15.117:0/602850794 >> 172.21.15.117:0/602850794 conn(0x7f46300fc6d0 msgr2=0x7f46300feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.093+0000 7f4637c00700 1 -- 172.21.15.117:0/602850794 shutdown_connections 2024-04-20T11:17:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.093+0000 7f4637c00700 1 -- 172.21.15.117:0/602850794 wait complete. 2024-04-20T11:17:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.094+0000 7f4637c00700 1 Processor -- start 2024-04-20T11:17:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f4637c00700 1 -- start start 2024-04-20T11:17:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f4637c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 0x7f4630198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f4637c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4630073bd0 0x7f4630199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f4637c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630199a00 con 0x7f46300732b0 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f4637c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463019d6d0 con 0x7f4630073bd0 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f4635200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 0x7f4630198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f4635200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 0x7f4630198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52866/0 (socket says 172.21.15.117:52866) 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f4635200700 1 -- 172.21.15.117:0/1937609373 learned_addr learned my addr 172.21.15.117:0/1937609373 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.095+0000 7f462fe00700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4630073bd0 0x7f4630199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.096+0000 7f4635200700 1 -- 172.21.15.117:0/1937609373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4630073bd0 msgr2=0x7f4630199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.096+0000 7f4635200700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4630073bd0 0x7f4630199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.096+0000 7f4635200700 1 -- 172.21.15.117:0/1937609373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240095f0 con 0x7f46300732b0 2024-04-20T11:17:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.096+0000 7f4635200700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 0x7f4630198de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46240092e0 tx=0x7f462400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:17.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.096+0000 7f462d600700 1 -- 172.21.15.117:0/1937609373 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462401d070 con 0x7f46300732b0 2024-04-20T11:17:17.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.096+0000 7f462d600700 1 -- 172.21.15.117:0/1937609373 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f462400f460 con 0x7f46300732b0 2024-04-20T11:17:17.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.096+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463019d950 con 0x7f46300732b0 2024-04-20T11:17:17.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.096+0000 7f462d600700 1 -- 172.21.15.117:0/1937609373 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4624017630 con 0x7f46300732b0 2024-04-20T11:17:17.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.097+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463019de40 con 0x7f46300732b0 2024-04-20T11:17:17.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.098+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4630074960 con 0x7f46300732b0 2024-04-20T11:17:17.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.099+0000 7f462d600700 1 -- 172.21.15.117:0/1937609373 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f462400f5d0 con 0x7f46300732b0 2024-04-20T11:17:17.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.099+0000 7f462d600700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f461806ecd0 0x7f4618071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.100+0000 7f462d600700 1 -- 172.21.15.117:0/1937609373 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f4624090330 con 0x7f46300732b0 2024-04-20T11:17:17.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.100+0000 7f462fe00700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f461806ecd0 0x7f4618071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.101+0000 7f462fe00700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f461806ecd0 0x7f4618071190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f461c006fd0 tx=0x7f461c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.104+0000 7f462d600700 1 -- 172.21.15.117:0/1937609373 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4624059120 con 0x7f46300732b0 2024-04-20T11:17:17.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.352+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4630107850 con 0x7f461806ecd0 2024-04-20T11:17:17.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.357+0000 7f462d600700 1 -- 172.21.15.117:0/1937609373 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4630107850 con 0x7f461806ecd0 2024-04-20T11:17:17.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f461806ecd0 msgr2=0x7f4618071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f461806ecd0 0x7f4618071190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f461c006fd0 tx=0x7f461c008040 comp rx=0 tx=0).stop 2024-04-20T11:17:17.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 msgr2=0x7f4630198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 0x7f4630198de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46240092e0 tx=0x7f462400be20 comp rx=0 tx=0).stop 2024-04-20T11:17:17.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 shutdown_connections 2024-04-20T11:17:17.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f461806ecd0 0x7f4618071190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300732b0 0x7f4630198de0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 --2- 172.21.15.117:0/1937609373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4630073bd0 0x7f4630199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.360+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 >> 172.21.15.117:0/1937609373 conn(0x7f46300fc6d0 msgr2=0x7f4630105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:17.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.361+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 shutdown_connections 2024-04-20T11:17:17.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.361+0000 7f4637c00700 1 -- 172.21.15.117:0/1937609373 wait complete. 2024-04-20T11:17:17.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.521+0000 7fd566e00700 1 -- 172.21.15.117:0/1460371659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 msgr2=0x7fd5600689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.521+0000 7fd566e00700 1 --2- 172.21.15.117:0/1460371659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 0x7fd5600689d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009910 tx=0x7fd54c009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:17.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.521+0000 7fd566e00700 1 -- 172.21.15.117:0/1460371659 shutdown_connections 2024-04-20T11:17:17.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.521+0000 7fd566e00700 1 --2- 172.21.15.117:0/1460371659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 0x7fd5600689d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.521+0000 7fd566e00700 1 --2- 172.21.15.117:0/1460371659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5601069d0 0x7fd560106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.521+0000 7fd566e00700 1 -- 172.21.15.117:0/1460371659 >> 172.21.15.117:0/1460371659 conn(0x7fd560075480 msgr2=0x7fd560075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:17.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.521+0000 7fd566e00700 1 -- 172.21.15.117:0/1460371659 shutdown_connections 2024-04-20T11:17:17.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.522+0000 7fd566e00700 1 -- 172.21.15.117:0/1460371659 wait complete. 2024-04-20T11:17:17.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.522+0000 7fd566e00700 1 Processor -- start 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd566e00700 1 -- start start 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd566e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 0x7fd56019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd566e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5601069d0 0x7fd56019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd566e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019f390 con 0x7fd560068570 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd566e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5601989c0 con 0x7fd5601069d0 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd565a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 0x7fd56019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd565a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 0x7fd56019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52882/0 (socket says 172.21.15.117:52882) 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd565a00700 1 -- 172.21.15.117:0/1409552929 learned_addr learned my addr 172.21.15.117:0/1409552929 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.523+0000 7fd565000700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5601069d0 0x7fd56019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.524+0000 7fd565a00700 1 -- 172.21.15.117:0/1409552929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5601069d0 msgr2=0x7fd56019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.524+0000 7fd565a00700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5601069d0 0x7fd56019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.524+0000 7fd565a00700 1 -- 172.21.15.117:0/1409552929 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c0095f0 con 0x7fd560068570 2024-04-20T11:17:17.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.524+0000 7fd565a00700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 0x7fd56019e830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd55400e920 tx=0x7fd55400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:17.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.524+0000 7fd55e000700 1 -- 172.21.15.117:0/1409552929 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd55400c9d0 con 0x7fd560068570 2024-04-20T11:17:17.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.525+0000 7fd55e000700 1 -- 172.21.15.117:0/1409552929 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd554010430 con 0x7fd560068570 2024-04-20T11:17:17.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.525+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd560198ca0 con 0x7fd560068570 2024-04-20T11:17:17.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.525+0000 7fd55e000700 1 -- 172.21.15.117:0/1409552929 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd554018630 con 0x7fd560068570 2024-04-20T11:17:17.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.525+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5601991f0 con 0x7fd560068570 2024-04-20T11:17:17.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.526+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56010af70 con 0x7fd560068570 2024-04-20T11:17:17.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.527+0000 7fd55e000700 1 -- 172.21.15.117:0/1409552929 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd5540105a0 con 0x7fd560068570 2024-04-20T11:17:17.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.528+0000 7fd55e000700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd54806ecc0 0x7fd548071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.528+0000 7fd55e000700 1 -- 172.21.15.117:0/1409552929 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fd554014070 con 0x7fd560068570 2024-04-20T11:17:17.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.529+0000 7fd565000700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd54806ecc0 0x7fd548071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.529+0000 7fd565000700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd54806ecc0 0x7fd548071180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd560199b30 tx=0x7fd54c005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:17.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.532+0000 7fd55e000700 1 -- 172.21.15.117:0/1409552929 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd55405bf00 con 0x7fd560068570 2024-04-20T11:17:17.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.768+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd56010dbb0 con 0x7fd54806ecc0 2024-04-20T11:17:17.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.799+0000 7fd55e000700 1 -- 172.21.15.117:0/1409552929 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd56010dbb0 con 0x7fd54806ecc0 2024-04-20T11:17:17.799 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:17:17.799 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (5m) 13s ago 19m 14.2M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:17:17.799 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (20m) 13s ago 20m 9017k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (18m) 110s ago 18m 8270k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (2m) 13s ago 20m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (115s) 110s ago 18m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (5m) 13s ago 19m 74.3M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (12m) 13s ago 12m 18.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (12m) 13s ago 12m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (12m) 110s ago 12m 17.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (12m) 110s ago 12m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (8m) 13s ago 21m 506M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (7m) 110s ago 18m 450M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:17:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (3m) 13s ago 21m 51.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (3m) 110s ago 18m 33.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 13s ago 19m 7872k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (6m) 110s ago 18m 7377k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (84s) 13s ago 17m 80.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (52s) 13s ago 16m 82.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (21s) 13s ago 16m 11.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (15m) 110s ago 15m 244M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ff75c097f92 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (15m) 110s ago 15m 215M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:17:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (14m) 110s ago 14m 202M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:17:17.802 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (6m) 13s ago 19m 69.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:17:17.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd54806ecc0 msgr2=0x7fd548071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd54806ecc0 0x7fd548071180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd560199b30 tx=0x7fd54c005a90 comp rx=0 tx=0).stop 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 msgr2=0x7fd56019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 0x7fd56019e830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd55400e920 tx=0x7fd55400ec30 comp rx=0 tx=0).stop 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 shutdown_connections 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd54806ecc0 0x7fd548071180 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd560068570 0x7fd56019e830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 --2- 172.21.15.117:0/1409552929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5601069d0 0x7fd56019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.802+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 >> 172.21.15.117:0/1409552929 conn(0x7fd560075480 msgr2=0x7fd5601044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.803+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 shutdown_connections 2024-04-20T11:17:17.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.803+0000 7fd566e00700 1 -- 172.21.15.117:0/1409552929 wait complete. 2024-04-20T11:17:17.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.948+0000 7fb90ea00700 1 -- 172.21.15.117:0/726243001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb908100e20 msgr2=0x7fb908101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.948+0000 7fb90ea00700 1 --2- 172.21.15.117:0/726243001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb908100e20 0x7fb908101200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009910 tx=0x7fb8f8009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:17.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.948+0000 7fb90ea00700 1 -- 172.21.15.117:0/726243001 shutdown_connections 2024-04-20T11:17:17.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.948+0000 7fb90ea00700 1 --2- 172.21.15.117:0/726243001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9081017d0 0x7fb908105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.948+0000 7fb90ea00700 1 --2- 172.21.15.117:0/726243001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb908100e20 0x7fb908101200 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.948+0000 7fb90ea00700 1 -- 172.21.15.117:0/726243001 >> 172.21.15.117:0/726243001 conn(0x7fb9080fc6f0 msgr2=0x7fb9080feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:17.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.948+0000 7fb90ea00700 1 -- 172.21.15.117:0/726243001 shutdown_connections 2024-04-20T11:17:17.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.948+0000 7fb90ea00700 1 -- 172.21.15.117:0/726243001 wait complete. 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.949+0000 7fb90ea00700 1 Processor -- start 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.949+0000 7fb90ea00700 1 -- start start 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.950+0000 7fb90ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb908100e20 0x7fb9080731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.950+0000 7fb90ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9081017d0 0x7fb908073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.950+0000 7fb90ea00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908073c60 con 0x7fb9081017d0 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.950+0000 7fb90ea00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908073da0 con 0x7fb908100e20 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.950+0000 7fb907400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9081017d0 0x7fb908073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.950+0000 7fb907400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9081017d0 0x7fb908073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52890/0 (socket says 172.21.15.117:52890) 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.950+0000 7fb907400700 1 -- 172.21.15.117:0/2158265896 learned_addr learned my addr 172.21.15.117:0/2158265896 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.950+0000 7fb907e00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb908100e20 0x7fb9080731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb907400700 1 -- 172.21.15.117:0/2158265896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb908100e20 msgr2=0x7fb9080731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb907400700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb908100e20 0x7fb9080731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb907400700 1 -- 172.21.15.117:0/2158265896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80095f0 con 0x7fb9081017d0 2024-04-20T11:17:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb907400700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9081017d0 0x7fb908073720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400c8f0 tx=0x7fb8f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb904c00700 1 -- 172.21.15.117:0/2158265896 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8f4007a50 con 0x7fb9081017d0 2024-04-20T11:17:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb904c00700 1 -- 172.21.15.117:0/2158265896 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8f4007bb0 con 0x7fb9081017d0 2024-04-20T11:17:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9081a2d50 con 0x7fb9081017d0 2024-04-20T11:17:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb904c00700 1 -- 172.21.15.117:0/2158265896 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8f4018630 con 0x7fb9081017d0 2024-04-20T11:17:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.951+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9081a30b0 con 0x7fb9081017d0 2024-04-20T11:17:17.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.952+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9081091c0 con 0x7fb9081017d0 2024-04-20T11:17:17.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.954+0000 7fb904c00700 1 -- 172.21.15.117:0/2158265896 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb8f4007d20 con 0x7fb9081017d0 2024-04-20T11:17:17.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.955+0000 7fb904c00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb8ec06ecc0 0x7fb8ec071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:17.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.955+0000 7fb904c00700 1 -- 172.21.15.117:0/2158265896 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fb8f408ed20 con 0x7fb9081017d0 2024-04-20T11:17:17.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.955+0000 7fb907e00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb8ec06ecc0 0x7fb8ec071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:17.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.956+0000 7fb907e00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb8ec06ecc0 0x7fb8ec071180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b5c0 tx=0x7fb8f801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:17.960+0000 7fb904c00700 1 -- 172.21.15.117:0/2158265896 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb8f4057b90 con 0x7fb9081017d0 2024-04-20T11:17:18.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.258+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb908073f60 con 0x7fb9081017d0 2024-04-20T11:17:18.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.259+0000 7fb904c00700 1 -- 172.21.15.117:0/2158265896 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fb8f405b250 con 0x7fb9081017d0 2024-04-20T11:17:18.258 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3, 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:17:18.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:17:18.260 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:17:18.260 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 7, 2024-04-20T11:17:18.260 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 7 2024-04-20T11:17:18.260 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:17:18.260 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:17:18.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb8ec06ecc0 msgr2=0x7fb8ec071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:18.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb8ec06ecc0 0x7fb8ec071180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b5c0 tx=0x7fb8f801a040 comp rx=0 tx=0).stop 2024-04-20T11:17:18.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9081017d0 msgr2=0x7fb908073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:18.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9081017d0 0x7fb908073720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400c8f0 tx=0x7fb8f400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 shutdown_connections 2024-04-20T11:17:18.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb8ec06ecc0 0x7fb8ec071180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb908100e20 0x7fb9080731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 --2- 172.21.15.117:0/2158265896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9081017d0 0x7fb908073720 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.261+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 >> 172.21.15.117:0/2158265896 conn(0x7fb9080fc6f0 msgr2=0x7fb9080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:18.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.262+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 shutdown_connections 2024-04-20T11:17:18.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.262+0000 7fb90ea00700 1 -- 172.21.15.117:0/2158265896 wait complete. 2024-04-20T11:17:18.314 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:18 smithi173 ceph-mon[74165]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:18.315 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:18 smithi173 ceph-mon[74165]: pgmap v125: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:18 smithi117 ceph-mon[99036]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:18 smithi117 ceph-mon[99036]: pgmap v125: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:18.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.420+0000 7fb604000700 1 -- 172.21.15.117:0/4150227081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc100e10 msgr2=0x7fb5fc1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:18.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.420+0000 7fb604000700 1 --2- 172.21.15.117:0/4150227081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc100e10 0x7fb5fc1011f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009910 tx=0x7fb5f0009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:18.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.420+0000 7fb604000700 1 -- 172.21.15.117:0/4150227081 shutdown_connections 2024-04-20T11:17:18.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.420+0000 7fb604000700 1 --2- 172.21.15.117:0/4150227081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc1017c0 0x7fb5fc105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.420+0000 7fb604000700 1 --2- 172.21.15.117:0/4150227081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc100e10 0x7fb5fc1011f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.420+0000 7fb604000700 1 -- 172.21.15.117:0/4150227081 >> 172.21.15.117:0/4150227081 conn(0x7fb5fc0fc6c0 msgr2=0x7fb5fc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:18.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.420+0000 7fb604000700 1 -- 172.21.15.117:0/4150227081 shutdown_connections 2024-04-20T11:17:18.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.420+0000 7fb604000700 1 -- 172.21.15.117:0/4150227081 wait complete. 2024-04-20T11:17:18.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.421+0000 7fb604000700 1 Processor -- start 2024-04-20T11:17:18.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.421+0000 7fb604000700 1 -- start start 2024-04-20T11:17:18.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.422+0000 7fb604000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc100e10 0x7fb5fc0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:18.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.422+0000 7fb604000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc1017c0 0x7fb5fc073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.422+0000 7fb604000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0751b0 con 0x7fb5fc1017c0 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.422+0000 7fb604000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc073c60 con 0x7fb5fc100e10 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.422+0000 7fb600c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc1017c0 0x7fb5fc073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.422+0000 7fb600c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc1017c0 0x7fb5fc073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52906/0 (socket says 172.21.15.117:52906) 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.422+0000 7fb600c00700 1 -- 172.21.15.117:0/3600717379 learned_addr learned my addr 172.21.15.117:0/3600717379 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.422+0000 7fb601600700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc100e10 0x7fb5fc0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb600c00700 1 -- 172.21.15.117:0/3600717379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc100e10 msgr2=0x7fb5fc0731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb600c00700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc100e10 0x7fb5fc0731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb600c00700 1 -- 172.21.15.117:0/3600717379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00095f0 con 0x7fb5fc1017c0 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb601600700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc100e10 0x7fb5fc0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:17:18.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb600c00700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc1017c0 0x7fb5fc073720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c960 tx=0x7fb5e800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:18.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb5fa000700 1 -- 172.21.15.117:0/3600717379 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5e8007a70 con 0x7fb5fc1017c0 2024-04-20T11:17:18.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb5fa000700 1 -- 172.21.15.117:0/3600717379 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5e8007bd0 con 0x7fb5fc1017c0 2024-04-20T11:17:18.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb5fa000700 1 -- 172.21.15.117:0/3600717379 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5e80186c0 con 0x7fb5fc1017c0 2024-04-20T11:17:18.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.423+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc073f10 con 0x7fb5fc1017c0 2024-04-20T11:17:18.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.424+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc1a2f40 con 0x7fb5fc1017c0 2024-04-20T11:17:18.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.425+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc1091b0 con 0x7fb5fc1017c0 2024-04-20T11:17:18.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.427+0000 7fb5fa000700 1 -- 172.21.15.117:0/3600717379 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb5e800faa0 con 0x7fb5fc1017c0 2024-04-20T11:17:18.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.428+0000 7fb5fa000700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5e406ecc0 0x7fb5e4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:18.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.428+0000 7fb5fa000700 1 -- 172.21.15.117:0/3600717379 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fb5e808edc0 con 0x7fb5fc1017c0 2024-04-20T11:17:18.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.428+0000 7fb601600700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5e406ecc0 0x7fb5e4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:18.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.429+0000 7fb601600700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5e406ecc0 0x7fb5e4071180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000b5c0 tx=0x7fb5f001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:18.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.433+0000 7fb5fa000700 1 -- 172.21.15.117:0/3600717379 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb5e8092030 con 0x7fb5fc1017c0 2024-04-20T11:17:18.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.703+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb5fc10dde0 con 0x7fb5fc1017c0 2024-04-20T11:17:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.704+0000 7fb5fa000700 1 -- 172.21.15.117:0/3600717379 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7fb5e805b760 con 0x7fb5fc1017c0 2024-04-20T11:17:18.705 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:17:18.705 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:17:18.705 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:17:18.705 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:17:18.705 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:17:18.705 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:17:18.705 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:17:18.705 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:17:18.706 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:17:18.707 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:17:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:17:18.709 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:17:18.709 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:17:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.708+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5e406ecc0 msgr2=0x7fb5e4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.708+0000 7fb604000700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5e406ecc0 0x7fb5e4071180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000b5c0 tx=0x7fb5f001a040 comp rx=0 tx=0).stop 2024-04-20T11:17:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.708+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc1017c0 msgr2=0x7fb5fc073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.708+0000 7fb604000700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc1017c0 0x7fb5fc073720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c960 tx=0x7fb5e800cd20 comp rx=0 tx=0).stop 2024-04-20T11:17:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.709+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 shutdown_connections 2024-04-20T11:17:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.709+0000 7fb604000700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb5e406ecc0 0x7fb5e4071180 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.709+0000 7fb604000700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc100e10 0x7fb5fc0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.709+0000 7fb604000700 1 --2- 172.21.15.117:0/3600717379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5fc1017c0 0x7fb5fc073720 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.709+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 >> 172.21.15.117:0/3600717379 conn(0x7fb5fc0fc6c0 msgr2=0x7fb5fc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.709+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 shutdown_connections 2024-04-20T11:17:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.709+0000 7fb604000700 1 -- 172.21.15.117:0/3600717379 wait complete. 2024-04-20T11:17:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:17:18.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.861+0000 7ff731400700 1 -- 172.21.15.117:0/4068652442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073310 msgr2=0x7ff72c0736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:18.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.861+0000 7ff731400700 1 --2- 172.21.15.117:0/4068652442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073310 0x7ff72c0736f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009910 tx=0x7ff71c009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:18.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.861+0000 7ff731400700 1 -- 172.21.15.117:0/4068652442 shutdown_connections 2024-04-20T11:17:18.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.861+0000 7ff731400700 1 --2- 172.21.15.117:0/4068652442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff72c073c30 0x7ff72c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.861+0000 7ff731400700 1 --2- 172.21.15.117:0/4068652442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073310 0x7ff72c0736f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.861+0000 7ff731400700 1 -- 172.21.15.117:0/4068652442 >> 172.21.15.117:0/4068652442 conn(0x7ff72c0fc6c0 msgr2=0x7ff72c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:18.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.861+0000 7ff731400700 1 -- 172.21.15.117:0/4068652442 shutdown_connections 2024-04-20T11:17:18.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.861+0000 7ff731400700 1 -- 172.21.15.117:0/4068652442 wait complete. 2024-04-20T11:17:18.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.863+0000 7ff731400700 1 Processor -- start 2024-04-20T11:17:18.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.863+0000 7ff731400700 1 -- start start 2024-04-20T11:17:18.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.863+0000 7ff731400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff72c073310 0x7ff72c198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:18.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.863+0000 7ff731400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073c30 0x7ff72c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:18.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.863+0000 7ff731400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c199a20 con 0x7ff72c073c30 2024-04-20T11:17:18.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.863+0000 7ff731400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c19d6f0 con 0x7ff72c073310 2024-04-20T11:17:18.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff729e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073c30 0x7ff72c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:18.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff729e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073c30 0x7ff72c199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52922/0 (socket says 172.21.15.117:52922) 2024-04-20T11:17:18.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff729e00700 1 -- 172.21.15.117:0/3824594342 learned_addr learned my addr 172.21.15.117:0/3824594342 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:18.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff72a800700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff72c073310 0x7ff72c198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:18.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff729e00700 1 -- 172.21.15.117:0/3824594342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff72c073310 msgr2=0x7ff72c198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:18.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff729e00700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff72c073310 0x7ff72c198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:18.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff729e00700 1 -- 172.21.15.117:0/3824594342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0095f0 con 0x7ff72c073c30 2024-04-20T11:17:18.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff729e00700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073c30 0x7ff72c199340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff71800b4b0 tx=0x7ff71800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:18.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.864+0000 7ff723400700 1 -- 172.21.15.117:0/3824594342 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff718011800 con 0x7ff72c073c30 2024-04-20T11:17:18.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.865+0000 7ff723400700 1 -- 172.21.15.117:0/3824594342 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff718011e20 con 0x7ff72c073c30 2024-04-20T11:17:18.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.865+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c19d9d0 con 0x7ff72c073c30 2024-04-20T11:17:18.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.865+0000 7ff723400700 1 -- 172.21.15.117:0/3824594342 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71800f2e0 con 0x7ff72c073c30 2024-04-20T11:17:18.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.865+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c19df20 con 0x7ff72c073c30 2024-04-20T11:17:18.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.866+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c0749c0 con 0x7ff72c073c30 2024-04-20T11:17:18.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.867+0000 7ff723400700 1 -- 172.21.15.117:0/3824594342 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff718010380 con 0x7ff72c073c30 2024-04-20T11:17:18.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.868+0000 7ff723400700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff71006ecc0 0x7ff710071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:18.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.868+0000 7ff723400700 1 -- 172.21.15.117:0/3824594342 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6285+0+0 (secure 0 0 0) 0x7ff71808e920 con 0x7ff72c073c30 2024-04-20T11:17:18.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.868+0000 7ff72a800700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff71006ecc0 0x7ff710071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:18.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.869+0000 7ff72a800700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff71006ecc0 0x7ff710071180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00b5c0 tx=0x7ff71c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:18.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:18.873+0000 7ff723400700 1 -- 172.21.15.117:0/3824594342 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff7180577b0 con 0x7ff72c073c30 2024-04-20T11:17:19.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.112+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff72c1077a0 con 0x7ff71006ecc0 2024-04-20T11:17:19.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.114+0000 7ff723400700 1 -- 172.21.15.117:0/3824594342 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff72c1077a0 con 0x7ff71006ecc0 2024-04-20T11:17:19.114 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:17:19.114 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:17:19.114 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:17:19.114 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:17:19.114 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:17:19.114 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:17:19.114 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:17:19.115 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:17:19.115 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:17:19.115 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "9/23 daemons upgraded", 2024-04-20T11:17:19.115 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading osd daemons", 2024-04-20T11:17:19.115 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:17:19.115 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:17:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff71006ecc0 msgr2=0x7ff710071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff71006ecc0 0x7ff710071180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00b5c0 tx=0x7ff71c01a040 comp rx=0 tx=0).stop 2024-04-20T11:17:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073c30 msgr2=0x7ff72c199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073c30 0x7ff72c199340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff71800b4b0 tx=0x7ff71800b870 comp rx=0 tx=0).stop 2024-04-20T11:17:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 shutdown_connections 2024-04-20T11:17:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff71006ecc0 0x7ff710071180 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff72c073310 0x7ff72c198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 --2- 172.21.15.117:0/3824594342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c073c30 0x7ff72c199340 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.117+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 >> 172.21.15.117:0/3824594342 conn(0x7ff72c0fc6c0 msgr2=0x7ff72c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.118+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 shutdown_connections 2024-04-20T11:17:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.118+0000 7ff731400700 1 -- 172.21.15.117:0/3824594342 wait complete. 2024-04-20T11:17:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.261+0000 7fa154a00700 1 -- 172.21.15.117:0/607419265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 msgr2=0x7fa150103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.261+0000 7fa154a00700 1 --2- 172.21.15.117:0/607419265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 0x7fa150103200 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa140009910 tx=0x7fa140009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.261+0000 7fa154a00700 1 -- 172.21.15.117:0/607419265 shutdown_connections 2024-04-20T11:17:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.261+0000 7fa154a00700 1 --2- 172.21.15.117:0/607419265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1501037d0 0x7fa150107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.261+0000 7fa154a00700 1 --2- 172.21.15.117:0/607419265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 0x7fa150103200 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.261+0000 7fa154a00700 1 -- 172.21.15.117:0/607419265 >> 172.21.15.117:0/607419265 conn(0x7fa1500fe6d0 msgr2=0x7fa150100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.261+0000 7fa154a00700 1 -- 172.21.15.117:0/607419265 shutdown_connections 2024-04-20T11:17:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.262+0000 7fa154a00700 1 -- 172.21.15.117:0/607419265 wait complete. 2024-04-20T11:17:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.262+0000 7fa154a00700 1 Processor -- start 2024-04-20T11:17:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa154a00700 1 -- start start 2024-04-20T11:17:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa154a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 0x7fa15019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa154a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1501037d0 0x7fa15019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa154a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15019f3e0 con 0x7fa150102e20 2024-04-20T11:17:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa154a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150198a10 con 0x7fa1501037d0 2024-04-20T11:17:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa14e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 0x7fa15019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa14e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 0x7fa15019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52940/0 (socket says 172.21.15.117:52940) 2024-04-20T11:17:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa14e800700 1 -- 172.21.15.117:0/319025909 learned_addr learned my addr 172.21.15.117:0/319025909 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.263+0000 7fa14de00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1501037d0 0x7fa15019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa14e800700 1 -- 172.21.15.117:0/319025909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1501037d0 msgr2=0x7fa15019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa14e800700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1501037d0 0x7fa15019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa14e800700 1 -- 172.21.15.117:0/319025909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400095f0 con 0x7fa150102e20 2024-04-20T11:17:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa14de00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1501037d0 0x7fa15019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:17:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa14e800700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 0x7fa15019e880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1400092e0 tx=0x7fa14000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa147400700 1 -- 172.21.15.117:0/319025909 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa14001d070 con 0x7fa150102e20 2024-04-20T11:17:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa147400700 1 -- 172.21.15.117:0/319025909 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1400056e0 con 0x7fa150102e20 2024-04-20T11:17:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa147400700 1 -- 172.21.15.117:0/319025909 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa14000fe80 con 0x7fa150102e20 2024-04-20T11:17:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa150198c90 con 0x7fa150102e20 2024-04-20T11:17:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.264+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa150199180 con 0x7fa150102e20 2024-04-20T11:17:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.265+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa150104300 con 0x7fa150102e20 2024-04-20T11:17:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.266+0000 7fa147400700 1 -- 172.21.15.117:0/319025909 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa14000f6f0 con 0x7fa150102e20 2024-04-20T11:17:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.267+0000 7fa147400700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa13406ecc0 0x7fa134071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.268+0000 7fa147400700 1 -- 172.21.15.117:0/319025909 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fa1400902a0 con 0x7fa150102e20 2024-04-20T11:17:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.268+0000 7fa14de00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa13406ecc0 0x7fa134071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.268+0000 7fa14de00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa13406ecc0 0x7fa134071180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005950 tx=0x7fa13c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:19.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.272+0000 7fa147400700 1 -- 172.21.15.117:0/319025909 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa14005cbc0 con 0x7fa150102e20 2024-04-20T11:17:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:19 smithi117 ceph-mon[99036]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:19 smithi117 ceph-mon[99036]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:19 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:19 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2158265896' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:19 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3600717379' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:17:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:19 smithi173 ceph-mon[74165]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:19 smithi173 ceph-mon[74165]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:19 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:19 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2158265896' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:19 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3600717379' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:17:19.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.572+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa150062510 con 0x7fa150102e20 2024-04-20T11:17:19.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.572+0000 7fa147400700 1 -- 172.21.15.117:0/319025909 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fa150062510 con 0x7fa150102e20 2024-04-20T11:17:19.572 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:17:19.572 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:17:19.572 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:17:19.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa13406ecc0 msgr2=0x7fa134071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:19.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa13406ecc0 0x7fa134071180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005950 tx=0x7fa13c0058e0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 msgr2=0x7fa15019e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:19.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 0x7fa15019e880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1400092e0 tx=0x7fa14000be20 comp rx=0 tx=0).stop 2024-04-20T11:17:19.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 shutdown_connections 2024-04-20T11:17:19.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa13406ecc0 0x7fa134071180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa150102e20 0x7fa15019e880 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 --2- 172.21.15.117:0/319025909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1501037d0 0x7fa15019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:19.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 >> 172.21.15.117:0/319025909 conn(0x7fa1500fe6d0 msgr2=0x7fa150106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:19.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.575+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 shutdown_connections 2024-04-20T11:17:19.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:19.576+0000 7fa154a00700 1 -- 172.21.15.117:0/319025909 wait complete. 2024-04-20T11:17:20.113 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:19 smithi173 systemd[1]: Stopping Ceph osd.3 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:17:20.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:20 smithi117 ceph-mon[99036]: pgmap v126: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:20 smithi117 ceph-mon[99036]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:20 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/319025909' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:17:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:20 smithi173 ceph-mon[74165]: pgmap v126: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:20 smithi173 ceph-mon[74165]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:20 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/319025909' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:17:20.412 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:20 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[29765]: 2024-04-20T11:17:20.340+0000 7f7fd1600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:17:20.412 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:20 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[29765]: 2024-04-20T11:17:20.340+0000 7f7fd1600700 -1 osd.3 65 *** Got signal Terminated *** 2024-04-20T11:17:20.412 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:20 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[29765]: 2024-04-20T11:17:20.340+0000 7f7fd1600700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:17:21.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:21 smithi117 ceph-mon[99036]: osd.3 marked itself down and dead 2024-04-20T11:17:21.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:21 smithi173 ceph-mon[74165]: osd.3 marked itself down and dead 2024-04-20T11:17:22.080 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:21 smithi173 podman[83246]: 2024-04-20 11:17:21.754085688 +0000 UTC m=+1.619841471 container died 6ff75c097f92d91ebe69069029c766b90307809214b5e6f1ae8be6a880f35158 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, GIT_CLEAN=True, ceph=True, RELEASE=reef-646b8c7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-20T11:17:22.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:22 smithi117 ceph-mon[99036]: pgmap v127: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:22.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:22 smithi117 ceph-mon[99036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:17:22.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:22 smithi117 ceph-mon[99036]: osdmap e66: 6 total, 5 up, 6 in 2024-04-20T11:17:22.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:22 smithi173 ceph-mon[74165]: pgmap v127: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:22.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:22 smithi173 ceph-mon[74165]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:17:22.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:22 smithi173 ceph-mon[74165]: osdmap e66: 6 total, 5 up, 6 in 2024-04-20T11:17:22.412 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:22 smithi173 podman[83246]: 2024-04-20 11:17:22.216258117 +0000 UTC m=+2.082013914 container cleanup 6ff75c097f92d91ebe69069029c766b90307809214b5e6f1ae8be6a880f35158 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-646b8c7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:17:22.412 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:22 smithi173 bash[83246]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3 2024-04-20T11:17:22.911 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:22 smithi173 podman[83264]: 2024-04-20 11:17:22.469302289 +0000 UTC m=+0.698621435 container remove 6ff75c097f92d91ebe69069029c766b90307809214b5e6f1ae8be6a880f35158 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-646b8c7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-20T11:17:23.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:23 smithi173 ceph-mon[74165]: osdmap e67: 6 total, 5 up, 6 in 2024-04-20T11:17:23.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:23 smithi173 podman[83353]: 2024-04-20 11:17:23.234978004 +0000 UTC m=+0.036239974 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:17:23.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:23 smithi173 podman[83353]: 2024-04-20 11:17:23.385215248 +0000 UTC m=+0.186477236 container create cd6efbb858ab83b9a807a5d33dc84c5f6486824e7c20bbb77b0e6712a868a9c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:17:23.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:23 smithi117 ceph-mon[99036]: osdmap e67: 6 total, 5 up, 6 in 2024-04-20T11:17:24.052 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:23 smithi173 podman[83353]: 2024-04-20 11:17:23.689665444 +0000 UTC m=+0.490927432 container init cd6efbb858ab83b9a807a5d33dc84c5f6486824e7c20bbb77b0e6712a868a9c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-20T11:17:24.052 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:23 smithi173 podman[83353]: 2024-04-20 11:17:23.700661554 +0000 UTC m=+0.501923505 container start cd6efbb858ab83b9a807a5d33dc84c5f6486824e7c20bbb77b0e6712a868a9c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-20T11:17:24.052 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:23 smithi173 podman[83353]: 2024-04-20 11:17:23.788080541 +0000 UTC m=+0.589342559 container attach cd6efbb858ab83b9a807a5d33dc84c5f6486824e7c20bbb77b0e6712a868a9c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-20T11:17:24.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:24 smithi173 podman[83353]: 2024-04-20 11:17:24.15357308 +0000 UTC m=+0.954835101 container died cd6efbb858ab83b9a807a5d33dc84c5f6486824e7c20bbb77b0e6712a868a9c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:17:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:24 smithi173 ceph-mon[74165]: pgmap v130: 65 pgs: 13 peering, 13 stale+active+clean, 39 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:24 smithi173 ceph-mon[74165]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-20T11:17:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:24 smithi117 ceph-mon[99036]: pgmap v130: 65 pgs: 13 peering, 13 stale+active+clean, 39 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:24 smithi117 ceph-mon[99036]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-20T11:17:25.161 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:24 smithi173 podman[83380]: 2024-04-20 11:17:24.900050299 +0000 UTC m=+0.725403054 container remove cd6efbb858ab83b9a807a5d33dc84c5f6486824e7c20bbb77b0e6712a868a9c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-20T11:17:25.161 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:24 smithi173 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-20T11:17:25.162 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:24 smithi173 systemd[1]: Stopped Ceph osd.3 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:17:25.162 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:24 smithi173 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.3.service: Consumed 2min 20.447s CPU time. 2024-04-20T11:17:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:25 smithi117 ceph-mon[99036]: pgmap v131: 65 pgs: 8 active+undersized+degraded, 13 peering, 3 stale+active+clean, 10 active+undersized, 31 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 36/324 objects degraded (11.111%) 2024-04-20T11:17:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:25 smithi117 ceph-mon[99036]: Health check failed: Degraded data redundancy: 36/324 objects degraded (11.111%), 8 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:25.661 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:25 smithi173 systemd[1]: Starting Ceph osd.3 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:17:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:25 smithi173 ceph-mon[74165]: pgmap v131: 65 pgs: 8 active+undersized+degraded, 13 peering, 3 stale+active+clean, 10 active+undersized, 31 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 36/324 objects degraded (11.111%) 2024-04-20T11:17:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:25 smithi173 ceph-mon[74165]: Health check failed: Degraded data redundancy: 36/324 objects degraded (11.111%), 8 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:25.989 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:25 smithi173 podman[83479]: 2024-04-20 11:17:25.605824102 +0000 UTC m=+0.036229199 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:17:25.989 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:25 smithi173 podman[83479]: 2024-04-20 11:17:25.734568403 +0000 UTC m=+0.164973521 container create cf8df8adb9aaf036b5a5f8c07159c2a194a45e03b687fa0be082ac008028dd13 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-20T11:17:25.989 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:25 smithi173 podman[83479]: 2024-04-20 11:17:25.988883268 +0000 UTC m=+0.419288396 container init cf8df8adb9aaf036b5a5f8c07159c2a194a45e03b687fa0be082ac008028dd13 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-20T11:17:26.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 podman[83479]: 2024-04-20 11:17:26.000741984 +0000 UTC m=+0.431147072 container start cf8df8adb9aaf036b5a5f8c07159c2a194a45e03b687fa0be082ac008028dd13 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-20T11:17:26.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 podman[83479]: 2024-04-20 11:17:26.104144769 +0000 UTC m=+0.534549917 container attach cf8df8adb9aaf036b5a5f8c07159c2a194a45e03b687fa0be082ac008028dd13 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-20T11:17:27.363 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:17:27.363 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:17:27.363 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:17:27.363 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:17:27.363 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T11:17:27.363 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-20T11:17:27.363 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 bash[83479]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:26 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate[83494]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-20T11:17:27.364 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:27 smithi173 podman[83479]: 2024-04-20 11:17:27.020189086 +0000 UTC m=+1.450594197 container died cf8df8adb9aaf036b5a5f8c07159c2a194a45e03b687fa0be082ac008028dd13 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:17:28.057 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:27 smithi173 podman[83634]: 2024-04-20 11:17:27.691188739 +0000 UTC m=+0.651706098 container remove cf8df8adb9aaf036b5a5f8c07159c2a194a45e03b687fa0be082ac008028dd13 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-20T11:17:28.356 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:28 smithi173 ceph-mon[74165]: pgmap v132: 65 pgs: 11 active+undersized+degraded, 13 peering, 14 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:17:28.356 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:28.357 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:17:28.357 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:28 smithi173 podman[83695]: 2024-04-20 11:17:27.963340188 +0000 UTC m=+0.036277799 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:17:28.357 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:28 smithi173 podman[83695]: 2024-04-20 11:17:28.092432467 +0000 UTC m=+0.165370140 container create 266ace61d6c6c17a1ec028b58759ce36a1c10a43c0a836295147b1d920abdf7b (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-20T11:17:28.357 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:28 smithi173 podman[83695]: 2024-04-20 11:17:28.356612556 +0000 UTC m=+0.429550209 container init 266ace61d6c6c17a1ec028b58759ce36a1c10a43c0a836295147b1d920abdf7b (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-20T11:17:28.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:28 smithi117 ceph-mon[99036]: pgmap v132: 65 pgs: 11 active+undersized+degraded, 13 peering, 14 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/324 objects degraded (14.198%) 2024-04-20T11:17:28.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:17:28.644 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:28 smithi173 podman[83695]: 2024-04-20 11:17:28.368471035 +0000 UTC m=+0.441408651 container start 266ace61d6c6c17a1ec028b58759ce36a1c10a43c0a836295147b1d920abdf7b (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:17:28.644 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:28 smithi173 ceph-osd[83714]: -- 172.21.15.173:0/2647883526 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d3c897680 con 0x562d3c861000 2024-04-20T11:17:28.644 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:28 smithi173 bash[83695]: 266ace61d6c6c17a1ec028b58759ce36a1c10a43c0a836295147b1d920abdf7b 2024-04-20T11:17:28.644 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:28 smithi173 systemd[1]: Started Ceph osd.3 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:17:29.410 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:28 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:17:28.948+0000 7f22fc0627c0 -1 Falling back to public interface 2024-04-20T11:17:29.882 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:29 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:29.882 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:29 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:29.882 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:29 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:29.882 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:29 smithi173 ceph-mon[74165]: pgmap v133: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:29.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:29 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:29.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:29 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:29.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:29 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:29.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:29 smithi117 ceph-mon[99036]: pgmap v133: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:30.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:30 smithi117 ceph-mon[99036]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-20T11:17:30.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:30 smithi173 ceph-mon[74165]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-20T11:17:31.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:31 smithi117 ceph-mon[99036]: pgmap v134: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:31.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:31 smithi173 ceph-mon[74165]: pgmap v134: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:34 smithi117 ceph-mon[99036]: pgmap v135: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:34.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:34 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 68/324 objects degraded (20.988%), 19 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:34.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:34 smithi173 ceph-mon[74165]: pgmap v135: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:34 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 68/324 objects degraded (20.988%), 19 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:35.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:34 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:17:34.936+0000 7f22fc0627c0 -1 osd.3 65 log_to_monitors true 2024-04-20T11:17:35.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:17:35 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:17:35.172+0000 7f22e6400700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:17:35.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:35 smithi173 ceph-mon[74165]: from='osd.3 [v2:172.21.15.173:6800/151782600,v1:172.21.15.173:6801/151782600]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:17:35.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:35 smithi173 ceph-mon[74165]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:17:35.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:35.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:35 smithi117 ceph-mon[99036]: from='osd.3 [v2:172.21.15.173:6800/151782600,v1:172.21.15.173:6801/151782600]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:17:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:35 smithi117 ceph-mon[99036]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:17:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:36.370 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:36 smithi173 ceph-mon[74165]: pgmap v136: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:36.370 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:36 smithi173 ceph-mon[74165]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T11:17:36.370 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:36 smithi173 ceph-mon[74165]: osdmap e68: 6 total, 5 up, 6 in 2024-04-20T11:17:36.370 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:36 smithi173 ceph-mon[74165]: from='osd.3 [v2:172.21.15.173:6800/151782600,v1:172.21.15.173:6801/151782600]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:17:36.370 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:36 smithi173 ceph-mon[74165]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:17:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:36 smithi117 ceph-mon[99036]: pgmap v136: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:36 smithi117 ceph-mon[99036]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T11:17:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:36 smithi117 ceph-mon[99036]: osdmap e68: 6 total, 5 up, 6 in 2024-04-20T11:17:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:36 smithi117 ceph-mon[99036]: from='osd.3 [v2:172.21.15.173:6800/151782600,v1:172.21.15.173:6801/151782600]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:17:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:36 smithi117 ceph-mon[99036]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:17:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:37 smithi173 ceph-mon[74165]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:17:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:37 smithi173 ceph-mon[74165]: osd.3 [v2:172.21.15.173:6800/151782600,v1:172.21.15.173:6801/151782600] boot 2024-04-20T11:17:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:37 smithi173 ceph-mon[74165]: osdmap e69: 6 total, 6 up, 6 in 2024-04-20T11:17:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:37 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:17:37.454 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:37 smithi117 ceph-mon[99036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:17:37.455 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:37 smithi117 ceph-mon[99036]: osd.3 [v2:172.21.15.173:6800/151782600,v1:172.21.15.173:6801/151782600] boot 2024-04-20T11:17:37.455 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:37 smithi117 ceph-mon[99036]: osdmap e69: 6 total, 6 up, 6 in 2024-04-20T11:17:37.455 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:37 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:17:38.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:38 smithi117 ceph-mon[99036]: pgmap v139: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:38 smithi117 ceph-mon[99036]: osdmap e70: 6 total, 6 up, 6 in 2024-04-20T11:17:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:38 smithi173 ceph-mon[74165]: pgmap v139: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 68/324 objects degraded (20.988%) 2024-04-20T11:17:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:38 smithi173 ceph-mon[74165]: osdmap e70: 6 total, 6 up, 6 in 2024-04-20T11:17:39.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:39 smithi173 ceph-mon[74165]: pgmap v141: 65 pgs: 10 peering, 11 active+undersized+degraded, 9 active+undersized, 35 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/324 objects degraded (11.728%) 2024-04-20T11:17:39.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:39 smithi117 ceph-mon[99036]: pgmap v141: 65 pgs: 10 peering, 11 active+undersized+degraded, 9 active+undersized, 35 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/324 objects degraded (11.728%) 2024-04-20T11:17:40.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:40 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 38/324 objects degraded (11.728%), 11 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:40 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 38/324 objects degraded (11.728%), 11 pgs degraded (PG_DEGRADED) 2024-04-20T11:17:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:41.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:41.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:41.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:41.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:41.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-20T11:17:41.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-20T11:17:41.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-20T11:17:41.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:41 smithi117 ceph-mon[99036]: pgmap v142: 65 pgs: 10 peering, 1 active+undersized+degraded, 3 active+undersized, 51 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3/324 objects degraded (0.926%) 2024-04-20T11:17:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-20T11:17:41.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:41 smithi173 ceph-mon[74165]: pgmap v142: 65 pgs: 10 peering, 1 active+undersized+degraded, 3 active+undersized, 51 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3/324 objects degraded (0.926%) 2024-04-20T11:17:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:44 smithi117 ceph-mon[99036]: pgmap v143: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:17:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:44 smithi117 ceph-mon[99036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/324 objects degraded (0.926%), 1 pg degraded) 2024-04-20T11:17:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:44 smithi173 ceph-mon[74165]: pgmap v143: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:17:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:44 smithi173 ceph-mon[74165]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/324 objects degraded (0.926%), 1 pg degraded) 2024-04-20T11:17:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:46 smithi117 ceph-mon[99036]: pgmap v144: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:46 smithi173 ceph-mon[74165]: pgmap v144: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:48.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:48 smithi117 ceph-mon[99036]: pgmap v145: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:48 smithi173 ceph-mon[74165]: pgmap v145: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:49.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:49.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.743+0000 7fbc05a00700 1 -- 172.21.15.117:0/2318581756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00107520 msgr2=0x7fbc00107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:49.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.743+0000 7fbc05a00700 1 --2- 172.21.15.117:0/2318581756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00107520 0x7fbc00107900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009910 tx=0x7fbbf0009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:49.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.743+0000 7fbc05a00700 1 -- 172.21.15.117:0/2318581756 shutdown_connections 2024-04-20T11:17:49.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.743+0000 7fbc05a00700 1 --2- 172.21.15.117:0/2318581756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc00108e10 0x7fbc00109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:49.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.743+0000 7fbc05a00700 1 --2- 172.21.15.117:0/2318581756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00107520 0x7fbc00107900 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:49.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.743+0000 7fbc05a00700 1 -- 172.21.15.117:0/2318581756 >> 172.21.15.117:0/2318581756 conn(0x7fbc000768f0 msgr2=0x7fbc00078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:49.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.743+0000 7fbc05a00700 1 -- 172.21.15.117:0/2318581756 shutdown_connections 2024-04-20T11:17:49.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.743+0000 7fbc05a00700 1 -- 172.21.15.117:0/2318581756 wait complete. 2024-04-20T11:17:49.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.744+0000 7fbc05a00700 1 Processor -- start 2024-04-20T11:17:49.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.744+0000 7fbc05a00700 1 -- start start 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.744+0000 7fbc05a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc00107520 0x7fbc0019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.744+0000 7fbc05a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00108e10 0x7fbc0019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.744+0000 7fbc05a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0019ddd0 con 0x7fbc00108e10 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.744+0000 7fbc05a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc001a1aa0 con 0x7fbc00107520 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.745+0000 7fbbfe000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00108e10 0x7fbc0019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.745+0000 7fbbfe000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00108e10 0x7fbc0019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56168/0 (socket says 172.21.15.117:56168) 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.745+0000 7fbbfe000700 1 -- 172.21.15.117:0/1097557307 learned_addr learned my addr 172.21.15.117:0/1097557307 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.745+0000 7fbbfea00700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc00107520 0x7fbc0019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.745+0000 7fbbfe000700 1 -- 172.21.15.117:0/1097557307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc00107520 msgr2=0x7fbc0019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.745+0000 7fbbfe000700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc00107520 0x7fbc0019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:49.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.745+0000 7fbbfe000700 1 -- 172.21.15.117:0/1097557307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00095f0 con 0x7fbc00108e10 2024-04-20T11:17:49.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.746+0000 7fbbfe000700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00108e10 0x7fbc0019d6f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800c8a0 tx=0x7fbbe800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:49.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.746+0000 7fbbf7400700 1 -- 172.21.15.117:0/1097557307 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbe800cea0 con 0x7fbc00108e10 2024-04-20T11:17:49.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.746+0000 7fbbf7400700 1 -- 172.21.15.117:0/1097557307 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbe8004830 con 0x7fbc00108e10 2024-04-20T11:17:49.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.746+0000 7fbbf7400700 1 -- 172.21.15.117:0/1097557307 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbe80055b0 con 0x7fbc00108e10 2024-04-20T11:17:49.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.746+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc001a1d80 con 0x7fbc00108e10 2024-04-20T11:17:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.746+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc001a2350 con 0x7fbc00108e10 2024-04-20T11:17:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.747+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0010f870 con 0x7fbc00108e10 2024-04-20T11:17:49.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.749+0000 7fbbf7400700 1 -- 172.21.15.117:0/1097557307 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbbe80049a0 con 0x7fbc00108e10 2024-04-20T11:17:49.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.750+0000 7fbbf7400700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbe406ecc0 0x7fbbe4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:49.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.750+0000 7fbbf7400700 1 -- 172.21.15.117:0/1097557307 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fbbe808e970 con 0x7fbc00108e10 2024-04-20T11:17:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.750+0000 7fbbfea00700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbe406ecc0 0x7fbbe4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.751+0000 7fbbfea00700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbe406ecc0 0x7fbbe4071180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0005f50 tx=0x7fbbf001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:49.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:49.755+0000 7fbbf7400700 1 -- 172.21.15.117:0/1097557307 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbbe8057880 con 0x7fbc00108e10 2024-04-20T11:17:50.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.001+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc0019e530 con 0x7fbbe406ecc0 2024-04-20T11:17:50.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.004+0000 7fbbf7400700 1 -- 172.21.15.117:0/1097557307 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbc0019e530 con 0x7fbbe406ecc0 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbe406ecc0 msgr2=0x7fbbe4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbe406ecc0 0x7fbbe4071180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0005f50 tx=0x7fbbf001a040 comp rx=0 tx=0).stop 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00108e10 msgr2=0x7fbc0019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00108e10 0x7fbc0019d6f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800c8a0 tx=0x7fbbe800cc60 comp rx=0 tx=0).stop 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 shutdown_connections 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbe406ecc0 0x7fbbe4071180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc00107520 0x7fbc0019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 --2- 172.21.15.117:0/1097557307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00108e10 0x7fbc0019d6f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.007+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 >> 172.21.15.117:0/1097557307 conn(0x7fbc000768f0 msgr2=0x7fbc00077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:50.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.008+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 shutdown_connections 2024-04-20T11:17:50.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.008+0000 7fbc05a00700 1 -- 172.21.15.117:0/1097557307 wait complete. 2024-04-20T11:17:50.025 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:17:50.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.165+0000 7fe8ce800700 1 -- 172.21.15.117:0/1997820139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8103110 msgr2=0x7fe8c81034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.165+0000 7fe8ce800700 1 --2- 172.21.15.117:0/1997820139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8103110 0x7fe8c81034f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009910 tx=0x7fe8b8009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:50.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.165+0000 7fe8ce800700 1 -- 172.21.15.117:0/1997820139 shutdown_connections 2024-04-20T11:17:50.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.165+0000 7fe8ce800700 1 --2- 172.21.15.117:0/1997820139 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8c8104a00 0x7fe8c8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.165+0000 7fe8ce800700 1 --2- 172.21.15.117:0/1997820139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8103110 0x7fe8c81034f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.165+0000 7fe8ce800700 1 -- 172.21.15.117:0/1997820139 >> 172.21.15.117:0/1997820139 conn(0x7fe8c80fe880 msgr2=0x7fe8c8100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:50.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.165+0000 7fe8ce800700 1 -- 172.21.15.117:0/1997820139 shutdown_connections 2024-04-20T11:17:50.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.165+0000 7fe8ce800700 1 -- 172.21.15.117:0/1997820139 wait complete. 2024-04-20T11:17:50.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.166+0000 7fe8ce800700 1 Processor -- start 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.166+0000 7fe8ce800700 1 -- start start 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.166+0000 7fe8ce800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8c8103110 0x7fe8c8198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.166+0000 7fe8ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8104a00 0x7fe8c8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.166+0000 7fe8ce800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81999e0 con 0x7fe8c8104a00 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.166+0000 7fe8ce800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c819d660 con 0x7fe8c8103110 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.167+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8104a00 0x7fe8c8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.167+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8104a00 0x7fe8c8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56196/0 (socket says 172.21.15.117:56196) 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.167+0000 7fe8bea00700 1 -- 172.21.15.117:0/559994693 learned_addr learned my addr 172.21.15.117:0/559994693 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.167+0000 7fe8c7400700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8c8103110 0x7fe8c8198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.167+0000 7fe8bea00700 1 -- 172.21.15.117:0/559994693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8c8103110 msgr2=0x7fe8c8198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.167+0000 7fe8bea00700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8c8103110 0x7fe8c8198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.167+0000 7fe8bea00700 1 -- 172.21.15.117:0/559994693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80095f0 con 0x7fe8c8104a00 2024-04-20T11:17:50.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.168+0000 7fe8bea00700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8104a00 0x7fe8c8199300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000d8d0 tx=0x7fe8b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:50.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.168+0000 7fe8c4c00700 1 -- 172.21.15.117:0/559994693 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8b0009940 con 0x7fe8c8104a00 2024-04-20T11:17:50.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.168+0000 7fe8c4c00700 1 -- 172.21.15.117:0/559994693 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8b0010460 con 0x7fe8c8104a00 2024-04-20T11:17:50.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.168+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c819d940 con 0x7fe8c8104a00 2024-04-20T11:17:50.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.168+0000 7fe8c4c00700 1 -- 172.21.15.117:0/559994693 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8b000f5f0 con 0x7fe8c8104a00 2024-04-20T11:17:50.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.168+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c819df10 con 0x7fe8c8104a00 2024-04-20T11:17:50.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.169+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c810b460 con 0x7fe8c8104a00 2024-04-20T11:17:50.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.170+0000 7fe8c4c00700 1 -- 172.21.15.117:0/559994693 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe8b00105d0 con 0x7fe8c8104a00 2024-04-20T11:17:50.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.171+0000 7fe8c4c00700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8ac06ecd0 0x7fe8ac071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:50.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.171+0000 7fe8c4c00700 1 -- 172.21.15.117:0/559994693 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fe8b008eda0 con 0x7fe8c8104a00 2024-04-20T11:17:50.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.172+0000 7fe8c7400700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8ac06ecd0 0x7fe8ac071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:50.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.172+0000 7fe8c7400700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8ac06ecd0 0x7fe8ac071190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b5c0 tx=0x7fe8b801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:50.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.177+0000 7fe8c4c00700 1 -- 172.21.15.117:0/559994693 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe8b0057c30 con 0x7fe8c8104a00 2024-04-20T11:17:50.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:50 smithi117 ceph-mon[99036]: pgmap v146: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:50 smithi173 ceph-mon[74165]: pgmap v146: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:50.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.418+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8c819a080 con 0x7fe8ac06ecd0 2024-04-20T11:17:50.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.421+0000 7fe8c4c00700 1 -- 172.21.15.117:0/559994693 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe8c819a080 con 0x7fe8ac06ecd0 2024-04-20T11:17:50.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8ac06ecd0 msgr2=0x7fe8ac071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8ac06ecd0 0x7fe8ac071190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b5c0 tx=0x7fe8b801a040 comp rx=0 tx=0).stop 2024-04-20T11:17:50.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8104a00 msgr2=0x7fe8c8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8104a00 0x7fe8c8199300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000d8d0 tx=0x7fe8b000dc90 comp rx=0 tx=0).stop 2024-04-20T11:17:50.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 shutdown_connections 2024-04-20T11:17:50.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8ac06ecd0 0x7fe8ac071190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8c8103110 0x7fe8c8198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 --2- 172.21.15.117:0/559994693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8104a00 0x7fe8c8199300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.424+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 >> 172.21.15.117:0/559994693 conn(0x7fe8c80fe880 msgr2=0x7fe8c80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:50.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.425+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 shutdown_connections 2024-04-20T11:17:50.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.425+0000 7fe8ce800700 1 -- 172.21.15.117:0/559994693 wait complete. 2024-04-20T11:17:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.579+0000 7fc24f000700 1 -- 172.21.15.117:0/26749849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2481070b0 msgr2=0x7fc248107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.579+0000 7fc24f000700 1 --2- 172.21.15.117:0/26749849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2481070b0 0x7fc248107510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc234009910 tx=0x7fc234009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:50.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.579+0000 7fc247e00700 1 -- 172.21.15.117:0/26749849 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2340054b0 con 0x7fc2481070b0 2024-04-20T11:17:50.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.580+0000 7fc24f000700 1 -- 172.21.15.117:0/26749849 shutdown_connections 2024-04-20T11:17:50.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.580+0000 7fc24f000700 1 --2- 172.21.15.117:0/26749849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2481070b0 0x7fc248107510 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.580+0000 7fc24f000700 1 --2- 172.21.15.117:0/26749849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc24810d0c0 0x7fc24810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.580+0000 7fc24f000700 1 -- 172.21.15.117:0/26749849 >> 172.21.15.117:0/26749849 conn(0x7fc248076830 msgr2=0x7fc248078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:50.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.580+0000 7fc24f000700 1 -- 172.21.15.117:0/26749849 shutdown_connections 2024-04-20T11:17:50.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.580+0000 7fc24f000700 1 -- 172.21.15.117:0/26749849 wait complete. 2024-04-20T11:17:50.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.581+0000 7fc24f000700 1 Processor -- start 2024-04-20T11:17:50.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.581+0000 7fc24f000700 1 -- start start 2024-04-20T11:17:50.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24f000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2481070b0 0x7fc24819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:50.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24f000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc24810d0c0 0x7fc24819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:50.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24f000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24819da90 con 0x7fc24810d0c0 2024-04-20T11:17:50.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24f000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24819dbd0 con 0x7fc2481070b0 2024-04-20T11:17:50.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc24810d0c0 0x7fc24819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:50.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc24810d0c0 0x7fc24819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56214/0 (socket says 172.21.15.117:56214) 2024-04-20T11:17:50.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24d200700 1 -- 172.21.15.117:0/443656510 learned_addr learned my addr 172.21.15.117:0/443656510 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:50.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24dc00700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2481070b0 0x7fc24819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:50.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24d200700 1 -- 172.21.15.117:0/443656510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2481070b0 msgr2=0x7fc24819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24d200700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2481070b0 0x7fc24819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24d200700 1 -- 172.21.15.117:0/443656510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2340095f0 con 0x7fc24810d0c0 2024-04-20T11:17:50.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.582+0000 7fc24dc00700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2481070b0 0x7fc24819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:17:50.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.583+0000 7fc24d200700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc24810d0c0 0x7fc24819d440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2340092e0 tx=0x7fc23400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:50.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.583+0000 7fc246000700 1 -- 172.21.15.117:0/443656510 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc23401d070 con 0x7fc24810d0c0 2024-04-20T11:17:50.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.583+0000 7fc246000700 1 -- 172.21.15.117:0/443656510 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc23400f460 con 0x7fc24810d0c0 2024-04-20T11:17:50.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.583+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2481a1900 con 0x7fc24810d0c0 2024-04-20T11:17:50.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.583+0000 7fc246000700 1 -- 172.21.15.117:0/443656510 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc234017630 con 0x7fc24810d0c0 2024-04-20T11:17:50.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.583+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2481a1df0 con 0x7fc24810d0c0 2024-04-20T11:17:50.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.584+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc248108770 con 0x7fc24810d0c0 2024-04-20T11:17:50.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.585+0000 7fc246000700 1 -- 172.21.15.117:0/443656510 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc23400f5d0 con 0x7fc24810d0c0 2024-04-20T11:17:50.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.586+0000 7fc246000700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc23006ed10 0x7fc2300711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:50.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.586+0000 7fc246000700 1 -- 172.21.15.117:0/443656510 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fc234090300 con 0x7fc24810d0c0 2024-04-20T11:17:50.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.587+0000 7fc24dc00700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc23006ed10 0x7fc2300711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:50.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.587+0000 7fc24dc00700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc23006ed10 0x7fc2300711d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc23c005fd0 tx=0x7fc23c005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:50.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.591+0000 7fc246000700 1 -- 172.21.15.117:0/443656510 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc234059230 con 0x7fc24810d0c0 2024-04-20T11:17:50.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.819+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc248114280 con 0x7fc23006ed10 2024-04-20T11:17:50.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.849+0000 7fc246000700 1 -- 172.21.15.117:0/443656510 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fc248114280 con 0x7fc23006ed10 2024-04-20T11:17:50.850 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (6m) 46s ago 20m 14.2M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (20m) 46s ago 20m 9017k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (19m) 17s ago 19m 8434k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (2m) 46s ago 20m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (2m) 17s ago 19m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (5m) 46s ago 20m 74.3M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (13m) 46s ago 13m 18.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (13m) 46s ago 13m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (12m) 17s ago 12m 18.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:17:50.851 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (13m) 17s ago 13m 25.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (9m) 46s ago 22m 506M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (7m) 17s ago 19m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (4m) 46s ago 22m 51.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (3m) 17s ago 18m 35.5M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (7m) 46s ago 20m 7872k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (7m) 17s ago 19m 7713k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (117s) 46s ago 17m 80.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (85s) 46s ago 17m 82.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (54s) 46s ago 16m 11.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (22s) 17s ago 16m 11.4M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (15m) 17s ago 15m 216M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e08c85cecb50 2024-04-20T11:17:50.852 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (15m) 17s ago 14m 203M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:17:50.853 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (6m) 46s ago 19m 69.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:17:50.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc23006ed10 msgr2=0x7fc2300711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc23006ed10 0x7fc2300711d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc23c005fd0 tx=0x7fc23c005e20 comp rx=0 tx=0).stop 2024-04-20T11:17:50.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc24810d0c0 msgr2=0x7fc24819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:50.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc24810d0c0 0x7fc24819d440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2340092e0 tx=0x7fc23400be20 comp rx=0 tx=0).stop 2024-04-20T11:17:50.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 shutdown_connections 2024-04-20T11:17:50.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc23006ed10 0x7fc2300711d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2481070b0 0x7fc24819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 --2- 172.21.15.117:0/443656510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc24810d0c0 0x7fc24819d440 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:50.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.853+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 >> 172.21.15.117:0/443656510 conn(0x7fc248076830 msgr2=0x7fc24810c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:50.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.854+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 shutdown_connections 2024-04-20T11:17:50.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:50.854+0000 7fc24f000700 1 -- 172.21.15.117:0/443656510 wait complete. 2024-04-20T11:17:51.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.005+0000 7ffbe6a00700 1 -- 172.21.15.117:0/1989935942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 msgr2=0x7ffbe01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.005+0000 7ffbe6a00700 1 --2- 172.21.15.117:0/1989935942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 0x7ffbe01034f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009910 tx=0x7ffbd0009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:51.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.005+0000 7ffbe6a00700 1 -- 172.21.15.117:0/1989935942 shutdown_connections 2024-04-20T11:17:51.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.005+0000 7ffbe6a00700 1 --2- 172.21.15.117:0/1989935942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe0104a00 0x7ffbe0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.005+0000 7ffbe6a00700 1 --2- 172.21.15.117:0/1989935942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 0x7ffbe01034f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.005+0000 7ffbe6a00700 1 -- 172.21.15.117:0/1989935942 >> 172.21.15.117:0/1989935942 conn(0x7ffbe00fe880 msgr2=0x7ffbe0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:51.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.005+0000 7ffbe6a00700 1 -- 172.21.15.117:0/1989935942 shutdown_connections 2024-04-20T11:17:51.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.005+0000 7ffbe6a00700 1 -- 172.21.15.117:0/1989935942 wait complete. 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.006+0000 7ffbe6a00700 1 Processor -- start 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.006+0000 7ffbe6a00700 1 -- start start 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.007+0000 7ffbe6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 0x7ffbe0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.007+0000 7ffbe6a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe0104a00 0x7ffbe01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.007+0000 7ffbe6a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe0199a80 con 0x7ffbe0103110 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.007+0000 7ffbe6a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe019d750 con 0x7ffbe0104a00 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.007+0000 7ffbdfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 0x7ffbe0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.007+0000 7ffbdfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 0x7ffbe0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56226/0 (socket says 172.21.15.117:56226) 2024-04-20T11:17:51.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.007+0000 7ffbdfe00700 1 -- 172.21.15.117:0/720633219 learned_addr learned my addr 172.21.15.117:0/720633219 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbdfe00700 1 -- 172.21.15.117:0/720633219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe0104a00 msgr2=0x7ffbe01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbdfe00700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe0104a00 0x7ffbe01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbdfe00700 1 -- 172.21.15.117:0/720633219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd00095f0 con 0x7ffbe0103110 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbdfe00700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 0x7ffbe0198e60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00092e0 tx=0x7ffbd000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbdcc00700 1 -- 172.21.15.117:0/720633219 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbd001d070 con 0x7ffbe0103110 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbdcc00700 1 -- 172.21.15.117:0/720633219 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbd000f460 con 0x7ffbe0103110 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe019d9d0 con 0x7ffbe0103110 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbdcc00700 1 -- 172.21.15.117:0/720633219 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbd0017630 con 0x7ffbe0103110 2024-04-20T11:17:51.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.008+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe019df40 con 0x7ffbe0103110 2024-04-20T11:17:51.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.010+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe010b480 con 0x7ffbe0103110 2024-04-20T11:17:51.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.011+0000 7ffbdcc00700 1 -- 172.21.15.117:0/720633219 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ffbd000f5d0 con 0x7ffbe0103110 2024-04-20T11:17:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.011+0000 7ffbdcc00700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffbc406ecd0 0x7ffbc4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.012+0000 7ffbdcc00700 1 -- 172.21.15.117:0/720633219 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6285+0+0 (secure 0 0 0) 0x7ffbd0090270 con 0x7ffbe0103110 2024-04-20T11:17:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.012+0000 7ffbdf400700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffbc406ecd0 0x7ffbc4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:51.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.013+0000 7ffbdf400700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffbc406ecd0 0x7ffbc4071190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffbe019a3c0 tx=0x7ffbcc006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:51.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.017+0000 7ffbdcc00700 1 -- 172.21.15.117:0/720633219 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ffbd00590e0 con 0x7ffbe0103110 2024-04-20T11:17:51.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.314+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffbe019a170 con 0x7ffbe0103110 2024-04-20T11:17:51.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.315+0000 7ffbdcc00700 1 -- 172.21.15.117:0/720633219 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7ffbd005c7a0 con 0x7ffbe0103110 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2, 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:17:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:17:51.317 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:17:51.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:17:51.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:17:51.317 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:17:51.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6, 2024-04-20T11:17:51.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 8 2024-04-20T11:17:51.317 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:17:51.317 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:17:51.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffbc406ecd0 msgr2=0x7ffbc4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffbc406ecd0 0x7ffbc4071190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffbe019a3c0 tx=0x7ffbcc006d20 comp rx=0 tx=0).stop 2024-04-20T11:17:51.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 msgr2=0x7ffbe0198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 0x7ffbe0198e60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00092e0 tx=0x7ffbd000be20 comp rx=0 tx=0).stop 2024-04-20T11:17:51.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 shutdown_connections 2024-04-20T11:17:51.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffbc406ecd0 0x7ffbc4071190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe0103110 0x7ffbe0198e60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 --2- 172.21.15.117:0/720633219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe0104a00 0x7ffbe01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.317+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 >> 172.21.15.117:0/720633219 conn(0x7ffbe00fe880 msgr2=0x7ffbe00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:51.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.318+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 shutdown_connections 2024-04-20T11:17:51.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.318+0000 7ffbe6a00700 1 -- 172.21.15.117:0/720633219 wait complete. 2024-04-20T11:17:51.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:51 smithi173 ceph-mon[74165]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:51.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.473+0000 7fd189200700 1 -- 172.21.15.117:0/805185411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1841017c0 msgr2=0x7fd184105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.473+0000 7fd189200700 1 --2- 172.21.15.117:0/805185411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1841017c0 0x7fd184105810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd170009910 tx=0x7fd170009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:51.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.473+0000 7fd189200700 1 -- 172.21.15.117:0/805185411 shutdown_connections 2024-04-20T11:17:51.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.473+0000 7fd189200700 1 --2- 172.21.15.117:0/805185411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1841017c0 0x7fd184105810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.473+0000 7fd189200700 1 --2- 172.21.15.117:0/805185411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd184100e10 0x7fd1841011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.473+0000 7fd189200700 1 -- 172.21.15.117:0/805185411 >> 172.21.15.117:0/805185411 conn(0x7fd1840fc6c0 msgr2=0x7fd1840feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:51.475 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:51 smithi117 ceph-mon[99036]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:51.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.473+0000 7fd189200700 1 -- 172.21.15.117:0/805185411 shutdown_connections 2024-04-20T11:17:51.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.473+0000 7fd189200700 1 -- 172.21.15.117:0/805185411 wait complete. 2024-04-20T11:17:51.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.474+0000 7fd189200700 1 Processor -- start 2024-04-20T11:17:51.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.475+0000 7fd189200700 1 -- start start 2024-04-20T11:17:51.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.475+0000 7fd189200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd184100e10 0x7fd18419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.475+0000 7fd189200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1841017c0 0x7fd18419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.475+0000 7fd189200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18419f5a0 con 0x7fd184100e10 2024-04-20T11:17:51.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.475+0000 7fd189200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd184198b20 con 0x7fd1841017c0 2024-04-20T11:17:51.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.475+0000 7fd181e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd184100e10 0x7fd18419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:51.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.475+0000 7fd181e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd184100e10 0x7fd18419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56234/0 (socket says 172.21.15.117:56234) 2024-04-20T11:17:51.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.475+0000 7fd181e00700 1 -- 172.21.15.117:0/2945839308 learned_addr learned my addr 172.21.15.117:0/2945839308 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:51.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.476+0000 7fd181400700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1841017c0 0x7fd18419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:51.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.476+0000 7fd181e00700 1 -- 172.21.15.117:0/2945839308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1841017c0 msgr2=0x7fd18419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.476+0000 7fd181e00700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1841017c0 0x7fd18419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.476+0000 7fd181e00700 1 -- 172.21.15.117:0/2945839308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700095f0 con 0x7fd184100e10 2024-04-20T11:17:51.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.476+0000 7fd181400700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1841017c0 0x7fd18419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:17:51.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.476+0000 7fd181e00700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd184100e10 0x7fd18419e980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd17400c8a0 tx=0x7fd17400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:51.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.477+0000 7fd17aa00700 1 -- 172.21.15.117:0/2945839308 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd17400cea0 con 0x7fd184100e10 2024-04-20T11:17:51.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.477+0000 7fd17aa00700 1 -- 172.21.15.117:0/2945839308 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd174004830 con 0x7fd184100e10 2024-04-20T11:17:51.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.477+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd184198e00 con 0x7fd184100e10 2024-04-20T11:17:51.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.477+0000 7fd17aa00700 1 -- 172.21.15.117:0/2945839308 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1740055b0 con 0x7fd184100e10 2024-04-20T11:17:51.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.477+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd184199350 con 0x7fd184100e10 2024-04-20T11:17:51.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.478+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd184103f80 con 0x7fd184100e10 2024-04-20T11:17:51.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.480+0000 7fd17aa00700 1 -- 172.21.15.117:0/2945839308 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd1740049a0 con 0x7fd184100e10 2024-04-20T11:17:51.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.481+0000 7fd17aa00700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd16806ecc0 0x7fd168071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.481+0000 7fd17aa00700 1 -- 172.21.15.117:0/2945839308 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fd17408ead0 con 0x7fd184100e10 2024-04-20T11:17:51.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.481+0000 7fd181400700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd16806ecc0 0x7fd168071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:51.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.482+0000 7fd181400700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd16806ecc0 0x7fd168071180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd170006010 tx=0x7fd17000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:51.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.486+0000 7fd17aa00700 1 -- 172.21.15.117:0/2945839308 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd17405b430 con 0x7fd184100e10 2024-04-20T11:17:51.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.759+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd18410ddb0 con 0x7fd184100e10 2024-04-20T11:17:51.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.760+0000 7fd17aa00700 1 -- 172.21.15.117:0/2945839308 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7fd17401d780 con 0x7fd184100e10 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:17:51.762 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:17:51.763 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:17:51.764 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:17:51.765 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:17:51.765 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:17:51.765 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:17:51.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.763+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd16806ecc0 msgr2=0x7fd168071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.763+0000 7fd189200700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd16806ecc0 0x7fd168071180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd170006010 tx=0x7fd17000b540 comp rx=0 tx=0).stop 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.763+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd184100e10 msgr2=0x7fd18419e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.763+0000 7fd189200700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd184100e10 0x7fd18419e980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd17400c8a0 tx=0x7fd17400cc60 comp rx=0 tx=0).stop 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.764+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 shutdown_connections 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.764+0000 7fd189200700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd16806ecc0 0x7fd168071180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.764+0000 7fd189200700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd184100e10 0x7fd18419e980 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.764+0000 7fd189200700 1 --2- 172.21.15.117:0/2945839308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1841017c0 0x7fd18419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.764+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 >> 172.21.15.117:0/2945839308 conn(0x7fd1840fc6c0 msgr2=0x7fd1840fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.764+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 shutdown_connections 2024-04-20T11:17:51.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.764+0000 7fd189200700 1 -- 172.21.15.117:0/2945839308 wait complete. 2024-04-20T11:17:51.767 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:17:51.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.913+0000 7fae40600700 1 -- 172.21.15.117:0/1894326242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 msgr2=0x7fae38104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.913+0000 7fae40600700 1 --2- 172.21.15.117:0/1894326242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 0x7fae38104e80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae24009910 tx=0x7fae24009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:51.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.914+0000 7fae40600700 1 -- 172.21.15.117:0/1894326242 shutdown_connections 2024-04-20T11:17:51.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.914+0000 7fae40600700 1 --2- 172.21.15.117:0/1894326242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 0x7fae38104e80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.914+0000 7fae40600700 1 --2- 172.21.15.117:0/1894326242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae38103110 0x7fae381034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.914+0000 7fae40600700 1 -- 172.21.15.117:0/1894326242 >> 172.21.15.117:0/1894326242 conn(0x7fae380fe880 msgr2=0x7fae38100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:51.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.914+0000 7fae40600700 1 -- 172.21.15.117:0/1894326242 shutdown_connections 2024-04-20T11:17:51.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.914+0000 7fae40600700 1 -- 172.21.15.117:0/1894326242 wait complete. 2024-04-20T11:17:51.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.915+0000 7fae40600700 1 Processor -- start 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.915+0000 7fae40600700 1 -- start start 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.915+0000 7fae40600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae38103110 0x7fae3819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.915+0000 7fae40600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 0x7fae3819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.915+0000 7fae40600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3819f5b0 con 0x7fae38104a00 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.915+0000 7fae40600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38198b90 con 0x7fae38103110 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.916+0000 7fae3d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 0x7fae3819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.916+0000 7fae3d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 0x7fae3819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56248/0 (socket says 172.21.15.117:56248) 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.916+0000 7fae3d200700 1 -- 172.21.15.117:0/4070092583 learned_addr learned my addr 172.21.15.117:0/4070092583 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.916+0000 7fae3dc00700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae38103110 0x7fae3819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.916+0000 7fae3d200700 1 -- 172.21.15.117:0/4070092583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae38103110 msgr2=0x7fae3819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.916+0000 7fae3d200700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae38103110 0x7fae3819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:51.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.916+0000 7fae3d200700 1 -- 172.21.15.117:0/4070092583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae240095f0 con 0x7fae38104a00 2024-04-20T11:17:51.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.916+0000 7fae3d200700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 0x7fae3819ef90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae2400bd70 tx=0x7fae2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:51.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.917+0000 7fae36000700 1 -- 172.21.15.117:0/4070092583 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae2401d070 con 0x7fae38104a00 2024-04-20T11:17:51.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.917+0000 7fae36000700 1 -- 172.21.15.117:0/4070092583 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae2400f460 con 0x7fae38104a00 2024-04-20T11:17:51.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.917+0000 7fae36000700 1 -- 172.21.15.117:0/4070092583 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae24017630 con 0x7fae38104a00 2024-04-20T11:17:51.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.917+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae38198e10 con 0x7fae38104a00 2024-04-20T11:17:51.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.917+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae38199380 con 0x7fae38104a00 2024-04-20T11:17:51.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.918+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3810b480 con 0x7fae38104a00 2024-04-20T11:17:51.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.919+0000 7fae36000700 1 -- 172.21.15.117:0/4070092583 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fae2400f5d0 con 0x7fae38104a00 2024-04-20T11:17:51.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.920+0000 7fae36000700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae2006ecc0 0x7fae20071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:51.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.921+0000 7fae36000700 1 -- 172.21.15.117:0/4070092583 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fae24090210 con 0x7fae38104a00 2024-04-20T11:17:51.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.921+0000 7fae3dc00700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae2006ecc0 0x7fae20071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:51.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.921+0000 7fae3dc00700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae2006ecc0 0x7fae20071180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fae2c005fd0 tx=0x7fae2c00a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:51.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:51.925+0000 7fae36000700 1 -- 172.21.15.117:0/4070092583 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fae2405cc50 con 0x7fae38104a00 2024-04-20T11:17:52.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.157+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae38199fe0 con 0x7fae2006ecc0 2024-04-20T11:17:52.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.159+0000 7fae36000700 1 -- 172.21.15.117:0/4070092583 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fae38199fe0 con 0x7fae2006ecc0 2024-04-20T11:17:52.160 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:17:52.160 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:17:52.160 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:17:52.160 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:17:52.160 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:17:52.160 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:17:52.160 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:17:52.161 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:17:52.161 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:17:52.161 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "10/23 daemons upgraded", 2024-04-20T11:17:52.161 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading osd daemons", 2024-04-20T11:17:52.161 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:17:52.161 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:17:52.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.161+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae2006ecc0 msgr2=0x7fae20071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:52.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.161+0000 7fae40600700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae2006ecc0 0x7fae20071180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fae2c005fd0 tx=0x7fae2c00a560 comp rx=0 tx=0).stop 2024-04-20T11:17:52.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.161+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 msgr2=0x7fae3819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:52.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.161+0000 7fae40600700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 0x7fae3819ef90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae2400bd70 tx=0x7fae2400be50 comp rx=0 tx=0).stop 2024-04-20T11:17:52.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.162+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 shutdown_connections 2024-04-20T11:17:52.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.162+0000 7fae40600700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae2006ecc0 0x7fae20071180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.162+0000 7fae40600700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae38103110 0x7fae3819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.162+0000 7fae40600700 1 --2- 172.21.15.117:0/4070092583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae38104a00 0x7fae3819ef90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.162+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 >> 172.21.15.117:0/4070092583 conn(0x7fae380fe880 msgr2=0x7fae380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:52.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.162+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 shutdown_connections 2024-04-20T11:17:52.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.162+0000 7fae40600700 1 -- 172.21.15.117:0/4070092583 wait complete. 2024-04-20T11:17:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.304+0000 7fc304e00700 1 -- 172.21.15.117:0/27835734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 msgr2=0x7fc30010ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.304+0000 7fc304e00700 1 --2- 172.21.15.117:0/27835734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 0x7fc30010ab80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009910 tx=0x7fc2ec009c20 comp rx=0 tx=0).stop 2024-04-20T11:17:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.305+0000 7fc304e00700 1 -- 172.21.15.117:0/27835734 shutdown_connections 2024-04-20T11:17:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.305+0000 7fc304e00700 1 --2- 172.21.15.117:0/27835734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 0x7fc30010ab80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.305+0000 7fc304e00700 1 --2- 172.21.15.117:0/27835734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc300101ce0 0x7fc3001020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.305+0000 7fc304e00700 1 -- 172.21.15.117:0/27835734 >> 172.21.15.117:0/27835734 conn(0x7fc3000fb5a0 msgr2=0x7fc3000fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.305+0000 7fc304e00700 1 -- 172.21.15.117:0/27835734 shutdown_connections 2024-04-20T11:17:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.305+0000 7fc304e00700 1 -- 172.21.15.117:0/27835734 wait complete. 2024-04-20T11:17:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.306+0000 7fc304e00700 1 Processor -- start 2024-04-20T11:17:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.306+0000 7fc304e00700 1 -- start start 2024-04-20T11:17:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.306+0000 7fc304e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc300101ce0 0x7fc30019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.306+0000 7fc304e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 0x7fc30019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.306+0000 7fc304e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30019d360 con 0x7fc300102690 2024-04-20T11:17:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.306+0000 7fc304e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3001969d0 con 0x7fc300101ce0 2024-04-20T11:17:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.307+0000 7fc2fd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 0x7fc30019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.307+0000 7fc2fd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 0x7fc30019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56272/0 (socket says 172.21.15.117:56272) 2024-04-20T11:17:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.307+0000 7fc2fd400700 1 -- 172.21.15.117:0/384680315 learned_addr learned my addr 172.21.15.117:0/384680315 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.307+0000 7fc2fd400700 1 -- 172.21.15.117:0/384680315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc300101ce0 msgr2=0x7fc30019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.307+0000 7fc2fde00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc300101ce0 0x7fc30019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.307+0000 7fc2fd400700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc300101ce0 0x7fc30019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.307+0000 7fc2fd400700 1 -- 172.21.15.117:0/384680315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0095f0 con 0x7fc300102690 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.307+0000 7fc2fd400700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 0x7fc30019cdd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005b40 tx=0x7fc2ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.308+0000 7fc2f6a00700 1 -- 172.21.15.117:0/384680315 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2ec01d070 con 0x7fc300102690 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.308+0000 7fc2f6a00700 1 -- 172.21.15.117:0/384680315 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2ec00f460 con 0x7fc300102690 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.308+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc300196c50 con 0x7fc300102690 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.308+0000 7fc2f6a00700 1 -- 172.21.15.117:0/384680315 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2ec017630 con 0x7fc300102690 2024-04-20T11:17:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.308+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc300197140 con 0x7fc300102690 2024-04-20T11:17:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.309+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc300105490 con 0x7fc300102690 2024-04-20T11:17:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.310+0000 7fc2f6a00700 1 -- 172.21.15.117:0/384680315 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc2ec003a60 con 0x7fc300102690 2024-04-20T11:17:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.311+0000 7fc2f6a00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc2e406ecd0 0x7fc2e4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.311+0000 7fc2f6a00700 1 -- 172.21.15.117:0/384680315 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fc2ec090140 con 0x7fc300102690 2024-04-20T11:17:52.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.311+0000 7fc2fde00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc2e406ecd0 0x7fc2e4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:52.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.312+0000 7fc2fde00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc2e406ecd0 0x7fc2e4071190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000ba60 tx=0x7fc2f0005d50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:52.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.316+0000 7fc2f6a00700 1 -- 172.21.15.117:0/384680315 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc2ec05cb80 con 0x7fc300102690 2024-04-20T11:17:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:52 smithi173 ceph-mon[74165]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:52 smithi173 ceph-mon[74165]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:52 smithi173 ceph-mon[74165]: pgmap v147: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:52 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/720633219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:52 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2945839308' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:17:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.632+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc3000699a0 con 0x7fc300102690 2024-04-20T11:17:52.634 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:52 smithi117 ceph-mon[99036]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:52.634 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:52 smithi117 ceph-mon[99036]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:52.634 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:52 smithi117 ceph-mon[99036]: pgmap v147: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:52.634 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:52 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/720633219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:17:52.634 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:52 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2945839308' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:17:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.633+0000 7fc2f6a00700 1 -- 172.21.15.117:0/384680315 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc2ec05c710 con 0x7fc300102690 2024-04-20T11:17:52.635 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:17:52.635 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:17:52.635 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:17:52.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc2e406ecd0 msgr2=0x7fc2e4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:52.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc2e406ecd0 0x7fc2e4071190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000ba60 tx=0x7fc2f0005d50 comp rx=0 tx=0).stop 2024-04-20T11:17:52.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 msgr2=0x7fc30019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:52.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 0x7fc30019cdd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005b40 tx=0x7fc2ec00be20 comp rx=0 tx=0).stop 2024-04-20T11:17:52.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 shutdown_connections 2024-04-20T11:17:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc2e406ecd0 0x7fc2e4071190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc300101ce0 0x7fc30019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 --2- 172.21.15.117:0/384680315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300102690 0x7fc30019cdd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.637+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 >> 172.21.15.117:0/384680315 conn(0x7fc3000fb5a0 msgr2=0x7fc300103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.638+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 shutdown_connections 2024-04-20T11:17:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:17:52.638+0000 7fc304e00700 1 -- 172.21.15.117:0/384680315 wait complete. 2024-04-20T11:17:53.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:53 smithi117 ceph-mon[99036]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:53.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:53 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/384680315' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:17:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:53 smithi173 ceph-mon[74165]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:53 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/384680315' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:17:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:54 smithi117 ceph-mon[99036]: pgmap v148: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:54 smithi173 ceph-mon[74165]: pgmap v148: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:55.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:55 smithi117 ceph-mon[99036]: pgmap v149: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:55 smithi173 ceph-mon[74165]: pgmap v149: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:56 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-20T11:17:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:56 smithi117 ceph-mon[99036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-20T11:17:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:56 smithi117 ceph-mon[99036]: Upgrade: osd.4 is safe to restart 2024-04-20T11:17:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:56 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-20T11:17:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:56 smithi173 ceph-mon[74165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-20T11:17:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:56 smithi173 ceph-mon[74165]: Upgrade: osd.4 is safe to restart 2024-04-20T11:17:57.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:57 smithi173 ceph-mon[74165]: pgmap v150: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:57 smithi117 ceph-mon[99036]: pgmap v150: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:17:59.324 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:59 smithi173 ceph-mon[74165]: Upgrade: Updating osd.4 2024-04-20T11:17:59.324 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:59.324 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:17:59.324 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:59.325 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:17:59.325 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:59.325 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:17:59 smithi173 ceph-mon[74165]: Deploying daemon osd.4 on smithi173 2024-04-20T11:17:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:59 smithi117 ceph-mon[99036]: Upgrade: Updating osd.4 2024-04-20T11:17:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:17:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:17:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:17:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:17:59 smithi117 ceph-mon[99036]: Deploying daemon osd.4 on smithi173 2024-04-20T11:18:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:00 smithi173 ceph-mon[74165]: pgmap v151: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:00.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:00 smithi117 ceph-mon[99036]: pgmap v151: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:02.299 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:02 smithi173 ceph-mon[74165]: pgmap v152: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:02.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:02 smithi117 ceph-mon[99036]: pgmap v152: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:03.411 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:02 smithi173 systemd[1]: Stopping Ceph osd.4 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:18:03.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:03 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[36531]: 2024-04-20T11:18:03.418+0000 7f3cc5600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:18:03.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:03 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[36531]: 2024-04-20T11:18:03.419+0000 7f3cc5600700 -1 osd.4 70 *** Got signal Terminated *** 2024-04-20T11:18:03.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:03 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[36531]: 2024-04-20T11:18:03.419+0000 7f3cc5600700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:18:04.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:04 smithi117 ceph-mon[99036]: pgmap v153: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:04 smithi117 ceph-mon[99036]: osd.4 marked itself down and dead 2024-04-20T11:18:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:04 smithi173 ceph-mon[74165]: pgmap v153: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:04 smithi173 ceph-mon[74165]: osd.4 marked itself down and dead 2024-04-20T11:18:04.866 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:04 smithi173 podman[88144]: 2024-04-20 11:18:04.550901316 +0000 UTC m=+1.265041213 container died e08c85cecb503112ee088769f8e224e02d1ae83c1cc53b13df777986fcaf8f7d (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, RELEASE=reef-646b8c7, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True) 2024-04-20T11:18:05.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:05 smithi173 ceph-mon[74165]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:18:05.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:05 smithi173 ceph-mon[74165]: osdmap e71: 6 total, 5 up, 6 in 2024-04-20T11:18:05.161 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:04 smithi173 podman[88144]: 2024-04-20 11:18:04.994444607 +0000 UTC m=+1.708584525 container cleanup e08c85cecb503112ee088769f8e224e02d1ae83c1cc53b13df777986fcaf8f7d (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=reef-646b8c7, GIT_CLEAN=True) 2024-04-20T11:18:05.161 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:04 smithi173 bash[88144]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4 2024-04-20T11:18:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:05 smithi117 ceph-mon[99036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:18:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:05 smithi117 ceph-mon[99036]: osdmap e71: 6 total, 5 up, 6 in 2024-04-20T11:18:05.661 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:05 smithi173 podman[88163]: 2024-04-20 11:18:05.212473791 +0000 UTC m=+0.656263663 container remove e08c85cecb503112ee088769f8e224e02d1ae83c1cc53b13df777986fcaf8f7d (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-646b8c7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:18:06.341 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:06 smithi173 ceph-mon[74165]: pgmap v155: 65 pgs: 6 stale+active+clean, 59 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:06.341 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:06 smithi173 ceph-mon[74165]: osdmap e72: 6 total, 5 up, 6 in 2024-04-20T11:18:06.342 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:06 smithi173 podman[88250]: 2024-04-20 11:18:05.92716464 +0000 UTC m=+0.038078805 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:18:06.342 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:06 smithi173 podman[88250]: 2024-04-20 11:18:06.038369991 +0000 UTC m=+0.149284186 container create e75f3956f24bc0ae435d32737465ecf7f97d4e5d2656ad22ca4c6f971a14ac80 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-20T11:18:06.342 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:06 smithi173 podman[88250]: 2024-04-20 11:18:06.341744761 +0000 UTC m=+0.452658963 container init e75f3956f24bc0ae435d32737465ecf7f97d4e5d2656ad22ca4c6f971a14ac80 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-20T11:18:06.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:06 smithi117 ceph-mon[99036]: pgmap v155: 65 pgs: 6 stale+active+clean, 59 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:06 smithi117 ceph-mon[99036]: osdmap e72: 6 total, 5 up, 6 in 2024-04-20T11:18:06.661 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:06 smithi173 podman[88250]: 2024-04-20 11:18:06.355943751 +0000 UTC m=+0.466857980 container start e75f3956f24bc0ae435d32737465ecf7f97d4e5d2656ad22ca4c6f971a14ac80 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:18:06.661 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:06 smithi173 podman[88250]: 2024-04-20 11:18:06.449018286 +0000 UTC m=+0.559932521 container attach e75f3956f24bc0ae435d32737465ecf7f97d4e5d2656ad22ca4c6f971a14ac80 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-20T11:18:07.124 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:06 smithi173 podman[88250]: 2024-04-20 11:18:06.812021779 +0000 UTC m=+0.922935984 container died e75f3956f24bc0ae435d32737465ecf7f97d4e5d2656ad22ca4c6f971a14ac80 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-20T11:18:07.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:07 smithi117 ceph-mon[99036]: Health check failed: Degraded data redundancy: 38/324 objects degraded (11.728%), 9 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:07.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:07 smithi173 ceph-mon[74165]: Health check failed: Degraded data redundancy: 38/324 objects degraded (11.728%), 9 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:07.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:07 smithi173 podman[88276]: 2024-04-20 11:18:07.586045033 +0000 UTC m=+0.753098630 container remove e75f3956f24bc0ae435d32737465ecf7f97d4e5d2656ad22ca4c6f971a14ac80 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-20T11:18:07.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:07 smithi173 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-20T11:18:07.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:07 smithi173 systemd[1]: Stopped Ceph osd.4 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:18:07.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:07 smithi173 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.4.service: Consumed 1min 56.819s CPU time. 2024-04-20T11:18:08.375 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:08 smithi173 systemd[1]: Starting Ceph osd.4 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:18:08.375 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:08 smithi173 ceph-mon[74165]: pgmap v157: 65 pgs: 9 active+undersized+degraded, 3 stale+active+clean, 10 active+undersized, 43 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/324 objects degraded (11.728%) 2024-04-20T11:18:08.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:08 smithi117 ceph-mon[99036]: pgmap v157: 65 pgs: 9 active+undersized+degraded, 3 stale+active+clean, 10 active+undersized, 43 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/324 objects degraded (11.728%) 2024-04-20T11:18:08.641 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:08 smithi173 podman[88378]: 2024-04-20 11:18:08.283041359 +0000 UTC m=+0.037387124 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:18:08.641 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:08 smithi173 podman[88378]: 2024-04-20 11:18:08.386920944 +0000 UTC m=+0.141266715 container create 352651e9e124a11fa291152ebf2e55d3694d9ed04e550089d648750f7a2fbbae (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-20T11:18:08.641 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:08 smithi173 podman[88378]: 2024-04-20 11:18:08.640760859 +0000 UTC m=+0.395106640 container init 352651e9e124a11fa291152ebf2e55d3694d9ed04e550089d648750f7a2fbbae (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:18:08.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:08 smithi173 podman[88378]: 2024-04-20 11:18:08.651420664 +0000 UTC m=+0.405766433 container start 352651e9e124a11fa291152ebf2e55d3694d9ed04e550089d648750f7a2fbbae (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-20T11:18:08.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:08 smithi173 podman[88378]: 2024-04-20 11:18:08.739142362 +0000 UTC m=+0.493488139 container attach 352651e9e124a11fa291152ebf2e55d3694d9ed04e550089d648750f7a2fbbae (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-20T11:18:09.912 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 bash[88378]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate[88393]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-20T11:18:09.913 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:09 smithi173 podman[88378]: 2024-04-20 11:18:09.636739924 +0000 UTC m=+1.391085725 container died 352651e9e124a11fa291152ebf2e55d3694d9ed04e550089d648750f7a2fbbae (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:18:10.269 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:10 smithi173 ceph-mon[74165]: pgmap v158: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:10.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:10 smithi117 ceph-mon[99036]: pgmap v158: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:10.661 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:10 smithi173 podman[88533]: 2024-04-20 11:18:10.376453943 +0000 UTC m=+0.725984971 container remove 352651e9e124a11fa291152ebf2e55d3694d9ed04e550089d648750f7a2fbbae (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-activate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-20T11:18:11.072 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:10 smithi173 podman[88592]: 2024-04-20 11:18:10.645031238 +0000 UTC m=+0.033173890 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:18:11.072 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:10 smithi173 podman[88592]: 2024-04-20 11:18:10.760633024 +0000 UTC m=+0.148775713 container create 91012d4ad129120ee028046a91008e94d73a7e38e17a3855ff7bd84439dd103f (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2) 2024-04-20T11:18:11.072 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:11 smithi173 podman[88592]: 2024-04-20 11:18:11.072601824 +0000 UTC m=+0.460744519 container init 91012d4ad129120ee028046a91008e94d73a7e38e17a3855ff7bd84439dd103f (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-20T11:18:11.411 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:11 smithi173 podman[88592]: 2024-04-20 11:18:11.083316705 +0000 UTC m=+0.471459404 container start 91012d4ad129120ee028046a91008e94d73a7e38e17a3855ff7bd84439dd103f (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:18:11.411 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:11 smithi173 ceph-osd[88610]: -- 172.21.15.173:0/3421169956 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560000a25680 con 0x5600009ef000 2024-04-20T11:18:11.412 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:11 smithi173 bash[88592]: 91012d4ad129120ee028046a91008e94d73a7e38e17a3855ff7bd84439dd103f 2024-04-20T11:18:11.412 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:11 smithi173 systemd[1]: Started Ceph osd.4 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:18:11.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:11 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:18:11.658+0000 7f148d6467c0 -1 Falling back to public interface 2024-04-20T11:18:12.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:12 smithi117 ceph-mon[99036]: pgmap v159: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:12.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:12 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:12.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:12 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:12.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:12 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:12 smithi173 ceph-mon[74165]: pgmap v159: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:12 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:12 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:12 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:14.379 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:14 smithi173 ceph-mon[74165]: pgmap v160: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:14.379 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:14.379 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:18:14.379 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:14 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 55/324 objects degraded (16.975%), 15 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:14 smithi117 ceph-mon[99036]: pgmap v160: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:18:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:14 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 55/324 objects degraded (16.975%), 15 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:16 smithi117 ceph-mon[99036]: pgmap v161: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:16.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:16 smithi173 ceph-mon[74165]: pgmap v161: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:17.661 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:17 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:18:17.262+0000 7f148d6467c0 -1 osd.4 70 log_to_monitors true 2024-04-20T11:18:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:18 smithi117 ceph-mon[99036]: pgmap v162: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:18 smithi117 ceph-mon[99036]: from='osd.4 [v2:172.21.15.173:6808/3480065294,v1:172.21.15.173:6809/3480065294]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:18:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:18 smithi117 ceph-mon[99036]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:18:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:18 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:18.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:18 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:18.410 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:18:18 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:18:18.078+0000 7f147ca00700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:18:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:18 smithi173 ceph-mon[74165]: pgmap v162: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:18 smithi173 ceph-mon[74165]: from='osd.4 [v2:172.21.15.173:6808/3480065294,v1:172.21.15.173:6809/3480065294]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:18:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:18 smithi173 ceph-mon[74165]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:18:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:18 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:18 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:19.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:19 smithi173 ceph-mon[74165]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T11:18:19.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:19 smithi173 ceph-mon[74165]: osdmap e73: 6 total, 5 up, 6 in 2024-04-20T11:18:19.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:19 smithi173 ceph-mon[74165]: from='osd.4 [v2:172.21.15.173:6808/3480065294,v1:172.21.15.173:6809/3480065294]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:18:19.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:19 smithi173 ceph-mon[74165]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:18:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:19 smithi117 ceph-mon[99036]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T11:18:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:19 smithi117 ceph-mon[99036]: osdmap e73: 6 total, 5 up, 6 in 2024-04-20T11:18:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:19 smithi117 ceph-mon[99036]: from='osd.4 [v2:172.21.15.173:6808/3480065294,v1:172.21.15.173:6809/3480065294]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:18:19.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:19 smithi117 ceph-mon[99036]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:18:20.338 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:20 smithi173 ceph-mon[74165]: pgmap v164: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:20.338 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:20 smithi173 ceph-mon[74165]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:18:20.338 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:20 smithi173 ceph-mon[74165]: osd.4 [v2:172.21.15.173:6808/3480065294,v1:172.21.15.173:6809/3480065294] boot 2024-04-20T11:18:20.339 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:20 smithi173 ceph-mon[74165]: osdmap e74: 6 total, 6 up, 6 in 2024-04-20T11:18:20.339 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:20 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:18:20.339 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:20 smithi173 ceph-mon[74165]: osdmap e75: 6 total, 6 up, 6 in 2024-04-20T11:18:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:20 smithi117 ceph-mon[99036]: pgmap v164: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/324 objects degraded (16.975%) 2024-04-20T11:18:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:20 smithi117 ceph-mon[99036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:18:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:20 smithi117 ceph-mon[99036]: osd.4 [v2:172.21.15.173:6808/3480065294,v1:172.21.15.173:6809/3480065294] boot 2024-04-20T11:18:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:20 smithi117 ceph-mon[99036]: osdmap e74: 6 total, 6 up, 6 in 2024-04-20T11:18:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:20 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:18:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:20 smithi117 ceph-mon[99036]: osdmap e75: 6 total, 6 up, 6 in 2024-04-20T11:18:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:21 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 29/324 objects degraded (8.951%), 10 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:21.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:21 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 29/324 objects degraded (8.951%), 10 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:22.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:22 smithi117 ceph-mon[99036]: pgmap v167: 65 pgs: 10 active+undersized+degraded, 14 active+undersized, 41 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29/324 objects degraded (8.951%) 2024-04-20T11:18:22.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:22 smithi173 ceph-mon[74165]: pgmap v167: 65 pgs: 10 active+undersized+degraded, 14 active+undersized, 41 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29/324 objects degraded (8.951%) 2024-04-20T11:18:22.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.811+0000 7faad7e00700 1 -- 172.21.15.117:0/1839218724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073310 msgr2=0x7faad00736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:22.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.811+0000 7faad7e00700 1 --2- 172.21.15.117:0/1839218724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073310 0x7faad00736f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faac4009910 tx=0x7faac4009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:22.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.811+0000 7faad7e00700 1 -- 172.21.15.117:0/1839218724 shutdown_connections 2024-04-20T11:18:22.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.811+0000 7faad7e00700 1 --2- 172.21.15.117:0/1839218724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad0073c30 0x7faad010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:22.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.811+0000 7faad7e00700 1 --2- 172.21.15.117:0/1839218724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073310 0x7faad00736f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:22.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.811+0000 7faad7e00700 1 -- 172.21.15.117:0/1839218724 >> 172.21.15.117:0/1839218724 conn(0x7faad00fc6a0 msgr2=0x7faad00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:22.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.811+0000 7faad7e00700 1 -- 172.21.15.117:0/1839218724 shutdown_connections 2024-04-20T11:18:22.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.811+0000 7faad7e00700 1 -- 172.21.15.117:0/1839218724 wait complete. 2024-04-20T11:18:22.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.813+0000 7faad7e00700 1 Processor -- start 2024-04-20T11:18:22.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.813+0000 7faad7e00700 1 -- start start 2024-04-20T11:18:22.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.813+0000 7faad7e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad0073310 0x7faad0198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:22.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.813+0000 7faad7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073c30 0x7faad0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:22.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.813+0000 7faad7e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad0199a20 con 0x7faad0073c30 2024-04-20T11:18:22.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.813+0000 7faad7e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad019d6f0 con 0x7faad0073310 2024-04-20T11:18:22.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.814+0000 7faad4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073c30 0x7faad0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:22.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.814+0000 7faad4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073c30 0x7faad0199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44978/0 (socket says 172.21.15.117:44978) 2024-04-20T11:18:22.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.814+0000 7faad4a00700 1 -- 172.21.15.117:0/3725261346 learned_addr learned my addr 172.21.15.117:0/3725261346 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:22.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.814+0000 7faad5400700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad0073310 0x7faad0198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.814+0000 7faad4a00700 1 -- 172.21.15.117:0/3725261346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad0073310 msgr2=0x7faad0198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.814+0000 7faad4a00700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad0073310 0x7faad0198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.814+0000 7faad4a00700 1 -- 172.21.15.117:0/3725261346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40095f0 con 0x7faad0073c30 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.815+0000 7faad4a00700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073c30 0x7faad0199340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faabc00b4b0 tx=0x7faabc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.815+0000 7faace000700 1 -- 172.21.15.117:0/3725261346 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faabc011800 con 0x7faad0073c30 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.815+0000 7faace000700 1 -- 172.21.15.117:0/3725261346 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faabc011e20 con 0x7faad0073c30 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.815+0000 7faace000700 1 -- 172.21.15.117:0/3725261346 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faabc00f2e0 con 0x7faad0073c30 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.815+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad019d9d0 con 0x7faad0073c30 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.815+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad019ded0 con 0x7faad0073c30 2024-04-20T11:18:22.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.816+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad00749c0 con 0x7faad0073c30 2024-04-20T11:18:22.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.817+0000 7faace000700 1 -- 172.21.15.117:0/3725261346 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7faabc010380 con 0x7faad0073c30 2024-04-20T11:18:22.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.818+0000 7faace000700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:22.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.818+0000 7faace000700 1 -- 172.21.15.117:0/3725261346 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6285+0+0 (secure 0 0 0) 0x7faabc08e940 con 0x7faad0073c30 2024-04-20T11:18:22.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.819+0000 7faad5400700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:22.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.819+0000 7faad5400700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faac4006010 tx=0x7faac401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:22.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:22.823+0000 7faace000700 1 -- 172.21.15.117:0/3725261346 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7faabc057730 con 0x7faad0073c30 2024-04-20T11:18:23.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.061+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faad0107820 con 0x7faab806ecd0 2024-04-20T11:18:23.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.066+0000 7faace000700 1 -- 172.21.15.117:0/3725261346 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7faad0107820 con 0x7faab806ecd0 2024-04-20T11:18:23.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 msgr2=0x7faab8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faac4006010 tx=0x7faac401a040 comp rx=0 tx=0).stop 2024-04-20T11:18:23.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073c30 msgr2=0x7faad0199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073c30 0x7faad0199340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faabc00b4b0 tx=0x7faabc00b7c0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 shutdown_connections 2024-04-20T11:18:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad0073310 0x7faad0198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 --2- 172.21.15.117:0/3725261346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad0073c30 0x7faad0199340 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 >> 172.21.15.117:0/3725261346 conn(0x7faad00fc6a0 msgr2=0x7faad0105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 shutdown_connections 2024-04-20T11:18:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.069+0000 7faad7e00700 1 -- 172.21.15.117:0/3725261346 wait complete. 2024-04-20T11:18:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:18:23.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.226+0000 7f5cf3800700 1 -- 172.21.15.117:0/474600689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 msgr2=0x7f5cec107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.226+0000 7f5cf3800700 1 --2- 172.21.15.117:0/474600689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 0x7f5cec107a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009910 tx=0x7f5cd8009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:23.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.226+0000 7f5cf3800700 1 -- 172.21.15.117:0/474600689 shutdown_connections 2024-04-20T11:18:23.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.226+0000 7f5cf3800700 1 --2- 172.21.15.117:0/474600689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 0x7f5cec107a20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.226+0000 7f5cf3800700 1 --2- 172.21.15.117:0/474600689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec103020 0x7f5cec103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.226+0000 7f5cf3800700 1 -- 172.21.15.117:0/474600689 >> 172.21.15.117:0/474600689 conn(0x7f5cec0fe8b0 msgr2=0x7f5cec100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:23.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.226+0000 7f5cf3800700 1 -- 172.21.15.117:0/474600689 shutdown_connections 2024-04-20T11:18:23.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.226+0000 7f5cf3800700 1 -- 172.21.15.117:0/474600689 wait complete. 2024-04-20T11:18:23.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.227+0000 7f5cf3800700 1 Processor -- start 2024-04-20T11:18:23.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.227+0000 7f5cf3800700 1 -- start start 2024-04-20T11:18:23.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.227+0000 7f5cf3800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec103020 0x7f5cec198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:23.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.227+0000 7f5cf3800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 0x7f5cec1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:23.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.227+0000 7f5cf3800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1999b0 con 0x7f5cec1039d0 2024-04-20T11:18:23.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.227+0000 7f5cf3800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec19d680 con 0x7f5cec103020 2024-04-20T11:18:23.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.228+0000 7f5cebe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 0x7f5cec1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:23.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.228+0000 7f5cebe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 0x7f5cec1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44988/0 (socket says 172.21.15.117:44988) 2024-04-20T11:18:23.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.228+0000 7f5cebe00700 1 -- 172.21.15.117:0/2034429925 learned_addr learned my addr 172.21.15.117:0/2034429925 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:23.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.228+0000 7f5cf0e00700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec103020 0x7f5cec198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:23.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.228+0000 7f5cebe00700 1 -- 172.21.15.117:0/2034429925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec103020 msgr2=0x7f5cec198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.228+0000 7f5cebe00700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec103020 0x7f5cec198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.228+0000 7f5cebe00700 1 -- 172.21.15.117:0/2034429925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80095f0 con 0x7f5cec1039d0 2024-04-20T11:18:23.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.229+0000 7f5cebe00700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 0x7f5cec1992d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80092e0 tx=0x7f5cd800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:23.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.229+0000 7f5ce9600700 1 -- 172.21.15.117:0/2034429925 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd801d070 con 0x7f5cec1039d0 2024-04-20T11:18:23.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.229+0000 7f5ce9600700 1 -- 172.21.15.117:0/2034429925 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cd800f460 con 0x7f5cec1039d0 2024-04-20T11:18:23.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.229+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec19d900 con 0x7f5cec1039d0 2024-04-20T11:18:23.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.229+0000 7f5ce9600700 1 -- 172.21.15.117:0/2034429925 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd8017630 con 0x7f5cec1039d0 2024-04-20T11:18:23.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.229+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec19ddf0 con 0x7f5cec1039d0 2024-04-20T11:18:23.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.230+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec10b370 con 0x7f5cec1039d0 2024-04-20T11:18:23.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.231+0000 7f5ce9600700 1 -- 172.21.15.117:0/2034429925 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5cd800f5d0 con 0x7f5cec1039d0 2024-04-20T11:18:23.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.232+0000 7f5ce9600700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cd406ecc0 0x7f5cd4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:23.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.232+0000 7f5ce9600700 1 -- 172.21.15.117:0/2034429925 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f5cd80902a0 con 0x7f5cec1039d0 2024-04-20T11:18:23.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.233+0000 7f5cf0e00700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cd406ecc0 0x7f5cd4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:23.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.233+0000 7f5cf0e00700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cd406ecc0 0x7f5cd4071180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0006fd0 tx=0x7f5ce0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:23.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.238+0000 7f5ce9600700 1 -- 172.21.15.117:0/2034429925 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5cd8059090 con 0x7f5cec1039d0 2024-04-20T11:18:23.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.480+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cec110040 con 0x7f5cd406ecc0 2024-04-20T11:18:23.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.485+0000 7f5ce9600700 1 -- 172.21.15.117:0/2034429925 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5cec110040 con 0x7f5cd406ecc0 2024-04-20T11:18:23.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cd406ecc0 msgr2=0x7f5cd4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cd406ecc0 0x7f5cd4071180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0006fd0 tx=0x7f5ce0008040 comp rx=0 tx=0).stop 2024-04-20T11:18:23.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 msgr2=0x7f5cec1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 0x7f5cec1992d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80092e0 tx=0x7f5cd800be20 comp rx=0 tx=0).stop 2024-04-20T11:18:23.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 shutdown_connections 2024-04-20T11:18:23.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cd406ecc0 0x7f5cd4071180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec103020 0x7f5cec198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 --2- 172.21.15.117:0/2034429925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cec1039d0 0x7f5cec1992d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.488+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 >> 172.21.15.117:0/2034429925 conn(0x7f5cec0fe8b0 msgr2=0x7f5cec0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:23.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.489+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 shutdown_connections 2024-04-20T11:18:23.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.489+0000 7f5cf3800700 1 -- 172.21.15.117:0/2034429925 wait complete. 2024-04-20T11:18:23.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.644+0000 7fc7d0600700 1 -- 172.21.15.117:0/1502503472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c81026e0 msgr2=0x7fc7c810abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.644+0000 7fc7d0600700 1 --2- 172.21.15.117:0/1502503472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c81026e0 0x7fc7c810abd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009940 tx=0x7fc7b4009c50 comp rx=0 tx=0).stop 2024-04-20T11:18:23.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.644+0000 7fc7d0600700 1 -- 172.21.15.117:0/1502503472 shutdown_connections 2024-04-20T11:18:23.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.644+0000 7fc7d0600700 1 --2- 172.21.15.117:0/1502503472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c81026e0 0x7fc7c810abd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.644+0000 7fc7d0600700 1 --2- 172.21.15.117:0/1502503472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c8101dc0 0x7fc7c81021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.644+0000 7fc7d0600700 1 -- 172.21.15.117:0/1502503472 >> 172.21.15.117:0/1502503472 conn(0x7fc7c8076750 msgr2=0x7fc7c80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:23.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.645+0000 7fc7d0600700 1 -- 172.21.15.117:0/1502503472 shutdown_connections 2024-04-20T11:18:23.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.645+0000 7fc7d0600700 1 -- 172.21.15.117:0/1502503472 wait complete. 2024-04-20T11:18:23.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7d0600700 1 Processor -- start 2024-04-20T11:18:23.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7d0600700 1 -- start start 2024-04-20T11:18:23.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7d0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c8101dc0 0x7fc7c80ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:23.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7d0600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c81026e0 0x7fc7c80fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:23.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7d0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81019f0 con 0x7fc7c8101dc0 2024-04-20T11:18:23.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7d0600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81004a0 con 0x7fc7c81026e0 2024-04-20T11:18:23.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7cdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c8101dc0 0x7fc7c80ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:23.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7cdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c8101dc0 0x7fc7c80ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45010/0 (socket says 172.21.15.117:45010) 2024-04-20T11:18:23.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.646+0000 7fc7cdc00700 1 -- 172.21.15.117:0/1654590693 learned_addr learned my addr 172.21.15.117:0/1654590693 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:23.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.647+0000 7fc7cdc00700 1 -- 172.21.15.117:0/1654590693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c81026e0 msgr2=0x7fc7c80fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.647+0000 7fc7cdc00700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c81026e0 0x7fc7c80fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.647+0000 7fc7cdc00700 1 -- 172.21.15.117:0/1654590693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b40095f0 con 0x7fc7c8101dc0 2024-04-20T11:18:23.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.647+0000 7fc7cdc00700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c8101dc0 0x7fc7c80ffa20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c930 tx=0x7fc7bc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:23.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.647+0000 7fc7c6000700 1 -- 172.21.15.117:0/1654590693 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7bc007ad0 con 0x7fc7c8101dc0 2024-04-20T11:18:23.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.647+0000 7fc7c6000700 1 -- 172.21.15.117:0/1654590693 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7bc007c30 con 0x7fc7c8101dc0 2024-04-20T11:18:23.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.647+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c8100750 con 0x7fc7c8101dc0 2024-04-20T11:18:23.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.648+0000 7fc7c6000700 1 -- 172.21.15.117:0/1654590693 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7bc018610 con 0x7fc7c8101dc0 2024-04-20T11:18:23.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.648+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81a2fa0 con 0x7fc7c8101dc0 2024-04-20T11:18:23.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.649+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c8068dd0 con 0x7fc7c8101dc0 2024-04-20T11:18:23.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.650+0000 7fc7c6000700 1 -- 172.21.15.117:0/1654590693 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc7bc00f450 con 0x7fc7c8101dc0 2024-04-20T11:18:23.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.651+0000 7fc7c6000700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7b006ecc0 0x7fc7b0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:23.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.651+0000 7fc7c6000700 1 -- 172.21.15.117:0/1654590693 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fc7bc08ef70 con 0x7fc7c8101dc0 2024-04-20T11:18:23.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.651+0000 7fc7cd200700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7b006ecc0 0x7fc7b0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:23.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.652+0000 7fc7cd200700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7b006ecc0 0x7fc7b0071180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4006010 tx=0x7fc7b4000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.656+0000 7fc7c6000700 1 -- 172.21.15.117:0/1654590693 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc7bc057de0 con 0x7fc7c8101dc0 2024-04-20T11:18:23.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.894+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7c810dda0 con 0x7fc7b006ecc0 2024-04-20T11:18:23.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.917+0000 7fc7c6000700 1 -- 172.21.15.117:0/1654590693 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fc7c810dda0 con 0x7fc7b006ecc0 2024-04-20T11:18:23.917 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:18:23.917 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (6m) 79s ago 20m 14.2M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:18:23.917 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (21m) 79s ago 21m 9017k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:18:23.917 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (19m) 8s ago 19m 8724k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:18:23.917 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (3m) 79s ago 21m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:18:23.917 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (3m) 8s ago 19m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:18:23.917 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (6m) 79s ago 20m 74.3M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:18:23.917 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (13m) 79s ago 13m 18.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (13m) 79s ago 13m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (13m) 8s ago 13m 18.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (13m) 8s ago 13m 25.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (10m) 79s ago 22m 506M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (8m) 8s ago 19m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (4m) 79s ago 22m 51.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (4m) 8s ago 19m 36.0M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (8m) 79s ago 21m 7872k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (8m) 8s ago 19m 8342k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (2m) 79s ago 18m 80.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (118s) 79s ago 17m 82.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (87s) 79s ago 17m 11.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (55s) 8s ago 16m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:18:23.918 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (13s) 8s ago 16m 11.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:18:23.919 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (15m) 8s ago 15m 204M 2240M 18.2.2-1237-g646b8c7d 9900ed1a6b81 e34232ef8172 2024-04-20T11:18:23.919 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (7m) 79s ago 20m 69.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:18:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7b006ecc0 msgr2=0x7fc7b0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7b006ecc0 0x7fc7b0071180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4006010 tx=0x7fc7b4000bc0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c8101dc0 msgr2=0x7fc7c80ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c8101dc0 0x7fc7c80ffa20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c930 tx=0x7fc7bc00ccf0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 shutdown_connections 2024-04-20T11:18:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7b006ecc0 0x7fc7b0071180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7c8101dc0 0x7fc7c80ffa20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 --2- 172.21.15.117:0/1654590693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c81026e0 0x7fc7c80fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:23.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 >> 172.21.15.117:0/1654590693 conn(0x7fc7c8076750 msgr2=0x7fc7c81030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:23.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.920+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 shutdown_connections 2024-04-20T11:18:23.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:23.921+0000 7fc7d0600700 1 -- 172.21.15.117:0/1654590693 wait complete. 2024-04-20T11:18:24.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.091+0000 7f7ba6400700 1 -- 172.21.15.117:0/3059663143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba0073bd0 msgr2=0x7f7ba010cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:24.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.091+0000 7f7ba6400700 1 --2- 172.21.15.117:0/3059663143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba0073bd0 0x7f7ba010cf20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009910 tx=0x7f7b8c009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:24.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.091+0000 7f7ba6400700 1 -- 172.21.15.117:0/3059663143 shutdown_connections 2024-04-20T11:18:24.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.091+0000 7f7ba6400700 1 --2- 172.21.15.117:0/3059663143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba0073bd0 0x7f7ba010cf20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.091+0000 7f7ba6400700 1 --2- 172.21.15.117:0/3059663143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba00732b0 0x7f7ba0073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.091+0000 7f7ba6400700 1 -- 172.21.15.117:0/3059663143 >> 172.21.15.117:0/3059663143 conn(0x7f7ba00fc6b0 msgr2=0x7f7ba00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:24.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.091+0000 7f7ba6400700 1 -- 172.21.15.117:0/3059663143 shutdown_connections 2024-04-20T11:18:24.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.091+0000 7f7ba6400700 1 -- 172.21.15.117:0/3059663143 wait complete. 2024-04-20T11:18:24.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.093+0000 7f7ba6400700 1 Processor -- start 2024-04-20T11:18:24.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.093+0000 7f7ba6400700 1 -- start start 2024-04-20T11:18:24.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.093+0000 7f7ba6400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba00732b0 0x7f7ba0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.093+0000 7f7ba6400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba0073bd0 0x7f7ba0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.093+0000 7f7ba6400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba01998e0 con 0x7f7ba00732b0 2024-04-20T11:18:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.093+0000 7f7ba6400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba019c550 con 0x7f7ba0073bd0 2024-04-20T11:18:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.094+0000 7f7b9f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba00732b0 0x7f7ba0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.094+0000 7f7b9f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba00732b0 0x7f7ba0198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39076/0 (socket says 172.21.15.117:39076) 2024-04-20T11:18:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.094+0000 7f7b9f400700 1 -- 172.21.15.117:0/1534888210 learned_addr learned my addr 172.21.15.117:0/1534888210 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:24.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.094+0000 7f7b9ea00700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba0073bd0 0x7f7ba0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:24.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.094+0000 7f7b9f400700 1 -- 172.21.15.117:0/1534888210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba0073bd0 msgr2=0x7f7ba0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:24.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.094+0000 7f7b9f400700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba0073bd0 0x7f7ba0199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.094+0000 7f7b9f400700 1 -- 172.21.15.117:0/1534888210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0095f0 con 0x7f7ba00732b0 2024-04-20T11:18:24.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.095+0000 7f7b9f400700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba00732b0 0x7f7ba0198d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000c8a0 tx=0x7f7b9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:24.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.095+0000 7f7b97e00700 1 -- 172.21.15.117:0/1534888210 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b9000cea0 con 0x7f7ba00732b0 2024-04-20T11:18:24.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.095+0000 7f7b97e00700 1 -- 172.21.15.117:0/1534888210 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b90004830 con 0x7f7ba00732b0 2024-04-20T11:18:24.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.095+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba019c830 con 0x7f7ba00732b0 2024-04-20T11:18:24.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.095+0000 7f7b97e00700 1 -- 172.21.15.117:0/1534888210 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b900055b0 con 0x7f7ba00732b0 2024-04-20T11:18:24.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.095+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba019cd80 con 0x7f7ba00732b0 2024-04-20T11:18:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.097+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba0074960 con 0x7f7ba00732b0 2024-04-20T11:18:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.098+0000 7f7b97e00700 1 -- 172.21.15.117:0/1534888210 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7b90007ad0 con 0x7f7ba00732b0 2024-04-20T11:18:24.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.098+0000 7f7b97e00700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b8406ecc0 0x7f7b84071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:24.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.099+0000 7f7b97e00700 1 -- 172.21.15.117:0/1534888210 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f7b9008eaa0 con 0x7f7ba00732b0 2024-04-20T11:18:24.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.099+0000 7f7b9ea00700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b8406ecc0 0x7f7b84071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.100+0000 7f7b9ea00700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b8406ecc0 0x7f7b84071180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ba019a230 tx=0x7f7b8c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:24.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.104+0000 7f7b97e00700 1 -- 172.21.15.117:0/1534888210 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7b90057930 con 0x7f7ba00732b0 2024-04-20T11:18:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: pgmap v168: 65 pgs: 3 active+undersized+degraded, 7 active+undersized, 55 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 10/324 objects degraded (3.086%) 2024-04-20T11:18:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:24.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:24 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-20T11:18:24.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.429+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7ba0107810 con 0x7f7ba00732b0 2024-04-20T11:18:24.429 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: pgmap v168: 65 pgs: 3 active+undersized+degraded, 7 active+undersized, 55 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 10/324 objects degraded (3.086%) 2024-04-20T11:18:24.429 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:24.430 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:24 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.430+0000 7f7b97e00700 1 -- 172.21.15.117:0/1534888210 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f7b9005aff0 con 0x7f7ba00732b0 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 5 2024-04-20T11:18:24.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:18:24.432 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:18:24.432 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:18:24.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:18:24.432 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:18:24.432 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 5, 2024-04-20T11:18:24.432 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 9 2024-04-20T11:18:24.432 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:18:24.432 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:18:24.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b8406ecc0 msgr2=0x7f7b84071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:24.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b8406ecc0 0x7f7b84071180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ba019a230 tx=0x7f7b8c01a040 comp rx=0 tx=0).stop 2024-04-20T11:18:24.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba00732b0 msgr2=0x7f7ba0198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:24.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba00732b0 0x7f7ba0198d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000c8a0 tx=0x7f7b9000cc60 comp rx=0 tx=0).stop 2024-04-20T11:18:24.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 shutdown_connections 2024-04-20T11:18:24.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b8406ecc0 0x7f7b84071180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ba00732b0 0x7f7ba0198d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 --2- 172.21.15.117:0/1534888210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba0073bd0 0x7f7ba0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 >> 172.21.15.117:0/1534888210 conn(0x7f7ba00fc6b0 msgr2=0x7f7ba0105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:24.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 shutdown_connections 2024-04-20T11:18:24.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.433+0000 7f7ba6400700 1 -- 172.21.15.117:0/1534888210 wait complete. 2024-04-20T11:18:24.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.589+0000 7ffba2000700 1 -- 172.21.15.117:0/1413698368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c102730 msgr2=0x7ffb9c10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.589+0000 7ffba2000700 1 --2- 172.21.15.117:0/1413698368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c102730 0x7ffb9c10ac20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009910 tx=0x7ffb84009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.589+0000 7ffba2000700 1 -- 172.21.15.117:0/1413698368 shutdown_connections 2024-04-20T11:18:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.589+0000 7ffba2000700 1 --2- 172.21.15.117:0/1413698368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c102730 0x7ffb9c10ac20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.589+0000 7ffba2000700 1 --2- 172.21.15.117:0/1413698368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c101e10 0x7ffb9c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.589+0000 7ffba2000700 1 -- 172.21.15.117:0/1413698368 >> 172.21.15.117:0/1413698368 conn(0x7ffb9c0fb5c0 msgr2=0x7ffb9c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.589+0000 7ffba2000700 1 -- 172.21.15.117:0/1413698368 shutdown_connections 2024-04-20T11:18:24.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.589+0000 7ffba2000700 1 -- 172.21.15.117:0/1413698368 wait complete. 2024-04-20T11:18:24.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.590+0000 7ffba2000700 1 Processor -- start 2024-04-20T11:18:24.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.590+0000 7ffba2000700 1 -- start start 2024-04-20T11:18:24.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.591+0000 7ffba2000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c101e10 0x7ffb9c199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:24.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.591+0000 7ffba2000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c102730 0x7ffb9c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:24.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.591+0000 7ffba2000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c199d60 con 0x7ffb9c101e10 2024-04-20T11:18:24.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.591+0000 7ffba2000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c19da30 con 0x7ffb9c102730 2024-04-20T11:18:24.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.591+0000 7ffb9b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c101e10 0x7ffb9c199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:24.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.591+0000 7ffb9b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c101e10 0x7ffb9c199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39092/0 (socket says 172.21.15.117:39092) 2024-04-20T11:18:24.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.591+0000 7ffb9b400700 1 -- 172.21.15.117:0/3376739123 learned_addr learned my addr 172.21.15.117:0/3376739123 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:24.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffb92a00700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c102730 0x7ffb9c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:24.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffb9b400700 1 -- 172.21.15.117:0/3376739123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c102730 msgr2=0x7ffb9c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:24.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffb9b400700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c102730 0x7ffb9c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffb9b400700 1 -- 172.21.15.117:0/3376739123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb840095f0 con 0x7ffb9c101e10 2024-04-20T11:18:24.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffb9b400700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c101e10 0x7ffb9c199140 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00b4b0 tx=0x7ffb8c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:24.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffb98c00700 1 -- 172.21.15.117:0/3376739123 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb8c011800 con 0x7ffb9c101e10 2024-04-20T11:18:24.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffb98c00700 1 -- 172.21.15.117:0/3376739123 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb8c011e20 con 0x7ffb9c101e10 2024-04-20T11:18:24.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c19dd10 con 0x7ffb9c101e10 2024-04-20T11:18:24.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffb98c00700 1 -- 172.21.15.117:0/3376739123 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb8c00f2e0 con 0x7ffb9c101e10 2024-04-20T11:18:24.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.592+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c19e260 con 0x7ffb9c101e10 2024-04-20T11:18:24.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.594+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c069660 con 0x7ffb9c101e10 2024-04-20T11:18:24.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.595+0000 7ffb98c00700 1 -- 172.21.15.117:0/3376739123 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ffb8c010380 con 0x7ffb9c101e10 2024-04-20T11:18:24.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.596+0000 7ffb98c00700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffb8006ecc0 0x7ffb80071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:24.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.596+0000 7ffb98c00700 1 -- 172.21.15.117:0/3376739123 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6285+0+0 (secure 0 0 0) 0x7ffb8c08e920 con 0x7ffb9c101e10 2024-04-20T11:18:24.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.597+0000 7ffb92a00700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffb8006ecc0 0x7ffb80071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:24.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.597+0000 7ffb92a00700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffb8006ecc0 0x7ffb80071180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c19a6a0 tx=0x7ffb8401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:24.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.601+0000 7ffb98c00700 1 -- 172.21.15.117:0/3376739123 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ffb8c0577b0 con 0x7ffb9c101e10 2024-04-20T11:18:24.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.869+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffb9c06a6a0 con 0x7ffb9c101e10 2024-04-20T11:18:24.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.870+0000 7ffb98c00700 1 -- 172.21.15.117:0/3376739123 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7ffb8c05ae70 con 0x7ffb9c101e10 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:18:24.870 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:18:24.871 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:18:24.872 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:18:24.873 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:18:24.873 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffb8006ecc0 msgr2=0x7ffb80071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffb8006ecc0 0x7ffb80071180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c19a6a0 tx=0x7ffb8401a040 comp rx=0 tx=0).stop 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c101e10 msgr2=0x7ffb9c199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c101e10 0x7ffb9c199140 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00b4b0 tx=0x7ffb8c00b870 comp rx=0 tx=0).stop 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 shutdown_connections 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ffb8006ecc0 0x7ffb80071180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb9c101e10 0x7ffb9c199140 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 --2- 172.21.15.117:0/3376739123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c102730 0x7ffb9c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:24.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.874+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 >> 172.21.15.117:0/3376739123 conn(0x7ffb9c0fb5c0 msgr2=0x7ffb9c0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:24.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.875+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 shutdown_connections 2024-04-20T11:18:24.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:24.875+0000 7ffba2000700 1 -- 172.21.15.117:0/3376739123 wait complete. 2024-04-20T11:18:24.875 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:18:25.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.026+0000 7fa40f600700 1 -- 172.21.15.117:0/996977983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 msgr2=0x7fa408109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:25.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.026+0000 7fa40f600700 1 --2- 172.21.15.117:0/996977983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 0x7fa408109290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009910 tx=0x7fa3f4009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:25.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.026+0000 7fa40f600700 1 -- 172.21.15.117:0/996977983 shutdown_connections 2024-04-20T11:18:25.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.026+0000 7fa40f600700 1 --2- 172.21.15.117:0/996977983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 0x7fa408109290 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.026+0000 7fa40f600700 1 --2- 172.21.15.117:0/996977983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408107520 0x7fa408107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.026+0000 7fa40f600700 1 -- 172.21.15.117:0/996977983 >> 172.21.15.117:0/996977983 conn(0x7fa4080768f0 msgr2=0x7fa408078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:25.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.027+0000 7fa40f600700 1 -- 172.21.15.117:0/996977983 shutdown_connections 2024-04-20T11:18:25.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.027+0000 7fa40f600700 1 -- 172.21.15.117:0/996977983 wait complete. 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.028+0000 7fa40f600700 1 Processor -- start 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.028+0000 7fa40f600700 1 -- start start 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.028+0000 7fa40f600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408107520 0x7fa40819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.028+0000 7fa40f600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 0x7fa40819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.028+0000 7fa40f600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40819ded0 con 0x7fa408108e10 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.028+0000 7fa40f600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4081a1ba0 con 0x7fa408107520 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.029+0000 7fa407e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 0x7fa40819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.029+0000 7fa407e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 0x7fa40819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39114/0 (socket says 172.21.15.117:39114) 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.029+0000 7fa407e00700 1 -- 172.21.15.117:0/2717901404 learned_addr learned my addr 172.21.15.117:0/2717901404 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.029+0000 7fa40cc00700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408107520 0x7fa40819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.029+0000 7fa407e00700 1 -- 172.21.15.117:0/2717901404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408107520 msgr2=0x7fa40819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.029+0000 7fa407e00700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408107520 0x7fa40819d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.029+0000 7fa407e00700 1 -- 172.21.15.117:0/2717901404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40095f0 con 0x7fa408108e10 2024-04-20T11:18:25.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.029+0000 7fa407e00700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 0x7fa40819d7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40092e0 tx=0x7fa3f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:25.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.030+0000 7fa405600700 1 -- 172.21.15.117:0/2717901404 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3f401d070 con 0x7fa408108e10 2024-04-20T11:18:25.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.030+0000 7fa405600700 1 -- 172.21.15.117:0/2717901404 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3f400f460 con 0x7fa408108e10 2024-04-20T11:18:25.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.030+0000 7fa405600700 1 -- 172.21.15.117:0/2717901404 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3f4017630 con 0x7fa408108e10 2024-04-20T11:18:25.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.030+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4081a1e20 con 0x7fa408108e10 2024-04-20T11:18:25.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.030+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4081a2390 con 0x7fa408108e10 2024-04-20T11:18:25.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.031+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40810f890 con 0x7fa408108e10 2024-04-20T11:18:25.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.032+0000 7fa405600700 1 -- 172.21.15.117:0/2717901404 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa3f400f5d0 con 0x7fa408108e10 2024-04-20T11:18:25.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.033+0000 7fa405600700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa3f006ecd0 0x7fa3f0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:25.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.033+0000 7fa405600700 1 -- 172.21.15.117:0/2717901404 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fa3f4090290 con 0x7fa408108e10 2024-04-20T11:18:25.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.033+0000 7fa40cc00700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa3f006ecd0 0x7fa3f0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:25.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.034+0000 7fa40cc00700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa3f006ecd0 0x7fa3f0071190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc006fd0 tx=0x7fa3fc008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:25.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.038+0000 7fa405600700 1 -- 172.21.15.117:0/2717901404 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa3f4059100 con 0x7fa408108e10 2024-04-20T11:18:25.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.287+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa40819e630 con 0x7fa3f006ecd0 2024-04-20T11:18:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.290+0000 7fa405600700 1 -- 172.21.15.117:0/2717901404 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa40819e630 con 0x7fa3f006ecd0 2024-04-20T11:18:25.289 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:18:25.289 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:18:25.289 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "11/23 daemons upgraded", 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading osd daemons", 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:18:25.290 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:18:25.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa3f006ecd0 msgr2=0x7fa3f0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:25.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa3f006ecd0 0x7fa3f0071190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc006fd0 tx=0x7fa3fc008040 comp rx=0 tx=0).stop 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 msgr2=0x7fa40819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 0x7fa40819d7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40092e0 tx=0x7fa3f400be20 comp rx=0 tx=0).stop 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 shutdown_connections 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa3f006ecd0 0x7fa3f0071190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408107520 0x7fa40819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 --2- 172.21.15.117:0/2717901404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa408108e10 0x7fa40819d7f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 >> 172.21.15.117:0/2717901404 conn(0x7fa4080768f0 msgr2=0x7fa4080775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 shutdown_connections 2024-04-20T11:18:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.293+0000 7fa40f600700 1 -- 172.21.15.117:0/2717901404 wait complete. 2024-04-20T11:18:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:25 smithi173 ceph-mon[74165]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:25 smithi173 ceph-mon[74165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-20T11:18:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:25 smithi173 ceph-mon[74165]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-04-20T11:18:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:25 smithi173 ceph-mon[74165]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:25 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1534888210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:25 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3376739123' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:18:25.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.439+0000 7f6f0be00700 1 -- 172.21.15.117:0/2065910829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 msgr2=0x7f6f04107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:25.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.439+0000 7f6f0be00700 1 --2- 172.21.15.117:0/2065910829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 0x7f6f04107900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009910 tx=0x7f6ef8009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:25.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.439+0000 7f6f0be00700 1 -- 172.21.15.117:0/2065910829 shutdown_connections 2024-04-20T11:18:25.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.439+0000 7f6f0be00700 1 --2- 172.21.15.117:0/2065910829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f04108e10 0x7f6f04109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.439+0000 7f6f0be00700 1 --2- 172.21.15.117:0/2065910829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 0x7f6f04107900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.439+0000 7f6f0be00700 1 -- 172.21.15.117:0/2065910829 >> 172.21.15.117:0/2065910829 conn(0x7f6f040768f0 msgr2=0x7f6f04078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:25.440 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:25 smithi117 ceph-mon[99036]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:25.440 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:25 smithi117 ceph-mon[99036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-20T11:18:25.440 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:25 smithi117 ceph-mon[99036]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-04-20T11:18:25.440 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:25 smithi117 ceph-mon[99036]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:25.440 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:25 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1534888210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:25.440 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:25 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3376739123' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:18:25.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.440+0000 7f6f0be00700 1 -- 172.21.15.117:0/2065910829 shutdown_connections 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.440+0000 7f6f0be00700 1 -- 172.21.15.117:0/2065910829 wait complete. 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f0be00700 1 Processor -- start 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f0be00700 1 -- start start 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f0be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 0x7f6f0419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f0be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f04108e10 0x7f6f0419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f0be00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0419de90 con 0x7f6f04107520 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f0be00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f041a1b60 con 0x7f6f04108e10 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f09400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 0x7f6f0419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f09400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 0x7f6f0419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39118/0 (socket says 172.21.15.117:39118) 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.441+0000 7f6f09400700 1 -- 172.21.15.117:0/2756329491 learned_addr learned my addr 172.21.15.117:0/2756329491 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.442+0000 7f6f08a00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f04108e10 0x7f6f0419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.442+0000 7f6f09400700 1 -- 172.21.15.117:0/2756329491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f04108e10 msgr2=0x7f6f0419d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:25.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.442+0000 7f6f09400700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f04108e10 0x7f6f0419d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.442+0000 7f6f09400700 1 -- 172.21.15.117:0/2756329491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef80095f0 con 0x7f6f04107520 2024-04-20T11:18:25.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.442+0000 7f6f08a00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f04108e10 0x7f6f0419d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:18:25.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.442+0000 7f6f09400700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 0x7f6f0419d270 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800bd70 tx=0x7f6ef800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:25.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.442+0000 7f6f02000700 1 -- 172.21.15.117:0/2756329491 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef801d070 con 0x7f6f04107520 2024-04-20T11:18:25.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.443+0000 7f6f02000700 1 -- 172.21.15.117:0/2756329491 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ef800f460 con 0x7f6f04107520 2024-04-20T11:18:25.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.443+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f041a1de0 con 0x7f6f04107520 2024-04-20T11:18:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.443+0000 7f6f02000700 1 -- 172.21.15.117:0/2756329491 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef8017630 con 0x7f6f04107520 2024-04-20T11:18:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.443+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f041a2350 con 0x7f6f04107520 2024-04-20T11:18:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.444+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0410f890 con 0x7f6f04107520 2024-04-20T11:18:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.445+0000 7f6f02000700 1 -- 172.21.15.117:0/2756329491 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6ef800f5d0 con 0x7f6f04107520 2024-04-20T11:18:25.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.446+0000 7f6f02000700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6eec06ecc0 0x7f6eec071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:25.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.446+0000 7f6f02000700 1 -- 172.21.15.117:0/2756329491 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f6ef8090290 con 0x7f6f04107520 2024-04-20T11:18:25.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.447+0000 7f6f08a00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6eec06ecc0 0x7f6eec071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:25.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.447+0000 7f6f08a00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6eec06ecc0 0x7f6eec071180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f0419e7d0 tx=0x7f6ef0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:25.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.450+0000 7f6f02000700 1 -- 172.21.15.117:0/2756329491 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6ef8059100 con 0x7f6f04107520 2024-04-20T11:18:25.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.745+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6f0419e580 con 0x7f6f04107520 2024-04-20T11:18:25.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.746+0000 7f6f02000700 1 -- 172.21.15.117:0/2756329491 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f6ef805c7c0 con 0x7f6f04107520 2024-04-20T11:18:25.745 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:18:25.745 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:18:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:18:25.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6eec06ecc0 msgr2=0x7f6eec071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:25.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6eec06ecc0 0x7f6eec071180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f0419e7d0 tx=0x7f6ef0008040 comp rx=0 tx=0).stop 2024-04-20T11:18:25.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 msgr2=0x7f6f0419d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:25.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 0x7f6f0419d270 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800bd70 tx=0x7f6ef800be50 comp rx=0 tx=0).stop 2024-04-20T11:18:25.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 shutdown_connections 2024-04-20T11:18:25.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6eec06ecc0 0x7f6eec071180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f04107520 0x7f6f0419d270 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 --2- 172.21.15.117:0/2756329491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f04108e10 0x7f6f0419d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:25.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 >> 172.21.15.117:0/2756329491 conn(0x7f6f040768f0 msgr2=0x7f6f04078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:25.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 shutdown_connections 2024-04-20T11:18:25.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:25.749+0000 7f6f0be00700 1 -- 172.21.15.117:0/2756329491 wait complete. 2024-04-20T11:18:26.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:26 smithi117 ceph-mon[99036]: pgmap v169: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:26.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:26 smithi117 ceph-mon[99036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/324 objects degraded (3.086%), 3 pgs degraded) 2024-04-20T11:18:26.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:26 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2756329491' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:18:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:26 smithi173 ceph-mon[74165]: pgmap v169: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:26 smithi173 ceph-mon[74165]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/324 objects degraded (3.086%), 3 pgs degraded) 2024-04-20T11:18:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:26 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2756329491' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:18:27.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:27 smithi117 ceph-mon[99036]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:27 smithi173 ceph-mon[74165]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:28 smithi117 ceph-mon[99036]: pgmap v170: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:18:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:28 smithi173 ceph-mon[74165]: pgmap v170: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:18:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:29 smithi117 ceph-mon[99036]: pgmap v171: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:29 smithi173 ceph-mon[74165]: pgmap v171: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:32.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:32 smithi117 ceph-mon[99036]: pgmap v172: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:32 smithi173 ceph-mon[74165]: pgmap v172: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:34 smithi173 ceph-mon[74165]: pgmap v173: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:34 smithi117 ceph-mon[99036]: pgmap v173: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:34.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:36 smithi173 ceph-mon[74165]: pgmap v174: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:36 smithi117 ceph-mon[99036]: pgmap v174: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:38.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:38 smithi117 ceph-mon[99036]: pgmap v175: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:38 smithi173 ceph-mon[74165]: pgmap v175: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:39.454 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:39 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-20T11:18:39.507 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:39 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-20T11:18:40.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:40 smithi173 ceph-mon[74165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-20T11:18:40.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:40 smithi173 ceph-mon[74165]: Upgrade: osd.5 is safe to restart 2024-04-20T11:18:40.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:40 smithi173 ceph-mon[74165]: pgmap v176: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:40.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:40 smithi117 ceph-mon[99036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-20T11:18:40.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:40 smithi117 ceph-mon[99036]: Upgrade: osd.5 is safe to restart 2024-04-20T11:18:40.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:40 smithi117 ceph-mon[99036]: pgmap v176: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:41.988 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:41 smithi173 ceph-mon[74165]: Upgrade: Updating osd.5 2024-04-20T11:18:41.988 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:41.989 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:18:41.989 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:41.989 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:41 smithi173 ceph-mon[74165]: Deploying daemon osd.5 on smithi173 2024-04-20T11:18:41.989 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:41 smithi173 ceph-mon[74165]: pgmap v177: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:42.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:41 smithi117 ceph-mon[99036]: Upgrade: Updating osd.5 2024-04-20T11:18:42.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:42.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:18:42.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:42.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:41 smithi117 ceph-mon[99036]: Deploying daemon osd.5 on smithi173 2024-04-20T11:18:42.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:41 smithi117 ceph-mon[99036]: pgmap v177: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:43.333 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:18:43.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:18:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:44 smithi173 ceph-mon[74165]: pgmap v178: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:44 smithi117 ceph-mon[99036]: pgmap v178: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:45.642 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:45 smithi173 systemd[1]: Stopping Ceph osd.5 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:18:45.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:45 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[43171]: 2024-04-20T11:18:45.849+0000 7fc057200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:18:45.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:45 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[43171]: 2024-04-20T11:18:45.849+0000 7fc057200700 -1 osd.5 75 *** Got signal Terminated *** 2024-04-20T11:18:45.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:45 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[43171]: 2024-04-20T11:18:45.849+0000 7fc057200700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:18:46.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:46 smithi117 ceph-mon[99036]: pgmap v179: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:46 smithi117 ceph-mon[99036]: osd.5 marked itself down and dead 2024-04-20T11:18:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:46 smithi173 ceph-mon[74165]: pgmap v179: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:46 smithi173 ceph-mon[74165]: osd.5 marked itself down and dead 2024-04-20T11:18:46.975 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:46 smithi173 podman[93464]: 2024-04-20 11:18:46.673693448 +0000 UTC m=+0.953198258 container died e34232ef8172a352ad0a0326c7ab24e6abcd52235cd22750c75781046fce3233 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=reef-646b8c7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-20T11:18:47.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:47 smithi117 ceph-mon[99036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:18:47.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:47 smithi117 ceph-mon[99036]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-20T11:18:47.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:47 smithi117 ceph-mon[99036]: osdmap e76: 6 total, 5 up, 6 in 2024-04-20T11:18:47.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:47 smithi117 ceph-mon[99036]: osdmap e77: 6 total, 5 up, 6 in 2024-04-20T11:18:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:47 smithi173 ceph-mon[74165]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-20T11:18:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:47 smithi173 ceph-mon[74165]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-20T11:18:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:47 smithi173 ceph-mon[74165]: osdmap e76: 6 total, 5 up, 6 in 2024-04-20T11:18:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:47 smithi173 ceph-mon[74165]: osdmap e77: 6 total, 5 up, 6 in 2024-04-20T11:18:47.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:47 smithi173 podman[93464]: 2024-04-20 11:18:47.093215988 +0000 UTC m=+1.372720815 container cleanup e34232ef8172a352ad0a0326c7ab24e6abcd52235cd22750c75781046fce3233 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, RELEASE=reef-646b8c7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T11:18:47.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:47 smithi173 bash[93464]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5 2024-04-20T11:18:47.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:47 smithi173 podman[93483]: 2024-04-20 11:18:47.511269781 +0000 UTC m=+0.819352803 container remove e34232ef8172a352ad0a0326c7ab24e6abcd52235cd22750c75781046fce3233 (image=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=reef-646b8c7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-20T11:18:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:48 smithi173 ceph-mon[74165]: pgmap v181: 65 pgs: 13 stale+active+clean, 52 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:48.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:48 smithi173 podman[93573]: 2024-04-20 11:18:48.274690699 +0000 UTC m=+0.032443634 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:18:48.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:48 smithi173 podman[93573]: 2024-04-20 11:18:48.387017656 +0000 UTC m=+0.144770628 container create 16b0c881ff595f0e54ff2bc6ee8354c28cf05cdf81ff1981bc6da4506d297379 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T11:18:48.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:48 smithi117 ceph-mon[99036]: pgmap v181: 65 pgs: 13 stale+active+clean, 52 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-20T11:18:48.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:48 smithi173 podman[93573]: 2024-04-20 11:18:48.699272095 +0000 UTC m=+0.457025074 container init 16b0c881ff595f0e54ff2bc6ee8354c28cf05cdf81ff1981bc6da4506d297379 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-20T11:18:48.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:48 smithi173 podman[93573]: 2024-04-20 11:18:48.711359852 +0000 UTC m=+0.469112830 container start 16b0c881ff595f0e54ff2bc6ee8354c28cf05cdf81ff1981bc6da4506d297379 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T11:18:48.912 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:48 smithi173 podman[93573]: 2024-04-20 11:18:48.797906882 +0000 UTC m=+0.555659831 container attach 16b0c881ff595f0e54ff2bc6ee8354c28cf05cdf81ff1981bc6da4506d297379 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-20T11:18:49.572 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:49 smithi173 podman[93573]: 2024-04-20 11:18:49.16316379 +0000 UTC m=+0.920916772 container died 16b0c881ff595f0e54ff2bc6ee8354c28cf05cdf81ff1981bc6da4506d297379 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:18:49.573 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:49 smithi173 ceph-mon[74165]: pgmap v183: 65 pgs: 3 active+undersized+degraded, 8 peering, 5 stale+active+clean, 4 active+undersized, 45 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 10/324 objects degraded (3.086%) 2024-04-20T11:18:49.573 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:49 smithi173 ceph-mon[74165]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-20T11:18:49.573 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:49 smithi173 ceph-mon[74165]: Health check failed: Degraded data redundancy: 10/324 objects degraded (3.086%), 3 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:49.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:49 smithi117 ceph-mon[99036]: pgmap v183: 65 pgs: 3 active+undersized+degraded, 8 peering, 5 stale+active+clean, 4 active+undersized, 45 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 10/324 objects degraded (3.086%) 2024-04-20T11:18:49.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:49 smithi117 ceph-mon[99036]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-20T11:18:49.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:49 smithi117 ceph-mon[99036]: Health check failed: Degraded data redundancy: 10/324 objects degraded (3.086%), 3 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:50.161 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:49 smithi173 podman[93599]: 2024-04-20 11:18:49.901005111 +0000 UTC m=+0.716546108 container remove 16b0c881ff595f0e54ff2bc6ee8354c28cf05cdf81ff1981bc6da4506d297379 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:18:50.161 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:49 smithi173 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-20T11:18:50.161 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:49 smithi173 systemd[1]: Stopped Ceph osd.5 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:18:50.162 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:49 smithi173 systemd[1]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.5.service: Consumed 1min 58.994s CPU time. 2024-04-20T11:18:50.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:50 smithi173 systemd[1]: Starting Ceph osd.5 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:18:51.039 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:50 smithi173 podman[93698]: 2024-04-20 11:18:50.608380018 +0000 UTC m=+0.037473767 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:18:51.039 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:50 smithi173 podman[93698]: 2024-04-20 11:18:50.765702515 +0000 UTC m=+0.194796304 container create 035c03b7a5ef7619d0f6c981ef018b88ceaf82d26069ffcb51d47f80826f7d9e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-20T11:18:51.039 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 podman[93698]: 2024-04-20 11:18:51.039215695 +0000 UTC m=+0.468309501 container init 035c03b7a5ef7619d0f6c981ef018b88ceaf82d26069ffcb51d47f80826f7d9e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-20T11:18:51.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 podman[93698]: 2024-04-20 11:18:51.051106735 +0000 UTC m=+0.480200470 container start 035c03b7a5ef7619d0f6c981ef018b88ceaf82d26069ffcb51d47f80826f7d9e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:18:51.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 podman[93698]: 2024-04-20 11:18:51.154670729 +0000 UTC m=+0.583764521 container attach 035c03b7a5ef7619d0f6c981ef018b88ceaf82d26069ffcb51d47f80826f7d9e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T11:18:52.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:52 smithi117 ceph-mon[99036]: pgmap v184: 65 pgs: 9 active+undersized+degraded, 8 peering, 2 stale+active+clean, 8 active+undersized, 38 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/324 objects degraded (11.728%) 2024-04-20T11:18:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:52 smithi173 ceph-mon[74165]: pgmap v184: 65 pgs: 9 active+undersized+degraded, 8 peering, 2 stale+active+clean, 8 active+undersized, 38 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/324 objects degraded (11.728%) 2024-04-20T11:18:52.412 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:18:52.412 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:18:52.412 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:18:52.412 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-20T11:18:52.412 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T11:18:52.412 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-20T11:18:52.412 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 bash[93698]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:51 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate[93713]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-20T11:18:52.413 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:52 smithi173 podman[93698]: 2024-04-20 11:18:52.023634526 +0000 UTC m=+1.452728251 container died 035c03b7a5ef7619d0f6c981ef018b88ceaf82d26069ffcb51d47f80826f7d9e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:18:53.149 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:52 smithi173 podman[93845]: 2024-04-20 11:18:52.858338314 +0000 UTC m=+0.819796842 container remove 035c03b7a5ef7619d0f6c981ef018b88ceaf82d26069ffcb51d47f80826f7d9e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-20T11:18:53.401 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:53 smithi173 ceph-mon[74165]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-04-20T11:18:53.401 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:53 smithi173 podman[93904]: 2024-04-20 11:18:53.111997259 +0000 UTC m=+0.034694346 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:18:53.401 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:53 smithi173 podman[93904]: 2024-04-20 11:18:53.234340302 +0000 UTC m=+0.157037426 container create b9aa7862f8f5625ad7dd79be23c15a90335ca97f4ec539ba5a2b94b5b38feca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-20T11:18:53.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:53 smithi117 ceph-mon[99036]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-04-20T11:18:53.914 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:53 smithi173 podman[93904]: 2024-04-20 11:18:53.537575125 +0000 UTC m=+0.460272256 container init b9aa7862f8f5625ad7dd79be23c15a90335ca97f4ec539ba5a2b94b5b38feca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-20T11:18:53.914 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:53 smithi173 podman[93904]: 2024-04-20 11:18:53.549417654 +0000 UTC m=+0.472114784 container start b9aa7862f8f5625ad7dd79be23c15a90335ca97f4ec539ba5a2b94b5b38feca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-20T11:18:53.914 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:53 smithi173 ceph-osd[93922]: -- 172.21.15.173:0/1549427411 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560dc861f680 con 0x560dc85e8c00 2024-04-20T11:18:53.914 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:53 smithi173 bash[93904]: b9aa7862f8f5625ad7dd79be23c15a90335ca97f4ec539ba5a2b94b5b38feca2 2024-04-20T11:18:53.914 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:53 smithi173 systemd[1]: Started Ceph osd.5 for 59ecfd22-ff04-11ee-bc92-c7b262605968. 2024-04-20T11:18:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:54 smithi173 ceph-mon[74165]: pgmap v185: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:18:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:54 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:54 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:54 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:54.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:54 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:18:54.116+0000 7f24048527c0 -1 Falling back to public interface 2024-04-20T11:18:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:54 smithi117 ceph-mon[99036]: pgmap v185: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:18:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:54 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:54 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:54 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:55 smithi173 ceph-mon[74165]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:18:55.920 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:55.921 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:55.921 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:55 smithi117 ceph-mon[99036]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:18:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.921+0000 7fee08c00700 1 -- 172.21.15.117:0/1301952413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 msgr2=0x7fee04109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.921+0000 7fee08c00700 1 --2- 172.21.15.117:0/1301952413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 0x7fee04109290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009910 tx=0x7fedf0009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.921+0000 7fee08c00700 1 -- 172.21.15.117:0/1301952413 shutdown_connections 2024-04-20T11:18:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.921+0000 7fee08c00700 1 --2- 172.21.15.117:0/1301952413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 0x7fee04109290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.921+0000 7fee08c00700 1 --2- 172.21.15.117:0/1301952413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee04107520 0x7fee04107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.921+0000 7fee08c00700 1 -- 172.21.15.117:0/1301952413 >> 172.21.15.117:0/1301952413 conn(0x7fee040768f0 msgr2=0x7fee04078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.921+0000 7fee08c00700 1 -- 172.21.15.117:0/1301952413 shutdown_connections 2024-04-20T11:18:55.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.921+0000 7fee08c00700 1 -- 172.21.15.117:0/1301952413 wait complete. 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.922+0000 7fee08c00700 1 Processor -- start 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.922+0000 7fee08c00700 1 -- start start 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.923+0000 7fee08c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee04107520 0x7fee0419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.923+0000 7fee08c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 0x7fee0419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.923+0000 7fee08c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0419ddf0 con 0x7fee04108e10 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.923+0000 7fee08c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee041a1a70 con 0x7fee04107520 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.923+0000 7fee01400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 0x7fee0419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.923+0000 7fee01400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 0x7fee0419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34524/0 (socket says 172.21.15.117:34524) 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.923+0000 7fee01400700 1 -- 172.21.15.117:0/1400763664 learned_addr learned my addr 172.21.15.117:0/1400763664 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:55.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fee01e00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee04107520 0x7fee0419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:55.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fee01400700 1 -- 172.21.15.117:0/1400763664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee04107520 msgr2=0x7fee0419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:55.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fee01400700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee04107520 0x7fee0419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:55.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fee01400700 1 -- 172.21.15.117:0/1400763664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00095f0 con 0x7fee04108e10 2024-04-20T11:18:55.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fee01400700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 0x7fee0419d710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fedf00092e0 tx=0x7fedf000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:55.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fedfaa00700 1 -- 172.21.15.117:0/1400763664 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedf001d070 con 0x7fee04108e10 2024-04-20T11:18:55.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fedfaa00700 1 -- 172.21.15.117:0/1400763664 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedf00056e0 con 0x7fee04108e10 2024-04-20T11:18:55.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fedfaa00700 1 -- 172.21.15.117:0/1400763664 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedf000fe80 con 0x7fee04108e10 2024-04-20T11:18:55.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.924+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee041a1cf0 con 0x7fee04108e10 2024-04-20T11:18:55.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.925+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee041a2260 con 0x7fee04108e10 2024-04-20T11:18:55.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.926+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0410f870 con 0x7fee04108e10 2024-04-20T11:18:55.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.927+0000 7fedfaa00700 1 -- 172.21.15.117:0/1400763664 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fedf000f6f0 con 0x7fee04108e10 2024-04-20T11:18:55.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.928+0000 7fedfaa00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fede806ecd0 0x7fede8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:55.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.928+0000 7fedfaa00700 1 -- 172.21.15.117:0/1400763664 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fedf0090180 con 0x7fee04108e10 2024-04-20T11:18:55.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.928+0000 7fee01e00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fede806ecd0 0x7fede8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:55.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.929+0000 7fee01e00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fede806ecd0 0x7fede8071190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005950 tx=0x7fedf40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:55.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:55.933+0000 7fedfaa00700 1 -- 172.21.15.117:0/1400763664 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fedf0058ff0 con 0x7fee04108e10 2024-04-20T11:18:56.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.181+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee0419e490 con 0x7fede806ecd0 2024-04-20T11:18:56.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.183+0000 7fedfaa00700 1 -- 172.21.15.117:0/1400763664 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fee0419e490 con 0x7fede806ecd0 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fede806ecd0 msgr2=0x7fede8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fede806ecd0 0x7fede8071190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005950 tx=0x7fedf40058e0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 msgr2=0x7fee0419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 0x7fee0419d710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fedf00092e0 tx=0x7fedf000be20 comp rx=0 tx=0).stop 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 shutdown_connections 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fede806ecd0 0x7fede8071190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee04107520 0x7fee0419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 --2- 172.21.15.117:0/1400763664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee04108e10 0x7fee0419d710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 >> 172.21.15.117:0/1400763664 conn(0x7fee040768f0 msgr2=0x7fee04077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 shutdown_connections 2024-04-20T11:18:56.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.186+0000 7fee08c00700 1 -- 172.21.15.117:0/1400763664 wait complete. 2024-04-20T11:18:56.201 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:18:56.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.334+0000 7fb9ab800700 1 -- 172.21.15.117:0/1453076372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 msgr2=0x7fb9a41034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:56.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.334+0000 7fb9ab800700 1 --2- 172.21.15.117:0/1453076372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 0x7fb9a41034f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb994009910 tx=0x7fb994009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:56.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.334+0000 7fb9ab800700 1 -- 172.21.15.117:0/1453076372 shutdown_connections 2024-04-20T11:18:56.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.334+0000 7fb9ab800700 1 --2- 172.21.15.117:0/1453076372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4104a00 0x7fb9a4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.334+0000 7fb9ab800700 1 --2- 172.21.15.117:0/1453076372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 0x7fb9a41034f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.334+0000 7fb9ab800700 1 -- 172.21.15.117:0/1453076372 >> 172.21.15.117:0/1453076372 conn(0x7fb9a40fe880 msgr2=0x7fb9a4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:56.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.334+0000 7fb9ab800700 1 -- 172.21.15.117:0/1453076372 shutdown_connections 2024-04-20T11:18:56.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.334+0000 7fb9ab800700 1 -- 172.21.15.117:0/1453076372 wait complete. 2024-04-20T11:18:56.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.335+0000 7fb9ab800700 1 Processor -- start 2024-04-20T11:18:56.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.335+0000 7fb9ab800700 1 -- start start 2024-04-20T11:18:56.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.335+0000 7fb9ab800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 0x7fb9a4198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:56.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.335+0000 7fb9ab800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4104a00 0x7fb9a4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:56.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.335+0000 7fb9ab800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a41999e0 con 0x7fb9a4103110 2024-04-20T11:18:56.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.335+0000 7fb9ab800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419d660 con 0x7fb9a4104a00 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb9a8e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 0x7fb9a4198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb9a8e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 0x7fb9a4198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34550/0 (socket says 172.21.15.117:34550) 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb9a8e00700 1 -- 172.21.15.117:0/3614207973 learned_addr learned my addr 172.21.15.117:0/3614207973 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb99be00700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4104a00 0x7fb9a4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb9a8e00700 1 -- 172.21.15.117:0/3614207973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4104a00 msgr2=0x7fb9a4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb9a8e00700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4104a00 0x7fb9a4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb9a8e00700 1 -- 172.21.15.117:0/3614207973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9940095f0 con 0x7fb9a4103110 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb99be00700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4104a00 0x7fb9a4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.336+0000 7fb9a8e00700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 0x7fb9a4198dc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb99400bd70 tx=0x7fb99400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.337+0000 7fb9a2000700 1 -- 172.21.15.117:0/3614207973 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb99401d070 con 0x7fb9a4103110 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.337+0000 7fb9a2000700 1 -- 172.21.15.117:0/3614207973 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9940056e0 con 0x7fb9a4103110 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.337+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a419d8e0 con 0x7fb9a4103110 2024-04-20T11:18:56.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.337+0000 7fb9a2000700 1 -- 172.21.15.117:0/3614207973 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb994017400 con 0x7fb9a4103110 2024-04-20T11:18:56.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.337+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a419de50 con 0x7fb9a4103110 2024-04-20T11:18:56.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.338+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a410b480 con 0x7fb9a4103110 2024-04-20T11:18:56.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.339+0000 7fb9a2000700 1 -- 172.21.15.117:0/3614207973 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb994003a60 con 0x7fb9a4103110 2024-04-20T11:18:56.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.340+0000 7fb9a2000700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb98406ecc0 0x7fb984071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:56.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.340+0000 7fb9a2000700 1 -- 172.21.15.117:0/3614207973 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fb9940901f0 con 0x7fb9a4103110 2024-04-20T11:18:56.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.340+0000 7fb99be00700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb98406ecc0 0x7fb984071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:56.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.341+0000 7fb99be00700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb98406ecc0 0x7fb984071180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb98c005fd0 tx=0x7fb98c005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:56.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.345+0000 7fb9a2000700 1 -- 172.21.15.117:0/3614207973 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb994059060 con 0x7fb9a4103110 2024-04-20T11:18:56.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.574+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9a419a080 con 0x7fb98406ecc0 2024-04-20T11:18:56.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.578+0000 7fb9a2000700 1 -- 172.21.15.117:0/3614207973 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb9a419a080 con 0x7fb98406ecc0 2024-04-20T11:18:56.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb98406ecc0 msgr2=0x7fb984071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:56.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb98406ecc0 0x7fb984071180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb98c005fd0 tx=0x7fb98c005ee0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 msgr2=0x7fb9a4198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:56.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 0x7fb9a4198dc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb99400bd70 tx=0x7fb99400be50 comp rx=0 tx=0).stop 2024-04-20T11:18:56.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 shutdown_connections 2024-04-20T11:18:56.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb98406ecc0 0x7fb984071180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4103110 0x7fb9a4198dc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 --2- 172.21.15.117:0/3614207973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4104a00 0x7fb9a4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 >> 172.21.15.117:0/3614207973 conn(0x7fb9a40fe880 msgr2=0x7fb9a40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:56.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 shutdown_connections 2024-04-20T11:18:56.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.581+0000 7fb9ab800700 1 -- 172.21.15.117:0/3614207973 wait complete. 2024-04-20T11:18:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.738+0000 7f1d2b200700 1 -- 172.21.15.117:0/1577936897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d24105a80 msgr2=0x7f1d24105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.738+0000 7f1d2b200700 1 --2- 172.21.15.117:0/1577936897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d24105a80 0x7f1d24105e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009870 tx=0x7f1d14009b80 comp rx=0 tx=0).stop 2024-04-20T11:18:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.738+0000 7f1d2b200700 1 -- 172.21.15.117:0/1577936897 shutdown_connections 2024-04-20T11:18:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.738+0000 7f1d2b200700 1 --2- 172.21.15.117:0/1577936897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d240ffa80 0x7f1d240fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.738+0000 7f1d2b200700 1 --2- 172.21.15.117:0/1577936897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d24105a80 0x7f1d24105e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.738+0000 7f1d2b200700 1 -- 172.21.15.117:0/1577936897 >> 172.21.15.117:0/1577936897 conn(0x7f1d24076750 msgr2=0x7f1d240fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:56.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.738+0000 7f1d2b200700 1 -- 172.21.15.117:0/1577936897 shutdown_connections 2024-04-20T11:18:56.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.738+0000 7f1d2b200700 1 -- 172.21.15.117:0/1577936897 wait complete. 2024-04-20T11:18:56.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.739+0000 7f1d2b200700 1 Processor -- start 2024-04-20T11:18:56.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.739+0000 7f1d2b200700 1 -- start start 2024-04-20T11:18:56.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.740+0000 7f1d2b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d240ffa80 0x7f1d24198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:56.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.740+0000 7f1d2b200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d24105a80 0x7f1d24199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:56.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.740+0000 7f1d2b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d241998e0 con 0x7f1d240ffa80 2024-04-20T11:18:56.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.740+0000 7f1d2b200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2419c550 con 0x7f1d24105a80 2024-04-20T11:18:56.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.740+0000 7f1d23e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d240ffa80 0x7f1d24198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:56.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.740+0000 7f1d23e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d240ffa80 0x7f1d24198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34568/0 (socket says 172.21.15.117:34568) 2024-04-20T11:18:56.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.740+0000 7f1d23e00700 1 -- 172.21.15.117:0/772849604 learned_addr learned my addr 172.21.15.117:0/772849604 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:56.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.740+0000 7f1d23400700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d24105a80 0x7f1d24199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:56.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d23e00700 1 -- 172.21.15.117:0/772849604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d24105a80 msgr2=0x7f1d24199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:56.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d23e00700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d24105a80 0x7f1d24199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:56.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d23e00700 1 -- 172.21.15.117:0/772849604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d14009520 con 0x7f1d240ffa80 2024-04-20T11:18:56.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d23e00700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d240ffa80 0x7f1d24198d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1d14004700 tx=0x7f1d1400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:56.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d20c00700 1 -- 172.21.15.117:0/772849604 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d1401c070 con 0x7f1d240ffa80 2024-04-20T11:18:56.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d20c00700 1 -- 172.21.15.117:0/772849604 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d1400fb70 con 0x7f1d240ffa80 2024-04-20T11:18:56.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2419c7d0 con 0x7f1d240ffa80 2024-04-20T11:18:56.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d20c00700 1 -- 172.21.15.117:0/772849604 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d14022d20 con 0x7f1d240ffa80 2024-04-20T11:18:56.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.741+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2419ccc0 con 0x7f1d240ffa80 2024-04-20T11:18:56.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.742+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d241011c0 con 0x7f1d240ffa80 2024-04-20T11:18:56.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.743+0000 7f1d20c00700 1 -- 172.21.15.117:0/772849604 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1d14020070 con 0x7f1d240ffa80 2024-04-20T11:18:56.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.744+0000 7f1d20c00700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d0806ecc0 0x7f1d08071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:56.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.744+0000 7f1d20c00700 1 -- 172.21.15.117:0/772849604 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f1d1408ff80 con 0x7f1d240ffa80 2024-04-20T11:18:56.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.745+0000 7f1d23400700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d0806ecc0 0x7f1d08071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:56.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.745+0000 7f1d23400700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d0806ecc0 0x7f1d08071180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1d2419a230 tx=0x7f1d10009310 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:56.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.749+0000 7f1d20c00700 1 -- 172.21.15.117:0/772849604 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1d14058d70 con 0x7f1d240ffa80 2024-04-20T11:18:56.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.981+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d2410cc80 con 0x7f1d0806ecc0 2024-04-20T11:18:56.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:56.998+0000 7f1d20c00700 1 -- 172.21.15.117:0/772849604 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1d2410cc80 con 0x7f1d0806ecc0 2024-04-20T11:18:56.998 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:18:56.998 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (7m) 112s ago 21m 14.2M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:18:56.998 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (21m) 112s ago 21m 9017k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (20m) 41s ago 20m 8724k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (3m) 112s ago 21m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (3m) 41s ago 20m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (6m) 112s ago 21m 74.3M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (14m) 112s ago 14m 18.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (14m) 112s ago 14m 150M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cbc432e03e31 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (14m) 41s ago 14m 18.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (14m) 41s ago 14m 25.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (10m) 112s ago 23m 506M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:18:56.999 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (9m) 41s ago 20m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (5m) 112s ago 23m 51.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (4m) 41s ago 20m 36.0M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (8m) 112s ago 21m 7872k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (8m) 41s ago 20m 8342k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (3m) 112s ago 19m 80.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (2m) 112s ago 18m 82.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (2m) 112s ago 17m 11.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (89s) 41s ago 17m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (46s) 41s ago 16m 11.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:18:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 starting - - - 2240M 2024-04-20T11:18:57.001 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (8m) 112s ago 20m 69.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:18:57.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d0806ecc0 msgr2=0x7f1d08071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d0806ecc0 0x7f1d08071180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1d2419a230 tx=0x7f1d10009310 comp rx=0 tx=0).stop 2024-04-20T11:18:57.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d240ffa80 msgr2=0x7f1d24198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d240ffa80 0x7f1d24198d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1d14004700 tx=0x7f1d1400b5b0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 shutdown_connections 2024-04-20T11:18:57.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d0806ecc0 0x7f1d08071180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d240ffa80 0x7f1d24198d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 --2- 172.21.15.117:0/772849604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d24105a80 0x7f1d24199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 >> 172.21.15.117:0/772849604 conn(0x7f1d24076750 msgr2=0x7f1d24103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:57.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.001+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 shutdown_connections 2024-04-20T11:18:57.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.002+0000 7f1d2b200700 1 -- 172.21.15.117:0/772849604 wait complete. 2024-04-20T11:18:57.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.157+0000 7fe6cdc00700 1 -- 172.21.15.117:0/613671738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 msgr2=0x7fe6c810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.157+0000 7fe6cdc00700 1 --2- 172.21.15.117:0/613671738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 0x7fe6c810cf10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009910 tx=0x7fe6b4009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:57.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.157+0000 7fe6cdc00700 1 -- 172.21.15.117:0/613671738 shutdown_connections 2024-04-20T11:18:57.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.157+0000 7fe6cdc00700 1 --2- 172.21.15.117:0/613671738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 0x7fe6c810cf10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.157+0000 7fe6cdc00700 1 --2- 172.21.15.117:0/613671738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6c80732b0 0x7fe6c8073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.157+0000 7fe6cdc00700 1 -- 172.21.15.117:0/613671738 >> 172.21.15.117:0/613671738 conn(0x7fe6c80fc6a0 msgr2=0x7fe6c80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:57.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.157+0000 7fe6cdc00700 1 -- 172.21.15.117:0/613671738 shutdown_connections 2024-04-20T11:18:57.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.157+0000 7fe6cdc00700 1 -- 172.21.15.117:0/613671738 wait complete. 2024-04-20T11:18:57.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.158+0000 7fe6cdc00700 1 Processor -- start 2024-04-20T11:18:57.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.158+0000 7fe6cdc00700 1 -- start start 2024-04-20T11:18:57.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.158+0000 7fe6cdc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6c80732b0 0x7fe6c8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:57.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.158+0000 7fe6cdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 0x7fe6c8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:57.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.158+0000 7fe6cdc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c81998d0 con 0x7fe6c8073bd0 2024-04-20T11:18:57.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.158+0000 7fe6cdc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c819c540 con 0x7fe6c80732b0 2024-04-20T11:18:57.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.159+0000 7fe6c6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 0x7fe6c8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.159+0000 7fe6c6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 0x7fe6c8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34582/0 (socket says 172.21.15.117:34582) 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.159+0000 7fe6c6000700 1 -- 172.21.15.117:0/1626553069 learned_addr learned my addr 172.21.15.117:0/1626553069 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.159+0000 7fe6c6000700 1 -- 172.21.15.117:0/1626553069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6c80732b0 msgr2=0x7fe6c8198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.159+0000 7fe6c6000700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6c80732b0 0x7fe6c8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.159+0000 7fe6c6000700 1 -- 172.21.15.117:0/1626553069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40095f0 con 0x7fe6c8073bd0 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.159+0000 7fe6c6000700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 0x7fe6c8199240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4000c00 tx=0x7fe6b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.160+0000 7fe6bf400700 1 -- 172.21.15.117:0/1626553069 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6b401d070 con 0x7fe6c8073bd0 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.160+0000 7fe6bf400700 1 -- 172.21.15.117:0/1626553069 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6b40056e0 con 0x7fe6c8073bd0 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.160+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c819c7c0 con 0x7fe6c8073bd0 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.160+0000 7fe6bf400700 1 -- 172.21.15.117:0/1626553069 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6b400fe80 con 0x7fe6c8073bd0 2024-04-20T11:18:57.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.160+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c819ccb0 con 0x7fe6c8073bd0 2024-04-20T11:18:57.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.161+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c8074960 con 0x7fe6c8073bd0 2024-04-20T11:18:57.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.162+0000 7fe6bf400700 1 -- 172.21.15.117:0/1626553069 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe6b400f6f0 con 0x7fe6c8073bd0 2024-04-20T11:18:57.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.163+0000 7fe6bf400700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6ac06ecd0 0x7fe6ac071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:57.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.163+0000 7fe6bf400700 1 -- 172.21.15.117:0/1626553069 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fe6b4090230 con 0x7fe6c8073bd0 2024-04-20T11:18:57.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.163+0000 7fe6c6a00700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6ac06ecd0 0x7fe6ac071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:57.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.164+0000 7fe6c6a00700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6ac06ecd0 0x7fe6ac071190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8005950 tx=0x7fe6b80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:57.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.168+0000 7fe6bf400700 1 -- 172.21.15.117:0/1626553069 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe6b4059020 con 0x7fe6c8073bd0 2024-04-20T11:18:57.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.476+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe6c8107770 con 0x7fe6c8073bd0 2024-04-20T11:18:57.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.477+0000 7fe6bf400700 1 -- 172.21.15.117:0/1626553069 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7fe6b405c6e0 con 0x7fe6c8073bd0 2024-04-20T11:18:57.477 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:18:57.477 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 5 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-20T11:18:57.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:18:57.479 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:18:57.479 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4, 2024-04-20T11:18:57.479 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 9 2024-04-20T11:18:57.479 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:18:57.479 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:18:57.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6ac06ecd0 msgr2=0x7fe6ac071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6ac06ecd0 0x7fe6ac071190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8005950 tx=0x7fe6b80058e0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 msgr2=0x7fe6c8199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 0x7fe6c8199240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4000c00 tx=0x7fe6b400be20 comp rx=0 tx=0).stop 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 shutdown_connections 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6ac06ecd0 0x7fe6ac071190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6c80732b0 0x7fe6c8198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 --2- 172.21.15.117:0/1626553069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6c8073bd0 0x7fe6c8199240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.480+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 >> 172.21.15.117:0/1626553069 conn(0x7fe6c80fc6a0 msgr2=0x7fe6c81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.481+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 shutdown_connections 2024-04-20T11:18:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.481+0000 7fe6cdc00700 1 -- 172.21.15.117:0/1626553069 wait complete. 2024-04-20T11:18:57.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.636+0000 7fbb8bc00700 1 -- 172.21.15.117:0/4267180961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb84100e10 msgr2=0x7fbb841011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.636+0000 7fbb8bc00700 1 --2- 172.21.15.117:0/4267180961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb84100e10 0x7fbb841011f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009910 tx=0x7fbb74009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:57.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.637+0000 7fbb8bc00700 1 -- 172.21.15.117:0/4267180961 shutdown_connections 2024-04-20T11:18:57.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.637+0000 7fbb8bc00700 1 --2- 172.21.15.117:0/4267180961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb841017c0 0x7fbb84105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.637+0000 7fbb8bc00700 1 --2- 172.21.15.117:0/4267180961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb84100e10 0x7fbb841011f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.637+0000 7fbb8bc00700 1 -- 172.21.15.117:0/4267180961 >> 172.21.15.117:0/4267180961 conn(0x7fbb840fc6c0 msgr2=0x7fbb840feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:57.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.637+0000 7fbb8bc00700 1 -- 172.21.15.117:0/4267180961 shutdown_connections 2024-04-20T11:18:57.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.637+0000 7fbb8bc00700 1 -- 172.21.15.117:0/4267180961 wait complete. 2024-04-20T11:18:57.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.638+0000 7fbb8bc00700 1 Processor -- start 2024-04-20T11:18:57.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.638+0000 7fbb8bc00700 1 -- start start 2024-04-20T11:18:57.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.638+0000 7fbb8bc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb84100e10 0x7fbb840731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:57.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.638+0000 7fbb8bc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb841017c0 0x7fbb84073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:57.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.638+0000 7fbb8bc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84073c60 con 0x7fbb841017c0 2024-04-20T11:18:57.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.638+0000 7fbb8bc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84073da0 con 0x7fbb84100e10 2024-04-20T11:18:57.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.639+0000 7fbb7be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb841017c0 0x7fbb84073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:57.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.639+0000 7fbb7be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb841017c0 0x7fbb84073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34588/0 (socket says 172.21.15.117:34588) 2024-04-20T11:18:57.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.639+0000 7fbb7be00700 1 -- 172.21.15.117:0/3889589743 learned_addr learned my addr 172.21.15.117:0/3889589743 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:57.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.639+0000 7fbb7be00700 1 -- 172.21.15.117:0/3889589743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb84100e10 msgr2=0x7fbb840731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.639+0000 7fbb7be00700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb84100e10 0x7fbb840731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.639+0000 7fbb7be00700 1 -- 172.21.15.117:0/3889589743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740095f0 con 0x7fbb841017c0 2024-04-20T11:18:57.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.639+0000 7fbb7be00700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb841017c0 0x7fbb84073720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00d900 tx=0x7fbb6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:57.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.640+0000 7fbb82000700 1 -- 172.21.15.117:0/3889589743 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb6c009900 con 0x7fbb841017c0 2024-04-20T11:18:57.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.640+0000 7fbb82000700 1 -- 172.21.15.117:0/3889589743 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb6c010460 con 0x7fbb841017c0 2024-04-20T11:18:57.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.640+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb841a2cc0 con 0x7fbb841017c0 2024-04-20T11:18:57.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.640+0000 7fbb82000700 1 -- 172.21.15.117:0/3889589743 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb6c00f5f0 con 0x7fbb841017c0 2024-04-20T11:18:57.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.640+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb841a3020 con 0x7fbb841017c0 2024-04-20T11:18:57.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.641+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb841091b0 con 0x7fbb841017c0 2024-04-20T11:18:57.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.642+0000 7fbb82000700 1 -- 172.21.15.117:0/3889589743 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbb6c0105d0 con 0x7fbb841017c0 2024-04-20T11:18:57.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.643+0000 7fbb82000700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6806ecc0 0x7fbb68071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:57.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.643+0000 7fbb82000700 1 -- 172.21.15.117:0/3889589743 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fbb6c08ebb0 con 0x7fbb841017c0 2024-04-20T11:18:57.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.643+0000 7fbb89200700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6806ecc0 0x7fbb68071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:57.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.644+0000 7fbb89200700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6806ecc0 0x7fbb68071180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb74006010 tx=0x7fbb7400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:57.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.648+0000 7fbb82000700 1 -- 172.21.15.117:0/3889589743 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbb6c057a20 con 0x7fbb841017c0 2024-04-20T11:18:57.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.924+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbb841022f0 con 0x7fbb841017c0 2024-04-20T11:18:57.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.924+0000 7fbb82000700 1 -- 172.21.15.117:0/3889589743 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1899 (secure 0 0 0) 0x7fbb6c017780 con 0x7fbb841017c0 2024-04-20T11:18:57.924 INFO:teuthology.orchestra.run.smithi117.stdout:e12 2024-04-20T11:18:57.924 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 12 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:04:55.549816+0000 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:18:57.925 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=14460,1=24253} 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:18:57.926 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:14460} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.117:6826/361271135,v1:172.21.15.117:6827/361271135] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{1:24253} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.173:6824/3563933435,v1:172.21.15.173:6825/3563933435] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:18:57.927 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6806ecc0 msgr2=0x7fbb68071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6806ecc0 0x7fbb68071180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb74006010 tx=0x7fbb7400b540 comp rx=0 tx=0).stop 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb841017c0 msgr2=0x7fbb84073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb841017c0 0x7fbb84073720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00d900 tx=0x7fbb6c00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 shutdown_connections 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6806ecc0 0x7fbb68071180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb84100e10 0x7fbb840731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 --2- 172.21.15.117:0/3889589743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb841017c0 0x7fbb84073720 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.927+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 >> 172.21.15.117:0/3889589743 conn(0x7fbb840fc6c0 msgr2=0x7fbb840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:57.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.928+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 shutdown_connections 2024-04-20T11:18:57.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:57.928+0000 7fbb8bc00700 1 -- 172.21.15.117:0/3889589743 wait complete. 2024-04-20T11:18:57.929 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-04-20T11:18:58.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.082+0000 7f82a7400700 1 -- 172.21.15.117:0/3722309517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0108e10 msgr2=0x7f82a0109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:58.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.082+0000 7f82a7400700 1 --2- 172.21.15.117:0/3722309517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0108e10 0x7f82a0109290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f828c009910 tx=0x7f828c009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:58.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.082+0000 7f82a7400700 1 -- 172.21.15.117:0/3722309517 shutdown_connections 2024-04-20T11:18:58.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.082+0000 7f82a7400700 1 --2- 172.21.15.117:0/3722309517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0108e10 0x7f82a0109290 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.082+0000 7f82a7400700 1 --2- 172.21.15.117:0/3722309517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0107520 0x7f82a0107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.082+0000 7f82a7400700 1 -- 172.21.15.117:0/3722309517 >> 172.21.15.117:0/3722309517 conn(0x7f82a00768f0 msgr2=0x7f82a0078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:58.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.082+0000 7f82a7400700 1 -- 172.21.15.117:0/3722309517 shutdown_connections 2024-04-20T11:18:58.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.082+0000 7f82a7400700 1 -- 172.21.15.117:0/3722309517 wait complete. 2024-04-20T11:18:58.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.083+0000 7f82a7400700 1 Processor -- start 2024-04-20T11:18:58.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.084+0000 7f82a7400700 1 -- start start 2024-04-20T11:18:58.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.084+0000 7f82a7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0107520 0x7f82a019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:58.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.084+0000 7f82a7400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0108e10 0x7f82a019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:58.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.084+0000 7f82a7400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a019ded0 con 0x7f82a0107520 2024-04-20T11:18:58.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.084+0000 7f82a7400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a01a1ba0 con 0x7f82a0108e10 2024-04-20T11:18:58.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.084+0000 7f82a4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0107520 0x7f82a019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.084+0000 7f82a4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0107520 0x7f82a019d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34606/0 (socket says 172.21.15.117:34606) 2024-04-20T11:18:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.084+0000 7f82a4a00700 1 -- 172.21.15.117:0/3464291120 learned_addr learned my addr 172.21.15.117:0/3464291120 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f829fe00700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0108e10 0x7f82a019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f82a4a00700 1 -- 172.21.15.117:0/3464291120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0108e10 msgr2=0x7f82a019d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f82a4a00700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0108e10 0x7f82a019d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f82a4a00700 1 -- 172.21.15.117:0/3464291120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0095f0 con 0x7f82a0107520 2024-04-20T11:18:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f82a4a00700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0107520 0x7f82a019d2b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f829400b4b0 tx=0x7f829400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f829d600700 1 -- 172.21.15.117:0/3464291120 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8294011800 con 0x7f82a0107520 2024-04-20T11:18:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a01a1e80 con 0x7f82a0107520 2024-04-20T11:18:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f829d600700 1 -- 172.21.15.117:0/3464291120 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8294011e20 con 0x7f82a0107520 2024-04-20T11:18:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.085+0000 7f829d600700 1 -- 172.21.15.117:0/3464291120 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f829400f2e0 con 0x7f82a0107520 2024-04-20T11:18:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.086+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a01a2450 con 0x7f82a0107520 2024-04-20T11:18:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.087+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a010f870 con 0x7f82a0107520 2024-04-20T11:18:58.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.088+0000 7f829d600700 1 -- 172.21.15.117:0/3464291120 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8294010380 con 0x7f82a0107520 2024-04-20T11:18:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.088+0000 7f829d600700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecd0 0x7f8288071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.089+0000 7f829d600700 1 -- 172.21.15.117:0/3464291120 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f829408e9c0 con 0x7f82a0107520 2024-04-20T11:18:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.089+0000 7f829fe00700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecd0 0x7f8288071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:58.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.089+0000 7f829fe00700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecd0 0x7f8288071190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f828c005f50 tx=0x7f828c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.093+0000 7f829d600700 1 -- 172.21.15.117:0/3464291120 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f82940577b0 con 0x7f82a0107520 2024-04-20T11:18:58.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.332+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82a019e630 con 0x7f828806ecd0 2024-04-20T11:18:58.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:58 smithi117 ceph-mon[99036]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:58.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:58 smithi117 ceph-mon[99036]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:58.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:58 smithi117 ceph-mon[99036]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:58.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:58 smithi117 ceph-mon[99036]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:18:58.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:58 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1626553069' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:58.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:58 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3889589743' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:18:58.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:58.333 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:18:58.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.334+0000 7f829d600700 1 -- 172.21.15.117:0/3464291120 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f82a019e630 con 0x7f828806ecd0 2024-04-20T11:18:58.334 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:18:58.334 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:18:58.334 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:18:58.334 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:18:58.334 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:18:58.334 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:18:58.335 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:18:58.335 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr" 2024-04-20T11:18:58.335 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:18:58.335 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "11/23 daemons upgraded", 2024-04-20T11:18:58.335 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading osd daemons", 2024-04-20T11:18:58.335 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:18:58.335 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:18:58.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecd0 msgr2=0x7f8288071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:58.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecd0 0x7f8288071190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f828c005f50 tx=0x7f828c01a040 comp rx=0 tx=0).stop 2024-04-20T11:18:58.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0107520 msgr2=0x7f82a019d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:58.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0107520 0x7f82a019d2b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f829400b4b0 tx=0x7f829400b870 comp rx=0 tx=0).stop 2024-04-20T11:18:58.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 shutdown_connections 2024-04-20T11:18:58.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecd0 0x7f8288071190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a0107520 0x7f82a019d2b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 --2- 172.21.15.117:0/3464291120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0108e10 0x7f82a019d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.338+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 >> 172.21.15.117:0/3464291120 conn(0x7f82a00768f0 msgr2=0x7f82a00775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:58.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.339+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 shutdown_connections 2024-04-20T11:18:58.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.339+0000 7f82a7400700 1 -- 172.21.15.117:0/3464291120 wait complete. 2024-04-20T11:18:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:58 smithi173 ceph-mon[74165]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:58 smithi173 ceph-mon[74165]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:58 smithi173 ceph-mon[74165]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:58 smithi173 ceph-mon[74165]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:18:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:58 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1626553069' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:18:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:58 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3889589743' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:18:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:18:58.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.506+0000 7f8b0ac00700 1 -- 172.21.15.117:0/813378843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 msgr2=0x7f8b04109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:58.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.506+0000 7f8b0ac00700 1 --2- 172.21.15.117:0/813378843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 0x7f8b04109290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009910 tx=0x7f8af0009c20 comp rx=0 tx=0).stop 2024-04-20T11:18:58.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.506+0000 7f8b0ac00700 1 -- 172.21.15.117:0/813378843 shutdown_connections 2024-04-20T11:18:58.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.506+0000 7f8b0ac00700 1 --2- 172.21.15.117:0/813378843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 0x7f8b04109290 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.506+0000 7f8b0ac00700 1 --2- 172.21.15.117:0/813378843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b04107520 0x7f8b04107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.506+0000 7f8b0ac00700 1 -- 172.21.15.117:0/813378843 >> 172.21.15.117:0/813378843 conn(0x7f8b040768f0 msgr2=0x7f8b04078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:58.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.506+0000 7f8b0ac00700 1 -- 172.21.15.117:0/813378843 shutdown_connections 2024-04-20T11:18:58.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.506+0000 7f8b0ac00700 1 -- 172.21.15.117:0/813378843 wait complete. 2024-04-20T11:18:58.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.507+0000 7f8b0ac00700 1 Processor -- start 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.507+0000 7f8b0ac00700 1 -- start start 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.508+0000 7f8b0ac00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b04107520 0x7f8b0419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.508+0000 7f8b0ac00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 0x7f8b0419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.508+0000 7f8b0ac00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0419ddf0 con 0x7f8b04108e10 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.508+0000 7f8b0ac00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b041a1a70 con 0x7f8b04107520 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.508+0000 7f8b03400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 0x7f8b0419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.508+0000 7f8b03e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b04107520 0x7f8b0419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.508+0000 7f8b03400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 0x7f8b0419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34626/0 (socket says 172.21.15.117:34626) 2024-04-20T11:18:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.508+0000 7f8b03400700 1 -- 172.21.15.117:0/2524549837 learned_addr learned my addr 172.21.15.117:0/2524549837 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:18:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b03400700 1 -- 172.21.15.117:0/2524549837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b04107520 msgr2=0x7f8b0419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b03400700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b04107520 0x7f8b0419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b03400700 1 -- 172.21.15.117:0/2524549837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00095f0 con 0x7f8b04108e10 2024-04-20T11:18:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b03400700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 0x7f8b0419d710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8af00092e0 tx=0x7f8af000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b00c00700 1 -- 172.21.15.117:0/2524549837 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8af001d070 con 0x7f8b04108e10 2024-04-20T11:18:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b00c00700 1 -- 172.21.15.117:0/2524549837 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8af00052a0 con 0x7f8b04108e10 2024-04-20T11:18:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b041a1cf0 con 0x7f8b04108e10 2024-04-20T11:18:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b00c00700 1 -- 172.21.15.117:0/2524549837 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8af0017400 con 0x7f8b04108e10 2024-04-20T11:18:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.509+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b041a2260 con 0x7f8b04108e10 2024-04-20T11:18:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.510+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0410f870 con 0x7f8b04108e10 2024-04-20T11:18:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.512+0000 7f8b00c00700 1 -- 172.21.15.117:0/2524549837 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8af0003710 con 0x7f8b04108e10 2024-04-20T11:18:58.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.513+0000 7f8b00c00700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ae806ecd0 0x7f8ae8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:58.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.513+0000 7f8b00c00700 1 -- 172.21.15.117:0/2524549837 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f8af00900b0 con 0x7f8b04108e10 2024-04-20T11:18:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.513+0000 7f8b03e00700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ae806ecd0 0x7f8ae8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.514+0000 7f8b03e00700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ae806ecd0 0x7f8ae8071190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009c60 tx=0x7f8af4009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:58.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.518+0000 7f8b00c00700 1 -- 172.21.15.117:0/2524549837 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8af0058ea0 con 0x7f8b04108e10 2024-04-20T11:18:58.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.823+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8b0419e490 con 0x7f8b04108e10 2024-04-20T11:18:58.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.823+0000 7f8b00c00700 1 -- 172.21.15.117:0/2524549837 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1480 (secure 0 0 0) 0x7f8af005c560 con 0x7f8b04108e10 2024-04-20T11:18:58.823 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded 2024-04-20T11:18:58.823 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:18:58.823 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:18:58.823 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-20T11:18:58.823 INFO:teuthology.orchestra.run.smithi117.stdout: osd.5 (root=default,host=smithi173) is down 2024-04-20T11:18:58.823 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-04-20T11:18:58.824 INFO:teuthology.orchestra.run.smithi117.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-04-20T11:18:58.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.826+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ae806ecd0 msgr2=0x7f8ae8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:58.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.826+0000 7f8b0ac00700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ae806ecd0 0x7f8ae8071190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009c60 tx=0x7f8af4009260 comp rx=0 tx=0).stop 2024-04-20T11:18:58.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 msgr2=0x7f8b0419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:58.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 0x7f8b0419d710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8af00092e0 tx=0x7f8af000b610 comp rx=0 tx=0).stop 2024-04-20T11:18:58.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 shutdown_connections 2024-04-20T11:18:58.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ae806ecd0 0x7f8ae8071190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b04107520 0x7f8b0419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 --2- 172.21.15.117:0/2524549837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b04108e10 0x7f8b0419d710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:58.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 >> 172.21.15.117:0/2524549837 conn(0x7f8b040768f0 msgr2=0x7f8b04077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:58.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 shutdown_connections 2024-04-20T11:18:58.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:18:58.827+0000 7f8b0ac00700 1 -- 172.21.15.117:0/2524549837 wait complete. 2024-04-20T11:18:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:59 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:59 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2524549837' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:18:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:18:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:59.410 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:18:59 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:18:59.345+0000 7f24048527c0 -1 osd.5 75 log_to_monitors true 2024-04-20T11:18:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:59 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded (PG_DEGRADED) 2024-04-20T11:18:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:59 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2524549837' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:18:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:18:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:18:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:00 smithi117 ceph-mon[99036]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:00 smithi117 ceph-mon[99036]: pgmap v188: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:19:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:00 smithi117 ceph-mon[99036]: from='osd.5 [v2:172.21.15.173:6816/3281347854,v1:172.21.15.173:6817/3281347854]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:19:00.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:00 smithi117 ceph-mon[99036]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:19:00.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:19:00 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:19:00.142+0000 7f23eec00700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:19:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:00 smithi173 ceph-mon[74165]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:00 smithi173 ceph-mon[74165]: pgmap v188: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:19:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:00 smithi173 ceph-mon[74165]: from='osd.5 [v2:172.21.15.173:6816/3281347854,v1:172.21.15.173:6817/3281347854]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:19:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:00 smithi173 ceph-mon[74165]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:19:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:01 smithi117 ceph-mon[99036]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T11:19:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:01 smithi117 ceph-mon[99036]: osdmap e78: 6 total, 5 up, 6 in 2024-04-20T11:19:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:01 smithi117 ceph-mon[99036]: from='osd.5 [v2:172.21.15.173:6816/3281347854,v1:172.21.15.173:6817/3281347854]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:19:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:01 smithi117 ceph-mon[99036]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:19:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:01 smithi173 ceph-mon[74165]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T11:19:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:01 smithi173 ceph-mon[74165]: osdmap e78: 6 total, 5 up, 6 in 2024-04-20T11:19:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:01 smithi173 ceph-mon[74165]: from='osd.5 [v2:172.21.15.173:6816/3281347854,v1:172.21.15.173:6817/3281347854]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:19:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:01 smithi173 ceph-mon[74165]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-20T11:19:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:02.287 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:02 smithi173 ceph-mon[74165]: pgmap v190: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:19:02.287 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:02 smithi173 ceph-mon[74165]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:19:02.287 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:02 smithi173 ceph-mon[74165]: osd.5 [v2:172.21.15.173:6816/3281347854,v1:172.21.15.173:6817/3281347854] boot 2024-04-20T11:19:02.287 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:02 smithi173 ceph-mon[74165]: osdmap e79: 6 total, 6 up, 6 in 2024-04-20T11:19:02.287 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:02 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:19:02.287 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:02 smithi173 ceph-mon[74165]: osdmap e80: 6 total, 6 up, 6 in 2024-04-20T11:19:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:02 smithi117 ceph-mon[99036]: pgmap v190: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-20T11:19:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:02 smithi117 ceph-mon[99036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-20T11:19:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:02 smithi117 ceph-mon[99036]: osd.5 [v2:172.21.15.173:6816/3281347854,v1:172.21.15.173:6817/3281347854] boot 2024-04-20T11:19:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:02 smithi117 ceph-mon[99036]: osdmap e79: 6 total, 6 up, 6 in 2024-04-20T11:19:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:02 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:19:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:02 smithi117 ceph-mon[99036]: osdmap e80: 6 total, 6 up, 6 in 2024-04-20T11:19:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:04 smithi173 ceph-mon[74165]: pgmap v193: 65 pgs: 4 peering, 12 active+undersized+degraded, 12 active+undersized, 37 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 45/324 objects degraded (13.889%) 2024-04-20T11:19:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:04 smithi173 ceph-mon[74165]: Health check update: Degraded data redundancy: 45/324 objects degraded (13.889%), 12 pgs degraded (PG_DEGRADED) 2024-04-20T11:19:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:04 smithi117 ceph-mon[99036]: pgmap v193: 65 pgs: 4 peering, 12 active+undersized+degraded, 12 active+undersized, 37 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 45/324 objects degraded (13.889%) 2024-04-20T11:19:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:04 smithi117 ceph-mon[99036]: Health check update: Degraded data redundancy: 45/324 objects degraded (13.889%), 12 pgs degraded (PG_DEGRADED) 2024-04-20T11:19:06.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:06 smithi173 ceph-mon[74165]: pgmap v194: 65 pgs: 4 peering, 10 active+undersized+degraded, 11 active+undersized, 40 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 40/324 objects degraded (12.346%) 2024-04-20T11:19:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:06 smithi117 ceph-mon[99036]: pgmap v194: 65 pgs: 4 peering, 10 active+undersized+degraded, 11 active+undersized, 40 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 40/324 objects degraded (12.346%) 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:07.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all osd 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-20T11:19:07.886 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-20T11:19:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: Upgrade: Setting require_osd_release to 19 squid 2024-04-20T11:19:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-20T11:19:07.887 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-mon[99036]: pgmap v195: 65 pgs: 4 peering, 61 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all osd 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-20T11:19:07.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: Upgrade: Setting require_osd_release to 19 squid 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-20T11:19:07.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:07 smithi173 ceph-mon[74165]: pgmap v195: 65 pgs: 4 peering, 61 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:08.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:07 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:19:07.882+0000 7fe6a6e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 40/324 objects degraded (12.346%), 10 pgs degraded) 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: osdmap e81: 6 total, 6 up, 6 in 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: osdmap e82: 6 total, 6 up, 6 in 2024-04-20T11:19:08.735 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:08 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-20T11:19:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 40/324 objects degraded (12.346%), 10 pgs degraded) 2024-04-20T11:19:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-20T11:19:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-20T11:19:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: osdmap e81: 6 total, 6 up, 6 in 2024-04-20T11:19:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-20T11:19:08.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T11:19:08.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T11:19:08.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:19:08.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T11:19:08.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: osdmap e82: 6 total, 6 up, 6 in 2024-04-20T11:19:08.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:08 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-20T11:19:10.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:09 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.173:6824/2867925301,v1:172.21.15.173:6825/2867925301] up:boot 2024-04-20T11:19:10.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:09 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6826/175884674,v1:172.21.15.117:6827/175884674] up:boot 2024-04-20T11:19:10.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:09 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:19:10.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:19:10.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:19:10.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:09 smithi117 ceph-mon[99036]: pgmap v198: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:09 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.173:6824/2867925301,v1:172.21.15.173:6825/2867925301] up:boot 2024-04-20T11:19:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:09 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6826/175884674,v1:172.21.15.117:6827/175884674] up:boot 2024-04-20T11:19:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:09 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:19:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:19:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:19:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:09 smithi173 ceph-mon[74165]: pgmap v198: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:11.630 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:11 smithi117 ceph-mon[99036]: Upgrade: Updating mds.cephfs.smithi117.umapup 2024-04-20T11:19:11.630 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:11 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:11.631 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:11 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.umapup", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:11.631 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:11 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:11.631 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:11 smithi117 ceph-mon[99036]: Deploying daemon mds.cephfs.smithi117.umapup on smithi117 2024-04-20T11:19:11.631 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:11 smithi117 ceph-mon[99036]: pgmap v199: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:11 smithi173 ceph-mon[74165]: Upgrade: Updating mds.cephfs.smithi117.umapup 2024-04-20T11:19:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:11 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:11 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.umapup", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:11 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:11 smithi173 ceph-mon[74165]: Deploying daemon mds.cephfs.smithi117.umapup on smithi117 2024-04-20T11:19:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:11 smithi173 ceph-mon[74165]: pgmap v199: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:19:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:14 smithi117 ceph-mon[99036]: pgmap v200: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:19:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:14 smithi173 ceph-mon[74165]: pgmap v200: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:15.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:15 smithi117 ceph-mon[99036]: osdmap e83: 6 total, 6 up, 6 in 2024-04-20T11:19:15.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:15 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-20T11:19:15.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:15 smithi173 ceph-mon[74165]: osdmap e83: 6 total, 6 up, 6 in 2024-04-20T11:19:15.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:15 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-20T11:19:16.351 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:16 smithi117 ceph-mon[99036]: pgmap v202: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:16.394 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:16 smithi173 ceph-mon[74165]: pgmap v202: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:18.342 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:18 smithi117 ceph-mon[99036]: pgmap v203: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:18 smithi173 ceph-mon[74165]: pgmap v203: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:19.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:19 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:19.855 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:19 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:19.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:19 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:19.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:19 smithi117 ceph-mon[99036]: pgmap v204: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:19.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:19 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:boot 2024-04-20T11:19:19.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:19 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:19:19.856 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:19 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:19:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:19 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:19 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:19.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:19 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:19.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:19 smithi173 ceph-mon[74165]: pgmap v204: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:19.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:19 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:boot 2024-04-20T11:19:19.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:19 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:19:19.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:19 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:19:22.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:22 smithi117 ceph-mon[99036]: pgmap v205: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:22.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:22 smithi173 ceph-mon[74165]: pgmap v205: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:24.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:24 smithi117 ceph-mon[99036]: pgmap v206: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:24 smithi173 ceph-mon[74165]: pgmap v206: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:26.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:26 smithi117 ceph-mon[99036]: pgmap v207: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:26.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:26.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:26 smithi173 ceph-mon[74165]: pgmap v207: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:28 smithi117 ceph-mon[99036]: pgmap v208: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:19:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:28 smithi173 ceph-mon[74165]: pgmap v208: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:19:29.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.000+0000 7f17ce200700 1 -- 172.21.15.117:0/456760335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 msgr2=0x7f17c810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.000+0000 7f17ce200700 1 --2- 172.21.15.117:0/456760335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 0x7f17c810abf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009910 tx=0x7f17b0009c20 comp rx=0 tx=0).stop 2024-04-20T11:19:29.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.000+0000 7f17ce200700 1 -- 172.21.15.117:0/456760335 shutdown_connections 2024-04-20T11:19:29.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.000+0000 7f17ce200700 1 --2- 172.21.15.117:0/456760335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 0x7f17c810abf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.000+0000 7f17ce200700 1 --2- 172.21.15.117:0/456760335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17c8101de0 0x7f17c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.000+0000 7f17ce200700 1 -- 172.21.15.117:0/456760335 >> 172.21.15.117:0/456760335 conn(0x7f17c8076750 msgr2=0x7f17c80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:29.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.001+0000 7f17ce200700 1 -- 172.21.15.117:0/456760335 shutdown_connections 2024-04-20T11:19:29.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.001+0000 7f17ce200700 1 -- 172.21.15.117:0/456760335 wait complete. 2024-04-20T11:19:29.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.002+0000 7f17ce200700 1 Processor -- start 2024-04-20T11:19:29.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.002+0000 7f17ce200700 1 -- start start 2024-04-20T11:19:29.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.002+0000 7f17ce200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17c8101de0 0x7f17c80ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.002+0000 7f17ce200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 0x7f17c80fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.002+0000 7f17ce200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c81004c0 con 0x7f17c8102700 2024-04-20T11:19:29.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.002+0000 7f17ce200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8100600 con 0x7f17c8101de0 2024-04-20T11:19:29.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.003+0000 7f17bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 0x7f17c80fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.003+0000 7f17bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 0x7f17c80fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44324/0 (socket says 172.21.15.117:44324) 2024-04-20T11:19:29.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.003+0000 7f17bfe00700 1 -- 172.21.15.117:0/1094646672 learned_addr learned my addr 172.21.15.117:0/1094646672 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:19:29.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.003+0000 7f17c7400700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17c8101de0 0x7f17c80ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.003+0000 7f17bfe00700 1 -- 172.21.15.117:0/1094646672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17c8101de0 msgr2=0x7f17c80ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.003+0000 7f17bfe00700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17c8101de0 0x7f17c80ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.003+0000 7f17bfe00700 1 -- 172.21.15.117:0/1094646672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00095f0 con 0x7f17c8102700 2024-04-20T11:19:29.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.004+0000 7f17bfe00700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 0x7f17c80fff80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f17b000bd70 tx=0x7f17b000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:29.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.004+0000 7f17c4c00700 1 -- 172.21.15.117:0/1094646672 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b001d070 con 0x7f17c8102700 2024-04-20T11:19:29.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.004+0000 7f17c4c00700 1 -- 172.21.15.117:0/1094646672 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17b00056e0 con 0x7f17c8102700 2024-04-20T11:19:29.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.004+0000 7f17c4c00700 1 -- 172.21.15.117:0/1094646672 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b0017400 con 0x7f17c8102700 2024-04-20T11:19:29.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.004+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c8100880 con 0x7f17c8102700 2024-04-20T11:19:29.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.004+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c81a2f40 con 0x7f17c8102700 2024-04-20T11:19:29.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.005+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c8068db0 con 0x7f17c8102700 2024-04-20T11:19:29.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.006+0000 7f17c4c00700 1 -- 172.21.15.117:0/1094646672 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f17b000f460 con 0x7f17c8102700 2024-04-20T11:19:29.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.007+0000 7f17c4c00700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17ac06ecc0 0x7f17ac071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.007+0000 7f17c4c00700 1 -- 172.21.15.117:0/1094646672 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f17b0090230 con 0x7f17c8102700 2024-04-20T11:19:29.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.008+0000 7f17c7400700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17ac06ecc0 0x7f17ac071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.008+0000 7f17c7400700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17ac06ecc0 0x7f17ac071180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005950 tx=0x7f17b80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:29.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.013+0000 7f17c4c00700 1 -- 172.21.15.117:0/1094646672 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f17b00590b0 con 0x7f17c8102700 2024-04-20T11:19:29.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.270+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17c810dda0 con 0x7f17ac06ecc0 2024-04-20T11:19:29.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.273+0000 7f17c4c00700 1 -- 172.21.15.117:0/1094646672 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f17c810dda0 con 0x7f17ac06ecc0 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17ac06ecc0 msgr2=0x7f17ac071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17ac06ecc0 0x7f17ac071180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005950 tx=0x7f17b80058e0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 msgr2=0x7f17c80fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 0x7f17c80fff80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f17b000bd70 tx=0x7f17b000be50 comp rx=0 tx=0).stop 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 shutdown_connections 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17ac06ecc0 0x7f17ac071180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17c8101de0 0x7f17c80ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 --2- 172.21.15.117:0/1094646672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17c8102700 0x7f17c80fff80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 >> 172.21.15.117:0/1094646672 conn(0x7f17c8076750 msgr2=0x7f17c8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:29.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 shutdown_connections 2024-04-20T11:19:29.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.276+0000 7f17ce200700 1 -- 172.21.15.117:0/1094646672 wait complete. 2024-04-20T11:19:29.293 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:19:29.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.436+0000 7fa841400700 1 -- 172.21.15.117:0/3327195644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c100e10 msgr2=0x7fa83c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.436+0000 7fa841400700 1 --2- 172.21.15.117:0/3327195644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c100e10 0x7fa83c1011f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009910 tx=0x7fa82c009c20 comp rx=0 tx=0).stop 2024-04-20T11:19:29.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.436+0000 7fa841400700 1 -- 172.21.15.117:0/3327195644 shutdown_connections 2024-04-20T11:19:29.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.436+0000 7fa841400700 1 --2- 172.21.15.117:0/3327195644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa83c1017c0 0x7fa83c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.436+0000 7fa841400700 1 --2- 172.21.15.117:0/3327195644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c100e10 0x7fa83c1011f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.436+0000 7fa841400700 1 -- 172.21.15.117:0/3327195644 >> 172.21.15.117:0/3327195644 conn(0x7fa83c0fc6c0 msgr2=0x7fa83c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:29.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.436+0000 7fa841400700 1 -- 172.21.15.117:0/3327195644 shutdown_connections 2024-04-20T11:19:29.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.437+0000 7fa841400700 1 -- 172.21.15.117:0/3327195644 wait complete. 2024-04-20T11:19:29.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.437+0000 7fa841400700 1 Processor -- start 2024-04-20T11:19:29.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.438+0000 7fa841400700 1 -- start start 2024-04-20T11:19:29.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.438+0000 7fa841400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa83c100e10 0x7fa83c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.438+0000 7fa841400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c1017c0 0x7fa83c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.438+0000 7fa841400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c199ab0 con 0x7fa83c1017c0 2024-04-20T11:19:29.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.438+0000 7fa841400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c19e6c0 con 0x7fa83c100e10 2024-04-20T11:19:29.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.438+0000 7fa839e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c1017c0 0x7fa83c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.438+0000 7fa839e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c1017c0 0x7fa83c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44346/0 (socket says 172.21.15.117:44346) 2024-04-20T11:19:29.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.438+0000 7fa839e00700 1 -- 172.21.15.117:0/3319340769 learned_addr learned my addr 172.21.15.117:0/3319340769 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:19:29.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.439+0000 7fa83a800700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa83c100e10 0x7fa83c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.439+0000 7fa839e00700 1 -- 172.21.15.117:0/3319340769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa83c100e10 msgr2=0x7fa83c198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.439+0000 7fa839e00700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa83c100e10 0x7fa83c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.439+0000 7fa839e00700 1 -- 172.21.15.117:0/3319340769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0095f0 con 0x7fa83c1017c0 2024-04-20T11:19:29.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.439+0000 7fa839e00700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c1017c0 0x7fa83c199420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa82800d8d0 tx=0x7fa82800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:29.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.439+0000 7fa833400700 1 -- 172.21.15.117:0/3319340769 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa828004a00 con 0x7fa83c1017c0 2024-04-20T11:19:29.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.439+0000 7fa833400700 1 -- 172.21.15.117:0/3319340769 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8280054e0 con 0x7fa83c1017c0 2024-04-20T11:19:29.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.439+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c19e9a0 con 0x7fa83c1017c0 2024-04-20T11:19:29.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.440+0000 7fa833400700 1 -- 172.21.15.117:0/3319340769 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa828009dd0 con 0x7fa83c1017c0 2024-04-20T11:19:29.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.440+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c19eef0 con 0x7fa83c1017c0 2024-04-20T11:19:29.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.441+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c1022f0 con 0x7fa83c1017c0 2024-04-20T11:19:29.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.442+0000 7fa833400700 1 -- 172.21.15.117:0/3319340769 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa828004fa0 con 0x7fa83c1017c0 2024-04-20T11:19:29.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.443+0000 7fa833400700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa82006ecc0 0x7fa820071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.443+0000 7fa833400700 1 -- 172.21.15.117:0/3319340769 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa82808e7b0 con 0x7fa83c1017c0 2024-04-20T11:19:29.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.444+0000 7fa83a800700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa82006ecc0 0x7fa820071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.444+0000 7fa83a800700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa82006ecc0 0x7fa820071180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00b5c0 tx=0x7fa82c005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:29.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.448+0000 7fa833400700 1 -- 172.21.15.117:0/3319340769 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa828057590 con 0x7fa83c1017c0 2024-04-20T11:19:29.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.695+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa83c10de70 con 0x7fa82006ecc0 2024-04-20T11:19:29.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.697+0000 7fa833400700 1 -- 172.21.15.117:0/3319340769 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa83c10de70 con 0x7fa82006ecc0 2024-04-20T11:19:29.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa82006ecc0 msgr2=0x7fa820071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa82006ecc0 0x7fa820071180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00b5c0 tx=0x7fa82c005960 comp rx=0 tx=0).stop 2024-04-20T11:19:29.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c1017c0 msgr2=0x7fa83c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c1017c0 0x7fa83c199420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa82800d8d0 tx=0x7fa82800dc90 comp rx=0 tx=0).stop 2024-04-20T11:19:29.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 shutdown_connections 2024-04-20T11:19:29.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa82006ecc0 0x7fa820071180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa83c100e10 0x7fa83c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 --2- 172.21.15.117:0/3319340769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa83c1017c0 0x7fa83c199420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 >> 172.21.15.117:0/3319340769 conn(0x7fa83c0fc6c0 msgr2=0x7fa83c074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:29.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 shutdown_connections 2024-04-20T11:19:29.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.701+0000 7fa841400700 1 -- 172.21.15.117:0/3319340769 wait complete. 2024-04-20T11:19:29.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.856+0000 7fa22e200700 1 -- 172.21.15.117:0/639055345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228103110 msgr2=0x7fa2281034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.856+0000 7fa22e200700 1 --2- 172.21.15.117:0/639055345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228103110 0x7fa2281034f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa218009910 tx=0x7fa218009c20 comp rx=0 tx=0).stop 2024-04-20T11:19:29.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.856+0000 7fa22e200700 1 -- 172.21.15.117:0/639055345 shutdown_connections 2024-04-20T11:19:29.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.856+0000 7fa22e200700 1 --2- 172.21.15.117:0/639055345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa228104a00 0x7fa228104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.856+0000 7fa22e200700 1 --2- 172.21.15.117:0/639055345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228103110 0x7fa2281034f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.856+0000 7fa22e200700 1 -- 172.21.15.117:0/639055345 >> 172.21.15.117:0/639055345 conn(0x7fa2280fe880 msgr2=0x7fa228100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:29.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.856+0000 7fa22e200700 1 -- 172.21.15.117:0/639055345 shutdown_connections 2024-04-20T11:19:29.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.856+0000 7fa22e200700 1 -- 172.21.15.117:0/639055345 wait complete. 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.857+0000 7fa22e200700 1 Processor -- start 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.857+0000 7fa22e200700 1 -- start start 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa22e200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa228103110 0x7fa228198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa22e200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228104a00 0x7fa228199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa22e200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2281999e0 con 0x7fa228104a00 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa22e200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22819d660 con 0x7fa228103110 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa226a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228104a00 0x7fa228199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa226a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228104a00 0x7fa228199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44368/0 (socket says 172.21.15.117:44368) 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa226a00700 1 -- 172.21.15.117:0/2894748856 learned_addr learned my addr 172.21.15.117:0/2894748856 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa227400700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa228103110 0x7fa228198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa226a00700 1 -- 172.21.15.117:0/2894748856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa228103110 msgr2=0x7fa228198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:29.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.858+0000 7fa226a00700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa228103110 0x7fa228198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:29.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.859+0000 7fa226a00700 1 -- 172.21.15.117:0/2894748856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180095f0 con 0x7fa228104a00 2024-04-20T11:19:29.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.859+0000 7fa226a00700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228104a00 0x7fa228199300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa21400d8d0 tx=0x7fa21400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.859+0000 7fa21fe00700 1 -- 172.21.15.117:0/2894748856 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa214009940 con 0x7fa228104a00 2024-04-20T11:19:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.859+0000 7fa21fe00700 1 -- 172.21.15.117:0/2894748856 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa214010460 con 0x7fa228104a00 2024-04-20T11:19:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.859+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22819d940 con 0x7fa228104a00 2024-04-20T11:19:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.859+0000 7fa21fe00700 1 -- 172.21.15.117:0/2894748856 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa21400f5f0 con 0x7fa228104a00 2024-04-20T11:19:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.859+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22819df10 con 0x7fa228104a00 2024-04-20T11:19:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.860+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22810b460 con 0x7fa228104a00 2024-04-20T11:19:29.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.862+0000 7fa21fe00700 1 -- 172.21.15.117:0/2894748856 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa2140105d0 con 0x7fa228104a00 2024-04-20T11:19:29.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.863+0000 7fa21fe00700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20c06ecd0 0x7fa20c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:29.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.863+0000 7fa21fe00700 1 -- 172.21.15.117:0/2894748856 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa21408ede0 con 0x7fa228104a00 2024-04-20T11:19:29.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.863+0000 7fa227400700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20c06ecd0 0x7fa20c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:29.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.864+0000 7fa227400700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20c06ecd0 0x7fa20c071190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa218006010 tx=0x7fa21800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:29.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:29.868+0000 7fa21fe00700 1 -- 172.21.15.117:0/2894748856 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa214057ba0 con 0x7fa228104a00 2024-04-20T11:19:30.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.094+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa22819a080 con 0x7fa20c06ecd0 2024-04-20T11:19:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.106+0000 7fa21fe00700 1 -- 172.21.15.117:0/2894748856 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa22819a080 con 0x7fa20c06ecd0 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (7m) 4s ago 21m 14.7M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (22m) 4s ago 22m 9336k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (20m) 31s ago 20m 8950k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (4m) 4s ago 22m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (4m) 31s ago 20m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (7m) 4s ago 21m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (14m) 4s ago 14m 19.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 09fdfa2f2105 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (12s) 4s ago 14m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (14m) 31s ago 14m 18.6M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (14m) 31s ago 14m 25.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:19:30.107 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (11m) 4s ago 23m 510M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (9m) 31s ago 20m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (5m) 4s ago 23m 56.9M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (5m) 31s ago 20m 38.1M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (9m) 4s ago 22m 8400k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (9m) 31s ago 20m 8128k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (3m) 4s ago 19m 84.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (3m) 4s ago 19m 86.2M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (2m) 4s ago 18m 83.7M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (2m) 31s ago 17m 88.5M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (79s) 31s ago 17m 83.3M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (36s) 31s ago 16m 11.2M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:19:30.108 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (8m) 4s ago 21m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:19:30.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.108+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20c06ecd0 msgr2=0x7fa20c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.108+0000 7fa22e200700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20c06ecd0 0x7fa20c071190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa218006010 tx=0x7fa21800b540 comp rx=0 tx=0).stop 2024-04-20T11:19:30.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.108+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228104a00 msgr2=0x7fa228199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.108+0000 7fa22e200700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228104a00 0x7fa228199300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa21400d8d0 tx=0x7fa21400dc90 comp rx=0 tx=0).stop 2024-04-20T11:19:30.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.109+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 shutdown_connections 2024-04-20T11:19:30.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.109+0000 7fa22e200700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20c06ecd0 0x7fa20c071190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.109+0000 7fa22e200700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa228103110 0x7fa228198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.109+0000 7fa22e200700 1 --2- 172.21.15.117:0/2894748856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa228104a00 0x7fa228199300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.109+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 >> 172.21.15.117:0/2894748856 conn(0x7fa2280fe880 msgr2=0x7fa2280ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:30.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.109+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 shutdown_connections 2024-04-20T11:19:30.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.109+0000 7fa22e200700 1 -- 172.21.15.117:0/2894748856 wait complete. 2024-04-20T11:19:30.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.259+0000 7f5083800700 1 -- 172.21.15.117:0/3120940857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 msgr2=0x7f507c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.259+0000 7f5083800700 1 --2- 172.21.15.117:0/3120940857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 0x7f507c1011f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5070009910 tx=0x7f5070009c20 comp rx=0 tx=0).stop 2024-04-20T11:19:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.259+0000 7f5083800700 1 -- 172.21.15.117:0/3120940857 shutdown_connections 2024-04-20T11:19:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.259+0000 7f5083800700 1 --2- 172.21.15.117:0/3120940857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1017c0 0x7f507c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.259+0000 7f5083800700 1 --2- 172.21.15.117:0/3120940857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 0x7f507c1011f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.259+0000 7f5083800700 1 -- 172.21.15.117:0/3120940857 >> 172.21.15.117:0/3120940857 conn(0x7f507c0fc6e0 msgr2=0x7f507c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.259+0000 7f5083800700 1 -- 172.21.15.117:0/3120940857 shutdown_connections 2024-04-20T11:19:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.260+0000 7f5083800700 1 -- 172.21.15.117:0/3120940857 wait complete. 2024-04-20T11:19:30.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.261+0000 7f5083800700 1 Processor -- start 2024-04-20T11:19:30.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.261+0000 7f5083800700 1 -- start start 2024-04-20T11:19:30.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.261+0000 7f5083800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 0x7f507c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.261+0000 7f5083800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1017c0 0x7f507c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.261+0000 7f5083800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c19d3c0 con 0x7f507c100e10 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.261+0000 7f5083800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1969a0 con 0x7f507c1017c0 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.262+0000 7f5080e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 0x7f507c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.262+0000 7f5080e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 0x7f507c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44386/0 (socket says 172.21.15.117:44386) 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.262+0000 7f5080e00700 1 -- 172.21.15.117:0/3303500206 learned_addr learned my addr 172.21.15.117:0/3303500206 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.262+0000 7f507be00700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1017c0 0x7f507c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.262+0000 7f5080e00700 1 -- 172.21.15.117:0/3303500206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1017c0 msgr2=0x7f507c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.262+0000 7f5080e00700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1017c0 0x7f507c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.262+0000 7f5080e00700 1 -- 172.21.15.117:0/3303500206 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700095f0 con 0x7f507c100e10 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.263+0000 7f5080e00700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 0x7f507c19c860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f50700092e0 tx=0x7f507000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:30.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.263+0000 7f5079600700 1 -- 172.21.15.117:0/3303500206 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f507001d070 con 0x7f507c100e10 2024-04-20T11:19:30.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.263+0000 7f5079600700 1 -- 172.21.15.117:0/3303500206 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50700056e0 con 0x7f507c100e10 2024-04-20T11:19:30.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.263+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c196c20 con 0x7f507c100e10 2024-04-20T11:19:30.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.263+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c197110 con 0x7f507c100e10 2024-04-20T11:19:30.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.263+0000 7f5079600700 1 -- 172.21.15.117:0/3303500206 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f507000fe80 con 0x7f507c100e10 2024-04-20T11:19:30.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.265+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c1091b0 con 0x7f507c100e10 2024-04-20T11:19:30.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.266+0000 7f5079600700 1 -- 172.21.15.117:0/3303500206 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f507000f6f0 con 0x7f507c100e10 2024-04-20T11:19:30.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.266+0000 7f5079600700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f506406ecc0 0x7f5064071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.267+0000 7f5079600700 1 -- 172.21.15.117:0/3303500206 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5070090320 con 0x7f507c100e10 2024-04-20T11:19:30.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.267+0000 7f507be00700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f506406ecc0 0x7f5064071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.268+0000 7f507be00700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f506406ecc0 0x7f5064071180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5068005950 tx=0x7f50680058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:30.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.275+0000 7f5079600700 1 -- 172.21.15.117:0/3303500206 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f507005cb90 con 0x7f507c100e10 2024-04-20T11:19:30.377 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:30 smithi117 ceph-mon[99036]: pgmap v209: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:30 smithi173 ceph-mon[74165]: pgmap v209: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:30.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.609+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f507c197e60 con 0x7f507c100e10 2024-04-20T11:19:30.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.609+0000 7f5079600700 1 -- 172.21.15.117:0/3303500206 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f507005c720 con 0x7f507c100e10 2024-04-20T11:19:30.610 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:19:30.610 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3, 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 1 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:19:30.611 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:19:30.612 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3, 2024-04-20T11:19:30.612 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 11 2024-04-20T11:19:30.612 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:19:30.612 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f506406ecc0 msgr2=0x7f5064071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f506406ecc0 0x7f5064071180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5068005950 tx=0x7f50680058e0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 msgr2=0x7f507c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 0x7f507c19c860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f50700092e0 tx=0x7f507000be20 comp rx=0 tx=0).stop 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 shutdown_connections 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f506406ecc0 0x7f5064071180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f507c100e10 0x7f507c19c860 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 --2- 172.21.15.117:0/3303500206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1017c0 0x7f507c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 >> 172.21.15.117:0/3303500206 conn(0x7f507c0fc6e0 msgr2=0x7f507c0fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 shutdown_connections 2024-04-20T11:19:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.612+0000 7f5083800700 1 -- 172.21.15.117:0/3303500206 wait complete. 2024-04-20T11:19:30.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.762+0000 7f55e9000700 1 -- 172.21.15.117:0/2511259233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4100e10 msgr2=0x7f55e41011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.762+0000 7f55e9000700 1 --2- 172.21.15.117:0/2511259233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4100e10 0x7f55e41011f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009910 tx=0x7f55d4009c20 comp rx=0 tx=0).stop 2024-04-20T11:19:30.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.762+0000 7f55e9000700 1 -- 172.21.15.117:0/2511259233 shutdown_connections 2024-04-20T11:19:30.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.762+0000 7f55e9000700 1 --2- 172.21.15.117:0/2511259233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e41017c0 0x7f55e4105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.762+0000 7f55e9000700 1 --2- 172.21.15.117:0/2511259233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4100e10 0x7f55e41011f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.762+0000 7f55e9000700 1 -- 172.21.15.117:0/2511259233 >> 172.21.15.117:0/2511259233 conn(0x7f55e40fc6c0 msgr2=0x7f55e40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:30.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.762+0000 7f55e9000700 1 -- 172.21.15.117:0/2511259233 shutdown_connections 2024-04-20T11:19:30.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.762+0000 7f55e9000700 1 -- 172.21.15.117:0/2511259233 wait complete. 2024-04-20T11:19:30.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.763+0000 7f55e9000700 1 Processor -- start 2024-04-20T11:19:30.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.764+0000 7f55e9000700 1 -- start start 2024-04-20T11:19:30.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.764+0000 7f55e9000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e4100e10 0x7f55e40731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.764+0000 7f55e9000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e41017c0 0x7f55e4073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.764+0000 7f55e9000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e4073c60 con 0x7f55e41017c0 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.764+0000 7f55e9000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e4073dd0 con 0x7f55e4100e10 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.764+0000 7f55e1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e41017c0 0x7f55e4073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.764+0000 7f55e1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e41017c0 0x7f55e4073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44404/0 (socket says 172.21.15.117:44404) 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.764+0000 7f55e1400700 1 -- 172.21.15.117:0/2861704689 learned_addr learned my addr 172.21.15.117:0/2861704689 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.765+0000 7f55e1e00700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e4100e10 0x7f55e40731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.765+0000 7f55e1400700 1 -- 172.21.15.117:0/2861704689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e4100e10 msgr2=0x7f55e40731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.765+0000 7f55e1400700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e4100e10 0x7f55e40731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.765+0000 7f55e1400700 1 -- 172.21.15.117:0/2861704689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d40095f0 con 0x7f55e41017c0 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.765+0000 7f55e1400700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e41017c0 0x7f55e4073720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b480 tx=0x7f55d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.765+0000 7f55daa00700 1 -- 172.21.15.117:0/2861704689 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55d00118c0 con 0x7f55e41017c0 2024-04-20T11:19:30.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.765+0000 7f55daa00700 1 -- 172.21.15.117:0/2861704689 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55d0010380 con 0x7f55e41017c0 2024-04-20T11:19:30.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.765+0000 7f55daa00700 1 -- 172.21.15.117:0/2861704689 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55d000f4f0 con 0x7f55e41017c0 2024-04-20T11:19:30.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.766+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e41a2df0 con 0x7f55e41017c0 2024-04-20T11:19:30.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.766+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e41a3150 con 0x7f55e41017c0 2024-04-20T11:19:30.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.767+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e41091b0 con 0x7f55e41017c0 2024-04-20T11:19:30.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.768+0000 7f55daa00700 1 -- 172.21.15.117:0/2861704689 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f55d00104f0 con 0x7f55e41017c0 2024-04-20T11:19:30.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.769+0000 7f55daa00700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f55c806ecc0 0x7f55c8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.769+0000 7f55daa00700 1 -- 172.21.15.117:0/2861704689 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f55d008ea40 con 0x7f55e41017c0 2024-04-20T11:19:30.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.770+0000 7f55e1e00700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f55c806ecc0 0x7f55c8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.770+0000 7f55e1e00700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f55c806ecc0 0x7f55c8071180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f55d400b5c0 tx=0x7f55d401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:30.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:30.774+0000 7f55daa00700 1 -- 172.21.15.117:0/2861704689 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f55d0057800 con 0x7f55e41017c0 2024-04-20T11:19:31.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.071+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f55e41022f0 con 0x7f55e41017c0 2024-04-20T11:19:31.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.072+0000 7f55daa00700 1 -- 172.21.15.117:0/2861704689 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f55d005aec0 con 0x7f55e41017c0 2024-04-20T11:19:31.073 INFO:teuthology.orchestra.run.smithi117.stdout:e16 2024-04-20T11:19:31.073 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:19:31.073 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:19:31.073 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:19:31.073 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:19:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:19:31.075 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:19:31.076 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:19:31.077 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:14472} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.117:6828/1089389303,v1:172.21.15.117:6829/1089389303] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:19:31.077 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:19:31.077 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:19:31.077 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44241} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/2867925301,v1:172.21.15.173:6825/2867925301] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f55c806ecc0 msgr2=0x7f55c8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f55c806ecc0 0x7f55c8071180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f55d400b5c0 tx=0x7f55d401a040 comp rx=0 tx=0).stop 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e41017c0 msgr2=0x7f55e4073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e41017c0 0x7f55e4073720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b480 tx=0x7f55d000b840 comp rx=0 tx=0).stop 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 shutdown_connections 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f55c806ecc0 0x7f55c8071180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e4100e10 0x7f55e40731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 --2- 172.21.15.117:0/2861704689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e41017c0 0x7f55e4073720 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 >> 172.21.15.117:0/2861704689 conn(0x7f55e40fc6c0 msgr2=0x7f55e40fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 shutdown_connections 2024-04-20T11:19:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.075+0000 7f55e9000700 1 -- 172.21.15.117:0/2861704689 wait complete. 2024-04-20T11:19:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 16 2024-04-20T11:19:31.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.223+0000 7f70b6a00700 1 -- 172.21.15.117:0/790280893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 msgr2=0x7f70b0105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.223+0000 7f70b6a00700 1 --2- 172.21.15.117:0/790280893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 0x7f70b0105ea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009910 tx=0x7f70a0009c20 comp rx=0 tx=0).stop 2024-04-20T11:19:31.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.223+0000 7f70b6a00700 1 -- 172.21.15.117:0/790280893 shutdown_connections 2024-04-20T11:19:31.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.223+0000 7f70b6a00700 1 --2- 172.21.15.117:0/790280893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70b00ffac0 0x7f70b00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.223+0000 7f70b6a00700 1 --2- 172.21.15.117:0/790280893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 0x7f70b0105ea0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.223+0000 7f70b6a00700 1 -- 172.21.15.117:0/790280893 >> 172.21.15.117:0/790280893 conn(0x7f70b00fb590 msgr2=0x7f70b00fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.223+0000 7f70b6a00700 1 -- 172.21.15.117:0/790280893 shutdown_connections 2024-04-20T11:19:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.223+0000 7f70b6a00700 1 -- 172.21.15.117:0/790280893 wait complete. 2024-04-20T11:19:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.225+0000 7f70b6a00700 1 Processor -- start 2024-04-20T11:19:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.225+0000 7f70b6a00700 1 -- start start 2024-04-20T11:19:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.225+0000 7f70b6a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70b00ffac0 0x7f70b019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.225+0000 7f70b6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 0x7f70b019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.225+0000 7f70b6a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b019d360 con 0x7f70b0105ac0 2024-04-20T11:19:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.225+0000 7f70b6a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0196940 con 0x7f70b00ffac0 2024-04-20T11:19:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.226+0000 7f70af400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 0x7f70b019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.226+0000 7f70af400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 0x7f70b019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44422/0 (socket says 172.21.15.117:44422) 2024-04-20T11:19:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.226+0000 7f70af400700 1 -- 172.21.15.117:0/1053569070 learned_addr learned my addr 172.21.15.117:0/1053569070 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:19:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.226+0000 7f70afe00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70b00ffac0 0x7f70b019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.226+0000 7f70af400700 1 -- 172.21.15.117:0/1053569070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70b00ffac0 msgr2=0x7f70b019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.226+0000 7f70af400700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70b00ffac0 0x7f70b019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.226+0000 7f70af400700 1 -- 172.21.15.117:0/1053569070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00095f0 con 0x7f70b0105ac0 2024-04-20T11:19:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.227+0000 7f70af400700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 0x7f70b019cd40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f709c00c8a0 tx=0x7f709c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.227+0000 7f70acc00700 1 -- 172.21.15.117:0/1053569070 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709c00cea0 con 0x7f70b0105ac0 2024-04-20T11:19:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.227+0000 7f70acc00700 1 -- 172.21.15.117:0/1053569070 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f709c004830 con 0x7f70b0105ac0 2024-04-20T11:19:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.227+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b0196c20 con 0x7f70b0105ac0 2024-04-20T11:19:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.227+0000 7f70acc00700 1 -- 172.21.15.117:0/1053569070 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709c0055b0 con 0x7f70b0105ac0 2024-04-20T11:19:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.227+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b0197170 con 0x7f70b0105ac0 2024-04-20T11:19:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.228+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70900052f0 con 0x7f70b0105ac0 2024-04-20T11:19:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.229+0000 7f70acc00700 1 -- 172.21.15.117:0/1053569070 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f709c00f450 con 0x7f70b0105ac0 2024-04-20T11:19:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.230+0000 7f70acc00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f709406ecc0 0x7f7094071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.230+0000 7f70acc00700 1 -- 172.21.15.117:0/1053569070 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f709c08ea20 con 0x7f70b0105ac0 2024-04-20T11:19:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.231+0000 7f70afe00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f709406ecc0 0x7f7094071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.231+0000 7f70afe00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f709406ecc0 0x7f7094071180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70a0006010 tx=0x7f70a000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:31.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.236+0000 7f70acc00700 1 -- 172.21.15.117:0/1053569070 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f709c05b4d0 con 0x7f70b0105ac0 2024-04-20T11:19:31.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:31 smithi117 ceph-mon[99036]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:31.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:31 smithi117 ceph-mon[99036]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:31.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:31 smithi117 ceph-mon[99036]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:31.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:31 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3303500206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:31 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2861704689' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:19:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:31 smithi173 ceph-mon[74165]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:31 smithi173 ceph-mon[74165]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:31 smithi173 ceph-mon[74165]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:31 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3303500206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:31 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2861704689' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:19:31.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.483+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7090000bc0 con 0x7f709406ecc0 2024-04-20T11:19:31.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.485+0000 7f70acc00700 1 -- 172.21.15.117:0/1053569070 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7090000bc0 con 0x7f709406ecc0 2024-04-20T11:19:31.486 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:19:31.486 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "13/23 daemons upgraded", 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:19:31.487 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:19:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.488+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f709406ecc0 msgr2=0x7f7094071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.488+0000 7f70b6a00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f709406ecc0 0x7f7094071180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70a0006010 tx=0x7f70a000b540 comp rx=0 tx=0).stop 2024-04-20T11:19:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.488+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 msgr2=0x7f70b019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.488+0000 7f70b6a00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 0x7f70b019cd40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f709c00c8a0 tx=0x7f709c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:19:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.489+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 shutdown_connections 2024-04-20T11:19:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.489+0000 7f70b6a00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f709406ecc0 0x7f7094071180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.489+0000 7f70b6a00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70b00ffac0 0x7f70b019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.489+0000 7f70b6a00700 1 --2- 172.21.15.117:0/1053569070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b0105ac0 0x7f70b019cd40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.489+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 >> 172.21.15.117:0/1053569070 conn(0x7f70b00fb590 msgr2=0x7f70b0069300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:31.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.489+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 shutdown_connections 2024-04-20T11:19:31.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.489+0000 7f70b6a00700 1 -- 172.21.15.117:0/1053569070 wait complete. 2024-04-20T11:19:31.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.636+0000 7f0fcf800700 1 -- 172.21.15.117:0/2367631496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 msgr2=0x7f0fc810abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.636+0000 7f0fcf800700 1 --2- 172.21.15.117:0/2367631496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 0x7f0fc810abb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009940 tx=0x7f0fb4009c50 comp rx=0 tx=0).stop 2024-04-20T11:19:31.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.636+0000 7f0fcf800700 1 -- 172.21.15.117:0/2367631496 shutdown_connections 2024-04-20T11:19:31.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.636+0000 7f0fcf800700 1 --2- 172.21.15.117:0/2367631496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 0x7f0fc810abb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.636+0000 7f0fcf800700 1 --2- 172.21.15.117:0/2367631496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc8101da0 0x7f0fc8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.636+0000 7f0fcf800700 1 -- 172.21.15.117:0/2367631496 >> 172.21.15.117:0/2367631496 conn(0x7f0fc8076750 msgr2=0x7f0fc80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:31.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.636+0000 7f0fcf800700 1 -- 172.21.15.117:0/2367631496 shutdown_connections 2024-04-20T11:19:31.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.637+0000 7f0fcf800700 1 -- 172.21.15.117:0/2367631496 wait complete. 2024-04-20T11:19:31.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.638+0000 7f0fcf800700 1 Processor -- start 2024-04-20T11:19:31.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.638+0000 7f0fcf800700 1 -- start start 2024-04-20T11:19:31.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.638+0000 7f0fcf800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc8101da0 0x7f0fc8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:31.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.638+0000 7f0fcf800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 0x7f0fc8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:31.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.638+0000 7f0fcf800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81998f0 con 0x7f0fc81026c0 2024-04-20T11:19:31.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.638+0000 7f0fcf800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc819c560 con 0x7f0fc8101da0 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.638+0000 7f0fc7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 0x7f0fc8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.638+0000 7f0fc7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 0x7f0fc8199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44440/0 (socket says 172.21.15.117:44440) 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.639+0000 7f0fc7e00700 1 -- 172.21.15.117:0/1756932192 learned_addr learned my addr 172.21.15.117:0/1756932192 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.639+0000 7f0fcce00700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc8101da0 0x7f0fc8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.639+0000 7f0fc7e00700 1 -- 172.21.15.117:0/1756932192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc8101da0 msgr2=0x7f0fc8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.639+0000 7f0fc7e00700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc8101da0 0x7f0fc8198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.639+0000 7f0fc7e00700 1 -- 172.21.15.117:0/1756932192 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb40095f0 con 0x7f0fc81026c0 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.639+0000 7f0fcce00700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc8101da0 0x7f0fc8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:19:31.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.639+0000 7f0fc7e00700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 0x7f0fc8199260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400b580 tx=0x7f0fb40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:31.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.640+0000 7f0fc5600700 1 -- 172.21.15.117:0/1756932192 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fb401d070 con 0x7f0fc81026c0 2024-04-20T11:19:31.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.640+0000 7f0fc5600700 1 -- 172.21.15.117:0/1756932192 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fb400bed0 con 0x7f0fc81026c0 2024-04-20T11:19:31.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.640+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc819c7e0 con 0x7f0fc81026c0 2024-04-20T11:19:31.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.640+0000 7f0fc5600700 1 -- 172.21.15.117:0/1756932192 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fb4021410 con 0x7f0fc81026c0 2024-04-20T11:19:31.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.640+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc819ccd0 con 0x7f0fc81026c0 2024-04-20T11:19:31.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.641+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc810a710 con 0x7f0fc81026c0 2024-04-20T11:19:31.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.642+0000 7f0fc5600700 1 -- 172.21.15.117:0/1756932192 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0fb400f810 con 0x7f0fc81026c0 2024-04-20T11:19:31.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.643+0000 7f0fc5600700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0fb006ecc0 0x7f0fb0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:31.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.643+0000 7f0fc5600700 1 -- 172.21.15.117:0/1756932192 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0fb4090180 con 0x7f0fc81026c0 2024-04-20T11:19:31.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.643+0000 7f0fcce00700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0fb006ecc0 0x7f0fb0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:31.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.644+0000 7f0fcce00700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0fb006ecc0 0x7f0fb0071180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc005fd0 tx=0x7f0fbc005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:31.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.648+0000 7f0fc5600700 1 -- 172.21.15.117:0/1756932192 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0fb4058ec0 con 0x7f0fc81026c0 2024-04-20T11:19:31.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.968+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0fc8199ff0 con 0x7f0fc81026c0 2024-04-20T11:19:31.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.969+0000 7f0fc5600700 1 -- 172.21.15.117:0/1756932192 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f0fb405c580 con 0x7f0fc81026c0 2024-04-20T11:19:31.970 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:19:31.970 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:19:31.970 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:19:31.970 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:19:31.970 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:19:31.970 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:19:31.970 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:19:31.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0fb006ecc0 msgr2=0x7f0fb0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0fb006ecc0 0x7f0fb0071180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc005fd0 tx=0x7f0fbc005dc0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 msgr2=0x7f0fc8199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:31.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 0x7f0fc8199260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400b580 tx=0x7f0fb40056a0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 shutdown_connections 2024-04-20T11:19:31.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0fb006ecc0 0x7f0fb0071180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc8101da0 0x7f0fc8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 --2- 172.21.15.117:0/1756932192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc81026c0 0x7f0fc8199260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:31.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.971+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 >> 172.21.15.117:0/1756932192 conn(0x7f0fc8076750 msgr2=0x7f0fc8101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:31.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.972+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 shutdown_connections 2024-04-20T11:19:31.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:19:31.972+0000 7f0fcf800700 1 -- 172.21.15.117:0/1756932192 wait complete. 2024-04-20T11:19:32.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:32 smithi117 ceph-mon[99036]: pgmap v210: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:32.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:32 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1756932192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:19:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:32 smithi173 ceph-mon[74165]: pgmap v210: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:32 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1756932192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:19:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:33 smithi117 ceph-mon[99036]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:33 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:33 smithi173 ceph-mon[74165]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:33 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:34 smithi173 ceph-mon[74165]: pgmap v211: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:34.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:34.477 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:34 smithi117 ceph-mon[99036]: pgmap v211: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:34.477 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:34.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:34.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:34.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:34.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:19:36.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:36 smithi117 ceph-mon[99036]: pgmap v212: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:36.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:36 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:36 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.miaplu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:36.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:36 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:36 smithi173 ceph-mon[74165]: pgmap v212: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:36 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:36 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi117.miaplu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:36 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:37 smithi173 ceph-mon[74165]: Upgrade: Updating mds.cephfs.smithi117.miaplu 2024-04-20T11:19:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:37 smithi173 ceph-mon[74165]: Deploying daemon mds.cephfs.smithi117.miaplu on smithi117 2024-04-20T11:19:37.592 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:37 smithi117 ceph-mon[99036]: Upgrade: Updating mds.cephfs.smithi117.miaplu 2024-04-20T11:19:37.592 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:37 smithi117 ceph-mon[99036]: Deploying daemon mds.cephfs.smithi117.miaplu on smithi117 2024-04-20T11:19:38.313 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:38 smithi117 ceph-mon[99036]: pgmap v213: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:38 smithi173 ceph-mon[74165]: pgmap v213: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:39.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:39 smithi117 ceph-mon[99036]: pgmap v214: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:39.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:39 smithi173 ceph-mon[74165]: pgmap v214: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:40.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:40 smithi117 ceph-mon[99036]: osdmap e84: 6 total, 6 up, 6 in 2024-04-20T11:19:40.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:40 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-20T11:19:40.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:40 smithi173 ceph-mon[74165]: osdmap e84: 6 total, 6 up, 6 in 2024-04-20T11:19:40.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:40 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-20T11:19:41.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:41 smithi117 ceph-mon[99036]: pgmap v216: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:41.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:41 smithi173 ceph-mon[74165]: pgmap v216: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:43.371 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:19:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:19:44.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:44 smithi173 ceph-mon[74165]: pgmap v217: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:44 smithi117 ceph-mon[99036]: pgmap v217: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:46.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:46 smithi117 ceph-mon[99036]: pgmap v218: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:46.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:46.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:46.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:46 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:46 smithi173 ceph-mon[74165]: pgmap v218: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:46 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:47.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:47 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:boot 2024-04-20T11:19:47.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:47 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:19:47.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:47 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:19:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:47 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:boot 2024-04-20T11:19:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:47 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:19:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:47 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:19:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:48 smithi173 ceph-mon[74165]: pgmap v219: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:48.565 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:48 smithi117 ceph-mon[99036]: pgmap v219: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:50 smithi173 ceph-mon[74165]: pgmap v220: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:50.592 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:50 smithi117 ceph-mon[99036]: pgmap v220: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:52 smithi173 ceph-mon[74165]: pgmap v221: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:52.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:52 smithi117 ceph-mon[99036]: pgmap v221: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:53.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:53 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:53 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:53 smithi117 ceph-mon[99036]: pgmap v222: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:53 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:53 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:53 smithi173 ceph-mon[74165]: pgmap v222: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:55.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:55.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:55 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:55.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:55 smithi117 ceph-mon[99036]: pgmap v223: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:55 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:19:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:55 smithi173 ceph-mon[74165]: pgmap v223: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:58.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:58 smithi117 ceph-mon[99036]: pgmap v224: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:58.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:19:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:19:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:58 smithi173 ceph-mon[74165]: pgmap v224: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:19:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:19:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:20:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:19:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:20:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:19:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:19:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:19:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:19:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:19:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:19:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-mon[99036]: pgmap v225: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-mon[99036]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-mon[99036]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-mon[99036]: fs cephfs is degraded 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-mon[99036]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-mon[99036]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-mon[99036]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:20:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:00 smithi117 ceph-mon[99036]: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:20:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:00 smithi173 ceph-mon[74165]: pgmap v225: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:00 smithi173 ceph-mon[74165]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:20:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:00 smithi173 ceph-mon[74165]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:20:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:00 smithi173 ceph-mon[74165]: fs cephfs is degraded 2024-04-20T11:20:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:00 smithi173 ceph-mon[74165]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:20:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:00 smithi173 ceph-mon[74165]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:20:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:00 smithi173 ceph-mon[74165]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:20:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:00 smithi173 ceph-mon[74165]: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:20:01.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:01.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:01.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:01.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:01.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:01.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:20:01.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:01.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:20:01.629 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:01.630 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:01.630 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:01.630 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:02.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.142+0000 7f477ca00700 1 -- 172.21.15.117:0/3684490783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 msgr2=0x7f477810bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.142+0000 7f477ca00700 1 --2- 172.21.15.117:0/3684490783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 0x7f477810bc50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4760009910 tx=0x7f4760009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:02.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.142+0000 7f477ca00700 1 -- 172.21.15.117:0/3684490783 shutdown_connections 2024-04-20T11:20:02.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.142+0000 7f477ca00700 1 --2- 172.21.15.117:0/3684490783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 0x7f477810bc50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.142+0000 7f477ca00700 1 --2- 172.21.15.117:0/3684490783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4778107250 0x7f4778107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.142+0000 7f477ca00700 1 -- 172.21.15.117:0/3684490783 >> 172.21.15.117:0/3684490783 conn(0x7f4778076810 msgr2=0x7f4778078c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:02.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.143+0000 7f477ca00700 1 -- 172.21.15.117:0/3684490783 shutdown_connections 2024-04-20T11:20:02.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.143+0000 7f477ca00700 1 -- 172.21.15.117:0/3684490783 wait complete. 2024-04-20T11:20:02.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.144+0000 7f477ca00700 1 Processor -- start 2024-04-20T11:20:02.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.144+0000 7f477ca00700 1 -- start start 2024-04-20T11:20:02.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.144+0000 7f477ca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4778107250 0x7f477819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.144+0000 7f477ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 0x7f477819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.144+0000 7f477ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477819dbc0 con 0x7f4778107c00 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.144+0000 7f477ca00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47781a1840 con 0x7f4778107250 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.145+0000 7f476fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 0x7f477819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.145+0000 7f476fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 0x7f477819d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39268/0 (socket says 172.21.15.117:39268) 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.145+0000 7f476fe00700 1 -- 172.21.15.117:0/2557489751 learned_addr learned my addr 172.21.15.117:0/2557489751 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.145+0000 7f4776800700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4778107250 0x7f477819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.145+0000 7f476fe00700 1 -- 172.21.15.117:0/2557489751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4778107250 msgr2=0x7f477819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.145+0000 7f476fe00700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4778107250 0x7f477819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.145+0000 7f476fe00700 1 -- 172.21.15.117:0/2557489751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47600095f0 con 0x7f4778107c00 2024-04-20T11:20:02.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.145+0000 7f4776800700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4778107250 0x7f477819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:20:02.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.146+0000 7f476fe00700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 0x7f477819d4e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f47600092e0 tx=0x7f476000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:02.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.146+0000 7f476f400700 1 -- 172.21.15.117:0/2557489751 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f476001d070 con 0x7f4778107c00 2024-04-20T11:20:02.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.146+0000 7f476f400700 1 -- 172.21.15.117:0/2557489751 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f476000f460 con 0x7f4778107c00 2024-04-20T11:20:02.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.146+0000 7f476f400700 1 -- 172.21.15.117:0/2557489751 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4760017610 con 0x7f4778107c00 2024-04-20T11:20:02.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.146+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47781a1ac0 con 0x7f4778107c00 2024-04-20T11:20:02.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.146+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47781a1fb0 con 0x7f4778107c00 2024-04-20T11:20:02.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.147+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477810f650 con 0x7f4778107c00 2024-04-20T11:20:02.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.148+0000 7f476f400700 1 -- 172.21.15.117:0/2557489751 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f476000f5d0 con 0x7f4778107c00 2024-04-20T11:20:02.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.149+0000 7f476f400700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f475c06ecc0 0x7f475c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.149+0000 7f476f400700 1 -- 172.21.15.117:0/2557489751 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f4760090230 con 0x7f4778107c00 2024-04-20T11:20:02.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.150+0000 7f4776800700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f475c06ecc0 0x7f475c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.150+0000 7f4776800700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f475c06ecc0 0x7f475c071180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4768005950 tx=0x7f4768005f50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:02.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.156+0000 7f476f400700 1 -- 172.21.15.117:0/2557489751 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4760059090 con 0x7f4778107c00 2024-04-20T11:20:02.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.403+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47781141f0 con 0x7f475c06ecc0 2024-04-20T11:20:02.403 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:02 smithi117 ceph-mon[99036]: pgmap v226: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:02.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.405+0000 7f476f400700 1 -- 172.21.15.117:0/2557489751 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f47781141f0 con 0x7f475c06ecc0 2024-04-20T11:20:02.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.407+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f475c06ecc0 msgr2=0x7f475c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.407+0000 7f477ca00700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f475c06ecc0 0x7f475c071180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4768005950 tx=0x7f4768005f50 comp rx=0 tx=0).stop 2024-04-20T11:20:02.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.407+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 msgr2=0x7f477819d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.407+0000 7f477ca00700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 0x7f477819d4e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f47600092e0 tx=0x7f476000be20 comp rx=0 tx=0).stop 2024-04-20T11:20:02.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.408+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 shutdown_connections 2024-04-20T11:20:02.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.408+0000 7f477ca00700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f475c06ecc0 0x7f475c071180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.408+0000 7f477ca00700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4778107250 0x7f477819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.408+0000 7f477ca00700 1 --2- 172.21.15.117:0/2557489751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4778107c00 0x7f477819d4e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.408+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 >> 172.21.15.117:0/2557489751 conn(0x7f4778076810 msgr2=0x7f4778076bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:02.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.408+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 shutdown_connections 2024-04-20T11:20:02.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.408+0000 7f477ca00700 1 -- 172.21.15.117:0/2557489751 wait complete. 2024-04-20T11:20:02.424 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:20:02.529 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:02 smithi173 ceph-mon[74165]: pgmap v226: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:02.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.568+0000 7ff7d2c00700 1 -- 172.21.15.117:0/1324215201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc104a00 msgr2=0x7ff7cc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.568+0000 7ff7d2c00700 1 --2- 172.21.15.117:0/1324215201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc104a00 0x7ff7cc104e80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009910 tx=0x7ff7b8009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:02.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.568+0000 7ff7d2c00700 1 -- 172.21.15.117:0/1324215201 shutdown_connections 2024-04-20T11:20:02.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.568+0000 7ff7d2c00700 1 --2- 172.21.15.117:0/1324215201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc104a00 0x7ff7cc104e80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.568+0000 7ff7d2c00700 1 --2- 172.21.15.117:0/1324215201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7cc103110 0x7ff7cc1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.568+0000 7ff7d2c00700 1 -- 172.21.15.117:0/1324215201 >> 172.21.15.117:0/1324215201 conn(0x7ff7cc0fe880 msgr2=0x7ff7cc100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:02.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.569+0000 7ff7d2c00700 1 -- 172.21.15.117:0/1324215201 shutdown_connections 2024-04-20T11:20:02.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.569+0000 7ff7d2c00700 1 -- 172.21.15.117:0/1324215201 wait complete. 2024-04-20T11:20:02.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.570+0000 7ff7d2c00700 1 Processor -- start 2024-04-20T11:20:02.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.570+0000 7ff7d2c00700 1 -- start start 2024-04-20T11:20:02.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.570+0000 7ff7d2c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc103110 0x7ff7cc198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.570+0000 7ff7d2c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7cc104a00 0x7ff7cc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.570+0000 7ff7d2c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc199980 con 0x7ff7cc103110 2024-04-20T11:20:02.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.570+0000 7ff7d2c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc19d650 con 0x7ff7cc104a00 2024-04-20T11:20:02.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.571+0000 7ff7cbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc103110 0x7ff7cc198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.571+0000 7ff7cbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc103110 0x7ff7cc198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39290/0 (socket says 172.21.15.117:39290) 2024-04-20T11:20:02.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.571+0000 7ff7cbe00700 1 -- 172.21.15.117:0/3831747564 learned_addr learned my addr 172.21.15.117:0/3831747564 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:02.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.571+0000 7ff7cb400700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7cc104a00 0x7ff7cc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.571+0000 7ff7cbe00700 1 -- 172.21.15.117:0/3831747564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7cc104a00 msgr2=0x7ff7cc1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.571+0000 7ff7cbe00700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7cc104a00 0x7ff7cc1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.571+0000 7ff7cbe00700 1 -- 172.21.15.117:0/3831747564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80095f0 con 0x7ff7cc103110 2024-04-20T11:20:02.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.571+0000 7ff7cbe00700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc103110 0x7ff7cc198d60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00d900 tx=0x7ff7bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:02.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.572+0000 7ff7c8c00700 1 -- 172.21.15.117:0/3831747564 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7bc004a00 con 0x7ff7cc103110 2024-04-20T11:20:02.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.572+0000 7ff7c8c00700 1 -- 172.21.15.117:0/3831747564 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7bc0054e0 con 0x7ff7cc103110 2024-04-20T11:20:02.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.572+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc19d930 con 0x7ff7cc103110 2024-04-20T11:20:02.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.572+0000 7ff7c8c00700 1 -- 172.21.15.117:0/3831747564 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7bc009d60 con 0x7ff7cc103110 2024-04-20T11:20:02.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.572+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc19df00 con 0x7ff7cc103110 2024-04-20T11:20:02.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.573+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc10b480 con 0x7ff7cc103110 2024-04-20T11:20:02.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.574+0000 7ff7c8c00700 1 -- 172.21.15.117:0/3831747564 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff7bc005000 con 0x7ff7cc103110 2024-04-20T11:20:02.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.575+0000 7ff7c8c00700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff7b006ecd0 0x7ff7b0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.575+0000 7ff7c8c00700 1 -- 172.21.15.117:0/3831747564 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff7bc08f0a0 con 0x7ff7cc103110 2024-04-20T11:20:02.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.575+0000 7ff7cb400700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff7b006ecd0 0x7ff7b0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.576+0000 7ff7cb400700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff7b006ecd0 0x7ff7b0071190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005b40 tx=0x7ff7b801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:02.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.580+0000 7ff7c8c00700 1 -- 172.21.15.117:0/3831747564 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff7bc057de0 con 0x7ff7cc103110 2024-04-20T11:20:02.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.824+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7cc1100d0 con 0x7ff7b006ecd0 2024-04-20T11:20:02.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.826+0000 7ff7c8c00700 1 -- 172.21.15.117:0/3831747564 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff7cc1100d0 con 0x7ff7b006ecd0 2024-04-20T11:20:02.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff7b006ecd0 msgr2=0x7ff7b0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff7b006ecd0 0x7ff7b0071190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005b40 tx=0x7ff7b801a040 comp rx=0 tx=0).stop 2024-04-20T11:20:02.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc103110 msgr2=0x7ff7cc198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc103110 0x7ff7cc198d60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00d900 tx=0x7ff7bc00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 shutdown_connections 2024-04-20T11:20:02.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff7b006ecd0 0x7ff7b0071190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7cc103110 0x7ff7cc198d60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 --2- 172.21.15.117:0/3831747564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7cc104a00 0x7ff7cc1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.829+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 >> 172.21.15.117:0/3831747564 conn(0x7ff7cc0fe880 msgr2=0x7ff7cc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:02.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.830+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 shutdown_connections 2024-04-20T11:20:02.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.830+0000 7ff7d2c00700 1 -- 172.21.15.117:0/3831747564 wait complete. 2024-04-20T11:20:02.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.993+0000 7fd0f6000700 1 -- 172.21.15.117:0/3278712733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 msgr2=0x7fd0f00736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.993+0000 7fd0f6000700 1 --2- 172.21.15.117:0/3278712733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 0x7fd0f00736f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009910 tx=0x7fd0e4009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:02.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.993+0000 7fd0f6000700 1 -- 172.21.15.117:0/3278712733 shutdown_connections 2024-04-20T11:20:02.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.993+0000 7fd0f6000700 1 --2- 172.21.15.117:0/3278712733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f0073c30 0x7fd0f010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.993+0000 7fd0f6000700 1 --2- 172.21.15.117:0/3278712733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 0x7fd0f00736f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.993+0000 7fd0f6000700 1 -- 172.21.15.117:0/3278712733 >> 172.21.15.117:0/3278712733 conn(0x7fd0f00fc500 msgr2=0x7fd0f00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:02.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.993+0000 7fd0f6000700 1 -- 172.21.15.117:0/3278712733 shutdown_connections 2024-04-20T11:20:02.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.993+0000 7fd0f6000700 1 -- 172.21.15.117:0/3278712733 wait complete. 2024-04-20T11:20:02.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.994+0000 7fd0f6000700 1 Processor -- start 2024-04-20T11:20:02.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.995+0000 7fd0f6000700 1 -- start start 2024-04-20T11:20:02.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.995+0000 7fd0f6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 0x7fd0f019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.995+0000 7fd0f6000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f0073c30 0x7fd0f019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.995+0000 7fd0f6000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f019f320 con 0x7fd0f0073310 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.995+0000 7fd0f6000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f0198990 con 0x7fd0f0073c30 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.995+0000 7fd0f4c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 0x7fd0f019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.995+0000 7fd0f4c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 0x7fd0f019e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39308/0 (socket says 172.21.15.117:39308) 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.995+0000 7fd0f4c00700 1 -- 172.21.15.117:0/3681185758 learned_addr learned my addr 172.21.15.117:0/3681185758 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0efe00700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f0073c30 0x7fd0f019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0f4c00700 1 -- 172.21.15.117:0/3681185758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f0073c30 msgr2=0x7fd0f019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0f4c00700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f0073c30 0x7fd0f019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0f4c00700 1 -- 172.21.15.117:0/3681185758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40095f0 con 0x7fd0f0073310 2024-04-20T11:20:02.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0f4c00700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 0x7fd0f019e850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40092e0 tx=0x7fd0e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:02.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0ed600700 1 -- 172.21.15.117:0/3681185758 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e401d070 con 0x7fd0f0073310 2024-04-20T11:20:02.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0ed600700 1 -- 172.21.15.117:0/3681185758 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0e400f460 con 0x7fd0f0073310 2024-04-20T11:20:02.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f0198c10 con 0x7fd0f0073310 2024-04-20T11:20:02.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.996+0000 7fd0ed600700 1 -- 172.21.15.117:0/3681185758 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e4017630 con 0x7fd0f0073310 2024-04-20T11:20:02.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.997+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f0199100 con 0x7fd0f0073310 2024-04-20T11:20:02.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.998+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f0107650 con 0x7fd0f0073310 2024-04-20T11:20:02.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:02.999+0000 7fd0ed600700 1 -- 172.21.15.117:0/3681185758 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd0e400f5d0 con 0x7fd0f0073310 2024-04-20T11:20:02.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.000+0000 7fd0ed600700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0d806ecc0 0x7fd0d8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.000+0000 7fd0ed600700 1 -- 172.21.15.117:0/3681185758 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd0e4090280 con 0x7fd0f0073310 2024-04-20T11:20:02.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.000+0000 7fd0efe00700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0d806ecc0 0x7fd0d8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.001+0000 7fd0efe00700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0d806ecc0 0x7fd0d8071180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005ea0 tx=0x7fd0dc005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.005+0000 7fd0ed600700 1 -- 172.21.15.117:0/3681185758 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd0e405cc10 con 0x7fd0f0073310 2024-04-20T11:20:03.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.243+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0f0199e50 con 0x7fd0d806ecc0 2024-04-20T11:20:03.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.275+0000 7fd0ed600700 1 -- 172.21.15.117:0/3681185758 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd0f0199e50 con 0x7fd0d806ecc0 2024-04-20T11:20:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:20:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (8m) 10s ago 22m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:20:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (22m) 10s ago 22m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:20:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (21m) 64s ago 21m 8950k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:20:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (4m) 10s ago 22m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:20:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (4m) 64s ago 21m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:20:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (7m) 10s ago 22m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:20:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (18s) 10s ago 15m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (45s) 10s ago 15m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (15m) 64s ago 15m 18.6M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (15m) 64s ago 15m 25.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 94e375cc4bc0 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (11m) 10s ago 24m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (10m) 64s ago 21m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (6m) 10s ago 24m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (5m) 64s ago 21m 38.1M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (9m) 10s ago 22m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (9m) 64s ago 21m 8128k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (4m) 10s ago 20m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (3m) 10s ago 19m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (3m) 10s ago 18m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (2m) 64s ago 18m 88.5M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:20:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (112s) 64s ago 17m 83.3M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:20:03.277 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (70s) 64s ago 17m 11.2M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:20:03.277 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (9m) 10s ago 21m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:20:03.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0d806ecc0 msgr2=0x7fd0d8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0d806ecc0 0x7fd0d8071180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005ea0 tx=0x7fd0dc005e30 comp rx=0 tx=0).stop 2024-04-20T11:20:03.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 msgr2=0x7fd0f019e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 0x7fd0f019e850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40092e0 tx=0x7fd0e400be20 comp rx=0 tx=0).stop 2024-04-20T11:20:03.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 shutdown_connections 2024-04-20T11:20:03.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0d806ecc0 0x7fd0d8071180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0073310 0x7fd0f019e850 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 --2- 172.21.15.117:0/3681185758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f0073c30 0x7fd0f019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 >> 172.21.15.117:0/3681185758 conn(0x7fd0f00fc500 msgr2=0x7fd0f0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:03.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.278+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 shutdown_connections 2024-04-20T11:20:03.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.279+0000 7fd0f6000700 1 -- 172.21.15.117:0/3681185758 wait complete. 2024-04-20T11:20:03.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.428+0000 7f0f83e00700 1 -- 172.21.15.117:0/3554504484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 msgr2=0x7f0f7c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.428+0000 7f0f83e00700 1 --2- 172.21.15.117:0/3554504484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 0x7f0f7c105810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009910 tx=0x7f0f68009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:03.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.428+0000 7f0f83e00700 1 -- 172.21.15.117:0/3554504484 shutdown_connections 2024-04-20T11:20:03.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.428+0000 7f0f83e00700 1 --2- 172.21.15.117:0/3554504484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 0x7f0f7c105810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.428+0000 7f0f83e00700 1 --2- 172.21.15.117:0/3554504484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f7c100e10 0x7f0f7c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.428+0000 7f0f83e00700 1 -- 172.21.15.117:0/3554504484 >> 172.21.15.117:0/3554504484 conn(0x7f0f7c0fc6a0 msgr2=0x7f0f7c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:03.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.429+0000 7f0f83e00700 1 -- 172.21.15.117:0/3554504484 shutdown_connections 2024-04-20T11:20:03.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.429+0000 7f0f83e00700 1 -- 172.21.15.117:0/3554504484 wait complete. 2024-04-20T11:20:03.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.430+0000 7f0f83e00700 1 Processor -- start 2024-04-20T11:20:03.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.430+0000 7f0f83e00700 1 -- start start 2024-04-20T11:20:03.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.430+0000 7f0f83e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f7c100e10 0x7f0f7c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.430+0000 7f0f83e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 0x7f0f7c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.430+0000 7f0f83e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c199990 con 0x7f0f7c1017c0 2024-04-20T11:20:03.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.430+0000 7f0f83e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c19d660 con 0x7f0f7c100e10 2024-04-20T11:20:03.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.431+0000 7f0f80a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 0x7f0f7c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.431+0000 7f0f80a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 0x7f0f7c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39318/0 (socket says 172.21.15.117:39318) 2024-04-20T11:20:03.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.431+0000 7f0f80a00700 1 -- 172.21.15.117:0/3675517780 learned_addr learned my addr 172.21.15.117:0/3675517780 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:03.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.431+0000 7f0f81400700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f7c100e10 0x7f0f7c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.431+0000 7f0f80a00700 1 -- 172.21.15.117:0/3675517780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f7c100e10 msgr2=0x7f0f7c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.431+0000 7f0f80a00700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f7c100e10 0x7f0f7c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.431+0000 7f0f80a00700 1 -- 172.21.15.117:0/3675517780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680095f0 con 0x7f0f7c1017c0 2024-04-20T11:20:03.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.431+0000 7f0f80a00700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 0x7f0f7c1992b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f68005b40 tx=0x7f0f6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.432+0000 7f0f7a000700 1 -- 172.21.15.117:0/3675517780 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f6801d070 con 0x7f0f7c1017c0 2024-04-20T11:20:03.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.432+0000 7f0f7a000700 1 -- 172.21.15.117:0/3675517780 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f6800f460 con 0x7f0f7c1017c0 2024-04-20T11:20:03.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.432+0000 7f0f7a000700 1 -- 172.21.15.117:0/3675517780 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f68017630 con 0x7f0f7c1017c0 2024-04-20T11:20:03.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.432+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c19d8e0 con 0x7f0f7c1017c0 2024-04-20T11:20:03.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.432+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c19ddd0 con 0x7f0f7c1017c0 2024-04-20T11:20:03.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.433+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c104f90 con 0x7f0f7c1017c0 2024-04-20T11:20:03.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.434+0000 7f0f7a000700 1 -- 172.21.15.117:0/3675517780 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0f68003a60 con 0x7f0f7c1017c0 2024-04-20T11:20:03.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.435+0000 7f0f7a000700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f6406ecd0 0x7f0f64071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.435+0000 7f0f7a000700 1 -- 172.21.15.117:0/3675517780 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0f680902a0 con 0x7f0f7c1017c0 2024-04-20T11:20:03.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.436+0000 7f0f81400700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f6406ecd0 0x7f0f64071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.436+0000 7f0f81400700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f6406ecd0 0x7f0f64071190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0f70005fd0 tx=0x7f0f7000a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.440+0000 7f0f7a000700 1 -- 172.21.15.117:0/3675517780 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0f68059060 con 0x7f0f7c1017c0 2024-04-20T11:20:03.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.752+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f7c1022f0 con 0x7f0f7c1017c0 2024-04-20T11:20:03.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.753+0000 7f0f7a000700 1 -- 172.21.15.117:0/3675517780 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f0f6805c720 con 0x7f0f7c1017c0 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2, 2024-04-20T11:20:03.753 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:20:03.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:20:03.754 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:20:03.754 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2, 2024-04-20T11:20:03.754 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 12 2024-04-20T11:20:03.754 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:20:03.754 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:20:03.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.755+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f6406ecd0 msgr2=0x7f0f64071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.755+0000 7f0f83e00700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f6406ecd0 0x7f0f64071190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0f70005fd0 tx=0x7f0f7000a560 comp rx=0 tx=0).stop 2024-04-20T11:20:03.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.755+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 msgr2=0x7f0f7c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.755+0000 7f0f83e00700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 0x7f0f7c1992b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f68005b40 tx=0x7f0f6800be20 comp rx=0 tx=0).stop 2024-04-20T11:20:03.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.756+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 shutdown_connections 2024-04-20T11:20:03.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.756+0000 7f0f83e00700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f6406ecd0 0x7f0f64071190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.756+0000 7f0f83e00700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f7c100e10 0x7f0f7c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.756+0000 7f0f83e00700 1 --2- 172.21.15.117:0/3675517780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f7c1017c0 0x7f0f7c1992b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.756+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 >> 172.21.15.117:0/3675517780 conn(0x7f0f7c0fc6a0 msgr2=0x7f0f7c0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:03.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.756+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 shutdown_connections 2024-04-20T11:20:03.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.756+0000 7f0f83e00700 1 -- 172.21.15.117:0/3675517780 wait complete. 2024-04-20T11:20:03.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.900+0000 7f0b11c00700 1 -- 172.21.15.117:0/2866095747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 msgr2=0x7f0b0c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.900+0000 7f0b11c00700 1 --2- 172.21.15.117:0/2866095747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 0x7f0b0c101200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009910 tx=0x7f0afc009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:03.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.900+0000 7f0b11c00700 1 -- 172.21.15.117:0/2866095747 shutdown_connections 2024-04-20T11:20:03.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.900+0000 7f0b11c00700 1 --2- 172.21.15.117:0/2866095747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b0c1017d0 0x7f0b0c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.900+0000 7f0b11c00700 1 --2- 172.21.15.117:0/2866095747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 0x7f0b0c101200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.900+0000 7f0b11c00700 1 -- 172.21.15.117:0/2866095747 >> 172.21.15.117:0/2866095747 conn(0x7f0b0c0fc6b0 msgr2=0x7f0b0c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:03.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.900+0000 7f0b11c00700 1 -- 172.21.15.117:0/2866095747 shutdown_connections 2024-04-20T11:20:03.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.900+0000 7f0b11c00700 1 -- 172.21.15.117:0/2866095747 wait complete. 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.901+0000 7f0b11c00700 1 Processor -- start 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.901+0000 7f0b11c00700 1 -- start start 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b11c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b11c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b0c1017d0 0x7f0b0c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b11c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c199a00 con 0x7f0b0c100e20 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b11c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c19d680 con 0x7f0b0c1017d0 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b0aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b0aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48806/0 (socket says 172.21.15.117:48806) 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b0aa00700 1 -- 172.21.15.117:0/3628508157 learned_addr learned my addr 172.21.15.117:0/3628508157 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b0a000700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b0c1017d0 0x7f0b0c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b0aa00700 1 -- 172.21.15.117:0/3628508157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b0c1017d0 msgr2=0x7f0b0c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b0aa00700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b0c1017d0 0x7f0b0c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.902+0000 7f0b0aa00700 1 -- 172.21.15.117:0/3628508157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc0095f0 con 0x7f0b0c100e20 2024-04-20T11:20:03.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.903+0000 7f0b0aa00700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0afc006010 tx=0x7f0afc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.903+0000 7f0b03400700 1 -- 172.21.15.117:0/3628508157 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0afc01d070 con 0x7f0b0c100e20 2024-04-20T11:20:03.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.903+0000 7f0b03400700 1 -- 172.21.15.117:0/3628508157 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0afc00f460 con 0x7f0b0c100e20 2024-04-20T11:20:03.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.903+0000 7f0b03400700 1 -- 172.21.15.117:0/3628508157 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0afc017630 con 0x7f0b0c100e20 2024-04-20T11:20:03.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.903+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c19d900 con 0x7f0b0c100e20 2024-04-20T11:20:03.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.903+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c19ddf0 con 0x7f0b0c100e20 2024-04-20T11:20:03.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.904+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c102300 con 0x7f0b0c100e20 2024-04-20T11:20:03.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.905+0000 7f0b03400700 1 -- 172.21.15.117:0/3628508157 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0afc00f5d0 con 0x7f0b0c100e20 2024-04-20T11:20:03.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.906+0000 7f0b03400700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0af006ecc0 0x7f0af0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.906+0000 7f0b03400700 1 -- 172.21.15.117:0/3628508157 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0afc090350 con 0x7f0b0c100e20 2024-04-20T11:20:03.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.907+0000 7f0b0a000700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0af006ecc0 0x7f0af0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.907+0000 7f0b0a000700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0af006ecc0 0x7f0af0071180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0af8006fd0 tx=0x7f0af8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:03.911+0000 7f0b03400700 1 -- 172.21.15.117:0/3628508157 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0afc0591d0 con 0x7f0b0c100e20 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: Upgrade: Updating mds.cephfs.smithi173.pybzht 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pybzht", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: Deploying daemon mds.cephfs.smithi173.pybzht on smithi173 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: pgmap v227: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:04.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:03 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3675517780' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:04.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.181+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b0c10de60 con 0x7f0b0c100e20 2024-04-20T11:20:04.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.182+0000 7f0b03400700 1 -- 172.21.15.117:0/3628508157 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1887 (secure 0 0 0) 0x7f0afc05c890 con 0x7f0b0c100e20 2024-04-20T11:20:04.182 INFO:teuthology.orchestra.run.smithi117.stdout:e18 2024-04-20T11:20:04.182 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:20:04.182 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:20:04.183 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:20:04.184 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:20:04.185 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44241} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/2867925301,v1:172.21.15.173:6825/2867925301] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:20:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0af006ecc0 msgr2=0x7f0af0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0af006ecc0 0x7f0af0071180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0af8006fd0 tx=0x7f0af8008040 comp rx=0 tx=0).stop 2024-04-20T11:20:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 msgr2=0x7f0b0c198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0afc006010 tx=0x7f0afc00be20 comp rx=0 tx=0).stop 2024-04-20T11:20:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 shutdown_connections 2024-04-20T11:20:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0af006ecc0 0x7f0af0071180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198de0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 --2- 172.21.15.117:0/3628508157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b0c1017d0 0x7f0b0c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.186+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 >> 172.21.15.117:0/3628508157 conn(0x7f0b0c0fc6b0 msgr2=0x7f0b0c074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.187+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 shutdown_connections 2024-04-20T11:20:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.187+0000 7f0b11c00700 1 -- 172.21.15.117:0/3628508157 wait complete. 2024-04-20T11:20:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 18 2024-04-20T11:20:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.344+0000 7fbc6d000700 1 -- 172.21.15.117:0/4262753154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 msgr2=0x7fbc68109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.344+0000 7fbc6d000700 1 --2- 172.21.15.117:0/4262753154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 0x7fbc68109290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009910 tx=0x7fbc54009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.344+0000 7fbc6d000700 1 -- 172.21.15.117:0/4262753154 shutdown_connections 2024-04-20T11:20:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.344+0000 7fbc6d000700 1 --2- 172.21.15.117:0/4262753154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 0x7fbc68109290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.344+0000 7fbc6d000700 1 --2- 172.21.15.117:0/4262753154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc68107520 0x7fbc68107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.344+0000 7fbc6d000700 1 -- 172.21.15.117:0/4262753154 >> 172.21.15.117:0/4262753154 conn(0x7fbc680768f0 msgr2=0x7fbc68078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.345+0000 7fbc6d000700 1 -- 172.21.15.117:0/4262753154 shutdown_connections 2024-04-20T11:20:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.345+0000 7fbc6d000700 1 -- 172.21.15.117:0/4262753154 wait complete. 2024-04-20T11:20:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.346+0000 7fbc6d000700 1 Processor -- start 2024-04-20T11:20:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.346+0000 7fbc6d000700 1 -- start start 2024-04-20T11:20:04.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.346+0000 7fbc6d000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc68107520 0x7fbc6819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:04.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.346+0000 7fbc6d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 0x7fbc6819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:04.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.346+0000 7fbc6d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6819ddf0 con 0x7fbc68108e10 2024-04-20T11:20:04.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.346+0000 7fbc6d000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc681a1a70 con 0x7fbc68107520 2024-04-20T11:20:04.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.347+0000 7fbc65400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 0x7fbc6819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:04.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.347+0000 7fbc65400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 0x7fbc6819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48826/0 (socket says 172.21.15.117:48826) 2024-04-20T11:20:04.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.347+0000 7fbc65400700 1 -- 172.21.15.117:0/2837301612 learned_addr learned my addr 172.21.15.117:0/2837301612 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.347+0000 7fbc65e00700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc68107520 0x7fbc6819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.347+0000 7fbc65400700 1 -- 172.21.15.117:0/2837301612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc68107520 msgr2=0x7fbc6819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.347+0000 7fbc65400700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc68107520 0x7fbc6819d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.347+0000 7fbc65400700 1 -- 172.21.15.117:0/2837301612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540095f0 con 0x7fbc68108e10 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.347+0000 7fbc65400700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 0x7fbc6819d710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbc54006010 tx=0x7fbc5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.348+0000 7fbc5ea00700 1 -- 172.21.15.117:0/2837301612 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5401d070 con 0x7fbc68108e10 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.348+0000 7fbc5ea00700 1 -- 172.21.15.117:0/2837301612 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc5400f460 con 0x7fbc68108e10 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.348+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc681a1cf0 con 0x7fbc68108e10 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.348+0000 7fbc5ea00700 1 -- 172.21.15.117:0/2837301612 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc54017630 con 0x7fbc68108e10 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.348+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc681a2260 con 0x7fbc68108e10 2024-04-20T11:20:04.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.349+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6810f870 con 0x7fbc68108e10 2024-04-20T11:20:04.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.350+0000 7fbc5ea00700 1 -- 172.21.15.117:0/2837301612 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbc5400f5d0 con 0x7fbc68108e10 2024-04-20T11:20:04.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.351+0000 7fbc5ea00700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc4c06ecd0 0x7fbc4c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:04.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.351+0000 7fbc5ea00700 1 -- 172.21.15.117:0/2837301612 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbc54090350 con 0x7fbc68108e10 2024-04-20T11:20:04.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.351+0000 7fbc65e00700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc4c06ecd0 0x7fbc4c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:04.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.352+0000 7fbc65e00700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc4c06ecd0 0x7fbc4c071190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbc58005fd0 tx=0x7fbc5800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:04.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.356+0000 7fbc5ea00700 1 -- 172.21.15.117:0/2837301612 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbc540591d0 con 0x7fbc68108e10 2024-04-20T11:20:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: Upgrade: Updating mds.cephfs.smithi173.pybzht 2024-04-20T11:20:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pybzht", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:20:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: Deploying daemon mds.cephfs.smithi173.pybzht on smithi173 2024-04-20T11:20:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: pgmap v227: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:04.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:03 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3675517780' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:04.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.604+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc6819e500 con 0x7fbc4c06ecd0 2024-04-20T11:20:04.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.606+0000 7fbc5ea00700 1 -- 172.21.15.117:0/2837301612 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbc6819e500 con 0x7fbc4c06ecd0 2024-04-20T11:20:04.605 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:20:04.605 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:20:04.605 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "14/23 daemons upgraded", 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:20:04.606 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:20:04.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc4c06ecd0 msgr2=0x7fbc4c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:04.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc4c06ecd0 0x7fbc4c071190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbc58005fd0 tx=0x7fbc5800a560 comp rx=0 tx=0).stop 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 msgr2=0x7fbc6819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 0x7fbc6819d710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbc54006010 tx=0x7fbc5400be20 comp rx=0 tx=0).stop 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 shutdown_connections 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc4c06ecd0 0x7fbc4c071190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc68107520 0x7fbc6819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 --2- 172.21.15.117:0/2837301612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc68108e10 0x7fbc6819d710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 >> 172.21.15.117:0/2837301612 conn(0x7fbc680768f0 msgr2=0x7fbc68077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 shutdown_connections 2024-04-20T11:20:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.609+0000 7fbc6d000700 1 -- 172.21.15.117:0/2837301612 wait complete. 2024-04-20T11:20:04.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.752+0000 7ff179000700 1 -- 172.21.15.117:0/3686034711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174103110 msgr2=0x7ff1741034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:04.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.752+0000 7ff179000700 1 --2- 172.21.15.117:0/3686034711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174103110 0x7ff1741034f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff164009910 tx=0x7ff164009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:04.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.752+0000 7ff179000700 1 -- 172.21.15.117:0/3686034711 shutdown_connections 2024-04-20T11:20:04.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.752+0000 7ff179000700 1 --2- 172.21.15.117:0/3686034711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff174104a00 0x7ff174104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.752+0000 7ff179000700 1 --2- 172.21.15.117:0/3686034711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174103110 0x7ff1741034f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.752+0000 7ff179000700 1 -- 172.21.15.117:0/3686034711 >> 172.21.15.117:0/3686034711 conn(0x7ff1740fe880 msgr2=0x7ff174100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:04.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.752+0000 7ff179000700 1 -- 172.21.15.117:0/3686034711 shutdown_connections 2024-04-20T11:20:04.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.752+0000 7ff179000700 1 -- 172.21.15.117:0/3686034711 wait complete. 2024-04-20T11:20:04.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.753+0000 7ff179000700 1 Processor -- start 2024-04-20T11:20:04.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.753+0000 7ff179000700 1 -- start start 2024-04-20T11:20:04.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff179000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff174103110 0x7ff174198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:04.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff179000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174104a00 0x7ff1741993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:04.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff179000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff174199a80 con 0x7ff174104a00 2024-04-20T11:20:04.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff179000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17419d750 con 0x7ff174103110 2024-04-20T11:20:04.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff171400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174104a00 0x7ff1741993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:04.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff171400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174104a00 0x7ff1741993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48838/0 (socket says 172.21.15.117:48838) 2024-04-20T11:20:04.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff171400700 1 -- 172.21.15.117:0/2153833690 learned_addr learned my addr 172.21.15.117:0/2153833690 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:04.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff171e00700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff174103110 0x7ff174198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:04.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff171400700 1 -- 172.21.15.117:0/2153833690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff174103110 msgr2=0x7ff174198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:04.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff171400700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff174103110 0x7ff174198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:04.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.754+0000 7ff171400700 1 -- 172.21.15.117:0/2153833690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1640095f0 con 0x7ff174104a00 2024-04-20T11:20:04.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.755+0000 7ff171400700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174104a00 0x7ff1741993a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff16000b4b0 tx=0x7ff16000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:04.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.755+0000 7ff16aa00700 1 -- 172.21.15.117:0/2153833690 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff160011800 con 0x7ff174104a00 2024-04-20T11:20:04.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.755+0000 7ff16aa00700 1 -- 172.21.15.117:0/2153833690 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff160011e20 con 0x7ff174104a00 2024-04-20T11:20:04.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.755+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17419da30 con 0x7ff174104a00 2024-04-20T11:20:04.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.755+0000 7ff16aa00700 1 -- 172.21.15.117:0/2153833690 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff16000f2e0 con 0x7ff174104a00 2024-04-20T11:20:04.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.755+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17419e000 con 0x7ff174104a00 2024-04-20T11:20:04.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.756+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17410b460 con 0x7ff174104a00 2024-04-20T11:20:04.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.758+0000 7ff16aa00700 1 -- 172.21.15.117:0/2153833690 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff160010380 con 0x7ff174104a00 2024-04-20T11:20:04.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.759+0000 7ff16aa00700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff15806ecd0 0x7ff158071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:04.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.759+0000 7ff16aa00700 1 -- 172.21.15.117:0/2153833690 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff16008ea70 con 0x7ff174104a00 2024-04-20T11:20:04.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.759+0000 7ff171e00700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff15806ecd0 0x7ff158071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:04.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.759+0000 7ff171e00700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff15806ecd0 0x7ff158071190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff164006010 tx=0x7ff16400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:04.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:04.764+0000 7ff16aa00700 1 -- 172.21.15.117:0/2153833690 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff1600577b0 con 0x7ff174104a00 2024-04-20T11:20:05.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.072+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff17419a170 con 0x7ff174104a00 2024-04-20T11:20:05.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.073+0000 7ff16aa00700 1 -- 172.21.15.117:0/2153833690 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7ff16005ae70 con 0x7ff174104a00 2024-04-20T11:20:05.072 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:20:05.073 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:20:05.073 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:20:05.073 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:20:05.073 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:20:05.073 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:20:05.073 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:20:05.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff15806ecd0 msgr2=0x7ff158071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff15806ecd0 0x7ff158071190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff164006010 tx=0x7ff16400b540 comp rx=0 tx=0).stop 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174104a00 msgr2=0x7ff1741993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174104a00 0x7ff1741993a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff16000b4b0 tx=0x7ff16000b870 comp rx=0 tx=0).stop 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 shutdown_connections 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff15806ecd0 0x7ff158071190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff174103110 0x7ff174198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 --2- 172.21.15.117:0/2153833690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff174104a00 0x7ff1741993a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 >> 172.21.15.117:0/2153833690 conn(0x7ff1740fe880 msgr2=0x7ff1740ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 shutdown_connections 2024-04-20T11:20:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:05.076+0000 7ff179000700 1 -- 172.21.15.117:0/2153833690 wait complete. 2024-04-20T11:20:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:04 smithi173 ceph-mon[74165]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:05.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:04 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3628508157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:20:05.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:04 smithi117 ceph-mon[99036]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:04 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3628508157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:20:06.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:05 smithi173 ceph-mon[74165]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:06.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:05 smithi173 ceph-mon[74165]: pgmap v228: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:06.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:05 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2153833690' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:20:06.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:05 smithi117 ceph-mon[99036]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:05 smithi117 ceph-mon[99036]: pgmap v228: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:05 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2153833690' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:20:07.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:07 smithi117 ceph-mon[99036]: osdmap e85: 6 total, 6 up, 6 in 2024-04-20T11:20:07.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:07 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-20T11:20:07.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:07 smithi173 ceph-mon[74165]: osdmap e85: 6 total, 6 up, 6 in 2024-04-20T11:20:07.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:07 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-20T11:20:08.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:08 smithi117 ceph-mon[99036]: pgmap v230: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:08.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:08 smithi173 ceph-mon[74165]: pgmap v230: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:10.381 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:10 smithi173 ceph-mon[74165]: pgmap v231: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:10.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:10 smithi117 ceph-mon[99036]: pgmap v231: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:11 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:11 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:11 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:20:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:11 smithi117 ceph-mon[99036]: pgmap v232: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:11 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] up:boot 2024-04-20T11:20:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:11 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:20:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:11 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:20:11.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:11 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:11.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:11 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:11.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:11 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:20:11.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:11 smithi173 ceph-mon[74165]: pgmap v232: 65 pgs: 65 active+clean; 260 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:11.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:11 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] up:boot 2024-04-20T11:20:11.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:11 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:20:11.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:11 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:20:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:20:13.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:20:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:14 smithi117 ceph-mon[99036]: pgmap v233: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:14.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:14 smithi173 ceph-mon[74165]: pgmap v233: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:16.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:16 smithi117 ceph-mon[99036]: pgmap v234: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:16.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:16 smithi173 ceph-mon[74165]: pgmap v234: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:18 smithi117 ceph-mon[99036]: pgmap v235: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:18 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:18 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:18 smithi173 ceph-mon[74165]: pgmap v235: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:18 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:18 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:20:20.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:20 smithi117 ceph-mon[99036]: pgmap v236: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:20 smithi173 ceph-mon[74165]: pgmap v236: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:22.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:22 smithi117 ceph-mon[99036]: pgmap v237: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:22.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:22 smithi173 ceph-mon[74165]: pgmap v237: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:24.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:24 smithi117 ceph-mon[99036]: pgmap v238: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:24 smithi173 ceph-mon[74165]: pgmap v238: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:26.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:26 smithi117 ceph-mon[99036]: pgmap v239: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:26 smithi173 ceph-mon[74165]: pgmap v239: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:28.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:28 smithi117 ceph-mon[99036]: pgmap v240: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:20:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:28 smithi173 ceph-mon[74165]: pgmap v240: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:20:30.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:30 smithi117 ceph-mon[99036]: pgmap v241: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:30 smithi173 ceph-mon[74165]: pgmap v241: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:32.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:32 smithi117 ceph-mon[99036]: pgmap v242: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:32.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:32 smithi173 ceph-mon[74165]: pgmap v242: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:34.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:34 smithi173 ceph-mon[74165]: pgmap v243: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:34 smithi117 ceph-mon[99036]: pgmap v243: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:35.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.250+0000 7f062ce00700 1 -- 172.21.15.117:0/3292334046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 msgr2=0x7f0628103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:35.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.250+0000 7f062ce00700 1 --2- 172.21.15.117:0/3292334046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 0x7f0628103f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f061400aa20 tx=0x7f061400ad30 comp rx=0 tx=0).stop 2024-04-20T11:20:35.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.250+0000 7f062ce00700 1 -- 172.21.15.117:0/3292334046 shutdown_connections 2024-04-20T11:20:35.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.250+0000 7f062ce00700 1 --2- 172.21.15.117:0/3292334046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 0x7f0628103f20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.250+0000 7f062ce00700 1 --2- 172.21.15.117:0/3292334046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06280ff520 0x7f06280ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.250+0000 7f062ce00700 1 -- 172.21.15.117:0/3292334046 >> 172.21.15.117:0/3292334046 conn(0x7f0628075f00 msgr2=0x7f06280fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:35.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.250+0000 7f062ce00700 1 -- 172.21.15.117:0/3292334046 shutdown_connections 2024-04-20T11:20:35.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.250+0000 7f062ce00700 1 -- 172.21.15.117:0/3292334046 wait complete. 2024-04-20T11:20:35.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.251+0000 7f062ce00700 1 Processor -- start 2024-04-20T11:20:35.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.251+0000 7f062ce00700 1 -- start start 2024-04-20T11:20:35.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f062ce00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06280ff520 0x7f0628198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:35.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f062ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 0x7f06281992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:35.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f062ce00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06281999b0 con 0x7f06280ffed0 2024-04-20T11:20:35.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f062ce00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062819d680 con 0x7f06280ff520 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f0625400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 0x7f06281992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f0625400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 0x7f06281992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47912/0 (socket says 172.21.15.117:47912) 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f0625400700 1 -- 172.21.15.117:0/3246613447 learned_addr learned my addr 172.21.15.117:0/3246613447 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f0625e00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06280ff520 0x7f0628198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f0625400700 1 -- 172.21.15.117:0/3246613447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06280ff520 msgr2=0x7f0628198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.252+0000 7f0625400700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06280ff520 0x7f0628198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.253+0000 7f0625400700 1 -- 172.21.15.117:0/3246613447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061400a700 con 0x7f06280ffed0 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.253+0000 7f0625400700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 0x7f06281992d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0614006010 tx=0x7f06140056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.253+0000 7f061ea00700 1 -- 172.21.15.117:0/3246613447 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0614004c30 con 0x7f06280ffed0 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.253+0000 7f061ea00700 1 -- 172.21.15.117:0/3246613447 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0614004d90 con 0x7f06280ffed0 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.253+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062819d900 con 0x7f06280ffed0 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.253+0000 7f061ea00700 1 -- 172.21.15.117:0/3246613447 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0614016910 con 0x7f06280ffed0 2024-04-20T11:20:35.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.253+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062819ddf0 con 0x7f06280ffed0 2024-04-20T11:20:35.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.255+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0628107920 con 0x7f06280ffed0 2024-04-20T11:20:35.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.255+0000 7f061ea00700 1 -- 172.21.15.117:0/3246613447 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f061400eca0 con 0x7f06280ffed0 2024-04-20T11:20:35.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.256+0000 7f061ea00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f060c06ecd0 0x7f060c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:35.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.257+0000 7f061ea00700 1 -- 172.21.15.117:0/3246613447 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0614012070 con 0x7f06280ffed0 2024-04-20T11:20:35.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.257+0000 7f0625e00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f060c06ecd0 0x7f060c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:35.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.257+0000 7f0625e00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f060c06ecd0 0x7f060c071190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0618005950 tx=0x7f0618005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:35.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.262+0000 7f061ea00700 1 -- 172.21.15.117:0/3246613447 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0614059d70 con 0x7f06280ffed0 2024-04-20T11:20:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.505+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f062819a110 con 0x7f060c06ecd0 2024-04-20T11:20:35.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.509+0000 7f061ea00700 1 -- 172.21.15.117:0/3246613447 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f062819a110 con 0x7f060c06ecd0 2024-04-20T11:20:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.511+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f060c06ecd0 msgr2=0x7f060c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.511+0000 7f062ce00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f060c06ecd0 0x7f060c071190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0618005950 tx=0x7f0618005cf0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.511+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 msgr2=0x7f06281992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.511+0000 7f062ce00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 0x7f06281992d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0614006010 tx=0x7f06140056c0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.512+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 shutdown_connections 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.512+0000 7f062ce00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f060c06ecd0 0x7f060c071190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.512+0000 7f062ce00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06280ff520 0x7f0628198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.512+0000 7f062ce00700 1 --2- 172.21.15.117:0/3246613447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06280ffed0 0x7f06281992d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.512+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 >> 172.21.15.117:0/3246613447 conn(0x7f0628075f00 msgr2=0x7f0628102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.512+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 shutdown_connections 2024-04-20T11:20:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.512+0000 7f062ce00700 1 -- 172.21.15.117:0/3246613447 wait complete. 2024-04-20T11:20:35.528 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:20:35.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.667+0000 7f675b800700 1 -- 172.21.15.117:0/225624030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 msgr2=0x7f67541011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:35.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.667+0000 7f675b800700 1 --2- 172.21.15.117:0/225624030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 0x7f67541011f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6748009910 tx=0x7f6748009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:35.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.667+0000 7f675b800700 1 -- 172.21.15.117:0/225624030 shutdown_connections 2024-04-20T11:20:35.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.667+0000 7f675b800700 1 --2- 172.21.15.117:0/225624030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67541017c0 0x7f6754105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.667+0000 7f675b800700 1 --2- 172.21.15.117:0/225624030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 0x7f67541011f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.667+0000 7f675b800700 1 -- 172.21.15.117:0/225624030 >> 172.21.15.117:0/225624030 conn(0x7f67540fc6a0 msgr2=0x7f67540feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:35.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.667+0000 7f675b800700 1 -- 172.21.15.117:0/225624030 shutdown_connections 2024-04-20T11:20:35.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.668+0000 7f675b800700 1 -- 172.21.15.117:0/225624030 wait complete. 2024-04-20T11:20:35.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f675b800700 1 Processor -- start 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f675b800700 1 -- start start 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f675b800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 0x7f6754198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f675b800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67541017c0 0x7f67541993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f675b800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6754199a90 con 0x7f6754100e10 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f675b800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675419d710 con 0x7f67541017c0 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f6758e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 0x7f6754198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f6758e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 0x7f6754198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47916/0 (socket says 172.21.15.117:47916) 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.669+0000 7f6758e00700 1 -- 172.21.15.117:0/3928237259 learned_addr learned my addr 172.21.15.117:0/3928237259 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.670+0000 7f6758e00700 1 -- 172.21.15.117:0/3928237259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67541017c0 msgr2=0x7f67541993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.670+0000 7f6758e00700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67541017c0 0x7f67541993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.670+0000 7f6758e00700 1 -- 172.21.15.117:0/3928237259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67480095f0 con 0x7f6754100e10 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.670+0000 7f6758e00700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 0x7f6754198e70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f67480092e0 tx=0x7f674800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:35.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.670+0000 7f6751600700 1 -- 172.21.15.117:0/3928237259 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f674801d070 con 0x7f6754100e10 2024-04-20T11:20:35.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.670+0000 7f6751600700 1 -- 172.21.15.117:0/3928237259 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f674800f460 con 0x7f6754100e10 2024-04-20T11:20:35.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.670+0000 7f6751600700 1 -- 172.21.15.117:0/3928237259 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6748017630 con 0x7f6754100e10 2024-04-20T11:20:35.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.670+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675419d990 con 0x7f6754100e10 2024-04-20T11:20:35.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.671+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675419de80 con 0x7f6754100e10 2024-04-20T11:20:35.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.672+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67541022f0 con 0x7f6754100e10 2024-04-20T11:20:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.673+0000 7f6751600700 1 -- 172.21.15.117:0/3928237259 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f674800f5d0 con 0x7f6754100e10 2024-04-20T11:20:35.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.674+0000 7f6751600700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f673c06ecd0 0x7f673c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:35.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.674+0000 7f6751600700 1 -- 172.21.15.117:0/3928237259 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6748090340 con 0x7f6754100e10 2024-04-20T11:20:35.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.674+0000 7f6753e00700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f673c06ecd0 0x7f673c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:35.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.675+0000 7f6753e00700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f673c06ecd0 0x7f673c071190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f675419a380 tx=0x7f6740006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:35.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.679+0000 7f6751600700 1 -- 172.21.15.117:0/3928237259 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6748059100 con 0x7f6754100e10 2024-04-20T11:20:35.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.928+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f675410df90 con 0x7f673c06ecd0 2024-04-20T11:20:35.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.931+0000 7f6751600700 1 -- 172.21.15.117:0/3928237259 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f675410df90 con 0x7f673c06ecd0 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f673c06ecd0 msgr2=0x7f673c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f673c06ecd0 0x7f673c071190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f675419a380 tx=0x7f6740006d20 comp rx=0 tx=0).stop 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 msgr2=0x7f6754198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 0x7f6754198e70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f67480092e0 tx=0x7f674800be20 comp rx=0 tx=0).stop 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 shutdown_connections 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f673c06ecd0 0x7f673c071190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754100e10 0x7f6754198e70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 --2- 172.21.15.117:0/3928237259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67541017c0 0x7f67541993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 >> 172.21.15.117:0/3928237259 conn(0x7f67540fc6a0 msgr2=0x7f6754074470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.934+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 shutdown_connections 2024-04-20T11:20:35.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:35.935+0000 7f675b800700 1 -- 172.21.15.117:0/3928237259 wait complete. 2024-04-20T11:20:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.093+0000 7f3032000700 1 -- 172.21.15.117:0/1662048160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c073bd0 msgr2=0x7f302c10cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.093+0000 7f3032000700 1 --2- 172.21.15.117:0/1662048160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c073bd0 0x7f302c10cf10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3018009910 tx=0x7f3018009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.093+0000 7f3032000700 1 -- 172.21.15.117:0/1662048160 shutdown_connections 2024-04-20T11:20:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.093+0000 7f3032000700 1 --2- 172.21.15.117:0/1662048160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c073bd0 0x7f302c10cf10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.093+0000 7f3032000700 1 --2- 172.21.15.117:0/1662048160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f302c0732b0 0x7f302c073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.093+0000 7f3032000700 1 -- 172.21.15.117:0/1662048160 >> 172.21.15.117:0/1662048160 conn(0x7f302c0fc6a0 msgr2=0x7f302c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.093+0000 7f3032000700 1 -- 172.21.15.117:0/1662048160 shutdown_connections 2024-04-20T11:20:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.093+0000 7f3032000700 1 -- 172.21.15.117:0/1662048160 wait complete. 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.094+0000 7f3032000700 1 Processor -- start 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.094+0000 7f3032000700 1 -- start start 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.095+0000 7f3032000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c0732b0 0x7f302c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.095+0000 7f3032000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f302c073bd0 0x7f302c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.095+0000 7f3032000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c199a30 con 0x7f302c0732b0 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.095+0000 7f3032000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c19d700 con 0x7f302c073bd0 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.095+0000 7f302b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c0732b0 0x7f302c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.095+0000 7f302b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c0732b0 0x7f302c198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47932/0 (socket says 172.21.15.117:47932) 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.095+0000 7f302b400700 1 -- 172.21.15.117:0/1899253684 learned_addr learned my addr 172.21.15.117:0/1899253684 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:36.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.096+0000 7f302aa00700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f302c073bd0 0x7f302c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.096+0000 7f302b400700 1 -- 172.21.15.117:0/1899253684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f302c073bd0 msgr2=0x7f302c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.096+0000 7f302b400700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f302c073bd0 0x7f302c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.096+0000 7f302b400700 1 -- 172.21.15.117:0/1899253684 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30180095f0 con 0x7f302c0732b0 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.096+0000 7f302b400700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c0732b0 0x7f302c198e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f301c00b4b0 tx=0x7f301c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.096+0000 7f3023e00700 1 -- 172.21.15.117:0/1899253684 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c011800 con 0x7f302c0732b0 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.096+0000 7f3023e00700 1 -- 172.21.15.117:0/1899253684 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f301c011e20 con 0x7f302c0732b0 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.097+0000 7f3023e00700 1 -- 172.21.15.117:0/1899253684 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c00f2e0 con 0x7f302c0732b0 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.097+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c19d9e0 con 0x7f302c0732b0 2024-04-20T11:20:36.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.097+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c19df30 con 0x7f302c0732b0 2024-04-20T11:20:36.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.098+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c074960 con 0x7f302c0732b0 2024-04-20T11:20:36.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.099+0000 7f3023e00700 1 -- 172.21.15.117:0/1899253684 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f301c010380 con 0x7f302c0732b0 2024-04-20T11:20:36.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.100+0000 7f3023e00700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301006ecd0 0x7f3010071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.101+0000 7f3023e00700 1 -- 172.21.15.117:0/1899253684 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f301c08e960 con 0x7f302c0732b0 2024-04-20T11:20:36.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.101+0000 7f302aa00700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301006ecd0 0x7f3010071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:36.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.102+0000 7f302aa00700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301006ecd0 0x7f3010071190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3018005f50 tx=0x7f301801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:36.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.106+0000 7f3023e00700 1 -- 172.21.15.117:0/1899253684 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f301c057870 con 0x7f302c0732b0 2024-04-20T11:20:36.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.333+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f302c107800 con 0x7f301006ecd0 2024-04-20T11:20:36.334 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:36 smithi117 ceph-mon[99036]: pgmap v244: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:36.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.350+0000 7f3023e00700 1 -- 172.21.15.117:0/1899253684 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f302c107800 con 0x7f301006ecd0 2024-04-20T11:20:36.351 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:20:36.351 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (9m) 43s ago 23m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (23m) 43s ago 23m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (22m) 21s ago 22m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (5m) 43s ago 23m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (5m) 21s ago 22m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (8m) 43s ago 22m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (51s) 43s ago 15m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (78s) 43s ago 15m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (15m) 21s ago 15m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (26s) 21s ago 15m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (12m) 43s ago 24m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:20:36.352 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (10m) 21s ago 21m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (7m) 43s ago 25m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (6m) 21s ago 21m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (10m) 43s ago 23m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (10m) 21s ago 21m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (4m) 43s ago 20m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (4m) 43s ago 20m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (3m) 43s ago 19m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (3m) 21s ago 18m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (2m) 21s ago 18m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (103s) 21s ago 17m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:20:36.353 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (9m) 43s ago 22m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:20:36.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301006ecd0 msgr2=0x7f3010071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301006ecd0 0x7f3010071190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3018005f50 tx=0x7f301801a040 comp rx=0 tx=0).stop 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c0732b0 msgr2=0x7f302c198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c0732b0 0x7f302c198e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f301c00b4b0 tx=0x7f301c00b870 comp rx=0 tx=0).stop 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 shutdown_connections 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301006ecd0 0x7f3010071190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f302c0732b0 0x7f302c198e10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 --2- 172.21.15.117:0/1899253684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f302c073bd0 0x7f302c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 >> 172.21.15.117:0/1899253684 conn(0x7f302c0fc6a0 msgr2=0x7f302c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.353+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 shutdown_connections 2024-04-20T11:20:36.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.354+0000 7f3032000700 1 -- 172.21.15.117:0/1899253684 wait complete. 2024-04-20T11:20:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:36 smithi173 ceph-mon[74165]: pgmap v244: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:36.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.507+0000 7fbfd6a00700 1 -- 172.21.15.117:0/2074619573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 msgr2=0x7fbfd0103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.507+0000 7fbfd6a00700 1 --2- 172.21.15.117:0/2074619573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 0x7fbfd0103400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009910 tx=0x7fbfc0009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:36.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.507+0000 7fbfd6a00700 1 -- 172.21.15.117:0/2074619573 shutdown_connections 2024-04-20T11:20:36.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.507+0000 7fbfd6a00700 1 --2- 172.21.15.117:0/2074619573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfd01039d0 0x7fbfd0107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.507+0000 7fbfd6a00700 1 --2- 172.21.15.117:0/2074619573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 0x7fbfd0103400 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.507+0000 7fbfd6a00700 1 -- 172.21.15.117:0/2074619573 >> 172.21.15.117:0/2074619573 conn(0x7fbfd00fe8b0 msgr2=0x7fbfd0100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:36.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.507+0000 7fbfd6a00700 1 -- 172.21.15.117:0/2074619573 shutdown_connections 2024-04-20T11:20:36.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.507+0000 7fbfd6a00700 1 -- 172.21.15.117:0/2074619573 wait complete. 2024-04-20T11:20:36.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.508+0000 7fbfd6a00700 1 Processor -- start 2024-04-20T11:20:36.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.509+0000 7fbfd6a00700 1 -- start start 2024-04-20T11:20:36.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.509+0000 7fbfd6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 0x7fbfd019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.509+0000 7fbfd6a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfd01039d0 0x7fbfd019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.509+0000 7fbfd6a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd019f530 con 0x7fbfd0103020 2024-04-20T11:20:36.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.509+0000 7fbfd6a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0198b10 con 0x7fbfd01039d0 2024-04-20T11:20:36.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.509+0000 7fbfcfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 0x7fbfd019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.509+0000 7fbfcfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 0x7fbfd019e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47948/0 (socket says 172.21.15.117:47948) 2024-04-20T11:20:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.509+0000 7fbfcfe00700 1 -- 172.21.15.117:0/1287523837 learned_addr learned my addr 172.21.15.117:0/1287523837 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.510+0000 7fbfcfe00700 1 -- 172.21.15.117:0/1287523837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfd01039d0 msgr2=0x7fbfd019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.510+0000 7fbfcfe00700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfd01039d0 0x7fbfd019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.510+0000 7fbfcfe00700 1 -- 172.21.15.117:0/1287523837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00095f0 con 0x7fbfd0103020 2024-04-20T11:20:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.510+0000 7fbfcfe00700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 0x7fbfd019e9d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00092e0 tx=0x7fbfc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:36.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.510+0000 7fbfccc00700 1 -- 172.21.15.117:0/1287523837 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc001d070 con 0x7fbfd0103020 2024-04-20T11:20:36.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.510+0000 7fbfccc00700 1 -- 172.21.15.117:0/1287523837 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfc00056e0 con 0x7fbfd0103020 2024-04-20T11:20:36.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.510+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd0198d90 con 0x7fbfd0103020 2024-04-20T11:20:36.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.510+0000 7fbfccc00700 1 -- 172.21.15.117:0/1287523837 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc000fe80 con 0x7fbfd0103020 2024-04-20T11:20:36.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.511+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd0199280 con 0x7fbfd0103020 2024-04-20T11:20:36.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.512+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd010b3c0 con 0x7fbfd0103020 2024-04-20T11:20:36.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.513+0000 7fbfccc00700 1 -- 172.21.15.117:0/1287523837 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbfc000f6f0 con 0x7fbfd0103020 2024-04-20T11:20:36.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.514+0000 7fbfccc00700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbfb406ecc0 0x7fbfb4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.514+0000 7fbfccc00700 1 -- 172.21.15.117:0/1287523837 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbfc0090190 con 0x7fbfd0103020 2024-04-20T11:20:36.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.514+0000 7fbfcf400700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbfb406ecc0 0x7fbfb4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:36.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.515+0000 7fbfcf400700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbfb406ecc0 0x7fbfb4071180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005950 tx=0x7fbfb80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:36.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.519+0000 7fbfccc00700 1 -- 172.21.15.117:0/1287523837 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbfc005cb20 con 0x7fbfd0103020 2024-04-20T11:20:36.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.829+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbfd0104500 con 0x7fbfd0103020 2024-04-20T11:20:36.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.830+0000 7fbfccc00700 1 -- 172.21.15.117:0/1287523837 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fbfc005c6b0 con 0x7fbfd0103020 2024-04-20T11:20:36.830 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:20:36.831 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:20:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:20:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:20:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:20:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:20:36.832 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:20:36.832 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:20:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbfb406ecc0 msgr2=0x7fbfb4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbfb406ecc0 0x7fbfb4071180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005950 tx=0x7fbfb80058e0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 msgr2=0x7fbfd019e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 0x7fbfd019e9d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00092e0 tx=0x7fbfc000be20 comp rx=0 tx=0).stop 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 shutdown_connections 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbfb406ecc0 0x7fbfb4071180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103020 0x7fbfd019e9d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 --2- 172.21.15.117:0/1287523837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfd01039d0 0x7fbfd019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 >> 172.21.15.117:0/1287523837 conn(0x7fbfd00fe8b0 msgr2=0x7fbfd00ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 shutdown_connections 2024-04-20T11:20:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.833+0000 7fbfd6a00700 1 -- 172.21.15.117:0/1287523837 wait complete. 2024-04-20T11:20:36.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.983+0000 7fb307200700 1 -- 172.21.15.117:0/2460871344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 msgr2=0x7fb300109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.983+0000 7fb307200700 1 --2- 172.21.15.117:0/2460871344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 0x7fb300109290 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009910 tx=0x7fb2ec009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:36.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.983+0000 7fb307200700 1 -- 172.21.15.117:0/2460871344 shutdown_connections 2024-04-20T11:20:36.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.983+0000 7fb307200700 1 --2- 172.21.15.117:0/2460871344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 0x7fb300109290 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.983+0000 7fb307200700 1 --2- 172.21.15.117:0/2460871344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb300107520 0x7fb300107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.983+0000 7fb307200700 1 -- 172.21.15.117:0/2460871344 >> 172.21.15.117:0/2460871344 conn(0x7fb3000768f0 msgr2=0x7fb300078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:36.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.983+0000 7fb307200700 1 -- 172.21.15.117:0/2460871344 shutdown_connections 2024-04-20T11:20:36.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.983+0000 7fb307200700 1 -- 172.21.15.117:0/2460871344 wait complete. 2024-04-20T11:20:36.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.984+0000 7fb307200700 1 Processor -- start 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.984+0000 7fb307200700 1 -- start start 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.984+0000 7fb307200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb300107520 0x7fb30019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.984+0000 7fb307200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 0x7fb30019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.984+0000 7fb307200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30019ded0 con 0x7fb300108e10 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.984+0000 7fb307200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001a1ba0 con 0x7fb300107520 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 0x7fb30019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 0x7fb30019d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47970/0 (socket says 172.21.15.117:47970) 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ff400700 1 -- 172.21.15.117:0/3166265338 learned_addr learned my addr 172.21.15.117:0/3166265338 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ffe00700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb300107520 0x7fb30019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ff400700 1 -- 172.21.15.117:0/3166265338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb300107520 msgr2=0x7fb30019d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ff400700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb300107520 0x7fb30019d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ff400700 1 -- 172.21.15.117:0/3166265338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0095f0 con 0x7fb300108e10 2024-04-20T11:20:36.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ffe00700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb300107520 0x7fb30019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:20:36.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.985+0000 7fb2ff400700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 0x7fb30019d7f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0092e0 tx=0x7fb2ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:36.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.986+0000 7fb2fcc00700 1 -- 172.21.15.117:0/3166265338 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2ec01d070 con 0x7fb300108e10 2024-04-20T11:20:36.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.986+0000 7fb2fcc00700 1 -- 172.21.15.117:0/3166265338 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2ec00f460 con 0x7fb300108e10 2024-04-20T11:20:36.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.986+0000 7fb2fcc00700 1 -- 172.21.15.117:0/3166265338 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2ec017630 con 0x7fb300108e10 2024-04-20T11:20:36.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.986+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3001a1e20 con 0x7fb300108e10 2024-04-20T11:20:36.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.986+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3001a2390 con 0x7fb300108e10 2024-04-20T11:20:36.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.987+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30010f890 con 0x7fb300108e10 2024-04-20T11:20:36.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.988+0000 7fb2fcc00700 1 -- 172.21.15.117:0/3166265338 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb2ec00f5d0 con 0x7fb300108e10 2024-04-20T11:20:36.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.989+0000 7fb2fcc00700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb2e406ecd0 0x7fb2e4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:36.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.989+0000 7fb2fcc00700 1 -- 172.21.15.117:0/3166265338 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb2ec090320 con 0x7fb300108e10 2024-04-20T11:20:36.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.989+0000 7fb2ffe00700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb2e406ecd0 0x7fb2e4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:36.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.990+0000 7fb2ffe00700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb2e406ecd0 0x7fb2e4071190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0007900 tx=0x7fb2f0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:36.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:36.994+0000 7fb2fcc00700 1 -- 172.21.15.117:0/3166265338 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb2ec0590e0 con 0x7fb300108e10 2024-04-20T11:20:37.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.266+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb30019e5c0 con 0x7fb300108e10 2024-04-20T11:20:37.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.267+0000 7fb2fcc00700 1 -- 172.21.15.117:0/3166265338 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fb2ec05c7a0 con 0x7fb300108e10 2024-04-20T11:20:37.268 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:20:37.268 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:20:37.268 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:20:37.268 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:20:37.268 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:20:37.268 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:20:37.268 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:20:37.268 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:20:37.269 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:20:37.270 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:20:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:20:37.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb2e406ecd0 msgr2=0x7fb2e4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:37.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb2e406ecd0 0x7fb2e4071190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0007900 tx=0x7fb2f0008040 comp rx=0 tx=0).stop 2024-04-20T11:20:37.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 msgr2=0x7fb30019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:37.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 0x7fb30019d7f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0092e0 tx=0x7fb2ec00be20 comp rx=0 tx=0).stop 2024-04-20T11:20:37.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 shutdown_connections 2024-04-20T11:20:37.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb2e406ecd0 0x7fb2e4071190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb300107520 0x7fb30019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 --2- 172.21.15.117:0/3166265338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb300108e10 0x7fb30019d7f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 >> 172.21.15.117:0/3166265338 conn(0x7fb3000768f0 msgr2=0x7fb3000775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:37.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 shutdown_connections 2024-04-20T11:20:37.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.270+0000 7fb307200700 1 -- 172.21.15.117:0/3166265338 wait complete. 2024-04-20T11:20:37.273 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:20:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:37 smithi173 ceph-mon[74165]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:37 smithi173 ceph-mon[74165]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:37 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1287523837' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:37.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.421+0000 7f5e33000700 1 -- 172.21.15.117:0/2294149822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 msgr2=0x7f5e2c0736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:37.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.421+0000 7f5e33000700 1 --2- 172.21.15.117:0/2294149822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 0x7f5e2c0736f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009910 tx=0x7f5e1c009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:37.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.422+0000 7f5e33000700 1 -- 172.21.15.117:0/2294149822 shutdown_connections 2024-04-20T11:20:37.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.422+0000 7f5e33000700 1 --2- 172.21.15.117:0/2294149822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e2c073c30 0x7f5e2c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.422+0000 7f5e33000700 1 --2- 172.21.15.117:0/2294149822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 0x7f5e2c0736f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.422+0000 7f5e33000700 1 -- 172.21.15.117:0/2294149822 >> 172.21.15.117:0/2294149822 conn(0x7f5e2c0fc6c0 msgr2=0x7f5e2c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:37.423 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:37 smithi117 ceph-mon[99036]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:37.423 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:37 smithi117 ceph-mon[99036]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:37.423 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:37 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1287523837' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:20:37.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.422+0000 7f5e33000700 1 -- 172.21.15.117:0/2294149822 shutdown_connections 2024-04-20T11:20:37.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.422+0000 7f5e33000700 1 -- 172.21.15.117:0/2294149822 wait complete. 2024-04-20T11:20:37.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.424+0000 7f5e33000700 1 Processor -- start 2024-04-20T11:20:37.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.424+0000 7f5e33000700 1 -- start start 2024-04-20T11:20:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.425+0000 7f5e33000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 0x7f5e2c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.425+0000 7f5e33000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e2c073c30 0x7f5e2c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.425+0000 7f5e33000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c1998d0 con 0x7f5e2c073310 2024-04-20T11:20:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.425+0000 7f5e33000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c19c540 con 0x7f5e2c073c30 2024-04-20T11:20:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.426+0000 7f5e2be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 0x7f5e2c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.426+0000 7f5e2be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 0x7f5e2c198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47986/0 (socket says 172.21.15.117:47986) 2024-04-20T11:20:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.426+0000 7f5e2be00700 1 -- 172.21.15.117:0/175896732 learned_addr learned my addr 172.21.15.117:0/175896732 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:37.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.426+0000 7f5e2be00700 1 -- 172.21.15.117:0/175896732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e2c073c30 msgr2=0x7f5e2c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:37.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.426+0000 7f5e2be00700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e2c073c30 0x7f5e2c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.426+0000 7f5e2be00700 1 -- 172.21.15.117:0/175896732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c0095f0 con 0x7f5e2c073310 2024-04-20T11:20:37.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.426+0000 7f5e2be00700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 0x7f5e2c198d00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0092e0 tx=0x7f5e1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:37.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.427+0000 7f5e28c00700 1 -- 172.21.15.117:0/175896732 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1c01d070 con 0x7f5e2c073310 2024-04-20T11:20:37.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.427+0000 7f5e28c00700 1 -- 172.21.15.117:0/175896732 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e1c00f460 con 0x7f5e2c073310 2024-04-20T11:20:37.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.427+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c19c7c0 con 0x7f5e2c073310 2024-04-20T11:20:37.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.427+0000 7f5e28c00700 1 -- 172.21.15.117:0/175896732 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1c017630 con 0x7f5e2c073310 2024-04-20T11:20:37.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.427+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c19ccb0 con 0x7f5e2c073310 2024-04-20T11:20:37.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.428+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c0749c0 con 0x7f5e2c073310 2024-04-20T11:20:37.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.430+0000 7f5e28c00700 1 -- 172.21.15.117:0/175896732 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5e1c00f5d0 con 0x7f5e2c073310 2024-04-20T11:20:37.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.431+0000 7f5e28c00700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1006ecc0 0x7f5e10071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:37.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.432+0000 7f5e28c00700 1 -- 172.21.15.117:0/175896732 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5e1c090320 con 0x7f5e2c073310 2024-04-20T11:20:37.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.433+0000 7f5e2b400700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1006ecc0 0x7f5e10071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:37.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.436+0000 7f5e2b400700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1006ecc0 0x7f5e10071180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c19a220 tx=0x7f5e18005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:37.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.438+0000 7f5e28c00700 1 -- 172.21.15.117:0/175896732 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5e1c0590e0 con 0x7f5e2c073310 2024-04-20T11:20:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.669+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e2c107780 con 0x7f5e1006ecc0 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.672+0000 7f5e28c00700 1 -- 172.21.15.117:0/175896732 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5e2c107780 con 0x7f5e1006ecc0 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:20:37.673 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:20:37.674 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:20:37.674 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:20:37.674 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:20:37.674 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:20:37.674 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:20:37.674 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:20:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1006ecc0 msgr2=0x7f5e10071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1006ecc0 0x7f5e10071180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c19a220 tx=0x7f5e18005c80 comp rx=0 tx=0).stop 2024-04-20T11:20:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 msgr2=0x7f5e2c198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 0x7f5e2c198d00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0092e0 tx=0x7f5e1c00be20 comp rx=0 tx=0).stop 2024-04-20T11:20:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 shutdown_connections 2024-04-20T11:20:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1006ecc0 0x7f5e10071180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e2c073310 0x7f5e2c198d00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 --2- 172.21.15.117:0/175896732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e2c073c30 0x7f5e2c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.675+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 >> 172.21.15.117:0/175896732 conn(0x7f5e2c0fc6c0 msgr2=0x7f5e2c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.676+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 shutdown_connections 2024-04-20T11:20:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.676+0000 7f5e33000700 1 -- 172.21.15.117:0/175896732 wait complete. 2024-04-20T11:20:37.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.835+0000 7fc106800700 1 -- 172.21.15.117:0/2212318560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc100073310 msgr2=0x7fc1000736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:37.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.835+0000 7fc106800700 1 --2- 172.21.15.117:0/2212318560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc100073310 0x7fc1000736f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009910 tx=0x7fc0f0009c20 comp rx=0 tx=0).stop 2024-04-20T11:20:37.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.835+0000 7fc106800700 1 -- 172.21.15.117:0/2212318560 shutdown_connections 2024-04-20T11:20:37.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.835+0000 7fc106800700 1 --2- 172.21.15.117:0/2212318560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc100073cc0 0x7fc10010cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.835+0000 7fc106800700 1 --2- 172.21.15.117:0/2212318560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc100073310 0x7fc1000736f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.835+0000 7fc106800700 1 -- 172.21.15.117:0/2212318560 >> 172.21.15.117:0/2212318560 conn(0x7fc1000fc6f0 msgr2=0x7fc1000feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:37.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.835+0000 7fc106800700 1 -- 172.21.15.117:0/2212318560 shutdown_connections 2024-04-20T11:20:37.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.835+0000 7fc106800700 1 -- 172.21.15.117:0/2212318560 wait complete. 2024-04-20T11:20:37.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.836+0000 7fc106800700 1 Processor -- start 2024-04-20T11:20:37.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.836+0000 7fc106800700 1 -- start start 2024-04-20T11:20:37.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.837+0000 7fc106800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc100073310 0x7fc10019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:37.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.837+0000 7fc106800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc100073cc0 0x7fc10019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:37.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.837+0000 7fc106800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019f4f0 con 0x7fc100073310 2024-04-20T11:20:37.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.837+0000 7fc106800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc100198ad0 con 0x7fc100073cc0 2024-04-20T11:20:37.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.837+0000 7fc0ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc100073310 0x7fc10019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:37.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.837+0000 7fc0ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc100073310 0x7fc10019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48004/0 (socket says 172.21.15.117:48004) 2024-04-20T11:20:37.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.837+0000 7fc0ff400700 1 -- 172.21.15.117:0/3754611964 learned_addr learned my addr 172.21.15.117:0/3754611964 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:20:37.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc0ff400700 1 -- 172.21.15.117:0/3754611964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc100073cc0 msgr2=0x7fc10019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:37.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc0ff400700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc100073cc0 0x7fc10019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:37.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc0ff400700 1 -- 172.21.15.117:0/3754611964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f00095f0 con 0x7fc100073310 2024-04-20T11:20:37.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc0ff400700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc100073310 0x7fc10019e990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000b7d0 tx=0x7fc0f0005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:37.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc0fcc00700 1 -- 172.21.15.117:0/3754611964 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0f001d070 con 0x7fc100073310 2024-04-20T11:20:37.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc100198d50 con 0x7fc100073310 2024-04-20T11:20:37.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc0fcc00700 1 -- 172.21.15.117:0/3754611964 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0f00052a0 con 0x7fc100073310 2024-04-20T11:20:37.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc0fcc00700 1 -- 172.21.15.117:0/3754611964 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0f0017400 con 0x7fc100073310 2024-04-20T11:20:37.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.838+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1001992a0 con 0x7fc100073310 2024-04-20T11:20:37.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.839+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1001077d0 con 0x7fc100073310 2024-04-20T11:20:37.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.841+0000 7fc0fcc00700 1 -- 172.21.15.117:0/3754611964 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc0f000f820 con 0x7fc100073310 2024-04-20T11:20:37.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.842+0000 7fc0fcc00700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0e406ecc0 0x7fc0e4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:37.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.842+0000 7fc0fcc00700 1 -- 172.21.15.117:0/3754611964 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc0f00901a0 con 0x7fc100073310 2024-04-20T11:20:37.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.842+0000 7fc0f7e00700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0e406ecc0 0x7fc0e4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:37.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.843+0000 7fc0f7e00700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0e406ecc0 0x7fc0e4071180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8005950 tx=0x7fc0e80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:37.847+0000 7fc0fcc00700 1 -- 172.21.15.117:0/3754611964 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc0f005cab0 con 0x7fc100073310 2024-04-20T11:20:38.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.168+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc10010ff70 con 0x7fc100073310 2024-04-20T11:20:38.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.169+0000 7fc0fcc00700 1 -- 172.21.15.117:0/3754611964 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fc0f005c640 con 0x7fc100073310 2024-04-20T11:20:38.170 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:20:38.170 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:20:38.170 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:20:38.170 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:20:38.170 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:20:38.170 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:20:38.170 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:20:38.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0e406ecc0 msgr2=0x7fc0e4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:38.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0e406ecc0 0x7fc0e4071180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8005950 tx=0x7fc0e80058e0 comp rx=0 tx=0).stop 2024-04-20T11:20:38.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc100073310 msgr2=0x7fc10019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:38.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc100073310 0x7fc10019e990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000b7d0 tx=0x7fc0f0005ee0 comp rx=0 tx=0).stop 2024-04-20T11:20:38.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 shutdown_connections 2024-04-20T11:20:38.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0e406ecc0 0x7fc0e4071180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:38.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc100073310 0x7fc10019e990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:38.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 --2- 172.21.15.117:0/3754611964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc100073cc0 0x7fc10019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:38.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.172+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 >> 172.21.15.117:0/3754611964 conn(0x7fc1000fc6f0 msgr2=0x7fc100105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:38.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.173+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 shutdown_connections 2024-04-20T11:20:38.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:20:38.173+0000 7fc106800700 1 -- 172.21.15.117:0/3754611964 wait complete. 2024-04-20T11:20:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:38 smithi173 ceph-mon[74165]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:38 smithi173 ceph-mon[74165]: pgmap v245: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:38 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3166265338' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:20:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:38 smithi117 ceph-mon[99036]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:38 smithi117 ceph-mon[99036]: pgmap v245: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:38 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3166265338' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:20:39.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:39 smithi117 ceph-mon[99036]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:39.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:39 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3754611964' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:20:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:39 smithi173 ceph-mon[74165]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:39 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3754611964' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:20:40.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:40 smithi117 ceph-mon[99036]: pgmap v246: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:40 smithi173 ceph-mon[74165]: pgmap v246: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:42 smithi117 ceph-mon[99036]: pgmap v247: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:42 smithi173 ceph-mon[74165]: pgmap v247: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:20:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:20:44.258 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:44 smithi173 ceph-mon[74165]: pgmap v248: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:44 smithi117 ceph-mon[99036]: pgmap v248: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:45 smithi117 ceph-mon[99036]: pgmap v249: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:45 smithi173 ceph-mon[74165]: pgmap v249: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:48.343 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:48 smithi117 ceph-mon[99036]: pgmap v250: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:48 smithi173 ceph-mon[74165]: pgmap v250: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:50 smithi173 ceph-mon[74165]: pgmap v251: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:50.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:50 smithi117 ceph-mon[99036]: pgmap v251: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:52 smithi173 ceph-mon[74165]: pgmap v252: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:52.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:52 smithi117 ceph-mon[99036]: pgmap v252: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:54 smithi117 ceph-mon[99036]: pgmap v253: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:54 smithi173 ceph-mon[74165]: pgmap v253: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:56.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:56 smithi117 ceph-mon[99036]: pgmap v254: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:56 smithi173 ceph-mon[74165]: pgmap v254: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:58 smithi117 ceph-mon[99036]: pgmap v255: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:20:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:20:58.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:58 smithi173 ceph-mon[74165]: pgmap v255: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:20:58.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:20:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:21:00.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:00 smithi117 ceph-mon[99036]: pgmap v256: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:00 smithi173 ceph-mon[74165]: pgmap v256: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:02.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:02 smithi173 ceph-mon[74165]: pgmap v257: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:02.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:02 smithi117 ceph-mon[99036]: pgmap v257: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:04.610 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:04 smithi173 ceph-mon[74165]: pgmap v258: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:04 smithi117 ceph-mon[99036]: pgmap v258: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:06 smithi117 ceph-mon[99036]: pgmap v259: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:06 smithi173 ceph-mon[74165]: pgmap v259: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:08.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.345+0000 7f9c06e00700 1 -- 172.21.15.117:0/4023123915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 msgr2=0x7f9c00107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:08.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.345+0000 7f9c06e00700 1 --2- 172.21.15.117:0/4023123915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 0x7f9c00107900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009910 tx=0x7f9bf0009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:08.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.345+0000 7f9c06e00700 1 -- 172.21.15.117:0/4023123915 shutdown_connections 2024-04-20T11:21:08.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.345+0000 7f9c06e00700 1 --2- 172.21.15.117:0/4023123915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c00108e10 0x7f9c00109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.345+0000 7f9c06e00700 1 --2- 172.21.15.117:0/4023123915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 0x7f9c00107900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.345+0000 7f9c06e00700 1 -- 172.21.15.117:0/4023123915 >> 172.21.15.117:0/4023123915 conn(0x7f9c000768f0 msgr2=0x7f9c00078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:08.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.345+0000 7f9c06e00700 1 -- 172.21.15.117:0/4023123915 shutdown_connections 2024-04-20T11:21:08.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.346+0000 7f9c06e00700 1 -- 172.21.15.117:0/4023123915 wait complete. 2024-04-20T11:21:08.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.347+0000 7f9c06e00700 1 Processor -- start 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.347+0000 7f9c06e00700 1 -- start start 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.347+0000 7f9c06e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 0x7f9c0019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.347+0000 7f9c06e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c00108e10 0x7f9c0019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.347+0000 7f9c06e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0019ddd0 con 0x7f9c00107520 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.347+0000 7f9c06e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c001a1aa0 con 0x7f9c00108e10 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.347+0000 7f9bffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 0x7f9c0019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.348+0000 7f9bffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 0x7f9c0019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34920/0 (socket says 172.21.15.117:34920) 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.348+0000 7f9bffe00700 1 -- 172.21.15.117:0/2102978976 learned_addr learned my addr 172.21.15.117:0/2102978976 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.348+0000 7f9bf7e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c00108e10 0x7f9c0019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:08.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.348+0000 7f9bffe00700 1 -- 172.21.15.117:0/2102978976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c00108e10 msgr2=0x7f9c0019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:08.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.348+0000 7f9bffe00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c00108e10 0x7f9c0019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.348+0000 7f9bffe00700 1 -- 172.21.15.117:0/2102978976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00095f0 con 0x7f9c00107520 2024-04-20T11:21:08.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.348+0000 7f9bf7e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c00108e10 0x7f9c0019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:21:08.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.348+0000 7f9bffe00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 0x7f9c0019d1b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0000c00 tx=0x7f9bf000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:08.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.349+0000 7f9bfd600700 1 -- 172.21.15.117:0/2102978976 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf001d070 con 0x7f9c00107520 2024-04-20T11:21:08.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.349+0000 7f9bfd600700 1 -- 172.21.15.117:0/2102978976 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bf00056e0 con 0x7f9c00107520 2024-04-20T11:21:08.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.349+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c001a1d20 con 0x7f9c00107520 2024-04-20T11:21:08.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.349+0000 7f9bfd600700 1 -- 172.21.15.117:0/2102978976 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf000fe80 con 0x7f9c00107520 2024-04-20T11:21:08.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.349+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c001a2290 con 0x7f9c00107520 2024-04-20T11:21:08.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.350+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0010f890 con 0x7f9c00107520 2024-04-20T11:21:08.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.351+0000 7f9bfd600700 1 -- 172.21.15.117:0/2102978976 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9bf000f6f0 con 0x7f9c00107520 2024-04-20T11:21:08.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.352+0000 7f9bfd600700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9be406ecd0 0x7f9be4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:08.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.352+0000 7f9bfd600700 1 -- 172.21.15.117:0/2102978976 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9bf0090200 con 0x7f9c00107520 2024-04-20T11:21:08.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.352+0000 7f9bf7e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9be406ecd0 0x7f9be4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:08.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.353+0000 7f9bf7e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9be406ecd0 0x7f9be4071190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9be80094f0 tx=0x7f9be8009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:08.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.357+0000 7f9bfd600700 1 -- 172.21.15.117:0/2102978976 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9bf0058fc0 con 0x7f9c00107520 2024-04-20T11:21:08.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.602+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c001144c0 con 0x7f9be406ecd0 2024-04-20T11:21:08.603 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:08 smithi117 ceph-mon[99036]: pgmap v260: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:08.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.604+0000 7f9bfd600700 1 -- 172.21.15.117:0/2102978976 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9c001144c0 con 0x7f9be406ecd0 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9be406ecd0 msgr2=0x7f9be4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9be406ecd0 0x7f9be4071190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9be80094f0 tx=0x7f9be8009040 comp rx=0 tx=0).stop 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 msgr2=0x7f9c0019d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 0x7f9c0019d1b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0000c00 tx=0x7f9bf000be20 comp rx=0 tx=0).stop 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 shutdown_connections 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9be406ecd0 0x7f9be4071190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c00107520 0x7f9c0019d1b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 --2- 172.21.15.117:0/2102978976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c00108e10 0x7f9c0019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 >> 172.21.15.117:0/2102978976 conn(0x7f9c000768f0 msgr2=0x7f9c00077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:08.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 shutdown_connections 2024-04-20T11:21:08.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.607+0000 7f9c06e00700 1 -- 172.21.15.117:0/2102978976 wait complete. 2024-04-20T11:21:08.624 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:21:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:08 smithi173 ceph-mon[74165]: pgmap v260: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.769+0000 7f0a7bc00700 1 -- 172.21.15.117:0/219278661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 msgr2=0x7f0a74101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.769+0000 7f0a7bc00700 1 --2- 172.21.15.117:0/219278661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 0x7f0a74101200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009910 tx=0x7f0a68009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.770+0000 7f0a7bc00700 1 -- 172.21.15.117:0/219278661 shutdown_connections 2024-04-20T11:21:08.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.770+0000 7f0a7bc00700 1 --2- 172.21.15.117:0/219278661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a741017d0 0x7f0a74105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.770+0000 7f0a7bc00700 1 --2- 172.21.15.117:0/219278661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 0x7f0a74101200 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.770+0000 7f0a7bc00700 1 -- 172.21.15.117:0/219278661 >> 172.21.15.117:0/219278661 conn(0x7f0a740fc6f0 msgr2=0x7f0a740feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:08.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.770+0000 7f0a7bc00700 1 -- 172.21.15.117:0/219278661 shutdown_connections 2024-04-20T11:21:08.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.770+0000 7f0a7bc00700 1 -- 172.21.15.117:0/219278661 wait complete. 2024-04-20T11:21:08.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.771+0000 7f0a7bc00700 1 Processor -- start 2024-04-20T11:21:08.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.771+0000 7f0a7bc00700 1 -- start start 2024-04-20T11:21:08.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.771+0000 7f0a7bc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 0x7f0a74198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:08.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.771+0000 7f0a7bc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a741017d0 0x7f0a74199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:08.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.771+0000 7f0a7bc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74199a40 con 0x7f0a74100e20 2024-04-20T11:21:08.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.771+0000 7f0a7bc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7419d710 con 0x7f0a741017d0 2024-04-20T11:21:08.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.772+0000 7f0a79200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 0x7f0a74198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:08.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.772+0000 7f0a79200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 0x7f0a74198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34932/0 (socket says 172.21.15.117:34932) 2024-04-20T11:21:08.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.772+0000 7f0a79200700 1 -- 172.21.15.117:0/3162236848 learned_addr learned my addr 172.21.15.117:0/3162236848 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:08.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.772+0000 7f0a73e00700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a741017d0 0x7f0a74199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:08.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.772+0000 7f0a79200700 1 -- 172.21.15.117:0/3162236848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a741017d0 msgr2=0x7f0a74199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:08.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.772+0000 7f0a79200700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a741017d0 0x7f0a74199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:08.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.772+0000 7f0a79200700 1 -- 172.21.15.117:0/3162236848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680095f0 con 0x7f0a74100e20 2024-04-20T11:21:08.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.773+0000 7f0a79200700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 0x7f0a74198e20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800bd70 tx=0x7f0a6800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:08.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.773+0000 7f0a71600700 1 -- 172.21.15.117:0/3162236848 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a6801d070 con 0x7f0a74100e20 2024-04-20T11:21:08.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.773+0000 7f0a71600700 1 -- 172.21.15.117:0/3162236848 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a6800f460 con 0x7f0a74100e20 2024-04-20T11:21:08.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.773+0000 7f0a71600700 1 -- 172.21.15.117:0/3162236848 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a68017630 con 0x7f0a74100e20 2024-04-20T11:21:08.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.773+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7419d990 con 0x7f0a74100e20 2024-04-20T11:21:08.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.773+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7419de80 con 0x7f0a74100e20 2024-04-20T11:21:08.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.774+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a74102300 con 0x7f0a74100e20 2024-04-20T11:21:08.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.775+0000 7f0a71600700 1 -- 172.21.15.117:0/3162236848 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0a6800f5d0 con 0x7f0a74100e20 2024-04-20T11:21:08.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.777+0000 7f0a71600700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0a5c06ecc0 0x7f0a5c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:08.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.777+0000 7f0a71600700 1 -- 172.21.15.117:0/3162236848 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0a68090360 con 0x7f0a74100e20 2024-04-20T11:21:08.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.777+0000 7f0a73e00700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0a5c06ecc0 0x7f0a5c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:08.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.778+0000 7f0a73e00700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0a5c06ecc0 0x7f0a5c071180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a60006fd0 tx=0x7f0a60008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:08.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:08.782+0000 7f0a71600700 1 -- 172.21.15.117:0/3162236848 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0a68059120 con 0x7f0a74100e20 2024-04-20T11:21:09.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.030+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a7410de60 con 0x7f0a5c06ecc0 2024-04-20T11:21:09.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.033+0000 7f0a71600700 1 -- 172.21.15.117:0/3162236848 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0a7410de60 con 0x7f0a5c06ecc0 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0a5c06ecc0 msgr2=0x7f0a5c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0a5c06ecc0 0x7f0a5c071180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a60006fd0 tx=0x7f0a60008040 comp rx=0 tx=0).stop 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 msgr2=0x7f0a74198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 0x7f0a74198e20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800bd70 tx=0x7f0a6800be50 comp rx=0 tx=0).stop 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 shutdown_connections 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0a5c06ecc0 0x7f0a5c071180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a74100e20 0x7f0a74198e20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 --2- 172.21.15.117:0/3162236848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a741017d0 0x7f0a74199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 >> 172.21.15.117:0/3162236848 conn(0x7f0a740fc6f0 msgr2=0x7f0a740fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 shutdown_connections 2024-04-20T11:21:09.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.036+0000 7f0a7bc00700 1 -- 172.21.15.117:0/3162236848 wait complete. 2024-04-20T11:21:09.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.188+0000 7f5f3f000700 1 -- 172.21.15.117:0/3509611158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073ba0 msgr2=0x7f5f381114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.188+0000 7f5f3f000700 1 --2- 172.21.15.117:0/3509611158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073ba0 0x7f5f381114a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009910 tx=0x7f5f24009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:09.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.188+0000 7f5f3f000700 1 -- 172.21.15.117:0/3509611158 shutdown_connections 2024-04-20T11:21:09.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.188+0000 7f5f3f000700 1 --2- 172.21.15.117:0/3509611158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073ba0 0x7f5f381114a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.188+0000 7f5f3f000700 1 --2- 172.21.15.117:0/3509611158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f38073280 0x7f5f38073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.188+0000 7f5f3f000700 1 -- 172.21.15.117:0/3509611158 >> 172.21.15.117:0/3509611158 conn(0x7f5f380fc500 msgr2=0x7f5f380fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:09.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.189+0000 7f5f3f000700 1 -- 172.21.15.117:0/3509611158 shutdown_connections 2024-04-20T11:21:09.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.189+0000 7f5f3f000700 1 -- 172.21.15.117:0/3509611158 wait complete. 2024-04-20T11:21:09.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.189+0000 7f5f3f000700 1 Processor -- start 2024-04-20T11:21:09.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.190+0000 7f5f3f000700 1 -- start start 2024-04-20T11:21:09.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.190+0000 7f5f3f000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073280 0x7f5f3819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:09.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.190+0000 7f5f3f000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f38073ba0 0x7f5f3819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:09.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.190+0000 7f5f3f000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3819dae0 con 0x7f5f38073280 2024-04-20T11:21:09.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.190+0000 7f5f3f000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381a17b0 con 0x7f5f38073ba0 2024-04-20T11:21:09.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.190+0000 7f5f3dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073280 0x7f5f3819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:09.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.190+0000 7f5f3dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073280 0x7f5f3819cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34946/0 (socket says 172.21.15.117:34946) 2024-04-20T11:21:09.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.190+0000 7f5f3dc00700 1 -- 172.21.15.117:0/2530046118 learned_addr learned my addr 172.21.15.117:0/2530046118 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:09.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f3dc00700 1 -- 172.21.15.117:0/2530046118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f38073ba0 msgr2=0x7f5f3819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f3dc00700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f38073ba0 0x7f5f3819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f3dc00700 1 -- 172.21.15.117:0/2530046118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240095f0 con 0x7f5f38073280 2024-04-20T11:21:09.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f3dc00700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073280 0x7f5f3819cec0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00d8d0 tx=0x7f5f2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:09.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f36000700 1 -- 172.21.15.117:0/2530046118 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2c009940 con 0x7f5f38073280 2024-04-20T11:21:09.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f381a1a90 con 0x7f5f38073280 2024-04-20T11:21:09.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f36000700 1 -- 172.21.15.117:0/2530046118 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f2c010460 con 0x7f5f38073280 2024-04-20T11:21:09.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f36000700 1 -- 172.21.15.117:0/2530046118 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2c00f5f0 con 0x7f5f38073280 2024-04-20T11:21:09.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.191+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f381a1fe0 con 0x7f5f38073280 2024-04-20T11:21:09.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.192+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f380749c0 con 0x7f5f38073280 2024-04-20T11:21:09.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.193+0000 7f5f36000700 1 -- 172.21.15.117:0/2530046118 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5f2c0105d0 con 0x7f5f38073280 2024-04-20T11:21:09.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.194+0000 7f5f36000700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5f2006ecc0 0x7f5f20071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:09.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.194+0000 7f5f36000700 1 -- 172.21.15.117:0/2530046118 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5f2c08ede0 con 0x7f5f38073280 2024-04-20T11:21:09.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.195+0000 7f5f3d200700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5f2006ecc0 0x7f5f20071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:09.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.195+0000 7f5f3d200700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5f2006ecc0 0x7f5f20071180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f2400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:09.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.199+0000 7f5f36000700 1 -- 172.21.15.117:0/2530046118 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5f2c057ba0 con 0x7f5f38073280 2024-04-20T11:21:09.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:09 smithi117 ceph-mon[99036]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:09.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:09 smithi117 ceph-mon[99036]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:09.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:09 smithi117 ceph-mon[99036]: pgmap v261: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:09.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.436+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f381142a0 con 0x7f5f2006ecc0 2024-04-20T11:21:09.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.467+0000 7f5f36000700 1 -- 172.21.15.117:0/2530046118 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f5f381142a0 con 0x7f5f2006ecc0 2024-04-20T11:21:09.468 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:21:09.468 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (9m) 76s ago 23m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (24m) 76s ago 24m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (22m) 54s ago 22m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (5m) 76s ago 23m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (5m) 54s ago 22m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (8m) 76s ago 23m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (84s) 76s ago 16m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (111s) 76s ago 16m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (16m) 54s ago 16m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (59s) 54s ago 16m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (12m) 76s ago 25m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (11m) 54s ago 22m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (7m) 76s ago 25m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (7m) 54s ago 22m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:21:09.469 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (10m) 76s ago 23m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:21:09.470 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (10m) 54s ago 22m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:21:09.470 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (5m) 76s ago 21m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:21:09.470 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (4m) 76s ago 20m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:21:09.470 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (4m) 76s ago 20m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:21:09.470 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (3m) 54s ago 19m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:21:09.470 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (2m) 54s ago 18m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:21:09.470 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (2m) 54s ago 18m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:21:09.470 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (10m) 76s ago 23m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:21:09.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.470+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5f2006ecc0 msgr2=0x7f5f20071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5f2006ecc0 0x7f5f20071180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f2400b540 comp rx=0 tx=0).stop 2024-04-20T11:21:09.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073280 msgr2=0x7f5f3819cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073280 0x7f5f3819cec0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00d8d0 tx=0x7f5f2c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:21:09.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 shutdown_connections 2024-04-20T11:21:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5f2006ecc0 0x7f5f20071180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38073280 0x7f5f3819cec0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 --2- 172.21.15.117:0/2530046118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f38073ba0 0x7f5f3819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 >> 172.21.15.117:0/2530046118 conn(0x7f5f380fc500 msgr2=0x7f5f38100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 shutdown_connections 2024-04-20T11:21:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.471+0000 7f5f3f000700 1 -- 172.21.15.117:0/2530046118 wait complete. 2024-04-20T11:21:09.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.627+0000 7fbd19800700 1 -- 172.21.15.117:0/1183615624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd141037d0 msgr2=0x7fbd14107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.627+0000 7fbd19800700 1 --2- 172.21.15.117:0/1183615624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd141037d0 0x7fbd14107820 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009910 tx=0x7fbd00009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:09.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.628+0000 7fbd19800700 1 -- 172.21.15.117:0/1183615624 shutdown_connections 2024-04-20T11:21:09.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.628+0000 7fbd19800700 1 --2- 172.21.15.117:0/1183615624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd141037d0 0x7fbd14107820 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.628+0000 7fbd19800700 1 --2- 172.21.15.117:0/1183615624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd14102e20 0x7fbd14103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.628+0000 7fbd19800700 1 -- 172.21.15.117:0/1183615624 >> 172.21.15.117:0/1183615624 conn(0x7fbd140fe6d0 msgr2=0x7fbd14100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.628+0000 7fbd19800700 1 -- 172.21.15.117:0/1183615624 shutdown_connections 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.628+0000 7fbd19800700 1 -- 172.21.15.117:0/1183615624 wait complete. 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.629+0000 7fbd19800700 1 Processor -- start 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.629+0000 7fbd19800700 1 -- start start 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.629+0000 7fbd19800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd14102e20 0x7fbd1419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.629+0000 7fbd19800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd141037d0 0x7fbd1419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.629+0000 7fbd19800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1419f360 con 0x7fbd14102e20 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.629+0000 7fbd19800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd14198940 con 0x7fbd141037d0 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.630+0000 7fbd13e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd14102e20 0x7fbd1419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.630+0000 7fbd13e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd14102e20 0x7fbd1419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34958/0 (socket says 172.21.15.117:34958) 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.630+0000 7fbd13e00700 1 -- 172.21.15.117:0/2342531739 learned_addr learned my addr 172.21.15.117:0/2342531739 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.630+0000 7fbd13400700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd141037d0 0x7fbd1419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:09.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.630+0000 7fbd13e00700 1 -- 172.21.15.117:0/2342531739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd141037d0 msgr2=0x7fbd1419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.630+0000 7fbd13e00700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd141037d0 0x7fbd1419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.630+0000 7fbd13e00700 1 -- 172.21.15.117:0/2342531739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000095f0 con 0x7fbd14102e20 2024-04-20T11:21:09.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.630+0000 7fbd13e00700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd14102e20 0x7fbd1419e800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400d8d0 tx=0x7fbd0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:09.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.631+0000 7fbd10c00700 1 -- 172.21.15.117:0/2342531739 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd04009940 con 0x7fbd14102e20 2024-04-20T11:21:09.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.631+0000 7fbd10c00700 1 -- 172.21.15.117:0/2342531739 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd04010460 con 0x7fbd14102e20 2024-04-20T11:21:09.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.631+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd14198c20 con 0x7fbd14102e20 2024-04-20T11:21:09.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.631+0000 7fbd10c00700 1 -- 172.21.15.117:0/2342531739 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0400f5f0 con 0x7fbd14102e20 2024-04-20T11:21:09.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.631+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd14199170 con 0x7fbd14102e20 2024-04-20T11:21:09.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.632+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1410b220 con 0x7fbd14102e20 2024-04-20T11:21:09.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.633+0000 7fbd10c00700 1 -- 172.21.15.117:0/2342531739 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbd040105d0 con 0x7fbd14102e20 2024-04-20T11:21:09.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.634+0000 7fbd10c00700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbcf806ecc0 0x7fbcf8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:09.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.634+0000 7fbd10c00700 1 -- 172.21.15.117:0/2342531739 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbd0408ee50 con 0x7fbd14102e20 2024-04-20T11:21:09.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.634+0000 7fbd13400700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbcf806ecc0 0x7fbcf8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:09.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.635+0000 7fbd13400700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbcf806ecc0 0x7fbcf8071180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd00006010 tx=0x7fbd0000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:09.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.639+0000 7fbd10c00700 1 -- 172.21.15.117:0/2342531739 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbd0405b760 con 0x7fbd14102e20 2024-04-20T11:21:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:09 smithi173 ceph-mon[74165]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:09 smithi173 ceph-mon[74165]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:09 smithi173 ceph-mon[74165]: pgmap v261: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:09.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.934+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbd14199e00 con 0x7fbd14102e20 2024-04-20T11:21:09.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.935+0000 7fbd10c00700 1 -- 172.21.15.117:0/2342531739 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fbd0405b2f0 con 0x7fbd14102e20 2024-04-20T11:21:09.936 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:21:09.936 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:21:09.936 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:21:09.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:21:09.936 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:21:09.936 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:21:09.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:21:09.938 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbcf806ecc0 msgr2=0x7fbcf8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbcf806ecc0 0x7fbcf8071180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd00006010 tx=0x7fbd0000b540 comp rx=0 tx=0).stop 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd14102e20 msgr2=0x7fbd1419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd14102e20 0x7fbd1419e800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400d8d0 tx=0x7fbd0400dc90 comp rx=0 tx=0).stop 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 shutdown_connections 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbcf806ecc0 0x7fbcf8071180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd14102e20 0x7fbd1419e800 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 --2- 172.21.15.117:0/2342531739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd141037d0 0x7fbd1419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 >> 172.21.15.117:0/2342531739 conn(0x7fbd140fe6d0 msgr2=0x7fbd140ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:09.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 shutdown_connections 2024-04-20T11:21:09.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:09.938+0000 7fbd19800700 1 -- 172.21.15.117:0/2342531739 wait complete. 2024-04-20T11:21:10.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.082+0000 7fb73a200700 1 -- 172.21.15.117:0/904902658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 msgr2=0x7fb734107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.082+0000 7fb73a200700 1 --2- 172.21.15.117:0/904902658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 0x7fb734107a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00b0a0 tx=0x7fb71c00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.082+0000 7fb73a200700 1 -- 172.21.15.117:0/904902658 shutdown_connections 2024-04-20T11:21:10.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.082+0000 7fb73a200700 1 --2- 172.21.15.117:0/904902658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 0x7fb734107a20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.082+0000 7fb73a200700 1 --2- 172.21.15.117:0/904902658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb734103020 0x7fb734103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.082+0000 7fb73a200700 1 -- 172.21.15.117:0/904902658 >> 172.21.15.117:0/904902658 conn(0x7fb7340fe8b0 msgr2=0x7fb734100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:10.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.082+0000 7fb73a200700 1 -- 172.21.15.117:0/904902658 shutdown_connections 2024-04-20T11:21:10.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.082+0000 7fb73a200700 1 -- 172.21.15.117:0/904902658 wait complete. 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.083+0000 7fb73a200700 1 Processor -- start 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.083+0000 7fb73a200700 1 -- start start 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb73a200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb734103020 0x7fb734198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb73a200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 0x7fb734199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb73a200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734199920 con 0x7fb7341039d0 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb73a200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73419c540 con 0x7fb734103020 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb72aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 0x7fb734199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb72aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 0x7fb734199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34982/0 (socket says 172.21.15.117:34982) 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb72aa00700 1 -- 172.21.15.117:0/2547808959 learned_addr learned my addr 172.21.15.117:0/2547808959 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb733400700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb734103020 0x7fb734198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb72aa00700 1 -- 172.21.15.117:0/2547808959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb734103020 msgr2=0x7fb734198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb72aa00700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb734103020 0x7fb734198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.084+0000 7fb72aa00700 1 -- 172.21.15.117:0/2547808959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c009d00 con 0x7fb7341039d0 2024-04-20T11:21:10.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.085+0000 7fb72aa00700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 0x7fb734199290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00b070 tx=0x7fb71c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:10.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.085+0000 7fb730c00700 1 -- 172.21.15.117:0/2547808959 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb71c00e040 con 0x7fb7341039d0 2024-04-20T11:21:10.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.085+0000 7fb730c00700 1 -- 172.21.15.117:0/2547808959 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb71c003de0 con 0x7fb7341039d0 2024-04-20T11:21:10.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.085+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73419c7c0 con 0x7fb7341039d0 2024-04-20T11:21:10.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.085+0000 7fb730c00700 1 -- 172.21.15.117:0/2547808959 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb71c01d6b0 con 0x7fb7341039d0 2024-04-20T11:21:10.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.085+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73419ccb0 con 0x7fb7341039d0 2024-04-20T11:21:10.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.086+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73410b420 con 0x7fb7341039d0 2024-04-20T11:21:10.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.087+0000 7fb730c00700 1 -- 172.21.15.117:0/2547808959 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb71c004040 con 0x7fb7341039d0 2024-04-20T11:21:10.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.088+0000 7fb730c00700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb71806ecc0 0x7fb718071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.088+0000 7fb730c00700 1 -- 172.21.15.117:0/2547808959 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb71c090680 con 0x7fb7341039d0 2024-04-20T11:21:10.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.088+0000 7fb733400700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb71806ecc0 0x7fb718071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.089+0000 7fb733400700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb71806ecc0 0x7fb718071180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb724005fd0 tx=0x7fb72400a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:10.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.093+0000 7fb730c00700 1 -- 172.21.15.117:0/2547808959 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb71c059440 con 0x7fb7341039d0 2024-04-20T11:21:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.356+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb73410ffc0 con 0x7fb7341039d0 2024-04-20T11:21:10.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.356+0000 7fb730c00700 1 -- 172.21.15.117:0/2547808959 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fb71c017770 con 0x7fb7341039d0 2024-04-20T11:21:10.357 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:21:10.357 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:21:10.358 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:21:10.359 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:21:10.360 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:21:10.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb71806ecc0 msgr2=0x7fb718071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb71806ecc0 0x7fb718071180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb724005fd0 tx=0x7fb72400a560 comp rx=0 tx=0).stop 2024-04-20T11:21:10.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 msgr2=0x7fb734199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 0x7fb734199290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00b070 tx=0x7fb71c007bd0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 shutdown_connections 2024-04-20T11:21:10.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb71806ecc0 0x7fb718071180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb734103020 0x7fb734198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 --2- 172.21.15.117:0/2547808959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7341039d0 0x7fb734199290 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.359+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 >> 172.21.15.117:0/2547808959 conn(0x7fb7340fe8b0 msgr2=0x7fb7340fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:10.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.360+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 shutdown_connections 2024-04-20T11:21:10.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.360+0000 7fb73a200700 1 -- 172.21.15.117:0/2547808959 wait complete. 2024-04-20T11:21:10.362 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:21:10.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.518+0000 7fab2ba00700 1 -- 172.21.15.117:0/2414836917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 msgr2=0x7fab24107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.518+0000 7fab2ba00700 1 --2- 172.21.15.117:0/2414836917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 0x7fab24107a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fab10009910 tx=0x7fab10009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:10.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.518+0000 7fab2ba00700 1 -- 172.21.15.117:0/2414836917 shutdown_connections 2024-04-20T11:21:10.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.518+0000 7fab2ba00700 1 --2- 172.21.15.117:0/2414836917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 0x7fab24107a00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.518+0000 7fab2ba00700 1 --2- 172.21.15.117:0/2414836917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24103000 0x7fab241033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.518+0000 7fab2ba00700 1 -- 172.21.15.117:0/2414836917 >> 172.21.15.117:0/2414836917 conn(0x7fab240fe8b0 msgr2=0x7fab24100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:10.520 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:10 smithi117 ceph-mon[99036]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:10.520 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:10 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2342531739' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.519+0000 7fab2ba00700 1 -- 172.21.15.117:0/2414836917 shutdown_connections 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.519+0000 7fab2ba00700 1 -- 172.21.15.117:0/2414836917 wait complete. 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.520+0000 7fab2ba00700 1 Processor -- start 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.520+0000 7fab2ba00700 1 -- start start 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.520+0000 7fab2ba00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24103000 0x7fab24198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.520+0000 7fab2ba00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 0x7fab24199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.520+0000 7fab2ba00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab24199a50 con 0x7fab241039b0 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.520+0000 7fab2ba00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2419d720 con 0x7fab24103000 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.521+0000 7fab23e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 0x7fab24199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.521+0000 7fab23e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 0x7fab24199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34996/0 (socket says 172.21.15.117:34996) 2024-04-20T11:21:10.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.521+0000 7fab23e00700 1 -- 172.21.15.117:0/3116921656 learned_addr learned my addr 172.21.15.117:0/3116921656 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.521+0000 7fab29000700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24103000 0x7fab24198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.521+0000 7fab23e00700 1 -- 172.21.15.117:0/3116921656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24103000 msgr2=0x7fab24198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.521+0000 7fab23e00700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24103000 0x7fab24198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.521+0000 7fab23e00700 1 -- 172.21.15.117:0/3116921656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab100095f0 con 0x7fab241039b0 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.522+0000 7fab23e00700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 0x7fab24199370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fab100092e0 tx=0x7fab1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.522+0000 7fab21600700 1 -- 172.21.15.117:0/3116921656 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1001d070 con 0x7fab241039b0 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.522+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2419d9a0 con 0x7fab241039b0 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.522+0000 7fab21600700 1 -- 172.21.15.117:0/3116921656 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab1000f460 con 0x7fab241039b0 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.522+0000 7fab21600700 1 -- 172.21.15.117:0/3116921656 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab10017610 con 0x7fab241039b0 2024-04-20T11:21:10.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.522+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2419de90 con 0x7fab241039b0 2024-04-20T11:21:10.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.523+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2410b350 con 0x7fab241039b0 2024-04-20T11:21:10.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.525+0000 7fab21600700 1 -- 172.21.15.117:0/3116921656 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fab1000f5d0 con 0x7fab241039b0 2024-04-20T11:21:10.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.526+0000 7fab21600700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab0c06ecc0 0x7fab0c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.527+0000 7fab21600700 1 -- 172.21.15.117:0/3116921656 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fab10090580 con 0x7fab241039b0 2024-04-20T11:21:10.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.527+0000 7fab29000700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab0c06ecc0 0x7fab0c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.527+0000 7fab29000700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab0c06ecc0 0x7fab0c071180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab18006fd0 tx=0x7fab18008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:10.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.530+0000 7fab21600700 1 -- 172.21.15.117:0/3116921656 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fab100593e0 con 0x7fab241039b0 2024-04-20T11:21:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:10 smithi173 ceph-mon[74165]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:10 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2342531739' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:21:10.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.775+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab24110020 con 0x7fab0c06ecc0 2024-04-20T11:21:10.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.777+0000 7fab21600700 1 -- 172.21.15.117:0/3116921656 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fab24110020 con 0x7fab0c06ecc0 2024-04-20T11:21:10.778 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:21:10.778 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:21:10.778 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:21:10.779 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:21:10.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab0c06ecc0 msgr2=0x7fab0c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab0c06ecc0 0x7fab0c071180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab18006fd0 tx=0x7fab18008040 comp rx=0 tx=0).stop 2024-04-20T11:21:10.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 msgr2=0x7fab24199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 0x7fab24199370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fab100092e0 tx=0x7fab1000be20 comp rx=0 tx=0).stop 2024-04-20T11:21:10.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 shutdown_connections 2024-04-20T11:21:10.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab0c06ecc0 0x7fab0c071180 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24103000 0x7fab24198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 --2- 172.21.15.117:0/3116921656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab241039b0 0x7fab24199370 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.780+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 >> 172.21.15.117:0/3116921656 conn(0x7fab240fe8b0 msgr2=0x7fab240ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:10.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.781+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 shutdown_connections 2024-04-20T11:21:10.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.781+0000 7fab2ba00700 1 -- 172.21.15.117:0/3116921656 wait complete. 2024-04-20T11:21:10.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.930+0000 7faa4dc00700 1 -- 172.21.15.117:0/3751260566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48073ba0 msgr2=0x7faa48111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.930+0000 7faa4dc00700 1 --2- 172.21.15.117:0/3751260566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48073ba0 0x7faa48111680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faa30009910 tx=0x7faa30009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:10.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.930+0000 7faa4dc00700 1 -- 172.21.15.117:0/3751260566 shutdown_connections 2024-04-20T11:21:10.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.930+0000 7faa4dc00700 1 --2- 172.21.15.117:0/3751260566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa48073ba0 0x7faa48111680 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.930+0000 7faa4dc00700 1 --2- 172.21.15.117:0/3751260566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48073280 0x7faa48073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.930+0000 7faa4dc00700 1 -- 172.21.15.117:0/3751260566 >> 172.21.15.117:0/3751260566 conn(0x7faa480fc6c0 msgr2=0x7faa480feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:10.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.930+0000 7faa4dc00700 1 -- 172.21.15.117:0/3751260566 shutdown_connections 2024-04-20T11:21:10.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.930+0000 7faa4dc00700 1 -- 172.21.15.117:0/3751260566 wait complete. 2024-04-20T11:21:10.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.931+0000 7faa4dc00700 1 Processor -- start 2024-04-20T11:21:10.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.931+0000 7faa4dc00700 1 -- start start 2024-04-20T11:21:10.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.932+0000 7faa4dc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48073280 0x7faa4819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.932+0000 7faa4dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4819d8e0 0x7faa481a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.932+0000 7faa4dc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4819df00 con 0x7faa4819d8e0 2024-04-20T11:21:10.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.932+0000 7faa4dc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4819e070 con 0x7faa48073280 2024-04-20T11:21:10.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.932+0000 7faa3fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4819d8e0 0x7faa481a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.932+0000 7faa3fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4819d8e0 0x7faa481a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35012/0 (socket says 172.21.15.117:35012) 2024-04-20T11:21:10.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.932+0000 7faa3fe00700 1 -- 172.21.15.117:0/592432489 learned_addr learned my addr 172.21.15.117:0/592432489 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:10.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.932+0000 7faa46a00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48073280 0x7faa4819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa3fe00700 1 -- 172.21.15.117:0/592432489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48073280 msgr2=0x7faa4819d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:10.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa3fe00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48073280 0x7faa4819d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:10.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa3fe00700 1 -- 172.21.15.117:0/592432489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300095f0 con 0x7faa4819d8e0 2024-04-20T11:21:10.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa46a00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48073280 0x7faa4819d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:21:10.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa3fe00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4819d8e0 0x7faa481a1c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa3000b5c0 tx=0x7faa3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:10.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa3f400700 1 -- 172.21.15.117:0/592432489 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa3001d070 con 0x7faa4819d8e0 2024-04-20T11:21:10.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa3f400700 1 -- 172.21.15.117:0/592432489 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa3000f460 con 0x7faa4819d8e0 2024-04-20T11:21:10.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa3f400700 1 -- 172.21.15.117:0/592432489 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa30005720 con 0x7faa4819d8e0 2024-04-20T11:21:10.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa481a2290 con 0x7faa4819d8e0 2024-04-20T11:21:10.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.933+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa481a27e0 con 0x7faa4819d8e0 2024-04-20T11:21:10.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.934+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa481a2ab0 con 0x7faa4819d8e0 2024-04-20T11:21:10.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.935+0000 7faa3f400700 1 -- 172.21.15.117:0/592432489 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7faa3000f5d0 con 0x7faa4819d8e0 2024-04-20T11:21:10.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.936+0000 7faa3f400700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faa2c06ecc0 0x7faa2c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:10.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.936+0000 7faa3f400700 1 -- 172.21.15.117:0/592432489 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7faa30090160 con 0x7faa4819d8e0 2024-04-20T11:21:10.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.937+0000 7faa46a00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faa2c06ecc0 0x7faa2c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:10.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.937+0000 7faa46a00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faa2c06ecc0 0x7faa2c071180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa38007900 tx=0x7faa38008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:10.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:10.941+0000 7faa3f400700 1 -- 172.21.15.117:0/592432489 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7faa30058f20 con 0x7faa4819d8e0 2024-04-20T11:21:11.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.261+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faa4819e6b0 con 0x7faa4819d8e0 2024-04-20T11:21:11.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.269+0000 7faa3f400700 1 -- 172.21.15.117:0/592432489 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7faa3005c5e0 con 0x7faa4819d8e0 2024-04-20T11:21:11.270 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:21:11.270 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:21:11.270 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:21:11.270 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:21:11.270 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:21:11.270 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:21:11.270 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:21:11.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.272+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faa2c06ecc0 msgr2=0x7faa2c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.272+0000 7faa4dc00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faa2c06ecc0 0x7faa2c071180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa38007900 tx=0x7faa38008040 comp rx=0 tx=0).stop 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4819d8e0 msgr2=0x7faa481a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4819d8e0 0x7faa481a1c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa3000b5c0 tx=0x7faa3000be20 comp rx=0 tx=0).stop 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 shutdown_connections 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faa2c06ecc0 0x7faa2c071180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48073280 0x7faa4819d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 --2- 172.21.15.117:0/592432489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4819d8e0 0x7faa481a1c90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 >> 172.21.15.117:0/592432489 conn(0x7faa480fc6c0 msgr2=0x7faa48100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 shutdown_connections 2024-04-20T11:21:11.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:11.273+0000 7faa4dc00700 1 -- 172.21.15.117:0/592432489 wait complete. 2024-04-20T11:21:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:11 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2547808959' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:21:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:11 smithi117 ceph-mon[99036]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:11 smithi117 ceph-mon[99036]: pgmap v262: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:11 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2547808959' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:21:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:11 smithi173 ceph-mon[74165]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:11 smithi173 ceph-mon[74165]: pgmap v262: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:12.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:12 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/592432489' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:21:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:12 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/592432489' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:21:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:21:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:13 smithi117 ceph-mon[99036]: pgmap v263: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:21:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:13 smithi173 ceph-mon[74165]: pgmap v263: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:16 smithi173 ceph-mon[74165]: pgmap v264: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:16 smithi117 ceph-mon[99036]: pgmap v264: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:18 smithi173 ceph-mon[74165]: pgmap v265: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:18.443 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:18 smithi117 ceph-mon[99036]: pgmap v265: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:20 smithi173 ceph-mon[74165]: pgmap v266: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:20 smithi117 ceph-mon[99036]: pgmap v266: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:22.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:22 smithi117 ceph-mon[99036]: pgmap v267: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:22 smithi173 ceph-mon[74165]: pgmap v267: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:24.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:24 smithi117 ceph-mon[99036]: pgmap v268: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:24 smithi173 ceph-mon[74165]: pgmap v268: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:26.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:26 smithi117 ceph-mon[99036]: pgmap v269: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:26 smithi173 ceph-mon[74165]: pgmap v269: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:28 smithi117 ceph-mon[99036]: pgmap v270: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:21:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:28 smithi173 ceph-mon[74165]: pgmap v270: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:21:29.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:29 smithi117 ceph-mon[99036]: pgmap v271: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:29 smithi173 ceph-mon[74165]: pgmap v271: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:32 smithi173 ceph-mon[74165]: pgmap v272: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:32 smithi117 ceph-mon[99036]: pgmap v272: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:34.454 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:34 smithi117 ceph-mon[99036]: pgmap v273: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:34 smithi173 ceph-mon[74165]: pgmap v273: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:36 smithi117 ceph-mon[99036]: pgmap v274: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:36 smithi173 ceph-mon[74165]: pgmap v274: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:38.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:38 smithi117 ceph-mon[99036]: pgmap v275: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:38.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:38 smithi173 ceph-mon[74165]: pgmap v275: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:40.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:40 smithi117 ceph-mon[99036]: pgmap v276: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:40 smithi173 ceph-mon[74165]: pgmap v276: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:41.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.443+0000 7f017fc00700 1 -- 172.21.15.117:0/4024677873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 msgr2=0x7f01781034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:41.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.443+0000 7f017fc00700 1 --2- 172.21.15.117:0/4024677873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 0x7f01781034f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f016c009910 tx=0x7f016c009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:41.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.444+0000 7f017fc00700 1 -- 172.21.15.117:0/4024677873 shutdown_connections 2024-04-20T11:21:41.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.444+0000 7f017fc00700 1 --2- 172.21.15.117:0/4024677873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0178104a00 0x7f0178104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.444+0000 7f017fc00700 1 --2- 172.21.15.117:0/4024677873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 0x7f01781034f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.444+0000 7f017fc00700 1 -- 172.21.15.117:0/4024677873 >> 172.21.15.117:0/4024677873 conn(0x7f01780fe880 msgr2=0x7f0178100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:41.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.444+0000 7f017fc00700 1 -- 172.21.15.117:0/4024677873 shutdown_connections 2024-04-20T11:21:41.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.444+0000 7f017fc00700 1 -- 172.21.15.117:0/4024677873 wait complete. 2024-04-20T11:21:41.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.445+0000 7f017fc00700 1 Processor -- start 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.445+0000 7f017fc00700 1 -- start start 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.445+0000 7f017fc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 0x7f0178198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017fc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0178104a00 0x7f01781993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017fc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0178199a80 con 0x7f0178103110 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017fc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017819d750 con 0x7f0178104a00 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 0x7f0178198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 0x7f0178198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33644/0 (socket says 172.21.15.117:33644) 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017d200700 1 -- 172.21.15.117:0/25030284 learned_addr learned my addr 172.21.15.117:0/25030284 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f0177e00700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0178104a00 0x7f01781993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017d200700 1 -- 172.21.15.117:0/25030284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0178104a00 msgr2=0x7f01781993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017d200700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0178104a00 0x7f01781993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.446+0000 7f017d200700 1 -- 172.21.15.117:0/25030284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0095f0 con 0x7f0178103110 2024-04-20T11:21:41.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.447+0000 7f017d200700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 0x7f0178198e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f016c0092e0 tx=0x7f016c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:41.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.447+0000 7f0175600700 1 -- 172.21.15.117:0/25030284 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f016c01d070 con 0x7f0178103110 2024-04-20T11:21:41.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.447+0000 7f0175600700 1 -- 172.21.15.117:0/25030284 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f016c00f460 con 0x7f0178103110 2024-04-20T11:21:41.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.447+0000 7f0175600700 1 -- 172.21.15.117:0/25030284 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f016c017630 con 0x7f0178103110 2024-04-20T11:21:41.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.447+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017819d9d0 con 0x7f0178103110 2024-04-20T11:21:41.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.447+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017819df40 con 0x7f0178103110 2024-04-20T11:21:41.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.448+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017810b480 con 0x7f0178103110 2024-04-20T11:21:41.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.449+0000 7f0175600700 1 -- 172.21.15.117:0/25030284 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f016c00f5d0 con 0x7f0178103110 2024-04-20T11:21:41.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.450+0000 7f0175600700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f016006ecd0 0x7f0160071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:41.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.450+0000 7f0175600700 1 -- 172.21.15.117:0/25030284 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f016c090340 con 0x7f0178103110 2024-04-20T11:21:41.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.451+0000 7f0177e00700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f016006ecd0 0x7f0160071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:41.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.451+0000 7f0177e00700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f016006ecd0 0x7f0160071190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0164006fd0 tx=0x7f0164008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:41.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.455+0000 7f0175600700 1 -- 172.21.15.117:0/25030284 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f016c059100 con 0x7f0178103110 2024-04-20T11:21:41.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.694+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f017819a170 con 0x7f016006ecd0 2024-04-20T11:21:41.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.696+0000 7f0175600700 1 -- 172.21.15.117:0/25030284 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f017819a170 con 0x7f016006ecd0 2024-04-20T11:21:41.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.698+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f016006ecd0 msgr2=0x7f0160071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:41.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f016006ecd0 0x7f0160071190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0164006fd0 tx=0x7f0164008040 comp rx=0 tx=0).stop 2024-04-20T11:21:41.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 msgr2=0x7f0178198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:41.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 0x7f0178198e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f016c0092e0 tx=0x7f016c00be20 comp rx=0 tx=0).stop 2024-04-20T11:21:41.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 shutdown_connections 2024-04-20T11:21:41.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f016006ecd0 0x7f0160071190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0178103110 0x7f0178198e60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 --2- 172.21.15.117:0/25030284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0178104a00 0x7f01781993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 >> 172.21.15.117:0/25030284 conn(0x7f01780fe880 msgr2=0x7f01780ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:41.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 shutdown_connections 2024-04-20T11:21:41.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.699+0000 7f017fc00700 1 -- 172.21.15.117:0/25030284 wait complete. 2024-04-20T11:21:41.713 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:21:41.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.853+0000 7fcbcf400700 1 -- 172.21.15.117:0/983918186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 msgr2=0x7fcbc80736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:41.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.853+0000 7fcbcf400700 1 --2- 172.21.15.117:0/983918186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 0x7fcbc80736f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009910 tx=0x7fcbbc009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:41.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.853+0000 7fcbcf400700 1 -- 172.21.15.117:0/983918186 shutdown_connections 2024-04-20T11:21:41.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.853+0000 7fcbcf400700 1 --2- 172.21.15.117:0/983918186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbc8073c30 0x7fcbc810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.853+0000 7fcbcf400700 1 --2- 172.21.15.117:0/983918186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 0x7fcbc80736f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.853+0000 7fcbcf400700 1 -- 172.21.15.117:0/983918186 >> 172.21.15.117:0/983918186 conn(0x7fcbc80fc6c0 msgr2=0x7fcbc80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:41.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.853+0000 7fcbcf400700 1 -- 172.21.15.117:0/983918186 shutdown_connections 2024-04-20T11:21:41.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.853+0000 7fcbcf400700 1 -- 172.21.15.117:0/983918186 wait complete. 2024-04-20T11:21:41.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.854+0000 7fcbcf400700 1 Processor -- start 2024-04-20T11:21:41.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.854+0000 7fcbcf400700 1 -- start start 2024-04-20T11:21:41.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 0x7fcbc8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:41.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcf400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbc8073c30 0x7fcbc8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:41.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcf400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc81998d0 con 0x7fcbc8073310 2024-04-20T11:21:41.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcf400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc819c540 con 0x7fcbc8073c30 2024-04-20T11:21:41.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 0x7fcbc8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:41.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 0x7fcbc8198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33660/0 (socket says 172.21.15.117:33660) 2024-04-20T11:21:41.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcca00700 1 -- 172.21.15.117:0/885731140 learned_addr learned my addr 172.21.15.117:0/885731140 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:41.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcca00700 1 -- 172.21.15.117:0/885731140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbc8073c30 msgr2=0x7fcbc8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:41.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbc7e00700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbc8073c30 0x7fcbc8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:41.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcca00700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbc8073c30 0x7fcbc8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:41.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.855+0000 7fcbcca00700 1 -- 172.21.15.117:0/885731140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc0095f0 con 0x7fcbc8073310 2024-04-20T11:21:41.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.856+0000 7fcbcca00700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 0x7fcbc8198d00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0092e0 tx=0x7fcbbc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:41.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.856+0000 7fcbc5600700 1 -- 172.21.15.117:0/885731140 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbbc01d070 con 0x7fcbc8073310 2024-04-20T11:21:41.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.856+0000 7fcbc5600700 1 -- 172.21.15.117:0/885731140 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbbc00f460 con 0x7fcbc8073310 2024-04-20T11:21:41.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.856+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc819c7c0 con 0x7fcbc8073310 2024-04-20T11:21:41.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.856+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc819ccb0 con 0x7fcbc8073310 2024-04-20T11:21:41.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.856+0000 7fcbc5600700 1 -- 172.21.15.117:0/885731140 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbbc017630 con 0x7fcbc8073310 2024-04-20T11:21:41.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.857+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc80749c0 con 0x7fcbc8073310 2024-04-20T11:21:41.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.858+0000 7fcbc5600700 1 -- 172.21.15.117:0/885731140 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcbbc00f5d0 con 0x7fcbc8073310 2024-04-20T11:21:41.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.859+0000 7fcbc5600700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcbb006ecc0 0x7fcbb0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:41.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.859+0000 7fcbc5600700 1 -- 172.21.15.117:0/885731140 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcbbc0903f0 con 0x7fcbc8073310 2024-04-20T11:21:41.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.860+0000 7fcbc7e00700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcbb006ecc0 0x7fcbb0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:41.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.860+0000 7fcbc7e00700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcbb006ecc0 0x7fcbb0071180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbc819a220 tx=0x7fcbb4005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:41.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:41.864+0000 7fcbc5600700 1 -- 172.21.15.117:0/885731140 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcbbc059130 con 0x7fcbc8073310 2024-04-20T11:21:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.103+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbc8107750 con 0x7fcbb006ecc0 2024-04-20T11:21:42.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.106+0000 7fcbc5600700 1 -- 172.21.15.117:0/885731140 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcbc8107750 con 0x7fcbb006ecc0 2024-04-20T11:21:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.108+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcbb006ecc0 msgr2=0x7fcbb0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.108+0000 7fcbcf400700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcbb006ecc0 0x7fcbb0071180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbc819a220 tx=0x7fcbb4005cf0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.108+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 msgr2=0x7fcbc8198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.108+0000 7fcbcf400700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 0x7fcbc8198d00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0092e0 tx=0x7fcbbc00be20 comp rx=0 tx=0).stop 2024-04-20T11:21:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.109+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 shutdown_connections 2024-04-20T11:21:42.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.109+0000 7fcbcf400700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcbb006ecc0 0x7fcbb0071180 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.109+0000 7fcbcf400700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc8073310 0x7fcbc8198d00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.109+0000 7fcbcf400700 1 --2- 172.21.15.117:0/885731140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbc8073c30 0x7fcbc8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.109+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 >> 172.21.15.117:0/885731140 conn(0x7fcbc80fc6c0 msgr2=0x7fcbc8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:42.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.109+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 shutdown_connections 2024-04-20T11:21:42.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.109+0000 7fcbcf400700 1 -- 172.21.15.117:0/885731140 wait complete. 2024-04-20T11:21:42.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.270+0000 7f0bee400700 1 -- 172.21.15.117:0/2903224636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 msgr2=0x7f0be8105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:42.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.270+0000 7f0bee400700 1 --2- 172.21.15.117:0/2903224636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 0x7f0be8105e90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009910 tx=0x7f0bd8009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:42.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.271+0000 7f0bee400700 1 -- 172.21.15.117:0/2903224636 shutdown_connections 2024-04-20T11:21:42.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.271+0000 7f0bee400700 1 --2- 172.21.15.117:0/2903224636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be80ffab0 0x7f0be80fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.271+0000 7f0bee400700 1 --2- 172.21.15.117:0/2903224636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 0x7f0be8105e90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.271+0000 7f0bee400700 1 -- 172.21.15.117:0/2903224636 >> 172.21.15.117:0/2903224636 conn(0x7f0be80fb5c0 msgr2=0x7f0be80fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:42.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.271+0000 7f0bee400700 1 -- 172.21.15.117:0/2903224636 shutdown_connections 2024-04-20T11:21:42.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.271+0000 7f0bee400700 1 -- 172.21.15.117:0/2903224636 wait complete. 2024-04-20T11:21:42.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.272+0000 7f0bee400700 1 Processor -- start 2024-04-20T11:21:42.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.272+0000 7f0bee400700 1 -- start start 2024-04-20T11:21:42.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.272+0000 7f0bee400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be80ffab0 0x7f0be819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:42.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.272+0000 7f0bee400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 0x7f0be819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:42.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.272+0000 7f0bee400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be819d2f0 con 0x7f0be8105ab0 2024-04-20T11:21:42.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.272+0000 7f0bee400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be81968d0 con 0x7f0be80ffab0 2024-04-20T11:21:42.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.273+0000 7f0be6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 0x7f0be819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.273+0000 7f0be6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 0x7f0be819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33684/0 (socket says 172.21.15.117:33684) 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.273+0000 7f0be6a00700 1 -- 172.21.15.117:0/477862574 learned_addr learned my addr 172.21.15.117:0/477862574 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.273+0000 7f0be6a00700 1 -- 172.21.15.117:0/477862574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be80ffab0 msgr2=0x7f0be819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.273+0000 7f0be7400700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be80ffab0 0x7f0be819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.273+0000 7f0be6a00700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be80ffab0 0x7f0be819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.273+0000 7f0be6a00700 1 -- 172.21.15.117:0/477862574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd80095f0 con 0x7f0be8105ab0 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.273+0000 7f0be6a00700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 0x7f0be819ccd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400d8d0 tx=0x7f0bd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.274+0000 7f0bdfe00700 1 -- 172.21.15.117:0/477862574 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bd4009940 con 0x7f0be8105ab0 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.274+0000 7f0bdfe00700 1 -- 172.21.15.117:0/477862574 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bd4010460 con 0x7f0be8105ab0 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.274+0000 7f0bdfe00700 1 -- 172.21.15.117:0/477862574 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bd400f5f0 con 0x7f0be8105ab0 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.274+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be8196bb0 con 0x7f0be8105ab0 2024-04-20T11:21:42.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.274+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be8197100 con 0x7f0be8105ab0 2024-04-20T11:21:42.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.275+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be81090c0 con 0x7f0be8105ab0 2024-04-20T11:21:42.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.276+0000 7f0bdfe00700 1 -- 172.21.15.117:0/477862574 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0bd40105d0 con 0x7f0be8105ab0 2024-04-20T11:21:42.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.277+0000 7f0bdfe00700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0bcc06ecc0 0x7f0bcc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:42.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.277+0000 7f0bdfe00700 1 -- 172.21.15.117:0/477862574 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0bd408edf0 con 0x7f0be8105ab0 2024-04-20T11:21:42.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.278+0000 7f0be7400700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0bcc06ecc0 0x7f0bcc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:42.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.278+0000 7f0be7400700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0bcc06ecc0 0x7f0bcc071180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8006010 tx=0x7f0bd800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:42.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.282+0000 7f0bdfe00700 1 -- 172.21.15.117:0/477862574 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0bd405b6e0 con 0x7f0be8105ab0 2024-04-20T11:21:42.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:42 smithi117 ceph-mon[99036]: pgmap v277: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:42.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.517+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0be8197e00 con 0x7f0bcc06ecc0 2024-04-20T11:21:42.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.547+0000 7f0bdfe00700 1 -- 172.21.15.117:0/477862574 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f0be8197e00 con 0x7f0bcc06ecc0 2024-04-20T11:21:42.547 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (10m) 109s ago 24m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (24m) 109s ago 24m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (23m) 87s ago 23m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (6m) 109s ago 24m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (6m) 87s ago 23m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (9m) 109s ago 23m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (117s) 109s ago 16m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (2m) 109s ago 17m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (16m) 87s ago 16m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (92s) 87s ago 17m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (13m) 109s ago 26m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (11m) 87s ago 22m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (8m) 109s ago 26m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:21:42.548 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (7m) 87s ago 22m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (11m) 109s ago 24m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (11m) 87s ago 23m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (5m) 109s ago 21m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (5m) 109s ago 21m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (4m) 109s ago 20m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (4m) 87s ago 20m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (3m) 87s ago 19m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (2m) 87s ago 18m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:21:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (10m) 109s ago 23m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:21:42.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0bcc06ecc0 msgr2=0x7f0bcc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:42.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0bcc06ecc0 0x7f0bcc071180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8006010 tx=0x7f0bd800b540 comp rx=0 tx=0).stop 2024-04-20T11:21:42.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 msgr2=0x7f0be819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:42.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 0x7f0be819ccd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400d8d0 tx=0x7f0bd400dc90 comp rx=0 tx=0).stop 2024-04-20T11:21:42.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 shutdown_connections 2024-04-20T11:21:42.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0bcc06ecc0 0x7f0bcc071180 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be80ffab0 0x7f0be819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 --2- 172.21.15.117:0/477862574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be8105ab0 0x7f0be819ccd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.550+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 >> 172.21.15.117:0/477862574 conn(0x7f0be80fb5c0 msgr2=0x7f0be8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:42.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.551+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 shutdown_connections 2024-04-20T11:21:42.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.551+0000 7f0bee400700 1 -- 172.21.15.117:0/477862574 wait complete. 2024-04-20T11:21:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:42 smithi173 ceph-mon[74165]: pgmap v277: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:42.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.706+0000 7f82a9600700 1 -- 172.21.15.117:0/3647534252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 msgr2=0x7f82a410d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:42.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.706+0000 7f82a9600700 1 --2- 172.21.15.117:0/3647534252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 0x7f82a410d020 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f828c009910 tx=0x7f828c009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:42.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.706+0000 7f82a9600700 1 -- 172.21.15.117:0/3647534252 shutdown_connections 2024-04-20T11:21:42.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.706+0000 7f82a9600700 1 --2- 172.21.15.117:0/3647534252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 0x7f82a410d020 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.706+0000 7f82a9600700 1 --2- 172.21.15.117:0/3647534252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a4068570 0x7f82a4068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.706+0000 7f82a9600700 1 -- 172.21.15.117:0/3647534252 >> 172.21.15.117:0/3647534252 conn(0x7f82a4075480 msgr2=0x7f82a4075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:42.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.706+0000 7f82a9600700 1 -- 172.21.15.117:0/3647534252 shutdown_connections 2024-04-20T11:21:42.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.706+0000 7f82a9600700 1 -- 172.21.15.117:0/3647534252 wait complete. 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.707+0000 7f82a9600700 1 Processor -- start 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.708+0000 7f82a9600700 1 -- start start 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.708+0000 7f82a9600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a4068570 0x7f82a419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.708+0000 7f82a9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 0x7f82a419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.708+0000 7f82a9600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a419f3a0 con 0x7f82a4068e90 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.708+0000 7f82a9600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a4198980 con 0x7f82a4068570 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.708+0000 7f829b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 0x7f82a419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.708+0000 7f829b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 0x7f82a419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33704/0 (socket says 172.21.15.117:33704) 2024-04-20T11:21:42.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.708+0000 7f829b400700 1 -- 172.21.15.117:0/1745462931 learned_addr learned my addr 172.21.15.117:0/1745462931 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:42.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f829b400700 1 -- 172.21.15.117:0/1745462931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a4068570 msgr2=0x7f82a419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:42.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f82a3e00700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a4068570 0x7f82a419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:42.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f829b400700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a4068570 0x7f82a419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:42.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f829b400700 1 -- 172.21.15.117:0/1745462931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0095f0 con 0x7f82a4068e90 2024-04-20T11:21:42.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f829b400700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 0x7f82a419ed80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f828c0092e0 tx=0x7f828c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:42.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f82a1600700 1 -- 172.21.15.117:0/1745462931 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828c01d070 con 0x7f82a4068e90 2024-04-20T11:21:42.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f82a1600700 1 -- 172.21.15.117:0/1745462931 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f828c00f460 con 0x7f82a4068e90 2024-04-20T11:21:42.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a4198c00 con 0x7f82a4068e90 2024-04-20T11:21:42.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.709+0000 7f82a1600700 1 -- 172.21.15.117:0/1745462931 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828c017630 con 0x7f82a4068e90 2024-04-20T11:21:42.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.710+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a41990f0 con 0x7f82a4068e90 2024-04-20T11:21:42.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.711+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a41079a0 con 0x7f82a4068e90 2024-04-20T11:21:42.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.712+0000 7f82a1600700 1 -- 172.21.15.117:0/1745462931 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f828c00f5d0 con 0x7f82a4068e90 2024-04-20T11:21:42.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.713+0000 7f82a1600700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecc0 0x7f8288071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:42.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.713+0000 7f82a1600700 1 -- 172.21.15.117:0/1745462931 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f828c0903f0 con 0x7f82a4068e90 2024-04-20T11:21:42.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.713+0000 7f82a3e00700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecc0 0x7f8288071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:42.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.714+0000 7f82a3e00700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecc0 0x7f8288071180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82940060b0 tx=0x7f8294008640 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:42.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:42.717+0000 7f82a1600700 1 -- 172.21.15.117:0/1745462931 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f828c05cc60 con 0x7f82a4068e90 2024-04-20T11:21:43.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.014+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f82a4199e40 con 0x7f82a4068e90 2024-04-20T11:21:43.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.015+0000 7f82a1600700 1 -- 172.21.15.117:0/1745462931 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f828c05c7f0 con 0x7f82a4068e90 2024-04-20T11:21:43.014 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:21:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:21:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:21:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:21:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:21:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:21:43.016 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:21:43.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.017+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecc0 msgr2=0x7f8288071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.017+0000 7f82a9600700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecc0 0x7f8288071180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82940060b0 tx=0x7f8294008640 comp rx=0 tx=0).stop 2024-04-20T11:21:43.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 msgr2=0x7f82a419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 0x7f82a419ed80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f828c0092e0 tx=0x7f828c00be20 comp rx=0 tx=0).stop 2024-04-20T11:21:43.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 shutdown_connections 2024-04-20T11:21:43.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f828806ecc0 0x7f8288071180 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a4068570 0x7f82a419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 --2- 172.21.15.117:0/1745462931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82a4068e90 0x7f82a419ed80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 >> 172.21.15.117:0/1745462931 conn(0x7f82a4075480 msgr2=0x7f82a40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:43.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 shutdown_connections 2024-04-20T11:21:43.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.018+0000 7f82a9600700 1 -- 172.21.15.117:0/1745462931 wait complete. 2024-04-20T11:21:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.165+0000 7fa9e3400700 1 -- 172.21.15.117:0/1005169831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 msgr2=0x7fa9dc107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.165+0000 7fa9e3400700 1 --2- 172.21.15.117:0/1005169831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 0x7fa9dc107900 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009910 tx=0x7fa9d0009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.165+0000 7fa9e3400700 1 -- 172.21.15.117:0/1005169831 shutdown_connections 2024-04-20T11:21:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.165+0000 7fa9e3400700 1 --2- 172.21.15.117:0/1005169831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9dc108e10 0x7fa9dc109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.165+0000 7fa9e3400700 1 --2- 172.21.15.117:0/1005169831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 0x7fa9dc107900 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.165+0000 7fa9e3400700 1 -- 172.21.15.117:0/1005169831 >> 172.21.15.117:0/1005169831 conn(0x7fa9dc0768f0 msgr2=0x7fa9dc078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.166+0000 7fa9e3400700 1 -- 172.21.15.117:0/1005169831 shutdown_connections 2024-04-20T11:21:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.166+0000 7fa9e3400700 1 -- 172.21.15.117:0/1005169831 wait complete. 2024-04-20T11:21:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e3400700 1 Processor -- start 2024-04-20T11:21:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e3400700 1 -- start start 2024-04-20T11:21:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 0x7fa9dc19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e3400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9dc108e10 0x7fa9dc19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e3400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19ddd0 con 0x7fa9dc107520 2024-04-20T11:21:43.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e3400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1a1aa0 con 0x7fa9dc108e10 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 0x7fa9dc19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e0a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 0x7fa9dc19d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33722/0 (socket says 172.21.15.117:33722) 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.167+0000 7fa9e0a00700 1 -- 172.21.15.117:0/1989412840 learned_addr learned my addr 172.21.15.117:0/1989412840 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.168+0000 7fa9e0a00700 1 -- 172.21.15.117:0/1989412840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9dc108e10 msgr2=0x7fa9dc19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.168+0000 7fa9e0a00700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9dc108e10 0x7fa9dc19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.168+0000 7fa9e0a00700 1 -- 172.21.15.117:0/1989412840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d00095f0 con 0x7fa9dc107520 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.168+0000 7fa9e0a00700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 0x7fa9dc19d1b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0000c00 tx=0x7fa9d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.168+0000 7fa9d9600700 1 -- 172.21.15.117:0/1989412840 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9d001d070 con 0x7fa9dc107520 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.168+0000 7fa9d9600700 1 -- 172.21.15.117:0/1989412840 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9d00056e0 con 0x7fa9dc107520 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.168+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc1a1d20 con 0x7fa9dc107520 2024-04-20T11:21:43.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.168+0000 7fa9d9600700 1 -- 172.21.15.117:0/1989412840 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9d000fe80 con 0x7fa9dc107520 2024-04-20T11:21:43.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.169+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc1a2290 con 0x7fa9dc107520 2024-04-20T11:21:43.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.170+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc10f890 con 0x7fa9dc107520 2024-04-20T11:21:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.171+0000 7fa9d9600700 1 -- 172.21.15.117:0/1989412840 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa9d000f6f0 con 0x7fa9dc107520 2024-04-20T11:21:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.171+0000 7fa9d9600700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa9c406ecd0 0x7fa9c4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.172+0000 7fa9d9600700 1 -- 172.21.15.117:0/1989412840 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa9d0090240 con 0x7fa9dc107520 2024-04-20T11:21:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.172+0000 7fa9dbe00700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa9c406ecd0 0x7fa9c4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.172+0000 7fa9dbe00700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa9c406ecd0 0x7fa9c4071190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005950 tx=0x7fa9c80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:43.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.176+0000 7fa9d9600700 1 -- 172.21.15.117:0/1989412840 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa9d0059020 con 0x7fa9dc107520 2024-04-20T11:21:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:43 smithi117 ceph-mon[99036]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:43 smithi117 ceph-mon[99036]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:43 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1745462931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:21:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:21:43.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.433+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa9dc19e4c0 con 0x7fa9dc107520 2024-04-20T11:21:43.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.434+0000 7fa9d9600700 1 -- 172.21.15.117:0/1989412840 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fa9d005c6e0 con 0x7fa9dc107520 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:21:43.435 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:21:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:21:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:21:43.438 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:21:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:21:43.438 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:21:43.438 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:21:43.438 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:21:43.438 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:21:43.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa9c406ecd0 msgr2=0x7fa9c4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa9c406ecd0 0x7fa9c4071190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005950 tx=0x7fa9c80058e0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 msgr2=0x7fa9dc19d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 0x7fa9dc19d1b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0000c00 tx=0x7fa9d000be20 comp rx=0 tx=0).stop 2024-04-20T11:21:43.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 shutdown_connections 2024-04-20T11:21:43.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa9c406ecd0 0x7fa9c4071190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9dc107520 0x7fa9dc19d1b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 --2- 172.21.15.117:0/1989412840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9dc108e10 0x7fa9dc19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 >> 172.21.15.117:0/1989412840 conn(0x7fa9dc0768f0 msgr2=0x7fa9dc077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:43.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 shutdown_connections 2024-04-20T11:21:43.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.438+0000 7fa9e3400700 1 -- 172.21.15.117:0/1989412840 wait complete. 2024-04-20T11:21:43.440 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:21:43.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.594+0000 7f3d58200700 1 -- 172.21.15.117:0/355747820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50103110 msgr2=0x7f3d501034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.594+0000 7f3d58200700 1 --2- 172.21.15.117:0/355747820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50103110 0x7f3d501034f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009910 tx=0x7f3d44009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:43.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.595+0000 7f3d58200700 1 -- 172.21.15.117:0/355747820 shutdown_connections 2024-04-20T11:21:43.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.595+0000 7f3d58200700 1 --2- 172.21.15.117:0/355747820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d50104a00 0x7f3d50104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.595+0000 7f3d58200700 1 --2- 172.21.15.117:0/355747820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50103110 0x7f3d501034f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.595+0000 7f3d58200700 1 -- 172.21.15.117:0/355747820 >> 172.21.15.117:0/355747820 conn(0x7f3d500fe880 msgr2=0x7f3d50100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:43.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.595+0000 7f3d58200700 1 -- 172.21.15.117:0/355747820 shutdown_connections 2024-04-20T11:21:43.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.595+0000 7f3d58200700 1 -- 172.21.15.117:0/355747820 wait complete. 2024-04-20T11:21:43.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.596+0000 7f3d58200700 1 Processor -- start 2024-04-20T11:21:43.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.596+0000 7f3d58200700 1 -- start start 2024-04-20T11:21:43.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.596+0000 7f3d58200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d50103110 0x7f3d50198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.596+0000 7f3d58200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50104a00 0x7f3d50199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.596+0000 7f3d58200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d501999e0 con 0x7f3d50104a00 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.596+0000 7f3d58200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5019d660 con 0x7f3d50103110 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.597+0000 7f3d54e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50104a00 0x7f3d50199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.597+0000 7f3d54e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50104a00 0x7f3d50199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33730/0 (socket says 172.21.15.117:33730) 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.597+0000 7f3d54e00700 1 -- 172.21.15.117:0/2997190021 learned_addr learned my addr 172.21.15.117:0/2997190021 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.597+0000 7f3d55800700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d50103110 0x7f3d50198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.597+0000 7f3d54e00700 1 -- 172.21.15.117:0/2997190021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d50103110 msgr2=0x7f3d50198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.597+0000 7f3d54e00700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d50103110 0x7f3d50198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.597+0000 7f3d54e00700 1 -- 172.21.15.117:0/2997190021 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440095f0 con 0x7f3d50104a00 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.597+0000 7f3d54e00700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50104a00 0x7f3d50199300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c8a0 tx=0x7f3d3c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:43.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.598+0000 7f3d4e000700 1 -- 172.21.15.117:0/2997190021 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d3c00cea0 con 0x7f3d50104a00 2024-04-20T11:21:43.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.598+0000 7f3d4e000700 1 -- 172.21.15.117:0/2997190021 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d3c004830 con 0x7f3d50104a00 2024-04-20T11:21:43.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.598+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5019d940 con 0x7f3d50104a00 2024-04-20T11:21:43.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.598+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5019df10 con 0x7f3d50104a00 2024-04-20T11:21:43.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.598+0000 7f3d4e000700 1 -- 172.21.15.117:0/2997190021 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d3c0055b0 con 0x7f3d50104a00 2024-04-20T11:21:43.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.599+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5010b460 con 0x7f3d50104a00 2024-04-20T11:21:43.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.600+0000 7f3d4e000700 1 -- 172.21.15.117:0/2997190021 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3d3c0049a0 con 0x7f3d50104a00 2024-04-20T11:21:43.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.601+0000 7f3d4e000700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3806ecd0 0x7f3d38071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.601+0000 7f3d4e000700 1 -- 172.21.15.117:0/2997190021 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3d3c08ef70 con 0x7f3d50104a00 2024-04-20T11:21:43.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.602+0000 7f3d55800700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3806ecd0 0x7f3d38071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.602+0000 7f3d55800700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3806ecd0 0x7f3d38071190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400b5c0 tx=0x7f3d4401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:43.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.606+0000 7f3d4e000700 1 -- 172.21.15.117:0/2997190021 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3d3c057db0 con 0x7f3d50104a00 2024-04-20T11:21:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:43 smithi173 ceph-mon[74165]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:43 smithi173 ceph-mon[74165]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:43 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1745462931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:21:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:21:43.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.836+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d5019a080 con 0x7f3d3806ecd0 2024-04-20T11:21:43.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.838+0000 7f3d4e000700 1 -- 172.21.15.117:0/2997190021 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3d5019a080 con 0x7f3d3806ecd0 2024-04-20T11:21:43.837 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:21:43.838 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.840+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3806ecd0 msgr2=0x7f3d38071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.840+0000 7f3d58200700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3806ecd0 0x7f3d38071190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400b5c0 tx=0x7f3d4401a040 comp rx=0 tx=0).stop 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.840+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50104a00 msgr2=0x7f3d50199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.840+0000 7f3d58200700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50104a00 0x7f3d50199300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c8a0 tx=0x7f3d3c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.840+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 shutdown_connections 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.840+0000 7f3d58200700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3806ecd0 0x7f3d38071190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.840+0000 7f3d58200700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d50103110 0x7f3d50198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.841+0000 7f3d58200700 1 --2- 172.21.15.117:0/2997190021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d50104a00 0x7f3d50199300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.841+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 >> 172.21.15.117:0/2997190021 conn(0x7f3d500fe880 msgr2=0x7f3d500ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:43.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.841+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 shutdown_connections 2024-04-20T11:21:43.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:43.841+0000 7f3d58200700 1 -- 172.21.15.117:0/2997190021 wait complete. 2024-04-20T11:21:44.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.000+0000 7f3d54400700 1 -- 172.21.15.117:0/4140236782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 msgr2=0x7f3d4c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.000+0000 7f3d54400700 1 --2- 172.21.15.117:0/4140236782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 0x7f3d4c109290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009910 tx=0x7f3d38009c20 comp rx=0 tx=0).stop 2024-04-20T11:21:44.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.000+0000 7f3d54400700 1 -- 172.21.15.117:0/4140236782 shutdown_connections 2024-04-20T11:21:44.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.000+0000 7f3d54400700 1 --2- 172.21.15.117:0/4140236782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 0x7f3d4c109290 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.000+0000 7f3d54400700 1 --2- 172.21.15.117:0/4140236782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d4c107520 0x7f3d4c107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.000+0000 7f3d54400700 1 -- 172.21.15.117:0/4140236782 >> 172.21.15.117:0/4140236782 conn(0x7f3d4c0768f0 msgr2=0x7f3d4c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:44.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.000+0000 7f3d54400700 1 -- 172.21.15.117:0/4140236782 shutdown_connections 2024-04-20T11:21:44.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.000+0000 7f3d54400700 1 -- 172.21.15.117:0/4140236782 wait complete. 2024-04-20T11:21:44.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.001+0000 7f3d54400700 1 Processor -- start 2024-04-20T11:21:44.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.001+0000 7f3d54400700 1 -- start start 2024-04-20T11:21:44.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d54400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d4c107520 0x7f3d4c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:44.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d54400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 0x7f3d4c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:44.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d54400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c19ddf0 con 0x7f3d4c108e10 2024-04-20T11:21:44.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d54400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c1a1a70 con 0x7f3d4c107520 2024-04-20T11:21:44.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d51000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 0x7f3d4c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:44.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d51000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 0x7f3d4c19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51916/0 (socket says 172.21.15.117:51916) 2024-04-20T11:21:44.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d51000700 1 -- 172.21.15.117:0/2826628802 learned_addr learned my addr 172.21.15.117:0/2826628802 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d51a00700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d4c107520 0x7f3d4c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d51000700 1 -- 172.21.15.117:0/2826628802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d4c107520 msgr2=0x7f3d4c19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d51000700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d4c107520 0x7f3d4c19d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.002+0000 7f3d51000700 1 -- 172.21.15.117:0/2826628802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d380095f0 con 0x7f3d4c108e10 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.003+0000 7f3d51000700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 0x7f3d4c19d710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800bd70 tx=0x7f3d3800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.003+0000 7f3d4a000700 1 -- 172.21.15.117:0/2826628802 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d3801d070 con 0x7f3d4c108e10 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.003+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c1a1cf0 con 0x7f3d4c108e10 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.003+0000 7f3d4a000700 1 -- 172.21.15.117:0/2826628802 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d3800f460 con 0x7f3d4c108e10 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.003+0000 7f3d4a000700 1 -- 172.21.15.117:0/2826628802 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d38017630 con 0x7f3d4c108e10 2024-04-20T11:21:44.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.003+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c1a2260 con 0x7f3d4c108e10 2024-04-20T11:21:44.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.004+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c10f870 con 0x7f3d4c108e10 2024-04-20T11:21:44.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.005+0000 7f3d4a000700 1 -- 172.21.15.117:0/2826628802 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3d3800f5d0 con 0x7f3d4c108e10 2024-04-20T11:21:44.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.006+0000 7f3d4a000700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3406ecc0 0x7f3d34071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:44.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.006+0000 7f3d4a000700 1 -- 172.21.15.117:0/2826628802 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3d38090330 con 0x7f3d4c108e10 2024-04-20T11:21:44.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.007+0000 7f3d51a00700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3406ecc0 0x7f3d34071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:44.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.007+0000 7f3d51a00700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3406ecc0 0x7f3d34071180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005950 tx=0x7f3d40005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:44.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.011+0000 7f3d4a000700 1 -- 172.21.15.117:0/2826628802 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3d380590f0 con 0x7f3d4c108e10 2024-04-20T11:21:44.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.323+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3d4c19e490 con 0x7f3d4c108e10 2024-04-20T11:21:44.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.324+0000 7f3d4a000700 1 -- 172.21.15.117:0/2826628802 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f3d3805c7b0 con 0x7f3d4c108e10 2024-04-20T11:21:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:21:44.323 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:21:44.324 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:21:44.324 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:21:44.324 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:21:44.324 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:21:44.324 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:21:44.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.326+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3406ecc0 msgr2=0x7f3d34071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.326+0000 7f3d54400700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3406ecc0 0x7f3d34071180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005950 tx=0x7f3d40005cf0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.326+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 msgr2=0x7f3d4c19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.326+0000 7f3d54400700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 0x7f3d4c19d710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800bd70 tx=0x7f3d3800be50 comp rx=0 tx=0).stop 2024-04-20T11:21:44.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.327+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 shutdown_connections 2024-04-20T11:21:44.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.327+0000 7f3d54400700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3406ecc0 0x7f3d34071180 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.327+0000 7f3d54400700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d4c107520 0x7f3d4c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.327+0000 7f3d54400700 1 --2- 172.21.15.117:0/2826628802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d4c108e10 0x7f3d4c19d710 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.327+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 >> 172.21.15.117:0/2826628802 conn(0x7f3d4c0768f0 msgr2=0x7f3d4c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:44.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.327+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 shutdown_connections 2024-04-20T11:21:44.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:21:44.327+0000 7f3d54400700 1 -- 172.21.15.117:0/2826628802 wait complete. 2024-04-20T11:21:44.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:44 smithi117 ceph-mon[99036]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:44.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:44 smithi117 ceph-mon[99036]: pgmap v278: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:44.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:44 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1989412840' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:21:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:44 smithi173 ceph-mon[74165]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:44 smithi173 ceph-mon[74165]: pgmap v278: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:44 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1989412840' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:21:45.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:45 smithi117 ceph-mon[99036]: from='client.34580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:45.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:45 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2826628802' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:21:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:45 smithi173 ceph-mon[74165]: from='client.34580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:45 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2826628802' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:21:46.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:46 smithi117 ceph-mon[99036]: pgmap v279: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:46 smithi173 ceph-mon[74165]: pgmap v279: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:47.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:47 smithi117 ceph-mon[99036]: pgmap v280: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:47 smithi173 ceph-mon[74165]: pgmap v280: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:50 smithi117 ceph-mon[99036]: pgmap v281: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:50 smithi173 ceph-mon[74165]: pgmap v281: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:52.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:52 smithi117 ceph-mon[99036]: pgmap v282: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:52 smithi173 ceph-mon[74165]: pgmap v282: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:54 smithi117 ceph-mon[99036]: pgmap v283: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:54 smithi173 ceph-mon[74165]: pgmap v283: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:56.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:56 smithi117 ceph-mon[99036]: pgmap v284: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:56 smithi173 ceph-mon[74165]: pgmap v284: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:58 smithi117 ceph-mon[99036]: pgmap v285: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:21:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:21:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:58 smithi173 ceph-mon[74165]: pgmap v285: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:21:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:21:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:22:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:00 smithi117 ceph-mon[99036]: pgmap v286: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:00 smithi173 ceph-mon[74165]: pgmap v286: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:02.562 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:02 smithi173 ceph-mon[74165]: pgmap v287: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:02.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:02 smithi117 ceph-mon[99036]: pgmap v287: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:04 smithi117 ceph-mon[99036]: pgmap v288: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:04 smithi173 ceph-mon[74165]: pgmap v288: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:06.530 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:06 smithi173 ceph-mon[74165]: pgmap v289: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:06 smithi117 ceph-mon[99036]: pgmap v289: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:07.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:07 smithi117 ceph-mon[99036]: pgmap v290: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:07 smithi173 ceph-mon[74165]: pgmap v290: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:10.570 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:10 smithi117 ceph-mon[99036]: pgmap v291: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:10 smithi173 ceph-mon[74165]: pgmap v291: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:12.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:12 smithi117 ceph-mon[99036]: pgmap v292: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:12 smithi173 ceph-mon[74165]: pgmap v292: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:22:13.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:22:14.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.496+0000 7fcdc8200700 1 -- 172.21.15.117:0/2748493018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc01017c0 msgr2=0x7fcdc0105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:14.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.496+0000 7fcdc8200700 1 --2- 172.21.15.117:0/2748493018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc01017c0 0x7fcdc0105810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009910 tx=0x7fcdac009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:14.497 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:14 smithi117 ceph-mon[99036]: pgmap v293: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:14.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.496+0000 7fcdc8200700 1 -- 172.21.15.117:0/2748493018 shutdown_connections 2024-04-20T11:22:14.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.496+0000 7fcdc8200700 1 --2- 172.21.15.117:0/2748493018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc01017c0 0x7fcdc0105810 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.496+0000 7fcdc8200700 1 --2- 172.21.15.117:0/2748493018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdc0100e10 0x7fcdc01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.496+0000 7fcdc8200700 1 -- 172.21.15.117:0/2748493018 >> 172.21.15.117:0/2748493018 conn(0x7fcdc00fc6c0 msgr2=0x7fcdc00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:14.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.496+0000 7fcdc8200700 1 -- 172.21.15.117:0/2748493018 shutdown_connections 2024-04-20T11:22:14.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.496+0000 7fcdc8200700 1 -- 172.21.15.117:0/2748493018 wait complete. 2024-04-20T11:22:14.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.497+0000 7fcdc8200700 1 Processor -- start 2024-04-20T11:22:14.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.497+0000 7fcdc8200700 1 -- start start 2024-04-20T11:22:14.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.497+0000 7fcdc8200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc0100e10 0x7fcdc00731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:14.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc8200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdc01017c0 0x7fcdc0073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:14.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc8200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0073c60 con 0x7fcdc0100e10 2024-04-20T11:22:14.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc8200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0073da0 con 0x7fcdc01017c0 2024-04-20T11:22:14.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc5800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc0100e10 0x7fcdc00731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:14.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc5800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc0100e10 0x7fcdc00731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52696/0 (socket says 172.21.15.117:52696) 2024-04-20T11:22:14.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc5800700 1 -- 172.21.15.117:0/2695063920 learned_addr learned my addr 172.21.15.117:0/2695063920 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:14.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc4e00700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdc01017c0 0x7fcdc0073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:14.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc5800700 1 -- 172.21.15.117:0/2695063920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdc01017c0 msgr2=0x7fcdc0073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:14.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc5800700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdc01017c0 0x7fcdc0073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.498+0000 7fcdc5800700 1 -- 172.21.15.117:0/2695063920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0095f0 con 0x7fcdc0100e10 2024-04-20T11:22:14.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.499+0000 7fcdc5800700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc0100e10 0x7fcdc00731e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40060b0 tx=0x7fcdb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:14.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.499+0000 7fcdbe000700 1 -- 172.21.15.117:0/2695063920 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdb400dda0 con 0x7fcdc0100e10 2024-04-20T11:22:14.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.499+0000 7fcdbe000700 1 -- 172.21.15.117:0/2695063920 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdb400f040 con 0x7fcdc0100e10 2024-04-20T11:22:14.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.499+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc01a2cf0 con 0x7fcdc0100e10 2024-04-20T11:22:14.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.499+0000 7fcdbe000700 1 -- 172.21.15.117:0/2695063920 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdb4014790 con 0x7fcdc0100e10 2024-04-20T11:22:14.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.499+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc01a3050 con 0x7fcdc0100e10 2024-04-20T11:22:14.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.500+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0109210 con 0x7fcdc0100e10 2024-04-20T11:22:14.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.501+0000 7fcdbe000700 1 -- 172.21.15.117:0/2695063920 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcdb40148f0 con 0x7fcdc0100e10 2024-04-20T11:22:14.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.502+0000 7fcdbe000700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcda806ecc0 0x7fcda8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:14.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.502+0000 7fcdbe000700 1 -- 172.21.15.117:0/2695063920 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcdb408e8a0 con 0x7fcdc0100e10 2024-04-20T11:22:14.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.502+0000 7fcdc4e00700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcda806ecc0 0x7fcda8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:14.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.503+0000 7fcdc4e00700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcda806ecc0 0x7fcda8071180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcdac006010 tx=0x7fcdac00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:14.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.507+0000 7fcdbe000700 1 -- 172.21.15.117:0/2695063920 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcdb4057710 con 0x7fcdc0100e10 2024-04-20T11:22:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:14 smithi173 ceph-mon[74165]: pgmap v293: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:14.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.751+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdc0073f60 con 0x7fcda806ecc0 2024-04-20T11:22:14.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.754+0000 7fcdbe000700 1 -- 172.21.15.117:0/2695063920 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcdc0073f60 con 0x7fcda806ecc0 2024-04-20T11:22:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcda806ecc0 msgr2=0x7fcda8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcda806ecc0 0x7fcda8071180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcdac006010 tx=0x7fcdac00b540 comp rx=0 tx=0).stop 2024-04-20T11:22:14.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc0100e10 msgr2=0x7fcdc00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:14.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc0100e10 0x7fcdc00731e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40060b0 tx=0x7fcdb400d3b0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 shutdown_connections 2024-04-20T11:22:14.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcda806ecc0 0x7fcda8071180 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdc0100e10 0x7fcdc00731e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 --2- 172.21.15.117:0/2695063920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdc01017c0 0x7fcdc0073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.757+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 >> 172.21.15.117:0/2695063920 conn(0x7fcdc00fc6c0 msgr2=0x7fcdc00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:14.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.758+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 shutdown_connections 2024-04-20T11:22:14.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.758+0000 7fcdc8200700 1 -- 172.21.15.117:0/2695063920 wait complete. 2024-04-20T11:22:14.775 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:22:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.918+0000 7fe9fa600700 1 -- 172.21.15.117:0/1646478839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 msgr2=0x7fe9f4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.918+0000 7fe9fa600700 1 --2- 172.21.15.117:0/1646478839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 0x7fe9f4109290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009910 tx=0x7fe9e0009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.918+0000 7fe9fa600700 1 -- 172.21.15.117:0/1646478839 shutdown_connections 2024-04-20T11:22:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.918+0000 7fe9fa600700 1 --2- 172.21.15.117:0/1646478839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 0x7fe9f4109290 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.918+0000 7fe9fa600700 1 --2- 172.21.15.117:0/1646478839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9f4107520 0x7fe9f4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.918+0000 7fe9fa600700 1 -- 172.21.15.117:0/1646478839 >> 172.21.15.117:0/1646478839 conn(0x7fe9f40768f0 msgr2=0x7fe9f4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.919+0000 7fe9fa600700 1 -- 172.21.15.117:0/1646478839 shutdown_connections 2024-04-20T11:22:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.919+0000 7fe9fa600700 1 -- 172.21.15.117:0/1646478839 wait complete. 2024-04-20T11:22:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.920+0000 7fe9fa600700 1 Processor -- start 2024-04-20T11:22:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.920+0000 7fe9fa600700 1 -- start start 2024-04-20T11:22:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.921+0000 7fe9fa600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9f4107520 0x7fe9f419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.921+0000 7fe9fa600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 0x7fe9f419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.921+0000 7fe9fa600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f419ddf0 con 0x7fe9f4108e10 2024-04-20T11:22:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.921+0000 7fe9fa600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f41a1a70 con 0x7fe9f4107520 2024-04-20T11:22:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.921+0000 7fe9f2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 0x7fe9f419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.921+0000 7fe9f2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 0x7fe9f419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52710/0 (socket says 172.21.15.117:52710) 2024-04-20T11:22:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.921+0000 7fe9f2a00700 1 -- 172.21.15.117:0/957973716 learned_addr learned my addr 172.21.15.117:0/957973716 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9f3400700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9f4107520 0x7fe9f419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9f2a00700 1 -- 172.21.15.117:0/957973716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9f4107520 msgr2=0x7fe9f419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9f2a00700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9f4107520 0x7fe9f419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9f2a00700 1 -- 172.21.15.117:0/957973716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00095f0 con 0x7fe9f4108e10 2024-04-20T11:22:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9f2a00700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 0x7fe9f419d710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000bd70 tx=0x7fe9e000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:14.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9ebe00700 1 -- 172.21.15.117:0/957973716 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9e001d070 con 0x7fe9f4108e10 2024-04-20T11:22:14.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9ebe00700 1 -- 172.21.15.117:0/957973716 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9e000f460 con 0x7fe9f4108e10 2024-04-20T11:22:14.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f41a1cf0 con 0x7fe9f4108e10 2024-04-20T11:22:14.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.922+0000 7fe9ebe00700 1 -- 172.21.15.117:0/957973716 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9e0017630 con 0x7fe9f4108e10 2024-04-20T11:22:14.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.923+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f41a2260 con 0x7fe9f4108e10 2024-04-20T11:22:14.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.924+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f410f870 con 0x7fe9f4108e10 2024-04-20T11:22:14.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.925+0000 7fe9ebe00700 1 -- 172.21.15.117:0/957973716 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe9e000f5d0 con 0x7fe9f4108e10 2024-04-20T11:22:14.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.926+0000 7fe9ebe00700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe9d806ecc0 0x7fe9d8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.926+0000 7fe9ebe00700 1 -- 172.21.15.117:0/957973716 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe9e0090330 con 0x7fe9f4108e10 2024-04-20T11:22:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.926+0000 7fe9f3400700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe9d806ecc0 0x7fe9d8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.927+0000 7fe9f3400700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe9d806ecc0 0x7fe9d8071180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005950 tx=0x7fe9e4005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:14.931+0000 7fe9ebe00700 1 -- 172.21.15.117:0/957973716 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe9e00590f0 con 0x7fe9f4108e10 2024-04-20T11:22:15.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.179+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9f419e490 con 0x7fe9d806ecc0 2024-04-20T11:22:15.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.182+0000 7fe9ebe00700 1 -- 172.21.15.117:0/957973716 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe9f419e490 con 0x7fe9d806ecc0 2024-04-20T11:22:15.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe9d806ecc0 msgr2=0x7fe9d8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:15.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe9d806ecc0 0x7fe9d8071180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005950 tx=0x7fe9e4005cf0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 msgr2=0x7fe9f419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:15.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 0x7fe9f419d710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000bd70 tx=0x7fe9e000be50 comp rx=0 tx=0).stop 2024-04-20T11:22:15.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 shutdown_connections 2024-04-20T11:22:15.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe9d806ecc0 0x7fe9d8071180 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9f4107520 0x7fe9f419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 --2- 172.21.15.117:0/957973716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f4108e10 0x7fe9f419d710 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.185+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 >> 172.21.15.117:0/957973716 conn(0x7fe9f40768f0 msgr2=0x7fe9f4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:15.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.186+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 shutdown_connections 2024-04-20T11:22:15.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.186+0000 7fe9fa600700 1 -- 172.21.15.117:0/957973716 wait complete. 2024-04-20T11:22:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.346+0000 7fedb9800700 1 -- 172.21.15.117:0/2721652200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 msgr2=0x7fedb4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.346+0000 7fedb9800700 1 --2- 172.21.15.117:0/2721652200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 0x7fedb4101200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feda4009910 tx=0x7feda4009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.346+0000 7fedb9800700 1 -- 172.21.15.117:0/2721652200 shutdown_connections 2024-04-20T11:22:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.346+0000 7fedb9800700 1 --2- 172.21.15.117:0/2721652200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb41017d0 0x7fedb4105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.346+0000 7fedb9800700 1 --2- 172.21.15.117:0/2721652200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 0x7fedb4101200 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.346+0000 7fedb9800700 1 -- 172.21.15.117:0/2721652200 >> 172.21.15.117:0/2721652200 conn(0x7fedb40fc6b0 msgr2=0x7fedb40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.346+0000 7fedb9800700 1 -- 172.21.15.117:0/2721652200 shutdown_connections 2024-04-20T11:22:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.346+0000 7fedb9800700 1 -- 172.21.15.117:0/2721652200 wait complete. 2024-04-20T11:22:15.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.347+0000 7fedb9800700 1 Processor -- start 2024-04-20T11:22:15.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.348+0000 7fedb9800700 1 -- start start 2024-04-20T11:22:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.348+0000 7fedb9800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 0x7fedb419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.348+0000 7fedb9800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb41017d0 0x7fedb419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.348+0000 7fedb9800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb419d370 con 0x7fedb4100e20 2024-04-20T11:22:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.348+0000 7fedb9800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb4196950 con 0x7fedb41017d0 2024-04-20T11:22:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.348+0000 7fedb2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 0x7fedb419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.349+0000 7fedb2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 0x7fedb419c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52728/0 (socket says 172.21.15.117:52728) 2024-04-20T11:22:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.349+0000 7fedb2a00700 1 -- 172.21.15.117:0/417865444 learned_addr learned my addr 172.21.15.117:0/417865444 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.349+0000 7fedb2000700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb41017d0 0x7fedb419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.349+0000 7fedb2a00700 1 -- 172.21.15.117:0/417865444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb41017d0 msgr2=0x7fedb419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.349+0000 7fedb2a00700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb41017d0 0x7fedb419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.349+0000 7fedb2a00700 1 -- 172.21.15.117:0/417865444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40095f0 con 0x7fedb4100e20 2024-04-20T11:22:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.349+0000 7fedb2a00700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 0x7fedb419c810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feda4000c00 tx=0x7feda400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.350+0000 7fedab400700 1 -- 172.21.15.117:0/417865444 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feda401d070 con 0x7fedb4100e20 2024-04-20T11:22:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.350+0000 7fedab400700 1 -- 172.21.15.117:0/417865444 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feda40056e0 con 0x7fedb4100e20 2024-04-20T11:22:15.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.350+0000 7fedab400700 1 -- 172.21.15.117:0/417865444 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feda400fe80 con 0x7fedb4100e20 2024-04-20T11:22:15.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.350+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb4196bd0 con 0x7fedb4100e20 2024-04-20T11:22:15.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.350+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb41970c0 con 0x7fedb4100e20 2024-04-20T11:22:15.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.351+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb41091c0 con 0x7fedb4100e20 2024-04-20T11:22:15.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.352+0000 7fedab400700 1 -- 172.21.15.117:0/417865444 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7feda400f6f0 con 0x7fedb4100e20 2024-04-20T11:22:15.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.353+0000 7fedab400700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fed9806ecc0 0x7fed98071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:15.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.353+0000 7fedab400700 1 -- 172.21.15.117:0/417865444 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7feda4090200 con 0x7fedb4100e20 2024-04-20T11:22:15.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.353+0000 7fedb2000700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fed9806ecc0 0x7fed98071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:15.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.354+0000 7fedb2000700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fed9806ecc0 0x7fed98071180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feda0005950 tx=0x7feda00058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:15.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.358+0000 7fedab400700 1 -- 172.21.15.117:0/417865444 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7feda405caf0 con 0x7fedb4100e20 2024-04-20T11:22:15.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.588+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fedb410ddf0 con 0x7fed9806ecc0 2024-04-20T11:22:15.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.603+0000 7fedab400700 1 -- 172.21.15.117:0/417865444 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fedb410ddf0 con 0x7fed9806ecc0 2024-04-20T11:22:15.604 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:22:15.604 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (10m) 2m ago 24m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:22:15.604 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (25m) 2m ago 25m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (23m) 2m ago 23m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (7m) 2m ago 25m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (6m) 2m ago 23m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (10m) 2m ago 24m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (2m) 2m ago 17m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (2m) 2m ago 17m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (17m) 2m ago 17m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (2m) 2m ago 17m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (13m) 2m ago 26m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (12m) 2m ago 23m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (8m) 2m ago 26m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (8m) 2m ago 23m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (12m) 2m ago 24m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:22:15.605 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (11m) 2m ago 23m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:22:15.606 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (6m) 2m ago 22m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:22:15.606 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (5m) 2m ago 21m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:22:15.606 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (5m) 2m ago 21m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:22:15.606 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (4m) 2m ago 20m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:22:15.606 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (4m) 2m ago 20m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:22:15.606 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (3m) 2m ago 19m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:22:15.606 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (11m) 2m ago 24m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fed9806ecc0 msgr2=0x7fed98071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fed9806ecc0 0x7fed98071180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feda0005950 tx=0x7feda00058e0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 msgr2=0x7fedb419c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 0x7fedb419c810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feda4000c00 tx=0x7feda400be20 comp rx=0 tx=0).stop 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 shutdown_connections 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fed9806ecc0 0x7fed98071180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb4100e20 0x7fedb419c810 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 --2- 172.21.15.117:0/417865444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb41017d0 0x7fedb419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 >> 172.21.15.117:0/417865444 conn(0x7fedb40fc6b0 msgr2=0x7fedb40fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 shutdown_connections 2024-04-20T11:22:15.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.606+0000 7fedb9800700 1 -- 172.21.15.117:0/417865444 wait complete. 2024-04-20T11:22:15.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.764+0000 7f72b9e00700 1 -- 172.21.15.117:0/4000609208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 msgr2=0x7f72b40736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:15.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.764+0000 7f72b9e00700 1 --2- 172.21.15.117:0/4000609208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 0x7f72b40736f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009910 tx=0x7f72a4009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:15.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.765+0000 7f72b9e00700 1 -- 172.21.15.117:0/4000609208 shutdown_connections 2024-04-20T11:22:15.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.765+0000 7f72b9e00700 1 --2- 172.21.15.117:0/4000609208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b4073c30 0x7f72b410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.765+0000 7f72b9e00700 1 --2- 172.21.15.117:0/4000609208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 0x7f72b40736f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.765+0000 7f72b9e00700 1 -- 172.21.15.117:0/4000609208 >> 172.21.15.117:0/4000609208 conn(0x7f72b40fc6c0 msgr2=0x7f72b40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:15.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.765+0000 7f72b9e00700 1 -- 172.21.15.117:0/4000609208 shutdown_connections 2024-04-20T11:22:15.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.765+0000 7f72b9e00700 1 -- 172.21.15.117:0/4000609208 wait complete. 2024-04-20T11:22:15.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.766+0000 7f72b9e00700 1 Processor -- start 2024-04-20T11:22:15.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.766+0000 7f72b9e00700 1 -- start start 2024-04-20T11:22:15.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.767+0000 7f72b9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 0x7f72b4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:15.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.767+0000 7f72b9e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b4073c30 0x7f72b4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:15.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.767+0000 7f72b9e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b4199a20 con 0x7f72b4073310 2024-04-20T11:22:15.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.767+0000 7f72b9e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b419d6f0 con 0x7f72b4073c30 2024-04-20T11:22:15.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.767+0000 7f72b2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 0x7f72b4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:15.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.767+0000 7f72b2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 0x7f72b4198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52746/0 (socket says 172.21.15.117:52746) 2024-04-20T11:22:15.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.767+0000 7f72b2a00700 1 -- 172.21.15.117:0/639991890 learned_addr learned my addr 172.21.15.117:0/639991890 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b2000700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b4073c30 0x7f72b4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b2a00700 1 -- 172.21.15.117:0/639991890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b4073c30 msgr2=0x7f72b4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b2a00700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b4073c30 0x7f72b4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b2a00700 1 -- 172.21.15.117:0/639991890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40095f0 con 0x7f72b4073310 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b2a00700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 0x7f72b4198e00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72a40092e0 tx=0x7f72a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b8a00700 1 -- 172.21.15.117:0/639991890 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72a401d070 con 0x7f72b4073310 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b8a00700 1 -- 172.21.15.117:0/639991890 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72a400f460 con 0x7f72b4073310 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b419d970 con 0x7f72b4073310 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.768+0000 7f72b8a00700 1 -- 172.21.15.117:0/639991890 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72a4017630 con 0x7f72b4073310 2024-04-20T11:22:15.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.769+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b419de60 con 0x7f72b4073310 2024-04-20T11:22:15.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.770+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b40749c0 con 0x7f72b4073310 2024-04-20T11:22:15.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.771+0000 7f72b8a00700 1 -- 172.21.15.117:0/639991890 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f72a400f5d0 con 0x7f72b4073310 2024-04-20T11:22:15.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.772+0000 7f72b8a00700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f729806ecc0 0x7f7298071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:15.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.772+0000 7f72b8a00700 1 -- 172.21.15.117:0/639991890 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f72a4090320 con 0x7f72b4073310 2024-04-20T11:22:15.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.772+0000 7f72b2000700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f729806ecc0 0x7f7298071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:15.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.773+0000 7f72b2000700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f729806ecc0 0x7f7298071180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006fd0 tx=0x7f72a0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:15.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:15.776+0000 7f72b8a00700 1 -- 172.21.15.117:0/639991890 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f72a40590e0 con 0x7f72b4073310 2024-04-20T11:22:16.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.084+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f72b4107750 con 0x7f72b4073310 2024-04-20T11:22:16.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.085+0000 7f72b8a00700 1 -- 172.21.15.117:0/639991890 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f72a405c7a0 con 0x7f72b4073310 2024-04-20T11:22:16.086 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:22:16.086 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:22:16.086 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:22:16.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:22:16.086 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:22:16.086 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:22:16.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:22:16.086 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:22:16.087 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:22:16.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f729806ecc0 msgr2=0x7f7298071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f729806ecc0 0x7f7298071180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006fd0 tx=0x7f72a0008040 comp rx=0 tx=0).stop 2024-04-20T11:22:16.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 msgr2=0x7f72b4198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 0x7f72b4198e00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72a40092e0 tx=0x7f72a400be20 comp rx=0 tx=0).stop 2024-04-20T11:22:16.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 shutdown_connections 2024-04-20T11:22:16.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f729806ecc0 0x7f7298071180 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72b4073310 0x7f72b4198e00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 --2- 172.21.15.117:0/639991890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b4073c30 0x7f72b4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 >> 172.21.15.117:0/639991890 conn(0x7f72b40fc6c0 msgr2=0x7f72b4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:16.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 shutdown_connections 2024-04-20T11:22:16.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.088+0000 7f72b9e00700 1 -- 172.21.15.117:0/639991890 wait complete. 2024-04-20T11:22:16.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.233+0000 7f184f200700 1 -- 172.21.15.117:0/4193015064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850103810 msgr2=0x7f1850107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.233+0000 7f184f200700 1 --2- 172.21.15.117:0/4193015064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850103810 0x7f1850107860 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f183c009910 tx=0x7f183c009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.233+0000 7f184f200700 1 -- 172.21.15.117:0/4193015064 shutdown_connections 2024-04-20T11:22:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.233+0000 7f184f200700 1 --2- 172.21.15.117:0/4193015064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850103810 0x7f1850107860 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.233+0000 7f184f200700 1 --2- 172.21.15.117:0/4193015064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1850102e60 0x7f1850103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.233+0000 7f184f200700 1 -- 172.21.15.117:0/4193015064 >> 172.21.15.117:0/4193015064 conn(0x7f18500fe710 msgr2=0x7f1850100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.233+0000 7f184f200700 1 -- 172.21.15.117:0/4193015064 shutdown_connections 2024-04-20T11:22:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.233+0000 7f184f200700 1 -- 172.21.15.117:0/4193015064 wait complete. 2024-04-20T11:22:16.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.235+0000 7f184f200700 1 Processor -- start 2024-04-20T11:22:16.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.235+0000 7f184f200700 1 -- start start 2024-04-20T11:22:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.235+0000 7f184f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850102e60 0x7f185019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.235+0000 7f184f200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1850103810 0x7f185019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.235+0000 7f184f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019f390 con 0x7f1850102e60 2024-04-20T11:22:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.235+0000 7f184f200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18501989c0 con 0x7f1850103810 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.236+0000 7f184de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850102e60 0x7f185019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.236+0000 7f184de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850102e60 0x7f185019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52758/0 (socket says 172.21.15.117:52758) 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.236+0000 7f184de00700 1 -- 172.21.15.117:0/719587423 learned_addr learned my addr 172.21.15.117:0/719587423 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.236+0000 7f184de00700 1 -- 172.21.15.117:0/719587423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1850103810 msgr2=0x7f185019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.236+0000 7f184de00700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1850103810 0x7f185019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.236+0000 7f184de00700 1 -- 172.21.15.117:0/719587423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c0095f0 con 0x7f1850102e60 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.236+0000 7f184de00700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850102e60 0x7f185019e830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f184000c8a0 tx=0x7f184000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.236+0000 7f1846a00700 1 -- 172.21.15.117:0/719587423 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184000cea0 con 0x7f1850102e60 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.237+0000 7f1846a00700 1 -- 172.21.15.117:0/719587423 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1840004830 con 0x7f1850102e60 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.237+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1850198ca0 con 0x7f1850102e60 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.237+0000 7f1846a00700 1 -- 172.21.15.117:0/719587423 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18400055b0 con 0x7f1850102e60 2024-04-20T11:22:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.237+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18501991f0 con 0x7f1850102e60 2024-04-20T11:22:16.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.238+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185010b260 con 0x7f1850102e60 2024-04-20T11:22:16.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.239+0000 7f1846a00700 1 -- 172.21.15.117:0/719587423 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f18400049a0 con 0x7f1850102e60 2024-04-20T11:22:16.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.240+0000 7f1846a00700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f183406ecd0 0x7f1834071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:16.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.240+0000 7f1846a00700 1 -- 172.21.15.117:0/719587423 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f184008ea20 con 0x7f1850102e60 2024-04-20T11:22:16.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.240+0000 7f184d400700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f183406ecd0 0x7f1834071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:16.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.241+0000 7f184d400700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f183406ecd0 0x7f1834071190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f183c006010 tx=0x7f183c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:16.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.244+0000 7f1846a00700 1 -- 172.21.15.117:0/719587423 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f184005b3b0 con 0x7f1850102e60 2024-04-20T11:22:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:16 smithi117 ceph-mon[99036]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:16 smithi117 ceph-mon[99036]: pgmap v294: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:16 smithi117 ceph-mon[99036]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:16 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/639991890' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:22:16.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.513+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f185010fe00 con 0x7f1850102e60 2024-04-20T11:22:16.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.514+0000 7f1846a00700 1 -- 172.21.15.117:0/719587423 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f184001d070 con 0x7f1850102e60 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:22:16.516 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:22:16.517 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:22:16.518 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:16.519 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:16.519 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:16.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f183406ecd0 msgr2=0x7f1834071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f183406ecd0 0x7f1834071190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f183c006010 tx=0x7f183c00b540 comp rx=0 tx=0).stop 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850102e60 msgr2=0x7f185019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850102e60 0x7f185019e830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f184000c8a0 tx=0x7f184000cc60 comp rx=0 tx=0).stop 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 shutdown_connections 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f183406ecd0 0x7f1834071190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850102e60 0x7f185019e830 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 --2- 172.21.15.117:0/719587423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1850103810 0x7f185019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 >> 172.21.15.117:0/719587423 conn(0x7f18500fe710 msgr2=0x7f18500ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 shutdown_connections 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.517+0000 7f184f200700 1 -- 172.21.15.117:0/719587423 wait complete. 2024-04-20T11:22:16.520 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:22:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.660+0000 7f4cc4c00700 1 -- 172.21.15.117:0/3279855822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107440 msgr2=0x7f4cc0107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.660+0000 7f4cc4c00700 1 --2- 172.21.15.117:0/3279855822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107440 0x7f4cc0107820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009910 tx=0x7f4cb0009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.660+0000 7f4cc4c00700 1 -- 172.21.15.117:0/3279855822 shutdown_connections 2024-04-20T11:22:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.660+0000 7f4cc4c00700 1 --2- 172.21.15.117:0/3279855822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cc0107df0 0x7f4cc010be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.660+0000 7f4cc4c00700 1 --2- 172.21.15.117:0/3279855822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107440 0x7f4cc0107820 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.660+0000 7f4cc4c00700 1 -- 172.21.15.117:0/3279855822 >> 172.21.15.117:0/3279855822 conn(0x7f4cc0076a10 msgr2=0x7f4cc0078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:16.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.660+0000 7f4cc4c00700 1 -- 172.21.15.117:0/3279855822 shutdown_connections 2024-04-20T11:22:16.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.660+0000 7f4cc4c00700 1 -- 172.21.15.117:0/3279855822 wait complete. 2024-04-20T11:22:16.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:16 smithi173 ceph-mon[74165]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:16.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:16 smithi173 ceph-mon[74165]: pgmap v294: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:16.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:16 smithi173 ceph-mon[74165]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:16.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:16 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/639991890' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:22:16.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.662+0000 7f4cc4c00700 1 Processor -- start 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.662+0000 7f4cc4c00700 1 -- start start 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.663+0000 7f4cc4c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cc0107440 0x7f4cc019d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.663+0000 7f4cc4c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107df0 0x7f4cc019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.663+0000 7f4cc4c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc019dd70 con 0x7f4cc0107df0 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.663+0000 7f4cc4c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc01a19f0 con 0x7f4cc0107440 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.663+0000 7f4cbd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107df0 0x7f4cc019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.663+0000 7f4cbd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107df0 0x7f4cc019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52780/0 (socket says 172.21.15.117:52780) 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.663+0000 7f4cbd400700 1 -- 172.21.15.117:0/2342309749 learned_addr learned my addr 172.21.15.117:0/2342309749 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:16.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.663+0000 7f4cbde00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cc0107440 0x7f4cc019d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cbd400700 1 -- 172.21.15.117:0/2342309749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cc0107440 msgr2=0x7f4cc019d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cbd400700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cc0107440 0x7f4cc019d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cbd400700 1 -- 172.21.15.117:0/2342309749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb00095f0 con 0x7f4cc0107df0 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cbd400700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107df0 0x7f4cc019d690 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00c8f0 tx=0x7f4cac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cb6a00700 1 -- 172.21.15.117:0/2342309749 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cac007a50 con 0x7f4cc0107df0 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cb6a00700 1 -- 172.21.15.117:0/2342309749 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cac007bb0 con 0x7f4cc0107df0 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc01a1cd0 con 0x7f4cc0107df0 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cb6a00700 1 -- 172.21.15.117:0/2342309749 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cac018630 con 0x7f4cc0107df0 2024-04-20T11:22:16.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.664+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc01a2220 con 0x7f4cc0107df0 2024-04-20T11:22:16.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.666+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc010f7e0 con 0x7f4cc0107df0 2024-04-20T11:22:16.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.667+0000 7f4cb6a00700 1 -- 172.21.15.117:0/2342309749 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4cac007d20 con 0x7f4cc0107df0 2024-04-20T11:22:16.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.668+0000 7f4cb6a00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4ca406ecc0 0x7f4ca4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:16.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.668+0000 7f4cb6a00700 1 -- 172.21.15.117:0/2342309749 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f4cac08eec0 con 0x7f4cc0107df0 2024-04-20T11:22:16.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.668+0000 7f4cbde00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4ca406ecc0 0x7f4ca4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:16.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.668+0000 7f4cbde00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4ca406ecc0 0x7f4ca4071180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0006010 tx=0x7f4cb000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:16.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.672+0000 7f4cb6a00700 1 -- 172.21.15.117:0/2342309749 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4cac057c00 con 0x7f4cc0107df0 2024-04-20T11:22:16.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.915+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cc0114400 con 0x7f4ca406ecc0 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.917+0000 7f4cb6a00700 1 -- 172.21.15.117:0/2342309749 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4cc0114400 con 0x7f4ca406ecc0 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:22:16.918 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:22:16.919 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:22:16.919 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:22:16.919 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:22:16.919 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:22:16.919 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:22:16.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4ca406ecc0 msgr2=0x7f4ca4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4ca406ecc0 0x7f4ca4071180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0006010 tx=0x7f4cb000b540 comp rx=0 tx=0).stop 2024-04-20T11:22:16.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107df0 msgr2=0x7f4cc019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:16.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107df0 0x7f4cc019d690 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00c8f0 tx=0x7f4cac00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 shutdown_connections 2024-04-20T11:22:16.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4ca406ecc0 0x7f4ca4071180 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cc0107440 0x7f4cc019d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 --2- 172.21.15.117:0/2342309749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0107df0 0x7f4cc019d690 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:16.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.920+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 >> 172.21.15.117:0/2342309749 conn(0x7f4cc0076a10 msgr2=0x7f4cc0076df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:16.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.921+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 shutdown_connections 2024-04-20T11:22:16.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:16.921+0000 7f4cc4c00700 1 -- 172.21.15.117:0/2342309749 wait complete. 2024-04-20T11:22:17.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.075+0000 7f385f200700 1 -- 172.21.15.117:0/903303597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860106a20 msgr2=0x7f3860106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.075+0000 7f385f200700 1 --2- 172.21.15.117:0/903303597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860106a20 0x7f3860106e00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3850009910 tx=0x7f3850009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.075+0000 7f385f200700 1 -- 172.21.15.117:0/903303597 shutdown_connections 2024-04-20T11:22:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.075+0000 7f385f200700 1 --2- 172.21.15.117:0/903303597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3860100a20 0x7f3860100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.075+0000 7f385f200700 1 --2- 172.21.15.117:0/903303597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860106a20 0x7f3860106e00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.075+0000 7f385f200700 1 -- 172.21.15.117:0/903303597 >> 172.21.15.117:0/903303597 conn(0x7f38600fc530 msgr2=0x7f38600fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.076+0000 7f385f200700 1 -- 172.21.15.117:0/903303597 shutdown_connections 2024-04-20T11:22:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.076+0000 7f385f200700 1 -- 172.21.15.117:0/903303597 wait complete. 2024-04-20T11:22:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.077+0000 7f385f200700 1 Processor -- start 2024-04-20T11:22:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.077+0000 7f385f200700 1 -- start start 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.077+0000 7f385f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860100a20 0x7f386019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.077+0000 7f385f200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3860106a20 0x7f386019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.077+0000 7f385f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386019f370 con 0x7f3860100a20 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.077+0000 7f385f200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860198950 con 0x7f3860106a20 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.077+0000 7f385de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860100a20 0x7f386019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f385de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860100a20 0x7f386019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52806/0 (socket says 172.21.15.117:52806) 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f385de00700 1 -- 172.21.15.117:0/546167890 learned_addr learned my addr 172.21.15.117:0/546167890 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f385de00700 1 -- 172.21.15.117:0/546167890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3860106a20 msgr2=0x7f386019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f385de00700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3860106a20 0x7f386019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f385de00700 1 -- 172.21.15.117:0/546167890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500095f0 con 0x7f3860100a20 2024-04-20T11:22:17.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f385de00700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860100a20 0x7f386019e810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3850006010 tx=0x7f385000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f3856a00700 1 -- 172.21.15.117:0/546167890 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f385001d070 con 0x7f3860100a20 2024-04-20T11:22:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860198bd0 con 0x7f3860100a20 2024-04-20T11:22:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.078+0000 7f3856a00700 1 -- 172.21.15.117:0/546167890 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f385000f460 con 0x7f3860100a20 2024-04-20T11:22:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.079+0000 7f3856a00700 1 -- 172.21.15.117:0/546167890 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3850017630 con 0x7f3860100a20 2024-04-20T11:22:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.079+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38601990c0 con 0x7f3860100a20 2024-04-20T11:22:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.080+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860104cd0 con 0x7f3860100a20 2024-04-20T11:22:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.081+0000 7f3856a00700 1 -- 172.21.15.117:0/546167890 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f385000f5d0 con 0x7f3860100a20 2024-04-20T11:22:17.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.081+0000 7f3856a00700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f384406ecd0 0x7f3844071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:17.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.082+0000 7f3856a00700 1 -- 172.21.15.117:0/546167890 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f38500903f0 con 0x7f3860100a20 2024-04-20T11:22:17.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.082+0000 7f385d400700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f384406ecd0 0x7f3844071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:17.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.082+0000 7f385d400700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f384406ecd0 0x7f3844071190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3860199b10 tx=0x7f384c005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:17.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.086+0000 7f3856a00700 1 -- 172.21.15.117:0/546167890 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f385005cd20 con 0x7f3860100a20 2024-04-20T11:22:17.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.410+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3860199e10 con 0x7f3860100a20 2024-04-20T11:22:17.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.411+0000 7f3856a00700 1 -- 172.21.15.117:0/546167890 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f3850020370 con 0x7f3860100a20 2024-04-20T11:22:17.411 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:22:17.412 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:22:17.412 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:22:17.412 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:22:17.412 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:22:17.412 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:22:17.412 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f384406ecd0 msgr2=0x7f3844071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f384406ecd0 0x7f3844071190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3860199b10 tx=0x7f384c005c80 comp rx=0 tx=0).stop 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860100a20 msgr2=0x7f386019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860100a20 0x7f386019e810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3850006010 tx=0x7f385000be20 comp rx=0 tx=0).stop 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 shutdown_connections 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f384406ecd0 0x7f3844071190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3860100a20 0x7f386019e810 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 --2- 172.21.15.117:0/546167890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3860106a20 0x7f386019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 >> 172.21.15.117:0/546167890 conn(0x7f38600fc530 msgr2=0x7f38600732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 shutdown_connections 2024-04-20T11:22:17.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:17.414+0000 7f385f200700 1 -- 172.21.15.117:0/546167890 wait complete. 2024-04-20T11:22:17.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:17 smithi117 ceph-mon[99036]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:17.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:17 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/719587423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:22:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:17 smithi173 ceph-mon[74165]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:17 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/719587423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:22:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:18 smithi117 ceph-mon[99036]: from='client.34608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:18.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:18 smithi117 ceph-mon[99036]: pgmap v295: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:18.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:18 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/546167890' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:22:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:18 smithi173 ceph-mon[74165]: from='client.34608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:18 smithi173 ceph-mon[74165]: pgmap v295: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:18 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/546167890' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:22:19.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:19 smithi117 ceph-mon[99036]: pgmap v296: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:19 smithi173 ceph-mon[74165]: pgmap v296: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:22.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:22 smithi173 ceph-mon[74165]: pgmap v297: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:22.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:22 smithi117 ceph-mon[99036]: pgmap v297: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:24 smithi117 ceph-mon[99036]: pgmap v298: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:24 smithi173 ceph-mon[74165]: pgmap v298: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:26.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:26 smithi117 ceph-mon[99036]: pgmap v299: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:26 smithi173 ceph-mon[74165]: pgmap v299: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:28 smithi117 ceph-mon[99036]: pgmap v300: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:22:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:28 smithi173 ceph-mon[74165]: pgmap v300: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:22:30.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:30 smithi117 ceph-mon[99036]: pgmap v301: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:30 smithi173 ceph-mon[74165]: pgmap v301: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:32.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:32 smithi173 ceph-mon[74165]: pgmap v302: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:32 smithi117 ceph-mon[99036]: pgmap v302: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:34 smithi117 ceph-mon[99036]: pgmap v303: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:34 smithi173 ceph-mon[74165]: pgmap v303: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:36 smithi117 ceph-mon[99036]: pgmap v304: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:36 smithi173 ceph-mon[74165]: pgmap v304: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:37.486 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:37 smithi173 ceph-mon[74165]: pgmap v305: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:37.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:37 smithi117 ceph-mon[99036]: pgmap v305: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:40.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:40 smithi117 ceph-mon[99036]: pgmap v306: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:40 smithi173 ceph-mon[74165]: pgmap v306: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:42 smithi117 ceph-mon[99036]: pgmap v307: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:42.646 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:42 smithi173 ceph-mon[74165]: pgmap v307: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:22:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:22:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:44 smithi117 ceph-mon[99036]: pgmap v308: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:44 smithi173 ceph-mon[74165]: pgmap v308: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:46.618 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:46 smithi117 ceph-mon[99036]: pgmap v309: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:46 smithi173 ceph-mon[74165]: pgmap v309: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:47.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.585+0000 7f5a61a00700 1 -- 172.21.15.117:0/2887912824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 msgr2=0x7f5a5c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.585+0000 7f5a61a00700 1 --2- 172.21.15.117:0/2887912824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 0x7f5a5c101210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009910 tx=0x7f5a4c009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:47.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.585+0000 7f5a61a00700 1 -- 172.21.15.117:0/2887912824 shutdown_connections 2024-04-20T11:22:47.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.585+0000 7f5a61a00700 1 --2- 172.21.15.117:0/2887912824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a5c1017e0 0x7f5a5c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.585+0000 7f5a61a00700 1 --2- 172.21.15.117:0/2887912824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 0x7f5a5c101210 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.585+0000 7f5a61a00700 1 -- 172.21.15.117:0/2887912824 >> 172.21.15.117:0/2887912824 conn(0x7f5a5c0fc700 msgr2=0x7f5a5c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:47.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.585+0000 7f5a61a00700 1 -- 172.21.15.117:0/2887912824 shutdown_connections 2024-04-20T11:22:47.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.585+0000 7f5a61a00700 1 -- 172.21.15.117:0/2887912824 wait complete. 2024-04-20T11:22:47.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.586+0000 7f5a61a00700 1 Processor -- start 2024-04-20T11:22:47.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.586+0000 7f5a61a00700 1 -- start start 2024-04-20T11:22:47.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a61a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 0x7f5a5c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a61a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a5c1017e0 0x7f5a5c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a61a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c1998d0 con 0x7f5a5c100e30 2024-04-20T11:22:47.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a61a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c19c540 con 0x7f5a5c1017e0 2024-04-20T11:22:47.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a5aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 0x7f5a5c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a5aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 0x7f5a5c198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56772/0 (socket says 172.21.15.117:56772) 2024-04-20T11:22:47.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a5aa00700 1 -- 172.21.15.117:0/1155410086 learned_addr learned my addr 172.21.15.117:0/1155410086 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:47.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a5aa00700 1 -- 172.21.15.117:0/1155410086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a5c1017e0 msgr2=0x7f5a5c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a5aa00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a5c1017e0 0x7f5a5c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.587+0000 7f5a5aa00700 1 -- 172.21.15.117:0/1155410086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c0095f0 con 0x7f5a5c100e30 2024-04-20T11:22:47.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.588+0000 7f5a5aa00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 0x7f5a5c198d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0092e0 tx=0x7f5a4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:47.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.588+0000 7f5a53400700 1 -- 172.21.15.117:0/1155410086 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4c01d070 con 0x7f5a5c100e30 2024-04-20T11:22:47.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.588+0000 7f5a53400700 1 -- 172.21.15.117:0/1155410086 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a4c00f460 con 0x7f5a5c100e30 2024-04-20T11:22:47.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.588+0000 7f5a53400700 1 -- 172.21.15.117:0/1155410086 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4c017630 con 0x7f5a5c100e30 2024-04-20T11:22:47.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.588+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c19c7c0 con 0x7f5a5c100e30 2024-04-20T11:22:47.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.588+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c19cc60 con 0x7f5a5c100e30 2024-04-20T11:22:47.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.589+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c104fb0 con 0x7f5a5c100e30 2024-04-20T11:22:47.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.591+0000 7f5a53400700 1 -- 172.21.15.117:0/1155410086 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5a4c00f5d0 con 0x7f5a5c100e30 2024-04-20T11:22:47.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.591+0000 7f5a53400700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5a4006ecd0 0x7f5a40071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.592+0000 7f5a53400700 1 -- 172.21.15.117:0/1155410086 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5a4c0904b0 con 0x7f5a5c100e30 2024-04-20T11:22:47.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.592+0000 7f5a53e00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5a4006ecd0 0x7f5a40071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.593+0000 7f5a53e00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5a4006ecd0 0x7f5a40071190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c19a220 tx=0x7f5a44005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:47.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.597+0000 7f5a53400700 1 -- 172.21.15.117:0/1155410086 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5a4c059290 con 0x7f5a5c100e30 2024-04-20T11:22:47.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.845+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a5c19a040 con 0x7f5a4006ecd0 2024-04-20T11:22:47.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.847+0000 7f5a53400700 1 -- 172.21.15.117:0/1155410086 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5a5c19a040 con 0x7f5a4006ecd0 2024-04-20T11:22:47.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5a4006ecd0 msgr2=0x7f5a40071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5a4006ecd0 0x7f5a40071190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c19a220 tx=0x7f5a44005c80 comp rx=0 tx=0).stop 2024-04-20T11:22:47.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 msgr2=0x7f5a5c198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 0x7f5a5c198d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0092e0 tx=0x7f5a4c00be20 comp rx=0 tx=0).stop 2024-04-20T11:22:47.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 shutdown_connections 2024-04-20T11:22:47.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5a4006ecd0 0x7f5a40071190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c100e30 0x7f5a5c198d00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 --2- 172.21.15.117:0/1155410086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a5c1017e0 0x7f5a5c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 >> 172.21.15.117:0/1155410086 conn(0x7f5a5c0fc700 msgr2=0x7f5a5c0fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:47.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 shutdown_connections 2024-04-20T11:22:47.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:47.850+0000 7f5a61a00700 1 -- 172.21.15.117:0/1155410086 wait complete. 2024-04-20T11:22:47.867 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:22:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.010+0000 7fc69ae00700 1 -- 172.21.15.117:0/765060946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 msgr2=0x7fc69410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.010+0000 7fc69ae00700 1 --2- 172.21.15.117:0/765060946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 0x7fc69410be40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc680009910 tx=0x7fc680009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.010+0000 7fc69ae00700 1 -- 172.21.15.117:0/765060946 shutdown_connections 2024-04-20T11:22:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.010+0000 7fc69ae00700 1 --2- 172.21.15.117:0/765060946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 0x7fc69410be40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.010+0000 7fc69ae00700 1 --2- 172.21.15.117:0/765060946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc694107440 0x7fc694107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.010+0000 7fc69ae00700 1 -- 172.21.15.117:0/765060946 >> 172.21.15.117:0/765060946 conn(0x7fc694076a10 msgr2=0x7fc694078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.010+0000 7fc69ae00700 1 -- 172.21.15.117:0/765060946 shutdown_connections 2024-04-20T11:22:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.010+0000 7fc69ae00700 1 -- 172.21.15.117:0/765060946 wait complete. 2024-04-20T11:22:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.011+0000 7fc69ae00700 1 Processor -- start 2024-04-20T11:22:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.011+0000 7fc69ae00700 1 -- start start 2024-04-20T11:22:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc69ae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc694107440 0x7fc69419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc69ae00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 0x7fc69419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc69ae00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69419deb0 con 0x7fc694107df0 2024-04-20T11:22:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc69ae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6941a1b80 con 0x7fc694107440 2024-04-20T11:22:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc693400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 0x7fc69419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc693400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 0x7fc69419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56786/0 (socket says 172.21.15.117:56786) 2024-04-20T11:22:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc693400700 1 -- 172.21.15.117:0/193179631 learned_addr learned my addr 172.21.15.117:0/193179631 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc693400700 1 -- 172.21.15.117:0/193179631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc694107440 msgr2=0x7fc69419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc693400700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc694107440 0x7fc69419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.012+0000 7fc693400700 1 -- 172.21.15.117:0/193179631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800095f0 con 0x7fc694107df0 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.013+0000 7fc693400700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 0x7fc69419d7d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc6800092e0 tx=0x7fc68000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.013+0000 7fc690c00700 1 -- 172.21.15.117:0/193179631 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc68001d070 con 0x7fc694107df0 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.013+0000 7fc690c00700 1 -- 172.21.15.117:0/193179631 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc68000f460 con 0x7fc694107df0 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.013+0000 7fc690c00700 1 -- 172.21.15.117:0/193179631 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc680017610 con 0x7fc694107df0 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.013+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6941a1e00 con 0x7fc694107df0 2024-04-20T11:22:48.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.013+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6941a22f0 con 0x7fc694107df0 2024-04-20T11:22:48.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.014+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc694108920 con 0x7fc694107df0 2024-04-20T11:22:48.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.016+0000 7fc690c00700 1 -- 172.21.15.117:0/193179631 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc68000f5d0 con 0x7fc694107df0 2024-04-20T11:22:48.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.017+0000 7fc690c00700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc67806ecc0 0x7fc678071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.017+0000 7fc690c00700 1 -- 172.21.15.117:0/193179631 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc680090230 con 0x7fc694107df0 2024-04-20T11:22:48.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.017+0000 7fc693e00700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc67806ecc0 0x7fc678071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.018+0000 7fc693e00700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc67806ecc0 0x7fc678071180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6840097b0 tx=0x7fc684006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.021+0000 7fc690c00700 1 -- 172.21.15.117:0/193179631 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc680059090 con 0x7fc694107df0 2024-04-20T11:22:48.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.263+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc694114460 con 0x7fc67806ecc0 2024-04-20T11:22:48.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.266+0000 7fc690c00700 1 -- 172.21.15.117:0/193179631 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc694114460 con 0x7fc67806ecc0 2024-04-20T11:22:48.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.268+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc67806ecc0 msgr2=0x7fc678071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.268+0000 7fc69ae00700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc67806ecc0 0x7fc678071180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6840097b0 tx=0x7fc684006d20 comp rx=0 tx=0).stop 2024-04-20T11:22:48.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.268+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 msgr2=0x7fc69419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.268+0000 7fc69ae00700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 0x7fc69419d7d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc6800092e0 tx=0x7fc68000be20 comp rx=0 tx=0).stop 2024-04-20T11:22:48.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.268+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 shutdown_connections 2024-04-20T11:22:48.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.268+0000 7fc69ae00700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc67806ecc0 0x7fc678071180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.269+0000 7fc69ae00700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc694107440 0x7fc69419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.269+0000 7fc69ae00700 1 --2- 172.21.15.117:0/193179631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc694107df0 0x7fc69419d7d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.269+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 >> 172.21.15.117:0/193179631 conn(0x7fc694076a10 msgr2=0x7fc694077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:48.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.269+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 shutdown_connections 2024-04-20T11:22:48.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.269+0000 7fc69ae00700 1 -- 172.21.15.117:0/193179631 wait complete. 2024-04-20T11:22:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.422+0000 7fae05000700 1 -- 172.21.15.117:0/3892117950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00108e10 msgr2=0x7fae00109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.422+0000 7fae05000700 1 --2- 172.21.15.117:0/3892117950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00108e10 0x7fae00109290 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fadec009910 tx=0x7fadec009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:48.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.422+0000 7fae05000700 1 -- 172.21.15.117:0/3892117950 shutdown_connections 2024-04-20T11:22:48.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.422+0000 7fae05000700 1 --2- 172.21.15.117:0/3892117950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00108e10 0x7fae00109290 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.422+0000 7fae05000700 1 --2- 172.21.15.117:0/3892117950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae00107520 0x7fae00107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.422+0000 7fae05000700 1 -- 172.21.15.117:0/3892117950 >> 172.21.15.117:0/3892117950 conn(0x7fae000768f0 msgr2=0x7fae00078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:48.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.422+0000 7fae05000700 1 -- 172.21.15.117:0/3892117950 shutdown_connections 2024-04-20T11:22:48.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.422+0000 7fae05000700 1 -- 172.21.15.117:0/3892117950 wait complete. 2024-04-20T11:22:48.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.423+0000 7fae05000700 1 Processor -- start 2024-04-20T11:22:48.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.423+0000 7fae05000700 1 -- start start 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.424+0000 7fae05000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00107520 0x7fae0019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.424+0000 7fae05000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae00108e10 0x7fae0019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.424+0000 7fae05000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0019ddf0 con 0x7fae00107520 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.424+0000 7fae05000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae001a1a70 con 0x7fae00108e10 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.424+0000 7fadfde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00107520 0x7fae0019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.424+0000 7fadfde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00107520 0x7fae0019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56808/0 (socket says 172.21.15.117:56808) 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.424+0000 7fadfde00700 1 -- 172.21.15.117:0/1306531063 learned_addr learned my addr 172.21.15.117:0/1306531063 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.424+0000 7fadfd400700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae00108e10 0x7fae0019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fadfde00700 1 -- 172.21.15.117:0/1306531063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae00108e10 msgr2=0x7fae0019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fadfde00700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae00108e10 0x7fae0019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fadfde00700 1 -- 172.21.15.117:0/1306531063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec0095f0 con 0x7fae00107520 2024-04-20T11:22:48.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fadfde00700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00107520 0x7fae0019d1d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c8a0 tx=0x7fadf000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:48.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fadf6a00700 1 -- 172.21.15.117:0/1306531063 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf000cea0 con 0x7fae00107520 2024-04-20T11:22:48.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae001a1d50 con 0x7fae00107520 2024-04-20T11:22:48.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fadf6a00700 1 -- 172.21.15.117:0/1306531063 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadf0004830 con 0x7fae00107520 2024-04-20T11:22:48.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fadf6a00700 1 -- 172.21.15.117:0/1306531063 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf00055b0 con 0x7fae00107520 2024-04-20T11:22:48.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.425+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae001a2320 con 0x7fae00107520 2024-04-20T11:22:48.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.426+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0010f870 con 0x7fae00107520 2024-04-20T11:22:48.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.428+0000 7fadf6a00700 1 -- 172.21.15.117:0/1306531063 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fadf00049a0 con 0x7fae00107520 2024-04-20T11:22:48.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.429+0000 7fadf6a00700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fade406ecd0 0x7fade4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.429+0000 7fadf6a00700 1 -- 172.21.15.117:0/1306531063 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fadf008eac0 con 0x7fae00107520 2024-04-20T11:22:48.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.429+0000 7fadfd400700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fade406ecd0 0x7fade4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:48.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.430+0000 7fadfd400700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fade406ecd0 0x7fade4071190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fadec006010 tx=0x7fadec00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:48.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.433+0000 7fadf6a00700 1 -- 172.21.15.117:0/1306531063 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fadf0057880 con 0x7fae00107520 2024-04-20T11:22:48.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:48 smithi117 ceph-mon[99036]: pgmap v310: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:48.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.660+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae0019e500 con 0x7fade406ecd0 2024-04-20T11:22:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:48 smithi173 ceph-mon[74165]: pgmap v310: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:48.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.691+0000 7fadf6a00700 1 -- 172.21.15.117:0/1306531063 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fae0019e500 con 0x7fade406ecd0 2024-04-20T11:22:48.692 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:22:48.692 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (11m) 2m ago 25m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:22:48.692 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (25m) 2m ago 25m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:22:48.692 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (24m) 2m ago 24m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:22:48.692 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (7m) 2m ago 25m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:22:48.693 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (7m) 2m ago 24m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:22:48.693 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (10m) 2m ago 24m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:22:48.693 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (3m) 2m ago 18m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:22:48.693 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (3m) 2m ago 18m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:22:48.693 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (17m) 2m ago 17m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:22:48.693 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (2m) 2m ago 18m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:22:48.694 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (14m) 2m ago 27m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:22:48.694 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (12m) 2m ago 24m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:22:48.694 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (9m) 2m ago 27m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:22:48.694 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (8m) 2m ago 23m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:22:48.694 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (12m) 2m ago 25m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:22:48.695 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (12m) 2m ago 24m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:22:48.695 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (6m) 2m ago 22m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:22:48.695 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (6m) 2m ago 22m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:22:48.695 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (5m) 2m ago 21m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:22:48.695 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (5m) 2m ago 21m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:22:48.695 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (4m) 2m ago 20m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:22:48.695 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (3m) 2m ago 19m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:22:48.696 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (11m) 2m ago 24m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:22:48.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fade406ecd0 msgr2=0x7fade4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fade406ecd0 0x7fade4071190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fadec006010 tx=0x7fadec00b540 comp rx=0 tx=0).stop 2024-04-20T11:22:48.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00107520 msgr2=0x7fae0019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00107520 0x7fae0019d1d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c8a0 tx=0x7fadf000cc60 comp rx=0 tx=0).stop 2024-04-20T11:22:48.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 shutdown_connections 2024-04-20T11:22:48.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fade406ecd0 0x7fade4071190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae00107520 0x7fae0019d1d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 --2- 172.21.15.117:0/1306531063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae00108e10 0x7fae0019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 >> 172.21.15.117:0/1306531063 conn(0x7fae000768f0 msgr2=0x7fae00077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.694+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 shutdown_connections 2024-04-20T11:22:48.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.695+0000 7fae05000700 1 -- 172.21.15.117:0/1306531063 wait complete. 2024-04-20T11:22:48.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.834+0000 7f2bf5400700 1 -- 172.21.15.117:0/2050406983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 msgr2=0x7f2bf01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.834+0000 7f2bf5400700 1 --2- 172.21.15.117:0/2050406983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 0x7f2bf01034f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009910 tx=0x7f2be0009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:48.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.835+0000 7f2bf5400700 1 -- 172.21.15.117:0/2050406983 shutdown_connections 2024-04-20T11:22:48.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.835+0000 7f2bf5400700 1 --2- 172.21.15.117:0/2050406983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bf0104a00 0x7f2bf0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.835+0000 7f2bf5400700 1 --2- 172.21.15.117:0/2050406983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 0x7f2bf01034f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.835+0000 7f2bf5400700 1 -- 172.21.15.117:0/2050406983 >> 172.21.15.117:0/2050406983 conn(0x7f2bf00fe880 msgr2=0x7f2bf0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:48.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.835+0000 7f2bf5400700 1 -- 172.21.15.117:0/2050406983 shutdown_connections 2024-04-20T11:22:48.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.835+0000 7f2bf5400700 1 -- 172.21.15.117:0/2050406983 wait complete. 2024-04-20T11:22:48.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.836+0000 7f2bf5400700 1 Processor -- start 2024-04-20T11:22:48.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.836+0000 7f2bf5400700 1 -- start start 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bf5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 0x7f2bf0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bf5400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bf0104a00 0x7f2bf01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bf5400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf0199a80 con 0x7f2bf0103110 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bf5400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf019d750 con 0x7f2bf0104a00 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bee800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 0x7f2bf0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bee800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 0x7f2bf0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56830/0 (socket says 172.21.15.117:56830) 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bee800700 1 -- 172.21.15.117:0/3097071946 learned_addr learned my addr 172.21.15.117:0/3097071946 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2be5e00700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bf0104a00 0x7f2bf01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bee800700 1 -- 172.21.15.117:0/3097071946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bf0104a00 msgr2=0x7f2bf01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bee800700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bf0104a00 0x7f2bf01993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:48.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.837+0000 7f2bee800700 1 -- 172.21.15.117:0/3097071946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be00095f0 con 0x7f2bf0103110 2024-04-20T11:22:48.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.838+0000 7f2bee800700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 0x7f2bf0198e60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006010 tx=0x7f2be000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:48.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.838+0000 7f2be7e00700 1 -- 172.21.15.117:0/3097071946 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2be001d070 con 0x7f2bf0103110 2024-04-20T11:22:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.838+0000 7f2be7e00700 1 -- 172.21.15.117:0/3097071946 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2be000f460 con 0x7f2bf0103110 2024-04-20T11:22:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.838+0000 7f2be7e00700 1 -- 172.21.15.117:0/3097071946 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2be0017630 con 0x7f2bf0103110 2024-04-20T11:22:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.838+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf019d9d0 con 0x7f2bf0103110 2024-04-20T11:22:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.838+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf019def0 con 0x7f2bf0103110 2024-04-20T11:22:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.839+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf010b480 con 0x7f2bf0103110 2024-04-20T11:22:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.840+0000 7f2be7e00700 1 -- 172.21.15.117:0/3097071946 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2be000f5d0 con 0x7f2bf0103110 2024-04-20T11:22:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.841+0000 7f2be7e00700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2bd406ecc0 0x7f2bd4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.841+0000 7f2be7e00700 1 -- 172.21.15.117:0/3097071946 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2be00903a0 con 0x7f2bf0103110 2024-04-20T11:22:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.842+0000 7f2be5e00700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2bd406ecc0 0x7f2bd4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.842+0000 7f2be5e00700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2bd406ecc0 0x7f2bd4071180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8006fd0 tx=0x7f2bd8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:48.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:48.847+0000 7f2be7e00700 1 -- 172.21.15.117:0/3097071946 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2be00591a0 con 0x7f2bf0103110 2024-04-20T11:22:49.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.166+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2bf004ebe0 con 0x7f2bf0103110 2024-04-20T11:22:49.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.167+0000 7f2be7e00700 1 -- 172.21.15.117:0/3097071946 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f2be005c860 con 0x7f2bf0103110 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:22:49.167 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:22:49.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:22:49.168 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:22:49.168 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:22:49.168 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:22:49.168 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:22:49.168 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:22:49.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.169+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2bd406ecc0 msgr2=0x7f2bd4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.169+0000 7f2bf5400700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2bd406ecc0 0x7f2bd4071180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8006fd0 tx=0x7f2bd8008040 comp rx=0 tx=0).stop 2024-04-20T11:22:49.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.169+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 msgr2=0x7f2bf0198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.169+0000 7f2bf5400700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 0x7f2bf0198e60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006010 tx=0x7f2be000be20 comp rx=0 tx=0).stop 2024-04-20T11:22:49.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.170+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 shutdown_connections 2024-04-20T11:22:49.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.170+0000 7f2bf5400700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2bd406ecc0 0x7f2bd4071180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.170+0000 7f2bf5400700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0103110 0x7f2bf0198e60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.170+0000 7f2bf5400700 1 --2- 172.21.15.117:0/3097071946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bf0104a00 0x7f2bf01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.170+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 >> 172.21.15.117:0/3097071946 conn(0x7f2bf00fe880 msgr2=0x7f2bf00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:49.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.170+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 shutdown_connections 2024-04-20T11:22:49.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.170+0000 7f2bf5400700 1 -- 172.21.15.117:0/3097071946 wait complete. 2024-04-20T11:22:49.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.325+0000 7f6334a00700 1 -- 172.21.15.117:0/1147016923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 msgr2=0x7f633010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.325+0000 7f6334a00700 1 --2- 172.21.15.117:0/1147016923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 0x7f633010cf40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6318009910 tx=0x7f6318009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:49.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.325+0000 7f6334a00700 1 -- 172.21.15.117:0/1147016923 shutdown_connections 2024-04-20T11:22:49.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.325+0000 7f6334a00700 1 --2- 172.21.15.117:0/1147016923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 0x7f633010cf40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.325+0000 7f6334a00700 1 --2- 172.21.15.117:0/1147016923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63300ff8d0 0x7f63300ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.325+0000 7f6334a00700 1 -- 172.21.15.117:0/1147016923 >> 172.21.15.117:0/1147016923 conn(0x7f6330076750 msgr2=0x7f63300fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:49.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.326+0000 7f6334a00700 1 -- 172.21.15.117:0/1147016923 shutdown_connections 2024-04-20T11:22:49.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.326+0000 7f6334a00700 1 -- 172.21.15.117:0/1147016923 wait complete. 2024-04-20T11:22:49.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6334a00700 1 Processor -- start 2024-04-20T11:22:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6334a00700 1 -- start start 2024-04-20T11:22:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6334a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63300ff8d0 0x7f633019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6334a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 0x7f633019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6334a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633019f550 con 0x7f6330100280 2024-04-20T11:22:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6334a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330198b30 con 0x7f63300ff8d0 2024-04-20T11:22:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6327e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 0x7f633019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6327e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 0x7f633019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56844/0 (socket says 172.21.15.117:56844) 2024-04-20T11:22:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.327+0000 7f6327e00700 1 -- 172.21.15.117:0/290153814 learned_addr learned my addr 172.21.15.117:0/290153814 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.328+0000 7f6327e00700 1 -- 172.21.15.117:0/290153814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63300ff8d0 msgr2=0x7f633019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.328+0000 7f6327e00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63300ff8d0 0x7f633019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.328+0000 7f6327e00700 1 -- 172.21.15.117:0/290153814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180095f0 con 0x7f6330100280 2024-04-20T11:22:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.328+0000 7f6327e00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 0x7f633019ef30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63180092e0 tx=0x7f63180056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.328+0000 7f6326a00700 1 -- 172.21.15.117:0/290153814 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631801d070 con 0x7f6330100280 2024-04-20T11:22:49.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.328+0000 7f6326a00700 1 -- 172.21.15.117:0/290153814 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f631800bed0 con 0x7f6330100280 2024-04-20T11:22:49.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.328+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6330198db0 con 0x7f6330100280 2024-04-20T11:22:49.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.328+0000 7f6326a00700 1 -- 172.21.15.117:0/290153814 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6318021410 con 0x7f6330100280 2024-04-20T11:22:49.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.329+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6330199300 con 0x7f6330100280 2024-04-20T11:22:49.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.330+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63301078c0 con 0x7f6330100280 2024-04-20T11:22:49.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.331+0000 7f6326a00700 1 -- 172.21.15.117:0/290153814 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f631800f460 con 0x7f6330100280 2024-04-20T11:22:49.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.332+0000 7f6326a00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f631406ecc0 0x7f6314071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:49.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.332+0000 7f6326a00700 1 -- 172.21.15.117:0/290153814 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6318090120 con 0x7f6330100280 2024-04-20T11:22:49.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.332+0000 7f632de00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f631406ecc0 0x7f6314071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:49.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.333+0000 7f632de00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f631406ecc0 0x7f6314071180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f632000a990 tx=0x7f6320005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:49.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.336+0000 7f6326a00700 1 -- 172.21.15.117:0/290153814 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f631805c990 con 0x7f6330100280 2024-04-20T11:22:49.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.600+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f633010ff50 con 0x7f6330100280 2024-04-20T11:22:49.600 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:49 smithi117 ceph-mon[99036]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:49.600 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:49 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3097071946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:22:49.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.601+0000 7f6326a00700 1 -- 172.21.15.117:0/290153814 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f631805c520 con 0x7f6330100280 2024-04-20T11:22:49.601 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:22:49.601 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:22:49.601 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:22:49.601 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:22:49.601 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:22:49.602 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:22:49.603 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:49.604 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f631406ecc0 msgr2=0x7f6314071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f631406ecc0 0x7f6314071180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f632000a990 tx=0x7f6320005c80 comp rx=0 tx=0).stop 2024-04-20T11:22:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 msgr2=0x7f633019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 0x7f633019ef30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63180092e0 tx=0x7f63180056a0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 shutdown_connections 2024-04-20T11:22:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f631406ecc0 0x7f6314071180 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63300ff8d0 0x7f633019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 --2- 172.21.15.117:0/290153814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6330100280 0x7f633019ef30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.604+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 >> 172.21.15.117:0/290153814 conn(0x7f6330076750 msgr2=0x7f63300fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.605+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 shutdown_connections 2024-04-20T11:22:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.605+0000 7f6334a00700 1 -- 172.21.15.117:0/290153814 wait complete. 2024-04-20T11:22:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:22:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:49 smithi173 ceph-mon[74165]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:49 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3097071946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:22:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.747+0000 7fe222200700 1 -- 172.21.15.117:0/4013228329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073280 msgr2=0x7fe21c073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.747+0000 7fe222200700 1 --2- 172.21.15.117:0/4013228329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073280 0x7fe21c073660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe210009910 tx=0x7fe210009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.747+0000 7fe222200700 1 -- 172.21.15.117:0/4013228329 shutdown_connections 2024-04-20T11:22:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.747+0000 7fe222200700 1 --2- 172.21.15.117:0/4013228329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe21c073ba0 0x7fe21c1114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.747+0000 7fe222200700 1 --2- 172.21.15.117:0/4013228329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073280 0x7fe21c073660 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.747+0000 7fe222200700 1 -- 172.21.15.117:0/4013228329 >> 172.21.15.117:0/4013228329 conn(0x7fe21c0fc500 msgr2=0x7fe21c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.747+0000 7fe222200700 1 -- 172.21.15.117:0/4013228329 shutdown_connections 2024-04-20T11:22:49.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.747+0000 7fe222200700 1 -- 172.21.15.117:0/4013228329 wait complete. 2024-04-20T11:22:49.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.749+0000 7fe222200700 1 Processor -- start 2024-04-20T11:22:49.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.749+0000 7fe222200700 1 -- start start 2024-04-20T11:22:49.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.749+0000 7fe222200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe21c073280 0x7fe21c19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:49.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.749+0000 7fe222200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073ba0 0x7fe21c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.749+0000 7fe222200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c19dc80 con 0x7fe21c073ba0 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.749+0000 7fe222200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1a1950 con 0x7fe21c073280 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.750+0000 7fe21be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073ba0 0x7fe21c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.750+0000 7fe21be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073ba0 0x7fe21c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56864/0 (socket says 172.21.15.117:56864) 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.750+0000 7fe21be00700 1 -- 172.21.15.117:0/3240587066 learned_addr learned my addr 172.21.15.117:0/3240587066 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.750+0000 7fe220e00700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe21c073280 0x7fe21c19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.750+0000 7fe21be00700 1 -- 172.21.15.117:0/3240587066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe21c073280 msgr2=0x7fe21c19d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.750+0000 7fe21be00700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe21c073280 0x7fe21c19d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.750+0000 7fe21be00700 1 -- 172.21.15.117:0/3240587066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100095f0 con 0x7fe21c073ba0 2024-04-20T11:22:49.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.750+0000 7fe21be00700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073ba0 0x7fe21c19d5a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe20800b4b0 tx=0x7fe20800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:49.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.751+0000 7fe219600700 1 -- 172.21.15.117:0/3240587066 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe208011800 con 0x7fe21c073ba0 2024-04-20T11:22:49.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.751+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c1a1c30 con 0x7fe21c073ba0 2024-04-20T11:22:49.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.751+0000 7fe219600700 1 -- 172.21.15.117:0/3240587066 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe208011e20 con 0x7fe21c073ba0 2024-04-20T11:22:49.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.751+0000 7fe219600700 1 -- 172.21.15.117:0/3240587066 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20800f2e0 con 0x7fe21c073ba0 2024-04-20T11:22:49.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.751+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c1a2180 con 0x7fe21c073ba0 2024-04-20T11:22:49.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.752+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21c0749c0 con 0x7fe21c073ba0 2024-04-20T11:22:49.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.753+0000 7fe219600700 1 -- 172.21.15.117:0/3240587066 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe208010380 con 0x7fe21c073ba0 2024-04-20T11:22:49.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.755+0000 7fe219600700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20406ecc0 0x7fe204071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:49.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.755+0000 7fe219600700 1 -- 172.21.15.117:0/3240587066 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe20808edd0 con 0x7fe21c073ba0 2024-04-20T11:22:49.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.755+0000 7fe220e00700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20406ecc0 0x7fe204071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:49.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.756+0000 7fe220e00700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20406ecc0 0x7fe204071180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe210006010 tx=0x7fe21000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:49.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.758+0000 7fe219600700 1 -- 172.21.15.117:0/3240587066 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe208057c10 con 0x7fe21c073ba0 2024-04-20T11:22:49.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.990+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe21c1141f0 con 0x7fe20406ecc0 2024-04-20T11:22:49.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.993+0000 7fe219600700 1 -- 172.21.15.117:0/3240587066 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe21c1141f0 con 0x7fe20406ecc0 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:22:49.994 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:22:49.995 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:22:49.995 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20406ecc0 msgr2=0x7fe204071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20406ecc0 0x7fe204071180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe210006010 tx=0x7fe21000b540 comp rx=0 tx=0).stop 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073ba0 msgr2=0x7fe21c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073ba0 0x7fe21c19d5a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe20800b4b0 tx=0x7fe20800b870 comp rx=0 tx=0).stop 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 shutdown_connections 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20406ecc0 0x7fe204071180 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe21c073280 0x7fe21c19d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 --2- 172.21.15.117:0/3240587066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21c073ba0 0x7fe21c19d5a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 >> 172.21.15.117:0/3240587066 conn(0x7fe21c0fc500 msgr2=0x7fe21c100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 shutdown_connections 2024-04-20T11:22:49.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:49.996+0000 7fe222200700 1 -- 172.21.15.117:0/3240587066 wait complete. 2024-04-20T11:22:50.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.151+0000 7fb42cc00700 1 -- 172.21.15.117:0/1878603454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428107520 msgr2=0x7fb428107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:50.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.151+0000 7fb42cc00700 1 --2- 172.21.15.117:0/1878603454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428107520 0x7fb428107900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb418009910 tx=0x7fb418009c20 comp rx=0 tx=0).stop 2024-04-20T11:22:50.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.151+0000 7fb42cc00700 1 -- 172.21.15.117:0/1878603454 shutdown_connections 2024-04-20T11:22:50.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.151+0000 7fb42cc00700 1 --2- 172.21.15.117:0/1878603454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb428108e10 0x7fb428109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:50.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.151+0000 7fb42cc00700 1 --2- 172.21.15.117:0/1878603454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428107520 0x7fb428107900 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:50.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.151+0000 7fb42cc00700 1 -- 172.21.15.117:0/1878603454 >> 172.21.15.117:0/1878603454 conn(0x7fb4280768f0 msgr2=0x7fb428078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:50.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.151+0000 7fb42cc00700 1 -- 172.21.15.117:0/1878603454 shutdown_connections 2024-04-20T11:22:50.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.151+0000 7fb42cc00700 1 -- 172.21.15.117:0/1878603454 wait complete. 2024-04-20T11:22:50.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.152+0000 7fb42cc00700 1 Processor -- start 2024-04-20T11:22:50.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.152+0000 7fb42cc00700 1 -- start start 2024-04-20T11:22:50.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.153+0000 7fb42cc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb428107520 0x7fb42819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.153+0000 7fb42cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428108e10 0x7fb42819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.153+0000 7fb42cc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42819de90 con 0x7fb428108e10 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.153+0000 7fb42cc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4281a1b60 con 0x7fb428107520 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.153+0000 7fb425400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428108e10 0x7fb42819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.153+0000 7fb425400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428108e10 0x7fb42819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56872/0 (socket says 172.21.15.117:56872) 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.153+0000 7fb425400700 1 -- 172.21.15.117:0/394998911 learned_addr learned my addr 172.21.15.117:0/394998911 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.153+0000 7fb425e00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb428107520 0x7fb42819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb425400700 1 -- 172.21.15.117:0/394998911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb428107520 msgr2=0x7fb42819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb425400700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb428107520 0x7fb42819d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb425400700 1 -- 172.21.15.117:0/394998911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4180095f0 con 0x7fb428108e10 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb425e00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb428107520 0x7fb42819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb425400700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428108e10 0x7fb42819d7b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb41400b4b0 tx=0x7fb41400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:50.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb41ea00700 1 -- 172.21.15.117:0/394998911 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb414011800 con 0x7fb428108e10 2024-04-20T11:22:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb41ea00700 1 -- 172.21.15.117:0/394998911 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb414011e20 con 0x7fb428108e10 2024-04-20T11:22:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb41ea00700 1 -- 172.21.15.117:0/394998911 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41400f2e0 con 0x7fb428108e10 2024-04-20T11:22:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4281a1e40 con 0x7fb428108e10 2024-04-20T11:22:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.154+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4281a2410 con 0x7fb428108e10 2024-04-20T11:22:50.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.156+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42810f870 con 0x7fb428108e10 2024-04-20T11:22:50.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.157+0000 7fb41ea00700 1 -- 172.21.15.117:0/394998911 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb414010380 con 0x7fb428108e10 2024-04-20T11:22:50.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.158+0000 7fb41ea00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb40c06ecd0 0x7fb40c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:50.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.158+0000 7fb41ea00700 1 -- 172.21.15.117:0/394998911 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb41408ea00 con 0x7fb428108e10 2024-04-20T11:22:50.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.158+0000 7fb425e00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb40c06ecd0 0x7fb40c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:50.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.159+0000 7fb425e00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb40c06ecd0 0x7fb40c071190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb418006010 tx=0x7fb41800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:50.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.163+0000 7fb41ea00700 1 -- 172.21.15.117:0/394998911 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb414057870 con 0x7fb428108e10 2024-04-20T11:22:50.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:50 smithi117 ceph-mon[99036]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:50 smithi117 ceph-mon[99036]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:50 smithi117 ceph-mon[99036]: pgmap v311: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:50 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/290153814' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:22:50.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.478+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb428108920 con 0x7fb428108e10 2024-04-20T11:22:50.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.478+0000 7fb41ea00700 1 -- 172.21.15.117:0/394998911 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fb41405af30 con 0x7fb428108e10 2024-04-20T11:22:50.479 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:22:50.479 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:22:50.479 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:22:50.479 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:22:50.479 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:22:50.479 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:22:50.480 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:22:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb40c06ecd0 msgr2=0x7fb40c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb40c06ecd0 0x7fb40c071190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb418006010 tx=0x7fb41800b540 comp rx=0 tx=0).stop 2024-04-20T11:22:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428108e10 msgr2=0x7fb42819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:50.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428108e10 0x7fb42819d7b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb41400b4b0 tx=0x7fb41400b870 comp rx=0 tx=0).stop 2024-04-20T11:22:50.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 shutdown_connections 2024-04-20T11:22:50.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb40c06ecd0 0x7fb40c071190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:50.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb428107520 0x7fb42819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:50.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 --2- 172.21.15.117:0/394998911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb428108e10 0x7fb42819d7b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:50.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 >> 172.21.15.117:0/394998911 conn(0x7fb4280768f0 msgr2=0x7fb428078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:50.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 shutdown_connections 2024-04-20T11:22:50.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:22:50.482+0000 7fb42cc00700 1 -- 172.21.15.117:0/394998911 wait complete. 2024-04-20T11:22:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:50 smithi173 ceph-mon[74165]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:50 smithi173 ceph-mon[74165]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:50 smithi173 ceph-mon[74165]: pgmap v311: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:50 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/290153814' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:22:51.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:51 smithi117 ceph-mon[99036]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:51 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/394998911' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:22:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:51 smithi173 ceph-mon[74165]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:51 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/394998911' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:22:52.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:52 smithi117 ceph-mon[99036]: pgmap v312: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:52 smithi173 ceph-mon[74165]: pgmap v312: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:53.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:53 smithi117 ceph-mon[99036]: pgmap v313: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:53 smithi173 ceph-mon[74165]: pgmap v313: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:56.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:56 smithi117 ceph-mon[99036]: pgmap v314: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:56 smithi173 ceph-mon[74165]: pgmap v314: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:58 smithi173 ceph-mon[74165]: pgmap v315: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:22:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:22:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:58 smithi117 ceph-mon[99036]: pgmap v315: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:22:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:22:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:00.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:00 smithi117 ceph-mon[99036]: pgmap v316: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:00 smithi173 ceph-mon[74165]: pgmap v316: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:02.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:02 smithi117 ceph-mon[99036]: pgmap v317: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:02.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:02 smithi173 ceph-mon[74165]: pgmap v317: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:04 smithi117 ceph-mon[99036]: pgmap v318: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:04 smithi173 ceph-mon[74165]: pgmap v318: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:06 smithi117 ceph-mon[99036]: pgmap v319: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:06 smithi173 ceph-mon[74165]: pgmap v319: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:08.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:08 smithi117 ceph-mon[99036]: pgmap v320: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:08 smithi173 ceph-mon[74165]: pgmap v320: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:09.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:09 smithi117 ceph-mon[99036]: pgmap v321: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:09 smithi173 ceph-mon[74165]: pgmap v321: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:12.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:12 smithi117 ceph-mon[99036]: pgmap v322: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:12 smithi173 ceph-mon[74165]: pgmap v322: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:13.604 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:14.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:14 smithi117 ceph-mon[99036]: pgmap v323: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:14 smithi173 ceph-mon[74165]: pgmap v323: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:16.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:16 smithi117 ceph-mon[99036]: pgmap v324: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:16 smithi173 ceph-mon[74165]: pgmap v324: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:18.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:18 smithi117 ceph-mon[99036]: pgmap v325: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:18 smithi173 ceph-mon[74165]: pgmap v325: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:20 smithi117 ceph-mon[99036]: pgmap v326: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.651+0000 7f624e800700 1 -- 172.21.15.117:0/2027453862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248104a00 msgr2=0x7f6248104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.651+0000 7f624e800700 1 --2- 172.21.15.117:0/2027453862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248104a00 0x7f6248104e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6230009910 tx=0x7f6230009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.651+0000 7f624e800700 1 -- 172.21.15.117:0/2027453862 shutdown_connections 2024-04-20T11:23:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.651+0000 7f624e800700 1 --2- 172.21.15.117:0/2027453862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248104a00 0x7f6248104e80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.651+0000 7f624e800700 1 --2- 172.21.15.117:0/2027453862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6248103110 0x7f62481034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.651+0000 7f624e800700 1 -- 172.21.15.117:0/2027453862 >> 172.21.15.117:0/2027453862 conn(0x7f62480fe880 msgr2=0x7f6248100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.651+0000 7f624e800700 1 -- 172.21.15.117:0/2027453862 shutdown_connections 2024-04-20T11:23:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.651+0000 7f624e800700 1 -- 172.21.15.117:0/2027453862 wait complete. 2024-04-20T11:23:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.652+0000 7f624e800700 1 Processor -- start 2024-04-20T11:23:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.652+0000 7f624e800700 1 -- start start 2024-04-20T11:23:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.652+0000 7f624e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248103110 0x7f624819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.652+0000 7f624e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6248104a00 0x7f624819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.652+0000 7f624e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624819f5b0 con 0x7f6248103110 2024-04-20T11:23:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.652+0000 7f624e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6248198b90 con 0x7f6248104a00 2024-04-20T11:23:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.653+0000 7f6247400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248103110 0x7f624819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.653+0000 7f6247400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248103110 0x7f624819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60730/0 (socket says 172.21.15.117:60730) 2024-04-20T11:23:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.653+0000 7f6247400700 1 -- 172.21.15.117:0/2883720885 learned_addr learned my addr 172.21.15.117:0/2883720885 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.653+0000 7f623ea00700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6248104a00 0x7f624819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.653+0000 7f6247400700 1 -- 172.21.15.117:0/2883720885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6248104a00 msgr2=0x7f624819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.653+0000 7f6247400700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6248104a00 0x7f624819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.653+0000 7f6247400700 1 -- 172.21.15.117:0/2883720885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300095f0 con 0x7f6248103110 2024-04-20T11:23:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.654+0000 7f6247400700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248103110 0x7f624819ea50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f623800c8a0 tx=0x7f623800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.654+0000 7f6244c00700 1 -- 172.21.15.117:0/2883720885 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f623800cea0 con 0x7f6248103110 2024-04-20T11:23:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.654+0000 7f6244c00700 1 -- 172.21.15.117:0/2883720885 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6238004830 con 0x7f6248103110 2024-04-20T11:23:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.654+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6248198e70 con 0x7f6248103110 2024-04-20T11:23:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.654+0000 7f6244c00700 1 -- 172.21.15.117:0/2883720885 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62380055b0 con 0x7f6248103110 2024-04-20T11:23:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.654+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6248199440 con 0x7f6248103110 2024-04-20T11:23:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.655+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624810b480 con 0x7f6248103110 2024-04-20T11:23:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.656+0000 7f6244c00700 1 -- 172.21.15.117:0/2883720885 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f62380049a0 con 0x7f6248103110 2024-04-20T11:23:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.658+0000 7f6244c00700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f622c06ecd0 0x7f622c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.658+0000 7f6244c00700 1 -- 172.21.15.117:0/2883720885 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f623808ea20 con 0x7f6248103110 2024-04-20T11:23:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.658+0000 7f623ea00700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f622c06ecd0 0x7f622c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.659+0000 7f623ea00700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f622c06ecd0 0x7f622c071190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6230005f50 tx=0x7f623001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:20.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:20 smithi173 ceph-mon[74165]: pgmap v326: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:20.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.663+0000 7f6244c00700 1 -- 172.21.15.117:0/2883720885 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f623805b3b0 con 0x7f6248103110 2024-04-20T11:23:20.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.901+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6248199fe0 con 0x7f622c06ecd0 2024-04-20T11:23:20.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.904+0000 7f6244c00700 1 -- 172.21.15.117:0/2883720885 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6248199fe0 con 0x7f622c06ecd0 2024-04-20T11:23:20.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.906+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f622c06ecd0 msgr2=0x7f622c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:20.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.906+0000 7f624e800700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f622c06ecd0 0x7f622c071190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6230005f50 tx=0x7f623001a040 comp rx=0 tx=0).stop 2024-04-20T11:23:20.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.906+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248103110 msgr2=0x7f624819ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:20.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.906+0000 7f624e800700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248103110 0x7f624819ea50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f623800c8a0 tx=0x7f623800cc60 comp rx=0 tx=0).stop 2024-04-20T11:23:20.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.907+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 shutdown_connections 2024-04-20T11:23:20.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.907+0000 7f624e800700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f622c06ecd0 0x7f622c071190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:20.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.907+0000 7f624e800700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6248103110 0x7f624819ea50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:20.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.907+0000 7f624e800700 1 --2- 172.21.15.117:0/2883720885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6248104a00 0x7f624819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:20.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.907+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 >> 172.21.15.117:0/2883720885 conn(0x7f62480fe880 msgr2=0x7f62480ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:20.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.907+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 shutdown_connections 2024-04-20T11:23:20.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:20.907+0000 7f624e800700 1 -- 172.21.15.117:0/2883720885 wait complete. 2024-04-20T11:23:20.921 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:23:21.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.051+0000 7f81dfc00700 1 -- 172.21.15.117:0/1050347468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8107520 msgr2=0x7f81d8107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.051+0000 7f81dfc00700 1 --2- 172.21.15.117:0/1050347468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8107520 0x7f81d8107900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009910 tx=0x7f81cc009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:21.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.051+0000 7f81dfc00700 1 -- 172.21.15.117:0/1050347468 shutdown_connections 2024-04-20T11:23:21.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.051+0000 7f81dfc00700 1 --2- 172.21.15.117:0/1050347468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d8108e10 0x7f81d8109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.051+0000 7f81dfc00700 1 --2- 172.21.15.117:0/1050347468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8107520 0x7f81d8107900 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.051+0000 7f81dfc00700 1 -- 172.21.15.117:0/1050347468 >> 172.21.15.117:0/1050347468 conn(0x7f81d80768f0 msgr2=0x7f81d8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:21.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.051+0000 7f81dfc00700 1 -- 172.21.15.117:0/1050347468 shutdown_connections 2024-04-20T11:23:21.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.051+0000 7f81dfc00700 1 -- 172.21.15.117:0/1050347468 wait complete. 2024-04-20T11:23:21.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.052+0000 7f81dfc00700 1 Processor -- start 2024-04-20T11:23:21.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.052+0000 7f81dfc00700 1 -- start start 2024-04-20T11:23:21.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.053+0000 7f81dfc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d8107520 0x7f81d819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.053+0000 7f81dfc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8108e10 0x7f81d819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.053+0000 7f81dfc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d819ddd0 con 0x7f81d8108e10 2024-04-20T11:23:21.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.053+0000 7f81dfc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81a1aa0 con 0x7f81d8107520 2024-04-20T11:23:21.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.053+0000 7f81d7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8108e10 0x7f81d819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.053+0000 7f81d7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8108e10 0x7f81d819d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60738/0 (socket says 172.21.15.117:60738) 2024-04-20T11:23:21.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.053+0000 7f81d7e00700 1 -- 172.21.15.117:0/995716675 learned_addr learned my addr 172.21.15.117:0/995716675 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.053+0000 7f81dd200700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d8107520 0x7f81d819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81d7e00700 1 -- 172.21.15.117:0/995716675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d8107520 msgr2=0x7f81d819d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81d7e00700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d8107520 0x7f81d819d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81d7e00700 1 -- 172.21.15.117:0/995716675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0095f0 con 0x7f81d8108e10 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81dd200700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d8107520 0x7f81d819d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81d7e00700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8108e10 0x7f81d819d6f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81c400d8d0 tx=0x7f81c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81d5600700 1 -- 172.21.15.117:0/995716675 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81c4009940 con 0x7f81d8108e10 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81d5600700 1 -- 172.21.15.117:0/995716675 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81c4010460 con 0x7f81d8108e10 2024-04-20T11:23:21.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81d5600700 1 -- 172.21.15.117:0/995716675 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81c400f5f0 con 0x7f81d8108e10 2024-04-20T11:23:21.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d81a1d80 con 0x7f81d8108e10 2024-04-20T11:23:21.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.054+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d81a2350 con 0x7f81d8108e10 2024-04-20T11:23:21.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.055+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d810f870 con 0x7f81d8108e10 2024-04-20T11:23:21.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.056+0000 7f81d5600700 1 -- 172.21.15.117:0/995716675 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f81c40105d0 con 0x7f81d8108e10 2024-04-20T11:23:21.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.057+0000 7f81d5600700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f81c006ecd0 0x7f81c0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.057+0000 7f81d5600700 1 -- 172.21.15.117:0/995716675 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f81c408ede0 con 0x7f81d8108e10 2024-04-20T11:23:21.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.058+0000 7f81dd200700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f81c006ecd0 0x7f81c0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.058+0000 7f81dd200700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f81c006ecd0 0x7f81c0071190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009fd0 tx=0x7f81cc00b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:21.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.062+0000 7f81d5600700 1 -- 172.21.15.117:0/995716675 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f81c4057ba0 con 0x7f81d8108e10 2024-04-20T11:23:21.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.314+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81d819e530 con 0x7f81c006ecd0 2024-04-20T11:23:21.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.317+0000 7f81d5600700 1 -- 172.21.15.117:0/995716675 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f81d819e530 con 0x7f81c006ecd0 2024-04-20T11:23:21.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f81c006ecd0 msgr2=0x7f81c0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f81c006ecd0 0x7f81c0071190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009fd0 tx=0x7f81cc00b560 comp rx=0 tx=0).stop 2024-04-20T11:23:21.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8108e10 msgr2=0x7f81d819d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8108e10 0x7f81d819d6f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81c400d8d0 tx=0x7f81c400dc90 comp rx=0 tx=0).stop 2024-04-20T11:23:21.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 shutdown_connections 2024-04-20T11:23:21.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f81c006ecd0 0x7f81c0071190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d8107520 0x7f81d819d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 --2- 172.21.15.117:0/995716675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d8108e10 0x7f81d819d6f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.319+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 >> 172.21.15.117:0/995716675 conn(0x7f81d80768f0 msgr2=0x7f81d8077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:21.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.320+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 shutdown_connections 2024-04-20T11:23:21.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.320+0000 7f81dfc00700 1 -- 172.21.15.117:0/995716675 wait complete. 2024-04-20T11:23:21.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.465+0000 7f372a800700 1 -- 172.21.15.117:0/3791345027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724103040 msgr2=0x7f3724103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.465+0000 7f372a800700 1 --2- 172.21.15.117:0/3791345027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724103040 0x7f3724103420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3714009910 tx=0x7f3714009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.465+0000 7f372a800700 1 -- 172.21.15.117:0/3791345027 shutdown_connections 2024-04-20T11:23:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.465+0000 7f372a800700 1 --2- 172.21.15.117:0/3791345027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37241039f0 0x7f3724107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.465+0000 7f372a800700 1 --2- 172.21.15.117:0/3791345027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3724103040 0x7f3724103420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.465+0000 7f372a800700 1 -- 172.21.15.117:0/3791345027 >> 172.21.15.117:0/3791345027 conn(0x7f37240fe8d0 msgr2=0x7f3724100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.466+0000 7f372a800700 1 -- 172.21.15.117:0/3791345027 shutdown_connections 2024-04-20T11:23:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.466+0000 7f372a800700 1 -- 172.21.15.117:0/3791345027 wait complete. 2024-04-20T11:23:21.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f372a800700 1 Processor -- start 2024-04-20T11:23:21.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f372a800700 1 -- start start 2024-04-20T11:23:21.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f372a800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3724103040 0x7f3724198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f372a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37241039f0 0x7f37241992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f372a800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37241999a0 con 0x7f37241039f0 2024-04-20T11:23:21.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f372a800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372419d670 con 0x7f3724103040 2024-04-20T11:23:21.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f3722a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37241039f0 0x7f37241992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f3722a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37241039f0 0x7f37241992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60762/0 (socket says 172.21.15.117:60762) 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.467+0000 7f3722a00700 1 -- 172.21.15.117:0/2507533565 learned_addr learned my addr 172.21.15.117:0/2507533565 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f3723400700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3724103040 0x7f3724198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f3722a00700 1 -- 172.21.15.117:0/2507533565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3724103040 msgr2=0x7f3724198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f3722a00700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3724103040 0x7f3724198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f3722a00700 1 -- 172.21.15.117:0/2507533565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140095f0 con 0x7f37241039f0 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f3723400700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3724103040 0x7f3724198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f3722a00700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37241039f0 0x7f37241992c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f371000c8f0 tx=0x7f371000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f371be00700 1 -- 172.21.15.117:0/2507533565 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3710007a50 con 0x7f37241039f0 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f371be00700 1 -- 172.21.15.117:0/2507533565 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3710007bb0 con 0x7f37241039f0 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f371be00700 1 -- 172.21.15.117:0/2507533565 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3710018630 con 0x7f37241039f0 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.468+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372419d950 con 0x7f37241039f0 2024-04-20T11:23:21.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.469+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372419dea0 con 0x7f37241039f0 2024-04-20T11:23:21.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.470+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372410b3e0 con 0x7f37241039f0 2024-04-20T11:23:21.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.471+0000 7f371be00700 1 -- 172.21.15.117:0/2507533565 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3710007d20 con 0x7f37241039f0 2024-04-20T11:23:21.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.472+0000 7f371be00700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f370806ecc0 0x7f3708071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.472+0000 7f371be00700 1 -- 172.21.15.117:0/2507533565 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f371008edd0 con 0x7f37241039f0 2024-04-20T11:23:21.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.472+0000 7f3723400700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f370806ecc0 0x7f3708071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.473+0000 7f3723400700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f370806ecc0 0x7f3708071180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3714005f50 tx=0x7f3714005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:21.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.477+0000 7f371be00700 1 -- 172.21.15.117:0/2507533565 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3710057b90 con 0x7f37241039f0 2024-04-20T11:23:21.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:21 smithi117 ceph-mon[99036]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:21 smithi117 ceph-mon[99036]: pgmap v327: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:21 smithi173 ceph-mon[74165]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:21 smithi173 ceph-mon[74165]: pgmap v327: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:21.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.697+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3724110010 con 0x7f370806ecc0 2024-04-20T11:23:21.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.710+0000 7f371be00700 1 -- 172.21.15.117:0/2507533565 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3724110010 con 0x7f370806ecc0 2024-04-20T11:23:21.710 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:23:21.710 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (11m) 3m ago 25m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:23:21.710 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (26m) 3m ago 26m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:23:21.710 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (24m) 3m ago 24m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (8m) 3m ago 26m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (7m) 3m ago 24m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (11m) 3m ago 25m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (3m) 3m ago 18m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (4m) 3m ago 18m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (18m) 3m ago 18m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (3m) 3m ago 18m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (14m) 3m ago 27m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (13m) 3m ago 24m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (9m) 3m ago 27m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (9m) 3m ago 24m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (13m) 3m ago 26m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:23:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (12m) 3m ago 24m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:23:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (7m) 3m ago 23m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:23:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (6m) 3m ago 22m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:23:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (6m) 3m ago 22m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:23:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (5m) 3m ago 21m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:23:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (5m) 3m ago 21m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:23:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (4m) 3m ago 20m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:23:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (12m) 3m ago 25m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:23:21.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f370806ecc0 msgr2=0x7f3708071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f370806ecc0 0x7f3708071180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3714005f50 tx=0x7f3714005dc0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37241039f0 msgr2=0x7f37241992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37241039f0 0x7f37241992c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f371000c8f0 tx=0x7f371000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 shutdown_connections 2024-04-20T11:23:21.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f370806ecc0 0x7f3708071180 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3724103040 0x7f3724198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 --2- 172.21.15.117:0/2507533565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37241039f0 0x7f37241992c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 >> 172.21.15.117:0/2507533565 conn(0x7f37240fe8d0 msgr2=0x7f37240fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:21.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 shutdown_connections 2024-04-20T11:23:21.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.713+0000 7f372a800700 1 -- 172.21.15.117:0/2507533565 wait complete. 2024-04-20T11:23:21.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.863+0000 7fdd96200700 1 -- 172.21.15.117:0/1194611399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90101d40 msgr2=0x7fdd90102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.863+0000 7fdd96200700 1 --2- 172.21.15.117:0/1194611399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90101d40 0x7fdd90102120 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000b0a0 tx=0x7fdd8000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.863+0000 7fdd96200700 1 -- 172.21.15.117:0/1194611399 shutdown_connections 2024-04-20T11:23:21.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.863+0000 7fdd96200700 1 --2- 172.21.15.117:0/1194611399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd90102660 0x7fdd9010ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.863+0000 7fdd96200700 1 --2- 172.21.15.117:0/1194611399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90101d40 0x7fdd90102120 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.863+0000 7fdd96200700 1 -- 172.21.15.117:0/1194611399 >> 172.21.15.117:0/1194611399 conn(0x7fdd90076750 msgr2=0x7fdd900fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:21.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.863+0000 7fdd96200700 1 -- 172.21.15.117:0/1194611399 shutdown_connections 2024-04-20T11:23:21.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.863+0000 7fdd96200700 1 -- 172.21.15.117:0/1194611399 wait complete. 2024-04-20T11:23:21.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.864+0000 7fdd96200700 1 Processor -- start 2024-04-20T11:23:21.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.864+0000 7fdd96200700 1 -- start start 2024-04-20T11:23:21.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.865+0000 7fdd96200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd90101d40 0x7fdd90198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.865+0000 7fdd96200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90102660 0x7fdd901992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.865+0000 7fdd96200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90199990 con 0x7fdd90102660 2024-04-20T11:23:21.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.865+0000 7fdd96200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9019d660 con 0x7fdd90101d40 2024-04-20T11:23:21.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.865+0000 7fdd87e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90102660 0x7fdd901992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.865+0000 7fdd87e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90102660 0x7fdd901992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60788/0 (socket says 172.21.15.117:60788) 2024-04-20T11:23:21.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.865+0000 7fdd87e00700 1 -- 172.21.15.117:0/3281448980 learned_addr learned my addr 172.21.15.117:0/3281448980 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd8f400700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd90101d40 0x7fdd90198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd87e00700 1 -- 172.21.15.117:0/3281448980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd90101d40 msgr2=0x7fdd90198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd87e00700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd90101d40 0x7fdd90198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd87e00700 1 -- 172.21.15.117:0/3281448980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80009d00 con 0x7fdd90102660 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd8f400700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd90101d40 0x7fdd90198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd87e00700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90102660 0x7fdd901992b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c8f0 tx=0x7fdd7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd8cc00700 1 -- 172.21.15.117:0/3281448980 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd78007ad0 con 0x7fdd90102660 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd8cc00700 1 -- 172.21.15.117:0/3281448980 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd7800ce90 con 0x7fdd90102660 2024-04-20T11:23:21.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9019d940 con 0x7fdd90102660 2024-04-20T11:23:21.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd8cc00700 1 -- 172.21.15.117:0/3281448980 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd78018630 con 0x7fdd90102660 2024-04-20T11:23:21.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.866+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9019de90 con 0x7fdd90102660 2024-04-20T11:23:21.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.868+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd90068e50 con 0x7fdd90102660 2024-04-20T11:23:21.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.869+0000 7fdd8cc00700 1 -- 172.21.15.117:0/3281448980 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdd78007c30 con 0x7fdd90102660 2024-04-20T11:23:21.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.870+0000 7fdd8cc00700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd7406ecc0 0x7fdd74071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:21.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.870+0000 7fdd8cc00700 1 -- 172.21.15.117:0/3281448980 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdd7808eeb0 con 0x7fdd90102660 2024-04-20T11:23:21.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.870+0000 7fdd8f400700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd7406ecc0 0x7fdd74071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:21.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.871+0000 7fdd8f400700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd7406ecc0 0x7fdd74071180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009480 tx=0x7fdd80006000 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:21.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:21.874+0000 7fdd8cc00700 1 -- 172.21.15.117:0/3281448980 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdd78057bf0 con 0x7fdd90102660 2024-04-20T11:23:22.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.188+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdd90062510 con 0x7fdd90102660 2024-04-20T11:23:22.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.189+0000 7fdd8cc00700 1 -- 172.21.15.117:0/3281448980 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fdd7805b2b0 con 0x7fdd90102660 2024-04-20T11:23:22.189 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:23:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:23:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:23:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:23:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:23:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:23:22.190 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd7406ecc0 msgr2=0x7fdd74071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd7406ecc0 0x7fdd74071180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009480 tx=0x7fdd80006000 comp rx=0 tx=0).stop 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90102660 msgr2=0x7fdd901992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90102660 0x7fdd901992b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c8f0 tx=0x7fdd7800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 shutdown_connections 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd7406ecc0 0x7fdd74071180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd90101d40 0x7fdd90198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 --2- 172.21.15.117:0/3281448980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd90102660 0x7fdd901992b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 >> 172.21.15.117:0/3281448980 conn(0x7fdd90076750 msgr2=0x7fdd9010a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:22.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 shutdown_connections 2024-04-20T11:23:22.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.192+0000 7fdd96200700 1 -- 172.21.15.117:0/3281448980 wait complete. 2024-04-20T11:23:22.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.337+0000 7f47b4e00700 1 -- 172.21.15.117:0/2592204317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 msgr2=0x7f47b01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:22.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.337+0000 7f47b4e00700 1 --2- 172.21.15.117:0/2592204317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 0x7f47b01034f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009910 tx=0x7f47a0009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:22.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.337+0000 7f47b4e00700 1 -- 172.21.15.117:0/2592204317 shutdown_connections 2024-04-20T11:23:22.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.337+0000 7f47b4e00700 1 --2- 172.21.15.117:0/2592204317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47b0104a00 0x7f47b0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.338+0000 7f47b4e00700 1 --2- 172.21.15.117:0/2592204317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 0x7f47b01034f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.338+0000 7f47b4e00700 1 -- 172.21.15.117:0/2592204317 >> 172.21.15.117:0/2592204317 conn(0x7f47b00fe880 msgr2=0x7f47b0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:22.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.338+0000 7f47b4e00700 1 -- 172.21.15.117:0/2592204317 shutdown_connections 2024-04-20T11:23:22.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.338+0000 7f47b4e00700 1 -- 172.21.15.117:0/2592204317 wait complete. 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.339+0000 7f47b4e00700 1 Processor -- start 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.339+0000 7f47b4e00700 1 -- start start 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.339+0000 7f47b4e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 0x7f47b0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47b4e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47b0104a00 0x7f47b01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47b4e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0199a80 con 0x7f47b0103110 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47b4e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b019d750 con 0x7f47b0104a00 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47ade00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 0x7f47b0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47ade00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 0x7f47b0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60798/0 (socket says 172.21.15.117:60798) 2024-04-20T11:23:22.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47ade00700 1 -- 172.21.15.117:0/1354912724 learned_addr learned my addr 172.21.15.117:0/1354912724 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:22.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47ade00700 1 -- 172.21.15.117:0/1354912724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47b0104a00 msgr2=0x7f47b01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:22.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47ade00700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47b0104a00 0x7f47b01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.340+0000 7f47ade00700 1 -- 172.21.15.117:0/1354912724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a00095f0 con 0x7f47b0103110 2024-04-20T11:23:22.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.341+0000 7f47ade00700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 0x7f47b0198e60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f47a00092e0 tx=0x7f47a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:22.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.341+0000 7f47a7400700 1 -- 172.21.15.117:0/1354912724 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a001d070 con 0x7f47b0103110 2024-04-20T11:23:22.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.341+0000 7f47a7400700 1 -- 172.21.15.117:0/1354912724 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47a000f460 con 0x7f47b0103110 2024-04-20T11:23:22.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.341+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b019d9d0 con 0x7f47b0103110 2024-04-20T11:23:22.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.341+0000 7f47a7400700 1 -- 172.21.15.117:0/1354912724 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a0017630 con 0x7f47b0103110 2024-04-20T11:23:22.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.341+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b019df40 con 0x7f47b0103110 2024-04-20T11:23:22.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.342+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b010b480 con 0x7f47b0103110 2024-04-20T11:23:22.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.343+0000 7f47a7400700 1 -- 172.21.15.117:0/1354912724 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f47a000f5d0 con 0x7f47b0103110 2024-04-20T11:23:22.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.344+0000 7f47a7400700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f479406ecd0 0x7f4794071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:22.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.344+0000 7f47a7400700 1 -- 172.21.15.117:0/1354912724 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f47a00903e0 con 0x7f47b0103110 2024-04-20T11:23:22.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.344+0000 7f47a5400700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f479406ecd0 0x7f4794071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:22.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.345+0000 7f47a5400700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f479406ecd0 0x7f4794071190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f47b019a3c0 tx=0x7f4798006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:22.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.349+0000 7f47a7400700 1 -- 172.21.15.117:0/1354912724 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f47a0059120 con 0x7f47b0103110 2024-04-20T11:23:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.618+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f47b019a170 con 0x7f47b0103110 2024-04-20T11:23:22.619 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:22 smithi117 ceph-mon[99036]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:22.619 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:22 smithi117 ceph-mon[99036]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:22.619 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:22 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3281448980' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:23:22.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.619+0000 7f47a7400700 1 -- 172.21.15.117:0/1354912724 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f47a005c7e0 con 0x7f47b0103110 2024-04-20T11:23:22.620 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:23:22.620 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:23:22.620 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:23:22.620 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:23:22.620 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:23:22.620 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:23:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:23:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:23:22.623 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:23:22.623 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:23:22.623 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:23:22.623 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:23:22.623 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:23:22.623 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:23:22.623 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:23:22.623 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f479406ecd0 msgr2=0x7f4794071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f479406ecd0 0x7f4794071190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f47b019a3c0 tx=0x7f4798006d20 comp rx=0 tx=0).stop 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 msgr2=0x7f47b0198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 0x7f47b0198e60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f47a00092e0 tx=0x7f47a000be20 comp rx=0 tx=0).stop 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 shutdown_connections 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f479406ecd0 0x7f4794071190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b0103110 0x7f47b0198e60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 --2- 172.21.15.117:0/1354912724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47b0104a00 0x7f47b01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.623+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 >> 172.21.15.117:0/1354912724 conn(0x7f47b00fe880 msgr2=0x7f47b00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.624+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 shutdown_connections 2024-04-20T11:23:22.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.624+0000 7f47b4e00700 1 -- 172.21.15.117:0/1354912724 wait complete. 2024-04-20T11:23:22.625 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:23:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:22 smithi173 ceph-mon[74165]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:22 smithi173 ceph-mon[74165]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:22 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3281448980' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:23:22.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.771+0000 7fc646200700 1 -- 172.21.15.117:0/1949945876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc640068570 msgr2=0x7fc6400689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:22.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.771+0000 7fc646200700 1 --2- 172.21.15.117:0/1949945876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc640068570 0x7fc6400689d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009910 tx=0x7fc62c009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:22.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.771+0000 7fc646200700 1 -- 172.21.15.117:0/1949945876 shutdown_connections 2024-04-20T11:23:22.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.771+0000 7fc646200700 1 --2- 172.21.15.117:0/1949945876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc640068570 0x7fc6400689d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.771+0000 7fc646200700 1 --2- 172.21.15.117:0/1949945876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6401069d0 0x7fc640106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.771+0000 7fc646200700 1 -- 172.21.15.117:0/1949945876 >> 172.21.15.117:0/1949945876 conn(0x7fc640075480 msgr2=0x7fc640075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:22.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.772+0000 7fc646200700 1 -- 172.21.15.117:0/1949945876 shutdown_connections 2024-04-20T11:23:22.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.772+0000 7fc646200700 1 -- 172.21.15.117:0/1949945876 wait complete. 2024-04-20T11:23:22.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.773+0000 7fc646200700 1 Processor -- start 2024-04-20T11:23:22.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.773+0000 7fc646200700 1 -- start start 2024-04-20T11:23:22.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.773+0000 7fc646200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc640068570 0x7fc64019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.773+0000 7fc646200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6401069d0 0x7fc64019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.773+0000 7fc646200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64019f360 con 0x7fc6401069d0 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.773+0000 7fc646200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc640198940 con 0x7fc640068570 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.774+0000 7fc63fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6401069d0 0x7fc64019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.774+0000 7fc63fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6401069d0 0x7fc64019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60822/0 (socket says 172.21.15.117:60822) 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.774+0000 7fc63fe00700 1 -- 172.21.15.117:0/1258850121 learned_addr learned my addr 172.21.15.117:0/1258850121 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.774+0000 7fc644e00700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc640068570 0x7fc64019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.774+0000 7fc63fe00700 1 -- 172.21.15.117:0/1258850121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc640068570 msgr2=0x7fc64019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.774+0000 7fc63fe00700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc640068570 0x7fc64019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.774+0000 7fc63fe00700 1 -- 172.21.15.117:0/1258850121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c0095f0 con 0x7fc6401069d0 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.774+0000 7fc644e00700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc640068570 0x7fc64019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:22.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.775+0000 7fc63fe00700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6401069d0 0x7fc64019ed40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc62c000c00 tx=0x7fc62c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:22.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.776+0000 7fc63d600700 1 -- 172.21.15.117:0/1258850121 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62c01d070 con 0x7fc6401069d0 2024-04-20T11:23:22.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.776+0000 7fc63d600700 1 -- 172.21.15.117:0/1258850121 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc62c0056e0 con 0x7fc6401069d0 2024-04-20T11:23:22.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.776+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc640198bc0 con 0x7fc6401069d0 2024-04-20T11:23:22.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.776+0000 7fc63d600700 1 -- 172.21.15.117:0/1258850121 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62c00fe80 con 0x7fc6401069d0 2024-04-20T11:23:22.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.776+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6401990b0 con 0x7fc6401069d0 2024-04-20T11:23:22.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.777+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64010af70 con 0x7fc6401069d0 2024-04-20T11:23:22.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.779+0000 7fc63d600700 1 -- 172.21.15.117:0/1258850121 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc62c00f6f0 con 0x7fc6401069d0 2024-04-20T11:23:22.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.780+0000 7fc63d600700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc62806ecc0 0x7fc628071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:22.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.780+0000 7fc63d600700 1 -- 172.21.15.117:0/1258850121 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc62c0901c0 con 0x7fc6401069d0 2024-04-20T11:23:22.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.781+0000 7fc644e00700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc62806ecc0 0x7fc628071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:22.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.781+0000 7fc644e00700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc62806ecc0 0x7fc628071180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc634005fd0 tx=0x7fc634005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:22.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:22.785+0000 7fc63d600700 1 -- 172.21.15.117:0/1258850121 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc62c05caf0 con 0x7fc6401069d0 2024-04-20T11:23:23.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.028+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc64010dbb0 con 0x7fc62806ecc0 2024-04-20T11:23:23.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.030+0000 7fc63d600700 1 -- 172.21.15.117:0/1258850121 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc64010dbb0 con 0x7fc62806ecc0 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:23:23.030 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:23:23.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.032+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc62806ecc0 msgr2=0x7fc628071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:23.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.032+0000 7fc646200700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc62806ecc0 0x7fc628071180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc634005fd0 tx=0x7fc634005ee0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.032+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6401069d0 msgr2=0x7fc64019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:23.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.032+0000 7fc646200700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6401069d0 0x7fc64019ed40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc62c000c00 tx=0x7fc62c00be20 comp rx=0 tx=0).stop 2024-04-20T11:23:23.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.033+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 shutdown_connections 2024-04-20T11:23:23.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.033+0000 7fc646200700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc62806ecc0 0x7fc628071180 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.033+0000 7fc646200700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc640068570 0x7fc64019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.033+0000 7fc646200700 1 --2- 172.21.15.117:0/1258850121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6401069d0 0x7fc64019ed40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.033+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 >> 172.21.15.117:0/1258850121 conn(0x7fc640075480 msgr2=0x7fc640104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:23.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.033+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 shutdown_connections 2024-04-20T11:23:23.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.033+0000 7fc646200700 1 -- 172.21.15.117:0/1258850121 wait complete. 2024-04-20T11:23:23.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.188+0000 7fe22b200700 1 -- 172.21.15.117:0/3410881931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224104a00 msgr2=0x7fe224104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:23.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.188+0000 7fe22b200700 1 --2- 172.21.15.117:0/3410881931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224104a00 0x7fe224104e80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe210009910 tx=0x7fe210009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:23.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.188+0000 7fe22b200700 1 -- 172.21.15.117:0/3410881931 shutdown_connections 2024-04-20T11:23:23.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.188+0000 7fe22b200700 1 --2- 172.21.15.117:0/3410881931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224104a00 0x7fe224104e80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.188+0000 7fe22b200700 1 --2- 172.21.15.117:0/3410881931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe224103110 0x7fe2241034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.188+0000 7fe22b200700 1 -- 172.21.15.117:0/3410881931 >> 172.21.15.117:0/3410881931 conn(0x7fe2240fe880 msgr2=0x7fe224100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:23.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.188+0000 7fe22b200700 1 -- 172.21.15.117:0/3410881931 shutdown_connections 2024-04-20T11:23:23.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.188+0000 7fe22b200700 1 -- 172.21.15.117:0/3410881931 wait complete. 2024-04-20T11:23:23.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.189+0000 7fe22b200700 1 Processor -- start 2024-04-20T11:23:23.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.189+0000 7fe22b200700 1 -- start start 2024-04-20T11:23:23.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe22b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224103110 0x7fe22419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:23.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe22b200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe224104a00 0x7fe22419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:23.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe22b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22419f5b0 con 0x7fe224103110 2024-04-20T11:23:23.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe22b200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe224198b90 con 0x7fe224104a00 2024-04-20T11:23:23.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe223e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224103110 0x7fe22419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:23.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe223e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224103110 0x7fe22419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60836/0 (socket says 172.21.15.117:60836) 2024-04-20T11:23:23.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe223e00700 1 -- 172.21.15.117:0/1951719391 learned_addr learned my addr 172.21.15.117:0/1951719391 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe223e00700 1 -- 172.21.15.117:0/1951719391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe224104a00 msgr2=0x7fe22419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.190+0000 7fe223400700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe224104a00 0x7fe22419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe223e00700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe224104a00 0x7fe22419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe223e00700 1 -- 172.21.15.117:0/1951719391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100095f0 con 0x7fe224103110 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe223400700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe224104a00 0x7fe22419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe223e00700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224103110 0x7fe22419ea50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c8a0 tx=0x7fe21400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe220c00700 1 -- 172.21.15.117:0/1951719391 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe21400cea0 con 0x7fe224103110 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe220c00700 1 -- 172.21.15.117:0/1951719391 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe214004830 con 0x7fe224103110 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe220c00700 1 -- 172.21.15.117:0/1951719391 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2140055b0 con 0x7fe224103110 2024-04-20T11:23:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe224198e70 con 0x7fe224103110 2024-04-20T11:23:23.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.191+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe224199440 con 0x7fe224103110 2024-04-20T11:23:23.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.193+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22410b480 con 0x7fe224103110 2024-04-20T11:23:23.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.194+0000 7fe220c00700 1 -- 172.21.15.117:0/1951719391 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe2140049a0 con 0x7fe224103110 2024-04-20T11:23:23.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.195+0000 7fe220c00700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20806ecd0 0x7fe208071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:23.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.195+0000 7fe220c00700 1 -- 172.21.15.117:0/1951719391 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe21408eac0 con 0x7fe224103110 2024-04-20T11:23:23.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.195+0000 7fe223400700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20806ecd0 0x7fe208071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:23.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.196+0000 7fe223400700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20806ecd0 0x7fe208071190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe22419a230 tx=0x7fe21000b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:23.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.199+0000 7fe220c00700 1 -- 172.21.15.117:0/1951719391 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe21405b3b0 con 0x7fe224103110 2024-04-20T11:23:23.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.513+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe22419a0a0 con 0x7fe224103110 2024-04-20T11:23:23.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.514+0000 7fe220c00700 1 -- 172.21.15.117:0/1951719391 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe21401d090 con 0x7fe224103110 2024-04-20T11:23:23.513 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:23:23.513 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:23:23.513 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:23:23.514 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:23:23.514 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:23:23.514 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:23:23.514 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:23:23.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20806ecd0 msgr2=0x7fe208071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:23.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20806ecd0 0x7fe208071190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe22419a230 tx=0x7fe21000b560 comp rx=0 tx=0).stop 2024-04-20T11:23:23.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224103110 msgr2=0x7fe22419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:23.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224103110 0x7fe22419ea50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c8a0 tx=0x7fe21400cc60 comp rx=0 tx=0).stop 2024-04-20T11:23:23.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 shutdown_connections 2024-04-20T11:23:23.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe20806ecd0 0x7fe208071190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe224103110 0x7fe22419ea50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 --2- 172.21.15.117:0/1951719391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe224104a00 0x7fe22419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:23.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 >> 172.21.15.117:0/1951719391 conn(0x7fe2240fe880 msgr2=0x7fe2240ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:23.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 shutdown_connections 2024-04-20T11:23:23.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:23.517+0000 7fe22b200700 1 -- 172.21.15.117:0/1951719391 wait complete. 2024-04-20T11:23:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:23 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1354912724' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:23:23.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:23 smithi117 ceph-mon[99036]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:23.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:23 smithi117 ceph-mon[99036]: pgmap v328: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:23 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1354912724' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:23:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:23 smithi173 ceph-mon[74165]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:23 smithi173 ceph-mon[74165]: pgmap v328: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:24.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:24 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1951719391' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:23:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:24 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1951719391' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:23:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:25 smithi117 ceph-mon[99036]: pgmap v329: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:25 smithi173 ceph-mon[74165]: pgmap v329: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:28 smithi117 ceph-mon[99036]: pgmap v330: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:28 smithi173 ceph-mon[74165]: pgmap v330: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:30.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:30 smithi117 ceph-mon[99036]: pgmap v331: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:30.667 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:30 smithi173 ceph-mon[74165]: pgmap v331: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:32 smithi117 ceph-mon[99036]: pgmap v332: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:32 smithi173 ceph-mon[74165]: pgmap v332: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:34 smithi117 ceph-mon[99036]: pgmap v333: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:34 smithi173 ceph-mon[74165]: pgmap v333: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:36 smithi117 ceph-mon[99036]: pgmap v334: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:36 smithi173 ceph-mon[74165]: pgmap v334: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:38.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:38 smithi117 ceph-mon[99036]: pgmap v335: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:38 smithi173 ceph-mon[74165]: pgmap v335: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:39 smithi173 ceph-mon[74165]: pgmap v336: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:39.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:39 smithi117 ceph-mon[99036]: pgmap v336: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:42 smithi117 ceph-mon[99036]: pgmap v337: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:42 smithi173 ceph-mon[74165]: pgmap v337: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:44.569 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:44 smithi173 ceph-mon[74165]: pgmap v338: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:44 smithi117 ceph-mon[99036]: pgmap v338: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:46.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:46 smithi117 ceph-mon[99036]: pgmap v339: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:46 smithi173 ceph-mon[74165]: pgmap v339: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:48.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:48 smithi117 ceph-mon[99036]: pgmap v340: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:48 smithi173 ceph-mon[74165]: pgmap v340: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:49.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:49 smithi117 ceph-mon[99036]: pgmap v341: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:49 smithi173 ceph-mon[74165]: pgmap v341: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:52.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:52 smithi117 ceph-mon[99036]: pgmap v342: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:52 smithi173 ceph-mon[74165]: pgmap v342: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:53.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.685+0000 7f3b40e00700 1 -- 172.21.15.117:0/2975951853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c1017c0 msgr2=0x7f3b3c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.685+0000 7f3b40e00700 1 --2- 172.21.15.117:0/2975951853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c1017c0 0x7f3b3c105810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009910 tx=0x7f3b28009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:53.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.685+0000 7f3b40e00700 1 -- 172.21.15.117:0/2975951853 shutdown_connections 2024-04-20T11:23:53.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.685+0000 7f3b40e00700 1 --2- 172.21.15.117:0/2975951853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c1017c0 0x7f3b3c105810 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.685+0000 7f3b40e00700 1 --2- 172.21.15.117:0/2975951853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c100e10 0x7f3b3c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.685+0000 7f3b40e00700 1 -- 172.21.15.117:0/2975951853 >> 172.21.15.117:0/2975951853 conn(0x7f3b3c0fc6c0 msgr2=0x7f3b3c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.685+0000 7f3b40e00700 1 -- 172.21.15.117:0/2975951853 shutdown_connections 2024-04-20T11:23:53.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.685+0000 7f3b40e00700 1 -- 172.21.15.117:0/2975951853 wait complete. 2024-04-20T11:23:53.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.687+0000 7f3b40e00700 1 Processor -- start 2024-04-20T11:23:53.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.687+0000 7f3b40e00700 1 -- start start 2024-04-20T11:23:53.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.688+0000 7f3b40e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c100e10 0x7f3b3c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.688+0000 7f3b40e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1017c0 0x7f3b3c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.688+0000 7f3b40e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c199b60 con 0x7f3b3c100e10 2024-04-20T11:23:53.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.688+0000 7f3b40e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c19d7e0 con 0x7f3b3c1017c0 2024-04-20T11:23:53.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.689+0000 7f3b39e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c100e10 0x7f3b3c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.689+0000 7f3b39e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c100e10 0x7f3b3c198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50922/0 (socket says 172.21.15.117:50922) 2024-04-20T11:23:53.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.689+0000 7f3b39e00700 1 -- 172.21.15.117:0/1428216773 learned_addr learned my addr 172.21.15.117:0/1428216773 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:53.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.689+0000 7f3b39400700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1017c0 0x7f3b3c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.689+0000 7f3b39e00700 1 -- 172.21.15.117:0/1428216773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1017c0 msgr2=0x7f3b3c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.689+0000 7f3b39e00700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1017c0 0x7f3b3c199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.689+0000 7f3b39e00700 1 -- 172.21.15.117:0/1428216773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280095f0 con 0x7f3b3c100e10 2024-04-20T11:23:53.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.690+0000 7f3b39e00700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c100e10 0x7f3b3c198f40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00b4b0 tx=0x7f3b2c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.690+0000 7f3b32a00700 1 -- 172.21.15.117:0/1428216773 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b2c011800 con 0x7f3b3c100e10 2024-04-20T11:23:53.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.690+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c19dac0 con 0x7f3b3c100e10 2024-04-20T11:23:53.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.690+0000 7f3b32a00700 1 -- 172.21.15.117:0/1428216773 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b2c011e20 con 0x7f3b3c100e10 2024-04-20T11:23:53.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.690+0000 7f3b32a00700 1 -- 172.21.15.117:0/1428216773 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b2c00f2e0 con 0x7f3b3c100e10 2024-04-20T11:23:53.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.690+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c19e010 con 0x7f3b3c100e10 2024-04-20T11:23:53.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.691+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c1022f0 con 0x7f3b3c100e10 2024-04-20T11:23:53.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.693+0000 7f3b32a00700 1 -- 172.21.15.117:0/1428216773 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3b2c010380 con 0x7f3b3c100e10 2024-04-20T11:23:53.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.694+0000 7f3b32a00700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3b2006ecc0 0x7f3b20071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.694+0000 7f3b32a00700 1 -- 172.21.15.117:0/1428216773 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3b2c08ea00 con 0x7f3b3c100e10 2024-04-20T11:23:53.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.694+0000 7f3b39400700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3b2006ecc0 0x7f3b20071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.695+0000 7f3b39400700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3b2006ecc0 0x7f3b20071180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c19a450 tx=0x7f3b2801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.700+0000 7f3b32a00700 1 -- 172.21.15.117:0/1428216773 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3b2c057870 con 0x7f3b3c100e10 2024-04-20T11:23:53.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.947+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b3c073430 con 0x7f3b2006ecc0 2024-04-20T11:23:53.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.951+0000 7f3b32a00700 1 -- 172.21.15.117:0/1428216773 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3b3c073430 con 0x7f3b2006ecc0 2024-04-20T11:23:53.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.953+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3b2006ecc0 msgr2=0x7f3b20071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.953+0000 7f3b40e00700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3b2006ecc0 0x7f3b20071180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c19a450 tx=0x7f3b2801a040 comp rx=0 tx=0).stop 2024-04-20T11:23:53.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.953+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c100e10 msgr2=0x7f3b3c198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.953+0000 7f3b40e00700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c100e10 0x7f3b3c198f40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00b4b0 tx=0x7f3b2c00b870 comp rx=0 tx=0).stop 2024-04-20T11:23:53.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.953+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 shutdown_connections 2024-04-20T11:23:53.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.953+0000 7f3b40e00700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3b2006ecc0 0x7f3b20071180 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.954+0000 7f3b40e00700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3c100e10 0x7f3b3c198f40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.954+0000 7f3b40e00700 1 --2- 172.21.15.117:0/1428216773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1017c0 0x7f3b3c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.954+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 >> 172.21.15.117:0/1428216773 conn(0x7f3b3c0fc6c0 msgr2=0x7f3b3c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.954+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 shutdown_connections 2024-04-20T11:23:53.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:53.954+0000 7f3b40e00700 1 -- 172.21.15.117:0/1428216773 wait complete. 2024-04-20T11:23:53.970 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:23:54.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.119+0000 7fd3f6000700 1 -- 172.21.15.117:0/347020373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073b20 msgr2=0x7fd3f010cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.119+0000 7fd3f6000700 1 --2- 172.21.15.117:0/347020373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073b20 0x7fd3f010cf30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009910 tx=0x7fd3dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:54.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.120+0000 7fd3f6000700 1 -- 172.21.15.117:0/347020373 shutdown_connections 2024-04-20T11:23:54.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.120+0000 7fd3f6000700 1 --2- 172.21.15.117:0/347020373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073b20 0x7fd3f010cf30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.120+0000 7fd3f6000700 1 --2- 172.21.15.117:0/347020373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f0073200 0x7fd3f00735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.120+0000 7fd3f6000700 1 -- 172.21.15.117:0/347020373 >> 172.21.15.117:0/347020373 conn(0x7fd3f00fc6a0 msgr2=0x7fd3f00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:54.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.120+0000 7fd3f6000700 1 -- 172.21.15.117:0/347020373 shutdown_connections 2024-04-20T11:23:54.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.120+0000 7fd3f6000700 1 -- 172.21.15.117:0/347020373 wait complete. 2024-04-20T11:23:54.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.121+0000 7fd3f6000700 1 Processor -- start 2024-04-20T11:23:54.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.121+0000 7fd3f6000700 1 -- start start 2024-04-20T11:23:54.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.121+0000 7fd3f6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073200 0x7fd3f0198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.121+0000 7fd3f6000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f0073b20 0x7fd3f0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.121+0000 7fd3f6000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0199a20 con 0x7fd3f0073200 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.121+0000 7fd3f6000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f019d6f0 con 0x7fd3f0073b20 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3ef400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073200 0x7fd3f0198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3ef400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073200 0x7fd3f0198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56698/0 (socket says 172.21.15.117:56698) 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3ef400700 1 -- 172.21.15.117:0/414147354 learned_addr learned my addr 172.21.15.117:0/414147354 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3eea00700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f0073b20 0x7fd3f0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3ef400700 1 -- 172.21.15.117:0/414147354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f0073b20 msgr2=0x7fd3f0199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3ef400700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f0073b20 0x7fd3f0199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3ef400700 1 -- 172.21.15.117:0/414147354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0095f0 con 0x7fd3f0073200 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3eea00700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f0073b20 0x7fd3f0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:54.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.122+0000 7fd3ef400700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073200 0x7fd3f0198e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000b4b0 tx=0x7fd3e000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:54.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.123+0000 7fd3e7e00700 1 -- 172.21.15.117:0/414147354 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3e0011800 con 0x7fd3f0073200 2024-04-20T11:23:54.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.123+0000 7fd3e7e00700 1 -- 172.21.15.117:0/414147354 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3e0011e20 con 0x7fd3f0073200 2024-04-20T11:23:54.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.123+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f019d9d0 con 0x7fd3f0073200 2024-04-20T11:23:54.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.123+0000 7fd3e7e00700 1 -- 172.21.15.117:0/414147354 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3e000f2e0 con 0x7fd3f0073200 2024-04-20T11:23:54.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.123+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f019df20 con 0x7fd3f0073200 2024-04-20T11:23:54.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.124+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f0074940 con 0x7fd3f0073200 2024-04-20T11:23:54.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.125+0000 7fd3e7e00700 1 -- 172.21.15.117:0/414147354 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd3e0011960 con 0x7fd3f0073200 2024-04-20T11:23:54.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.126+0000 7fd3e7e00700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd3d406ecd0 0x7fd3d4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.126+0000 7fd3e7e00700 1 -- 172.21.15.117:0/414147354 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd3e008e660 con 0x7fd3f0073200 2024-04-20T11:23:54.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.126+0000 7fd3eea00700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd3d406ecd0 0x7fd3d4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.127+0000 7fd3eea00700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd3d406ecd0 0x7fd3d4071190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc006010 tx=0x7fd3dc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:54.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.130+0000 7fd3e7e00700 1 -- 172.21.15.117:0/414147354 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd3e0057440 con 0x7fd3f0073200 2024-04-20T11:23:54.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.363+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3f0107840 con 0x7fd3d406ecd0 2024-04-20T11:23:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.367+0000 7fd3e7e00700 1 -- 172.21.15.117:0/414147354 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd3f0107840 con 0x7fd3d406ecd0 2024-04-20T11:23:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.369+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd3d406ecd0 msgr2=0x7fd3d4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.369+0000 7fd3f6000700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd3d406ecd0 0x7fd3d4071190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc006010 tx=0x7fd3dc00b540 comp rx=0 tx=0).stop 2024-04-20T11:23:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.369+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073200 msgr2=0x7fd3f0198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.369+0000 7fd3f6000700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073200 0x7fd3f0198e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000b4b0 tx=0x7fd3e000b870 comp rx=0 tx=0).stop 2024-04-20T11:23:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.370+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 shutdown_connections 2024-04-20T11:23:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.370+0000 7fd3f6000700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd3d406ecd0 0x7fd3d4071190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.370+0000 7fd3f6000700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f0073200 0x7fd3f0198e00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.370+0000 7fd3f6000700 1 --2- 172.21.15.117:0/414147354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f0073b20 0x7fd3f0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.370+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 >> 172.21.15.117:0/414147354 conn(0x7fd3f00fc6a0 msgr2=0x7fd3f0105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.370+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 shutdown_connections 2024-04-20T11:23:54.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.370+0000 7fd3f6000700 1 -- 172.21.15.117:0/414147354 wait complete. 2024-04-20T11:23:54.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.513+0000 7fa16d200700 1 -- 172.21.15.117:0/2880782809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 msgr2=0x7fa168100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.513+0000 7fa16d200700 1 --2- 172.21.15.117:0/2880782809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 0x7fa168100e30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa154009910 tx=0x7fa154009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:54.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.513+0000 7fa16d200700 1 -- 172.21.15.117:0/2880782809 shutdown_connections 2024-04-20T11:23:54.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.513+0000 7fa16d200700 1 --2- 172.21.15.117:0/2880782809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 0x7fa168100e30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.513+0000 7fa16d200700 1 --2- 172.21.15.117:0/2880782809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1681069b0 0x7fa168106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.513+0000 7fa16d200700 1 -- 172.21.15.117:0/2880782809 >> 172.21.15.117:0/2880782809 conn(0x7fa1680fc4e0 msgr2=0x7fa1680fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:54.515 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:54 smithi117 ceph-mon[99036]: pgmap v343: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.514+0000 7fa16d200700 1 -- 172.21.15.117:0/2880782809 shutdown_connections 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.514+0000 7fa16d200700 1 -- 172.21.15.117:0/2880782809 wait complete. 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa16d200700 1 Processor -- start 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa16d200700 1 -- start start 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa16d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 0x7fa16819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa16d200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1681069b0 0x7fa16819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa16d200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16819d170 con 0x7fa1681009b0 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa16d200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168196750 con 0x7fa1681069b0 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa167400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 0x7fa16819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa167400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 0x7fa16819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56726/0 (socket says 172.21.15.117:56726) 2024-04-20T11:23:54.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.515+0000 7fa167400700 1 -- 172.21.15.117:0/2650591896 learned_addr learned my addr 172.21.15.117:0/2650591896 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:54.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.516+0000 7fa166a00700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1681069b0 0x7fa16819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.516+0000 7fa167400700 1 -- 172.21.15.117:0/2650591896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1681069b0 msgr2=0x7fa16819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.516+0000 7fa167400700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1681069b0 0x7fa16819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.516+0000 7fa167400700 1 -- 172.21.15.117:0/2650591896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540095f0 con 0x7fa1681009b0 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.516+0000 7fa167400700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 0x7fa16819c610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa15800d8d0 tx=0x7fa15800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.517+0000 7fa15fe00700 1 -- 172.21.15.117:0/2650591896 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa158009940 con 0x7fa1681009b0 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.517+0000 7fa15fe00700 1 -- 172.21.15.117:0/2650591896 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa158010460 con 0x7fa1681009b0 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.517+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa168196a30 con 0x7fa1681009b0 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.517+0000 7fa15fe00700 1 -- 172.21.15.117:0/2650591896 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa15800f5f0 con 0x7fa1681009b0 2024-04-20T11:23:54.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.517+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa168196f80 con 0x7fa1681009b0 2024-04-20T11:23:54.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.518+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168108fd0 con 0x7fa1681009b0 2024-04-20T11:23:54.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.519+0000 7fa15fe00700 1 -- 172.21.15.117:0/2650591896 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa1580105d0 con 0x7fa1681009b0 2024-04-20T11:23:54.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.520+0000 7fa15fe00700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa14c06ecc0 0x7fa14c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.520+0000 7fa15fe00700 1 -- 172.21.15.117:0/2650591896 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa15808ede0 con 0x7fa1681009b0 2024-04-20T11:23:54.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.520+0000 7fa166a00700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa14c06ecc0 0x7fa14c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.521+0000 7fa166a00700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa14c06ecc0 0x7fa14c071180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa154006010 tx=0x7fa15400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:54.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.525+0000 7fa15fe00700 1 -- 172.21.15.117:0/2650591896 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa15805b6d0 con 0x7fa1681009b0 2024-04-20T11:23:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:54 smithi173 ceph-mon[74165]: pgmap v343: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:54.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.742+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa168197cb0 con 0x7fa14c06ecc0 2024-04-20T11:23:54.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.759+0000 7fa15fe00700 1 -- 172.21.15.117:0/2650591896 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa168197cb0 con 0x7fa14c06ecc0 2024-04-20T11:23:54.760 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:23:54.760 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (12m) 4m ago 26m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:23:54.760 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (26m) 4m ago 26m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:23:54.760 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (25m) 3m ago 25m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (8m) 4m ago 26m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (8m) 3m ago 25m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (11m) 4m ago 26m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (4m) 4m ago 19m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (4m) 4m ago 19m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (19m) 3m ago 19m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (3m) 3m ago 19m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (15m) 4m ago 28m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (14m) 3m ago 25m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (10m) 4m ago 28m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (9m) 3m ago 25m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (13m) 4m ago 26m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (13m) 3m ago 25m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (8m) 4m ago 24m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:23:54.761 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (7m) 4m ago 23m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:23:54.762 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (6m) 4m ago 22m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:23:54.762 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (6m) 3m ago 22m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:23:54.762 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (5m) 3m ago 21m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:23:54.762 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (5m) 3m ago 21m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:23:54.762 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (13m) 4m ago 25m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:23:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa14c06ecc0 msgr2=0x7fa14c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa14c06ecc0 0x7fa14c071180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa154006010 tx=0x7fa15400b540 comp rx=0 tx=0).stop 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 msgr2=0x7fa16819c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 0x7fa16819c610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa15800d8d0 tx=0x7fa15800dc90 comp rx=0 tx=0).stop 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 shutdown_connections 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa14c06ecc0 0x7fa14c071180 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1681009b0 0x7fa16819c610 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 --2- 172.21.15.117:0/2650591896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1681069b0 0x7fa16819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 >> 172.21.15.117:0/2650591896 conn(0x7fa1680fc4e0 msgr2=0x7fa1680fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 shutdown_connections 2024-04-20T11:23:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.762+0000 7fa16d200700 1 -- 172.21.15.117:0/2650591896 wait complete. 2024-04-20T11:23:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.918+0000 7fb22c000700 1 -- 172.21.15.117:0/2859956947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 msgr2=0x7fb2240757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.918+0000 7fb22c000700 1 --2- 172.21.15.117:0/2859956947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 0x7fb2240757f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb218009910 tx=0x7fb218009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.918+0000 7fb22c000700 1 -- 172.21.15.117:0/2859956947 shutdown_connections 2024-04-20T11:23:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.918+0000 7fb22c000700 1 --2- 172.21.15.117:0/2859956947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb224075d30 0x7fb224111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.918+0000 7fb22c000700 1 --2- 172.21.15.117:0/2859956947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 0x7fb2240757f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.918+0000 7fb22c000700 1 -- 172.21.15.117:0/2859956947 >> 172.21.15.117:0/2859956947 conn(0x7fb2240fe8d0 msgr2=0x7fb224100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.918+0000 7fb22c000700 1 -- 172.21.15.117:0/2859956947 shutdown_connections 2024-04-20T11:23:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.918+0000 7fb22c000700 1 -- 172.21.15.117:0/2859956947 wait complete. 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.919+0000 7fb22c000700 1 Processor -- start 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.919+0000 7fb22c000700 1 -- start start 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.920+0000 7fb22c000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 0x7fb22419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.920+0000 7fb22c000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb224075d30 0x7fb22419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.920+0000 7fb22c000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22419dde0 con 0x7fb224075410 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.920+0000 7fb22c000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241a1ab0 con 0x7fb224075d30 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.920+0000 7fb229600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 0x7fb22419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.920+0000 7fb229600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 0x7fb22419d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56744/0 (socket says 172.21.15.117:56744) 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.920+0000 7fb229600700 1 -- 172.21.15.117:0/766686010 learned_addr learned my addr 172.21.15.117:0/766686010 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.920+0000 7fb228c00700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb224075d30 0x7fb22419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.921+0000 7fb229600700 1 -- 172.21.15.117:0/766686010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb224075d30 msgr2=0x7fb22419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.921+0000 7fb229600700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb224075d30 0x7fb22419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.921+0000 7fb229600700 1 -- 172.21.15.117:0/766686010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180095f0 con 0x7fb224075410 2024-04-20T11:23:54.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.921+0000 7fb229600700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 0x7fb22419d1c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb2180092e0 tx=0x7fb2180056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:54.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.921+0000 7fb222000700 1 -- 172.21.15.117:0/766686010 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb21801d070 con 0x7fb224075410 2024-04-20T11:23:54.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.921+0000 7fb222000700 1 -- 172.21.15.117:0/766686010 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb21800bed0 con 0x7fb224075410 2024-04-20T11:23:54.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.921+0000 7fb222000700 1 -- 172.21.15.117:0/766686010 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb218021410 con 0x7fb224075410 2024-04-20T11:23:54.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.921+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2241a1d90 con 0x7fb224075410 2024-04-20T11:23:54.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.922+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2241a22e0 con 0x7fb224075410 2024-04-20T11:23:54.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.923+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb224076b50 con 0x7fb224075410 2024-04-20T11:23:54.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.924+0000 7fb222000700 1 -- 172.21.15.117:0/766686010 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb21800f460 con 0x7fb224075410 2024-04-20T11:23:54.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.924+0000 7fb222000700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb20c06ecd0 0x7fb20c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:54.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.924+0000 7fb222000700 1 -- 172.21.15.117:0/766686010 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb218090090 con 0x7fb224075410 2024-04-20T11:23:54.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.925+0000 7fb228c00700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb20c06ecd0 0x7fb20c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:54.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.925+0000 7fb228c00700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb20c06ecd0 0x7fb20c071190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb210006fd0 tx=0x7fb210008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:54.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:54.929+0000 7fb222000700 1 -- 172.21.15.117:0/766686010 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb218058e50 con 0x7fb224075410 2024-04-20T11:23:55.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.244+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb22404ebb0 con 0x7fb224075410 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.244+0000 7fb222000700 1 -- 172.21.15.117:0/766686010 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fb21805c510 con 0x7fb224075410 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:23:55.245 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:23:55.246 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:23:55.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb20c06ecd0 msgr2=0x7fb20c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:55.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb20c06ecd0 0x7fb20c071190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb210006fd0 tx=0x7fb210008040 comp rx=0 tx=0).stop 2024-04-20T11:23:55.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 msgr2=0x7fb22419d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:55.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 0x7fb22419d1c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb2180092e0 tx=0x7fb2180056a0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 shutdown_connections 2024-04-20T11:23:55.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb20c06ecd0 0x7fb20c071190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb224075410 0x7fb22419d1c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 --2- 172.21.15.117:0/766686010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb224075d30 0x7fb22419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 >> 172.21.15.117:0/766686010 conn(0x7fb2240fe8d0 msgr2=0x7fb2240fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:55.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 shutdown_connections 2024-04-20T11:23:55.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.247+0000 7fb22c000700 1 -- 172.21.15.117:0/766686010 wait complete. 2024-04-20T11:23:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.398+0000 7ff2ac000700 1 -- 172.21.15.117:0/635697369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4104a00 msgr2=0x7ff2a4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.398+0000 7ff2ac000700 1 --2- 172.21.15.117:0/635697369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4104a00 0x7ff2a4104e80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff294009910 tx=0x7ff294009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.398+0000 7ff2ac000700 1 -- 172.21.15.117:0/635697369 shutdown_connections 2024-04-20T11:23:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.398+0000 7ff2ac000700 1 --2- 172.21.15.117:0/635697369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4104a00 0x7ff2a4104e80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.398+0000 7ff2ac000700 1 --2- 172.21.15.117:0/635697369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2a4103110 0x7ff2a41034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.398+0000 7ff2ac000700 1 -- 172.21.15.117:0/635697369 >> 172.21.15.117:0/635697369 conn(0x7ff2a40fe880 msgr2=0x7ff2a4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.398+0000 7ff2ac000700 1 -- 172.21.15.117:0/635697369 shutdown_connections 2024-04-20T11:23:55.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.398+0000 7ff2ac000700 1 -- 172.21.15.117:0/635697369 wait complete. 2024-04-20T11:23:55.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.399+0000 7ff2ac000700 1 Processor -- start 2024-04-20T11:23:55.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.399+0000 7ff2ac000700 1 -- start start 2024-04-20T11:23:55.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2ac000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4103110 0x7ff2a419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:55.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2ac000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2a4104a00 0x7ff2a419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:55.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2ac000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419f5b0 con 0x7ff2a4103110 2024-04-20T11:23:55.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2ac000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4198b90 con 0x7ff2a4104a00 2024-04-20T11:23:55.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2a9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4103110 0x7ff2a419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:55.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2a9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4103110 0x7ff2a419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56772/0 (socket says 172.21.15.117:56772) 2024-04-20T11:23:55.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2a9600700 1 -- 172.21.15.117:0/2169219167 learned_addr learned my addr 172.21.15.117:0/2169219167 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2a8c00700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2a4104a00 0x7ff2a419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2a9600700 1 -- 172.21.15.117:0/2169219167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2a4104a00 msgr2=0x7ff2a419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2a9600700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2a4104a00 0x7ff2a419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.400+0000 7ff2a9600700 1 -- 172.21.15.117:0/2169219167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940095f0 con 0x7ff2a4103110 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.401+0000 7ff2a9600700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4103110 0x7ff2a419ea50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff29800d8d0 tx=0x7ff29800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.401+0000 7ff2a2000700 1 -- 172.21.15.117:0/2169219167 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff298009940 con 0x7ff2a4103110 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.401+0000 7ff2a2000700 1 -- 172.21.15.117:0/2169219167 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff298010460 con 0x7ff2a4103110 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.401+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a4198e70 con 0x7ff2a4103110 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.401+0000 7ff2a2000700 1 -- 172.21.15.117:0/2169219167 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29800f5f0 con 0x7ff2a4103110 2024-04-20T11:23:55.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.401+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a4199440 con 0x7ff2a4103110 2024-04-20T11:23:55.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.402+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a410b480 con 0x7ff2a4103110 2024-04-20T11:23:55.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.403+0000 7ff2a2000700 1 -- 172.21.15.117:0/2169219167 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff2980105d0 con 0x7ff2a4103110 2024-04-20T11:23:55.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.405+0000 7ff2a2000700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28c06ecc0 0x7ff28c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:55.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.405+0000 7ff2a2000700 1 -- 172.21.15.117:0/2169219167 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff29808ed40 con 0x7ff2a4103110 2024-04-20T11:23:55.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.405+0000 7ff2a8c00700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28c06ecc0 0x7ff28c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:55.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.406+0000 7ff2a8c00700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28c06ecc0 0x7ff28c071180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b5c0 tx=0x7ff294005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:55.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.410+0000 7ff2a2000700 1 -- 172.21.15.117:0/2169219167 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff29805b6d0 con 0x7ff2a4103110 2024-04-20T11:23:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:55 smithi117 ceph-mon[99036]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:55 smithi173 ceph-mon[74165]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:55.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.680+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff2a4199fe0 con 0x7ff2a4103110 2024-04-20T11:23:55.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.681+0000 7ff2a2000700 1 -- 172.21.15.117:0/2169219167 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7ff298016770 con 0x7ff2a4103110 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:23:55.683 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:23:55.684 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:23:55.685 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:23:55.686 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:23:55.686 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:23:55.686 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28c06ecc0 msgr2=0x7ff28c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28c06ecc0 0x7ff28c071180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b5c0 tx=0x7ff294005960 comp rx=0 tx=0).stop 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4103110 msgr2=0x7ff2a419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4103110 0x7ff2a419ea50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff29800d8d0 tx=0x7ff29800dc90 comp rx=0 tx=0).stop 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 shutdown_connections 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff28c06ecc0 0x7ff28c071180 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4103110 0x7ff2a419ea50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 --2- 172.21.15.117:0/2169219167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2a4104a00 0x7ff2a419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 >> 172.21.15.117:0/2169219167 conn(0x7ff2a40fe880 msgr2=0x7ff2a40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 shutdown_connections 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.684+0000 7ff2ac000700 1 -- 172.21.15.117:0/2169219167 wait complete. 2024-04-20T11:23:55.687 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:23:55.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.832+0000 7f5137800700 1 -- 172.21.15.117:0/2861520170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 msgr2=0x7f5130104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:55.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.832+0000 7f5137800700 1 --2- 172.21.15.117:0/2861520170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 0x7f5130104920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f511c009910 tx=0x7f511c009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:55.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.832+0000 7f5137800700 1 -- 172.21.15.117:0/2861520170 shutdown_connections 2024-04-20T11:23:55.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.832+0000 7f5137800700 1 --2- 172.21.15.117:0/2861520170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 0x7f5130104920 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.832+0000 7f5137800700 1 --2- 172.21.15.117:0/2861520170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51300ffb30 0x7f51300fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.832+0000 7f5137800700 1 -- 172.21.15.117:0/2861520170 >> 172.21.15.117:0/2861520170 conn(0x7f5130076750 msgr2=0x7f51300fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:55.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.833+0000 7f5137800700 1 -- 172.21.15.117:0/2861520170 shutdown_connections 2024-04-20T11:23:55.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.833+0000 7f5137800700 1 -- 172.21.15.117:0/2861520170 wait complete. 2024-04-20T11:23:55.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f5137800700 1 Processor -- start 2024-04-20T11:23:55.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f5137800700 1 -- start start 2024-04-20T11:23:55.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f5137800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51300ffb30 0x7f513019d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f5137800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 0x7f513019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f5137800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513019de90 con 0x7f5130100450 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f5137800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51301a1b60 con 0x7f51300ffb30 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f512fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 0x7f513019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f512fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 0x7f513019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56790/0 (socket says 172.21.15.117:56790) 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.834+0000 7f512fe00700 1 -- 172.21.15.117:0/1093553678 learned_addr learned my addr 172.21.15.117:0/1093553678 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.835+0000 7f5134e00700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51300ffb30 0x7f513019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.835+0000 7f512fe00700 1 -- 172.21.15.117:0/1093553678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51300ffb30 msgr2=0x7f513019d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.835+0000 7f512fe00700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51300ffb30 0x7f513019d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.835+0000 7f512fe00700 1 -- 172.21.15.117:0/1093553678 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c0095f0 con 0x7f5130100450 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.835+0000 7f5134e00700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51300ffb30 0x7f513019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:55.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.835+0000 7f512fe00700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 0x7f513019d7b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f511c0092e0 tx=0x7f511c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:55.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.835+0000 7f512d600700 1 -- 172.21.15.117:0/1093553678 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511c01d070 con 0x7f5130100450 2024-04-20T11:23:55.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.836+0000 7f512d600700 1 -- 172.21.15.117:0/1093553678 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f511c003d50 con 0x7f5130100450 2024-04-20T11:23:55.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.836+0000 7f512d600700 1 -- 172.21.15.117:0/1093553678 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511c017400 con 0x7f5130100450 2024-04-20T11:23:55.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.836+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51301a1de0 con 0x7f5130100450 2024-04-20T11:23:55.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.836+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51301a22d0 con 0x7f5130100450 2024-04-20T11:23:55.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.837+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130068e50 con 0x7f5130100450 2024-04-20T11:23:55.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.838+0000 7f512d600700 1 -- 172.21.15.117:0/1093553678 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f511c003890 con 0x7f5130100450 2024-04-20T11:23:55.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.839+0000 7f512d600700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f511806ecc0 0x7f5118071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:55.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.839+0000 7f512d600700 1 -- 172.21.15.117:0/1093553678 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f511c090260 con 0x7f5130100450 2024-04-20T11:23:55.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.839+0000 7f5134e00700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f511806ecc0 0x7f5118071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:55.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.839+0000 7f5134e00700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f511806ecc0 0x7f5118071180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5124007900 tx=0x7f5124008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:55.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:55.843+0000 7f512d600700 1 -- 172.21.15.117:0/1093553678 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f511c0590f0 con 0x7f5130100450 2024-04-20T11:23:56.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.084+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5130114530 con 0x7f511806ecc0 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.087+0000 7f512d600700 1 -- 172.21.15.117:0/1093553678 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5130114530 con 0x7f511806ecc0 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:23:56.088 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:23:56.089 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:23:56.089 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:23:56.089 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:23:56.089 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:23:56.089 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:23:56.089 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:23:56.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f511806ecc0 msgr2=0x7f5118071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f511806ecc0 0x7f5118071180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5124007900 tx=0x7f5124008040 comp rx=0 tx=0).stop 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 msgr2=0x7f513019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 0x7f513019d7b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f511c0092e0 tx=0x7f511c00be20 comp rx=0 tx=0).stop 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 shutdown_connections 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f511806ecc0 0x7f5118071180 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51300ffb30 0x7f513019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 --2- 172.21.15.117:0/1093553678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5130100450 0x7f513019d7b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 >> 172.21.15.117:0/1093553678 conn(0x7f5130076750 msgr2=0x7f513010e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 shutdown_connections 2024-04-20T11:23:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.090+0000 7f5137800700 1 -- 172.21.15.117:0/1093553678 wait complete. 2024-04-20T11:23:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.249+0000 7f15e7400700 1 -- 172.21.15.117:0/608633504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e0103020 msgr2=0x7f15e0103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.249+0000 7f15e7400700 1 --2- 172.21.15.117:0/608633504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e0103020 0x7f15e0103400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009910 tx=0x7f15d4009c20 comp rx=0 tx=0).stop 2024-04-20T11:23:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.249+0000 7f15e7400700 1 -- 172.21.15.117:0/608633504 shutdown_connections 2024-04-20T11:23:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.249+0000 7f15e7400700 1 --2- 172.21.15.117:0/608633504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e01039d0 0x7f15e0107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.249+0000 7f15e7400700 1 --2- 172.21.15.117:0/608633504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e0103020 0x7f15e0103400 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.249+0000 7f15e7400700 1 -- 172.21.15.117:0/608633504 >> 172.21.15.117:0/608633504 conn(0x7f15e00fe8b0 msgr2=0x7f15e0100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.249+0000 7f15e7400700 1 -- 172.21.15.117:0/608633504 shutdown_connections 2024-04-20T11:23:56.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.249+0000 7f15e7400700 1 -- 172.21.15.117:0/608633504 wait complete. 2024-04-20T11:23:56.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.250+0000 7f15e7400700 1 Processor -- start 2024-04-20T11:23:56.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.250+0000 7f15e7400700 1 -- start start 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.250+0000 7f15e7400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e0103020 0x7f15e019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.250+0000 7f15e7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e01039d0 0x7f15e019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.250+0000 7f15e7400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e019f5c0 con 0x7f15e01039d0 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.250+0000 7f15e7400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e0198bf0 con 0x7f15e0103020 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.251+0000 7f15dfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e01039d0 0x7f15e019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.251+0000 7f15dfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e01039d0 0x7f15e019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56812/0 (socket says 172.21.15.117:56812) 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.251+0000 7f15dfe00700 1 -- 172.21.15.117:0/666015289 learned_addr learned my addr 172.21.15.117:0/666015289 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.251+0000 7f15e4a00700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e0103020 0x7f15e019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.251+0000 7f15dfe00700 1 -- 172.21.15.117:0/666015289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e0103020 msgr2=0x7f15e019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.251+0000 7f15dfe00700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e0103020 0x7f15e019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.251+0000 7f15dfe00700 1 -- 172.21.15.117:0/666015289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40095f0 con 0x7f15e01039d0 2024-04-20T11:23:56.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.251+0000 7f15dfe00700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e01039d0 0x7f15e019efa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00c8f0 tx=0x7f15cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:56.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.252+0000 7f15dd600700 1 -- 172.21.15.117:0/666015289 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15cc007a50 con 0x7f15e01039d0 2024-04-20T11:23:56.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.252+0000 7f15dd600700 1 -- 172.21.15.117:0/666015289 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15cc007bb0 con 0x7f15e01039d0 2024-04-20T11:23:56.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.252+0000 7f15dd600700 1 -- 172.21.15.117:0/666015289 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15cc018630 con 0x7f15e01039d0 2024-04-20T11:23:56.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.252+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e0198ed0 con 0x7f15e01039d0 2024-04-20T11:23:56.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.252+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0199420 con 0x7f15e01039d0 2024-04-20T11:23:56.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.253+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e010b3c0 con 0x7f15e01039d0 2024-04-20T11:23:56.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.254+0000 7f15dd600700 1 -- 172.21.15.117:0/666015289 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f15cc007d20 con 0x7f15e01039d0 2024-04-20T11:23:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.255+0000 7f15dd600700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15c806ecd0 0x7f15c8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.255+0000 7f15dd600700 1 -- 172.21.15.117:0/666015289 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f15cc08edd0 con 0x7f15e01039d0 2024-04-20T11:23:56.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.255+0000 7f15e4a00700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15c806ecd0 0x7f15c8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:56.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.256+0000 7f15e4a00700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15c806ecd0 0x7f15c8071190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15d4006010 tx=0x7f15d400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:56.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.260+0000 7f15dd600700 1 -- 172.21.15.117:0/666015289 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f15cc05b6c0 con 0x7f15e01039d0 2024-04-20T11:23:56.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:56 smithi117 ceph-mon[99036]: from='client.34676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:56 smithi117 ceph-mon[99036]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:56 smithi117 ceph-mon[99036]: pgmap v344: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:56 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/766686010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:23:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:56 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2169219167' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:23:56.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.551+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f15e0199ff0 con 0x7f15e01039d0 2024-04-20T11:23:56.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.552+0000 7f15dd600700 1 -- 172.21.15.117:0/666015289 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f15cc05b250 con 0x7f15e01039d0 2024-04-20T11:23:56.553 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:23:56.553 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:23:56.553 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:23:56.553 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:23:56.553 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:23:56.553 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:23:56.553 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:23:56.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15c806ecd0 msgr2=0x7f15c8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:56.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15c806ecd0 0x7f15c8071190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15d4006010 tx=0x7f15d400b540 comp rx=0 tx=0).stop 2024-04-20T11:23:56.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e01039d0 msgr2=0x7f15e019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:56.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e01039d0 0x7f15e019efa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00c8f0 tx=0x7f15cc00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 shutdown_connections 2024-04-20T11:23:56.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15c806ecd0 0x7f15c8071190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e0103020 0x7f15e019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 --2- 172.21.15.117:0/666015289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15e01039d0 0x7f15e019efa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:56.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 >> 172.21.15.117:0/666015289 conn(0x7f15e00fe8b0 msgr2=0x7f15e00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:56.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 shutdown_connections 2024-04-20T11:23:56.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:23:56.555+0000 7f15e7400700 1 -- 172.21.15.117:0/666015289 wait complete. 2024-04-20T11:23:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:56 smithi173 ceph-mon[74165]: from='client.34676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:56 smithi173 ceph-mon[74165]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:56 smithi173 ceph-mon[74165]: pgmap v344: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:56 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/766686010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:23:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:56 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2169219167' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:23:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:57 smithi117 ceph-mon[99036]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:57.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:57 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/666015289' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:23:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:57 smithi173 ceph-mon[74165]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:23:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:57 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/666015289' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:23:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:58 smithi117 ceph-mon[99036]: pgmap v345: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:58 smithi173 ceph-mon[74165]: pgmap v345: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:23:59.349 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:23:59 smithi173 ceph-mon[74165]: pgmap v346: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:23:59.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:23:59 smithi117 ceph-mon[99036]: pgmap v346: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:02.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:02 smithi117 ceph-mon[99036]: pgmap v347: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:02 smithi173 ceph-mon[74165]: pgmap v347: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:04 smithi117 ceph-mon[99036]: pgmap v348: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:04 smithi173 ceph-mon[74165]: pgmap v348: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:06 smithi117 ceph-mon[99036]: pgmap v349: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:06 smithi173 ceph-mon[74165]: pgmap v349: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:08.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:08 smithi117 ceph-mon[99036]: pgmap v350: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:08 smithi173 ceph-mon[74165]: pgmap v350: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:10.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:10 smithi117 ceph-mon[99036]: pgmap v351: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:10 smithi173 ceph-mon[74165]: pgmap v351: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:11 smithi117 ceph-mon[99036]: pgmap v352: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:11 smithi173 ceph-mon[74165]: pgmap v352: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:24:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:24:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:14 smithi117 ceph-mon[99036]: pgmap v353: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:14 smithi173 ceph-mon[74165]: pgmap v353: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:16 smithi117 ceph-mon[99036]: pgmap v354: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:16.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:16 smithi173 ceph-mon[74165]: pgmap v354: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:18 smithi117 ceph-mon[99036]: pgmap v355: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:18 smithi173 ceph-mon[74165]: pgmap v355: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:20 smithi117 ceph-mon[99036]: pgmap v356: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:20 smithi173 ceph-mon[74165]: pgmap v356: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:22 smithi117 ceph-mon[99036]: pgmap v357: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:22 smithi173 ceph-mon[74165]: pgmap v357: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:24.558 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:24 smithi117 ceph-mon[99036]: pgmap v358: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:24 smithi173 ceph-mon[74165]: pgmap v358: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:25 smithi117 ceph-mon[99036]: pgmap v359: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:25 smithi173 ceph-mon[74165]: pgmap v359: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:26.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.726+0000 7fd9a9600700 1 -- 172.21.15.117:0/2012423782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a41017c0 msgr2=0x7fd9a4105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.726+0000 7fd9a9600700 1 --2- 172.21.15.117:0/2012423782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a41017c0 0x7fd9a4105810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd990009910 tx=0x7fd990009c20 comp rx=0 tx=0).stop 2024-04-20T11:24:26.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.727+0000 7fd9a9600700 1 -- 172.21.15.117:0/2012423782 shutdown_connections 2024-04-20T11:24:26.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.727+0000 7fd9a9600700 1 --2- 172.21.15.117:0/2012423782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a41017c0 0x7fd9a4105810 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.727+0000 7fd9a9600700 1 --2- 172.21.15.117:0/2012423782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a4100e10 0x7fd9a41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.727+0000 7fd9a9600700 1 -- 172.21.15.117:0/2012423782 >> 172.21.15.117:0/2012423782 conn(0x7fd9a40fc6a0 msgr2=0x7fd9a40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:26.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.727+0000 7fd9a9600700 1 -- 172.21.15.117:0/2012423782 shutdown_connections 2024-04-20T11:24:26.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.727+0000 7fd9a9600700 1 -- 172.21.15.117:0/2012423782 wait complete. 2024-04-20T11:24:26.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.728+0000 7fd9a9600700 1 Processor -- start 2024-04-20T11:24:26.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.728+0000 7fd9a9600700 1 -- start start 2024-04-20T11:24:26.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.728+0000 7fd9a9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a4100e10 0x7fd9a40731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.728+0000 7fd9a9600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a41017c0 0x7fd9a4073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.728+0000 7fd9a9600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4073c60 con 0x7fd9a4100e10 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.728+0000 7fd9a9600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4073da0 con 0x7fd9a41017c0 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a4100e10 0x7fd9a40731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a4100e10 0x7fd9a40731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43606/0 (socket says 172.21.15.117:43606) 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2a00700 1 -- 172.21.15.117:0/3987828770 learned_addr learned my addr 172.21.15.117:0/3987828770 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2000700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a41017c0 0x7fd9a4073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2a00700 1 -- 172.21.15.117:0/3987828770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a41017c0 msgr2=0x7fd9a4073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2a00700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a41017c0 0x7fd9a4073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2a00700 1 -- 172.21.15.117:0/3987828770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9900095f0 con 0x7fd9a4100e10 2024-04-20T11:24:26.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2000700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a41017c0 0x7fd9a4073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:24:26.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.729+0000 7fd9a2a00700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a4100e10 0x7fd9a40731e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd99400c8f0 tx=0x7fd99400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:26.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.730+0000 7fd99b400700 1 -- 172.21.15.117:0/3987828770 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd994007a70 con 0x7fd9a4100e10 2024-04-20T11:24:26.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.730+0000 7fd99b400700 1 -- 172.21.15.117:0/3987828770 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd994004cf0 con 0x7fd9a4100e10 2024-04-20T11:24:26.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.730+0000 7fd99b400700 1 -- 172.21.15.117:0/3987828770 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd994005610 con 0x7fd9a4100e10 2024-04-20T11:24:26.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.730+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a41a2d50 con 0x7fd9a4100e10 2024-04-20T11:24:26.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.730+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a41a30b0 con 0x7fd9a4100e10 2024-04-20T11:24:26.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.731+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a4109210 con 0x7fd9a4100e10 2024-04-20T11:24:26.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.732+0000 7fd99b400700 1 -- 172.21.15.117:0/3987828770 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd994004750 con 0x7fd9a4100e10 2024-04-20T11:24:26.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.733+0000 7fd99b400700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd98806ecd0 0x7fd988071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.733+0000 7fd99b400700 1 -- 172.21.15.117:0/3987828770 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd994013070 con 0x7fd9a4100e10 2024-04-20T11:24:26.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.734+0000 7fd9a2000700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd98806ecd0 0x7fd988071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.734+0000 7fd9a2000700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd98806ecd0 0x7fd988071190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd990006010 tx=0x7fd99000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:26.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.738+0000 7fd99b400700 1 -- 172.21.15.117:0/3987828770 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd9940579e0 con 0x7fd9a4100e10 2024-04-20T11:24:26.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.975+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9a410ddd0 con 0x7fd98806ecd0 2024-04-20T11:24:26.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.978+0000 7fd99b400700 1 -- 172.21.15.117:0/3987828770 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd9a410ddd0 con 0x7fd98806ecd0 2024-04-20T11:24:26.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd98806ecd0 msgr2=0x7fd988071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd98806ecd0 0x7fd988071190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd990006010 tx=0x7fd99000b540 comp rx=0 tx=0).stop 2024-04-20T11:24:26.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a4100e10 msgr2=0x7fd9a40731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a4100e10 0x7fd9a40731e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd99400c8f0 tx=0x7fd99400ccb0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 shutdown_connections 2024-04-20T11:24:26.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd98806ecd0 0x7fd988071190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a4100e10 0x7fd9a40731e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 --2- 172.21.15.117:0/3987828770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a41017c0 0x7fd9a4073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.981+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 >> 172.21.15.117:0/3987828770 conn(0x7fd9a40fc6a0 msgr2=0x7fd9a40fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:26.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.982+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 shutdown_connections 2024-04-20T11:24:26.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:26.982+0000 7fd9a9600700 1 -- 172.21.15.117:0/3987828770 wait complete. 2024-04-20T11:24:26.996 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:24:27.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.139+0000 7fcd9e800700 1 -- 172.21.15.117:0/536862994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 msgr2=0x7fcd981011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.139+0000 7fcd9e800700 1 --2- 172.21.15.117:0/536862994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 0x7fcd981011f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009910 tx=0x7fcd88009c20 comp rx=0 tx=0).stop 2024-04-20T11:24:27.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.139+0000 7fcd9e800700 1 -- 172.21.15.117:0/536862994 shutdown_connections 2024-04-20T11:24:27.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.139+0000 7fcd9e800700 1 --2- 172.21.15.117:0/536862994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd981017c0 0x7fcd98105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.139+0000 7fcd9e800700 1 --2- 172.21.15.117:0/536862994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 0x7fcd981011f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.139+0000 7fcd9e800700 1 -- 172.21.15.117:0/536862994 >> 172.21.15.117:0/536862994 conn(0x7fcd980fc6a0 msgr2=0x7fcd980feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.139+0000 7fcd9e800700 1 -- 172.21.15.117:0/536862994 shutdown_connections 2024-04-20T11:24:27.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.139+0000 7fcd9e800700 1 -- 172.21.15.117:0/536862994 wait complete. 2024-04-20T11:24:27.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.140+0000 7fcd9e800700 1 Processor -- start 2024-04-20T11:24:27.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.141+0000 7fcd9e800700 1 -- start start 2024-04-20T11:24:27.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.141+0000 7fcd9e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 0x7fcd9819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.141+0000 7fcd9e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd981017c0 0x7fcd9819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.141+0000 7fcd9e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9819d360 con 0x7fcd98100e10 2024-04-20T11:24:27.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.141+0000 7fcd9e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd98196940 con 0x7fcd981017c0 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.141+0000 7fcd97400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 0x7fcd9819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.141+0000 7fcd97400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 0x7fcd9819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43636/0 (socket says 172.21.15.117:43636) 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.141+0000 7fcd97400700 1 -- 172.21.15.117:0/1670599654 learned_addr learned my addr 172.21.15.117:0/1670599654 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.142+0000 7fcd97400700 1 -- 172.21.15.117:0/1670599654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd981017c0 msgr2=0x7fcd9819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.142+0000 7fcd97400700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd981017c0 0x7fcd9819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.142+0000 7fcd97400700 1 -- 172.21.15.117:0/1670599654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd880095f0 con 0x7fcd98100e10 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.142+0000 7fcd97400700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 0x7fcd9819c800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcd88000c00 tx=0x7fcd8800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.142+0000 7fcd8fe00700 1 -- 172.21.15.117:0/1670599654 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd8801d070 con 0x7fcd98100e10 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.142+0000 7fcd8fe00700 1 -- 172.21.15.117:0/1670599654 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd880056e0 con 0x7fcd98100e10 2024-04-20T11:24:27.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.143+0000 7fcd8fe00700 1 -- 172.21.15.117:0/1670599654 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd8800fe80 con 0x7fcd98100e10 2024-04-20T11:24:27.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.143+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd98196bc0 con 0x7fcd98100e10 2024-04-20T11:24:27.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.143+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd981970b0 con 0x7fcd98100e10 2024-04-20T11:24:27.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.144+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd981091b0 con 0x7fcd98100e10 2024-04-20T11:24:27.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.145+0000 7fcd8fe00700 1 -- 172.21.15.117:0/1670599654 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcd8800f6f0 con 0x7fcd98100e10 2024-04-20T11:24:27.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.146+0000 7fcd8fe00700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7c06ecd0 0x7fcd7c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.146+0000 7fcd8fe00700 1 -- 172.21.15.117:0/1670599654 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcd88090230 con 0x7fcd98100e10 2024-04-20T11:24:27.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.146+0000 7fcd96a00700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7c06ecd0 0x7fcd7c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.147+0000 7fcd96a00700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7c06ecd0 0x7fcd7c071190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcd84005950 tx=0x7fcd840058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.151+0000 7fcd8fe00700 1 -- 172.21.15.117:0/1670599654 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcd8805cb20 con 0x7fcd98100e10 2024-04-20T11:24:27.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.393+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd98197ea0 con 0x7fcd7c06ecd0 2024-04-20T11:24:27.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.395+0000 7fcd8fe00700 1 -- 172.21.15.117:0/1670599654 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcd98197ea0 con 0x7fcd7c06ecd0 2024-04-20T11:24:27.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7c06ecd0 msgr2=0x7fcd7c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7c06ecd0 0x7fcd7c071190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcd84005950 tx=0x7fcd840058e0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 msgr2=0x7fcd9819c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 0x7fcd9819c800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcd88000c00 tx=0x7fcd8800be20 comp rx=0 tx=0).stop 2024-04-20T11:24:27.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 shutdown_connections 2024-04-20T11:24:27.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7c06ecd0 0x7fcd7c071190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd98100e10 0x7fcd9819c800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 --2- 172.21.15.117:0/1670599654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd981017c0 0x7fcd9819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 >> 172.21.15.117:0/1670599654 conn(0x7fcd980fc6a0 msgr2=0x7fcd980fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 shutdown_connections 2024-04-20T11:24:27.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.398+0000 7fcd9e800700 1 -- 172.21.15.117:0/1670599654 wait complete. 2024-04-20T11:24:27.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.543+0000 7f949a600700 1 -- 172.21.15.117:0/1585888877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 msgr2=0x7f94941011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.543+0000 7f949a600700 1 --2- 172.21.15.117:0/1585888877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 0x7f94941011f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9484009910 tx=0x7f9484009c20 comp rx=0 tx=0).stop 2024-04-20T11:24:27.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.543+0000 7f949a600700 1 -- 172.21.15.117:0/1585888877 shutdown_connections 2024-04-20T11:24:27.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.543+0000 7f949a600700 1 --2- 172.21.15.117:0/1585888877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941017c0 0x7f9494105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.543+0000 7f949a600700 1 --2- 172.21.15.117:0/1585888877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 0x7f94941011f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.543+0000 7f949a600700 1 -- 172.21.15.117:0/1585888877 >> 172.21.15.117:0/1585888877 conn(0x7f94940fc6c0 msgr2=0x7f94940feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.543+0000 7f949a600700 1 -- 172.21.15.117:0/1585888877 shutdown_connections 2024-04-20T11:24:27.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.543+0000 7f949a600700 1 -- 172.21.15.117:0/1585888877 wait complete. 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.544+0000 7f949a600700 1 Processor -- start 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.545+0000 7f949a600700 1 -- start start 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.545+0000 7f949a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 0x7f949419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.545+0000 7f949a600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941017c0 0x7f949419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.545+0000 7f949a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949419d360 con 0x7f9494100e10 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.545+0000 7f949a600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494196940 con 0x7f94941017c0 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.545+0000 7f9493400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 0x7f949419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.545+0000 7f9493400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 0x7f949419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43662/0 (socket says 172.21.15.117:43662) 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.545+0000 7f9493400700 1 -- 172.21.15.117:0/1153251519 learned_addr learned my addr 172.21.15.117:0/1153251519 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.546+0000 7f9493400700 1 -- 172.21.15.117:0/1153251519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941017c0 msgr2=0x7f949419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.546+0000 7f9493400700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941017c0 0x7f949419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.546+0000 7f9493400700 1 -- 172.21.15.117:0/1153251519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840095f0 con 0x7f9494100e10 2024-04-20T11:24:27.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.546+0000 7f9493400700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 0x7f949419c800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9484000c00 tx=0x7f948400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.546+0000 7f9490c00700 1 -- 172.21.15.117:0/1153251519 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f948401d070 con 0x7f9494100e10 2024-04-20T11:24:27.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.546+0000 7f9490c00700 1 -- 172.21.15.117:0/1153251519 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94840056e0 con 0x7f9494100e10 2024-04-20T11:24:27.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.546+0000 7f9490c00700 1 -- 172.21.15.117:0/1153251519 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f948400fe80 con 0x7f9494100e10 2024-04-20T11:24:27.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.546+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9494196bc0 con 0x7f9494100e10 2024-04-20T11:24:27.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.547+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94941970b0 con 0x7f9494100e10 2024-04-20T11:24:27.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.547+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94941091b0 con 0x7f9494100e10 2024-04-20T11:24:27.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.549+0000 7f9490c00700 1 -- 172.21.15.117:0/1153251519 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f948400f6f0 con 0x7f9494100e10 2024-04-20T11:24:27.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.549+0000 7f9490c00700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f947806ecc0 0x7f9478071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.550+0000 7f9490c00700 1 -- 172.21.15.117:0/1153251519 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9484090200 con 0x7f9494100e10 2024-04-20T11:24:27.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.550+0000 7f948be00700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f947806ecc0 0x7f9478071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.550+0000 7f948be00700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f947806ecc0 0x7f9478071180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f947c005950 tx=0x7f947c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.554+0000 7f9490c00700 1 -- 172.21.15.117:0/1153251519 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f948405caf0 con 0x7f9494100e10 2024-04-20T11:24:27.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.788+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f949410dde0 con 0x7f947806ecc0 2024-04-20T11:24:27.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.806+0000 7f9490c00700 1 -- 172.21.15.117:0/1153251519 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f949410dde0 con 0x7f947806ecc0 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (12m) 4m ago 26m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (27m) 4m ago 27m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (25m) 4m ago 25m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (9m) 4m ago 27m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (9m) 4m ago 25m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (12m) 4m ago 26m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (4m) 4m ago 19m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (5m) 4m ago 19m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (19m) 4m ago 19m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:24:27.807 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (4m) 4m ago 19m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (16m) 4m ago 28m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (14m) 4m ago 25m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (10m) 4m ago 28m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (10m) 4m ago 25m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (14m) 4m ago 27m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (14m) 4m ago 25m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (8m) 4m ago 24m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (8m) 4m ago 23m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (7m) 4m ago 23m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (6m) 4m ago 22m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (6m) 4m ago 22m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (5m) 4m ago 21m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:24:27.808 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (13m) 4m ago 26m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f947806ecc0 msgr2=0x7f9478071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f947806ecc0 0x7f9478071180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f947c005950 tx=0x7f947c0058e0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 msgr2=0x7f949419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 0x7f949419c800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9484000c00 tx=0x7f948400be20 comp rx=0 tx=0).stop 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 shutdown_connections 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f947806ecc0 0x7f9478071180 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9494100e10 0x7f949419c800 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 --2- 172.21.15.117:0/1153251519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941017c0 0x7f949419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 >> 172.21.15.117:0/1153251519 conn(0x7f94940fc6c0 msgr2=0x7f94940fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 shutdown_connections 2024-04-20T11:24:27.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.809+0000 7f949a600700 1 -- 172.21.15.117:0/1153251519 wait complete. 2024-04-20T11:24:27.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.963+0000 7fc83c000700 1 -- 172.21.15.117:0/107064087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107df0 msgr2=0x7fc83410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.963+0000 7fc83c000700 1 --2- 172.21.15.117:0/107064087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107df0 0x7fc83410be40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc820009910 tx=0x7fc820009c20 comp rx=0 tx=0).stop 2024-04-20T11:24:27.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.963+0000 7fc83c000700 1 -- 172.21.15.117:0/107064087 shutdown_connections 2024-04-20T11:24:27.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.963+0000 7fc83c000700 1 --2- 172.21.15.117:0/107064087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107df0 0x7fc83410be40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.963+0000 7fc83c000700 1 --2- 172.21.15.117:0/107064087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc834107440 0x7fc834107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.963+0000 7fc83c000700 1 -- 172.21.15.117:0/107064087 >> 172.21.15.117:0/107064087 conn(0x7fc834076a10 msgr2=0x7fc834078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.964+0000 7fc83c000700 1 -- 172.21.15.117:0/107064087 shutdown_connections 2024-04-20T11:24:27.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.964+0000 7fc83c000700 1 -- 172.21.15.117:0/107064087 wait complete. 2024-04-20T11:24:27.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.965+0000 7fc83c000700 1 Processor -- start 2024-04-20T11:24:27.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.965+0000 7fc83c000700 1 -- start start 2024-04-20T11:24:27.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.965+0000 7fc83c000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107440 0x7fc83419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.965+0000 7fc83c000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc834107df0 0x7fc83419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.965+0000 7fc83c000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419dc80 con 0x7fc834107440 2024-04-20T11:24:27.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.965+0000 7fc83c000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419ddc0 con 0x7fc834107df0 2024-04-20T11:24:27.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.966+0000 7fc839600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107440 0x7fc83419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.966+0000 7fc839600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107440 0x7fc83419d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43684/0 (socket says 172.21.15.117:43684) 2024-04-20T11:24:27.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.966+0000 7fc839600700 1 -- 172.21.15.117:0/852763280 learned_addr learned my addr 172.21.15.117:0/852763280 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:24:27.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.966+0000 7fc838c00700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc834107df0 0x7fc83419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.966+0000 7fc839600700 1 -- 172.21.15.117:0/852763280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc834107df0 msgr2=0x7fc83419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.966+0000 7fc839600700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc834107df0 0x7fc83419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.966+0000 7fc839600700 1 -- 172.21.15.117:0/852763280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8200095f0 con 0x7fc834107440 2024-04-20T11:24:27.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.966+0000 7fc839600700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107440 0x7fc83419d0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c930 tx=0x7fc82800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.967+0000 7fc832000700 1 -- 172.21.15.117:0/852763280 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc828007ad0 con 0x7fc834107440 2024-04-20T11:24:27.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.967+0000 7fc832000700 1 -- 172.21.15.117:0/852763280 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc828007c30 con 0x7fc834107440 2024-04-20T11:24:27.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.967+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8341a1b50 con 0x7fc834107440 2024-04-20T11:24:27.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.967+0000 7fc832000700 1 -- 172.21.15.117:0/852763280 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc828018610 con 0x7fc834107440 2024-04-20T11:24:27.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.967+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8341a20a0 con 0x7fc834107440 2024-04-20T11:24:27.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.968+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83410f840 con 0x7fc834107440 2024-04-20T11:24:27.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.969+0000 7fc832000700 1 -- 172.21.15.117:0/852763280 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc82800f450 con 0x7fc834107440 2024-04-20T11:24:27.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.970+0000 7fc832000700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc81c06ecd0 0x7fc81c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.970+0000 7fc832000700 1 -- 172.21.15.117:0/852763280 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc82808f0f0 con 0x7fc834107440 2024-04-20T11:24:27.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.970+0000 7fc838c00700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc81c06ecd0 0x7fc81c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.971+0000 7fc838c00700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc81c06ecd0 0x7fc81c071190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc820006010 tx=0x7fc82000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:27.975+0000 7fc832000700 1 -- 172.21.15.117:0/852763280 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc828057e30 con 0x7fc834107440 2024-04-20T11:24:28.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.304+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc83419e4a0 con 0x7fc834107440 2024-04-20T11:24:28.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.305+0000 7fc832000700 1 -- 172.21.15.117:0/852763280 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fc82805b4f0 con 0x7fc834107440 2024-04-20T11:24:28.305 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:24:28.305 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:24:28.305 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:24:28.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:24:28.305 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:24:28.306 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:24:28.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc81c06ecd0 msgr2=0x7fc81c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc81c06ecd0 0x7fc81c071190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc820006010 tx=0x7fc82000b540 comp rx=0 tx=0).stop 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107440 msgr2=0x7fc83419d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107440 0x7fc83419d0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c930 tx=0x7fc82800ccf0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 shutdown_connections 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc81c06ecd0 0x7fc81c071190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc834107440 0x7fc83419d0f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 --2- 172.21.15.117:0/852763280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc834107df0 0x7fc83419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 >> 172.21.15.117:0/852763280 conn(0x7fc834076a10 msgr2=0x7fc834076df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 shutdown_connections 2024-04-20T11:24:28.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.308+0000 7fc83c000700 1 -- 172.21.15.117:0/852763280 wait complete. 2024-04-20T11:24:28.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.462+0000 7fab8ec00700 1 -- 172.21.15.117:0/2922001810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 msgr2=0x7fab88109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:28.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.462+0000 7fab8ec00700 1 --2- 172.21.15.117:0/2922001810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 0x7fab88109290 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fab70009910 tx=0x7fab70009c20 comp rx=0 tx=0).stop 2024-04-20T11:24:28.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.462+0000 7fab8ec00700 1 -- 172.21.15.117:0/2922001810 shutdown_connections 2024-04-20T11:24:28.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.462+0000 7fab8ec00700 1 --2- 172.21.15.117:0/2922001810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 0x7fab88109290 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.462+0000 7fab8ec00700 1 --2- 172.21.15.117:0/2922001810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab88107520 0x7fab88107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.462+0000 7fab8ec00700 1 -- 172.21.15.117:0/2922001810 >> 172.21.15.117:0/2922001810 conn(0x7fab880768f0 msgr2=0x7fab88078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:28.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.462+0000 7fab8ec00700 1 -- 172.21.15.117:0/2922001810 shutdown_connections 2024-04-20T11:24:28.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.462+0000 7fab8ec00700 1 -- 172.21.15.117:0/2922001810 wait complete. 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.463+0000 7fab8ec00700 1 Processor -- start 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.463+0000 7fab8ec00700 1 -- start start 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.463+0000 7fab8ec00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab88107520 0x7fab8819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.463+0000 7fab8ec00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 0x7fab8819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab8ec00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8819ddf0 con 0x7fab88108e10 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab8ec00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab881a1a70 con 0x7fab88107520 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab7fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 0x7fab8819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab7fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 0x7fab8819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43700/0 (socket says 172.21.15.117:43700) 2024-04-20T11:24:28.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab7fe00700 1 -- 172.21.15.117:0/1167086593 learned_addr learned my addr 172.21.15.117:0/1167086593 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:24:28.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab87e00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab88107520 0x7fab8819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:28.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab7fe00700 1 -- 172.21.15.117:0/1167086593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab88107520 msgr2=0x7fab8819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab7fe00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab88107520 0x7fab8819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab7fe00700 1 -- 172.21.15.117:0/1167086593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab700095f0 con 0x7fab88108e10 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.464+0000 7fab7fe00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 0x7fab8819d710 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab700092e0 tx=0x7fab7000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.465+0000 7fab85600700 1 -- 172.21.15.117:0/1167086593 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab7001d070 con 0x7fab88108e10 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.465+0000 7fab85600700 1 -- 172.21.15.117:0/1167086593 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab700056e0 con 0x7fab88108e10 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.465+0000 7fab85600700 1 -- 172.21.15.117:0/1167086593 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab7000fe80 con 0x7fab88108e10 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.465+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab881a1cf0 con 0x7fab88108e10 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.465+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab881a2260 con 0x7fab88108e10 2024-04-20T11:24:28.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.466+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8810f870 con 0x7fab88108e10 2024-04-20T11:24:28.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.467+0000 7fab85600700 1 -- 172.21.15.117:0/1167086593 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fab7000f6f0 con 0x7fab88108e10 2024-04-20T11:24:28.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.468+0000 7fab85600700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab6c06ecd0 0x7fab6c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:28.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.468+0000 7fab85600700 1 -- 172.21.15.117:0/1167086593 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fab70090190 con 0x7fab88108e10 2024-04-20T11:24:28.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.468+0000 7fab87e00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab6c06ecd0 0x7fab6c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:28.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.469+0000 7fab87e00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab6c06ecd0 0x7fab6c071190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab78005950 tx=0x7fab780058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:28.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.473+0000 7fab85600700 1 -- 172.21.15.117:0/1167086593 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fab70058ff0 con 0x7fab88108e10 2024-04-20T11:24:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:28 smithi117 ceph-mon[99036]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:24:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:28 smithi117 ceph-mon[99036]: pgmap v360: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:24:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:28 smithi173 ceph-mon[74165]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:24:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:28 smithi173 ceph-mon[74165]: pgmap v360: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:24:28.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.745+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab88108920 con 0x7fab88108e10 2024-04-20T11:24:28.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.745+0000 7fab85600700 1 -- 172.21.15.117:0/1167086593 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fab7005c6b0 con 0x7fab88108e10 2024-04-20T11:24:28.745 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:24:28.746 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:24:28.747 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:24:28.748 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:24:28.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab6c06ecd0 msgr2=0x7fab6c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:28.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab6c06ecd0 0x7fab6c071190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab78005950 tx=0x7fab780058e0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 msgr2=0x7fab8819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 0x7fab8819d710 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab700092e0 tx=0x7fab7000be20 comp rx=0 tx=0).stop 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 shutdown_connections 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fab6c06ecd0 0x7fab6c071190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab88107520 0x7fab8819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 --2- 172.21.15.117:0/1167086593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab88108e10 0x7fab8819d710 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 >> 172.21.15.117:0/1167086593 conn(0x7fab880768f0 msgr2=0x7fab88077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 shutdown_connections 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.748+0000 7fab8ec00700 1 -- 172.21.15.117:0/1167086593 wait complete. 2024-04-20T11:24:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:24:28.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.905+0000 7f174f000700 1 -- 172.21.15.117:0/1076109254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 msgr2=0x7f1748104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:28.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.905+0000 7f174f000700 1 --2- 172.21.15.117:0/1076109254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 0x7f1748104e80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1730009910 tx=0x7f1730009c20 comp rx=0 tx=0).stop 2024-04-20T11:24:28.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.905+0000 7f174f000700 1 -- 172.21.15.117:0/1076109254 shutdown_connections 2024-04-20T11:24:28.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.905+0000 7f174f000700 1 --2- 172.21.15.117:0/1076109254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 0x7f1748104e80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.905+0000 7f174f000700 1 --2- 172.21.15.117:0/1076109254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1748103110 0x7f17481034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.905+0000 7f174f000700 1 -- 172.21.15.117:0/1076109254 >> 172.21.15.117:0/1076109254 conn(0x7f17480fe880 msgr2=0x7f1748100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:28.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.905+0000 7f174f000700 1 -- 172.21.15.117:0/1076109254 shutdown_connections 2024-04-20T11:24:28.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.905+0000 7f174f000700 1 -- 172.21.15.117:0/1076109254 wait complete. 2024-04-20T11:24:28.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.906+0000 7f174f000700 1 Processor -- start 2024-04-20T11:24:28.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.907+0000 7f174f000700 1 -- start start 2024-04-20T11:24:28.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.907+0000 7f174f000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1748103110 0x7f174819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:28.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.907+0000 7f174f000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 0x7f174819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:28.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.907+0000 7f174f000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174819f5b0 con 0x7f1748104a00 2024-04-20T11:24:28.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.907+0000 7f174f000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748198b90 con 0x7f1748103110 2024-04-20T11:24:28.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.907+0000 7f1747400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 0x7f174819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.907+0000 7f1747400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 0x7f174819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43708/0 (socket says 172.21.15.117:43708) 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.907+0000 7f1747400700 1 -- 172.21.15.117:0/1315001254 learned_addr learned my addr 172.21.15.117:0/1315001254 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.908+0000 7f1747e00700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1748103110 0x7f174819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.908+0000 7f1747400700 1 -- 172.21.15.117:0/1315001254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1748103110 msgr2=0x7f174819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.908+0000 7f1747400700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1748103110 0x7f174819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.908+0000 7f1747400700 1 -- 172.21.15.117:0/1315001254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17300095f0 con 0x7f1748104a00 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.908+0000 7f1747e00700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1748103110 0x7f174819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.908+0000 7f1747400700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 0x7f174819ef90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f17300092e0 tx=0x7f173000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.909+0000 7f1744c00700 1 -- 172.21.15.117:0/1315001254 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173001d070 con 0x7f1748104a00 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.909+0000 7f1744c00700 1 -- 172.21.15.117:0/1315001254 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f173000f460 con 0x7f1748104a00 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.909+0000 7f1744c00700 1 -- 172.21.15.117:0/1315001254 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1730017630 con 0x7f1748104a00 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.909+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1748198e10 con 0x7f1748104a00 2024-04-20T11:24:28.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.909+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748199380 con 0x7f1748104a00 2024-04-20T11:24:28.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.910+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174810b480 con 0x7f1748104a00 2024-04-20T11:24:28.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.911+0000 7f1744c00700 1 -- 172.21.15.117:0/1315001254 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f173000f5d0 con 0x7f1748104a00 2024-04-20T11:24:28.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.912+0000 7f1744c00700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f172c06ecd0 0x7f172c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:28.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.912+0000 7f1744c00700 1 -- 172.21.15.117:0/1315001254 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1730090340 con 0x7f1748104a00 2024-04-20T11:24:28.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.912+0000 7f1747e00700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f172c06ecd0 0x7f172c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:28.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.913+0000 7f1747e00700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f172c06ecd0 0x7f172c071190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1738005fd0 tx=0x7f1738005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:28.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:28.917+0000 7f1744c00700 1 -- 172.21.15.117:0/1315001254 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f173005cc30 con 0x7f1748104a00 2024-04-20T11:24:29.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.165+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1748199fe0 con 0x7f172c06ecd0 2024-04-20T11:24:29.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.168+0000 7f1744c00700 1 -- 172.21.15.117:0/1315001254 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1748199fe0 con 0x7f172c06ecd0 2024-04-20T11:24:29.167 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:24:29.167 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:24:29.168 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:24:29.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f172c06ecd0 msgr2=0x7f172c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:29.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f172c06ecd0 0x7f172c071190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1738005fd0 tx=0x7f1738005dc0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 msgr2=0x7f174819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 0x7f174819ef90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f17300092e0 tx=0x7f173000be20 comp rx=0 tx=0).stop 2024-04-20T11:24:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 shutdown_connections 2024-04-20T11:24:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f172c06ecd0 0x7f172c071190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1748103110 0x7f174819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 --2- 172.21.15.117:0/1315001254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1748104a00 0x7f174819ef90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 >> 172.21.15.117:0/1315001254 conn(0x7f17480fe880 msgr2=0x7f17480ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 shutdown_connections 2024-04-20T11:24:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.171+0000 7f174f000700 1 -- 172.21.15.117:0/1315001254 wait complete. 2024-04-20T11:24:29.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.325+0000 7f2179c00700 1 -- 172.21.15.117:0/1008361055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 msgr2=0x7f2174107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:29.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.325+0000 7f2179c00700 1 --2- 172.21.15.117:0/1008361055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 0x7f2174107900 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2164009910 tx=0x7f2164009c20 comp rx=0 tx=0).stop 2024-04-20T11:24:29.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.326+0000 7f2179c00700 1 -- 172.21.15.117:0/1008361055 shutdown_connections 2024-04-20T11:24:29.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.326+0000 7f2179c00700 1 --2- 172.21.15.117:0/1008361055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2174108e10 0x7f2174109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.326+0000 7f2179c00700 1 --2- 172.21.15.117:0/1008361055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 0x7f2174107900 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.326+0000 7f2179c00700 1 -- 172.21.15.117:0/1008361055 >> 172.21.15.117:0/1008361055 conn(0x7f21740768f0 msgr2=0x7f2174078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:29.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.326+0000 7f2179c00700 1 -- 172.21.15.117:0/1008361055 shutdown_connections 2024-04-20T11:24:29.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.326+0000 7f2179c00700 1 -- 172.21.15.117:0/1008361055 wait complete. 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.327+0000 7f2179c00700 1 Processor -- start 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.327+0000 7f2179c00700 1 -- start start 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.327+0000 7f2179c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 0x7f217419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.327+0000 7f2179c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2174108e10 0x7f217419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.327+0000 7f2179c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217419ddd0 con 0x7f2174107520 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.327+0000 7f2179c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21741a1aa0 con 0x7f2174108e10 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.327+0000 7f2172a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 0x7f217419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.328+0000 7f2172a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 0x7f217419d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43728/0 (socket says 172.21.15.117:43728) 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.328+0000 7f2172a00700 1 -- 172.21.15.117:0/207432531 learned_addr learned my addr 172.21.15.117:0/207432531 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:24:29.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.328+0000 7f2172000700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2174108e10 0x7f217419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:29.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.328+0000 7f2172a00700 1 -- 172.21.15.117:0/207432531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2174108e10 msgr2=0x7f217419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:29.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.328+0000 7f2172a00700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2174108e10 0x7f217419d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.328+0000 7f2172a00700 1 -- 172.21.15.117:0/207432531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640095f0 con 0x7f2174107520 2024-04-20T11:24:29.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.328+0000 7f2172a00700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 0x7f217419d1b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2164006010 tx=0x7f216400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:29.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.329+0000 7f216b400700 1 -- 172.21.15.117:0/207432531 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f216401d070 con 0x7f2174107520 2024-04-20T11:24:29.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.329+0000 7f216b400700 1 -- 172.21.15.117:0/207432531 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f216400f460 con 0x7f2174107520 2024-04-20T11:24:29.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.329+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21741a1d20 con 0x7f2174107520 2024-04-20T11:24:29.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.329+0000 7f216b400700 1 -- 172.21.15.117:0/207432531 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2164017630 con 0x7f2174107520 2024-04-20T11:24:29.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.329+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21741a2290 con 0x7f2174107520 2024-04-20T11:24:29.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.330+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217410f890 con 0x7f2174107520 2024-04-20T11:24:29.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.331+0000 7f216b400700 1 -- 172.21.15.117:0/207432531 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f216400f5d0 con 0x7f2174107520 2024-04-20T11:24:29.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.332+0000 7f216b400700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f215806ecd0 0x7f2158071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:29.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.332+0000 7f216b400700 1 -- 172.21.15.117:0/207432531 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2164090310 con 0x7f2174107520 2024-04-20T11:24:29.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.332+0000 7f2172000700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f215806ecd0 0x7f2158071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:29.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.333+0000 7f2172000700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f215806ecd0 0x7f2158071190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2160005fd0 tx=0x7f216000a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:29.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.337+0000 7f216b400700 1 -- 172.21.15.117:0/207432531 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2164059190 con 0x7f2174107520 2024-04-20T11:24:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:29 smithi117 ceph-mon[99036]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:24:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:29 smithi117 ceph-mon[99036]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:24:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:29 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/852763280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:24:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:29 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1167086593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:24:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.649+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f217419e4c0 con 0x7f2174107520 2024-04-20T11:24:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.649+0000 7f216b400700 1 -- 172.21.15.117:0/207432531 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f216405c850 con 0x7f2174107520 2024-04-20T11:24:29.649 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:24:29.649 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:24:29.649 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:24:29.649 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:24:29.649 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:24:29.649 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:24:29.649 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:24:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.652+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f215806ecd0 msgr2=0x7f2158071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.652+0000 7f2179c00700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f215806ecd0 0x7f2158071190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2160005fd0 tx=0x7f216000a560 comp rx=0 tx=0).stop 2024-04-20T11:24:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.652+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 msgr2=0x7f217419d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.652+0000 7f2179c00700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 0x7f217419d1b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2164006010 tx=0x7f216400be20 comp rx=0 tx=0).stop 2024-04-20T11:24:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.652+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 shutdown_connections 2024-04-20T11:24:29.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.653+0000 7f2179c00700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f215806ecd0 0x7f2158071190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.653+0000 7f2179c00700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2174107520 0x7f217419d1b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.653+0000 7f2179c00700 1 --2- 172.21.15.117:0/207432531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2174108e10 0x7f217419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:29.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.653+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 >> 172.21.15.117:0/207432531 conn(0x7f21740768f0 msgr2=0x7f2174077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:29.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.653+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 shutdown_connections 2024-04-20T11:24:29.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:29.653+0000 7f2179c00700 1 -- 172.21.15.117:0/207432531 wait complete. 2024-04-20T11:24:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:29 smithi173 ceph-mon[74165]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:24:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:29 smithi173 ceph-mon[74165]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:24:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:29 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/852763280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:24:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:29 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1167086593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:24:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:30 smithi117 ceph-mon[99036]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:24:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:30 smithi117 ceph-mon[99036]: pgmap v361: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:30 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/207432531' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:24:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:30 smithi173 ceph-mon[74165]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:24:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:30 smithi173 ceph-mon[74165]: pgmap v361: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:30 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/207432531' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:24:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:32 smithi117 ceph-mon[99036]: pgmap v362: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:32.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:32 smithi173 ceph-mon[74165]: pgmap v362: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:33.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:33 smithi117 ceph-mon[99036]: pgmap v363: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:33 smithi173 ceph-mon[74165]: pgmap v363: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:36 smithi173 ceph-mon[74165]: pgmap v364: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:36 smithi117 ceph-mon[99036]: pgmap v364: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:38.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:38 smithi117 ceph-mon[99036]: pgmap v365: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:38 smithi173 ceph-mon[74165]: pgmap v365: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:40.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:40 smithi117 ceph-mon[99036]: pgmap v366: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:40 smithi173 ceph-mon[74165]: pgmap v366: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:42 smithi117 ceph-mon[99036]: pgmap v367: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:42 smithi173 ceph-mon[74165]: pgmap v367: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:24:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:24:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:44 smithi117 ceph-mon[99036]: pgmap v368: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:44 smithi173 ceph-mon[74165]: pgmap v368: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:45 smithi117 ceph-mon[99036]: pgmap v369: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:45 smithi173 ceph-mon[74165]: pgmap v369: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:48.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:48 smithi117 ceph-mon[99036]: pgmap v370: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:48 smithi173 ceph-mon[74165]: pgmap v370: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:50.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:50 smithi117 ceph-mon[99036]: pgmap v371: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:50 smithi173 ceph-mon[74165]: pgmap v371: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:52.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:52 smithi117 ceph-mon[99036]: pgmap v372: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:52.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:52 smithi173 ceph-mon[74165]: pgmap v372: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:54 smithi117 ceph-mon[99036]: pgmap v373: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:54 smithi173 ceph-mon[74165]: pgmap v373: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:55.525 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:55 smithi117 ceph-mon[99036]: pgmap v374: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:55 smithi173 ceph-mon[74165]: pgmap v374: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:58 smithi117 ceph-mon[99036]: pgmap v375: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:24:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:24:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:58 smithi173 ceph-mon[74165]: pgmap v375: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:24:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:24:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:24:59.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.820+0000 7f93fa400700 1 -- 172.21.15.117:0/636895463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 msgr2=0x7f93f4106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.820+0000 7f93fa400700 1 --2- 172.21.15.117:0/636895463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 0x7f93f4106db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009910 tx=0x7f93e4009c20 comp rx=0 tx=0).stop 2024-04-20T11:24:59.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.820+0000 7f93fa400700 1 -- 172.21.15.117:0/636895463 shutdown_connections 2024-04-20T11:24:59.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.820+0000 7f93fa400700 1 --2- 172.21.15.117:0/636895463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93f41009d0 0x7f93f4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.820+0000 7f93fa400700 1 --2- 172.21.15.117:0/636895463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 0x7f93f4106db0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.820+0000 7f93fa400700 1 -- 172.21.15.117:0/636895463 >> 172.21.15.117:0/636895463 conn(0x7f93f40fc500 msgr2=0x7f93f40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.820+0000 7f93fa400700 1 -- 172.21.15.117:0/636895463 shutdown_connections 2024-04-20T11:24:59.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.820+0000 7f93fa400700 1 -- 172.21.15.117:0/636895463 wait complete. 2024-04-20T11:24:59.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.821+0000 7f93fa400700 1 Processor -- start 2024-04-20T11:24:59.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.822+0000 7f93fa400700 1 -- start start 2024-04-20T11:24:59.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.822+0000 7f93fa400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93f41009d0 0x7f93f419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.822+0000 7f93fa400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 0x7f93f419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.822+0000 7f93fa400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419f3a0 con 0x7f93f41069d0 2024-04-20T11:24:59.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.822+0000 7f93fa400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4198980 con 0x7f93f41009d0 2024-04-20T11:24:59.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.822+0000 7f93ebe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 0x7f93f419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.822+0000 7f93ebe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 0x7f93f419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51100/0 (socket says 172.21.15.117:51100) 2024-04-20T11:24:59.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.822+0000 7f93ebe00700 1 -- 172.21.15.117:0/2128237763 learned_addr learned my addr 172.21.15.117:0/2128237763 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:24:59.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93f9000700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93f41009d0 0x7f93f419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93ebe00700 1 -- 172.21.15.117:0/2128237763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93f41009d0 msgr2=0x7f93f419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93ebe00700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93f41009d0 0x7f93f419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93ebe00700 1 -- 172.21.15.117:0/2128237763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40095f0 con 0x7f93f41069d0 2024-04-20T11:24:59.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93ebe00700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 0x7f93f419ed80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c8a0 tx=0x7f93e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93f2000700 1 -- 172.21.15.117:0/2128237763 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93e000cea0 con 0x7f93f41069d0 2024-04-20T11:24:59.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93f2000700 1 -- 172.21.15.117:0/2128237763 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93e0004830 con 0x7f93f41069d0 2024-04-20T11:24:59.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f4198c60 con 0x7f93f41069d0 2024-04-20T11:24:59.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93f2000700 1 -- 172.21.15.117:0/2128237763 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93e00055b0 con 0x7f93f41069d0 2024-04-20T11:24:59.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.823+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f41991b0 con 0x7f93f41069d0 2024-04-20T11:24:59.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.825+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4104c80 con 0x7f93f41069d0 2024-04-20T11:24:59.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.826+0000 7f93f2000700 1 -- 172.21.15.117:0/2128237763 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f93e00049a0 con 0x7f93f41069d0 2024-04-20T11:24:59.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.826+0000 7f93f2000700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93d806ecc0 0x7f93d8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.827+0000 7f93f2000700 1 -- 172.21.15.117:0/2128237763 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f93e008eac0 con 0x7f93f41069d0 2024-04-20T11:24:59.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.827+0000 7f93f9000700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93d806ecc0 0x7f93d8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.827+0000 7f93f9000700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93d806ecc0 0x7f93d8071180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b5c0 tx=0x7f93e4005380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:24:59.832+0000 7f93f2000700 1 -- 172.21.15.117:0/2128237763 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f93e005b3b0 con 0x7f93f41069d0 2024-04-20T11:25:00.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.082+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93f410dbd0 con 0x7f93d806ecc0 2024-04-20T11:25:00.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.083+0000 7f93f2000700 1 -- 172.21.15.117:0/2128237763 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f93f410dbd0 con 0x7f93d806ecc0 2024-04-20T11:25:00.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93d806ecc0 msgr2=0x7f93d8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93d806ecc0 0x7f93d8071180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b5c0 tx=0x7f93e4005380 comp rx=0 tx=0).stop 2024-04-20T11:25:00.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 msgr2=0x7f93f419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 0x7f93f419ed80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c8a0 tx=0x7f93e000cc60 comp rx=0 tx=0).stop 2024-04-20T11:25:00.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 shutdown_connections 2024-04-20T11:25:00.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93d806ecc0 0x7f93d8071180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93f41009d0 0x7f93f419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 --2- 172.21.15.117:0/2128237763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f41069d0 0x7f93f419ed80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 >> 172.21.15.117:0/2128237763 conn(0x7f93f40fc500 msgr2=0x7f93f40733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:00.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 shutdown_connections 2024-04-20T11:25:00.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.087+0000 7f93fa400700 1 -- 172.21.15.117:0/2128237763 wait complete. 2024-04-20T11:25:00.104 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:25:00.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.250+0000 7f744ee00700 1 -- 172.21.15.117:0/124632315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 msgr2=0x7f7448105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.250+0000 7f744ee00700 1 --2- 172.21.15.117:0/124632315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 0x7f7448105820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7434009910 tx=0x7f7434009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:00.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.250+0000 7f744ee00700 1 -- 172.21.15.117:0/124632315 shutdown_connections 2024-04-20T11:25:00.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.250+0000 7f744ee00700 1 --2- 172.21.15.117:0/124632315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 0x7f7448105820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.250+0000 7f744ee00700 1 --2- 172.21.15.117:0/124632315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7448100e20 0x7f7448101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.250+0000 7f744ee00700 1 -- 172.21.15.117:0/124632315 >> 172.21.15.117:0/124632315 conn(0x7f74480fc6f0 msgr2=0x7f74480feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:00.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.250+0000 7f744ee00700 1 -- 172.21.15.117:0/124632315 shutdown_connections 2024-04-20T11:25:00.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.250+0000 7f744ee00700 1 -- 172.21.15.117:0/124632315 wait complete. 2024-04-20T11:25:00.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.251+0000 7f744ee00700 1 Processor -- start 2024-04-20T11:25:00.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.251+0000 7f744ee00700 1 -- start start 2024-04-20T11:25:00.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.252+0000 7f744ee00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7448100e20 0x7f74480731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.252+0000 7f744ee00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 0x7f7448073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.252+0000 7f744ee00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448073c60 con 0x7f74481017d0 2024-04-20T11:25:00.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.252+0000 7f744ee00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448073da0 con 0x7f7448100e20 2024-04-20T11:25:00.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.252+0000 7f7447400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 0x7f7448073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.252+0000 7f7447400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 0x7f7448073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51112/0 (socket says 172.21.15.117:51112) 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.252+0000 7f7447400700 1 -- 172.21.15.117:0/2191028669 learned_addr learned my addr 172.21.15.117:0/2191028669 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.252+0000 7f7447e00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7448100e20 0x7f74480731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f7447400700 1 -- 172.21.15.117:0/2191028669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7448100e20 msgr2=0x7f74480731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f7447400700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7448100e20 0x7f74480731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f7447400700 1 -- 172.21.15.117:0/2191028669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340095f0 con 0x7f74481017d0 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f7447400700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 0x7f7448073720 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7434005b40 tx=0x7f743400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f7444c00700 1 -- 172.21.15.117:0/2191028669 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743401d070 con 0x7f74481017d0 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f7444c00700 1 -- 172.21.15.117:0/2191028669 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f743400f460 con 0x7f74481017d0 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f7444c00700 1 -- 172.21.15.117:0/2191028669 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7434017630 con 0x7f74481017d0 2024-04-20T11:25:00.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7448074020 con 0x7f74481017d0 2024-04-20T11:25:00.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.253+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74481a2f40 con 0x7f74481017d0 2024-04-20T11:25:00.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.255+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7448109220 con 0x7f74481017d0 2024-04-20T11:25:00.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.256+0000 7f7444c00700 1 -- 172.21.15.117:0/2191028669 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7434003a60 con 0x7f74481017d0 2024-04-20T11:25:00.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.257+0000 7f7444c00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f742c06ecc0 0x7f742c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.258+0000 7f7444c00700 1 -- 172.21.15.117:0/2191028669 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f74340902a0 con 0x7f74481017d0 2024-04-20T11:25:00.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.258+0000 7f7447e00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f742c06ecc0 0x7f742c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.258+0000 7f7447e00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f742c06ecc0 0x7f742c071180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7438005fd0 tx=0x7f743800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:00.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.262+0000 7f7444c00700 1 -- 172.21.15.117:0/2191028669 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7434059060 con 0x7f74481017d0 2024-04-20T11:25:00.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:00 smithi117 ceph-mon[99036]: pgmap v376: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:00.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.510+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f744810ddc0 con 0x7f742c06ecc0 2024-04-20T11:25:00.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.512+0000 7f7444c00700 1 -- 172.21.15.117:0/2191028669 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f744810ddc0 con 0x7f742c06ecc0 2024-04-20T11:25:00.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f742c06ecc0 msgr2=0x7f742c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f742c06ecc0 0x7f742c071180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7438005fd0 tx=0x7f743800a560 comp rx=0 tx=0).stop 2024-04-20T11:25:00.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 msgr2=0x7f7448073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 0x7f7448073720 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7434005b40 tx=0x7f743400be20 comp rx=0 tx=0).stop 2024-04-20T11:25:00.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 shutdown_connections 2024-04-20T11:25:00.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f742c06ecc0 0x7f742c071180 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7448100e20 0x7f74480731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 --2- 172.21.15.117:0/2191028669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74481017d0 0x7f7448073720 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 >> 172.21.15.117:0/2191028669 conn(0x7f74480fc6f0 msgr2=0x7f74480fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:00.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 shutdown_connections 2024-04-20T11:25:00.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.515+0000 7f744ee00700 1 -- 172.21.15.117:0/2191028669 wait complete. 2024-04-20T11:25:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:00 smithi173 ceph-mon[74165]: pgmap v376: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:00.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.668+0000 7f92b8000700 1 -- 172.21.15.117:0/407292423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b0073bd0 msgr2=0x7f92b010cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.668+0000 7f92b8000700 1 --2- 172.21.15.117:0/407292423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b0073bd0 0x7f92b010cf10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f929c009910 tx=0x7f929c009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:00.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.668+0000 7f92b8000700 1 -- 172.21.15.117:0/407292423 shutdown_connections 2024-04-20T11:25:00.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.668+0000 7f92b8000700 1 --2- 172.21.15.117:0/407292423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b0073bd0 0x7f92b010cf10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.668+0000 7f92b8000700 1 --2- 172.21.15.117:0/407292423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b00732b0 0x7f92b0073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.668+0000 7f92b8000700 1 -- 172.21.15.117:0/407292423 >> 172.21.15.117:0/407292423 conn(0x7f92b00fc6a0 msgr2=0x7f92b00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:00.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.668+0000 7f92b8000700 1 -- 172.21.15.117:0/407292423 shutdown_connections 2024-04-20T11:25:00.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.669+0000 7f92b8000700 1 -- 172.21.15.117:0/407292423 wait complete. 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.669+0000 7f92b8000700 1 Processor -- start 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.670+0000 7f92b8000700 1 -- start start 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.670+0000 7f92b8000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00732b0 0x7f92b0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.670+0000 7f92b8000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b0073bd0 0x7f92b0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.670+0000 7f92b8000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b0199a30 con 0x7f92b00732b0 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.670+0000 7f92b8000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b019d700 con 0x7f92b0073bd0 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.670+0000 7f92b5600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00732b0 0x7f92b0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.670+0000 7f92b5600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00732b0 0x7f92b0198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51134/0 (socket says 172.21.15.117:51134) 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.670+0000 7f92b5600700 1 -- 172.21.15.117:0/2632199732 learned_addr learned my addr 172.21.15.117:0/2632199732 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92b4c00700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b0073bd0 0x7f92b0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92b5600700 1 -- 172.21.15.117:0/2632199732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b0073bd0 msgr2=0x7f92b0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92b5600700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b0073bd0 0x7f92b0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92b5600700 1 -- 172.21.15.117:0/2632199732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c0095f0 con 0x7f92b00732b0 2024-04-20T11:25:00.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92b5600700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00732b0 0x7f92b0198e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b4b0 tx=0x7f92a400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:00.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92ae000700 1 -- 172.21.15.117:0/2632199732 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a4011800 con 0x7f92b00732b0 2024-04-20T11:25:00.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92ae000700 1 -- 172.21.15.117:0/2632199732 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92a4011e20 con 0x7f92b00732b0 2024-04-20T11:25:00.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b019d9e0 con 0x7f92b00732b0 2024-04-20T11:25:00.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92ae000700 1 -- 172.21.15.117:0/2632199732 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a400f2e0 con 0x7f92b00732b0 2024-04-20T11:25:00.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.671+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b019df30 con 0x7f92b00732b0 2024-04-20T11:25:00.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.673+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b0074960 con 0x7f92b00732b0 2024-04-20T11:25:00.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.674+0000 7f92ae000700 1 -- 172.21.15.117:0/2632199732 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f92a4010380 con 0x7f92b00732b0 2024-04-20T11:25:00.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.674+0000 7f92ae000700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f929806ecd0 0x7f9298071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.675+0000 7f92ae000700 1 -- 172.21.15.117:0/2632199732 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f92a408e8f0 con 0x7f92b00732b0 2024-04-20T11:25:00.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.675+0000 7f92b4c00700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f929806ecd0 0x7f9298071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.675+0000 7f92b4c00700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f929806ecd0 0x7f9298071190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f929c006010 tx=0x7f929c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:00.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.679+0000 7f92ae000700 1 -- 172.21.15.117:0/2632199732 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f92a4057750 con 0x7f92b00732b0 2024-04-20T11:25:00.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.914+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92b01077b0 con 0x7f929806ecd0 2024-04-20T11:25:00.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.946+0000 7f92ae000700 1 -- 172.21.15.117:0/2632199732 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f92b01077b0 con 0x7f929806ecd0 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (13m) 5m ago 27m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (27m) 5m ago 27m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (26m) 4m ago 26m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (9m) 5m ago 27m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (9m) 4m ago 26m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (12m) 5m ago 27m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (5m) 5m ago 20m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (5m) 5m ago 20m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (20m) 4m ago 20m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (4m) 4m ago 20m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:25:00.947 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (16m) 5m ago 29m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (15m) 4m ago 26m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (11m) 5m ago 29m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (10m) 4m ago 26m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (14m) 5m ago 27m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (14m) 4m ago 26m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (9m) 5m ago 25m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (8m) 5m ago 24m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (8m) 5m ago 23m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (7m) 4m ago 23m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (6m) 4m ago 22m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (6m) 4m ago 22m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:25:00.948 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (14m) 5m ago 26m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:25:00.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f929806ecd0 msgr2=0x7f9298071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f929806ecd0 0x7f9298071190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f929c006010 tx=0x7f929c00b540 comp rx=0 tx=0).stop 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00732b0 msgr2=0x7f92b0198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00732b0 0x7f92b0198e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b4b0 tx=0x7f92a400b870 comp rx=0 tx=0).stop 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 shutdown_connections 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f929806ecd0 0x7f9298071190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00732b0 0x7f92b0198e10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 --2- 172.21.15.117:0/2632199732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b0073bd0 0x7f92b0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 >> 172.21.15.117:0/2632199732 conn(0x7f92b00fc6a0 msgr2=0x7f92b01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 shutdown_connections 2024-04-20T11:25:00.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:00.950+0000 7f92b8000700 1 -- 172.21.15.117:0/2632199732 wait complete. 2024-04-20T11:25:01.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.100+0000 7fa133800700 1 -- 172.21.15.117:0/2167078095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 msgr2=0x7fa12c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.100+0000 7fa133800700 1 --2- 172.21.15.117:0/2167078095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 0x7fa12c104e80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa118009910 tx=0x7fa118009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:01.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.100+0000 7fa133800700 1 -- 172.21.15.117:0/2167078095 shutdown_connections 2024-04-20T11:25:01.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.100+0000 7fa133800700 1 --2- 172.21.15.117:0/2167078095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 0x7fa12c104e80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.100+0000 7fa133800700 1 --2- 172.21.15.117:0/2167078095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12c103110 0x7fa12c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.100+0000 7fa133800700 1 -- 172.21.15.117:0/2167078095 >> 172.21.15.117:0/2167078095 conn(0x7fa12c0fe880 msgr2=0x7fa12c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.100+0000 7fa133800700 1 -- 172.21.15.117:0/2167078095 shutdown_connections 2024-04-20T11:25:01.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.100+0000 7fa133800700 1 -- 172.21.15.117:0/2167078095 wait complete. 2024-04-20T11:25:01.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.101+0000 7fa133800700 1 Processor -- start 2024-04-20T11:25:01.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.101+0000 7fa133800700 1 -- start start 2024-04-20T11:25:01.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.102+0000 7fa133800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12c103110 0x7fa12c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.102+0000 7fa133800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 0x7fa12c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.102+0000 7fa133800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c199ac0 con 0x7fa12c104a00 2024-04-20T11:25:01.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.102+0000 7fa133800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c19d740 con 0x7fa12c103110 2024-04-20T11:25:01.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.102+0000 7fa12be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 0x7fa12c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.102+0000 7fa12be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 0x7fa12c1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51156/0 (socket says 172.21.15.117:51156) 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.102+0000 7fa12be00700 1 -- 172.21.15.117:0/1895401887 learned_addr learned my addr 172.21.15.117:0/1895401887 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.102+0000 7fa130e00700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12c103110 0x7fa12c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa12be00700 1 -- 172.21.15.117:0/1895401887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12c103110 msgr2=0x7fa12c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa12be00700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12c103110 0x7fa12c198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa12be00700 1 -- 172.21.15.117:0/1895401887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180095f0 con 0x7fa12c104a00 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa12be00700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 0x7fa12c1993e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa1180092e0 tx=0x7fa11800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa129600700 1 -- 172.21.15.117:0/1895401887 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa11801d070 con 0x7fa12c104a00 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa129600700 1 -- 172.21.15.117:0/1895401887 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa11800f460 con 0x7fa12c104a00 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c19d9c0 con 0x7fa12c104a00 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa129600700 1 -- 172.21.15.117:0/1895401887 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa118017630 con 0x7fa12c104a00 2024-04-20T11:25:01.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.103+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c19df30 con 0x7fa12c104a00 2024-04-20T11:25:01.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.104+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c10b480 con 0x7fa12c104a00 2024-04-20T11:25:01.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.105+0000 7fa129600700 1 -- 172.21.15.117:0/1895401887 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa11800f5d0 con 0x7fa12c104a00 2024-04-20T11:25:01.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.107+0000 7fa129600700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa11406ecd0 0x7fa114071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.107+0000 7fa129600700 1 -- 172.21.15.117:0/1895401887 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa1180903e0 con 0x7fa12c104a00 2024-04-20T11:25:01.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.107+0000 7fa130e00700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa11406ecd0 0x7fa114071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.108+0000 7fa130e00700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa11406ecd0 0x7fa114071190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.112+0000 7fa129600700 1 -- 172.21.15.117:0/1895401887 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa118059120 con 0x7fa12c104a00 2024-04-20T11:25:01.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.430+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa12c19a160 con 0x7fa12c104a00 2024-04-20T11:25:01.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.431+0000 7fa129600700 1 -- 172.21.15.117:0/1895401887 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fa11805c7e0 con 0x7fa12c104a00 2024-04-20T11:25:01.431 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:25:01.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:25:01.433 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:25:01.433 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:25:01.433 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:25:01.433 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:25:01.433 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:25:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa11406ecd0 msgr2=0x7fa114071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa11406ecd0 0x7fa114071190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120008040 comp rx=0 tx=0).stop 2024-04-20T11:25:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 msgr2=0x7fa12c1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 0x7fa12c1993e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa1180092e0 tx=0x7fa11800be20 comp rx=0 tx=0).stop 2024-04-20T11:25:01.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 shutdown_connections 2024-04-20T11:25:01.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa11406ecd0 0x7fa114071190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12c103110 0x7fa12c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 --2- 172.21.15.117:0/1895401887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa12c104a00 0x7fa12c1993e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 >> 172.21.15.117:0/1895401887 conn(0x7fa12c0fe880 msgr2=0x7fa12c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 shutdown_connections 2024-04-20T11:25:01.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.434+0000 7fa133800700 1 -- 172.21.15.117:0/1895401887 wait complete. 2024-04-20T11:25:01.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.577+0000 7f9261e00700 1 -- 172.21.15.117:0/4055109426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 msgr2=0x7f925c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.577+0000 7f9261e00700 1 --2- 172.21.15.117:0/4055109426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 0x7f925c10be40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9244009910 tx=0x7f9244009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:01.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.577+0000 7f9261e00700 1 -- 172.21.15.117:0/4055109426 shutdown_connections 2024-04-20T11:25:01.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.577+0000 7f9261e00700 1 --2- 172.21.15.117:0/4055109426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 0x7f925c10be40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.577+0000 7f9261e00700 1 --2- 172.21.15.117:0/4055109426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f925c107440 0x7f925c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.577+0000 7f9261e00700 1 -- 172.21.15.117:0/4055109426 >> 172.21.15.117:0/4055109426 conn(0x7f925c0769f0 msgr2=0x7f925c078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.579 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:01 smithi117 ceph-mon[99036]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.577+0000 7f9261e00700 1 -- 172.21.15.117:0/4055109426 shutdown_connections 2024-04-20T11:25:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.578+0000 7f9261e00700 1 -- 172.21.15.117:0/4055109426 wait complete. 2024-04-20T11:25:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.578+0000 7f9261e00700 1 Processor -- start 2024-04-20T11:25:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.579+0000 7f9261e00700 1 -- start start 2024-04-20T11:25:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.579+0000 7f9261e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f925c107440 0x7f925c19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.579+0000 7f9261e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 0x7f925c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.579+0000 7f9261e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c19de50 con 0x7f925c107df0 2024-04-20T11:25:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.579+0000 7f9261e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c1a1b20 con 0x7f925c107440 2024-04-20T11:25:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.579+0000 7f9252000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 0x7f925c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.579+0000 7f9252000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 0x7f925c19d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51170/0 (socket says 172.21.15.117:51170) 2024-04-20T11:25:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.579+0000 7f9252000700 1 -- 172.21.15.117:0/219108379 learned_addr learned my addr 172.21.15.117:0/219108379 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.580+0000 7f925aa00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f925c107440 0x7f925c19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.580+0000 7f9252000700 1 -- 172.21.15.117:0/219108379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f925c107440 msgr2=0x7f925c19d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.580+0000 7f9252000700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f925c107440 0x7f925c19d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.580+0000 7f9252000700 1 -- 172.21.15.117:0/219108379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92440095f0 con 0x7f925c107df0 2024-04-20T11:25:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.580+0000 7f9252000700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 0x7f925c19d770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92440092e0 tx=0x7f924400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.581+0000 7f9260a00700 1 -- 172.21.15.117:0/219108379 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f924401d070 con 0x7f925c107df0 2024-04-20T11:25:01.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.581+0000 7f9260a00700 1 -- 172.21.15.117:0/219108379 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f924400f460 con 0x7f925c107df0 2024-04-20T11:25:01.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.581+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c1a1da0 con 0x7f925c107df0 2024-04-20T11:25:01.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.581+0000 7f9260a00700 1 -- 172.21.15.117:0/219108379 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9244017610 con 0x7f925c107df0 2024-04-20T11:25:01.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.581+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c1a2290 con 0x7f925c107df0 2024-04-20T11:25:01.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.582+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c108920 con 0x7f925c107df0 2024-04-20T11:25:01.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.585+0000 7f9260a00700 1 -- 172.21.15.117:0/219108379 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f924400f5d0 con 0x7f925c107df0 2024-04-20T11:25:01.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.586+0000 7f9260a00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924006ecc0 0x7f9240071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.586+0000 7f9260a00700 1 -- 172.21.15.117:0/219108379 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9244090430 con 0x7f925c107df0 2024-04-20T11:25:01.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.586+0000 7f925aa00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924006ecc0 0x7f9240071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.587+0000 7f925aa00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924006ecc0 0x7f9240071180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f924c009f20 tx=0x7f924c0094f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.588+0000 7f9260a00700 1 -- 172.21.15.117:0/219108379 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9244059330 con 0x7f925c107df0 2024-04-20T11:25:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:01 smithi173 ceph-mon[74165]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:01.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.840+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f925c062510 con 0x7f925c107df0 2024-04-20T11:25:01.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.841+0000 7f9260a00700 1 -- 172.21.15.117:0/219108379 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f924405c9f0 con 0x7f925c107df0 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:25:01.842 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:25:01.843 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:25:01.844 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:25:01.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:25:01.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:25:01.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.844+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924006ecc0 msgr2=0x7f9240071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.844+0000 7f9261e00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924006ecc0 0x7f9240071180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f924c009f20 tx=0x7f924c0094f0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.844+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 msgr2=0x7f925c19d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.844+0000 7f9261e00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 0x7f925c19d770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92440092e0 tx=0x7f924400be20 comp rx=0 tx=0).stop 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.846+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 shutdown_connections 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.846+0000 7f9261e00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924006ecc0 0x7f9240071180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.846+0000 7f9261e00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f925c107440 0x7f925c19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.846+0000 7f9261e00700 1 --2- 172.21.15.117:0/219108379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f925c107df0 0x7f925c19d770 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.846+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 >> 172.21.15.117:0/219108379 conn(0x7f925c0769f0 msgr2=0x7f925c077d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.846+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 shutdown_connections 2024-04-20T11:25:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:01.846+0000 7f9261e00700 1 -- 172.21.15.117:0/219108379 wait complete. 2024-04-20T11:25:01.847 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:25:02.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.004+0000 7f99f2600700 1 -- 172.21.15.117:0/1076179236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec107520 msgr2=0x7f99ec107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:02.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.004+0000 7f99f2600700 1 --2- 172.21.15.117:0/1076179236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec107520 0x7f99ec107900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009910 tx=0x7f99dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:02.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.004+0000 7f99f2600700 1 -- 172.21.15.117:0/1076179236 shutdown_connections 2024-04-20T11:25:02.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.004+0000 7f99f2600700 1 --2- 172.21.15.117:0/1076179236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec108e10 0x7f99ec109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.004+0000 7f99f2600700 1 --2- 172.21.15.117:0/1076179236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec107520 0x7f99ec107900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.004+0000 7f99f2600700 1 -- 172.21.15.117:0/1076179236 >> 172.21.15.117:0/1076179236 conn(0x7f99ec0768f0 msgr2=0x7f99ec078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:02.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.004+0000 7f99f2600700 1 -- 172.21.15.117:0/1076179236 shutdown_connections 2024-04-20T11:25:02.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.004+0000 7f99f2600700 1 -- 172.21.15.117:0/1076179236 wait complete. 2024-04-20T11:25:02.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.005+0000 7f99f2600700 1 Processor -- start 2024-04-20T11:25:02.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.006+0000 7f99f2600700 1 -- start start 2024-04-20T11:25:02.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.006+0000 7f99f2600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec107520 0x7f99ec19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.006+0000 7f99f2600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec108e10 0x7f99ec19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.006+0000 7f99f2600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec19ddd0 con 0x7f99ec108e10 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.006+0000 7f99f2600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec1a1aa0 con 0x7f99ec107520 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.006+0000 7f99eaa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec108e10 0x7f99ec19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.006+0000 7f99eaa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec108e10 0x7f99ec19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51190/0 (socket says 172.21.15.117:51190) 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.006+0000 7f99eaa00700 1 -- 172.21.15.117:0/3964628612 learned_addr learned my addr 172.21.15.117:0/3964628612 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.007+0000 7f99eb400700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec107520 0x7f99ec19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.007+0000 7f99eaa00700 1 -- 172.21.15.117:0/3964628612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec107520 msgr2=0x7f99ec19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.007+0000 7f99eaa00700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec107520 0x7f99ec19d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.007+0000 7f99eaa00700 1 -- 172.21.15.117:0/3964628612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0095f0 con 0x7f99ec108e10 2024-04-20T11:25:02.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.007+0000 7f99eaa00700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec108e10 0x7f99ec19d6f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99d800d8d0 tx=0x7f99d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:02.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.007+0000 7f99e3e00700 1 -- 172.21.15.117:0/3964628612 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99d8009940 con 0x7f99ec108e10 2024-04-20T11:25:02.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.007+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ec1a1d80 con 0x7f99ec108e10 2024-04-20T11:25:02.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.007+0000 7f99e3e00700 1 -- 172.21.15.117:0/3964628612 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99d8010460 con 0x7f99ec108e10 2024-04-20T11:25:02.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.008+0000 7f99e3e00700 1 -- 172.21.15.117:0/3964628612 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99d800f5f0 con 0x7f99ec108e10 2024-04-20T11:25:02.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.008+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ec1a2350 con 0x7f99ec108e10 2024-04-20T11:25:02.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.009+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ec10f870 con 0x7f99ec108e10 2024-04-20T11:25:02.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.010+0000 7f99e3e00700 1 -- 172.21.15.117:0/3964628612 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f99d80105d0 con 0x7f99ec108e10 2024-04-20T11:25:02.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.011+0000 7f99e3e00700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99d006ecd0 0x7f99d0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:02.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.011+0000 7f99e3e00700 1 -- 172.21.15.117:0/3964628612 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f99d808ee50 con 0x7f99ec108e10 2024-04-20T11:25:02.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.012+0000 7f99eb400700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99d006ecd0 0x7f99d0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:02.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.012+0000 7f99eb400700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99d006ecd0 0x7f99d0071190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b5c0 tx=0x7f99dc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:02.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.016+0000 7f99e3e00700 1 -- 172.21.15.117:0/3964628612 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f99d8057c30 con 0x7f99ec108e10 2024-04-20T11:25:02.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.263+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99ec19e4c0 con 0x7f99d006ecd0 2024-04-20T11:25:02.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.266+0000 7f99e3e00700 1 -- 172.21.15.117:0/3964628612 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f99ec19e4c0 con 0x7f99d006ecd0 2024-04-20T11:25:02.265 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:25:02.266 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:25:02.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99d006ecd0 msgr2=0x7f99d0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99d006ecd0 0x7f99d0071190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b5c0 tx=0x7f99dc01a040 comp rx=0 tx=0).stop 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec108e10 msgr2=0x7f99ec19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec108e10 0x7f99ec19d6f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99d800d8d0 tx=0x7f99d800dc90 comp rx=0 tx=0).stop 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 shutdown_connections 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99d006ecd0 0x7f99d0071190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec107520 0x7f99ec19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 --2- 172.21.15.117:0/3964628612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ec108e10 0x7f99ec19d6f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 >> 172.21.15.117:0/3964628612 conn(0x7f99ec0768f0 msgr2=0x7f99ec077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 shutdown_connections 2024-04-20T11:25:02.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.269+0000 7f99f2600700 1 -- 172.21.15.117:0/3964628612 wait complete. 2024-04-20T11:25:02.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.426+0000 7f3cc9600700 1 -- 172.21.15.117:0/2078771209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 msgr2=0x7f3cc4103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:02.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.426+0000 7f3cc9600700 1 --2- 172.21.15.117:0/2078771209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 0x7f3cc4103240 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009910 tx=0x7f3cb4009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:02.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.426+0000 7f3cc9600700 1 -- 172.21.15.117:0/2078771209 shutdown_connections 2024-04-20T11:25:02.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.426+0000 7f3cc9600700 1 --2- 172.21.15.117:0/2078771209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc4103810 0x7f3cc4107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.426+0000 7f3cc9600700 1 --2- 172.21.15.117:0/2078771209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 0x7f3cc4103240 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.426+0000 7f3cc9600700 1 -- 172.21.15.117:0/2078771209 >> 172.21.15.117:0/2078771209 conn(0x7f3cc40fe710 msgr2=0x7f3cc4100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:02.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.427+0000 7f3cc9600700 1 -- 172.21.15.117:0/2078771209 shutdown_connections 2024-04-20T11:25:02.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.427+0000 7f3cc9600700 1 -- 172.21.15.117:0/2078771209 wait complete. 2024-04-20T11:25:02.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.428+0000 7f3cc9600700 1 Processor -- start 2024-04-20T11:25:02.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.428+0000 7f3cc9600700 1 -- start start 2024-04-20T11:25:02.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.428+0000 7f3cc9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 0x7f3cc419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:02.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.428+0000 7f3cc9600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc4103810 0x7f3cc419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:02.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.428+0000 7f3cc9600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc419f4c0 con 0x7f3cc4102e60 2024-04-20T11:25:02.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.428+0000 7f3cc9600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4198aa0 con 0x7f3cc4103810 2024-04-20T11:25:02.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.428+0000 7f3cc3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 0x7f3cc419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:02.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.428+0000 7f3cc3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 0x7f3cc419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51218/0 (socket says 172.21.15.117:51218) 2024-04-20T11:25:02.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.429+0000 7f3cc3e00700 1 -- 172.21.15.117:0/3891506500 learned_addr learned my addr 172.21.15.117:0/3891506500 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:02.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.429+0000 7f3cc3400700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc4103810 0x7f3cc419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:02.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.429+0000 7f3cc3e00700 1 -- 172.21.15.117:0/3891506500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc4103810 msgr2=0x7f3cc419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:02.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.429+0000 7f3cc3e00700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc4103810 0x7f3cc419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.429+0000 7f3cc3e00700 1 -- 172.21.15.117:0/3891506500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40095f0 con 0x7f3cc4102e60 2024-04-20T11:25:02.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.429+0000 7f3cc3e00700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 0x7f3cc419e960 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006010 tx=0x7f3cb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:02.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.430+0000 7f3cc0c00700 1 -- 172.21.15.117:0/3891506500 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb401d070 con 0x7f3cc4102e60 2024-04-20T11:25:02.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.430+0000 7f3cc0c00700 1 -- 172.21.15.117:0/3891506500 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cb400f460 con 0x7f3cc4102e60 2024-04-20T11:25:02.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.430+0000 7f3cc0c00700 1 -- 172.21.15.117:0/3891506500 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb4017630 con 0x7f3cc4102e60 2024-04-20T11:25:02.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.430+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc4198d20 con 0x7f3cc4102e60 2024-04-20T11:25:02.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.430+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc4199210 con 0x7f3cc4102e60 2024-04-20T11:25:02.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.431+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc410b200 con 0x7f3cc4102e60 2024-04-20T11:25:02.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.432+0000 7f3cc0c00700 1 -- 172.21.15.117:0/3891506500 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3cb400f5d0 con 0x7f3cc4102e60 2024-04-20T11:25:02.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.433+0000 7f3cc0c00700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ca806ecd0 0x7f3ca8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:02.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.433+0000 7f3cc0c00700 1 -- 172.21.15.117:0/3891506500 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3cb4090350 con 0x7f3cc4102e60 2024-04-20T11:25:02.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.433+0000 7f3cc3400700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ca806ecd0 0x7f3ca8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:02.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.434+0000 7f3cc3400700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ca806ecd0 0x7f3ca8071190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0006fd0 tx=0x7f3cb0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:02.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.438+0000 7f3cc0c00700 1 -- 172.21.15.117:0/3891506500 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3cb405cd00 con 0x7f3cc4102e60 2024-04-20T11:25:02.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:02 smithi117 ceph-mon[99036]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:02.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:02 smithi117 ceph-mon[99036]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:02 smithi117 ceph-mon[99036]: pgmap v377: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:02 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1895401887' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:25:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:02 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/219108379' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:25:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:02 smithi173 ceph-mon[74165]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:02 smithi173 ceph-mon[74165]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:02 smithi173 ceph-mon[74165]: pgmap v377: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:02 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1895401887' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:25:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:02 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/219108379' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:25:02.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.747+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3cc4104340 con 0x7f3cc4102e60 2024-04-20T11:25:02.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.748+0000 7f3cc0c00700 1 -- 172.21.15.117:0/3891506500 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f3cb405c890 con 0x7f3cc4102e60 2024-04-20T11:25:02.747 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:25:02.747 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:25:02.748 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:25:02.748 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:25:02.748 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:25:02.748 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:25:02.748 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:25:02.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.750+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ca806ecd0 msgr2=0x7f3ca8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:02.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.750+0000 7f3cc9600700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ca806ecd0 0x7f3ca8071190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0006fd0 tx=0x7f3cb0008040 comp rx=0 tx=0).stop 2024-04-20T11:25:02.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.750+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 msgr2=0x7f3cc419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:02.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.750+0000 7f3cc9600700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 0x7f3cc419e960 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006010 tx=0x7f3cb400be20 comp rx=0 tx=0).stop 2024-04-20T11:25:02.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.751+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 shutdown_connections 2024-04-20T11:25:02.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.751+0000 7f3cc9600700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ca806ecd0 0x7f3ca8071190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.751+0000 7f3cc9600700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc4102e60 0x7f3cc419e960 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.751+0000 7f3cc9600700 1 --2- 172.21.15.117:0/3891506500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc4103810 0x7f3cc419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:02.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.751+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 >> 172.21.15.117:0/3891506500 conn(0x7f3cc40fe710 msgr2=0x7f3cc40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:02.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.751+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 shutdown_connections 2024-04-20T11:25:02.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:02.751+0000 7f3cc9600700 1 -- 172.21.15.117:0/3891506500 wait complete. 2024-04-20T11:25:03.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:03 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3891506500' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:25:03.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:03 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3891506500' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:25:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:04 smithi117 ceph-mon[99036]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:04 smithi117 ceph-mon[99036]: pgmap v378: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:04 smithi173 ceph-mon[74165]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:04 smithi173 ceph-mon[74165]: pgmap v378: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:05 smithi117 ceph-mon[99036]: pgmap v379: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:05 smithi173 ceph-mon[74165]: pgmap v379: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:08.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:08 smithi117 ceph-mon[99036]: pgmap v380: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:08 smithi173 ceph-mon[74165]: pgmap v380: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:10.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:10 smithi117 ceph-mon[99036]: pgmap v381: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:10 smithi173 ceph-mon[74165]: pgmap v381: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:12.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:12 smithi117 ceph-mon[99036]: pgmap v382: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:12 smithi173 ceph-mon[74165]: pgmap v382: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:25:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:25:14.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:14 smithi117 ceph-mon[99036]: pgmap v383: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:14 smithi173 ceph-mon[74165]: pgmap v383: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:15 smithi117 ceph-mon[99036]: pgmap v384: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:15 smithi173 ceph-mon[74165]: pgmap v384: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:18 smithi117 ceph-mon[99036]: pgmap v385: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:18.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:18 smithi173 ceph-mon[74165]: pgmap v385: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:20.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:20 smithi117 ceph-mon[99036]: pgmap v386: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:20 smithi173 ceph-mon[74165]: pgmap v386: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:22.592 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:22 smithi173 ceph-mon[74165]: pgmap v387: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:22.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:22 smithi117 ceph-mon[99036]: pgmap v387: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:23 smithi117 ceph-mon[99036]: pgmap v388: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:23 smithi173 ceph-mon[74165]: pgmap v388: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:26.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:26 smithi117 ceph-mon[99036]: pgmap v389: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:26 smithi173 ceph-mon[74165]: pgmap v389: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:28 smithi117 ceph-mon[99036]: pgmap v390: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:25:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:28 smithi173 ceph-mon[74165]: pgmap v390: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:25:30.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:30 smithi117 ceph-mon[99036]: pgmap v391: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:30 smithi173 ceph-mon[74165]: pgmap v391: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:31.603 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:31 smithi117 ceph-mon[99036]: pgmap v392: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:31 smithi173 ceph-mon[74165]: pgmap v392: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:32.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.920+0000 7fac45800700 1 -- 172.21.15.117:0/3280447422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac401017d0 msgr2=0x7fac40105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.920+0000 7fac45800700 1 --2- 172.21.15.117:0/3280447422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac401017d0 0x7fac40105820 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac28009910 tx=0x7fac28009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:32.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.920+0000 7fac45800700 1 -- 172.21.15.117:0/3280447422 shutdown_connections 2024-04-20T11:25:32.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.920+0000 7fac45800700 1 --2- 172.21.15.117:0/3280447422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac401017d0 0x7fac40105820 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.920+0000 7fac45800700 1 --2- 172.21.15.117:0/3280447422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac40100e20 0x7fac40101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.920+0000 7fac45800700 1 -- 172.21.15.117:0/3280447422 >> 172.21.15.117:0/3280447422 conn(0x7fac400fc6d0 msgr2=0x7fac400feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.920+0000 7fac45800700 1 -- 172.21.15.117:0/3280447422 shutdown_connections 2024-04-20T11:25:32.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.921+0000 7fac45800700 1 -- 172.21.15.117:0/3280447422 wait complete. 2024-04-20T11:25:32.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.922+0000 7fac45800700 1 Processor -- start 2024-04-20T11:25:32.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.922+0000 7fac45800700 1 -- start start 2024-04-20T11:25:32.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.922+0000 7fac45800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac40100e20 0x7fac4019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.922+0000 7fac45800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac401017d0 0x7fac4019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.922+0000 7fac45800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4019f550 con 0x7fac40100e20 2024-04-20T11:25:32.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.922+0000 7fac45800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac40198b30 con 0x7fac401017d0 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.922+0000 7fac3ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac40100e20 0x7fac4019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.923+0000 7fac3ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac40100e20 0x7fac4019e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56866/0 (socket says 172.21.15.117:56866) 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.923+0000 7fac3ea00700 1 -- 172.21.15.117:0/1954416925 learned_addr learned my addr 172.21.15.117:0/1954416925 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.923+0000 7fac36000700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac401017d0 0x7fac4019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.923+0000 7fac3ea00700 1 -- 172.21.15.117:0/1954416925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac401017d0 msgr2=0x7fac4019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.923+0000 7fac3ea00700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac401017d0 0x7fac4019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.923+0000 7fac3ea00700 1 -- 172.21.15.117:0/1954416925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280095f0 con 0x7fac40100e20 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.923+0000 7fac3ea00700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac40100e20 0x7fac4019e9f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fac3000ada0 tx=0x7fac3000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.923+0000 7fac37e00700 1 -- 172.21.15.117:0/1954416925 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac3000f070 con 0x7fac40100e20 2024-04-20T11:25:32.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.924+0000 7fac37e00700 1 -- 172.21.15.117:0/1954416925 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac30007bf0 con 0x7fac40100e20 2024-04-20T11:25:32.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.924+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac40198e10 con 0x7fac40100e20 2024-04-20T11:25:32.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.924+0000 7fac37e00700 1 -- 172.21.15.117:0/1954416925 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac30005690 con 0x7fac40100e20 2024-04-20T11:25:32.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.924+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac40199360 con 0x7fac40100e20 2024-04-20T11:25:32.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.925+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac40104fa0 con 0x7fac40100e20 2024-04-20T11:25:32.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.926+0000 7fac37e00700 1 -- 172.21.15.117:0/1954416925 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fac300058f0 con 0x7fac40100e20 2024-04-20T11:25:32.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.927+0000 7fac37e00700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac2406ecc0 0x7fac24071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.927+0000 7fac37e00700 1 -- 172.21.15.117:0/1954416925 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fac3008e7f0 con 0x7fac40100e20 2024-04-20T11:25:32.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.928+0000 7fac36000700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac2406ecc0 0x7fac24071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.928+0000 7fac36000700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac2406ecc0 0x7fac24071180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fac28006010 tx=0x7fac2800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:32.933+0000 7fac37e00700 1 -- 172.21.15.117:0/1954416925 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fac3005b0e0 con 0x7fac40100e20 2024-04-20T11:25:33.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.184+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac4019a090 con 0x7fac2406ecc0 2024-04-20T11:25:33.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.187+0000 7fac37e00700 1 -- 172.21.15.117:0/1954416925 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fac4019a090 con 0x7fac2406ecc0 2024-04-20T11:25:33.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac2406ecc0 msgr2=0x7fac24071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac2406ecc0 0x7fac24071180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fac28006010 tx=0x7fac2800b540 comp rx=0 tx=0).stop 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac40100e20 msgr2=0x7fac4019e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac40100e20 0x7fac4019e9f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fac3000ada0 tx=0x7fac3000c330 comp rx=0 tx=0).stop 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 shutdown_connections 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac2406ecc0 0x7fac24071180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac40100e20 0x7fac4019e9f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 --2- 172.21.15.117:0/1954416925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac401017d0 0x7fac4019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 >> 172.21.15.117:0/1954416925 conn(0x7fac400fc6d0 msgr2=0x7fac400fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 shutdown_connections 2024-04-20T11:25:33.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.190+0000 7fac45800700 1 -- 172.21.15.117:0/1954416925 wait complete. 2024-04-20T11:25:33.207 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:25:33.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.351+0000 7fd9df600700 1 -- 172.21.15.117:0/437603602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8103110 msgr2=0x7fd9d81034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.351+0000 7fd9df600700 1 --2- 172.21.15.117:0/437603602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8103110 0x7fd9d81034f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009910 tx=0x7fd9cc009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:33.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.351+0000 7fd9df600700 1 -- 172.21.15.117:0/437603602 shutdown_connections 2024-04-20T11:25:33.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.351+0000 7fd9df600700 1 --2- 172.21.15.117:0/437603602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8104a00 0x7fd9d8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.351+0000 7fd9df600700 1 --2- 172.21.15.117:0/437603602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8103110 0x7fd9d81034f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.351+0000 7fd9df600700 1 -- 172.21.15.117:0/437603602 >> 172.21.15.117:0/437603602 conn(0x7fd9d80fe880 msgr2=0x7fd9d8100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.351+0000 7fd9df600700 1 -- 172.21.15.117:0/437603602 shutdown_connections 2024-04-20T11:25:33.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.351+0000 7fd9df600700 1 -- 172.21.15.117:0/437603602 wait complete. 2024-04-20T11:25:33.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.352+0000 7fd9df600700 1 Processor -- start 2024-04-20T11:25:33.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.352+0000 7fd9df600700 1 -- start start 2024-04-20T11:25:33.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.353+0000 7fd9df600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8103110 0x7fd9d8198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.353+0000 7fd9df600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8104a00 0x7fd9d8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.353+0000 7fd9df600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81999e0 con 0x7fd9d8104a00 2024-04-20T11:25:33.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.353+0000 7fd9df600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d819d660 con 0x7fd9d8103110 2024-04-20T11:25:33.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.353+0000 7fd9d7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8104a00 0x7fd9d8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.353+0000 7fd9d7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8104a00 0x7fd9d8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56878/0 (socket says 172.21.15.117:56878) 2024-04-20T11:25:33.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.353+0000 7fd9d7e00700 1 -- 172.21.15.117:0/2371772386 learned_addr learned my addr 172.21.15.117:0/2371772386 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:33.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.353+0000 7fd9dcc00700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8103110 0x7fd9d8198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9d7e00700 1 -- 172.21.15.117:0/2371772386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8103110 msgr2=0x7fd9d8198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9d7e00700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8103110 0x7fd9d8198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9d7e00700 1 -- 172.21.15.117:0/2371772386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0095f0 con 0x7fd9d8104a00 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9d7e00700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8104a00 0x7fd9d8199300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c8a0 tx=0x7fd9c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9d5600700 1 -- 172.21.15.117:0/2371772386 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9c400cea0 con 0x7fd9d8104a00 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9d5600700 1 -- 172.21.15.117:0/2371772386 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9c4004830 con 0x7fd9d8104a00 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d819d940 con 0x7fd9d8104a00 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9d5600700 1 -- 172.21.15.117:0/2371772386 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9c40055b0 con 0x7fd9d8104a00 2024-04-20T11:25:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.354+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d819df10 con 0x7fd9d8104a00 2024-04-20T11:25:33.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.356+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d810b460 con 0x7fd9d8104a00 2024-04-20T11:25:33.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.357+0000 7fd9d5600700 1 -- 172.21.15.117:0/2371772386 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd9c40049a0 con 0x7fd9d8104a00 2024-04-20T11:25:33.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.358+0000 7fd9d5600700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd9c006ecd0 0x7fd9c0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.358+0000 7fd9d5600700 1 -- 172.21.15.117:0/2371772386 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd9c408ebc0 con 0x7fd9d8104a00 2024-04-20T11:25:33.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.358+0000 7fd9dcc00700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd9c006ecd0 0x7fd9c0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.359+0000 7fd9dcc00700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd9c006ecd0 0x7fd9c0071190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b5c0 tx=0x7fd9cc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.363+0000 7fd9d5600700 1 -- 172.21.15.117:0/2371772386 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd9c4057900 con 0x7fd9d8104a00 2024-04-20T11:25:33.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.611+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9d819a080 con 0x7fd9c006ecd0 2024-04-20T11:25:33.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.613+0000 7fd9d5600700 1 -- 172.21.15.117:0/2371772386 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd9d819a080 con 0x7fd9c006ecd0 2024-04-20T11:25:33.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.615+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd9c006ecd0 msgr2=0x7fd9c0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd9c006ecd0 0x7fd9c0071190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b5c0 tx=0x7fd9cc01a040 comp rx=0 tx=0).stop 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8104a00 msgr2=0x7fd9d8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8104a00 0x7fd9d8199300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c8a0 tx=0x7fd9c400cc60 comp rx=0 tx=0).stop 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 shutdown_connections 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd9c006ecd0 0x7fd9c0071190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8103110 0x7fd9d8198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 --2- 172.21.15.117:0/2371772386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d8104a00 0x7fd9d8199300 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c8a0 tx=0x7fd9c400cc60 comp rx=0 tx=0).stop 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 >> 172.21.15.117:0/2371772386 conn(0x7fd9d80fe880 msgr2=0x7fd9d80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 shutdown_connections 2024-04-20T11:25:33.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.616+0000 7fd9df600700 1 -- 172.21.15.117:0/2371772386 wait complete. 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.764+0000 7f3f3be00700 1 -- 172.21.15.117:0/2717480467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c101bd0 msgr2=0x7f3f3c101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.764+0000 7f3f3be00700 1 --2- 172.21.15.117:0/2717480467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c101bd0 0x7f3f3c101fb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009910 tx=0x7f3f2c009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.764+0000 7f3f3be00700 1 -- 172.21.15.117:0/2717480467 shutdown_connections 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.764+0000 7f3f3be00700 1 --2- 172.21.15.117:0/2717480467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f3c1024f0 0x7f3f3c10a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.764+0000 7f3f3be00700 1 --2- 172.21.15.117:0/2717480467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c101bd0 0x7f3f3c101fb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.764+0000 7f3f3be00700 1 -- 172.21.15.117:0/2717480467 >> 172.21.15.117:0/2717480467 conn(0x7f3f3c0fb420 msgr2=0x7f3f3c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.764+0000 7f3f3be00700 1 -- 172.21.15.117:0/2717480467 shutdown_connections 2024-04-20T11:25:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.764+0000 7f3f3be00700 1 -- 172.21.15.117:0/2717480467 wait complete. 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.765+0000 7f3f3be00700 1 Processor -- start 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.765+0000 7f3f3be00700 1 -- start start 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.766+0000 7f3f3be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f3c101bd0 0x7f3f3c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.766+0000 7f3f3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c1024f0 0x7f3f3c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.766+0000 7f3f3be00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c19d240 con 0x7f3f3c1024f0 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.766+0000 7f3f3be00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c1967b0 con 0x7f3f3c101bd0 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.766+0000 7f3f3a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c1024f0 0x7f3f3c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.766+0000 7f3f3a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c1024f0 0x7f3f3c19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56886/0 (socket says 172.21.15.117:56886) 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.766+0000 7f3f3a000700 1 -- 172.21.15.117:0/1993977357 learned_addr learned my addr 172.21.15.117:0/1993977357 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.766+0000 7f3f3aa00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f3c101bd0 0x7f3f3c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f3a000700 1 -- 172.21.15.117:0/1993977357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f3c101bd0 msgr2=0x7f3f3c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f3a000700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f3c101bd0 0x7f3f3c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f3a000700 1 -- 172.21.15.117:0/1993977357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0095f0 con 0x7f3f3c1024f0 2024-04-20T11:25:33.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f3a000700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c1024f0 0x7f3f3c19cbb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c8f0 tx=0x7f3f2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f40a00700 1 -- 172.21.15.117:0/1993977357 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f28007ad0 con 0x7f3f3c1024f0 2024-04-20T11:25:33.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f40a00700 1 -- 172.21.15.117:0/1993977357 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f2800ce90 con 0x7f3f3c1024f0 2024-04-20T11:25:33.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c196a90 con 0x7f3f3c1024f0 2024-04-20T11:25:33.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f40a00700 1 -- 172.21.15.117:0/1993977357 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f28018630 con 0x7f3f3c1024f0 2024-04-20T11:25:33.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.767+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c196fe0 con 0x7f3f3c1024f0 2024-04-20T11:25:33.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.769+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c1052f0 con 0x7f3f3c1024f0 2024-04-20T11:25:33.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.770+0000 7f3f40a00700 1 -- 172.21.15.117:0/1993977357 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3f28007c30 con 0x7f3f3c1024f0 2024-04-20T11:25:33.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.770+0000 7f3f40a00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f2006ecc0 0x7f3f20071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.771+0000 7f3f40a00700 1 -- 172.21.15.117:0/1993977357 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3f2808ed90 con 0x7f3f3c1024f0 2024-04-20T11:25:33.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.771+0000 7f3f3aa00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f2006ecc0 0x7f3f20071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.772+0000 7f3f3aa00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f2006ecc0 0x7f3f20071180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00b5c0 tx=0x7f3f2c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.776+0000 7f3f40a00700 1 -- 172.21.15.117:0/1993977357 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3f2805b720 con 0x7f3f3c1024f0 2024-04-20T11:25:33.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:33.993+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f3c197ce0 con 0x7f3f2006ecc0 2024-04-20T11:25:34.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.010+0000 7f3f40a00700 1 -- 172.21.15.117:0/1993977357 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3f3c197ce0 con 0x7f3f2006ecc0 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (14m) 5m ago 28m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (28m) 5m ago 28m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (27m) 5m ago 27m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (10m) 5m ago 28m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (10m) 5m ago 26m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (13m) 5m ago 27m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (5m) 5m ago 20m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (6m) 5m ago 20m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:25:34.011 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (20m) 5m ago 20m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (5m) 5m ago 20m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (17m) 5m ago 29m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (15m) 5m ago 26m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (12m) 5m ago 30m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (11m) 5m ago 26m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (15m) 5m ago 28m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (15m) 5m ago 26m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (9m) 5m ago 25m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (9m) 5m ago 25m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (8m) 5m ago 24m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (8m) 5m ago 23m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (7m) 5m ago 23m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (6m) 5m ago 22m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:25:34.012 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (14m) 5m ago 27m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:25:34.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f2006ecc0 msgr2=0x7f3f20071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f2006ecc0 0x7f3f20071180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00b5c0 tx=0x7f3f2c01a040 comp rx=0 tx=0).stop 2024-04-20T11:25:34.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c1024f0 msgr2=0x7f3f3c19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c1024f0 0x7f3f3c19cbb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c8f0 tx=0x7f3f2800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 shutdown_connections 2024-04-20T11:25:34.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f2006ecc0 0x7f3f20071180 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f3c101bd0 0x7f3f3c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 --2- 172.21.15.117:0/1993977357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c1024f0 0x7f3f3c19cbb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.013+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 >> 172.21.15.117:0/1993977357 conn(0x7f3f3c0fb420 msgr2=0x7f3f3c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.014+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 shutdown_connections 2024-04-20T11:25:34.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.014+0000 7f3f3be00700 1 -- 172.21.15.117:0/1993977357 wait complete. 2024-04-20T11:25:34.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.169+0000 7f9a66c00700 1 -- 172.21.15.117:0/2365142132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60107520 msgr2=0x7f9a60107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.169+0000 7f9a66c00700 1 --2- 172.21.15.117:0/2365142132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60107520 0x7f9a60107900 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009910 tx=0x7f9a50009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:34.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.169+0000 7f9a66c00700 1 -- 172.21.15.117:0/2365142132 shutdown_connections 2024-04-20T11:25:34.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.169+0000 7f9a66c00700 1 --2- 172.21.15.117:0/2365142132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a60108e10 0x7f9a60109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.169+0000 7f9a66c00700 1 --2- 172.21.15.117:0/2365142132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60107520 0x7f9a60107900 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.169+0000 7f9a66c00700 1 -- 172.21.15.117:0/2365142132 >> 172.21.15.117:0/2365142132 conn(0x7f9a600768f0 msgr2=0x7f9a60078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.169+0000 7f9a66c00700 1 -- 172.21.15.117:0/2365142132 shutdown_connections 2024-04-20T11:25:34.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.169+0000 7f9a66c00700 1 -- 172.21.15.117:0/2365142132 wait complete. 2024-04-20T11:25:34.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.170+0000 7f9a66c00700 1 Processor -- start 2024-04-20T11:25:34.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.170+0000 7f9a66c00700 1 -- start start 2024-04-20T11:25:34.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a66c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a60107520 0x7f9a6019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a66c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60108e10 0x7f9a6019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a66c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6019ddd0 con 0x7f9a60108e10 2024-04-20T11:25:34.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a66c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601a1aa0 con 0x7f9a60107520 2024-04-20T11:25:34.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a5f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60108e10 0x7f9a6019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a5f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60108e10 0x7f9a6019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40974/0 (socket says 172.21.15.117:40974) 2024-04-20T11:25:34.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a5f400700 1 -- 172.21.15.117:0/2380342391 learned_addr learned my addr 172.21.15.117:0/2380342391 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a5f400700 1 -- 172.21.15.117:0/2380342391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a60107520 msgr2=0x7f9a6019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.171+0000 7f9a5f400700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a60107520 0x7f9a6019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.172+0000 7f9a5f400700 1 -- 172.21.15.117:0/2380342391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500095f0 con 0x7f9a60108e10 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.172+0000 7f9a5f400700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60108e10 0x7f9a6019d6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c8a0 tx=0x7f9a4c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.172+0000 7f9a5cc00700 1 -- 172.21.15.117:0/2380342391 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4c00cea0 con 0x7f9a60108e10 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.172+0000 7f9a5cc00700 1 -- 172.21.15.117:0/2380342391 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a4c004830 con 0x7f9a60108e10 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.172+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a601a1d80 con 0x7f9a60108e10 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.172+0000 7f9a5cc00700 1 -- 172.21.15.117:0/2380342391 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4c0055b0 con 0x7f9a60108e10 2024-04-20T11:25:34.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.172+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a601a2350 con 0x7f9a60108e10 2024-04-20T11:25:34.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.174+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6010f870 con 0x7f9a60108e10 2024-04-20T11:25:34.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.175+0000 7f9a5cc00700 1 -- 172.21.15.117:0/2380342391 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9a4c0049a0 con 0x7f9a60108e10 2024-04-20T11:25:34.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.176+0000 7f9a5cc00700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a4406ecd0 0x7f9a44071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.176+0000 7f9a5cc00700 1 -- 172.21.15.117:0/2380342391 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9a4c08eac0 con 0x7f9a60108e10 2024-04-20T11:25:34.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.176+0000 7f9a5fe00700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a4406ecd0 0x7f9a44071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.177+0000 7f9a5fe00700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a4406ecd0 0x7f9a44071190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006010 tx=0x7f9a5000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.181+0000 7f9a5cc00700 1 -- 172.21.15.117:0/2380342391 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9a4c057880 con 0x7f9a60108e10 2024-04-20T11:25:34.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.480+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9a6019e4c0 con 0x7f9a60108e10 2024-04-20T11:25:34.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.481+0000 7f9a5cc00700 1 -- 172.21.15.117:0/2380342391 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f9a4c01d070 con 0x7f9a60108e10 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:25:34.482 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:25:34.483 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:25:34.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a4406ecd0 msgr2=0x7f9a44071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a4406ecd0 0x7f9a44071190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006010 tx=0x7f9a5000b540 comp rx=0 tx=0).stop 2024-04-20T11:25:34.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60108e10 msgr2=0x7f9a6019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60108e10 0x7f9a6019d6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c8a0 tx=0x7f9a4c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:25:34.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 shutdown_connections 2024-04-20T11:25:34.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a4406ecd0 0x7f9a44071190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a60107520 0x7f9a6019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 --2- 172.21.15.117:0/2380342391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60108e10 0x7f9a6019d6f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 >> 172.21.15.117:0/2380342391 conn(0x7f9a600768f0 msgr2=0x7f9a60077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 shutdown_connections 2024-04-20T11:25:34.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.484+0000 7f9a66c00700 1 -- 172.21.15.117:0/2380342391 wait complete. 2024-04-20T11:25:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:34 smithi117 ceph-mon[99036]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:34 smithi117 ceph-mon[99036]: pgmap v393: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:34.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.641+0000 7f9106200700 1 -- 172.21.15.117:0/3159073547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 msgr2=0x7f91000736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.641+0000 7f9106200700 1 --2- 172.21.15.117:0/3159073547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 0x7f91000736f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b0a0 tx=0x7f90f000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.642+0000 7f9106200700 1 -- 172.21.15.117:0/3159073547 shutdown_connections 2024-04-20T11:25:34.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.642+0000 7f9106200700 1 --2- 172.21.15.117:0/3159073547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9100073c30 0x7f910010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.642+0000 7f9106200700 1 --2- 172.21.15.117:0/3159073547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 0x7f91000736f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.642+0000 7f9106200700 1 -- 172.21.15.117:0/3159073547 >> 172.21.15.117:0/3159073547 conn(0x7f91000fc6c0 msgr2=0x7f91000feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.642+0000 7f9106200700 1 -- 172.21.15.117:0/3159073547 shutdown_connections 2024-04-20T11:25:34.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.642+0000 7f9106200700 1 -- 172.21.15.117:0/3159073547 wait complete. 2024-04-20T11:25:34.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.643+0000 7f9106200700 1 Processor -- start 2024-04-20T11:25:34.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.643+0000 7f9106200700 1 -- start start 2024-04-20T11:25:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f9106200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 0x7f9100198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f9106200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9100073c30 0x7f91001992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f9106200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100199980 con 0x7f9100073310 2024-04-20T11:25:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f9106200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910019d650 con 0x7f9100073c30 2024-04-20T11:25:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f90ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 0x7f9100198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f90ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 0x7f9100198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41002/0 (socket says 172.21.15.117:41002) 2024-04-20T11:25:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f90ff400700 1 -- 172.21.15.117:0/2162694059 learned_addr learned my addr 172.21.15.117:0/2162694059 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:34.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f90ff400700 1 -- 172.21.15.117:0/2162694059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9100073c30 msgr2=0x7f91001992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f90f6a00700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9100073c30 0x7f91001992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f90ff400700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9100073c30 0x7f91001992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.644+0000 7f90ff400700 1 -- 172.21.15.117:0/2162694059 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0009d00 con 0x7f9100073310 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.645+0000 7f90ff400700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 0x7f9100198d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b070 tx=0x7f90f0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.645+0000 7f90fcc00700 1 -- 172.21.15.117:0/2162694059 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f000e040 con 0x7f9100073310 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.645+0000 7f90fcc00700 1 -- 172.21.15.117:0/2162694059 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90f0004040 con 0x7f9100073310 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.645+0000 7f90fcc00700 1 -- 172.21.15.117:0/2162694059 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f001d6b0 con 0x7f9100073310 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.645+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910019d8d0 con 0x7f9100073310 2024-04-20T11:25:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.645+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910019ddc0 con 0x7f9100073310 2024-04-20T11:25:34.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.646+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91000749c0 con 0x7f9100073310 2024-04-20T11:25:34.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.647+0000 7f90fcc00700 1 -- 172.21.15.117:0/2162694059 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f90f00046b0 con 0x7f9100073310 2024-04-20T11:25:34.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.648+0000 7f90fcc00700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f90e406ecc0 0x7f90e4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.648+0000 7f90fcc00700 1 -- 172.21.15.117:0/2162694059 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f90f0090410 con 0x7f9100073310 2024-04-20T11:25:34.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.649+0000 7f90f6a00700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f90e406ecc0 0x7f90e4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.649+0000 7f90f6a00700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f90e406ecc0 0x7f90e4071180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005950 tx=0x7f90e80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.653+0000 7f90fcc00700 1 -- 172.21.15.117:0/2162694059 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f90f00591d0 con 0x7f9100073310 2024-04-20T11:25:34.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:34 smithi173 ceph-mon[74165]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:34 smithi173 ceph-mon[74165]: pgmap v393: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:34.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.906+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f91001077f0 con 0x7f9100073310 2024-04-20T11:25:34.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.907+0000 7f90fcc00700 1 -- 172.21.15.117:0/2162694059 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f90f0017770 con 0x7f9100073310 2024-04-20T11:25:34.909 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:25:34.909 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:25:34.909 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:25:34.909 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:25:34.909 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:25:34.909 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:25:34.909 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:25:34.909 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:25:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:25:34.911 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:25:34.912 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:25:34.912 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:25:34.912 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:25:34.912 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:25:34.912 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:25:34.912 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.909+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f90e406ecc0 msgr2=0x7f90e4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.909+0000 7f9106200700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f90e406ecc0 0x7f90e4071180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005950 tx=0x7f90e80058e0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.909+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 msgr2=0x7f9100198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.909+0000 7f9106200700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 0x7f9100198d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b070 tx=0x7f90f0003ce0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.910+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 shutdown_connections 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.910+0000 7f9106200700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f90e406ecc0 0x7f90e4071180 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.910+0000 7f9106200700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9100073310 0x7f9100198d60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.910+0000 7f9106200700 1 --2- 172.21.15.117:0/2162694059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9100073c30 0x7f91001992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.910+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 >> 172.21.15.117:0/2162694059 conn(0x7f91000fc6c0 msgr2=0x7f9100105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.910+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 shutdown_connections 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:34.910+0000 7f9106200700 1 -- 172.21.15.117:0/2162694059 wait complete. 2024-04-20T11:25:34.913 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.060+0000 7f6dd9e00700 1 -- 172.21.15.117:0/2080978459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd41017c0 msgr2=0x7f6dd4105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.060+0000 7f6dd9e00700 1 --2- 172.21.15.117:0/2080978459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd41017c0 0x7f6dd4105810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009910 tx=0x7f6dbc009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.060+0000 7f6dd9e00700 1 -- 172.21.15.117:0/2080978459 shutdown_connections 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.060+0000 7f6dd9e00700 1 --2- 172.21.15.117:0/2080978459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd41017c0 0x7f6dd4105810 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.060+0000 7f6dd9e00700 1 --2- 172.21.15.117:0/2080978459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dd4100e10 0x7f6dd41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.060+0000 7f6dd9e00700 1 -- 172.21.15.117:0/2080978459 >> 172.21.15.117:0/2080978459 conn(0x7f6dd40fc6c0 msgr2=0x7f6dd40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.060+0000 7f6dd9e00700 1 -- 172.21.15.117:0/2080978459 shutdown_connections 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.060+0000 7f6dd9e00700 1 -- 172.21.15.117:0/2080978459 wait complete. 2024-04-20T11:25:35.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.061+0000 7f6dd9e00700 1 Processor -- start 2024-04-20T11:25:35.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.061+0000 7f6dd9e00700 1 -- start start 2024-04-20T11:25:35.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4100e10 0x7f6dd419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd9e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dd41017c0 0x7f6dd419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd9e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd419d340 con 0x7f6dd4100e10 2024-04-20T11:25:35.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd9e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd4196970 con 0x7f6dd41017c0 2024-04-20T11:25:35.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4100e10 0x7f6dd419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4100e10 0x7f6dd419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41030/0 (socket says 172.21.15.117:41030) 2024-04-20T11:25:35.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd2a00700 1 -- 172.21.15.117:0/3135297800 learned_addr learned my addr 172.21.15.117:0/3135297800 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:35.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd2a00700 1 -- 172.21.15.117:0/3135297800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dd41017c0 msgr2=0x7f6dd419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd2a00700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dd41017c0 0x7f6dd419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.062+0000 7f6dd2a00700 1 -- 172.21.15.117:0/3135297800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0095f0 con 0x7f6dd4100e10 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.063+0000 7f6dd2a00700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4100e10 0x7f6dd419c7e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400ada0 tx=0x7f6dc400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.063+0000 7f6dd8a00700 1 -- 172.21.15.117:0/3135297800 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dc400f070 con 0x7f6dd4100e10 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.063+0000 7f6dd8a00700 1 -- 172.21.15.117:0/3135297800 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dc4007bf0 con 0x7f6dd4100e10 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.063+0000 7f6dd8a00700 1 -- 172.21.15.117:0/3135297800 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dc4005690 con 0x7f6dd4100e10 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.063+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd4196c50 con 0x7f6dd4100e10 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.063+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd41971a0 con 0x7f6dd4100e10 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.064+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd4109210 con 0x7f6dd4100e10 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.065+0000 7f6dd8a00700 1 -- 172.21.15.117:0/3135297800 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6dc4005850 con 0x7f6dd4100e10 2024-04-20T11:25:35.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.067+0000 7f6dd8a00700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6db806ecc0 0x7f6db8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.067+0000 7f6dd8a00700 1 -- 172.21.15.117:0/3135297800 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6dc408e830 con 0x7f6dd4100e10 2024-04-20T11:25:35.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.067+0000 7f6dca000700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6db806ecc0 0x7f6db8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.067+0000 7f6dca000700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6db806ecc0 0x7f6db8071180 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006010 tx=0x7f6dbc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.072+0000 7f6dd8a00700 1 -- 172.21.15.117:0/3135297800 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6dc405b120 con 0x7f6dd4100e10 2024-04-20T11:25:35.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.317+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dd410ddb0 con 0x7f6db806ecc0 2024-04-20T11:25:35.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.320+0000 7f6dd8a00700 1 -- 172.21.15.117:0/3135297800 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6dd410ddb0 con 0x7f6db806ecc0 2024-04-20T11:25:35.320 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:25:35.320 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:25:35.321 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6db806ecc0 msgr2=0x7f6db8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6db806ecc0 0x7f6db8071180 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006010 tx=0x7f6dbc00b540 comp rx=0 tx=0).stop 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4100e10 msgr2=0x7f6dd419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4100e10 0x7f6dd419c7e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400ada0 tx=0x7f6dc400c330 comp rx=0 tx=0).stop 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 shutdown_connections 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6db806ecc0 0x7f6db8071180 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4100e10 0x7f6dd419c7e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 --2- 172.21.15.117:0/3135297800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dd41017c0 0x7f6dd419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 >> 172.21.15.117:0/3135297800 conn(0x7f6dd40fc6c0 msgr2=0x7f6dd40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 shutdown_connections 2024-04-20T11:25:35.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.323+0000 7f6dd9e00700 1 -- 172.21.15.117:0/3135297800 wait complete. 2024-04-20T11:25:35.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.480+0000 7fd81ae00700 1 -- 172.21.15.117:0/2626293731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd814103020 msgr2=0x7fd814103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.480+0000 7fd81ae00700 1 --2- 172.21.15.117:0/2626293731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd814103020 0x7fd814103400 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd804009910 tx=0x7fd804009c20 comp rx=0 tx=0).stop 2024-04-20T11:25:35.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.480+0000 7fd81ae00700 1 -- 172.21.15.117:0/2626293731 shutdown_connections 2024-04-20T11:25:35.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.480+0000 7fd81ae00700 1 --2- 172.21.15.117:0/2626293731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8141039d0 0x7fd814107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.480+0000 7fd81ae00700 1 --2- 172.21.15.117:0/2626293731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd814103020 0x7fd814103400 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.480+0000 7fd81ae00700 1 -- 172.21.15.117:0/2626293731 >> 172.21.15.117:0/2626293731 conn(0x7fd8140fe8b0 msgr2=0x7fd814100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.480+0000 7fd81ae00700 1 -- 172.21.15.117:0/2626293731 shutdown_connections 2024-04-20T11:25:35.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.480+0000 7fd81ae00700 1 -- 172.21.15.117:0/2626293731 wait complete. 2024-04-20T11:25:35.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.481+0000 7fd81ae00700 1 Processor -- start 2024-04-20T11:25:35.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.482+0000 7fd81ae00700 1 -- start start 2024-04-20T11:25:35.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.482+0000 7fd81ae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814103020 0x7fd814198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.482+0000 7fd81ae00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8141039d0 0x7fd814199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.482+0000 7fd81ae00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141998f0 con 0x7fd8141039d0 2024-04-20T11:25:35.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.482+0000 7fd81ae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81419c560 con 0x7fd814103020 2024-04-20T11:25:35.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.482+0000 7fd813400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8141039d0 0x7fd814199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.482+0000 7fd813400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8141039d0 0x7fd814199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41050/0 (socket says 172.21.15.117:41050) 2024-04-20T11:25:35.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.482+0000 7fd813400700 1 -- 172.21.15.117:0/3749953879 learned_addr learned my addr 172.21.15.117:0/3749953879 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:25:35.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd813e00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814103020 0x7fd814198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd813400700 1 -- 172.21.15.117:0/3749953879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814103020 msgr2=0x7fd814198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd813400700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814103020 0x7fd814198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd813400700 1 -- 172.21.15.117:0/3749953879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040095f0 con 0x7fd8141039d0 2024-04-20T11:25:35.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd813400700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8141039d0 0x7fd814199260 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd80000d900 tx=0x7fd80000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd810c00700 1 -- 172.21.15.117:0/3749953879 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd800009900 con 0x7fd8141039d0 2024-04-20T11:25:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd810c00700 1 -- 172.21.15.117:0/3749953879 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd800010460 con 0x7fd8141039d0 2024-04-20T11:25:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81419c840 con 0x7fd8141039d0 2024-04-20T11:25:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd810c00700 1 -- 172.21.15.117:0/3749953879 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd80000f5f0 con 0x7fd8141039d0 2024-04-20T11:25:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.483+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81419cd90 con 0x7fd8141039d0 2024-04-20T11:25:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.484+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81410b3c0 con 0x7fd8141039d0 2024-04-20T11:25:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.486+0000 7fd810c00700 1 -- 172.21.15.117:0/3749953879 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd8000105d0 con 0x7fd8141039d0 2024-04-20T11:25:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.487+0000 7fd810c00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd7f806ecc0 0x7fd7f8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.487+0000 7fd810c00700 1 -- 172.21.15.117:0/3749953879 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd80008ec70 con 0x7fd8141039d0 2024-04-20T11:25:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.487+0000 7fd813e00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd7f806ecc0 0x7fd7f8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.488+0000 7fd813e00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd7f806ecc0 0x7fd7f8071180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd80400b5c0 tx=0x7fd8040052a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.492+0000 7fd810c00700 1 -- 172.21.15.117:0/3749953879 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd800095080 con 0x7fd8141039d0 2024-04-20T11:25:35.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:35 smithi117 ceph-mon[99036]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:35 smithi117 ceph-mon[99036]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:35 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2380342391' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:25:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:35 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2162694059' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:25:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:35 smithi173 ceph-mon[74165]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:35 smithi173 ceph-mon[74165]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:35 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2380342391' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:25:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:35 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2162694059' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:25:35.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.789+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd81410fff0 con 0x7fd8141039d0 2024-04-20T11:25:35.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.790+0000 7fd810c00700 1 -- 172.21.15.117:0/3749953879 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd8000171c0 con 0x7fd8141039d0 2024-04-20T11:25:35.790 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:25:35.790 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:25:35.791 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:25:35.791 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:25:35.791 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:25:35.791 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:25:35.791 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:25:35.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.792+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd7f806ecc0 msgr2=0x7fd7f8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.792+0000 7fd81ae00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd7f806ecc0 0x7fd7f8071180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd80400b5c0 tx=0x7fd8040052a0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.792+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8141039d0 msgr2=0x7fd814199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.792+0000 7fd81ae00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8141039d0 0x7fd814199260 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd80000d900 tx=0x7fd80000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.793+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 shutdown_connections 2024-04-20T11:25:35.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.793+0000 7fd81ae00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd7f806ecc0 0x7fd7f8071180 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.793+0000 7fd81ae00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814103020 0x7fd814198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.793+0000 7fd81ae00700 1 --2- 172.21.15.117:0/3749953879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8141039d0 0x7fd814199260 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.793+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 >> 172.21.15.117:0/3749953879 conn(0x7fd8140fe8b0 msgr2=0x7fd8140ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.793+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 shutdown_connections 2024-04-20T11:25:35.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:25:35.793+0000 7fd81ae00700 1 -- 172.21.15.117:0/3749953879 wait complete. 2024-04-20T11:25:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:36 smithi117 ceph-mon[99036]: pgmap v394: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:36 smithi117 ceph-mon[99036]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:36 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3749953879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:25:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:36 smithi173 ceph-mon[74165]: pgmap v394: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:36 smithi173 ceph-mon[74165]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:25:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:36 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3749953879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:25:37.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:37 smithi117 ceph-mon[99036]: pgmap v395: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:37 smithi173 ceph-mon[74165]: pgmap v395: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:40.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:40 smithi117 ceph-mon[99036]: pgmap v396: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:40 smithi173 ceph-mon[74165]: pgmap v396: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:42 smithi117 ceph-mon[99036]: pgmap v397: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:42 smithi173 ceph-mon[74165]: pgmap v397: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:25:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:25:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:44 smithi117 ceph-mon[99036]: pgmap v398: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:44 smithi173 ceph-mon[74165]: pgmap v398: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:45 smithi117 ceph-mon[99036]: pgmap v399: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:45 smithi173 ceph-mon[74165]: pgmap v399: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:48.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:48 smithi117 ceph-mon[99036]: pgmap v400: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:48 smithi173 ceph-mon[74165]: pgmap v400: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:50.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:50 smithi117 ceph-mon[99036]: pgmap v401: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:50 smithi173 ceph-mon[74165]: pgmap v401: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:51.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:51 smithi117 ceph-mon[99036]: pgmap v402: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:51.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:51 smithi173 ceph-mon[74165]: pgmap v402: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:54 smithi117 ceph-mon[99036]: pgmap v403: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:54 smithi173 ceph-mon[74165]: pgmap v403: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:56.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:56 smithi117 ceph-mon[99036]: pgmap v404: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:56 smithi173 ceph-mon[74165]: pgmap v404: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:57.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:57 smithi117 ceph-mon[99036]: pgmap v405: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:57 smithi173 ceph-mon[74165]: pgmap v405: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:25:58.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:25:59.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:25:59 smithi117 ceph-mon[99036]: pgmap v406: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:25:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:25:59 smithi173 ceph-mon[74165]: pgmap v406: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:02.574 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:02 smithi117 ceph-mon[99036]: pgmap v407: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:02 smithi173 ceph-mon[74165]: pgmap v407: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:04 smithi117 ceph-mon[99036]: pgmap v408: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:04 smithi173 ceph-mon[74165]: pgmap v408: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:05 smithi117 ceph-mon[99036]: pgmap v409: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:05 smithi173 ceph-mon[74165]: pgmap v409: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:05.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.962+0000 7fdab1c00700 1 -- 172.21.15.117:0/2830037533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 msgr2=0x7fdaac105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.962+0000 7fdab1c00700 1 --2- 172.21.15.117:0/2830037533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 0x7fdaac105810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fda94009910 tx=0x7fda94009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:05.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.962+0000 7fdab1c00700 1 -- 172.21.15.117:0/2830037533 shutdown_connections 2024-04-20T11:26:05.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.962+0000 7fdab1c00700 1 --2- 172.21.15.117:0/2830037533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 0x7fdaac105810 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.962+0000 7fdab1c00700 1 --2- 172.21.15.117:0/2830037533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac100e10 0x7fdaac1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.962+0000 7fdab1c00700 1 -- 172.21.15.117:0/2830037533 >> 172.21.15.117:0/2830037533 conn(0x7fdaac0fc6c0 msgr2=0x7fdaac0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.962+0000 7fdab1c00700 1 -- 172.21.15.117:0/2830037533 shutdown_connections 2024-04-20T11:26:05.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.963+0000 7fdab1c00700 1 -- 172.21.15.117:0/2830037533 wait complete. 2024-04-20T11:26:05.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.963+0000 7fdab1c00700 1 Processor -- start 2024-04-20T11:26:05.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.964+0000 7fdab1c00700 1 -- start start 2024-04-20T11:26:05.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.964+0000 7fdab1c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac100e10 0x7fdaac0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.964+0000 7fdab1c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 0x7fdaac073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.964+0000 7fdab1c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac073c60 con 0x7fdaac1017c0 2024-04-20T11:26:05.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.964+0000 7fdab1c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac073da0 con 0x7fdaac100e10 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.964+0000 7fdaa2000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 0x7fdaac073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.964+0000 7fdaa2000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 0x7fdaac073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34102/0 (socket says 172.21.15.117:34102) 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.964+0000 7fdaa2000700 1 -- 172.21.15.117:0/3502242989 learned_addr learned my addr 172.21.15.117:0/3502242989 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.965+0000 7fdaaaa00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac100e10 0x7fdaac0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.965+0000 7fdaa2000700 1 -- 172.21.15.117:0/3502242989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac100e10 msgr2=0x7fdaac0731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.965+0000 7fdaa2000700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac100e10 0x7fdaac0731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.965+0000 7fdaa2000700 1 -- 172.21.15.117:0/3502242989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda940095f0 con 0x7fdaac1017c0 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.965+0000 7fdaaaa00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac100e10 0x7fdaac0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:05.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.965+0000 7fdaa2000700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 0x7fdaac073720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fda94005b40 tx=0x7fda9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.966+0000 7fdaa3e00700 1 -- 172.21.15.117:0/3502242989 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9401d070 con 0x7fdaac1017c0 2024-04-20T11:26:05.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.966+0000 7fdaa3e00700 1 -- 172.21.15.117:0/3502242989 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda9400f460 con 0x7fdaac1017c0 2024-04-20T11:26:05.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.966+0000 7fdaa3e00700 1 -- 172.21.15.117:0/3502242989 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda94017630 con 0x7fdaac1017c0 2024-04-20T11:26:05.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.966+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac074020 con 0x7fdaac1017c0 2024-04-20T11:26:05.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.966+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac1a2f20 con 0x7fdaac1017c0 2024-04-20T11:26:05.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.967+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaac109210 con 0x7fdaac1017c0 2024-04-20T11:26:05.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.968+0000 7fdaa3e00700 1 -- 172.21.15.117:0/3502242989 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fda94003a60 con 0x7fdaac1017c0 2024-04-20T11:26:05.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.969+0000 7fdaa3e00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda9006ecc0 0x7fda90071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.969+0000 7fdaa3e00700 1 -- 172.21.15.117:0/3502242989 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fda94090290 con 0x7fdaac1017c0 2024-04-20T11:26:05.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.969+0000 7fdaaaa00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda9006ecc0 0x7fda90071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.970+0000 7fdaaaa00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda9006ecc0 0x7fda90071180 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0060b0 tx=0x7fda9c0076a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:05.974+0000 7fdaa3e00700 1 -- 172.21.15.117:0/3502242989 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fda94059050 con 0x7fdaac1017c0 2024-04-20T11:26:06.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.222+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdaac0746e0 con 0x7fda9006ecc0 2024-04-20T11:26:06.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.224+0000 7fdaa3e00700 1 -- 172.21.15.117:0/3502242989 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdaac0746e0 con 0x7fda9006ecc0 2024-04-20T11:26:06.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda9006ecc0 msgr2=0x7fda90071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda9006ecc0 0x7fda90071180 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0060b0 tx=0x7fda9c0076a0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 msgr2=0x7fdaac073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 0x7fdaac073720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fda94005b40 tx=0x7fda9400be20 comp rx=0 tx=0).stop 2024-04-20T11:26:06.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 shutdown_connections 2024-04-20T11:26:06.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fda9006ecc0 0x7fda90071180 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac100e10 0x7fdaac0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 --2- 172.21.15.117:0/3502242989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaac1017c0 0x7fdaac073720 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 >> 172.21.15.117:0/3502242989 conn(0x7fdaac0fc6c0 msgr2=0x7fdaac0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 shutdown_connections 2024-04-20T11:26:06.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.227+0000 7fdab1c00700 1 -- 172.21.15.117:0/3502242989 wait complete. 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:26:06.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.384+0000 7f9481c00700 1 -- 172.21.15.117:0/2339592120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 msgr2=0x7f947c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.384+0000 7f9481c00700 1 --2- 172.21.15.117:0/2339592120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 0x7f947c105810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9464009910 tx=0x7f9464009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:06.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.384+0000 7f9481c00700 1 -- 172.21.15.117:0/2339592120 shutdown_connections 2024-04-20T11:26:06.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.384+0000 7f9481c00700 1 --2- 172.21.15.117:0/2339592120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 0x7f947c105810 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.384+0000 7f9481c00700 1 --2- 172.21.15.117:0/2339592120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947c100e10 0x7f947c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.384+0000 7f9481c00700 1 -- 172.21.15.117:0/2339592120 >> 172.21.15.117:0/2339592120 conn(0x7f947c0fc6c0 msgr2=0x7f947c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.384+0000 7f9481c00700 1 -- 172.21.15.117:0/2339592120 shutdown_connections 2024-04-20T11:26:06.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.384+0000 7f9481c00700 1 -- 172.21.15.117:0/2339592120 wait complete. 2024-04-20T11:26:06.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.385+0000 7f9481c00700 1 Processor -- start 2024-04-20T11:26:06.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.385+0000 7f9481c00700 1 -- start start 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9481c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947c100e10 0x7f947c0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9481c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 0x7f947c073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9481c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c073c60 con 0x7f947c1017c0 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9481c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c073da0 con 0x7f947c100e10 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9473e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 0x7f947c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9473e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 0x7f947c073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34114/0 (socket says 172.21.15.117:34114) 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9473e00700 1 -- 172.21.15.117:0/3962920344 learned_addr learned my addr 172.21.15.117:0/3962920344 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f947aa00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947c100e10 0x7f947c0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9473e00700 1 -- 172.21.15.117:0/3962920344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947c100e10 msgr2=0x7f947c0731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9473e00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947c100e10 0x7f947c0731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.386+0000 7f9473e00700 1 -- 172.21.15.117:0/3962920344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640095f0 con 0x7f947c1017c0 2024-04-20T11:26:06.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.387+0000 7f9473e00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 0x7f947c073720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9464005b40 tx=0x7f946400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.387+0000 7f9473400700 1 -- 172.21.15.117:0/3962920344 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946401d070 con 0x7f947c1017c0 2024-04-20T11:26:06.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.387+0000 7f9473400700 1 -- 172.21.15.117:0/3962920344 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f946400f460 con 0x7f947c1017c0 2024-04-20T11:26:06.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.387+0000 7f9473400700 1 -- 172.21.15.117:0/3962920344 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9464017630 con 0x7f947c1017c0 2024-04-20T11:26:06.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.387+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c074020 con 0x7f947c1017c0 2024-04-20T11:26:06.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.387+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c1a3060 con 0x7f947c1017c0 2024-04-20T11:26:06.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.389+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c1022f0 con 0x7f947c1017c0 2024-04-20T11:26:06.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.389+0000 7f9473400700 1 -- 172.21.15.117:0/3962920344 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9464003a60 con 0x7f947c1017c0 2024-04-20T11:26:06.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.390+0000 7f9473400700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f946006ecc0 0x7f9460071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.391+0000 7f9473400700 1 -- 172.21.15.117:0/3962920344 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f94640901b0 con 0x7f947c1017c0 2024-04-20T11:26:06.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.391+0000 7f947aa00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f946006ecc0 0x7f9460071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.391+0000 7f947aa00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f946006ecc0 0x7f9460071180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f946c006fd0 tx=0x7f946c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.395+0000 7f9473400700 1 -- 172.21.15.117:0/3962920344 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9464059010 con 0x7f947c1017c0 2024-04-20T11:26:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.628+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f947c10dea0 con 0x7f946006ecc0 2024-04-20T11:26:06.628 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:06 smithi117 ceph-mon[99036]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:06.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.631+0000 7f9473400700 1 -- 172.21.15.117:0/3962920344 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f947c10dea0 con 0x7f946006ecc0 2024-04-20T11:26:06.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f946006ecc0 msgr2=0x7f9460071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f946006ecc0 0x7f9460071180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f946c006fd0 tx=0x7f946c008040 comp rx=0 tx=0).stop 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 msgr2=0x7f947c073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 0x7f947c073720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9464005b40 tx=0x7f946400be20 comp rx=0 tx=0).stop 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 shutdown_connections 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f946006ecc0 0x7f9460071180 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947c100e10 0x7f947c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 --2- 172.21.15.117:0/3962920344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f947c1017c0 0x7f947c073720 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 >> 172.21.15.117:0/3962920344 conn(0x7f947c0fc6c0 msgr2=0x7f947c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 shutdown_connections 2024-04-20T11:26:06.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.634+0000 7f9481c00700 1 -- 172.21.15.117:0/3962920344 wait complete. 2024-04-20T11:26:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:06 smithi173 ceph-mon[74165]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:06.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.788+0000 7fa761200700 1 -- 172.21.15.117:0/1589338692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 msgr2=0x7fa75c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.788+0000 7fa761200700 1 --2- 172.21.15.117:0/1589338692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 0x7fa75c1033e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009910 tx=0x7fa74c009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:06.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.789+0000 7fa761200700 1 -- 172.21.15.117:0/1589338692 shutdown_connections 2024-04-20T11:26:06.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.789+0000 7fa761200700 1 --2- 172.21.15.117:0/1589338692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75c1039b0 0x7fa75c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.789+0000 7fa761200700 1 --2- 172.21.15.117:0/1589338692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 0x7fa75c1033e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.789+0000 7fa761200700 1 -- 172.21.15.117:0/1589338692 >> 172.21.15.117:0/1589338692 conn(0x7fa75c0fe8b0 msgr2=0x7fa75c100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.789+0000 7fa761200700 1 -- 172.21.15.117:0/1589338692 shutdown_connections 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.789+0000 7fa761200700 1 -- 172.21.15.117:0/1589338692 wait complete. 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.790+0000 7fa761200700 1 Processor -- start 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.790+0000 7fa761200700 1 -- start start 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.790+0000 7fa761200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 0x7fa75c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.790+0000 7fa761200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75c1039b0 0x7fa75c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.790+0000 7fa761200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c19f540 con 0x7fa75c103000 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.790+0000 7fa761200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c198b70 con 0x7fa75c1039b0 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.791+0000 7fa759e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 0x7fa75c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.791+0000 7fa759e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 0x7fa75c19e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34134/0 (socket says 172.21.15.117:34134) 2024-04-20T11:26:06.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.791+0000 7fa759e00700 1 -- 172.21.15.117:0/4057969838 learned_addr learned my addr 172.21.15.117:0/4057969838 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.791+0000 7fa753e00700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75c1039b0 0x7fa75c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.791+0000 7fa759e00700 1 -- 172.21.15.117:0/4057969838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75c1039b0 msgr2=0x7fa75c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.791+0000 7fa759e00700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75c1039b0 0x7fa75c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.791+0000 7fa759e00700 1 -- 172.21.15.117:0/4057969838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c0095f0 con 0x7fa75c103000 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.792+0000 7fa759e00700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 0x7fa75c19e9e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00bd70 tx=0x7fa74c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.792+0000 7fa752a00700 1 -- 172.21.15.117:0/4057969838 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74c01d070 con 0x7fa75c103000 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.792+0000 7fa752a00700 1 -- 172.21.15.117:0/4057969838 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa74c0056e0 con 0x7fa75c103000 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.792+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c198df0 con 0x7fa75c103000 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.792+0000 7fa752a00700 1 -- 172.21.15.117:0/4057969838 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74c017400 con 0x7fa75c103000 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.792+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c1992e0 con 0x7fa75c103000 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.793+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c10b3a0 con 0x7fa75c103000 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.794+0000 7fa752a00700 1 -- 172.21.15.117:0/4057969838 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa74c003a60 con 0x7fa75c103000 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.795+0000 7fa752a00700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa74006ecc0 0x7fa740071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.795+0000 7fa752a00700 1 -- 172.21.15.117:0/4057969838 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa74c090330 con 0x7fa75c103000 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.795+0000 7fa753e00700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa74006ecc0 0x7fa740071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.796+0000 7fa753e00700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa74006ecc0 0x7fa740071180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa744005950 tx=0x7fa7440058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:06.800+0000 7fa752a00700 1 -- 172.21.15.117:0/4057969838 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa74c05cba0 con 0x7fa75c103000 2024-04-20T11:26:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.038+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa75c19a050 con 0x7fa74006ecc0 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.055+0000 7fa752a00700 1 -- 172.21.15.117:0/4057969838 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa75c19a050 con 0x7fa74006ecc0 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (14m) 6m ago 28m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (28m) 6m ago 28m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (27m) 5m ago 27m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (10m) 6m ago 28m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (10m) 5m ago 27m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (13m) 6m ago 28m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (6m) 6m ago 21m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (6m) 6m ago 21m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (21m) 5m ago 21m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (5m) 5m ago 21m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (17m) 6m ago 30m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (16m) 5m ago 27m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (12m) 6m ago 30m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (11m) 5m ago 27m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (15m) 6m ago 28m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (15m) 5m ago 27m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (10m) 6m ago 26m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (9m) 6m ago 25m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (9m) 6m ago 25m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (8m) 5m ago 24m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (7m) 5m ago 23m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (7m) 5m ago 23m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (15m) 6m ago 28m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa74006ecc0 msgr2=0x7fa740071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa74006ecc0 0x7fa740071180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa744005950 tx=0x7fa7440058e0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 msgr2=0x7fa75c19e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 0x7fa75c19e9e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00bd70 tx=0x7fa74c00be50 comp rx=0 tx=0).stop 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 shutdown_connections 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa74006ecc0 0x7fa740071180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa75c103000 0x7fa75c19e9e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 --2- 172.21.15.117:0/4057969838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75c1039b0 0x7fa75c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 >> 172.21.15.117:0/4057969838 conn(0x7fa75c0fe8b0 msgr2=0x7fa75c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 shutdown_connections 2024-04-20T11:26:07.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.058+0000 7fa761200700 1 -- 172.21.15.117:0/4057969838 wait complete. 2024-04-20T11:26:07.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.213+0000 7fec77600700 1 -- 172.21.15.117:0/1827543978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70101d70 msgr2=0x7fec70102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.213+0000 7fec77600700 1 --2- 172.21.15.117:0/1827543978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70101d70 0x7fec70102150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fec64009910 tx=0x7fec64009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:07.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.213+0000 7fec77600700 1 -- 172.21.15.117:0/1827543978 shutdown_connections 2024-04-20T11:26:07.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.213+0000 7fec77600700 1 --2- 172.21.15.117:0/1827543978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec70102690 0x7fec7010ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.213+0000 7fec77600700 1 --2- 172.21.15.117:0/1827543978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70101d70 0x7fec70102150 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.213+0000 7fec77600700 1 -- 172.21.15.117:0/1827543978 >> 172.21.15.117:0/1827543978 conn(0x7fec700fb5c0 msgr2=0x7fec700fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.213+0000 7fec77600700 1 -- 172.21.15.117:0/1827543978 shutdown_connections 2024-04-20T11:26:07.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.213+0000 7fec77600700 1 -- 172.21.15.117:0/1827543978 wait complete. 2024-04-20T11:26:07.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.214+0000 7fec77600700 1 Processor -- start 2024-04-20T11:26:07.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.214+0000 7fec77600700 1 -- start start 2024-04-20T11:26:07.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.215+0000 7fec77600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec70101d70 0x7fec700ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.215+0000 7fec77600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70102690 0x7fec700fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.215+0000 7fec77600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70100450 con 0x7fec70102690 2024-04-20T11:26:07.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.215+0000 7fec77600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70100590 con 0x7fec70101d70 2024-04-20T11:26:07.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.215+0000 7fec6fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70102690 0x7fec700fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.215+0000 7fec6fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70102690 0x7fec700fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34166/0 (socket says 172.21.15.117:34166) 2024-04-20T11:26:07.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.215+0000 7fec6fe00700 1 -- 172.21.15.117:0/1670393490 learned_addr learned my addr 172.21.15.117:0/1670393490 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec74c00700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec70101d70 0x7fec700ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec6fe00700 1 -- 172.21.15.117:0/1670393490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec70101d70 msgr2=0x7fec700ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec6fe00700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec70101d70 0x7fec700ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec6fe00700 1 -- 172.21.15.117:0/1670393490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640095f0 con 0x7fec70102690 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec74c00700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec70101d70 0x7fec700ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec6fe00700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70102690 0x7fec700fff10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c8f0 tx=0x7fec5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec6d600700 1 -- 172.21.15.117:0/1670393490 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec5c007ad0 con 0x7fec70102690 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec6d600700 1 -- 172.21.15.117:0/1670393490 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec5c00ce90 con 0x7fec70102690 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec701a2cc0 con 0x7fec70102690 2024-04-20T11:26:07.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec6d600700 1 -- 172.21.15.117:0/1670393490 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec5c018630 con 0x7fec70102690 2024-04-20T11:26:07.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.216+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701a3020 con 0x7fec70102690 2024-04-20T11:26:07.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.218+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec700696e0 con 0x7fec70102690 2024-04-20T11:26:07.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.218+0000 7fec6d600700 1 -- 172.21.15.117:0/1670393490 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fec5c007c30 con 0x7fec70102690 2024-04-20T11:26:07.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.219+0000 7fec6d600700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec5806ecc0 0x7fec58071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.219+0000 7fec6d600700 1 -- 172.21.15.117:0/1670393490 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fec5c08ede0 con 0x7fec70102690 2024-04-20T11:26:07.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.220+0000 7fec74c00700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec5806ecc0 0x7fec58071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.220+0000 7fec74c00700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec5806ecc0 0x7fec58071180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec64006010 tx=0x7fec6400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.224+0000 7fec6d600700 1 -- 172.21.15.117:0/1670393490 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fec5c057b20 con 0x7fec70102690 2024-04-20T11:26:07.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.530+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec7010dde0 con 0x7fec70102690 2024-04-20T11:26:07.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.531+0000 7fec6d600700 1 -- 172.21.15.117:0/1670393490 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fec5c05b1e0 con 0x7fec70102690 2024-04-20T11:26:07.532 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:26:07.532 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:26:07.532 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:26:07.532 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:26:07.532 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:26:07.533 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:26:07.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:26:07.533 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:26:07.533 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:26:07.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:26:07.533 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:26:07.533 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:26:07.534 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:26:07.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:26:07.534 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:26:07.534 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:26:07.534 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:26:07.534 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:26:07.534 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:26:07.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec5806ecc0 msgr2=0x7fec58071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec5806ecc0 0x7fec58071180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec64006010 tx=0x7fec6400b540 comp rx=0 tx=0).stop 2024-04-20T11:26:07.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70102690 msgr2=0x7fec700fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70102690 0x7fec700fff10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c8f0 tx=0x7fec5c00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 shutdown_connections 2024-04-20T11:26:07.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec5806ecc0 0x7fec58071180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec70101d70 0x7fec700ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 --2- 172.21.15.117:0/1670393490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec70102690 0x7fec700fff10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.534+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 >> 172.21.15.117:0/1670393490 conn(0x7fec700fb5c0 msgr2=0x7fec70103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.535+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 shutdown_connections 2024-04-20T11:26:07.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.535+0000 7fec77600700 1 -- 172.21.15.117:0/1670393490 wait complete. 2024-04-20T11:26:07.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:07 smithi117 ceph-mon[99036]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:07.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:07 smithi117 ceph-mon[99036]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:07.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:07 smithi117 ceph-mon[99036]: pgmap v410: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:07 smithi173 ceph-mon[74165]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:07 smithi173 ceph-mon[74165]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:07 smithi173 ceph-mon[74165]: pgmap v410: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:07.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.690+0000 7f5899200700 1 -- 172.21.15.117:0/667466199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894104a00 msgr2=0x7f5894104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.690+0000 7f5899200700 1 --2- 172.21.15.117:0/667466199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894104a00 0x7f5894104e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f587c009910 tx=0x7f587c009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:07.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.690+0000 7f5899200700 1 -- 172.21.15.117:0/667466199 shutdown_connections 2024-04-20T11:26:07.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.690+0000 7f5899200700 1 --2- 172.21.15.117:0/667466199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894104a00 0x7f5894104e80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.690+0000 7f5899200700 1 --2- 172.21.15.117:0/667466199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5894103110 0x7f58941034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.690+0000 7f5899200700 1 -- 172.21.15.117:0/667466199 >> 172.21.15.117:0/667466199 conn(0x7f58940fe880 msgr2=0x7f5894100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.690+0000 7f5899200700 1 -- 172.21.15.117:0/667466199 shutdown_connections 2024-04-20T11:26:07.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.690+0000 7f5899200700 1 -- 172.21.15.117:0/667466199 wait complete. 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5899200700 1 Processor -- start 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5899200700 1 -- start start 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5899200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894103110 0x7f5894198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5899200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5894104a00 0x7f58941993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5899200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5894199ac0 con 0x7f5894103110 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5899200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589419d740 con 0x7f5894104a00 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5891e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894103110 0x7f5894198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5891e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894103110 0x7f5894198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34182/0 (socket says 172.21.15.117:34182) 2024-04-20T11:26:07.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.692+0000 7f5891e00700 1 -- 172.21.15.117:0/2229153785 learned_addr learned my addr 172.21.15.117:0/2229153785 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:07.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.693+0000 7f5891400700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5894104a00 0x7f58941993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.693+0000 7f5891e00700 1 -- 172.21.15.117:0/2229153785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5894104a00 msgr2=0x7f58941993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.693+0000 7f5891e00700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5894104a00 0x7f58941993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.693+0000 7f5891e00700 1 -- 172.21.15.117:0/2229153785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c0095f0 con 0x7f5894103110 2024-04-20T11:26:07.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.693+0000 7f5891e00700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894103110 0x7f5894198ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f588400b4b0 tx=0x7f588400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.694+0000 7f588aa00700 1 -- 172.21.15.117:0/2229153785 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5884011800 con 0x7f5894103110 2024-04-20T11:26:07.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.694+0000 7f588aa00700 1 -- 172.21.15.117:0/2229153785 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5884011e20 con 0x7f5894103110 2024-04-20T11:26:07.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.694+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589419da20 con 0x7f5894103110 2024-04-20T11:26:07.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.694+0000 7f588aa00700 1 -- 172.21.15.117:0/2229153785 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f588400f2e0 con 0x7f5894103110 2024-04-20T11:26:07.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.694+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589419dff0 con 0x7f5894103110 2024-04-20T11:26:07.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.695+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589410b460 con 0x7f5894103110 2024-04-20T11:26:07.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.696+0000 7f588aa00700 1 -- 172.21.15.117:0/2229153785 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5884010380 con 0x7f5894103110 2024-04-20T11:26:07.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.697+0000 7f588aa00700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f587806ecd0 0x7f5878071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.697+0000 7f588aa00700 1 -- 172.21.15.117:0/2229153785 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f588408ea00 con 0x7f5894103110 2024-04-20T11:26:07.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.697+0000 7f5891400700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f587806ecd0 0x7f5878071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.698+0000 7f5891400700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f587806ecd0 0x7f5878071190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f587c005100 tx=0x7f587c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.702+0000 7f588aa00700 1 -- 172.21.15.117:0/2229153785 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5884057870 con 0x7f5894103110 2024-04-20T11:26:07.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.993+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5894104510 con 0x7f5894103110 2024-04-20T11:26:07.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.994+0000 7f588aa00700 1 -- 172.21.15.117:0/2229153785 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f588405af30 con 0x7f5894103110 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:26:07.997 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:26:07.998 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:26:07.999 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:26:08.000 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:26:08.000 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:26:08.000 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:26:08.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.997+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f587806ecd0 msgr2=0x7f5878071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.997+0000 7f5899200700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f587806ecd0 0x7f5878071190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f587c005100 tx=0x7f587c01a040 comp rx=0 tx=0).stop 2024-04-20T11:26:08.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.997+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894103110 msgr2=0x7f5894198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.997+0000 7f5899200700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894103110 0x7f5894198ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f588400b4b0 tx=0x7f588400b870 comp rx=0 tx=0).stop 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.998+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 shutdown_connections 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.998+0000 7f5899200700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f587806ecd0 0x7f5878071190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.998+0000 7f5899200700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894103110 0x7f5894198ea0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.998+0000 7f5899200700 1 --2- 172.21.15.117:0/2229153785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5894104a00 0x7f58941993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.998+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 >> 172.21.15.117:0/2229153785 conn(0x7f58940fe880 msgr2=0x7f58940ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.998+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 shutdown_connections 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:07.998+0000 7f5899200700 1 -- 172.21.15.117:0/2229153785 wait complete. 2024-04-20T11:26:08.001 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:26:08.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.146+0000 7f2a3ca00700 1 -- 172.21.15.117:0/678122950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38103110 msgr2=0x7f2a381034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.146+0000 7f2a3ca00700 1 --2- 172.21.15.117:0/678122950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38103110 0x7f2a381034f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009910 tx=0x7f2a28009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:08.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.146+0000 7f2a3ca00700 1 -- 172.21.15.117:0/678122950 shutdown_connections 2024-04-20T11:26:08.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.146+0000 7f2a3ca00700 1 --2- 172.21.15.117:0/678122950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a38104a00 0x7f2a38104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.146+0000 7f2a3ca00700 1 --2- 172.21.15.117:0/678122950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38103110 0x7f2a381034f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.146+0000 7f2a3ca00700 1 -- 172.21.15.117:0/678122950 >> 172.21.15.117:0/678122950 conn(0x7f2a380fe880 msgr2=0x7f2a38100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.146+0000 7f2a3ca00700 1 -- 172.21.15.117:0/678122950 shutdown_connections 2024-04-20T11:26:08.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.146+0000 7f2a3ca00700 1 -- 172.21.15.117:0/678122950 wait complete. 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.147+0000 7f2a3ca00700 1 Processor -- start 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.147+0000 7f2a3ca00700 1 -- start start 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.148+0000 7f2a3ca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a38103110 0x7f2a38198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.148+0000 7f2a3ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38104a00 0x7f2a38199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.148+0000 7f2a3ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a381999e0 con 0x7f2a38104a00 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.148+0000 7f2a3ca00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819d660 con 0x7f2a38103110 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.148+0000 7f2a35400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38104a00 0x7f2a38199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.148+0000 7f2a35400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38104a00 0x7f2a38199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34190/0 (socket says 172.21.15.117:34190) 2024-04-20T11:26:08.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.148+0000 7f2a35400700 1 -- 172.21.15.117:0/788107364 learned_addr learned my addr 172.21.15.117:0/788107364 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:08.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.148+0000 7f2a35e00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a38103110 0x7f2a38198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a35400700 1 -- 172.21.15.117:0/788107364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a38103110 msgr2=0x7f2a38198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a35400700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a38103110 0x7f2a38198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a35400700 1 -- 172.21.15.117:0/788107364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a280095f0 con 0x7f2a38104a00 2024-04-20T11:26:08.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a35400700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38104a00 0x7f2a38199300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400d8d0 tx=0x7f2a2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a2ea00700 1 -- 172.21.15.117:0/788107364 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a24009940 con 0x7f2a38104a00 2024-04-20T11:26:08.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a2ea00700 1 -- 172.21.15.117:0/788107364 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a24010460 con 0x7f2a38104a00 2024-04-20T11:26:08.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a2ea00700 1 -- 172.21.15.117:0/788107364 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2400f5f0 con 0x7f2a38104a00 2024-04-20T11:26:08.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3819d940 con 0x7f2a38104a00 2024-04-20T11:26:08.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.149+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3819df10 con 0x7f2a38104a00 2024-04-20T11:26:08.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.150+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3810b460 con 0x7f2a38104a00 2024-04-20T11:26:08.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.152+0000 7f2a2ea00700 1 -- 172.21.15.117:0/788107364 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2a240105d0 con 0x7f2a38104a00 2024-04-20T11:26:08.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.153+0000 7f2a2ea00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a1c06ecc0 0x7f2a1c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.153+0000 7f2a2ea00700 1 -- 172.21.15.117:0/788107364 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2a2408ede0 con 0x7f2a38104a00 2024-04-20T11:26:08.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.153+0000 7f2a35e00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a1c06ecc0 0x7f2a1c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.154+0000 7f2a35e00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a1c06ecc0 0x7f2a1c071180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800b5c0 tx=0x7f2a2801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.158+0000 7f2a2ea00700 1 -- 172.21.15.117:0/788107364 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2a24057ba0 con 0x7f2a38104a00 2024-04-20T11:26:08.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.401+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a3819a080 con 0x7f2a1c06ecc0 2024-04-20T11:26:08.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.403+0000 7f2a2ea00700 1 -- 172.21.15.117:0/788107364 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2a3819a080 con 0x7f2a1c06ecc0 2024-04-20T11:26:08.402 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:26:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:26:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:26:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:26:08.403 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a1c06ecc0 msgr2=0x7f2a1c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a1c06ecc0 0x7f2a1c071180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800b5c0 tx=0x7f2a2801a040 comp rx=0 tx=0).stop 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38104a00 msgr2=0x7f2a38199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38104a00 0x7f2a38199300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400d8d0 tx=0x7f2a2400dc90 comp rx=0 tx=0).stop 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 shutdown_connections 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a1c06ecc0 0x7f2a1c071180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a38103110 0x7f2a38198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 --2- 172.21.15.117:0/788107364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a38104a00 0x7f2a38199300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 >> 172.21.15.117:0/788107364 conn(0x7f2a380fe880 msgr2=0x7f2a380ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.405+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 shutdown_connections 2024-04-20T11:26:08.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.406+0000 7f2a3ca00700 1 -- 172.21.15.117:0/788107364 wait complete. 2024-04-20T11:26:08.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.561+0000 7fbfac400700 1 -- 172.21.15.117:0/3362767254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa4100e10 msgr2=0x7fbfa41011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.561+0000 7fbfac400700 1 --2- 172.21.15.117:0/3362767254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa4100e10 0x7fbfa41011f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009910 tx=0x7fbf98009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:08.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.561+0000 7fbfac400700 1 -- 172.21.15.117:0/3362767254 shutdown_connections 2024-04-20T11:26:08.561 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:08 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1670393490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:26:08.561 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:08 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2229153785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:26:08.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.561+0000 7fbfac400700 1 --2- 172.21.15.117:0/3362767254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfa41017c0 0x7fbfa4105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.561+0000 7fbfac400700 1 --2- 172.21.15.117:0/3362767254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa4100e10 0x7fbfa41011f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.561+0000 7fbfac400700 1 -- 172.21.15.117:0/3362767254 >> 172.21.15.117:0/3362767254 conn(0x7fbfa40fc6c0 msgr2=0x7fbfa40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.562+0000 7fbfac400700 1 -- 172.21.15.117:0/3362767254 shutdown_connections 2024-04-20T11:26:08.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.562+0000 7fbfac400700 1 -- 172.21.15.117:0/3362767254 wait complete. 2024-04-20T11:26:08.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.563+0000 7fbfac400700 1 Processor -- start 2024-04-20T11:26:08.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.563+0000 7fbfac400700 1 -- start start 2024-04-20T11:26:08.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.564+0000 7fbfac400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfa4100e10 0x7fbfa419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.564+0000 7fbfac400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa41017c0 0x7fbfa419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.564+0000 7fbfac400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419d360 con 0x7fbfa41017c0 2024-04-20T11:26:08.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.564+0000 7fbfac400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4196940 con 0x7fbfa4100e10 2024-04-20T11:26:08.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.564+0000 7fbfa9000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa41017c0 0x7fbfa419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.564+0000 7fbfa9000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa41017c0 0x7fbfa419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34214/0 (socket says 172.21.15.117:34214) 2024-04-20T11:26:08.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.564+0000 7fbfa9000700 1 -- 172.21.15.117:0/1819139050 learned_addr learned my addr 172.21.15.117:0/1819139050 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:08.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.564+0000 7fbfa9a00700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfa4100e10 0x7fbfa419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfa9000700 1 -- 172.21.15.117:0/1819139050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfa4100e10 msgr2=0x7fbfa419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfa9000700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfa4100e10 0x7fbfa419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfa9000700 1 -- 172.21.15.117:0/1819139050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf980095f0 con 0x7fbfa41017c0 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfa9a00700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfa4100e10 0x7fbfa419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfa9000700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa41017c0 0x7fbfa419cd40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000c8f0 tx=0x7fbf9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfa2000700 1 -- 172.21.15.117:0/1819139050 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf90007a50 con 0x7fbfa41017c0 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfa2000700 1 -- 172.21.15.117:0/1819139050 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf90007bb0 con 0x7fbfa41017c0 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfa2000700 1 -- 172.21.15.117:0/1819139050 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf90018630 con 0x7fbfa41017c0 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.565+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa4196c20 con 0x7fbfa41017c0 2024-04-20T11:26:08.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.566+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa4197170 con 0x7fbfa41017c0 2024-04-20T11:26:08.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.567+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa41091b0 con 0x7fbfa41017c0 2024-04-20T11:26:08.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.568+0000 7fbfa2000700 1 -- 172.21.15.117:0/1819139050 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbf9000f450 con 0x7fbfa41017c0 2024-04-20T11:26:08.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.569+0000 7fbfa2000700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf8c06ecc0 0x7fbf8c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.569+0000 7fbfa2000700 1 -- 172.21.15.117:0/1819139050 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbf9008ee10 con 0x7fbfa41017c0 2024-04-20T11:26:08.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.569+0000 7fbfa9a00700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf8c06ecc0 0x7fbf8c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.570+0000 7fbfa9a00700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf8c06ecc0 0x7fbf8c071180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b5c0 tx=0x7fbf9801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.574+0000 7fbfa2000700 1 -- 172.21.15.117:0/1819139050 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbf9005b720 con 0x7fbfa41017c0 2024-04-20T11:26:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:08 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1670393490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:26:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:08 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2229153785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:26:08.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.881+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbfa41022f0 con 0x7fbfa41017c0 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.882+0000 7fbfa2000700 1 -- 172.21.15.117:0/1819139050 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fbf9005b2b0 con 0x7fbfa41017c0 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.884+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf8c06ecc0 msgr2=0x7fbf8c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.884+0000 7fbfac400700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf8c06ecc0 0x7fbf8c071180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b5c0 tx=0x7fbf9801a040 comp rx=0 tx=0).stop 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.884+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa41017c0 msgr2=0x7fbfa419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.884+0000 7fbfac400700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa41017c0 0x7fbfa419cd40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000c8f0 tx=0x7fbf9000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.885+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 shutdown_connections 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.885+0000 7fbfac400700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf8c06ecc0 0x7fbf8c071180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.885+0000 7fbfac400700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfa4100e10 0x7fbfa419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.885+0000 7fbfac400700 1 --2- 172.21.15.117:0/1819139050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfa41017c0 0x7fbfa419cd40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.885+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 >> 172.21.15.117:0/1819139050 conn(0x7fbfa40fc6c0 msgr2=0x7fbfa40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.885+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 shutdown_connections 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:08.885+0000 7fbfac400700 1 -- 172.21.15.117:0/1819139050 wait complete. 2024-04-20T11:26:09.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:09 smithi117 ceph-mon[99036]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:09.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:09 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1819139050' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:26:09.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:09 smithi117 ceph-mon[99036]: pgmap v411: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:09 smithi173 ceph-mon[74165]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:09 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1819139050' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:26:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:09 smithi173 ceph-mon[74165]: pgmap v411: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:12.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:12 smithi117 ceph-mon[99036]: pgmap v412: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:12 smithi173 ceph-mon[74165]: pgmap v412: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:26:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:26:14.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:14 smithi117 ceph-mon[99036]: pgmap v413: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:14 smithi173 ceph-mon[74165]: pgmap v413: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:15 smithi117 ceph-mon[99036]: pgmap v414: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:15 smithi173 ceph-mon[74165]: pgmap v414: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:18 smithi117 ceph-mon[99036]: pgmap v415: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:18 smithi173 ceph-mon[74165]: pgmap v415: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:19.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:19 smithi117 ceph-mon[99036]: pgmap v416: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:19 smithi173 ceph-mon[74165]: pgmap v416: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:22.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:22 smithi117 ceph-mon[99036]: pgmap v417: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:22.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:22 smithi173 ceph-mon[74165]: pgmap v417: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:23 smithi117 ceph-mon[99036]: pgmap v418: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:23 smithi173 ceph-mon[74165]: pgmap v418: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:26.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:26 smithi117 ceph-mon[99036]: pgmap v419: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:26 smithi173 ceph-mon[74165]: pgmap v419: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:27 smithi117 ceph-mon[99036]: pgmap v420: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:27 smithi173 ceph-mon[74165]: pgmap v420: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:26:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:26:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:29 smithi117 ceph-mon[99036]: pgmap v421: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:29 smithi173 ceph-mon[74165]: pgmap v421: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:32 smithi117 ceph-mon[99036]: pgmap v422: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:32 smithi173 ceph-mon[74165]: pgmap v422: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:33.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:33 smithi117 ceph-mon[99036]: pgmap v423: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:33 smithi173 ceph-mon[74165]: pgmap v423: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:35.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:35 smithi117 ceph-mon[99036]: pgmap v424: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:35 smithi173 ceph-mon[74165]: pgmap v424: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:37.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:37 smithi117 ceph-mon[99036]: pgmap v425: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:37.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:37 smithi173 ceph-mon[74165]: pgmap v425: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:39.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.054+0000 7f0cb7200700 1 -- 172.21.15.117:0/3216392248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0102660 msgr2=0x7f0cb010ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.054+0000 7f0cb7200700 1 --2- 172.21.15.117:0/3216392248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0102660 0x7f0cb010ab50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009910 tx=0x7f0c98009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:39.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.054+0000 7f0cb7200700 1 -- 172.21.15.117:0/3216392248 shutdown_connections 2024-04-20T11:26:39.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.054+0000 7f0cb7200700 1 --2- 172.21.15.117:0/3216392248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0102660 0x7f0cb010ab50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.054+0000 7f0cb7200700 1 --2- 172.21.15.117:0/3216392248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cb0101d40 0x7f0cb0102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.054+0000 7f0cb7200700 1 -- 172.21.15.117:0/3216392248 >> 172.21.15.117:0/3216392248 conn(0x7f0cb0076750 msgr2=0x7f0cb00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.054+0000 7f0cb7200700 1 -- 172.21.15.117:0/3216392248 shutdown_connections 2024-04-20T11:26:39.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.054+0000 7f0cb7200700 1 -- 172.21.15.117:0/3216392248 wait complete. 2024-04-20T11:26:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.055+0000 7f0cb7200700 1 Processor -- start 2024-04-20T11:26:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.055+0000 7f0cb7200700 1 -- start start 2024-04-20T11:26:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cb7200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0101d40 0x7f0cb00ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cb7200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cb0102660 0x7f0cb00ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cb7200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0100420 con 0x7f0cb0101d40 2024-04-20T11:26:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cb7200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0100560 con 0x7f0cb0102660 2024-04-20T11:26:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cafe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0101d40 0x7f0cb00ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cafe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0101d40 0x7f0cb00ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56302/0 (socket says 172.21.15.117:56302) 2024-04-20T11:26:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cafe00700 1 -- 172.21.15.117:0/3411989140 learned_addr learned my addr 172.21.15.117:0/3411989140 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0ca7400700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cb0102660 0x7f0cb00ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cafe00700 1 -- 172.21.15.117:0/3411989140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cb0102660 msgr2=0x7f0cb00ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cafe00700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cb0102660 0x7f0cb00ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0cafe00700 1 -- 172.21.15.117:0/3411989140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980095f0 con 0x7f0cb0101d40 2024-04-20T11:26:39.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.056+0000 7f0ca7400700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cb0102660 0x7f0cb00ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:39.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.057+0000 7f0cafe00700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0101d40 0x7f0cb00ff9a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00060b0 tx=0x7f0ca000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.057+0000 7f0cad600700 1 -- 172.21.15.117:0/3411989140 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ca000dda0 con 0x7f0cb0101d40 2024-04-20T11:26:39.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.057+0000 7f0cad600700 1 -- 172.21.15.117:0/3411989140 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ca000f040 con 0x7f0cb0101d40 2024-04-20T11:26:39.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.057+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb01a2d00 con 0x7f0cb0101d40 2024-04-20T11:26:39.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.057+0000 7f0cad600700 1 -- 172.21.15.117:0/3411989140 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ca0014790 con 0x7f0cb0101d40 2024-04-20T11:26:39.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.057+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb01a3060 con 0x7f0cb0101d40 2024-04-20T11:26:39.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.058+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb0068e50 con 0x7f0cb0101d40 2024-04-20T11:26:39.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.059+0000 7f0cad600700 1 -- 172.21.15.117:0/3411989140 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0ca0014980 con 0x7f0cb0101d40 2024-04-20T11:26:39.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.060+0000 7f0cad600700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c9406ecc0 0x7f0c94071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.060+0000 7f0cad600700 1 -- 172.21.15.117:0/3411989140 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0ca008e950 con 0x7f0cb0101d40 2024-04-20T11:26:39.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.061+0000 7f0ca7400700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c9406ecc0 0x7f0c94071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.061+0000 7f0ca7400700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c9406ecc0 0x7f0c94071180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cb01010d0 tx=0x7f0c9800b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.065+0000 7f0cad600700 1 -- 172.21.15.117:0/3411989140 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0ca0057740 con 0x7f0cb0101d40 2024-04-20T11:26:39.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.304+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cb0100720 con 0x7f0c9406ecc0 2024-04-20T11:26:39.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.307+0000 7f0cad600700 1 -- 172.21.15.117:0/3411989140 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0cb0100720 con 0x7f0c9406ecc0 2024-04-20T11:26:39.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c9406ecc0 msgr2=0x7f0c94071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c9406ecc0 0x7f0c94071180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cb01010d0 tx=0x7f0c9800b560 comp rx=0 tx=0).stop 2024-04-20T11:26:39.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0101d40 msgr2=0x7f0cb00ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0101d40 0x7f0cb00ff9a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00060b0 tx=0x7f0ca000d3b0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 shutdown_connections 2024-04-20T11:26:39.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c9406ecc0 0x7f0c94071180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cb0101d40 0x7f0cb00ff9a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 --2- 172.21.15.117:0/3411989140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cb0102660 0x7f0cb00ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 >> 172.21.15.117:0/3411989140 conn(0x7f0cb0076750 msgr2=0x7f0cb0103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 shutdown_connections 2024-04-20T11:26:39.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.310+0000 7f0cb7200700 1 -- 172.21.15.117:0/3411989140 wait complete. 2024-04-20T11:26:39.326 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:26:39.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.467+0000 7f9138e00700 1 -- 172.21.15.117:0/1153626756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134107520 msgr2=0x7f9134107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.467+0000 7f9138e00700 1 --2- 172.21.15.117:0/1153626756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134107520 0x7f9134107900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f911c009910 tx=0x7f911c009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:39.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.467+0000 7f9138e00700 1 -- 172.21.15.117:0/1153626756 shutdown_connections 2024-04-20T11:26:39.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.467+0000 7f9138e00700 1 --2- 172.21.15.117:0/1153626756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134108e10 0x7f9134109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.467+0000 7f9138e00700 1 --2- 172.21.15.117:0/1153626756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134107520 0x7f9134107900 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.467+0000 7f9138e00700 1 -- 172.21.15.117:0/1153626756 >> 172.21.15.117:0/1153626756 conn(0x7f91340768f0 msgr2=0x7f9134078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.468+0000 7f9138e00700 1 -- 172.21.15.117:0/1153626756 shutdown_connections 2024-04-20T11:26:39.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.468+0000 7f9138e00700 1 -- 172.21.15.117:0/1153626756 wait complete. 2024-04-20T11:26:39.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.468+0000 7f9138e00700 1 Processor -- start 2024-04-20T11:26:39.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.469+0000 7f9138e00700 1 -- start start 2024-04-20T11:26:39.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.469+0000 7f9138e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134107520 0x7f913419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.469+0000 7f9138e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134108e10 0x7f913419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.469+0000 7f9138e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913419de90 con 0x7f9134108e10 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.469+0000 7f9138e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91341a1b60 con 0x7f9134107520 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.469+0000 7f9131400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134108e10 0x7f913419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.469+0000 7f9131400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134108e10 0x7f913419d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56324/0 (socket says 172.21.15.117:56324) 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.469+0000 7f9131400700 1 -- 172.21.15.117:0/2322968433 learned_addr learned my addr 172.21.15.117:0/2322968433 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f9131e00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134107520 0x7f913419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f9131400700 1 -- 172.21.15.117:0/2322968433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134107520 msgr2=0x7f913419d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f9131400700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134107520 0x7f913419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f9131400700 1 -- 172.21.15.117:0/2322968433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0095f0 con 0x7f9134108e10 2024-04-20T11:26:39.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f9131400700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134108e10 0x7f913419d7b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f912400b4b0 tx=0x7f912400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f912aa00700 1 -- 172.21.15.117:0/2322968433 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9124011800 con 0x7f9134108e10 2024-04-20T11:26:39.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f912aa00700 1 -- 172.21.15.117:0/2322968433 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9124011e20 con 0x7f9134108e10 2024-04-20T11:26:39.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91341a1e40 con 0x7f9134108e10 2024-04-20T11:26:39.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.470+0000 7f912aa00700 1 -- 172.21.15.117:0/2322968433 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912400f2e0 con 0x7f9134108e10 2024-04-20T11:26:39.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.471+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91341a2410 con 0x7f9134108e10 2024-04-20T11:26:39.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.472+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913410f870 con 0x7f9134108e10 2024-04-20T11:26:39.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.473+0000 7f912aa00700 1 -- 172.21.15.117:0/2322968433 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9124011960 con 0x7f9134108e10 2024-04-20T11:26:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.473+0000 7f912aa00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f911806ecc0 0x7f9118071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.474+0000 7f912aa00700 1 -- 172.21.15.117:0/2322968433 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f912408ea30 con 0x7f9134108e10 2024-04-20T11:26:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.474+0000 7f9131e00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f911806ecc0 0x7f9118071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.475+0000 7f9131e00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f911806ecc0 0x7f9118071180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f911c00b5c0 tx=0x7f911c005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.478+0000 7f912aa00700 1 -- 172.21.15.117:0/2322968433 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9124057810 con 0x7f9134108e10 2024-04-20T11:26:39.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:39 smithi117 ceph-mon[99036]: pgmap v426: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:39.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:39 smithi173 ceph-mon[74165]: pgmap v426: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:39.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.725+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f913419e580 con 0x7f911806ecc0 2024-04-20T11:26:39.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.727+0000 7f912aa00700 1 -- 172.21.15.117:0/2322968433 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f913419e580 con 0x7f911806ecc0 2024-04-20T11:26:39.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f911806ecc0 msgr2=0x7f9118071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f911806ecc0 0x7f9118071180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f911c00b5c0 tx=0x7f911c005960 comp rx=0 tx=0).stop 2024-04-20T11:26:39.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134108e10 msgr2=0x7f913419d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134108e10 0x7f913419d7b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f912400b4b0 tx=0x7f912400b870 comp rx=0 tx=0).stop 2024-04-20T11:26:39.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 shutdown_connections 2024-04-20T11:26:39.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f911806ecc0 0x7f9118071180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134107520 0x7f913419d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 --2- 172.21.15.117:0/2322968433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134108e10 0x7f913419d7b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.730+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 >> 172.21.15.117:0/2322968433 conn(0x7f91340768f0 msgr2=0x7f9134078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.731+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 shutdown_connections 2024-04-20T11:26:39.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.731+0000 7f9138e00700 1 -- 172.21.15.117:0/2322968433 wait complete. 2024-04-20T11:26:39.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.883+0000 7ff97c000700 1 -- 172.21.15.117:0/1802988620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97406feb0 msgr2=0x7ff974070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.883+0000 7ff97c000700 1 --2- 172.21.15.117:0/1802988620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97406feb0 0x7ff974070330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff960009910 tx=0x7ff960009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:39.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.883+0000 7ff97c000700 1 -- 172.21.15.117:0/1802988620 shutdown_connections 2024-04-20T11:26:39.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.883+0000 7ff97c000700 1 --2- 172.21.15.117:0/1802988620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97406feb0 0x7ff974070330 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.883+0000 7ff97c000700 1 --2- 172.21.15.117:0/1802988620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff97410f530 0x7ff97410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.883+0000 7ff97c000700 1 -- 172.21.15.117:0/1802988620 >> 172.21.15.117:0/1802988620 conn(0x7ff974072340 msgr2=0x7ff974072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.883+0000 7ff97c000700 1 -- 172.21.15.117:0/1802988620 shutdown_connections 2024-04-20T11:26:39.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.883+0000 7ff97c000700 1 -- 172.21.15.117:0/1802988620 wait complete. 2024-04-20T11:26:39.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.884+0000 7ff97c000700 1 Processor -- start 2024-04-20T11:26:39.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.884+0000 7ff97c000700 1 -- start start 2024-04-20T11:26:39.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.884+0000 7ff97c000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff97406feb0 0x7ff974108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.884+0000 7ff97c000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97410f530 0x7ff974108890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.884+0000 7ff97c000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974108dd0 con 0x7ff97410f530 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.884+0000 7ff97c000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974108f10 con 0x7ff97406feb0 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.885+0000 7ff978c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97410f530 0x7ff974108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.885+0000 7ff978c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97410f530 0x7ff974108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56338/0 (socket says 172.21.15.117:56338) 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.885+0000 7ff978c00700 1 -- 172.21.15.117:0/3329896353 learned_addr learned my addr 172.21.15.117:0/3329896353 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.885+0000 7ff979600700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff97406feb0 0x7ff974108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.885+0000 7ff978c00700 1 -- 172.21.15.117:0/3329896353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff97406feb0 msgr2=0x7ff974108350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.885+0000 7ff978c00700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff97406feb0 0x7ff974108350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.885+0000 7ff978c00700 1 -- 172.21.15.117:0/3329896353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9600095f0 con 0x7ff97410f530 2024-04-20T11:26:39.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.886+0000 7ff978c00700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97410f530 0x7ff974108890 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff9600092e0 tx=0x7ff96000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.886+0000 7ff972000700 1 -- 172.21.15.117:0/3329896353 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96001d070 con 0x7ff97410f530 2024-04-20T11:26:39.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.886+0000 7ff972000700 1 -- 172.21.15.117:0/3329896353 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9600056e0 con 0x7ff97410f530 2024-04-20T11:26:39.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.886+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff974109190 con 0x7ff97410f530 2024-04-20T11:26:39.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.886+0000 7ff972000700 1 -- 172.21.15.117:0/3329896353 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96000fe80 con 0x7ff97410f530 2024-04-20T11:26:39.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.886+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9741ab710 con 0x7ff97410f530 2024-04-20T11:26:39.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.887+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff974111870 con 0x7ff97410f530 2024-04-20T11:26:39.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.888+0000 7ff972000700 1 -- 172.21.15.117:0/3329896353 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff96000f6f0 con 0x7ff97410f530 2024-04-20T11:26:39.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.889+0000 7ff972000700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff95c06ecd0 0x7ff95c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.889+0000 7ff972000700 1 -- 172.21.15.117:0/3329896353 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff960090230 con 0x7ff97410f530 2024-04-20T11:26:39.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.889+0000 7ff979600700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff95c06ecd0 0x7ff95c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.890+0000 7ff979600700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff95c06ecd0 0x7ff95c071190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff968005950 tx=0x7ff9680058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:39.894+0000 7ff972000700 1 -- 172.21.15.117:0/3329896353 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff960058ff0 con 0x7ff97410f530 2024-04-20T11:26:40.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.111+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff974109880 con 0x7ff95c06ecd0 2024-04-20T11:26:40.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.143+0000 7ff972000700 1 -- 172.21.15.117:0/3329896353 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff974109880 con 0x7ff95c06ecd0 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (15m) 6m ago 29m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (29m) 6m ago 29m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (28m) 6m ago 28m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (11m) 6m ago 29m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (11m) 6m ago 28m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (14m) 6m ago 28m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (6m) 6m ago 21m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (7m) 6m ago 22m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (21m) 6m ago 21m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:26:40.144 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (6m) 6m ago 21m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (18m) 6m ago 30m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (16m) 6m ago 27m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (13m) 6m ago 31m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (12m) 6m ago 27m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (16m) 6m ago 29m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (16m) 6m ago 27m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (10m) 6m ago 26m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (10m) 6m ago 26m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (9m) 6m ago 25m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (9m) 6m ago 24m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (8m) 6m ago 24m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (7m) 6m ago 23m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:26:40.145 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (15m) 6m ago 28m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:26:40.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.146+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff95c06ecd0 msgr2=0x7ff95c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.146+0000 7ff97c000700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff95c06ecd0 0x7ff95c071190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff968005950 tx=0x7ff9680058e0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97410f530 msgr2=0x7ff974108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97410f530 0x7ff974108890 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff9600092e0 tx=0x7ff96000be20 comp rx=0 tx=0).stop 2024-04-20T11:26:40.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 shutdown_connections 2024-04-20T11:26:40.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff95c06ecd0 0x7ff95c071190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff97406feb0 0x7ff974108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 --2- 172.21.15.117:0/3329896353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff97410f530 0x7ff974108890 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 >> 172.21.15.117:0/3329896353 conn(0x7ff974072340 msgr2=0x7ff97410e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 shutdown_connections 2024-04-20T11:26:40.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.147+0000 7ff97c000700 1 -- 172.21.15.117:0/3329896353 wait complete. 2024-04-20T11:26:40.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.303+0000 7f5493c00700 1 -- 172.21.15.117:0/123377290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c103040 msgr2=0x7f548c103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.303+0000 7f5493c00700 1 --2- 172.21.15.117:0/123377290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c103040 0x7f548c103420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5480009910 tx=0x7f5480009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:40.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.303+0000 7f5493c00700 1 -- 172.21.15.117:0/123377290 shutdown_connections 2024-04-20T11:26:40.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.303+0000 7f5493c00700 1 --2- 172.21.15.117:0/123377290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c1039f0 0x7f548c107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.303+0000 7f5493c00700 1 --2- 172.21.15.117:0/123377290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c103040 0x7f548c103420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.303+0000 7f5493c00700 1 -- 172.21.15.117:0/123377290 >> 172.21.15.117:0/123377290 conn(0x7f548c0fe8d0 msgr2=0x7f548c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.303+0000 7f5493c00700 1 -- 172.21.15.117:0/123377290 shutdown_connections 2024-04-20T11:26:40.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.303+0000 7f5493c00700 1 -- 172.21.15.117:0/123377290 wait complete. 2024-04-20T11:26:40.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.304+0000 7f5493c00700 1 Processor -- start 2024-04-20T11:26:40.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.304+0000 7f5493c00700 1 -- start start 2024-04-20T11:26:40.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f5493c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c103040 0x7f548c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f5493c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c1039f0 0x7f548c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f5493c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c19f5b0 con 0x7f548c1039f0 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f5493c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c198b90 con 0x7f548c103040 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f548be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c1039f0 0x7f548c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f548be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c1039f0 0x7f548c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56360/0 (socket says 172.21.15.117:56360) 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f548be00700 1 -- 172.21.15.117:0/919913529 learned_addr learned my addr 172.21.15.117:0/919913529 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f5491200700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c103040 0x7f548c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f548be00700 1 -- 172.21.15.117:0/919913529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c103040 msgr2=0x7f548c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f548be00700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c103040 0x7f548c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f548be00700 1 -- 172.21.15.117:0/919913529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54800095f0 con 0x7f548c1039f0 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.305+0000 7f5491200700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c103040 0x7f548c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:40.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.306+0000 7f548be00700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c1039f0 0x7f548c19ef90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f547800c8f0 tx=0x7f547800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.306+0000 7f5489600700 1 -- 172.21.15.117:0/919913529 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5478007a50 con 0x7f548c1039f0 2024-04-20T11:26:40.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.306+0000 7f5489600700 1 -- 172.21.15.117:0/919913529 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5478007bb0 con 0x7f548c1039f0 2024-04-20T11:26:40.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.306+0000 7f5489600700 1 -- 172.21.15.117:0/919913529 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5478018630 con 0x7f548c1039f0 2024-04-20T11:26:40.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.306+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c198e70 con 0x7f548c1039f0 2024-04-20T11:26:40.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.306+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c1993c0 con 0x7f548c1039f0 2024-04-20T11:26:40.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.307+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c10b3e0 con 0x7f548c1039f0 2024-04-20T11:26:40.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.308+0000 7f5489600700 1 -- 172.21.15.117:0/919913529 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5478007d20 con 0x7f548c1039f0 2024-04-20T11:26:40.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.309+0000 7f5489600700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f547406ecd0 0x7f5474071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.309+0000 7f5489600700 1 -- 172.21.15.117:0/919913529 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f547808eec0 con 0x7f548c1039f0 2024-04-20T11:26:40.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.310+0000 7f5491200700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f547406ecd0 0x7f5474071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.310+0000 7f5491200700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f547406ecd0 0x7f5474071190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5480009fd0 tx=0x7f548000b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.314+0000 7f5489600700 1 -- 172.21.15.117:0/919913529 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f547805b730 con 0x7f548c1039f0 2024-04-20T11:26:40.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:40 smithi117 ceph-mon[99036]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:40.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:40 smithi117 ceph-mon[99036]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:40.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:40 smithi117 ceph-mon[99036]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:40.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.626+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f548c199fe0 con 0x7f548c1039f0 2024-04-20T11:26:40.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.627+0000 7f5489600700 1 -- 172.21.15.117:0/919913529 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f547805b2c0 con 0x7f548c1039f0 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:26:40.628 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:26:40.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:26:40.629 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:26:40.629 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:26:40.629 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:26:40.629 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:26:40.629 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:26:40.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f547406ecd0 msgr2=0x7f5474071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f547406ecd0 0x7f5474071190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5480009fd0 tx=0x7f548000b560 comp rx=0 tx=0).stop 2024-04-20T11:26:40.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c1039f0 msgr2=0x7f548c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c1039f0 0x7f548c19ef90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f547800c8f0 tx=0x7f547800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 shutdown_connections 2024-04-20T11:26:40.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f547406ecd0 0x7f5474071190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c103040 0x7f548c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 --2- 172.21.15.117:0/919913529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c1039f0 0x7f548c19ef90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 >> 172.21.15.117:0/919913529 conn(0x7f548c0fe8d0 msgr2=0x7f548c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 shutdown_connections 2024-04-20T11:26:40.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.630+0000 7f5493c00700 1 -- 172.21.15.117:0/919913529 wait complete. 2024-04-20T11:26:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:40 smithi173 ceph-mon[74165]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:40 smithi173 ceph-mon[74165]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:40 smithi173 ceph-mon[74165]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:40.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.783+0000 7fd270e00700 1 -- 172.21.15.117:0/2046455225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 msgr2=0x7fd26c103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.783+0000 7fd270e00700 1 --2- 172.21.15.117:0/2046455225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 0x7fd26c103200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009910 tx=0x7fd25c009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:40.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.784+0000 7fd270e00700 1 -- 172.21.15.117:0/2046455225 shutdown_connections 2024-04-20T11:26:40.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.784+0000 7fd270e00700 1 --2- 172.21.15.117:0/2046455225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd26c1037d0 0x7fd26c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.784+0000 7fd270e00700 1 --2- 172.21.15.117:0/2046455225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 0x7fd26c103200 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.784+0000 7fd270e00700 1 -- 172.21.15.117:0/2046455225 >> 172.21.15.117:0/2046455225 conn(0x7fd26c0fe6d0 msgr2=0x7fd26c100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.784+0000 7fd270e00700 1 -- 172.21.15.117:0/2046455225 shutdown_connections 2024-04-20T11:26:40.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.784+0000 7fd270e00700 1 -- 172.21.15.117:0/2046455225 wait complete. 2024-04-20T11:26:40.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.785+0000 7fd270e00700 1 Processor -- start 2024-04-20T11:26:40.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.785+0000 7fd270e00700 1 -- start start 2024-04-20T11:26:40.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.785+0000 7fd270e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 0x7fd26c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.785+0000 7fd270e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd26c1037d0 0x7fd26c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.785+0000 7fd270e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c19f3b0 con 0x7fd26c102e20 2024-04-20T11:26:40.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.785+0000 7fd270e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c198990 con 0x7fd26c1037d0 2024-04-20T11:26:40.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.786+0000 7fd26b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 0x7fd26c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.786+0000 7fd26b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 0x7fd26c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56372/0 (socket says 172.21.15.117:56372) 2024-04-20T11:26:40.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.786+0000 7fd26b400700 1 -- 172.21.15.117:0/2812676123 learned_addr learned my addr 172.21.15.117:0/2812676123 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:40.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.786+0000 7fd263e00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd26c1037d0 0x7fd26c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.786+0000 7fd26b400700 1 -- 172.21.15.117:0/2812676123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd26c1037d0 msgr2=0x7fd26c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.786+0000 7fd26b400700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd26c1037d0 0x7fd26c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.786+0000 7fd26b400700 1 -- 172.21.15.117:0/2812676123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0095f0 con 0x7fd26c102e20 2024-04-20T11:26:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.787+0000 7fd26b400700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 0x7fd26c19e850 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd25c000c00 tx=0x7fd25c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.787+0000 7fd268c00700 1 -- 172.21.15.117:0/2812676123 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd25c01d070 con 0x7fd26c102e20 2024-04-20T11:26:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.787+0000 7fd268c00700 1 -- 172.21.15.117:0/2812676123 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd25c0056e0 con 0x7fd26c102e20 2024-04-20T11:26:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.787+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c198c10 con 0x7fd26c102e20 2024-04-20T11:26:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.787+0000 7fd268c00700 1 -- 172.21.15.117:0/2812676123 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd25c00fe80 con 0x7fd26c102e20 2024-04-20T11:26:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.787+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c199100 con 0x7fd26c102e20 2024-04-20T11:26:40.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.788+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c10b1c0 con 0x7fd26c102e20 2024-04-20T11:26:40.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.789+0000 7fd268c00700 1 -- 172.21.15.117:0/2812676123 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd25c00f6f0 con 0x7fd26c102e20 2024-04-20T11:26:40.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.790+0000 7fd268c00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd25006ecc0 0x7fd250071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.790+0000 7fd268c00700 1 -- 172.21.15.117:0/2812676123 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd25c090190 con 0x7fd26c102e20 2024-04-20T11:26:40.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.791+0000 7fd263e00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd25006ecc0 0x7fd250071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.791+0000 7fd263e00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd25006ecc0 0x7fd250071180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd254005950 tx=0x7fd2540058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:40.795+0000 7fd268c00700 1 -- 172.21.15.117:0/2812676123 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd25c05cb20 con 0x7fd26c102e20 2024-04-20T11:26:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.067+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd26c199e50 con 0x7fd26c102e20 2024-04-20T11:26:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.067+0000 7fd268c00700 1 -- 172.21.15.117:0/2812676123 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fd25c05c6b0 con 0x7fd26c102e20 2024-04-20T11:26:41.068 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:26:41.068 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:26:41.068 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:26:41.068 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:26:41.068 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:26:41.069 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:26:41.070 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:26:41.071 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:26:41.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd25006ecc0 msgr2=0x7fd250071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd25006ecc0 0x7fd250071180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd254005950 tx=0x7fd2540058e0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 msgr2=0x7fd26c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 0x7fd26c19e850 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd25c000c00 tx=0x7fd25c00be20 comp rx=0 tx=0).stop 2024-04-20T11:26:41.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 shutdown_connections 2024-04-20T11:26:41.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd25006ecc0 0x7fd250071180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c102e20 0x7fd26c19e850 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 --2- 172.21.15.117:0/2812676123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd26c1037d0 0x7fd26c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.070+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 >> 172.21.15.117:0/2812676123 conn(0x7fd26c0fe6d0 msgr2=0x7fd26c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.071+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 shutdown_connections 2024-04-20T11:26:41.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.071+0000 7fd270e00700 1 -- 172.21.15.117:0/2812676123 wait complete. 2024-04-20T11:26:41.073 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:26:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.229+0000 7f93d3400700 1 -- 172.21.15.117:0/4238929073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 msgr2=0x7f93cc10ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.229+0000 7f93d3400700 1 --2- 172.21.15.117:0/4238929073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 0x7f93cc10ab50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009910 tx=0x7f93b8009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.229+0000 7f93d3400700 1 -- 172.21.15.117:0/4238929073 shutdown_connections 2024-04-20T11:26:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.229+0000 7f93d3400700 1 --2- 172.21.15.117:0/4238929073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 0x7f93cc10ab50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.229+0000 7f93d3400700 1 --2- 172.21.15.117:0/4238929073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc101d40 0x7f93cc102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.229+0000 7f93d3400700 1 -- 172.21.15.117:0/4238929073 >> 172.21.15.117:0/4238929073 conn(0x7f93cc076750 msgr2=0x7f93cc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.229+0000 7f93d3400700 1 -- 172.21.15.117:0/4238929073 shutdown_connections 2024-04-20T11:26:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.230+0000 7f93d3400700 1 -- 172.21.15.117:0/4238929073 wait complete. 2024-04-20T11:26:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.231+0000 7f93d3400700 1 Processor -- start 2024-04-20T11:26:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.231+0000 7f93d3400700 1 -- start start 2024-04-20T11:26:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.231+0000 7f93d3400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc101d40 0x7f93cc198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.231+0000 7f93d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 0x7f93cc199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.231+0000 7f93d3400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc199840 con 0x7f93cc102660 2024-04-20T11:26:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.231+0000 7f93d3400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc1999b0 con 0x7f93cc101d40 2024-04-20T11:26:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.232+0000 7f93cbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 0x7f93cc199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.232+0000 7f93cbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 0x7f93cc199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56400/0 (socket says 172.21.15.117:56400) 2024-04-20T11:26:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.232+0000 7f93cbe00700 1 -- 172.21.15.117:0/1930146833 learned_addr learned my addr 172.21.15.117:0/1930146833 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.232+0000 7f93d0a00700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc101d40 0x7f93cc198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.232+0000 7f93cbe00700 1 -- 172.21.15.117:0/1930146833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc101d40 msgr2=0x7f93cc198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.232+0000 7f93cbe00700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc101d40 0x7f93cc198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.232+0000 7f93cbe00700 1 -- 172.21.15.117:0/1930146833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80095f0 con 0x7f93cc102660 2024-04-20T11:26:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.233+0000 7f93cbe00700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 0x7f93cc199240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f93b8005b40 tx=0x7f93b800bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.233+0000 7f93c9600700 1 -- 172.21.15.117:0/1930146833 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93b801d070 con 0x7f93cc102660 2024-04-20T11:26:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.233+0000 7f93c9600700 1 -- 172.21.15.117:0/1930146833 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93b8003d50 con 0x7f93cc102660 2024-04-20T11:26:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.233+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93cc19c630 con 0x7f93cc102660 2024-04-20T11:26:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.233+0000 7f93c9600700 1 -- 172.21.15.117:0/1930146833 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93b800fe90 con 0x7f93cc102660 2024-04-20T11:26:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.233+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93cc19cb20 con 0x7f93cc102660 2024-04-20T11:26:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.234+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93cc068e50 con 0x7f93cc102660 2024-04-20T11:26:41.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.235+0000 7f93c9600700 1 -- 172.21.15.117:0/1930146833 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f93b8003890 con 0x7f93cc102660 2024-04-20T11:26:41.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.236+0000 7f93c9600700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93b406ed10 0x7f93b40711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.237+0000 7f93c9600700 1 -- 172.21.15.117:0/1930146833 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f93b8090300 con 0x7f93cc102660 2024-04-20T11:26:41.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.237+0000 7f93d0a00700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93b406ed10 0x7f93b40711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.238+0000 7f93d0a00700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93b406ed10 0x7f93b40711d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005ea0 tx=0x7f93c0005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.242+0000 7f93c9600700 1 -- 172.21.15.117:0/1930146833 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f93b8059040 con 0x7f93cc102660 2024-04-20T11:26:41.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.496+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93cc10de10 con 0x7f93b406ed10 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.498+0000 7f93c9600700 1 -- 172.21.15.117:0/1930146833 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f93cc10de10 con 0x7f93b406ed10 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:26:41.499 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:26:41.500 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:26:41.500 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:26:41.500 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:26:41.500 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93b406ed10 msgr2=0x7f93b40711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93b406ed10 0x7f93b40711d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005ea0 tx=0x7f93c0005e30 comp rx=0 tx=0).stop 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 msgr2=0x7f93cc199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 0x7f93cc199240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f93b8005b40 tx=0x7f93b800bc10 comp rx=0 tx=0).stop 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 shutdown_connections 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93b406ed10 0x7f93b40711d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc101d40 0x7f93cc198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 --2- 172.21.15.117:0/1930146833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93cc102660 0x7f93cc199240 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 >> 172.21.15.117:0/1930146833 conn(0x7f93cc076750 msgr2=0x7f93cc100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.501+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 shutdown_connections 2024-04-20T11:26:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.502+0000 7f93d3400700 1 -- 172.21.15.117:0/1930146833 wait complete. 2024-04-20T11:26:41.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:41 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/919913529' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:26:41.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:41 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2812676123' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:26:41.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:41 smithi117 ceph-mon[99036]: pgmap v427: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:41.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.653+0000 7effd1a00700 1 -- 172.21.15.117:0/1226802585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 msgr2=0x7effcc109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.653+0000 7effd1a00700 1 --2- 172.21.15.117:0/1226802585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 0x7effcc109290 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7effb4009910 tx=0x7effb4009c20 comp rx=0 tx=0).stop 2024-04-20T11:26:41.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.653+0000 7effd1a00700 1 -- 172.21.15.117:0/1226802585 shutdown_connections 2024-04-20T11:26:41.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.653+0000 7effd1a00700 1 --2- 172.21.15.117:0/1226802585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 0x7effcc109290 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.653+0000 7effd1a00700 1 --2- 172.21.15.117:0/1226802585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc107520 0x7effcc107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.653+0000 7effd1a00700 1 -- 172.21.15.117:0/1226802585 >> 172.21.15.117:0/1226802585 conn(0x7effcc0768f0 msgr2=0x7effcc078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.654+0000 7effd1a00700 1 -- 172.21.15.117:0/1226802585 shutdown_connections 2024-04-20T11:26:41.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.654+0000 7effd1a00700 1 -- 172.21.15.117:0/1226802585 wait complete. 2024-04-20T11:26:41.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.655+0000 7effd1a00700 1 Processor -- start 2024-04-20T11:26:41.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.655+0000 7effd1a00700 1 -- start start 2024-04-20T11:26:41.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.655+0000 7effd1a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc107520 0x7effcc19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.655+0000 7effd1a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 0x7effcc19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.655+0000 7effd1a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc19ded0 con 0x7effcc108e10 2024-04-20T11:26:41.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.655+0000 7effd1a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc1a1ba0 con 0x7effcc107520 2024-04-20T11:26:41.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.655+0000 7effc3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 0x7effcc19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.656+0000 7effc3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 0x7effcc19d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56422/0 (socket says 172.21.15.117:56422) 2024-04-20T11:26:41.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.656+0000 7effc3e00700 1 -- 172.21.15.117:0/471130818 learned_addr learned my addr 172.21.15.117:0/471130818 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:26:41.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.656+0000 7effc3e00700 1 -- 172.21.15.117:0/471130818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc107520 msgr2=0x7effcc19d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.656+0000 7effcaa00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc107520 0x7effcc19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.656+0000 7effc3e00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc107520 0x7effcc19d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.656+0000 7effc3e00700 1 -- 172.21.15.117:0/471130818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40095f0 con 0x7effcc108e10 2024-04-20T11:26:41.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.656+0000 7effc3e00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 0x7effcc19d7f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7effb4006010 tx=0x7effb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.656+0000 7effc3400700 1 -- 172.21.15.117:0/471130818 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effb401d070 con 0x7effcc108e10 2024-04-20T11:26:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.657+0000 7effc3400700 1 -- 172.21.15.117:0/471130818 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effb400f460 con 0x7effcc108e10 2024-04-20T11:26:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.657+0000 7effc3400700 1 -- 172.21.15.117:0/471130818 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effb4017630 con 0x7effcc108e10 2024-04-20T11:26:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.657+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc1a1e20 con 0x7effcc108e10 2024-04-20T11:26:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.657+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc1a22f0 con 0x7effcc108e10 2024-04-20T11:26:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.658+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effcc10f890 con 0x7effcc108e10 2024-04-20T11:26:41.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.659+0000 7effc3400700 1 -- 172.21.15.117:0/471130818 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7effb400f5d0 con 0x7effcc108e10 2024-04-20T11:26:41.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.660+0000 7effc3400700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ecc0 0x7effb0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.660+0000 7effc3400700 1 -- 172.21.15.117:0/471130818 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7effb4090360 con 0x7effcc108e10 2024-04-20T11:26:41.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.660+0000 7effcaa00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ecc0 0x7effb0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:41 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/919913529' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:26:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:41 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2812676123' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:26:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:41 smithi173 ceph-mon[74165]: pgmap v427: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:41.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.661+0000 7effcaa00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ecc0 0x7effb0071180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.664+0000 7effc3400700 1 -- 172.21.15.117:0/471130818 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7effb40591e0 con 0x7effcc108e10 2024-04-20T11:26:41.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.956+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7effcc19e5c0 con 0x7effcc108e10 2024-04-20T11:26:41.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.956+0000 7effc3400700 1 -- 172.21.15.117:0/471130818 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7effb405c8a0 con 0x7effcc108e10 2024-04-20T11:26:41.957 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:26:41.957 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:26:41.957 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:26:41.957 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:26:41.957 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:26:41.957 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:26:41.957 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:26:41.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.959+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ecc0 msgr2=0x7effb0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.959+0000 7effd1a00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ecc0 0x7effb0071180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc008040 comp rx=0 tx=0).stop 2024-04-20T11:26:41.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.959+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 msgr2=0x7effcc19d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.959+0000 7effd1a00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 0x7effcc19d7f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7effb4006010 tx=0x7effb400be20 comp rx=0 tx=0).stop 2024-04-20T11:26:41.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.960+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 shutdown_connections 2024-04-20T11:26:41.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.960+0000 7effd1a00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ecc0 0x7effb0071180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.960+0000 7effd1a00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc107520 0x7effcc19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.960+0000 7effd1a00700 1 --2- 172.21.15.117:0/471130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc108e10 0x7effcc19d7f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.960+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 >> 172.21.15.117:0/471130818 conn(0x7effcc0768f0 msgr2=0x7effcc0775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.960+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 shutdown_connections 2024-04-20T11:26:41.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:26:41.960+0000 7effd1a00700 1 -- 172.21.15.117:0/471130818 wait complete. 2024-04-20T11:26:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:42 smithi117 ceph-mon[99036]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:42 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/471130818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:26:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:42 smithi173 ceph-mon[74165]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:26:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:42 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/471130818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:26:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:26:43.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:43 smithi117 ceph-mon[99036]: pgmap v428: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:26:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:43 smithi173 ceph-mon[74165]: pgmap v428: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:45 smithi173 ceph-mon[74165]: pgmap v429: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:45 smithi117 ceph-mon[99036]: pgmap v429: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:47.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:47 smithi117 ceph-mon[99036]: pgmap v430: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:47.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:47 smithi173 ceph-mon[74165]: pgmap v430: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:49.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:49 smithi117 ceph-mon[99036]: pgmap v431: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:49.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:49 smithi173 ceph-mon[74165]: pgmap v431: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:51.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:51 smithi117 ceph-mon[99036]: pgmap v432: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:51 smithi173 ceph-mon[74165]: pgmap v432: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:53.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:53 smithi117 ceph-mon[99036]: pgmap v433: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:53 smithi173 ceph-mon[74165]: pgmap v433: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:55.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:55 smithi117 ceph-mon[99036]: pgmap v434: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:55 smithi173 ceph-mon[74165]: pgmap v434: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:57 smithi117 ceph-mon[99036]: pgmap v435: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:57.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:57 smithi173 ceph-mon[74165]: pgmap v435: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:26:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:26:59.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:26:59 smithi117 ceph-mon[99036]: pgmap v436: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:26:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:26:59 smithi173 ceph-mon[74165]: pgmap v436: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:01.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:01 smithi117 ceph-mon[99036]: pgmap v437: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:01 smithi173 ceph-mon[74165]: pgmap v437: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:03.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:03 smithi117 ceph-mon[99036]: pgmap v438: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:03 smithi173 ceph-mon[74165]: pgmap v438: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:05 smithi117 ceph-mon[99036]: pgmap v439: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:05.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:05 smithi173 ceph-mon[74165]: pgmap v439: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:07.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:07 smithi117 ceph-mon[99036]: pgmap v440: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:07 smithi173 ceph-mon[74165]: pgmap v440: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:09.636 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:09 smithi117 ceph-mon[99036]: pgmap v441: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:09 smithi173 ceph-mon[74165]: pgmap v441: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:11.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:11 smithi117 ceph-mon[99036]: pgmap v442: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:11 smithi173 ceph-mon[74165]: pgmap v442: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:12.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.134+0000 7fc804800700 1 -- 172.21.15.117:0/2633915533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 msgr2=0x7fc7fc107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.134+0000 7fc804800700 1 --2- 172.21.15.117:0/2633915533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 0x7fc7fc107a20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009910 tx=0x7fc7ec009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:12.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.134+0000 7fc804800700 1 -- 172.21.15.117:0/2633915533 shutdown_connections 2024-04-20T11:27:12.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.134+0000 7fc804800700 1 --2- 172.21.15.117:0/2633915533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 0x7fc7fc107a20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.134+0000 7fc804800700 1 --2- 172.21.15.117:0/2633915533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103020 0x7fc7fc103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.134+0000 7fc804800700 1 -- 172.21.15.117:0/2633915533 >> 172.21.15.117:0/2633915533 conn(0x7fc7fc0fe8b0 msgr2=0x7fc7fc100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.134+0000 7fc804800700 1 -- 172.21.15.117:0/2633915533 shutdown_connections 2024-04-20T11:27:12.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.134+0000 7fc804800700 1 -- 172.21.15.117:0/2633915533 wait complete. 2024-04-20T11:27:12.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.136+0000 7fc804800700 1 Processor -- start 2024-04-20T11:27:12.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.136+0000 7fc804800700 1 -- start start 2024-04-20T11:27:12.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.136+0000 7fc804800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103020 0x7fc7fc198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.136+0000 7fc804800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 0x7fc7fc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.136+0000 7fc804800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc199a30 con 0x7fc7fc1039d0 2024-04-20T11:27:12.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.136+0000 7fc804800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc19d700 con 0x7fc7fc103020 2024-04-20T11:27:12.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.137+0000 7fc801400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 0x7fc7fc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.137+0000 7fc801400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 0x7fc7fc199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39206/0 (socket says 172.21.15.117:39206) 2024-04-20T11:27:12.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.137+0000 7fc801400700 1 -- 172.21.15.117:0/3988483842 learned_addr learned my addr 172.21.15.117:0/3988483842 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:12.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.137+0000 7fc801e00700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103020 0x7fc7fc198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.137+0000 7fc801400700 1 -- 172.21.15.117:0/3988483842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103020 msgr2=0x7fc7fc198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.137+0000 7fc801400700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103020 0x7fc7fc198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.137+0000 7fc801400700 1 -- 172.21.15.117:0/3988483842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec0095f0 con 0x7fc7fc1039d0 2024-04-20T11:27:12.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.138+0000 7fc801400700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 0x7fc7fc199350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005b40 tx=0x7fc7ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.138+0000 7fc7f6a00700 1 -- 172.21.15.117:0/3988483842 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7ec01d070 con 0x7fc7fc1039d0 2024-04-20T11:27:12.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.138+0000 7fc7f6a00700 1 -- 172.21.15.117:0/3988483842 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7ec00f460 con 0x7fc7fc1039d0 2024-04-20T11:27:12.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.138+0000 7fc7f6a00700 1 -- 172.21.15.117:0/3988483842 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7ec017630 con 0x7fc7fc1039d0 2024-04-20T11:27:12.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.138+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7fc19d980 con 0x7fc7fc1039d0 2024-04-20T11:27:12.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.138+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc19de70 con 0x7fc7fc1039d0 2024-04-20T11:27:12.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.139+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7fc10b370 con 0x7fc7fc1039d0 2024-04-20T11:27:12.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.141+0000 7fc7f6a00700 1 -- 172.21.15.117:0/3988483842 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc7ec003a60 con 0x7fc7fc1039d0 2024-04-20T11:27:12.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.142+0000 7fc7f6a00700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7e406ecc0 0x7fc7e4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.142+0000 7fc7f6a00700 1 -- 172.21.15.117:0/3988483842 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc7ec090380 con 0x7fc7fc1039d0 2024-04-20T11:27:12.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.142+0000 7fc801e00700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7e406ecc0 0x7fc7e4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.142+0000 7fc801e00700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7e406ecc0 0x7fc7e4071180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006fd0 tx=0x7fc7f0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.147+0000 7fc7f6a00700 1 -- 172.21.15.117:0/3988483842 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc7ec0590c0 con 0x7fc7fc1039d0 2024-04-20T11:27:12.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.391+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7fc110040 con 0x7fc7e406ecc0 2024-04-20T11:27:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.393+0000 7fc7f6a00700 1 -- 172.21.15.117:0/3988483842 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc7fc110040 con 0x7fc7e406ecc0 2024-04-20T11:27:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7e406ecc0 msgr2=0x7fc7e4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7e406ecc0 0x7fc7e4071180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006fd0 tx=0x7fc7f0008040 comp rx=0 tx=0).stop 2024-04-20T11:27:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 msgr2=0x7fc7fc199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 0x7fc7fc199350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005b40 tx=0x7fc7ec00be20 comp rx=0 tx=0).stop 2024-04-20T11:27:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 shutdown_connections 2024-04-20T11:27:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc7e406ecc0 0x7fc7e4071180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7fc103020 0x7fc7fc198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 --2- 172.21.15.117:0/3988483842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7fc1039d0 0x7fc7fc199350 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 >> 172.21.15.117:0/3988483842 conn(0x7fc7fc0fe8b0 msgr2=0x7fc7fc0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.396+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 shutdown_connections 2024-04-20T11:27:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.397+0000 7fc804800700 1 -- 172.21.15.117:0/3988483842 wait complete. 2024-04-20T11:27:12.412 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:27:12.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.555+0000 7fd127400700 1 -- 172.21.15.117:0/3049981313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd120103020 msgr2=0x7fd120103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.555+0000 7fd127400700 1 --2- 172.21.15.117:0/3049981313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd120103020 0x7fd120103400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd110009910 tx=0x7fd110009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:12.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.555+0000 7fd127400700 1 -- 172.21.15.117:0/3049981313 shutdown_connections 2024-04-20T11:27:12.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.555+0000 7fd127400700 1 --2- 172.21.15.117:0/3049981313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1201039d0 0x7fd120107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.555+0000 7fd127400700 1 --2- 172.21.15.117:0/3049981313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd120103020 0x7fd120103400 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.555+0000 7fd127400700 1 -- 172.21.15.117:0/3049981313 >> 172.21.15.117:0/3049981313 conn(0x7fd1200fe890 msgr2=0x7fd120100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.555+0000 7fd127400700 1 -- 172.21.15.117:0/3049981313 shutdown_connections 2024-04-20T11:27:12.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.556+0000 7fd127400700 1 -- 172.21.15.117:0/3049981313 wait complete. 2024-04-20T11:27:12.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.557+0000 7fd127400700 1 Processor -- start 2024-04-20T11:27:12.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.557+0000 7fd127400700 1 -- start start 2024-04-20T11:27:12.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.557+0000 7fd127400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd120103020 0x7fd120198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.557+0000 7fd127400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1201039d0 0x7fd120199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.557+0000 7fd127400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd120199a50 con 0x7fd1201039d0 2024-04-20T11:27:12.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.557+0000 7fd127400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12019d720 con 0x7fd120103020 2024-04-20T11:27:12.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.557+0000 7fd117e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1201039d0 0x7fd120199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd117e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1201039d0 0x7fd120199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39216/0 (socket says 172.21.15.117:39216) 2024-04-20T11:27:12.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd117e00700 1 -- 172.21.15.117:0/2452731838 learned_addr learned my addr 172.21.15.117:0/2452731838 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:12.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd124a00700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd120103020 0x7fd120198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd117e00700 1 -- 172.21.15.117:0/2452731838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd120103020 msgr2=0x7fd120198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd117e00700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd120103020 0x7fd120198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd117e00700 1 -- 172.21.15.117:0/2452731838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1100095f0 con 0x7fd1201039d0 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd117e00700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1201039d0 0x7fd120199370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd10800b480 tx=0x7fd10800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd11e000700 1 -- 172.21.15.117:0/2452731838 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1080118c0 con 0x7fd1201039d0 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd11e000700 1 -- 172.21.15.117:0/2452731838 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd108010380 con 0x7fd1201039d0 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.558+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12019da00 con 0x7fd1201039d0 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.559+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12019df50 con 0x7fd1201039d0 2024-04-20T11:27:12.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.559+0000 7fd11e000700 1 -- 172.21.15.117:0/2452731838 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd10800f4f0 con 0x7fd1201039d0 2024-04-20T11:27:12.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.560+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12010b3c0 con 0x7fd1201039d0 2024-04-20T11:27:12.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.561+0000 7fd11e000700 1 -- 172.21.15.117:0/2452731838 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd1080104f0 con 0x7fd1201039d0 2024-04-20T11:27:12.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.562+0000 7fd11e000700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd10406ecd0 0x7fd104071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.562+0000 7fd11e000700 1 -- 172.21.15.117:0/2452731838 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd10808eb40 con 0x7fd1201039d0 2024-04-20T11:27:12.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.562+0000 7fd124a00700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd10406ecd0 0x7fd104071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.563+0000 7fd124a00700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd10406ecd0 0x7fd104071190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd110006010 tx=0x7fd11000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.567+0000 7fd11e000700 1 -- 172.21.15.117:0/2452731838 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd108057880 con 0x7fd1201039d0 2024-04-20T11:27:12.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.794+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd120110060 con 0x7fd10406ecd0 2024-04-20T11:27:12.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.798+0000 7fd11e000700 1 -- 172.21.15.117:0/2452731838 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd120110060 con 0x7fd10406ecd0 2024-04-20T11:27:12.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd10406ecd0 msgr2=0x7fd104071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd10406ecd0 0x7fd104071190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd110006010 tx=0x7fd11000b540 comp rx=0 tx=0).stop 2024-04-20T11:27:12.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1201039d0 msgr2=0x7fd120199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1201039d0 0x7fd120199370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd10800b480 tx=0x7fd10800b840 comp rx=0 tx=0).stop 2024-04-20T11:27:12.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 shutdown_connections 2024-04-20T11:27:12.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd10406ecd0 0x7fd104071190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd120103020 0x7fd120198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 --2- 172.21.15.117:0/2452731838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1201039d0 0x7fd120199370 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 >> 172.21.15.117:0/2452731838 conn(0x7fd1200fe890 msgr2=0x7fd1200ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 shutdown_connections 2024-04-20T11:27:12.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.801+0000 7fd127400700 1 -- 172.21.15.117:0/2452731838 wait complete. 2024-04-20T11:27:12.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.957+0000 7f51ab200700 1 -- 172.21.15.117:0/3137513387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 msgr2=0x7f51a4100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.957+0000 7f51ab200700 1 --2- 172.21.15.117:0/3137513387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 0x7f51a4100e50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5194009910 tx=0x7f5194009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:12.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.957+0000 7f51ab200700 1 -- 172.21.15.117:0/3137513387 shutdown_connections 2024-04-20T11:27:12.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.957+0000 7f51ab200700 1 --2- 172.21.15.117:0/3137513387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 0x7f51a4100e50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.957+0000 7f51ab200700 1 --2- 172.21.15.117:0/3137513387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a41069d0 0x7f51a4106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.957+0000 7f51ab200700 1 -- 172.21.15.117:0/3137513387 >> 172.21.15.117:0/3137513387 conn(0x7f51a40fc500 msgr2=0x7f51a40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:12.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.957+0000 7f51ab200700 1 -- 172.21.15.117:0/3137513387 shutdown_connections 2024-04-20T11:27:12.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.957+0000 7f51ab200700 1 -- 172.21.15.117:0/3137513387 wait complete. 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.958+0000 7f51ab200700 1 Processor -- start 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.959+0000 7f51ab200700 1 -- start start 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.959+0000 7f51ab200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 0x7f51a4196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.959+0000 7f51ab200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a41069d0 0x7f51a4196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.959+0000 7f51ab200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4197610 con 0x7f51a41009d0 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.959+0000 7f51ab200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a419b360 con 0x7f51a41069d0 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.959+0000 7f51a9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 0x7f51a4196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.959+0000 7f51a9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 0x7f51a4196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39226/0 (socket says 172.21.15.117:39226) 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.959+0000 7f51a9e00700 1 -- 172.21.15.117:0/2880387912 learned_addr learned my addr 172.21.15.117:0/2880387912 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:12.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.960+0000 7f51a9400700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a41069d0 0x7f51a4196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.960+0000 7f51a9e00700 1 -- 172.21.15.117:0/2880387912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a41069d0 msgr2=0x7f51a4196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:12.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.960+0000 7f51a9e00700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a41069d0 0x7f51a4196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:12.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.960+0000 7f51a9e00700 1 -- 172.21.15.117:0/2880387912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940095f0 con 0x7f51a41009d0 2024-04-20T11:27:12.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.960+0000 7f51a9e00700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 0x7f51a4196a40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f519800b4b0 tx=0x7f519800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.961+0000 7f519ea00700 1 -- 172.21.15.117:0/2880387912 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5198011800 con 0x7f51a41009d0 2024-04-20T11:27:12.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.961+0000 7f519ea00700 1 -- 172.21.15.117:0/2880387912 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5198011e20 con 0x7f51a41009d0 2024-04-20T11:27:12.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.961+0000 7f519ea00700 1 -- 172.21.15.117:0/2880387912 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f519800f2e0 con 0x7f51a41009d0 2024-04-20T11:27:12.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.961+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a419b640 con 0x7f51a41009d0 2024-04-20T11:27:12.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.961+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a419bb90 con 0x7f51a41009d0 2024-04-20T11:27:12.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.962+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4108ff0 con 0x7f51a41009d0 2024-04-20T11:27:12.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.963+0000 7f519ea00700 1 -- 172.21.15.117:0/2880387912 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5198010380 con 0x7f51a41009d0 2024-04-20T11:27:12.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.964+0000 7f519ea00700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518c06ecc0 0x7f518c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:12.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.964+0000 7f519ea00700 1 -- 172.21.15.117:0/2880387912 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f519808ea70 con 0x7f51a41009d0 2024-04-20T11:27:12.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.964+0000 7f51a9400700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518c06ecc0 0x7f518c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:12.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.965+0000 7f51a9400700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518c06ecc0 0x7f518c071180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f51a4197fa0 tx=0x7f519401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:12.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:12.969+0000 7f519ea00700 1 -- 172.21.15.117:0/2880387912 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f519805b2e0 con 0x7f51a41009d0 2024-04-20T11:27:13.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.217+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51a410dc50 con 0x7f518c06ecc0 2024-04-20T11:27:13.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.236+0000 7f519ea00700 1 -- 172.21.15.117:0/2880387912 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f51a410dc50 con 0x7f518c06ecc0 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (15m) 7m ago 29m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (30m) 7m ago 30m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (28m) 6m ago 28m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (11m) 7m ago 30m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (11m) 6m ago 28m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (14m) 7m ago 29m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (7m) 7m ago 22m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (7m) 7m ago 22m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:27:13.237 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (22m) 6m ago 22m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (7m) 6m ago 22m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (18m) 7m ago 31m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (17m) 6m ago 28m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (13m) 7m ago 31m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (13m) 6m ago 28m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (16m) 7m ago 29m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (16m) 6m ago 28m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (11m) 7m ago 27m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (10m) 7m ago 26m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (10m) 7m ago 26m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (9m) 6m ago 25m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (9m) 6m ago 24m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (8m) 6m ago 24m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:27:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (16m) 7m ago 29m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:27:13.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518c06ecc0 msgr2=0x7f518c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518c06ecc0 0x7f518c071180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f51a4197fa0 tx=0x7f519401a040 comp rx=0 tx=0).stop 2024-04-20T11:27:13.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 msgr2=0x7f51a4196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 0x7f51a4196a40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f519800b4b0 tx=0x7f519800b870 comp rx=0 tx=0).stop 2024-04-20T11:27:13.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 shutdown_connections 2024-04-20T11:27:13.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f518c06ecc0 0x7f518c071180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a41009d0 0x7f51a4196a40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 --2- 172.21.15.117:0/2880387912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a41069d0 0x7f51a4196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.239+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 >> 172.21.15.117:0/2880387912 conn(0x7f51a40fc500 msgr2=0x7f51a40fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.240+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 shutdown_connections 2024-04-20T11:27:13.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.240+0000 7f51ab200700 1 -- 172.21.15.117:0/2880387912 wait complete. 2024-04-20T11:27:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:27:13.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.390+0000 7f8972600700 1 -- 172.21.15.117:0/3573149520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 msgr2=0x7f896c107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.390+0000 7f8972600700 1 --2- 172.21.15.117:0/3573149520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 0x7f896c107a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8954009910 tx=0x7f8954009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:13.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.391+0000 7f8972600700 1 -- 172.21.15.117:0/3573149520 shutdown_connections 2024-04-20T11:27:13.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.391+0000 7f8972600700 1 --2- 172.21.15.117:0/3573149520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 0x7f896c107a20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.391+0000 7f8972600700 1 --2- 172.21.15.117:0/3573149520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c103020 0x7f896c103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.391+0000 7f8972600700 1 -- 172.21.15.117:0/3573149520 >> 172.21.15.117:0/3573149520 conn(0x7f896c0fe8b0 msgr2=0x7f896c100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.391+0000 7f8972600700 1 -- 172.21.15.117:0/3573149520 shutdown_connections 2024-04-20T11:27:13.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.391+0000 7f8972600700 1 -- 172.21.15.117:0/3573149520 wait complete. 2024-04-20T11:27:13.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.392+0000 7f8972600700 1 Processor -- start 2024-04-20T11:27:13.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.393+0000 7f8972600700 1 -- start start 2024-04-20T11:27:13.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.393+0000 7f8972600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c103020 0x7f896c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.393+0000 7f8972600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 0x7f896c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.393+0000 7f8972600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1998e0 con 0x7f896c1039d0 2024-04-20T11:27:13.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.393+0000 7f8972600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c19c550 con 0x7f896c103020 2024-04-20T11:27:13.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.393+0000 7f8963e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 0x7f896c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.393+0000 7f8963e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 0x7f896c199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39248/0 (socket says 172.21.15.117:39248) 2024-04-20T11:27:13.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.393+0000 7f8963e00700 1 -- 172.21.15.117:0/94086331 learned_addr learned my addr 172.21.15.117:0/94086331 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:13.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.394+0000 7f8963e00700 1 -- 172.21.15.117:0/94086331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c103020 msgr2=0x7f896c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.394+0000 7f896b400700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c103020 0x7f896c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.394+0000 7f8963e00700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c103020 0x7f896c198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.394+0000 7f8963e00700 1 -- 172.21.15.117:0/94086331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540095f0 con 0x7f896c1039d0 2024-04-20T11:27:13.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.394+0000 7f8963e00700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 0x7f896c199250 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8954005b40 tx=0x7f895400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.395+0000 7f8968c00700 1 -- 172.21.15.117:0/94086331 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f895401d070 con 0x7f896c1039d0 2024-04-20T11:27:13.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.395+0000 7f8968c00700 1 -- 172.21.15.117:0/94086331 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f895400f460 con 0x7f896c1039d0 2024-04-20T11:27:13.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.395+0000 7f8968c00700 1 -- 172.21.15.117:0/94086331 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8954017630 con 0x7f896c1039d0 2024-04-20T11:27:13.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.395+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c19c7d0 con 0x7f896c1039d0 2024-04-20T11:27:13.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.395+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c19ccc0 con 0x7f896c1039d0 2024-04-20T11:27:13.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.396+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c10b420 con 0x7f896c1039d0 2024-04-20T11:27:13.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.397+0000 7f8968c00700 1 -- 172.21.15.117:0/94086331 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f895400f5d0 con 0x7f896c1039d0 2024-04-20T11:27:13.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.398+0000 7f8968c00700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f895006ecc0 0x7f8950071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.398+0000 7f8968c00700 1 -- 172.21.15.117:0/94086331 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f89540902d0 con 0x7f896c1039d0 2024-04-20T11:27:13.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.398+0000 7f896b400700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f895006ecc0 0x7f8950071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.399+0000 7f896b400700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f895006ecc0 0x7f8950071180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f895c005950 tx=0x7f895c005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.402+0000 7f8968c00700 1 -- 172.21.15.117:0/94086331 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8954059090 con 0x7f896c1039d0 2024-04-20T11:27:13.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:27:13.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.725+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f896c199fe0 con 0x7f896c1039d0 2024-04-20T11:27:13.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.726+0000 7f8968c00700 1 -- 172.21.15.117:0/94086331 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f895405c750 con 0x7f896c1039d0 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:27:13.727 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:27:13.728 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:27:13.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:27:13.728 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:27:13.728 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:27:13.728 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:27:13.728 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:27:13.728 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:27:13.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f895006ecc0 msgr2=0x7f8950071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f895006ecc0 0x7f8950071180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f895c005950 tx=0x7f895c005cf0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 msgr2=0x7f896c199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 0x7f896c199250 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8954005b40 tx=0x7f895400be20 comp rx=0 tx=0).stop 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 shutdown_connections 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f895006ecc0 0x7f8950071180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c103020 0x7f896c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 --2- 172.21.15.117:0/94086331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1039d0 0x7f896c199250 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 >> 172.21.15.117:0/94086331 conn(0x7f896c0fe8b0 msgr2=0x7f896c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 shutdown_connections 2024-04-20T11:27:13.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.729+0000 7f8972600700 1 -- 172.21.15.117:0/94086331 wait complete. 2024-04-20T11:27:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.885+0000 7fba1bc00700 1 -- 172.21.15.117:0/3330140554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba141026e0 msgr2=0x7fba1410abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.885+0000 7fba1bc00700 1 --2- 172.21.15.117:0/3330140554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba141026e0 0x7fba1410abd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fba00009940 tx=0x7fba00009c50 comp rx=0 tx=0).stop 2024-04-20T11:27:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.885+0000 7fba1bc00700 1 -- 172.21.15.117:0/3330140554 shutdown_connections 2024-04-20T11:27:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.885+0000 7fba1bc00700 1 --2- 172.21.15.117:0/3330140554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba141026e0 0x7fba1410abd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.885+0000 7fba1bc00700 1 --2- 172.21.15.117:0/3330140554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba14101dc0 0x7fba141021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.885+0000 7fba1bc00700 1 -- 172.21.15.117:0/3330140554 >> 172.21.15.117:0/3330140554 conn(0x7fba14076750 msgr2=0x7fba140fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.885+0000 7fba1bc00700 1 -- 172.21.15.117:0/3330140554 shutdown_connections 2024-04-20T11:27:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.885+0000 7fba1bc00700 1 -- 172.21.15.117:0/3330140554 wait complete. 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.886+0000 7fba1bc00700 1 Processor -- start 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.886+0000 7fba1bc00700 1 -- start start 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba1bc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba14101dc0 0x7fba140ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba1bc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba141026e0 0x7fba140fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba1bc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba141004a0 con 0x7fba14101dc0 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba1bc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba141005e0 con 0x7fba141026e0 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba19200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba14101dc0 0x7fba140ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba19200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba14101dc0 0x7fba140ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50130/0 (socket says 172.21.15.117:50130) 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba19200700 1 -- 172.21.15.117:0/4135702648 learned_addr learned my addr 172.21.15.117:0/4135702648 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba13e00700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba141026e0 0x7fba140fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba19200700 1 -- 172.21.15.117:0/4135702648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba141026e0 msgr2=0x7fba140fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.887+0000 7fba19200700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba141026e0 0x7fba140fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.888+0000 7fba19200700 1 -- 172.21.15.117:0/4135702648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba000095f0 con 0x7fba14101dc0 2024-04-20T11:27:13.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.888+0000 7fba19200700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba14101dc0 0x7fba140ffa20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fba0800b4b0 tx=0x7fba0800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.888+0000 7fba11600700 1 -- 172.21.15.117:0/4135702648 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba08011800 con 0x7fba14101dc0 2024-04-20T11:27:13.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.888+0000 7fba11600700 1 -- 172.21.15.117:0/4135702648 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba08011e20 con 0x7fba14101dc0 2024-04-20T11:27:13.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.888+0000 7fba11600700 1 -- 172.21.15.117:0/4135702648 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0800f2e0 con 0x7fba14101dc0 2024-04-20T11:27:13.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.888+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba141a2e00 con 0x7fba14101dc0 2024-04-20T11:27:13.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.888+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba141a3160 con 0x7fba14101dc0 2024-04-20T11:27:13.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.889+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba14068dd0 con 0x7fba14101dc0 2024-04-20T11:27:13.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.890+0000 7fba11600700 1 -- 172.21.15.117:0/4135702648 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fba08010380 con 0x7fba14101dc0 2024-04-20T11:27:13.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.891+0000 7fba11600700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb9fc06ecc0 0x7fb9fc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:13.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.892+0000 7fba11600700 1 -- 172.21.15.117:0/4135702648 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fba0808e930 con 0x7fba14101dc0 2024-04-20T11:27:13.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.892+0000 7fba13e00700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb9fc06ecc0 0x7fb9fc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:13.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.892+0000 7fba13e00700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb9fc06ecc0 0x7fb9fc071180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fba00006010 tx=0x7fba00005c40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:13.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:13.897+0000 7fba11600700 1 -- 172.21.15.117:0/4135702648 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fba08057790 con 0x7fba14101dc0 2024-04-20T11:27:14.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.161+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fba1410de20 con 0x7fba14101dc0 2024-04-20T11:27:14.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.162+0000 7fba11600700 1 -- 172.21.15.117:0/4135702648 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fba0805ae50 con 0x7fba14101dc0 2024-04-20T11:27:14.162 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:27:14.162 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:27:14.163 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:27:14.164 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:27:14.165 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:27:14.165 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:27:14.165 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:27:14.165 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:27:14.165 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:27:14.165 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:27:14.165 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:27:14.165 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:27:14.166 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:27:14.166 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:27:14.166 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:27:14.166 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:27:14.166 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:27:14.166 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb9fc06ecc0 msgr2=0x7fb9fc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb9fc06ecc0 0x7fb9fc071180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fba00006010 tx=0x7fba00005c40 comp rx=0 tx=0).stop 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba14101dc0 msgr2=0x7fba140ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba14101dc0 0x7fba140ffa20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fba0800b4b0 tx=0x7fba0800b870 comp rx=0 tx=0).stop 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 shutdown_connections 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb9fc06ecc0 0x7fb9fc071180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba14101dc0 0x7fba140ffa20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 --2- 172.21.15.117:0/4135702648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba141026e0 0x7fba140fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 >> 172.21.15.117:0/4135702648 conn(0x7fba14076750 msgr2=0x7fba141030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:14.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 shutdown_connections 2024-04-20T11:27:14.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.165+0000 7fba1bc00700 1 -- 172.21.15.117:0/4135702648 wait complete. 2024-04-20T11:27:14.168 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:27:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.318+0000 7f8d2c800700 1 -- 172.21.15.117:0/1128225211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 msgr2=0x7f8d2410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.318+0000 7f8d2c800700 1 --2- 172.21.15.117:0/1128225211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 0x7f8d2410aba0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009910 tx=0x7f8d14009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.318+0000 7f8d2c800700 1 -- 172.21.15.117:0/1128225211 shutdown_connections 2024-04-20T11:27:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.319+0000 7f8d2c800700 1 --2- 172.21.15.117:0/1128225211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 0x7f8d2410aba0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.319+0000 7f8d2c800700 1 --2- 172.21.15.117:0/1128225211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d24101d90 0x7f8d24102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.319+0000 7f8d2c800700 1 -- 172.21.15.117:0/1128225211 >> 172.21.15.117:0/1128225211 conn(0x7f8d240fb5c0 msgr2=0x7f8d240fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.319+0000 7f8d2c800700 1 -- 172.21.15.117:0/1128225211 shutdown_connections 2024-04-20T11:27:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.319+0000 7f8d2c800700 1 -- 172.21.15.117:0/1128225211 wait complete. 2024-04-20T11:27:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.320+0000 7f8d2c800700 1 Processor -- start 2024-04-20T11:27:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.320+0000 7f8d2c800700 1 -- start start 2024-04-20T11:27:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.321+0000 7f8d2c800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d24101d90 0x7f8d240ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.321+0000 7f8d2c800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 0x7f8d240fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.321+0000 7f8d2c800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d24100470 con 0x7f8d241026b0 2024-04-20T11:27:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.321+0000 7f8d2c800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241005b0 con 0x7f8d24101d90 2024-04-20T11:27:14.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.321+0000 7f8d29400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 0x7f8d240fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.321+0000 7f8d29400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 0x7f8d240fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50154/0 (socket says 172.21.15.117:50154) 2024-04-20T11:27:14.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.321+0000 7f8d29400700 1 -- 172.21.15.117:0/3291195672 learned_addr learned my addr 172.21.15.117:0/3291195672 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.321+0000 7f8d29e00700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d24101d90 0x7f8d240ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.322+0000 7f8d29400700 1 -- 172.21.15.117:0/3291195672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d24101d90 msgr2=0x7f8d240ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.322+0000 7f8d29400700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d24101d90 0x7f8d240ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.322+0000 7f8d29400700 1 -- 172.21.15.117:0/3291195672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140095f0 con 0x7f8d241026b0 2024-04-20T11:27:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.322+0000 7f8d29400700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 0x7f8d240fff30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400bd70 tx=0x7f8d1400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.322+0000 7f8d1ea00700 1 -- 172.21.15.117:0/3291195672 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d1401d070 con 0x7f8d241026b0 2024-04-20T11:27:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.322+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d24100830 con 0x7f8d241026b0 2024-04-20T11:27:14.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.322+0000 7f8d1ea00700 1 -- 172.21.15.117:0/3291195672 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d1400f460 con 0x7f8d241026b0 2024-04-20T11:27:14.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.322+0000 7f8d1ea00700 1 -- 172.21.15.117:0/3291195672 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d14017610 con 0x7f8d241026b0 2024-04-20T11:27:14.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.323+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d241a3280 con 0x7f8d241026b0 2024-04-20T11:27:14.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.324+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d240696e0 con 0x7f8d241026b0 2024-04-20T11:27:14.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.325+0000 7f8d1ea00700 1 -- 172.21.15.117:0/3291195672 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8d1400fa50 con 0x7f8d241026b0 2024-04-20T11:27:14.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.326+0000 7f8d1ea00700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8d0c06ecc0 0x7f8d0c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.326+0000 7f8d1ea00700 1 -- 172.21.15.117:0/3291195672 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8d14090320 con 0x7f8d241026b0 2024-04-20T11:27:14.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.326+0000 7f8d29e00700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8d0c06ecc0 0x7f8d0c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.327+0000 7f8d29e00700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8d0c06ecc0 0x7f8d0c071180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005fd0 tx=0x7f8d1800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.331+0000 7f8d1ea00700 1 -- 172.21.15.117:0/3291195672 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8d14059120 con 0x7f8d241026b0 2024-04-20T11:27:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:14 smithi117 ceph-mon[99036]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:14 smithi117 ceph-mon[99036]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:14 smithi117 ceph-mon[99036]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:14 smithi117 ceph-mon[99036]: pgmap v443: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:14 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/94086331' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:27:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:14 smithi173 ceph-mon[74165]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:14 smithi173 ceph-mon[74165]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:14.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:14 smithi173 ceph-mon[74165]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:14.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:14 smithi173 ceph-mon[74165]: pgmap v443: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:14.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:14 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/94086331' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:27:14.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.560+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d2406a5e0 con 0x7f8d0c06ecc0 2024-04-20T11:27:14.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.562+0000 7f8d1ea00700 1 -- 172.21.15.117:0/3291195672 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8d2406a5e0 con 0x7f8d0c06ecc0 2024-04-20T11:27:14.564 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:27:14.564 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:27:14.564 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:27:14.564 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:27:14.564 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:27:14.564 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:27:14.564 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:27:14.565 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:27:14.565 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:27:14.565 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:27:14.565 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:27:14.565 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:27:14.565 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:27:14.565 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:27:14.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8d0c06ecc0 msgr2=0x7f8d0c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8d0c06ecc0 0x7f8d0c071180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005fd0 tx=0x7f8d1800a560 comp rx=0 tx=0).stop 2024-04-20T11:27:14.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 msgr2=0x7f8d240fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 0x7f8d240fff30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400bd70 tx=0x7f8d1400be50 comp rx=0 tx=0).stop 2024-04-20T11:27:14.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 shutdown_connections 2024-04-20T11:27:14.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8d0c06ecc0 0x7f8d0c071180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d24101d90 0x7f8d240ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 --2- 172.21.15.117:0/3291195672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d241026b0 0x7f8d240fff30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.565+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 >> 172.21.15.117:0/3291195672 conn(0x7f8d240fb5c0 msgr2=0x7f8d24103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:14.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.566+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 shutdown_connections 2024-04-20T11:27:14.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.566+0000 7f8d2c800700 1 -- 172.21.15.117:0/3291195672 wait complete. 2024-04-20T11:27:14.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.719+0000 7f4786600700 1 -- 172.21.15.117:0/3651736239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780104a00 msgr2=0x7f4780104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.719+0000 7f4786600700 1 --2- 172.21.15.117:0/3651736239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780104a00 0x7f4780104e80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f476c009910 tx=0x7f476c009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:14.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.719+0000 7f4786600700 1 -- 172.21.15.117:0/3651736239 shutdown_connections 2024-04-20T11:27:14.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.719+0000 7f4786600700 1 --2- 172.21.15.117:0/3651736239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780104a00 0x7f4780104e80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.719+0000 7f4786600700 1 --2- 172.21.15.117:0/3651736239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4780103110 0x7f47801034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.719+0000 7f4786600700 1 -- 172.21.15.117:0/3651736239 >> 172.21.15.117:0/3651736239 conn(0x7f47800fe880 msgr2=0x7f4780100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:14.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.719+0000 7f4786600700 1 -- 172.21.15.117:0/3651736239 shutdown_connections 2024-04-20T11:27:14.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.719+0000 7f4786600700 1 -- 172.21.15.117:0/3651736239 wait complete. 2024-04-20T11:27:14.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.720+0000 7f4786600700 1 Processor -- start 2024-04-20T11:27:14.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.720+0000 7f4786600700 1 -- start start 2024-04-20T11:27:14.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f4786600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780103110 0x7f478019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f4786600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4780104a00 0x7f478019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f4786600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f5b0 con 0x7f4780103110 2024-04-20T11:27:14.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f4786600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4780198b90 con 0x7f4780104a00 2024-04-20T11:27:14.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f477f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780103110 0x7f478019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f477f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780103110 0x7f478019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50174/0 (socket says 172.21.15.117:50174) 2024-04-20T11:27:14.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f477f400700 1 -- 172.21.15.117:0/1996436571 learned_addr learned my addr 172.21.15.117:0/1996436571 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:14.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f477ea00700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4780104a00 0x7f478019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f477f400700 1 -- 172.21.15.117:0/1996436571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4780104a00 msgr2=0x7f478019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f477f400700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4780104a00 0x7f478019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.721+0000 7f477f400700 1 -- 172.21.15.117:0/1996436571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0095f0 con 0x7f4780103110 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.722+0000 7f477f400700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780103110 0x7f478019ea50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f477000d8d0 tx=0x7f477000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.722+0000 7f4777e00700 1 -- 172.21.15.117:0/1996436571 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4770009940 con 0x7f4780103110 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.722+0000 7f4777e00700 1 -- 172.21.15.117:0/1996436571 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4770010460 con 0x7f4780103110 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.722+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4780198e70 con 0x7f4780103110 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.722+0000 7f4777e00700 1 -- 172.21.15.117:0/1996436571 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f477000f5f0 con 0x7f4780103110 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.722+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4780199440 con 0x7f4780103110 2024-04-20T11:27:14.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.723+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478010b480 con 0x7f4780103110 2024-04-20T11:27:14.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.724+0000 7f4777e00700 1 -- 172.21.15.117:0/1996436571 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f47700105d0 con 0x7f4780103110 2024-04-20T11:27:14.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.726+0000 7f4777e00700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f476406ecd0 0x7f4764071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:14.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.726+0000 7f4777e00700 1 -- 172.21.15.117:0/1996436571 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f477008ede0 con 0x7f4780103110 2024-04-20T11:27:14.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.726+0000 7f477ea00700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f476406ecd0 0x7f4764071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:14.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.727+0000 7f477ea00700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f476406ecd0 0x7f4764071190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f476c005b40 tx=0x7f476c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:14.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:14.731+0000 7f4777e00700 1 -- 172.21.15.117:0/1996436571 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f477005b6d0 con 0x7f4780103110 2024-04-20T11:27:15.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.045+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4780199fe0 con 0x7f4780103110 2024-04-20T11:27:15.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.046+0000 7f4777e00700 1 -- 172.21.15.117:0/1996436571 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f47700161c0 con 0x7f4780103110 2024-04-20T11:27:15.046 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:27:15.046 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:27:15.047 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:27:15.047 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:27:15.047 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:27:15.047 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:27:15.047 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:27:15.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.048+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f476406ecd0 msgr2=0x7f4764071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f476406ecd0 0x7f4764071190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f476c005b40 tx=0x7f476c01a040 comp rx=0 tx=0).stop 2024-04-20T11:27:15.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780103110 msgr2=0x7f478019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:15.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780103110 0x7f478019ea50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f477000d8d0 tx=0x7f477000dc90 comp rx=0 tx=0).stop 2024-04-20T11:27:15.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 shutdown_connections 2024-04-20T11:27:15.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f476406ecd0 0x7f4764071190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780103110 0x7f478019ea50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 --2- 172.21.15.117:0/1996436571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4780104a00 0x7f478019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:15.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 >> 172.21.15.117:0/1996436571 conn(0x7f47800fe880 msgr2=0x7f47800ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:15.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 shutdown_connections 2024-04-20T11:27:15.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:15.049+0000 7f4786600700 1 -- 172.21.15.117:0/1996436571 wait complete. 2024-04-20T11:27:15.372 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:15 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4135702648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:27:15.372 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:15 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1996436571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:27:15.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:15 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4135702648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:27:15.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:15 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1996436571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:27:16.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:16 smithi117 ceph-mon[99036]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:16 smithi117 ceph-mon[99036]: pgmap v444: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:16 smithi173 ceph-mon[74165]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:16 smithi173 ceph-mon[74165]: pgmap v444: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:17.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:17 smithi117 ceph-mon[99036]: pgmap v445: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:17 smithi173 ceph-mon[74165]: pgmap v445: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:19.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:19 smithi117 ceph-mon[99036]: pgmap v446: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:19 smithi173 ceph-mon[74165]: pgmap v446: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:21.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:21 smithi117 ceph-mon[99036]: pgmap v447: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:21.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:21 smithi173 ceph-mon[74165]: pgmap v447: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:23 smithi117 ceph-mon[99036]: pgmap v448: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:23 smithi173 ceph-mon[74165]: pgmap v448: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:25.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:25 smithi117 ceph-mon[99036]: pgmap v449: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:25 smithi173 ceph-mon[74165]: pgmap v449: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:27 smithi117 ceph-mon[99036]: pgmap v450: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:27 smithi173 ceph-mon[74165]: pgmap v450: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:27:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:27:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:29 smithi117 ceph-mon[99036]: pgmap v451: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:29.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:29 smithi173 ceph-mon[74165]: pgmap v451: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:31.597 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:31 smithi173 ceph-mon[74165]: pgmap v452: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:31.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:31 smithi117 ceph-mon[99036]: pgmap v452: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:33.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:33 smithi117 ceph-mon[99036]: pgmap v453: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:33 smithi173 ceph-mon[74165]: pgmap v453: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:35.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:35 smithi117 ceph-mon[99036]: pgmap v454: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:35 smithi173 ceph-mon[74165]: pgmap v454: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:37.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:37 smithi117 ceph-mon[99036]: pgmap v455: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:37 smithi173 ceph-mon[74165]: pgmap v455: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:39.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:39 smithi117 ceph-mon[99036]: pgmap v456: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:39.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:39 smithi173 ceph-mon[74165]: pgmap v456: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:41.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:41 smithi117 ceph-mon[99036]: pgmap v457: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:41.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:41 smithi173 ceph-mon[74165]: pgmap v457: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:43.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:27:43.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:27:44.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:44 smithi117 ceph-mon[99036]: pgmap v458: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:44.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:44 smithi173 ceph-mon[74165]: pgmap v458: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.222+0000 7f7c06600700 1 -- 172.21.15.117:0/2772904070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00108e10 msgr2=0x7f7c00109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.222+0000 7f7c06600700 1 --2- 172.21.15.117:0/2772904070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00108e10 0x7f7c00109290 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009910 tx=0x7f7bec009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.223+0000 7f7c06600700 1 -- 172.21.15.117:0/2772904070 shutdown_connections 2024-04-20T11:27:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.223+0000 7f7c06600700 1 --2- 172.21.15.117:0/2772904070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00108e10 0x7f7c00109290 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.223+0000 7f7c06600700 1 --2- 172.21.15.117:0/2772904070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c00107520 0x7f7c00107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.223+0000 7f7c06600700 1 -- 172.21.15.117:0/2772904070 >> 172.21.15.117:0/2772904070 conn(0x7f7c000768f0 msgr2=0x7f7c00078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.223+0000 7f7c06600700 1 -- 172.21.15.117:0/2772904070 shutdown_connections 2024-04-20T11:27:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.223+0000 7f7c06600700 1 -- 172.21.15.117:0/2772904070 wait complete. 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.224+0000 7f7c06600700 1 Processor -- start 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.224+0000 7f7c06600700 1 -- start start 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.224+0000 7f7c06600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00107520 0x7f7c0019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.224+0000 7f7c06600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c00108e10 0x7f7c0019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.224+0000 7f7c06600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0019ddf0 con 0x7f7c00107520 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.224+0000 7f7c06600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c001a1a70 con 0x7f7c00108e10 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.224+0000 7f7bff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00107520 0x7f7c0019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.225+0000 7f7bff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00107520 0x7f7c0019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58212/0 (socket says 172.21.15.117:58212) 2024-04-20T11:27:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.225+0000 7f7bff400700 1 -- 172.21.15.117:0/3687354686 learned_addr learned my addr 172.21.15.117:0/3687354686 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.225+0000 7f7bfea00700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c00108e10 0x7f7c0019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.225+0000 7f7bff400700 1 -- 172.21.15.117:0/3687354686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c00108e10 msgr2=0x7f7c0019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.225+0000 7f7bff400700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c00108e10 0x7f7c0019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.225+0000 7f7bff400700 1 -- 172.21.15.117:0/3687354686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0095f0 con 0x7f7c00107520 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.225+0000 7f7bff400700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00107520 0x7f7c0019d1d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c8a0 tx=0x7f7bf000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.226+0000 7f7bf7e00700 1 -- 172.21.15.117:0/3687354686 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf000cea0 con 0x7f7c00107520 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.226+0000 7f7bf7e00700 1 -- 172.21.15.117:0/3687354686 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf0004830 con 0x7f7c00107520 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.226+0000 7f7bf7e00700 1 -- 172.21.15.117:0/3687354686 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf00055b0 con 0x7f7c00107520 2024-04-20T11:27:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.226+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c001a1d50 con 0x7f7c00107520 2024-04-20T11:27:45.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.226+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c001a2320 con 0x7f7c00107520 2024-04-20T11:27:45.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.227+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0010f870 con 0x7f7c00107520 2024-04-20T11:27:45.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.228+0000 7f7bf7e00700 1 -- 172.21.15.117:0/3687354686 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7bf00049a0 con 0x7f7c00107520 2024-04-20T11:27:45.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.229+0000 7f7bf7e00700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7be406ecd0 0x7f7be4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.229+0000 7f7bf7e00700 1 -- 172.21.15.117:0/3687354686 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7bf008ea20 con 0x7f7c00107520 2024-04-20T11:27:45.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.229+0000 7f7bfea00700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7be406ecd0 0x7f7be4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.230+0000 7f7bfea00700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7be406ecd0 0x7f7be4071190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006010 tx=0x7f7bec00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.234+0000 7f7bf7e00700 1 -- 172.21.15.117:0/3687354686 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7bf0057880 con 0x7f7c00107520 2024-04-20T11:27:45.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.486+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c0019e500 con 0x7f7be406ecd0 2024-04-20T11:27:45.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:45 smithi117 ceph-mon[99036]: pgmap v459: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:45.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.488+0000 7f7bf7e00700 1 -- 172.21.15.117:0/3687354686 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7c0019e500 con 0x7f7be406ecd0 2024-04-20T11:27:45.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.491+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7be406ecd0 msgr2=0x7f7be4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.491+0000 7f7c06600700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7be406ecd0 0x7f7be4071190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006010 tx=0x7f7bec00b540 comp rx=0 tx=0).stop 2024-04-20T11:27:45.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.491+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00107520 msgr2=0x7f7c0019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.491+0000 7f7c06600700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00107520 0x7f7c0019d1d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c8a0 tx=0x7f7bf000cc60 comp rx=0 tx=0).stop 2024-04-20T11:27:45.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.491+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 shutdown_connections 2024-04-20T11:27:45.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.491+0000 7f7c06600700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7be406ecd0 0x7f7be4071190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.491+0000 7f7c06600700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c00107520 0x7f7c0019d1d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.491+0000 7f7c06600700 1 --2- 172.21.15.117:0/3687354686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c00108e10 0x7f7c0019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.492+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 >> 172.21.15.117:0/3687354686 conn(0x7f7c000768f0 msgr2=0x7f7c00077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:45.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.492+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 shutdown_connections 2024-04-20T11:27:45.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.492+0000 7f7c06600700 1 -- 172.21.15.117:0/3687354686 wait complete. 2024-04-20T11:27:45.507 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:27:45.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.652+0000 7f890cc00700 1 -- 172.21.15.117:0/3845143360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 msgr2=0x7f8908105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.652+0000 7f890cc00700 1 --2- 172.21.15.117:0/3845143360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 0x7f8908105810 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009910 tx=0x7f88f4009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.652+0000 7f890cc00700 1 -- 172.21.15.117:0/3845143360 shutdown_connections 2024-04-20T11:27:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.652+0000 7f890cc00700 1 --2- 172.21.15.117:0/3845143360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 0x7f8908105810 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.652+0000 7f890cc00700 1 --2- 172.21.15.117:0/3845143360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8908100e10 0x7f89081011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.652+0000 7f890cc00700 1 -- 172.21.15.117:0/3845143360 >> 172.21.15.117:0/3845143360 conn(0x7f89080fc6a0 msgr2=0x7f89080feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.652+0000 7f890cc00700 1 -- 172.21.15.117:0/3845143360 shutdown_connections 2024-04-20T11:27:45.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.652+0000 7f890cc00700 1 -- 172.21.15.117:0/3845143360 wait complete. 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.653+0000 7f890cc00700 1 Processor -- start 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.653+0000 7f890cc00700 1 -- start start 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.653+0000 7f890cc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8908100e10 0x7f890819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.653+0000 7f890cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 0x7f890819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.653+0000 7f890cc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890819d470 con 0x7f89081017c0 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.653+0000 7f890cc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8908196a50 con 0x7f8908100e10 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.654+0000 7f8905400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 0x7f890819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.654+0000 7f8905400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 0x7f890819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58236/0 (socket says 172.21.15.117:58236) 2024-04-20T11:27:45.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.654+0000 7f8905400700 1 -- 172.21.15.117:0/2998899389 learned_addr learned my addr 172.21.15.117:0/2998899389 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.654+0000 7f8905e00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8908100e10 0x7f890819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.654+0000 7f8905400700 1 -- 172.21.15.117:0/2998899389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8908100e10 msgr2=0x7f890819c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.654+0000 7f8905400700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8908100e10 0x7f890819c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.654+0000 7f8905400700 1 -- 172.21.15.117:0/2998899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40095f0 con 0x7f89081017c0 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.654+0000 7f8905400700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 0x7f890819ce50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f88f40092e0 tx=0x7f88f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.655+0000 7f88fea00700 1 -- 172.21.15.117:0/2998899389 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f401d070 con 0x7f89081017c0 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.655+0000 7f88fea00700 1 -- 172.21.15.117:0/2998899389 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88f400f460 con 0x7f89081017c0 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.655+0000 7f88fea00700 1 -- 172.21.15.117:0/2998899389 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f4017610 con 0x7f89081017c0 2024-04-20T11:27:45.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.655+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8908196cd0 con 0x7f89081017c0 2024-04-20T11:27:45.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.655+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89081971c0 con 0x7f89081017c0 2024-04-20T11:27:45.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.656+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8908109160 con 0x7f89081017c0 2024-04-20T11:27:45.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.657+0000 7f88fea00700 1 -- 172.21.15.117:0/2998899389 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f88f400f5d0 con 0x7f89081017c0 2024-04-20T11:27:45.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.658+0000 7f88fea00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f88ec06ecd0 0x7f88ec071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:45.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.658+0000 7f88fea00700 1 -- 172.21.15.117:0/2998899389 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f88f40902d0 con 0x7f89081017c0 2024-04-20T11:27:45.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.659+0000 7f8905e00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f88ec06ecd0 0x7f88ec071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:45.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.659+0000 7f8905e00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f88ec06ecd0 0x7f88ec071190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88f8006fd0 tx=0x7f88f8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:45.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:45 smithi173 ceph-mon[74165]: pgmap v459: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:45.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.663+0000 7f88fea00700 1 -- 172.21.15.117:0/2998899389 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f88f405cbc0 con 0x7f89081017c0 2024-04-20T11:27:45.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.919+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8908197f80 con 0x7f88ec06ecd0 2024-04-20T11:27:45.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.922+0000 7f88fea00700 1 -- 172.21.15.117:0/2998899389 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8908197f80 con 0x7f88ec06ecd0 2024-04-20T11:27:45.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f88ec06ecd0 msgr2=0x7f88ec071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f88ec06ecd0 0x7f88ec071190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88f8006fd0 tx=0x7f88f8008040 comp rx=0 tx=0).stop 2024-04-20T11:27:45.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 msgr2=0x7f890819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:45.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 0x7f890819ce50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f88f40092e0 tx=0x7f88f400be20 comp rx=0 tx=0).stop 2024-04-20T11:27:45.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 shutdown_connections 2024-04-20T11:27:45.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f88ec06ecd0 0x7f88ec071190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8908100e10 0x7f890819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 --2- 172.21.15.117:0/2998899389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89081017c0 0x7f890819ce50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:45.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.924+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 >> 172.21.15.117:0/2998899389 conn(0x7f89080fc6a0 msgr2=0x7f89080fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:45.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.925+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 shutdown_connections 2024-04-20T11:27:45.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:45.925+0000 7f890cc00700 1 -- 172.21.15.117:0/2998899389 wait complete. 2024-04-20T11:27:46.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.067+0000 7f3670a00700 1 -- 172.21.15.117:0/3069997457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c107520 msgr2=0x7f366c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.067+0000 7f3670a00700 1 --2- 172.21.15.117:0/3069997457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c107520 0x7f366c107900 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f365c009910 tx=0x7f365c009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:46.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.067+0000 7f3670a00700 1 -- 172.21.15.117:0/3069997457 shutdown_connections 2024-04-20T11:27:46.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.067+0000 7f3670a00700 1 --2- 172.21.15.117:0/3069997457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c108e10 0x7f366c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.067+0000 7f3670a00700 1 --2- 172.21.15.117:0/3069997457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c107520 0x7f366c107900 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.067+0000 7f3670a00700 1 -- 172.21.15.117:0/3069997457 >> 172.21.15.117:0/3069997457 conn(0x7f366c0768f0 msgr2=0x7f366c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.067+0000 7f3670a00700 1 -- 172.21.15.117:0/3069997457 shutdown_connections 2024-04-20T11:27:46.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.067+0000 7f3670a00700 1 -- 172.21.15.117:0/3069997457 wait complete. 2024-04-20T11:27:46.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.068+0000 7f3670a00700 1 Processor -- start 2024-04-20T11:27:46.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.068+0000 7f3670a00700 1 -- start start 2024-04-20T11:27:46.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.069+0000 7f3670a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c107520 0x7f366c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.069+0000 7f3670a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c108e10 0x7f366c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.069+0000 7f3670a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c19de90 con 0x7f366c108e10 2024-04-20T11:27:46.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.069+0000 7f3670a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c1a1b60 con 0x7f366c107520 2024-04-20T11:27:46.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.069+0000 7f3669400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c108e10 0x7f366c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.069+0000 7f3669400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c108e10 0x7f366c19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58256/0 (socket says 172.21.15.117:58256) 2024-04-20T11:27:46.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.069+0000 7f3669400700 1 -- 172.21.15.117:0/2315770022 learned_addr learned my addr 172.21.15.117:0/2315770022 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:46.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.069+0000 7f3669e00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c107520 0x7f366c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3669400700 1 -- 172.21.15.117:0/2315770022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c107520 msgr2=0x7f366c19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3669400700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c107520 0x7f366c19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3669400700 1 -- 172.21.15.117:0/2315770022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0095f0 con 0x7f366c108e10 2024-04-20T11:27:46.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3669400700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c108e10 0x7f366c19d7b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f365800b4b0 tx=0x7f365800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3662a00700 1 -- 172.21.15.117:0/2315770022 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3658011800 con 0x7f366c108e10 2024-04-20T11:27:46.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c1a1e40 con 0x7f366c108e10 2024-04-20T11:27:46.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3662a00700 1 -- 172.21.15.117:0/2315770022 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3658011e20 con 0x7f366c108e10 2024-04-20T11:27:46.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3662a00700 1 -- 172.21.15.117:0/2315770022 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365800f2e0 con 0x7f366c108e10 2024-04-20T11:27:46.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.070+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c1a2410 con 0x7f366c108e10 2024-04-20T11:27:46.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.072+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c10f870 con 0x7f366c108e10 2024-04-20T11:27:46.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.073+0000 7f3662a00700 1 -- 172.21.15.117:0/2315770022 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3658010380 con 0x7f366c108e10 2024-04-20T11:27:46.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.074+0000 7f3662a00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecd0 0x7f3650071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.074+0000 7f3662a00700 1 -- 172.21.15.117:0/2315770022 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f365808ea00 con 0x7f366c108e10 2024-04-20T11:27:46.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.074+0000 7f3669e00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecd0 0x7f3650071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.075+0000 7f3669e00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecd0 0x7f3650071190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f365c00b5c0 tx=0x7f365c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.079+0000 7f3662a00700 1 -- 172.21.15.117:0/2315770022 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3658057870 con 0x7f366c108e10 2024-04-20T11:27:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.312+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f366c19e580 con 0x7f365006ecd0 2024-04-20T11:27:46.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.346+0000 7f3662a00700 1 -- 172.21.15.117:0/2315770022 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f366c19e580 con 0x7f365006ecd0 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (16m) 7m ago 30m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (30m) 7m ago 30m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (29m) 7m ago 29m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (12m) 7m ago 30m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (12m) 7m ago 29m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (15m) 7m ago 29m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (8m) 7m ago 22m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (8m) 7m ago 23m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (22m) 7m ago 22m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (7m) 7m ago 23m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:27:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (19m) 7m ago 32m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (17m) 7m ago 28m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (14m) 7m ago 32m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (13m) 7m ago 28m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (17m) 7m ago 30m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (17m) 7m ago 29m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (11m) 7m ago 27m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (11m) 7m ago 27m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (10m) 7m ago 26m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (10m) 7m ago 26m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (9m) 7m ago 25m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (8m) 7m ago 24m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:27:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (16m) 7m ago 29m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:27:46.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.349+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecd0 msgr2=0x7f3650071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.349+0000 7f3670a00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecd0 0x7f3650071190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f365c00b5c0 tx=0x7f365c01a040 comp rx=0 tx=0).stop 2024-04-20T11:27:46.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.349+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c108e10 msgr2=0x7f366c19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.349+0000 7f3670a00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c108e10 0x7f366c19d7b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f365800b4b0 tx=0x7f365800b870 comp rx=0 tx=0).stop 2024-04-20T11:27:46.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.350+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 shutdown_connections 2024-04-20T11:27:46.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.350+0000 7f3670a00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecd0 0x7f3650071190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.350+0000 7f3670a00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c107520 0x7f366c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.350+0000 7f3670a00700 1 --2- 172.21.15.117:0/2315770022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c108e10 0x7f366c19d7b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.350+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 >> 172.21.15.117:0/2315770022 conn(0x7f366c0768f0 msgr2=0x7f366c078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.350+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 shutdown_connections 2024-04-20T11:27:46.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.350+0000 7f3670a00700 1 -- 172.21.15.117:0/2315770022 wait complete. 2024-04-20T11:27:46.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.504+0000 7fd2d2a00700 1 -- 172.21.15.117:0/2583672644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073c30 msgr2=0x7fd2cc10ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.504+0000 7fd2d2a00700 1 --2- 172.21.15.117:0/2583672644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073c30 0x7fd2cc10ceb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009910 tx=0x7fd2b8009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:46.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.505+0000 7fd2d2a00700 1 -- 172.21.15.117:0/2583672644 shutdown_connections 2024-04-20T11:27:46.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.505+0000 7fd2d2a00700 1 --2- 172.21.15.117:0/2583672644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073c30 0x7fd2cc10ceb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.505+0000 7fd2d2a00700 1 --2- 172.21.15.117:0/2583672644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc073310 0x7fd2cc0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.505+0000 7fd2d2a00700 1 -- 172.21.15.117:0/2583672644 >> 172.21.15.117:0/2583672644 conn(0x7fd2cc0fc6a0 msgr2=0x7fd2cc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.505+0000 7fd2d2a00700 1 -- 172.21.15.117:0/2583672644 shutdown_connections 2024-04-20T11:27:46.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.505+0000 7fd2d2a00700 1 -- 172.21.15.117:0/2583672644 wait complete. 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.506+0000 7fd2d2a00700 1 Processor -- start 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.506+0000 7fd2d2a00700 1 -- start start 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.506+0000 7fd2d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073310 0x7fd2cc19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.506+0000 7fd2d2a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc073c30 0x7fd2cc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.506+0000 7fd2d2a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19f4c0 con 0x7fd2cc073310 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.506+0000 7fd2d2a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc198b30 con 0x7fd2cc073c30 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073310 0x7fd2cc19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073310 0x7fd2cc19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58266/0 (socket says 172.21.15.117:58266) 2024-04-20T11:27:46.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cbe00700 1 -- 172.21.15.117:0/128978224 learned_addr learned my addr 172.21.15.117:0/128978224 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:46.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cb400700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc073c30 0x7fd2cc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cbe00700 1 -- 172.21.15.117:0/128978224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc073c30 msgr2=0x7fd2cc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cbe00700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc073c30 0x7fd2cc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cbe00700 1 -- 172.21.15.117:0/128978224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80095f0 con 0x7fd2cc073310 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cb400700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc073c30 0x7fd2cc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.507+0000 7fd2cbe00700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073310 0x7fd2cc19e9f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c930 tx=0x7fd2bc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.508+0000 7fd2c8c00700 1 -- 172.21.15.117:0/128978224 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2bc007ad0 con 0x7fd2cc073310 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.508+0000 7fd2c8c00700 1 -- 172.21.15.117:0/128978224 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2bc007c30 con 0x7fd2cc073310 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.508+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc198e10 con 0x7fd2cc073310 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.508+0000 7fd2c8c00700 1 -- 172.21.15.117:0/128978224 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2bc018610 con 0x7fd2cc073310 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.508+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc199360 con 0x7fd2cc073310 2024-04-20T11:27:46.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.509+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc107750 con 0x7fd2cc073310 2024-04-20T11:27:46.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.510+0000 7fd2c8c00700 1 -- 172.21.15.117:0/128978224 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd2bc00f450 con 0x7fd2cc073310 2024-04-20T11:27:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.511+0000 7fd2c8c00700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd2b006ecd0 0x7fd2b0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.511+0000 7fd2c8c00700 1 -- 172.21.15.117:0/128978224 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd2bc08f110 con 0x7fd2cc073310 2024-04-20T11:27:46.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.511+0000 7fd2cb400700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd2b006ecd0 0x7fd2b0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.512+0000 7fd2cb400700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd2b006ecd0 0x7fd2b0071190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8006010 tx=0x7fd2b800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.516+0000 7fd2c8c00700 1 -- 172.21.15.117:0/128978224 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd2bc05b980 con 0x7fd2cc073310 2024-04-20T11:27:46.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:46 smithi117 ceph-mon[99036]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:46.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:46 smithi117 ceph-mon[99036]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:46 smithi173 ceph-mon[74165]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:46 smithi173 ceph-mon[74165]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:46.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.824+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd2cc10ffd0 con 0x7fd2cc073310 2024-04-20T11:27:46.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.825+0000 7fd2c8c00700 1 -- 172.21.15.117:0/128978224 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fd2bc05b510 con 0x7fd2cc073310 2024-04-20T11:27:46.824 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:27:46.824 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:27:46.824 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:27:46.825 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:27:46.826 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd2b006ecd0 msgr2=0x7fd2b0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd2b006ecd0 0x7fd2b0071190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8006010 tx=0x7fd2b800b540 comp rx=0 tx=0).stop 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073310 msgr2=0x7fd2cc19e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073310 0x7fd2cc19e9f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c930 tx=0x7fd2bc00ccf0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 shutdown_connections 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd2b006ecd0 0x7fd2b0071190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc073310 0x7fd2cc19e9f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 --2- 172.21.15.117:0/128978224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc073c30 0x7fd2cc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 >> 172.21.15.117:0/128978224 conn(0x7fd2cc0fc6a0 msgr2=0x7fd2cc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 shutdown_connections 2024-04-20T11:27:46.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.829+0000 7fd2d2a00700 1 -- 172.21.15.117:0/128978224 wait complete. 2024-04-20T11:27:46.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.969+0000 7f81ab200700 1 -- 172.21.15.117:0/442286101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 msgr2=0x7f81a4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.969+0000 7f81ab200700 1 --2- 172.21.15.117:0/442286101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 0x7f81a4109290 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8190009910 tx=0x7f8190009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:46.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.969+0000 7f81ab200700 1 -- 172.21.15.117:0/442286101 shutdown_connections 2024-04-20T11:27:46.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.969+0000 7f81ab200700 1 --2- 172.21.15.117:0/442286101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 0x7f81a4109290 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.969+0000 7f81ab200700 1 --2- 172.21.15.117:0/442286101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4107520 0x7f81a4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.969+0000 7f81ab200700 1 -- 172.21.15.117:0/442286101 >> 172.21.15.117:0/442286101 conn(0x7f81a40768f0 msgr2=0x7f81a4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:46.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.969+0000 7f81ab200700 1 -- 172.21.15.117:0/442286101 shutdown_connections 2024-04-20T11:27:46.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.969+0000 7f81ab200700 1 -- 172.21.15.117:0/442286101 wait complete. 2024-04-20T11:27:46.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.970+0000 7f81ab200700 1 Processor -- start 2024-04-20T11:27:46.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.970+0000 7f81ab200700 1 -- start start 2024-04-20T11:27:46.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81ab200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4107520 0x7f81a419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81ab200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 0x7f81a419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81ab200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a419ded0 con 0x7f81a4108e10 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81ab200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41a1ba0 con 0x7f81a4107520 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81a3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 0x7f81a419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81a3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 0x7f81a419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58286/0 (socket says 172.21.15.117:58286) 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81a3400700 1 -- 172.21.15.117:0/151318994 learned_addr learned my addr 172.21.15.117:0/151318994 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81a3400700 1 -- 172.21.15.117:0/151318994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4107520 msgr2=0x7f81a419d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81a3e00700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4107520 0x7f81a419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81a3400700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4107520 0x7f81a419d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.971+0000 7f81a3400700 1 -- 172.21.15.117:0/151318994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81900095f0 con 0x7f81a4108e10 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.972+0000 7f81a3400700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 0x7f81a419d7f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8190006010 tx=0x7f819000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.972+0000 7f81a0c00700 1 -- 172.21.15.117:0/151318994 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f819001d070 con 0x7f81a4108e10 2024-04-20T11:27:46.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.972+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a41a1e20 con 0x7f81a4108e10 2024-04-20T11:27:46.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.972+0000 7f81a0c00700 1 -- 172.21.15.117:0/151318994 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f819000f460 con 0x7f81a4108e10 2024-04-20T11:27:46.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.972+0000 7f81a0c00700 1 -- 172.21.15.117:0/151318994 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8190017630 con 0x7f81a4108e10 2024-04-20T11:27:46.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.972+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a41a2390 con 0x7f81a4108e10 2024-04-20T11:27:46.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.973+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a410f890 con 0x7f81a4108e10 2024-04-20T11:27:46.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.975+0000 7f81a0c00700 1 -- 172.21.15.117:0/151318994 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f819000f5d0 con 0x7f81a4108e10 2024-04-20T11:27:46.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.975+0000 7f81a0c00700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f818806ecd0 0x7f8188071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:46.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.976+0000 7f81a0c00700 1 -- 172.21.15.117:0/151318994 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f81900903f0 con 0x7f81a4108e10 2024-04-20T11:27:46.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.976+0000 7f81a3e00700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f818806ecd0 0x7f8188071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:46.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.976+0000 7f81a3e00700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f818806ecd0 0x7f8188071190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8194006fd0 tx=0x7f8194008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:46.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:46.980+0000 7f81a0c00700 1 -- 172.21.15.117:0/151318994 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f81900591f0 con 0x7f81a4108e10 2024-04-20T11:27:47.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.256+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f81a419e5c0 con 0x7f81a4108e10 2024-04-20T11:27:47.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.257+0000 7f81a0c00700 1 -- 172.21.15.117:0/151318994 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f8190020370 con 0x7f81a4108e10 2024-04-20T11:27:47.257 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:27:47.257 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:27:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:27:47.259 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:27:47.260 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:27:47.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f818806ecd0 msgr2=0x7f8188071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f818806ecd0 0x7f8188071190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8194006fd0 tx=0x7f8194008040 comp rx=0 tx=0).stop 2024-04-20T11:27:47.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 msgr2=0x7f81a419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 0x7f81a419d7f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8190006010 tx=0x7f819000be20 comp rx=0 tx=0).stop 2024-04-20T11:27:47.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 shutdown_connections 2024-04-20T11:27:47.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f818806ecd0 0x7f8188071190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4107520 0x7f81a419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 --2- 172.21.15.117:0/151318994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a4108e10 0x7f81a419d7f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 >> 172.21.15.117:0/151318994 conn(0x7f81a40768f0 msgr2=0x7f81a40775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:47.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 shutdown_connections 2024-04-20T11:27:47.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.260+0000 7f81ab200700 1 -- 172.21.15.117:0/151318994 wait complete. 2024-04-20T11:27:47.262 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.405+0000 7fd281800700 1 -- 172.21.15.117:0/4263699222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 msgr2=0x7fd27c103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.405+0000 7fd281800700 1 --2- 172.21.15.117:0/4263699222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 0x7fd27c103400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009910 tx=0x7fd26c009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.405+0000 7fd281800700 1 -- 172.21.15.117:0/4263699222 shutdown_connections 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.405+0000 7fd281800700 1 --2- 172.21.15.117:0/4263699222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd27c1039d0 0x7fd27c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.405+0000 7fd281800700 1 --2- 172.21.15.117:0/4263699222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 0x7fd27c103400 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.405+0000 7fd281800700 1 -- 172.21.15.117:0/4263699222 >> 172.21.15.117:0/4263699222 conn(0x7fd27c0fe8b0 msgr2=0x7fd27c100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.405+0000 7fd281800700 1 -- 172.21.15.117:0/4263699222 shutdown_connections 2024-04-20T11:27:47.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.405+0000 7fd281800700 1 -- 172.21.15.117:0/4263699222 wait complete. 2024-04-20T11:27:47.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.406+0000 7fd281800700 1 Processor -- start 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.406+0000 7fd281800700 1 -- start start 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd281800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 0x7fd27c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd281800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd27c1039d0 0x7fd27c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd281800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c199a10 con 0x7fd27c103020 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd281800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19d6e0 con 0x7fd27c1039d0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd27aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 0x7fd27c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd27aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 0x7fd27c198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58292/0 (socket says 172.21.15.117:58292) 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd27aa00700 1 -- 172.21.15.117:0/2298589112 learned_addr learned my addr 172.21.15.117:0/2298589112 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd27aa00700 1 -- 172.21.15.117:0/2298589112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd27c1039d0 msgr2=0x7fd27c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd27aa00700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd27c1039d0 0x7fd27c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.407+0000 7fd27aa00700 1 -- 172.21.15.117:0/2298589112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0095f0 con 0x7fd27c103020 2024-04-20T11:27:47.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.408+0000 7fd27aa00700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 0x7fd27c198df0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0092e0 tx=0x7fd26c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.408+0000 7fd273400700 1 -- 172.21.15.117:0/2298589112 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26c01d070 con 0x7fd27c103020 2024-04-20T11:27:47.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.408+0000 7fd273400700 1 -- 172.21.15.117:0/2298589112 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd26c00f460 con 0x7fd27c103020 2024-04-20T11:27:47.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.408+0000 7fd273400700 1 -- 172.21.15.117:0/2298589112 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26c017630 con 0x7fd27c103020 2024-04-20T11:27:47.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.408+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c19d960 con 0x7fd27c103020 2024-04-20T11:27:47.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.408+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c19de50 con 0x7fd27c103020 2024-04-20T11:27:47.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.409+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c10b3c0 con 0x7fd27c103020 2024-04-20T11:27:47.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.410+0000 7fd273400700 1 -- 172.21.15.117:0/2298589112 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd26c00f5d0 con 0x7fd27c103020 2024-04-20T11:27:47.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.411+0000 7fd273400700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26006ecc0 0x7fd260071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.411+0000 7fd273400700 1 -- 172.21.15.117:0/2298589112 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd26c0903e0 con 0x7fd27c103020 2024-04-20T11:27:47.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.411+0000 7fd27a000700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26006ecc0 0x7fd260071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.412+0000 7fd27a000700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26006ecc0 0x7fd260071180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd27c19a350 tx=0x7fd268006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.416+0000 7fd273400700 1 -- 172.21.15.117:0/2298589112 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd26c059120 con 0x7fd27c103020 2024-04-20T11:27:47.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:47 smithi117 ceph-mon[99036]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:47.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:47 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/128978224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:27:47.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:47 smithi117 ceph-mon[99036]: pgmap v460: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:47.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:47 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/151318994' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:27:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:47 smithi173 ceph-mon[74165]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:47 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/128978224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:27:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:47 smithi173 ceph-mon[74165]: pgmap v460: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:47 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/151318994' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:27:47.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.661+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd27c110060 con 0x7fd26006ecc0 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.663+0000 7fd273400700 1 -- 172.21.15.117:0/2298589112 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd27c110060 con 0x7fd26006ecc0 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:27:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:27:47.664 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:27:47.664 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:27:47.664 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:27:47.664 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:27:47.664 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:27:47.664 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:27:47.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26006ecc0 msgr2=0x7fd260071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26006ecc0 0x7fd260071180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd27c19a350 tx=0x7fd268006d20 comp rx=0 tx=0).stop 2024-04-20T11:27:47.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 msgr2=0x7fd27c198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 0x7fd27c198df0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0092e0 tx=0x7fd26c00be20 comp rx=0 tx=0).stop 2024-04-20T11:27:47.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 shutdown_connections 2024-04-20T11:27:47.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26006ecc0 0x7fd260071180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd27c103020 0x7fd27c198df0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 --2- 172.21.15.117:0/2298589112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd27c1039d0 0x7fd27c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 >> 172.21.15.117:0/2298589112 conn(0x7fd27c0fe8b0 msgr2=0x7fd27c0ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:47.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 shutdown_connections 2024-04-20T11:27:47.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.666+0000 7fd281800700 1 -- 172.21.15.117:0/2298589112 wait complete. 2024-04-20T11:27:47.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.820+0000 7f9162c00700 1 -- 172.21.15.117:0/2166647846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 msgr2=0x7f915c10cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.820+0000 7f9162c00700 1 --2- 172.21.15.117:0/2166647846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 0x7f915c10cf30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9148009910 tx=0x7f9148009c20 comp rx=0 tx=0).stop 2024-04-20T11:27:47.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.820+0000 7f9162c00700 1 -- 172.21.15.117:0/2166647846 shutdown_connections 2024-04-20T11:27:47.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.820+0000 7f9162c00700 1 --2- 172.21.15.117:0/2166647846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 0x7f915c10cf30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.820+0000 7f9162c00700 1 --2- 172.21.15.117:0/2166647846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f915c073200 0x7f915c0735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.820+0000 7f9162c00700 1 -- 172.21.15.117:0/2166647846 >> 172.21.15.117:0/2166647846 conn(0x7f915c0fc6a0 msgr2=0x7f915c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:47.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.820+0000 7f9162c00700 1 -- 172.21.15.117:0/2166647846 shutdown_connections 2024-04-20T11:27:47.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.820+0000 7f9162c00700 1 -- 172.21.15.117:0/2166647846 wait complete. 2024-04-20T11:27:47.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.821+0000 7f9162c00700 1 Processor -- start 2024-04-20T11:27:47.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.821+0000 7f9162c00700 1 -- start start 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.821+0000 7f9162c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f915c073200 0x7f915c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f9162c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 0x7f915c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f9162c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c1998c0 con 0x7f915c073b20 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f9162c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c19c530 con 0x7f915c073200 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f915b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 0x7f915c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f915b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 0x7f915c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58324/0 (socket says 172.21.15.117:58324) 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f915b400700 1 -- 172.21.15.117:0/962816111 learned_addr learned my addr 172.21.15.117:0/962816111 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f915be00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f915c073200 0x7f915c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f915b400700 1 -- 172.21.15.117:0/962816111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f915c073200 msgr2=0x7f915c198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f915b400700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f915c073200 0x7f915c198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:47.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.822+0000 7f915b400700 1 -- 172.21.15.117:0/962816111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91480095f0 con 0x7f915c073b20 2024-04-20T11:27:47.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.823+0000 7f915b400700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 0x7f915c199230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9148006010 tx=0x7f914800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.823+0000 7f9158c00700 1 -- 172.21.15.117:0/962816111 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914801d070 con 0x7f915c073b20 2024-04-20T11:27:47.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.823+0000 7f9158c00700 1 -- 172.21.15.117:0/962816111 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f914800f460 con 0x7f915c073b20 2024-04-20T11:27:47.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.823+0000 7f9158c00700 1 -- 172.21.15.117:0/962816111 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9148017630 con 0x7f915c073b20 2024-04-20T11:27:47.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.823+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c19c7b0 con 0x7f915c073b20 2024-04-20T11:27:47.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.823+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c19cca0 con 0x7f915c073b20 2024-04-20T11:27:47.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.824+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c074940 con 0x7f915c073b20 2024-04-20T11:27:47.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.826+0000 7f9158c00700 1 -- 172.21.15.117:0/962816111 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f914800f5d0 con 0x7f915c073b20 2024-04-20T11:27:47.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.827+0000 7f9158c00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f914006ecc0 0x7f9140071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:47.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.827+0000 7f9158c00700 1 -- 172.21.15.117:0/962816111 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f91480903a0 con 0x7f915c073b20 2024-04-20T11:27:47.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.827+0000 7f915be00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f914006ecc0 0x7f9140071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:47.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.828+0000 7f915be00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f914006ecc0 0x7f9140071180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f914c005fd0 tx=0x7f914c00a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:47.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:47.831+0000 7f9158c00700 1 -- 172.21.15.117:0/962816111 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f91480591a0 con 0x7f915c073b20 2024-04-20T11:27:48.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.144+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f915c04ebb0 con 0x7f915c073b20 2024-04-20T11:27:48.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.145+0000 7f9158c00700 1 -- 172.21.15.117:0/962816111 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f914805c860 con 0x7f915c073b20 2024-04-20T11:27:48.144 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:27:48.145 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:27:48.145 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:27:48.145 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:27:48.145 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:27:48.145 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:27:48.145 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f914006ecc0 msgr2=0x7f9140071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f914006ecc0 0x7f9140071180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f914c005fd0 tx=0x7f914c00a560 comp rx=0 tx=0).stop 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 msgr2=0x7f915c199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 0x7f915c199230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9148006010 tx=0x7f914800be20 comp rx=0 tx=0).stop 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 shutdown_connections 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f914006ecc0 0x7f9140071180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f915c073200 0x7f915c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 --2- 172.21.15.117:0/962816111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c073b20 0x7f915c199230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 >> 172.21.15.117:0/962816111 conn(0x7f915c0fc6a0 msgr2=0x7f915c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 shutdown_connections 2024-04-20T11:27:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:27:48.148+0000 7f9162c00700 1 -- 172.21.15.117:0/962816111 wait complete. 2024-04-20T11:27:48.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:48 smithi117 ceph-mon[99036]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:48.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:48 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/962816111' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:27:48.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:48 smithi173 ceph-mon[74165]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:27:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:48 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/962816111' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:27:49.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:49 smithi117 ceph-mon[99036]: pgmap v461: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:49 smithi173 ceph-mon[74165]: pgmap v461: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:51.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:51 smithi117 ceph-mon[99036]: pgmap v462: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:51 smithi173 ceph-mon[74165]: pgmap v462: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:53.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:53 smithi117 ceph-mon[99036]: pgmap v463: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:53 smithi173 ceph-mon[74165]: pgmap v463: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:55.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:55 smithi117 ceph-mon[99036]: pgmap v464: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:55 smithi173 ceph-mon[74165]: pgmap v464: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:57 smithi117 ceph-mon[99036]: pgmap v465: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:57 smithi173 ceph-mon[74165]: pgmap v465: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:27:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:27:59.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:27:59 smithi117 ceph-mon[99036]: pgmap v466: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:27:59.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:27:59 smithi173 ceph-mon[74165]: pgmap v466: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:01.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:01 smithi117 ceph-mon[99036]: pgmap v467: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:01 smithi173 ceph-mon[74165]: pgmap v467: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:03.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:03 smithi117 ceph-mon[99036]: pgmap v468: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:03.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:03 smithi173 ceph-mon[74165]: pgmap v468: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:05 smithi173 ceph-mon[74165]: pgmap v469: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:05 smithi117 ceph-mon[99036]: pgmap v469: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:07.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:07 smithi117 ceph-mon[99036]: pgmap v470: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:07.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:07 smithi173 ceph-mon[74165]: pgmap v470: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:09.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:09 smithi117 ceph-mon[99036]: pgmap v471: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:09 smithi173 ceph-mon[74165]: pgmap v471: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:11.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:11 smithi117 ceph-mon[99036]: pgmap v472: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:11.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:11 smithi173 ceph-mon[74165]: pgmap v472: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:28:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:28:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:14 smithi117 ceph-mon[99036]: pgmap v473: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:14 smithi173 ceph-mon[74165]: pgmap v473: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:15 smithi117 ceph-mon[99036]: pgmap v474: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:15.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:15 smithi173 ceph-mon[74165]: pgmap v474: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:17.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:17 smithi117 ceph-mon[99036]: pgmap v475: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:17 smithi173 ceph-mon[74165]: pgmap v475: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:18.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.318+0000 7f546f600700 1 -- 172.21.15.117:0/1922033981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075410 msgr2=0x7f54680757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.318+0000 7f546f600700 1 --2- 172.21.15.117:0/1922033981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075410 0x7f54680757f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f545c009910 tx=0x7f545c009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:18.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.318+0000 7f546f600700 1 -- 172.21.15.117:0/1922033981 shutdown_connections 2024-04-20T11:28:18.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.318+0000 7f546f600700 1 --2- 172.21.15.117:0/1922033981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5468075d30 0x7f5468111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.318+0000 7f546f600700 1 --2- 172.21.15.117:0/1922033981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075410 0x7f54680757f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.318+0000 7f546f600700 1 -- 172.21.15.117:0/1922033981 >> 172.21.15.117:0/1922033981 conn(0x7f54680fe8b0 msgr2=0x7f5468100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:18.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.318+0000 7f546f600700 1 -- 172.21.15.117:0/1922033981 shutdown_connections 2024-04-20T11:28:18.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.318+0000 7f546f600700 1 -- 172.21.15.117:0/1922033981 wait complete. 2024-04-20T11:28:18.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.319+0000 7f546f600700 1 Processor -- start 2024-04-20T11:28:18.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.320+0000 7f546f600700 1 -- start start 2024-04-20T11:28:18.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.320+0000 7f546f600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5468075410 0x7f546819d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.320+0000 7f546f600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075d30 0x7f546819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.320+0000 7f546f600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546819dda0 con 0x7f5468075d30 2024-04-20T11:28:18.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.320+0000 7f546f600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681a1a70 con 0x7f5468075410 2024-04-20T11:28:18.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.320+0000 7f5467e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075d30 0x7f546819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.320+0000 7f5467e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075d30 0x7f546819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53464/0 (socket says 172.21.15.117:53464) 2024-04-20T11:28:18.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.320+0000 7f5467e00700 1 -- 172.21.15.117:0/878845090 learned_addr learned my addr 172.21.15.117:0/878845090 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:18.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.321+0000 7f546cc00700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5468075410 0x7f546819d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.321+0000 7f5467e00700 1 -- 172.21.15.117:0/878845090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5468075410 msgr2=0x7f546819d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.321+0000 7f5467e00700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5468075410 0x7f546819d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.321+0000 7f5467e00700 1 -- 172.21.15.117:0/878845090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0095f0 con 0x7f5468075d30 2024-04-20T11:28:18.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.321+0000 7f5467e00700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075d30 0x7f546819d6c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f545400b4b0 tx=0x7f545400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:18.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.322+0000 7f5465600700 1 -- 172.21.15.117:0/878845090 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5454011800 con 0x7f5468075d30 2024-04-20T11:28:18.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.322+0000 7f5465600700 1 -- 172.21.15.117:0/878845090 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5454011e20 con 0x7f5468075d30 2024-04-20T11:28:18.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.322+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54681a1d50 con 0x7f5468075d30 2024-04-20T11:28:18.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.322+0000 7f5465600700 1 -- 172.21.15.117:0/878845090 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f545400f2e0 con 0x7f5468075d30 2024-04-20T11:28:18.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.322+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54681a22a0 con 0x7f5468075d30 2024-04-20T11:28:18.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.323+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5468076b50 con 0x7f5468075d30 2024-04-20T11:28:18.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.324+0000 7f5465600700 1 -- 172.21.15.117:0/878845090 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5454010380 con 0x7f5468075d30 2024-04-20T11:28:18.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.325+0000 7f5465600700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f545006ecd0 0x7f5450071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.326+0000 7f5465600700 1 -- 172.21.15.117:0/878845090 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f545408ea70 con 0x7f5468075d30 2024-04-20T11:28:18.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.326+0000 7f546cc00700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f545006ecd0 0x7f5450071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.326+0000 7f546cc00700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f545006ecd0 0x7f5450071190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f545c00b5c0 tx=0x7f545c005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:18.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.331+0000 7f5465600700 1 -- 172.21.15.117:0/878845090 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f54540577b0 con 0x7f5468075d30 2024-04-20T11:28:18.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.579+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54681144c0 con 0x7f545006ecd0 2024-04-20T11:28:18.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.581+0000 7f5465600700 1 -- 172.21.15.117:0/878845090 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f54681144c0 con 0x7f545006ecd0 2024-04-20T11:28:18.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f545006ecd0 msgr2=0x7f5450071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f545006ecd0 0x7f5450071190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f545c00b5c0 tx=0x7f545c005a90 comp rx=0 tx=0).stop 2024-04-20T11:28:18.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075d30 msgr2=0x7f546819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075d30 0x7f546819d6c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f545400b4b0 tx=0x7f545400b870 comp rx=0 tx=0).stop 2024-04-20T11:28:18.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 shutdown_connections 2024-04-20T11:28:18.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f545006ecd0 0x7f5450071190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5468075410 0x7f546819d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 --2- 172.21.15.117:0/878845090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5468075d30 0x7f546819d6c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 >> 172.21.15.117:0/878845090 conn(0x7f54680fe8b0 msgr2=0x7f5468100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:18.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 shutdown_connections 2024-04-20T11:28:18.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.584+0000 7f546f600700 1 -- 172.21.15.117:0/878845090 wait complete. 2024-04-20T11:28:18.600 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:28:18.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.747+0000 7f8ea8a00700 1 -- 172.21.15.117:0/1980480184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4108e10 msgr2=0x7f8ea4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.747+0000 7f8ea8a00700 1 --2- 172.21.15.117:0/1980480184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4108e10 0x7f8ea4109290 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009910 tx=0x7f8e90009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:18.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.748+0000 7f8ea8a00700 1 -- 172.21.15.117:0/1980480184 shutdown_connections 2024-04-20T11:28:18.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.748+0000 7f8ea8a00700 1 --2- 172.21.15.117:0/1980480184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4108e10 0x7f8ea4109290 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.748+0000 7f8ea8a00700 1 --2- 172.21.15.117:0/1980480184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ea4107520 0x7f8ea4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.748+0000 7f8ea8a00700 1 -- 172.21.15.117:0/1980480184 >> 172.21.15.117:0/1980480184 conn(0x7f8ea40768f0 msgr2=0x7f8ea4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:18.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.748+0000 7f8ea8a00700 1 -- 172.21.15.117:0/1980480184 shutdown_connections 2024-04-20T11:28:18.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.748+0000 7f8ea8a00700 1 -- 172.21.15.117:0/1980480184 wait complete. 2024-04-20T11:28:18.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.749+0000 7f8ea8a00700 1 Processor -- start 2024-04-20T11:28:18.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.749+0000 7f8ea8a00700 1 -- start start 2024-04-20T11:28:18.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.749+0000 7f8ea8a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4107520 0x7f8ea419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.749+0000 7f8ea8a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ea4108e10 0x7f8ea419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.749+0000 7f8ea8a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea419ddf0 con 0x7f8ea4107520 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.749+0000 7f8ea8a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41a1a70 con 0x7f8ea4108e10 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.750+0000 7f8ea1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4107520 0x7f8ea419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.750+0000 7f8ea1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4107520 0x7f8ea419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53482/0 (socket says 172.21.15.117:53482) 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.750+0000 7f8ea1e00700 1 -- 172.21.15.117:0/2669239365 learned_addr learned my addr 172.21.15.117:0/2669239365 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.750+0000 7f8ea1400700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ea4108e10 0x7f8ea419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.750+0000 7f8ea1e00700 1 -- 172.21.15.117:0/2669239365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ea4108e10 msgr2=0x7f8ea419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.750+0000 7f8ea1e00700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ea4108e10 0x7f8ea419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.750+0000 7f8ea1e00700 1 -- 172.21.15.117:0/2669239365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e900095f0 con 0x7f8ea4107520 2024-04-20T11:28:18.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.750+0000 7f8ea1e00700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4107520 0x7f8ea419d1d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400d8d0 tx=0x7f8e9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:18.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.751+0000 7f8e9aa00700 1 -- 172.21.15.117:0/2669239365 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e94009940 con 0x7f8ea4107520 2024-04-20T11:28:18.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.751+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea41a1d50 con 0x7f8ea4107520 2024-04-20T11:28:18.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.751+0000 7f8e9aa00700 1 -- 172.21.15.117:0/2669239365 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e94010460 con 0x7f8ea4107520 2024-04-20T11:28:18.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.751+0000 7f8e9aa00700 1 -- 172.21.15.117:0/2669239365 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9400f5f0 con 0x7f8ea4107520 2024-04-20T11:28:18.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.751+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea41a2320 con 0x7f8ea4107520 2024-04-20T11:28:18.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.752+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea410f870 con 0x7f8ea4107520 2024-04-20T11:28:18.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.753+0000 7f8e9aa00700 1 -- 172.21.15.117:0/2669239365 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8e940105d0 con 0x7f8ea4107520 2024-04-20T11:28:18.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.754+0000 7f8e9aa00700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e8806ecd0 0x7f8e88071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:18.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.754+0000 7f8e9aa00700 1 -- 172.21.15.117:0/2669239365 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8e9408edb0 con 0x7f8ea4107520 2024-04-20T11:28:18.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.754+0000 7f8ea1400700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e8806ecd0 0x7f8e88071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:18.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.755+0000 7f8ea1400700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e8806ecd0 0x7f8e88071190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8e90006010 tx=0x7f8e9000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:18.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.759+0000 7f8e9aa00700 1 -- 172.21.15.117:0/2669239365 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8e94057b70 con 0x7f8ea4107520 2024-04-20T11:28:19.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:18.999+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ea41144c0 con 0x7f8e8806ecd0 2024-04-20T11:28:19.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.001+0000 7f8e9aa00700 1 -- 172.21.15.117:0/2669239365 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8ea41144c0 con 0x7f8e8806ecd0 2024-04-20T11:28:19.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e8806ecd0 msgr2=0x7f8e88071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e8806ecd0 0x7f8e88071190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8e90006010 tx=0x7f8e9000b540 comp rx=0 tx=0).stop 2024-04-20T11:28:19.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4107520 msgr2=0x7f8ea419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4107520 0x7f8ea419d1d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400d8d0 tx=0x7f8e9400dc90 comp rx=0 tx=0).stop 2024-04-20T11:28:19.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 shutdown_connections 2024-04-20T11:28:19.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e8806ecd0 0x7f8e88071190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ea4107520 0x7f8ea419d1d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 --2- 172.21.15.117:0/2669239365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ea4108e10 0x7f8ea419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 >> 172.21.15.117:0/2669239365 conn(0x7f8ea40768f0 msgr2=0x7f8ea4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:19.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 shutdown_connections 2024-04-20T11:28:19.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.004+0000 7f8ea8a00700 1 -- 172.21.15.117:0/2669239365 wait complete. 2024-04-20T11:28:19.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.161+0000 7fb281800700 1 -- 172.21.15.117:0/2237453820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 msgr2=0x7fb27c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.161+0000 7fb281800700 1 --2- 172.21.15.117:0/2237453820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 0x7fb27c109290 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb268009910 tx=0x7fb268009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:19.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.161+0000 7fb281800700 1 -- 172.21.15.117:0/2237453820 shutdown_connections 2024-04-20T11:28:19.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.161+0000 7fb281800700 1 --2- 172.21.15.117:0/2237453820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 0x7fb27c109290 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.161+0000 7fb281800700 1 --2- 172.21.15.117:0/2237453820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb27c107520 0x7fb27c107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.161+0000 7fb281800700 1 -- 172.21.15.117:0/2237453820 >> 172.21.15.117:0/2237453820 conn(0x7fb27c0768f0 msgr2=0x7fb27c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:19.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.161+0000 7fb281800700 1 -- 172.21.15.117:0/2237453820 shutdown_connections 2024-04-20T11:28:19.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.161+0000 7fb281800700 1 -- 172.21.15.117:0/2237453820 wait complete. 2024-04-20T11:28:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.162+0000 7fb281800700 1 Processor -- start 2024-04-20T11:28:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.162+0000 7fb281800700 1 -- start start 2024-04-20T11:28:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb281800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb27c107520 0x7fb27c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb281800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 0x7fb27c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb281800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c19ddf0 con 0x7fb27c108e10 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb281800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c1a1a70 con 0x7fb27c107520 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb27a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 0x7fb27c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb27a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 0x7fb27c19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53504/0 (socket says 172.21.15.117:53504) 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb27a000700 1 -- 172.21.15.117:0/528172188 learned_addr learned my addr 172.21.15.117:0/528172188 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb27aa00700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb27c107520 0x7fb27c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb27a000700 1 -- 172.21.15.117:0/528172188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb27c107520 msgr2=0x7fb27c19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb27a000700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb27c107520 0x7fb27c19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb27a000700 1 -- 172.21.15.117:0/528172188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2680095f0 con 0x7fb27c108e10 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.163+0000 7fb27aa00700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb27c107520 0x7fb27c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.164+0000 7fb27a000700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 0x7fb27c19d710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb268006010 tx=0x7fb26800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.164+0000 7fb273400700 1 -- 172.21.15.117:0/528172188 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb26801d070 con 0x7fb27c108e10 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.164+0000 7fb273400700 1 -- 172.21.15.117:0/528172188 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb26800f460 con 0x7fb27c108e10 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.164+0000 7fb273400700 1 -- 172.21.15.117:0/528172188 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb268017630 con 0x7fb27c108e10 2024-04-20T11:28:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.164+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c1a1cf0 con 0x7fb27c108e10 2024-04-20T11:28:19.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.164+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c1a21c0 con 0x7fb27c108e10 2024-04-20T11:28:19.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.165+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27c10f870 con 0x7fb27c108e10 2024-04-20T11:28:19.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.166+0000 7fb273400700 1 -- 172.21.15.117:0/528172188 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb26800f5d0 con 0x7fb27c108e10 2024-04-20T11:28:19.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.167+0000 7fb273400700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb26006ecc0 0x7fb260071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.167+0000 7fb273400700 1 -- 172.21.15.117:0/528172188 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb268090370 con 0x7fb27c108e10 2024-04-20T11:28:19.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.168+0000 7fb27aa00700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb26006ecc0 0x7fb260071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.168+0000 7fb27aa00700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb26006ecc0 0x7fb260071180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb26c005fd0 tx=0x7fb26c005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.172+0000 7fb273400700 1 -- 172.21.15.117:0/528172188 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb2680591f0 con 0x7fb27c108e10 2024-04-20T11:28:19.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.409+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb27c19e490 con 0x7fb26006ecc0 2024-04-20T11:28:19.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.463+0000 7fb273400700 1 -- 172.21.15.117:0/528172188 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb27c19e490 con 0x7fb26006ecc0 2024-04-20T11:28:19.464 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:28:19.464 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (16m) 8m ago 30m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:28:19.464 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (31m) 8m ago 31m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:28:19.464 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (29m) 8m ago 29m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:28:19.464 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (13m) 8m ago 31m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:28:19.464 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (12m) 8m ago 29m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:28:19.464 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (16m) 8m ago 30m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (8m) 8m ago 23m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (9m) 8m ago 23m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (23m) 8m ago 23m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (8m) 8m ago 23m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (19m) 8m ago 32m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (18m) 8m ago 29m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (14m) 8m ago 32m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (14m) 8m ago 29m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (18m) 8m ago 31m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (17m) 8m ago 29m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (12m) 8m ago 28m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (11m) 8m ago 27m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (11m) 8m ago 27m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:28:19.465 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (10m) 8m ago 26m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:28:19.466 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (10m) 8m ago 26m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:28:19.466 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (9m) 8m ago 25m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:28:19.466 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (17m) 8m ago 30m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb26006ecc0 msgr2=0x7fb260071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb26006ecc0 0x7fb260071180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb26c005fd0 tx=0x7fb26c005dc0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 msgr2=0x7fb27c19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 0x7fb27c19d710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb268006010 tx=0x7fb26800be20 comp rx=0 tx=0).stop 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 shutdown_connections 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb26006ecc0 0x7fb260071180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb27c107520 0x7fb27c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 --2- 172.21.15.117:0/528172188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb27c108e10 0x7fb27c19d710 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 >> 172.21.15.117:0/528172188 conn(0x7fb27c0768f0 msgr2=0x7fb27c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 shutdown_connections 2024-04-20T11:28:19.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.466+0000 7fb281800700 1 -- 172.21.15.117:0/528172188 wait complete. 2024-04-20T11:28:19.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.619+0000 7f25d4200700 1 -- 172.21.15.117:0/2230602110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc0ff8d0 msgr2=0x7f25cc0ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.619+0000 7f25d4200700 1 --2- 172.21.15.117:0/2230602110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc0ff8d0 0x7f25cc0ffcb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f25c000b0a0 tx=0x7f25c000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.619+0000 7f25d4200700 1 -- 172.21.15.117:0/2230602110 shutdown_connections 2024-04-20T11:28:19.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.619+0000 7f25d4200700 1 --2- 172.21.15.117:0/2230602110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25cc100280 0x7f25cc10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.619+0000 7f25d4200700 1 --2- 172.21.15.117:0/2230602110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc0ff8d0 0x7f25cc0ffcb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.619+0000 7f25d4200700 1 -- 172.21.15.117:0/2230602110 >> 172.21.15.117:0/2230602110 conn(0x7f25cc076750 msgr2=0x7f25cc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:19.620 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:19 smithi117 ceph-mon[99036]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:19.621 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:19 smithi117 ceph-mon[99036]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:19.621 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:19 smithi117 ceph-mon[99036]: pgmap v476: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:19.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.619+0000 7f25d4200700 1 -- 172.21.15.117:0/2230602110 shutdown_connections 2024-04-20T11:28:19.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.619+0000 7f25d4200700 1 -- 172.21.15.117:0/2230602110 wait complete. 2024-04-20T11:28:19.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.620+0000 7f25d4200700 1 Processor -- start 2024-04-20T11:28:19.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.620+0000 7f25d4200700 1 -- start start 2024-04-20T11:28:19.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.621+0000 7f25d4200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25cc0ff8d0 0x7f25cc19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.621+0000 7f25d4200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc100280 0x7f25cc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.621+0000 7f25d4200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc19f520 con 0x7f25cc100280 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.621+0000 7f25d4200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc198b00 con 0x7f25cc0ff8d0 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.621+0000 7f25d0e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc100280 0x7f25cc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.621+0000 7f25d0e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc100280 0x7f25cc19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53520/0 (socket says 172.21.15.117:53520) 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.621+0000 7f25d0e00700 1 -- 172.21.15.117:0/3755207984 learned_addr learned my addr 172.21.15.117:0/3755207984 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25d1800700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25cc0ff8d0 0x7f25cc19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25d0e00700 1 -- 172.21.15.117:0/3755207984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25cc0ff8d0 msgr2=0x7f25cc19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25d0e00700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25cc0ff8d0 0x7f25cc19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25d0e00700 1 -- 172.21.15.117:0/3755207984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0009d00 con 0x7f25cc100280 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25d1800700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25cc0ff8d0 0x7f25cc19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:19.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25d0e00700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc100280 0x7f25cc19ef00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f25b800d8d0 tx=0x7f25b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25ca000700 1 -- 172.21.15.117:0/3755207984 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b8009940 con 0x7f25cc100280 2024-04-20T11:28:19.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25ca000700 1 -- 172.21.15.117:0/3755207984 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25b8010460 con 0x7f25cc100280 2024-04-20T11:28:19.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25d4200700 1 -- 172.21.15.117:0/3755207984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc198de0 con 0x7f25cc100280 2024-04-20T11:28:19.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25ca000700 1 -- 172.21.15.117:0/3755207984 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b800f5f0 con 0x7f25cc100280 2024-04-20T11:28:19.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.622+0000 7f25d4200700 1 -- 172.21.15.117:0/3755207984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc199330 con 0x7f25cc100280 2024-04-20T11:28:19.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.624+0000 7f25bf400700 1 -- 172.21.15.117:0/3755207984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc068e50 con 0x7f25cc100280 2024-04-20T11:28:19.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.625+0000 7f25ca000700 1 -- 172.21.15.117:0/3755207984 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f25b80105d0 con 0x7f25cc100280 2024-04-20T11:28:19.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.626+0000 7f25ca000700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f25b406ecc0 0x7f25b4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:19.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.626+0000 7f25ca000700 1 -- 172.21.15.117:0/3755207984 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f25b808eb80 con 0x7f25cc100280 2024-04-20T11:28:19.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.627+0000 7f25d1800700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f25b406ecc0 0x7f25b4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:19.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.627+0000 7f25d1800700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f25b406ecc0 0x7f25b4071180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009480 tx=0x7f25c000b7d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:19.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.630+0000 7f25ca000700 1 -- 172.21.15.117:0/3755207984 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f25b805b470 con 0x7f25cc100280 2024-04-20T11:28:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:19 smithi173 ceph-mon[74165]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:19 smithi173 ceph-mon[74165]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:19 smithi173 ceph-mon[74165]: pgmap v476: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:19.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.921+0000 7f25bf400700 1 -- 172.21.15.117:0/3755207984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f25cc062510 con 0x7f25cc100280 2024-04-20T11:28:19.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.922+0000 7f25ca000700 1 -- 172.21.15.117:0/3755207984 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f25b805b000 con 0x7f25cc100280 2024-04-20T11:28:19.923 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:28:19.923 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:28:19.923 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:28:19.924 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:28:19.925 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 -- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f25b406ecc0 msgr2=0x7f25b4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f25b406ecc0 0x7f25b4071180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009480 tx=0x7f25c000b7d0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 -- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc100280 msgr2=0x7f25cc19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc100280 0x7f25cc19ef00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f25b800d8d0 tx=0x7f25b800dc90 comp rx=0 tx=0).stop 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 -- 172.21.15.117:0/3755207984 shutdown_connections 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f25b406ecc0 0x7f25b4071180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25cc0ff8d0 0x7f25cc19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 --2- 172.21.15.117:0/3755207984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25cc100280 0x7f25cc19ef00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 -- 172.21.15.117:0/3755207984 >> 172.21.15.117:0/3755207984 conn(0x7f25cc076750 msgr2=0x7f25cc0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 -- 172.21.15.117:0/3755207984 shutdown_connections 2024-04-20T11:28:19.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:19.925+0000 7f25bf400700 1 -- 172.21.15.117:0/3755207984 wait complete. 2024-04-20T11:28:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.080+0000 7ff26f000700 1 -- 172.21.15.117:0/2476009952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 msgr2=0x7ff268101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.080+0000 7ff26f000700 1 --2- 172.21.15.117:0/2476009952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 0x7ff268101200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff258009910 tx=0x7ff258009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.080+0000 7ff26f000700 1 -- 172.21.15.117:0/2476009952 shutdown_connections 2024-04-20T11:28:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.080+0000 7ff26f000700 1 --2- 172.21.15.117:0/2476009952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2681017d0 0x7ff268105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.080+0000 7ff26f000700 1 --2- 172.21.15.117:0/2476009952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 0x7ff268101200 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.080+0000 7ff26f000700 1 -- 172.21.15.117:0/2476009952 >> 172.21.15.117:0/2476009952 conn(0x7ff2680fc6b0 msgr2=0x7ff2680fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.080+0000 7ff26f000700 1 -- 172.21.15.117:0/2476009952 shutdown_connections 2024-04-20T11:28:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.080+0000 7ff26f000700 1 -- 172.21.15.117:0/2476009952 wait complete. 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.081+0000 7ff26f000700 1 Processor -- start 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.081+0000 7ff26f000700 1 -- start start 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff26f000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 0x7ff2680731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff26f000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2681017d0 0x7ff268073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff26f000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff268073c60 con 0x7ff268100e20 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff26f000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff268073da0 con 0x7ff2681017d0 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff267e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 0x7ff2680731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff267e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 0x7ff2680731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53532/0 (socket says 172.21.15.117:53532) 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff267e00700 1 -- 172.21.15.117:0/4258493433 learned_addr learned my addr 172.21.15.117:0/4258493433 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff267400700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2681017d0 0x7ff268073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff267e00700 1 -- 172.21.15.117:0/4258493433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2681017d0 msgr2=0x7ff268073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff267e00700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2681017d0 0x7ff268073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.082+0000 7ff267e00700 1 -- 172.21.15.117:0/4258493433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2580095f0 con 0x7ff268100e20 2024-04-20T11:28:20.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.083+0000 7ff267e00700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 0x7ff2680731e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff2580092e0 tx=0x7ff25800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.083+0000 7ff264c00700 1 -- 172.21.15.117:0/4258493433 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff25801d070 con 0x7ff268100e20 2024-04-20T11:28:20.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.083+0000 7ff264c00700 1 -- 172.21.15.117:0/4258493433 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff25800f460 con 0x7ff268100e20 2024-04-20T11:28:20.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.083+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff268074020 con 0x7ff268100e20 2024-04-20T11:28:20.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.083+0000 7ff264c00700 1 -- 172.21.15.117:0/4258493433 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff258017630 con 0x7ff268100e20 2024-04-20T11:28:20.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.083+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2681a2f50 con 0x7ff268100e20 2024-04-20T11:28:20.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.085+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2681091c0 con 0x7ff268100e20 2024-04-20T11:28:20.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.086+0000 7ff264c00700 1 -- 172.21.15.117:0/4258493433 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff25800f5d0 con 0x7ff268100e20 2024-04-20T11:28:20.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.087+0000 7ff264c00700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff24c06ecc0 0x7ff24c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.087+0000 7ff264c00700 1 -- 172.21.15.117:0/4258493433 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff258090400 con 0x7ff268100e20 2024-04-20T11:28:20.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.087+0000 7ff267400700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff24c06ecc0 0x7ff24c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.088+0000 7ff267400700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff24c06ecc0 0x7ff24c071180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff254005fd0 tx=0x7ff25400a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.091+0000 7ff264c00700 1 -- 172.21.15.117:0/4258493433 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff258059140 con 0x7ff268100e20 2024-04-20T11:28:20.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.372+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff26810ddf0 con 0x7ff268100e20 2024-04-20T11:28:20.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.373+0000 7ff264c00700 1 -- 172.21.15.117:0/4258493433 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7ff25805c800 con 0x7ff268100e20 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:28:20.374 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:28:20.375 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:28:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:28:20.377 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:28:20.377 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:28:20.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.375+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff24c06ecc0 msgr2=0x7ff24c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.375+0000 7ff26f000700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff24c06ecc0 0x7ff24c071180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff254005fd0 tx=0x7ff25400a560 comp rx=0 tx=0).stop 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.375+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 msgr2=0x7ff2680731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.375+0000 7ff26f000700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 0x7ff2680731e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff2580092e0 tx=0x7ff25800be20 comp rx=0 tx=0).stop 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.376+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 shutdown_connections 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.376+0000 7ff26f000700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff24c06ecc0 0x7ff24c071180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.376+0000 7ff26f000700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff268100e20 0x7ff2680731e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.376+0000 7ff26f000700 1 --2- 172.21.15.117:0/4258493433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2681017d0 0x7ff268073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.376+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 >> 172.21.15.117:0/4258493433 conn(0x7ff2680fc6b0 msgr2=0x7ff2680fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.376+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 shutdown_connections 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.376+0000 7ff26f000700 1 -- 172.21.15.117:0/4258493433 wait complete. 2024-04-20T11:28:20.378 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:28:20.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.537+0000 7f2821000700 1 -- 172.21.15.117:0/637664012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c1003b0 msgr2=0x7f281c10d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.537+0000 7f2821000700 1 --2- 172.21.15.117:0/637664012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c1003b0 0x7f281c10d070 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2808009910 tx=0x7f2808009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:20.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.537+0000 7f2821000700 1 -- 172.21.15.117:0/637664012 shutdown_connections 2024-04-20T11:28:20.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.537+0000 7f2821000700 1 --2- 172.21.15.117:0/637664012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c1003b0 0x7f281c10d070 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.537+0000 7f2821000700 1 --2- 172.21.15.117:0/637664012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c0ffa00 0x7f281c0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.537+0000 7f2821000700 1 -- 172.21.15.117:0/637664012 >> 172.21.15.117:0/637664012 conn(0x7f281c0fb640 msgr2=0x7f281c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:20.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.538+0000 7f2821000700 1 -- 172.21.15.117:0/637664012 shutdown_connections 2024-04-20T11:28:20.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.538+0000 7f2821000700 1 -- 172.21.15.117:0/637664012 wait complete. 2024-04-20T11:28:20.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.539+0000 7f2821000700 1 Processor -- start 2024-04-20T11:28:20.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.539+0000 7f2821000700 1 -- start start 2024-04-20T11:28:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2821000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c0ffa00 0x7f281c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2821000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c1003b0 0x7f281c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2821000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c199940 con 0x7f281c0ffa00 2024-04-20T11:28:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2821000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c199a80 con 0x7f281c1003b0 2024-04-20T11:28:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2819e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c0ffa00 0x7f281c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2819e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c0ffa00 0x7f281c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53550/0 (socket says 172.21.15.117:53550) 2024-04-20T11:28:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2819e00700 1 -- 172.21.15.117:0/4139873872 learned_addr learned my addr 172.21.15.117:0/4139873872 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:20.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2819400700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c1003b0 0x7f281c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2819e00700 1 -- 172.21.15.117:0/4139873872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c1003b0 msgr2=0x7f281c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2819e00700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c1003b0 0x7f281c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.540+0000 7f2819e00700 1 -- 172.21.15.117:0/4139873872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28080095f0 con 0x7f281c0ffa00 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.541+0000 7f2819e00700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c0ffa00 0x7f281c198db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c930 tx=0x7f280c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.541+0000 7f2812a00700 1 -- 172.21.15.117:0/4139873872 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f280c007ad0 con 0x7f281c0ffa00 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.541+0000 7f2812a00700 1 -- 172.21.15.117:0/4139873872 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f280c007c30 con 0x7f281c0ffa00 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.541+0000 7f2812a00700 1 -- 172.21.15.117:0/4139873872 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f280c018610 con 0x7f281c0ffa00 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.541+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c19d810 con 0x7f281c0ffa00 2024-04-20T11:28:20.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.541+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c19dd10 con 0x7f281c0ffa00 2024-04-20T11:28:20.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.542+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c10aef0 con 0x7f281c0ffa00 2024-04-20T11:28:20.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.544+0000 7f2812a00700 1 -- 172.21.15.117:0/4139873872 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f280c00f450 con 0x7f281c0ffa00 2024-04-20T11:28:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.545+0000 7f2812a00700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f280006ecc0 0x7f2800071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.545+0000 7f2812a00700 1 -- 172.21.15.117:0/4139873872 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f280c08f0f0 con 0x7f281c0ffa00 2024-04-20T11:28:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.545+0000 7f2819400700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f280006ecc0 0x7f2800071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.546+0000 7f2819400700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f280006ecc0 0x7f2800071180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f281c19a310 tx=0x7f280800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.550+0000 7f2812a00700 1 -- 172.21.15.117:0/4139873872 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f280c057e30 con 0x7f281c0ffa00 2024-04-20T11:28:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:20 smithi117 ceph-mon[99036]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:20 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3755207984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:28:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:20 smithi173 ceph-mon[74165]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:20 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3755207984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:28:20.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.794+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f281c19a130 con 0x7f280006ecc0 2024-04-20T11:28:20.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.796+0000 7f2812a00700 1 -- 172.21.15.117:0/4139873872 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f281c19a130 con 0x7f280006ecc0 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:28:20.797 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:28:20.798 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:28:20.798 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:28:20.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f280006ecc0 msgr2=0x7f2800071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f280006ecc0 0x7f2800071180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f281c19a310 tx=0x7f280800b540 comp rx=0 tx=0).stop 2024-04-20T11:28:20.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c0ffa00 msgr2=0x7f281c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c0ffa00 0x7f281c198db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c930 tx=0x7f280c00ccf0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 shutdown_connections 2024-04-20T11:28:20.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f280006ecc0 0x7f2800071180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f281c0ffa00 0x7f281c198db0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 --2- 172.21.15.117:0/4139873872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c1003b0 0x7f281c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.799+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 >> 172.21.15.117:0/4139873872 conn(0x7f281c0fb640 msgr2=0x7f281c108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:20.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.800+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 shutdown_connections 2024-04-20T11:28:20.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.800+0000 7f2821000700 1 -- 172.21.15.117:0/4139873872 wait complete. 2024-04-20T11:28:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.952+0000 7f1d9ce00700 1 -- 172.21.15.117:0/4038769590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 msgr2=0x7f1d9810ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.952+0000 7f1d9ce00700 1 --2- 172.21.15.117:0/4038769590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 0x7f1d9810ab50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009910 tx=0x7f1d84009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.952+0000 7f1d9ce00700 1 -- 172.21.15.117:0/4038769590 shutdown_connections 2024-04-20T11:28:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.952+0000 7f1d9ce00700 1 --2- 172.21.15.117:0/4038769590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 0x7f1d9810ab50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.952+0000 7f1d9ce00700 1 --2- 172.21.15.117:0/4038769590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d98101d40 0x7f1d98102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.952+0000 7f1d9ce00700 1 -- 172.21.15.117:0/4038769590 >> 172.21.15.117:0/4038769590 conn(0x7f1d98076750 msgr2=0x7f1d980fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.952+0000 7f1d9ce00700 1 -- 172.21.15.117:0/4038769590 shutdown_connections 2024-04-20T11:28:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.952+0000 7f1d9ce00700 1 -- 172.21.15.117:0/4038769590 wait complete. 2024-04-20T11:28:20.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.953+0000 7f1d9ce00700 1 Processor -- start 2024-04-20T11:28:20.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.953+0000 7f1d9ce00700 1 -- start start 2024-04-20T11:28:20.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.954+0000 7f1d9ce00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d98101d40 0x7f1d98198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.954+0000 7f1d9ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 0x7f1d98199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.954+0000 7f1d9ce00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981998f0 con 0x7f1d98102660 2024-04-20T11:28:20.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.954+0000 7f1d9ce00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9819c560 con 0x7f1d98101d40 2024-04-20T11:28:20.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.954+0000 7f1d95400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 0x7f1d98199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.954+0000 7f1d95400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 0x7f1d98199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53570/0 (socket says 172.21.15.117:53570) 2024-04-20T11:28:20.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.954+0000 7f1d95400700 1 -- 172.21.15.117:0/2745793372 learned_addr learned my addr 172.21.15.117:0/2745793372 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:20.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.954+0000 7f1d95e00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d98101d40 0x7f1d98198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d95400700 1 -- 172.21.15.117:0/2745793372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d98101d40 msgr2=0x7f1d98198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:20.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d95400700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d98101d40 0x7f1d98198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:20.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d95400700 1 -- 172.21.15.117:0/2745793372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d840095f0 con 0x7f1d98102660 2024-04-20T11:28:20.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d95e00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d98101d40 0x7f1d98198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:20.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d95400700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 0x7f1d98199260 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005b40 tx=0x7f1d8400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d8ea00700 1 -- 172.21.15.117:0/2745793372 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d8401d070 con 0x7f1d98102660 2024-04-20T11:28:20.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d8ea00700 1 -- 172.21.15.117:0/2745793372 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d84003d50 con 0x7f1d98102660 2024-04-20T11:28:20.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d8ea00700 1 -- 172.21.15.117:0/2745793372 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d8400fe90 con 0x7f1d98102660 2024-04-20T11:28:20.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9819c7e0 con 0x7f1d98102660 2024-04-20T11:28:20.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.955+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9819ccd0 con 0x7f1d98102660 2024-04-20T11:28:20.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.956+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d981096b0 con 0x7f1d98102660 2024-04-20T11:28:20.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.957+0000 7f1d8ea00700 1 -- 172.21.15.117:0/2745793372 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1d84003890 con 0x7f1d98102660 2024-04-20T11:28:20.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.958+0000 7f1d8ea00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d7c06ecc0 0x7f1d7c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:20.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.958+0000 7f1d8ea00700 1 -- 172.21.15.117:0/2745793372 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1d84090320 con 0x7f1d98102660 2024-04-20T11:28:20.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.959+0000 7f1d95e00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d7c06ecc0 0x7f1d7c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:20.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.959+0000 7f1d95e00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d7c06ecc0 0x7f1d7c071180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005fd0 tx=0x7f1d88005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:20.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:20.963+0000 7f1d8ea00700 1 -- 172.21.15.117:0/2745793372 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1d84059060 con 0x7f1d98102660 2024-04-20T11:28:21.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.261+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1d98068e50 con 0x7f1d98102660 2024-04-20T11:28:21.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.263+0000 7f1d8ea00700 1 -- 172.21.15.117:0/2745793372 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f1d8405c720 con 0x7f1d98102660 2024-04-20T11:28:21.263 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:28:21.263 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:28:21.263 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:28:21.263 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:28:21.263 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:28:21.263 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:28:21.264 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:28:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.265+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d7c06ecc0 msgr2=0x7f1d7c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.265+0000 7f1d9ce00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d7c06ecc0 0x7f1d7c071180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005fd0 tx=0x7f1d88005dc0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 msgr2=0x7f1d98199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 0x7f1d98199260 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005b40 tx=0x7f1d8400bc10 comp rx=0 tx=0).stop 2024-04-20T11:28:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 shutdown_connections 2024-04-20T11:28:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1d7c06ecc0 0x7f1d7c071180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d98101d40 0x7f1d98198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 --2- 172.21.15.117:0/2745793372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98102660 0x7f1d98199260 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 >> 172.21.15.117:0/2745793372 conn(0x7f1d98076750 msgr2=0x7f1d9810a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 shutdown_connections 2024-04-20T11:28:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:21.266+0000 7f1d9ce00700 1 -- 172.21.15.117:0/2745793372 wait complete. 2024-04-20T11:28:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:21 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4258493433' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:28:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:21 smithi117 ceph-mon[99036]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:21 smithi117 ceph-mon[99036]: pgmap v477: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:21 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2745793372' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:28:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:21 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4258493433' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:28:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:21 smithi173 ceph-mon[74165]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:21 smithi173 ceph-mon[74165]: pgmap v477: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:21 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2745793372' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:28:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:23 smithi117 ceph-mon[99036]: pgmap v478: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:23.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:23 smithi173 ceph-mon[74165]: pgmap v478: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:25 smithi117 ceph-mon[99036]: pgmap v479: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:25 smithi173 ceph-mon[74165]: pgmap v479: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:27 smithi117 ceph-mon[99036]: pgmap v480: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:27.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:27 smithi173 ceph-mon[74165]: pgmap v480: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:28:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:28:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:29 smithi117 ceph-mon[99036]: pgmap v481: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:29.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:29 smithi173 ceph-mon[74165]: pgmap v481: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:31.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:31 smithi117 ceph-mon[99036]: pgmap v482: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:31 smithi173 ceph-mon[74165]: pgmap v482: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:33.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:33 smithi117 ceph-mon[99036]: pgmap v483: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:33 smithi173 ceph-mon[74165]: pgmap v483: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:35.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:35 smithi117 ceph-mon[99036]: pgmap v484: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:35 smithi173 ceph-mon[74165]: pgmap v484: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:37.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:37 smithi117 ceph-mon[99036]: pgmap v485: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:37.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:37 smithi173 ceph-mon[74165]: pgmap v485: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:39.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:39 smithi117 ceph-mon[99036]: pgmap v486: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:39.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:39 smithi173 ceph-mon[74165]: pgmap v486: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:41.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:41 smithi117 ceph-mon[99036]: pgmap v487: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:41.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:41 smithi173 ceph-mon[74165]: pgmap v487: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:43.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:28:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:28:44.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:44 smithi117 ceph-mon[99036]: pgmap v488: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:44 smithi173 ceph-mon[74165]: pgmap v488: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:45 smithi117 ceph-mon[99036]: pgmap v489: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:45.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:45 smithi173 ceph-mon[74165]: pgmap v489: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:47.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:47 smithi117 ceph-mon[99036]: pgmap v490: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:47 smithi173 ceph-mon[74165]: pgmap v490: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:49.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:49 smithi117 ceph-mon[99036]: pgmap v491: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:49.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:49 smithi173 ceph-mon[74165]: pgmap v491: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:51.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.442+0000 7f5b25a00700 1 -- 172.21.15.117:0/2196541279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073ba0 msgr2=0x7f5b201114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.442+0000 7f5b25a00700 1 --2- 172.21.15.117:0/2196541279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073ba0 0x7f5b201114a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009910 tx=0x7f5b0c009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:51.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.442+0000 7f5b25a00700 1 -- 172.21.15.117:0/2196541279 shutdown_connections 2024-04-20T11:28:51.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.442+0000 7f5b25a00700 1 --2- 172.21.15.117:0/2196541279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073ba0 0x7f5b201114a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.442+0000 7f5b25a00700 1 --2- 172.21.15.117:0/2196541279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b20073280 0x7f5b20073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.442+0000 7f5b25a00700 1 -- 172.21.15.117:0/2196541279 >> 172.21.15.117:0/2196541279 conn(0x7f5b200fc500 msgr2=0x7f5b200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:51.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.442+0000 7f5b25a00700 1 -- 172.21.15.117:0/2196541279 shutdown_connections 2024-04-20T11:28:51.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.442+0000 7f5b25a00700 1 -- 172.21.15.117:0/2196541279 wait complete. 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.443+0000 7f5b25a00700 1 Processor -- start 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.444+0000 7f5b25a00700 1 -- start start 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.444+0000 7f5b25a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073280 0x7f5b2019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.444+0000 7f5b25a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b20073ba0 0x7f5b2019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.444+0000 7f5b25a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019dae0 con 0x7f5b20073280 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.444+0000 7f5b25a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b201a17b0 con 0x7f5b20073ba0 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.444+0000 7f5b1fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073280 0x7f5b2019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.444+0000 7f5b1fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073280 0x7f5b2019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45364/0 (socket says 172.21.15.117:45364) 2024-04-20T11:28:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.444+0000 7f5b1fe00700 1 -- 172.21.15.117:0/1022891817 learned_addr learned my addr 172.21.15.117:0/1022891817 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.445+0000 7f5b1f400700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b20073ba0 0x7f5b2019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.445+0000 7f5b1fe00700 1 -- 172.21.15.117:0/1022891817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b20073ba0 msgr2=0x7f5b2019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.445+0000 7f5b1fe00700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b20073ba0 0x7f5b2019d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.445+0000 7f5b1fe00700 1 -- 172.21.15.117:0/1022891817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0095f0 con 0x7f5b20073280 2024-04-20T11:28:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.445+0000 7f5b1fe00700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073280 0x7f5b2019cec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000ada0 tx=0x7f5b1000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.445+0000 7f5b1cc00700 1 -- 172.21.15.117:0/1022891817 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b1000f070 con 0x7f5b20073280 2024-04-20T11:28:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.446+0000 7f5b1cc00700 1 -- 172.21.15.117:0/1022891817 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b10007bf0 con 0x7f5b20073280 2024-04-20T11:28:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.446+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b201a1a90 con 0x7f5b20073280 2024-04-20T11:28:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.446+0000 7f5b1cc00700 1 -- 172.21.15.117:0/1022891817 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b10005690 con 0x7f5b20073280 2024-04-20T11:28:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.446+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a1fe0 con 0x7f5b20073280 2024-04-20T11:28:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.447+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b200749c0 con 0x7f5b20073280 2024-04-20T11:28:51.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.448+0000 7f5b1cc00700 1 -- 172.21.15.117:0/1022891817 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5b10005850 con 0x7f5b20073280 2024-04-20T11:28:51.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.449+0000 7f5b1cc00700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b0406ecc0 0x7f5b04071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.449+0000 7f5b1cc00700 1 -- 172.21.15.117:0/1022891817 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5b1008e790 con 0x7f5b20073280 2024-04-20T11:28:51.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.449+0000 7f5b1f400700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b0406ecc0 0x7f5b04071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.450+0000 7f5b1f400700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b0406ecc0 0x7f5b04071180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c006010 tx=0x7f5b0c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.455+0000 7f5b1cc00700 1 -- 172.21.15.117:0/1022891817 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5b100575f0 con 0x7f5b20073280 2024-04-20T11:28:51.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:51 smithi117 ceph-mon[99036]: pgmap v492: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:51.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:51 smithi173 ceph-mon[74165]: pgmap v492: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:51.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.703+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b201142a0 con 0x7f5b0406ecc0 2024-04-20T11:28:51.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.706+0000 7f5b1cc00700 1 -- 172.21.15.117:0/1022891817 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5b201142a0 con 0x7f5b0406ecc0 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b0406ecc0 msgr2=0x7f5b04071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b0406ecc0 0x7f5b04071180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c006010 tx=0x7f5b0c00b540 comp rx=0 tx=0).stop 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073280 msgr2=0x7f5b2019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073280 0x7f5b2019cec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000ada0 tx=0x7f5b1000c330 comp rx=0 tx=0).stop 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 shutdown_connections 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b0406ecc0 0x7f5b04071180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20073280 0x7f5b2019cec0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 --2- 172.21.15.117:0/1022891817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b20073ba0 0x7f5b2019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.709+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 >> 172.21.15.117:0/1022891817 conn(0x7f5b200fc500 msgr2=0x7f5b20100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:51.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.710+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 shutdown_connections 2024-04-20T11:28:51.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.710+0000 7f5b25a00700 1 -- 172.21.15.117:0/1022891817 wait complete. 2024-04-20T11:28:51.726 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:28:51.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.870+0000 7f58e4600700 1 -- 172.21.15.117:0/3811108296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc100e10 msgr2=0x7f58dc1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.870+0000 7f58e4600700 1 --2- 172.21.15.117:0/3811108296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc100e10 0x7f58dc1011f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009910 tx=0x7f58d0009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.870+0000 7f58e4600700 1 -- 172.21.15.117:0/3811108296 shutdown_connections 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.870+0000 7f58e4600700 1 --2- 172.21.15.117:0/3811108296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58dc1017c0 0x7f58dc105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.870+0000 7f58e4600700 1 --2- 172.21.15.117:0/3811108296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc100e10 0x7f58dc1011f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.870+0000 7f58e4600700 1 -- 172.21.15.117:0/3811108296 >> 172.21.15.117:0/3811108296 conn(0x7f58dc0fc6a0 msgr2=0x7f58dc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.870+0000 7f58e4600700 1 -- 172.21.15.117:0/3811108296 shutdown_connections 2024-04-20T11:28:51.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.870+0000 7f58e4600700 1 -- 172.21.15.117:0/3811108296 wait complete. 2024-04-20T11:28:51.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.871+0000 7f58e4600700 1 Processor -- start 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.871+0000 7f58e4600700 1 -- start start 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.871+0000 7f58e4600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58dc100e10 0x7f58dc198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e4600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc1017c0 0x7f58dc199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e4600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc1998d0 con 0x7f58dc1017c0 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e4600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc19c540 con 0x7f58dc100e10 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e1200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc1017c0 0x7f58dc199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e1200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc1017c0 0x7f58dc199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45386/0 (socket says 172.21.15.117:45386) 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e1200700 1 -- 172.21.15.117:0/3452263621 learned_addr learned my addr 172.21.15.117:0/3452263621 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e1200700 1 -- 172.21.15.117:0/3452263621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58dc100e10 msgr2=0x7f58dc198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e1200700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58dc100e10 0x7f58dc198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.872+0000 7f58e1200700 1 -- 172.21.15.117:0/3452263621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d00095f0 con 0x7f58dc1017c0 2024-04-20T11:28:51.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.873+0000 7f58e1200700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc1017c0 0x7f58dc199240 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c8f0 tx=0x7f58c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.873+0000 7f58da000700 1 -- 172.21.15.117:0/3452263621 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58c8007a50 con 0x7f58dc1017c0 2024-04-20T11:28:51.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.873+0000 7f58da000700 1 -- 172.21.15.117:0/3452263621 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58c8007bb0 con 0x7f58dc1017c0 2024-04-20T11:28:51.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.873+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc19c820 con 0x7f58dc1017c0 2024-04-20T11:28:51.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.873+0000 7f58da000700 1 -- 172.21.15.117:0/3452263621 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58c8018630 con 0x7f58dc1017c0 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.873+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc19cd70 con 0x7f58dc1017c0 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.874+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc104f90 con 0x7f58dc1017c0 2024-04-20T11:28:51.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.875+0000 7f58da000700 1 -- 172.21.15.117:0/3452263621 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f58c8007d20 con 0x7f58dc1017c0 2024-04-20T11:28:51.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.876+0000 7f58da000700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58c406ecc0 0x7f58c4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:51.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.876+0000 7f58da000700 1 -- 172.21.15.117:0/3452263621 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f58c808eec0 con 0x7f58dc1017c0 2024-04-20T11:28:51.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.876+0000 7f58e1c00700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58c406ecc0 0x7f58c4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:51.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.877+0000 7f58e1c00700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58c406ecc0 0x7f58c4071180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b5c0 tx=0x7f58d0005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:51.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:51.881+0000 7f58da000700 1 -- 172.21.15.117:0/3452263621 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f58c8057c00 con 0x7f58dc1017c0 2024-04-20T11:28:52.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.121+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58dc10dde0 con 0x7f58c406ecc0 2024-04-20T11:28:52.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.124+0000 7f58da000700 1 -- 172.21.15.117:0/3452263621 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f58dc10dde0 con 0x7f58c406ecc0 2024-04-20T11:28:52.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.127+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58c406ecc0 msgr2=0x7f58c4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.127+0000 7f58e4600700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58c406ecc0 0x7f58c4071180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b5c0 tx=0x7f58d0005e20 comp rx=0 tx=0).stop 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.127+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc1017c0 msgr2=0x7f58dc199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.127+0000 7f58e4600700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc1017c0 0x7f58dc199240 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c8f0 tx=0x7f58c800ccb0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.128+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 shutdown_connections 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.128+0000 7f58e4600700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58c406ecc0 0x7f58c4071180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.128+0000 7f58e4600700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58dc100e10 0x7f58dc198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.128+0000 7f58e4600700 1 --2- 172.21.15.117:0/3452263621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58dc1017c0 0x7f58dc199240 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.128+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 >> 172.21.15.117:0/3452263621 conn(0x7f58dc0fc6a0 msgr2=0x7f58dc0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.128+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 shutdown_connections 2024-04-20T11:28:52.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.128+0000 7f58e4600700 1 -- 172.21.15.117:0/3452263621 wait complete. 2024-04-20T11:28:52.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.284+0000 7f3738c00700 1 -- 172.21.15.117:0/557569221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 msgr2=0x7f3734103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.284+0000 7f3738c00700 1 --2- 172.21.15.117:0/557569221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 0x7f3734103400 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3724009910 tx=0x7f3724009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:52.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.284+0000 7f3738c00700 1 -- 172.21.15.117:0/557569221 shutdown_connections 2024-04-20T11:28:52.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.284+0000 7f3738c00700 1 --2- 172.21.15.117:0/557569221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37341039d0 0x7f3734107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.284+0000 7f3738c00700 1 --2- 172.21.15.117:0/557569221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 0x7f3734103400 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.284+0000 7f3738c00700 1 -- 172.21.15.117:0/557569221 >> 172.21.15.117:0/557569221 conn(0x7f37340fe8b0 msgr2=0x7f3734100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:52.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.284+0000 7f3738c00700 1 -- 172.21.15.117:0/557569221 shutdown_connections 2024-04-20T11:28:52.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.284+0000 7f3738c00700 1 -- 172.21.15.117:0/557569221 wait complete. 2024-04-20T11:28:52.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.285+0000 7f3738c00700 1 Processor -- start 2024-04-20T11:28:52.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.286+0000 7f3738c00700 1 -- start start 2024-04-20T11:28:52.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.286+0000 7f3738c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 0x7f373419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.286+0000 7f3738c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37341039d0 0x7f373419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.286+0000 7f3738c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373419f5c0 con 0x7f3734103020 2024-04-20T11:28:52.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.286+0000 7f3738c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3734198bf0 con 0x7f37341039d0 2024-04-20T11:28:52.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.286+0000 7f3731e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 0x7f373419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.286+0000 7f3731e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 0x7f373419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45396/0 (socket says 172.21.15.117:45396) 2024-04-20T11:28:52.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.286+0000 7f3731e00700 1 -- 172.21.15.117:0/743258595 learned_addr learned my addr 172.21.15.117:0/743258595 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.287+0000 7f3731e00700 1 -- 172.21.15.117:0/743258595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37341039d0 msgr2=0x7f373419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.287+0000 7f3731e00700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37341039d0 0x7f373419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.287+0000 7f3731e00700 1 -- 172.21.15.117:0/743258595 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240095f0 con 0x7f3734103020 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.287+0000 7f3731e00700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 0x7f373419ea60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3724006010 tx=0x7f372400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.287+0000 7f372aa00700 1 -- 172.21.15.117:0/743258595 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f372401d070 con 0x7f3734103020 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.287+0000 7f372aa00700 1 -- 172.21.15.117:0/743258595 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f372400f460 con 0x7f3734103020 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.287+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3734198e70 con 0x7f3734103020 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.287+0000 7f372aa00700 1 -- 172.21.15.117:0/743258595 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3724017630 con 0x7f3734103020 2024-04-20T11:28:52.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.288+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3734199360 con 0x7f3734103020 2024-04-20T11:28:52.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.289+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373410b3c0 con 0x7f3734103020 2024-04-20T11:28:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.290+0000 7f372aa00700 1 -- 172.21.15.117:0/743258595 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f372400f5d0 con 0x7f3734103020 2024-04-20T11:28:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.291+0000 7f372aa00700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f371806ecd0 0x7f3718071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.291+0000 7f372aa00700 1 -- 172.21.15.117:0/743258595 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f37240903c0 con 0x7f3734103020 2024-04-20T11:28:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.292+0000 7f3731400700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f371806ecd0 0x7f3718071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.292+0000 7f3731400700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f371806ecd0 0x7f3718071190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f373419a240 tx=0x7f3720005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.296+0000 7f372aa00700 1 -- 172.21.15.117:0/743258595 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f372405ccf0 con 0x7f3734103020 2024-04-20T11:28:52.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:52 smithi117 ceph-mon[99036]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:52.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:52 smithi117 ceph-mon[99036]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:52.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.534+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f373410fff0 con 0x7f371806ecd0 2024-04-20T11:28:52.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.552+0000 7f372aa00700 1 -- 172.21.15.117:0/743258595 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f373410fff0 con 0x7f371806ecd0 2024-04-20T11:28:52.552 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:28:52.552 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (17m) 8m ago 31m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:28:52.552 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (31m) 8m ago 31m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:28:52.552 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (30m) 8m ago 30m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:28:52.552 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (13m) 8m ago 31m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:28:52.552 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (13m) 8m ago 30m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:28:52.552 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (16m) 8m ago 31m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (9m) 8m ago 24m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (9m) 8m ago 24m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (24m) 8m ago 24m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (8m) 8m ago 24m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (20m) 8m ago 33m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (18m) 8m ago 30m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (15m) 8m ago 33m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (14m) 8m ago 29m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (18m) 8m ago 31m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (18m) 8m ago 30m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (12m) 8m ago 29m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:28:52.553 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (12m) 8m ago 28m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:28:52.554 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (11m) 8m ago 27m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:28:52.554 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (11m) 8m ago 27m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:28:52.554 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (10m) 8m ago 26m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:28:52.554 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (9m) 8m ago 26m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:28:52.554 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (17m) 8m ago 30m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:28:52.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f371806ecd0 msgr2=0x7f3718071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f371806ecd0 0x7f3718071190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f373419a240 tx=0x7f3720005c80 comp rx=0 tx=0).stop 2024-04-20T11:28:52.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 msgr2=0x7f373419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 0x7f373419ea60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3724006010 tx=0x7f372400be20 comp rx=0 tx=0).stop 2024-04-20T11:28:52.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 shutdown_connections 2024-04-20T11:28:52.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f371806ecd0 0x7f3718071190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3734103020 0x7f373419ea60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 --2- 172.21.15.117:0/743258595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37341039d0 0x7f373419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.556+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 >> 172.21.15.117:0/743258595 conn(0x7f37340fe8b0 msgr2=0x7f37340fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:52.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.557+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 shutdown_connections 2024-04-20T11:28:52.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.557+0000 7f3738c00700 1 -- 172.21.15.117:0/743258595 wait complete. 2024-04-20T11:28:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:52 smithi173 ceph-mon[74165]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:52 smithi173 ceph-mon[74165]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:52.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.707+0000 7f22af200700 1 -- 172.21.15.117:0/479882151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 msgr2=0x7f22a8105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.707+0000 7f22af200700 1 --2- 172.21.15.117:0/479882151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 0x7f22a8105810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2294009910 tx=0x7f2294009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:52.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.707+0000 7f22af200700 1 -- 172.21.15.117:0/479882151 shutdown_connections 2024-04-20T11:28:52.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.707+0000 7f22af200700 1 --2- 172.21.15.117:0/479882151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 0x7f22a8105810 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.707+0000 7f22af200700 1 --2- 172.21.15.117:0/479882151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a8100e10 0x7f22a81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.707+0000 7f22af200700 1 -- 172.21.15.117:0/479882151 >> 172.21.15.117:0/479882151 conn(0x7f22a80fc6c0 msgr2=0x7f22a80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:52.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.707+0000 7f22af200700 1 -- 172.21.15.117:0/479882151 shutdown_connections 2024-04-20T11:28:52.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.707+0000 7f22af200700 1 -- 172.21.15.117:0/479882151 wait complete. 2024-04-20T11:28:52.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.708+0000 7f22af200700 1 Processor -- start 2024-04-20T11:28:52.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.709+0000 7f22af200700 1 -- start start 2024-04-20T11:28:52.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.709+0000 7f22af200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a8100e10 0x7f22a819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.709+0000 7f22af200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 0x7f22a819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.709+0000 7f22af200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a819f5a0 con 0x7f22a81017c0 2024-04-20T11:28:52.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.709+0000 7f22af200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a8198b20 con 0x7f22a8100e10 2024-04-20T11:28:52.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.709+0000 7f22a7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 0x7f22a819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.709+0000 7f22a7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 0x7f22a819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45416/0 (socket says 172.21.15.117:45416) 2024-04-20T11:28:52.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.709+0000 7f22a7400700 1 -- 172.21.15.117:0/3792343095 learned_addr learned my addr 172.21.15.117:0/3792343095 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:52.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22a7e00700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a8100e10 0x7f22a819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22a7400700 1 -- 172.21.15.117:0/3792343095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a8100e10 msgr2=0x7f22a819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:52.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22a7400700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a8100e10 0x7f22a819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:52.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22a7400700 1 -- 172.21.15.117:0/3792343095 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940095f0 con 0x7f22a81017c0 2024-04-20T11:28:52.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22a7400700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 0x7f22a819eec0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f229400bd70 tx=0x7f229400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:52.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22a4c00700 1 -- 172.21.15.117:0/3792343095 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f229401d070 con 0x7f22a81017c0 2024-04-20T11:28:52.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22a4c00700 1 -- 172.21.15.117:0/3792343095 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f229400f460 con 0x7f22a81017c0 2024-04-20T11:28:52.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a8198da0 con 0x7f22a81017c0 2024-04-20T11:28:52.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.710+0000 7f22a4c00700 1 -- 172.21.15.117:0/3792343095 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2294017630 con 0x7f22a81017c0 2024-04-20T11:28:52.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.711+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a8199290 con 0x7f22a81017c0 2024-04-20T11:28:52.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.712+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a8103f80 con 0x7f22a81017c0 2024-04-20T11:28:52.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.713+0000 7f22a4c00700 1 -- 172.21.15.117:0/3792343095 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f229400f5d0 con 0x7f22a81017c0 2024-04-20T11:28:52.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.713+0000 7f22a4c00700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f228c06ecc0 0x7f228c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:52.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.714+0000 7f22a4c00700 1 -- 172.21.15.117:0/3792343095 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2294090280 con 0x7f22a81017c0 2024-04-20T11:28:52.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.714+0000 7f22a7e00700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f228c06ecc0 0x7f228c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:52.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.714+0000 7f22a7e00700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f228c06ecc0 0x7f228c071180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f22980060b0 tx=0x7f2298008640 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:52.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:52.718+0000 7f22a4c00700 1 -- 172.21.15.117:0/3792343095 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f229405cb90 con 0x7f22a81017c0 2024-04-20T11:28:53.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.028+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f22a81022f0 con 0x7f22a81017c0 2024-04-20T11:28:53.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.029+0000 7f22a4c00700 1 -- 172.21.15.117:0/3792343095 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f2294005740 con 0x7f22a81017c0 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:28:53.029 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:28:53.030 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:28:53.030 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:28:53.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:28:53.030 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:28:53.030 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:28:53.030 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:28:53.030 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:28:53.030 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:28:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f228c06ecc0 msgr2=0x7f228c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f228c06ecc0 0x7f228c071180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f22980060b0 tx=0x7f2298008640 comp rx=0 tx=0).stop 2024-04-20T11:28:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 msgr2=0x7f22a819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 0x7f22a819eec0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f229400bd70 tx=0x7f229400be50 comp rx=0 tx=0).stop 2024-04-20T11:28:53.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 shutdown_connections 2024-04-20T11:28:53.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f228c06ecc0 0x7f228c071180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a8100e10 0x7f22a819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 --2- 172.21.15.117:0/3792343095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22a81017c0 0x7f22a819eec0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 >> 172.21.15.117:0/3792343095 conn(0x7f22a80fc6c0 msgr2=0x7f22a80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:53.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 shutdown_connections 2024-04-20T11:28:53.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.032+0000 7f22af200700 1 -- 172.21.15.117:0/3792343095 wait complete. 2024-04-20T11:28:53.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.183+0000 7fb569a00700 1 -- 172.21.15.117:0/1762420116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb564107ba0 msgr2=0x7fb56410bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.183+0000 7fb569a00700 1 --2- 172.21.15.117:0/1762420116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb564107ba0 0x7fb56410bbf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009910 tx=0x7fb54c009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:53.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.183+0000 7fb569a00700 1 -- 172.21.15.117:0/1762420116 shutdown_connections 2024-04-20T11:28:53.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.183+0000 7fb569a00700 1 --2- 172.21.15.117:0/1762420116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb564107ba0 0x7fb56410bbf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.183+0000 7fb569a00700 1 --2- 172.21.15.117:0/1762420116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5641071f0 0x7fb5641075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.183+0000 7fb569a00700 1 -- 172.21.15.117:0/1762420116 >> 172.21.15.117:0/1762420116 conn(0x7fb5640767d0 msgr2=0x7fb564078bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:53.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.183+0000 7fb569a00700 1 -- 172.21.15.117:0/1762420116 shutdown_connections 2024-04-20T11:28:53.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.183+0000 7fb569a00700 1 -- 172.21.15.117:0/1762420116 wait complete. 2024-04-20T11:28:53.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.184+0000 7fb569a00700 1 Processor -- start 2024-04-20T11:28:53.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.184+0000 7fb569a00700 1 -- start start 2024-04-20T11:28:53.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.184+0000 7fb569a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5641071f0 0x7fb56419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.184+0000 7fb569a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb564107ba0 0x7fb56419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.184+0000 7fb569a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56419db80 con 0x7fb5641071f0 2024-04-20T11:28:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.184+0000 7fb569a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5641a1850 con 0x7fb564107ba0 2024-04-20T11:28:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.184+0000 7fb563e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5641071f0 0x7fb56419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.185+0000 7fb563e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5641071f0 0x7fb56419cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45428/0 (socket says 172.21.15.117:45428) 2024-04-20T11:28:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.185+0000 7fb563e00700 1 -- 172.21.15.117:0/604579562 learned_addr learned my addr 172.21.15.117:0/604579562 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.185+0000 7fb563e00700 1 -- 172.21.15.117:0/604579562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb564107ba0 msgr2=0x7fb56419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.185+0000 7fb563e00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb564107ba0 0x7fb56419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.185+0000 7fb563e00700 1 -- 172.21.15.117:0/604579562 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c0095f0 con 0x7fb5641071f0 2024-04-20T11:28:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.185+0000 7fb563e00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5641071f0 0x7fb56419cf60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d8d0 tx=0x7fb55400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.185+0000 7fb561600700 1 -- 172.21.15.117:0/604579562 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb554009940 con 0x7fb5641071f0 2024-04-20T11:28:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.185+0000 7fb561600700 1 -- 172.21.15.117:0/604579562 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb554010460 con 0x7fb5641071f0 2024-04-20T11:28:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.186+0000 7fb561600700 1 -- 172.21.15.117:0/604579562 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb55400f5f0 con 0x7fb5641071f0 2024-04-20T11:28:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.186+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5641a1b30 con 0x7fb5641071f0 2024-04-20T11:28:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.186+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5641a2080 con 0x7fb5641071f0 2024-04-20T11:28:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.187+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56410f5f0 con 0x7fb5641071f0 2024-04-20T11:28:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.188+0000 7fb561600700 1 -- 172.21.15.117:0/604579562 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb5540105d0 con 0x7fb5641071f0 2024-04-20T11:28:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.189+0000 7fb561600700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb54806ecd0 0x7fb548071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.189+0000 7fb561600700 1 -- 172.21.15.117:0/604579562 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb55408edf0 con 0x7fb5641071f0 2024-04-20T11:28:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.189+0000 7fb55be00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb54806ecd0 0x7fb548071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.190+0000 7fb55be00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb54806ecd0 0x7fb548071190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006010 tx=0x7fb54c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:53.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.193+0000 7fb561600700 1 -- 172.21.15.117:0/604579562 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb554057bb0 con 0x7fb5641071f0 2024-04-20T11:28:53.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.468+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb56419e310 con 0x7fb5641071f0 2024-04-20T11:28:53.468 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:53 smithi117 ceph-mon[99036]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:53.468 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:53 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3792343095' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:28:53.468 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:53 smithi117 ceph-mon[99036]: pgmap v493: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:53.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.469+0000 7fb561600700 1 -- 172.21.15.117:0/604579562 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fb554016770 con 0x7fb5641071f0 2024-04-20T11:28:53.469 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:28:53.469 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:28:53.469 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:28:53.469 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:28:53.470 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:28:53.471 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:28:53.472 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:28:53.472 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:28:53.472 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:28:53.472 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:28:53.472 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:28:53.472 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:28:53.472 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:28:53.472 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb54806ecd0 msgr2=0x7fb548071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb54806ecd0 0x7fb548071190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006010 tx=0x7fb54c00b540 comp rx=0 tx=0).stop 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5641071f0 msgr2=0x7fb56419cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5641071f0 0x7fb56419cf60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d8d0 tx=0x7fb55400dc90 comp rx=0 tx=0).stop 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 shutdown_connections 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb54806ecd0 0x7fb548071190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5641071f0 0x7fb56419cf60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 --2- 172.21.15.117:0/604579562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb564107ba0 0x7fb56419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.472+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 >> 172.21.15.117:0/604579562 conn(0x7fb5640767d0 msgr2=0x7fb564076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.473+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 shutdown_connections 2024-04-20T11:28:53.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.473+0000 7fb569a00700 1 -- 172.21.15.117:0/604579562 wait complete. 2024-04-20T11:28:53.474 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:28:53.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.629+0000 7f088d800700 1 -- 172.21.15.117:0/98490335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 msgr2=0x7f0888109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.629+0000 7f088d800700 1 --2- 172.21.15.117:0/98490335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 0x7f0888109290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0870009910 tx=0x7f0870009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:53.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.629+0000 7f088d800700 1 -- 172.21.15.117:0/98490335 shutdown_connections 2024-04-20T11:28:53.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.629+0000 7f088d800700 1 --2- 172.21.15.117:0/98490335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 0x7f0888109290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.629+0000 7f088d800700 1 --2- 172.21.15.117:0/98490335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0888107520 0x7f0888107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.629+0000 7f088d800700 1 -- 172.21.15.117:0/98490335 >> 172.21.15.117:0/98490335 conn(0x7f08880768f0 msgr2=0x7f0888078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:53.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.629+0000 7f088d800700 1 -- 172.21.15.117:0/98490335 shutdown_connections 2024-04-20T11:28:53.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.629+0000 7f088d800700 1 -- 172.21.15.117:0/98490335 wait complete. 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.630+0000 7f088d800700 1 Processor -- start 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.630+0000 7f088d800700 1 -- start start 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.631+0000 7f088d800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0888107520 0x7f088819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.631+0000 7f088d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 0x7f088819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.631+0000 7f088d800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088819ddf0 con 0x7f0888108e10 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.631+0000 7f088d800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08881a1a70 con 0x7f0888107520 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.631+0000 7f0886000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 0x7f088819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.631+0000 7f0886000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 0x7f088819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45444/0 (socket says 172.21.15.117:45444) 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.631+0000 7f0886000700 1 -- 172.21.15.117:0/3291545146 learned_addr learned my addr 172.21.15.117:0/3291545146 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f0886000700 1 -- 172.21.15.117:0/3291545146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0888107520 msgr2=0x7f088819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.631+0000 7f0886a00700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0888107520 0x7f088819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f0886000700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0888107520 0x7f088819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f0886000700 1 -- 172.21.15.117:0/3291545146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08700095f0 con 0x7f0888108e10 2024-04-20T11:28:53.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f0886000700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 0x7f088819d710 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f087000bd70 tx=0x7f087000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:53.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f087f400700 1 -- 172.21.15.117:0/3291545146 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087001d070 con 0x7f0888108e10 2024-04-20T11:28:53.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f087f400700 1 -- 172.21.15.117:0/3291545146 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f087000f460 con 0x7f0888108e10 2024-04-20T11:28:53.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f087f400700 1 -- 172.21.15.117:0/3291545146 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0870017630 con 0x7f0888108e10 2024-04-20T11:28:53.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08881a1cf0 con 0x7f0888108e10 2024-04-20T11:28:53.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.632+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08881a2260 con 0x7f0888108e10 2024-04-20T11:28:53.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.633+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088810f870 con 0x7f0888108e10 2024-04-20T11:28:53.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.634+0000 7f087f400700 1 -- 172.21.15.117:0/3291545146 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f087000f5d0 con 0x7f0888108e10 2024-04-20T11:28:53.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.635+0000 7f087f400700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f086c06ecc0 0x7f086c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:53.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.635+0000 7f087f400700 1 -- 172.21.15.117:0/3291545146 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f08700902c0 con 0x7f0888108e10 2024-04-20T11:28:53.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.636+0000 7f0886a00700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f086c06ecc0 0x7f086c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:53.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.636+0000 7f0886a00700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f086c06ecc0 0x7f086c071180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0878005950 tx=0x7f0878005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:53.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.641+0000 7f087f400700 1 -- 172.21.15.117:0/3291545146 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0870059120 con 0x7f0888108e10 2024-04-20T11:28:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:53 smithi173 ceph-mon[74165]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:53 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3792343095' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:28:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:53 smithi173 ceph-mon[74165]: pgmap v493: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:53.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.888+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f088819e500 con 0x7f086c06ecc0 2024-04-20T11:28:53.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.892+0000 7f087f400700 1 -- 172.21.15.117:0/3291545146 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f088819e500 con 0x7f086c06ecc0 2024-04-20T11:28:53.891 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:28:53.891 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:28:53.891 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:28:53.892 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:28:53.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.894+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f086c06ecc0 msgr2=0x7f086c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.894+0000 7f088d800700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f086c06ecc0 0x7f086c071180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0878005950 tx=0x7f0878005cf0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.894+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 msgr2=0x7f088819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:53.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.894+0000 7f088d800700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 0x7f088819d710 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f087000bd70 tx=0x7f087000be50 comp rx=0 tx=0).stop 2024-04-20T11:28:53.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.895+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 shutdown_connections 2024-04-20T11:28:53.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.895+0000 7f088d800700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f086c06ecc0 0x7f086c071180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.895+0000 7f088d800700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0888107520 0x7f088819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.895+0000 7f088d800700 1 --2- 172.21.15.117:0/3291545146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0888108e10 0x7f088819d710 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:53.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.895+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 >> 172.21.15.117:0/3291545146 conn(0x7f08880768f0 msgr2=0x7f0888077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:53.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.895+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 shutdown_connections 2024-04-20T11:28:53.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:53.895+0000 7f088d800700 1 -- 172.21.15.117:0/3291545146 wait complete. 2024-04-20T11:28:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.055+0000 7f595c600700 1 -- 172.21.15.117:0/2289128786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 msgr2=0x7f5954105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.055+0000 7f595c600700 1 --2- 172.21.15.117:0/2289128786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 0x7f5954105820 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5940009910 tx=0x7f5940009c20 comp rx=0 tx=0).stop 2024-04-20T11:28:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.055+0000 7f595c600700 1 -- 172.21.15.117:0/2289128786 shutdown_connections 2024-04-20T11:28:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.055+0000 7f595c600700 1 --2- 172.21.15.117:0/2289128786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 0x7f5954105820 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.055+0000 7f595c600700 1 --2- 172.21.15.117:0/2289128786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954100e20 0x7f5954101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.055+0000 7f595c600700 1 -- 172.21.15.117:0/2289128786 >> 172.21.15.117:0/2289128786 conn(0x7f59540fc6d0 msgr2=0x7f59540feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.055+0000 7f595c600700 1 -- 172.21.15.117:0/2289128786 shutdown_connections 2024-04-20T11:28:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.055+0000 7f595c600700 1 -- 172.21.15.117:0/2289128786 wait complete. 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.056+0000 7f595c600700 1 Processor -- start 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.056+0000 7f595c600700 1 -- start start 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.057+0000 7f595c600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954100e20 0x7f595419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.057+0000 7f595c600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 0x7f595419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.057+0000 7f595c600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595419d380 con 0x7f59541017d0 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.057+0000 7f595c600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5954196960 con 0x7f5954100e20 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.057+0000 7f5959200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 0x7f595419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.057+0000 7f5959200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 0x7f595419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59474/0 (socket says 172.21.15.117:59474) 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.057+0000 7f5959200700 1 -- 172.21.15.117:0/2964052765 learned_addr learned my addr 172.21.15.117:0/2964052765 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.057+0000 7f5959c00700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954100e20 0x7f595419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f5959200700 1 -- 172.21.15.117:0/2964052765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954100e20 msgr2=0x7f595419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f5959200700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954100e20 0x7f595419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f5959200700 1 -- 172.21.15.117:0/2964052765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400095f0 con 0x7f59541017d0 2024-04-20T11:28:54.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f5959c00700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954100e20 0x7f595419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:54.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f5959200700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 0x7f595419cd60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f59400092e0 tx=0x7f594000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:54.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f5952000700 1 -- 172.21.15.117:0/2964052765 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594001d070 con 0x7f59541017d0 2024-04-20T11:28:54.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f5952000700 1 -- 172.21.15.117:0/2964052765 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f594000f460 con 0x7f59541017d0 2024-04-20T11:28:54.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5954196be0 con 0x7f59541017d0 2024-04-20T11:28:54.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f5952000700 1 -- 172.21.15.117:0/2964052765 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5940017630 con 0x7f59541017d0 2024-04-20T11:28:54.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.058+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59541970d0 con 0x7f59541017d0 2024-04-20T11:28:54.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.059+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5954109220 con 0x7f59541017d0 2024-04-20T11:28:54.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.061+0000 7f5952000700 1 -- 172.21.15.117:0/2964052765 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f594000f5d0 con 0x7f59541017d0 2024-04-20T11:28:54.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.061+0000 7f5952000700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f593c06ecc0 0x7f593c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:54.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.062+0000 7f5952000700 1 -- 172.21.15.117:0/2964052765 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5940090390 con 0x7f59541017d0 2024-04-20T11:28:54.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.062+0000 7f5959c00700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f593c06ecc0 0x7f593c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:54.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.063+0000 7f5959c00700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f593c06ecc0 0x7f593c071180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5948005fd0 tx=0x7f5948005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:54.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.066+0000 7f5952000700 1 -- 172.21.15.117:0/2964052765 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f594005cc00 con 0x7f59541017d0 2024-04-20T11:28:54.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.382+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5954102300 con 0x7f59541017d0 2024-04-20T11:28:54.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.383+0000 7f5952000700 1 -- 172.21.15.117:0/2964052765 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5940005740 con 0x7f59541017d0 2024-04-20T11:28:54.383 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:28:54.383 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:28:54.383 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:28:54.383 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:28:54.383 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:28:54.383 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:28:54.383 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:28:54.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.385+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f593c06ecc0 msgr2=0x7f593c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.385+0000 7f595c600700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f593c06ecc0 0x7f593c071180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5948005fd0 tx=0x7f5948005dc0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.385+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 msgr2=0x7f595419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:54.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.385+0000 7f595c600700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 0x7f595419cd60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f59400092e0 tx=0x7f594000be20 comp rx=0 tx=0).stop 2024-04-20T11:28:54.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.386+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 shutdown_connections 2024-04-20T11:28:54.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.386+0000 7f595c600700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f593c06ecc0 0x7f593c071180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.386+0000 7f595c600700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954100e20 0x7f595419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.386+0000 7f595c600700 1 --2- 172.21.15.117:0/2964052765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59541017d0 0x7f595419cd60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:54.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.386+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 >> 172.21.15.117:0/2964052765 conn(0x7f59540fc6d0 msgr2=0x7f59540fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:54.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.386+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 shutdown_connections 2024-04-20T11:28:54.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:28:54.386+0000 7f595c600700 1 -- 172.21.15.117:0/2964052765 wait complete. 2024-04-20T11:28:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:54 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/604579562' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:28:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:54 smithi173 ceph-mon[74165]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:54 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/604579562' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:28:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:54 smithi117 ceph-mon[99036]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:28:55.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:55 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2964052765' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:28:55.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:55 smithi117 ceph-mon[99036]: pgmap v494: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:55 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2964052765' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:28:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:55 smithi173 ceph-mon[74165]: pgmap v494: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:57 smithi117 ceph-mon[99036]: pgmap v495: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:57 smithi173 ceph-mon[74165]: pgmap v495: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:28:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:28:59.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:28:59 smithi117 ceph-mon[99036]: pgmap v496: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:28:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:28:59 smithi173 ceph-mon[74165]: pgmap v496: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:01.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:01 smithi117 ceph-mon[99036]: pgmap v497: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:01 smithi173 ceph-mon[74165]: pgmap v497: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:03.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:03 smithi117 ceph-mon[99036]: pgmap v498: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:03 smithi173 ceph-mon[74165]: pgmap v498: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:05 smithi117 ceph-mon[99036]: pgmap v499: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:05.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:05 smithi173 ceph-mon[74165]: pgmap v499: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:07.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:07 smithi117 ceph-mon[99036]: pgmap v500: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:07 smithi173 ceph-mon[74165]: pgmap v500: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:09.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:09 smithi117 ceph-mon[99036]: pgmap v501: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:09.644 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:09 smithi173 ceph-mon[74165]: pgmap v501: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:11.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:11 smithi117 ceph-mon[99036]: pgmap v502: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:11 smithi173 ceph-mon[74165]: pgmap v502: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:29:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:29:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:14 smithi117 ceph-mon[99036]: pgmap v503: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:14 smithi173 ceph-mon[74165]: pgmap v503: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:15 smithi117 ceph-mon[99036]: pgmap v504: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:15 smithi173 ceph-mon[74165]: pgmap v504: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:17.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:17 smithi117 ceph-mon[99036]: pgmap v505: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:17 smithi173 ceph-mon[74165]: pgmap v505: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:19.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:19 smithi117 ceph-mon[99036]: pgmap v506: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:19.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:19 smithi173 ceph-mon[74165]: pgmap v506: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:21.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:21 smithi117 ceph-mon[99036]: pgmap v507: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:21.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:21 smithi173 ceph-mon[74165]: pgmap v507: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:23 smithi117 ceph-mon[99036]: pgmap v508: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:23.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:23 smithi173 ceph-mon[74165]: pgmap v508: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.556+0000 7f443e200700 1 -- 172.21.15.117:0/1639701849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381039f0 msgr2=0x7f4438107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.556+0000 7f443e200700 1 --2- 172.21.15.117:0/1639701849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381039f0 0x7f4438107a40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4424009910 tx=0x7f4424009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.556+0000 7f443e200700 1 -- 172.21.15.117:0/1639701849 shutdown_connections 2024-04-20T11:29:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.556+0000 7f443e200700 1 --2- 172.21.15.117:0/1639701849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381039f0 0x7f4438107a40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.556+0000 7f443e200700 1 --2- 172.21.15.117:0/1639701849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4438103040 0x7f4438103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.556+0000 7f443e200700 1 -- 172.21.15.117:0/1639701849 >> 172.21.15.117:0/1639701849 conn(0x7f44380fe8d0 msgr2=0x7f4438100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.556+0000 7f443e200700 1 -- 172.21.15.117:0/1639701849 shutdown_connections 2024-04-20T11:29:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.556+0000 7f443e200700 1 -- 172.21.15.117:0/1639701849 wait complete. 2024-04-20T11:29:24.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.557+0000 7f443e200700 1 Processor -- start 2024-04-20T11:29:24.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.557+0000 7f443e200700 1 -- start start 2024-04-20T11:29:24.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.558+0000 7f443e200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438103040 0x7f4438198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.558+0000 7f443e200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44381039f0 0x7f4438199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.558+0000 7f443e200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438199a10 con 0x7f4438103040 2024-04-20T11:29:24.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.558+0000 7f443e200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819d6e0 con 0x7f44381039f0 2024-04-20T11:29:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.558+0000 7f4437400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438103040 0x7f4438198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.558+0000 7f4437400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438103040 0x7f4438198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57986/0 (socket says 172.21.15.117:57986) 2024-04-20T11:29:24.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.558+0000 7f4437400700 1 -- 172.21.15.117:0/2398715238 learned_addr learned my addr 172.21.15.117:0/2398715238 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.558+0000 7f4436a00700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44381039f0 0x7f4438199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f4437400700 1 -- 172.21.15.117:0/2398715238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44381039f0 msgr2=0x7f4438199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f4437400700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44381039f0 0x7f4438199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f4437400700 1 -- 172.21.15.117:0/2398715238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240095f0 con 0x7f4438103040 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f4437400700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438103040 0x7f4438198df0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f442800b4b0 tx=0x7f442800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f442fe00700 1 -- 172.21.15.117:0/2398715238 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4428011800 con 0x7f4438103040 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f442fe00700 1 -- 172.21.15.117:0/2398715238 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4428011e20 con 0x7f4438103040 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443819d9c0 con 0x7f4438103040 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f442fe00700 1 -- 172.21.15.117:0/2398715238 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f442800f2e0 con 0x7f4438103040 2024-04-20T11:29:24.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.559+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443819df10 con 0x7f4438103040 2024-04-20T11:29:24.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.560+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4438104520 con 0x7f4438103040 2024-04-20T11:29:24.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.561+0000 7f442fe00700 1 -- 172.21.15.117:0/2398715238 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4428010380 con 0x7f4438103040 2024-04-20T11:29:24.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.562+0000 7f442fe00700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecd0 0x7f441c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.562+0000 7f442fe00700 1 -- 172.21.15.117:0/2398715238 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f442808ea00 con 0x7f4438103040 2024-04-20T11:29:24.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.563+0000 7f4436a00700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecd0 0x7f441c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.564+0000 7f4436a00700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecd0 0x7f441c071190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f442400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:24.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.568+0000 7f442fe00700 1 -- 172.21.15.117:0/2398715238 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4428057870 con 0x7f4438103040 2024-04-20T11:29:24.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.813+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4438110140 con 0x7f441c06ecd0 2024-04-20T11:29:24.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.817+0000 7f442fe00700 1 -- 172.21.15.117:0/2398715238 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4438110140 con 0x7f441c06ecd0 2024-04-20T11:29:24.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.820+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecd0 msgr2=0x7f441c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.820+0000 7f443e200700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecd0 0x7f441c071190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f442400b540 comp rx=0 tx=0).stop 2024-04-20T11:29:24.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.820+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438103040 msgr2=0x7f4438198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.820+0000 7f443e200700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438103040 0x7f4438198df0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f442800b4b0 tx=0x7f442800b870 comp rx=0 tx=0).stop 2024-04-20T11:29:24.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.820+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 shutdown_connections 2024-04-20T11:29:24.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.820+0000 7f443e200700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecd0 0x7f441c071190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.821+0000 7f443e200700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438103040 0x7f4438198df0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.821+0000 7f443e200700 1 --2- 172.21.15.117:0/2398715238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44381039f0 0x7f4438199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.821+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 >> 172.21.15.117:0/2398715238 conn(0x7f44380fe8d0 msgr2=0x7f4438106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:24.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.821+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 shutdown_connections 2024-04-20T11:29:24.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.821+0000 7f443e200700 1 -- 172.21.15.117:0/2398715238 wait complete. 2024-04-20T11:29:24.837 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.984+0000 7f0dc8a00700 1 -- 172.21.15.117:0/3480604936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 msgr2=0x7f0dc410cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.984+0000 7f0dc8a00700 1 --2- 172.21.15.117:0/3480604936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 0x7f0dc410cec0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009910 tx=0x7f0db0009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.984+0000 7f0dc8a00700 1 -- 172.21.15.117:0/3480604936 shutdown_connections 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.984+0000 7f0dc8a00700 1 --2- 172.21.15.117:0/3480604936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 0x7f0dc410cec0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.984+0000 7f0dc8a00700 1 --2- 172.21.15.117:0/3480604936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dc4073310 0x7f0dc40736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.984+0000 7f0dc8a00700 1 -- 172.21.15.117:0/3480604936 >> 172.21.15.117:0/3480604936 conn(0x7f0dc40fc6f0 msgr2=0x7f0dc40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.985+0000 7f0dc8a00700 1 -- 172.21.15.117:0/3480604936 shutdown_connections 2024-04-20T11:29:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.985+0000 7f0dc8a00700 1 -- 172.21.15.117:0/3480604936 wait complete. 2024-04-20T11:29:24.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.986+0000 7f0dc8a00700 1 Processor -- start 2024-04-20T11:29:24.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.986+0000 7f0dc8a00700 1 -- start start 2024-04-20T11:29:24.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.986+0000 7f0dc8a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dc4073310 0x7f0dc4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.986+0000 7f0dc8a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 0x7f0dc4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.986+0000 7f0dc8a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc41998c0 con 0x7f0dc4073c30 2024-04-20T11:29:24.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.986+0000 7f0dc8a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc419c530 con 0x7f0dc4073310 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.986+0000 7f0dc1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 0x7f0dc4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.987+0000 7f0dc1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 0x7f0dc4199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58002/0 (socket says 172.21.15.117:58002) 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.987+0000 7f0dc1400700 1 -- 172.21.15.117:0/451713701 learned_addr learned my addr 172.21.15.117:0/451713701 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.987+0000 7f0dc1e00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dc4073310 0x7f0dc4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.987+0000 7f0dc1400700 1 -- 172.21.15.117:0/451713701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dc4073310 msgr2=0x7f0dc4198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.987+0000 7f0dc1400700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dc4073310 0x7f0dc4198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.987+0000 7f0dc1400700 1 -- 172.21.15.117:0/451713701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db00095f0 con 0x7f0dc4073c30 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.987+0000 7f0dc1400700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 0x7f0dc4199230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0db000bd70 tx=0x7f0db000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.988+0000 7f0dbaa00700 1 -- 172.21.15.117:0/451713701 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db001d070 con 0x7f0dc4073c30 2024-04-20T11:29:24.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.988+0000 7f0dbaa00700 1 -- 172.21.15.117:0/451713701 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0db00056e0 con 0x7f0dc4073c30 2024-04-20T11:29:24.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.988+0000 7f0dbaa00700 1 -- 172.21.15.117:0/451713701 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db0017400 con 0x7f0dc4073c30 2024-04-20T11:29:24.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.988+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc419c7b0 con 0x7f0dc4073c30 2024-04-20T11:29:24.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.988+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc419cca0 con 0x7f0dc4073c30 2024-04-20T11:29:24.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.989+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc40749c0 con 0x7f0dc4073c30 2024-04-20T11:29:24.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.990+0000 7f0dbaa00700 1 -- 172.21.15.117:0/451713701 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0db0003a60 con 0x7f0dc4073c30 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.991+0000 7f0dbaa00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0da806ecc0 0x7f0da8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.991+0000 7f0dbaa00700 1 -- 172.21.15.117:0/451713701 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0db0090280 con 0x7f0dc4073c30 2024-04-20T11:29:24.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.991+0000 7f0dc1e00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0da806ecc0 0x7f0da8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:24.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.992+0000 7f0dc1e00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0da806ecc0 0x7f0da8071180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0db4005950 tx=0x7f0db40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:24.996+0000 7f0dbaa00700 1 -- 172.21.15.117:0/451713701 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0db0059040 con 0x7f0dc4073c30 2024-04-20T11:29:25.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.244+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dc4107760 con 0x7f0da806ecc0 2024-04-20T11:29:25.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.246+0000 7f0dbaa00700 1 -- 172.21.15.117:0/451713701 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0dc4107760 con 0x7f0da806ecc0 2024-04-20T11:29:25.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0da806ecc0 msgr2=0x7f0da8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0da806ecc0 0x7f0da8071180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0db4005950 tx=0x7f0db40058e0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 msgr2=0x7f0dc4199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 0x7f0dc4199230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0db000bd70 tx=0x7f0db000be50 comp rx=0 tx=0).stop 2024-04-20T11:29:25.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 shutdown_connections 2024-04-20T11:29:25.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0da806ecc0 0x7f0da8071180 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dc4073310 0x7f0dc4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 --2- 172.21.15.117:0/451713701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc4073c30 0x7f0dc4199230 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.249+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 >> 172.21.15.117:0/451713701 conn(0x7f0dc40fc6f0 msgr2=0x7f0dc41053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:25.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.250+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 shutdown_connections 2024-04-20T11:29:25.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.250+0000 7f0dc8a00700 1 -- 172.21.15.117:0/451713701 wait complete. 2024-04-20T11:29:25.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.407+0000 7f1bc1400700 1 -- 172.21.15.117:0/1777902592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 msgr2=0x7f1bbc105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.407+0000 7f1bc1400700 1 --2- 172.21.15.117:0/1777902592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 0x7f1bbc105810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009910 tx=0x7f1ba4009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:25.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.407+0000 7f1bc1400700 1 -- 172.21.15.117:0/1777902592 shutdown_connections 2024-04-20T11:29:25.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.407+0000 7f1bc1400700 1 --2- 172.21.15.117:0/1777902592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 0x7f1bbc105810 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.407+0000 7f1bc1400700 1 --2- 172.21.15.117:0/1777902592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bbc100e10 0x7f1bbc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.407+0000 7f1bc1400700 1 -- 172.21.15.117:0/1777902592 >> 172.21.15.117:0/1777902592 conn(0x7f1bbc0fc6c0 msgr2=0x7f1bbc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:25.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.407+0000 7f1bc1400700 1 -- 172.21.15.117:0/1777902592 shutdown_connections 2024-04-20T11:29:25.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.407+0000 7f1bc1400700 1 -- 172.21.15.117:0/1777902592 wait complete. 2024-04-20T11:29:25.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.408+0000 7f1bc1400700 1 Processor -- start 2024-04-20T11:29:25.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.408+0000 7f1bc1400700 1 -- start start 2024-04-20T11:29:25.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bc1400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bbc100e10 0x7f1bbc19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bc1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 0x7f1bbc19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bc1400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc19d450 con 0x7f1bbc1017c0 2024-04-20T11:29:25.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bc1400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc196a90 con 0x7f1bbc100e10 2024-04-20T11:29:25.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bb1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 0x7f1bbc19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bb1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 0x7f1bbc19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58018/0 (socket says 172.21.15.117:58018) 2024-04-20T11:29:25.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bb1e00700 1 -- 172.21.15.117:0/2895525359 learned_addr learned my addr 172.21.15.117:0/2895525359 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bba800700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bbc100e10 0x7f1bbc19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bb1e00700 1 -- 172.21.15.117:0/2895525359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bbc100e10 msgr2=0x7f1bbc19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bb1e00700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bbc100e10 0x7f1bbc19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.409+0000 7f1bb1e00700 1 -- 172.21.15.117:0/2895525359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba40095f0 con 0x7f1bbc1017c0 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.410+0000 7f1bb1e00700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 0x7f1bbc19cd70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40092e0 tx=0x7f1ba400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.410+0000 7f1bb3e00700 1 -- 172.21.15.117:0/2895525359 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ba401d070 con 0x7f1bbc1017c0 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.410+0000 7f1bb3e00700 1 -- 172.21.15.117:0/2895525359 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ba400f460 con 0x7f1bbc1017c0 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.410+0000 7f1bb3e00700 1 -- 172.21.15.117:0/2895525359 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ba4017630 con 0x7f1bbc1017c0 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.410+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc196d10 con 0x7f1bbc1017c0 2024-04-20T11:29:25.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.410+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc197200 con 0x7f1bbc1017c0 2024-04-20T11:29:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:25 smithi173 ceph-mon[74165]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:25 smithi173 ceph-mon[74165]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:25 smithi173 ceph-mon[74165]: pgmap v509: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:25.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.411+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc1022f0 con 0x7f1bbc1017c0 2024-04-20T11:29:25.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.412+0000 7f1bb3e00700 1 -- 172.21.15.117:0/2895525359 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1ba400f5d0 con 0x7f1bbc1017c0 2024-04-20T11:29:25.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.413+0000 7f1bb3e00700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1ba006ecc0 0x7f1ba0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.413+0000 7f1bb3e00700 1 -- 172.21.15.117:0/2895525359 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1ba4090260 con 0x7f1bbc1017c0 2024-04-20T11:29:25.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.414+0000 7f1bba800700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1ba006ecc0 0x7f1ba0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.414+0000 7f1bba800700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1ba006ecc0 0x7f1ba0071180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc068850 tx=0x7f1bac0094f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.418+0000 7f1bb3e00700 1 -- 172.21.15.117:0/2895525359 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1ba405cb50 con 0x7f1bbc1017c0 2024-04-20T11:29:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:25 smithi117 ceph-mon[99036]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:25 smithi117 ceph-mon[99036]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:25 smithi117 ceph-mon[99036]: pgmap v509: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:25.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.659+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bbc10dea0 con 0x7f1ba006ecc0 2024-04-20T11:29:25.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.691+0000 7f1bb3e00700 1 -- 172.21.15.117:0/2895525359 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1bbc10dea0 con 0x7f1ba006ecc0 2024-04-20T11:29:25.691 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:29:25.691 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (17m) 9m ago 31m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:29:25.691 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (32m) 9m ago 32m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:29:25.691 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (30m) 9m ago 30m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (14m) 9m ago 32m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (14m) 9m ago 30m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (17m) 9m ago 31m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (9m) 9m ago 24m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (10m) 9m ago 24m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (24m) 9m ago 24m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (9m) 9m ago 24m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (21m) 9m ago 33m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (19m) 9m ago 30m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (15m) 9m ago 33m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (15m) 9m ago 30m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (19m) 9m ago 32m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (19m) 9m ago 30m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (13m) 9m ago 29m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:29:25.692 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (13m) 9m ago 28m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:29:25.693 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (12m) 9m ago 28m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:29:25.693 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (11m) 9m ago 27m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:29:25.693 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (11m) 9m ago 27m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:29:25.693 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (10m) 9m ago 26m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:29:25.693 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (18m) 9m ago 31m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:29:25.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.694+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1ba006ecc0 msgr2=0x7f1ba0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.694+0000 7f1bc1400700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1ba006ecc0 0x7f1ba0071180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc068850 tx=0x7f1bac0094f0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.694+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 msgr2=0x7f1bbc19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.694+0000 7f1bc1400700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 0x7f1bbc19cd70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40092e0 tx=0x7f1ba400be20 comp rx=0 tx=0).stop 2024-04-20T11:29:25.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.695+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 shutdown_connections 2024-04-20T11:29:25.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.695+0000 7f1bc1400700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1ba006ecc0 0x7f1ba0071180 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.695+0000 7f1bc1400700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bbc100e10 0x7f1bbc19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.695+0000 7f1bc1400700 1 --2- 172.21.15.117:0/2895525359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bbc1017c0 0x7f1bbc19cd70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.695+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 >> 172.21.15.117:0/2895525359 conn(0x7f1bbc0fc6c0 msgr2=0x7f1bbc0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:25.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.695+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 shutdown_connections 2024-04-20T11:29:25.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.695+0000 7f1bc1400700 1 -- 172.21.15.117:0/2895525359 wait complete. 2024-04-20T11:29:25.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.853+0000 7f3cece00700 1 -- 172.21.15.117:0/3817681046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce8105a80 msgr2=0x7f3ce8105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.853+0000 7f3cece00700 1 --2- 172.21.15.117:0/3817681046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce8105a80 0x7f3ce8105e60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009910 tx=0x7f3cd8009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:25.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.853+0000 7f3cece00700 1 -- 172.21.15.117:0/3817681046 shutdown_connections 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.853+0000 7f3cece00700 1 --2- 172.21.15.117:0/3817681046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce80ffa80 0x7f3ce80fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.853+0000 7f3cece00700 1 --2- 172.21.15.117:0/3817681046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce8105a80 0x7f3ce8105e60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.853+0000 7f3cece00700 1 -- 172.21.15.117:0/3817681046 >> 172.21.15.117:0/3817681046 conn(0x7f3ce8076750 msgr2=0x7f3ce80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.853+0000 7f3cece00700 1 -- 172.21.15.117:0/3817681046 shutdown_connections 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.853+0000 7f3cece00700 1 -- 172.21.15.117:0/3817681046 wait complete. 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.854+0000 7f3cece00700 1 Processor -- start 2024-04-20T11:29:25.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.854+0000 7f3cece00700 1 -- start start 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.855+0000 7f3cece00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce80ffa80 0x7f3ce80fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.855+0000 7f3cece00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce8105a80 0x7f3ce80fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.855+0000 7f3cece00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80fc5c0 con 0x7f3ce80ffa80 2024-04-20T11:29:25.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.855+0000 7f3cece00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80fc700 con 0x7f3ce8105a80 2024-04-20T11:29:25.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.855+0000 7f3ce5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce80ffa80 0x7f3ce80fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.855+0000 7f3ce5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce80ffa80 0x7f3ce80fbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58036/0 (socket says 172.21.15.117:58036) 2024-04-20T11:29:25.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.855+0000 7f3ce5e00700 1 -- 172.21.15.117:0/1002062338 learned_addr learned my addr 172.21.15.117:0/1002062338 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3ce5400700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce8105a80 0x7f3ce80fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3ce5e00700 1 -- 172.21.15.117:0/1002062338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce8105a80 msgr2=0x7f3ce80fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3ce5e00700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce8105a80 0x7f3ce80fc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3ce5e00700 1 -- 172.21.15.117:0/1002062338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80095f0 con 0x7f3ce80ffa80 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3ce5e00700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce80ffa80 0x7f3ce80fbb40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80092e0 tx=0x7f3cd800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3cdea00700 1 -- 172.21.15.117:0/1002062338 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cd801d070 con 0x7f3ce80ffa80 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3cdea00700 1 -- 172.21.15.117:0/1002062338 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cd80056e0 con 0x7f3ce80ffa80 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce80fc980 con 0x7f3ce80ffa80 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.856+0000 7f3cdea00700 1 -- 172.21.15.117:0/1002062338 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cd800fe80 con 0x7f3ce80ffa80 2024-04-20T11:29:25.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.857+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce81a2f10 con 0x7f3ce80ffa80 2024-04-20T11:29:25.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.858+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce81011c0 con 0x7f3ce80ffa80 2024-04-20T11:29:25.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.859+0000 7f3cdea00700 1 -- 172.21.15.117:0/1002062338 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3cd800f6f0 con 0x7f3ce80ffa80 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.860+0000 7f3cdea00700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ccc06ecc0 0x7f3ccc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.860+0000 7f3cdea00700 1 -- 172.21.15.117:0/1002062338 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3cd8090250 con 0x7f3ce80ffa80 2024-04-20T11:29:25.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.860+0000 7f3ce5400700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ccc06ecc0 0x7f3ccc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:25.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.861+0000 7f3ce5400700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ccc06ecc0 0x7f3ccc071180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005950 tx=0x7f3cd40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:25.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:25.865+0000 7f3cdea00700 1 -- 172.21.15.117:0/1002062338 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3cd8059030 con 0x7f3ce80ffa80 2024-04-20T11:29:26.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.160+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3ce810cd40 con 0x7f3ce80ffa80 2024-04-20T11:29:26.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.161+0000 7f3cdea00700 1 -- 172.21.15.117:0/1002062338 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f3cd805c6f0 con 0x7f3ce80ffa80 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:29:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:29:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:29:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:29:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:29:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:29:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:29:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:29:26.162 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:29:26.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ccc06ecc0 msgr2=0x7f3ccc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ccc06ecc0 0x7f3ccc071180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005950 tx=0x7f3cd40058e0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce80ffa80 msgr2=0x7f3ce80fbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce80ffa80 0x7f3ce80fbb40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80092e0 tx=0x7f3cd800be20 comp rx=0 tx=0).stop 2024-04-20T11:29:26.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 shutdown_connections 2024-04-20T11:29:26.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3ccc06ecc0 0x7f3ccc071180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ce80ffa80 0x7f3ce80fbb40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 --2- 172.21.15.117:0/1002062338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce8105a80 0x7f3ce80fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 >> 172.21.15.117:0/1002062338 conn(0x7f3ce8076750 msgr2=0x7f3ce80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 shutdown_connections 2024-04-20T11:29:26.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.165+0000 7f3cece00700 1 -- 172.21.15.117:0/1002062338 wait complete. 2024-04-20T11:29:26.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.320+0000 7f772e600700 1 -- 172.21.15.117:0/2797536839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 msgr2=0x7f7728107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.320+0000 7f772e600700 1 --2- 172.21.15.117:0/2797536839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 0x7f7728107900 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7718009910 tx=0x7f7718009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:26.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.320+0000 7f772e600700 1 -- 172.21.15.117:0/2797536839 shutdown_connections 2024-04-20T11:29:26.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.320+0000 7f772e600700 1 --2- 172.21.15.117:0/2797536839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7728108e10 0x7f7728109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.320+0000 7f772e600700 1 --2- 172.21.15.117:0/2797536839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 0x7f7728107900 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.320+0000 7f772e600700 1 -- 172.21.15.117:0/2797536839 >> 172.21.15.117:0/2797536839 conn(0x7f77280768f0 msgr2=0x7f7728078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.320+0000 7f772e600700 1 -- 172.21.15.117:0/2797536839 shutdown_connections 2024-04-20T11:29:26.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.320+0000 7f772e600700 1 -- 172.21.15.117:0/2797536839 wait complete. 2024-04-20T11:29:26.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.321+0000 7f772e600700 1 Processor -- start 2024-04-20T11:29:26.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.321+0000 7f772e600700 1 -- start start 2024-04-20T11:29:26.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.322+0000 7f772e600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 0x7f772819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.322+0000 7f772e600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7728108e10 0x7f772819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.322+0000 7f772e600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772819dd40 con 0x7f7728107520 2024-04-20T11:29:26.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.322+0000 7f772e600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772819de80 con 0x7f7728108e10 2024-04-20T11:29:26.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.322+0000 7f7727400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 0x7f772819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.322+0000 7f7727400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 0x7f772819d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58050/0 (socket says 172.21.15.117:58050) 2024-04-20T11:29:26.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.322+0000 7f7727400700 1 -- 172.21.15.117:0/538996850 learned_addr learned my addr 172.21.15.117:0/538996850 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f7726a00700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7728108e10 0x7f772819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f7727400700 1 -- 172.21.15.117:0/538996850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7728108e10 msgr2=0x7f772819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f7727400700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7728108e10 0x7f772819d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f7727400700 1 -- 172.21.15.117:0/538996850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180095f0 con 0x7f7728107520 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f7727400700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 0x7f772819d1b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7718006010 tx=0x7f771800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f771fe00700 1 -- 172.21.15.117:0/538996850 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f771801d070 con 0x7f7728107520 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f771fe00700 1 -- 172.21.15.117:0/538996850 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f771800f460 con 0x7f7728107520 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f771fe00700 1 -- 172.21.15.117:0/538996850 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7718017630 con 0x7f7728107520 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77281a1bb0 con 0x7f7728107520 2024-04-20T11:29:26.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.323+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77281a2080 con 0x7f7728107520 2024-04-20T11:29:26.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.325+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772810f890 con 0x7f7728107520 2024-04-20T11:29:26.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.326+0000 7f771fe00700 1 -- 172.21.15.117:0/538996850 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f771800f5d0 con 0x7f7728107520 2024-04-20T11:29:26.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.327+0000 7f771fe00700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f770c06ecc0 0x7f770c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.327+0000 7f771fe00700 1 -- 172.21.15.117:0/538996850 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7718090370 con 0x7f7728107520 2024-04-20T11:29:26.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.327+0000 7f7726a00700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f770c06ecc0 0x7f770c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.328+0000 7f7726a00700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f770c06ecc0 0x7f770c071180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f772819e710 tx=0x7f7714005d50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.332+0000 7f771fe00700 1 -- 172.21.15.117:0/538996850 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f77180591f0 con 0x7f7728107520 2024-04-20T11:29:26.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.596+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f772819e4c0 con 0x7f7728107520 2024-04-20T11:29:26.596 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:26 smithi117 ceph-mon[99036]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:26.596 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:26 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1002062338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:29:26.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.597+0000 7f771fe00700 1 -- 172.21.15.117:0/538996850 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f771805c8b0 con 0x7f7728107520 2024-04-20T11:29:26.597 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:29:26.597 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:29:26.597 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:29:26.597 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:29:26.598 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:29:26.599 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:29:26.600 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.600+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f770c06ecc0 msgr2=0x7f770c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.600+0000 7f772e600700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f770c06ecc0 0x7f770c071180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f772819e710 tx=0x7f7714005d50 comp rx=0 tx=0).stop 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.600+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 msgr2=0x7f772819d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.601+0000 7f772e600700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 0x7f772819d1b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7718006010 tx=0x7f771800be20 comp rx=0 tx=0).stop 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.601+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 shutdown_connections 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.601+0000 7f772e600700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f770c06ecc0 0x7f770c071180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.601+0000 7f772e600700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7728107520 0x7f772819d1b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.601+0000 7f772e600700 1 --2- 172.21.15.117:0/538996850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7728108e10 0x7f772819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.601+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 >> 172.21.15.117:0/538996850 conn(0x7f77280768f0 msgr2=0x7f7728077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.601+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 shutdown_connections 2024-04-20T11:29:26.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.601+0000 7f772e600700 1 -- 172.21.15.117:0/538996850 wait complete. 2024-04-20T11:29:26.602 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:29:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:26 smithi173 ceph-mon[74165]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:26 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1002062338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.758+0000 7fcd94800700 1 -- 172.21.15.117:0/106899623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c108e10 msgr2=0x7fcd8c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.758+0000 7fcd94800700 1 --2- 172.21.15.117:0/106899623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c108e10 0x7fcd8c109290 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009910 tx=0x7fcd7c009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.758+0000 7fcd94800700 1 -- 172.21.15.117:0/106899623 shutdown_connections 2024-04-20T11:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.758+0000 7fcd94800700 1 --2- 172.21.15.117:0/106899623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c108e10 0x7fcd8c109290 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.758+0000 7fcd94800700 1 --2- 172.21.15.117:0/106899623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8c107520 0x7fcd8c107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.758+0000 7fcd94800700 1 -- 172.21.15.117:0/106899623 >> 172.21.15.117:0/106899623 conn(0x7fcd8c0768f0 msgr2=0x7fcd8c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.758+0000 7fcd94800700 1 -- 172.21.15.117:0/106899623 shutdown_connections 2024-04-20T11:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.758+0000 7fcd94800700 1 -- 172.21.15.117:0/106899623 wait complete. 2024-04-20T11:29:26.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.759+0000 7fcd94800700 1 Processor -- start 2024-04-20T11:29:26.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd94800700 1 -- start start 2024-04-20T11:29:26.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd94800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c107520 0x7fcd8c19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd94800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8c108e10 0x7fcd8c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd94800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c19ded0 con 0x7fcd8c107520 2024-04-20T11:29:26.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd94800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1a1ba0 con 0x7fcd8c108e10 2024-04-20T11:29:26.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd91e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c107520 0x7fcd8c19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd91e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c107520 0x7fcd8c19d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58070/0 (socket says 172.21.15.117:58070) 2024-04-20T11:29:26.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd91e00700 1 -- 172.21.15.117:0/1172119454 learned_addr learned my addr 172.21.15.117:0/1172119454 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.760+0000 7fcd91400700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8c108e10 0x7fcd8c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.761+0000 7fcd91e00700 1 -- 172.21.15.117:0/1172119454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8c108e10 msgr2=0x7fcd8c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.761+0000 7fcd91e00700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8c108e10 0x7fcd8c19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.761+0000 7fcd91e00700 1 -- 172.21.15.117:0/1172119454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c0095f0 con 0x7fcd8c107520 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.761+0000 7fcd91e00700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c107520 0x7fcd8c19d2b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b4b0 tx=0x7fcd8000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.761+0000 7fcd86a00700 1 -- 172.21.15.117:0/1172119454 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd80011800 con 0x7fcd8c107520 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.761+0000 7fcd86a00700 1 -- 172.21.15.117:0/1172119454 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd80011e20 con 0x7fcd8c107520 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.761+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c1a1e80 con 0x7fcd8c107520 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.761+0000 7fcd86a00700 1 -- 172.21.15.117:0/1172119454 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd8000f2e0 con 0x7fcd8c107520 2024-04-20T11:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.762+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1a2450 con 0x7fcd8c107520 2024-04-20T11:29:26.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.763+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c10f870 con 0x7fcd8c107520 2024-04-20T11:29:26.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.763+0000 7fcd86a00700 1 -- 172.21.15.117:0/1172119454 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcd80010380 con 0x7fcd8c107520 2024-04-20T11:29:26.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.765+0000 7fcd86a00700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7406ecd0 0x7fcd74071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:26.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.765+0000 7fcd86a00700 1 -- 172.21.15.117:0/1172119454 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcd8008e990 con 0x7fcd8c107520 2024-04-20T11:29:26.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.765+0000 7fcd91400700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7406ecd0 0x7fcd74071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:26.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.766+0000 7fcd91400700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7406ecd0 0x7fcd74071190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c005f50 tx=0x7fcd7c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:26.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:26.770+0000 7fcd86a00700 1 -- 172.21.15.117:0/1172119454 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcd80057750 con 0x7fcd8c107520 2024-04-20T11:29:27.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.018+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd8c19e5c0 con 0x7fcd7406ecd0 2024-04-20T11:29:27.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.020+0000 7fcd86a00700 1 -- 172.21.15.117:0/1172119454 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcd8c19e5c0 con 0x7fcd7406ecd0 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:29:27.020 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:29:27.021 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:29:27.021 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:29:27.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.023+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7406ecd0 msgr2=0x7fcd74071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.023+0000 7fcd94800700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7406ecd0 0x7fcd74071190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c005f50 tx=0x7fcd7c01a040 comp rx=0 tx=0).stop 2024-04-20T11:29:27.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.023+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c107520 msgr2=0x7fcd8c19d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.023+0000 7fcd94800700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c107520 0x7fcd8c19d2b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b4b0 tx=0x7fcd8000b870 comp rx=0 tx=0).stop 2024-04-20T11:29:27.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.024+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 shutdown_connections 2024-04-20T11:29:27.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.024+0000 7fcd94800700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd7406ecd0 0x7fcd74071190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.024+0000 7fcd94800700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd8c107520 0x7fcd8c19d2b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.024+0000 7fcd94800700 1 --2- 172.21.15.117:0/1172119454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8c108e10 0x7fcd8c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.024+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 >> 172.21.15.117:0/1172119454 conn(0x7fcd8c0768f0 msgr2=0x7fcd8c0775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:27.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.024+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 shutdown_connections 2024-04-20T11:29:27.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.024+0000 7fcd94800700 1 -- 172.21.15.117:0/1172119454 wait complete. 2024-04-20T11:29:27.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.182+0000 7f316d200700 1 -- 172.21.15.117:0/1051672602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 msgr2=0x7f31680fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.182+0000 7f316d200700 1 --2- 172.21.15.117:0/1051672602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 0x7f31680fff00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3154009910 tx=0x7f3154009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.182+0000 7f316d200700 1 -- 172.21.15.117:0/1051672602 shutdown_connections 2024-04-20T11:29:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.182+0000 7f316d200700 1 --2- 172.21.15.117:0/1051672602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 0x7f31680fff00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.182+0000 7f316d200700 1 --2- 172.21.15.117:0/1051672602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3168105a80 0x7f3168105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.182+0000 7f316d200700 1 -- 172.21.15.117:0/1051672602 >> 172.21.15.117:0/1051672602 conn(0x7f3168076750 msgr2=0x7f31680fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.182+0000 7f316d200700 1 -- 172.21.15.117:0/1051672602 shutdown_connections 2024-04-20T11:29:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.182+0000 7f316d200700 1 -- 172.21.15.117:0/1051672602 wait complete. 2024-04-20T11:29:27.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.183+0000 7f316d200700 1 Processor -- start 2024-04-20T11:29:27.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.184+0000 7f316d200700 1 -- start start 2024-04-20T11:29:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.184+0000 7f316d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 0x7f316819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.184+0000 7f316d200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3168105a80 0x7f316819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.184+0000 7f316d200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316819d420 con 0x7f31680ffa80 2024-04-20T11:29:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.184+0000 7f316d200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3168196a50 con 0x7f3168105a80 2024-04-20T11:29:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.184+0000 7f3165e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 0x7f316819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.184+0000 7f3165e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 0x7f316819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58086/0 (socket says 172.21.15.117:58086) 2024-04-20T11:29:27.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.184+0000 7f3165e00700 1 -- 172.21.15.117:0/4174319760 learned_addr learned my addr 172.21.15.117:0/4174319760 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:27.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.185+0000 7f3165e00700 1 -- 172.21.15.117:0/4174319760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3168105a80 msgr2=0x7f316819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.185+0000 7f3165e00700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3168105a80 0x7f316819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.185+0000 7f3165e00700 1 -- 172.21.15.117:0/4174319760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31540095f0 con 0x7f31680ffa80 2024-04-20T11:29:27.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.185+0000 7f3165e00700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 0x7f316819c8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f315800b4b0 tx=0x7f315800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:27.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.185+0000 7f315ea00700 1 -- 172.21.15.117:0/4174319760 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3158011800 con 0x7f31680ffa80 2024-04-20T11:29:27.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.185+0000 7f315ea00700 1 -- 172.21.15.117:0/4174319760 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3158011e20 con 0x7f31680ffa80 2024-04-20T11:29:27.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.185+0000 7f315ea00700 1 -- 172.21.15.117:0/4174319760 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f315800f2e0 con 0x7f31680ffa80 2024-04-20T11:29:27.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.186+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3168196d30 con 0x7f31680ffa80 2024-04-20T11:29:27.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.186+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3168197230 con 0x7f31680ffa80 2024-04-20T11:29:27.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.187+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31681011c0 con 0x7f31680ffa80 2024-04-20T11:29:27.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.188+0000 7f315ea00700 1 -- 172.21.15.117:0/4174319760 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3158010380 con 0x7f31680ffa80 2024-04-20T11:29:27.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.189+0000 7f315ea00700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f314c06ecc0 0x7f314c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:27.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.189+0000 7f315ea00700 1 -- 172.21.15.117:0/4174319760 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f315808ea70 con 0x7f31680ffa80 2024-04-20T11:29:27.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.189+0000 7f3165400700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f314c06ecc0 0x7f314c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:27.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.190+0000 7f3165400700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f314c06ecc0 0x7f314c071180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3154006010 tx=0x7f315400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:27.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.193+0000 7f315ea00700 1 -- 172.21.15.117:0/4174319760 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f315805b2e0 con 0x7f31680ffa80 2024-04-20T11:29:27.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.504+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3168062510 con 0x7f31680ffa80 2024-04-20T11:29:27.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.505+0000 7f315ea00700 1 -- 172.21.15.117:0/4174319760 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f315805ae70 con 0x7f31680ffa80 2024-04-20T11:29:27.504 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:29:27.504 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:29:27.505 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:29:27.505 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:29:27.505 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:29:27.505 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:29:27.505 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:29:27.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f314c06ecc0 msgr2=0x7f314c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f314c06ecc0 0x7f314c071180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3154006010 tx=0x7f315400b540 comp rx=0 tx=0).stop 2024-04-20T11:29:27.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 msgr2=0x7f316819c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:27.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 0x7f316819c8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f315800b4b0 tx=0x7f315800b870 comp rx=0 tx=0).stop 2024-04-20T11:29:27.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 shutdown_connections 2024-04-20T11:29:27.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f314c06ecc0 0x7f314c071180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31680ffa80 0x7f316819c8c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 --2- 172.21.15.117:0/4174319760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3168105a80 0x7f316819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:27.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 >> 172.21.15.117:0/4174319760 conn(0x7f3168076750 msgr2=0x7f31680fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:27.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 shutdown_connections 2024-04-20T11:29:27.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:27.508+0000 7f316d200700 1 -- 172.21.15.117:0/4174319760 wait complete. 2024-04-20T11:29:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:27 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/538996850' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:29:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:27 smithi117 ceph-mon[99036]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:27 smithi117 ceph-mon[99036]: pgmap v510: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:27 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/538996850' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:29:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:27 smithi173 ceph-mon[74165]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:27 smithi173 ceph-mon[74165]: pgmap v510: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:28 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4174319760' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:29:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:29:28.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:28 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4174319760' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:29:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:29:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:29 smithi117 ceph-mon[99036]: pgmap v511: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:29.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:29 smithi173 ceph-mon[74165]: pgmap v511: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:31.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:31 smithi117 ceph-mon[99036]: pgmap v512: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:31.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:31 smithi173 ceph-mon[74165]: pgmap v512: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:33.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:33 smithi117 ceph-mon[99036]: pgmap v513: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:33 smithi173 ceph-mon[74165]: pgmap v513: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:35.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:35 smithi117 ceph-mon[99036]: pgmap v514: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:35 smithi173 ceph-mon[74165]: pgmap v514: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:37.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:37 smithi117 ceph-mon[99036]: pgmap v515: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:37 smithi173 ceph-mon[74165]: pgmap v515: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:39.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:39 smithi117 ceph-mon[99036]: pgmap v516: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:39 smithi173 ceph-mon[74165]: pgmap v516: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:41.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:41 smithi117 ceph-mon[99036]: pgmap v517: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:41 smithi173 ceph-mon[74165]: pgmap v517: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:43.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:29:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:29:44.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:44 smithi117 ceph-mon[99036]: pgmap v518: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:44.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:44 smithi173 ceph-mon[74165]: pgmap v518: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:45 smithi117 ceph-mon[99036]: pgmap v519: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:45.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:45 smithi173 ceph-mon[74165]: pgmap v519: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:47.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:47 smithi117 ceph-mon[99036]: pgmap v520: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:47.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:47 smithi173 ceph-mon[74165]: pgmap v520: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:49.596 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:49 smithi117 ceph-mon[99036]: pgmap v521: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:49.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:49 smithi173 ceph-mon[74165]: pgmap v521: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:51.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:51 smithi117 ceph-mon[99036]: pgmap v522: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:51.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:51 smithi173 ceph-mon[74165]: pgmap v522: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:53.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:53 smithi117 ceph-mon[99036]: pgmap v523: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:53.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:53 smithi173 ceph-mon[74165]: pgmap v523: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:55.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:55 smithi117 ceph-mon[99036]: pgmap v524: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:55.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:55 smithi173 ceph-mon[74165]: pgmap v524: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:57 smithi117 ceph-mon[99036]: pgmap v525: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:57 smithi173 ceph-mon[74165]: pgmap v525: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:57.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.679+0000 7efd16c00700 1 -- 172.21.15.117:0/2666204984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073b20 msgr2=0x7efd1010cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:57.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.679+0000 7efd16c00700 1 --2- 172.21.15.117:0/2666204984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073b20 0x7efd1010cf30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009910 tx=0x7efcfc009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:57.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.680+0000 7efd16c00700 1 -- 172.21.15.117:0/2666204984 shutdown_connections 2024-04-20T11:29:57.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.680+0000 7efd16c00700 1 --2- 172.21.15.117:0/2666204984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073b20 0x7efd1010cf30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:57.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.680+0000 7efd16c00700 1 --2- 172.21.15.117:0/2666204984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd10073200 0x7efd100735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:57.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.680+0000 7efd16c00700 1 -- 172.21.15.117:0/2666204984 >> 172.21.15.117:0/2666204984 conn(0x7efd100fc6a0 msgr2=0x7efd100feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:57.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.680+0000 7efd16c00700 1 -- 172.21.15.117:0/2666204984 shutdown_connections 2024-04-20T11:29:57.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.680+0000 7efd16c00700 1 -- 172.21.15.117:0/2666204984 wait complete. 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.681+0000 7efd16c00700 1 Processor -- start 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.681+0000 7efd16c00700 1 -- start start 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.681+0000 7efd16c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073200 0x7efd10198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.681+0000 7efd16c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd10073b20 0x7efd10199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.681+0000 7efd16c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd10199a20 con 0x7efd10073200 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.681+0000 7efd16c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1019d6f0 con 0x7efd10073b20 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.682+0000 7efd0fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073200 0x7efd10198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.682+0000 7efd0fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073200 0x7efd10198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56464/0 (socket says 172.21.15.117:56464) 2024-04-20T11:29:57.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.682+0000 7efd0fe00700 1 -- 172.21.15.117:0/944230271 learned_addr learned my addr 172.21.15.117:0/944230271 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.682+0000 7efd0f400700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd10073b20 0x7efd10199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.682+0000 7efd0fe00700 1 -- 172.21.15.117:0/944230271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd10073b20 msgr2=0x7efd10199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.682+0000 7efd0fe00700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd10073b20 0x7efd10199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.682+0000 7efd0fe00700 1 -- 172.21.15.117:0/944230271 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0095f0 con 0x7efd10073200 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.682+0000 7efd0f400700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd10073b20 0x7efd10199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.683+0000 7efd0fe00700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073200 0x7efd10198e00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efd0000b4b0 tx=0x7efd0000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.683+0000 7efd0cc00700 1 -- 172.21.15.117:0/944230271 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd00011800 con 0x7efd10073200 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.683+0000 7efd0cc00700 1 -- 172.21.15.117:0/944230271 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd00011e20 con 0x7efd10073200 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.683+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1019d9d0 con 0x7efd10073200 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.683+0000 7efd0cc00700 1 -- 172.21.15.117:0/944230271 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd0000f2e0 con 0x7efd10073200 2024-04-20T11:29:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.683+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1019df20 con 0x7efd10073200 2024-04-20T11:29:57.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.684+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd10074940 con 0x7efd10073200 2024-04-20T11:29:57.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.685+0000 7efd0cc00700 1 -- 172.21.15.117:0/944230271 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efd00010380 con 0x7efd10073200 2024-04-20T11:29:57.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.686+0000 7efd0cc00700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efcf406ecc0 0x7efcf4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:57.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.686+0000 7efd0cc00700 1 -- 172.21.15.117:0/944230271 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efd0008ea70 con 0x7efd10073200 2024-04-20T11:29:57.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.687+0000 7efd0f400700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efcf406ecc0 0x7efcf4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:57.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.687+0000 7efd0f400700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efcf406ecc0 0x7efcf4071180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efcfc005f50 tx=0x7efcfc005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:57.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.691+0000 7efd0cc00700 1 -- 172.21.15.117:0/944230271 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efd000577b0 con 0x7efd10073200 2024-04-20T11:29:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.930+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd10107840 con 0x7efcf406ecc0 2024-04-20T11:29:57.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.933+0000 7efd0cc00700 1 -- 172.21.15.117:0/944230271 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efd10107840 con 0x7efcf406ecc0 2024-04-20T11:29:57.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efcf406ecc0 msgr2=0x7efcf4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:57.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efcf406ecc0 0x7efcf4071180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efcfc005f50 tx=0x7efcfc005dc0 comp rx=0 tx=0).stop 2024-04-20T11:29:57.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073200 msgr2=0x7efd10198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:57.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073200 0x7efd10198e00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efd0000b4b0 tx=0x7efd0000b870 comp rx=0 tx=0).stop 2024-04-20T11:29:57.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 shutdown_connections 2024-04-20T11:29:57.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efcf406ecc0 0x7efcf4071180 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:57.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10073200 0x7efd10198e00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:57.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 --2- 172.21.15.117:0/944230271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd10073b20 0x7efd10199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:57.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 >> 172.21.15.117:0/944230271 conn(0x7efd100fc6a0 msgr2=0x7efd10105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:57.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 shutdown_connections 2024-04-20T11:29:57.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:57.936+0000 7efd16c00700 1 -- 172.21.15.117:0/944230271 wait complete. 2024-04-20T11:29:57.951 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:29:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.084+0000 7f6682c00700 1 -- 172.21.15.117:0/3660088383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 msgr2=0x7f667c102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.084+0000 7f6682c00700 1 --2- 172.21.15.117:0/3660088383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 0x7f667c102120 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f666c009910 tx=0x7f666c009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.084+0000 7f6682c00700 1 -- 172.21.15.117:0/3660088383 shutdown_connections 2024-04-20T11:29:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.084+0000 7f6682c00700 1 --2- 172.21.15.117:0/3660088383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f667c102660 0x7f667c10ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.084+0000 7f6682c00700 1 --2- 172.21.15.117:0/3660088383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 0x7f667c102120 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.084+0000 7f6682c00700 1 -- 172.21.15.117:0/3660088383 >> 172.21.15.117:0/3660088383 conn(0x7f667c076750 msgr2=0x7f667c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.084+0000 7f6682c00700 1 -- 172.21.15.117:0/3660088383 shutdown_connections 2024-04-20T11:29:58.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.084+0000 7f6682c00700 1 -- 172.21.15.117:0/3660088383 wait complete. 2024-04-20T11:29:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.085+0000 7f6682c00700 1 Processor -- start 2024-04-20T11:29:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.086+0000 7f6682c00700 1 -- start start 2024-04-20T11:29:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.086+0000 7f6682c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 0x7f667c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.086+0000 7f6682c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f667c102660 0x7f667c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.086+0000 7f6682c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c19d480 con 0x7f667c101d40 2024-04-20T11:29:58.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.086+0000 7f6682c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c196a60 con 0x7f667c102660 2024-04-20T11:29:58.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.086+0000 7f667be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 0x7f667c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.086+0000 7f667be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 0x7f667c19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56488/0 (socket says 172.21.15.117:56488) 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.086+0000 7f667be00700 1 -- 172.21.15.117:0/1357633169 learned_addr learned my addr 172.21.15.117:0/1357633169 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f667b400700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f667c102660 0x7f667c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f667be00700 1 -- 172.21.15.117:0/1357633169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f667c102660 msgr2=0x7f667c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f667be00700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f667c102660 0x7f667c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f667be00700 1 -- 172.21.15.117:0/1357633169 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0095f0 con 0x7f667c101d40 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f667be00700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 0x7f667c19c920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f666c006010 tx=0x7f666c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f6678c00700 1 -- 172.21.15.117:0/1357633169 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c01d070 con 0x7f667c101d40 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f6678c00700 1 -- 172.21.15.117:0/1357633169 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f666c00f460 con 0x7f667c101d40 2024-04-20T11:29:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f6678c00700 1 -- 172.21.15.117:0/1357633169 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c017630 con 0x7f667c101d40 2024-04-20T11:29:58.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.087+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c196ce0 con 0x7f667c101d40 2024-04-20T11:29:58.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.088+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c1971d0 con 0x7f667c101d40 2024-04-20T11:29:58.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.089+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c105530 con 0x7f667c101d40 2024-04-20T11:29:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.089+0000 7f6678c00700 1 -- 172.21.15.117:0/1357633169 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f666c00f5d0 con 0x7f667c101d40 2024-04-20T11:29:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.090+0000 7f6678c00700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f666006ecc0 0x7f6660071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.090+0000 7f6678c00700 1 -- 172.21.15.117:0/1357633169 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f666c090300 con 0x7f667c101d40 2024-04-20T11:29:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.091+0000 7f667b400700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f666006ecc0 0x7f6660071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.091+0000 7f667b400700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f666006ecc0 0x7f6660071180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6668006fd0 tx=0x7f6668008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:58.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.095+0000 7f6678c00700 1 -- 172.21.15.117:0/1357633169 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f666c05ccd0 con 0x7f667c101d40 2024-04-20T11:29:58.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.327+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f667c197f20 con 0x7f666006ecc0 2024-04-20T11:29:58.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.331+0000 7f6678c00700 1 -- 172.21.15.117:0/1357633169 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f667c197f20 con 0x7f666006ecc0 2024-04-20T11:29:58.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f666006ecc0 msgr2=0x7f6660071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f666006ecc0 0x7f6660071180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6668006fd0 tx=0x7f6668008040 comp rx=0 tx=0).stop 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 msgr2=0x7f667c19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 0x7f667c19c920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f666c006010 tx=0x7f666c00be20 comp rx=0 tx=0).stop 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 shutdown_connections 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f666006ecc0 0x7f6660071180 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667c101d40 0x7f667c19c920 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 --2- 172.21.15.117:0/1357633169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f667c102660 0x7f667c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.333+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 >> 172.21.15.117:0/1357633169 conn(0x7f667c076750 msgr2=0x7f667c103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.334+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 shutdown_connections 2024-04-20T11:29:58.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.334+0000 7f6682c00700 1 -- 172.21.15.117:0/1357633169 wait complete. 2024-04-20T11:29:58.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.485+0000 7f66d7800700 1 -- 172.21.15.117:0/4090846579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0104a00 msgr2=0x7f66d0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.485+0000 7f66d7800700 1 --2- 172.21.15.117:0/4090846579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0104a00 0x7f66d0104e80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009910 tx=0x7f66bc009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:58.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.486+0000 7f66d7800700 1 -- 172.21.15.117:0/4090846579 shutdown_connections 2024-04-20T11:29:58.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.486+0000 7f66d7800700 1 --2- 172.21.15.117:0/4090846579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0104a00 0x7f66d0104e80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.486+0000 7f66d7800700 1 --2- 172.21.15.117:0/4090846579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66d0103110 0x7f66d01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.486+0000 7f66d7800700 1 -- 172.21.15.117:0/4090846579 >> 172.21.15.117:0/4090846579 conn(0x7f66d00fe880 msgr2=0x7f66d0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:58.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.486+0000 7f66d7800700 1 -- 172.21.15.117:0/4090846579 shutdown_connections 2024-04-20T11:29:58.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.486+0000 7f66d7800700 1 -- 172.21.15.117:0/4090846579 wait complete. 2024-04-20T11:29:58.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.487+0000 7f66d7800700 1 Processor -- start 2024-04-20T11:29:58.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.487+0000 7f66d7800700 1 -- start start 2024-04-20T11:29:58.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.487+0000 7f66d7800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0103110 0x7f66d0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.487+0000 7f66d7800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66d0104a00 0x7f66d01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.487+0000 7f66d7800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0199ac0 con 0x7f66d0103110 2024-04-20T11:29:58.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.487+0000 7f66d7800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d019d740 con 0x7f66d0104a00 2024-04-20T11:29:58.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.488+0000 7f66d4e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0103110 0x7f66d0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.488+0000 7f66d4e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0103110 0x7f66d0198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56520/0 (socket says 172.21.15.117:56520) 2024-04-20T11:29:58.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.488+0000 7f66d4e00700 1 -- 172.21.15.117:0/2236957738 learned_addr learned my addr 172.21.15.117:0/2236957738 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:58.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.488+0000 7f66cfe00700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66d0104a00 0x7f66d01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.488+0000 7f66d4e00700 1 -- 172.21.15.117:0/2236957738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66d0104a00 msgr2=0x7f66d01993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.488+0000 7f66d4e00700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66d0104a00 0x7f66d01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.488+0000 7f66d4e00700 1 -- 172.21.15.117:0/2236957738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0095f0 con 0x7f66d0103110 2024-04-20T11:29:58.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.488+0000 7f66d4e00700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0103110 0x7f66d0198ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f66c400b4b0 tx=0x7f66c400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:58.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.489+0000 7f66cd600700 1 -- 172.21.15.117:0/2236957738 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66c4011800 con 0x7f66d0103110 2024-04-20T11:29:58.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.489+0000 7f66cd600700 1 -- 172.21.15.117:0/2236957738 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66c4011e20 con 0x7f66d0103110 2024-04-20T11:29:58.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.489+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d019da20 con 0x7f66d0103110 2024-04-20T11:29:58.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.489+0000 7f66cd600700 1 -- 172.21.15.117:0/2236957738 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66c400f2e0 con 0x7f66d0103110 2024-04-20T11:29:58.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.489+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d019dff0 con 0x7f66d0103110 2024-04-20T11:29:58.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.490+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d010b460 con 0x7f66d0103110 2024-04-20T11:29:58.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.491+0000 7f66cd600700 1 -- 172.21.15.117:0/2236957738 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f66c4010380 con 0x7f66d0103110 2024-04-20T11:29:58.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.492+0000 7f66cd600700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66b806ecd0 0x7f66b8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.493+0000 7f66cd600700 1 -- 172.21.15.117:0/2236957738 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f66c408e9d0 con 0x7f66d0103110 2024-04-20T11:29:58.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.493+0000 7f66cfe00700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66b806ecd0 0x7f66b8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.493+0000 7f66cfe00700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66b806ecd0 0x7f66b8071190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66bc006010 tx=0x7f66bc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:58.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.497+0000 7f66cd600700 1 -- 172.21.15.117:0/2236957738 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f66c40577b0 con 0x7f66d0103110 2024-04-20T11:29:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:58 smithi117 ceph-mon[99036]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:29:58.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:58 smithi173 ceph-mon[74165]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:29:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.722+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f66d019a160 con 0x7f66b806ecd0 2024-04-20T11:29:58.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.740+0000 7f66cd600700 1 -- 172.21.15.117:0/2236957738 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f66d019a160 con 0x7f66b806ecd0 2024-04-20T11:29:58.740 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (18m) 10m ago 32m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (32m) 10m ago 32m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (31m) 9m ago 31m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (14m) 10m ago 32m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (14m) 9m ago 31m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (17m) 10m ago 32m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (10m) 10m ago 25m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (10m) 10m ago 25m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (25m) 9m ago 25m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:29:58.741 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (9m) 9m ago 25m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (21m) 10m ago 34m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (20m) 9m ago 31m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (16m) 10m ago 34m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (15m) 9m ago 31m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (19m) 10m ago 32m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (19m) 9m ago 31m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (14m) 10m ago 30m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (13m) 10m ago 29m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (13m) 10m ago 28m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:29:58.742 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (12m) 9m ago 28m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:29:58.743 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (11m) 9m ago 27m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:29:58.743 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (11m) 9m ago 27m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:29:58.743 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (19m) 10m ago 31m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:29:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.742+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66b806ecd0 msgr2=0x7f66b8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.742+0000 7f66d7800700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66b806ecd0 0x7f66b8071190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66bc006010 tx=0x7f66bc00b540 comp rx=0 tx=0).stop 2024-04-20T11:29:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.742+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0103110 msgr2=0x7f66d0198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.742+0000 7f66d7800700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0103110 0x7f66d0198ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f66c400b4b0 tx=0x7f66c400b870 comp rx=0 tx=0).stop 2024-04-20T11:29:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.743+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 shutdown_connections 2024-04-20T11:29:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.743+0000 7f66d7800700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66b806ecd0 0x7f66b8071190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.743+0000 7f66d7800700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d0103110 0x7f66d0198ea0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.743+0000 7f66d7800700 1 --2- 172.21.15.117:0/2236957738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66d0104a00 0x7f66d01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.743+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 >> 172.21.15.117:0/2236957738 conn(0x7f66d00fe880 msgr2=0x7f66d00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:58.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.743+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 shutdown_connections 2024-04-20T11:29:58.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.743+0000 7f66d7800700 1 -- 172.21.15.117:0/2236957738 wait complete. 2024-04-20T11:29:58.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.883+0000 7f5b9b200700 1 -- 172.21.15.117:0/1634691709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c10d0c0 msgr2=0x7f5b9c10d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.883+0000 7f5b9b200700 1 --2- 172.21.15.117:0/1634691709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c10d0c0 0x7f5b9c10d4a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009910 tx=0x7f5b8c009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:58.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.883+0000 7f5b9b200700 1 -- 172.21.15.117:0/1634691709 shutdown_connections 2024-04-20T11:29:58.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.883+0000 7f5b9b200700 1 --2- 172.21.15.117:0/1634691709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b9c1070b0 0x7f5b9c107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.883+0000 7f5b9b200700 1 --2- 172.21.15.117:0/1634691709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c10d0c0 0x7f5b9c10d4a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.883+0000 7f5b9b200700 1 -- 172.21.15.117:0/1634691709 >> 172.21.15.117:0/1634691709 conn(0x7f5b9c076830 msgr2=0x7f5b9c078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:58.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.883+0000 7f5b9b200700 1 -- 172.21.15.117:0/1634691709 shutdown_connections 2024-04-20T11:29:58.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.883+0000 7f5b9b200700 1 -- 172.21.15.117:0/1634691709 wait complete. 2024-04-20T11:29:58.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.884+0000 7f5b9b200700 1 Processor -- start 2024-04-20T11:29:58.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.884+0000 7f5b9b200700 1 -- start start 2024-04-20T11:29:58.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b9b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c1070b0 0x7f5b9c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b9b200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b9c10d0c0 0x7f5b9c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b9b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c19db50 con 0x7f5b9c1070b0 2024-04-20T11:29:58.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b9b200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c1a1820 con 0x7f5b9c10d0c0 2024-04-20T11:29:58.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b99e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c1070b0 0x7f5b9c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b99e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c1070b0 0x7f5b9c19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56548/0 (socket says 172.21.15.117:56548) 2024-04-20T11:29:58.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b99e00700 1 -- 172.21.15.117:0/1482768509 learned_addr learned my addr 172.21.15.117:0/1482768509 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:58.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b99400700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b9c10d0c0 0x7f5b9c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.885+0000 7f5b99e00700 1 -- 172.21.15.117:0/1482768509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b9c10d0c0 msgr2=0x7f5b9c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:58.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.886+0000 7f5b99e00700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b9c10d0c0 0x7f5b9c19d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:58.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.886+0000 7f5b99e00700 1 -- 172.21.15.117:0/1482768509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0095f0 con 0x7f5b9c1070b0 2024-04-20T11:29:58.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.886+0000 7f5b99e00700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c1070b0 0x7f5b9c19cf30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0092e0 tx=0x7f5b8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:58.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.886+0000 7f5b92a00700 1 -- 172.21.15.117:0/1482768509 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b8c01d070 con 0x7f5b9c1070b0 2024-04-20T11:29:58.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.886+0000 7f5b92a00700 1 -- 172.21.15.117:0/1482768509 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b8c00f460 con 0x7f5b9c1070b0 2024-04-20T11:29:58.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.886+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c1a1aa0 con 0x7f5b9c1070b0 2024-04-20T11:29:58.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.886+0000 7f5b92a00700 1 -- 172.21.15.117:0/1482768509 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b8c017630 con 0x7f5b9c1070b0 2024-04-20T11:29:58.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.886+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c1a1f90 con 0x7f5b9c1070b0 2024-04-20T11:29:58.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.887+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c10f5e0 con 0x7f5b9c1070b0 2024-04-20T11:29:58.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.888+0000 7f5b92a00700 1 -- 172.21.15.117:0/1482768509 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5b8c00f5d0 con 0x7f5b9c1070b0 2024-04-20T11:29:58.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.889+0000 7f5b92a00700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b8006ecd0 0x7f5b80071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:58.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.889+0000 7f5b92a00700 1 -- 172.21.15.117:0/1482768509 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5b8c090320 con 0x7f5b9c1070b0 2024-04-20T11:29:58.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.890+0000 7f5b99400700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b8006ecd0 0x7f5b80071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:58.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.890+0000 7f5b99400700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b8006ecd0 0x7f5b80071190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c19e490 tx=0x7f5b88005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:58.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:58.895+0000 7f5b92a00700 1 -- 172.21.15.117:0/1482768509 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5b8c0590e0 con 0x7f5b9c1070b0 2024-04-20T11:29:59.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.192+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5b9c108770 con 0x7f5b9c1070b0 2024-04-20T11:29:59.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.193+0000 7f5b92a00700 1 -- 172.21.15.117:0/1482768509 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f5b8c05c7a0 con 0x7f5b9c1070b0 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:29:59.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:29:59.195 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:29:59.195 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:29:59.195 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:29:59.195 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:29:59.195 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:29:59.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b8006ecd0 msgr2=0x7f5b80071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:59.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b8006ecd0 0x7f5b80071190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c19e490 tx=0x7f5b88005cf0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c1070b0 msgr2=0x7f5b9c19cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:59.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c1070b0 0x7f5b9c19cf30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0092e0 tx=0x7f5b8c00be20 comp rx=0 tx=0).stop 2024-04-20T11:29:59.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 shutdown_connections 2024-04-20T11:29:59.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b8006ecd0 0x7f5b80071190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b9c1070b0 0x7f5b9c19cf30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 --2- 172.21.15.117:0/1482768509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b9c10d0c0 0x7f5b9c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 >> 172.21.15.117:0/1482768509 conn(0x7f5b9c076830 msgr2=0x7f5b9c10c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:59.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 shutdown_connections 2024-04-20T11:29:59.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.196+0000 7f5b9b200700 1 -- 172.21.15.117:0/1482768509 wait complete. 2024-04-20T11:29:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.352+0000 7fa12bc00700 1 -- 172.21.15.117:0/1893634762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124102660 msgr2=0x7fa12410ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.352+0000 7fa12bc00700 1 --2- 172.21.15.117:0/1893634762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124102660 0x7fa12410ab50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009910 tx=0x7fa10c009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.352+0000 7fa12bc00700 1 -- 172.21.15.117:0/1893634762 shutdown_connections 2024-04-20T11:29:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.352+0000 7fa12bc00700 1 --2- 172.21.15.117:0/1893634762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124102660 0x7fa12410ab50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.352+0000 7fa12bc00700 1 --2- 172.21.15.117:0/1893634762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa124101d40 0x7fa124102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.352+0000 7fa12bc00700 1 -- 172.21.15.117:0/1893634762 >> 172.21.15.117:0/1893634762 conn(0x7fa124076750 msgr2=0x7fa1240fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.352+0000 7fa12bc00700 1 -- 172.21.15.117:0/1893634762 shutdown_connections 2024-04-20T11:29:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.352+0000 7fa12bc00700 1 -- 172.21.15.117:0/1893634762 wait complete. 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.353+0000 7fa12bc00700 1 Processor -- start 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.354+0000 7fa12bc00700 1 -- start start 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.354+0000 7fa12bc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124101d40 0x7fa12419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.354+0000 7fa12bc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa124102660 0x7fa12419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.354+0000 7fa12bc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12419d4c0 con 0x7fa124101d40 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.354+0000 7fa12bc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa124196aa0 con 0x7fa124102660 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.354+0000 7fa129200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124101d40 0x7fa12419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.354+0000 7fa129200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124101d40 0x7fa12419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56574/0 (socket says 172.21.15.117:56574) 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.354+0000 7fa129200700 1 -- 172.21.15.117:0/4042102284 learned_addr learned my addr 172.21.15.117:0/4042102284 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa11be00700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa124102660 0x7fa12419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa129200700 1 -- 172.21.15.117:0/4042102284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa124102660 msgr2=0x7fa12419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa129200700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa124102660 0x7fa12419cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa129200700 1 -- 172.21.15.117:0/4042102284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0095f0 con 0x7fa124101d40 2024-04-20T11:29:59.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa129200700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124101d40 0x7fa12419c960 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa11400b4b0 tx=0x7fa11400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:59.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa122000700 1 -- 172.21.15.117:0/4042102284 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa114011800 con 0x7fa124101d40 2024-04-20T11:29:59.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa122000700 1 -- 172.21.15.117:0/4042102284 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa114011e20 con 0x7fa124101d40 2024-04-20T11:29:59.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa122000700 1 -- 172.21.15.117:0/4042102284 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa11400f2e0 con 0x7fa124101d40 2024-04-20T11:29:59.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa124196d80 con 0x7fa124101d40 2024-04-20T11:29:59.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.355+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1241972d0 con 0x7fa124101d40 2024-04-20T11:29:59.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.356+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa124105530 con 0x7fa124101d40 2024-04-20T11:29:59.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.357+0000 7fa122000700 1 -- 172.21.15.117:0/4042102284 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa114010380 con 0x7fa124101d40 2024-04-20T11:29:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.358+0000 7fa122000700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa10806ecc0 0x7fa108071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:59.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.358+0000 7fa122000700 1 -- 172.21.15.117:0/4042102284 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa11408e960 con 0x7fa124101d40 2024-04-20T11:29:59.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.359+0000 7fa11be00700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa10806ecc0 0x7fa108071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:59.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.359+0000 7fa11be00700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa10806ecc0 0x7fa108071180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006010 tx=0x7fa10c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:59.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.363+0000 7fa122000700 1 -- 172.21.15.117:0/4042102284 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa11405b3a0 con 0x7fa124101d40 2024-04-20T11:29:59.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.636+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa12410de20 con 0x7fa124101d40 2024-04-20T11:29:59.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:59 smithi117 ceph-mon[99036]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:59.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:59 smithi117 ceph-mon[99036]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:59.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:59 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1482768509' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:29:59.637 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:29:59 smithi117 ceph-mon[99036]: pgmap v526: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:59.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.637+0000 7fa122000700 1 -- 172.21.15.117:0/4042102284 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fa11405af30 con 0x7fa124101d40 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:29:59.638 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:29:59.639 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:29:59.640 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:29:59.641 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:29:59.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa10806ecc0 msgr2=0x7fa108071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:59.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa10806ecc0 0x7fa108071180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006010 tx=0x7fa10c00b540 comp rx=0 tx=0).stop 2024-04-20T11:29:59.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124101d40 msgr2=0x7fa12419c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:59.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124101d40 0x7fa12419c960 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa11400b4b0 tx=0x7fa11400b870 comp rx=0 tx=0).stop 2024-04-20T11:29:59.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 shutdown_connections 2024-04-20T11:29:59.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa10806ecc0 0x7fa108071180 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa124101d40 0x7fa12419c960 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 --2- 172.21.15.117:0/4042102284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa124102660 0x7fa12419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 >> 172.21.15.117:0/4042102284 conn(0x7fa124076750 msgr2=0x7fa124103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:59.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 shutdown_connections 2024-04-20T11:29:59.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.641+0000 7fa12bc00700 1 -- 172.21.15.117:0/4042102284 wait complete. 2024-04-20T11:29:59.643 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:29:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:59 smithi173 ceph-mon[74165]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:59 smithi173 ceph-mon[74165]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:29:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:59 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1482768509' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:29:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:29:59 smithi173 ceph-mon[74165]: pgmap v526: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:29:59.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.794+0000 7f231ae00700 1 -- 172.21.15.117:0/1582501769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 msgr2=0x7f2314107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:59.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.794+0000 7f231ae00700 1 --2- 172.21.15.117:0/1582501769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 0x7f2314107900 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2304009910 tx=0x7f2304009c20 comp rx=0 tx=0).stop 2024-04-20T11:29:59.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.794+0000 7f231ae00700 1 -- 172.21.15.117:0/1582501769 shutdown_connections 2024-04-20T11:29:59.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.794+0000 7f231ae00700 1 --2- 172.21.15.117:0/1582501769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2314108e10 0x7f2314109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.794+0000 7f231ae00700 1 --2- 172.21.15.117:0/1582501769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 0x7f2314107900 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.794+0000 7f231ae00700 1 -- 172.21.15.117:0/1582501769 >> 172.21.15.117:0/1582501769 conn(0x7f23140768f0 msgr2=0x7f2314078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:59.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.794+0000 7f231ae00700 1 -- 172.21.15.117:0/1582501769 shutdown_connections 2024-04-20T11:29:59.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.794+0000 7f231ae00700 1 -- 172.21.15.117:0/1582501769 wait complete. 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.795+0000 7f231ae00700 1 Processor -- start 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.795+0000 7f231ae00700 1 -- start start 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.795+0000 7f231ae00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 0x7f231419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.795+0000 7f231ae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2314108e10 0x7f231419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.795+0000 7f231ae00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231419ddd0 con 0x7f2314107520 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.795+0000 7f231ae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141a1aa0 con 0x7f2314108e10 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.796+0000 7f2313e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 0x7f231419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.796+0000 7f2313e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 0x7f231419d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56598/0 (socket says 172.21.15.117:56598) 2024-04-20T11:29:59.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.796+0000 7f2313e00700 1 -- 172.21.15.117:0/989948162 learned_addr learned my addr 172.21.15.117:0/989948162 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.796+0000 7f2313e00700 1 -- 172.21.15.117:0/989948162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2314108e10 msgr2=0x7f231419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.796+0000 7f2313400700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2314108e10 0x7f231419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.796+0000 7f2313e00700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2314108e10 0x7f231419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.796+0000 7f2313e00700 1 -- 172.21.15.117:0/989948162 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23040095f0 con 0x7f2314107520 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.796+0000 7f2313e00700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 0x7f231419d1b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f23040092e0 tx=0x7f230400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.797+0000 7f2310c00700 1 -- 172.21.15.117:0/989948162 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f230401d070 con 0x7f2314107520 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.797+0000 7f2310c00700 1 -- 172.21.15.117:0/989948162 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f230400f460 con 0x7f2314107520 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.797+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23141a1d20 con 0x7f2314107520 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.797+0000 7f2310c00700 1 -- 172.21.15.117:0/989948162 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2304017630 con 0x7f2314107520 2024-04-20T11:29:59.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.797+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23141a2290 con 0x7f2314107520 2024-04-20T11:29:59.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.798+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231410f890 con 0x7f2314107520 2024-04-20T11:29:59.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.799+0000 7f2310c00700 1 -- 172.21.15.117:0/989948162 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f230400f5d0 con 0x7f2314107520 2024-04-20T11:29:59.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.800+0000 7f2310c00700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f22f806ecd0 0x7f22f8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:59.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.800+0000 7f2310c00700 1 -- 172.21.15.117:0/989948162 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f23040903f0 con 0x7f2314107520 2024-04-20T11:29:59.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.800+0000 7f2313400700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f22f806ecd0 0x7f22f8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:59.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.801+0000 7f2313400700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f22f806ecd0 0x7f22f8071190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f231419e710 tx=0x7f2300005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:59.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:29:59.805+0000 7f2310c00700 1 -- 172.21.15.117:0/989948162 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2304059130 con 0x7f2314107520 2024-04-20T11:30:00.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.047+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f231419e530 con 0x7f22f806ecd0 2024-04-20T11:30:00.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.049+0000 7f2310c00700 1 -- 172.21.15.117:0/989948162 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f231419e530 con 0x7f22f806ecd0 2024-04-20T11:30:00.049 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:30:00.049 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:30:00.050 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:30:00.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.051+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f22f806ecd0 msgr2=0x7f22f8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.051+0000 7f231ae00700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f22f806ecd0 0x7f22f8071190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f231419e710 tx=0x7f2300005cf0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.051+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 msgr2=0x7f231419d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.051+0000 7f231ae00700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 0x7f231419d1b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f23040092e0 tx=0x7f230400be20 comp rx=0 tx=0).stop 2024-04-20T11:30:00.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.052+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 shutdown_connections 2024-04-20T11:30:00.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.052+0000 7f231ae00700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f22f806ecd0 0x7f22f8071190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.052+0000 7f231ae00700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2314107520 0x7f231419d1b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.052+0000 7f231ae00700 1 --2- 172.21.15.117:0/989948162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2314108e10 0x7f231419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.052+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 >> 172.21.15.117:0/989948162 conn(0x7f23140768f0 msgr2=0x7f2314077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:00.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.052+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 shutdown_connections 2024-04-20T11:30:00.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.052+0000 7f231ae00700 1 -- 172.21.15.117:0/989948162 wait complete. 2024-04-20T11:30:00.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.195+0000 7fadc6000700 1 -- 172.21.15.117:0/1825209581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 msgr2=0x7fadc010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.195+0000 7fadc6000700 1 --2- 172.21.15.117:0/1825209581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 0x7fadc010d4a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009910 tx=0x7fadb4009c20 comp rx=0 tx=0).stop 2024-04-20T11:30:00.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.195+0000 7fadc6000700 1 -- 172.21.15.117:0/1825209581 shutdown_connections 2024-04-20T11:30:00.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.195+0000 7fadc6000700 1 --2- 172.21.15.117:0/1825209581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc01070b0 0x7fadc0107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.195+0000 7fadc6000700 1 --2- 172.21.15.117:0/1825209581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 0x7fadc010d4a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.195+0000 7fadc6000700 1 -- 172.21.15.117:0/1825209581 >> 172.21.15.117:0/1825209581 conn(0x7fadc0076830 msgr2=0x7fadc0078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:00.197 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:29:59.998+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:30:00.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.196+0000 7fadc6000700 1 -- 172.21.15.117:0/1825209581 shutdown_connections 2024-04-20T11:30:00.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.196+0000 7fadc6000700 1 -- 172.21.15.117:0/1825209581 wait complete. 2024-04-20T11:30:00.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.197+0000 7fadc6000700 1 Processor -- start 2024-04-20T11:30:00.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.197+0000 7fadc6000700 1 -- start start 2024-04-20T11:30:00.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.197+0000 7fadc6000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc01070b0 0x7fadc019d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.197+0000 7fadc6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 0x7fadc019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.197+0000 7fadc6000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc019dca0 con 0x7fadc010d0c0 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.197+0000 7fadc6000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc01a1920 con 0x7fadc01070b0 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadbfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 0x7fadc019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadbfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 0x7fadc019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56616/0 (socket says 172.21.15.117:56616) 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadbfe00700 1 -- 172.21.15.117:0/219168821 learned_addr learned my addr 172.21.15.117:0/219168821 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadc4c00700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc01070b0 0x7fadc019d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadbfe00700 1 -- 172.21.15.117:0/219168821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc01070b0 msgr2=0x7fadc019d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadbfe00700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc01070b0 0x7fadc019d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadbfe00700 1 -- 172.21.15.117:0/219168821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb40095f0 con 0x7fadc010d0c0 2024-04-20T11:30:00.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadc4c00700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc01070b0 0x7fadc019d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:00.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.198+0000 7fadbfe00700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 0x7fadc019d5c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fadac00b4b0 tx=0x7fadac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:00.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.199+0000 7fadbd600700 1 -- 172.21.15.117:0/219168821 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadac011800 con 0x7fadc010d0c0 2024-04-20T11:30:00.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.199+0000 7fadbd600700 1 -- 172.21.15.117:0/219168821 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadac011e20 con 0x7fadc010d0c0 2024-04-20T11:30:00.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.199+0000 7fadbd600700 1 -- 172.21.15.117:0/219168821 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadac00f2e0 con 0x7fadc010d0c0 2024-04-20T11:30:00.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.199+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc01a1c00 con 0x7fadc010d0c0 2024-04-20T11:30:00.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.199+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc01a2150 con 0x7fadc010d0c0 2024-04-20T11:30:00.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.200+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc010f5c0 con 0x7fadc010d0c0 2024-04-20T11:30:00.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.201+0000 7fadbd600700 1 -- 172.21.15.117:0/219168821 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fadac011960 con 0x7fadc010d0c0 2024-04-20T11:30:00.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.202+0000 7fadbd600700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fada806ecd0 0x7fada8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:00.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.202+0000 7fadbd600700 1 -- 172.21.15.117:0/219168821 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fadac08e5d0 con 0x7fadc010d0c0 2024-04-20T11:30:00.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.202+0000 7fadc4c00700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fada806ecd0 0x7fada8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:00.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.203+0000 7fadc4c00700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fada806ecd0 0x7fada8071190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009fd0 tx=0x7fadb400b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:00.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.207+0000 7fadbd600700 1 -- 172.21.15.117:0/219168821 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fadac057430 con 0x7fadc010d0c0 2024-04-20T11:30:00.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.520+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fadc0108770 con 0x7fadc010d0c0 2024-04-20T11:30:00.521 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:00 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4042102284' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:30:00.522 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:00 smithi117 ceph-mon[99036]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:30:00.522 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:00 smithi117 ceph-mon[99036]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:00.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.521+0000 7fadbd600700 1 -- 172.21.15.117:0/219168821 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fadac05aaf0 con 0x7fadc010d0c0 2024-04-20T11:30:00.522 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:30:00.522 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:30:00.522 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:30:00.522 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:30:00.522 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:30:00.523 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:30:00.523 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:30:00.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fada806ecd0 msgr2=0x7fada8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fada806ecd0 0x7fada8071190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009fd0 tx=0x7fadb400b560 comp rx=0 tx=0).stop 2024-04-20T11:30:00.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 msgr2=0x7fadc019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:00.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 0x7fadc019d5c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fadac00b4b0 tx=0x7fadac00b870 comp rx=0 tx=0).stop 2024-04-20T11:30:00.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 shutdown_connections 2024-04-20T11:30:00.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fada806ecd0 0x7fada8071190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc01070b0 0x7fadc019d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 --2- 172.21.15.117:0/219168821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc010d0c0 0x7fadc019d5c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:00.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.525+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 >> 172.21.15.117:0/219168821 conn(0x7fadc0076830 msgr2=0x7fadc00789e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:00.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.526+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 shutdown_connections 2024-04-20T11:30:00.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:00.526+0000 7fadc6000700 1 -- 172.21.15.117:0/219168821 wait complete. 2024-04-20T11:30:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:00 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4042102284' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:30:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:00 smithi173 ceph-mon[74165]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:30:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:00 smithi173 ceph-mon[74165]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:01 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/219168821' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:30:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:01 smithi173 ceph-mon[74165]: pgmap v527: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:01.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:01 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/219168821' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:30:01.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:01 smithi117 ceph-mon[99036]: pgmap v527: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:03.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:03 smithi117 ceph-mon[99036]: pgmap v528: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:03.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:03 smithi173 ceph-mon[74165]: pgmap v528: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:05 smithi117 ceph-mon[99036]: pgmap v529: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:05 smithi173 ceph-mon[74165]: pgmap v529: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:07.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:07 smithi117 ceph-mon[99036]: pgmap v530: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:07 smithi173 ceph-mon[74165]: pgmap v530: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:09.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:09 smithi117 ceph-mon[99036]: pgmap v531: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:09.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:09 smithi173 ceph-mon[74165]: pgmap v531: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:11.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:11 smithi117 ceph-mon[99036]: pgmap v532: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:11 smithi173 ceph-mon[74165]: pgmap v532: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:30:13.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:30:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:14 smithi117 ceph-mon[99036]: pgmap v533: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:14.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:14 smithi173 ceph-mon[74165]: pgmap v533: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:15 smithi117 ceph-mon[99036]: pgmap v534: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:15.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:15 smithi173 ceph-mon[74165]: pgmap v534: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:17.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:17 smithi117 ceph-mon[99036]: pgmap v535: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:17 smithi173 ceph-mon[74165]: pgmap v535: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:19.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:19 smithi117 ceph-mon[99036]: pgmap v536: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:19.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:19 smithi173 ceph-mon[74165]: pgmap v536: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:21.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:21 smithi117 ceph-mon[99036]: pgmap v537: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:21 smithi173 ceph-mon[74165]: pgmap v537: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:23 smithi117 ceph-mon[99036]: pgmap v538: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:23.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:23 smithi173 ceph-mon[74165]: pgmap v538: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:25 smithi117 ceph-mon[99036]: pgmap v539: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:25.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:25 smithi173 ceph-mon[74165]: pgmap v539: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:27 smithi117 ceph-mon[99036]: pgmap v540: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:27 smithi173 ceph-mon[74165]: pgmap v540: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:30:28.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:30:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:29 smithi117 ceph-mon[99036]: pgmap v541: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:29.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:29 smithi173 ceph-mon[74165]: pgmap v541: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.697+0000 7fc78b800700 1 -- 172.21.15.117:0/3849849424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784108e10 msgr2=0x7fc784109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.697+0000 7fc78b800700 1 --2- 172.21.15.117:0/3849849424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784108e10 0x7fc784109290 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc774009910 tx=0x7fc774009c20 comp rx=0 tx=0).stop 2024-04-20T11:30:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.697+0000 7fc78b800700 1 -- 172.21.15.117:0/3849849424 shutdown_connections 2024-04-20T11:30:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.697+0000 7fc78b800700 1 --2- 172.21.15.117:0/3849849424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784108e10 0x7fc784109290 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.697+0000 7fc78b800700 1 --2- 172.21.15.117:0/3849849424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc784107520 0x7fc784107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.697+0000 7fc78b800700 1 -- 172.21.15.117:0/3849849424 >> 172.21.15.117:0/3849849424 conn(0x7fc7840768f0 msgr2=0x7fc784078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.698+0000 7fc78b800700 1 -- 172.21.15.117:0/3849849424 shutdown_connections 2024-04-20T11:30:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.698+0000 7fc78b800700 1 -- 172.21.15.117:0/3849849424 wait complete. 2024-04-20T11:30:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.699+0000 7fc78b800700 1 Processor -- start 2024-04-20T11:30:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.699+0000 7fc78b800700 1 -- start start 2024-04-20T11:30:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc78b800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784107520 0x7fc78419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc78b800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc784108e10 0x7fc78419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc78b800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78419ddf0 con 0x7fc784107520 2024-04-20T11:30:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc78b800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841a1a70 con 0x7fc784108e10 2024-04-20T11:30:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc788e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784107520 0x7fc78419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc788e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784107520 0x7fc78419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50576/0 (socket says 172.21.15.117:50576) 2024-04-20T11:30:30.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc788e00700 1 -- 172.21.15.117:0/3335117543 learned_addr learned my addr 172.21.15.117:0/3335117543 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:30:30.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc77be00700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc784108e10 0x7fc78419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:30.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc788e00700 1 -- 172.21.15.117:0/3335117543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc784108e10 msgr2=0x7fc78419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc788e00700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc784108e10 0x7fc78419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.700+0000 7fc788e00700 1 -- 172.21.15.117:0/3335117543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7740095f0 con 0x7fc784107520 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.701+0000 7fc788e00700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784107520 0x7fc78419d1d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c8a0 tx=0x7fc76c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.701+0000 7fc782000700 1 -- 172.21.15.117:0/3335117543 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76c00cea0 con 0x7fc784107520 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.701+0000 7fc782000700 1 -- 172.21.15.117:0/3335117543 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc76c004830 con 0x7fc784107520 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.701+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7841a1d50 con 0x7fc784107520 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.701+0000 7fc782000700 1 -- 172.21.15.117:0/3335117543 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76c0055b0 con 0x7fc784107520 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.701+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7841a2320 con 0x7fc784107520 2024-04-20T11:30:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.702+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78410f870 con 0x7fc784107520 2024-04-20T11:30:30.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.703+0000 7fc782000700 1 -- 172.21.15.117:0/3335117543 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc76c0049a0 con 0x7fc784107520 2024-04-20T11:30:30.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.704+0000 7fc782000700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc76406ecd0 0x7fc764071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:30.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.704+0000 7fc782000700 1 -- 172.21.15.117:0/3335117543 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc76c08eac0 con 0x7fc784107520 2024-04-20T11:30:30.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.704+0000 7fc77be00700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc76406ecd0 0x7fc764071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:30.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.705+0000 7fc77be00700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc76406ecd0 0x7fc764071190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc77400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.709+0000 7fc782000700 1 -- 172.21.15.117:0/3335117543 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc76c057880 con 0x7fc784107520 2024-04-20T11:30:30.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.969+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7841144c0 con 0x7fc76406ecd0 2024-04-20T11:30:30.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.971+0000 7fc782000700 1 -- 172.21.15.117:0/3335117543 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc7841144c0 con 0x7fc76406ecd0 2024-04-20T11:30:30.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.973+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc76406ecd0 msgr2=0x7fc764071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:30.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.973+0000 7fc78b800700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc76406ecd0 0x7fc764071190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc77400b540 comp rx=0 tx=0).stop 2024-04-20T11:30:30.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784107520 msgr2=0x7fc78419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:30.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784107520 0x7fc78419d1d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c8a0 tx=0x7fc76c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:30:30.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 shutdown_connections 2024-04-20T11:30:30.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc76406ecd0 0x7fc764071190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:30.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc784107520 0x7fc78419d1d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:30.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 --2- 172.21.15.117:0/3335117543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc784108e10 0x7fc78419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:30.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 >> 172.21.15.117:0/3335117543 conn(0x7fc7840768f0 msgr2=0x7fc784077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:30.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 shutdown_connections 2024-04-20T11:30:30.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:30.974+0000 7fc78b800700 1 -- 172.21.15.117:0/3335117543 wait complete. 2024-04-20T11:30:30.994 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:30:31.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.137+0000 7f3762e00700 1 -- 172.21.15.117:0/1439104823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c1024f0 msgr2=0x7f375c10a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.137+0000 7f3762e00700 1 --2- 172.21.15.117:0/1439104823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c1024f0 0x7f375c10a9e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3748009910 tx=0x7f3748009c20 comp rx=0 tx=0).stop 2024-04-20T11:30:31.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.137+0000 7f3762e00700 1 -- 172.21.15.117:0/1439104823 shutdown_connections 2024-04-20T11:30:31.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.137+0000 7f3762e00700 1 --2- 172.21.15.117:0/1439104823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c1024f0 0x7f375c10a9e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.137+0000 7f3762e00700 1 --2- 172.21.15.117:0/1439104823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f375c101bd0 0x7f375c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.137+0000 7f3762e00700 1 -- 172.21.15.117:0/1439104823 >> 172.21.15.117:0/1439104823 conn(0x7f375c0fb420 msgr2=0x7f375c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:31.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.137+0000 7f3762e00700 1 -- 172.21.15.117:0/1439104823 shutdown_connections 2024-04-20T11:30:31.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.137+0000 7f3762e00700 1 -- 172.21.15.117:0/1439104823 wait complete. 2024-04-20T11:30:31.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.138+0000 7f3762e00700 1 Processor -- start 2024-04-20T11:30:31.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.138+0000 7f3762e00700 1 -- start start 2024-04-20T11:30:31.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3762e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c101bd0 0x7f375c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3762e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f375c1024f0 0x7f375c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3762e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c19d1d0 con 0x7f375c101bd0 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3762e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1967d0 con 0x7f375c1024f0 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3761a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c101bd0 0x7f375c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3761a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c101bd0 0x7f375c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50580/0 (socket says 172.21.15.117:50580) 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3761a00700 1 -- 172.21.15.117:0/1931206378 learned_addr learned my addr 172.21.15.117:0/1931206378 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3761000700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f375c1024f0 0x7f375c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3761a00700 1 -- 172.21.15.117:0/1931206378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f375c1024f0 msgr2=0x7f375c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3761a00700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f375c1024f0 0x7f375c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3761a00700 1 -- 172.21.15.117:0/1931206378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480095f0 con 0x7f375c101bd0 2024-04-20T11:30:31.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.139+0000 7f3761000700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f375c1024f0 0x7f375c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:31.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.140+0000 7f3761a00700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c101bd0 0x7f375c19c690 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f375000c930 tx=0x7f375000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:31.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.140+0000 7f375a000700 1 -- 172.21.15.117:0/1931206378 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3750007ad0 con 0x7f375c101bd0 2024-04-20T11:30:31.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.140+0000 7f375a000700 1 -- 172.21.15.117:0/1931206378 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3750007c30 con 0x7f375c101bd0 2024-04-20T11:30:31.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.140+0000 7f375a000700 1 -- 172.21.15.117:0/1931206378 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3750018610 con 0x7f375c101bd0 2024-04-20T11:30:31.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.140+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c196ab0 con 0x7f375c101bd0 2024-04-20T11:30:31.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.140+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c196fb0 con 0x7f375c101bd0 2024-04-20T11:30:31.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.141+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c1052f0 con 0x7f375c101bd0 2024-04-20T11:30:31.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.142+0000 7f375a000700 1 -- 172.21.15.117:0/1931206378 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f375000f450 con 0x7f375c101bd0 2024-04-20T11:30:31.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.143+0000 7f375a000700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f374406ecd0 0x7f3744071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.143+0000 7f375a000700 1 -- 172.21.15.117:0/1931206378 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f375008f0f0 con 0x7f375c101bd0 2024-04-20T11:30:31.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.144+0000 7f3761000700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f374406ecd0 0x7f3744071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:31.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.144+0000 7f3761000700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f374406ecd0 0x7f3744071190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3748006010 tx=0x7f374800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:31.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.148+0000 7f375a000700 1 -- 172.21.15.117:0/1931206378 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f375005b960 con 0x7f375c101bd0 2024-04-20T11:30:31.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.396+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f375c197c90 con 0x7f374406ecd0 2024-04-20T11:30:31.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.397+0000 7f375a000700 1 -- 172.21.15.117:0/1931206378 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f375c197c90 con 0x7f374406ecd0 2024-04-20T11:30:31.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.399+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f374406ecd0 msgr2=0x7f3744071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.399+0000 7f3762e00700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f374406ecd0 0x7f3744071190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3748006010 tx=0x7f374800b540 comp rx=0 tx=0).stop 2024-04-20T11:30:31.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c101bd0 msgr2=0x7f375c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c101bd0 0x7f375c19c690 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f375000c930 tx=0x7f375000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 shutdown_connections 2024-04-20T11:30:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f374406ecd0 0x7f3744071190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f375c101bd0 0x7f375c19c690 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 --2- 172.21.15.117:0/1931206378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f375c1024f0 0x7f375c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 >> 172.21.15.117:0/1931206378 conn(0x7f375c0fb420 msgr2=0x7f375c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 shutdown_connections 2024-04-20T11:30:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.400+0000 7f3762e00700 1 -- 172.21.15.117:0/1931206378 wait complete. 2024-04-20T11:30:31.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.547+0000 7f93b8600700 1 -- 172.21.15.117:0/1312718944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 msgr2=0x7f93b0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.547+0000 7f93b8600700 1 --2- 172.21.15.117:0/1312718944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 0x7f93b0104e80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f939c009910 tx=0x7f939c009c20 comp rx=0 tx=0).stop 2024-04-20T11:30:31.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.548+0000 7f93b8600700 1 -- 172.21.15.117:0/1312718944 shutdown_connections 2024-04-20T11:30:31.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.548+0000 7f93b8600700 1 --2- 172.21.15.117:0/1312718944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 0x7f93b0104e80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.548+0000 7f93b8600700 1 --2- 172.21.15.117:0/1312718944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93b0103110 0x7f93b01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.548+0000 7f93b8600700 1 -- 172.21.15.117:0/1312718944 >> 172.21.15.117:0/1312718944 conn(0x7f93b00fe880 msgr2=0x7f93b0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:31.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.548+0000 7f93b8600700 1 -- 172.21.15.117:0/1312718944 shutdown_connections 2024-04-20T11:30:31.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.548+0000 7f93b8600700 1 -- 172.21.15.117:0/1312718944 wait complete. 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.549+0000 7f93b8600700 1 Processor -- start 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.549+0000 7f93b8600700 1 -- start start 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.549+0000 7f93b8600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93b0103110 0x7f93b0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.549+0000 7f93b8600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 0x7f93b01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.549+0000 7f93b8600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0199ac0 con 0x7f93b0104a00 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.549+0000 7f93b8600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b019d740 con 0x7f93b0103110 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.550+0000 7f93b5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 0x7f93b01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.550+0000 7f93b5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 0x7f93b01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50600/0 (socket says 172.21.15.117:50600) 2024-04-20T11:30:31.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.550+0000 7f93b5200700 1 -- 172.21.15.117:0/860012933 learned_addr learned my addr 172.21.15.117:0/860012933 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:30:31.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.550+0000 7f93b5200700 1 -- 172.21.15.117:0/860012933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93b0103110 msgr2=0x7f93b0198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.550+0000 7f93b5200700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93b0103110 0x7f93b0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.550+0000 7f93b5200700 1 -- 172.21.15.117:0/860012933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c0095f0 con 0x7f93b0104a00 2024-04-20T11:30:31.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.550+0000 7f93b5200700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 0x7f93b01993e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f939c00bd70 tx=0x7f939c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:31.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.550+0000 7f93ae000700 1 -- 172.21.15.117:0/860012933 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f939c01d070 con 0x7f93b0104a00 2024-04-20T11:30:31.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.551+0000 7f93ae000700 1 -- 172.21.15.117:0/860012933 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f939c00f460 con 0x7f93b0104a00 2024-04-20T11:30:31.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.551+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b019d9c0 con 0x7f93b0104a00 2024-04-20T11:30:31.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.551+0000 7f93ae000700 1 -- 172.21.15.117:0/860012933 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f939c017630 con 0x7f93b0104a00 2024-04-20T11:30:31.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.551+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b019df30 con 0x7f93b0104a00 2024-04-20T11:30:31.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.552+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b010b480 con 0x7f93b0104a00 2024-04-20T11:30:31.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.553+0000 7f93ae000700 1 -- 172.21.15.117:0/860012933 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f939c00f5d0 con 0x7f93b0104a00 2024-04-20T11:30:31.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.554+0000 7f93ae000700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f939806ecc0 0x7f9398071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.554+0000 7f93ae000700 1 -- 172.21.15.117:0/860012933 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f939c0903e0 con 0x7f93b0104a00 2024-04-20T11:30:31.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.554+0000 7f93b5c00700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f939806ecc0 0x7f9398071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:31.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.555+0000 7f93b5c00700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f939806ecc0 0x7f9398071180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93a40097b0 tx=0x7f93a4006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:31.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.558+0000 7f93ae000700 1 -- 172.21.15.117:0/860012933 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f939c059120 con 0x7f93b0104a00 2024-04-20T11:30:31.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:31 smithi117 ceph-mon[99036]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:31.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:31 smithi117 ceph-mon[99036]: pgmap v542: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:31 smithi173 ceph-mon[74165]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:31 smithi173 ceph-mon[74165]: pgmap v542: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:31.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.783+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f93b019a160 con 0x7f939806ecc0 2024-04-20T11:30:31.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.801+0000 7f93ae000700 1 -- 172.21.15.117:0/860012933 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f93b019a160 con 0x7f939806ecc0 2024-04-20T11:30:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:30:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (18m) 10m ago 33m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:30:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (33m) 10m ago 33m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:30:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (32m) 10m ago 32m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:30:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (15m) 10m ago 33m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:30:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (15m) 10m ago 31m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (18m) 10m ago 32m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (10m) 10m ago 25m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (11m) 10m ago 25m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (25m) 10m ago 25m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (10m) 10m ago 25m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (22m) 10m ago 34m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (20m) 10m ago 31m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (16m) 10m ago 34m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (16m) 10m ago 31m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (20m) 10m ago 33m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (20m) 10m ago 31m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (14m) 10m ago 30m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:30:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (14m) 10m ago 30m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:30:31.803 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (13m) 10m ago 29m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:30:31.803 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (13m) 10m ago 28m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:30:31.803 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (12m) 10m ago 28m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:30:31.803 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (11m) 10m ago 27m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:30:31.803 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (19m) 10m ago 32m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f939806ecc0 msgr2=0x7f9398071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f939806ecc0 0x7f9398071180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93a40097b0 tx=0x7f93a4006d20 comp rx=0 tx=0).stop 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 msgr2=0x7f93b01993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 0x7f93b01993e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f939c00bd70 tx=0x7f939c00be50 comp rx=0 tx=0).stop 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 shutdown_connections 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f939806ecc0 0x7f9398071180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93b0103110 0x7f93b0198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 --2- 172.21.15.117:0/860012933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b0104a00 0x7f93b01993e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.804+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 >> 172.21.15.117:0/860012933 conn(0x7f93b00fe880 msgr2=0x7f93b00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.805+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 shutdown_connections 2024-04-20T11:30:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.805+0000 7f93b8600700 1 -- 172.21.15.117:0/860012933 wait complete. 2024-04-20T11:30:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.957+0000 7f6aece00700 1 -- 172.21.15.117:0/1182220969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 msgr2=0x7f6ae8103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.957+0000 7f6aece00700 1 --2- 172.21.15.117:0/1182220969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 0x7f6ae8103240 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009910 tx=0x7f6ad8009c20 comp rx=0 tx=0).stop 2024-04-20T11:30:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.957+0000 7f6aece00700 1 -- 172.21.15.117:0/1182220969 shutdown_connections 2024-04-20T11:30:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.957+0000 7f6aece00700 1 --2- 172.21.15.117:0/1182220969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ae8103810 0x7f6ae8107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.957+0000 7f6aece00700 1 --2- 172.21.15.117:0/1182220969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 0x7f6ae8103240 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.957+0000 7f6aece00700 1 -- 172.21.15.117:0/1182220969 >> 172.21.15.117:0/1182220969 conn(0x7f6ae80fe710 msgr2=0x7f6ae8100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.957+0000 7f6aece00700 1 -- 172.21.15.117:0/1182220969 shutdown_connections 2024-04-20T11:30:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.958+0000 7f6aece00700 1 -- 172.21.15.117:0/1182220969 wait complete. 2024-04-20T11:30:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.958+0000 7f6aece00700 1 Processor -- start 2024-04-20T11:30:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.959+0000 7f6aece00700 1 -- start start 2024-04-20T11:30:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.959+0000 7f6aece00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 0x7f6ae819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.959+0000 7f6aece00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ae8103810 0x7f6ae819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.959+0000 7f6aece00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae819f3a0 con 0x7f6ae8102e60 2024-04-20T11:30:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.959+0000 7f6aece00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae81989d0 con 0x7f6ae8103810 2024-04-20T11:30:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.959+0000 7f6ae7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 0x7f6ae819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.959+0000 7f6ae7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 0x7f6ae819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50614/0 (socket says 172.21.15.117:50614) 2024-04-20T11:30:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.959+0000 7f6ae7400700 1 -- 172.21.15.117:0/2977532232 learned_addr learned my addr 172.21.15.117:0/2977532232 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6ae7400700 1 -- 172.21.15.117:0/2977532232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ae8103810 msgr2=0x7f6ae819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6ae7400700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ae8103810 0x7f6ae819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6ae7400700 1 -- 172.21.15.117:0/2977532232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad80095f0 con 0x7f6ae8102e60 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6ae7400700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 0x7f6ae819e840 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800bd70 tx=0x7f6ad800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6adfe00700 1 -- 172.21.15.117:0/2977532232 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ad801d070 con 0x7f6ae8102e60 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6adfe00700 1 -- 172.21.15.117:0/2977532232 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ad80056e0 con 0x7f6ae8102e60 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae8198c50 con 0x7f6ae8102e60 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6adfe00700 1 -- 172.21.15.117:0/2977532232 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ad8017400 con 0x7f6ae8102e60 2024-04-20T11:30:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.960+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae8199140 con 0x7f6ae8102e60 2024-04-20T11:30:31.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.961+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae810b200 con 0x7f6ae8102e60 2024-04-20T11:30:31.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.962+0000 7f6adfe00700 1 -- 172.21.15.117:0/2977532232 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6ad8003a60 con 0x7f6ae8102e60 2024-04-20T11:30:31.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.963+0000 7f6adfe00700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6acc06ecc0 0x7f6acc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:31.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.963+0000 7f6adfe00700 1 -- 172.21.15.117:0/2977532232 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6ad80902a0 con 0x7f6ae8102e60 2024-04-20T11:30:31.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.963+0000 7f6ae6a00700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6acc06ecc0 0x7f6acc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:31.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.964+0000 7f6ae6a00700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6acc06ecc0 0x7f6acc071180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4005950 tx=0x7f6ad40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:31.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:31.968+0000 7f6adfe00700 1 -- 172.21.15.117:0/2977532232 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6ad805cb90 con 0x7f6ae8102e60 2024-04-20T11:30:32.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.281+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ae8199e40 con 0x7f6ae8102e60 2024-04-20T11:30:32.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.282+0000 7f6adfe00700 1 -- 172.21.15.117:0/2977532232 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f6ad805c720 con 0x7f6ae8102e60 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:30:32.282 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:30:32.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:30:32.283 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:30:32.283 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:30:32.283 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:30:32.283 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:30:32.283 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:30:32.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6acc06ecc0 msgr2=0x7f6acc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6acc06ecc0 0x7f6acc071180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4005950 tx=0x7f6ad40058e0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 msgr2=0x7f6ae819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 0x7f6ae819e840 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800bd70 tx=0x7f6ad800be50 comp rx=0 tx=0).stop 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 shutdown_connections 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6acc06ecc0 0x7f6acc071180 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ae8102e60 0x7f6ae819e840 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 --2- 172.21.15.117:0/2977532232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ae8103810 0x7f6ae819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 >> 172.21.15.117:0/2977532232 conn(0x7f6ae80fe710 msgr2=0x7f6ae80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 shutdown_connections 2024-04-20T11:30:32.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.285+0000 7f6aece00700 1 -- 172.21.15.117:0/2977532232 wait complete. 2024-04-20T11:30:32.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.432+0000 7fb338a00700 1 -- 172.21.15.117:0/4269950953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 msgr2=0x7fb334107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:32.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.432+0000 7fb338a00700 1 --2- 172.21.15.117:0/4269950953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 0x7fb334107900 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb324009910 tx=0x7fb324009c20 comp rx=0 tx=0).stop 2024-04-20T11:30:32.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.432+0000 7fb338a00700 1 -- 172.21.15.117:0/4269950953 shutdown_connections 2024-04-20T11:30:32.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.432+0000 7fb338a00700 1 --2- 172.21.15.117:0/4269950953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb334108e10 0x7fb334109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.432+0000 7fb338a00700 1 --2- 172.21.15.117:0/4269950953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 0x7fb334107900 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.432+0000 7fb338a00700 1 -- 172.21.15.117:0/4269950953 >> 172.21.15.117:0/4269950953 conn(0x7fb3340768f0 msgr2=0x7fb334078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:32.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.432+0000 7fb338a00700 1 -- 172.21.15.117:0/4269950953 shutdown_connections 2024-04-20T11:30:32.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.432+0000 7fb338a00700 1 -- 172.21.15.117:0/4269950953 wait complete. 2024-04-20T11:30:32.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.433+0000 7fb338a00700 1 Processor -- start 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.433+0000 7fb338a00700 1 -- start start 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb338a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 0x7fb33419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb338a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb334108e10 0x7fb33419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb338a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33419de90 con 0x7fb334107520 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb338a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3341a1af0 con 0x7fb334108e10 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb331e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 0x7fb33419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb331e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 0x7fb33419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50626/0 (socket says 172.21.15.117:50626) 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb331e00700 1 -- 172.21.15.117:0/616103702 learned_addr learned my addr 172.21.15.117:0/616103702 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb331e00700 1 -- 172.21.15.117:0/616103702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb334108e10 msgr2=0x7fb33419d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb331e00700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb334108e10 0x7fb33419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.434+0000 7fb331e00700 1 -- 172.21.15.117:0/616103702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240095f0 con 0x7fb334107520 2024-04-20T11:30:32.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.435+0000 7fb331e00700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 0x7fb33419d270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb324006010 tx=0x7fb32400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:32.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.435+0000 7fb32aa00700 1 -- 172.21.15.117:0/616103702 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb32401d070 con 0x7fb334107520 2024-04-20T11:30:32.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.435+0000 7fb32aa00700 1 -- 172.21.15.117:0/616103702 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb32400f460 con 0x7fb334107520 2024-04-20T11:30:32.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.435+0000 7fb32aa00700 1 -- 172.21.15.117:0/616103702 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb324017630 con 0x7fb334107520 2024-04-20T11:30:32.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.435+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3341a1d70 con 0x7fb334107520 2024-04-20T11:30:32.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.435+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3341a22e0 con 0x7fb334107520 2024-04-20T11:30:32.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.436+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33410f890 con 0x7fb334107520 2024-04-20T11:30:32.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.437+0000 7fb32aa00700 1 -- 172.21.15.117:0/616103702 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb32400f5d0 con 0x7fb334107520 2024-04-20T11:30:32.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.438+0000 7fb32aa00700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb31806ecd0 0x7fb318071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:32.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.438+0000 7fb32aa00700 1 -- 172.21.15.117:0/616103702 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb324090270 con 0x7fb334107520 2024-04-20T11:30:32.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.438+0000 7fb331400700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb31806ecd0 0x7fb318071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:32.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.439+0000 7fb331400700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb31806ecd0 0x7fb318071190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb334077ef0 tx=0x7fb320006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:32.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.443+0000 7fb32aa00700 1 -- 172.21.15.117:0/616103702 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb324059190 con 0x7fb334107520 2024-04-20T11:30:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:32 smithi117 ceph-mon[99036]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:32 smithi117 ceph-mon[99036]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:32 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2977532232' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:30:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:32 smithi173 ceph-mon[74165]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:32 smithi173 ceph-mon[74165]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:32 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2977532232' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:30:32.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.699+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb33419e580 con 0x7fb334107520 2024-04-20T11:30:32.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.699+0000 7fb32aa00700 1 -- 172.21.15.117:0/616103702 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fb32405c850 con 0x7fb334107520 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:30:32.700 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:30:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:30:32.702 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:30:32.703 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:30:32.703 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:30:32.703 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:30:32.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb31806ecd0 msgr2=0x7fb318071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb31806ecd0 0x7fb318071190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb334077ef0 tx=0x7fb320006d20 comp rx=0 tx=0).stop 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 msgr2=0x7fb33419d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 0x7fb33419d270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb324006010 tx=0x7fb32400be20 comp rx=0 tx=0).stop 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 shutdown_connections 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb31806ecd0 0x7fb318071190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb334107520 0x7fb33419d270 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 --2- 172.21.15.117:0/616103702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb334108e10 0x7fb33419d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 >> 172.21.15.117:0/616103702 conn(0x7fb3340768f0 msgr2=0x7fb334078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 shutdown_connections 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.702+0000 7fb338a00700 1 -- 172.21.15.117:0/616103702 wait complete. 2024-04-20T11:30:32.704 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:30:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.856+0000 7f4309800700 1 -- 172.21.15.117:0/3971041404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 msgr2=0x7f4304107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.856+0000 7f4309800700 1 --2- 172.21.15.117:0/3971041404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 0x7f4304107900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009910 tx=0x7f42f4009c20 comp rx=0 tx=0).stop 2024-04-20T11:30:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.856+0000 7f4309800700 1 -- 172.21.15.117:0/3971041404 shutdown_connections 2024-04-20T11:30:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.856+0000 7f4309800700 1 --2- 172.21.15.117:0/3971041404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4304108e10 0x7f4304109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.856+0000 7f4309800700 1 --2- 172.21.15.117:0/3971041404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 0x7f4304107900 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.856+0000 7f4309800700 1 -- 172.21.15.117:0/3971041404 >> 172.21.15.117:0/3971041404 conn(0x7f43040768f0 msgr2=0x7f4304078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.856+0000 7f4309800700 1 -- 172.21.15.117:0/3971041404 shutdown_connections 2024-04-20T11:30:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.856+0000 7f4309800700 1 -- 172.21.15.117:0/3971041404 wait complete. 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.857+0000 7f4309800700 1 Processor -- start 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.857+0000 7f4309800700 1 -- start start 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.858+0000 7f4309800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 0x7f430419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.858+0000 7f4309800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4304108e10 0x7f430419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.858+0000 7f4309800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430419ddf0 con 0x7f4304107520 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.858+0000 7f4309800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43041a1ac0 con 0x7f4304108e10 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.858+0000 7f4302a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 0x7f430419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.858+0000 7f4302a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 0x7f430419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50650/0 (socket says 172.21.15.117:50650) 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.858+0000 7f4302a00700 1 -- 172.21.15.117:0/2641404580 learned_addr learned my addr 172.21.15.117:0/2641404580 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.858+0000 7f4302000700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4304108e10 0x7f430419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f4302a00700 1 -- 172.21.15.117:0/2641404580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4304108e10 msgr2=0x7f430419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f4302a00700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4304108e10 0x7f430419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f4302a00700 1 -- 172.21.15.117:0/2641404580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40095f0 con 0x7f4304107520 2024-04-20T11:30:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f4302a00700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 0x7f430419d1d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f42f4000c00 tx=0x7f42f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f42fb400700 1 -- 172.21.15.117:0/2641404580 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f401d070 con 0x7f4304107520 2024-04-20T11:30:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f42fb400700 1 -- 172.21.15.117:0/2641404580 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42f40056e0 con 0x7f4304107520 2024-04-20T11:30:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f42fb400700 1 -- 172.21.15.117:0/2641404580 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f400fe80 con 0x7f4304107520 2024-04-20T11:30:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43041a1d40 con 0x7f4304107520 2024-04-20T11:30:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.859+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43041a2260 con 0x7f4304107520 2024-04-20T11:30:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.861+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430410f890 con 0x7f4304107520 2024-04-20T11:30:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.862+0000 7f42fb400700 1 -- 172.21.15.117:0/2641404580 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f42f400f6f0 con 0x7f4304107520 2024-04-20T11:30:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.863+0000 7f42fb400700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42e806ecd0 0x7f42e8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.863+0000 7f42fb400700 1 -- 172.21.15.117:0/2641404580 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f42f4090240 con 0x7f4304107520 2024-04-20T11:30:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.863+0000 7f4302000700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42e806ecd0 0x7f42e8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:32.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.864+0000 7f4302000700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42e806ecd0 0x7f42e8071190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f42f0005950 tx=0x7f42f00058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:32.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:32.868+0000 7f42fb400700 1 -- 172.21.15.117:0/2641404580 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f42f4059020 con 0x7f4304107520 2024-04-20T11:30:33.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.112+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f430419e4e0 con 0x7f42e806ecd0 2024-04-20T11:30:33.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.113+0000 7f42fb400700 1 -- 172.21.15.117:0/2641404580 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f430419e4e0 con 0x7f42e806ecd0 2024-04-20T11:30:33.113 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:30:33.113 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:30:33.113 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:30:33.113 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:30:33.113 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:30:33.113 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:30:33.113 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:30:33.114 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:30:33.114 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:30:33.114 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:30:33.114 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:30:33.114 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:30:33.114 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:30:33.114 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:30:33.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42e806ecd0 msgr2=0x7f42e8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:33.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42e806ecd0 0x7f42e8071190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f42f0005950 tx=0x7f42f00058e0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 msgr2=0x7f430419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:33.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 0x7f430419d1d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f42f4000c00 tx=0x7f42f400be20 comp rx=0 tx=0).stop 2024-04-20T11:30:33.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 shutdown_connections 2024-04-20T11:30:33.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42e806ecd0 0x7f42e8071190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4304107520 0x7f430419d1d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 --2- 172.21.15.117:0/2641404580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4304108e10 0x7f430419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 >> 172.21.15.117:0/2641404580 conn(0x7f43040768f0 msgr2=0x7f4304077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:33.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.116+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 shutdown_connections 2024-04-20T11:30:33.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.117+0000 7f4309800700 1 -- 172.21.15.117:0/2641404580 wait complete. 2024-04-20T11:30:33.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.267+0000 7fc0dd200700 1 -- 172.21.15.117:0/1422569402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d8100e10 msgr2=0x7fc0d81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:33.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.267+0000 7fc0dd200700 1 --2- 172.21.15.117:0/1422569402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d8100e10 0x7fc0d81011f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009910 tx=0x7fc0c8009c20 comp rx=0 tx=0).stop 2024-04-20T11:30:33.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.267+0000 7fc0dd200700 1 -- 172.21.15.117:0/1422569402 shutdown_connections 2024-04-20T11:30:33.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.267+0000 7fc0dd200700 1 --2- 172.21.15.117:0/1422569402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d81017c0 0x7fc0d8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.267+0000 7fc0dd200700 1 --2- 172.21.15.117:0/1422569402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d8100e10 0x7fc0d81011f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.267+0000 7fc0dd200700 1 -- 172.21.15.117:0/1422569402 >> 172.21.15.117:0/1422569402 conn(0x7fc0d80fc6c0 msgr2=0x7fc0d80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:33.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.267+0000 7fc0dd200700 1 -- 172.21.15.117:0/1422569402 shutdown_connections 2024-04-20T11:30:33.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.267+0000 7fc0dd200700 1 -- 172.21.15.117:0/1422569402 wait complete. 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.268+0000 7fc0dd200700 1 Processor -- start 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.268+0000 7fc0dd200700 1 -- start start 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.268+0000 7fc0dd200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d8100e10 0x7fc0d8198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.268+0000 7fc0dd200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d81017c0 0x7fc0d8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.268+0000 7fc0dd200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8199af0 con 0x7fc0d81017c0 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.268+0000 7fc0dd200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d819d7c0 con 0x7fc0d8100e10 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.269+0000 7fc0cd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d81017c0 0x7fc0d8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.269+0000 7fc0cd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d81017c0 0x7fc0d8199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50674/0 (socket says 172.21.15.117:50674) 2024-04-20T11:30:33.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.269+0000 7fc0cd400700 1 -- 172.21.15.117:0/1641818338 learned_addr learned my addr 172.21.15.117:0/1641818338 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.269+0000 7fc0d5e00700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d8100e10 0x7fc0d8198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.269+0000 7fc0cd400700 1 -- 172.21.15.117:0/1641818338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d8100e10 msgr2=0x7fc0d8198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.269+0000 7fc0cd400700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d8100e10 0x7fc0d8198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.269+0000 7fc0cd400700 1 -- 172.21.15.117:0/1641818338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c80095f0 con 0x7fc0d81017c0 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.269+0000 7fc0d5e00700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d8100e10 0x7fc0d8198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.270+0000 7fc0cd400700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d81017c0 0x7fc0d8199410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000b480 tx=0x7fc0c000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.270+0000 7fc0cf400700 1 -- 172.21.15.117:0/1641818338 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0c00118c0 con 0x7fc0d81017c0 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.270+0000 7fc0cf400700 1 -- 172.21.15.117:0/1641818338 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0c0010380 con 0x7fc0d81017c0 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.270+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d819daa0 con 0x7fc0d81017c0 2024-04-20T11:30:33.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.270+0000 7fc0cf400700 1 -- 172.21.15.117:0/1641818338 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0c000f4f0 con 0x7fc0d81017c0 2024-04-20T11:30:33.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.270+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d819dff0 con 0x7fc0d81017c0 2024-04-20T11:30:33.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.271+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d81022f0 con 0x7fc0d81017c0 2024-04-20T11:30:33.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.273+0000 7fc0cf400700 1 -- 172.21.15.117:0/1641818338 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc0c00104f0 con 0x7fc0d81017c0 2024-04-20T11:30:33.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.273+0000 7fc0cf400700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0bc06ecc0 0x7fc0bc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:33.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.274+0000 7fc0cf400700 1 -- 172.21.15.117:0/1641818338 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc0c008eb00 con 0x7fc0d81017c0 2024-04-20T11:30:33.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.274+0000 7fc0d5e00700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0bc06ecc0 0x7fc0bc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:33.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.274+0000 7fc0d5e00700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0bc06ecc0 0x7fc0bc071180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8006010 tx=0x7fc0c800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:33.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.278+0000 7fc0cf400700 1 -- 172.21.15.117:0/1641818338 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc0c0057840 con 0x7fc0d81017c0 2024-04-20T11:30:33.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.590+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc0d810def0 con 0x7fc0d81017c0 2024-04-20T11:30:33.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.591+0000 7fc0cf400700 1 -- 172.21.15.117:0/1641818338 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fc0c005af00 con 0x7fc0d81017c0 2024-04-20T11:30:33.590 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:30:33.590 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:30:33.591 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:30:33.591 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:30:33.591 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:30:33.591 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:30:33.591 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:30:33.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.593+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0bc06ecc0 msgr2=0x7fc0bc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:33.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.593+0000 7fc0dd200700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0bc06ecc0 0x7fc0bc071180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8006010 tx=0x7fc0c800b540 comp rx=0 tx=0).stop 2024-04-20T11:30:33.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.593+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d81017c0 msgr2=0x7fc0d8199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:33.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.593+0000 7fc0dd200700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d81017c0 0x7fc0d8199410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000b480 tx=0x7fc0c000b840 comp rx=0 tx=0).stop 2024-04-20T11:30:33.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.594+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 shutdown_connections 2024-04-20T11:30:33.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.594+0000 7fc0dd200700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc0bc06ecc0 0x7fc0bc071180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.594+0000 7fc0dd200700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d8100e10 0x7fc0d8198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.594+0000 7fc0dd200700 1 --2- 172.21.15.117:0/1641818338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0d81017c0 0x7fc0d8199410 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:33.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.594+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 >> 172.21.15.117:0/1641818338 conn(0x7fc0d80fc6c0 msgr2=0x7fc0d8074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:33.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.594+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 shutdown_connections 2024-04-20T11:30:33.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:30:33.594+0000 7fc0dd200700 1 -- 172.21.15.117:0/1641818338 wait complete. 2024-04-20T11:30:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:33 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/616103702' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:30:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:33 smithi173 ceph-mon[74165]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:33 smithi173 ceph-mon[74165]: pgmap v543: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:33.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:33 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/616103702' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:30:33.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:33 smithi117 ceph-mon[99036]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:30:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:33 smithi117 ceph-mon[99036]: pgmap v543: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:34 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1641818338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:30:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:34 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1641818338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:30:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:35 smithi173 ceph-mon[74165]: pgmap v544: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:35.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:35 smithi117 ceph-mon[99036]: pgmap v544: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:37.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:37 smithi117 ceph-mon[99036]: pgmap v545: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:37.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:37 smithi173 ceph-mon[74165]: pgmap v545: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:39.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:39 smithi117 ceph-mon[99036]: pgmap v546: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:39.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:39 smithi173 ceph-mon[74165]: pgmap v546: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:41.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:41 smithi117 ceph-mon[99036]: pgmap v547: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:41.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:41 smithi173 ceph-mon[74165]: pgmap v547: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:43.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:30:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:30:44.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:44 smithi117 ceph-mon[99036]: pgmap v548: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:44.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:44 smithi173 ceph-mon[74165]: pgmap v548: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:45 smithi117 ceph-mon[99036]: pgmap v549: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:45.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:45 smithi173 ceph-mon[74165]: pgmap v549: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:47.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:47 smithi117 ceph-mon[99036]: pgmap v550: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:47 smithi173 ceph-mon[74165]: pgmap v550: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:49.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:49 smithi117 ceph-mon[99036]: pgmap v551: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:49 smithi173 ceph-mon[74165]: pgmap v551: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:51.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:51 smithi117 ceph-mon[99036]: pgmap v552: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:51 smithi173 ceph-mon[74165]: pgmap v552: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:53.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:53 smithi117 ceph-mon[99036]: pgmap v553: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:53 smithi173 ceph-mon[74165]: pgmap v553: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:55.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:55 smithi117 ceph-mon[99036]: pgmap v554: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:55 smithi173 ceph-mon[74165]: pgmap v554: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:57 smithi117 ceph-mon[99036]: pgmap v555: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:57 smithi173 ceph-mon[74165]: pgmap v555: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:58.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:30:58.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:30:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:30:59 smithi173 ceph-mon[74165]: pgmap v556: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:30:59.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:30:59 smithi117 ceph-mon[99036]: pgmap v556: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:01.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:01 smithi117 ceph-mon[99036]: pgmap v557: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:01 smithi173 ceph-mon[74165]: pgmap v557: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:03.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.762+0000 7fb7edc00700 1 -- 172.21.15.117:0/3288375075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8107520 msgr2=0x7fb7e8107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:03.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.762+0000 7fb7edc00700 1 --2- 172.21.15.117:0/3288375075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8107520 0x7fb7e8107900 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009910 tx=0x7fb7d8009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:03.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.762+0000 7fb7edc00700 1 -- 172.21.15.117:0/3288375075 shutdown_connections 2024-04-20T11:31:03.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.762+0000 7fb7edc00700 1 --2- 172.21.15.117:0/3288375075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e8108e10 0x7fb7e8109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.762+0000 7fb7edc00700 1 --2- 172.21.15.117:0/3288375075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8107520 0x7fb7e8107900 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.762+0000 7fb7edc00700 1 -- 172.21.15.117:0/3288375075 >> 172.21.15.117:0/3288375075 conn(0x7fb7e80768f0 msgr2=0x7fb7e8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:03.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.762+0000 7fb7edc00700 1 -- 172.21.15.117:0/3288375075 shutdown_connections 2024-04-20T11:31:03.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.762+0000 7fb7edc00700 1 -- 172.21.15.117:0/3288375075 wait complete. 2024-04-20T11:31:03.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.763+0000 7fb7edc00700 1 Processor -- start 2024-04-20T11:31:03.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.764+0000 7fb7edc00700 1 -- start start 2024-04-20T11:31:03.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.764+0000 7fb7edc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e8107520 0x7fb7e819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:03.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.764+0000 7fb7edc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8108e10 0x7fb7e819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:03.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.764+0000 7fb7edc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e819de90 con 0x7fb7e8108e10 2024-04-20T11:31:03.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.764+0000 7fb7edc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e81a1b60 con 0x7fb7e8107520 2024-04-20T11:31:03.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.764+0000 7fb7e6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8108e10 0x7fb7e819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.764+0000 7fb7e6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8108e10 0x7fb7e819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54980/0 (socket says 172.21.15.117:54980) 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.764+0000 7fb7e6000700 1 -- 172.21.15.117:0/3882911660 learned_addr learned my addr 172.21.15.117:0/3882911660 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7e6a00700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e8107520 0x7fb7e819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7e6000700 1 -- 172.21.15.117:0/3882911660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e8107520 msgr2=0x7fb7e819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7e6000700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e8107520 0x7fb7e819d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7e6000700 1 -- 172.21.15.117:0/3882911660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d80095f0 con 0x7fb7e8108e10 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7e6000700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8108e10 0x7fb7e819d7b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400b4b0 tx=0x7fb7d400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7df400700 1 -- 172.21.15.117:0/3882911660 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d4011800 con 0x7fb7e8108e10 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7df400700 1 -- 172.21.15.117:0/3882911660 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7d4011e20 con 0x7fb7e8108e10 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e81a1e40 con 0x7fb7e8108e10 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7df400700 1 -- 172.21.15.117:0/3882911660 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d400f2e0 con 0x7fb7e8108e10 2024-04-20T11:31:03.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.765+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e81a2410 con 0x7fb7e8108e10 2024-04-20T11:31:03.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.767+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e810f870 con 0x7fb7e8108e10 2024-04-20T11:31:03.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.768+0000 7fb7df400700 1 -- 172.21.15.117:0/3882911660 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb7d4010380 con 0x7fb7e8108e10 2024-04-20T11:31:03.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.769+0000 7fb7df400700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7cc06ecd0 0x7fb7cc071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:03.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.769+0000 7fb7df400700 1 -- 172.21.15.117:0/3882911660 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb7d408ea70 con 0x7fb7e8108e10 2024-04-20T11:31:03.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.769+0000 7fb7e6a00700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7cc06ecd0 0x7fb7cc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:03.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.770+0000 7fb7e6a00700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7cc06ecd0 0x7fb7cc071190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006010 tx=0x7fb7d800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:03.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:03.774+0000 7fb7df400700 1 -- 172.21.15.117:0/3882911660 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb7d40577b0 con 0x7fb7e8108e10 2024-04-20T11:31:04.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.022+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7e8114540 con 0x7fb7cc06ecd0 2024-04-20T11:31:04.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.023+0000 7fb7df400700 1 -- 172.21.15.117:0/3882911660 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb7e8114540 con 0x7fb7cc06ecd0 2024-04-20T11:31:04.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7cc06ecd0 msgr2=0x7fb7cc071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7cc06ecd0 0x7fb7cc071190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006010 tx=0x7fb7d800b540 comp rx=0 tx=0).stop 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8108e10 msgr2=0x7fb7e819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8108e10 0x7fb7e819d7b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400b4b0 tx=0x7fb7d400b870 comp rx=0 tx=0).stop 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 shutdown_connections 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7cc06ecd0 0x7fb7cc071190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e8107520 0x7fb7e819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 --2- 172.21.15.117:0/3882911660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e8108e10 0x7fb7e819d7b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 >> 172.21.15.117:0/3882911660 conn(0x7fb7e80768f0 msgr2=0x7fb7e8078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 shutdown_connections 2024-04-20T11:31:04.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.026+0000 7fb7edc00700 1 -- 172.21.15.117:0/3882911660 wait complete. 2024-04-20T11:31:04.042 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:31:04.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.184+0000 7f61b6800700 1 -- 172.21.15.117:0/2807691732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 msgr2=0x7f61b0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.184+0000 7f61b6800700 1 --2- 172.21.15.117:0/2807691732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 0x7f61b0104e80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f619c009910 tx=0x7f619c009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:04.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.184+0000 7f61b6800700 1 -- 172.21.15.117:0/2807691732 shutdown_connections 2024-04-20T11:31:04.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.184+0000 7f61b6800700 1 --2- 172.21.15.117:0/2807691732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 0x7f61b0104e80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.184+0000 7f61b6800700 1 --2- 172.21.15.117:0/2807691732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61b0103110 0x7f61b01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.184+0000 7f61b6800700 1 -- 172.21.15.117:0/2807691732 >> 172.21.15.117:0/2807691732 conn(0x7f61b00fe880 msgr2=0x7f61b0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.185+0000 7f61b6800700 1 -- 172.21.15.117:0/2807691732 shutdown_connections 2024-04-20T11:31:04.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.185+0000 7f61b6800700 1 -- 172.21.15.117:0/2807691732 wait complete. 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.186+0000 7f61b6800700 1 Processor -- start 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.186+0000 7f61b6800700 1 -- start start 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.186+0000 7f61b6800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61b0103110 0x7f61b0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.186+0000 7f61b6800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 0x7f61b01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.186+0000 7f61b6800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0199ac0 con 0x7f61b0104a00 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.186+0000 7f61b6800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b019d740 con 0x7f61b0103110 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.187+0000 7f61aea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 0x7f61b01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.187+0000 7f61aea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 0x7f61b01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57666/0 (socket says 172.21.15.117:57666) 2024-04-20T11:31:04.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.187+0000 7f61aea00700 1 -- 172.21.15.117:0/3206093513 learned_addr learned my addr 172.21.15.117:0/3206093513 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:04.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.187+0000 7f61aea00700 1 -- 172.21.15.117:0/3206093513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61b0103110 msgr2=0x7f61b0198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.187+0000 7f61aea00700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61b0103110 0x7f61b0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.187+0000 7f61aea00700 1 -- 172.21.15.117:0/3206093513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c0095f0 con 0x7f61b0104a00 2024-04-20T11:31:04.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.187+0000 7f61aea00700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 0x7f61b01993e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f619c0092e0 tx=0x7f619c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.188+0000 7f61a7e00700 1 -- 172.21.15.117:0/3206093513 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f619c01d070 con 0x7f61b0104a00 2024-04-20T11:31:04.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.188+0000 7f61a7e00700 1 -- 172.21.15.117:0/3206093513 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f619c00f460 con 0x7f61b0104a00 2024-04-20T11:31:04.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.188+0000 7f61a7e00700 1 -- 172.21.15.117:0/3206093513 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f619c017630 con 0x7f61b0104a00 2024-04-20T11:31:04.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.188+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b019d9c0 con 0x7f61b0104a00 2024-04-20T11:31:04.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.188+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b019df30 con 0x7f61b0104a00 2024-04-20T11:31:04.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.189+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b010b480 con 0x7f61b0104a00 2024-04-20T11:31:04.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.190+0000 7f61a7e00700 1 -- 172.21.15.117:0/3206093513 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f619c00f5d0 con 0x7f61b0104a00 2024-04-20T11:31:04.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.191+0000 7f61a7e00700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f619406ecd0 0x7f6194071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.191+0000 7f61a7e00700 1 -- 172.21.15.117:0/3206093513 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f619c090350 con 0x7f61b0104a00 2024-04-20T11:31:04.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.191+0000 7f61af400700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f619406ecd0 0x7f6194071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.192+0000 7f61af400700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f619406ecd0 0x7f6194071190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f61a00097b0 tx=0x7f61a0006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.197+0000 7f61a7e00700 1 -- 172.21.15.117:0/3206093513 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f619c059130 con 0x7f61b0104a00 2024-04-20T11:31:04.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.448+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f61b019a160 con 0x7f619406ecd0 2024-04-20T11:31:04.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.451+0000 7f61a7e00700 1 -- 172.21.15.117:0/3206093513 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f61b019a160 con 0x7f619406ecd0 2024-04-20T11:31:04.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.455+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f619406ecd0 msgr2=0x7f6194071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.455+0000 7f61b6800700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f619406ecd0 0x7f6194071190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f61a00097b0 tx=0x7f61a0006d20 comp rx=0 tx=0).stop 2024-04-20T11:31:04.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.455+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 msgr2=0x7f61b01993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.455+0000 7f61b6800700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 0x7f61b01993e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f619c0092e0 tx=0x7f619c00be20 comp rx=0 tx=0).stop 2024-04-20T11:31:04.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.459+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 shutdown_connections 2024-04-20T11:31:04.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.459+0000 7f61b6800700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f619406ecd0 0x7f6194071190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.459+0000 7f61b6800700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61b0103110 0x7f61b0198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.459+0000 7f61b6800700 1 --2- 172.21.15.117:0/3206093513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0104a00 0x7f61b01993e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.459+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 >> 172.21.15.117:0/3206093513 conn(0x7f61b00fe880 msgr2=0x7f61b00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.459+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 shutdown_connections 2024-04-20T11:31:04.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.459+0000 7f61b6800700 1 -- 172.21.15.117:0/3206093513 wait complete. 2024-04-20T11:31:04.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.610+0000 7f3ded200700 1 -- 172.21.15.117:0/4045113095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 msgr2=0x7f3de8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.610+0000 7f3ded200700 1 --2- 172.21.15.117:0/4045113095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 0x7f3de8107a40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009910 tx=0x7f3dd4009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:04.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.610+0000 7f3ded200700 1 -- 172.21.15.117:0/4045113095 shutdown_connections 2024-04-20T11:31:04.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.610+0000 7f3ded200700 1 --2- 172.21.15.117:0/4045113095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 0x7f3de8107a40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.610+0000 7f3ded200700 1 --2- 172.21.15.117:0/4045113095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3de8103040 0x7f3de8103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.610+0000 7f3ded200700 1 -- 172.21.15.117:0/4045113095 >> 172.21.15.117:0/4045113095 conn(0x7f3de80fe8d0 msgr2=0x7f3de8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.610+0000 7f3ded200700 1 -- 172.21.15.117:0/4045113095 shutdown_connections 2024-04-20T11:31:04.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.610+0000 7f3ded200700 1 -- 172.21.15.117:0/4045113095 wait complete. 2024-04-20T11:31:04.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.611+0000 7f3ded200700 1 Processor -- start 2024-04-20T11:31:04.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.611+0000 7f3ded200700 1 -- start start 2024-04-20T11:31:04.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.611+0000 7f3ded200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3de8103040 0x7f3de8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.611+0000 7f3ded200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 0x7f3de8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.611+0000 7f3ded200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8199a70 con 0x7f3de81039f0 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.611+0000 7f3ded200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de819d740 con 0x7f3de8103040 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3de5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 0x7f3de8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3de5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 0x7f3de8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57684/0 (socket says 172.21.15.117:57684) 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3de5400700 1 -- 172.21.15.117:0/2472106960 learned_addr learned my addr 172.21.15.117:0/2472106960 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3de5400700 1 -- 172.21.15.117:0/2472106960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3de8103040 msgr2=0x7f3de8198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3de5400700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3de8103040 0x7f3de8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3de5400700 1 -- 172.21.15.117:0/2472106960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40095f0 con 0x7f3de81039f0 2024-04-20T11:31:04.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3de5400700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 0x7f3de8199390 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4005b40 tx=0x7f3dd400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3ddea00700 1 -- 172.21.15.117:0/2472106960 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dd401d070 con 0x7f3de81039f0 2024-04-20T11:31:04.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3ddea00700 1 -- 172.21.15.117:0/2472106960 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3dd400f460 con 0x7f3de81039f0 2024-04-20T11:31:04.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.612+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de819d9c0 con 0x7f3de81039f0 2024-04-20T11:31:04.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.613+0000 7f3ddea00700 1 -- 172.21.15.117:0/2472106960 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dd4017630 con 0x7f3de81039f0 2024-04-20T11:31:04.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.613+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de819deb0 con 0x7f3de81039f0 2024-04-20T11:31:04.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.614+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de8104520 con 0x7f3de81039f0 2024-04-20T11:31:04.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.615+0000 7f3ddea00700 1 -- 172.21.15.117:0/2472106960 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3dd4003a60 con 0x7f3de81039f0 2024-04-20T11:31:04.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.616+0000 7f3ddea00700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3dcc06ecc0 0x7f3dcc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:04.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.616+0000 7f3ddea00700 1 -- 172.21.15.117:0/2472106960 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3dd40903e0 con 0x7f3de81039f0 2024-04-20T11:31:04.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.616+0000 7f3de5e00700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3dcc06ecc0 0x7f3dcc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:04.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.617+0000 7f3de5e00700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3dcc06ecc0 0x7f3dcc071180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80097b0 tx=0x7f3dd8006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:04.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.621+0000 7f3ddea00700 1 -- 172.21.15.117:0/2472106960 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3dd4059120 con 0x7f3de81039f0 2024-04-20T11:31:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:04 smithi117 ceph-mon[99036]: pgmap v558: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:04 smithi117 ceph-mon[99036]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:04 smithi173 ceph-mon[74165]: pgmap v558: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:04 smithi173 ceph-mon[74165]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:04.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.853+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3de8110060 con 0x7f3dcc06ecc0 2024-04-20T11:31:04.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.885+0000 7f3ddea00700 1 -- 172.21.15.117:0/2472106960 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3de8110060 con 0x7f3dcc06ecc0 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (19m) 11m ago 33m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (33m) 11m ago 33m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (32m) 10m ago 32m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (15m) 11m ago 33m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (15m) 10m ago 32m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (18m) 11m ago 33m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (11m) 11m ago 26m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (11m) 11m ago 26m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (26m) 10m ago 26m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:31:04.885 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (10m) 10m ago 26m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (22m) 11m ago 35m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (21m) 10m ago 32m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (17m) 11m ago 35m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (16m) 10m ago 32m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (20m) 11m ago 33m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (20m) 10m ago 32m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (15m) 11m ago 31m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (14m) 11m ago 30m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (14m) 11m ago 29m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (13m) 10m ago 29m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (12m) 10m ago 28m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (12m) 10m ago 28m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:31:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (20m) 11m ago 33m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:31:04.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3dcc06ecc0 msgr2=0x7f3dcc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3dcc06ecc0 0x7f3dcc071180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80097b0 tx=0x7f3dd8006d20 comp rx=0 tx=0).stop 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 msgr2=0x7f3de8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 0x7f3de8199390 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4005b40 tx=0x7f3dd400be20 comp rx=0 tx=0).stop 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 shutdown_connections 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3dcc06ecc0 0x7f3dcc071180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3de8103040 0x7f3de8198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 --2- 172.21.15.117:0/2472106960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de81039f0 0x7f3de8199390 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 >> 172.21.15.117:0/2472106960 conn(0x7f3de80fe8d0 msgr2=0x7f3de80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 shutdown_connections 2024-04-20T11:31:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:04.888+0000 7f3ded200700 1 -- 172.21.15.117:0/2472106960 wait complete. 2024-04-20T11:31:05.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.039+0000 7f0c61a00700 1 -- 172.21.15.117:0/2635492525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 msgr2=0x7f0c5c073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.039+0000 7f0c61a00700 1 --2- 172.21.15.117:0/2635492525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 0x7f0c5c073690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009910 tx=0x7f0c4c009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:05.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.039+0000 7f0c61a00700 1 -- 172.21.15.117:0/2635492525 shutdown_connections 2024-04-20T11:31:05.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.039+0000 7f0c61a00700 1 --2- 172.21.15.117:0/2635492525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c073bd0 0x7f0c5c10cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.039+0000 7f0c61a00700 1 --2- 172.21.15.117:0/2635492525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 0x7f0c5c073690 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.039+0000 7f0c61a00700 1 -- 172.21.15.117:0/2635492525 >> 172.21.15.117:0/2635492525 conn(0x7f0c5c0fc500 msgr2=0x7f0c5c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.039+0000 7f0c61a00700 1 -- 172.21.15.117:0/2635492525 shutdown_connections 2024-04-20T11:31:05.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.039+0000 7f0c61a00700 1 -- 172.21.15.117:0/2635492525 wait complete. 2024-04-20T11:31:05.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.040+0000 7f0c61a00700 1 Processor -- start 2024-04-20T11:31:05.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.040+0000 7f0c61a00700 1 -- start start 2024-04-20T11:31:05.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c61a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 0x7f0c5c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c61a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c073bd0 0x7f0c5c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c61a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19f480 con 0x7f0c5c0732b0 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c61a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c198a60 con 0x7f0c5c073bd0 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c5be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 0x7f0c5c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c5be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 0x7f0c5c19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57694/0 (socket says 172.21.15.117:57694) 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c5be00700 1 -- 172.21.15.117:0/3367402324 learned_addr learned my addr 172.21.15.117:0/3367402324 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c5be00700 1 -- 172.21.15.117:0/3367402324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c073bd0 msgr2=0x7f0c5c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c5be00700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c073bd0 0x7f0c5c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.041+0000 7f0c5be00700 1 -- 172.21.15.117:0/3367402324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0095f0 con 0x7f0c5c0732b0 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.042+0000 7f0c5be00700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 0x7f0c5c19e920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0092e0 tx=0x7f0c4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.042+0000 7f0c58c00700 1 -- 172.21.15.117:0/3367402324 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c4c01d070 con 0x7f0c5c0732b0 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.042+0000 7f0c58c00700 1 -- 172.21.15.117:0/3367402324 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c4c00f460 con 0x7f0c5c0732b0 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.042+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c198ce0 con 0x7f0c5c0732b0 2024-04-20T11:31:05.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.042+0000 7f0c58c00700 1 -- 172.21.15.117:0/3367402324 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c4c017630 con 0x7f0c5c0732b0 2024-04-20T11:31:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.042+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c1991d0 con 0x7f0c5c0732b0 2024-04-20T11:31:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.043+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c1076b0 con 0x7f0c5c0732b0 2024-04-20T11:31:05.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.044+0000 7f0c58c00700 1 -- 172.21.15.117:0/3367402324 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0c4c00f5d0 con 0x7f0c5c0732b0 2024-04-20T11:31:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.045+0000 7f0c58c00700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c4006ecc0 0x7f0c40071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.045+0000 7f0c58c00700 1 -- 172.21.15.117:0/3367402324 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0c4c090320 con 0x7f0c5c0732b0 2024-04-20T11:31:05.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.045+0000 7f0c5b400700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c4006ecc0 0x7f0c40071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.046+0000 7f0c5b400700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c4006ecc0 0x7f0c40071180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c199c20 tx=0x7f0c48006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.050+0000 7f0c58c00700 1 -- 172.21.15.117:0/3367402324 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0c4c05cc10 con 0x7f0c5c0732b0 2024-04-20T11:31:05.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.361+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c5c074960 con 0x7f0c5c0732b0 2024-04-20T11:31:05.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.362+0000 7f0c58c00700 1 -- 172.21.15.117:0/3367402324 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f0c4c05c7a0 con 0x7f0c5c0732b0 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:31:05.362 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:31:05.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:31:05.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:31:05.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:31:05.363 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:31:05.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:31:05.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:31:05.363 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:31:05.363 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c4006ecc0 msgr2=0x7f0c40071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c4006ecc0 0x7f0c40071180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c199c20 tx=0x7f0c48006d20 comp rx=0 tx=0).stop 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 msgr2=0x7f0c5c19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 0x7f0c5c19e920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0092e0 tx=0x7f0c4c00be20 comp rx=0 tx=0).stop 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 shutdown_connections 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0c4006ecc0 0x7f0c40071180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5c0732b0 0x7f0c5c19e920 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 --2- 172.21.15.117:0/3367402324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c073bd0 0x7f0c5c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 >> 172.21.15.117:0/3367402324 conn(0x7f0c5c0fc500 msgr2=0x7f0c5c105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 shutdown_connections 2024-04-20T11:31:05.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.365+0000 7f0c61a00700 1 -- 172.21.15.117:0/3367402324 wait complete. 2024-04-20T11:31:05.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.524+0000 7fb3f5a00700 1 -- 172.21.15.117:0/178636427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 msgr2=0x7fb3f010abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.524+0000 7fb3f5a00700 1 --2- 172.21.15.117:0/178636427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 0x7fb3f010abb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009940 tx=0x7fb3dc009c50 comp rx=0 tx=0).stop 2024-04-20T11:31:05.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.524+0000 7fb3f5a00700 1 -- 172.21.15.117:0/178636427 shutdown_connections 2024-04-20T11:31:05.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.524+0000 7fb3f5a00700 1 --2- 172.21.15.117:0/178636427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 0x7fb3f010abb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.524+0000 7fb3f5a00700 1 --2- 172.21.15.117:0/178636427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f0101da0 0x7fb3f0102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.524+0000 7fb3f5a00700 1 -- 172.21.15.117:0/178636427 >> 172.21.15.117:0/178636427 conn(0x7fb3f0076750 msgr2=0x7fb3f00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.525+0000 7fb3f5a00700 1 -- 172.21.15.117:0/178636427 shutdown_connections 2024-04-20T11:31:05.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.525+0000 7fb3f5a00700 1 -- 172.21.15.117:0/178636427 wait complete. 2024-04-20T11:31:05.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.526+0000 7fb3f5a00700 1 Processor -- start 2024-04-20T11:31:05.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.526+0000 7fb3f5a00700 1 -- start start 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.526+0000 7fb3f5a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f0101da0 0x7fb3f0198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.526+0000 7fb3f5a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 0x7fb3f0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.526+0000 7fb3f5a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01998f0 con 0x7fb3f01026c0 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.526+0000 7fb3f5a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f019c560 con 0x7fb3f0101da0 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.526+0000 7fb3ee000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 0x7fb3f0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.527+0000 7fb3ee000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 0x7fb3f0199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57712/0 (socket says 172.21.15.117:57712) 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.527+0000 7fb3ee000700 1 -- 172.21.15.117:0/383662212 learned_addr learned my addr 172.21.15.117:0/383662212 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.527+0000 7fb3eea00700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f0101da0 0x7fb3f0198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.527+0000 7fb3ee000700 1 -- 172.21.15.117:0/383662212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f0101da0 msgr2=0x7fb3f0198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.527+0000 7fb3ee000700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f0101da0 0x7fb3f0198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.527+0000 7fb3ee000700 1 -- 172.21.15.117:0/383662212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc0095f0 con 0x7fb3f01026c0 2024-04-20T11:31:05.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.527+0000 7fb3ee000700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 0x7fb3f0199260 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00b870 tx=0x7fb3dc0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.528+0000 7fb3e7400700 1 -- 172.21.15.117:0/383662212 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3dc01d070 con 0x7fb3f01026c0 2024-04-20T11:31:05.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.528+0000 7fb3e7400700 1 -- 172.21.15.117:0/383662212 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3dc00bed0 con 0x7fb3f01026c0 2024-04-20T11:31:05.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.528+0000 7fb3e7400700 1 -- 172.21.15.117:0/383662212 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3dc021410 con 0x7fb3f01026c0 2024-04-20T11:31:05.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.528+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f019c7e0 con 0x7fb3f01026c0 2024-04-20T11:31:05.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.528+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f019ccd0 con 0x7fb3f01026c0 2024-04-20T11:31:05.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.529+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f010a710 con 0x7fb3f01026c0 2024-04-20T11:31:05.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.530+0000 7fb3e7400700 1 -- 172.21.15.117:0/383662212 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb3dc00f810 con 0x7fb3f01026c0 2024-04-20T11:31:05.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.531+0000 7fb3e7400700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb3d406ecc0 0x7fb3d4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.531+0000 7fb3e7400700 1 -- 172.21.15.117:0/383662212 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb3dc090030 con 0x7fb3f01026c0 2024-04-20T11:31:05.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.531+0000 7fb3eea00700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb3d406ecc0 0x7fb3d4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.532+0000 7fb3eea00700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb3d406ecc0 0x7fb3d4071180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0005950 tx=0x7fb3e0005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.536+0000 7fb3e7400700 1 -- 172.21.15.117:0/383662212 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb3dc058e10 con 0x7fb3f01026c0 2024-04-20T11:31:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:05 smithi117 ceph-mon[99036]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:05 smithi117 ceph-mon[99036]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:05 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3367402324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:31:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:05 smithi173 ceph-mon[74165]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:05 smithi173 ceph-mon[74165]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:05 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3367402324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:31:05.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.811+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb3f0199ff0 con 0x7fb3f01026c0 2024-04-20T11:31:05.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.811+0000 7fb3e7400700 1 -- 172.21.15.117:0/383662212 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fb3dc05c4d0 con 0x7fb3f01026c0 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:31:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:31:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:31:05.814 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:31:05.815 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:31:05.815 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:31:05.815 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:31:05.815 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:31:05.815 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:31:05.815 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.814+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb3d406ecc0 msgr2=0x7fb3d4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.814+0000 7fb3f5a00700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb3d406ecc0 0x7fb3d4071180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0005950 tx=0x7fb3e0005cf0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.814+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 msgr2=0x7fb3f0199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.814+0000 7fb3f5a00700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 0x7fb3f0199260 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00b870 tx=0x7fb3dc0056a0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.815+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 shutdown_connections 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.815+0000 7fb3f5a00700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb3d406ecc0 0x7fb3d4071180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.815+0000 7fb3f5a00700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f0101da0 0x7fb3f0198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.815+0000 7fb3f5a00700 1 --2- 172.21.15.117:0/383662212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f01026c0 0x7fb3f0199260 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.815+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 >> 172.21.15.117:0/383662212 conn(0x7fb3f0076750 msgr2=0x7fb3f0101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.815+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 shutdown_connections 2024-04-20T11:31:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.815+0000 7fb3f5a00700 1 -- 172.21.15.117:0/383662212 wait complete. 2024-04-20T11:31:05.817 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:31:05.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.974+0000 7f0e0b200700 1 -- 172.21.15.117:0/963078573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04100a20 msgr2=0x7f0e04100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.974+0000 7f0e0b200700 1 --2- 172.21.15.117:0/963078573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04100a20 0x7f0e04100ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009910 tx=0x7f0df4009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:05.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.974+0000 7f0e0b200700 1 -- 172.21.15.117:0/963078573 shutdown_connections 2024-04-20T11:31:05.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.974+0000 7f0e0b200700 1 --2- 172.21.15.117:0/963078573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04100a20 0x7f0e04100ea0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.974+0000 7f0e0b200700 1 --2- 172.21.15.117:0/963078573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e04106a20 0x7f0e04106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.974+0000 7f0e0b200700 1 -- 172.21.15.117:0/963078573 >> 172.21.15.117:0/963078573 conn(0x7f0e040fc530 msgr2=0x7f0e040fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:05.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.975+0000 7f0e0b200700 1 -- 172.21.15.117:0/963078573 shutdown_connections 2024-04-20T11:31:05.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.975+0000 7f0e0b200700 1 -- 172.21.15.117:0/963078573 wait complete. 2024-04-20T11:31:05.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.976+0000 7f0e0b200700 1 Processor -- start 2024-04-20T11:31:05.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.976+0000 7f0e0b200700 1 -- start start 2024-04-20T11:31:05.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.976+0000 7f0e0b200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e04100a20 0x7f0e040731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.976+0000 7f0e0b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04106a20 0x7f0e04073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.976+0000 7f0e0b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04073c60 con 0x7f0e04106a20 2024-04-20T11:31:05.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.976+0000 7f0e0b200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04073da0 con 0x7f0e04100a20 2024-04-20T11:31:05.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.976+0000 7f0e09400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04106a20 0x7f0e04073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.977+0000 7f0e09400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04106a20 0x7f0e04073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57726/0 (socket says 172.21.15.117:57726) 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.977+0000 7f0e09400700 1 -- 172.21.15.117:0/2542581697 learned_addr learned my addr 172.21.15.117:0/2542581697 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.977+0000 7f0e09e00700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e04100a20 0x7f0e040731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.977+0000 7f0e09400700 1 -- 172.21.15.117:0/2542581697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e04100a20 msgr2=0x7f0e040731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.977+0000 7f0e09400700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e04100a20 0x7f0e040731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.977+0000 7f0e09400700 1 -- 172.21.15.117:0/2542581697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40095f0 con 0x7f0e04106a20 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.977+0000 7f0e09e00700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e04100a20 0x7f0e040731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.977+0000 7f0e09400700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04106a20 0x7f0e04073720 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0df40092e0 tx=0x7f0df400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.978+0000 7f0dfea00700 1 -- 172.21.15.117:0/2542581697 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df401d070 con 0x7f0e04106a20 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.978+0000 7f0dfea00700 1 -- 172.21.15.117:0/2542581697 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0df40056e0 con 0x7f0e04106a20 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.978+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e04074020 con 0x7f0e04106a20 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.978+0000 7f0dfea00700 1 -- 172.21.15.117:0/2542581697 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df400fe80 con 0x7f0e04106a20 2024-04-20T11:31:05.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.978+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e041a2db0 con 0x7f0e04106a20 2024-04-20T11:31:05.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.979+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e04109040 con 0x7f0e04106a20 2024-04-20T11:31:05.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.980+0000 7f0dfea00700 1 -- 172.21.15.117:0/2542581697 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0df400f6f0 con 0x7f0e04106a20 2024-04-20T11:31:05.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.981+0000 7f0dfea00700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dec06ecd0 0x7f0dec071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:05.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.981+0000 7f0dfea00700 1 -- 172.21.15.117:0/2542581697 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0df4090300 con 0x7f0e04106a20 2024-04-20T11:31:05.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.981+0000 7f0e09e00700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dec06ecd0 0x7f0dec071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:05.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.982+0000 7f0e09e00700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dec06ecd0 0x7f0dec071190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0df8005fd0 tx=0x7f0df8005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:05.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:05.986+0000 7f0dfea00700 1 -- 172.21.15.117:0/2542581697 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0df4059040 con 0x7f0e04106a20 2024-04-20T11:31:06.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.217+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e0410dc20 con 0x7f0dec06ecd0 2024-04-20T11:31:06.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.220+0000 7f0dfea00700 1 -- 172.21.15.117:0/2542581697 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0e0410dc20 con 0x7f0dec06ecd0 2024-04-20T11:31:06.219 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:31:06.220 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:31:06.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dec06ecd0 msgr2=0x7f0dec071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dec06ecd0 0x7f0dec071190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0df8005fd0 tx=0x7f0df8005ee0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04106a20 msgr2=0x7f0e04073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04106a20 0x7f0e04073720 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0df40092e0 tx=0x7f0df400be20 comp rx=0 tx=0).stop 2024-04-20T11:31:06.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 shutdown_connections 2024-04-20T11:31:06.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dec06ecd0 0x7f0dec071190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e04100a20 0x7f0e040731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 --2- 172.21.15.117:0/2542581697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e04106a20 0x7f0e04073720 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 >> 172.21.15.117:0/2542581697 conn(0x7f0e040fc530 msgr2=0x7f0e040fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:06.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 shutdown_connections 2024-04-20T11:31:06.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.223+0000 7f0e0b200700 1 -- 172.21.15.117:0/2542581697 wait complete. 2024-04-20T11:31:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.378+0000 7f5f04200700 1 -- 172.21.15.117:0/1847867008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc1017c0 msgr2=0x7f5efc105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.378+0000 7f5f04200700 1 --2- 172.21.15.117:0/1847867008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc1017c0 0x7f5efc105810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009910 tx=0x7f5ee8009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.378+0000 7f5f04200700 1 -- 172.21.15.117:0/1847867008 shutdown_connections 2024-04-20T11:31:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.378+0000 7f5f04200700 1 --2- 172.21.15.117:0/1847867008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc1017c0 0x7f5efc105810 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.378+0000 7f5f04200700 1 --2- 172.21.15.117:0/1847867008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5efc100e10 0x7f5efc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.378+0000 7f5f04200700 1 -- 172.21.15.117:0/1847867008 >> 172.21.15.117:0/1847867008 conn(0x7f5efc0fc6c0 msgr2=0x7f5efc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.378+0000 7f5f04200700 1 -- 172.21.15.117:0/1847867008 shutdown_connections 2024-04-20T11:31:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.379+0000 7f5f04200700 1 -- 172.21.15.117:0/1847867008 wait complete. 2024-04-20T11:31:06.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.380+0000 7f5f04200700 1 Processor -- start 2024-04-20T11:31:06.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.380+0000 7f5f04200700 1 -- start start 2024-04-20T11:31:06.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.381+0000 7f5f04200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc100e10 0x7f5efc0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.381+0000 7f5f04200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5efc1017c0 0x7f5efc073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.381+0000 7f5f04200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc073c60 con 0x7f5efc100e10 2024-04-20T11:31:06.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.381+0000 7f5f04200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc073da0 con 0x7f5efc1017c0 2024-04-20T11:31:06.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.381+0000 7f5f01800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc100e10 0x7f5efc0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.381+0000 7f5f01800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc100e10 0x7f5efc0731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57740/0 (socket says 172.21.15.117:57740) 2024-04-20T11:31:06.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.381+0000 7f5f01800700 1 -- 172.21.15.117:0/3477489450 learned_addr learned my addr 172.21.15.117:0/3477489450 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:06.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.381+0000 7f5f00e00700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5efc1017c0 0x7f5efc073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.382+0000 7f5f01800700 1 -- 172.21.15.117:0/3477489450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5efc1017c0 msgr2=0x7f5efc073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.382+0000 7f5f01800700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5efc1017c0 0x7f5efc073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.382+0000 7f5f01800700 1 -- 172.21.15.117:0/3477489450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80095f0 con 0x7f5efc100e10 2024-04-20T11:31:06.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.382+0000 7f5f00e00700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5efc1017c0 0x7f5efc073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:06.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.382+0000 7f5f01800700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc100e10 0x7f5efc0731e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c8a0 tx=0x7f5ef000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:06.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.382+0000 7f5efa000700 1 -- 172.21.15.117:0/3477489450 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ef000cea0 con 0x7f5efc100e10 2024-04-20T11:31:06.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.382+0000 7f5efa000700 1 -- 172.21.15.117:0/3477489450 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ef0004830 con 0x7f5efc100e10 2024-04-20T11:31:06.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.383+0000 7f5efa000700 1 -- 172.21.15.117:0/3477489450 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ef00055b0 con 0x7f5efc100e10 2024-04-20T11:31:06.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.383+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc1a2cf0 con 0x7f5efc100e10 2024-04-20T11:31:06.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.383+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc1a3050 con 0x7f5efc100e10 2024-04-20T11:31:06.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.384+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc109210 con 0x7f5efc100e10 2024-04-20T11:31:06.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.385+0000 7f5efa000700 1 -- 172.21.15.117:0/3477489450 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5ef00049a0 con 0x7f5efc100e10 2024-04-20T11:31:06.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.386+0000 7f5efa000700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ee406ecc0 0x7f5ee4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:06.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.386+0000 7f5efa000700 1 -- 172.21.15.117:0/3477489450 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5ef008ee50 con 0x7f5efc100e10 2024-04-20T11:31:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.387+0000 7f5f00e00700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ee406ecc0 0x7f5ee4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:06.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.387+0000 7f5f00e00700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ee406ecc0 0x7f5ee4071180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0748c0 tx=0x7f5ee801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:06.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.390+0000 7f5efa000700 1 -- 172.21.15.117:0/3477489450 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5ef0057d50 con 0x7f5efc100e10 2024-04-20T11:31:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:06 smithi117 ceph-mon[99036]: pgmap v559: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:06 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/383662212' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:31:06.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:06 smithi117 ceph-mon[99036]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:06 smithi173 ceph-mon[74165]: pgmap v559: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:06 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/383662212' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:31:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:06 smithi173 ceph-mon[74165]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:06.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.683+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5efc1022f0 con 0x7f5efc100e10 2024-04-20T11:31:06.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.684+0000 7f5efa000700 1 -- 172.21.15.117:0/3477489450 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5ef001d090 con 0x7f5efc100e10 2024-04-20T11:31:06.683 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:31:06.683 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:31:06.683 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:31:06.684 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:31:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:31:06.684 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:31:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:31:06.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.686+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ee406ecc0 msgr2=0x7f5ee4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.686+0000 7f5f04200700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ee406ecc0 0x7f5ee4071180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0748c0 tx=0x7f5ee801a040 comp rx=0 tx=0).stop 2024-04-20T11:31:06.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.686+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc100e10 msgr2=0x7f5efc0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:06.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.686+0000 7f5f04200700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc100e10 0x7f5efc0731e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c8a0 tx=0x7f5ef000cc60 comp rx=0 tx=0).stop 2024-04-20T11:31:06.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.687+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 shutdown_connections 2024-04-20T11:31:06.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.687+0000 7f5f04200700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ee406ecc0 0x7f5ee4071180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.687+0000 7f5f04200700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5efc100e10 0x7f5efc0731e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.687+0000 7f5f04200700 1 --2- 172.21.15.117:0/3477489450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5efc1017c0 0x7f5efc073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:06.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.687+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 >> 172.21.15.117:0/3477489450 conn(0x7f5efc0fc6c0 msgr2=0x7f5efc0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:06.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.687+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 shutdown_connections 2024-04-20T11:31:06.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:06.687+0000 7f5f04200700 1 -- 172.21.15.117:0/3477489450 wait complete. 2024-04-20T11:31:07.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:07 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3477489450' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:31:07.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:07 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3477489450' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:31:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:08 smithi173 ceph-mon[74165]: pgmap v560: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:08.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:08 smithi117 ceph-mon[99036]: pgmap v560: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:10.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:10 smithi173 ceph-mon[74165]: pgmap v561: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:11.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:10 smithi117 ceph-mon[99036]: pgmap v561: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:13.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:12 smithi117 ceph-mon[99036]: pgmap v562: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:13.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:12 smithi173 ceph-mon[74165]: pgmap v562: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:31:14.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:31:15.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:14 smithi117 ceph-mon[99036]: pgmap v563: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:15.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:14 smithi173 ceph-mon[74165]: pgmap v563: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:17.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:16 smithi117 ceph-mon[99036]: pgmap v564: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:17.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:16 smithi173 ceph-mon[74165]: pgmap v564: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:18 smithi173 ceph-mon[74165]: pgmap v565: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:19.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:18 smithi117 ceph-mon[99036]: pgmap v565: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:21.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:20 smithi117 ceph-mon[99036]: pgmap v566: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:21.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:20 smithi173 ceph-mon[74165]: pgmap v566: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:22.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:22 smithi117 ceph-mon[99036]: pgmap v567: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:23.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:22 smithi173 ceph-mon[74165]: pgmap v567: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:25.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:24 smithi117 ceph-mon[99036]: pgmap v568: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:25.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:24 smithi173 ceph-mon[74165]: pgmap v568: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:27.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:26 smithi117 ceph-mon[99036]: pgmap v569: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:27.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:26 smithi173 ceph-mon[74165]: pgmap v569: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:29.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:28 smithi117 ceph-mon[99036]: pgmap v570: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:29.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:31:29.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:28 smithi173 ceph-mon[74165]: pgmap v570: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:29.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:31:31.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:30 smithi117 ceph-mon[99036]: pgmap v571: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:31.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:30 smithi173 ceph-mon[74165]: pgmap v571: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:33.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:32 smithi117 ceph-mon[99036]: pgmap v572: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:33.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:32 smithi173 ceph-mon[74165]: pgmap v572: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:35.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:34 smithi117 ceph-mon[99036]: pgmap v573: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:34 smithi173 ceph-mon[74165]: pgmap v573: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:36.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.855+0000 7f092f000700 1 -- 172.21.15.117:0/2838180530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f092810d0c0 msgr2=0x7f092810d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.855+0000 7f092f000700 1 --2- 172.21.15.117:0/2838180530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f092810d0c0 0x7f092810d4a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f091c009910 tx=0x7f091c009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:36.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.855+0000 7f092f000700 1 -- 172.21.15.117:0/2838180530 shutdown_connections 2024-04-20T11:31:36.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.855+0000 7f092f000700 1 --2- 172.21.15.117:0/2838180530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09281070b0 0x7f0928107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.855+0000 7f092f000700 1 --2- 172.21.15.117:0/2838180530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f092810d0c0 0x7f092810d4a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.855+0000 7f092f000700 1 -- 172.21.15.117:0/2838180530 >> 172.21.15.117:0/2838180530 conn(0x7f0928076830 msgr2=0x7f0928078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:36.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.856+0000 7f092f000700 1 -- 172.21.15.117:0/2838180530 shutdown_connections 2024-04-20T11:31:36.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.856+0000 7f092f000700 1 -- 172.21.15.117:0/2838180530 wait complete. 2024-04-20T11:31:36.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.857+0000 7f092f000700 1 Processor -- start 2024-04-20T11:31:36.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.858+0000 7f092f000700 1 -- start start 2024-04-20T11:31:36.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.858+0000 7f092f000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09281070b0 0x7f092819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.858+0000 7f092f000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092810d0c0 0x7f092819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.858+0000 7f092f000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092819db50 con 0x7f09281070b0 2024-04-20T11:31:36.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.858+0000 7f092f000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09281a1820 con 0x7f092810d0c0 2024-04-20T11:31:36.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.858+0000 7f092dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09281070b0 0x7f092819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.858+0000 7f092dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09281070b0 0x7f092819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34634/0 (socket says 172.21.15.117:34634) 2024-04-20T11:31:36.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.858+0000 7f092dc00700 1 -- 172.21.15.117:0/3283680556 learned_addr learned my addr 172.21.15.117:0/3283680556 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:36.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.859+0000 7f092d200700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092810d0c0 0x7f092819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.859+0000 7f092dc00700 1 -- 172.21.15.117:0/3283680556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092810d0c0 msgr2=0x7f092819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:36.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.859+0000 7f092dc00700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092810d0c0 0x7f092819d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:36.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.859+0000 7f092dc00700 1 -- 172.21.15.117:0/3283680556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0095f0 con 0x7f09281070b0 2024-04-20T11:31:36.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.859+0000 7f092dc00700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09281070b0 0x7f092819cf30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f091c006010 tx=0x7f091c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.859+0000 7f0926000700 1 -- 172.21.15.117:0/3283680556 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091c01d070 con 0x7f09281070b0 2024-04-20T11:31:36.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.860+0000 7f0926000700 1 -- 172.21.15.117:0/3283680556 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f091c00f460 con 0x7f09281070b0 2024-04-20T11:31:36.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.860+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09281a1aa0 con 0x7f09281070b0 2024-04-20T11:31:36.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.860+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09281a1f90 con 0x7f09281070b0 2024-04-20T11:31:36.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.860+0000 7f0926000700 1 -- 172.21.15.117:0/3283680556 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091c017630 con 0x7f09281070b0 2024-04-20T11:31:36.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.861+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092810f5e0 con 0x7f09281070b0 2024-04-20T11:31:36.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.862+0000 7f0926000700 1 -- 172.21.15.117:0/3283680556 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f091c00f5d0 con 0x7f09281070b0 2024-04-20T11:31:36.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.863+0000 7f0926000700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f091006ecc0 0x7f0910071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:36.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.863+0000 7f0926000700 1 -- 172.21.15.117:0/3283680556 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f091c090370 con 0x7f09281070b0 2024-04-20T11:31:36.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.863+0000 7f092d200700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f091006ecc0 0x7f0910071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:36.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.864+0000 7f092d200700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f091006ecc0 0x7f0910071180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f092819e490 tx=0x7f0914005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:36.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:36.868+0000 7f0926000700 1 -- 172.21.15.117:0/3283680556 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f091c059170 con 0x7f09281070b0 2024-04-20T11:31:37.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.114+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f092819e240 con 0x7f091006ecc0 2024-04-20T11:31:37.116 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:36 smithi117 ceph-mon[99036]: pgmap v574: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:37.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.117+0000 7f0926000700 1 -- 172.21.15.117:0/3283680556 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f092819e240 con 0x7f091006ecc0 2024-04-20T11:31:37.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.119+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f091006ecc0 msgr2=0x7f0910071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.119+0000 7f092f000700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f091006ecc0 0x7f0910071180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f092819e490 tx=0x7f0914005cf0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.119+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09281070b0 msgr2=0x7f092819cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.119+0000 7f092f000700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09281070b0 0x7f092819cf30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f091c006010 tx=0x7f091c00be20 comp rx=0 tx=0).stop 2024-04-20T11:31:37.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.120+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 shutdown_connections 2024-04-20T11:31:37.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.120+0000 7f092f000700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f091006ecc0 0x7f0910071180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.120+0000 7f092f000700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09281070b0 0x7f092819cf30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.120+0000 7f092f000700 1 --2- 172.21.15.117:0/3283680556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092810d0c0 0x7f092819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.120+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 >> 172.21.15.117:0/3283680556 conn(0x7f0928076830 msgr2=0x7f092810c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:37.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.120+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 shutdown_connections 2024-04-20T11:31:37.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.120+0000 7f092f000700 1 -- 172.21.15.117:0/3283680556 wait complete. 2024-04-20T11:31:37.136 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:31:37.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:36 smithi173 ceph-mon[74165]: pgmap v574: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:37.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.285+0000 7fd28d800700 1 -- 172.21.15.117:0/218104282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 msgr2=0x7fd288106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.285+0000 7fd28d800700 1 --2- 172.21.15.117:0/218104282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 0x7fd288106e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd278009910 tx=0x7fd278009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:37.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.285+0000 7fd28d800700 1 -- 172.21.15.117:0/218104282 shutdown_connections 2024-04-20T11:31:37.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.285+0000 7fd28d800700 1 --2- 172.21.15.117:0/218104282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288100a20 0x7fd288100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.285+0000 7fd28d800700 1 --2- 172.21.15.117:0/218104282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 0x7fd288106e00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.285+0000 7fd28d800700 1 -- 172.21.15.117:0/218104282 >> 172.21.15.117:0/218104282 conn(0x7fd2880fc530 msgr2=0x7fd2880fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:37.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.285+0000 7fd28d800700 1 -- 172.21.15.117:0/218104282 shutdown_connections 2024-04-20T11:31:37.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.285+0000 7fd28d800700 1 -- 172.21.15.117:0/218104282 wait complete. 2024-04-20T11:31:37.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.286+0000 7fd28d800700 1 Processor -- start 2024-04-20T11:31:37.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.287+0000 7fd28d800700 1 -- start start 2024-04-20T11:31:37.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.287+0000 7fd28d800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288100a20 0x7fd28819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.287+0000 7fd28d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 0x7fd28819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.287+0000 7fd28d800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28819d0f0 con 0x7fd288106a20 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.287+0000 7fd28d800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd288196760 con 0x7fd288100a20 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.287+0000 7fd287400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 0x7fd28819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.287+0000 7fd287400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 0x7fd28819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34652/0 (socket says 172.21.15.117:34652) 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.287+0000 7fd287400700 1 -- 172.21.15.117:0/1062359691 learned_addr learned my addr 172.21.15.117:0/1062359691 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd287e00700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288100a20 0x7fd28819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd287400700 1 -- 172.21.15.117:0/1062359691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288100a20 msgr2=0x7fd28819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd287400700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288100a20 0x7fd28819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd287400700 1 -- 172.21.15.117:0/1062359691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780095f0 con 0x7fd288106a20 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd287e00700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288100a20 0x7fd28819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:37.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd287400700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 0x7fd28819cb60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd27400c930 tx=0x7fd27400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:37.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd284c00700 1 -- 172.21.15.117:0/1062359691 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd274007ad0 con 0x7fd288106a20 2024-04-20T11:31:37.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd284c00700 1 -- 172.21.15.117:0/1062359691 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd274007c30 con 0x7fd288106a20 2024-04-20T11:31:37.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd284c00700 1 -- 172.21.15.117:0/1062359691 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd274018610 con 0x7fd288106a20 2024-04-20T11:31:37.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.288+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd288196a40 con 0x7fd288106a20 2024-04-20T11:31:37.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.289+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd288196ef0 con 0x7fd288106a20 2024-04-20T11:31:37.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.290+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd288109020 con 0x7fd288106a20 2024-04-20T11:31:37.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.291+0000 7fd284c00700 1 -- 172.21.15.117:0/1062359691 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd27400f450 con 0x7fd288106a20 2024-04-20T11:31:37.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.292+0000 7fd284c00700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26c06ecd0 0x7fd26c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.292+0000 7fd284c00700 1 -- 172.21.15.117:0/1062359691 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd27408f0f0 con 0x7fd288106a20 2024-04-20T11:31:37.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.292+0000 7fd287e00700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26c06ecd0 0x7fd26c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.293+0000 7fd287e00700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26c06ecd0 0x7fd26c071190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd278009fd0 tx=0x7fd27800b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:37.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.297+0000 7fd284c00700 1 -- 172.21.15.117:0/1062359691 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd27405b960 con 0x7fd288106a20 2024-04-20T11:31:37.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.553+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd288197c90 con 0x7fd26c06ecd0 2024-04-20T11:31:37.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.555+0000 7fd284c00700 1 -- 172.21.15.117:0/1062359691 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd288197c90 con 0x7fd26c06ecd0 2024-04-20T11:31:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26c06ecd0 msgr2=0x7fd26c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26c06ecd0 0x7fd26c071190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd278009fd0 tx=0x7fd27800b560 comp rx=0 tx=0).stop 2024-04-20T11:31:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 msgr2=0x7fd28819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 0x7fd28819cb60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd27400c930 tx=0x7fd27400ccf0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 shutdown_connections 2024-04-20T11:31:37.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd26c06ecd0 0x7fd26c071190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd288100a20 0x7fd28819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 --2- 172.21.15.117:0/1062359691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd288106a20 0x7fd28819cb60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 >> 172.21.15.117:0/1062359691 conn(0x7fd2880fc530 msgr2=0x7fd2880fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:37.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 shutdown_connections 2024-04-20T11:31:37.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.558+0000 7fd28d800700 1 -- 172.21.15.117:0/1062359691 wait complete. 2024-04-20T11:31:37.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.706+0000 7f1810a00700 1 -- 172.21.15.117:0/2039447937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107df0 msgr2=0x7f180c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.706+0000 7f1810a00700 1 --2- 172.21.15.117:0/2039447937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107df0 0x7f180c10be40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009910 tx=0x7f17f8009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:37.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.706+0000 7f1810a00700 1 -- 172.21.15.117:0/2039447937 shutdown_connections 2024-04-20T11:31:37.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.706+0000 7f1810a00700 1 --2- 172.21.15.117:0/2039447937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107df0 0x7f180c10be40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.706+0000 7f1810a00700 1 --2- 172.21.15.117:0/2039447937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180c107440 0x7f180c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.706+0000 7f1810a00700 1 -- 172.21.15.117:0/2039447937 >> 172.21.15.117:0/2039447937 conn(0x7f180c076a10 msgr2=0x7f180c078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:37.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.706+0000 7f1810a00700 1 -- 172.21.15.117:0/2039447937 shutdown_connections 2024-04-20T11:31:37.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.706+0000 7f1810a00700 1 -- 172.21.15.117:0/2039447937 wait complete. 2024-04-20T11:31:37.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.707+0000 7f1810a00700 1 Processor -- start 2024-04-20T11:31:37.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.707+0000 7f1810a00700 1 -- start start 2024-04-20T11:31:37.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.707+0000 7f1810a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107440 0x7f180c19d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.707+0000 7f1810a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180c107df0 0x7f180c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.707+0000 7f1810a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c19dd70 con 0x7f180c107440 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.707+0000 7f1810a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c1a19f0 con 0x7f180c107df0 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.708+0000 7f1809e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107440 0x7f180c19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.708+0000 7f1809e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107440 0x7f180c19d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34676/0 (socket says 172.21.15.117:34676) 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.708+0000 7f1809e00700 1 -- 172.21.15.117:0/4253794896 learned_addr learned my addr 172.21.15.117:0/4253794896 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.708+0000 7f1809e00700 1 -- 172.21.15.117:0/4253794896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180c107df0 msgr2=0x7f180c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.708+0000 7f1809400700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180c107df0 0x7f180c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.708+0000 7f1809e00700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180c107df0 0x7f180c19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.708+0000 7f1809e00700 1 -- 172.21.15.117:0/4253794896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f80095f0 con 0x7f180c107440 2024-04-20T11:31:37.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.708+0000 7f1809e00700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107440 0x7f180c19d150 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00c8a0 tx=0x7f17fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:37.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.709+0000 7f1802a00700 1 -- 172.21.15.117:0/4253794896 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17fc00cea0 con 0x7f180c107440 2024-04-20T11:31:37.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.709+0000 7f1802a00700 1 -- 172.21.15.117:0/4253794896 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17fc004830 con 0x7f180c107440 2024-04-20T11:31:37.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.709+0000 7f1802a00700 1 -- 172.21.15.117:0/4253794896 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17fc0055b0 con 0x7f180c107440 2024-04-20T11:31:37.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.709+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c1a1cd0 con 0x7f180c107440 2024-04-20T11:31:37.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.709+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c1a2220 con 0x7f180c107440 2024-04-20T11:31:37.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.710+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c10f840 con 0x7f180c107440 2024-04-20T11:31:37.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.711+0000 7f1802a00700 1 -- 172.21.15.117:0/4253794896 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f17fc0049a0 con 0x7f180c107440 2024-04-20T11:31:37.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.712+0000 7f1802a00700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17f006ecc0 0x7f17f0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:37.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.712+0000 7f1802a00700 1 -- 172.21.15.117:0/4253794896 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f17fc08eac0 con 0x7f180c107440 2024-04-20T11:31:37.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.712+0000 7f1809400700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17f006ecc0 0x7f17f0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:37.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.713+0000 7f1809400700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17f006ecc0 0x7f17f0071180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17f800b5c0 tx=0x7f17f8005380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:37.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.717+0000 7f1802a00700 1 -- 172.21.15.117:0/4253794896 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f17fc057880 con 0x7f180c107440 2024-04-20T11:31:37.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:37 smithi117 ceph-mon[99036]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:37.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.953+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f180c1143e0 con 0x7f17f006ecc0 2024-04-20T11:31:37.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.984+0000 7f1802a00700 1 -- 172.21.15.117:0/4253794896 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f180c1143e0 con 0x7f17f006ecc0 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (20m) 11m ago 34m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (34m) 11m ago 34m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (33m) 11m ago 33m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (16m) 11m ago 34m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (16m) 11m ago 33m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (19m) 11m ago 33m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (11m) 11m ago 26m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (12m) 11m ago 26m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (26m) 11m ago 26m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (11m) 11m ago 26m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:31:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (23m) 11m ago 35m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (21m) 11m ago 32m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (18m) 11m ago 36m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (17m) 11m ago 32m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (21m) 11m ago 34m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (21m) 11m ago 32m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (15m) 11m ago 31m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (15m) 11m ago 31m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (14m) 11m ago 30m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (14m) 11m ago 29m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (13m) 11m ago 29m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (12m) 11m ago 28m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:31:37.987 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (20m) 11m ago 33m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:31:37.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.987+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17f006ecc0 msgr2=0x7f17f0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.987+0000 7f1810a00700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17f006ecc0 0x7f17f0071180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17f800b5c0 tx=0x7f17f8005380 comp rx=0 tx=0).stop 2024-04-20T11:31:37.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.987+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107440 msgr2=0x7f180c19d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:37.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.987+0000 7f1810a00700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107440 0x7f180c19d150 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00c8a0 tx=0x7f17fc00cc60 comp rx=0 tx=0).stop 2024-04-20T11:31:37.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.988+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 shutdown_connections 2024-04-20T11:31:37.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.988+0000 7f1810a00700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f17f006ecc0 0x7f17f0071180 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.988+0000 7f1810a00700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f180c107440 0x7f180c19d150 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.988+0000 7f1810a00700 1 --2- 172.21.15.117:0/4253794896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180c107df0 0x7f180c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:37.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.988+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 >> 172.21.15.117:0/4253794896 conn(0x7f180c076a10 msgr2=0x7f180c076df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:37.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.988+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 shutdown_connections 2024-04-20T11:31:37.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:37.988+0000 7f1810a00700 1 -- 172.21.15.117:0/4253794896 wait complete. 2024-04-20T11:31:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.143+0000 7fa36f000700 1 -- 172.21.15.117:0/3522027356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368100a20 msgr2=0x7fa368100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.143+0000 7fa36f000700 1 --2- 172.21.15.117:0/3522027356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368100a20 0x7fa368100ea0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa358009910 tx=0x7fa358009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.143+0000 7fa36f000700 1 -- 172.21.15.117:0/3522027356 shutdown_connections 2024-04-20T11:31:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.143+0000 7fa36f000700 1 --2- 172.21.15.117:0/3522027356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368100a20 0x7fa368100ea0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.143+0000 7fa36f000700 1 --2- 172.21.15.117:0/3522027356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa368106a20 0x7fa368106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.143+0000 7fa36f000700 1 -- 172.21.15.117:0/3522027356 >> 172.21.15.117:0/3522027356 conn(0x7fa3680fc530 msgr2=0x7fa3680fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.143+0000 7fa36f000700 1 -- 172.21.15.117:0/3522027356 shutdown_connections 2024-04-20T11:31:38.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.144+0000 7fa36f000700 1 -- 172.21.15.117:0/3522027356 wait complete. 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.144+0000 7fa36f000700 1 Processor -- start 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.145+0000 7fa36f000700 1 -- start start 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.145+0000 7fa36f000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa368100a20 0x7fa36819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.145+0000 7fa36f000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368106a20 0x7fa36819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.145+0000 7fa36f000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36819f5d0 con 0x7fa368106a20 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.145+0000 7fa36f000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa368198b50 con 0x7fa368100a20 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.145+0000 7fa36d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368106a20 0x7fa36819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.145+0000 7fa36d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368106a20 0x7fa36819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34698/0 (socket says 172.21.15.117:34698) 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.145+0000 7fa36d200700 1 -- 172.21.15.117:0/91992031 learned_addr learned my addr 172.21.15.117:0/91992031 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa36dc00700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa368100a20 0x7fa36819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa36d200700 1 -- 172.21.15.117:0/91992031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa368100a20 msgr2=0x7fa36819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa36d200700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa368100a20 0x7fa36819e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa36d200700 1 -- 172.21.15.117:0/91992031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580095f0 con 0x7fa368106a20 2024-04-20T11:31:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa36d200700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368106a20 0x7fa36819eef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa358006010 tx=0x7fa35800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa366000700 1 -- 172.21.15.117:0/91992031 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35801d070 con 0x7fa368106a20 2024-04-20T11:31:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa366000700 1 -- 172.21.15.117:0/91992031 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35800f460 con 0x7fa368106a20 2024-04-20T11:31:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa368198dd0 con 0x7fa368106a20 2024-04-20T11:31:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa366000700 1 -- 172.21.15.117:0/91992031 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa358017630 con 0x7fa368106a20 2024-04-20T11:31:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.146+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3681992c0 con 0x7fa368106a20 2024-04-20T11:31:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.148+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa368104cd0 con 0x7fa368106a20 2024-04-20T11:31:38.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.149+0000 7fa366000700 1 -- 172.21.15.117:0/91992031 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa35800f5d0 con 0x7fa368106a20 2024-04-20T11:31:38.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.149+0000 7fa366000700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa35006ecd0 0x7fa350071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.150+0000 7fa366000700 1 -- 172.21.15.117:0/91992031 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa358090350 con 0x7fa368106a20 2024-04-20T11:31:38.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.150+0000 7fa36dc00700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa35006ecd0 0x7fa350071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.150+0000 7fa36dc00700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa35006ecd0 0x7fa350071190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa35c006fd0 tx=0x7fa35c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.154+0000 7fa366000700 1 -- 172.21.15.117:0/91992031 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa35805cd20 con 0x7fa368106a20 2024-04-20T11:31:38.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:37 smithi173 ceph-mon[74165]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:38.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.451+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa368199ff0 con 0x7fa368106a20 2024-04-20T11:31:38.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.452+0000 7fa366000700 1 -- 172.21.15.117:0/91992031 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fa358020370 con 0x7fa368106a20 2024-04-20T11:31:38.452 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:31:38.452 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:31:38.452 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:31:38.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:31:38.453 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:31:38.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.454+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa35006ecd0 msgr2=0x7fa350071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.454+0000 7fa36f000700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa35006ecd0 0x7fa350071190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa35c006fd0 tx=0x7fa35c008040 comp rx=0 tx=0).stop 2024-04-20T11:31:38.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368106a20 msgr2=0x7fa36819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368106a20 0x7fa36819eef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa358006010 tx=0x7fa35800be20 comp rx=0 tx=0).stop 2024-04-20T11:31:38.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 shutdown_connections 2024-04-20T11:31:38.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa35006ecd0 0x7fa350071190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa368100a20 0x7fa36819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 --2- 172.21.15.117:0/91992031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368106a20 0x7fa36819eef0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 >> 172.21.15.117:0/91992031 conn(0x7fa3680fc530 msgr2=0x7fa368073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:38.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 shutdown_connections 2024-04-20T11:31:38.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.455+0000 7fa36f000700 1 -- 172.21.15.117:0/91992031 wait complete. 2024-04-20T11:31:38.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.599+0000 7f45c1c00700 1 -- 172.21.15.117:0/2594721859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc102e60 msgr2=0x7f45bc103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.599+0000 7f45c1c00700 1 --2- 172.21.15.117:0/2594721859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc102e60 0x7f45bc103240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009910 tx=0x7f45ac009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:38.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.599+0000 7f45c1c00700 1 -- 172.21.15.117:0/2594721859 shutdown_connections 2024-04-20T11:31:38.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.599+0000 7f45c1c00700 1 --2- 172.21.15.117:0/2594721859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45bc103810 0x7f45bc107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.599+0000 7f45c1c00700 1 --2- 172.21.15.117:0/2594721859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc102e60 0x7f45bc103240 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.599+0000 7f45c1c00700 1 -- 172.21.15.117:0/2594721859 >> 172.21.15.117:0/2594721859 conn(0x7f45bc0fe710 msgr2=0x7f45bc100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:38.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.599+0000 7f45c1c00700 1 -- 172.21.15.117:0/2594721859 shutdown_connections 2024-04-20T11:31:38.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.599+0000 7f45c1c00700 1 -- 172.21.15.117:0/2594721859 wait complete. 2024-04-20T11:31:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.600+0000 7f45c1c00700 1 Processor -- start 2024-04-20T11:31:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.600+0000 7f45c1c00700 1 -- start start 2024-04-20T11:31:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.601+0000 7f45c1c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45bc102e60 0x7f45bc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.601+0000 7f45c1c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc103810 0x7f45bc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.601+0000 7f45c1c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc19f380 con 0x7f45bc103810 2024-04-20T11:31:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.601+0000 7f45c1c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc1989b0 con 0x7f45bc102e60 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.601+0000 7f45bb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc103810 0x7f45bc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.601+0000 7f45bb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc103810 0x7f45bc19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34722/0 (socket says 172.21.15.117:34722) 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.601+0000 7f45bb400700 1 -- 172.21.15.117:0/4218800419 learned_addr learned my addr 172.21.15.117:0/4218800419 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.601+0000 7f45bbe00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45bc102e60 0x7f45bc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45bb400700 1 -- 172.21.15.117:0/4218800419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45bc102e60 msgr2=0x7f45bc19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45bb400700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45bc102e60 0x7f45bc19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45bb400700 1 -- 172.21.15.117:0/4218800419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac0095f0 con 0x7f45bc103810 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45bb400700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc103810 0x7f45bc19ed60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e8f0 tx=0x7f45a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45b8c00700 1 -- 172.21.15.117:0/4218800419 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45a8009790 con 0x7f45bc103810 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45b8c00700 1 -- 172.21.15.117:0/4218800419 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45a8004cf0 con 0x7f45bc103810 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45b8c00700 1 -- 172.21.15.117:0/4218800419 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45a8005690 con 0x7f45bc103810 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc198c90 con 0x7f45bc103810 2024-04-20T11:31:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.602+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc199190 con 0x7f45bc103810 2024-04-20T11:31:38.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.603+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc10b200 con 0x7f45bc103810 2024-04-20T11:31:38.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.605+0000 7f45b8c00700 1 -- 172.21.15.117:0/4218800419 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f45a8004750 con 0x7f45bc103810 2024-04-20T11:31:38.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.606+0000 7f45b8c00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f45a006ecd0 0x7f45a0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:38.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.606+0000 7f45b8c00700 1 -- 172.21.15.117:0/4218800419 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f45a8014070 con 0x7f45bc103810 2024-04-20T11:31:38.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.606+0000 7f45bbe00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f45a006ecd0 0x7f45a0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:38.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.607+0000 7f45bbe00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f45a006ecd0 0x7f45a0071190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b760 tx=0x7f45ac009f90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:38.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.610+0000 7f45b8c00700 1 -- 172.21.15.117:0/4218800419 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f45a805bb40 con 0x7f45bc103810 2024-04-20T11:31:38.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.886+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f45bc104340 con 0x7f45bc103810 2024-04-20T11:31:38.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.887+0000 7f45b8c00700 1 -- 172.21.15.117:0/4218800419 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f45a805b6d0 con 0x7f45bc103810 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:31:38.888 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:31:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:31:38.890 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:31:38.891 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:31:38.891 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:31:38.891 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:31:38.891 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:31:38.891 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:31:38.891 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:31:38.891 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:31:38.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f45a006ecd0 msgr2=0x7f45a0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f45a006ecd0 0x7f45a0071190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b760 tx=0x7f45ac009f90 comp rx=0 tx=0).stop 2024-04-20T11:31:38.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc103810 msgr2=0x7f45bc19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:38.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc103810 0x7f45bc19ed60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e8f0 tx=0x7f45a800ecb0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 shutdown_connections 2024-04-20T11:31:38.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f45a006ecd0 0x7f45a0071190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45bc102e60 0x7f45bc19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 --2- 172.21.15.117:0/4218800419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45bc103810 0x7f45bc19ed60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:38.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.891+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 >> 172.21.15.117:0/4218800419 conn(0x7f45bc0fe710 msgr2=0x7f45bc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:38.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.892+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 shutdown_connections 2024-04-20T11:31:38.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:38.892+0000 7f45c1c00700 1 -- 172.21.15.117:0/4218800419 wait complete. 2024-04-20T11:31:38.893 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:31:39.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.035+0000 7f1829a00700 1 -- 172.21.15.117:0/817422734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 msgr2=0x7f1824107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.035+0000 7f1829a00700 1 --2- 172.21.15.117:0/817422734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 0x7f1824107a20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f180c009910 tx=0x7f180c009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:39.036 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:38 smithi117 ceph-mon[99036]: pgmap v575: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:39.036 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:38 smithi117 ceph-mon[99036]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:39.036 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:38 smithi117 ceph-mon[99036]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:39.036 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:38 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/91992031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.035+0000 7f1829a00700 1 -- 172.21.15.117:0/817422734 shutdown_connections 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.035+0000 7f1829a00700 1 --2- 172.21.15.117:0/817422734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 0x7f1824107a20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.035+0000 7f1829a00700 1 --2- 172.21.15.117:0/817422734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1824103020 0x7f1824103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.035+0000 7f1829a00700 1 -- 172.21.15.117:0/817422734 >> 172.21.15.117:0/817422734 conn(0x7f18240fe8b0 msgr2=0x7f1824100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.035+0000 7f1829a00700 1 -- 172.21.15.117:0/817422734 shutdown_connections 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.035+0000 7f1829a00700 1 -- 172.21.15.117:0/817422734 wait complete. 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.036+0000 7f1829a00700 1 Processor -- start 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.036+0000 7f1829a00700 1 -- start start 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.037+0000 7f1829a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1824103020 0x7f1824198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.037+0000 7f1829a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 0x7f1824199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.037+0000 7f1829a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1824199a30 con 0x7f18241039d0 2024-04-20T11:31:39.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.037+0000 7f1829a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182419d700 con 0x7f1824103020 2024-04-20T11:31:39.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.037+0000 7f1822000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 0x7f1824199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.037+0000 7f1822000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 0x7f1824199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34732/0 (socket says 172.21.15.117:34732) 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.037+0000 7f1822000700 1 -- 172.21.15.117:0/1680200436 learned_addr learned my addr 172.21.15.117:0/1680200436 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f1822000700 1 -- 172.21.15.117:0/1680200436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1824103020 msgr2=0x7f1824198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f1822000700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1824103020 0x7f1824198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f1822000700 1 -- 172.21.15.117:0/1680200436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0095f0 con 0x7f18241039d0 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f1822000700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 0x7f1824199350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f180c0092e0 tx=0x7f180c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f181b400700 1 -- 172.21.15.117:0/1680200436 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180c01d070 con 0x7f18241039d0 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182419d980 con 0x7f18241039d0 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f181b400700 1 -- 172.21.15.117:0/1680200436 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f180c00f460 con 0x7f18241039d0 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f181b400700 1 -- 172.21.15.117:0/1680200436 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180c017610 con 0x7f18241039d0 2024-04-20T11:31:39.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.038+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182419de70 con 0x7f18241039d0 2024-04-20T11:31:39.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.039+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182410b370 con 0x7f18241039d0 2024-04-20T11:31:39.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.041+0000 7f181b400700 1 -- 172.21.15.117:0/1680200436 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f180c00f5d0 con 0x7f18241039d0 2024-04-20T11:31:39.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.042+0000 7f181b400700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f180806ecc0 0x7f1808071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.042+0000 7f181b400700 1 -- 172.21.15.117:0/1680200436 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f180c090580 con 0x7f18241039d0 2024-04-20T11:31:39.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.042+0000 7f1822a00700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f180806ecc0 0x7f1808071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.043+0000 7f1822a00700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f180806ecc0 0x7f1808071180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f18140097b0 tx=0x7f1814006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:39.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.046+0000 7f181b400700 1 -- 172.21.15.117:0/1680200436 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f180c0593e0 con 0x7f18241039d0 2024-04-20T11:31:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:38 smithi173 ceph-mon[74165]: pgmap v575: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:38 smithi173 ceph-mon[74165]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:38 smithi173 ceph-mon[74165]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:39.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:38 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/91992031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:31:39.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.275+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f182419a190 con 0x7f180806ecc0 2024-04-20T11:31:39.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.278+0000 7f181b400700 1 -- 172.21.15.117:0/1680200436 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f182419a190 con 0x7f180806ecc0 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:31:39.279 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:31:39.280 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:31:39.280 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:31:39.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f180806ecc0 msgr2=0x7f1808071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f180806ecc0 0x7f1808071180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f18140097b0 tx=0x7f1814006d20 comp rx=0 tx=0).stop 2024-04-20T11:31:39.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 msgr2=0x7f1824199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 0x7f1824199350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f180c0092e0 tx=0x7f180c00be20 comp rx=0 tx=0).stop 2024-04-20T11:31:39.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 shutdown_connections 2024-04-20T11:31:39.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f180806ecc0 0x7f1808071180 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1824103020 0x7f1824198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 --2- 172.21.15.117:0/1680200436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18241039d0 0x7f1824199350 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 >> 172.21.15.117:0/1680200436 conn(0x7f18240fe8b0 msgr2=0x7f18240ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:39.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 shutdown_connections 2024-04-20T11:31:39.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.281+0000 7f1829a00700 1 -- 172.21.15.117:0/1680200436 wait complete. 2024-04-20T11:31:39.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.432+0000 7f1b3d200700 1 -- 172.21.15.117:0/2819819045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 msgr2=0x7f1b38102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.432+0000 7f1b3d200700 1 --2- 172.21.15.117:0/2819819045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 0x7f1b38102170 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009910 tx=0x7f1b28009c20 comp rx=0 tx=0).stop 2024-04-20T11:31:39.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.432+0000 7f1b3d200700 1 -- 172.21.15.117:0/2819819045 shutdown_connections 2024-04-20T11:31:39.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.432+0000 7f1b3d200700 1 --2- 172.21.15.117:0/2819819045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b381026b0 0x7f1b3810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.432+0000 7f1b3d200700 1 --2- 172.21.15.117:0/2819819045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 0x7f1b38102170 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.432+0000 7f1b3d200700 1 -- 172.21.15.117:0/2819819045 >> 172.21.15.117:0/2819819045 conn(0x7f1b380fb5c0 msgr2=0x7f1b380fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:39.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.432+0000 7f1b3d200700 1 -- 172.21.15.117:0/2819819045 shutdown_connections 2024-04-20T11:31:39.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.432+0000 7f1b3d200700 1 -- 172.21.15.117:0/2819819045 wait complete. 2024-04-20T11:31:39.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.433+0000 7f1b3d200700 1 Processor -- start 2024-04-20T11:31:39.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.433+0000 7f1b3d200700 1 -- start start 2024-04-20T11:31:39.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b3d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 0x7f1b380ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b3d200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b381026b0 0x7f1b380fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b3d200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b38100470 con 0x7f1b38101d90 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b3d200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b381005e0 con 0x7f1b381026b0 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b35e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 0x7f1b380ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b35e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 0x7f1b380ff9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34754/0 (socket says 172.21.15.117:34754) 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b35e00700 1 -- 172.21.15.117:0/1588447800 learned_addr learned my addr 172.21.15.117:0/1588447800 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b35400700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b381026b0 0x7f1b380fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b35e00700 1 -- 172.21.15.117:0/1588447800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b381026b0 msgr2=0x7f1b380fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b35e00700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b381026b0 0x7f1b380fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.434+0000 7f1b35e00700 1 -- 172.21.15.117:0/1588447800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280095f0 con 0x7f1b38101d90 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.435+0000 7f1b35e00700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 0x7f1b380ff9f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800bda0 tx=0x7f1b2800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.435+0000 7f1b2ea00700 1 -- 172.21.15.117:0/1588447800 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b2801d070 con 0x7f1b38101d90 2024-04-20T11:31:39.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.435+0000 7f1b2ea00700 1 -- 172.21.15.117:0/1588447800 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b2800f460 con 0x7f1b38101d90 2024-04-20T11:31:39.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.435+0000 7f1b2ea00700 1 -- 172.21.15.117:0/1588447800 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b28017630 con 0x7f1b38101d90 2024-04-20T11:31:39.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.435+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b38100860 con 0x7f1b38101d90 2024-04-20T11:31:39.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.435+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b381a3420 con 0x7f1b38101d90 2024-04-20T11:31:39.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.436+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b380696e0 con 0x7f1b38101d90 2024-04-20T11:31:39.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.437+0000 7f1b2ea00700 1 -- 172.21.15.117:0/1588447800 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1b2800f5d0 con 0x7f1b38101d90 2024-04-20T11:31:39.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.438+0000 7f1b2ea00700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1b1c06ecc0 0x7f1b1c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:39.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.438+0000 7f1b2ea00700 1 -- 172.21.15.117:0/1588447800 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1b280901b0 con 0x7f1b38101d90 2024-04-20T11:31:39.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.439+0000 7f1b35400700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1b1c06ecc0 0x7f1b1c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:39.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.439+0000 7f1b35400700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1b1c06ecc0 0x7f1b1c071180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1b24006fd0 tx=0x7f1b24008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:39.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.443+0000 7f1b2ea00700 1 -- 172.21.15.117:0/1588447800 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1b28058f70 con 0x7f1b38101d90 2024-04-20T11:31:39.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.740+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1b38100860 con 0x7f1b38101d90 2024-04-20T11:31:39.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.741+0000 7f1b2ea00700 1 -- 172.21.15.117:0/1588447800 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f1b38100860 con 0x7f1b38101d90 2024-04-20T11:31:39.742 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:31:39.742 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:31:39.742 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:31:39.742 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:31:39.742 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:31:39.742 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:31:39.742 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:31:39.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1b1c06ecc0 msgr2=0x7f1b1c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1b1c06ecc0 0x7f1b1c071180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1b24006fd0 tx=0x7f1b24008040 comp rx=0 tx=0).stop 2024-04-20T11:31:39.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 msgr2=0x7f1b380ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:39.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 0x7f1b380ff9f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800bda0 tx=0x7f1b2800be80 comp rx=0 tx=0).stop 2024-04-20T11:31:39.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 shutdown_connections 2024-04-20T11:31:39.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1b1c06ecc0 0x7f1b1c071180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b38101d90 0x7f1b380ff9f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 --2- 172.21.15.117:0/1588447800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b381026b0 0x7f1b380fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:39.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 >> 172.21.15.117:0/1588447800 conn(0x7f1b380fb5c0 msgr2=0x7f1b381033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:39.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.744+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 shutdown_connections 2024-04-20T11:31:39.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:31:39.745+0000 7f1b3d200700 1 -- 172.21.15.117:0/1588447800 wait complete. 2024-04-20T11:31:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:39 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4218800419' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:31:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:39 smithi117 ceph-mon[99036]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:40.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:39 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1588447800' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:31:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:39 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4218800419' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:31:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:39 smithi173 ceph-mon[74165]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:31:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:39 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1588447800' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:31:41.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:40 smithi117 ceph-mon[99036]: pgmap v576: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:41.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:40 smithi173 ceph-mon[74165]: pgmap v576: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:43.064 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:42 smithi117 ceph-mon[99036]: pgmap v577: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:43.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:42 smithi173 ceph-mon[74165]: pgmap v577: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:44.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:31:44.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:31:45.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:44 smithi117 ceph-mon[99036]: pgmap v578: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:45.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:44 smithi173 ceph-mon[74165]: pgmap v578: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:46 smithi117 ceph-mon[99036]: pgmap v579: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:46 smithi173 ceph-mon[74165]: pgmap v579: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:49.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:48 smithi117 ceph-mon[99036]: pgmap v580: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:49.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:48 smithi173 ceph-mon[74165]: pgmap v580: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:51.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:50 smithi117 ceph-mon[99036]: pgmap v581: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:51.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:50 smithi173 ceph-mon[74165]: pgmap v581: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:53.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:52 smithi117 ceph-mon[99036]: pgmap v582: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:52 smithi173 ceph-mon[74165]: pgmap v582: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:55.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:54 smithi117 ceph-mon[99036]: pgmap v583: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:55.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:54 smithi173 ceph-mon[74165]: pgmap v583: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:57.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:56 smithi117 ceph-mon[99036]: pgmap v584: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:57.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:56 smithi173 ceph-mon[74165]: pgmap v584: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:59.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:58 smithi117 ceph-mon[99036]: pgmap v585: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:59.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:31:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:31:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:58 smithi173 ceph-mon[74165]: pgmap v585: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:31:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:31:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:01.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:00 smithi117 ceph-mon[99036]: pgmap v586: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:01.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:00 smithi173 ceph-mon[74165]: pgmap v586: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:03.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:02 smithi117 ceph-mon[99036]: pgmap v587: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:03.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:02 smithi173 ceph-mon[74165]: pgmap v587: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:05.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:04 smithi117 ceph-mon[99036]: pgmap v588: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:05.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:04 smithi173 ceph-mon[74165]: pgmap v588: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:07.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:06 smithi117 ceph-mon[99036]: pgmap v589: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:06 smithi173 ceph-mon[74165]: pgmap v589: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:09.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:08 smithi117 ceph-mon[99036]: pgmap v590: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:08 smithi173 ceph-mon[74165]: pgmap v590: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:09.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.917+0000 7f5909a00700 1 -- 172.21.15.117:0/3681611330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073310 msgr2=0x7f59040736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:09.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.917+0000 7f5909a00700 1 --2- 172.21.15.117:0/3681611330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073310 0x7f59040736f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009910 tx=0x7f58f4009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:09.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.917+0000 7f5909a00700 1 -- 172.21.15.117:0/3681611330 shutdown_connections 2024-04-20T11:32:09.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.917+0000 7f5909a00700 1 --2- 172.21.15.117:0/3681611330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5904073c30 0x7f590410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:09.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.917+0000 7f5909a00700 1 --2- 172.21.15.117:0/3681611330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073310 0x7f59040736f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:09.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.917+0000 7f5909a00700 1 -- 172.21.15.117:0/3681611330 >> 172.21.15.117:0/3681611330 conn(0x7f59040fc6a0 msgr2=0x7f59040feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:09.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.917+0000 7f5909a00700 1 -- 172.21.15.117:0/3681611330 shutdown_connections 2024-04-20T11:32:09.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.917+0000 7f5909a00700 1 -- 172.21.15.117:0/3681611330 wait complete. 2024-04-20T11:32:09.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.918+0000 7f5909a00700 1 Processor -- start 2024-04-20T11:32:09.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.918+0000 7f5909a00700 1 -- start start 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.918+0000 7f5909a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5904073310 0x7f5904198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.918+0000 7f5909a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073c30 0x7f5904199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.918+0000 7f5909a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5904199a20 con 0x7f5904073c30 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.918+0000 7f5909a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590419d6f0 con 0x7f5904073310 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.919+0000 7f5902000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073c30 0x7f5904199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.919+0000 7f5902000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073c30 0x7f5904199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41076/0 (socket says 172.21.15.117:41076) 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.919+0000 7f5902000700 1 -- 172.21.15.117:0/240195282 learned_addr learned my addr 172.21.15.117:0/240195282 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.919+0000 7f5902a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5904073310 0x7f5904198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.919+0000 7f5902000700 1 -- 172.21.15.117:0/240195282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5904073310 msgr2=0x7f5904198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.919+0000 7f5902000700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5904073310 0x7f5904198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.919+0000 7f5902000700 1 -- 172.21.15.117:0/240195282 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40095f0 con 0x7f5904073c30 2024-04-20T11:32:09.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.919+0000 7f5902a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5904073310 0x7f5904198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:09.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.920+0000 7f5902000700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073c30 0x7f5904199340 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b4b0 tx=0x7f58f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:09.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.920+0000 7f58fb400700 1 -- 172.21.15.117:0/240195282 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f0011800 con 0x7f5904073c30 2024-04-20T11:32:09.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.920+0000 7f58fb400700 1 -- 172.21.15.117:0/240195282 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58f0011e20 con 0x7f5904073c30 2024-04-20T11:32:09.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.920+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590419d9d0 con 0x7f5904073c30 2024-04-20T11:32:09.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.920+0000 7f58fb400700 1 -- 172.21.15.117:0/240195282 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f000f2e0 con 0x7f5904073c30 2024-04-20T11:32:09.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.920+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590419df20 con 0x7f5904073c30 2024-04-20T11:32:09.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.921+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59040749c0 con 0x7f5904073c30 2024-04-20T11:32:09.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.923+0000 7f58fb400700 1 -- 172.21.15.117:0/240195282 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f58f0010380 con 0x7f5904073c30 2024-04-20T11:32:09.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.924+0000 7f58fb400700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58e806ecd0 0x7f58e8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:09.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.924+0000 7f58fb400700 1 -- 172.21.15.117:0/240195282 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f58f008ea70 con 0x7f5904073c30 2024-04-20T11:32:09.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.924+0000 7f5902a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58e806ecd0 0x7f58e8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:09.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.925+0000 7f5902a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58e806ecd0 0x7f58e8071190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006010 tx=0x7f58f400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:09.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:09.929+0000 7f58fb400700 1 -- 172.21.15.117:0/240195282 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f58f00577b0 con 0x7f5904073c30 2024-04-20T11:32:10.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.177+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5904107860 con 0x7f58e806ecd0 2024-04-20T11:32:10.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.179+0000 7f58fb400700 1 -- 172.21.15.117:0/240195282 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f58f4005480 con 0x7f58e806ecd0 2024-04-20T11:32:10.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.182+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58e806ecd0 msgr2=0x7f58e8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:10.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.182+0000 7f5909a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58e806ecd0 0x7f58e8071190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006010 tx=0x7f58f400b540 comp rx=0 tx=0).stop 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.182+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073c30 msgr2=0x7f5904199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.182+0000 7f5909a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073c30 0x7f5904199340 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b4b0 tx=0x7f58f000b870 comp rx=0 tx=0).stop 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.183+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 shutdown_connections 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.183+0000 7f5909a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f58e806ecd0 0x7f58e8071190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.183+0000 7f5909a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5904073310 0x7f5904198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.183+0000 7f5909a00700 1 --2- 172.21.15.117:0/240195282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5904073c30 0x7f5904199340 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.183+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 >> 172.21.15.117:0/240195282 conn(0x7f59040fc6a0 msgr2=0x7f5904105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.183+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 shutdown_connections 2024-04-20T11:32:10.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.183+0000 7f5909a00700 1 -- 172.21.15.117:0/240195282 wait complete. 2024-04-20T11:32:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:32:10.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.346+0000 7f3f29000700 1 -- 172.21.15.117:0/4026073478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24103110 msgr2=0x7f3f241034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:10.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.346+0000 7f3f29000700 1 --2- 172.21.15.117:0/4026073478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24103110 0x7f3f241034f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009910 tx=0x7f3f14009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:10.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.346+0000 7f3f29000700 1 -- 172.21.15.117:0/4026073478 shutdown_connections 2024-04-20T11:32:10.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.346+0000 7f3f29000700 1 --2- 172.21.15.117:0/4026073478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f24104a00 0x7f3f24104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.346+0000 7f3f29000700 1 --2- 172.21.15.117:0/4026073478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24103110 0x7f3f241034f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.346+0000 7f3f29000700 1 -- 172.21.15.117:0/4026073478 >> 172.21.15.117:0/4026073478 conn(0x7f3f240fe880 msgr2=0x7f3f24100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:10.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.346+0000 7f3f29000700 1 -- 172.21.15.117:0/4026073478 shutdown_connections 2024-04-20T11:32:10.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.346+0000 7f3f29000700 1 -- 172.21.15.117:0/4026073478 wait complete. 2024-04-20T11:32:10.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.347+0000 7f3f29000700 1 Processor -- start 2024-04-20T11:32:10.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.348+0000 7f3f29000700 1 -- start start 2024-04-20T11:32:10.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.348+0000 7f3f29000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f24103110 0x7f3f24198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:10.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.348+0000 7f3f29000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24104a00 0x7f3f24199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:10.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.348+0000 7f3f29000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24199a00 con 0x7f3f24104a00 2024-04-20T11:32:10.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.348+0000 7f3f29000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2419d680 con 0x7f3f24103110 2024-04-20T11:32:10.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.348+0000 7f3f21400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24104a00 0x7f3f24199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:10.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.348+0000 7f3f21400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24104a00 0x7f3f24199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41088/0 (socket says 172.21.15.117:41088) 2024-04-20T11:32:10.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.348+0000 7f3f21400700 1 -- 172.21.15.117:0/2557627664 learned_addr learned my addr 172.21.15.117:0/2557627664 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:10.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f21e00700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f24103110 0x7f3f24198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:10.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f21400700 1 -- 172.21.15.117:0/2557627664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f24103110 msgr2=0x7f3f24198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:10.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f21400700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f24103110 0x7f3f24198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f21400700 1 -- 172.21.15.117:0/2557627664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f140095f0 con 0x7f3f24104a00 2024-04-20T11:32:10.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f21e00700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f24103110 0x7f3f24198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:10.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f21400700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24104a00 0x7f3f24199320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000c8f0 tx=0x7f3f1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:10.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f1aa00700 1 -- 172.21.15.117:0/2557627664 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f10007a70 con 0x7f3f24104a00 2024-04-20T11:32:10.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f1aa00700 1 -- 172.21.15.117:0/2557627664 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f10004cf0 con 0x7f3f24104a00 2024-04-20T11:32:10.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f1aa00700 1 -- 172.21.15.117:0/2557627664 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f10005610 con 0x7f3f24104a00 2024-04-20T11:32:10.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.349+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2419d960 con 0x7f3f24104a00 2024-04-20T11:32:10.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.350+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2419df30 con 0x7f3f24104a00 2024-04-20T11:32:10.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.351+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2410b460 con 0x7f3f24104a00 2024-04-20T11:32:10.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.352+0000 7f3f1aa00700 1 -- 172.21.15.117:0/2557627664 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3f10004750 con 0x7f3f24104a00 2024-04-20T11:32:10.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.353+0000 7f3f1aa00700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f0806ecd0 0x7f3f08071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:10.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.353+0000 7f3f1aa00700 1 -- 172.21.15.117:0/2557627664 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3f10013070 con 0x7f3f24104a00 2024-04-20T11:32:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.353+0000 7f3f21e00700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f0806ecd0 0x7f3f08071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.354+0000 7f3f21e00700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f0806ecd0 0x7f3f08071190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f14006010 tx=0x7f3f1400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:10.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.358+0000 7f3f1aa00700 1 -- 172.21.15.117:0/2557627664 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3f100579e0 con 0x7f3f24104a00 2024-04-20T11:32:10.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.600+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f2419a0a0 con 0x7f3f0806ecd0 2024-04-20T11:32:10.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.603+0000 7f3f1aa00700 1 -- 172.21.15.117:0/2557627664 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3f2419a0a0 con 0x7f3f0806ecd0 2024-04-20T11:32:10.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.606+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f0806ecd0 msgr2=0x7f3f08071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:10.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.606+0000 7f3f29000700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f0806ecd0 0x7f3f08071190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f14006010 tx=0x7f3f1400b540 comp rx=0 tx=0).stop 2024-04-20T11:32:10.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.606+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24104a00 msgr2=0x7f3f24199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:10.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.606+0000 7f3f29000700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24104a00 0x7f3f24199320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000c8f0 tx=0x7f3f1000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.607+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 shutdown_connections 2024-04-20T11:32:10.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.607+0000 7f3f29000700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3f0806ecd0 0x7f3f08071190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.607+0000 7f3f29000700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f24103110 0x7f3f24198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.607+0000 7f3f29000700 1 --2- 172.21.15.117:0/2557627664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f24104a00 0x7f3f24199320 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.607+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 >> 172.21.15.117:0/2557627664 conn(0x7f3f240fe880 msgr2=0x7f3f240ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:10.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.607+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 shutdown_connections 2024-04-20T11:32:10.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.607+0000 7f3f29000700 1 -- 172.21.15.117:0/2557627664 wait complete. 2024-04-20T11:32:10.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.763+0000 7fa802000700 1 -- 172.21.15.117:0/3515598020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc102660 msgr2=0x7fa7fc10ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:10.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.763+0000 7fa802000700 1 --2- 172.21.15.117:0/3515598020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc102660 0x7fa7fc10ab50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009910 tx=0x7fa7e8009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:10.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.763+0000 7fa802000700 1 -- 172.21.15.117:0/3515598020 shutdown_connections 2024-04-20T11:32:10.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.763+0000 7fa802000700 1 --2- 172.21.15.117:0/3515598020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc102660 0x7fa7fc10ab50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.763+0000 7fa802000700 1 --2- 172.21.15.117:0/3515598020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7fc101d40 0x7fa7fc102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.763+0000 7fa802000700 1 -- 172.21.15.117:0/3515598020 >> 172.21.15.117:0/3515598020 conn(0x7fa7fc076750 msgr2=0x7fa7fc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:10.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.763+0000 7fa802000700 1 -- 172.21.15.117:0/3515598020 shutdown_connections 2024-04-20T11:32:10.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.763+0000 7fa802000700 1 -- 172.21.15.117:0/3515598020 wait complete. 2024-04-20T11:32:10.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.764+0000 7fa802000700 1 Processor -- start 2024-04-20T11:32:10.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.764+0000 7fa802000700 1 -- start start 2024-04-20T11:32:10.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa802000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc101d40 0x7fa7fc19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:10.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa802000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7fc102660 0x7fa7fc19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:10.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa802000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc19d4a0 con 0x7fa7fc101d40 2024-04-20T11:32:10.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa802000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc196a80 con 0x7fa7fc102660 2024-04-20T11:32:10.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa7fb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc101d40 0x7fa7fc19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:10.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa7fb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc101d40 0x7fa7fc19c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41110/0 (socket says 172.21.15.117:41110) 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa7fb400700 1 -- 172.21.15.117:0/2376517995 learned_addr learned my addr 172.21.15.117:0/2376517995 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa7faa00700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7fc102660 0x7fa7fc19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa7fb400700 1 -- 172.21.15.117:0/2376517995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7fc102660 msgr2=0x7fa7fc19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa7fb400700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7fc102660 0x7fa7fc19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.765+0000 7fa7fb400700 1 -- 172.21.15.117:0/2376517995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80095f0 con 0x7fa7fc101d40 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.766+0000 7fa7fb400700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc101d40 0x7fa7fc19c940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00b4d0 tx=0x7fa7ec00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.766+0000 7fa7f3e00700 1 -- 172.21.15.117:0/2376517995 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7ec011800 con 0x7fa7fc101d40 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.766+0000 7fa7f3e00700 1 -- 172.21.15.117:0/2376517995 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7ec011e20 con 0x7fa7fc101d40 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.766+0000 7fa7f3e00700 1 -- 172.21.15.117:0/2376517995 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7ec00f2e0 con 0x7fa7fc101d40 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.766+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc196d60 con 0x7fa7fc101d40 2024-04-20T11:32:10.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.766+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc1972b0 con 0x7fa7fc101d40 2024-04-20T11:32:10.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.767+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7fc105530 con 0x7fa7fc101d40 2024-04-20T11:32:10.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.768+0000 7fa7f3e00700 1 -- 172.21.15.117:0/2376517995 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa7ec010380 con 0x7fa7fc101d40 2024-04-20T11:32:10.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.769+0000 7fa7f3e00700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa7e006ecc0 0x7fa7e0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:10.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.769+0000 7fa7f3e00700 1 -- 172.21.15.117:0/2376517995 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa7ec08e950 con 0x7fa7fc101d40 2024-04-20T11:32:10.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.770+0000 7fa7faa00700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa7e006ecc0 0x7fa7e0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:10.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.770+0000 7fa7faa00700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa7e006ecc0 0x7fa7e0071180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc197c40 tx=0x7fa7e800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:10.774+0000 7fa7f3e00700 1 -- 172.21.15.117:0/2376517995 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa7ec05b240 con 0x7fa7fc101d40 2024-04-20T11:32:11.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.010+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7fc197fb0 con 0x7fa7e006ecc0 2024-04-20T11:32:11.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.026+0000 7fa7f3e00700 1 -- 172.21.15.117:0/2376517995 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa7fc197fb0 con 0x7fa7e006ecc0 2024-04-20T11:32:11.025 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:32:11.025 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (20m) 12m ago 34m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:32:11.025 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (35m) 12m ago 35m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (33m) 11m ago 33m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (16m) 12m ago 34m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (16m) 11m ago 33m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (19m) 12m ago 34m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (12m) 12m ago 27m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (12m) 12m ago 27m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (27m) 11m ago 27m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (12m) 11m ago 27m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (23m) 12m ago 36m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (22m) 11m ago 33m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (18m) 12m ago 36m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (18m) 11m ago 33m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (21m) 12m ago 34m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:32:11.026 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (21m) 11m ago 33m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:32:11.027 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (16m) 12m ago 32m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:32:11.027 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (15m) 12m ago 31m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:32:11.027 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (15m) 12m ago 31m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:32:11.027 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (14m) 11m ago 30m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:32:11.027 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (14m) 11m ago 29m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:32:11.027 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (13m) 11m ago 29m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:32:11.027 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (21m) 12m ago 34m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa7e006ecc0 msgr2=0x7fa7e0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa7e006ecc0 0x7fa7e0071180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc197c40 tx=0x7fa7e800b540 comp rx=0 tx=0).stop 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc101d40 msgr2=0x7fa7fc19c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc101d40 0x7fa7fc19c940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00b4d0 tx=0x7fa7ec00b890 comp rx=0 tx=0).stop 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 shutdown_connections 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa7e006ecc0 0x7fa7e0071180 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7fc101d40 0x7fa7fc19c940 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 --2- 172.21.15.117:0/2376517995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7fc102660 0x7fa7fc19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.028+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 >> 172.21.15.117:0/2376517995 conn(0x7fa7fc076750 msgr2=0x7fa7fc103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.029+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 shutdown_connections 2024-04-20T11:32:11.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.029+0000 7fa802000700 1 -- 172.21.15.117:0/2376517995 wait complete. 2024-04-20T11:32:11.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:10 smithi117 ceph-mon[99036]: pgmap v591: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:11.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:10 smithi117 ceph-mon[99036]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:10 smithi173 ceph-mon[74165]: pgmap v591: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:10 smithi173 ceph-mon[74165]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:11.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.183+0000 7f98f2c00700 1 -- 172.21.15.117:0/1647344175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec100e40 msgr2=0x7f98ec101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.183+0000 7f98f2c00700 1 --2- 172.21.15.117:0/1647344175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec100e40 0x7f98ec101220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009910 tx=0x7f98dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:11.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.183+0000 7f98f2c00700 1 -- 172.21.15.117:0/1647344175 shutdown_connections 2024-04-20T11:32:11.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.183+0000 7f98f2c00700 1 --2- 172.21.15.117:0/1647344175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98ec1017f0 0x7f98ec105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.183+0000 7f98f2c00700 1 --2- 172.21.15.117:0/1647344175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec100e40 0x7f98ec101220 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.183+0000 7f98f2c00700 1 -- 172.21.15.117:0/1647344175 >> 172.21.15.117:0/1647344175 conn(0x7f98ec0fc6d0 msgr2=0x7f98ec0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:11.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.184+0000 7f98f2c00700 1 -- 172.21.15.117:0/1647344175 shutdown_connections 2024-04-20T11:32:11.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.184+0000 7f98f2c00700 1 -- 172.21.15.117:0/1647344175 wait complete. 2024-04-20T11:32:11.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.185+0000 7f98f2c00700 1 Processor -- start 2024-04-20T11:32:11.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.185+0000 7f98f2c00700 1 -- start start 2024-04-20T11:32:11.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.185+0000 7f98f2c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98ec100e40 0x7f98ec193550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:11.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.185+0000 7f98f2c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec1017f0 0x7f98ec193a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:11.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.185+0000 7f98f2c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec197590 con 0x7f98ec1017f0 2024-04-20T11:32:11.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.185+0000 7f98f2c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec197700 con 0x7f98ec100e40 2024-04-20T11:32:11.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.186+0000 7f98eb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec1017f0 0x7f98ec193a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:11.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.186+0000 7f98eb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec1017f0 0x7f98ec193a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41136/0 (socket says 172.21.15.117:41136) 2024-04-20T11:32:11.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.186+0000 7f98eb400700 1 -- 172.21.15.117:0/3400227400 learned_addr learned my addr 172.21.15.117:0/3400227400 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:11.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.186+0000 7f98ebe00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98ec100e40 0x7f98ec193550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:11.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.186+0000 7f98eb400700 1 -- 172.21.15.117:0/3400227400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98ec100e40 msgr2=0x7f98ec193550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.186+0000 7f98eb400700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98ec100e40 0x7f98ec193550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.186+0000 7f98eb400700 1 -- 172.21.15.117:0/3400227400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc0095f0 con 0x7f98ec1017f0 2024-04-20T11:32:11.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.186+0000 7f98eb400700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec1017f0 0x7f98ec193a90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c960 tx=0x7f98d800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:11.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.187+0000 7f98e8c00700 1 -- 172.21.15.117:0/3400227400 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d8007a70 con 0x7f98ec1017f0 2024-04-20T11:32:11.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.187+0000 7f98e8c00700 1 -- 172.21.15.117:0/3400227400 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98d8007bd0 con 0x7f98ec1017f0 2024-04-20T11:32:11.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.187+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec1940f0 con 0x7f98ec1017f0 2024-04-20T11:32:11.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.187+0000 7f98e8c00700 1 -- 172.21.15.117:0/3400227400 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d80186c0 con 0x7f98ec1017f0 2024-04-20T11:32:11.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.187+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec1a7490 con 0x7f98ec1017f0 2024-04-20T11:32:11.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.188+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec104fc0 con 0x7f98ec1017f0 2024-04-20T11:32:11.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.189+0000 7f98e8c00700 1 -- 172.21.15.117:0/3400227400 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f98d800faa0 con 0x7f98ec1017f0 2024-04-20T11:32:11.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.190+0000 7f98e8c00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f98d006ecd0 0x7f98d0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:11.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.190+0000 7f98e8c00700 1 -- 172.21.15.117:0/3400227400 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f98d808ee70 con 0x7f98ec1017f0 2024-04-20T11:32:11.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.191+0000 7f98ebe00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f98d006ecd0 0x7f98d0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:11.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.191+0000 7f98ebe00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f98d006ecd0 0x7f98d0071190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00b5c0 tx=0x7f98dc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:11.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.195+0000 7f98e8c00700 1 -- 172.21.15.117:0/3400227400 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f98d805b760 con 0x7f98ec1017f0 2024-04-20T11:32:11.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.509+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f98ec102320 con 0x7f98ec1017f0 2024-04-20T11:32:11.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.510+0000 7f98e8c00700 1 -- 172.21.15.117:0/3400227400 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f98d805b2f0 con 0x7f98ec1017f0 2024-04-20T11:32:11.510 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:32:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:32:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:32:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:32:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:32:11.511 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:32:11.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.514+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f98d006ecd0 msgr2=0x7f98d0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.514+0000 7f98f2c00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f98d006ecd0 0x7f98d0071190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00b5c0 tx=0x7f98dc01a040 comp rx=0 tx=0).stop 2024-04-20T11:32:11.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.514+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec1017f0 msgr2=0x7f98ec193a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.514+0000 7f98f2c00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec1017f0 0x7f98ec193a90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c960 tx=0x7f98d800cd20 comp rx=0 tx=0).stop 2024-04-20T11:32:11.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.515+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 shutdown_connections 2024-04-20T11:32:11.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.515+0000 7f98f2c00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f98d006ecd0 0x7f98d0071190 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.515+0000 7f98f2c00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98ec100e40 0x7f98ec193550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.515+0000 7f98f2c00700 1 --2- 172.21.15.117:0/3400227400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec1017f0 0x7f98ec193a90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.515+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 >> 172.21.15.117:0/3400227400 conn(0x7f98ec0fc6d0 msgr2=0x7f98ec0fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:11.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.515+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 shutdown_connections 2024-04-20T11:32:11.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.515+0000 7f98f2c00700 1 -- 172.21.15.117:0/3400227400 wait complete. 2024-04-20T11:32:11.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.671+0000 7fd025e00700 1 -- 172.21.15.117:0/1627485703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 msgr2=0x7fd020109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.671+0000 7fd025e00700 1 --2- 172.21.15.117:0/1627485703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 0x7fd020109290 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009910 tx=0x7fd00c009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:11.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.671+0000 7fd025e00700 1 -- 172.21.15.117:0/1627485703 shutdown_connections 2024-04-20T11:32:11.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.671+0000 7fd025e00700 1 --2- 172.21.15.117:0/1627485703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 0x7fd020109290 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.671+0000 7fd025e00700 1 --2- 172.21.15.117:0/1627485703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd020107520 0x7fd020107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.671+0000 7fd025e00700 1 -- 172.21.15.117:0/1627485703 >> 172.21.15.117:0/1627485703 conn(0x7fd0200768f0 msgr2=0x7fd020078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:11.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.671+0000 7fd025e00700 1 -- 172.21.15.117:0/1627485703 shutdown_connections 2024-04-20T11:32:11.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.671+0000 7fd025e00700 1 -- 172.21.15.117:0/1627485703 wait complete. 2024-04-20T11:32:11.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.672+0000 7fd025e00700 1 Processor -- start 2024-04-20T11:32:11.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.672+0000 7fd025e00700 1 -- start start 2024-04-20T11:32:11.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.672+0000 7fd025e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd020107520 0x7fd02019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd025e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 0x7fd02019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd025e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02019ded0 con 0x7fd020108e10 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd025e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201a1ba0 con 0x7fd020107520 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd01e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 0x7fd02019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd01e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 0x7fd02019d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41158/0 (socket says 172.21.15.117:41158) 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd01e000700 1 -- 172.21.15.117:0/3064613646 learned_addr learned my addr 172.21.15.117:0/3064613646 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd01ea00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd020107520 0x7fd02019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd01e000700 1 -- 172.21.15.117:0/3064613646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd020107520 msgr2=0x7fd02019d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd01e000700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd020107520 0x7fd02019d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd01e000700 1 -- 172.21.15.117:0/3064613646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0095f0 con 0x7fd020108e10 2024-04-20T11:32:11.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.673+0000 7fd01ea00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd020107520 0x7fd02019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.674+0000 7fd01e000700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 0x7fd02019d7f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0092e0 tx=0x7fd00c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.674+0000 7fd024a00700 1 -- 172.21.15.117:0/3064613646 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd00c01d070 con 0x7fd020108e10 2024-04-20T11:32:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.674+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0201a1e20 con 0x7fd020108e10 2024-04-20T11:32:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.674+0000 7fd024a00700 1 -- 172.21.15.117:0/3064613646 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd00c00f460 con 0x7fd020108e10 2024-04-20T11:32:11.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.674+0000 7fd024a00700 1 -- 172.21.15.117:0/3064613646 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd00c017630 con 0x7fd020108e10 2024-04-20T11:32:11.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.674+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0201a2390 con 0x7fd020108e10 2024-04-20T11:32:11.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.675+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd02010f890 con 0x7fd020108e10 2024-04-20T11:32:11.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.676+0000 7fd024a00700 1 -- 172.21.15.117:0/3064613646 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd00c00f5d0 con 0x7fd020108e10 2024-04-20T11:32:11.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.677+0000 7fd024a00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd00406ecd0 0x7fd004071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:11.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.677+0000 7fd024a00700 1 -- 172.21.15.117:0/3064613646 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd00c0903e0 con 0x7fd020108e10 2024-04-20T11:32:11.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.678+0000 7fd01ea00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd00406ecd0 0x7fd004071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:11.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.678+0000 7fd01ea00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd00406ecd0 0x7fd004071190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd010007900 tx=0x7fd010008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:11.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.682+0000 7fd024a00700 1 -- 172.21.15.117:0/3064613646 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd00c059120 con 0x7fd020108e10 2024-04-20T11:32:11.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.943+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd020108920 con 0x7fd020108e10 2024-04-20T11:32:11.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.944+0000 7fd024a00700 1 -- 172.21.15.117:0/3064613646 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fd00c05c7e0 con 0x7fd020108e10 2024-04-20T11:32:11.944 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:32:11.944 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:32:11.944 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:32:11.944 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:32:11.944 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:32:11.945 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:32:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:32:11.947 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:32:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd00406ecd0 msgr2=0x7fd004071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd00406ecd0 0x7fd004071190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd010007900 tx=0x7fd010008040 comp rx=0 tx=0).stop 2024-04-20T11:32:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 msgr2=0x7fd02019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 0x7fd02019d7f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0092e0 tx=0x7fd00c00be20 comp rx=0 tx=0).stop 2024-04-20T11:32:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 shutdown_connections 2024-04-20T11:32:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd00406ecd0 0x7fd004071190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd020107520 0x7fd02019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 --2- 172.21.15.117:0/3064613646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd020108e10 0x7fd02019d7f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:11.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 >> 172.21.15.117:0/3064613646 conn(0x7fd0200768f0 msgr2=0x7fd0200775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:11.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 shutdown_connections 2024-04-20T11:32:11.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:11.947+0000 7fd025e00700 1 -- 172.21.15.117:0/3064613646 wait complete. 2024-04-20T11:32:11.949 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:32:12.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.103+0000 7fa2c5200700 1 -- 172.21.15.117:0/2018046698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0101e10 msgr2=0x7fa2c01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:12.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.103+0000 7fa2c5200700 1 --2- 172.21.15.117:0/2018046698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0101e10 0x7fa2c01021f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009910 tx=0x7fa2b0009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:12.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.103+0000 7fa2c5200700 1 -- 172.21.15.117:0/2018046698 shutdown_connections 2024-04-20T11:32:12.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.103+0000 7fa2c5200700 1 --2- 172.21.15.117:0/2018046698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2c0102730 0x7fa2c010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.103+0000 7fa2c5200700 1 --2- 172.21.15.117:0/2018046698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0101e10 0x7fa2c01021f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.103+0000 7fa2c5200700 1 -- 172.21.15.117:0/2018046698 >> 172.21.15.117:0/2018046698 conn(0x7fa2c00fb5c0 msgr2=0x7fa2c00fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:12.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.104+0000 7fa2c5200700 1 -- 172.21.15.117:0/2018046698 shutdown_connections 2024-04-20T11:32:12.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.104+0000 7fa2c5200700 1 -- 172.21.15.117:0/2018046698 wait complete. 2024-04-20T11:32:12.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.105+0000 7fa2c5200700 1 Processor -- start 2024-04-20T11:32:12.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.105+0000 7fa2c5200700 1 -- start start 2024-04-20T11:32:12.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.105+0000 7fa2c5200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2c0101e10 0x7fa2c019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:12.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.105+0000 7fa2c5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0102730 0x7fa2c019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:12.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.105+0000 7fa2c5200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c019d450 con 0x7fa2c0102730 2024-04-20T11:32:12.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.105+0000 7fa2c5200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0196a30 con 0x7fa2c0101e10 2024-04-20T11:32:12.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.106+0000 7fa2bd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0102730 0x7fa2c019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:12.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.106+0000 7fa2bd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0102730 0x7fa2c019ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41166/0 (socket says 172.21.15.117:41166) 2024-04-20T11:32:12.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.106+0000 7fa2bd400700 1 -- 172.21.15.117:0/4156888981 learned_addr learned my addr 172.21.15.117:0/4156888981 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:12.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.106+0000 7fa2bde00700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2c0101e10 0x7fa2c019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:12.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.106+0000 7fa2bd400700 1 -- 172.21.15.117:0/4156888981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2c0101e10 msgr2=0x7fa2c019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:12.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.106+0000 7fa2bd400700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2c0101e10 0x7fa2c019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.106+0000 7fa2bd400700 1 -- 172.21.15.117:0/4156888981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00095f0 con 0x7fa2c0102730 2024-04-20T11:32:12.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.106+0000 7fa2bd400700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0102730 0x7fa2c019ce30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00b480 tx=0x7fa2ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.107+0000 7fa2b6a00700 1 -- 172.21.15.117:0/4156888981 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ac0118c0 con 0x7fa2c0102730 2024-04-20T11:32:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.107+0000 7fa2b6a00700 1 -- 172.21.15.117:0/4156888981 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2ac010380 con 0x7fa2c0102730 2024-04-20T11:32:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.107+0000 7fa2b6a00700 1 -- 172.21.15.117:0/4156888981 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ac00f4f0 con 0x7fa2c0102730 2024-04-20T11:32:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.107+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c0196d10 con 0x7fa2c0102730 2024-04-20T11:32:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.107+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c0197260 con 0x7fa2c0102730 2024-04-20T11:32:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.108+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c0105570 con 0x7fa2c0102730 2024-04-20T11:32:12.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.110+0000 7fa2b6a00700 1 -- 172.21.15.117:0/4156888981 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa2ac0104f0 con 0x7fa2c0102730 2024-04-20T11:32:12.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.110+0000 7fa2b6a00700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa2a406ecc0 0x7fa2a4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:12.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.111+0000 7fa2b6a00700 1 -- 172.21.15.117:0/4156888981 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa2ac08ea40 con 0x7fa2c0102730 2024-04-20T11:32:12.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.111+0000 7fa2bde00700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa2a406ecc0 0x7fa2a4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:12.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.111+0000 7fa2bde00700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa2a406ecc0 0x7fa2a4071180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000b5c0 tx=0x7fa2b001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:12.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.115+0000 7fa2b6a00700 1 -- 172.21.15.117:0/4156888981 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa2ac05b330 con 0x7fa2c0102730 2024-04-20T11:32:12.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:11 smithi117 ceph-mon[99036]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:12.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:11 smithi117 ceph-mon[99036]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:12.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:11 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3400227400' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:32:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:11 smithi173 ceph-mon[74165]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:11 smithi173 ceph-mon[74165]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:12.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:11 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3400227400' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:32:12.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.373+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2c0197f60 con 0x7fa2a406ecc0 2024-04-20T11:32:12.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.375+0000 7fa2b6a00700 1 -- 172.21.15.117:0/4156888981 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa2c0197f60 con 0x7fa2a406ecc0 2024-04-20T11:32:12.374 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:32:12.375 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:32:12.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa2a406ecc0 msgr2=0x7fa2a4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa2a406ecc0 0x7fa2a4071180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000b5c0 tx=0x7fa2b001a040 comp rx=0 tx=0).stop 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0102730 msgr2=0x7fa2c019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0102730 0x7fa2c019ce30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00b480 tx=0x7fa2ac00b840 comp rx=0 tx=0).stop 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 shutdown_connections 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa2a406ecc0 0x7fa2a4071180 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2c0101e10 0x7fa2c019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 --2- 172.21.15.117:0/4156888981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2c0102730 0x7fa2c019ce30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 >> 172.21.15.117:0/4156888981 conn(0x7fa2c00fb5c0 msgr2=0x7fa2c01030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 shutdown_connections 2024-04-20T11:32:12.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.378+0000 7fa2c5200700 1 -- 172.21.15.117:0/4156888981 wait complete. 2024-04-20T11:32:12.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.529+0000 7fd4e9400700 1 -- 172.21.15.117:0/1689596006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e41039d0 msgr2=0x7fd4e4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:12.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.529+0000 7fd4e9400700 1 --2- 172.21.15.117:0/1689596006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e41039d0 0x7fd4e4107a20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009910 tx=0x7fd4d0009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:12.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.529+0000 7fd4e9400700 1 -- 172.21.15.117:0/1689596006 shutdown_connections 2024-04-20T11:32:12.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.529+0000 7fd4e9400700 1 --2- 172.21.15.117:0/1689596006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e41039d0 0x7fd4e4107a20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.529+0000 7fd4e9400700 1 --2- 172.21.15.117:0/1689596006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4e4103020 0x7fd4e4103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.529+0000 7fd4e9400700 1 -- 172.21.15.117:0/1689596006 >> 172.21.15.117:0/1689596006 conn(0x7fd4e40fe8b0 msgr2=0x7fd4e4100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:12.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.529+0000 7fd4e9400700 1 -- 172.21.15.117:0/1689596006 shutdown_connections 2024-04-20T11:32:12.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.529+0000 7fd4e9400700 1 -- 172.21.15.117:0/1689596006 wait complete. 2024-04-20T11:32:12.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.530+0000 7fd4e9400700 1 Processor -- start 2024-04-20T11:32:12.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.530+0000 7fd4e9400700 1 -- start start 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.530+0000 7fd4e9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4103020 0x7fd4e4198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e9400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4e41039d0 0x7fd4e4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e9400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e41998e0 con 0x7fd4e4103020 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e9400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e419c550 con 0x7fd4e41039d0 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e2800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4103020 0x7fd4e4198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e2800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4103020 0x7fd4e4198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41180/0 (socket says 172.21.15.117:41180) 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e2800700 1 -- 172.21.15.117:0/2012230988 learned_addr learned my addr 172.21.15.117:0/2012230988 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e1e00700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4e41039d0 0x7fd4e4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e2800700 1 -- 172.21.15.117:0/2012230988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4e41039d0 msgr2=0x7fd4e4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e2800700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4e41039d0 0x7fd4e4199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.531+0000 7fd4e2800700 1 -- 172.21.15.117:0/2012230988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00095f0 con 0x7fd4e4103020 2024-04-20T11:32:12.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.532+0000 7fd4e2800700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4103020 0x7fd4e4198d10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400c8a0 tx=0x7fd4d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:12.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.532+0000 7fd4db400700 1 -- 172.21.15.117:0/2012230988 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4d400cea0 con 0x7fd4e4103020 2024-04-20T11:32:12.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.532+0000 7fd4db400700 1 -- 172.21.15.117:0/2012230988 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4d4004830 con 0x7fd4e4103020 2024-04-20T11:32:12.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.532+0000 7fd4db400700 1 -- 172.21.15.117:0/2012230988 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4d40055b0 con 0x7fd4e4103020 2024-04-20T11:32:12.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.532+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e419c830 con 0x7fd4e4103020 2024-04-20T11:32:12.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.532+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e419cd80 con 0x7fd4e4103020 2024-04-20T11:32:12.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.533+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e410b420 con 0x7fd4e4103020 2024-04-20T11:32:12.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.534+0000 7fd4db400700 1 -- 172.21.15.117:0/2012230988 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd4d40049a0 con 0x7fd4e4103020 2024-04-20T11:32:12.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.535+0000 7fd4db400700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4c806ecc0 0x7fd4c8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:12.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.535+0000 7fd4db400700 1 -- 172.21.15.117:0/2012230988 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd4d408eac0 con 0x7fd4e4103020 2024-04-20T11:32:12.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.535+0000 7fd4e1e00700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4c806ecc0 0x7fd4c8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:12.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.536+0000 7fd4e1e00700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4c806ecc0 0x7fd4c8071180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd4e419a230 tx=0x7fd4d0005360 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.540+0000 7fd4db400700 1 -- 172.21.15.117:0/2012230988 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd4d4057880 con 0x7fd4e4103020 2024-04-20T11:32:12.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.850+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd4e4104500 con 0x7fd4e4103020 2024-04-20T11:32:12.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.851+0000 7fd4db400700 1 -- 172.21.15.117:0/2012230988 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd4d401d090 con 0x7fd4e4103020 2024-04-20T11:32:12.851 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:32:12.851 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:32:12.851 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:32:12.851 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:32:12.851 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:32:12.851 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:32:12.851 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:32:12.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4c806ecc0 msgr2=0x7fd4c8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:12.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4c806ecc0 0x7fd4c8071180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd4e419a230 tx=0x7fd4d0005360 comp rx=0 tx=0).stop 2024-04-20T11:32:12.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4103020 msgr2=0x7fd4e4198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:12.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4103020 0x7fd4e4198d10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400c8a0 tx=0x7fd4d400cc60 comp rx=0 tx=0).stop 2024-04-20T11:32:12.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 shutdown_connections 2024-04-20T11:32:12.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4c806ecc0 0x7fd4c8071180 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4103020 0x7fd4e4198d10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 --2- 172.21.15.117:0/2012230988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4e41039d0 0x7fd4e4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:12.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 >> 172.21.15.117:0/2012230988 conn(0x7fd4e40fe8b0 msgr2=0x7fd4e40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:12.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 shutdown_connections 2024-04-20T11:32:12.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:12.854+0000 7fd4e9400700 1 -- 172.21.15.117:0/2012230988 wait complete. 2024-04-20T11:32:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:12 smithi117 ceph-mon[99036]: pgmap v592: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:12 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3064613646' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:32:13.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:12 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2012230988' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:32:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:12 smithi173 ceph-mon[74165]: pgmap v592: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:12 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3064613646' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:32:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:12 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2012230988' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:32:14.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:13 smithi173 ceph-mon[74165]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:14.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:13 smithi117 ceph-mon[99036]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:15.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:14 smithi173 ceph-mon[74165]: pgmap v593: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:15.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:14 smithi117 ceph-mon[99036]: pgmap v593: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:17.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:16 smithi117 ceph-mon[99036]: pgmap v594: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:17.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:16 smithi173 ceph-mon[74165]: pgmap v594: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:19.183 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:18 smithi117 ceph-mon[99036]: pgmap v595: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:18 smithi173 ceph-mon[74165]: pgmap v595: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:21.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:20 smithi117 ceph-mon[99036]: pgmap v596: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:21.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:20 smithi173 ceph-mon[74165]: pgmap v596: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:23.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:22 smithi117 ceph-mon[99036]: pgmap v597: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:23.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:22 smithi173 ceph-mon[74165]: pgmap v597: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:25.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:24 smithi117 ceph-mon[99036]: pgmap v598: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:24 smithi173 ceph-mon[74165]: pgmap v598: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:27.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:26 smithi117 ceph-mon[99036]: pgmap v599: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:26 smithi173 ceph-mon[74165]: pgmap v599: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:29.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:28 smithi117 ceph-mon[99036]: pgmap v600: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:28 smithi173 ceph-mon[74165]: pgmap v600: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:31.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:30 smithi173 ceph-mon[74165]: pgmap v601: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:31.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:30 smithi117 ceph-mon[99036]: pgmap v601: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:33.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:32 smithi117 ceph-mon[99036]: pgmap v602: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:33.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:32 smithi173 ceph-mon[74165]: pgmap v602: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:35.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:34 smithi117 ceph-mon[99036]: pgmap v603: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:35.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:34 smithi173 ceph-mon[74165]: pgmap v603: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:37.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:36 smithi117 ceph-mon[99036]: pgmap v604: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:36 smithi173 ceph-mon[74165]: pgmap v604: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:39.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:38 smithi117 ceph-mon[99036]: pgmap v605: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:38 smithi173 ceph-mon[74165]: pgmap v605: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:41.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:40 smithi117 ceph-mon[99036]: pgmap v606: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:40 smithi173 ceph-mon[74165]: pgmap v606: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:43.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.025+0000 7f3252a00700 1 -- 172.21.15.117:0/1793436685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c1002e0 msgr2=0x7f324c10cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.025+0000 7f3252a00700 1 --2- 172.21.15.117:0/1793436685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c1002e0 0x7f324c10cfa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3238009910 tx=0x7f3238009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:43.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.025+0000 7f3252a00700 1 -- 172.21.15.117:0/1793436685 shutdown_connections 2024-04-20T11:32:43.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.025+0000 7f3252a00700 1 --2- 172.21.15.117:0/1793436685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c1002e0 0x7f324c10cfa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.025+0000 7f3252a00700 1 --2- 172.21.15.117:0/1793436685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324c0ff930 0x7f324c0ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.025+0000 7f3252a00700 1 -- 172.21.15.117:0/1793436685 >> 172.21.15.117:0/1793436685 conn(0x7f324c076750 msgr2=0x7f324c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:43.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.025+0000 7f3252a00700 1 -- 172.21.15.117:0/1793436685 shutdown_connections 2024-04-20T11:32:43.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.025+0000 7f3252a00700 1 -- 172.21.15.117:0/1793436685 wait complete. 2024-04-20T11:32:43.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.026+0000 7f3252a00700 1 Processor -- start 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.026+0000 7f3252a00700 1 -- start start 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f3252a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ff930 0x7f324c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f3252a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324c1002e0 0x7f324c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f3252a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c198b10 con 0x7f324c0ff930 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f3252a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c198c80 con 0x7f324c1002e0 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f324be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ff930 0x7f324c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f324be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ff930 0x7f324c19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40356/0 (socket says 172.21.15.117:40356) 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f324be00700 1 -- 172.21.15.117:0/3494692061 learned_addr learned my addr 172.21.15.117:0/3494692061 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f324b400700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324c1002e0 0x7f324c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f324be00700 1 -- 172.21.15.117:0/3494692061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324c1002e0 msgr2=0x7f324c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f324be00700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324c1002e0 0x7f324c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.027+0000 7f324be00700 1 -- 172.21.15.117:0/3494692061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380095f0 con 0x7f324c0ff930 2024-04-20T11:32:43.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.028+0000 7f324be00700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ff930 0x7f324c19ea60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c8a0 tx=0x7f323c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.028+0000 7f3248c00700 1 -- 172.21.15.117:0/3494692061 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f323c00cea0 con 0x7f324c0ff930 2024-04-20T11:32:43.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.028+0000 7f3248c00700 1 -- 172.21.15.117:0/3494692061 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f323c004830 con 0x7f324c0ff930 2024-04-20T11:32:43.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.028+0000 7f3248c00700 1 -- 172.21.15.117:0/3494692061 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f323c0055b0 con 0x7f324c0ff930 2024-04-20T11:32:43.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.028+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c198f60 con 0x7f324c0ff930 2024-04-20T11:32:43.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.028+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c1994b0 con 0x7f324c0ff930 2024-04-20T11:32:43.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.029+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c068df0 con 0x7f324c0ff930 2024-04-20T11:32:43.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.030+0000 7f3248c00700 1 -- 172.21.15.117:0/3494692061 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f323c0049a0 con 0x7f324c0ff930 2024-04-20T11:32:43.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.031+0000 7f3248c00700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f323006ecc0 0x7f3230071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.032+0000 7f3248c00700 1 -- 172.21.15.117:0/3494692061 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f323c08eac0 con 0x7f324c0ff930 2024-04-20T11:32:43.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.032+0000 7f324b400700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f323006ecc0 0x7f3230071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.032+0000 7f324b400700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f323006ecc0 0x7f3230071180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f32380059d0 tx=0x7f3238005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.037+0000 7f3248c00700 1 -- 172.21.15.117:0/3494692061 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f323c057880 con 0x7f324c0ff930 2024-04-20T11:32:43.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.278+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f324c199780 con 0x7f323006ecc0 2024-04-20T11:32:43.279 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:42 smithi117 ceph-mon[99036]: pgmap v607: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:43.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.280+0000 7f3248c00700 1 -- 172.21.15.117:0/3494692061 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f324c199780 con 0x7f323006ecc0 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f323006ecc0 msgr2=0x7f3230071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f323006ecc0 0x7f3230071180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f32380059d0 tx=0x7f3238005960 comp rx=0 tx=0).stop 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ff930 msgr2=0x7f324c19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ff930 0x7f324c19ea60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c8a0 tx=0x7f323c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 shutdown_connections 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f323006ecc0 0x7f3230071180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324c0ff930 0x7f324c19ea60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 --2- 172.21.15.117:0/3494692061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324c1002e0 0x7f324c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.283+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 >> 172.21.15.117:0/3494692061 conn(0x7f324c076750 msgr2=0x7f324c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:43.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.284+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 shutdown_connections 2024-04-20T11:32:43.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.284+0000 7f3252a00700 1 -- 172.21.15.117:0/3494692061 wait complete. 2024-04-20T11:32:43.299 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:32:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:42 smithi173 ceph-mon[74165]: pgmap v607: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:43.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.442+0000 7f50f1000700 1 -- 172.21.15.117:0/1025828033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec1039b0 msgr2=0x7f50ec107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.442+0000 7f50f1000700 1 --2- 172.21.15.117:0/1025828033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec1039b0 0x7f50ec107a00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009910 tx=0x7f50d8009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:43.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.442+0000 7f50f1000700 1 -- 172.21.15.117:0/1025828033 shutdown_connections 2024-04-20T11:32:43.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.442+0000 7f50f1000700 1 --2- 172.21.15.117:0/1025828033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec1039b0 0x7f50ec107a00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.442+0000 7f50f1000700 1 --2- 172.21.15.117:0/1025828033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50ec103000 0x7f50ec1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.442+0000 7f50f1000700 1 -- 172.21.15.117:0/1025828033 >> 172.21.15.117:0/1025828033 conn(0x7f50ec0fe8b0 msgr2=0x7f50ec100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:43.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.442+0000 7f50f1000700 1 -- 172.21.15.117:0/1025828033 shutdown_connections 2024-04-20T11:32:43.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.442+0000 7f50f1000700 1 -- 172.21.15.117:0/1025828033 wait complete. 2024-04-20T11:32:43.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.443+0000 7f50f1000700 1 Processor -- start 2024-04-20T11:32:43.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.443+0000 7f50f1000700 1 -- start start 2024-04-20T11:32:43.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.443+0000 7f50f1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec103000 0x7f50ec198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50f1000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50ec1039b0 0x7f50ec1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50f1000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec199ab0 con 0x7f50ec103000 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50f1000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec19d780 con 0x7f50ec1039b0 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50e9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec103000 0x7f50ec198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50e9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec103000 0x7f50ec198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40376/0 (socket says 172.21.15.117:40376) 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50e9e00700 1 -- 172.21.15.117:0/268105210 learned_addr learned my addr 172.21.15.117:0/268105210 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50e9400700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50ec1039b0 0x7f50ec1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50e9e00700 1 -- 172.21.15.117:0/268105210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50ec1039b0 msgr2=0x7f50ec1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50e9e00700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50ec1039b0 0x7f50ec1993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50e9e00700 1 -- 172.21.15.117:0/268105210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80095f0 con 0x7f50ec103000 2024-04-20T11:32:43.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.444+0000 7f50e9e00700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec103000 0x7f50ec198e90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00b4b0 tx=0x7f50dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.445+0000 7f50e2a00700 1 -- 172.21.15.117:0/268105210 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50dc011800 con 0x7f50ec103000 2024-04-20T11:32:43.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.445+0000 7f50e2a00700 1 -- 172.21.15.117:0/268105210 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50dc011e20 con 0x7f50ec103000 2024-04-20T11:32:43.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.445+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec19da60 con 0x7f50ec103000 2024-04-20T11:32:43.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.445+0000 7f50e2a00700 1 -- 172.21.15.117:0/268105210 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50dc00f2e0 con 0x7f50ec103000 2024-04-20T11:32:43.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.445+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec19dfb0 con 0x7f50ec103000 2024-04-20T11:32:43.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.446+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec10b330 con 0x7f50ec103000 2024-04-20T11:32:43.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.447+0000 7f50e2a00700 1 -- 172.21.15.117:0/268105210 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f50dc010380 con 0x7f50ec103000 2024-04-20T11:32:43.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.448+0000 7f50e2a00700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f50d006ecc0 0x7f50d0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.448+0000 7f50e2a00700 1 -- 172.21.15.117:0/268105210 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f50dc08e960 con 0x7f50ec103000 2024-04-20T11:32:43.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.448+0000 7f50e9400700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f50d006ecc0 0x7f50d0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.449+0000 7f50e9400700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f50d006ecc0 0x7f50d0071180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f50d8006010 tx=0x7f50d800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.453+0000 7f50e2a00700 1 -- 172.21.15.117:0/268105210 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f50dc057870 con 0x7f50ec103000 2024-04-20T11:32:43.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.700+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50ec19a260 con 0x7f50d006ecc0 2024-04-20T11:32:43.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.703+0000 7f50e2a00700 1 -- 172.21.15.117:0/268105210 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f50ec19a260 con 0x7f50d006ecc0 2024-04-20T11:32:43.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.705+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f50d006ecc0 msgr2=0x7f50d0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.705+0000 7f50f1000700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f50d006ecc0 0x7f50d0071180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f50d8006010 tx=0x7f50d800b540 comp rx=0 tx=0).stop 2024-04-20T11:32:43.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.705+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec103000 msgr2=0x7f50ec198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.705+0000 7f50f1000700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec103000 0x7f50ec198e90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00b4b0 tx=0x7f50dc00b870 comp rx=0 tx=0).stop 2024-04-20T11:32:43.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.705+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 shutdown_connections 2024-04-20T11:32:43.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.706+0000 7f50f1000700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f50d006ecc0 0x7f50d0071180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.706+0000 7f50f1000700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ec103000 0x7f50ec198e90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.706+0000 7f50f1000700 1 --2- 172.21.15.117:0/268105210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50ec1039b0 0x7f50ec1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.706+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 >> 172.21.15.117:0/268105210 conn(0x7f50ec0fe8b0 msgr2=0x7f50ec0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:43.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.706+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 shutdown_connections 2024-04-20T11:32:43.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.706+0000 7f50f1000700 1 -- 172.21.15.117:0/268105210 wait complete. 2024-04-20T11:32:43.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.859+0000 7faeec400700 1 -- 172.21.15.117:0/282866634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4103110 msgr2=0x7faee41034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.859+0000 7faeec400700 1 --2- 172.21.15.117:0/282866634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4103110 0x7faee41034f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faed8009910 tx=0x7faed8009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:43.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.859+0000 7faeec400700 1 -- 172.21.15.117:0/282866634 shutdown_connections 2024-04-20T11:32:43.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.859+0000 7faeec400700 1 --2- 172.21.15.117:0/282866634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee4104a00 0x7faee4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.859+0000 7faeec400700 1 --2- 172.21.15.117:0/282866634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4103110 0x7faee41034f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.859+0000 7faeec400700 1 -- 172.21.15.117:0/282866634 >> 172.21.15.117:0/282866634 conn(0x7faee40fe880 msgr2=0x7faee4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:43.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.859+0000 7faeec400700 1 -- 172.21.15.117:0/282866634 shutdown_connections 2024-04-20T11:32:43.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.859+0000 7faeec400700 1 -- 172.21.15.117:0/282866634 wait complete. 2024-04-20T11:32:43.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.860+0000 7faeec400700 1 Processor -- start 2024-04-20T11:32:43.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.860+0000 7faeec400700 1 -- start start 2024-04-20T11:32:43.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.861+0000 7faeec400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee4103110 0x7faee4198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.861+0000 7faeec400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4104a00 0x7faee41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.861+0000 7faeec400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee4199a80 con 0x7faee4104a00 2024-04-20T11:32:43.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.861+0000 7faeec400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee419d750 con 0x7faee4103110 2024-04-20T11:32:43.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.861+0000 7faee9000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4104a00 0x7faee41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.861+0000 7faee9000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4104a00 0x7faee41993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42504/0 (socket says 172.21.15.117:42504) 2024-04-20T11:32:43.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.861+0000 7faee9000700 1 -- 172.21.15.117:0/2774161406 learned_addr learned my addr 172.21.15.117:0/2774161406 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:43.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faee9a00700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee4103110 0x7faee4198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faee9000700 1 -- 172.21.15.117:0/2774161406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee4103110 msgr2=0x7faee4198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:43.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faee9000700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee4103110 0x7faee4198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:43.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faee9000700 1 -- 172.21.15.117:0/2774161406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed80095f0 con 0x7faee4104a00 2024-04-20T11:32:43.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faee9000700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4104a00 0x7faee41993a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faed000b4b0 tx=0x7faed000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faee2000700 1 -- 172.21.15.117:0/2774161406 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed0011800 con 0x7faee4104a00 2024-04-20T11:32:43.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faee2000700 1 -- 172.21.15.117:0/2774161406 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faed0011e20 con 0x7faee4104a00 2024-04-20T11:32:43.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee419da30 con 0x7faee4104a00 2024-04-20T11:32:43.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.862+0000 7faee2000700 1 -- 172.21.15.117:0/2774161406 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed000f2e0 con 0x7faee4104a00 2024-04-20T11:32:43.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.863+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee419e000 con 0x7faee4104a00 2024-04-20T11:32:43.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.864+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee410b460 con 0x7faee4104a00 2024-04-20T11:32:43.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.865+0000 7faee2000700 1 -- 172.21.15.117:0/2774161406 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7faed0010380 con 0x7faee4104a00 2024-04-20T11:32:43.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.866+0000 7faee2000700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faecc06ecd0 0x7faecc071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:43.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.866+0000 7faee2000700 1 -- 172.21.15.117:0/2774161406 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7faed008e990 con 0x7faee4104a00 2024-04-20T11:32:43.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.866+0000 7faee9a00700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faecc06ecd0 0x7faecc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:43.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.867+0000 7faee9a00700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faecc06ecd0 0x7faecc071190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faed800b5c0 tx=0x7faed801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:43.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:43.871+0000 7faee2000700 1 -- 172.21.15.117:0/2774161406 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7faed0057750 con 0x7faee4104a00 2024-04-20T11:32:44.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.113+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faee404ec50 con 0x7faecc06ecd0 2024-04-20T11:32:44.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.126+0000 7faee2000700 1 -- 172.21.15.117:0/2774161406 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7faee404ec50 con 0x7faecc06ecd0 2024-04-20T11:32:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:32:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (21m) 12m ago 35m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:32:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (35m) 12m ago 35m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:32:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (34m) 12m ago 34m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:32:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (17m) 12m ago 35m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:32:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (17m) 12m ago 34m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:32:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (20m) 12m ago 34m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:32:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (12m) 12m ago 27m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (13m) 12m ago 28m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (27m) 12m ago 27m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (12m) 12m ago 28m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (24m) 12m ago 37m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (22m) 12m ago 33m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (19m) 12m ago 37m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (18m) 12m ago 33m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (22m) 12m ago 35m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (22m) 12m ago 34m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (16m) 12m ago 32m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (16m) 12m ago 32m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (15m) 12m ago 31m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (15m) 12m ago 31m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (14m) 12m ago 30m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:32:44.128 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (13m) 12m ago 29m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:32:44.129 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (21m) 12m ago 34m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:32:44.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faecc06ecd0 msgr2=0x7faecc071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faecc06ecd0 0x7faecc071190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faed800b5c0 tx=0x7faed801a040 comp rx=0 tx=0).stop 2024-04-20T11:32:44.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4104a00 msgr2=0x7faee41993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4104a00 0x7faee41993a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faed000b4b0 tx=0x7faed000b870 comp rx=0 tx=0).stop 2024-04-20T11:32:44.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 shutdown_connections 2024-04-20T11:32:44.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faecc06ecd0 0x7faecc071190 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee4103110 0x7faee4198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 --2- 172.21.15.117:0/2774161406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faee4104a00 0x7faee41993a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 >> 172.21.15.117:0/2774161406 conn(0x7faee40fe880 msgr2=0x7faee40ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:44.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 shutdown_connections 2024-04-20T11:32:44.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.129+0000 7faeec400700 1 -- 172.21.15.117:0/2774161406 wait complete. 2024-04-20T11:32:44.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.283+0000 7f2ab2800700 1 -- 172.21.15.117:0/1180006020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac107520 msgr2=0x7f2aac107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.283+0000 7f2ab2800700 1 --2- 172.21.15.117:0/1180006020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac107520 0x7f2aac107900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009910 tx=0x7f2a9c009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:44.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.283+0000 7f2ab2800700 1 -- 172.21.15.117:0/1180006020 shutdown_connections 2024-04-20T11:32:44.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.283+0000 7f2ab2800700 1 --2- 172.21.15.117:0/1180006020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aac108e10 0x7f2aac109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.283+0000 7f2ab2800700 1 --2- 172.21.15.117:0/1180006020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac107520 0x7f2aac107900 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.283+0000 7f2ab2800700 1 -- 172.21.15.117:0/1180006020 >> 172.21.15.117:0/1180006020 conn(0x7f2aac0768f0 msgr2=0x7f2aac078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:44.285 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:44.285 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:44 smithi117 ceph-mon[99036]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:44.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.284+0000 7f2ab2800700 1 -- 172.21.15.117:0/1180006020 shutdown_connections 2024-04-20T11:32:44.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.284+0000 7f2ab2800700 1 -- 172.21.15.117:0/1180006020 wait complete. 2024-04-20T11:32:44.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.285+0000 7f2ab2800700 1 Processor -- start 2024-04-20T11:32:44.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.285+0000 7f2ab2800700 1 -- start start 2024-04-20T11:32:44.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.286+0000 7f2ab2800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aac107520 0x7f2aac19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.286+0000 7f2ab2800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac108e10 0x7f2aac19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.286+0000 7f2ab2800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac19de90 con 0x7f2aac108e10 2024-04-20T11:32:44.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.286+0000 7f2ab2800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac1a1b60 con 0x7f2aac107520 2024-04-20T11:32:44.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.286+0000 7f2aa3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac108e10 0x7f2aac19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.286+0000 7f2aa3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac108e10 0x7f2aac19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42514/0 (socket says 172.21.15.117:42514) 2024-04-20T11:32:44.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.286+0000 7f2aa3e00700 1 -- 172.21.15.117:0/2530967329 learned_addr learned my addr 172.21.15.117:0/2530967329 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:44.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.287+0000 7f2aab400700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aac107520 0x7f2aac19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.287+0000 7f2aa3e00700 1 -- 172.21.15.117:0/2530967329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aac107520 msgr2=0x7f2aac19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.287+0000 7f2aa3e00700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aac107520 0x7f2aac19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.287+0000 7f2aa3e00700 1 -- 172.21.15.117:0/2530967329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c0095f0 con 0x7f2aac108e10 2024-04-20T11:32:44.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.287+0000 7f2aa3e00700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac108e10 0x7f2aac19d7b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400b4b0 tx=0x7f2a9400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:44.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.287+0000 7f2aa8c00700 1 -- 172.21.15.117:0/2530967329 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a94011800 con 0x7f2aac108e10 2024-04-20T11:32:44.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.288+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac1a1e40 con 0x7f2aac108e10 2024-04-20T11:32:44.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.288+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac1a2410 con 0x7f2aac108e10 2024-04-20T11:32:44.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.288+0000 7f2aa8c00700 1 -- 172.21.15.117:0/2530967329 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a94011e20 con 0x7f2aac108e10 2024-04-20T11:32:44.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.288+0000 7f2aa8c00700 1 -- 172.21.15.117:0/2530967329 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a9401a400 con 0x7f2aac108e10 2024-04-20T11:32:44.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.289+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a880052f0 con 0x7f2aac108e10 2024-04-20T11:32:44.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.290+0000 7f2aa8c00700 1 -- 172.21.15.117:0/2530967329 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2a94011960 con 0x7f2aac108e10 2024-04-20T11:32:44.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.291+0000 7f2aa8c00700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a9006ecd0 0x7f2a90071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.291+0000 7f2aa8c00700 1 -- 172.21.15.117:0/2530967329 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2a9408e830 con 0x7f2aac108e10 2024-04-20T11:32:44.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.291+0000 7f2aab400700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a9006ecd0 0x7f2a90071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.292+0000 7f2aab400700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a9006ecd0 0x7f2a90071190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00b5c0 tx=0x7f2a9c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:44.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.296+0000 7f2aa8c00700 1 -- 172.21.15.117:0/2530967329 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2a94057690 con 0x7f2aac108e10 2024-04-20T11:32:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:44 smithi173 ceph-mon[74165]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:44.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.611+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2a88005db0 con 0x7f2aac108e10 2024-04-20T11:32:44.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.612+0000 7f2aa8c00700 1 -- 172.21.15.117:0/2530967329 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f2a9405ad50 con 0x7f2aac108e10 2024-04-20T11:32:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:32:44.613 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:32:44.613 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:32:44.614 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:32:44.615 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:32:44.615 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:32:44.615 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:32:44.615 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:32:44.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.615+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a9006ecd0 msgr2=0x7f2a90071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.615+0000 7f2ab2800700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a9006ecd0 0x7f2a90071190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00b5c0 tx=0x7f2a9c01a040 comp rx=0 tx=0).stop 2024-04-20T11:32:44.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac108e10 msgr2=0x7f2aac19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac108e10 0x7f2aac19d7b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400b4b0 tx=0x7f2a9400b870 comp rx=0 tx=0).stop 2024-04-20T11:32:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 shutdown_connections 2024-04-20T11:32:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2a9006ecd0 0x7f2a90071190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aac107520 0x7f2aac19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 --2- 172.21.15.117:0/2530967329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac108e10 0x7f2aac19d7b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 >> 172.21.15.117:0/2530967329 conn(0x7f2aac0768f0 msgr2=0x7f2aac078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 shutdown_connections 2024-04-20T11:32:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.616+0000 7f2ab2800700 1 -- 172.21.15.117:0/2530967329 wait complete. 2024-04-20T11:32:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.771+0000 7efd7aa00700 1 -- 172.21.15.117:0/944268391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74102660 msgr2=0x7efd7410ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.771+0000 7efd7aa00700 1 --2- 172.21.15.117:0/944268391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74102660 0x7efd7410ab50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009910 tx=0x7efd5c009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.771+0000 7efd7aa00700 1 -- 172.21.15.117:0/944268391 shutdown_connections 2024-04-20T11:32:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.771+0000 7efd7aa00700 1 --2- 172.21.15.117:0/944268391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74102660 0x7efd7410ab50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.771+0000 7efd7aa00700 1 --2- 172.21.15.117:0/944268391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd74101d40 0x7efd74102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.771+0000 7efd7aa00700 1 -- 172.21.15.117:0/944268391 >> 172.21.15.117:0/944268391 conn(0x7efd74076750 msgr2=0x7efd740fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.771+0000 7efd7aa00700 1 -- 172.21.15.117:0/944268391 shutdown_connections 2024-04-20T11:32:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.771+0000 7efd7aa00700 1 -- 172.21.15.117:0/944268391 wait complete. 2024-04-20T11:32:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.772+0000 7efd7aa00700 1 Processor -- start 2024-04-20T11:32:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.773+0000 7efd7aa00700 1 -- start start 2024-04-20T11:32:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.773+0000 7efd7aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74101d40 0x7efd740ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.773+0000 7efd7aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd74102660 0x7efd740ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.773+0000 7efd7aa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd74100420 con 0x7efd74101d40 2024-04-20T11:32:44.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.773+0000 7efd7aa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd74100560 con 0x7efd74102660 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.773+0000 7efd73e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74101d40 0x7efd740ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.773+0000 7efd73e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74101d40 0x7efd740ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42536/0 (socket says 172.21.15.117:42536) 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.773+0000 7efd73e00700 1 -- 172.21.15.117:0/2141624079 learned_addr learned my addr 172.21.15.117:0/2141624079 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.774+0000 7efd6be00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd74102660 0x7efd740ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.774+0000 7efd73e00700 1 -- 172.21.15.117:0/2141624079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd74102660 msgr2=0x7efd740ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.774+0000 7efd73e00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd74102660 0x7efd740ffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.774+0000 7efd73e00700 1 -- 172.21.15.117:0/2141624079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c0095f0 con 0x7efd74101d40 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.774+0000 7efd73e00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74101d40 0x7efd740ff9a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b4b0 tx=0x7efd6400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.774+0000 7efd71600700 1 -- 172.21.15.117:0/2141624079 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd64011800 con 0x7efd74101d40 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.774+0000 7efd71600700 1 -- 172.21.15.117:0/2141624079 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd64011e20 con 0x7efd74101d40 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.774+0000 7efd71600700 1 -- 172.21.15.117:0/2141624079 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6400f2e0 con 0x7efd74101d40 2024-04-20T11:32:44.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.775+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd741a2e20 con 0x7efd74101d40 2024-04-20T11:32:44.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.775+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd741a3180 con 0x7efd74101d40 2024-04-20T11:32:44.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.776+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd74068e50 con 0x7efd74101d40 2024-04-20T11:32:44.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.777+0000 7efd71600700 1 -- 172.21.15.117:0/2141624079 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efd64010380 con 0x7efd74101d40 2024-04-20T11:32:44.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.778+0000 7efd71600700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efd5806ecc0 0x7efd58071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:44.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.778+0000 7efd71600700 1 -- 172.21.15.117:0/2141624079 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efd6408e9a0 con 0x7efd74101d40 2024-04-20T11:32:44.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.778+0000 7efd6be00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efd5806ecc0 0x7efd58071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:44.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.779+0000 7efd6be00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efd5806ecc0 0x7efd58071180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd5c006010 tx=0x7efd5c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:44.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:44.783+0000 7efd71600700 1 -- 172.21.15.117:0/2141624079 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efd64057760 con 0x7efd74101d40 2024-04-20T11:32:45.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.051+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efd74100720 con 0x7efd74101d40 2024-04-20T11:32:45.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.052+0000 7efd71600700 1 -- 172.21.15.117:0/2141624079 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7efd74100720 con 0x7efd74101d40 2024-04-20T11:32:45.053 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:32:45.053 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:32:45.053 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:32:45.053 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:32:45.053 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:32:45.053 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:32:45.053 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:32:45.053 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:32:45.054 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:32:45.055 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:32:45.056 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:32:45.056 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:32:45.056 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:32:45.056 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:32:45.056 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:32:45.056 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:32:45.056 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efd5806ecc0 msgr2=0x7efd58071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efd5806ecc0 0x7efd58071180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd5c006010 tx=0x7efd5c00b540 comp rx=0 tx=0).stop 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74101d40 msgr2=0x7efd740ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74101d40 0x7efd740ff9a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b4b0 tx=0x7efd6400b870 comp rx=0 tx=0).stop 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 shutdown_connections 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efd5806ecc0 0x7efd58071180 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd74101d40 0x7efd740ff9a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 --2- 172.21.15.117:0/2141624079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd74102660 0x7efd740ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 >> 172.21.15.117:0/2141624079 conn(0x7efd74076750 msgr2=0x7efd74103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 shutdown_connections 2024-04-20T11:32:45.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.055+0000 7efd7aa00700 1 -- 172.21.15.117:0/2141624079 wait complete. 2024-04-20T11:32:45.058 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:32:45.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.218+0000 7fadf5400700 1 -- 172.21.15.117:0/1336119554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0103110 msgr2=0x7fadf01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.218+0000 7fadf5400700 1 --2- 172.21.15.117:0/1336119554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0103110 0x7fadf01034f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fade0009910 tx=0x7fade0009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:45.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.218+0000 7fadf5400700 1 -- 172.21.15.117:0/1336119554 shutdown_connections 2024-04-20T11:32:45.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.218+0000 7fadf5400700 1 --2- 172.21.15.117:0/1336119554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf0104a00 0x7fadf0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.218+0000 7fadf5400700 1 --2- 172.21.15.117:0/1336119554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0103110 0x7fadf01034f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.218+0000 7fadf5400700 1 -- 172.21.15.117:0/1336119554 >> 172.21.15.117:0/1336119554 conn(0x7fadf00fe880 msgr2=0x7fadf0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.218+0000 7fadf5400700 1 -- 172.21.15.117:0/1336119554 shutdown_connections 2024-04-20T11:32:45.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.218+0000 7fadf5400700 1 -- 172.21.15.117:0/1336119554 wait complete. 2024-04-20T11:32:45.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.219+0000 7fadf5400700 1 Processor -- start 2024-04-20T11:32:45.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.220+0000 7fadf5400700 1 -- start start 2024-04-20T11:32:45.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.220+0000 7fadf5400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf0103110 0x7fadf0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.220+0000 7fadf5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0104a00 0x7fadf01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.220+0000 7fadf5400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf0199a80 con 0x7fadf0104a00 2024-04-20T11:32:45.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.220+0000 7fadf5400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf019d750 con 0x7fadf0103110 2024-04-20T11:32:45.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.221+0000 7fade7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0104a00 0x7fadf01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.221+0000 7fade7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0104a00 0x7fadf01993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42552/0 (socket says 172.21.15.117:42552) 2024-04-20T11:32:45.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.221+0000 7fade7e00700 1 -- 172.21.15.117:0/3821652351 learned_addr learned my addr 172.21.15.117:0/3821652351 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.221+0000 7fadee800700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf0103110 0x7fadf0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.221+0000 7fade7e00700 1 -- 172.21.15.117:0/3821652351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf0103110 msgr2=0x7fadf0198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.221+0000 7fade7e00700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf0103110 0x7fadf0198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.221+0000 7fade7e00700 1 -- 172.21.15.117:0/3821652351 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00095f0 con 0x7fadf0104a00 2024-04-20T11:32:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.221+0000 7fade7e00700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0104a00 0x7fadf01993a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fadd800b4b0 tx=0x7fadd800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.222+0000 7fade7400700 1 -- 172.21.15.117:0/3821652351 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadd8011800 con 0x7fadf0104a00 2024-04-20T11:32:45.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.222+0000 7fade7400700 1 -- 172.21.15.117:0/3821652351 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadd8011e20 con 0x7fadf0104a00 2024-04-20T11:32:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.222+0000 7fade7400700 1 -- 172.21.15.117:0/3821652351 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadd800f2e0 con 0x7fadf0104a00 2024-04-20T11:32:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.222+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf019da30 con 0x7fadf0104a00 2024-04-20T11:32:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.222+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf019e000 con 0x7fadf0104a00 2024-04-20T11:32:45.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.223+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf010b460 con 0x7fadf0104a00 2024-04-20T11:32:45.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.224+0000 7fade7400700 1 -- 172.21.15.117:0/3821652351 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fadd8010380 con 0x7fadf0104a00 2024-04-20T11:32:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.225+0000 7fade7400700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fadd406ecd0 0x7fadd4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.225+0000 7fade7400700 1 -- 172.21.15.117:0/3821652351 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fadd808ea70 con 0x7fadf0104a00 2024-04-20T11:32:45.226 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:45 smithi117 ceph-mon[99036]: pgmap v608: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:45.226 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:45 smithi117 ceph-mon[99036]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:45.226 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:45 smithi117 ceph-mon[99036]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:45.226 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:45 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2530967329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:32:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.225+0000 7fadee800700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fadd406ecd0 0x7fadd4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.226+0000 7fadee800700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fadd406ecd0 0x7fadd4071190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fade000b5c0 tx=0x7fade001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.232+0000 7fade7400700 1 -- 172.21.15.117:0/3821652351 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fadd80577b0 con 0x7fadf0104a00 2024-04-20T11:32:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:45 smithi173 ceph-mon[74165]: pgmap v608: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:45 smithi173 ceph-mon[74165]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:45 smithi173 ceph-mon[74165]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:45 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2530967329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:32:45.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.473+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadf004ec50 con 0x7fadd406ecd0 2024-04-20T11:32:45.476 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:32:45.476 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:32:45.476 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:32:45.476 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:32:45.476 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:32:45.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.475+0000 7fade7400700 1 -- 172.21.15.117:0/3821652351 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fadf004ec50 con 0x7fadd406ecd0 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fadd406ecd0 msgr2=0x7fadd4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fadd406ecd0 0x7fadd4071190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fade000b5c0 tx=0x7fade001a040 comp rx=0 tx=0).stop 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0104a00 msgr2=0x7fadf01993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0104a00 0x7fadf01993a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fadd800b4b0 tx=0x7fadd800b870 comp rx=0 tx=0).stop 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 shutdown_connections 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fadd406ecd0 0x7fadd4071190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf0103110 0x7fadf0198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 --2- 172.21.15.117:0/3821652351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf0104a00 0x7fadf01993a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 >> 172.21.15.117:0/3821652351 conn(0x7fadf00fe880 msgr2=0x7fadf00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.478+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 shutdown_connections 2024-04-20T11:32:45.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.479+0000 7fadf5400700 1 -- 172.21.15.117:0/3821652351 wait complete. 2024-04-20T11:32:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.628+0000 7fdf1b200700 1 -- 172.21.15.117:0/3495420942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 msgr2=0x7fdf1c10d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.628+0000 7fdf1b200700 1 --2- 172.21.15.117:0/3495420942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 0x7fdf1c10d4a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009910 tx=0x7fdf0c009c20 comp rx=0 tx=0).stop 2024-04-20T11:32:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.628+0000 7fdf1b200700 1 -- 172.21.15.117:0/3495420942 shutdown_connections 2024-04-20T11:32:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.628+0000 7fdf1b200700 1 --2- 172.21.15.117:0/3495420942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c1070b0 0x7fdf1c107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.628+0000 7fdf1b200700 1 --2- 172.21.15.117:0/3495420942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 0x7fdf1c10d4a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.628+0000 7fdf1b200700 1 -- 172.21.15.117:0/3495420942 >> 172.21.15.117:0/3495420942 conn(0x7fdf1c076830 msgr2=0x7fdf1c078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.628+0000 7fdf1b200700 1 -- 172.21.15.117:0/3495420942 shutdown_connections 2024-04-20T11:32:45.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.628+0000 7fdf1b200700 1 -- 172.21.15.117:0/3495420942 wait complete. 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.629+0000 7fdf1b200700 1 Processor -- start 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.629+0000 7fdf1b200700 1 -- start start 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.629+0000 7fdf1b200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c1070b0 0x7fdf1c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.629+0000 7fdf1b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 0x7fdf1c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.629+0000 7fdf1b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c19db50 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.629+0000 7fdf1b200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c1a1820 con 0x7fdf1c1070b0 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.630+0000 7fdf13e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 0x7fdf1c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.630+0000 7fdf13e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 0x7fdf1c19d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42558/0 (socket says 172.21.15.117:42558) 2024-04-20T11:32:45.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.630+0000 7fdf13e00700 1 -- 172.21.15.117:0/3861045247 learned_addr learned my addr 172.21.15.117:0/3861045247 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:32:45.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.630+0000 7fdf19e00700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c1070b0 0x7fdf1c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.630+0000 7fdf13e00700 1 -- 172.21.15.117:0/3861045247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c1070b0 msgr2=0x7fdf1c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.630+0000 7fdf13e00700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c1070b0 0x7fdf1c19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.630+0000 7fdf13e00700 1 -- 172.21.15.117:0/3861045247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0095f0 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.630+0000 7fdf13e00700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 0x7fdf1c19d470 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c8a0 tx=0x7fdf0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.632+0000 7fdf12a00700 1 -- 172.21.15.117:0/3861045247 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0400cea0 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.632+0000 7fdf12a00700 1 -- 172.21.15.117:0/3861045247 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf04004830 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.632+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c1a1b00 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.632+0000 7fdf12a00700 1 -- 172.21.15.117:0/3861045247 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf040055b0 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.632+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c1a2050 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.633+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c10f5c0 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.634+0000 7fdf12a00700 1 -- 172.21.15.117:0/3861045247 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdf040049a0 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.635+0000 7fdf12a00700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf0006ecd0 0x7fdf00071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:45.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.635+0000 7fdf12a00700 1 -- 172.21.15.117:0/3861045247 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdf0408ebc0 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.635+0000 7fdf19e00700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf0006ecd0 0x7fdf00071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:45.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.636+0000 7fdf19e00700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf0006ecd0 0x7fdf00071190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00b5c0 tx=0x7fdf0c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:45.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.639+0000 7fdf12a00700 1 -- 172.21.15.117:0/3861045247 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdf04057900 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.930+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdf1c19e270 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.930+0000 7fdf12a00700 1 -- 172.21.15.117:0/3861045247 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fdf0401d1c0 con 0x7fdf1c10d0c0 2024-04-20T11:32:45.930 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:32:45.930 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:32:45.930 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:32:45.930 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:32:45.931 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:32:45.931 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:32:45.931 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:32:45.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.933+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf0006ecd0 msgr2=0x7fdf00071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.933+0000 7fdf1b200700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf0006ecd0 0x7fdf00071190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00b5c0 tx=0x7fdf0c01a040 comp rx=0 tx=0).stop 2024-04-20T11:32:45.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 msgr2=0x7fdf1c19d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:45.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 0x7fdf1c19d470 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c8a0 tx=0x7fdf0400cc60 comp rx=0 tx=0).stop 2024-04-20T11:32:45.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 shutdown_connections 2024-04-20T11:32:45.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf0006ecd0 0x7fdf00071190 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c1070b0 0x7fdf1c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 --2- 172.21.15.117:0/3861045247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c10d0c0 0x7fdf1c19d470 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:45.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 >> 172.21.15.117:0/3861045247 conn(0x7fdf1c076830 msgr2=0x7fdf1c10c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:45.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 shutdown_connections 2024-04-20T11:32:45.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:32:45.934+0000 7fdf1b200700 1 -- 172.21.15.117:0/3861045247 wait complete. 2024-04-20T11:32:46.357 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:46 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2141624079' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:32:46.358 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:46 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3861045247' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:32:46.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:46 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2141624079' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:32:46.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:46 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3861045247' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:32:47.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:47 smithi117 ceph-mon[99036]: pgmap v609: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:47.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:47 smithi117 ceph-mon[99036]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:47 smithi173 ceph-mon[74165]: pgmap v609: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:47 smithi173 ceph-mon[74165]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:49.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:49 smithi117 ceph-mon[99036]: pgmap v610: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:49.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:49 smithi173 ceph-mon[74165]: pgmap v610: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:51.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:51 smithi117 ceph-mon[99036]: pgmap v611: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:51.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:51 smithi173 ceph-mon[74165]: pgmap v611: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:53.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:53 smithi117 ceph-mon[99036]: pgmap v612: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:53.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:53 smithi173 ceph-mon[74165]: pgmap v612: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:55.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:55 smithi117 ceph-mon[99036]: pgmap v613: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:55 smithi173 ceph-mon[74165]: pgmap v613: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:57.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:57 smithi117 ceph-mon[99036]: pgmap v614: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:57.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:57 smithi173 ceph-mon[74165]: pgmap v614: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:58.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:58.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:32:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:32:59 smithi117 ceph-mon[99036]: pgmap v615: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:32:59.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:32:59 smithi173 ceph-mon[74165]: pgmap v615: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:01.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:01 smithi117 ceph-mon[99036]: pgmap v616: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:01 smithi173 ceph-mon[74165]: pgmap v616: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:03.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:03 smithi117 ceph-mon[99036]: pgmap v617: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:03 smithi173 ceph-mon[74165]: pgmap v617: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:05.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:05 smithi117 ceph-mon[99036]: pgmap v618: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:05 smithi173 ceph-mon[74165]: pgmap v618: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:07.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:07 smithi117 ceph-mon[99036]: pgmap v619: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:07.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:07 smithi173 ceph-mon[74165]: pgmap v619: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:09.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:09 smithi117 ceph-mon[99036]: pgmap v620: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:09.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:09 smithi173 ceph-mon[74165]: pgmap v620: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:11.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:11 smithi117 ceph-mon[99036]: pgmap v621: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:11 smithi173 ceph-mon[74165]: pgmap v621: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:13 smithi117 ceph-mon[99036]: pgmap v622: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:33:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:13 smithi173 ceph-mon[74165]: pgmap v622: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:33:15.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:15 smithi117 ceph-mon[99036]: pgmap v623: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:15.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:15 smithi173 ceph-mon[74165]: pgmap v623: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:16.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.113+0000 7f35c2c00700 1 -- 172.21.15.117:0/3909968498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 msgr2=0x7f35bc10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.113+0000 7f35c2c00700 1 --2- 172.21.15.117:0/3909968498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 0x7f35bc10cf00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009910 tx=0x7f35a8009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:16.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.113+0000 7f35c2c00700 1 -- 172.21.15.117:0/3909968498 shutdown_connections 2024-04-20T11:33:16.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.113+0000 7f35c2c00700 1 --2- 172.21.15.117:0/3909968498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 0x7f35bc10cf00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.113+0000 7f35c2c00700 1 --2- 172.21.15.117:0/3909968498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc0732b0 0x7f35bc073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.113+0000 7f35c2c00700 1 -- 172.21.15.117:0/3909968498 >> 172.21.15.117:0/3909968498 conn(0x7f35bc0fc6b0 msgr2=0x7f35bc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:16.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.113+0000 7f35c2c00700 1 -- 172.21.15.117:0/3909968498 shutdown_connections 2024-04-20T11:33:16.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.113+0000 7f35c2c00700 1 -- 172.21.15.117:0/3909968498 wait complete. 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.114+0000 7f35c2c00700 1 Processor -- start 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.114+0000 7f35c2c00700 1 -- start start 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.115+0000 7f35c2c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc0732b0 0x7f35bc19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.115+0000 7f35c2c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 0x7f35bc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.115+0000 7f35c2c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc19f540 con 0x7f35bc073bd0 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.115+0000 7f35c2c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc198b20 con 0x7f35bc0732b0 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.115+0000 7f35bb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 0x7f35bc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.115+0000 7f35bb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 0x7f35bc19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37398/0 (socket says 172.21.15.117:37398) 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.115+0000 7f35bb400700 1 -- 172.21.15.117:0/268962929 learned_addr learned my addr 172.21.15.117:0/268962929 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:16.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.115+0000 7f35bbe00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc0732b0 0x7f35bc19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.116+0000 7f35bb400700 1 -- 172.21.15.117:0/268962929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc0732b0 msgr2=0x7f35bc19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.116+0000 7f35bb400700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc0732b0 0x7f35bc19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.116+0000 7f35bb400700 1 -- 172.21.15.117:0/268962929 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80095f0 con 0x7f35bc073bd0 2024-04-20T11:33:16.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.116+0000 7f35bb400700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 0x7f35bc19ef20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f35a80092e0 tx=0x7f35a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.116+0000 7f35b8c00700 1 -- 172.21.15.117:0/268962929 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a801d070 con 0x7f35bc073bd0 2024-04-20T11:33:16.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.116+0000 7f35b8c00700 1 -- 172.21.15.117:0/268962929 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35a80056e0 con 0x7f35bc073bd0 2024-04-20T11:33:16.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.116+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc198da0 con 0x7f35bc073bd0 2024-04-20T11:33:16.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.116+0000 7f35b8c00700 1 -- 172.21.15.117:0/268962929 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a800fe80 con 0x7f35bc073bd0 2024-04-20T11:33:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.117+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc199290 con 0x7f35bc073bd0 2024-04-20T11:33:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.118+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc107810 con 0x7f35bc073bd0 2024-04-20T11:33:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.119+0000 7f35b8c00700 1 -- 172.21.15.117:0/268962929 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f35a800f6f0 con 0x7f35bc073bd0 2024-04-20T11:33:16.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.120+0000 7f35b8c00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f35a006ecd0 0x7f35a0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.120+0000 7f35b8c00700 1 -- 172.21.15.117:0/268962929 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f35a8090300 con 0x7f35bc073bd0 2024-04-20T11:33:16.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.120+0000 7f35bbe00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f35a006ecd0 0x7f35a0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.121+0000 7f35bbe00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f35a006ecd0 0x7f35a0071190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005950 tx=0x7f35ac0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.125+0000 7f35b8c00700 1 -- 172.21.15.117:0/268962929 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f35a805cb70 con 0x7f35bc073bd0 2024-04-20T11:33:16.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.374+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35bc10ffc0 con 0x7f35a006ecd0 2024-04-20T11:33:16.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.377+0000 7f35b8c00700 1 -- 172.21.15.117:0/268962929 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f35bc10ffc0 con 0x7f35a006ecd0 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f35a006ecd0 msgr2=0x7f35a0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f35a006ecd0 0x7f35a0071190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005950 tx=0x7f35ac0058e0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 msgr2=0x7f35bc19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 0x7f35bc19ef20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f35a80092e0 tx=0x7f35a800be20 comp rx=0 tx=0).stop 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 shutdown_connections 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f35a006ecd0 0x7f35a0071190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc0732b0 0x7f35bc19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 --2- 172.21.15.117:0/268962929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35bc073bd0 0x7f35bc19ef20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.380+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 >> 172.21.15.117:0/268962929 conn(0x7f35bc0fc6b0 msgr2=0x7f35bc105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:16.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.381+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 shutdown_connections 2024-04-20T11:33:16.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.381+0000 7f35c2c00700 1 -- 172.21.15.117:0/268962929 wait complete. 2024-04-20T11:33:16.397 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:33:16.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.530+0000 7f0cca600700 1 -- 172.21.15.117:0/3649162039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc41069d0 msgr2=0x7f0cc4106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.530+0000 7f0cca600700 1 --2- 172.21.15.117:0/3649162039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc41069d0 0x7f0cc4106db0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009910 tx=0x7f0cb8009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:16.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.530+0000 7f0cca600700 1 -- 172.21.15.117:0/3649162039 shutdown_connections 2024-04-20T11:33:16.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.530+0000 7f0cca600700 1 --2- 172.21.15.117:0/3649162039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cc4068570 0x7f0cc40689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.530+0000 7f0cca600700 1 --2- 172.21.15.117:0/3649162039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc41069d0 0x7f0cc4106db0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.530+0000 7f0cca600700 1 -- 172.21.15.117:0/3649162039 >> 172.21.15.117:0/3649162039 conn(0x7f0cc4075480 msgr2=0x7f0cc4075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:16.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.531+0000 7f0cca600700 1 -- 172.21.15.117:0/3649162039 shutdown_connections 2024-04-20T11:33:16.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.531+0000 7f0cca600700 1 -- 172.21.15.117:0/3649162039 wait complete. 2024-04-20T11:33:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cca600700 1 Processor -- start 2024-04-20T11:33:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cca600700 1 -- start start 2024-04-20T11:33:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cca600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc4068570 0x7f0cc40ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cca600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cc41069d0 0x7f0cc40fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cca600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4100500 con 0x7f0cc4068570 2024-04-20T11:33:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cca600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4100640 con 0x7f0cc41069d0 2024-04-20T11:33:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cc9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc4068570 0x7f0cc40ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cc9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc4068570 0x7f0cc40ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37418/0 (socket says 172.21.15.117:37418) 2024-04-20T11:33:16.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.532+0000 7f0cc9200700 1 -- 172.21.15.117:0/1426844378 learned_addr learned my addr 172.21.15.117:0/1426844378 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:16.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cc9200700 1 -- 172.21.15.117:0/1426844378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cc41069d0 msgr2=0x7f0cc40fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cc9200700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cc41069d0 0x7f0cc40fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cc9200700 1 -- 172.21.15.117:0/1426844378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb80095f0 con 0x7f0cc4068570 2024-04-20T11:33:16.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cc9200700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc4068570 0x7f0cc40ffa80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8006010 tx=0x7f0cb800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cc1600700 1 -- 172.21.15.117:0/1426844378 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb801d070 con 0x7f0cc4068570 2024-04-20T11:33:16.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cc1600700 1 -- 172.21.15.117:0/1426844378 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cb80056e0 con 0x7f0cc4068570 2024-04-20T11:33:16.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cc1600700 1 -- 172.21.15.117:0/1426844378 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb8017400 con 0x7f0cc4068570 2024-04-20T11:33:16.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc41008c0 con 0x7f0cc4068570 2024-04-20T11:33:16.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.533+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc41a2d50 con 0x7f0cc4068570 2024-04-20T11:33:16.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.535+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc4108f80 con 0x7f0cc4068570 2024-04-20T11:33:16.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.536+0000 7f0cc1600700 1 -- 172.21.15.117:0/1426844378 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0cb800f860 con 0x7f0cc4068570 2024-04-20T11:33:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.537+0000 7f0cc1600700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0cac06ecc0 0x7f0cac071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.537+0000 7f0cc1600700 1 -- 172.21.15.117:0/1426844378 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0cb8090270 con 0x7f0cc4068570 2024-04-20T11:33:16.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.537+0000 7f0cc3e00700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0cac06ecc0 0x7f0cac071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.538+0000 7f0cc3e00700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0cac06ecc0 0x7f0cac071180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4005950 tx=0x7f0cb40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.542+0000 7f0cc1600700 1 -- 172.21.15.117:0/1426844378 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0cb8059070 con 0x7f0cc4068570 2024-04-20T11:33:16.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.781+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cc410dbb0 con 0x7f0cac06ecc0 2024-04-20T11:33:16.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.783+0000 7f0cc1600700 1 -- 172.21.15.117:0/1426844378 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0cc410dbb0 con 0x7f0cac06ecc0 2024-04-20T11:33:16.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0cac06ecc0 msgr2=0x7f0cac071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0cac06ecc0 0x7f0cac071180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4005950 tx=0x7f0cb40058e0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc4068570 msgr2=0x7f0cc40ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc4068570 0x7f0cc40ffa80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8006010 tx=0x7f0cb800be20 comp rx=0 tx=0).stop 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 shutdown_connections 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0cac06ecc0 0x7f0cac071180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cc4068570 0x7f0cc40ffa80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 --2- 172.21.15.117:0/1426844378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0cc41069d0 0x7f0cc40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 >> 172.21.15.117:0/1426844378 conn(0x7f0cc4075480 msgr2=0x7f0cc40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 shutdown_connections 2024-04-20T11:33:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.786+0000 7f0cca600700 1 -- 172.21.15.117:0/1426844378 wait complete. 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.937+0000 7fa744400700 1 -- 172.21.15.117:0/3895215769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c1017c0 msgr2=0x7fa73c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.937+0000 7fa744400700 1 --2- 172.21.15.117:0/3895215769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c1017c0 0x7fa73c105810 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa728009910 tx=0x7fa728009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.937+0000 7fa744400700 1 -- 172.21.15.117:0/3895215769 shutdown_connections 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.937+0000 7fa744400700 1 --2- 172.21.15.117:0/3895215769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c1017c0 0x7fa73c105810 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.937+0000 7fa744400700 1 --2- 172.21.15.117:0/3895215769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa73c100e10 0x7fa73c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.937+0000 7fa744400700 1 -- 172.21.15.117:0/3895215769 >> 172.21.15.117:0/3895215769 conn(0x7fa73c0fc6a0 msgr2=0x7fa73c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.937+0000 7fa744400700 1 -- 172.21.15.117:0/3895215769 shutdown_connections 2024-04-20T11:33:16.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.937+0000 7fa744400700 1 -- 172.21.15.117:0/3895215769 wait complete. 2024-04-20T11:33:16.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.938+0000 7fa744400700 1 Processor -- start 2024-04-20T11:33:16.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.938+0000 7fa744400700 1 -- start start 2024-04-20T11:33:16.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa744400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c100e10 0x7fa73c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa744400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa73c1017c0 0x7fa73c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa744400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c19f550 con 0x7fa73c100e10 2024-04-20T11:33:16.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa744400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c198b80 con 0x7fa73c1017c0 2024-04-20T11:33:16.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa741a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c100e10 0x7fa73c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa741a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c100e10 0x7fa73c19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37426/0 (socket says 172.21.15.117:37426) 2024-04-20T11:33:16.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa741a00700 1 -- 172.21.15.117:0/3909148858 learned_addr learned my addr 172.21.15.117:0/3909148858 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:16.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa741000700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa73c1017c0 0x7fa73c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa741a00700 1 -- 172.21.15.117:0/3909148858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa73c1017c0 msgr2=0x7fa73c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa741a00700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa73c1017c0 0x7fa73c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.939+0000 7fa741a00700 1 -- 172.21.15.117:0/3909148858 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280095f0 con 0x7fa73c100e10 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.940+0000 7fa741a00700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c100e10 0x7fa73c19e9f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa7300060b0 tx=0x7fa73000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.940+0000 7fa73a000700 1 -- 172.21.15.117:0/3909148858 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa73000dda0 con 0x7fa73c100e10 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.940+0000 7fa73a000700 1 -- 172.21.15.117:0/3909148858 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa73000f040 con 0x7fa73c100e10 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.940+0000 7fa73a000700 1 -- 172.21.15.117:0/3909148858 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa730014790 con 0x7fa73c100e10 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.940+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c198e60 con 0x7fa73c100e10 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.940+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c1993b0 con 0x7fa73c100e10 2024-04-20T11:33:16.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.941+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c104f90 con 0x7fa73c100e10 2024-04-20T11:33:16.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.942+0000 7fa73a000700 1 -- 172.21.15.117:0/3909148858 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa730014980 con 0x7fa73c100e10 2024-04-20T11:33:16.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.943+0000 7fa73a000700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa72406ecd0 0x7fa724071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:16.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.943+0000 7fa73a000700 1 -- 172.21.15.117:0/3909148858 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa73008e970 con 0x7fa73c100e10 2024-04-20T11:33:16.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.943+0000 7fa741000700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa72406ecd0 0x7fa724071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:16.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.944+0000 7fa741000700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa72406ecd0 0x7fa724071190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa728006010 tx=0x7fa72800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:16.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:16.948+0000 7fa73a000700 1 -- 172.21.15.117:0/3909148858 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa73005b290 con 0x7fa73c100e10 2024-04-20T11:33:17.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.168+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa73c19a060 con 0x7fa72406ecd0 2024-04-20T11:33:17.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.199+0000 7fa73a000700 1 -- 172.21.15.117:0/3909148858 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa73c19a060 con 0x7fa72406ecd0 2024-04-20T11:33:17.200 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:33:17.200 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (21m) 13m ago 35m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:33:17.200 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (36m) 13m ago 36m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (34m) 13m ago 34m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (18m) 13m ago 36m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (17m) 13m ago 34m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (21m) 13m ago 35m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (13m) 13m ago 28m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (13m) 13m ago 28m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (28m) 13m ago 28m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (13m) 13m ago 28m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (24m) 13m ago 37m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (23m) 13m ago 34m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (19m) 13m ago 37m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (19m) 13m ago 34m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (23m) 13m ago 35m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:33:17.201 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (22m) 13m ago 34m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:33:17.202 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (17m) 13m ago 33m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:33:17.202 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (16m) 13m ago 32m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:33:17.202 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (16m) 13m ago 32m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:33:17.202 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (15m) 13m ago 31m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:33:17.202 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (15m) 13m ago 31m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:33:17.202 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (14m) 13m ago 30m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:33:17.202 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (22m) 13m ago 35m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:33:17.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa72406ecd0 msgr2=0x7fa724071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa72406ecd0 0x7fa724071190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa728006010 tx=0x7fa72800b540 comp rx=0 tx=0).stop 2024-04-20T11:33:17.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c100e10 msgr2=0x7fa73c19e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c100e10 0x7fa73c19e9f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa7300060b0 tx=0x7fa73000d3b0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 shutdown_connections 2024-04-20T11:33:17.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa72406ecd0 0x7fa724071190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c100e10 0x7fa73c19e9f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 --2- 172.21.15.117:0/3909148858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa73c1017c0 0x7fa73c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.202+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 >> 172.21.15.117:0/3909148858 conn(0x7fa73c0fc6a0 msgr2=0x7fa73c0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.203+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 shutdown_connections 2024-04-20T11:33:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.203+0000 7fa744400700 1 -- 172.21.15.117:0/3909148858 wait complete. 2024-04-20T11:33:17.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.352+0000 7f8c1aa00700 1 -- 172.21.15.117:0/3972109288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14108e10 msgr2=0x7f8c14109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.352+0000 7f8c1aa00700 1 --2- 172.21.15.117:0/3972109288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14108e10 0x7f8c14109290 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009910 tx=0x7f8c00009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:17.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.352+0000 7f8c1aa00700 1 -- 172.21.15.117:0/3972109288 shutdown_connections 2024-04-20T11:33:17.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.352+0000 7f8c1aa00700 1 --2- 172.21.15.117:0/3972109288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14108e10 0x7f8c14109290 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.352+0000 7f8c1aa00700 1 --2- 172.21.15.117:0/3972109288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c14107520 0x7f8c14107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.352+0000 7f8c1aa00700 1 -- 172.21.15.117:0/3972109288 >> 172.21.15.117:0/3972109288 conn(0x7f8c140768f0 msgr2=0x7f8c14078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:17.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.352+0000 7f8c1aa00700 1 -- 172.21.15.117:0/3972109288 shutdown_connections 2024-04-20T11:33:17.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.352+0000 7f8c1aa00700 1 -- 172.21.15.117:0/3972109288 wait complete. 2024-04-20T11:33:17.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.353+0000 7f8c1aa00700 1 Processor -- start 2024-04-20T11:33:17.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.353+0000 7f8c1aa00700 1 -- start start 2024-04-20T11:33:17.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.354+0000 7f8c1aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14107520 0x7f8c1419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.354+0000 7f8c1aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c14108e10 0x7f8c1419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.354+0000 7f8c1aa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1419ddf0 con 0x7f8c14107520 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.354+0000 7f8c1aa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141a1a70 con 0x7f8c14108e10 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.354+0000 7f8c13e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14107520 0x7f8c1419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.354+0000 7f8c13e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14107520 0x7f8c1419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37440/0 (socket says 172.21.15.117:37440) 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.354+0000 7f8c13e00700 1 -- 172.21.15.117:0/565730687 learned_addr learned my addr 172.21.15.117:0/565730687 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.354+0000 7f8c13400700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c14108e10 0x7f8c1419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c13e00700 1 -- 172.21.15.117:0/565730687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c14108e10 msgr2=0x7f8c1419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c13e00700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c14108e10 0x7f8c1419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c13e00700 1 -- 172.21.15.117:0/565730687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000095f0 con 0x7f8c14107520 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c13e00700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14107520 0x7f8c1419d1d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400d8d0 tx=0x7f8c0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c10c00700 1 -- 172.21.15.117:0/565730687 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c04009940 con 0x7f8c14107520 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c10c00700 1 -- 172.21.15.117:0/565730687 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c04010460 con 0x7f8c14107520 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c141a1d50 con 0x7f8c14107520 2024-04-20T11:33:17.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c10c00700 1 -- 172.21.15.117:0/565730687 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c0400f5f0 con 0x7f8c14107520 2024-04-20T11:33:17.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.355+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141a2320 con 0x7f8c14107520 2024-04-20T11:33:17.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.356+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1410f870 con 0x7f8c14107520 2024-04-20T11:33:17.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.357+0000 7f8c10c00700 1 -- 172.21.15.117:0/565730687 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8c040105d0 con 0x7f8c14107520 2024-04-20T11:33:17.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.358+0000 7f8c10c00700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8bf806ecd0 0x7f8bf8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.358+0000 7f8c10c00700 1 -- 172.21.15.117:0/565730687 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8c0408ede0 con 0x7f8c14107520 2024-04-20T11:33:17.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.359+0000 7f8c13400700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8bf806ecd0 0x7f8bf8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.359+0000 7f8c13400700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8bf806ecd0 0x7f8bf8071190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005b40 tx=0x7f8c0001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.363+0000 7f8c10c00700 1 -- 172.21.15.117:0/565730687 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8c04057ba0 con 0x7f8c14107520 2024-04-20T11:33:17.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:17 smithi117 ceph-mon[99036]: pgmap v624: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:17.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:17 smithi173 ceph-mon[74165]: pgmap v624: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.659+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8c1419e490 con 0x7f8c14107520 2024-04-20T11:33:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.660+0000 7f8c10c00700 1 -- 172.21.15.117:0/565730687 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f8c0405b260 con 0x7f8c14107520 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:33:17.661 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:33:17.662 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:33:17.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:33:17.662 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:33:17.662 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:33:17.662 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:33:17.662 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:33:17.662 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:33:17.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8bf806ecd0 msgr2=0x7f8bf8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8bf806ecd0 0x7f8bf8071190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005b40 tx=0x7f8c0001a040 comp rx=0 tx=0).stop 2024-04-20T11:33:17.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14107520 msgr2=0x7f8c1419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14107520 0x7f8c1419d1d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400d8d0 tx=0x7f8c0400dc90 comp rx=0 tx=0).stop 2024-04-20T11:33:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 shutdown_connections 2024-04-20T11:33:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8bf806ecd0 0x7f8bf8071190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c14107520 0x7f8c1419d1d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 --2- 172.21.15.117:0/565730687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c14108e10 0x7f8c1419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 >> 172.21.15.117:0/565730687 conn(0x7f8c140768f0 msgr2=0x7f8c14077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.663+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 shutdown_connections 2024-04-20T11:33:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.664+0000 7f8c1aa00700 1 -- 172.21.15.117:0/565730687 wait complete. 2024-04-20T11:33:17.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.819+0000 7f5c6b000700 1 -- 172.21.15.117:0/519931598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 msgr2=0x7f5c6410cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.819+0000 7f5c6b000700 1 --2- 172.21.15.117:0/519931598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 0x7f5c6410cd80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009910 tx=0x7f5c50009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:17.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.819+0000 7f5c6b000700 1 -- 172.21.15.117:0/519931598 shutdown_connections 2024-04-20T11:33:17.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.819+0000 7f5c6b000700 1 --2- 172.21.15.117:0/519931598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 0x7f5c6410cd80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.819+0000 7f5c6b000700 1 --2- 172.21.15.117:0/519931598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c640732b0 0x7f5c64073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.819+0000 7f5c6b000700 1 -- 172.21.15.117:0/519931598 >> 172.21.15.117:0/519931598 conn(0x7f5c640fc530 msgr2=0x7f5c640fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:17.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.819+0000 7f5c6b000700 1 -- 172.21.15.117:0/519931598 shutdown_connections 2024-04-20T11:33:17.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.819+0000 7f5c6b000700 1 -- 172.21.15.117:0/519931598 wait complete. 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.820+0000 7f5c6b000700 1 Processor -- start 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.820+0000 7f5c6b000700 1 -- start start 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.821+0000 7f5c6b000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c640732b0 0x7f5c6419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.821+0000 7f5c6b000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 0x7f5c6419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.821+0000 7f5c6b000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6419f4c0 con 0x7f5c64073bd0 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.821+0000 7f5c6b000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c64198af0 con 0x7f5c640732b0 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.821+0000 7f5c69200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 0x7f5c6419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.821+0000 7f5c69200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 0x7f5c6419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37448/0 (socket says 172.21.15.117:37448) 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.821+0000 7f5c69200700 1 -- 172.21.15.117:0/3872555783 learned_addr learned my addr 172.21.15.117:0/3872555783 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:17.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c69c00700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c640732b0 0x7f5c6419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c69200700 1 -- 172.21.15.117:0/3872555783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c640732b0 msgr2=0x7f5c6419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:17.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c69200700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c640732b0 0x7f5c6419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:17.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c69200700 1 -- 172.21.15.117:0/3872555783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c500095f0 con 0x7f5c64073bd0 2024-04-20T11:33:17.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c69200700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 0x7f5c6419eea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5c500092e0 tx=0x7f5c5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c62000700 1 -- 172.21.15.117:0/3872555783 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c5001d070 con 0x7f5c64073bd0 2024-04-20T11:33:17.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c62000700 1 -- 172.21.15.117:0/3872555783 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c5000f460 con 0x7f5c64073bd0 2024-04-20T11:33:17.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c64198d70 con 0x7f5c64073bd0 2024-04-20T11:33:17.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.822+0000 7f5c62000700 1 -- 172.21.15.117:0/3872555783 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c50017630 con 0x7f5c64073bd0 2024-04-20T11:33:17.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.823+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c64199260 con 0x7f5c64073bd0 2024-04-20T11:33:17.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.824+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c64107760 con 0x7f5c64073bd0 2024-04-20T11:33:17.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.825+0000 7f5c62000700 1 -- 172.21.15.117:0/3872555783 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5c5000f5d0 con 0x7f5c64073bd0 2024-04-20T11:33:17.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.826+0000 7f5c62000700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c4c06ecd0 0x7f5c4c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:17.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.826+0000 7f5c62000700 1 -- 172.21.15.117:0/3872555783 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5c50090340 con 0x7f5c64073bd0 2024-04-20T11:33:17.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.826+0000 7f5c69c00700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c4c06ecd0 0x7f5c4c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:17.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.827+0000 7f5c69c00700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c4c06ecd0 0x7f5c4c071190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5c58006fd0 tx=0x7f5c58008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:17.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:17.831+0000 7f5c62000700 1 -- 172.21.15.117:0/3872555783 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5c5005cc30 con 0x7f5c64073bd0 2024-04-20T11:33:18.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.100+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5c64074960 con 0x7f5c64073bd0 2024-04-20T11:33:18.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.102+0000 7f5c62000700 1 -- 172.21.15.117:0/3872555783 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f5c5005c7c0 con 0x7f5c64073bd0 2024-04-20T11:33:18.103 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:33:18.103 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:33:18.103 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:33:18.103 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:33:18.103 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:33:18.104 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:33:18.105 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:33:18.106 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:33:18.106 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:33:18.106 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:33:18.106 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:33:18.106 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:33:18.106 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:33:18.106 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:33:18.106 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c4c06ecd0 msgr2=0x7f5c4c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c4c06ecd0 0x7f5c4c071190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5c58006fd0 tx=0x7f5c58008040 comp rx=0 tx=0).stop 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 msgr2=0x7f5c6419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 0x7f5c6419eea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5c500092e0 tx=0x7f5c5000be20 comp rx=0 tx=0).stop 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 shutdown_connections 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c4c06ecd0 0x7f5c4c071190 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c640732b0 0x7f5c6419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 --2- 172.21.15.117:0/3872555783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c64073bd0 0x7f5c6419eea0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 >> 172.21.15.117:0/3872555783 conn(0x7f5c640fc530 msgr2=0x7f5c64105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:18.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.105+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 shutdown_connections 2024-04-20T11:33:18.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.106+0000 7f5c6b000700 1 -- 172.21.15.117:0/3872555783 wait complete. 2024-04-20T11:33:18.108 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:33:18.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.263+0000 7f291da00700 1 -- 172.21.15.117:0/1481537092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 msgr2=0x7f2918107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.263+0000 7f291da00700 1 --2- 172.21.15.117:0/1481537092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 0x7f2918107510 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2904009910 tx=0x7f2904009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:18.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.263+0000 7f291da00700 1 -- 172.21.15.117:0/1481537092 shutdown_connections 2024-04-20T11:33:18.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.263+0000 7f291da00700 1 --2- 172.21.15.117:0/1481537092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 0x7f2918107510 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.263+0000 7f291da00700 1 --2- 172.21.15.117:0/1481537092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f291810d0c0 0x7f291810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.263+0000 7f291da00700 1 -- 172.21.15.117:0/1481537092 >> 172.21.15.117:0/1481537092 conn(0x7f2918076830 msgr2=0x7f2918078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:18.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.263+0000 7f291da00700 1 -- 172.21.15.117:0/1481537092 shutdown_connections 2024-04-20T11:33:18.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.263+0000 7f291da00700 1 -- 172.21.15.117:0/1481537092 wait complete. 2024-04-20T11:33:18.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.264+0000 7f291da00700 1 Processor -- start 2024-04-20T11:33:18.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.264+0000 7f291da00700 1 -- start start 2024-04-20T11:33:18.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.264+0000 7f291da00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 0x7f291819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.264+0000 7f291da00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f291810d0c0 0x7f291819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.264+0000 7f291da00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291819db80 con 0x7f29181070b0 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.264+0000 7f291da00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29181a1850 con 0x7f291810d0c0 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2917e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 0x7f291819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2917e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 0x7f291819cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37458/0 (socket says 172.21.15.117:37458) 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2917e00700 1 -- 172.21.15.117:0/3173415235 learned_addr learned my addr 172.21.15.117:0/3173415235 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2917400700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f291810d0c0 0x7f291819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2917e00700 1 -- 172.21.15.117:0/3173415235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f291810d0c0 msgr2=0x7f291819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2917e00700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f291810d0c0 0x7f291819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2917e00700 1 -- 172.21.15.117:0/3173415235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040095f0 con 0x7f29181070b0 2024-04-20T11:33:18.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2917e00700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 0x7f291819cf60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f290800c8a0 tx=0x7f290800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.265+0000 7f2914c00700 1 -- 172.21.15.117:0/3173415235 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f290800cea0 con 0x7f29181070b0 2024-04-20T11:33:18.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.266+0000 7f2914c00700 1 -- 172.21.15.117:0/3173415235 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2908004830 con 0x7f29181070b0 2024-04-20T11:33:18.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.266+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29181a1b30 con 0x7f29181070b0 2024-04-20T11:33:18.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.266+0000 7f2914c00700 1 -- 172.21.15.117:0/3173415235 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29080055b0 con 0x7f29181070b0 2024-04-20T11:33:18.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.266+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29181a2080 con 0x7f29181070b0 2024-04-20T11:33:18.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.267+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291810f5c0 con 0x7f29181070b0 2024-04-20T11:33:18.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.268+0000 7f2914c00700 1 -- 172.21.15.117:0/3173415235 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f29080049a0 con 0x7f29181070b0 2024-04-20T11:33:18.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.269+0000 7f2914c00700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28fc06ecd0 0x7f28fc071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.269+0000 7f2914c00700 1 -- 172.21.15.117:0/3173415235 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f290808ea20 con 0x7f29181070b0 2024-04-20T11:33:18.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.269+0000 7f2917400700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28fc06ecd0 0x7f28fc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.270+0000 7f2917400700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28fc06ecd0 0x7f28fc071190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2904006010 tx=0x7f290400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.273+0000 7f2914c00700 1 -- 172.21.15.117:0/3173415235 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2908057880 con 0x7f29181070b0 2024-04-20T11:33:18.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:18 smithi117 ceph-mon[99036]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:18 smithi117 ceph-mon[99036]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:18 smithi117 ceph-mon[99036]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:18 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/565730687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:33:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:18 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3872555783' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:33:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:18 smithi173 ceph-mon[74165]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:18 smithi173 ceph-mon[74165]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:18 smithi173 ceph-mon[74165]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:18 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/565730687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:33:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:18 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3872555783' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:33:18.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.519+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2918114210 con 0x7f28fc06ecd0 2024-04-20T11:33:18.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.522+0000 7f2914c00700 1 -- 172.21.15.117:0/3173415235 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2918114210 con 0x7f28fc06ecd0 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:33:18.523 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:33:18.524 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:33:18.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.525+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28fc06ecd0 msgr2=0x7f28fc071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.525+0000 7f291da00700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28fc06ecd0 0x7f28fc071190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2904006010 tx=0x7f290400b540 comp rx=0 tx=0).stop 2024-04-20T11:33:18.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.525+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 msgr2=0x7f291819cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.525+0000 7f291da00700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 0x7f291819cf60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f290800c8a0 tx=0x7f290800cc60 comp rx=0 tx=0).stop 2024-04-20T11:33:18.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.526+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 shutdown_connections 2024-04-20T11:33:18.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.526+0000 7f291da00700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28fc06ecd0 0x7f28fc071190 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.526+0000 7f291da00700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29181070b0 0x7f291819cf60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.526+0000 7f291da00700 1 --2- 172.21.15.117:0/3173415235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f291810d0c0 0x7f291819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.526+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 >> 172.21.15.117:0/3173415235 conn(0x7f2918076830 msgr2=0x7f2918078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:18.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.526+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 shutdown_connections 2024-04-20T11:33:18.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.526+0000 7f291da00700 1 -- 172.21.15.117:0/3173415235 wait complete. 2024-04-20T11:33:18.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.684+0000 7fd536200700 1 -- 172.21.15.117:0/1566125333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 msgr2=0x7fd530103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.684+0000 7fd536200700 1 --2- 172.21.15.117:0/1566125333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 0x7fd530103410 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd520009910 tx=0x7fd520009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:18.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.684+0000 7fd536200700 1 -- 172.21.15.117:0/1566125333 shutdown_connections 2024-04-20T11:33:18.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.684+0000 7fd536200700 1 --2- 172.21.15.117:0/1566125333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5301039e0 0x7fd530107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.684+0000 7fd536200700 1 --2- 172.21.15.117:0/1566125333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 0x7fd530103410 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.684+0000 7fd536200700 1 -- 172.21.15.117:0/1566125333 >> 172.21.15.117:0/1566125333 conn(0x7fd5300fe8c0 msgr2=0x7fd530100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:18.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.684+0000 7fd536200700 1 -- 172.21.15.117:0/1566125333 shutdown_connections 2024-04-20T11:33:18.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.684+0000 7fd536200700 1 -- 172.21.15.117:0/1566125333 wait complete. 2024-04-20T11:33:18.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.686+0000 7fd536200700 1 Processor -- start 2024-04-20T11:33:18.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.686+0000 7fd536200700 1 -- start start 2024-04-20T11:33:18.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.686+0000 7fd536200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 0x7fd530198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.686+0000 7fd536200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5301039e0 0x7fd530199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.686+0000 7fd536200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd530199850 con 0x7fd530103030 2024-04-20T11:33:18.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.686+0000 7fd536200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5301999c0 con 0x7fd5301039e0 2024-04-20T11:33:18.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 0x7fd530198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 0x7fd530198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37462/0 (socket says 172.21.15.117:37462) 2024-04-20T11:33:18.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52f400700 1 -- 172.21.15.117:0/3376816551 learned_addr learned my addr 172.21.15.117:0/3376816551 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:18.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52ea00700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5301039e0 0x7fd530199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52f400700 1 -- 172.21.15.117:0/3376816551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5301039e0 msgr2=0x7fd530199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:18.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52f400700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5301039e0 0x7fd530199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:18.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52f400700 1 -- 172.21.15.117:0/3376816551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200095f0 con 0x7fd530103030 2024-04-20T11:33:18.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52ea00700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5301039e0 0x7fd530199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:18.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.687+0000 7fd52f400700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 0x7fd530198d10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd520006010 tx=0x7fd52000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.688+0000 7fd527e00700 1 -- 172.21.15.117:0/3376816551 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd52001d070 con 0x7fd530103030 2024-04-20T11:33:18.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.688+0000 7fd527e00700 1 -- 172.21.15.117:0/3376816551 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd52000f460 con 0x7fd530103030 2024-04-20T11:33:18.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.688+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53019c640 con 0x7fd530103030 2024-04-20T11:33:18.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.688+0000 7fd527e00700 1 -- 172.21.15.117:0/3376816551 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd520017630 con 0x7fd530103030 2024-04-20T11:33:18.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.688+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53019cb30 con 0x7fd530103030 2024-04-20T11:33:18.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.689+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd530104510 con 0x7fd530103030 2024-04-20T11:33:18.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.690+0000 7fd527e00700 1 -- 172.21.15.117:0/3376816551 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd52000f5d0 con 0x7fd530103030 2024-04-20T11:33:18.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.691+0000 7fd527e00700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd51406ecc0 0x7fd514071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:18.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.691+0000 7fd527e00700 1 -- 172.21.15.117:0/3376816551 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd520090320 con 0x7fd530103030 2024-04-20T11:33:18.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.692+0000 7fd52ea00700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd51406ecc0 0x7fd514071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:18.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.692+0000 7fd52ea00700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd51406ecc0 0x7fd514071180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:18.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:18.697+0000 7fd527e00700 1 -- 172.21.15.117:0/3376816551 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd5200591c0 con 0x7fd530103030 2024-04-20T11:33:19.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.005+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd530110000 con 0x7fd530103030 2024-04-20T11:33:19.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.006+0000 7fd527e00700 1 -- 172.21.15.117:0/3376816551 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd52005c880 con 0x7fd530103030 2024-04-20T11:33:19.007 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:33:19.007 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:33:19.007 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:33:19.007 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:33:19.007 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:33:19.008 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:33:19.008 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:33:19.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd51406ecc0 msgr2=0x7fd514071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:19.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd51406ecc0 0x7fd514071180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c005e20 comp rx=0 tx=0).stop 2024-04-20T11:33:19.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 msgr2=0x7fd530198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:19.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 0x7fd530198d10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd520006010 tx=0x7fd52000be20 comp rx=0 tx=0).stop 2024-04-20T11:33:19.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 shutdown_connections 2024-04-20T11:33:19.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd51406ecc0 0x7fd514071180 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd530103030 0x7fd530198d10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 --2- 172.21.15.117:0/3376816551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5301039e0 0x7fd530199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:19.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 >> 172.21.15.117:0/3376816551 conn(0x7fd5300fe8c0 msgr2=0x7fd5300ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:19.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 shutdown_connections 2024-04-20T11:33:19.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:19.009+0000 7fd536200700 1 -- 172.21.15.117:0/3376816551 wait complete. 2024-04-20T11:33:19.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:19 smithi117 ceph-mon[99036]: pgmap v625: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:19 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3376816551' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:33:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:19 smithi173 ceph-mon[74165]: pgmap v625: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:19 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3376816551' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:33:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:20 smithi173 ceph-mon[74165]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:20 smithi117 ceph-mon[99036]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:21.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:21 smithi117 ceph-mon[99036]: pgmap v626: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:21 smithi173 ceph-mon[74165]: pgmap v626: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:23 smithi117 ceph-mon[99036]: pgmap v627: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:23 smithi173 ceph-mon[74165]: pgmap v627: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:25.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:25 smithi117 ceph-mon[99036]: pgmap v628: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:25.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:25 smithi173 ceph-mon[74165]: pgmap v628: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:27.633 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:27 smithi173 ceph-mon[74165]: pgmap v629: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:27 smithi117 ceph-mon[99036]: pgmap v629: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:33:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:33:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:29 smithi117 ceph-mon[99036]: pgmap v630: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:29.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:29 smithi173 ceph-mon[74165]: pgmap v630: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:31.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:31 smithi117 ceph-mon[99036]: pgmap v631: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:31.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:31 smithi173 ceph-mon[74165]: pgmap v631: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:33.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:33 smithi117 ceph-mon[99036]: pgmap v632: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:33 smithi173 ceph-mon[74165]: pgmap v632: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:35.465 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:35 smithi173 ceph-mon[74165]: pgmap v633: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:35.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:35 smithi117 ceph-mon[99036]: pgmap v633: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:37.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:37 smithi117 ceph-mon[99036]: pgmap v634: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:37.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:37 smithi173 ceph-mon[74165]: pgmap v634: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:39.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:39 smithi117 ceph-mon[99036]: pgmap v635: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:39 smithi173 ceph-mon[74165]: pgmap v635: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:41.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:41 smithi117 ceph-mon[99036]: pgmap v636: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:41 smithi173 ceph-mon[74165]: pgmap v636: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:43 smithi173 ceph-mon[74165]: pgmap v637: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:33:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:43 smithi117 ceph-mon[99036]: pgmap v637: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:33:45.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:45 smithi117 ceph-mon[99036]: pgmap v638: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:45 smithi173 ceph-mon[74165]: pgmap v638: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:47.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:47 smithi117 ceph-mon[99036]: pgmap v639: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:47 smithi173 ceph-mon[74165]: pgmap v639: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:49.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.181+0000 7fe652e00700 1 -- 172.21.15.117:0/2457200132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 msgr2=0x7fe64c0735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.181+0000 7fe652e00700 1 --2- 172.21.15.117:0/2457200132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 0x7fe64c0735e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009910 tx=0x7fe63c009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:49.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.181+0000 7fe652e00700 1 -- 172.21.15.117:0/2457200132 shutdown_connections 2024-04-20T11:33:49.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.181+0000 7fe652e00700 1 --2- 172.21.15.117:0/2457200132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe64c073b20 0x7fe64c10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.181+0000 7fe652e00700 1 --2- 172.21.15.117:0/2457200132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 0x7fe64c0735e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.181+0000 7fe652e00700 1 -- 172.21.15.117:0/2457200132 >> 172.21.15.117:0/2457200132 conn(0x7fe64c0fc740 msgr2=0x7fe64c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:49.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.181+0000 7fe652e00700 1 -- 172.21.15.117:0/2457200132 shutdown_connections 2024-04-20T11:33:49.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.181+0000 7fe652e00700 1 -- 172.21.15.117:0/2457200132 wait complete. 2024-04-20T11:33:49.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.182+0000 7fe652e00700 1 Processor -- start 2024-04-20T11:33:49.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.182+0000 7fe652e00700 1 -- start start 2024-04-20T11:33:49.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe652e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 0x7fe64c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe652e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe64c073b20 0x7fe64c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe652e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c1999e0 con 0x7fe64c073200 2024-04-20T11:33:49.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe652e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c19d6b0 con 0x7fe64c073b20 2024-04-20T11:33:49.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe64be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 0x7fe64c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe64be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 0x7fe64c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58000/0 (socket says 172.21.15.117:58000) 2024-04-20T11:33:49.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe64be00700 1 -- 172.21.15.117:0/270324456 learned_addr learned my addr 172.21.15.117:0/270324456 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe64b400700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe64c073b20 0x7fe64c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe64be00700 1 -- 172.21.15.117:0/270324456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe64c073b20 msgr2=0x7fe64c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.183+0000 7fe64be00700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe64c073b20 0x7fe64c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.184+0000 7fe64be00700 1 -- 172.21.15.117:0/270324456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0095f0 con 0x7fe64c073200 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.184+0000 7fe64be00700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 0x7fe64c198dc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0092e0 tx=0x7fe63c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.184+0000 7fe648c00700 1 -- 172.21.15.117:0/270324456 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe63c01d070 con 0x7fe64c073200 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.184+0000 7fe648c00700 1 -- 172.21.15.117:0/270324456 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe63c00f460 con 0x7fe64c073200 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.184+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64c19d930 con 0x7fe64c073200 2024-04-20T11:33:49.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.184+0000 7fe648c00700 1 -- 172.21.15.117:0/270324456 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe63c017630 con 0x7fe64c073200 2024-04-20T11:33:49.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.184+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c19de20 con 0x7fe64c073200 2024-04-20T11:33:49.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.186+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64c074940 con 0x7fe64c073200 2024-04-20T11:33:49.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.187+0000 7fe648c00700 1 -- 172.21.15.117:0/270324456 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe63c00f5d0 con 0x7fe64c073200 2024-04-20T11:33:49.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.188+0000 7fe648c00700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe63006ecd0 0x7fe630071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.188+0000 7fe648c00700 1 -- 172.21.15.117:0/270324456 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe63c0903f0 con 0x7fe64c073200 2024-04-20T11:33:49.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.188+0000 7fe64b400700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe63006ecd0 0x7fe630071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.189+0000 7fe64b400700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe63006ecd0 0x7fe630071190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe638006fd0 tx=0x7fe638008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:49.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.193+0000 7fe648c00700 1 -- 172.21.15.117:0/270324456 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe63c059130 con 0x7fe64c073200 2024-04-20T11:33:49.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.439+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe64c1077f0 con 0x7fe63006ecd0 2024-04-20T11:33:49.439 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:49 smithi117 ceph-mon[99036]: pgmap v640: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:49.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.441+0000 7fe648c00700 1 -- 172.21.15.117:0/270324456 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe64c1077f0 con 0x7fe63006ecd0 2024-04-20T11:33:49.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe63006ecd0 msgr2=0x7fe630071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe63006ecd0 0x7fe630071190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe638006fd0 tx=0x7fe638008040 comp rx=0 tx=0).stop 2024-04-20T11:33:49.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 msgr2=0x7fe64c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 0x7fe64c198dc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0092e0 tx=0x7fe63c00be20 comp rx=0 tx=0).stop 2024-04-20T11:33:49.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 shutdown_connections 2024-04-20T11:33:49.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe63006ecd0 0x7fe630071190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe64c073200 0x7fe64c198dc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 --2- 172.21.15.117:0/270324456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe64c073b20 0x7fe64c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.444+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 >> 172.21.15.117:0/270324456 conn(0x7fe64c0fc740 msgr2=0x7fe64c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:49.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.445+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 shutdown_connections 2024-04-20T11:33:49.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.445+0000 7fe652e00700 1 -- 172.21.15.117:0/270324456 wait complete. 2024-04-20T11:33:49.460 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:33:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.601+0000 7f74fb200700 1 -- 172.21.15.117:0/2241616881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc100a20 msgr2=0x7f74fc100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.601+0000 7f74fb200700 1 --2- 172.21.15.117:0/2241616881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc100a20 0x7f74fc100ea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009910 tx=0x7f74e8009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.602+0000 7f74fb200700 1 -- 172.21.15.117:0/2241616881 shutdown_connections 2024-04-20T11:33:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.602+0000 7f74fb200700 1 --2- 172.21.15.117:0/2241616881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc100a20 0x7f74fc100ea0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.602+0000 7f74fb200700 1 --2- 172.21.15.117:0/2241616881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74fc106a20 0x7f74fc106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.602+0000 7f74fb200700 1 -- 172.21.15.117:0/2241616881 >> 172.21.15.117:0/2241616881 conn(0x7f74fc0fc530 msgr2=0x7f74fc0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.602+0000 7f74fb200700 1 -- 172.21.15.117:0/2241616881 shutdown_connections 2024-04-20T11:33:49.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.602+0000 7f74fb200700 1 -- 172.21.15.117:0/2241616881 wait complete. 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.603+0000 7f74fb200700 1 Processor -- start 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.603+0000 7f74fb200700 1 -- start start 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.604+0000 7f74fb200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74fc100a20 0x7f74fc19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.604+0000 7f74fb200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc106a20 0x7f74fc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.604+0000 7f74fb200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc198a30 con 0x7f74fc106a20 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.604+0000 7f74fb200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc198ba0 con 0x7f74fc100a20 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.604+0000 7f74f9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc106a20 0x7f74fc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.604+0000 7f74f9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc106a20 0x7f74fc19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58016/0 (socket says 172.21.15.117:58016) 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.604+0000 7f74f9400700 1 -- 172.21.15.117:0/203194513 learned_addr learned my addr 172.21.15.117:0/203194513 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f9400700 1 -- 172.21.15.117:0/203194513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74fc100a20 msgr2=0x7f74fc19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f9e00700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74fc100a20 0x7f74fc19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f9400700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74fc100a20 0x7f74fc19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f9400700 1 -- 172.21.15.117:0/203194513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e80095f0 con 0x7f74fc106a20 2024-04-20T11:33:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f9e00700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74fc100a20 0x7f74fc19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:33:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f9400700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc106a20 0x7f74fc19eec0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f74e80092e0 tx=0x7f74e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f2a00700 1 -- 172.21.15.117:0/203194513 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74e801d070 con 0x7f74fc106a20 2024-04-20T11:33:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f2a00700 1 -- 172.21.15.117:0/203194513 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74e800f460 con 0x7f74fc106a20 2024-04-20T11:33:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74f2a00700 1 -- 172.21.15.117:0/203194513 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74e8017630 con 0x7f74fc106a20 2024-04-20T11:33:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74fc198e20 con 0x7f74fc106a20 2024-04-20T11:33:49.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.605+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74fc199310 con 0x7f74fc106a20 2024-04-20T11:33:49.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.607+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74fc104cd0 con 0x7f74fc106a20 2024-04-20T11:33:49.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.608+0000 7f74f2a00700 1 -- 172.21.15.117:0/203194513 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f74e800f5d0 con 0x7f74fc106a20 2024-04-20T11:33:49.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.609+0000 7f74f2a00700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74e006ecd0 0x7f74e0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:49.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.609+0000 7f74f2a00700 1 -- 172.21.15.117:0/203194513 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f74e8090320 con 0x7f74fc106a20 2024-04-20T11:33:49.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.609+0000 7f74f9e00700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74e006ecd0 0x7f74e0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:49.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.610+0000 7f74f9e00700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74e006ecd0 0x7f74e0071190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f74ec005fd0 tx=0x7f74ec005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:49.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.613+0000 7f74f2a00700 1 -- 172.21.15.117:0/203194513 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f74e80590e0 con 0x7f74fc106a20 2024-04-20T11:33:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:49 smithi173 ceph-mon[74165]: pgmap v640: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:49.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.859+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74fc199f10 con 0x7f74e006ecd0 2024-04-20T11:33:49.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.861+0000 7f74f2a00700 1 -- 172.21.15.117:0/203194513 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f74fc199f10 con 0x7f74e006ecd0 2024-04-20T11:33:49.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.864+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74e006ecd0 msgr2=0x7f74e0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.864+0000 7f74fb200700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74e006ecd0 0x7f74e0071190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f74ec005fd0 tx=0x7f74ec005dc0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.864+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc106a20 msgr2=0x7f74fc19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:49.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.864+0000 7f74fb200700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc106a20 0x7f74fc19eec0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f74e80092e0 tx=0x7f74e800be20 comp rx=0 tx=0).stop 2024-04-20T11:33:49.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.865+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 shutdown_connections 2024-04-20T11:33:49.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.865+0000 7f74fb200700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74e006ecd0 0x7f74e0071190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.865+0000 7f74fb200700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74fc100a20 0x7f74fc19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.865+0000 7f74fb200700 1 --2- 172.21.15.117:0/203194513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc106a20 0x7f74fc19eec0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:49.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.865+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 >> 172.21.15.117:0/203194513 conn(0x7f74fc0fc530 msgr2=0x7f74fc0733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:49.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.865+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 shutdown_connections 2024-04-20T11:33:49.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:49.865+0000 7f74fb200700 1 -- 172.21.15.117:0/203194513 wait complete. 2024-04-20T11:33:50.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.014+0000 7f08fd000700 1 -- 172.21.15.117:0/3830691784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f81017c0 msgr2=0x7f08f8105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.014+0000 7f08fd000700 1 --2- 172.21.15.117:0/3830691784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f81017c0 0x7f08f8105810 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009910 tx=0x7f08e4009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:50.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.015+0000 7f08fd000700 1 -- 172.21.15.117:0/3830691784 shutdown_connections 2024-04-20T11:33:50.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.015+0000 7f08fd000700 1 --2- 172.21.15.117:0/3830691784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f81017c0 0x7f08f8105810 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.015+0000 7f08fd000700 1 --2- 172.21.15.117:0/3830691784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08f8100e10 0x7f08f81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.015+0000 7f08fd000700 1 -- 172.21.15.117:0/3830691784 >> 172.21.15.117:0/3830691784 conn(0x7f08f80fc6a0 msgr2=0x7f08f80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:50.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.015+0000 7f08fd000700 1 -- 172.21.15.117:0/3830691784 shutdown_connections 2024-04-20T11:33:50.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.015+0000 7f08fd000700 1 -- 172.21.15.117:0/3830691784 wait complete. 2024-04-20T11:33:50.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.016+0000 7f08fd000700 1 Processor -- start 2024-04-20T11:33:50.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.016+0000 7f08fd000700 1 -- start start 2024-04-20T11:33:50.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.016+0000 7f08fd000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f8100e10 0x7f08f80731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.016+0000 7f08fd000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08f81017c0 0x7f08f8073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.016+0000 7f08fd000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f8075160 con 0x7f08f8100e10 2024-04-20T11:33:50.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.016+0000 7f08fd000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f8073c60 con 0x7f08f81017c0 2024-04-20T11:33:50.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.017+0000 7f08f5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f8100e10 0x7f08f80731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:50.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.017+0000 7f08f5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f8100e10 0x7f08f80731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58026/0 (socket says 172.21.15.117:58026) 2024-04-20T11:33:50.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.017+0000 7f08f5e00700 1 -- 172.21.15.117:0/1314094137 learned_addr learned my addr 172.21.15.117:0/1314094137 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:50.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.017+0000 7f08f5e00700 1 -- 172.21.15.117:0/1314094137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08f81017c0 msgr2=0x7f08f8073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.017+0000 7f08f5e00700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08f81017c0 0x7f08f8073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.017+0000 7f08f5e00700 1 -- 172.21.15.117:0/1314094137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e40095f0 con 0x7f08f8100e10 2024-04-20T11:33:50.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.017+0000 7f08f5e00700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f8100e10 0x7f08f80731e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c930 tx=0x7f08e800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:50.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.018+0000 7f08eea00700 1 -- 172.21.15.117:0/1314094137 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08e8007ad0 con 0x7f08f8100e10 2024-04-20T11:33:50.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.018+0000 7f08eea00700 1 -- 172.21.15.117:0/1314094137 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08e8007c30 con 0x7f08f8100e10 2024-04-20T11:33:50.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.018+0000 7f08eea00700 1 -- 172.21.15.117:0/1314094137 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08e8018610 con 0x7f08f8100e10 2024-04-20T11:33:50.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.018+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f8073f40 con 0x7f08f8100e10 2024-04-20T11:33:50.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.018+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f81a2f90 con 0x7f08f8100e10 2024-04-20T11:33:50.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.019+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f8109210 con 0x7f08f8100e10 2024-04-20T11:33:50.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.020+0000 7f08eea00700 1 -- 172.21.15.117:0/1314094137 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f08e800f450 con 0x7f08f8100e10 2024-04-20T11:33:50.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.021+0000 7f08eea00700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f08dc06ecc0 0x7f08dc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.021+0000 7f08eea00700 1 -- 172.21.15.117:0/1314094137 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f08e808efe0 con 0x7f08f8100e10 2024-04-20T11:33:50.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.021+0000 7f08f5400700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f08dc06ecc0 0x7f08dc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:50.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.022+0000 7f08f5400700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f08dc06ecc0 0x7f08dc071180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f08e400b5c0 tx=0x7f08e4005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:50.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.025+0000 7f08eea00700 1 -- 172.21.15.117:0/1314094137 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f08e8057de0 con 0x7f08f8100e10 2024-04-20T11:33:50.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.252+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f08f810ddb0 con 0x7f08dc06ecc0 2024-04-20T11:33:50.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.274+0000 7f08eea00700 1 -- 172.21.15.117:0/1314094137 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f08f810ddb0 con 0x7f08dc06ecc0 2024-04-20T11:33:50.273 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:33:50.273 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (22m) 13m ago 36m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (36m) 13m ago 36m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (35m) 13m ago 35m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (18m) 13m ago 36m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (18m) 13m ago 35m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (21m) 13m ago 36m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (14m) 13m ago 29m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (14m) 13m ago 29m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (28m) 13m ago 28m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (13m) 13m ago 29m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (25m) 13m ago 38m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (23m) 13m ago 35m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (20m) 13m ago 38m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:33:50.274 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (19m) 13m ago 34m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (23m) 13m ago 36m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (23m) 13m ago 35m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (17m) 13m ago 33m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (17m) 13m ago 33m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (16m) 13m ago 32m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (16m) 13m ago 32m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (15m) 13m ago 31m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (14m) 13m ago 30m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:33:50.275 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (22m) 13m ago 35m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:33:50.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.276+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f08dc06ecc0 msgr2=0x7f08dc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.276+0000 7f08fd000700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f08dc06ecc0 0x7f08dc071180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f08e400b5c0 tx=0x7f08e4005960 comp rx=0 tx=0).stop 2024-04-20T11:33:50.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.276+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f8100e10 msgr2=0x7f08f80731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.276+0000 7f08fd000700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f8100e10 0x7f08f80731e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c930 tx=0x7f08e800ccf0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.277+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 shutdown_connections 2024-04-20T11:33:50.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.277+0000 7f08fd000700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f08dc06ecc0 0x7f08dc071180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.277+0000 7f08fd000700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08f8100e10 0x7f08f80731e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.277+0000 7f08fd000700 1 --2- 172.21.15.117:0/1314094137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08f81017c0 0x7f08f8073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.277+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 >> 172.21.15.117:0/1314094137 conn(0x7f08f80fc6a0 msgr2=0x7f08f80fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:50.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.277+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 shutdown_connections 2024-04-20T11:33:50.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.277+0000 7f08fd000700 1 -- 172.21.15.117:0/1314094137 wait complete. 2024-04-20T11:33:50.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.432+0000 7f9a0aa00700 1 -- 172.21.15.117:0/1296063610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04108e10 msgr2=0x7f9a04109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.432+0000 7f9a0aa00700 1 --2- 172.21.15.117:0/1296063610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04108e10 0x7f9a04109290 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009910 tx=0x7f99f0009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:50.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.432+0000 7f9a0aa00700 1 -- 172.21.15.117:0/1296063610 shutdown_connections 2024-04-20T11:33:50.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.432+0000 7f9a0aa00700 1 --2- 172.21.15.117:0/1296063610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04108e10 0x7f9a04109290 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.432+0000 7f9a0aa00700 1 --2- 172.21.15.117:0/1296063610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a04107520 0x7f9a04107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.432+0000 7f9a0aa00700 1 -- 172.21.15.117:0/1296063610 >> 172.21.15.117:0/1296063610 conn(0x7f9a040768f0 msgr2=0x7f9a04078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:50.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.432+0000 7f9a0aa00700 1 -- 172.21.15.117:0/1296063610 shutdown_connections 2024-04-20T11:33:50.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.432+0000 7f9a0aa00700 1 -- 172.21.15.117:0/1296063610 wait complete. 2024-04-20T11:33:50.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.433+0000 7f9a0aa00700 1 Processor -- start 2024-04-20T11:33:50.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.433+0000 7f9a0aa00700 1 -- start start 2024-04-20T11:33:50.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.433+0000 7f9a0aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04107520 0x7f9a0419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a0aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a04108e10 0x7f9a0419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a0aa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0419ded0 con 0x7f9a04107520 2024-04-20T11:33:50.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a0aa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041a1ba0 con 0x7f9a04108e10 2024-04-20T11:33:50.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a03e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04107520 0x7f9a0419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:50.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a03e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04107520 0x7f9a0419d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58046/0 (socket says 172.21.15.117:58046) 2024-04-20T11:33:50.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a03e00700 1 -- 172.21.15.117:0/2379620726 learned_addr learned my addr 172.21.15.117:0/2379620726 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:50.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a03400700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a04108e10 0x7f9a0419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a03e00700 1 -- 172.21.15.117:0/2379620726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a04108e10 msgr2=0x7f9a0419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a03e00700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a04108e10 0x7f9a0419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.434+0000 7f9a03e00700 1 -- 172.21.15.117:0/2379620726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00095f0 con 0x7f9a04107520 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.435+0000 7f9a03e00700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04107520 0x7f9a0419d2b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f99f400b4b0 tx=0x7f99f400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.435+0000 7f9a00c00700 1 -- 172.21.15.117:0/2379620726 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f4011800 con 0x7f9a04107520 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.435+0000 7f9a00c00700 1 -- 172.21.15.117:0/2379620726 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99f4011e20 con 0x7f9a04107520 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.435+0000 7f9a00c00700 1 -- 172.21.15.117:0/2379620726 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f400f2e0 con 0x7f9a04107520 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.435+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a041a1e80 con 0x7f9a04107520 2024-04-20T11:33:50.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.435+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a041a2450 con 0x7f9a04107520 2024-04-20T11:33:50.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.436+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0410f870 con 0x7f9a04107520 2024-04-20T11:33:50.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.437+0000 7f9a00c00700 1 -- 172.21.15.117:0/2379620726 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f99f4010380 con 0x7f9a04107520 2024-04-20T11:33:50.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.438+0000 7f9a00c00700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99e806ecc0 0x7f99e8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.438+0000 7f9a00c00700 1 -- 172.21.15.117:0/2379620726 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f99f408ea00 con 0x7f9a04107520 2024-04-20T11:33:50.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.439+0000 7f9a03400700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99e806ecc0 0x7f99e8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:50.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.439+0000 7f9a03400700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99e806ecc0 0x7f99e8071180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99f000b5c0 tx=0x7f99f0005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:50.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.443+0000 7f9a00c00700 1 -- 172.21.15.117:0/2379620726 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f99f4057870 con 0x7f9a04107520 2024-04-20T11:33:50.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.757+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9a0419e5c0 con 0x7f9a04107520 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.758+0000 7f9a00c00700 1 -- 172.21.15.117:0/2379620726 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f99f405af30 con 0x7f9a04107520 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:33:50.758 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:33:50.759 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:33:50.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99e806ecc0 msgr2=0x7f99e8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99e806ecc0 0x7f99e8071180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99f000b5c0 tx=0x7f99f0005960 comp rx=0 tx=0).stop 2024-04-20T11:33:50.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04107520 msgr2=0x7f9a0419d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04107520 0x7f9a0419d2b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f99f400b4b0 tx=0x7f99f400b870 comp rx=0 tx=0).stop 2024-04-20T11:33:50.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 shutdown_connections 2024-04-20T11:33:50.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f99e806ecc0 0x7f99e8071180 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a04107520 0x7f9a0419d2b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 --2- 172.21.15.117:0/2379620726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a04108e10 0x7f9a0419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 >> 172.21.15.117:0/2379620726 conn(0x7f9a040768f0 msgr2=0x7f9a040775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:50.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 shutdown_connections 2024-04-20T11:33:50.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.761+0000 7f9a0aa00700 1 -- 172.21.15.117:0/2379620726 wait complete. 2024-04-20T11:33:50.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.910+0000 7f9337200700 1 -- 172.21.15.117:0/3958223016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 msgr2=0x7f933810cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.910+0000 7f9337200700 1 --2- 172.21.15.117:0/3958223016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 0x7f933810cd70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9324009910 tx=0x7f9324009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:50.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.910+0000 7f9337200700 1 -- 172.21.15.117:0/3958223016 shutdown_connections 2024-04-20T11:33:50.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.910+0000 7f9337200700 1 --2- 172.21.15.117:0/3958223016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 0x7f933810cd70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.910+0000 7f9337200700 1 --2- 172.21.15.117:0/3958223016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93380ff700 0x7f93380ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.910+0000 7f9337200700 1 -- 172.21.15.117:0/3958223016 >> 172.21.15.117:0/3958223016 conn(0x7f93380fb3c0 msgr2=0x7f93380fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:50.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.910+0000 7f9337200700 1 -- 172.21.15.117:0/3958223016 shutdown_connections 2024-04-20T11:33:50.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.910+0000 7f9337200700 1 -- 172.21.15.117:0/3958223016 wait complete. 2024-04-20T11:33:50.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.911+0000 7f9337200700 1 Processor -- start 2024-04-20T11:33:50.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.911+0000 7f9337200700 1 -- start start 2024-04-20T11:33:50.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.912+0000 7f9337200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93380ff700 0x7f933819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.912+0000 7f9337200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 0x7f933819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.912+0000 7f9337200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338198980 con 0x7f9338100020 2024-04-20T11:33:50.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.912+0000 7f9337200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338198af0 con 0x7f93380ff700 2024-04-20T11:33:50.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.912+0000 7f9335400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 0x7f933819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:50.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.912+0000 7f9335400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 0x7f933819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58056/0 (socket says 172.21.15.117:58056) 2024-04-20T11:33:50.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.912+0000 7f9335400700 1 -- 172.21.15.117:0/2747992197 learned_addr learned my addr 172.21.15.117:0/2747992197 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.912+0000 7f9335e00700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93380ff700 0x7f933819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f9335400700 1 -- 172.21.15.117:0/2747992197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93380ff700 msgr2=0x7f933819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f9335400700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93380ff700 0x7f933819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f9335400700 1 -- 172.21.15.117:0/2747992197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240095f0 con 0x7f9338100020 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f9335400700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 0x7f933819ee10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9324006010 tx=0x7f932400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f932ea00700 1 -- 172.21.15.117:0/2747992197 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f932401d070 con 0x7f9338100020 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f932ea00700 1 -- 172.21.15.117:0/2747992197 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f932400f460 con 0x7f9338100020 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f932ea00700 1 -- 172.21.15.117:0/2747992197 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9324017630 con 0x7f9338100020 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9338198d70 con 0x7f9338100020 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.913+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93381991c0 con 0x7f9338100020 2024-04-20T11:33:50.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.914+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93380696e0 con 0x7f9338100020 2024-04-20T11:33:50.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.915+0000 7f932ea00700 1 -- 172.21.15.117:0/2747992197 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f932400f5d0 con 0x7f9338100020 2024-04-20T11:33:50.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.916+0000 7f932ea00700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f931c06ecc0 0x7f931c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:50.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.916+0000 7f932ea00700 1 -- 172.21.15.117:0/2747992197 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9324090370 con 0x7f9338100020 2024-04-20T11:33:50.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.917+0000 7f9335e00700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f931c06ecc0 0x7f931c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:50.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.917+0000 7f9335e00700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f931c06ecc0 0x7f931c071180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9328005fd0 tx=0x7f932800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:50.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:50.921+0000 7f932ea00700 1 -- 172.21.15.117:0/2747992197 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f93240591f0 con 0x7f9338100020 2024-04-20T11:33:51.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.175+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f933810fdc0 con 0x7f9338100020 2024-04-20T11:33:51.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.176+0000 7f932ea00700 1 -- 172.21.15.117:0/2747992197 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f932405c8b0 con 0x7f9338100020 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:33:51.177 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:33:51.178 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:33:51.179 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:33:51.180 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:33:51.180 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:33:51.180 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:33:51.180 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:33:51.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.180+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f931c06ecc0 msgr2=0x7f931c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.180+0000 7f9337200700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f931c06ecc0 0x7f931c071180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9328005fd0 tx=0x7f932800a560 comp rx=0 tx=0).stop 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.180+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 msgr2=0x7f933819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.180+0000 7f9337200700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 0x7f933819ee10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9324006010 tx=0x7f932400be20 comp rx=0 tx=0).stop 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.181+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 shutdown_connections 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.181+0000 7f9337200700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f931c06ecc0 0x7f931c071180 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.181+0000 7f9337200700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93380ff700 0x7f933819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.181+0000 7f9337200700 1 --2- 172.21.15.117:0/2747992197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9338100020 0x7f933819ee10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.181+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 >> 172.21.15.117:0/2747992197 conn(0x7f93380fb3c0 msgr2=0x7f93380fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.181+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 shutdown_connections 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.181+0000 7f9337200700 1 -- 172.21.15.117:0/2747992197 wait complete. 2024-04-20T11:33:51.181 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:33:51.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.338+0000 7fbdf6e00700 1 -- 172.21.15.117:0/3515932152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 msgr2=0x7fbdf0109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.338+0000 7fbdf6e00700 1 --2- 172.21.15.117:0/3515932152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 0x7fbdf0109290 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009910 tx=0x7fbddc009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:51.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.338+0000 7fbdf6e00700 1 -- 172.21.15.117:0/3515932152 shutdown_connections 2024-04-20T11:33:51.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.338+0000 7fbdf6e00700 1 --2- 172.21.15.117:0/3515932152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 0x7fbdf0109290 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.338+0000 7fbdf6e00700 1 --2- 172.21.15.117:0/3515932152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdf0107520 0x7fbdf0107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.338+0000 7fbdf6e00700 1 -- 172.21.15.117:0/3515932152 >> 172.21.15.117:0/3515932152 conn(0x7fbdf00768f0 msgr2=0x7fbdf0078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:51.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.338+0000 7fbdf6e00700 1 -- 172.21.15.117:0/3515932152 shutdown_connections 2024-04-20T11:33:51.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.338+0000 7fbdf6e00700 1 -- 172.21.15.117:0/3515932152 wait complete. 2024-04-20T11:33:51.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.339+0000 7fbdf6e00700 1 Processor -- start 2024-04-20T11:33:51.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.340+0000 7fbdf6e00700 1 -- start start 2024-04-20T11:33:51.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.340+0000 7fbdf6e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdf0107520 0x7fbdf019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.340+0000 7fbdf6e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 0x7fbdf019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.340+0000 7fbdf6e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf019ddf0 con 0x7fbdf0108e10 2024-04-20T11:33:51.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.340+0000 7fbdf6e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf01a1a70 con 0x7fbdf0107520 2024-04-20T11:33:51.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.340+0000 7fbdef400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 0x7fbdf019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.341+0000 7fbdef400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 0x7fbdf019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58072/0 (socket says 172.21.15.117:58072) 2024-04-20T11:33:51.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.341+0000 7fbdef400700 1 -- 172.21.15.117:0/2905673399 learned_addr learned my addr 172.21.15.117:0/2905673399 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:51.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.341+0000 7fbdefe00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdf0107520 0x7fbdf019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.341+0000 7fbdef400700 1 -- 172.21.15.117:0/2905673399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdf0107520 msgr2=0x7fbdf019d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.341+0000 7fbdef400700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdf0107520 0x7fbdf019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.341+0000 7fbdef400700 1 -- 172.21.15.117:0/2905673399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0095f0 con 0x7fbdf0108e10 2024-04-20T11:33:51.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.341+0000 7fbdef400700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 0x7fbdf019d710 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006010 tx=0x7fbddc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:51.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.341+0000 7fbdecc00700 1 -- 172.21.15.117:0/2905673399 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbddc01d070 con 0x7fbdf0108e10 2024-04-20T11:33:51.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.342+0000 7fbdecc00700 1 -- 172.21.15.117:0/2905673399 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbddc00f460 con 0x7fbdf0108e10 2024-04-20T11:33:51.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.342+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf01a1cf0 con 0x7fbdf0108e10 2024-04-20T11:33:51.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.342+0000 7fbdecc00700 1 -- 172.21.15.117:0/2905673399 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbddc017630 con 0x7fbdf0108e10 2024-04-20T11:33:51.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.342+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf01a2260 con 0x7fbdf0108e10 2024-04-20T11:33:51.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.343+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf010f870 con 0x7fbdf0108e10 2024-04-20T11:33:51.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.344+0000 7fbdecc00700 1 -- 172.21.15.117:0/2905673399 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbddc00f5d0 con 0x7fbdf0108e10 2024-04-20T11:33:51.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.345+0000 7fbdecc00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbdd406ecc0 0x7fbdd4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.345+0000 7fbdecc00700 1 -- 172.21.15.117:0/2905673399 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbddc090370 con 0x7fbdf0108e10 2024-04-20T11:33:51.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.345+0000 7fbdefe00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbdd406ecc0 0x7fbdd4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.346+0000 7fbdefe00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbdd406ecc0 0x7fbdd4071180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbde0005950 tx=0x7fbde0005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:51.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.350+0000 7fbdecc00700 1 -- 172.21.15.117:0/2905673399 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbddc0591f0 con 0x7fbdf0108e10 2024-04-20T11:33:51.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.583+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdf019e490 con 0x7fbdd406ecc0 2024-04-20T11:33:51.583 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:51 smithi117 ceph-mon[99036]: pgmap v641: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:51.583 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:51 smithi117 ceph-mon[99036]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:51.583 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:51 smithi117 ceph-mon[99036]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:51.583 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:51 smithi117 ceph-mon[99036]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:51.583 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:51 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2379620726' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:33:51.583 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:51 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2747992197' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:33:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.586+0000 7fbdecc00700 1 -- 172.21.15.117:0/2905673399 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbdf019e490 con 0x7fbdd406ecc0 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:33:51.586 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:33:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbdd406ecc0 msgr2=0x7fbdd4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbdd406ecc0 0x7fbdd4071180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbde0005950 tx=0x7fbde0005cf0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 msgr2=0x7fbdf019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 0x7fbdf019d710 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006010 tx=0x7fbddc00be20 comp rx=0 tx=0).stop 2024-04-20T11:33:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 shutdown_connections 2024-04-20T11:33:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbdd406ecc0 0x7fbdd4071180 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdf0107520 0x7fbdf019d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 --2- 172.21.15.117:0/2905673399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf0108e10 0x7fbdf019d710 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 >> 172.21.15.117:0/2905673399 conn(0x7fbdf00768f0 msgr2=0x7fbdf0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.590+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 shutdown_connections 2024-04-20T11:33:51.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.591+0000 7fbdf6e00700 1 -- 172.21.15.117:0/2905673399 wait complete. 2024-04-20T11:33:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:51 smithi173 ceph-mon[74165]: pgmap v641: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:51 smithi173 ceph-mon[74165]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:51 smithi173 ceph-mon[74165]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:51 smithi173 ceph-mon[74165]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:51 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2379620726' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:33:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:51 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2747992197' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:33:51.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.739+0000 7fa553c00700 1 -- 172.21.15.117:0/1806186150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 msgr2=0x7fa54c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.739+0000 7fa553c00700 1 --2- 172.21.15.117:0/1806186150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 0x7fa54c1034f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa540009910 tx=0x7fa540009c20 comp rx=0 tx=0).stop 2024-04-20T11:33:51.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.739+0000 7fa553c00700 1 -- 172.21.15.117:0/1806186150 shutdown_connections 2024-04-20T11:33:51.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.739+0000 7fa553c00700 1 --2- 172.21.15.117:0/1806186150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54c104a00 0x7fa54c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.739+0000 7fa553c00700 1 --2- 172.21.15.117:0/1806186150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 0x7fa54c1034f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.739+0000 7fa553c00700 1 -- 172.21.15.117:0/1806186150 >> 172.21.15.117:0/1806186150 conn(0x7fa54c0fe880 msgr2=0x7fa54c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:51.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.739+0000 7fa553c00700 1 -- 172.21.15.117:0/1806186150 shutdown_connections 2024-04-20T11:33:51.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.739+0000 7fa553c00700 1 -- 172.21.15.117:0/1806186150 wait complete. 2024-04-20T11:33:51.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.740+0000 7fa553c00700 1 Processor -- start 2024-04-20T11:33:51.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.740+0000 7fa553c00700 1 -- start start 2024-04-20T11:33:51.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa553c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 0x7fa54c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa553c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54c104a00 0x7fa54c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa553c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1999e0 con 0x7fa54c103110 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa553c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c19d660 con 0x7fa54c104a00 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa551200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 0x7fa54c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa551200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 0x7fa54c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58086/0 (socket says 172.21.15.117:58086) 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa551200700 1 -- 172.21.15.117:0/2799773156 learned_addr learned my addr 172.21.15.117:0/2799773156 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa551200700 1 -- 172.21.15.117:0/2799773156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54c104a00 msgr2=0x7fa54c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa54be00700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54c104a00 0x7fa54c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa551200700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54c104a00 0x7fa54c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:51.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.741+0000 7fa551200700 1 -- 172.21.15.117:0/2799773156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400095f0 con 0x7fa54c103110 2024-04-20T11:33:51.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.742+0000 7fa551200700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 0x7fa54c198dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa540000c00 tx=0x7fa54000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:51.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.742+0000 7fa549600700 1 -- 172.21.15.117:0/2799773156 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa54001d070 con 0x7fa54c103110 2024-04-20T11:33:51.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.742+0000 7fa549600700 1 -- 172.21.15.117:0/2799773156 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5400056e0 con 0x7fa54c103110 2024-04-20T11:33:51.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.742+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c19d8e0 con 0x7fa54c103110 2024-04-20T11:33:51.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.742+0000 7fa549600700 1 -- 172.21.15.117:0/2799773156 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa54000fe80 con 0x7fa54c103110 2024-04-20T11:33:51.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.742+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c19de50 con 0x7fa54c103110 2024-04-20T11:33:51.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.743+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c10b480 con 0x7fa54c103110 2024-04-20T11:33:51.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.744+0000 7fa549600700 1 -- 172.21.15.117:0/2799773156 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa54000f6f0 con 0x7fa54c103110 2024-04-20T11:33:51.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.745+0000 7fa549600700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa53406ecd0 0x7fa534071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:51.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.745+0000 7fa549600700 1 -- 172.21.15.117:0/2799773156 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa540090200 con 0x7fa54c103110 2024-04-20T11:33:51.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.745+0000 7fa54be00700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa53406ecd0 0x7fa534071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:51.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.746+0000 7fa54be00700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa53406ecd0 0x7fa534071190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa538005950 tx=0x7fa5380058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:51.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:51.750+0000 7fa549600700 1 -- 172.21.15.117:0/2799773156 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa540058fc0 con 0x7fa54c103110 2024-04-20T11:33:52.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.041+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa54c19a080 con 0x7fa54c103110 2024-04-20T11:33:52.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.042+0000 7fa549600700 1 -- 172.21.15.117:0/2799773156 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa54005c680 con 0x7fa54c103110 2024-04-20T11:33:52.041 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:33:52.042 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:33:52.042 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:33:52.042 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:33:52.042 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:33:52.042 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:33:52.042 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa53406ecd0 msgr2=0x7fa534071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa53406ecd0 0x7fa534071190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa538005950 tx=0x7fa5380058e0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 msgr2=0x7fa54c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 0x7fa54c198dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa540000c00 tx=0x7fa54000be20 comp rx=0 tx=0).stop 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 shutdown_connections 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa53406ecd0 0x7fa534071190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c103110 0x7fa54c198dc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 --2- 172.21.15.117:0/2799773156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54c104a00 0x7fa54c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 >> 172.21.15.117:0/2799773156 conn(0x7fa54c0fe880 msgr2=0x7fa54c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.045+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 shutdown_connections 2024-04-20T11:33:52.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:33:52.046+0000 7fa553c00700 1 -- 172.21.15.117:0/2799773156 wait complete. 2024-04-20T11:33:52.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:52 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2799773156' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:33:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:52 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2799773156' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:33:53.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:53 smithi117 ceph-mon[99036]: pgmap v642: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:53.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:53 smithi117 ceph-mon[99036]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:53 smithi173 ceph-mon[74165]: pgmap v642: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:53 smithi173 ceph-mon[74165]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:33:55.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:55 smithi117 ceph-mon[99036]: pgmap v643: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:55.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:55 smithi173 ceph-mon[74165]: pgmap v643: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:57 smithi117 ceph-mon[99036]: pgmap v644: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:57.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:57 smithi173 ceph-mon[74165]: pgmap v644: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:58.592 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:33:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:33:59.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:33:59 smithi117 ceph-mon[99036]: pgmap v645: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:33:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:33:59 smithi173 ceph-mon[74165]: pgmap v645: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:01.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:01 smithi117 ceph-mon[99036]: pgmap v646: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:01.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:01 smithi173 ceph-mon[74165]: pgmap v646: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:03.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:03 smithi117 ceph-mon[99036]: pgmap v647: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:03.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:03 smithi173 ceph-mon[74165]: pgmap v647: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:05.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:05 smithi117 ceph-mon[99036]: pgmap v648: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:05.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:05 smithi173 ceph-mon[74165]: pgmap v648: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:07.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:07 smithi117 ceph-mon[99036]: pgmap v649: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:07.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:07 smithi173 ceph-mon[74165]: pgmap v649: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:09.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:09 smithi117 ceph-mon[99036]: pgmap v650: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:09 smithi173 ceph-mon[74165]: pgmap v650: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:11.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:11 smithi117 ceph-mon[99036]: pgmap v651: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:11 smithi173 ceph-mon[74165]: pgmap v651: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:13.604 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:13 smithi173 ceph-mon[74165]: pgmap v652: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:13.604 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:34:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:13 smithi117 ceph-mon[99036]: pgmap v652: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:34:15.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:15 smithi117 ceph-mon[99036]: pgmap v653: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:15 smithi173 ceph-mon[74165]: pgmap v653: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:17.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:17 smithi117 ceph-mon[99036]: pgmap v654: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:17.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:17 smithi173 ceph-mon[74165]: pgmap v654: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:19.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:19 smithi117 ceph-mon[99036]: pgmap v655: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:19.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:19 smithi173 ceph-mon[74165]: pgmap v655: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:20 smithi117 ceph-mon[99036]: pgmap v656: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:20.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:20 smithi173 ceph-mon[74165]: pgmap v656: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:22.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.215+0000 7fe50fe00700 1 -- 172.21.15.117:0/3212905939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 msgr2=0x7fe510106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.215+0000 7fe50fe00700 1 --2- 172.21.15.117:0/3212905939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 0x7fe510106db0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe500009910 tx=0x7fe500009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:22.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.216+0000 7fe50fe00700 1 -- 172.21.15.117:0/3212905939 shutdown_connections 2024-04-20T11:34:22.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.216+0000 7fe50fe00700 1 --2- 172.21.15.117:0/3212905939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5101009d0 0x7fe510100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.216+0000 7fe50fe00700 1 --2- 172.21.15.117:0/3212905939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 0x7fe510106db0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.216+0000 7fe50fe00700 1 -- 172.21.15.117:0/3212905939 >> 172.21.15.117:0/3212905939 conn(0x7fe5100fc500 msgr2=0x7fe5100fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.216+0000 7fe50fe00700 1 -- 172.21.15.117:0/3212905939 shutdown_connections 2024-04-20T11:34:22.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.216+0000 7fe50fe00700 1 -- 172.21.15.117:0/3212905939 wait complete. 2024-04-20T11:34:22.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.218+0000 7fe50fe00700 1 Processor -- start 2024-04-20T11:34:22.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.218+0000 7fe50fe00700 1 -- start start 2024-04-20T11:34:22.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.219+0000 7fe50fe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5101009d0 0x7fe51019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.219+0000 7fe50fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 0x7fe51019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.219+0000 7fe50fe00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51019d2f0 con 0x7fe5101069d0 2024-04-20T11:34:22.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.219+0000 7fe50fe00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101968d0 con 0x7fe5101009d0 2024-04-20T11:34:22.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.219+0000 7fe506000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 0x7fe51019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.220+0000 7fe506000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 0x7fe51019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35228/0 (socket says 172.21.15.117:35228) 2024-04-20T11:34:22.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.220+0000 7fe506000700 1 -- 172.21.15.117:0/2524556437 learned_addr learned my addr 172.21.15.117:0/2524556437 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:22.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.220+0000 7fe50ea00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5101009d0 0x7fe51019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.220+0000 7fe506000700 1 -- 172.21.15.117:0/2524556437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5101009d0 msgr2=0x7fe51019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.220+0000 7fe506000700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5101009d0 0x7fe51019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.220+0000 7fe506000700 1 -- 172.21.15.117:0/2524556437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000095f0 con 0x7fe5101069d0 2024-04-20T11:34:22.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.220+0000 7fe506000700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 0x7fe51019ccd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800ba50 tx=0x7fe4f800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.221+0000 7fe507e00700 1 -- 172.21.15.117:0/2524556437 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4f800ca00 con 0x7fe5101069d0 2024-04-20T11:34:22.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.221+0000 7fe507e00700 1 -- 172.21.15.117:0/2524556437 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4f800cb60 con 0x7fe5101069d0 2024-04-20T11:34:22.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.221+0000 7fe507e00700 1 -- 172.21.15.117:0/2524556437 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4f80075a0 con 0x7fe5101069d0 2024-04-20T11:34:22.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.221+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe510196bb0 con 0x7fe5101069d0 2024-04-20T11:34:22.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.221+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe510197100 con 0x7fe5101069d0 2024-04-20T11:34:22.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.222+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe510108fd0 con 0x7fe5101069d0 2024-04-20T11:34:22.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.223+0000 7fe507e00700 1 -- 172.21.15.117:0/2524556437 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe4f800ccd0 con 0x7fe5101069d0 2024-04-20T11:34:22.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.224+0000 7fe507e00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe4f406ecc0 0x7fe4f4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.224+0000 7fe507e00700 1 -- 172.21.15.117:0/2524556437 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe4f8006e40 con 0x7fe5101069d0 2024-04-20T11:34:22.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.224+0000 7fe50ea00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe4f406ecc0 0x7fe4f4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.225+0000 7fe50ea00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe4f406ecc0 0x7fe4f4071180 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe500005f50 tx=0x7fe50001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.229+0000 7fe507e00700 1 -- 172.21.15.117:0/2524556437 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe4f805af20 con 0x7fe5101069d0 2024-04-20T11:34:22.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.482+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe510197e00 con 0x7fe4f406ecc0 2024-04-20T11:34:22.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:22 smithi117 ceph-mon[99036]: pgmap v657: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:22.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.484+0000 7fe507e00700 1 -- 172.21.15.117:0/2524556437 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe510197e00 con 0x7fe4f406ecc0 2024-04-20T11:34:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe4f406ecc0 msgr2=0x7fe4f4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe4f406ecc0 0x7fe4f4071180 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe500005f50 tx=0x7fe50001a040 comp rx=0 tx=0).stop 2024-04-20T11:34:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 msgr2=0x7fe51019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 0x7fe51019ccd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800ba50 tx=0x7fe4f800be10 comp rx=0 tx=0).stop 2024-04-20T11:34:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 shutdown_connections 2024-04-20T11:34:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe4f406ecc0 0x7fe4f4071180 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5101009d0 0x7fe51019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 --2- 172.21.15.117:0/2524556437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5101069d0 0x7fe51019ccd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.487+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 >> 172.21.15.117:0/2524556437 conn(0x7fe5100fc500 msgr2=0x7fe5100fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.488+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 shutdown_connections 2024-04-20T11:34:22.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.488+0000 7fe50fe00700 1 -- 172.21.15.117:0/2524556437 wait complete. 2024-04-20T11:34:22.504 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:34:22.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.643+0000 7f2906600700 1 -- 172.21.15.117:0/3773692576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29001070b0 msgr2=0x7f2900107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.643+0000 7f2906600700 1 --2- 172.21.15.117:0/3773692576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29001070b0 0x7f2900107510 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009910 tx=0x7f28ec009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:22.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.643+0000 7f2906600700 1 -- 172.21.15.117:0/3773692576 shutdown_connections 2024-04-20T11:34:22.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.643+0000 7f2906600700 1 --2- 172.21.15.117:0/3773692576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29001070b0 0x7f2900107510 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.643+0000 7f2906600700 1 --2- 172.21.15.117:0/3773692576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f290010d0c0 0x7f290010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.643+0000 7f2906600700 1 -- 172.21.15.117:0/3773692576 >> 172.21.15.117:0/3773692576 conn(0x7f2900076830 msgr2=0x7f2900078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.643+0000 7f2906600700 1 -- 172.21.15.117:0/3773692576 shutdown_connections 2024-04-20T11:34:22.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.643+0000 7f2906600700 1 -- 172.21.15.117:0/3773692576 wait complete. 2024-04-20T11:34:22.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.644+0000 7f2906600700 1 Processor -- start 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.644+0000 7f2906600700 1 -- start start 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f2906600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29001070b0 0x7f290019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f2906600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290010d0c0 0x7f290019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f2906600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290019db80 con 0x7f290010d0c0 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f2906600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29001a1850 con 0x7f29001070b0 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f28ffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290010d0c0 0x7f290019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f28ffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290010d0c0 0x7f290019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35242/0 (socket says 172.21.15.117:35242) 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f28ffe00700 1 -- 172.21.15.117:0/2571464983 learned_addr learned my addr 172.21.15.117:0/2571464983 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f28ffe00700 1 -- 172.21.15.117:0/2571464983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29001070b0 msgr2=0x7f290019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f28ffe00700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29001070b0 0x7f290019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.645+0000 7f28ffe00700 1 -- 172.21.15.117:0/2571464983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0095f0 con 0x7f290010d0c0 2024-04-20T11:34:22.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.646+0000 7f28ffe00700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290010d0c0 0x7f290019d4a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c00 tx=0x7f28ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.646+0000 7f28fd600700 1 -- 172.21.15.117:0/2571464983 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ec01d070 con 0x7f290010d0c0 2024-04-20T11:34:22.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.646+0000 7f28fd600700 1 -- 172.21.15.117:0/2571464983 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28ec0056e0 con 0x7f290010d0c0 2024-04-20T11:34:22.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.646+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29001a1ad0 con 0x7f290010d0c0 2024-04-20T11:34:22.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.646+0000 7f28fd600700 1 -- 172.21.15.117:0/2571464983 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ec00fe80 con 0x7f290010d0c0 2024-04-20T11:34:22.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.646+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29001a1fc0 con 0x7f290010d0c0 2024-04-20T11:34:22.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.647+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290010f5c0 con 0x7f290010d0c0 2024-04-20T11:34:22.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.648+0000 7f28fd600700 1 -- 172.21.15.117:0/2571464983 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f28ec00f6f0 con 0x7f290010d0c0 2024-04-20T11:34:22.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.649+0000 7f28fd600700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28e806ecd0 0x7f28e8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:22.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.649+0000 7f28fd600700 1 -- 172.21.15.117:0/2571464983 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f28ec0902e0 con 0x7f290010d0c0 2024-04-20T11:34:22.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.649+0000 7f2905200700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28e806ecd0 0x7f28e8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:22.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.650+0000 7f2905200700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28e806ecd0 0x7f28e8071190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f28f4005950 tx=0x7f28f40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.654+0000 7f28fd600700 1 -- 172.21.15.117:0/2571464983 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f28ec059020 con 0x7f290010d0c0 2024-04-20T11:34:22.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.896+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f290019e2e0 con 0x7f28e806ecd0 2024-04-20T11:34:22.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.898+0000 7f28fd600700 1 -- 172.21.15.117:0/2571464983 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f290019e2e0 con 0x7f28e806ecd0 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28e806ecd0 msgr2=0x7f28e8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28e806ecd0 0x7f28e8071190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f28f4005950 tx=0x7f28f40058e0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290010d0c0 msgr2=0x7f290019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290010d0c0 0x7f290019d4a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c00 tx=0x7f28ec00be20 comp rx=0 tx=0).stop 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 shutdown_connections 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f28e806ecd0 0x7f28e8071190 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29001070b0 0x7f290019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 --2- 172.21.15.117:0/2571464983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290010d0c0 0x7f290019d4a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 >> 172.21.15.117:0/2571464983 conn(0x7f2900076830 msgr2=0x7f2900078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:22.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.901+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 shutdown_connections 2024-04-20T11:34:22.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:22.902+0000 7f2906600700 1 -- 172.21.15.117:0/2571464983 wait complete. 2024-04-20T11:34:22.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:22 smithi173 ceph-mon[74165]: pgmap v657: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.068+0000 7f5750200700 1 -- 172.21.15.117:0/2581590638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 msgr2=0x7f5748111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.068+0000 7f5750200700 1 --2- 172.21.15.117:0/2581590638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 0x7f5748111680 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5734009910 tx=0x7f5734009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.068+0000 7f5750200700 1 -- 172.21.15.117:0/2581590638 shutdown_connections 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.068+0000 7f5750200700 1 --2- 172.21.15.117:0/2581590638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 0x7f5748111680 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.068+0000 7f5750200700 1 --2- 172.21.15.117:0/2581590638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5748073280 0x7f5748073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.068+0000 7f5750200700 1 -- 172.21.15.117:0/2581590638 >> 172.21.15.117:0/2581590638 conn(0x7f57480fc6a0 msgr2=0x7f57480feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.068+0000 7f5750200700 1 -- 172.21.15.117:0/2581590638 shutdown_connections 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.068+0000 7f5750200700 1 -- 172.21.15.117:0/2581590638 wait complete. 2024-04-20T11:34:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.069+0000 7f5750200700 1 Processor -- start 2024-04-20T11:34:23.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.070+0000 7f5750200700 1 -- start start 2024-04-20T11:34:23.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.070+0000 7f5750200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5748073280 0x7f574819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.070+0000 7f5750200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 0x7f574819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.070+0000 7f5750200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574819dd20 con 0x7f5748073ba0 2024-04-20T11:34:23.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.070+0000 7f5750200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57481a19f0 con 0x7f5748073280 2024-04-20T11:34:23.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.071+0000 7f574ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 0x7f574819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.071+0000 7f574ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 0x7f574819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35262/0 (socket says 172.21.15.117:35262) 2024-04-20T11:34:23.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.071+0000 7f574ce00700 1 -- 172.21.15.117:0/3247927581 learned_addr learned my addr 172.21.15.117:0/3247927581 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.071+0000 7f574d800700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5748073280 0x7f574819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.071+0000 7f574ce00700 1 -- 172.21.15.117:0/3247927581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5748073280 msgr2=0x7f574819d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.071+0000 7f574ce00700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5748073280 0x7f574819d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.071+0000 7f574ce00700 1 -- 172.21.15.117:0/3247927581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57340095f0 con 0x7f5748073ba0 2024-04-20T11:34:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.072+0000 7f574ce00700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 0x7f574819d640 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5734000c00 tx=0x7f573400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.072+0000 7f5746000700 1 -- 172.21.15.117:0/3247927581 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573401d070 con 0x7f5748073ba0 2024-04-20T11:34:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.072+0000 7f5746000700 1 -- 172.21.15.117:0/3247927581 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57340056e0 con 0x7f5748073ba0 2024-04-20T11:34:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.072+0000 7f5746000700 1 -- 172.21.15.117:0/3247927581 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573400fe80 con 0x7f5748073ba0 2024-04-20T11:34:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.072+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57481a1cd0 con 0x7f5748073ba0 2024-04-20T11:34:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.072+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57481a2220 con 0x7f5748073ba0 2024-04-20T11:34:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.073+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57480749c0 con 0x7f5748073ba0 2024-04-20T11:34:23.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.075+0000 7f5746000700 1 -- 172.21.15.117:0/3247927581 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f573400f6f0 con 0x7f5748073ba0 2024-04-20T11:34:23.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.076+0000 7f5746000700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ecd0 0x7f5730071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.076+0000 7f5746000700 1 -- 172.21.15.117:0/3247927581 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f57340901f0 con 0x7f5748073ba0 2024-04-20T11:34:23.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.076+0000 7f574d800700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ecd0 0x7f5730071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.077+0000 7f574d800700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ecd0 0x7f5730071190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f573c005950 tx=0x7f573c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.081+0000 7f5746000700 1 -- 172.21.15.117:0/3247927581 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5734058fb0 con 0x7f5748073ba0 2024-04-20T11:34:23.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.308+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f57481143e0 con 0x7f573006ecd0 2024-04-20T11:34:23.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.341+0000 7f5746000700 1 -- 172.21.15.117:0/3247927581 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f57481143e0 con 0x7f573006ecd0 2024-04-20T11:34:23.341 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:34:23.341 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (22m) 14m ago 36m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:34:23.341 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (37m) 14m ago 37m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (35m) 14m ago 35m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (19m) 14m ago 37m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (19m) 14m ago 35m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (22m) 14m ago 36m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (14m) 14m ago 29m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (15m) 14m ago 29m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (29m) 14m ago 29m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (14m) 14m ago 29m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (26m) 14m ago 38m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (24m) 14m ago 35m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (20m) 14m ago 38m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (20m) 14m ago 35m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (24m) 14m ago 37m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:34:23.342 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (23m) 14m ago 35m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:34:23.343 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (18m) 14m ago 34m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:34:23.343 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (17m) 14m ago 33m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:34:23.343 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (17m) 14m ago 33m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:34:23.343 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (16m) 14m ago 32m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:34:23.343 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (16m) 14m ago 32m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:34:23.343 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (15m) 14m ago 31m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:34:23.343 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (23m) 14m ago 36m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ecd0 msgr2=0x7f5730071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ecd0 0x7f5730071190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f573c005950 tx=0x7f573c0058e0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 msgr2=0x7f574819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 0x7f574819d640 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5734000c00 tx=0x7f573400be20 comp rx=0 tx=0).stop 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 shutdown_connections 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f573006ecd0 0x7f5730071190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5748073280 0x7f574819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 --2- 172.21.15.117:0/3247927581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5748073ba0 0x7f574819d640 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.344+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 >> 172.21.15.117:0/3247927581 conn(0x7f57480fc6a0 msgr2=0x7f57481008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.345+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 shutdown_connections 2024-04-20T11:34:23.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.345+0000 7f5750200700 1 -- 172.21.15.117:0/3247927581 wait complete. 2024-04-20T11:34:23.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.490+0000 7f443e000700 1 -- 172.21.15.117:0/3152468342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438100e10 msgr2=0x7f44381011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.490+0000 7f443e000700 1 --2- 172.21.15.117:0/3152468342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438100e10 0x7f44381011f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f442800b0a0 tx=0x7f442800b3b0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.490+0000 7f443e000700 1 -- 172.21.15.117:0/3152468342 shutdown_connections 2024-04-20T11:34:23.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.490+0000 7f443e000700 1 --2- 172.21.15.117:0/3152468342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44381017c0 0x7f4438105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.490+0000 7f443e000700 1 --2- 172.21.15.117:0/3152468342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4438100e10 0x7f44381011f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.490+0000 7f443e000700 1 -- 172.21.15.117:0/3152468342 >> 172.21.15.117:0/3152468342 conn(0x7f44380fc6c0 msgr2=0x7f44380feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:23.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.490+0000 7f443e000700 1 -- 172.21.15.117:0/3152468342 shutdown_connections 2024-04-20T11:34:23.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.490+0000 7f443e000700 1 -- 172.21.15.117:0/3152468342 wait complete. 2024-04-20T11:34:23.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.492+0000 7f443e000700 1 Processor -- start 2024-04-20T11:34:23.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.492+0000 7f443e000700 1 -- start start 2024-04-20T11:34:23.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.492+0000 7f443e000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4438100e10 0x7f4438198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.492+0000 7f443e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381017c0 0x7f4438199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.492+0000 7f443e000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44381999e0 con 0x7f44381017c0 2024-04-20T11:34:23.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.492+0000 7f443e000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819d660 con 0x7f4438100e10 2024-04-20T11:34:23.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.492+0000 7f442ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381017c0 0x7f4438199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.493+0000 7f442ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381017c0 0x7f4438199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35278/0 (socket says 172.21.15.117:35278) 2024-04-20T11:34:23.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.493+0000 7f442ea00700 1 -- 172.21.15.117:0/4294400868 learned_addr learned my addr 172.21.15.117:0/4294400868 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:23.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.493+0000 7f4437400700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4438100e10 0x7f4438198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.493+0000 7f442ea00700 1 -- 172.21.15.117:0/4294400868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4438100e10 msgr2=0x7f4438198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.493+0000 7f442ea00700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4438100e10 0x7f4438198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.493+0000 7f442ea00700 1 -- 172.21.15.117:0/4294400868 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4428009d00 con 0x7f44381017c0 2024-04-20T11:34:23.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.493+0000 7f442ea00700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381017c0 0x7f4438199300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f442000d8d0 tx=0x7f442000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.494+0000 7f4434c00700 1 -- 172.21.15.117:0/4294400868 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4420004a00 con 0x7f44381017c0 2024-04-20T11:34:23.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.494+0000 7f4434c00700 1 -- 172.21.15.117:0/4294400868 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44200054e0 con 0x7f44381017c0 2024-04-20T11:34:23.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.494+0000 7f4434c00700 1 -- 172.21.15.117:0/4294400868 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4420009d40 con 0x7f44381017c0 2024-04-20T11:34:23.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.494+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443819d940 con 0x7f44381017c0 2024-04-20T11:34:23.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.494+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443819de90 con 0x7f44381017c0 2024-04-20T11:34:23.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.495+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44381022f0 con 0x7f44381017c0 2024-04-20T11:34:23.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.496+0000 7f4434c00700 1 -- 172.21.15.117:0/4294400868 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4420005020 con 0x7f44381017c0 2024-04-20T11:34:23.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.497+0000 7f4434c00700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecc0 0x7f441c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.497+0000 7f4434c00700 1 -- 172.21.15.117:0/4294400868 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f442008e740 con 0x7f44381017c0 2024-04-20T11:34:23.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.497+0000 7f4437400700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecc0 0x7f441c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.498+0000 7f4437400700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecc0 0x7f441c071180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4428009480 tx=0x7f4428000f40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.502+0000 7f4434c00700 1 -- 172.21.15.117:0/4294400868 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4420057500 con 0x7f44381017c0 2024-04-20T11:34:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:23 smithi117 ceph-mon[99036]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:23 smithi117 ceph-mon[99036]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:23.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:23 smithi117 ceph-mon[99036]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:23.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.799+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f443810de00 con 0x7f44381017c0 2024-04-20T11:34:23.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.800+0000 7f4434c00700 1 -- 172.21.15.117:0/4294400868 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f442005abc0 con 0x7f44381017c0 2024-04-20T11:34:23.800 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:34:23.802 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:34:23.802 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:34:23.802 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:34:23.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.802+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecc0 msgr2=0x7f441c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.802+0000 7f443e000700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecc0 0x7f441c071180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4428009480 tx=0x7f4428000f40 comp rx=0 tx=0).stop 2024-04-20T11:34:23.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.802+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381017c0 msgr2=0x7f4438199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.802+0000 7f443e000700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381017c0 0x7f4438199300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f442000d8d0 tx=0x7f442000dc90 comp rx=0 tx=0).stop 2024-04-20T11:34:23.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.803+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 shutdown_connections 2024-04-20T11:34:23.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.803+0000 7f443e000700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f441c06ecc0 0x7f441c071180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.803+0000 7f443e000700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4438100e10 0x7f4438198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.803+0000 7f443e000700 1 --2- 172.21.15.117:0/4294400868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44381017c0 0x7f4438199300 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.803+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 >> 172.21.15.117:0/4294400868 conn(0x7f44380fc6c0 msgr2=0x7f4438074400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:23.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.803+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 shutdown_connections 2024-04-20T11:34:23.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.803+0000 7f443e000700 1 -- 172.21.15.117:0/4294400868 wait complete. 2024-04-20T11:34:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:23 smithi173 ceph-mon[74165]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:23 smithi173 ceph-mon[74165]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:23 smithi173 ceph-mon[74165]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:23.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.950+0000 7f0b61e00700 1 -- 172.21.15.117:0/2735485413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073200 msgr2=0x7f0b5c0735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.950+0000 7f0b61e00700 1 --2- 172.21.15.117:0/2735485413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073200 0x7f0b5c0735e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009910 tx=0x7f0b4c009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:23.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.950+0000 7f0b61e00700 1 -- 172.21.15.117:0/2735485413 shutdown_connections 2024-04-20T11:34:23.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.950+0000 7f0b61e00700 1 --2- 172.21.15.117:0/2735485413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b5c073b20 0x7f0b5c10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.950+0000 7f0b61e00700 1 --2- 172.21.15.117:0/2735485413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073200 0x7f0b5c0735e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.950+0000 7f0b61e00700 1 -- 172.21.15.117:0/2735485413 >> 172.21.15.117:0/2735485413 conn(0x7f0b5c0fc6c0 msgr2=0x7f0b5c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:23.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.951+0000 7f0b61e00700 1 -- 172.21.15.117:0/2735485413 shutdown_connections 2024-04-20T11:34:23.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.951+0000 7f0b61e00700 1 -- 172.21.15.117:0/2735485413 wait complete. 2024-04-20T11:34:23.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.952+0000 7f0b61e00700 1 Processor -- start 2024-04-20T11:34:23.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.952+0000 7f0b61e00700 1 -- start start 2024-04-20T11:34:23.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.952+0000 7f0b61e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b5c073200 0x7f0b5c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.952+0000 7f0b61e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073b20 0x7f0b5c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.952+0000 7f0b61e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c19f530 con 0x7f0b5c073b20 2024-04-20T11:34:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.952+0000 7f0b61e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c198b10 con 0x7f0b5c073200 2024-04-20T11:34:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.952+0000 7f0b5a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073b20 0x7f0b5c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.953+0000 7f0b5a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073b20 0x7f0b5c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50038/0 (socket says 172.21.15.117:50038) 2024-04-20T11:34:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.953+0000 7f0b5a000700 1 -- 172.21.15.117:0/3254723440 learned_addr learned my addr 172.21.15.117:0/3254723440 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.953+0000 7f0b5aa00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b5c073200 0x7f0b5c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.953+0000 7f0b5a000700 1 -- 172.21.15.117:0/3254723440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b5c073200 msgr2=0x7f0b5c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.953+0000 7f0b5a000700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b5c073200 0x7f0b5c19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.953+0000 7f0b5a000700 1 -- 172.21.15.117:0/3254723440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0095f0 con 0x7f0b5c073b20 2024-04-20T11:34:23.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.953+0000 7f0b5a000700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073b20 0x7f0b5c19ef10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c8a0 tx=0x7f0b4800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.953+0000 7f0b60a00700 1 -- 172.21.15.117:0/3254723440 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4800cea0 con 0x7f0b5c073b20 2024-04-20T11:34:23.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.954+0000 7f0b60a00700 1 -- 172.21.15.117:0/3254723440 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b48004830 con 0x7f0b5c073b20 2024-04-20T11:34:23.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.954+0000 7f0b60a00700 1 -- 172.21.15.117:0/3254723440 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b480055b0 con 0x7f0b5c073b20 2024-04-20T11:34:23.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.954+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c198df0 con 0x7f0b5c073b20 2024-04-20T11:34:23.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.954+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c199340 con 0x7f0b5c073b20 2024-04-20T11:34:23.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.955+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5c107860 con 0x7f0b5c073b20 2024-04-20T11:34:23.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.956+0000 7f0b60a00700 1 -- 172.21.15.117:0/3254723440 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0b480049a0 con 0x7f0b5c073b20 2024-04-20T11:34:23.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.957+0000 7f0b60a00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0b4006ecc0 0x7f0b40071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:23.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.957+0000 7f0b60a00700 1 -- 172.21.15.117:0/3254723440 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0b4808eac0 con 0x7f0b5c073b20 2024-04-20T11:34:23.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.958+0000 7f0b5aa00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0b4006ecc0 0x7f0b40071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:23.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.958+0000 7f0b5aa00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0b4006ecc0 0x7f0b40071180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b5c0 tx=0x7f0b4c005380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:23.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:23.963+0000 7f0b60a00700 1 -- 172.21.15.117:0/3254723440 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0b4805b3b0 con 0x7f0b5c073b20 2024-04-20T11:34:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.232+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b5c10ffd0 con 0x7f0b5c073b20 2024-04-20T11:34:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.233+0000 7f0b60a00700 1 -- 172.21.15.117:0/3254723440 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7f0b4801d070 con 0x7f0b5c073b20 2024-04-20T11:34:24.234 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:34:24.234 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:34:24.234 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:34:24.234 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:34:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:34:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:34:24.237 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:34:24.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.235+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0b4006ecc0 msgr2=0x7f0b40071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.235+0000 7f0b61e00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0b4006ecc0 0x7f0b40071180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b5c0 tx=0x7f0b4c005380 comp rx=0 tx=0).stop 2024-04-20T11:34:24.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.235+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073b20 msgr2=0x7f0b5c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.235+0000 7f0b61e00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073b20 0x7f0b5c19ef10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c8a0 tx=0x7f0b4800cc60 comp rx=0 tx=0).stop 2024-04-20T11:34:24.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.235+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 shutdown_connections 2024-04-20T11:34:24.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.236+0000 7f0b61e00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0b4006ecc0 0x7f0b40071180 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.236+0000 7f0b61e00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b5c073200 0x7f0b5c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.236+0000 7f0b61e00700 1 --2- 172.21.15.117:0/3254723440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5c073b20 0x7f0b5c19ef10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.236+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 >> 172.21.15.117:0/3254723440 conn(0x7f0b5c0fc6c0 msgr2=0x7f0b5c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:24.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.236+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 shutdown_connections 2024-04-20T11:34:24.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.236+0000 7f0b61e00700 1 -- 172.21.15.117:0/3254723440 wait complete. 2024-04-20T11:34:24.239 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:34:24.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.388+0000 7fcd11400700 1 -- 172.21.15.117:0/3688314380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c103110 msgr2=0x7fcd0c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.388+0000 7fcd11400700 1 --2- 172.21.15.117:0/3688314380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c103110 0x7fcd0c1034f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009910 tx=0x7fccfc009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:24.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.389+0000 7fcd11400700 1 -- 172.21.15.117:0/3688314380 shutdown_connections 2024-04-20T11:34:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.389+0000 7fcd11400700 1 --2- 172.21.15.117:0/3688314380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c104a00 0x7fcd0c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.389+0000 7fcd11400700 1 --2- 172.21.15.117:0/3688314380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c103110 0x7fcd0c1034f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.389+0000 7fcd11400700 1 -- 172.21.15.117:0/3688314380 >> 172.21.15.117:0/3688314380 conn(0x7fcd0c0fe880 msgr2=0x7fcd0c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.389+0000 7fcd11400700 1 -- 172.21.15.117:0/3688314380 shutdown_connections 2024-04-20T11:34:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.389+0000 7fcd11400700 1 -- 172.21.15.117:0/3688314380 wait complete. 2024-04-20T11:34:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.390+0000 7fcd11400700 1 Processor -- start 2024-04-20T11:34:24.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.390+0000 7fcd11400700 1 -- start start 2024-04-20T11:34:24.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.390+0000 7fcd11400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c103110 0x7fcd0c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.390+0000 7fcd11400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c104a00 0x7fcd0c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.390+0000 7fcd11400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c199950 con 0x7fcd0c104a00 2024-04-20T11:34:24.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.390+0000 7fcd11400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c199a90 con 0x7fcd0c103110 2024-04-20T11:34:24.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.391+0000 7fcd09e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c104a00 0x7fcd0c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.391+0000 7fcd09e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c104a00 0x7fcd0c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50064/0 (socket says 172.21.15.117:50064) 2024-04-20T11:34:24.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.391+0000 7fcd09e00700 1 -- 172.21.15.117:0/3031455871 learned_addr learned my addr 172.21.15.117:0/3031455871 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:24.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.391+0000 7fcd0a800700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c103110 0x7fcd0c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.391+0000 7fcd09e00700 1 -- 172.21.15.117:0/3031455871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c103110 msgr2=0x7fcd0c198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.391+0000 7fcd09e00700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c103110 0x7fcd0c198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.391+0000 7fcd09e00700 1 -- 172.21.15.117:0/3031455871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc0095f0 con 0x7fcd0c104a00 2024-04-20T11:34:24.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.391+0000 7fcd09e00700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c104a00 0x7fcd0c199300 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fccf800c930 tx=0x7fccf800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:24.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.392+0000 7fcd03400700 1 -- 172.21.15.117:0/3031455871 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf8007ad0 con 0x7fcd0c104a00 2024-04-20T11:34:24.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.392+0000 7fcd03400700 1 -- 172.21.15.117:0/3031455871 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccf8007c30 con 0x7fcd0c104a00 2024-04-20T11:34:24.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.392+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c19d7d0 con 0x7fcd0c104a00 2024-04-20T11:34:24.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.392+0000 7fcd03400700 1 -- 172.21.15.117:0/3031455871 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf8018610 con 0x7fcd0c104a00 2024-04-20T11:34:24.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.392+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c19dda0 con 0x7fcd0c104a00 2024-04-20T11:34:24.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.393+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0c10b460 con 0x7fcd0c104a00 2024-04-20T11:34:24.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.394+0000 7fcd03400700 1 -- 172.21.15.117:0/3031455871 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fccf800f450 con 0x7fcd0c104a00 2024-04-20T11:34:24.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.395+0000 7fcd03400700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccf006ecd0 0x7fccf0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.395+0000 7fcd03400700 1 -- 172.21.15.117:0/3031455871 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fccf808f0f0 con 0x7fcd0c104a00 2024-04-20T11:34:24.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.396+0000 7fcd0a800700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccf006ecd0 0x7fccf0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.396+0000 7fcd0a800700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccf006ecd0 0x7fccf0071190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00b5c0 tx=0x7fccfc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:24.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.400+0000 7fcd03400700 1 -- 172.21.15.117:0/3031455871 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fccf8057e30 con 0x7fcd0c104a00 2024-04-20T11:34:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:24 smithi117 ceph-mon[99036]: pgmap v658: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:24 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4294400868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:34:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:24 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3254723440' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:34:24.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.644+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd0c19a080 con 0x7fccf006ecd0 2024-04-20T11:34:24.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.648+0000 7fcd03400700 1 -- 172.21.15.117:0/3031455871 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcd0c19a080 con 0x7fccf006ecd0 2024-04-20T11:34:24.649 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:34:24.649 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:34:24.650 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.651+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccf006ecd0 msgr2=0x7fccf0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.651+0000 7fcd11400700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccf006ecd0 0x7fccf0071190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00b5c0 tx=0x7fccfc01a040 comp rx=0 tx=0).stop 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.651+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c104a00 msgr2=0x7fcd0c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.651+0000 7fcd11400700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c104a00 0x7fcd0c199300 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fccf800c930 tx=0x7fccf800ccf0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.652+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 shutdown_connections 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.652+0000 7fcd11400700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccf006ecd0 0x7fccf0071190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.652+0000 7fcd11400700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c103110 0x7fcd0c198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.652+0000 7fcd11400700 1 --2- 172.21.15.117:0/3031455871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd0c104a00 0x7fcd0c199300 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.652+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 >> 172.21.15.117:0/3031455871 conn(0x7fcd0c0fe880 msgr2=0x7fcd0c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:24.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.652+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 shutdown_connections 2024-04-20T11:34:24.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.652+0000 7fcd11400700 1 -- 172.21.15.117:0/3031455871 wait complete. 2024-04-20T11:34:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:24 smithi173 ceph-mon[74165]: pgmap v658: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:24 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4294400868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:34:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:24 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3254723440' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:34:24.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.804+0000 7f5b70000700 1 -- 172.21.15.117:0/1313181262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 msgr2=0x7f5b68107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.804+0000 7f5b70000700 1 --2- 172.21.15.117:0/1313181262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 0x7f5b68107900 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009910 tx=0x7f5b5c009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:24.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.805+0000 7f5b70000700 1 -- 172.21.15.117:0/1313181262 shutdown_connections 2024-04-20T11:34:24.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.805+0000 7f5b70000700 1 --2- 172.21.15.117:0/1313181262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68108e10 0x7f5b68109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.805+0000 7f5b70000700 1 --2- 172.21.15.117:0/1313181262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 0x7f5b68107900 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.805+0000 7f5b70000700 1 -- 172.21.15.117:0/1313181262 >> 172.21.15.117:0/1313181262 conn(0x7f5b680768f0 msgr2=0x7f5b68078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:24.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.805+0000 7f5b70000700 1 -- 172.21.15.117:0/1313181262 shutdown_connections 2024-04-20T11:34:24.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.805+0000 7f5b70000700 1 -- 172.21.15.117:0/1313181262 wait complete. 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b70000700 1 Processor -- start 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b70000700 1 -- start start 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b70000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 0x7f5b6819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b70000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68108e10 0x7f5b6819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b70000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6819de90 con 0x7f5b68107520 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b70000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681a1b60 con 0x7f5b68108e10 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b6d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 0x7f5b6819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b6d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 0x7f5b6819d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50092/0 (socket says 172.21.15.117:50092) 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.806+0000 7f5b6d600700 1 -- 172.21.15.117:0/43805218 learned_addr learned my addr 172.21.15.117:0/43805218 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.807+0000 7f5b6d600700 1 -- 172.21.15.117:0/43805218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68108e10 msgr2=0x7f5b6819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:24.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.807+0000 7f5b6d600700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68108e10 0x7f5b6819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:24.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.807+0000 7f5b6d600700 1 -- 172.21.15.117:0/43805218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0095f0 con 0x7f5b68107520 2024-04-20T11:34:24.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.807+0000 7f5b6d600700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 0x7f5b6819d270 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c006010 tx=0x7f5b5c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:24.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.807+0000 7f5b66000700 1 -- 172.21.15.117:0/43805218 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b5c01d070 con 0x7f5b68107520 2024-04-20T11:34:24.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.807+0000 7f5b66000700 1 -- 172.21.15.117:0/43805218 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b5c00f460 con 0x7f5b68107520 2024-04-20T11:34:24.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.807+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b681a1de0 con 0x7f5b68107520 2024-04-20T11:34:24.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.807+0000 7f5b66000700 1 -- 172.21.15.117:0/43805218 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b5c017630 con 0x7f5b68107520 2024-04-20T11:34:24.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.808+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b681a2350 con 0x7f5b68107520 2024-04-20T11:34:24.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.809+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6810f890 con 0x7f5b68107520 2024-04-20T11:34:24.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.809+0000 7f5b66000700 1 -- 172.21.15.117:0/43805218 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5b5c00f5d0 con 0x7f5b68107520 2024-04-20T11:34:24.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.810+0000 7f5b66000700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b5006ecd0 0x7f5b50071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:24.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.810+0000 7f5b66000700 1 -- 172.21.15.117:0/43805218 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5b5c0903c0 con 0x7f5b68107520 2024-04-20T11:34:24.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.811+0000 7f5b6cc00700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b5006ecd0 0x7f5b50071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:24.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.811+0000 7f5b6cc00700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b5006ecd0 0x7f5b50071190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5b6819e7d0 tx=0x7f5b54006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:24.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:24.815+0000 7f5b66000700 1 -- 172.21.15.117:0/43805218 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5b5c0591c0 con 0x7f5b68107520 2024-04-20T11:34:25.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.115+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5b6819e580 con 0x7f5b68107520 2024-04-20T11:34:25.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.116+0000 7f5b66000700 1 -- 172.21.15.117:0/43805218 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5b5c05c880 con 0x7f5b68107520 2024-04-20T11:34:25.116 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:34:25.116 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:34:25.116 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:34:25.116 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:34:25.116 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:34:25.117 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:34:25.117 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:34:25.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.118+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b5006ecd0 msgr2=0x7f5b50071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:25.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.118+0000 7f5b70000700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b5006ecd0 0x7f5b50071190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5b6819e7d0 tx=0x7f5b54006d20 comp rx=0 tx=0).stop 2024-04-20T11:34:25.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.118+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 msgr2=0x7f5b6819d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:25.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.118+0000 7f5b70000700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 0x7f5b6819d270 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c006010 tx=0x7f5b5c00be20 comp rx=0 tx=0).stop 2024-04-20T11:34:25.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.119+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 shutdown_connections 2024-04-20T11:34:25.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.119+0000 7f5b70000700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5b5006ecd0 0x7f5b50071190 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.119+0000 7f5b70000700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68107520 0x7f5b6819d270 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.119+0000 7f5b70000700 1 --2- 172.21.15.117:0/43805218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68108e10 0x7f5b6819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:25.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.119+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 >> 172.21.15.117:0/43805218 conn(0x7f5b680768f0 msgr2=0x7f5b68078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:25.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.119+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 shutdown_connections 2024-04-20T11:34:25.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:25.119+0000 7f5b70000700 1 -- 172.21.15.117:0/43805218 wait complete. 2024-04-20T11:34:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:25 smithi117 ceph-mon[99036]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:25 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/43805218' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:34:25.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:25 smithi173 ceph-mon[74165]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:25 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/43805218' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:34:26.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:26 smithi117 ceph-mon[99036]: pgmap v659: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:26.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:26 smithi173 ceph-mon[74165]: pgmap v659: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:28.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:28 smithi117 ceph-mon[99036]: pgmap v660: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:34:28.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:28 smithi173 ceph-mon[74165]: pgmap v660: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:34:30.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:30 smithi117 ceph-mon[99036]: pgmap v661: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:30.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:30 smithi173 ceph-mon[74165]: pgmap v661: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:32 smithi117 ceph-mon[99036]: pgmap v662: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:32 smithi173 ceph-mon[74165]: pgmap v662: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:34 smithi117 ceph-mon[99036]: pgmap v663: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:34 smithi173 ceph-mon[74165]: pgmap v663: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:36.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:36 smithi117 ceph-mon[99036]: pgmap v664: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:36.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:36 smithi173 ceph-mon[74165]: pgmap v664: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:38.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:38 smithi117 ceph-mon[99036]: pgmap v665: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:38 smithi173 ceph-mon[74165]: pgmap v665: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:40.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:40 smithi117 ceph-mon[99036]: pgmap v666: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:40.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:40 smithi173 ceph-mon[74165]: pgmap v666: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:42.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:42 smithi117 ceph-mon[99036]: pgmap v667: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:42 smithi173 ceph-mon[74165]: pgmap v667: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:43.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:34:43.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:34:44.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:44 smithi117 ceph-mon[99036]: pgmap v668: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:44.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:44 smithi173 ceph-mon[74165]: pgmap v668: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:46.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:46 smithi117 ceph-mon[99036]: pgmap v669: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:46.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:46 smithi173 ceph-mon[74165]: pgmap v669: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:48.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:48 smithi117 ceph-mon[99036]: pgmap v670: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:48.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:48 smithi173 ceph-mon[74165]: pgmap v670: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:50.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:50 smithi117 ceph-mon[99036]: pgmap v671: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:50 smithi173 ceph-mon[74165]: pgmap v671: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:52.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:52 smithi117 ceph-mon[99036]: pgmap v672: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:52.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:52 smithi173 ceph-mon[74165]: pgmap v672: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:55.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:54 smithi117 ceph-mon[99036]: pgmap v673: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:55.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:54 smithi173 ceph-mon[74165]: pgmap v673: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:55.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.289+0000 7f0efc800700 1 -- 172.21.15.117:0/105244772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef4100e10 msgr2=0x7f0ef41011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.289+0000 7f0efc800700 1 --2- 172.21.15.117:0/105244772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef4100e10 0x7f0ef41011f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009910 tx=0x7f0ee8009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:55.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.289+0000 7f0efc800700 1 -- 172.21.15.117:0/105244772 shutdown_connections 2024-04-20T11:34:55.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.289+0000 7f0efc800700 1 --2- 172.21.15.117:0/105244772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef41017c0 0x7f0ef4105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.289+0000 7f0efc800700 1 --2- 172.21.15.117:0/105244772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef4100e10 0x7f0ef41011f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.289+0000 7f0efc800700 1 -- 172.21.15.117:0/105244772 >> 172.21.15.117:0/105244772 conn(0x7f0ef40fc6a0 msgr2=0x7f0ef40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:55.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.289+0000 7f0efc800700 1 -- 172.21.15.117:0/105244772 shutdown_connections 2024-04-20T11:34:55.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.289+0000 7f0efc800700 1 -- 172.21.15.117:0/105244772 wait complete. 2024-04-20T11:34:55.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.290+0000 7f0efc800700 1 Processor -- start 2024-04-20T11:34:55.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.291+0000 7f0efc800700 1 -- start start 2024-04-20T11:34:55.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.291+0000 7f0efc800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef4100e10 0x7f0ef419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.291+0000 7f0efc800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef41017c0 0x7f0ef419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.291+0000 7f0efc800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef419f5c0 con 0x7f0ef41017c0 2024-04-20T11:34:55.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.291+0000 7f0efc800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4198bf0 con 0x7f0ef4100e10 2024-04-20T11:34:55.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.291+0000 7f0ef9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef41017c0 0x7f0ef419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.291+0000 7f0ef9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef41017c0 0x7f0ef419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39488/0 (socket says 172.21.15.117:39488) 2024-04-20T11:34:55.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.291+0000 7f0ef9400700 1 -- 172.21.15.117:0/220849902 learned_addr learned my addr 172.21.15.117:0/220849902 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:55.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0ef9e00700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef4100e10 0x7f0ef419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0ef9400700 1 -- 172.21.15.117:0/220849902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef4100e10 msgr2=0x7f0ef419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0ef9400700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef4100e10 0x7f0ef419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0ef9400700 1 -- 172.21.15.117:0/220849902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee80095f0 con 0x7f0ef41017c0 2024-04-20T11:34:55.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0ef9400700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef41017c0 0x7f0ef419efa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e8f0 tx=0x7f0ee400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0eeea00700 1 -- 172.21.15.117:0/220849902 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ee4009790 con 0x7f0ef41017c0 2024-04-20T11:34:55.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0eeea00700 1 -- 172.21.15.117:0/220849902 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ee4004cf0 con 0x7f0ef41017c0 2024-04-20T11:34:55.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0eeea00700 1 -- 172.21.15.117:0/220849902 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ee4005690 con 0x7f0ef41017c0 2024-04-20T11:34:55.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.292+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef4198ed0 con 0x7f0ef41017c0 2024-04-20T11:34:55.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.293+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef4199420 con 0x7f0ef41017c0 2024-04-20T11:34:55.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.294+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4104f90 con 0x7f0ef41017c0 2024-04-20T11:34:55.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.295+0000 7f0eeea00700 1 -- 172.21.15.117:0/220849902 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0ee4004750 con 0x7f0ef41017c0 2024-04-20T11:34:55.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.297+0000 7f0eeea00700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0edc06ecc0 0x7f0edc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.297+0000 7f0eeea00700 1 -- 172.21.15.117:0/220849902 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0ee4014070 con 0x7f0ef41017c0 2024-04-20T11:34:55.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.297+0000 7f0ef9e00700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0edc06ecc0 0x7f0edc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.298+0000 7f0ef9e00700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0edc06ecc0 0x7f0edc071180 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009910 tx=0x7f0ee8009f70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.302+0000 7f0eeea00700 1 -- 172.21.15.117:0/220849902 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0ee405bb60 con 0x7f0ef41017c0 2024-04-20T11:34:55.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.545+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ef419a060 con 0x7f0edc06ecc0 2024-04-20T11:34:55.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.547+0000 7f0eeea00700 1 -- 172.21.15.117:0/220849902 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0ef419a060 con 0x7f0edc06ecc0 2024-04-20T11:34:55.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0edc06ecc0 msgr2=0x7f0edc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0edc06ecc0 0x7f0edc071180 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009910 tx=0x7f0ee8009f70 comp rx=0 tx=0).stop 2024-04-20T11:34:55.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef41017c0 msgr2=0x7f0ef419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef41017c0 0x7f0ef419efa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e8f0 tx=0x7f0ee400ecb0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 shutdown_connections 2024-04-20T11:34:55.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0edc06ecc0 0x7f0edc071180 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef4100e10 0x7f0ef419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 --2- 172.21.15.117:0/220849902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef41017c0 0x7f0ef419efa0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.550+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 >> 172.21.15.117:0/220849902 conn(0x7f0ef40fc6a0 msgr2=0x7f0ef40fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:55.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.551+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 shutdown_connections 2024-04-20T11:34:55.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.551+0000 7f0efc800700 1 -- 172.21.15.117:0/220849902 wait complete. 2024-04-20T11:34:55.567 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:34:55.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.701+0000 7f1431000700 1 -- 172.21.15.117:0/77294672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c103020 msgr2=0x7f142c103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.701+0000 7f1431000700 1 --2- 172.21.15.117:0/77294672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c103020 0x7f142c103400 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f141c009910 tx=0x7f141c009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:55.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.701+0000 7f1431000700 1 -- 172.21.15.117:0/77294672 shutdown_connections 2024-04-20T11:34:55.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.701+0000 7f1431000700 1 --2- 172.21.15.117:0/77294672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c1039d0 0x7f142c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.701+0000 7f1431000700 1 --2- 172.21.15.117:0/77294672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c103020 0x7f142c103400 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.701+0000 7f1431000700 1 -- 172.21.15.117:0/77294672 >> 172.21.15.117:0/77294672 conn(0x7f142c0fe890 msgr2=0x7f142c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:55.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.702+0000 7f1431000700 1 -- 172.21.15.117:0/77294672 shutdown_connections 2024-04-20T11:34:55.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.702+0000 7f1431000700 1 -- 172.21.15.117:0/77294672 wait complete. 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.703+0000 7f1431000700 1 Processor -- start 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.703+0000 7f1431000700 1 -- start start 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.703+0000 7f1431000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c103020 0x7f142c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.703+0000 7f1431000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c1039d0 0x7f142c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.703+0000 7f1431000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c199a50 con 0x7f142c1039d0 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.703+0000 7f1431000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19d720 con 0x7f142c103020 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.703+0000 7f1429400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c1039d0 0x7f142c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.704+0000 7f1429400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c1039d0 0x7f142c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39494/0 (socket says 172.21.15.117:39494) 2024-04-20T11:34:55.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.704+0000 7f1429400700 1 -- 172.21.15.117:0/14790765 learned_addr learned my addr 172.21.15.117:0/14790765 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:55.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.704+0000 7f1429e00700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c103020 0x7f142c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.704+0000 7f1429400700 1 -- 172.21.15.117:0/14790765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c103020 msgr2=0x7f142c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.704+0000 7f1429400700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c103020 0x7f142c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.704+0000 7f1429400700 1 -- 172.21.15.117:0/14790765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0095f0 con 0x7f142c1039d0 2024-04-20T11:34:55.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.704+0000 7f1429400700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c1039d0 0x7f142c199370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f141800b480 tx=0x7f141800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.704+0000 7f1422a00700 1 -- 172.21.15.117:0/14790765 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14180118c0 con 0x7f142c1039d0 2024-04-20T11:34:55.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.705+0000 7f1422a00700 1 -- 172.21.15.117:0/14790765 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1418010380 con 0x7f142c1039d0 2024-04-20T11:34:55.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.705+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c19da00 con 0x7f142c1039d0 2024-04-20T11:34:55.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.705+0000 7f1422a00700 1 -- 172.21.15.117:0/14790765 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141800f4f0 con 0x7f142c1039d0 2024-04-20T11:34:55.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.705+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c19df50 con 0x7f142c1039d0 2024-04-20T11:34:55.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.706+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c10b3c0 con 0x7f142c1039d0 2024-04-20T11:34:55.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.707+0000 7f1422a00700 1 -- 172.21.15.117:0/14790765 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f14180104f0 con 0x7f142c1039d0 2024-04-20T11:34:55.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.708+0000 7f1422a00700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f141006ecd0 0x7f1410071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:55.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.708+0000 7f1422a00700 1 -- 172.21.15.117:0/14790765 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f141808eb00 con 0x7f142c1039d0 2024-04-20T11:34:55.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.708+0000 7f1429e00700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f141006ecd0 0x7f1410071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:55.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.709+0000 7f1429e00700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f141006ecd0 0x7f1410071190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b5c0 tx=0x7f141c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:55.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.713+0000 7f1422a00700 1 -- 172.21.15.117:0/14790765 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1418057840 con 0x7f142c1039d0 2024-04-20T11:34:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.943+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f142c110060 con 0x7f141006ecd0 2024-04-20T11:34:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.945+0000 7f1422a00700 1 -- 172.21.15.117:0/14790765 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f142c110060 con 0x7f141006ecd0 2024-04-20T11:34:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.947+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f141006ecd0 msgr2=0x7f1410071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.947+0000 7f1431000700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f141006ecd0 0x7f1410071190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b5c0 tx=0x7f141c01a040 comp rx=0 tx=0).stop 2024-04-20T11:34:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c1039d0 msgr2=0x7f142c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c1039d0 0x7f142c199370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f141800b480 tx=0x7f141800b840 comp rx=0 tx=0).stop 2024-04-20T11:34:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 shutdown_connections 2024-04-20T11:34:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f141006ecd0 0x7f1410071190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c103020 0x7f142c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 --2- 172.21.15.117:0/14790765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c1039d0 0x7f142c199370 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 >> 172.21.15.117:0/14790765 conn(0x7f142c0fe890 msgr2=0x7f142c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 shutdown_connections 2024-04-20T11:34:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:55.948+0000 7f1431000700 1 -- 172.21.15.117:0/14790765 wait complete. 2024-04-20T11:34:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.090+0000 7f1270a00700 1 -- 172.21.15.117:0/1475564702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 msgr2=0x7f126c100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.090+0000 7f1270a00700 1 --2- 172.21.15.117:0/1475564702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 0x7f126c100ea0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1258009910 tx=0x7f1258009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.090+0000 7f1270a00700 1 -- 172.21.15.117:0/1475564702 shutdown_connections 2024-04-20T11:34:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.090+0000 7f1270a00700 1 --2- 172.21.15.117:0/1475564702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 0x7f126c100ea0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.090+0000 7f1270a00700 1 --2- 172.21.15.117:0/1475564702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f126c106a20 0x7f126c106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.090+0000 7f1270a00700 1 -- 172.21.15.117:0/1475564702 >> 172.21.15.117:0/1475564702 conn(0x7f126c0fc530 msgr2=0x7f126c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.090+0000 7f1270a00700 1 -- 172.21.15.117:0/1475564702 shutdown_connections 2024-04-20T11:34:56.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.090+0000 7f1270a00700 1 -- 172.21.15.117:0/1475564702 wait complete. 2024-04-20T11:34:56.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.091+0000 7f1270a00700 1 Processor -- start 2024-04-20T11:34:56.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.091+0000 7f1270a00700 1 -- start start 2024-04-20T11:34:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.092+0000 7f1270a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 0x7f126c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.092+0000 7f1270a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f126c106a20 0x7f126c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.092+0000 7f1270a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c19d2e0 con 0x7f126c100a20 2024-04-20T11:34:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.092+0000 7f1270a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c1968c0 con 0x7f126c106a20 2024-04-20T11:34:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.092+0000 7f126b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 0x7f126c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.092+0000 7f126b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 0x7f126c19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39502/0 (socket says 172.21.15.117:39502) 2024-04-20T11:34:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.092+0000 7f126b400700 1 -- 172.21.15.117:0/1161136371 learned_addr learned my addr 172.21.15.117:0/1161136371 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f126b400700 1 -- 172.21.15.117:0/1161136371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f126c106a20 msgr2=0x7f126c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f126aa00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f126c106a20 0x7f126c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f126b400700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f126c106a20 0x7f126c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f126b400700 1 -- 172.21.15.117:0/1161136371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12580095f0 con 0x7f126c100a20 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f126b400700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 0x7f126c19c780 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b4b0 tx=0x7f125c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f1263e00700 1 -- 172.21.15.117:0/1161136371 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125c011800 con 0x7f126c100a20 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f1263e00700 1 -- 172.21.15.117:0/1161136371 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f125c011e20 con 0x7f126c100a20 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f1263e00700 1 -- 172.21.15.117:0/1161136371 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125c00f2e0 con 0x7f126c100a20 2024-04-20T11:34:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c196ba0 con 0x7f126c100a20 2024-04-20T11:34:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.093+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c1970f0 con 0x7f126c100a20 2024-04-20T11:34:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.094+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c109040 con 0x7f126c100a20 2024-04-20T11:34:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.095+0000 7f1263e00700 1 -- 172.21.15.117:0/1161136371 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f125c010380 con 0x7f126c100a20 2024-04-20T11:34:56.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.096+0000 7f1263e00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f125006ecd0 0x7f1250071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.096+0000 7f1263e00700 1 -- 172.21.15.117:0/1161136371 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f125c08e960 con 0x7f126c100a20 2024-04-20T11:34:56.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.097+0000 7f126aa00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f125006ecd0 0x7f1250071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.097+0000 7f126aa00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f125006ecd0 0x7f1250071190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1258006010 tx=0x7f125800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:56.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.101+0000 7f1263e00700 1 -- 172.21.15.117:0/1161136371 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f125c05b3a0 con 0x7f126c100a20 2024-04-20T11:34:56.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.330+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f126c10dca0 con 0x7f125006ecd0 2024-04-20T11:34:56.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.362+0000 7f1263e00700 1 -- 172.21.15.117:0/1161136371 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f126c10dca0 con 0x7f125006ecd0 2024-04-20T11:34:56.364 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:34:56.364 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (23m) 15m ago 37m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:34:56.364 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (37m) 15m ago 37m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:34:56.364 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (36m) 14m ago 36m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:34:56.364 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (19m) 15m ago 37m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:34:56.364 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (19m) 14m ago 36m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:34:56.364 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (22m) 15m ago 37m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (15m) 15m ago 30m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (15m) 15m ago 30m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (30m) 14m ago 30m 19.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 c77c43b792d7 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (14m) 14m ago 30m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (26m) 15m ago 39m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (25m) 14m ago 36m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (21m) 15m ago 39m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (20m) 14m ago 36m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (24m) 15m ago 37m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (24m) 14m ago 36m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (19m) 15m ago 35m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (18m) 15m ago 34m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (18m) 15m ago 33m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (17m) 14m ago 33m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:34:56.365 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (16m) 14m ago 32m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:34:56.366 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (16m) 14m ago 32m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:34:56.366 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (24m) 15m ago 36m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:34:56.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f125006ecd0 msgr2=0x7f1250071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f125006ecd0 0x7f1250071190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1258006010 tx=0x7f125800b540 comp rx=0 tx=0).stop 2024-04-20T11:34:56.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 msgr2=0x7f126c19c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 0x7f126c19c780 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b4b0 tx=0x7f125c00b870 comp rx=0 tx=0).stop 2024-04-20T11:34:56.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 shutdown_connections 2024-04-20T11:34:56.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f125006ecd0 0x7f1250071190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f126c100a20 0x7f126c19c780 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 --2- 172.21.15.117:0/1161136371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f126c106a20 0x7f126c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 >> 172.21.15.117:0/1161136371 conn(0x7f126c0fc530 msgr2=0x7f126c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:56.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 shutdown_connections 2024-04-20T11:34:56.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.365+0000 7f1270a00700 1 -- 172.21.15.117:0/1161136371 wait complete. 2024-04-20T11:34:56.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.514+0000 7fee17c00700 1 -- 172.21.15.117:0/3815114592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 msgr2=0x7fee10103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.514+0000 7fee17c00700 1 --2- 172.21.15.117:0/3815114592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 0x7fee10103400 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fee04009910 tx=0x7fee04009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:56.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.514+0000 7fee17c00700 1 -- 172.21.15.117:0/3815114592 shutdown_connections 2024-04-20T11:34:56.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.514+0000 7fee17c00700 1 --2- 172.21.15.117:0/3815114592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee101039d0 0x7fee10107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.514+0000 7fee17c00700 1 --2- 172.21.15.117:0/3815114592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 0x7fee10103400 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.514+0000 7fee17c00700 1 -- 172.21.15.117:0/3815114592 >> 172.21.15.117:0/3815114592 conn(0x7fee100fe8b0 msgr2=0x7fee10100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:56.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.514+0000 7fee17c00700 1 -- 172.21.15.117:0/3815114592 shutdown_connections 2024-04-20T11:34:56.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.514+0000 7fee17c00700 1 -- 172.21.15.117:0/3815114592 wait complete. 2024-04-20T11:34:56.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.515+0000 7fee17c00700 1 Processor -- start 2024-04-20T11:34:56.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.515+0000 7fee17c00700 1 -- start start 2024-04-20T11:34:56.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.515+0000 7fee17c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 0x7fee100753f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.515+0000 7fee17c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee101039d0 0x7fee10075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.515+0000 7fee17c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee100794c0 con 0x7fee10103020 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.515+0000 7fee17c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10075e70 con 0x7fee101039d0 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.516+0000 7fee15200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 0x7fee100753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.516+0000 7fee15200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 0x7fee100753f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39518/0 (socket says 172.21.15.117:39518) 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.516+0000 7fee15200700 1 -- 172.21.15.117:0/402597761 learned_addr learned my addr 172.21.15.117:0/402597761 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.516+0000 7fee15200700 1 -- 172.21.15.117:0/402597761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee101039d0 msgr2=0x7fee10075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.516+0000 7fee15200700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee101039d0 0x7fee10075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.516+0000 7fee15200700 1 -- 172.21.15.117:0/402597761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040095f0 con 0x7fee10103020 2024-04-20T11:34:56.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.516+0000 7fee15200700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 0x7fee100753f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fee04006010 tx=0x7fee0400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:56.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.517+0000 7fee0d600700 1 -- 172.21.15.117:0/402597761 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee0401d070 con 0x7fee10103020 2024-04-20T11:34:56.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.517+0000 7fee0d600700 1 -- 172.21.15.117:0/402597761 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee0400f460 con 0x7fee10103020 2024-04-20T11:34:56.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.517+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee100760f0 con 0x7fee10103020 2024-04-20T11:34:56.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.517+0000 7fee0d600700 1 -- 172.21.15.117:0/402597761 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee04017630 con 0x7fee10103020 2024-04-20T11:34:56.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.517+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee101a7570 con 0x7fee10103020 2024-04-20T11:34:56.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.518+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1010b3c0 con 0x7fee10103020 2024-04-20T11:34:56.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.519+0000 7fee0d600700 1 -- 172.21.15.117:0/402597761 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fee0400f5d0 con 0x7fee10103020 2024-04-20T11:34:56.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.520+0000 7fee0d600700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fedf806ecc0 0x7fedf8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.520+0000 7fee0d600700 1 -- 172.21.15.117:0/402597761 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fee040903c0 con 0x7fee10103020 2024-04-20T11:34:56.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.520+0000 7fee0fe00700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fedf806ecc0 0x7fedf8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.521+0000 7fee0fe00700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fedf806ecc0 0x7fedf8071180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee10076b20 tx=0x7fedfc006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:56.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.525+0000 7fee0d600700 1 -- 172.21.15.117:0/402597761 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fee040591c0 con 0x7fee10103020 2024-04-20T11:34:56.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.843+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fee10110060 con 0x7fee10103020 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.844+0000 7fee0d600700 1 -- 172.21.15.117:0/402597761 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fee0405c880 con 0x7fee10103020 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:34:56.845 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 3 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1, 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 13 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:34:56.846 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:34:56.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fedf806ecc0 msgr2=0x7fedf8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fedf806ecc0 0x7fedf8071180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee10076b20 tx=0x7fedfc006d20 comp rx=0 tx=0).stop 2024-04-20T11:34:56.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 msgr2=0x7fee100753f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 0x7fee100753f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fee04006010 tx=0x7fee0400be20 comp rx=0 tx=0).stop 2024-04-20T11:34:56.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 shutdown_connections 2024-04-20T11:34:56.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fedf806ecc0 0x7fedf8071180 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee10103020 0x7fee100753f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 --2- 172.21.15.117:0/402597761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee101039d0 0x7fee10075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 >> 172.21.15.117:0/402597761 conn(0x7fee100fe8b0 msgr2=0x7fee100ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:56.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 shutdown_connections 2024-04-20T11:34:56.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.848+0000 7fee17c00700 1 -- 172.21.15.117:0/402597761 wait complete. 2024-04-20T11:34:56.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.996+0000 7fe827800700 1 -- 172.21.15.117:0/3309813798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 msgr2=0x7fe8200fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:56.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.996+0000 7fe827800700 1 --2- 172.21.15.117:0/3309813798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 0x7fe8200fff00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009910 tx=0x7fe80c009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:56.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.996+0000 7fe827800700 1 -- 172.21.15.117:0/3309813798 shutdown_connections 2024-04-20T11:34:56.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.996+0000 7fe827800700 1 --2- 172.21.15.117:0/3309813798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 0x7fe8200fff00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.996+0000 7fe827800700 1 --2- 172.21.15.117:0/3309813798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe820105a80 0x7fe820105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:56.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.996+0000 7fe827800700 1 -- 172.21.15.117:0/3309813798 >> 172.21.15.117:0/3309813798 conn(0x7fe820076750 msgr2=0x7fe8200fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:56.997 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:56 smithi117 ceph-mon[99036]: pgmap v674: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:56.997 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:56 smithi117 ceph-mon[99036]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:56.997 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:56 smithi117 ceph-mon[99036]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:56.997 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:56 smithi117 ceph-mon[99036]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:56.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.996+0000 7fe827800700 1 -- 172.21.15.117:0/3309813798 shutdown_connections 2024-04-20T11:34:56.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.996+0000 7fe827800700 1 -- 172.21.15.117:0/3309813798 wait complete. 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.997+0000 7fe827800700 1 Processor -- start 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.997+0000 7fe827800700 1 -- start start 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.998+0000 7fe827800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 0x7fe82019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.998+0000 7fe827800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe820105a80 0x7fe82019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.998+0000 7fe827800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82019d3e0 con 0x7fe8200ffa80 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.998+0000 7fe827800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8201969c0 con 0x7fe820105a80 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.998+0000 7fe824e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 0x7fe82019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.998+0000 7fe824e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 0x7fe82019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39526/0 (socket says 172.21.15.117:39526) 2024-04-20T11:34:56.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.998+0000 7fe824e00700 1 -- 172.21.15.117:0/1524122171 learned_addr learned my addr 172.21.15.117:0/1524122171 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:57.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe81fe00700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe820105a80 0x7fe82019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe824e00700 1 -- 172.21.15.117:0/1524122171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe820105a80 msgr2=0x7fe82019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe824e00700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe820105a80 0x7fe82019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe824e00700 1 -- 172.21.15.117:0/1524122171 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0095f0 con 0x7fe8200ffa80 2024-04-20T11:34:57.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe824e00700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 0x7fe82019c880 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe81400d8d0 tx=0x7fe81400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe81d600700 1 -- 172.21.15.117:0/1524122171 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe814009940 con 0x7fe8200ffa80 2024-04-20T11:34:57.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe81d600700 1 -- 172.21.15.117:0/1524122171 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe814010460 con 0x7fe8200ffa80 2024-04-20T11:34:57.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe81d600700 1 -- 172.21.15.117:0/1524122171 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe81400f5f0 con 0x7fe8200ffa80 2024-04-20T11:34:57.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:56.999+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe820196ca0 con 0x7fe8200ffa80 2024-04-20T11:34:57.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.000+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8201971f0 con 0x7fe8200ffa80 2024-04-20T11:34:57.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.001+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8201090b0 con 0x7fe8200ffa80 2024-04-20T11:34:57.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.002+0000 7fe81d600700 1 -- 172.21.15.117:0/1524122171 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe8140105d0 con 0x7fe8200ffa80 2024-04-20T11:34:57.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.003+0000 7fe81d600700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe80806ecc0 0x7fe808071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.003+0000 7fe81d600700 1 -- 172.21.15.117:0/1524122171 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe81408ede0 con 0x7fe8200ffa80 2024-04-20T11:34:57.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.003+0000 7fe81fe00700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe80806ecc0 0x7fe808071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.003+0000 7fe81fe00700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe80806ecc0 0x7fe808071180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe80c006010 tx=0x7fe80c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.007+0000 7fe81d600700 1 -- 172.21.15.117:0/1524122171 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe81405b6d0 con 0x7fe8200ffa80 2024-04-20T11:34:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:56 smithi173 ceph-mon[74165]: pgmap v674: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:56 smithi173 ceph-mon[74165]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:56 smithi173 ceph-mon[74165]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:56 smithi173 ceph-mon[74165]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:57.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.271+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe820197e80 con 0x7fe8200ffa80 2024-04-20T11:34:57.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.272+0000 7fe81d600700 1 -- 172.21.15.117:0/1524122171 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1888 (secure 0 0 0) 0x7fe814016770 con 0x7fe8200ffa80 2024-04-20T11:34:57.274 INFO:teuthology.orchestra.run.smithi117.stdout:e20 2024-04-20T11:34:57.274 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:34:57.274 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:34:57.274 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:34:57.275 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:34:57.276 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.173:6826/46112047,v1:172.21.15.173:6827/46112047] compat {c=[1],r=[1],i=[7ff]}] 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:34:57.277 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe80806ecc0 msgr2=0x7fe808071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe80806ecc0 0x7fe808071180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe80c006010 tx=0x7fe80c00b540 comp rx=0 tx=0).stop 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 msgr2=0x7fe82019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 0x7fe82019c880 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe81400d8d0 tx=0x7fe81400dc90 comp rx=0 tx=0).stop 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 shutdown_connections 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe80806ecc0 0x7fe808071180 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8200ffa80 0x7fe82019c880 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 --2- 172.21.15.117:0/1524122171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe820105a80 0x7fe82019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 >> 172.21.15.117:0/1524122171 conn(0x7fe820076750 msgr2=0x7fe820068b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 shutdown_connections 2024-04-20T11:34:57.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.275+0000 7fe827800700 1 -- 172.21.15.117:0/1524122171 wait complete. 2024-04-20T11:34:57.279 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:34:57.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.426+0000 7f606ea00700 1 -- 172.21.15.117:0/706093972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 msgr2=0x7f606810ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.426+0000 7f606ea00700 1 --2- 172.21.15.117:0/706093972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 0x7f606810ab50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6054009910 tx=0x7f6054009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:57.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.426+0000 7f606ea00700 1 -- 172.21.15.117:0/706093972 shutdown_connections 2024-04-20T11:34:57.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.426+0000 7f606ea00700 1 --2- 172.21.15.117:0/706093972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 0x7f606810ab50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.426+0000 7f606ea00700 1 --2- 172.21.15.117:0/706093972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6068101d40 0x7f6068102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.426+0000 7f606ea00700 1 -- 172.21.15.117:0/706093972 >> 172.21.15.117:0/706093972 conn(0x7f6068076750 msgr2=0x7f60680fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.426+0000 7f606ea00700 1 -- 172.21.15.117:0/706093972 shutdown_connections 2024-04-20T11:34:57.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.426+0000 7f606ea00700 1 -- 172.21.15.117:0/706093972 wait complete. 2024-04-20T11:34:57.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.428+0000 7f606ea00700 1 Processor -- start 2024-04-20T11:34:57.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.428+0000 7f606ea00700 1 -- start start 2024-04-20T11:34:57.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.428+0000 7f606ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6068101d40 0x7f6068198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.428+0000 7f606ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 0x7f6068199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.428+0000 7f606ea00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6068199a30 con 0x7f6068102660 2024-04-20T11:34:57.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.428+0000 7f606ea00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606819d700 con 0x7f6068101d40 2024-04-20T11:34:57.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.429+0000 7f6067400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 0x7f6068199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.429+0000 7f6067400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 0x7f6068199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39552/0 (socket says 172.21.15.117:39552) 2024-04-20T11:34:57.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.429+0000 7f6067400700 1 -- 172.21.15.117:0/2473878356 learned_addr learned my addr 172.21.15.117:0/2473878356 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:57.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.429+0000 7f6067e00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6068101d40 0x7f6068198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.429+0000 7f6067400700 1 -- 172.21.15.117:0/2473878356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6068101d40 msgr2=0x7f6068198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.429+0000 7f6067400700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6068101d40 0x7f6068198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.429+0000 7f6067400700 1 -- 172.21.15.117:0/2473878356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60540095f0 con 0x7f6068102660 2024-04-20T11:34:57.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.429+0000 7f6067400700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 0x7f6068199350 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f60540092e0 tx=0x7f605400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.430+0000 7f6064c00700 1 -- 172.21.15.117:0/2473878356 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605401d070 con 0x7f6068102660 2024-04-20T11:34:57.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.430+0000 7f6064c00700 1 -- 172.21.15.117:0/2473878356 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6054003d50 con 0x7f6068102660 2024-04-20T11:34:57.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.430+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606819d980 con 0x7f6068102660 2024-04-20T11:34:57.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.430+0000 7f6064c00700 1 -- 172.21.15.117:0/2473878356 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6054017400 con 0x7f6068102660 2024-04-20T11:34:57.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.430+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606819de70 con 0x7f6068102660 2024-04-20T11:34:57.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.431+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606810a6b0 con 0x7f6068102660 2024-04-20T11:34:57.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.432+0000 7f6064c00700 1 -- 172.21.15.117:0/2473878356 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6054003890 con 0x7f6068102660 2024-04-20T11:34:57.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.433+0000 7f6064c00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f604c06ecc0 0x7f604c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.433+0000 7f6064c00700 1 -- 172.21.15.117:0/2473878356 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6054090270 con 0x7f6068102660 2024-04-20T11:34:57.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.433+0000 7f6067e00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f604c06ecc0 0x7f604c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.434+0000 7f6067e00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f604c06ecc0 0x7f604c071180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6058006fd0 tx=0x7f6058008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.437+0000 7f6064c00700 1 -- 172.21.15.117:0/2473878356 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6054058fb0 con 0x7f6068102660 2024-04-20T11:34:57.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.668+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f606810de20 con 0x7f604c06ecc0 2024-04-20T11:34:57.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.671+0000 7f6064c00700 1 -- 172.21.15.117:0/2473878356 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f606810de20 con 0x7f604c06ecc0 2024-04-20T11:34:57.671 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:34:57.671 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:34:57.671 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:34:57.671 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:34:57.671 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:34:57.672 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:34:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.673+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f604c06ecc0 msgr2=0x7f604c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.673+0000 7f606ea00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f604c06ecc0 0x7f604c071180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6058006fd0 tx=0x7f6058008040 comp rx=0 tx=0).stop 2024-04-20T11:34:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.673+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 msgr2=0x7f6068199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.674+0000 7f606ea00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 0x7f6068199350 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f60540092e0 tx=0x7f605400be20 comp rx=0 tx=0).stop 2024-04-20T11:34:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.674+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 shutdown_connections 2024-04-20T11:34:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.674+0000 7f606ea00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f604c06ecc0 0x7f604c071180 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.674+0000 7f606ea00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6068101d40 0x7f6068198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.674+0000 7f606ea00700 1 --2- 172.21.15.117:0/2473878356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6068102660 0x7f6068199350 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.674+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 >> 172.21.15.117:0/2473878356 conn(0x7f6068076750 msgr2=0x7f6068101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.674+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 shutdown_connections 2024-04-20T11:34:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.674+0000 7f606ea00700 1 -- 172.21.15.117:0/2473878356 wait complete. 2024-04-20T11:34:57.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.828+0000 7fe5eae00700 1 -- 172.21.15.117:0/1976073598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 msgr2=0x7fe5e4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.828+0000 7fe5eae00700 1 --2- 172.21.15.117:0/1976073598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 0x7fe5e4109290 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009910 tx=0x7fe5d0009c20 comp rx=0 tx=0).stop 2024-04-20T11:34:57.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.828+0000 7fe5eae00700 1 -- 172.21.15.117:0/1976073598 shutdown_connections 2024-04-20T11:34:57.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.828+0000 7fe5eae00700 1 --2- 172.21.15.117:0/1976073598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 0x7fe5e4109290 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.828+0000 7fe5eae00700 1 --2- 172.21.15.117:0/1976073598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e4107520 0x7fe5e4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.828+0000 7fe5eae00700 1 -- 172.21.15.117:0/1976073598 >> 172.21.15.117:0/1976073598 conn(0x7fe5e40768f0 msgr2=0x7fe5e4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:57.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.828+0000 7fe5eae00700 1 -- 172.21.15.117:0/1976073598 shutdown_connections 2024-04-20T11:34:57.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.828+0000 7fe5eae00700 1 -- 172.21.15.117:0/1976073598 wait complete. 2024-04-20T11:34:57.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.829+0000 7fe5eae00700 1 Processor -- start 2024-04-20T11:34:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.829+0000 7fe5eae00700 1 -- start start 2024-04-20T11:34:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5eae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e4107520 0x7fe5e419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5eae00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 0x7fe5e419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5eae00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e419ddf0 con 0x7fe5e4108e10 2024-04-20T11:34:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5eae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41a1a70 con 0x7fe5e4107520 2024-04-20T11:34:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5e3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 0x7fe5e419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5e3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 0x7fe5e419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39560/0 (socket says 172.21.15.117:39560) 2024-04-20T11:34:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5e3400700 1 -- 172.21.15.117:0/352738448 learned_addr learned my addr 172.21.15.117:0/352738448 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5e3400700 1 -- 172.21.15.117:0/352738448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e4107520 msgr2=0x7fe5e419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.830+0000 7fe5e3e00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e4107520 0x7fe5e419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.831+0000 7fe5e3400700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e4107520 0x7fe5e419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.831+0000 7fe5e3400700 1 -- 172.21.15.117:0/352738448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d00095f0 con 0x7fe5e4108e10 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.831+0000 7fe5e3400700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 0x7fe5e419d710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0000c00 tx=0x7fe5d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.831+0000 7fe5e0c00700 1 -- 172.21.15.117:0/352738448 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5d001d070 con 0x7fe5e4108e10 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.831+0000 7fe5e0c00700 1 -- 172.21.15.117:0/352738448 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5d00056e0 con 0x7fe5e4108e10 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.831+0000 7fe5e0c00700 1 -- 172.21.15.117:0/352738448 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5d000fe80 con 0x7fe5e4108e10 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.831+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e41a1cf0 con 0x7fe5e4108e10 2024-04-20T11:34:57.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.831+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e41a2210 con 0x7fe5e4108e10 2024-04-20T11:34:57.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.833+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e410f870 con 0x7fe5e4108e10 2024-04-20T11:34:57.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.834+0000 7fe5e0c00700 1 -- 172.21.15.117:0/352738448 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe5d000f6f0 con 0x7fe5e4108e10 2024-04-20T11:34:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.835+0000 7fe5e0c00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5c806ecd0 0x7fe5c8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:34:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.835+0000 7fe5e0c00700 1 -- 172.21.15.117:0/352738448 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe5d00902e0 con 0x7fe5e4108e10 2024-04-20T11:34:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.835+0000 7fe5e3e00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5c806ecd0 0x7fe5c8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:34:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.836+0000 7fe5e3e00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5c806ecd0 0x7fe5c8071190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005950 tx=0x7fe5d40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:34:57.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:57.840+0000 7fe5e0c00700 1 -- 172.21.15.117:0/352738448 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe5d0059020 con 0x7fe5e4108e10 2024-04-20T11:34:58.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:57 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/402597761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:34:58.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:57 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1524122171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:34:58.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.150+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe5e419e490 con 0x7fe5e4108e10 2024-04-20T11:34:58.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.151+0000 7fe5e0c00700 1 -- 172.21.15.117:0/352738448 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe5d005c6e0 con 0x7fe5e4108e10 2024-04-20T11:34:58.152 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:34:58.152 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:34:58.152 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:34:58.152 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:34:58.152 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:34:58.152 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:34:58.153 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:34:58.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5c806ecd0 msgr2=0x7fe5c8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5c806ecd0 0x7fe5c8071190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005950 tx=0x7fe5d40058e0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 msgr2=0x7fe5e419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:34:58.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 0x7fe5e419d710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0000c00 tx=0x7fe5d000be20 comp rx=0 tx=0).stop 2024-04-20T11:34:58.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 shutdown_connections 2024-04-20T11:34:58.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5c806ecd0 0x7fe5c8071190 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e4107520 0x7fe5e419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 --2- 172.21.15.117:0/352738448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e4108e10 0x7fe5e419d710 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:34:58.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 >> 172.21.15.117:0/352738448 conn(0x7fe5e40768f0 msgr2=0x7fe5e4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:34:58.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 shutdown_connections 2024-04-20T11:34:58.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:34:58.154+0000 7fe5eae00700 1 -- 172.21.15.117:0/352738448 wait complete. 2024-04-20T11:34:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:57 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/402597761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:34:58.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:57 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1524122171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:34:59.075 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:58 smithi117 ceph-mon[99036]: pgmap v675: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:59.075 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:58 smithi117 ceph-mon[99036]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:59.076 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:34:59.076 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:34:58 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/352738448' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:34:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:58 smithi173 ceph-mon[74165]: pgmap v675: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:34:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:58 smithi173 ceph-mon[74165]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:34:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:34:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:34:58 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/352738448' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:35:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:00 smithi173 ceph-mon[74165]: pgmap v676: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:01.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:00 smithi117 ceph-mon[99036]: pgmap v676: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:03.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:02 smithi117 ceph-mon[99036]: pgmap v677: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:03.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:02 smithi173 ceph-mon[74165]: pgmap v677: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:05.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:04 smithi117 ceph-mon[99036]: pgmap v678: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:05.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:04 smithi173 ceph-mon[74165]: pgmap v678: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:07.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:06 smithi117 ceph-mon[99036]: pgmap v679: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:06 smithi173 ceph-mon[74165]: pgmap v679: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:09.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:08 smithi117 ceph-mon[99036]: pgmap v680: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:08 smithi173 ceph-mon[74165]: pgmap v680: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:11.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:10 smithi117 ceph-mon[99036]: pgmap v681: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:10 smithi173 ceph-mon[74165]: pgmap v681: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:13.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:12 smithi117 ceph-mon[99036]: pgmap v682: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:12 smithi173 ceph-mon[74165]: pgmap v682: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:14.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:35:14.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:35:14.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:14 smithi117 ceph-mon[99036]: pgmap v683: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:15.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:14 smithi173 ceph-mon[74165]: pgmap v683: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:35:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:35:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: pgmap v684: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:35:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:16.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:15 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: pgmap v684: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:16.141 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:15 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:17.065 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:16 smithi173 ceph-mon[74165]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-20T11:35:17.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:16 smithi117 ceph-mon[99036]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-20T11:35:18.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:17 smithi117 ceph-mon[99036]: pgmap v685: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:18.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:17 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:18.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:17 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pfrgat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:35:18.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:17 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:35:18.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:17 smithi173 ceph-mon[74165]: pgmap v685: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:18.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:17 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:18.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:17 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi173.pfrgat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:35:18.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:17 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:35:19.066 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:18 smithi173 ceph-mon[74165]: Upgrade: Updating mds.cephfs.smithi173.pfrgat 2024-04-20T11:35:19.066 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:18 smithi173 ceph-mon[74165]: Deploying daemon mds.cephfs.smithi173.pfrgat on smithi173 2024-04-20T11:35:19.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:18 smithi117 ceph-mon[99036]: Upgrade: Updating mds.cephfs.smithi173.pfrgat 2024-04-20T11:35:19.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:18 smithi117 ceph-mon[99036]: Deploying daemon mds.cephfs.smithi173.pfrgat on smithi173 2024-04-20T11:35:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:19 smithi173 ceph-mon[74165]: pgmap v686: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:20.041 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:19 smithi117 ceph-mon[99036]: pgmap v686: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:22.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:21 smithi117 ceph-mon[99036]: osdmap e86: 6 total, 6 up, 6 in 2024-04-20T11:35:22.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:21 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-20T11:35:22.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:21 smithi117 ceph-mon[99036]: pgmap v688: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:22.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:21 smithi173 ceph-mon[74165]: osdmap e86: 6 total, 6 up, 6 in 2024-04-20T11:35:22.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:21 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-20T11:35:22.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:21 smithi173 ceph-mon[74165]: pgmap v688: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:24.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:23 smithi173 ceph-mon[74165]: pgmap v689: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:24.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:23 smithi117 ceph-mon[99036]: pgmap v689: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:26.160 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:25 smithi173 ceph-mon[74165]: pgmap v690: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:26.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:25 smithi117 ceph-mon[99036]: pgmap v690: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] up:boot 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:26 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] up:boot 2024-04-20T11:35:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:35:27.479 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:35:27.479 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.479 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.479 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:35:27.479 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.479 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.479 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:27.479 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:26 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:28.278 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:27 smithi173 ceph-mon[74165]: pgmap v691: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:28.282 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:27 smithi117 ceph-mon[99036]: pgmap v691: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.317+0000 7fe572800700 1 -- 172.21.15.117:0/1571285233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 msgr2=0x7fe56c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.317+0000 7fe572800700 1 --2- 172.21.15.117:0/1571285233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 0x7fe56c1011f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009910 tx=0x7fe55c009c20 comp rx=0 tx=0).stop 2024-04-20T11:35:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.317+0000 7fe572800700 1 -- 172.21.15.117:0/1571285233 shutdown_connections 2024-04-20T11:35:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.317+0000 7fe572800700 1 --2- 172.21.15.117:0/1571285233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe56c1017c0 0x7fe56c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.317+0000 7fe572800700 1 --2- 172.21.15.117:0/1571285233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 0x7fe56c1011f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.317+0000 7fe572800700 1 -- 172.21.15.117:0/1571285233 >> 172.21.15.117:0/1571285233 conn(0x7fe56c0fc6c0 msgr2=0x7fe56c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.317+0000 7fe572800700 1 -- 172.21.15.117:0/1571285233 shutdown_connections 2024-04-20T11:35:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.317+0000 7fe572800700 1 -- 172.21.15.117:0/1571285233 wait complete. 2024-04-20T11:35:28.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.318+0000 7fe572800700 1 Processor -- start 2024-04-20T11:35:28.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.318+0000 7fe572800700 1 -- start start 2024-04-20T11:35:28.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.319+0000 7fe572800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 0x7fe56c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.319+0000 7fe572800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe56c1017c0 0x7fe56c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.319+0000 7fe572800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c1999c0 con 0x7fe56c100e10 2024-04-20T11:35:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.319+0000 7fe572800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c19d5d0 con 0x7fe56c1017c0 2024-04-20T11:35:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.319+0000 7fe56b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 0x7fe56c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.319+0000 7fe56b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 0x7fe56c198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56156/0 (socket says 172.21.15.117:56156) 2024-04-20T11:35:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.319+0000 7fe56b400700 1 -- 172.21.15.117:0/1509530244 learned_addr learned my addr 172.21.15.117:0/1509530244 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:35:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.319+0000 7fe56aa00700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe56c1017c0 0x7fe56c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.320+0000 7fe56b400700 1 -- 172.21.15.117:0/1509530244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe56c1017c0 msgr2=0x7fe56c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.320+0000 7fe56b400700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe56c1017c0 0x7fe56c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.320+0000 7fe56b400700 1 -- 172.21.15.117:0/1509530244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c0095f0 con 0x7fe56c100e10 2024-04-20T11:35:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.320+0000 7fe56b400700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 0x7fe56c198da0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0092e0 tx=0x7fe55c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.320+0000 7fe563e00700 1 -- 172.21.15.117:0/1509530244 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe55c01d070 con 0x7fe56c100e10 2024-04-20T11:35:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.321+0000 7fe563e00700 1 -- 172.21.15.117:0/1509530244 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe55c0056e0 con 0x7fe56c100e10 2024-04-20T11:35:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.321+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56c19d850 con 0x7fe56c100e10 2024-04-20T11:35:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.321+0000 7fe563e00700 1 -- 172.21.15.117:0/1509530244 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe55c00fe10 con 0x7fe56c100e10 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.321+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56c19dd40 con 0x7fe56c100e10 2024-04-20T11:35:28.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.322+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56c1022f0 con 0x7fe56c100e10 2024-04-20T11:35:28.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.323+0000 7fe563e00700 1 -- 172.21.15.117:0/1509530244 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe55c00f680 con 0x7fe56c100e10 2024-04-20T11:35:28.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.324+0000 7fe563e00700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe55006ecc0 0x7fe550071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:28.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.324+0000 7fe563e00700 1 -- 172.21.15.117:0/1509530244 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe55c0901e0 con 0x7fe56c100e10 2024-04-20T11:35:28.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.325+0000 7fe56aa00700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe55006ecc0 0x7fe550071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:28.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.326+0000 7fe56aa00700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe55006ecc0 0x7fe550071180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe56c073230 tx=0x7fe5580073d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:28.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.330+0000 7fe563e00700 1 -- 172.21.15.117:0/1509530244 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe55c058fc0 con 0x7fe56c100e10 2024-04-20T11:35:28.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.579+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe56c10dde0 con 0x7fe55006ecc0 2024-04-20T11:35:28.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.582+0000 7fe563e00700 1 -- 172.21.15.117:0/1509530244 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe56c10dde0 con 0x7fe55006ecc0 2024-04-20T11:35:28.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe55006ecc0 msgr2=0x7fe550071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:28.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe55006ecc0 0x7fe550071180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe56c073230 tx=0x7fe5580073d0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 msgr2=0x7fe56c198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:28.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 0x7fe56c198da0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0092e0 tx=0x7fe55c00bd70 comp rx=0 tx=0).stop 2024-04-20T11:35:28.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 shutdown_connections 2024-04-20T11:35:28.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe55006ecc0 0x7fe550071180 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c100e10 0x7fe56c198da0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 --2- 172.21.15.117:0/1509530244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe56c1017c0 0x7fe56c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 >> 172.21.15.117:0/1509530244 conn(0x7fe56c0fc6c0 msgr2=0x7fe56c074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:28.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 shutdown_connections 2024-04-20T11:35:28.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.585+0000 7fe572800700 1 -- 172.21.15.117:0/1509530244 wait complete. 2024-04-20T11:35:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:35:28.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.743+0000 7f584fa00700 1 -- 172.21.15.117:0/105439701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 msgr2=0x7f5848103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.743+0000 7f584fa00700 1 --2- 172.21.15.117:0/105439701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 0x7f5848103420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f583c009910 tx=0x7f583c009c20 comp rx=0 tx=0).stop 2024-04-20T11:35:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.743+0000 7f584fa00700 1 -- 172.21.15.117:0/105439701 shutdown_connections 2024-04-20T11:35:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.743+0000 7f584fa00700 1 --2- 172.21.15.117:0/105439701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58481039f0 0x7f5848107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.743+0000 7f584fa00700 1 --2- 172.21.15.117:0/105439701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 0x7f5848103420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.743+0000 7f584fa00700 1 -- 172.21.15.117:0/105439701 >> 172.21.15.117:0/105439701 conn(0x7f58480fe8b0 msgr2=0x7f5848100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.743+0000 7f584fa00700 1 -- 172.21.15.117:0/105439701 shutdown_connections 2024-04-20T11:35:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.743+0000 7f584fa00700 1 -- 172.21.15.117:0/105439701 wait complete. 2024-04-20T11:35:28.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.744+0000 7f584fa00700 1 Processor -- start 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.744+0000 7f584fa00700 1 -- start start 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.744+0000 7f584fa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 0x7f5848198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.744+0000 7f584fa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58481039f0 0x7f58481992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.744+0000 7f584fa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848199930 con 0x7f5848103040 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.744+0000 7f584fa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819c550 con 0x7f58481039f0 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.745+0000 7f584d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 0x7f5848198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.745+0000 7f584d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 0x7f5848198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56172/0 (socket says 172.21.15.117:56172) 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.745+0000 7f584d000700 1 -- 172.21.15.117:0/3677825667 learned_addr learned my addr 172.21.15.117:0/3677825667 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.745+0000 7f5847e00700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58481039f0 0x7f58481992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.745+0000 7f584d000700 1 -- 172.21.15.117:0/3677825667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58481039f0 msgr2=0x7f58481992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.745+0000 7f584d000700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58481039f0 0x7f58481992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:28.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.745+0000 7f584d000700 1 -- 172.21.15.117:0/3677825667 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c0095f0 con 0x7f5848103040 2024-04-20T11:35:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.746+0000 7f584d000700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 0x7f5848198d60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f583c006010 tx=0x7f583c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.746+0000 7f5845600700 1 -- 172.21.15.117:0/3677825667 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f583c01d070 con 0x7f5848103040 2024-04-20T11:35:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.746+0000 7f5845600700 1 -- 172.21.15.117:0/3677825667 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f583c0056e0 con 0x7f5848103040 2024-04-20T11:35:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.746+0000 7f5845600700 1 -- 172.21.15.117:0/3677825667 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f583c00fdf0 con 0x7f5848103040 2024-04-20T11:35:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.746+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584819c7d0 con 0x7f5848103040 2024-04-20T11:35:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.746+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584819cc70 con 0x7f5848103040 2024-04-20T11:35:28.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.748+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584810b3e0 con 0x7f5848103040 2024-04-20T11:35:28.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.749+0000 7f5845600700 1 -- 172.21.15.117:0/3677825667 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f583c00f5f0 con 0x7f5848103040 2024-04-20T11:35:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.750+0000 7f5845600700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f583006ecc0 0x7f5830071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.751+0000 7f5845600700 1 -- 172.21.15.117:0/3677825667 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f583c0901b0 con 0x7f5848103040 2024-04-20T11:35:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.751+0000 7f5847e00700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f583006ecc0 0x7f5830071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:28.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.752+0000 7f5847e00700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f583006ecc0 0x7f5830071180 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f584819a230 tx=0x7f58340073d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:28.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:28.756+0000 7f5845600700 1 -- 172.21.15.117:0/3677825667 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f583c058f70 con 0x7f5848103040 2024-04-20T11:35:29.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.021+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5848199fe0 con 0x7f583006ecc0 2024-04-20T11:35:29.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.023+0000 7f5845600700 1 -- 172.21.15.117:0/3677825667 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5848199fe0 con 0x7f583006ecc0 2024-04-20T11:35:29.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f583006ecc0 msgr2=0x7f5830071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f583006ecc0 0x7f5830071180 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f584819a230 tx=0x7f58340073d0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 msgr2=0x7f5848198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 0x7f5848198d60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f583c006010 tx=0x7f583c00bd70 comp rx=0 tx=0).stop 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 shutdown_connections 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f583006ecc0 0x7f5830071180 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5848103040 0x7f5848198d60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 --2- 172.21.15.117:0/3677825667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58481039f0 0x7f58481992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 >> 172.21.15.117:0/3677825667 conn(0x7f58480fe8b0 msgr2=0x7f58480fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 shutdown_connections 2024-04-20T11:35:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.026+0000 7f584fa00700 1 -- 172.21.15.117:0/3677825667 wait complete. 2024-04-20T11:35:29.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.172+0000 7f3672a00700 1 -- 172.21.15.117:0/838579408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 msgr2=0x7f366c0ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.172+0000 7f3672a00700 1 --2- 172.21.15.117:0/838579408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 0x7f366c0ffcb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f365c009910 tx=0x7f365c009c20 comp rx=0 tx=0).stop 2024-04-20T11:35:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.173+0000 7f3672a00700 1 -- 172.21.15.117:0/838579408 shutdown_connections 2024-04-20T11:35:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.173+0000 7f3672a00700 1 --2- 172.21.15.117:0/838579408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c100280 0x7f366c10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.173+0000 7f3672a00700 1 --2- 172.21.15.117:0/838579408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 0x7f366c0ffcb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.173+0000 7f3672a00700 1 -- 172.21.15.117:0/838579408 >> 172.21.15.117:0/838579408 conn(0x7f366c076750 msgr2=0x7f366c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.173+0000 7f3672a00700 1 -- 172.21.15.117:0/838579408 shutdown_connections 2024-04-20T11:35:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.173+0000 7f3672a00700 1 -- 172.21.15.117:0/838579408 wait complete. 2024-04-20T11:35:29.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.174+0000 7f3672a00700 1 Processor -- start 2024-04-20T11:35:29.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.174+0000 7f3672a00700 1 -- start start 2024-04-20T11:35:29.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.174+0000 7f3672a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 0x7f366c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.174+0000 7f3672a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c100280 0x7f366c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.174+0000 7f3672a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c199a50 con 0x7f366c0ff8d0 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.174+0000 7f3672a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c19d720 con 0x7f366c100280 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.175+0000 7f366be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 0x7f366c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.175+0000 7f366be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 0x7f366c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56198/0 (socket says 172.21.15.117:56198) 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.175+0000 7f366be00700 1 -- 172.21.15.117:0/2125866260 learned_addr learned my addr 172.21.15.117:0/2125866260 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.175+0000 7f366b400700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c100280 0x7f366c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.175+0000 7f366be00700 1 -- 172.21.15.117:0/2125866260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c100280 msgr2=0x7f366c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.175+0000 7f366be00700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c100280 0x7f366c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.175+0000 7f366be00700 1 -- 172.21.15.117:0/2125866260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0095f0 con 0x7f366c0ff8d0 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.175+0000 7f366be00700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 0x7f366c198e30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f365c0092e0 tx=0x7f365c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.176+0000 7f3668c00700 1 -- 172.21.15.117:0/2125866260 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365c01d070 con 0x7f366c0ff8d0 2024-04-20T11:35:29.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.176+0000 7f3668c00700 1 -- 172.21.15.117:0/2125866260 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365c00bed0 con 0x7f366c0ff8d0 2024-04-20T11:35:29.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.176+0000 7f3668c00700 1 -- 172.21.15.117:0/2125866260 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365c021410 con 0x7f366c0ff8d0 2024-04-20T11:35:29.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.176+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c19d9a0 con 0x7f366c0ff8d0 2024-04-20T11:35:29.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.176+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c19def0 con 0x7f366c0ff8d0 2024-04-20T11:35:29.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.177+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c10cc20 con 0x7f366c0ff8d0 2024-04-20T11:35:29.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.178+0000 7f3668c00700 1 -- 172.21.15.117:0/2125866260 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f365c00f460 con 0x7f366c0ff8d0 2024-04-20T11:35:29.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.179+0000 7f3668c00700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecc0 0x7f3650071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.179+0000 7f3668c00700 1 -- 172.21.15.117:0/2125866260 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f365c0900e0 con 0x7f366c0ff8d0 2024-04-20T11:35:29.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.179+0000 7f366b400700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecc0 0x7f3650071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.180+0000 7f366b400700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecc0 0x7f3650071180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3658006fd0 tx=0x7f3658008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:29.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.184+0000 7f3668c00700 1 -- 172.21.15.117:0/2125866260 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f365c058ea0 con 0x7f366c0ff8d0 2024-04-20T11:35:29.326 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:29 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:29.326 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:29 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:35:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:29 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:29 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:35:29.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.415+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f366c19a140 con 0x7f365006ecc0 2024-04-20T11:35:29.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.429+0000 7f3668c00700 1 -- 172.21.15.117:0/2125866260 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f366c19a140 con 0x7f365006ecc0 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (23m) 15m ago 37m 14.8M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (38m) 15m ago 38m 9369k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (36m) 15m ago 36m 9248k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (20m) 15m ago 38m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (20m) 15m ago 36m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (23m) 15m ago 37m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (15m) 15m ago 30m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:35:29.430 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (16m) 15m ago 30m 11.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 starting - - - - 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (15m) 15m ago 30m 11.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (27m) 15m ago 39m 511M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (25m) 15m ago 36m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (21m) 15m ago 39m 57.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (21m) 15m ago 36m 41.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (25m) 15m ago 38m 8354k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (25m) 15m ago 36m 7738k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (19m) 15m ago 35m 85.1M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (19m) 15m ago 35m 86.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (18m) 15m ago 34m 83.8M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (18m) 15m ago 33m 89.8M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (17m) 15m ago 33m 83.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:35:29.431 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (16m) 15m ago 32m 76.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:35:29.432 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (24m) 15m ago 37m 72.5M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:35:29.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecc0 msgr2=0x7f3650071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecc0 0x7f3650071180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3658006fd0 tx=0x7f3658008040 comp rx=0 tx=0).stop 2024-04-20T11:35:29.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 msgr2=0x7f366c198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 0x7f366c198e30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f365c0092e0 tx=0x7f365c0056a0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 shutdown_connections 2024-04-20T11:35:29.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f365006ecc0 0x7f3650071180 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f366c0ff8d0 0x7f366c198e30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 --2- 172.21.15.117:0/2125866260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f366c100280 0x7f366c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 >> 172.21.15.117:0/2125866260 conn(0x7f366c076750 msgr2=0x7f366c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:29.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 shutdown_connections 2024-04-20T11:35:29.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.432+0000 7f3672a00700 1 -- 172.21.15.117:0/2125866260 wait complete. 2024-04-20T11:35:29.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.584+0000 7f80be600700 1 -- 172.21.15.117:0/2784973290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81017c0 msgr2=0x7f80b8105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.584+0000 7f80be600700 1 --2- 172.21.15.117:0/2784973290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81017c0 0x7f80b8105810 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009910 tx=0x7f80a4009c20 comp rx=0 tx=0).stop 2024-04-20T11:35:29.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.584+0000 7f80be600700 1 -- 172.21.15.117:0/2784973290 shutdown_connections 2024-04-20T11:35:29.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.584+0000 7f80be600700 1 --2- 172.21.15.117:0/2784973290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81017c0 0x7f80b8105810 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.584+0000 7f80be600700 1 --2- 172.21.15.117:0/2784973290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b8100e10 0x7f80b81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.584+0000 7f80be600700 1 -- 172.21.15.117:0/2784973290 >> 172.21.15.117:0/2784973290 conn(0x7f80b80fc6c0 msgr2=0x7f80b80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:29.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.584+0000 7f80be600700 1 -- 172.21.15.117:0/2784973290 shutdown_connections 2024-04-20T11:35:29.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.584+0000 7f80be600700 1 -- 172.21.15.117:0/2784973290 wait complete. 2024-04-20T11:35:29.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.586+0000 7f80be600700 1 Processor -- start 2024-04-20T11:35:29.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.586+0000 7f80be600700 1 -- start start 2024-04-20T11:35:29.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.586+0000 7f80be600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8100e10 0x7f80b8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.586+0000 7f80be600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b81017c0 0x7f80b8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.586+0000 7f80be600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b81999f0 con 0x7f80b8100e10 2024-04-20T11:35:29.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.586+0000 7f80be600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819d5b0 con 0x7f80b81017c0 2024-04-20T11:35:29.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.586+0000 7f80b7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8100e10 0x7f80b8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.587+0000 7f80b7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8100e10 0x7f80b8198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56214/0 (socket says 172.21.15.117:56214) 2024-04-20T11:35:29.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.587+0000 7f80b7400700 1 -- 172.21.15.117:0/1253703253 learned_addr learned my addr 172.21.15.117:0/1253703253 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:35:29.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.587+0000 7f80b6a00700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b81017c0 0x7f80b8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.587+0000 7f80b7400700 1 -- 172.21.15.117:0/1253703253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b81017c0 msgr2=0x7f80b8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.587+0000 7f80b7400700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b81017c0 0x7f80b8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.587+0000 7f80b7400700 1 -- 172.21.15.117:0/1253703253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40095f0 con 0x7f80b8100e10 2024-04-20T11:35:29.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.587+0000 7f80b7400700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8100e10 0x7f80b8198dd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d8d0 tx=0x7f80a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:29.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.587+0000 7f80afe00700 1 -- 172.21.15.117:0/1253703253 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a8009940 con 0x7f80b8100e10 2024-04-20T11:35:29.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.588+0000 7f80afe00700 1 -- 172.21.15.117:0/1253703253 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80a8010460 con 0x7f80b8100e10 2024-04-20T11:35:29.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.588+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b819d890 con 0x7f80b8100e10 2024-04-20T11:35:29.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.588+0000 7f80afe00700 1 -- 172.21.15.117:0/1253703253 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a800f5f0 con 0x7f80b8100e10 2024-04-20T11:35:29.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.588+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b819ddb0 con 0x7f80b8100e10 2024-04-20T11:35:29.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.589+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b81022f0 con 0x7f80b8100e10 2024-04-20T11:35:29.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.590+0000 7f80afe00700 1 -- 172.21.15.117:0/1253703253 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f80a80105d0 con 0x7f80b8100e10 2024-04-20T11:35:29.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.591+0000 7f80afe00700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f809c06ecc0 0x7f809c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:29.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.591+0000 7f80afe00700 1 -- 172.21.15.117:0/1253703253 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f80a808ed40 con 0x7f80b8100e10 2024-04-20T11:35:29.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.592+0000 7f80b6a00700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f809c06ecc0 0x7f809c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:29.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.592+0000 7f80b6a00700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f809c06ecc0 0x7f809c071180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f80b819a220 tx=0x7f80a401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:29.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.597+0000 7f80afe00700 1 -- 172.21.15.117:0/1253703253 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f80a8057ba0 con 0x7f80b8100e10 2024-04-20T11:35:29.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.941+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80b810ddb0 con 0x7f80b8100e10 2024-04-20T11:35:29.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.942+0000 7f80afe00700 1 -- 172.21.15.117:0/1253703253 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f80a805b260 con 0x7f80b8100e10 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:35:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:35:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:35:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:35:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:35:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:35:29.943 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:35:29.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f809c06ecc0 msgr2=0x7f809c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f809c06ecc0 0x7f809c071180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f80b819a220 tx=0x7f80a401a040 comp rx=0 tx=0).stop 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8100e10 msgr2=0x7f80b8198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8100e10 0x7f80b8198dd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d8d0 tx=0x7f80a800dc90 comp rx=0 tx=0).stop 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 shutdown_connections 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f809c06ecc0 0x7f809c071180 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8100e10 0x7f80b8198dd0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 --2- 172.21.15.117:0/1253703253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b81017c0 0x7f80b8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 >> 172.21.15.117:0/1253703253 conn(0x7f80b80fc6c0 msgr2=0x7f80b8074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 shutdown_connections 2024-04-20T11:35:29.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:29.945+0000 7f80be600700 1 -- 172.21.15.117:0/1253703253 wait complete. 2024-04-20T11:35:30.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.097+0000 7fd364800700 1 -- 172.21.15.117:0/2301272347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c107520 msgr2=0x7fd35c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.097+0000 7fd364800700 1 --2- 172.21.15.117:0/2301272347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c107520 0x7fd35c107900 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd350009910 tx=0x7fd350009c20 comp rx=0 tx=0).stop 2024-04-20T11:35:30.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.097+0000 7fd364800700 1 -- 172.21.15.117:0/2301272347 shutdown_connections 2024-04-20T11:35:30.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.097+0000 7fd364800700 1 --2- 172.21.15.117:0/2301272347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c108e10 0x7fd35c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.097+0000 7fd364800700 1 --2- 172.21.15.117:0/2301272347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c107520 0x7fd35c107900 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.097+0000 7fd364800700 1 -- 172.21.15.117:0/2301272347 >> 172.21.15.117:0/2301272347 conn(0x7fd35c0768f0 msgr2=0x7fd35c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:30.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.097+0000 7fd364800700 1 -- 172.21.15.117:0/2301272347 shutdown_connections 2024-04-20T11:35:30.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.097+0000 7fd364800700 1 -- 172.21.15.117:0/2301272347 wait complete. 2024-04-20T11:35:30.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.099+0000 7fd364800700 1 Processor -- start 2024-04-20T11:35:30.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.099+0000 7fd364800700 1 -- start start 2024-04-20T11:35:30.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.099+0000 7fd364800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c107520 0x7fd35c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd364800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c108e10 0x7fd35c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd364800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c19ddd0 con 0x7fd35c108e10 2024-04-20T11:35:30.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd364800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c1a1aa0 con 0x7fd35c107520 2024-04-20T11:35:30.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd361400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c108e10 0x7fd35c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd361400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c108e10 0x7fd35c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56222/0 (socket says 172.21.15.117:56222) 2024-04-20T11:35:30.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd361400700 1 -- 172.21.15.117:0/2975268899 learned_addr learned my addr 172.21.15.117:0/2975268899 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:35:30.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd361e00700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c107520 0x7fd35c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd361400700 1 -- 172.21.15.117:0/2975268899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c107520 msgr2=0x7fd35c19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd361400700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c107520 0x7fd35c19d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.100+0000 7fd361400700 1 -- 172.21.15.117:0/2975268899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3500095f0 con 0x7fd35c108e10 2024-04-20T11:35:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.101+0000 7fd361400700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c108e10 0x7fd35c19d6f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00c8a0 tx=0x7fd34c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.101+0000 7fd356a00700 1 -- 172.21.15.117:0/2975268899 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd34c00cea0 con 0x7fd35c108e10 2024-04-20T11:35:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.101+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c1a1d80 con 0x7fd35c108e10 2024-04-20T11:35:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.101+0000 7fd356a00700 1 -- 172.21.15.117:0/2975268899 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd34c004830 con 0x7fd35c108e10 2024-04-20T11:35:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.101+0000 7fd356a00700 1 -- 172.21.15.117:0/2975268899 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd34c0055b0 con 0x7fd35c108e10 2024-04-20T11:35:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.101+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c1a2350 con 0x7fd35c108e10 2024-04-20T11:35:30.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.102+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c10f870 con 0x7fd35c108e10 2024-04-20T11:35:30.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.104+0000 7fd356a00700 1 -- 172.21.15.117:0/2975268899 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd34c0049a0 con 0x7fd35c108e10 2024-04-20T11:35:30.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.105+0000 7fd356a00700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd34406ecd0 0x7fd344071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.106+0000 7fd356a00700 1 -- 172.21.15.117:0/2975268899 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd34c08eb20 con 0x7fd35c108e10 2024-04-20T11:35:30.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.106+0000 7fd361e00700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd34406ecd0 0x7fd344071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.107+0000 7fd361e00700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd34406ecd0 0x7fd344071190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd35000b5c0 tx=0x7fd35001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:30.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.110+0000 7fd356a00700 1 -- 172.21.15.117:0/2975268899 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd34c057900 con 0x7fd35c108e10 2024-04-20T11:35:30.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:30 smithi117 ceph-mon[99036]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:35:30.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:30 smithi117 ceph-mon[99036]: pgmap v692: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:30.345 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:30 smithi117 ceph-mon[99036]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:35:30.346 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:30 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1253703253' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:30.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.387+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd35c108920 con 0x7fd35c108e10 2024-04-20T11:35:30.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.388+0000 7fd356a00700 1 -- 172.21.15.117:0/2975268899 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fd34c01d780 con 0x7fd35c108e10 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:35:30.388 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:35:30.389 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:35:30.390 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:35:30.391 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:35:30.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.390+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd34406ecd0 msgr2=0x7fd344071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.390+0000 7fd364800700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd34406ecd0 0x7fd344071190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd35000b5c0 tx=0x7fd35001a040 comp rx=0 tx=0).stop 2024-04-20T11:35:30.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c108e10 msgr2=0x7fd35c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c108e10 0x7fd35c19d6f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00c8a0 tx=0x7fd34c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 shutdown_connections 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd34406ecd0 0x7fd344071190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c107520 0x7fd35c19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 --2- 172.21.15.117:0/2975268899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd35c108e10 0x7fd35c19d6f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 >> 172.21.15.117:0/2975268899 conn(0x7fd35c0768f0 msgr2=0x7fd35c077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 shutdown_connections 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.391+0000 7fd364800700 1 -- 172.21.15.117:0/2975268899 wait complete. 2024-04-20T11:35:30.392 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:35:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:30 smithi173 ceph-mon[74165]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:35:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:30 smithi173 ceph-mon[74165]: pgmap v692: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:30 smithi173 ceph-mon[74165]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:35:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:30 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1253703253' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:35:30.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.549+0000 7fbc18c00700 1 -- 172.21.15.117:0/893313613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141002e0 msgr2=0x7fbc1410cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.549+0000 7fbc18c00700 1 --2- 172.21.15.117:0/893313613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141002e0 0x7fbc1410cfa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009910 tx=0x7fbbfc009c20 comp rx=0 tx=0).stop 2024-04-20T11:35:30.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.550+0000 7fbc18c00700 1 -- 172.21.15.117:0/893313613 shutdown_connections 2024-04-20T11:35:30.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.550+0000 7fbc18c00700 1 --2- 172.21.15.117:0/893313613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc141002e0 0x7fbc1410cfa0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.550+0000 7fbc18c00700 1 --2- 172.21.15.117:0/893313613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc140ff930 0x7fbc140ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.550+0000 7fbc18c00700 1 -- 172.21.15.117:0/893313613 >> 172.21.15.117:0/893313613 conn(0x7fbc14076750 msgr2=0x7fbc140fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:30.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.550+0000 7fbc18c00700 1 -- 172.21.15.117:0/893313613 shutdown_connections 2024-04-20T11:35:30.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.550+0000 7fbc18c00700 1 -- 172.21.15.117:0/893313613 wait complete. 2024-04-20T11:35:30.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.551+0000 7fbc18c00700 1 Processor -- start 2024-04-20T11:35:30.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.551+0000 7fbc18c00700 1 -- start start 2024-04-20T11:35:30.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.551+0000 7fbc18c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc140ff930 0x7fbc1419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.551+0000 7fbc18c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc141002e0 0x7fbc1419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.551+0000 7fbc18c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc14198b10 con 0x7fbc140ff930 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.551+0000 7fbc18c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc14198c80 con 0x7fbc141002e0 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.552+0000 7fbc11e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc140ff930 0x7fbc1419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.552+0000 7fbc11e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc140ff930 0x7fbc1419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56244/0 (socket says 172.21.15.117:56244) 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.552+0000 7fbc11e00700 1 -- 172.21.15.117:0/605488033 learned_addr learned my addr 172.21.15.117:0/605488033 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.552+0000 7fbc09400700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc141002e0 0x7fbc1419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.552+0000 7fbc11e00700 1 -- 172.21.15.117:0/605488033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc141002e0 msgr2=0x7fbc1419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.552+0000 7fbc11e00700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc141002e0 0x7fbc1419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.552+0000 7fbc11e00700 1 -- 172.21.15.117:0/605488033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0095f0 con 0x7fbc140ff930 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.552+0000 7fbc11e00700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc140ff930 0x7fbc1419ea60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400d8d0 tx=0x7fbc0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.553+0000 7fbc0b400700 1 -- 172.21.15.117:0/605488033 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc04009940 con 0x7fbc140ff930 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.553+0000 7fbc0b400700 1 -- 172.21.15.117:0/605488033 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc04010460 con 0x7fbc140ff930 2024-04-20T11:35:30.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.553+0000 7fbc0b400700 1 -- 172.21.15.117:0/605488033 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc0400f5f0 con 0x7fbc140ff930 2024-04-20T11:35:30.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.553+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc14198f60 con 0x7fbc140ff930 2024-04-20T11:35:30.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.553+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc141994b0 con 0x7fbc140ff930 2024-04-20T11:35:30.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.554+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc14068df0 con 0x7fbc140ff930 2024-04-20T11:35:30.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.555+0000 7fbc0b400700 1 -- 172.21.15.117:0/605488033 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbc040105d0 con 0x7fbc140ff930 2024-04-20T11:35:30.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.556+0000 7fbc0b400700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbf806ecc0 0x7fbbf8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.556+0000 7fbc0b400700 1 -- 172.21.15.117:0/605488033 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbc0408ed40 con 0x7fbc140ff930 2024-04-20T11:35:30.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.556+0000 7fbc09400700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbf806ecc0 0x7fbbf8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.557+0000 7fbc09400700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbf806ecc0 0x7fbbf8071180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbc1419a280 tx=0x7fbbfc005ad0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:30.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.561+0000 7fbc0b400700 1 -- 172.21.15.117:0/605488033 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbc04057ba0 con 0x7fbc140ff930 2024-04-20T11:35:30.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.816+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc1410fef0 con 0x7fbbf806ecc0 2024-04-20T11:35:30.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.818+0000 7fbc0b400700 1 -- 172.21.15.117:0/605488033 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbc1410fef0 con 0x7fbbf806ecc0 2024-04-20T11:35:30.817 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:35:30.817 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "osd" 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "15/23 daemons upgraded", 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:35:30.818 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:35:30.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.820+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbf806ecc0 msgr2=0x7fbbf8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.820+0000 7fbc18c00700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbf806ecc0 0x7fbbf8071180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbc1419a280 tx=0x7fbbfc005ad0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.820+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc140ff930 msgr2=0x7fbc1419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.820+0000 7fbc18c00700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc140ff930 0x7fbc1419ea60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400d8d0 tx=0x7fbc0400dc90 comp rx=0 tx=0).stop 2024-04-20T11:35:30.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.821+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 shutdown_connections 2024-04-20T11:35:30.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.821+0000 7fbc18c00700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbf806ecc0 0x7fbbf8071180 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.821+0000 7fbc18c00700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc140ff930 0x7fbc1419ea60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.821+0000 7fbc18c00700 1 --2- 172.21.15.117:0/605488033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc141002e0 0x7fbc1419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.821+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 >> 172.21.15.117:0/605488033 conn(0x7fbc14076750 msgr2=0x7fbc140fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:30.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.821+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 shutdown_connections 2024-04-20T11:35:30.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.821+0000 7fbc18c00700 1 -- 172.21.15.117:0/605488033 wait complete. 2024-04-20T11:35:30.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.977+0000 7f9ffcc00700 1 -- 172.21.15.117:0/2914309622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff8101da0 msgr2=0x7f9ff8102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.977+0000 7f9ffcc00700 1 --2- 172.21.15.117:0/2914309622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff8101da0 0x7f9ff8102180 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009940 tx=0x7f9fe8009c50 comp rx=0 tx=0).stop 2024-04-20T11:35:30.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.977+0000 7f9ffcc00700 1 -- 172.21.15.117:0/2914309622 shutdown_connections 2024-04-20T11:35:30.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.977+0000 7f9ffcc00700 1 --2- 172.21.15.117:0/2914309622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff81026c0 0x7f9ff810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.977+0000 7f9ffcc00700 1 --2- 172.21.15.117:0/2914309622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff8101da0 0x7f9ff8102180 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.978+0000 7f9ffcc00700 1 -- 172.21.15.117:0/2914309622 >> 172.21.15.117:0/2914309622 conn(0x7f9ff8076750 msgr2=0x7f9ff80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:30.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.978+0000 7f9ffcc00700 1 -- 172.21.15.117:0/2914309622 shutdown_connections 2024-04-20T11:35:30.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.978+0000 7f9ffcc00700 1 -- 172.21.15.117:0/2914309622 wait complete. 2024-04-20T11:35:30.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.979+0000 7f9ffcc00700 1 Processor -- start 2024-04-20T11:35:30.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.979+0000 7f9ffcc00700 1 -- start start 2024-04-20T11:35:30.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.979+0000 7f9ffcc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff8101da0 0x7f9ff80ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.979+0000 7f9ffcc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff81026c0 0x7f9ff80fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.979+0000 7f9ffcc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff8100480 con 0x7f9ff81026c0 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.979+0000 7f9ffcc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff81005c0 con 0x7f9ff8101da0 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.980+0000 7f9ff5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff81026c0 0x7f9ff80fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.980+0000 7f9ff5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff81026c0 0x7f9ff80fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56272/0 (socket says 172.21.15.117:56272) 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.980+0000 7f9ff5400700 1 -- 172.21.15.117:0/453352408 learned_addr learned my addr 172.21.15.117:0/453352408 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.980+0000 7f9ff5e00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff8101da0 0x7f9ff80ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.980+0000 7f9ff5400700 1 -- 172.21.15.117:0/453352408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff8101da0 msgr2=0x7f9ff80ffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.980+0000 7f9ff5400700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff8101da0 0x7f9ff80ffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:30.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.980+0000 7f9ff5400700 1 -- 172.21.15.117:0/453352408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe80095f0 con 0x7f9ff81026c0 2024-04-20T11:35:30.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.981+0000 7f9ff5400700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff81026c0 0x7f9ff80fff40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400d900 tx=0x7f9fe400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:30.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.981+0000 7f9feea00700 1 -- 172.21.15.117:0/453352408 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe4009900 con 0x7f9ff81026c0 2024-04-20T11:35:30.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.981+0000 7f9feea00700 1 -- 172.21.15.117:0/453352408 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fe4010460 con 0x7f9ff81026c0 2024-04-20T11:35:30.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.981+0000 7f9feea00700 1 -- 172.21.15.117:0/453352408 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe400f5f0 con 0x7f9ff81026c0 2024-04-20T11:35:30.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.981+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff81a2ce0 con 0x7f9ff81026c0 2024-04-20T11:35:30.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.981+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff81a3040 con 0x7f9ff81026c0 2024-04-20T11:35:30.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.982+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff8068df0 con 0x7f9ff81026c0 2024-04-20T11:35:30.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.984+0000 7f9feea00700 1 -- 172.21.15.117:0/453352408 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9fe40105d0 con 0x7f9ff81026c0 2024-04-20T11:35:30.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.985+0000 7f9feea00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9fdc06ecc0 0x7f9fdc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:35:30.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.985+0000 7f9feea00700 1 -- 172.21.15.117:0/453352408 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9fe408eba0 con 0x7f9ff81026c0 2024-04-20T11:35:30.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.985+0000 7f9ff5e00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9fdc06ecc0 0x7f9fdc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:35:30.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.986+0000 7f9ff5e00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9fdc06ecc0 0x7f9fdc071180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005b40 tx=0x7f9fe8005ad0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:35:30.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:30.990+0000 7f9feea00700 1 -- 172.21.15.117:0/453352408 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9fe4057a00 con 0x7f9ff81026c0 2024-04-20T11:35:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.333+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9ff8062510 con 0x7f9ff81026c0 2024-04-20T11:35:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.334+0000 7f9feea00700 1 -- 172.21.15.117:0/453352408 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f9fe405b0c0 con 0x7f9ff81026c0 2024-04-20T11:35:31.334 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:35:31.334 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:35:31.334 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:35:31.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:35:31.335 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:35:31.336 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:35:31.336 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:31 smithi117 ceph-mon[99036]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:35:31.337 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:31 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2975268899' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:35:31.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9fdc06ecc0 msgr2=0x7f9fdc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:31.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9fdc06ecc0 0x7f9fdc071180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005b40 tx=0x7f9fe8005ad0 comp rx=0 tx=0).stop 2024-04-20T11:35:31.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff81026c0 msgr2=0x7f9ff80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:35:31.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff81026c0 0x7f9ff80fff40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400d900 tx=0x7f9fe400dcc0 comp rx=0 tx=0).stop 2024-04-20T11:35:31.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 shutdown_connections 2024-04-20T11:35:31.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9fdc06ecc0 0x7f9fdc071180 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:31.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff8101da0 0x7f9ff80ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:31.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 --2- 172.21.15.117:0/453352408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ff81026c0 0x7f9ff80fff40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:35:31.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 >> 172.21.15.117:0/453352408 conn(0x7f9ff8076750 msgr2=0x7f9ff8103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:35:31.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 shutdown_connections 2024-04-20T11:35:31.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:35:31.338+0000 7f9ffcc00700 1 -- 172.21.15.117:0/453352408 wait complete. 2024-04-20T11:35:31.397 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:31 smithi173 ceph-mon[74165]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:35:31.397 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:31 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2975268899' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:35:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:32 smithi117 ceph-mon[99036]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:35:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:32 smithi117 ceph-mon[99036]: pgmap v693: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:32 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/453352408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:35:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:32 smithi173 ceph-mon[74165]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:35:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:32 smithi173 ceph-mon[74165]: pgmap v693: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:32 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/453352408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:35:34.636 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:34 smithi173 ceph-mon[74165]: pgmap v694: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:34.636 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:34.636 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:34.636 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:34.636 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:34 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:34 smithi117 ceph-mon[99036]: pgmap v694: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:34 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:35:36.486 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:36 smithi173 ceph-mon[74165]: pgmap v695: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:36 smithi117 ceph-mon[99036]: pgmap v695: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:38.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:38 smithi117 ceph-mon[99036]: pgmap v696: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:38 smithi173 ceph-mon[74165]: pgmap v696: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:40.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:39 smithi117 ceph-mon[99036]: pgmap v697: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:39 smithi173 ceph-mon[74165]: pgmap v697: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:42.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:41 smithi117 ceph-mon[99036]: pgmap v698: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:42.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:41 smithi173 ceph-mon[74165]: pgmap v698: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:44.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:43 smithi117 ceph-mon[99036]: pgmap v699: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:35:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:43 smithi173 ceph-mon[74165]: pgmap v699: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:35:46.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:45 smithi117 ceph-mon[99036]: pgmap v700: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:45 smithi173 ceph-mon[74165]: pgmap v700: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:48.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:47 smithi173 ceph-mon[74165]: pgmap v701: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:48.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:47 smithi117 ceph-mon[99036]: pgmap v701: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:49 smithi117 ceph-mon[99036]: pgmap v702: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:49 smithi173 ceph-mon[74165]: pgmap v702: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:51 smithi117 ceph-mon[99036]: pgmap v703: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:51 smithi173 ceph-mon[74165]: pgmap v703: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:54.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:53 smithi117 ceph-mon[99036]: pgmap v704: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:53 smithi173 ceph-mon[74165]: pgmap v704: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:55 smithi117 ceph-mon[99036]: pgmap v705: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:56.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:55 smithi173 ceph-mon[74165]: pgmap v705: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:58.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:58 smithi117 ceph-mon[99036]: pgmap v706: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:58 smithi173 ceph-mon[74165]: pgmap v706: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:35:59.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:35:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:35:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:35:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:36:00.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:00 smithi117 ceph-mon[99036]: pgmap v707: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:00 smithi173 ceph-mon[74165]: pgmap v707: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:01.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.512+0000 7fdfdac00700 1 -- 172.21.15.117:0/2833643330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 msgr2=0x7fdfd4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.512+0000 7fdfdac00700 1 --2- 172.21.15.117:0/2833643330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 0x7fdfd4104e80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009910 tx=0x7fdfc0009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.512+0000 7fdfdac00700 1 -- 172.21.15.117:0/2833643330 shutdown_connections 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.512+0000 7fdfdac00700 1 --2- 172.21.15.117:0/2833643330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 0x7fdfd4104e80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.512+0000 7fdfdac00700 1 --2- 172.21.15.117:0/2833643330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd4103110 0x7fdfd41034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.512+0000 7fdfdac00700 1 -- 172.21.15.117:0/2833643330 >> 172.21.15.117:0/2833643330 conn(0x7fdfd40fe880 msgr2=0x7fdfd4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.513+0000 7fdfdac00700 1 -- 172.21.15.117:0/2833643330 shutdown_connections 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.513+0000 7fdfdac00700 1 -- 172.21.15.117:0/2833643330 wait complete. 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.514+0000 7fdfdac00700 1 Processor -- start 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.514+0000 7fdfdac00700 1 -- start start 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.514+0000 7fdfdac00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd4103110 0x7fdfd419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.514+0000 7fdfdac00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 0x7fdfd419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.514+0000 7fdfdac00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd419f520 con 0x7fdfd4104a00 2024-04-20T11:36:01.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.514+0000 7fdfdac00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4198b90 con 0x7fdfd4103110 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 0x7fdfd419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 0x7fdfd419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35080/0 (socket says 172.21.15.117:35080) 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3400700 1 -- 172.21.15.117:0/3901601843 learned_addr learned my addr 172.21.15.117:0/3901601843 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3e00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd4103110 0x7fdfd419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3400700 1 -- 172.21.15.117:0/3901601843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd4103110 msgr2=0x7fdfd419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3400700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd4103110 0x7fdfd419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3400700 1 -- 172.21.15.117:0/3901601843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc00095f0 con 0x7fdfd4104a00 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3e00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd4103110 0x7fdfd419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.515+0000 7fdfd3400700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 0x7fdfd419ef90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0006010 tx=0x7fdfc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.516+0000 7fdfd0c00700 1 -- 172.21.15.117:0/3901601843 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfc001d070 con 0x7fdfd4104a00 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.516+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd4198e10 con 0x7fdfd4104a00 2024-04-20T11:36:01.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.516+0000 7fdfd0c00700 1 -- 172.21.15.117:0/3901601843 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfc000f460 con 0x7fdfd4104a00 2024-04-20T11:36:01.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.516+0000 7fdfd0c00700 1 -- 172.21.15.117:0/3901601843 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfc0017630 con 0x7fdfd4104a00 2024-04-20T11:36:01.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.516+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd4199380 con 0x7fdfd4104a00 2024-04-20T11:36:01.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.517+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd410b480 con 0x7fdfd4104a00 2024-04-20T11:36:01.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.520+0000 7fdfd0c00700 1 -- 172.21.15.117:0/3901601843 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdfc000f5d0 con 0x7fdfd4104a00 2024-04-20T11:36:01.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.521+0000 7fdfd0c00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfb806ecd0 0x7fdfb8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.521+0000 7fdfd0c00700 1 -- 172.21.15.117:0/3901601843 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdfc00903f0 con 0x7fdfd4104a00 2024-04-20T11:36:01.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.521+0000 7fdfd3e00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfb806ecd0 0x7fdfb8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.522+0000 7fdfd3e00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfb806ecd0 0x7fdfb8071190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4005fd0 tx=0x7fdfc4005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.527+0000 7fdfd0c00700 1 -- 172.21.15.117:0/3901601843 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdfc005cd20 con 0x7fdfd4104a00 2024-04-20T11:36:01.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.777+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfd4199fe0 con 0x7fdfb806ecd0 2024-04-20T11:36:01.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.781+0000 7fdfd0c00700 1 -- 172.21.15.117:0/3901601843 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdfd4199fe0 con 0x7fdfb806ecd0 2024-04-20T11:36:01.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfb806ecd0 msgr2=0x7fdfb8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfb806ecd0 0x7fdfb8071190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4005fd0 tx=0x7fdfc4005e20 comp rx=0 tx=0).stop 2024-04-20T11:36:01.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 msgr2=0x7fdfd419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 0x7fdfd419ef90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0006010 tx=0x7fdfc000be20 comp rx=0 tx=0).stop 2024-04-20T11:36:01.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 shutdown_connections 2024-04-20T11:36:01.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfb806ecd0 0x7fdfb8071190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd4103110 0x7fdfd419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 --2- 172.21.15.117:0/3901601843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd4104a00 0x7fdfd419ef90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.783+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 >> 172.21.15.117:0/3901601843 conn(0x7fdfd40fe880 msgr2=0x7fdfd40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:01.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.784+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 shutdown_connections 2024-04-20T11:36:01.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.784+0000 7fdfdac00700 1 -- 172.21.15.117:0/3901601843 wait complete. 2024-04-20T11:36:01.800 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:36:01.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.944+0000 7f0fbc600700 1 -- 172.21.15.117:0/3397417231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 msgr2=0x7f0fb4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.944+0000 7f0fbc600700 1 --2- 172.21.15.117:0/3397417231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 0x7f0fb4109290 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009910 tx=0x7f0fa0009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:01.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.944+0000 7f0fbc600700 1 -- 172.21.15.117:0/3397417231 shutdown_connections 2024-04-20T11:36:01.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.944+0000 7f0fbc600700 1 --2- 172.21.15.117:0/3397417231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 0x7f0fb4109290 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.944+0000 7f0fbc600700 1 --2- 172.21.15.117:0/3397417231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fb4107520 0x7f0fb4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.944+0000 7f0fbc600700 1 -- 172.21.15.117:0/3397417231 >> 172.21.15.117:0/3397417231 conn(0x7f0fb40768f0 msgr2=0x7f0fb4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:01.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.944+0000 7f0fbc600700 1 -- 172.21.15.117:0/3397417231 shutdown_connections 2024-04-20T11:36:01.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.944+0000 7f0fbc600700 1 -- 172.21.15.117:0/3397417231 wait complete. 2024-04-20T11:36:01.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.945+0000 7f0fbc600700 1 Processor -- start 2024-04-20T11:36:01.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.945+0000 7f0fbc600700 1 -- start start 2024-04-20T11:36:01.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.945+0000 7f0fbc600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fb4107520 0x7f0fb419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.945+0000 7f0fbc600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 0x7f0fb419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.945+0000 7f0fbc600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb419ddf0 con 0x7f0fb4108e10 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.945+0000 7f0fbc600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41a1a70 con 0x7f0fb4107520 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.946+0000 7f0fb9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 0x7f0fb419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.946+0000 7f0fb9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 0x7f0fb419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35102/0 (socket says 172.21.15.117:35102) 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.946+0000 7f0fb9200700 1 -- 172.21.15.117:0/320091211 learned_addr learned my addr 172.21.15.117:0/320091211 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.946+0000 7f0fb9c00700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fb4107520 0x7f0fb419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.946+0000 7f0fb9200700 1 -- 172.21.15.117:0/320091211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fb4107520 msgr2=0x7f0fb419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.946+0000 7f0fb9200700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fb4107520 0x7f0fb419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.946+0000 7f0fb9200700 1 -- 172.21.15.117:0/320091211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa00095f0 con 0x7f0fb4108e10 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.947+0000 7f0fb9200700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 0x7f0fb419d710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0006010 tx=0x7f0fa000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.947+0000 7f0fb2000700 1 -- 172.21.15.117:0/320091211 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa001d070 con 0x7f0fb4108e10 2024-04-20T11:36:01.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.947+0000 7f0fb2000700 1 -- 172.21.15.117:0/320091211 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fa00056e0 con 0x7f0fb4108e10 2024-04-20T11:36:01.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.947+0000 7f0fb2000700 1 -- 172.21.15.117:0/320091211 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa0017400 con 0x7f0fb4108e10 2024-04-20T11:36:01.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.947+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb41a1cf0 con 0x7f0fb4108e10 2024-04-20T11:36:01.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.947+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb41a2260 con 0x7f0fb4108e10 2024-04-20T11:36:01.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.948+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb410f870 con 0x7f0fb4108e10 2024-04-20T11:36:01.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.949+0000 7f0fb2000700 1 -- 172.21.15.117:0/320091211 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0fa000f6f0 con 0x7f0fb4108e10 2024-04-20T11:36:01.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.950+0000 7f0fb2000700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f9c06ecc0 0x7f0f9c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:01.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.950+0000 7f0fb2000700 1 -- 172.21.15.117:0/320091211 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0fa0090110 con 0x7f0fb4108e10 2024-04-20T11:36:01.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.951+0000 7f0fb9c00700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f9c06ecc0 0x7f0f9c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:01.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.951+0000 7f0fb9c00700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f9c06ecc0 0x7f0f9c071180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8005950 tx=0x7f0fa80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:01.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:01.956+0000 7f0fb2000700 1 -- 172.21.15.117:0/320091211 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0fa0058f70 con 0x7f0fb4108e10 2024-04-20T11:36:02.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:02 smithi117 ceph-mon[99036]: pgmap v708: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:02.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.204+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fb419e500 con 0x7f0f9c06ecc0 2024-04-20T11:36:02.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.206+0000 7f0fb2000700 1 -- 172.21.15.117:0/320091211 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0fb419e500 con 0x7f0f9c06ecc0 2024-04-20T11:36:02.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f9c06ecc0 msgr2=0x7f0f9c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f9c06ecc0 0x7f0f9c071180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8005950 tx=0x7f0fa80058e0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 msgr2=0x7f0fb419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 0x7f0fb419d710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0006010 tx=0x7f0fa000be20 comp rx=0 tx=0).stop 2024-04-20T11:36:02.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 shutdown_connections 2024-04-20T11:36:02.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f9c06ecc0 0x7f0f9c071180 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fb4107520 0x7f0fb419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 --2- 172.21.15.117:0/320091211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb4108e10 0x7f0fb419d710 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.209+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 >> 172.21.15.117:0/320091211 conn(0x7f0fb40768f0 msgr2=0x7f0fb4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:02.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.210+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 shutdown_connections 2024-04-20T11:36:02.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.210+0000 7f0fbc600700 1 -- 172.21.15.117:0/320091211 wait complete. 2024-04-20T11:36:02.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.370+0000 7eff86c00700 1 -- 172.21.15.117:0/2631831239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 msgr2=0x7eff80073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.370+0000 7eff86c00700 1 --2- 172.21.15.117:0/2631831239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 0x7eff80073660 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7eff70009910 tx=0x7eff70009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:02.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.370+0000 7eff86c00700 1 -- 172.21.15.117:0/2631831239 shutdown_connections 2024-04-20T11:36:02.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.370+0000 7eff86c00700 1 --2- 172.21.15.117:0/2631831239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff80073ba0 0x7eff80111680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.370+0000 7eff86c00700 1 --2- 172.21.15.117:0/2631831239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 0x7eff80073660 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.370+0000 7eff86c00700 1 -- 172.21.15.117:0/2631831239 >> 172.21.15.117:0/2631831239 conn(0x7eff800fc6a0 msgr2=0x7eff800feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:02.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.371+0000 7eff86c00700 1 -- 172.21.15.117:0/2631831239 shutdown_connections 2024-04-20T11:36:02.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.371+0000 7eff86c00700 1 -- 172.21.15.117:0/2631831239 wait complete. 2024-04-20T11:36:02.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.372+0000 7eff86c00700 1 Processor -- start 2024-04-20T11:36:02.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.372+0000 7eff86c00700 1 -- start start 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.372+0000 7eff86c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 0x7eff8019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.372+0000 7eff86c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff80073ba0 0x7eff8019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.372+0000 7eff86c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8019dc80 con 0x7eff80073280 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.372+0000 7eff86c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff801a1950 con 0x7eff80073ba0 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.372+0000 7eff7fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 0x7eff8019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.373+0000 7eff7fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 0x7eff8019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35116/0 (socket says 172.21.15.117:35116) 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.373+0000 7eff7fe00700 1 -- 172.21.15.117:0/4237991697 learned_addr learned my addr 172.21.15.117:0/4237991697 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.373+0000 7eff7fe00700 1 -- 172.21.15.117:0/4237991697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff80073ba0 msgr2=0x7eff8019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.373+0000 7eff7fe00700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff80073ba0 0x7eff8019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.373+0000 7eff7fe00700 1 -- 172.21.15.117:0/4237991697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff700095f0 con 0x7eff80073280 2024-04-20T11:36:02.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.373+0000 7eff7fe00700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 0x7eff8019d060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7eff70006010 tx=0x7eff7000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:02.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.373+0000 7eff7cc00700 1 -- 172.21.15.117:0/4237991697 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7001d070 con 0x7eff80073280 2024-04-20T11:36:02.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.374+0000 7eff7cc00700 1 -- 172.21.15.117:0/4237991697 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff7000f460 con 0x7eff80073280 2024-04-20T11:36:02.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.374+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff801a1c30 con 0x7eff80073280 2024-04-20T11:36:02.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.374+0000 7eff7cc00700 1 -- 172.21.15.117:0/4237991697 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff70017630 con 0x7eff80073280 2024-04-20T11:36:02.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.374+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff801a2180 con 0x7eff80073280 2024-04-20T11:36:02.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.375+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8010ee00 con 0x7eff80073280 2024-04-20T11:36:02.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.376+0000 7eff7cc00700 1 -- 172.21.15.117:0/4237991697 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7eff7000f5d0 con 0x7eff80073280 2024-04-20T11:36:02.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.377+0000 7eff7cc00700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7eff6406ecd0 0x7eff64071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.378+0000 7eff7cc00700 1 -- 172.21.15.117:0/4237991697 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7eff700903d0 con 0x7eff80073280 2024-04-20T11:36:02.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.378+0000 7eff7f400700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7eff6406ecd0 0x7eff64071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.379+0000 7eff7f400700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7eff6406ecd0 0x7eff64071190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7eff8019e5c0 tx=0x7eff6c005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:02.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.382+0000 7eff7cc00700 1 -- 172.21.15.117:0/4237991697 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7eff700591d0 con 0x7eff80073280 2024-04-20T11:36:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:02 smithi173 ceph-mon[74165]: pgmap v708: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:02.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.613+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff80114350 con 0x7eff6406ecd0 2024-04-20T11:36:02.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.646+0000 7eff7cc00700 1 -- 172.21.15.117:0/4237991697 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7eff80114350 con 0x7eff6406ecd0 2024-04-20T11:36:02.723 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (24m) 28s ago 38m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (38m) 28s ago 38m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (37m) 31s ago 37m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (20m) 28s ago 38m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (20m) 31s ago 37m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (23m) 28s ago 38m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (16m) 28s ago 31m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (16m) 28s ago 31m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (37s) 31s ago 31m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:36:02.724 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (15m) 31s ago 31m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (27m) 28s ago 40m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (26m) 31s ago 37m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (22m) 28s ago 40m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (21m) 31s ago 37m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (25m) 28s ago 38m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (25m) 31s ago 37m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (20m) 28s ago 36m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (19m) 28s ago 35m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (19m) 28s ago 34m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (18m) 31s ago 34m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (17m) 31s ago 33m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (17m) 31s ago 33m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:36:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (25m) 28s ago 37m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:36:02.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7eff6406ecd0 msgr2=0x7eff64071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7eff6406ecd0 0x7eff64071190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7eff8019e5c0 tx=0x7eff6c005c80 comp rx=0 tx=0).stop 2024-04-20T11:36:02.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 msgr2=0x7eff8019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 0x7eff8019d060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7eff70006010 tx=0x7eff7000be20 comp rx=0 tx=0).stop 2024-04-20T11:36:02.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 shutdown_connections 2024-04-20T11:36:02.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7eff6406ecd0 0x7eff64071190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff80073280 0x7eff8019d060 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 --2- 172.21.15.117:0/4237991697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff80073ba0 0x7eff8019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.650+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 >> 172.21.15.117:0/4237991697 conn(0x7eff800fc6a0 msgr2=0x7eff80104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:02.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.651+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 shutdown_connections 2024-04-20T11:36:02.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.651+0000 7eff86c00700 1 -- 172.21.15.117:0/4237991697 wait complete. 2024-04-20T11:36:02.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.805+0000 7efdd4800700 1 -- 172.21.15.117:0/2102483019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc1026e0 msgr2=0x7efdcc10abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.805+0000 7efdd4800700 1 --2- 172.21.15.117:0/2102483019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc1026e0 0x7efdcc10abd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009940 tx=0x7efdbc009c50 comp rx=0 tx=0).stop 2024-04-20T11:36:02.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.805+0000 7efdd4800700 1 -- 172.21.15.117:0/2102483019 shutdown_connections 2024-04-20T11:36:02.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.805+0000 7efdd4800700 1 --2- 172.21.15.117:0/2102483019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc1026e0 0x7efdcc10abd0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.805+0000 7efdd4800700 1 --2- 172.21.15.117:0/2102483019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdcc101dc0 0x7efdcc1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.805+0000 7efdd4800700 1 -- 172.21.15.117:0/2102483019 >> 172.21.15.117:0/2102483019 conn(0x7efdcc076750 msgr2=0x7efdcc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:02.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.806+0000 7efdd4800700 1 -- 172.21.15.117:0/2102483019 shutdown_connections 2024-04-20T11:36:02.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.806+0000 7efdd4800700 1 -- 172.21.15.117:0/2102483019 wait complete. 2024-04-20T11:36:02.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.807+0000 7efdd4800700 1 Processor -- start 2024-04-20T11:36:02.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.807+0000 7efdd4800700 1 -- start start 2024-04-20T11:36:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.807+0000 7efdd4800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc101dc0 0x7efdcc0ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.807+0000 7efdd4800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdcc1026e0 0x7efdcc0fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.807+0000 7efdd4800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc1019f0 con 0x7efdcc101dc0 2024-04-20T11:36:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.807+0000 7efdd4800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc1004a0 con 0x7efdcc1026e0 2024-04-20T11:36:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.808+0000 7efdd1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc101dc0 0x7efdcc0ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.808+0000 7efdd1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc101dc0 0x7efdcc0ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35128/0 (socket says 172.21.15.117:35128) 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.808+0000 7efdd1e00700 1 -- 172.21.15.117:0/912740570 learned_addr learned my addr 172.21.15.117:0/912740570 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.808+0000 7efdd1400700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdcc1026e0 0x7efdcc0fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.808+0000 7efdd1e00700 1 -- 172.21.15.117:0/912740570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdcc1026e0 msgr2=0x7efdcc0fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.808+0000 7efdd1e00700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdcc1026e0 0x7efdcc0fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.808+0000 7efdd1e00700 1 -- 172.21.15.117:0/912740570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc0095f0 con 0x7efdcc101dc0 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.808+0000 7efdd1e00700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc101dc0 0x7efdcc0ffa20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efdc000c930 tx=0x7efdc000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.809+0000 7efdc6a00700 1 -- 172.21.15.117:0/912740570 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdc0007ad0 con 0x7efdcc101dc0 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.809+0000 7efdc6a00700 1 -- 172.21.15.117:0/912740570 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdc0007c30 con 0x7efdcc101dc0 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.809+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdcc100750 con 0x7efdcc101dc0 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.809+0000 7efdc6a00700 1 -- 172.21.15.117:0/912740570 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdc0018610 con 0x7efdcc101dc0 2024-04-20T11:36:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.809+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc1a2fa0 con 0x7efdcc101dc0 2024-04-20T11:36:02.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.810+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc068dd0 con 0x7efdcc101dc0 2024-04-20T11:36:02.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.811+0000 7efdc6a00700 1 -- 172.21.15.117:0/912740570 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efdc000f450 con 0x7efdcc101dc0 2024-04-20T11:36:02.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.812+0000 7efdc6a00700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdb406ecc0 0x7efdb4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:02.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.812+0000 7efdc6a00700 1 -- 172.21.15.117:0/912740570 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efdc008f070 con 0x7efdcc101dc0 2024-04-20T11:36:02.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.812+0000 7efdd1400700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdb406ecc0 0x7efdb4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:02.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.813+0000 7efdd1400700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdb406ecc0 0x7efdb4071180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009910 tx=0x7efdbc005ab0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:02.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:02.818+0000 7efdc6a00700 1 -- 172.21.15.117:0/912740570 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efdc0057e50 con 0x7efdcc101dc0 2024-04-20T11:36:03.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.136+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efdcc100f00 con 0x7efdcc101dc0 2024-04-20T11:36:03.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.137+0000 7efdc6a00700 1 -- 172.21.15.117:0/912740570 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efdc005b510 con 0x7efdcc101dc0 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:36:03.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:36:03.139 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:36:03.139 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:36:03.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:36:03.139 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:36:03.139 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:36:03.139 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:36:03.139 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:36:03.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdb406ecc0 msgr2=0x7efdb4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdb406ecc0 0x7efdb4071180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009910 tx=0x7efdbc005ab0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc101dc0 msgr2=0x7efdcc0ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc101dc0 0x7efdcc0ffa20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efdc000c930 tx=0x7efdc000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 shutdown_connections 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdb406ecc0 0x7efdb4071180 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdcc101dc0 0x7efdcc0ffa20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 --2- 172.21.15.117:0/912740570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdcc1026e0 0x7efdcc0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 >> 172.21.15.117:0/912740570 conn(0x7efdcc076750 msgr2=0x7efdcc1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 shutdown_connections 2024-04-20T11:36:03.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.140+0000 7efdd4800700 1 -- 172.21.15.117:0/912740570 wait complete. 2024-04-20T11:36:03.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.294+0000 7f5ee6c00700 1 -- 172.21.15.117:0/2561787595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0102660 msgr2=0x7f5ee010ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.294+0000 7f5ee6c00700 1 --2- 172.21.15.117:0/2561787595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0102660 0x7f5ee010ab50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8009910 tx=0x7f5ec8009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:03.295 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:03 smithi117 ceph-mon[99036]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:03.295 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:03 smithi117 ceph-mon[99036]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:03.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.295+0000 7f5ee6c00700 1 -- 172.21.15.117:0/2561787595 shutdown_connections 2024-04-20T11:36:03.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.295+0000 7f5ee6c00700 1 --2- 172.21.15.117:0/2561787595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0102660 0x7f5ee010ab50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.295+0000 7f5ee6c00700 1 --2- 172.21.15.117:0/2561787595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ee0101d40 0x7f5ee0102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.295+0000 7f5ee6c00700 1 -- 172.21.15.117:0/2561787595 >> 172.21.15.117:0/2561787595 conn(0x7f5ee0076750 msgr2=0x7f5ee00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:03.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.295+0000 7f5ee6c00700 1 -- 172.21.15.117:0/2561787595 shutdown_connections 2024-04-20T11:36:03.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.295+0000 7f5ee6c00700 1 -- 172.21.15.117:0/2561787595 wait complete. 2024-04-20T11:36:03.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.296+0000 7f5ee6c00700 1 Processor -- start 2024-04-20T11:36:03.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.296+0000 7f5ee6c00700 1 -- start start 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5ee6c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0101d40 0x7f5ee0198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5ee6c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ee0102660 0x7f5ee0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5ee6c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee01998d0 con 0x7f5ee0101d40 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5ee6c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee019c540 con 0x7f5ee0102660 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5edfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0101d40 0x7f5ee0198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5edfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0101d40 0x7f5ee0198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35156/0 (socket says 172.21.15.117:35156) 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5edfe00700 1 -- 172.21.15.117:0/291450436 learned_addr learned my addr 172.21.15.117:0/291450436 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5ed7400700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ee0102660 0x7f5ee0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5edfe00700 1 -- 172.21.15.117:0/291450436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ee0102660 msgr2=0x7f5ee0199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5edfe00700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ee0102660 0x7f5ee0199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5edfe00700 1 -- 172.21.15.117:0/291450436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80095f0 con 0x7f5ee0101d40 2024-04-20T11:36:03.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.297+0000 7f5ed7400700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ee0102660 0x7f5ee0199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:03.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.298+0000 7f5edfe00700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0101d40 0x7f5ee0198d00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000d8d0 tx=0x7f5ed000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:03.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.298+0000 7f5edd600700 1 -- 172.21.15.117:0/291450436 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed0009940 con 0x7f5ee0101d40 2024-04-20T11:36:03.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.298+0000 7f5edd600700 1 -- 172.21.15.117:0/291450436 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ed0010460 con 0x7f5ee0101d40 2024-04-20T11:36:03.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.298+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee019c820 con 0x7f5ee0101d40 2024-04-20T11:36:03.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.298+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee019cd70 con 0x7f5ee0101d40 2024-04-20T11:36:03.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.298+0000 7f5edd600700 1 -- 172.21.15.117:0/291450436 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed000f5f0 con 0x7f5ee0101d40 2024-04-20T11:36:03.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.299+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee010a6b0 con 0x7f5ee0101d40 2024-04-20T11:36:03.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.300+0000 7f5edd600700 1 -- 172.21.15.117:0/291450436 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5ed00105d0 con 0x7f5ee0101d40 2024-04-20T11:36:03.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.301+0000 7f5edd600700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ec406ecc0 0x7f5ec4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.302+0000 7f5edd600700 1 -- 172.21.15.117:0/291450436 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5ed008eef0 con 0x7f5ee0101d40 2024-04-20T11:36:03.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.302+0000 7f5ed7400700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ec406ecc0 0x7f5ec4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.302+0000 7f5ed7400700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ec406ecc0 0x7f5ec4071180 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5ee019a220 tx=0x7f5ec800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:03.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.306+0000 7f5edd600700 1 -- 172.21.15.117:0/291450436 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5ed0057c30 con 0x7f5ee0101d40 2024-04-20T11:36:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:03 smithi173 ceph-mon[74165]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:03 smithi173 ceph-mon[74165]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:03.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.572+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5ee019a070 con 0x7f5ee0101d40 2024-04-20T11:36:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.573+0000 7f5edd600700 1 -- 172.21.15.117:0/291450436 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f5ed005b2f0 con 0x7f5ee0101d40 2024-04-20T11:36:03.574 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:36:03.574 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:36:03.574 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:36:03.575 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:36:03.576 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:36:03.577 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:36:03.577 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:36:03.577 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:36:03.577 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:36:03.577 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:36:03.577 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:36:03.577 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:36:03.577 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:36:03.578 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:36:03.578 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:36:03.578 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:36:03.578 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:36:03.578 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:36:03.578 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:36:03.578 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:36:03.578 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:36:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ec406ecc0 msgr2=0x7f5ec4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ec406ecc0 0x7f5ec4071180 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5ee019a220 tx=0x7f5ec800b540 comp rx=0 tx=0).stop 2024-04-20T11:36:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0101d40 msgr2=0x7f5ee0198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0101d40 0x7f5ee0198d00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000d8d0 tx=0x7f5ed000dc90 comp rx=0 tx=0).stop 2024-04-20T11:36:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 shutdown_connections 2024-04-20T11:36:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ec406ecc0 0x7f5ec4071180 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee0101d40 0x7f5ee0198d00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 --2- 172.21.15.117:0/291450436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ee0102660 0x7f5ee0199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 >> 172.21.15.117:0/291450436 conn(0x7f5ee0076750 msgr2=0x7f5ee0100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 shutdown_connections 2024-04-20T11:36:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.576+0000 7f5ee6c00700 1 -- 172.21.15.117:0/291450436 wait complete. 2024-04-20T11:36:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:36:03.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.720+0000 7f3554200700 1 -- 172.21.15.117:0/1392474945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107df0 msgr2=0x7f354c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.720+0000 7f3554200700 1 --2- 172.21.15.117:0/1392474945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107df0 0x7f354c10be40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3538009910 tx=0x7f3538009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:03.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.720+0000 7f3554200700 1 -- 172.21.15.117:0/1392474945 shutdown_connections 2024-04-20T11:36:03.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.720+0000 7f3554200700 1 --2- 172.21.15.117:0/1392474945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107df0 0x7f354c10be40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.720+0000 7f3554200700 1 --2- 172.21.15.117:0/1392474945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f354c107440 0x7f354c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.721+0000 7f3554200700 1 -- 172.21.15.117:0/1392474945 >> 172.21.15.117:0/1392474945 conn(0x7f354c076a10 msgr2=0x7f354c078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:03.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.721+0000 7f3554200700 1 -- 172.21.15.117:0/1392474945 shutdown_connections 2024-04-20T11:36:03.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.721+0000 7f3554200700 1 -- 172.21.15.117:0/1392474945 wait complete. 2024-04-20T11:36:03.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.722+0000 7f3554200700 1 Processor -- start 2024-04-20T11:36:03.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.722+0000 7f3554200700 1 -- start start 2024-04-20T11:36:03.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.722+0000 7f3554200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107440 0x7f354c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.722+0000 7f3554200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f354c107df0 0x7f354c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.722+0000 7f3554200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c19dc60 con 0x7f354c107440 2024-04-20T11:36:03.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.722+0000 7f3554200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c19dda0 con 0x7f354c107df0 2024-04-20T11:36:03.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.723+0000 7f3551800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107440 0x7f354c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.723+0000 7f3551800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107440 0x7f354c19d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35172/0 (socket says 172.21.15.117:35172) 2024-04-20T11:36:03.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.723+0000 7f3550e00700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f354c107df0 0x7f354c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.723+0000 7f3551800700 1 -- 172.21.15.117:0/2048754227 learned_addr learned my addr 172.21.15.117:0/2048754227 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:03.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.723+0000 7f3551800700 1 -- 172.21.15.117:0/2048754227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f354c107df0 msgr2=0x7f354c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.723+0000 7f3551800700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f354c107df0 0x7f354c19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.724+0000 7f3551800700 1 -- 172.21.15.117:0/2048754227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35380095f0 con 0x7f354c107440 2024-04-20T11:36:03.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.724+0000 7f3551800700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107440 0x7f354c19d0d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f354000c930 tx=0x7f354000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:03.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.724+0000 7f354a000700 1 -- 172.21.15.117:0/2048754227 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3540007ad0 con 0x7f354c107440 2024-04-20T11:36:03.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.724+0000 7f354a000700 1 -- 172.21.15.117:0/2048754227 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f354000ce80 con 0x7f354c107440 2024-04-20T11:36:03.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.724+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c1a1b30 con 0x7f354c107440 2024-04-20T11:36:03.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.724+0000 7f354a000700 1 -- 172.21.15.117:0/2048754227 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3540018630 con 0x7f354c107440 2024-04-20T11:36:03.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.724+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c1a2050 con 0x7f354c107440 2024-04-20T11:36:03.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.725+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c10f790 con 0x7f354c107440 2024-04-20T11:36:03.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.726+0000 7f354a000700 1 -- 172.21.15.117:0/2048754227 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3540007c30 con 0x7f354c107440 2024-04-20T11:36:03.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.727+0000 7f354a000700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f353406ecd0 0x7f3534071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:03.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.727+0000 7f354a000700 1 -- 172.21.15.117:0/2048754227 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f354008eda0 con 0x7f354c107440 2024-04-20T11:36:03.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.728+0000 7f3550e00700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f353406ecd0 0x7f3534071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:03.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.728+0000 7f3550e00700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f353406ecd0 0x7f3534071190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3538009fd0 tx=0x7f353801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:03.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.732+0000 7f354a000700 1 -- 172.21.15.117:0/2048754227 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3540057c00 con 0x7f354c107440 2024-04-20T11:36:03.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.980+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f354c19e4f0 con 0x7f353406ecd0 2024-04-20T11:36:03.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.982+0000 7f354a000700 1 -- 172.21.15.117:0/2048754227 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f354c19e4f0 con 0x7f353406ecd0 2024-04-20T11:36:03.983 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:36:03.983 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:36:03.983 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:36:03.983 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:36:03.983 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:36:03.983 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:36:03.983 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:36:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:36:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:36:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:36:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:36:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:36:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:36:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:36:03.984 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:36:03.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f353406ecd0 msgr2=0x7f3534071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f353406ecd0 0x7f3534071190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3538009fd0 tx=0x7f353801a040 comp rx=0 tx=0).stop 2024-04-20T11:36:03.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107440 msgr2=0x7f354c19d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:03.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107440 0x7f354c19d0d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f354000c930 tx=0x7f354000cc40 comp rx=0 tx=0).stop 2024-04-20T11:36:03.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 shutdown_connections 2024-04-20T11:36:03.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f353406ecd0 0x7f3534071190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c107440 0x7f354c19d0d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 --2- 172.21.15.117:0/2048754227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f354c107df0 0x7f354c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:03.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 >> 172.21.15.117:0/2048754227 conn(0x7f354c076a10 msgr2=0x7f354c076df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:03.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 shutdown_connections 2024-04-20T11:36:03.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:03.985+0000 7f3554200700 1 -- 172.21.15.117:0/2048754227 wait complete. 2024-04-20T11:36:04.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.135+0000 7fcf8d800700 1 -- 172.21.15.117:0/78826499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 msgr2=0x7fcf8810ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.135+0000 7fcf8d800700 1 --2- 172.21.15.117:0/78826499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 0x7fcf8810ab50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009910 tx=0x7fcf74009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:04.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.135+0000 7fcf8d800700 1 -- 172.21.15.117:0/78826499 shutdown_connections 2024-04-20T11:36:04.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.135+0000 7fcf8d800700 1 --2- 172.21.15.117:0/78826499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 0x7fcf8810ab50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.135+0000 7fcf8d800700 1 --2- 172.21.15.117:0/78826499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf88101d40 0x7fcf88102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.135+0000 7fcf8d800700 1 -- 172.21.15.117:0/78826499 >> 172.21.15.117:0/78826499 conn(0x7fcf88076750 msgr2=0x7fcf880fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:04.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.135+0000 7fcf8d800700 1 -- 172.21.15.117:0/78826499 shutdown_connections 2024-04-20T11:36:04.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.135+0000 7fcf8d800700 1 -- 172.21.15.117:0/78826499 wait complete. 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.136+0000 7fcf8d800700 1 Processor -- start 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.136+0000 7fcf8d800700 1 -- start start 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.136+0000 7fcf8d800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf88101d40 0x7fcf880ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.136+0000 7fcf8d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 0x7fcf880ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.136+0000 7fcf8d800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88100420 con 0x7fcf88102660 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.136+0000 7fcf8d800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88100560 con 0x7fcf88101d40 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.137+0000 7fcf86000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 0x7fcf880ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.137+0000 7fcf86000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 0x7fcf880ffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56416/0 (socket says 172.21.15.117:56416) 2024-04-20T11:36:04.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.137+0000 7fcf86000700 1 -- 172.21.15.117:0/2236751639 learned_addr learned my addr 172.21.15.117:0/2236751639 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:04.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.137+0000 7fcf86a00700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf88101d40 0x7fcf880ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.137+0000 7fcf86000700 1 -- 172.21.15.117:0/2236751639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf88101d40 msgr2=0x7fcf880ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.137+0000 7fcf86000700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf88101d40 0x7fcf880ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.137+0000 7fcf86000700 1 -- 172.21.15.117:0/2236751639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740095f0 con 0x7fcf88102660 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.137+0000 7fcf86000700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 0x7fcf880ffee0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400b5c0 tx=0x7fcf7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.138+0000 7fcf7f400700 1 -- 172.21.15.117:0/2236751639 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf7401d070 con 0x7fcf88102660 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.138+0000 7fcf7f400700 1 -- 172.21.15.117:0/2236751639 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf74003d50 con 0x7fcf88102660 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.138+0000 7fcf7f400700 1 -- 172.21.15.117:0/2236751639 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf74017400 con 0x7fcf88102660 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.138+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf881007e0 con 0x7fcf88102660 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.138+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf881a2f80 con 0x7fcf88102660 2024-04-20T11:36:04.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.139+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf88068e50 con 0x7fcf88102660 2024-04-20T11:36:04.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.140+0000 7fcf7f400700 1 -- 172.21.15.117:0/2236751639 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcf74003890 con 0x7fcf88102660 2024-04-20T11:36:04.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.141+0000 7fcf7f400700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcf6c06ecc0 0x7fcf6c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:04.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.141+0000 7fcf7f400700 1 -- 172.21.15.117:0/2236751639 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcf74090230 con 0x7fcf88102660 2024-04-20T11:36:04.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.141+0000 7fcf86a00700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcf6c06ecc0 0x7fcf6c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:04.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.142+0000 7fcf86a00700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcf6c06ecc0 0x7fcf6c071180 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcf78005950 tx=0x7fcf78005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:04.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.146+0000 7fcf7f400700 1 -- 172.21.15.117:0/2236751639 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcf74059090 con 0x7fcf88102660 2024-04-20T11:36:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:04 smithi117 ceph-mon[99036]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:04 smithi117 ceph-mon[99036]: pgmap v709: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:04 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/912740570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:36:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:04 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/291450436' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:36:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:04 smithi173 ceph-mon[74165]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:04 smithi173 ceph-mon[74165]: pgmap v709: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:04 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/912740570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:36:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:04 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/291450436' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:36:04.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.459+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcf88100e80 con 0x7fcf88102660 2024-04-20T11:36:04.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.460+0000 7fcf7f400700 1 -- 172.21.15.117:0/2236751639 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fcf7405c750 con 0x7fcf88102660 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:36:04.461 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:36:04.462 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.462+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcf6c06ecc0 msgr2=0x7fcf6c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.462+0000 7fcf8d800700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcf6c06ecc0 0x7fcf6c071180 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcf78005950 tx=0x7fcf78005cf0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.462+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 msgr2=0x7fcf880ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.462+0000 7fcf8d800700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 0x7fcf880ffee0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400b5c0 tx=0x7fcf7400be20 comp rx=0 tx=0).stop 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.463+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 shutdown_connections 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.463+0000 7fcf8d800700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcf6c06ecc0 0x7fcf6c071180 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.463+0000 7fcf8d800700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf88101d40 0x7fcf880ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.463+0000 7fcf8d800700 1 --2- 172.21.15.117:0/2236751639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf88102660 0x7fcf880ffee0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.463+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 >> 172.21.15.117:0/2236751639 conn(0x7fcf88076750 msgr2=0x7fcf88103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.463+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 shutdown_connections 2024-04-20T11:36:04.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:04.463+0000 7fcf8d800700 1 -- 172.21.15.117:0/2236751639 wait complete. 2024-04-20T11:36:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:05 smithi117 ceph-mon[99036]: from='client.35310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:05 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2236751639' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:36:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:05 smithi173 ceph-mon[74165]: from='client.35310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:05 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2236751639' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:36:06.312 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:06 smithi173 ceph-mon[74165]: pgmap v710: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:06.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:06 smithi117 ceph-mon[99036]: pgmap v710: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:08.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:08 smithi117 ceph-mon[99036]: pgmap v711: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:08.418 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:08 smithi173 ceph-mon[74165]: pgmap v711: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:10.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:10 smithi117 ceph-mon[99036]: pgmap v712: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:10.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:10 smithi173 ceph-mon[74165]: pgmap v712: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:12.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:12 smithi117 ceph-mon[99036]: pgmap v713: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:12.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:12 smithi173 ceph-mon[74165]: pgmap v713: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:36:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:36:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:14 smithi117 ceph-mon[99036]: pgmap v714: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:14.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:14 smithi173 ceph-mon[74165]: pgmap v714: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:16.372 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:16 smithi173 ceph-mon[74165]: pgmap v715: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:16.388 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:16 smithi117 ceph-mon[99036]: pgmap v715: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:18.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:18 smithi117 ceph-mon[99036]: pgmap v716: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:18 smithi173 ceph-mon[74165]: pgmap v716: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:20.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:20 smithi117 ceph-mon[99036]: pgmap v717: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:20 smithi173 ceph-mon[74165]: pgmap v717: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:22.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:22 smithi117 ceph-mon[99036]: pgmap v718: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:22.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:22 smithi173 ceph-mon[74165]: pgmap v718: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:24.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:24 smithi117 ceph-mon[99036]: pgmap v719: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:24 smithi173 ceph-mon[74165]: pgmap v719: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:26.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:26 smithi117 ceph-mon[99036]: pgmap v720: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:26 smithi173 ceph-mon[74165]: pgmap v720: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:28 smithi173 ceph-mon[74165]: pgmap v721: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:36:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:28 smithi117 ceph-mon[99036]: pgmap v721: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:36:30.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:30 smithi173 ceph-mon[74165]: pgmap v722: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:30.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:30 smithi117 ceph-mon[99036]: pgmap v722: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:32.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:32 smithi173 ceph-mon[74165]: pgmap v723: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:32 smithi117 ceph-mon[99036]: pgmap v723: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:34.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.633+0000 7f8425200700 1 -- 172.21.15.117:0/2724651078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420108e10 msgr2=0x7f8420109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.633+0000 7f8425200700 1 --2- 172.21.15.117:0/2724651078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420108e10 0x7f8420109290 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f840c009910 tx=0x7f840c009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:34.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.633+0000 7f8425200700 1 -- 172.21.15.117:0/2724651078 shutdown_connections 2024-04-20T11:36:34.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.633+0000 7f8425200700 1 --2- 172.21.15.117:0/2724651078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420108e10 0x7f8420109290 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.634 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:34 smithi117 ceph-mon[99036]: pgmap v724: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.633+0000 7f8425200700 1 --2- 172.21.15.117:0/2724651078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8420107520 0x7f8420107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.633+0000 7f8425200700 1 -- 172.21.15.117:0/2724651078 >> 172.21.15.117:0/2724651078 conn(0x7f84200768f0 msgr2=0x7f8420078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.634+0000 7f8425200700 1 -- 172.21.15.117:0/2724651078 shutdown_connections 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.634+0000 7f8425200700 1 -- 172.21.15.117:0/2724651078 wait complete. 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f8425200700 1 Processor -- start 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f8425200700 1 -- start start 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f8425200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420107520 0x7f842019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f8425200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8420108e10 0x7f842019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f8425200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842019ddf0 con 0x7f8420107520 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f8425200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84201a1a70 con 0x7f8420108e10 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f841de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420107520 0x7f842019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f841de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420107520 0x7f842019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35966/0 (socket says 172.21.15.117:35966) 2024-04-20T11:36:34.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.635+0000 7f841de00700 1 -- 172.21.15.117:0/702559538 learned_addr learned my addr 172.21.15.117:0/702559538 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.636+0000 7f841d400700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8420108e10 0x7f842019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.636+0000 7f841de00700 1 -- 172.21.15.117:0/702559538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8420108e10 msgr2=0x7f842019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.636+0000 7f841de00700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8420108e10 0x7f842019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.636+0000 7f841de00700 1 -- 172.21.15.117:0/702559538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c0095f0 con 0x7f8420107520 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.636+0000 7f841de00700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420107520 0x7f842019d1d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f841000c8a0 tx=0x7f841000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.636+0000 7f8416a00700 1 -- 172.21.15.117:0/702559538 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f841000cea0 con 0x7f8420107520 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.637+0000 7f8416a00700 1 -- 172.21.15.117:0/702559538 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8410004830 con 0x7f8420107520 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.637+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84201a1d50 con 0x7f8420107520 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.637+0000 7f8416a00700 1 -- 172.21.15.117:0/702559538 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84100055b0 con 0x7f8420107520 2024-04-20T11:36:34.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.637+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84201a2320 con 0x7f8420107520 2024-04-20T11:36:34.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.638+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842010f870 con 0x7f8420107520 2024-04-20T11:36:34.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.639+0000 7f8416a00700 1 -- 172.21.15.117:0/702559538 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f84100049a0 con 0x7f8420107520 2024-04-20T11:36:34.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.640+0000 7f8416a00700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f840406ecd0 0x7f8404071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:34.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.640+0000 7f8416a00700 1 -- 172.21.15.117:0/702559538 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f841008eac0 con 0x7f8420107520 2024-04-20T11:36:34.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.640+0000 7f841d400700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f840406ecd0 0x7f8404071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:34.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.641+0000 7f841d400700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f840406ecd0 0x7f8404071190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f840c006010 tx=0x7f840c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:34.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.645+0000 7f8416a00700 1 -- 172.21.15.117:0/702559538 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8410057880 con 0x7f8420107520 2024-04-20T11:36:34.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:34 smithi173 ceph-mon[74165]: pgmap v724: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:34.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.892+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84201144c0 con 0x7f840406ecd0 2024-04-20T11:36:34.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.895+0000 7f8416a00700 1 -- 172.21.15.117:0/702559538 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f84201144c0 con 0x7f840406ecd0 2024-04-20T11:36:34.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.897+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f840406ecd0 msgr2=0x7f8404071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.897+0000 7f8425200700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f840406ecd0 0x7f8404071190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f840c006010 tx=0x7f840c00b540 comp rx=0 tx=0).stop 2024-04-20T11:36:34.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.897+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420107520 msgr2=0x7f842019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:34.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.897+0000 7f8425200700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420107520 0x7f842019d1d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f841000c8a0 tx=0x7f841000cc60 comp rx=0 tx=0).stop 2024-04-20T11:36:34.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.898+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 shutdown_connections 2024-04-20T11:36:34.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.898+0000 7f8425200700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f840406ecd0 0x7f8404071190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.898+0000 7f8425200700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8420107520 0x7f842019d1d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.898+0000 7f8425200700 1 --2- 172.21.15.117:0/702559538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8420108e10 0x7f842019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:34.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.898+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 >> 172.21.15.117:0/702559538 conn(0x7f84200768f0 msgr2=0x7f8420077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:34.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.898+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 shutdown_connections 2024-04-20T11:36:34.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:34.898+0000 7f8425200700 1 -- 172.21.15.117:0/702559538 wait complete. 2024-04-20T11:36:34.914 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:36:35.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.044+0000 7f1753200700 1 -- 172.21.15.117:0/3468506969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c1002e0 msgr2=0x7f174c10cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.044+0000 7f1753200700 1 --2- 172.21.15.117:0/3468506969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c1002e0 0x7f174c10cfa0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1734009910 tx=0x7f1734009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:35.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.045+0000 7f1753200700 1 -- 172.21.15.117:0/3468506969 shutdown_connections 2024-04-20T11:36:35.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.045+0000 7f1753200700 1 --2- 172.21.15.117:0/3468506969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c1002e0 0x7f174c10cfa0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.045+0000 7f1753200700 1 --2- 172.21.15.117:0/3468506969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c0ff930 0x7f174c0ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.045+0000 7f1753200700 1 -- 172.21.15.117:0/3468506969 >> 172.21.15.117:0/3468506969 conn(0x7f174c076750 msgr2=0x7f174c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:35.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.045+0000 7f1753200700 1 -- 172.21.15.117:0/3468506969 shutdown_connections 2024-04-20T11:36:35.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.045+0000 7f1753200700 1 -- 172.21.15.117:0/3468506969 wait complete. 2024-04-20T11:36:35.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.046+0000 7f1753200700 1 Processor -- start 2024-04-20T11:36:35.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.046+0000 7f1753200700 1 -- start start 2024-04-20T11:36:35.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.046+0000 7f1753200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c0ff930 0x7f174c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.046+0000 7f1753200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c1002e0 0x7f174c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.046+0000 7f1753200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c198b10 con 0x7f174c0ff930 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.046+0000 7f1753200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c198c80 con 0x7f174c1002e0 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.046+0000 7f174be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c0ff930 0x7f174c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.047+0000 7f174be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c0ff930 0x7f174c19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35984/0 (socket says 172.21.15.117:35984) 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.047+0000 7f174be00700 1 -- 172.21.15.117:0/3624775276 learned_addr learned my addr 172.21.15.117:0/3624775276 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.047+0000 7f174be00700 1 -- 172.21.15.117:0/3624775276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c1002e0 msgr2=0x7f174c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.047+0000 7f174be00700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c1002e0 0x7f174c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.047+0000 7f174be00700 1 -- 172.21.15.117:0/3624775276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17340095f0 con 0x7f174c0ff930 2024-04-20T11:36:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.047+0000 7f174be00700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c0ff930 0x7f174c19ea60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f173c00ada0 tx=0x7f173c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.047+0000 7f1749600700 1 -- 172.21.15.117:0/3624775276 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173c00f070 con 0x7f174c0ff930 2024-04-20T11:36:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.047+0000 7f1749600700 1 -- 172.21.15.117:0/3624775276 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f173c007bf0 con 0x7f174c0ff930 2024-04-20T11:36:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.048+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c198f60 con 0x7f174c0ff930 2024-04-20T11:36:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.048+0000 7f1749600700 1 -- 172.21.15.117:0/3624775276 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173c005690 con 0x7f174c0ff930 2024-04-20T11:36:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.048+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1994b0 con 0x7f174c0ff930 2024-04-20T11:36:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.049+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174c068df0 con 0x7f174c0ff930 2024-04-20T11:36:35.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.050+0000 7f1749600700 1 -- 172.21.15.117:0/3624775276 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f173c005850 con 0x7f174c0ff930 2024-04-20T11:36:35.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.051+0000 7f1749600700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f173006ecc0 0x7f1730071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.051+0000 7f1749600700 1 -- 172.21.15.117:0/3624775276 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f173c08e830 con 0x7f174c0ff930 2024-04-20T11:36:35.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.051+0000 7f1743400700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f173006ecc0 0x7f1730071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.052+0000 7f1743400700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f173006ecc0 0x7f1730071180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f173400b850 tx=0x7f173400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.055+0000 7f1749600700 1 -- 172.21.15.117:0/3624775276 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f173c0575f0 con 0x7f174c0ff930 2024-04-20T11:36:35.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.303+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f174c10ff40 con 0x7f173006ecc0 2024-04-20T11:36:35.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.306+0000 7f1749600700 1 -- 172.21.15.117:0/3624775276 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f174c10ff40 con 0x7f173006ecc0 2024-04-20T11:36:35.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f173006ecc0 msgr2=0x7f1730071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f173006ecc0 0x7f1730071180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f173400b850 tx=0x7f173400b540 comp rx=0 tx=0).stop 2024-04-20T11:36:35.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c0ff930 msgr2=0x7f174c19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c0ff930 0x7f174c19ea60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f173c00ada0 tx=0x7f173c00c330 comp rx=0 tx=0).stop 2024-04-20T11:36:35.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 shutdown_connections 2024-04-20T11:36:35.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f173006ecc0 0x7f1730071180 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c0ff930 0x7f174c19ea60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 --2- 172.21.15.117:0/3624775276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c1002e0 0x7f174c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.309+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 >> 172.21.15.117:0/3624775276 conn(0x7f174c076750 msgr2=0x7f174c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:35.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.310+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 shutdown_connections 2024-04-20T11:36:35.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.310+0000 7f1753200700 1 -- 172.21.15.117:0/3624775276 wait complete. 2024-04-20T11:36:35.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.466+0000 7f9661400700 1 -- 172.21.15.117:0/3924246597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c107520 msgr2=0x7f965c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.466+0000 7f9661400700 1 --2- 172.21.15.117:0/3924246597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c107520 0x7f965c107900 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f964c009910 tx=0x7f964c009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:35.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.466+0000 7f9661400700 1 -- 172.21.15.117:0/3924246597 shutdown_connections 2024-04-20T11:36:35.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.466+0000 7f9661400700 1 --2- 172.21.15.117:0/3924246597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f965c108e10 0x7f965c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.466+0000 7f9661400700 1 --2- 172.21.15.117:0/3924246597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c107520 0x7f965c107900 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.466+0000 7f9661400700 1 -- 172.21.15.117:0/3924246597 >> 172.21.15.117:0/3924246597 conn(0x7f965c0768f0 msgr2=0x7f965c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:35.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.466+0000 7f9661400700 1 -- 172.21.15.117:0/3924246597 shutdown_connections 2024-04-20T11:36:35.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.466+0000 7f9661400700 1 -- 172.21.15.117:0/3924246597 wait complete. 2024-04-20T11:36:35.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.467+0000 7f9661400700 1 Processor -- start 2024-04-20T11:36:35.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.467+0000 7f9661400700 1 -- start start 2024-04-20T11:36:35.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.468+0000 7f9661400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f965c107520 0x7f965c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.468+0000 7f9661400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c108e10 0x7f965c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.468+0000 7f9661400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c19ddd0 con 0x7f965c108e10 2024-04-20T11:36:35.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.468+0000 7f9661400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c1a1aa0 con 0x7f965c107520 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.468+0000 7f9653e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c108e10 0x7f965c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.468+0000 7f9653e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c108e10 0x7f965c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35992/0 (socket says 172.21.15.117:35992) 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.468+0000 7f9653e00700 1 -- 172.21.15.117:0/1643503126 learned_addr learned my addr 172.21.15.117:0/1643503126 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f965a800700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f965c107520 0x7f965c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f9653e00700 1 -- 172.21.15.117:0/1643503126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f965c107520 msgr2=0x7f965c19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f9653e00700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f965c107520 0x7f965c19d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f9653e00700 1 -- 172.21.15.117:0/1643503126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c0095f0 con 0x7f965c108e10 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f9653e00700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c108e10 0x7f965c19d6f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f964400c8a0 tx=0x7f964400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f9653400700 1 -- 172.21.15.117:0/1643503126 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964400cea0 con 0x7f965c108e10 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f9653400700 1 -- 172.21.15.117:0/1643503126 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9644004830 con 0x7f965c108e10 2024-04-20T11:36:35.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c1a1d80 con 0x7f965c108e10 2024-04-20T11:36:35.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.469+0000 7f9653400700 1 -- 172.21.15.117:0/1643503126 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96440055b0 con 0x7f965c108e10 2024-04-20T11:36:35.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.470+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c1a2350 con 0x7f965c108e10 2024-04-20T11:36:35.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.471+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c10f870 con 0x7f965c108e10 2024-04-20T11:36:35.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.472+0000 7f9653400700 1 -- 172.21.15.117:0/1643503126 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f96440049a0 con 0x7f965c108e10 2024-04-20T11:36:35.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.473+0000 7f9653400700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f964006ecd0 0x7f9640071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.473+0000 7f9653400700 1 -- 172.21.15.117:0/1643503126 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f964408eb20 con 0x7f965c108e10 2024-04-20T11:36:35.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.473+0000 7f965a800700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f964006ecd0 0x7f9640071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.474+0000 7f965a800700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f964006ecd0 0x7f9640071190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f964c006010 tx=0x7f964c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.477+0000 7f9653400700 1 -- 172.21.15.117:0/1643503126 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9644057900 con 0x7f965c108e10 2024-04-20T11:36:35.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.715+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f965c1144c0 con 0x7f964006ecd0 2024-04-20T11:36:35.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.746+0000 7f9653400700 1 -- 172.21.15.117:0/1643503126 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f965c1144c0 con 0x7f964006ecd0 2024-04-20T11:36:35.747 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:36:35.747 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (25m) 61s ago 39m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:36:35.747 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (39m) 61s ago 39m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:36:35.747 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (38m) 64s ago 38m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (21m) 61s ago 39m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (21m) 64s ago 37m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (24m) 61s ago 38m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (16m) 61s ago 31m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (17m) 61s ago 31m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (70s) 64s ago 31m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (16m) 64s ago 31m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (28m) 61s ago 40m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (26m) 64s ago 37m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (23m) 61s ago 41m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (22m) 64s ago 37m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (26m) 61s ago 39m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (26m) 64s ago 37m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:36:35.748 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (20m) 61s ago 36m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:36:35.749 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (20m) 61s ago 36m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:36:35.749 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (19m) 61s ago 35m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:36:35.749 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (19m) 64s ago 34m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:36:35.749 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (18m) 64s ago 34m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:36:35.749 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (17m) 64s ago 33m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:36:35.749 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (25m) 61s ago 38m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f964006ecd0 msgr2=0x7f9640071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f964006ecd0 0x7f9640071190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f964c006010 tx=0x7f964c00b540 comp rx=0 tx=0).stop 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c108e10 msgr2=0x7f965c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c108e10 0x7f965c19d6f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f964400c8a0 tx=0x7f964400cc60 comp rx=0 tx=0).stop 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 shutdown_connections 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f964006ecd0 0x7f9640071190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f965c107520 0x7f965c19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 --2- 172.21.15.117:0/1643503126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f965c108e10 0x7f965c19d6f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.749+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 >> 172.21.15.117:0/1643503126 conn(0x7f965c0768f0 msgr2=0x7f965c077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:35.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.750+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 shutdown_connections 2024-04-20T11:36:35.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.750+0000 7f9661400700 1 -- 172.21.15.117:0/1643503126 wait complete. 2024-04-20T11:36:35.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.893+0000 7f0032c00700 1 -- 172.21.15.117:0/1552876209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 msgr2=0x7f002c107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.893+0000 7f0032c00700 1 --2- 172.21.15.117:0/1552876209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 0x7f002c107a10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0018009910 tx=0x7f0018009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:35.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.894+0000 7f0032c00700 1 -- 172.21.15.117:0/1552876209 shutdown_connections 2024-04-20T11:36:35.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.894+0000 7f0032c00700 1 --2- 172.21.15.117:0/1552876209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 0x7f002c107a10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.894+0000 7f0032c00700 1 --2- 172.21.15.117:0/1552876209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f002c103010 0x7f002c1033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.894+0000 7f0032c00700 1 -- 172.21.15.117:0/1552876209 >> 172.21.15.117:0/1552876209 conn(0x7f002c0fe8c0 msgr2=0x7f002c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:35.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.894+0000 7f0032c00700 1 -- 172.21.15.117:0/1552876209 shutdown_connections 2024-04-20T11:36:35.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.894+0000 7f0032c00700 1 -- 172.21.15.117:0/1552876209 wait complete. 2024-04-20T11:36:35.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.895+0000 7f0032c00700 1 Processor -- start 2024-04-20T11:36:35.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.895+0000 7f0032c00700 1 -- start start 2024-04-20T11:36:35.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.895+0000 7f0032c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f002c103010 0x7f002c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.895+0000 7f0032c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 0x7f002c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.895+0000 7f0032c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c199a40 con 0x7f002c1039c0 2024-04-20T11:36:35.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.895+0000 7f0032c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c19d710 con 0x7f002c103010 2024-04-20T11:36:35.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 0x7f002c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 0x7f002c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36010/0 (socket says 172.21.15.117:36010) 2024-04-20T11:36:35.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002b400700 1 -- 172.21.15.117:0/971564865 learned_addr learned my addr 172.21.15.117:0/971564865 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:35.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002be00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f002c103010 0x7f002c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002b400700 1 -- 172.21.15.117:0/971564865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f002c103010 msgr2=0x7f002c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:35.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002b400700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f002c103010 0x7f002c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:35.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002b400700 1 -- 172.21.15.117:0/971564865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180095f0 con 0x7f002c1039c0 2024-04-20T11:36:35.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002be00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f002c103010 0x7f002c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:35.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.896+0000 7f002b400700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 0x7f002c199360 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0018005b40 tx=0x7f001800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.897+0000 7f0028c00700 1 -- 172.21.15.117:0/971564865 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001801d070 con 0x7f002c1039c0 2024-04-20T11:36:35.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.897+0000 7f0028c00700 1 -- 172.21.15.117:0/971564865 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f001800f460 con 0x7f002c1039c0 2024-04-20T11:36:35.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.897+0000 7f0028c00700 1 -- 172.21.15.117:0/971564865 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0018017630 con 0x7f002c1039c0 2024-04-20T11:36:35.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.897+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c19d990 con 0x7f002c1039c0 2024-04-20T11:36:35.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.897+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c19de80 con 0x7f002c1039c0 2024-04-20T11:36:35.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.898+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c10b360 con 0x7f002c1039c0 2024-04-20T11:36:35.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.899+0000 7f0028c00700 1 -- 172.21.15.117:0/971564865 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0018003a60 con 0x7f002c1039c0 2024-04-20T11:36:35.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.900+0000 7f0028c00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f001006ecc0 0x7f0010071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:35.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.900+0000 7f0028c00700 1 -- 172.21.15.117:0/971564865 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f00180902a0 con 0x7f002c1039c0 2024-04-20T11:36:35.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.900+0000 7f002be00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f001006ecc0 0x7f0010071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:35.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.901+0000 7f002be00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f001006ecc0 0x7f0010071180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f001c007900 tx=0x7f001c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:35.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:35.905+0000 7f0028c00700 1 -- 172.21.15.117:0/971564865 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0018059060 con 0x7f002c1039c0 2024-04-20T11:36:36.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.222+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f002c1044f0 con 0x7f002c1039c0 2024-04-20T11:36:36.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.223+0000 7f0028c00700 1 -- 172.21.15.117:0/971564865 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f001805c720 con 0x7f002c1039c0 2024-04-20T11:36:36.223 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:36:36.223 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:36:36.223 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:36:36.224 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:36:36.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.225+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f001006ecc0 msgr2=0x7f0010071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.225+0000 7f0032c00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f001006ecc0 0x7f0010071180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f001c007900 tx=0x7f001c008040 comp rx=0 tx=0).stop 2024-04-20T11:36:36.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.225+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 msgr2=0x7f002c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.225+0000 7f0032c00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 0x7f002c199360 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0018005b40 tx=0x7f001800be20 comp rx=0 tx=0).stop 2024-04-20T11:36:36.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.226+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 shutdown_connections 2024-04-20T11:36:36.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.226+0000 7f0032c00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f001006ecc0 0x7f0010071180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.226+0000 7f0032c00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f002c103010 0x7f002c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.226+0000 7f0032c00700 1 --2- 172.21.15.117:0/971564865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002c1039c0 0x7f002c199360 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.226+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 >> 172.21.15.117:0/971564865 conn(0x7f002c0fe8c0 msgr2=0x7f002c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:36.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.226+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 shutdown_connections 2024-04-20T11:36:36.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.226+0000 7f0032c00700 1 -- 172.21.15.117:0/971564865 wait complete. 2024-04-20T11:36:36.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.384+0000 7f50b6800700 1 -- 172.21.15.117:0/1450062748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0108e10 msgr2=0x7f50b0109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.384+0000 7f50b6800700 1 --2- 172.21.15.117:0/1450062748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0108e10 0x7f50b0109290 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f509c009910 tx=0x7f509c009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:36.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.384+0000 7f50b6800700 1 -- 172.21.15.117:0/1450062748 shutdown_connections 2024-04-20T11:36:36.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.384+0000 7f50b6800700 1 --2- 172.21.15.117:0/1450062748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0108e10 0x7f50b0109290 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.384+0000 7f50b6800700 1 --2- 172.21.15.117:0/1450062748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50b0107520 0x7f50b0107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.384+0000 7f50b6800700 1 -- 172.21.15.117:0/1450062748 >> 172.21.15.117:0/1450062748 conn(0x7f50b00768f0 msgr2=0x7f50b0078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:36.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.384+0000 7f50b6800700 1 -- 172.21.15.117:0/1450062748 shutdown_connections 2024-04-20T11:36:36.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.384+0000 7f50b6800700 1 -- 172.21.15.117:0/1450062748 wait complete. 2024-04-20T11:36:36.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.386+0000 7f50b6800700 1 Processor -- start 2024-04-20T11:36:36.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.386+0000 7f50b6800700 1 -- start start 2024-04-20T11:36:36.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.386+0000 7f50b6800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0107520 0x7f50b019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.386+0000 7f50b6800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50b0108e10 0x7f50b019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.386+0000 7f50b6800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b019dd60 con 0x7f50b0107520 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.386+0000 7f50b6800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b019dea0 con 0x7f50b0108e10 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.386+0000 7f50af400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0107520 0x7f50b019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.387+0000 7f50af400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0107520 0x7f50b019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36024/0 (socket says 172.21.15.117:36024) 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.387+0000 7f50af400700 1 -- 172.21.15.117:0/2946182940 learned_addr learned my addr 172.21.15.117:0/2946182940 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.387+0000 7f50aea00700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50b0108e10 0x7f50b019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.387+0000 7f50af400700 1 -- 172.21.15.117:0/2946182940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50b0108e10 msgr2=0x7f50b019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.387+0000 7f50af400700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50b0108e10 0x7f50b019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.387+0000 7f50af400700 1 -- 172.21.15.117:0/2946182940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0095f0 con 0x7f50b0107520 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.387+0000 7f50af400700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0107520 0x7f50b019d1d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c930 tx=0x7f50a000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.388+0000 7f50a7e00700 1 -- 172.21.15.117:0/2946182940 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a0007ad0 con 0x7f50b0107520 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.388+0000 7f50a7e00700 1 -- 172.21.15.117:0/2946182940 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50a0007c30 con 0x7f50b0107520 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.388+0000 7f50a7e00700 1 -- 172.21.15.117:0/2946182940 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a0018610 con 0x7f50b0107520 2024-04-20T11:36:36.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.388+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b01a1be0 con 0x7f50b0107520 2024-04-20T11:36:36.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.388+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b01a21b0 con 0x7f50b0107520 2024-04-20T11:36:36.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.389+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b010f870 con 0x7f50b0107520 2024-04-20T11:36:36.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.390+0000 7f50a7e00700 1 -- 172.21.15.117:0/2946182940 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f50a000f450 con 0x7f50b0107520 2024-04-20T11:36:36.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.391+0000 7f50a7e00700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f509406ecc0 0x7f5094071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.391+0000 7f50a7e00700 1 -- 172.21.15.117:0/2946182940 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f50a008efa0 con 0x7f50b0107520 2024-04-20T11:36:36.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.391+0000 7f50aea00700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f509406ecc0 0x7f5094071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.392+0000 7f50aea00700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f509406ecc0 0x7f5094071180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f509c005f50 tx=0x7f509c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.396+0000 7f50a7e00700 1 -- 172.21.15.117:0/2946182940 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f50a0057ea0 con 0x7f50b0107520 2024-04-20T11:36:36.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:36 smithi173 ceph-mon[74165]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:36.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:36 smithi173 ceph-mon[74165]: pgmap v725: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:36.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:36 smithi173 ceph-mon[74165]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:36 smithi117 ceph-mon[99036]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:36 smithi117 ceph-mon[99036]: pgmap v725: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:36 smithi117 ceph-mon[99036]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:36.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.668+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f50b019e530 con 0x7f50b0107520 2024-04-20T11:36:36.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.669+0000 7f50a7e00700 1 -- 172.21.15.117:0/2946182940 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f50a005b560 con 0x7f50b0107520 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:36:36.670 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:36:36.671 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:36:36.672 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:36:36.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f509406ecc0 msgr2=0x7f5094071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f509406ecc0 0x7f5094071180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f509c005f50 tx=0x7f509c01a040 comp rx=0 tx=0).stop 2024-04-20T11:36:36.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0107520 msgr2=0x7f50b019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0107520 0x7f50b019d1d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c930 tx=0x7f50a000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 shutdown_connections 2024-04-20T11:36:36.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f509406ecc0 0x7f5094071180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b0107520 0x7f50b019d1d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 --2- 172.21.15.117:0/2946182940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50b0108e10 0x7f50b019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.671+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 >> 172.21.15.117:0/2946182940 conn(0x7f50b00768f0 msgr2=0x7f50b0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:36.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.672+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 shutdown_connections 2024-04-20T11:36:36.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.672+0000 7f50b6800700 1 -- 172.21.15.117:0/2946182940 wait complete. 2024-04-20T11:36:36.674 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:36:36.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.828+0000 7f9058600700 1 -- 172.21.15.117:0/733281809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 msgr2=0x7f9050104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.828+0000 7f9058600700 1 --2- 172.21.15.117:0/733281809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 0x7f9050104e80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f903c009910 tx=0x7f903c009c20 comp rx=0 tx=0).stop 2024-04-20T11:36:36.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.828+0000 7f9058600700 1 -- 172.21.15.117:0/733281809 shutdown_connections 2024-04-20T11:36:36.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.828+0000 7f9058600700 1 --2- 172.21.15.117:0/733281809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 0x7f9050104e80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.828+0000 7f9058600700 1 --2- 172.21.15.117:0/733281809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9050103110 0x7f90501034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.828+0000 7f9058600700 1 -- 172.21.15.117:0/733281809 >> 172.21.15.117:0/733281809 conn(0x7f90500fe880 msgr2=0x7f9050100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:36.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.828+0000 7f9058600700 1 -- 172.21.15.117:0/733281809 shutdown_connections 2024-04-20T11:36:36.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.829+0000 7f9058600700 1 -- 172.21.15.117:0/733281809 wait complete. 2024-04-20T11:36:36.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.830+0000 7f9058600700 1 Processor -- start 2024-04-20T11:36:36.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.830+0000 7f9058600700 1 -- start start 2024-04-20T11:36:36.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.830+0000 7f9058600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9050103110 0x7f905019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.830+0000 7f9058600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 0x7f905019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.830+0000 7f9058600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905019f5b0 con 0x7f9050104a00 2024-04-20T11:36:36.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.830+0000 7f9058600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9050198b90 con 0x7f9050103110 2024-04-20T11:36:36.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 0x7f905019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 0x7f905019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36042/0 (socket says 172.21.15.117:36042) 2024-04-20T11:36:36.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055200700 1 -- 172.21.15.117:0/1704235506 learned_addr learned my addr 172.21.15.117:0/1704235506 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:36.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055200700 1 -- 172.21.15.117:0/1704235506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9050103110 msgr2=0x7f905019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055c00700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9050103110 0x7f905019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055200700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9050103110 0x7f905019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055200700 1 -- 172.21.15.117:0/1704235506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c0095f0 con 0x7f9050104a00 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055c00700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9050103110 0x7f905019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.831+0000 7f9055200700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 0x7f905019ef90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f903c0092e0 tx=0x7f903c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.832+0000 7f904e000700 1 -- 172.21.15.117:0/1704235506 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903c01d070 con 0x7f9050104a00 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.832+0000 7f904e000700 1 -- 172.21.15.117:0/1704235506 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f903c00f460 con 0x7f9050104a00 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.832+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9050198e10 con 0x7f9050104a00 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.832+0000 7f904e000700 1 -- 172.21.15.117:0/1704235506 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903c017630 con 0x7f9050104a00 2024-04-20T11:36:36.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.832+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9050199380 con 0x7f9050104a00 2024-04-20T11:36:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.833+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905010b480 con 0x7f9050104a00 2024-04-20T11:36:36.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.834+0000 7f904e000700 1 -- 172.21.15.117:0/1704235506 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f903c00f5d0 con 0x7f9050104a00 2024-04-20T11:36:36.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.835+0000 7f904e000700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f903806ecd0 0x7f9038071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:36.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.835+0000 7f904e000700 1 -- 172.21.15.117:0/1704235506 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f903c090340 con 0x7f9050104a00 2024-04-20T11:36:36.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.835+0000 7f9055c00700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f903806ecd0 0x7f9038071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:36.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.836+0000 7f9055c00700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f903806ecd0 0x7f9038071190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9044005fd0 tx=0x7f9044005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:36.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:36.840+0000 7f904e000700 1 -- 172.21.15.117:0/1704235506 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f903c05cc30 con 0x7f9050104a00 2024-04-20T11:36:37.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.085+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9050199fe0 con 0x7f903806ecd0 2024-04-20T11:36:37.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.088+0000 7f904e000700 1 -- 172.21.15.117:0/1704235506 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9050199fe0 con 0x7f903806ecd0 2024-04-20T11:36:37.088 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:36:37.088 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:36:37.088 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:36:37.088 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:36:37.089 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f903806ecd0 msgr2=0x7f9038071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f903806ecd0 0x7f9038071190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9044005fd0 tx=0x7f9044005dc0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 msgr2=0x7f905019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 0x7f905019ef90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f903c0092e0 tx=0x7f903c00be20 comp rx=0 tx=0).stop 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 shutdown_connections 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f903806ecd0 0x7f9038071190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9050103110 0x7f905019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 --2- 172.21.15.117:0/1704235506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9050104a00 0x7f905019ef90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 >> 172.21.15.117:0/1704235506 conn(0x7f90500fe880 msgr2=0x7f90500ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.090+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 shutdown_connections 2024-04-20T11:36:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.091+0000 7f9058600700 1 -- 172.21.15.117:0/1704235506 wait complete. 2024-04-20T11:36:37.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.241+0000 7f9ea4a00700 1 -- 172.21.15.117:0/2561387708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea01070b0 msgr2=0x7f9ea0107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.241+0000 7f9ea4a00700 1 --2- 172.21.15.117:0/2561387708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea01070b0 0x7f9ea0107510 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800b0a0 tx=0x7f9e8800b3b0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.241+0000 7f9ea4a00700 1 -- 172.21.15.117:0/2561387708 shutdown_connections 2024-04-20T11:36:37.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.241+0000 7f9ea4a00700 1 --2- 172.21.15.117:0/2561387708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea01070b0 0x7f9ea0107510 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.241+0000 7f9ea4a00700 1 --2- 172.21.15.117:0/2561387708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea010d0c0 0x7f9ea010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.241+0000 7f9ea4a00700 1 -- 172.21.15.117:0/2561387708 >> 172.21.15.117:0/2561387708 conn(0x7f9ea0076830 msgr2=0x7f9ea0078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:37.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.241+0000 7f9ea4a00700 1 -- 172.21.15.117:0/2561387708 shutdown_connections 2024-04-20T11:36:37.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.242+0000 7f9ea4a00700 1 -- 172.21.15.117:0/2561387708 wait complete. 2024-04-20T11:36:37.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9ea4a00700 1 Processor -- start 2024-04-20T11:36:37.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9ea4a00700 1 -- start start 2024-04-20T11:36:37.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9ea4a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea01070b0 0x7f9ea019d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9ea4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea010d0c0 0x7f9ea019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9ea4a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea019dd80 con 0x7f9ea010d0c0 2024-04-20T11:36:37.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9ea4a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01a1a50 con 0x7f9ea01070b0 2024-04-20T11:36:37.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9e96a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea010d0c0 0x7f9ea019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9e96a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea010d0c0 0x7f9ea019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36048/0 (socket says 172.21.15.117:36048) 2024-04-20T11:36:37.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.243+0000 7f9e96a00700 1 -- 172.21.15.117:0/144125150 learned_addr learned my addr 172.21.15.117:0/144125150 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:36:37.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9e96a00700 1 -- 172.21.15.117:0/144125150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea01070b0 msgr2=0x7f9ea019d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9e96a00700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea01070b0 0x7f9ea019d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9e96a00700 1 -- 172.21.15.117:0/144125150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e88009d00 con 0x7f9ea010d0c0 2024-04-20T11:36:37.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9e96a00700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea010d0c0 0x7f9ea019d6a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800b070 tx=0x7f9e88007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9e9cc00700 1 -- 172.21.15.117:0/144125150 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e8800e040 con 0x7f9ea010d0c0 2024-04-20T11:36:37.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9e9cc00700 1 -- 172.21.15.117:0/144125150 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e88003de0 con 0x7f9ea010d0c0 2024-04-20T11:36:37.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea01a1cd0 con 0x7f9ea010d0c0 2024-04-20T11:36:37.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea01a21c0 con 0x7f9ea010d0c0 2024-04-20T11:36:37.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.244+0000 7f9e9cc00700 1 -- 172.21.15.117:0/144125150 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e8801d6b0 con 0x7f9ea010d0c0 2024-04-20T11:36:37.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.245+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea010f5e0 con 0x7f9ea010d0c0 2024-04-20T11:36:37.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.246+0000 7f9e9cc00700 1 -- 172.21.15.117:0/144125150 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9e88004040 con 0x7f9ea010d0c0 2024-04-20T11:36:37.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.247+0000 7f9e9cc00700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e8406ecd0 0x7f9e84071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:36:37.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.247+0000 7f9e9cc00700 1 -- 172.21.15.117:0/144125150 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9e880906e0 con 0x7f9ea010d0c0 2024-04-20T11:36:37.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.248+0000 7f9e9f400700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e8406ecd0 0x7f9e84071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:36:37.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.248+0000 7f9e9f400700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e8406ecd0 0x7f9e84071190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9e900097b0 tx=0x7f9e90006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:36:37.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.252+0000 7f9e9cc00700 1 -- 172.21.15.117:0/144125150 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9e88059420 con 0x7f9ea010d0c0 2024-04-20T11:36:37.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:37 smithi117 ceph-mon[99036]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:37.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:37 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/971564865' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:36:37.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:37 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2946182940' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:36:37.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.567+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9ea0108770 con 0x7f9ea010d0c0 2024-04-20T11:36:37.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.568+0000 7f9e9cc00700 1 -- 172.21.15.117:0/144125150 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f9e8805cae0 con 0x7f9ea010d0c0 2024-04-20T11:36:37.569 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:36:37.569 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:36:37.569 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:36:37.569 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:36:37.569 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:36:37.569 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:36:37.570 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:36:37.570 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:36:37.570 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:36:37.570 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:36:37.570 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:36:37.570 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:36:37.570 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:36:37.571 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:36:37.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e8406ecd0 msgr2=0x7f9e84071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e8406ecd0 0x7f9e84071190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9e900097b0 tx=0x7f9e90006d20 comp rx=0 tx=0).stop 2024-04-20T11:36:37.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea010d0c0 msgr2=0x7f9ea019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:36:37.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea010d0c0 0x7f9ea019d6a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800b070 tx=0x7f9e88007bd0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 shutdown_connections 2024-04-20T11:36:37.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e8406ecd0 0x7f9e84071190 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea01070b0 0x7f9ea019d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 --2- 172.21.15.117:0/144125150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea010d0c0 0x7f9ea019d6a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:36:37.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 >> 172.21.15.117:0/144125150 conn(0x7f9ea0076830 msgr2=0x7f9ea0078a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:36:37.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 shutdown_connections 2024-04-20T11:36:37.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:36:37.571+0000 7f9ea4a00700 1 -- 172.21.15.117:0/144125150 wait complete. 2024-04-20T11:36:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:37 smithi173 ceph-mon[74165]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:37 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/971564865' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:36:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:37 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2946182940' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:36:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:38 smithi117 ceph-mon[99036]: pgmap v726: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:38 smithi117 ceph-mon[99036]: from='client.35338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:38 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/144125150' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:36:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:38 smithi173 ceph-mon[74165]: pgmap v726: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:38 smithi173 ceph-mon[74165]: from='client.35338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:36:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:38 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/144125150' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:36:40.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:40 smithi117 ceph-mon[99036]: pgmap v727: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:40 smithi173 ceph-mon[74165]: pgmap v727: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:42 smithi117 ceph-mon[99036]: pgmap v728: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:42.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:42 smithi173 ceph-mon[74165]: pgmap v728: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:43.535 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:36:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:36:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:44 smithi117 ceph-mon[99036]: pgmap v729: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:44.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:44 smithi173 ceph-mon[74165]: pgmap v729: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:46.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:46 smithi117 ceph-mon[99036]: pgmap v730: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:46 smithi173 ceph-mon[74165]: pgmap v730: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:48.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:48 smithi117 ceph-mon[99036]: pgmap v731: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:48.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:48 smithi173 ceph-mon[74165]: pgmap v731: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:50.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:50 smithi117 ceph-mon[99036]: pgmap v732: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:50.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:50 smithi173 ceph-mon[74165]: pgmap v732: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:52.509 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:52 smithi117 ceph-mon[99036]: pgmap v733: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:52 smithi173 ceph-mon[74165]: pgmap v733: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:54 smithi117 ceph-mon[99036]: pgmap v734: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:54 smithi173 ceph-mon[74165]: pgmap v734: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:56.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:56 smithi117 ceph-mon[99036]: pgmap v735: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:56.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:56 smithi173 ceph-mon[74165]: pgmap v735: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:58 smithi117 ceph-mon[99036]: pgmap v736: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:36:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:36:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:58 smithi173 ceph-mon[74165]: pgmap v736: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:36:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:36:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:37:00.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:00 smithi117 ceph-mon[99036]: pgmap v737: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:00 smithi173 ceph-mon[74165]: pgmap v737: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:02.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:02 smithi117 ceph-mon[99036]: pgmap v738: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:02 smithi173 ceph-mon[74165]: pgmap v738: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:04 smithi117 ceph-mon[99036]: pgmap v739: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:04 smithi173 ceph-mon[74165]: pgmap v739: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:06 smithi117 ceph-mon[99036]: pgmap v740: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:06 smithi173 ceph-mon[74165]: pgmap v740: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:07.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.740+0000 7fef9a800700 1 -- 172.21.15.117:0/97591319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 msgr2=0x7fef94101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:07.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.740+0000 7fef9a800700 1 --2- 172.21.15.117:0/97591319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 0x7fef94101200 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fef84009910 tx=0x7fef84009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:07.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.740+0000 7fef9a800700 1 -- 172.21.15.117:0/97591319 shutdown_connections 2024-04-20T11:37:07.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.740+0000 7fef9a800700 1 --2- 172.21.15.117:0/97591319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef941017d0 0x7fef94105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.740+0000 7fef9a800700 1 --2- 172.21.15.117:0/97591319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 0x7fef94101200 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.740+0000 7fef9a800700 1 -- 172.21.15.117:0/97591319 >> 172.21.15.117:0/97591319 conn(0x7fef940fc6b0 msgr2=0x7fef940fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:07.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.740+0000 7fef9a800700 1 -- 172.21.15.117:0/97591319 shutdown_connections 2024-04-20T11:37:07.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.741+0000 7fef9a800700 1 -- 172.21.15.117:0/97591319 wait complete. 2024-04-20T11:37:07.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.742+0000 7fef9a800700 1 Processor -- start 2024-04-20T11:37:07.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.742+0000 7fef9a800700 1 -- start start 2024-04-20T11:37:07.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.743+0000 7fef9a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 0x7fef9419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:07.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.743+0000 7fef9a800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef941017d0 0x7fef9419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:07.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.743+0000 7fef9a800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9419d370 con 0x7fef94100e20 2024-04-20T11:37:07.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.743+0000 7fef9a800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef94196950 con 0x7fef941017d0 2024-04-20T11:37:07.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.743+0000 7fef93400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 0x7fef9419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:07.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.743+0000 7fef93400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 0x7fef9419c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38896/0 (socket says 172.21.15.117:38896) 2024-04-20T11:37:07.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.743+0000 7fef93400700 1 -- 172.21.15.117:0/3299910303 learned_addr learned my addr 172.21.15.117:0/3299910303 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:07.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.744+0000 7fef8aa00700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef941017d0 0x7fef9419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:07.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.744+0000 7fef93400700 1 -- 172.21.15.117:0/3299910303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef941017d0 msgr2=0x7fef9419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:07.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.744+0000 7fef93400700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef941017d0 0x7fef9419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:07.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.744+0000 7fef93400700 1 -- 172.21.15.117:0/3299910303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840095f0 con 0x7fef94100e20 2024-04-20T11:37:07.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.744+0000 7fef8aa00700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef941017d0 0x7fef9419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:07.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.744+0000 7fef93400700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 0x7fef9419c810 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef840092e0 tx=0x7fef8400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:07.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.744+0000 7fef90c00700 1 -- 172.21.15.117:0/3299910303 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8401d070 con 0x7fef94100e20 2024-04-20T11:37:07.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.745+0000 7fef90c00700 1 -- 172.21.15.117:0/3299910303 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef840056e0 con 0x7fef94100e20 2024-04-20T11:37:07.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.745+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef94196bd0 con 0x7fef94100e20 2024-04-20T11:37:07.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.745+0000 7fef90c00700 1 -- 172.21.15.117:0/3299910303 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8400fe80 con 0x7fef94100e20 2024-04-20T11:37:07.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.745+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef941970c0 con 0x7fef94100e20 2024-04-20T11:37:07.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.746+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef941091c0 con 0x7fef94100e20 2024-04-20T11:37:07.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.747+0000 7fef90c00700 1 -- 172.21.15.117:0/3299910303 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fef8400f6f0 con 0x7fef94100e20 2024-04-20T11:37:07.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.748+0000 7fef90c00700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fef7806ed10 0x7fef780711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:07.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.748+0000 7fef90c00700 1 -- 172.21.15.117:0/3299910303 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fef84090340 con 0x7fef94100e20 2024-04-20T11:37:07.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.749+0000 7fef8aa00700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fef7806ed10 0x7fef780711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:07.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.750+0000 7fef8aa00700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fef7806ed10 0x7fef780711d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005fd0 tx=0x7fef7c005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:07.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:07.753+0000 7fef90c00700 1 -- 172.21.15.117:0/3299910303 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fef8405cbb0 con 0x7fef94100e20 2024-04-20T11:37:08.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.003+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef94197e80 con 0x7fef7806ed10 2024-04-20T11:37:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.006+0000 7fef90c00700 1 -- 172.21.15.117:0/3299910303 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fef94197e80 con 0x7fef7806ed10 2024-04-20T11:37:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fef7806ed10 msgr2=0x7fef780711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fef7806ed10 0x7fef780711d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005fd0 tx=0x7fef7c005ee0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 msgr2=0x7fef9419c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 0x7fef9419c810 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef840092e0 tx=0x7fef8400be20 comp rx=0 tx=0).stop 2024-04-20T11:37:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 shutdown_connections 2024-04-20T11:37:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fef7806ed10 0x7fef780711d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef94100e20 0x7fef9419c810 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 --2- 172.21.15.117:0/3299910303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef941017d0 0x7fef9419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 >> 172.21.15.117:0/3299910303 conn(0x7fef940fc6b0 msgr2=0x7fef940fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 shutdown_connections 2024-04-20T11:37:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.009+0000 7fef9a800700 1 -- 172.21.15.117:0/3299910303 wait complete. 2024-04-20T11:37:08.026 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:37:08.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.171+0000 7f629be00700 1 -- 172.21.15.117:0/3799722217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c1037d0 msgr2=0x7f629c107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.171+0000 7f629be00700 1 --2- 172.21.15.117:0/3799722217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c1037d0 0x7f629c107820 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6284009910 tx=0x7f6284009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.171+0000 7f629be00700 1 -- 172.21.15.117:0/3799722217 shutdown_connections 2024-04-20T11:37:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.171+0000 7f629be00700 1 --2- 172.21.15.117:0/3799722217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c1037d0 0x7f629c107820 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.171+0000 7f629be00700 1 --2- 172.21.15.117:0/3799722217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f629c102e20 0x7f629c103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.171+0000 7f629be00700 1 -- 172.21.15.117:0/3799722217 >> 172.21.15.117:0/3799722217 conn(0x7f629c0fe6d0 msgr2=0x7f629c100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.171+0000 7f629be00700 1 -- 172.21.15.117:0/3799722217 shutdown_connections 2024-04-20T11:37:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.171+0000 7f629be00700 1 -- 172.21.15.117:0/3799722217 wait complete. 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.172+0000 7f629be00700 1 Processor -- start 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.172+0000 7f629be00700 1 -- start start 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.172+0000 7f629be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c102e20 0x7f629c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f629c1037d0 0x7f629c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629be00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c19f360 con 0x7f629c102e20 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629be00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c198940 con 0x7f629c1037d0 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c102e20 0x7f629c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c102e20 0x7f629c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38916/0 (socket says 172.21.15.117:38916) 2024-04-20T11:37:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629aa00700 1 -- 172.21.15.117:0/3781922716 learned_addr learned my addr 172.21.15.117:0/3781922716 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f6293e00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f629c1037d0 0x7f629c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629aa00700 1 -- 172.21.15.117:0/3781922716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f629c1037d0 msgr2=0x7f629c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629aa00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f629c1037d0 0x7f629c19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.173+0000 7f629aa00700 1 -- 172.21.15.117:0/3781922716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840095f0 con 0x7f629c102e20 2024-04-20T11:37:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.174+0000 7f629aa00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c102e20 0x7f629c19e800 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f628c00c8a0 tx=0x7f628c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.174+0000 7f6293400700 1 -- 172.21.15.117:0/3781922716 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f628c00cea0 con 0x7f629c102e20 2024-04-20T11:37:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.174+0000 7f6293400700 1 -- 172.21.15.117:0/3781922716 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f628c004830 con 0x7f629c102e20 2024-04-20T11:37:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.174+0000 7f6293400700 1 -- 172.21.15.117:0/3781922716 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f628c0055b0 con 0x7f629c102e20 2024-04-20T11:37:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.174+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c198c20 con 0x7f629c102e20 2024-04-20T11:37:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.174+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c199170 con 0x7f629c102e20 2024-04-20T11:37:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.175+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c10b220 con 0x7f629c102e20 2024-04-20T11:37:08.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.176+0000 7f6293400700 1 -- 172.21.15.117:0/3781922716 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f628c0049a0 con 0x7f629c102e20 2024-04-20T11:37:08.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.178+0000 7f6293400700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f628006ecc0 0x7f6280071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:08.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.178+0000 7f6293400700 1 -- 172.21.15.117:0/3781922716 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f628c08eac0 con 0x7f629c102e20 2024-04-20T11:37:08.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.178+0000 7f6293e00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f628006ecc0 0x7f6280071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:08.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.179+0000 7f6293e00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f628006ecc0 0x7f6280071180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6284006010 tx=0x7f628400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:08.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.183+0000 7f6293400700 1 -- 172.21.15.117:0/3781922716 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f628c05b3b0 con 0x7f629c102e20 2024-04-20T11:37:08.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.433+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f629c199ea0 con 0x7f628006ecc0 2024-04-20T11:37:08.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.436+0000 7f6293400700 1 -- 172.21.15.117:0/3781922716 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f629c199ea0 con 0x7f628006ecc0 2024-04-20T11:37:08.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f628006ecc0 msgr2=0x7f6280071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f628006ecc0 0x7f6280071180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6284006010 tx=0x7f628400b540 comp rx=0 tx=0).stop 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c102e20 msgr2=0x7f629c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c102e20 0x7f629c19e800 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f628c00c8a0 tx=0x7f628c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 shutdown_connections 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f628006ecc0 0x7f6280071180 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f629c102e20 0x7f629c19e800 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 --2- 172.21.15.117:0/3781922716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f629c1037d0 0x7f629c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 >> 172.21.15.117:0/3781922716 conn(0x7f629c0fe6d0 msgr2=0x7f629c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 shutdown_connections 2024-04-20T11:37:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.439+0000 7f629be00700 1 -- 172.21.15.117:0/3781922716 wait complete. 2024-04-20T11:37:08.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.599+0000 7fce14800700 1 -- 172.21.15.117:0/1041237646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c100e20 msgr2=0x7fce0c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.599+0000 7fce14800700 1 --2- 172.21.15.117:0/1041237646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c100e20 0x7fce0c101200 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fce00009910 tx=0x7fce00009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:08.599 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:08 smithi117 ceph-mon[99036]: pgmap v741: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:08.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.599+0000 7fce14800700 1 -- 172.21.15.117:0/1041237646 shutdown_connections 2024-04-20T11:37:08.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.599+0000 7fce14800700 1 --2- 172.21.15.117:0/1041237646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0c1017d0 0x7fce0c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.599+0000 7fce14800700 1 --2- 172.21.15.117:0/1041237646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c100e20 0x7fce0c101200 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.599+0000 7fce14800700 1 -- 172.21.15.117:0/1041237646 >> 172.21.15.117:0/1041237646 conn(0x7fce0c0fc6f0 msgr2=0x7fce0c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:08.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.599+0000 7fce14800700 1 -- 172.21.15.117:0/1041237646 shutdown_connections 2024-04-20T11:37:08.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.599+0000 7fce14800700 1 -- 172.21.15.117:0/1041237646 wait complete. 2024-04-20T11:37:08.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.601+0000 7fce14800700 1 Processor -- start 2024-04-20T11:37:08.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.601+0000 7fce14800700 1 -- start start 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.601+0000 7fce14800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0c100e20 0x7fce0c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.601+0000 7fce14800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c1017d0 0x7fce0c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce14800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c19d3a0 con 0x7fce0c1017d0 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce14800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c1969d0 con 0x7fce0c100e20 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce11400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c1017d0 0x7fce0c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce11400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c1017d0 0x7fce0c19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38936/0 (socket says 172.21.15.117:38936) 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce11400700 1 -- 172.21.15.117:0/1022668879 learned_addr learned my addr 172.21.15.117:0/1022668879 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce11e00700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0c100e20 0x7fce0c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce11400700 1 -- 172.21.15.117:0/1022668879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0c100e20 msgr2=0x7fce0c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce11400700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0c100e20 0x7fce0c19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce11400700 1 -- 172.21.15.117:0/1022668879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce000095f0 con 0x7fce0c1017d0 2024-04-20T11:37:08.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.602+0000 7fce11e00700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0c100e20 0x7fce0c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:37:08.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.603+0000 7fce11400700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c1017d0 0x7fce0c19cd80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c8f0 tx=0x7fcdfc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:08.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.603+0000 7fce06a00700 1 -- 172.21.15.117:0/1022668879 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc007a50 con 0x7fce0c1017d0 2024-04-20T11:37:08.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.603+0000 7fce06a00700 1 -- 172.21.15.117:0/1022668879 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdfc007bb0 con 0x7fce0c1017d0 2024-04-20T11:37:08.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.603+0000 7fce06a00700 1 -- 172.21.15.117:0/1022668879 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc018630 con 0x7fce0c1017d0 2024-04-20T11:37:08.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.603+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c196cb0 con 0x7fce0c1017d0 2024-04-20T11:37:08.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.603+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c197200 con 0x7fce0c1017d0 2024-04-20T11:37:08.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.604+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c1091c0 con 0x7fce0c1017d0 2024-04-20T11:37:08.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.606+0000 7fce06a00700 1 -- 172.21.15.117:0/1022668879 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcdfc007d20 con 0x7fce0c1017d0 2024-04-20T11:37:08.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.607+0000 7fce06a00700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdf406ecc0 0x7fcdf4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:08.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.607+0000 7fce06a00700 1 -- 172.21.15.117:0/1022668879 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcdfc08edd0 con 0x7fce0c1017d0 2024-04-20T11:37:08.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.607+0000 7fce11e00700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdf406ecc0 0x7fcdf4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:08.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.608+0000 7fce11e00700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdf406ecc0 0x7fcdf4071180 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fce00005b40 tx=0x7fce0001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:08.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.612+0000 7fce06a00700 1 -- 172.21.15.117:0/1022668879 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcdfc05b6c0 con 0x7fce0c1017d0 2024-04-20T11:37:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:08 smithi173 ceph-mon[74165]: pgmap v741: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:08.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.844+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fce0c10ddf0 con 0x7fcdf406ecc0 2024-04-20T11:37:08.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.857+0000 7fce06a00700 1 -- 172.21.15.117:0/1022668879 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fce0c10ddf0 con 0x7fcdf406ecc0 2024-04-20T11:37:08.857 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:37:08.857 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (25m) 94s ago 39m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:37:08.857 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (40m) 94s ago 39m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:37:08.857 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (38m) 97s ago 38m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:37:08.857 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (21m) 94s ago 39m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:37:08.857 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (21m) 97s ago 38m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:37:08.857 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (24m) 94s ago 39m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (17m) 94s ago 32m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (17m) 94s ago 32m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (103s) 97s ago 32m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (16m) 97s ago 32m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (28m) 94s ago 41m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (27m) 97s ago 38m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (23m) 94s ago 41m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (23m) 97s ago 38m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (26m) 94s ago 39m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (26m) 97s ago 38m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (21m) 94s ago 37m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (20m) 94s ago 36m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (20m) 94s ago 36m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:37:08.858 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (19m) 97s ago 35m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:37:08.859 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (18m) 97s ago 34m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:37:08.859 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (18m) 97s ago 34m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:37:08.859 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (26m) 94s ago 39m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:37:08.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.859+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdf406ecc0 msgr2=0x7fcdf4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.859+0000 7fce14800700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdf406ecc0 0x7fcdf4071180 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fce00005b40 tx=0x7fce0001a040 comp rx=0 tx=0).stop 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.859+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c1017d0 msgr2=0x7fce0c19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.859+0000 7fce14800700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c1017d0 0x7fce0c19cd80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c8f0 tx=0x7fcdfc00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.860+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 shutdown_connections 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.860+0000 7fce14800700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdf406ecc0 0x7fcdf4071180 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.860+0000 7fce14800700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce0c100e20 0x7fce0c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.860+0000 7fce14800700 1 --2- 172.21.15.117:0/1022668879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce0c1017d0 0x7fce0c19cd80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.860+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 >> 172.21.15.117:0/1022668879 conn(0x7fce0c0fc6f0 msgr2=0x7fce0c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.860+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 shutdown_connections 2024-04-20T11:37:08.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:08.860+0000 7fce14800700 1 -- 172.21.15.117:0/1022668879 wait complete. 2024-04-20T11:37:09.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.009+0000 7fa225a00700 1 -- 172.21.15.117:0/4036149989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2201070b0 msgr2=0x7fa220107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.009+0000 7fa225a00700 1 --2- 172.21.15.117:0/4036149989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2201070b0 0x7fa220107510 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009910 tx=0x7fa20c009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.009+0000 7fa225a00700 1 -- 172.21.15.117:0/4036149989 shutdown_connections 2024-04-20T11:37:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.009+0000 7fa225a00700 1 --2- 172.21.15.117:0/4036149989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2201070b0 0x7fa220107510 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.009+0000 7fa225a00700 1 --2- 172.21.15.117:0/4036149989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22010d0c0 0x7fa22010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.009+0000 7fa225a00700 1 -- 172.21.15.117:0/4036149989 >> 172.21.15.117:0/4036149989 conn(0x7fa220076830 msgr2=0x7fa220078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.010+0000 7fa225a00700 1 -- 172.21.15.117:0/4036149989 shutdown_connections 2024-04-20T11:37:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.010+0000 7fa225a00700 1 -- 172.21.15.117:0/4036149989 wait complete. 2024-04-20T11:37:09.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.011+0000 7fa225a00700 1 Processor -- start 2024-04-20T11:37:09.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.011+0000 7fa225a00700 1 -- start start 2024-04-20T11:37:09.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.011+0000 7fa225a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2201070b0 0x7fa22019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.011+0000 7fa225a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa22010d0c0 0x7fa22019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.011+0000 7fa225a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22019db80 con 0x7fa22010d0c0 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.011+0000 7fa225a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201a1850 con 0x7fa2201070b0 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.011+0000 7fa21f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa22010d0c0 0x7fa22019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.012+0000 7fa21f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa22010d0c0 0x7fa22019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38946/0 (socket says 172.21.15.117:38946) 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.012+0000 7fa21f400700 1 -- 172.21.15.117:0/3289397304 learned_addr learned my addr 172.21.15.117:0/3289397304 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.012+0000 7fa21fe00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2201070b0 0x7fa22019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.012+0000 7fa21f400700 1 -- 172.21.15.117:0/3289397304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2201070b0 msgr2=0x7fa22019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.012+0000 7fa21f400700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2201070b0 0x7fa22019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.012+0000 7fa21f400700 1 -- 172.21.15.117:0/3289397304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0095f0 con 0x7fa22010d0c0 2024-04-20T11:37:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.012+0000 7fa21f400700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa22010d0c0 0x7fa22019d4a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00bd70 tx=0x7fa20c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.013+0000 7fa21cc00700 1 -- 172.21.15.117:0/3289397304 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa20c01d070 con 0x7fa22010d0c0 2024-04-20T11:37:09.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.013+0000 7fa21cc00700 1 -- 172.21.15.117:0/3289397304 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa20c0056e0 con 0x7fa22010d0c0 2024-04-20T11:37:09.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.013+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2201a1ad0 con 0x7fa22010d0c0 2024-04-20T11:37:09.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.013+0000 7fa21cc00700 1 -- 172.21.15.117:0/3289397304 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa20c017400 con 0x7fa22010d0c0 2024-04-20T11:37:09.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.013+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2201a1fc0 con 0x7fa22010d0c0 2024-04-20T11:37:09.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.014+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22010f5c0 con 0x7fa22010d0c0 2024-04-20T11:37:09.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.015+0000 7fa21cc00700 1 -- 172.21.15.117:0/3289397304 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa20c003a60 con 0x7fa22010d0c0 2024-04-20T11:37:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.016+0000 7fa21cc00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20406ecc0 0x7fa204071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.016+0000 7fa21cc00700 1 -- 172.21.15.117:0/3289397304 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa20c090280 con 0x7fa22010d0c0 2024-04-20T11:37:09.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.016+0000 7fa21fe00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20406ecc0 0x7fa204071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.017+0000 7fa21fe00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20406ecc0 0x7fa204071180 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa210005950 tx=0x7fa2100058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.021+0000 7fa21cc00700 1 -- 172.21.15.117:0/3289397304 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa20c059040 con 0x7fa22010d0c0 2024-04-20T11:37:09.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.323+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa22019e270 con 0x7fa22010d0c0 2024-04-20T11:37:09.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.324+0000 7fa21cc00700 1 -- 172.21.15.117:0/3289397304 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa20c05c700 con 0x7fa22010d0c0 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:37:09.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:37:09.325 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:37:09.325 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:37:09.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:37:09.325 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:37:09.325 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:37:09.325 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:37:09.325 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20406ecc0 msgr2=0x7fa204071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20406ecc0 0x7fa204071180 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa210005950 tx=0x7fa2100058e0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa22010d0c0 msgr2=0x7fa22019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa22010d0c0 0x7fa22019d4a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00bd70 tx=0x7fa20c00be50 comp rx=0 tx=0).stop 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 shutdown_connections 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa20406ecc0 0x7fa204071180 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2201070b0 0x7fa22019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 --2- 172.21.15.117:0/3289397304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa22010d0c0 0x7fa22019d4a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 >> 172.21.15.117:0/3289397304 conn(0x7fa220076830 msgr2=0x7fa220078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 shutdown_connections 2024-04-20T11:37:09.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.327+0000 7fa225a00700 1 -- 172.21.15.117:0/3289397304 wait complete. 2024-04-20T11:37:09.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.485+0000 7fa496200700 1 -- 172.21.15.117:0/3752748728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 msgr2=0x7fa490104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.485+0000 7fa496200700 1 --2- 172.21.15.117:0/3752748728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 0x7fa490104920 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa478009910 tx=0x7fa478009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:09.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.485+0000 7fa496200700 1 -- 172.21.15.117:0/3752748728 shutdown_connections 2024-04-20T11:37:09.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.485+0000 7fa496200700 1 --2- 172.21.15.117:0/3752748728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 0x7fa490104920 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.485+0000 7fa496200700 1 --2- 172.21.15.117:0/3752748728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900ffb30 0x7fa4900fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.485+0000 7fa496200700 1 -- 172.21.15.117:0/3752748728 >> 172.21.15.117:0/3752748728 conn(0x7fa490076750 msgr2=0x7fa4900fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:09.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.486+0000 7fa496200700 1 -- 172.21.15.117:0/3752748728 shutdown_connections 2024-04-20T11:37:09.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.486+0000 7fa496200700 1 -- 172.21.15.117:0/3752748728 wait complete. 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.486+0000 7fa496200700 1 Processor -- start 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.487+0000 7fa496200700 1 -- start start 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.487+0000 7fa496200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900ffb30 0x7fa49019d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.487+0000 7fa496200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 0x7fa49019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.487+0000 7fa496200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49019de90 con 0x7fa490100450 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.487+0000 7fa496200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901a1b60 con 0x7fa4900ffb30 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.487+0000 7fa486a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 0x7fa49019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.487+0000 7fa486a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 0x7fa49019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38960/0 (socket says 172.21.15.117:38960) 2024-04-20T11:37:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.487+0000 7fa486a00700 1 -- 172.21.15.117:0/2979683187 learned_addr learned my addr 172.21.15.117:0/2979683187 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:09.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa48f400700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900ffb30 0x7fa49019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa486a00700 1 -- 172.21.15.117:0/2979683187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900ffb30 msgr2=0x7fa49019d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa486a00700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900ffb30 0x7fa49019d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa486a00700 1 -- 172.21.15.117:0/2979683187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4780095f0 con 0x7fa490100450 2024-04-20T11:37:09.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa486a00700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 0x7fa49019d7b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa478005b40 tx=0x7fa47800bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa48cc00700 1 -- 172.21.15.117:0/2979683187 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47801d070 con 0x7fa490100450 2024-04-20T11:37:09.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa48cc00700 1 -- 172.21.15.117:0/2979683187 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa478003d50 con 0x7fa490100450 2024-04-20T11:37:09.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa48cc00700 1 -- 172.21.15.117:0/2979683187 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47800fe90 con 0x7fa490100450 2024-04-20T11:37:09.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.488+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4901a1de0 con 0x7fa490100450 2024-04-20T11:37:09.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.489+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4901a22d0 con 0x7fa490100450 2024-04-20T11:37:09.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.490+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa490068e50 con 0x7fa490100450 2024-04-20T11:37:09.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.491+0000 7fa48cc00700 1 -- 172.21.15.117:0/2979683187 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa478003890 con 0x7fa490100450 2024-04-20T11:37:09.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.492+0000 7fa48cc00700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa47406ecc0 0x7fa474071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.492+0000 7fa48cc00700 1 -- 172.21.15.117:0/2979683187 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa4780901f0 con 0x7fa490100450 2024-04-20T11:37:09.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.492+0000 7fa48f400700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa47406ecc0 0x7fa474071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.493+0000 7fa48f400700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa47406ecc0 0x7fa474071180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa480007dc0 tx=0x7fa4800072f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.496+0000 7fa48cc00700 1 -- 172.21.15.117:0/2979683187 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa478059050 con 0x7fa490100450 2024-04-20T11:37:09.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:09 smithi117 ceph-mon[99036]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:09 smithi173 ceph-mon[74165]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:09.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.779+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa4901144c0 con 0x7fa490100450 2024-04-20T11:37:09.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.780+0000 7fa48cc00700 1 -- 172.21.15.117:0/2979683187 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fa47805c710 con 0x7fa490100450 2024-04-20T11:37:09.781 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:37:09.781 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:37:09.781 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:37:09.781 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:37:09.781 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:37:09.781 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:37:09.782 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:37:09.783 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:37:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa47406ecc0 msgr2=0x7fa474071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa47406ecc0 0x7fa474071180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa480007dc0 tx=0x7fa4800072f0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 msgr2=0x7fa49019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 0x7fa49019d7b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa478005b40 tx=0x7fa47800bc10 comp rx=0 tx=0).stop 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 shutdown_connections 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa47406ecc0 0x7fa474071180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900ffb30 0x7fa49019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 --2- 172.21.15.117:0/2979683187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa490100450 0x7fa49019d7b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.782+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 >> 172.21.15.117:0/2979683187 conn(0x7fa490076750 msgr2=0x7fa49010e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.783+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 shutdown_connections 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.783+0000 7fa496200700 1 -- 172.21.15.117:0/2979683187 wait complete. 2024-04-20T11:37:09.786 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:37:09.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.935+0000 7f4f2be00700 1 -- 172.21.15.117:0/131933650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24104a00 msgr2=0x7f4f24104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.935+0000 7f4f2be00700 1 --2- 172.21.15.117:0/131933650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24104a00 0x7f4f24104e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009910 tx=0x7f4f10009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:09.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.935+0000 7f4f2be00700 1 -- 172.21.15.117:0/131933650 shutdown_connections 2024-04-20T11:37:09.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.935+0000 7f4f2be00700 1 --2- 172.21.15.117:0/131933650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24104a00 0x7f4f24104e80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.935+0000 7f4f2be00700 1 --2- 172.21.15.117:0/131933650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24103110 0x7f4f241034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.935+0000 7f4f2be00700 1 -- 172.21.15.117:0/131933650 >> 172.21.15.117:0/131933650 conn(0x7f4f240fe880 msgr2=0x7f4f24100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:09.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.935+0000 7f4f2be00700 1 -- 172.21.15.117:0/131933650 shutdown_connections 2024-04-20T11:37:09.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.935+0000 7f4f2be00700 1 -- 172.21.15.117:0/131933650 wait complete. 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.936+0000 7f4f2be00700 1 Processor -- start 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.937+0000 7f4f2be00700 1 -- start start 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.937+0000 7f4f2be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24103110 0x7f4f24198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.937+0000 7f4f2be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24104a00 0x7f4f241993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.937+0000 7f4f2be00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24199ac0 con 0x7f4f24103110 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.937+0000 7f4f2be00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2419d740 con 0x7f4f24104a00 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.937+0000 7f4f29400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24103110 0x7f4f24198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.937+0000 7f4f29400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24103110 0x7f4f24198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38978/0 (socket says 172.21.15.117:38978) 2024-04-20T11:37:09.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.937+0000 7f4f29400700 1 -- 172.21.15.117:0/1896188822 learned_addr learned my addr 172.21.15.117:0/1896188822 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:09.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f28a00700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24104a00 0x7f4f241993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f29400700 1 -- 172.21.15.117:0/1896188822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24104a00 msgr2=0x7f4f241993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:09.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f29400700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24104a00 0x7f4f241993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:09.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f29400700 1 -- 172.21.15.117:0/1896188822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f100095f0 con 0x7f4f24103110 2024-04-20T11:37:09.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f29400700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24103110 0x7f4f24198ea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800b4b0 tx=0x7f4f1800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f22000700 1 -- 172.21.15.117:0/1896188822 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f18011800 con 0x7f4f24103110 2024-04-20T11:37:09.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f22000700 1 -- 172.21.15.117:0/1896188822 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f18011e20 con 0x7f4f24103110 2024-04-20T11:37:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f22000700 1 -- 172.21.15.117:0/1896188822 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f1800f2e0 con 0x7f4f24103110 2024-04-20T11:37:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.938+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2419da20 con 0x7f4f24103110 2024-04-20T11:37:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.939+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2419dfa0 con 0x7f4f24103110 2024-04-20T11:37:09.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.939+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2410b460 con 0x7f4f24103110 2024-04-20T11:37:09.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.941+0000 7f4f22000700 1 -- 172.21.15.117:0/1896188822 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4f18010380 con 0x7f4f24103110 2024-04-20T11:37:09.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.941+0000 7f4f22000700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4f0c06ecc0 0x7f4f0c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:09.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.942+0000 7f4f22000700 1 -- 172.21.15.117:0/1896188822 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f4f1808ea70 con 0x7f4f24103110 2024-04-20T11:37:09.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.942+0000 7f4f28a00700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4f0c06ecc0 0x7f4f0c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:09.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.942+0000 7f4f28a00700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4f0c06ecc0 0x7f4f0c071180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b5c0 tx=0x7f4f10005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:09.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:09.946+0000 7f4f22000700 1 -- 172.21.15.117:0/1896188822 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4f180577b0 con 0x7f4f24103110 2024-04-20T11:37:10.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.193+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f2419a160 con 0x7f4f0c06ecc0 2024-04-20T11:37:10.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.195+0000 7f4f22000700 1 -- 172.21.15.117:0/1896188822 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4f2419a160 con 0x7f4f0c06ecc0 2024-04-20T11:37:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:37:10.194 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:37:10.194 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:37:10.194 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:37:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:37:10.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.197+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4f0c06ecc0 msgr2=0x7f4f0c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:10.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.197+0000 7f4f2be00700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4f0c06ecc0 0x7f4f0c071180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b5c0 tx=0x7f4f10005960 comp rx=0 tx=0).stop 2024-04-20T11:37:10.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.197+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24103110 msgr2=0x7f4f24198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:10.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.197+0000 7f4f2be00700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24103110 0x7f4f24198ea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800b4b0 tx=0x7f4f1800b870 comp rx=0 tx=0).stop 2024-04-20T11:37:10.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.197+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 shutdown_connections 2024-04-20T11:37:10.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.198+0000 7f4f2be00700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4f0c06ecc0 0x7f4f0c071180 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.198+0000 7f4f2be00700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f24103110 0x7f4f24198ea0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.198+0000 7f4f2be00700 1 --2- 172.21.15.117:0/1896188822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24104a00 0x7f4f241993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.198+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 >> 172.21.15.117:0/1896188822 conn(0x7f4f240fe880 msgr2=0x7f4f240ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:10.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.198+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 shutdown_connections 2024-04-20T11:37:10.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.198+0000 7f4f2be00700 1 -- 172.21.15.117:0/1896188822 wait complete. 2024-04-20T11:37:10.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.353+0000 7f58abc00700 1 -- 172.21.15.117:0/3487077195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 msgr2=0x7f58a4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.353+0000 7f58abc00700 1 --2- 172.21.15.117:0/3487077195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 0x7f58a4109290 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5890009910 tx=0x7f5890009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.353+0000 7f58abc00700 1 -- 172.21.15.117:0/3487077195 shutdown_connections 2024-04-20T11:37:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.353+0000 7f58abc00700 1 --2- 172.21.15.117:0/3487077195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 0x7f58a4109290 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.353+0000 7f58abc00700 1 --2- 172.21.15.117:0/3487077195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58a4107520 0x7f58a4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.353+0000 7f58abc00700 1 -- 172.21.15.117:0/3487077195 >> 172.21.15.117:0/3487077195 conn(0x7f58a40768f0 msgr2=0x7f58a4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.353+0000 7f58abc00700 1 -- 172.21.15.117:0/3487077195 shutdown_connections 2024-04-20T11:37:10.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.353+0000 7f58abc00700 1 -- 172.21.15.117:0/3487077195 wait complete. 2024-04-20T11:37:10.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.354+0000 7f58abc00700 1 Processor -- start 2024-04-20T11:37:10.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.354+0000 7f58abc00700 1 -- start start 2024-04-20T11:37:10.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.354+0000 7f58abc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58a4107520 0x7f58a419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:10.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58abc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 0x7f58a419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:10.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58abc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a419ded0 con 0x7f58a4108e10 2024-04-20T11:37:10.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58abc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41a1ba0 con 0x7f58a4107520 2024-04-20T11:37:10.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58a3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 0x7f58a419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:10.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58a3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 0x7f58a419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39004/0 (socket says 172.21.15.117:39004) 2024-04-20T11:37:10.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58a3e00700 1 -- 172.21.15.117:0/290488328 learned_addr learned my addr 172.21.15.117:0/290488328 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58a9200700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58a4107520 0x7f58a419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58a3e00700 1 -- 172.21.15.117:0/290488328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58a4107520 msgr2=0x7f58a419d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58a3e00700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58a4107520 0x7f58a419d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58a3e00700 1 -- 172.21.15.117:0/290488328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900095f0 con 0x7f58a4108e10 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.355+0000 7f58a9200700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58a4107520 0x7f58a419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.356+0000 7f58a3e00700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 0x7f58a419d7f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f58900092e0 tx=0x7f589000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.356+0000 7f58a1600700 1 -- 172.21.15.117:0/290488328 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f589001d070 con 0x7f58a4108e10 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.356+0000 7f58a1600700 1 -- 172.21.15.117:0/290488328 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f589000f460 con 0x7f58a4108e10 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.356+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a41a1e20 con 0x7f58a4108e10 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.356+0000 7f58a1600700 1 -- 172.21.15.117:0/290488328 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5890017630 con 0x7f58a4108e10 2024-04-20T11:37:10.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.356+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a41a2390 con 0x7f58a4108e10 2024-04-20T11:37:10.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.357+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a410f890 con 0x7f58a4108e10 2024-04-20T11:37:10.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.358+0000 7f58a1600700 1 -- 172.21.15.117:0/290488328 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f589000f5d0 con 0x7f58a4108e10 2024-04-20T11:37:10.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.359+0000 7f58a1600700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f588c06ecd0 0x7f588c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:10.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.359+0000 7f58a1600700 1 -- 172.21.15.117:0/290488328 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f58900903e0 con 0x7f58a4108e10 2024-04-20T11:37:10.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.359+0000 7f58a9200700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f588c06ecd0 0x7f588c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:10.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.360+0000 7f58a9200700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f588c06ecd0 0x7f588c071190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5898007900 tx=0x7f5898008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:10.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.364+0000 7f58a1600700 1 -- 172.21.15.117:0/290488328 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5890059120 con 0x7f58a4108e10 2024-04-20T11:37:10.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:10 smithi117 ceph-mon[99036]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:10 smithi117 ceph-mon[99036]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:10 smithi117 ceph-mon[99036]: pgmap v742: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:10 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3289397304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:37:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:10 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2979683187' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:37:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:10 smithi173 ceph-mon[74165]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:10 smithi173 ceph-mon[74165]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:10 smithi173 ceph-mon[74165]: pgmap v742: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:10 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3289397304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:37:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:10 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2979683187' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:37:10.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.672+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f58a4114540 con 0x7f58a4108e10 2024-04-20T11:37:10.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.673+0000 7f58a1600700 1 -- 172.21.15.117:0/290488328 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f589005c7e0 con 0x7f58a4108e10 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:37:10.673 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:37:10.674 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:37:10.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f588c06ecd0 msgr2=0x7f588c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:10.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f588c06ecd0 0x7f588c071190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5898007900 tx=0x7f5898008040 comp rx=0 tx=0).stop 2024-04-20T11:37:10.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 msgr2=0x7f58a419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:10.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 0x7f58a419d7f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f58900092e0 tx=0x7f589000be20 comp rx=0 tx=0).stop 2024-04-20T11:37:10.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 shutdown_connections 2024-04-20T11:37:10.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f588c06ecd0 0x7f588c071190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58a4107520 0x7f58a419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 --2- 172.21.15.117:0/290488328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58a4108e10 0x7f58a419d7f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:10.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 >> 172.21.15.117:0/290488328 conn(0x7f58a40768f0 msgr2=0x7f58a40775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:10.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 shutdown_connections 2024-04-20T11:37:10.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:10.677+0000 7f58abc00700 1 -- 172.21.15.117:0/290488328 wait complete. 2024-04-20T11:37:11.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:11 smithi117 ceph-mon[99036]: from='client.35366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:11 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/290488328' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:37:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:11 smithi173 ceph-mon[74165]: from='client.35366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:11 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/290488328' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:37:12.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:12 smithi117 ceph-mon[99036]: pgmap v743: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:12 smithi173 ceph-mon[74165]: pgmap v743: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:37:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:37:14.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:14 smithi117 ceph-mon[99036]: pgmap v744: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:14 smithi173 ceph-mon[74165]: pgmap v744: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:16 smithi117 ceph-mon[99036]: pgmap v745: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:16.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:16 smithi173 ceph-mon[74165]: pgmap v745: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:18 smithi117 ceph-mon[99036]: pgmap v746: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:18 smithi173 ceph-mon[74165]: pgmap v746: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:20 smithi117 ceph-mon[99036]: pgmap v747: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:20 smithi173 ceph-mon[74165]: pgmap v747: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:22.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:22 smithi117 ceph-mon[99036]: pgmap v748: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:22.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:22 smithi173 ceph-mon[74165]: pgmap v748: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:24 smithi117 ceph-mon[99036]: pgmap v749: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:24 smithi173 ceph-mon[74165]: pgmap v749: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:26.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:26 smithi117 ceph-mon[99036]: pgmap v750: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:26 smithi173 ceph-mon[74165]: pgmap v750: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:28 smithi117 ceph-mon[99036]: pgmap v751: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:37:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:28 smithi173 ceph-mon[74165]: pgmap v751: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:37:30.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:30 smithi117 ceph-mon[99036]: pgmap v752: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:30.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:30 smithi173 ceph-mon[74165]: pgmap v752: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:32 smithi173 ceph-mon[74165]: pgmap v753: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:32.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:32 smithi117 ceph-mon[99036]: pgmap v753: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:34 smithi173 ceph-mon[74165]: pgmap v754: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:34 smithi117 ceph-mon[99036]: pgmap v754: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:36 smithi173 ceph-mon[74165]: pgmap v755: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:36.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:36 smithi117 ceph-mon[99036]: pgmap v755: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:38.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:38 smithi117 ceph-mon[99036]: pgmap v756: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:38 smithi173 ceph-mon[74165]: pgmap v756: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:39.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:39 smithi117 ceph-mon[99036]: pgmap v757: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:39.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:39 smithi173 ceph-mon[74165]: pgmap v757: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:40.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.846+0000 7fbacb800700 1 -- 172.21.15.117:0/2094339645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 msgr2=0x7fbac4105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:40.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.846+0000 7fbacb800700 1 --2- 172.21.15.117:0/2094339645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 0x7fbac4105820 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009910 tx=0x7fbab0009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:40.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.846+0000 7fbacb800700 1 -- 172.21.15.117:0/2094339645 shutdown_connections 2024-04-20T11:37:40.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.846+0000 7fbacb800700 1 --2- 172.21.15.117:0/2094339645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 0x7fbac4105820 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.846+0000 7fbacb800700 1 --2- 172.21.15.117:0/2094339645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac4100e20 0x7fbac4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.846+0000 7fbacb800700 1 -- 172.21.15.117:0/2094339645 >> 172.21.15.117:0/2094339645 conn(0x7fbac40fc6b0 msgr2=0x7fbac40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:40.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.846+0000 7fbacb800700 1 -- 172.21.15.117:0/2094339645 shutdown_connections 2024-04-20T11:37:40.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.846+0000 7fbacb800700 1 -- 172.21.15.117:0/2094339645 wait complete. 2024-04-20T11:37:40.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.847+0000 7fbacb800700 1 Processor -- start 2024-04-20T11:37:40.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.847+0000 7fbacb800700 1 -- start start 2024-04-20T11:37:40.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.848+0000 7fbacb800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac4100e20 0x7fbac419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.848+0000 7fbacb800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 0x7fbac419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.848+0000 7fbacb800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac419d520 con 0x7fbac41017d0 2024-04-20T11:37:40.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.848+0000 7fbacb800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac4196b00 con 0x7fbac4100e20 2024-04-20T11:37:40.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.848+0000 7fbac3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 0x7fbac419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.848+0000 7fbac3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 0x7fbac419cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41746/0 (socket says 172.21.15.117:41746) 2024-04-20T11:37:40.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.848+0000 7fbac3e00700 1 -- 172.21.15.117:0/4032812473 learned_addr learned my addr 172.21.15.117:0/4032812473 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.848+0000 7fbac8e00700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac4100e20 0x7fbac419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.849+0000 7fbac3e00700 1 -- 172.21.15.117:0/4032812473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac4100e20 msgr2=0x7fbac419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.849+0000 7fbac3e00700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac4100e20 0x7fbac419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.849+0000 7fbac3e00700 1 -- 172.21.15.117:0/4032812473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab00095f0 con 0x7fbac41017d0 2024-04-20T11:37:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.849+0000 7fbac8e00700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac4100e20 0x7fbac419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:37:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.849+0000 7fbac3e00700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 0x7fbac419cf00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005b40 tx=0x7fbab000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.850+0000 7fbac1600700 1 -- 172.21.15.117:0/4032812473 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab001d070 con 0x7fbac41017d0 2024-04-20T11:37:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.850+0000 7fbac1600700 1 -- 172.21.15.117:0/4032812473 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbab000f460 con 0x7fbac41017d0 2024-04-20T11:37:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.850+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac4196d80 con 0x7fbac41017d0 2024-04-20T11:37:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.850+0000 7fbac1600700 1 -- 172.21.15.117:0/4032812473 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab0017630 con 0x7fbac41017d0 2024-04-20T11:37:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.850+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac4197270 con 0x7fbac41017d0 2024-04-20T11:37:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.851+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac4102300 con 0x7fbac41017d0 2024-04-20T11:37:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.852+0000 7fbac1600700 1 -- 172.21.15.117:0/4032812473 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbab0003a60 con 0x7fbac41017d0 2024-04-20T11:37:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.853+0000 7fbac1600700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbaac06ecc0 0x7fbaac071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.853+0000 7fbac1600700 1 -- 172.21.15.117:0/4032812473 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbab0090290 con 0x7fbac41017d0 2024-04-20T11:37:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.853+0000 7fbac8e00700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbaac06ecc0 0x7fbaac071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.854+0000 7fbac8e00700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbaac06ecc0 0x7fbaac071180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbab80099d0 tx=0x7fbab8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:40.858+0000 7fbac1600700 1 -- 172.21.15.117:0/4032812473 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbab005cb80 con 0x7fbac41017d0 2024-04-20T11:37:41.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.099+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbac410de40 con 0x7fbaac06ecc0 2024-04-20T11:37:41.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.102+0000 7fbac1600700 1 -- 172.21.15.117:0/4032812473 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbac410de40 con 0x7fbaac06ecc0 2024-04-20T11:37:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.105+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbaac06ecc0 msgr2=0x7fbaac071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.105+0000 7fbacb800700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbaac06ecc0 0x7fbaac071180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbab80099d0 tx=0x7fbab8008040 comp rx=0 tx=0).stop 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.105+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 msgr2=0x7fbac419cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.106+0000 7fbacb800700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 0x7fbac419cf00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005b40 tx=0x7fbab000be20 comp rx=0 tx=0).stop 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.106+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 shutdown_connections 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.106+0000 7fbacb800700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbaac06ecc0 0x7fbaac071180 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.106+0000 7fbacb800700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac4100e20 0x7fbac419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.106+0000 7fbacb800700 1 --2- 172.21.15.117:0/4032812473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbac41017d0 0x7fbac419cf00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.106+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 >> 172.21.15.117:0/4032812473 conn(0x7fbac40fc6b0 msgr2=0x7fbac40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.106+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 shutdown_connections 2024-04-20T11:37:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.106+0000 7fbacb800700 1 -- 172.21.15.117:0/4032812473 wait complete. 2024-04-20T11:37:41.122 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:37:41.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.265+0000 7fb1b1400700 1 -- 172.21.15.117:0/151075131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 msgr2=0x7fb1ac105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.265+0000 7fb1b1400700 1 --2- 172.21.15.117:0/151075131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 0x7fb1ac105810 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb198009910 tx=0x7fb198009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.265+0000 7fb1b1400700 1 -- 172.21.15.117:0/151075131 shutdown_connections 2024-04-20T11:37:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.265+0000 7fb1b1400700 1 --2- 172.21.15.117:0/151075131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 0x7fb1ac105810 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.265+0000 7fb1b1400700 1 --2- 172.21.15.117:0/151075131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1ac100e10 0x7fb1ac1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.265+0000 7fb1b1400700 1 -- 172.21.15.117:0/151075131 >> 172.21.15.117:0/151075131 conn(0x7fb1ac0fc6a0 msgr2=0x7fb1ac0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.265+0000 7fb1b1400700 1 -- 172.21.15.117:0/151075131 shutdown_connections 2024-04-20T11:37:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.265+0000 7fb1b1400700 1 -- 172.21.15.117:0/151075131 wait complete. 2024-04-20T11:37:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.266+0000 7fb1b1400700 1 Processor -- start 2024-04-20T11:37:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.266+0000 7fb1b1400700 1 -- start start 2024-04-20T11:37:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.266+0000 7fb1b1400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1ac100e10 0x7fb1ac0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.266+0000 7fb1b1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 0x7fb1ac073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.266+0000 7fb1b1400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac073c60 con 0x7fb1ac1017c0 2024-04-20T11:37:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.266+0000 7fb1b1400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac073da0 con 0x7fb1ac100e10 2024-04-20T11:37:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.267+0000 7fb1a9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 0x7fb1ac073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.267+0000 7fb1a9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 0x7fb1ac073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41772/0 (socket says 172.21.15.117:41772) 2024-04-20T11:37:41.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.267+0000 7fb1a9e00700 1 -- 172.21.15.117:0/3272123437 learned_addr learned my addr 172.21.15.117:0/3272123437 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.267+0000 7fb1aa800700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1ac100e10 0x7fb1ac0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.267+0000 7fb1a9e00700 1 -- 172.21.15.117:0/3272123437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1ac100e10 msgr2=0x7fb1ac0731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.267+0000 7fb1a9e00700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1ac100e10 0x7fb1ac0731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.267+0000 7fb1a9e00700 1 -- 172.21.15.117:0/3272123437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1980095f0 con 0x7fb1ac1017c0 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.268+0000 7fb1a9e00700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 0x7fb1ac073720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb1980092e0 tx=0x7fb19800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.268+0000 7fb1a3400700 1 -- 172.21.15.117:0/3272123437 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb19801d070 con 0x7fb1ac1017c0 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.268+0000 7fb1a3400700 1 -- 172.21.15.117:0/3272123437 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb19800f460 con 0x7fb1ac1017c0 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.268+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac074020 con 0x7fb1ac1017c0 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.268+0000 7fb1a3400700 1 -- 172.21.15.117:0/3272123437 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb198017610 con 0x7fb1ac1017c0 2024-04-20T11:37:41.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.268+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac1a2f40 con 0x7fb1ac1017c0 2024-04-20T11:37:41.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.269+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac109210 con 0x7fb1ac1017c0 2024-04-20T11:37:41.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.270+0000 7fb1a3400700 1 -- 172.21.15.117:0/3272123437 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb19800f5d0 con 0x7fb1ac1017c0 2024-04-20T11:37:41.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.271+0000 7fb1a3400700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb19006ecd0 0x7fb190071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.271+0000 7fb1a3400700 1 -- 172.21.15.117:0/3272123437 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb1980903a0 con 0x7fb1ac1017c0 2024-04-20T11:37:41.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.272+0000 7fb1aa800700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb19006ecd0 0x7fb190071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.272+0000 7fb1aa800700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb19006ecd0 0x7fb190071190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb19c005fd0 tx=0x7fb19c005db0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:41.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.276+0000 7fb1a3400700 1 -- 172.21.15.117:0/3272123437 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb1980590e0 con 0x7fb1ac1017c0 2024-04-20T11:37:41.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.521+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1ac10ddb0 con 0x7fb19006ecd0 2024-04-20T11:37:41.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.523+0000 7fb1a3400700 1 -- 172.21.15.117:0/3272123437 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb1ac10ddb0 con 0x7fb19006ecd0 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb19006ecd0 msgr2=0x7fb190071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb19006ecd0 0x7fb190071190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb19c005fd0 tx=0x7fb19c005db0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 msgr2=0x7fb1ac073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 0x7fb1ac073720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb1980092e0 tx=0x7fb19800be20 comp rx=0 tx=0).stop 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 shutdown_connections 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb19006ecd0 0x7fb190071190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1ac100e10 0x7fb1ac0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 --2- 172.21.15.117:0/3272123437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1017c0 0x7fb1ac073720 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.526+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 >> 172.21.15.117:0/3272123437 conn(0x7fb1ac0fc6a0 msgr2=0x7fb1ac0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:41.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.527+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 shutdown_connections 2024-04-20T11:37:41.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.527+0000 7fb1b1400700 1 -- 172.21.15.117:0/3272123437 wait complete. 2024-04-20T11:37:41.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.682+0000 7f0735200700 1 -- 172.21.15.117:0/3255107545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 msgr2=0x7f0730107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.682+0000 7f0735200700 1 --2- 172.21.15.117:0/3255107545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 0x7f0730107a20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f071c009910 tx=0x7f071c009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:41.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.682+0000 7f0735200700 1 -- 172.21.15.117:0/3255107545 shutdown_connections 2024-04-20T11:37:41.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.682+0000 7f0735200700 1 --2- 172.21.15.117:0/3255107545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 0x7f0730107a20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.682+0000 7f0735200700 1 --2- 172.21.15.117:0/3255107545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0730103020 0x7f0730103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.682+0000 7f0735200700 1 -- 172.21.15.117:0/3255107545 >> 172.21.15.117:0/3255107545 conn(0x7f07300fe890 msgr2=0x7f0730100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:41.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.682+0000 7f0735200700 1 -- 172.21.15.117:0/3255107545 shutdown_connections 2024-04-20T11:37:41.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.682+0000 7f0735200700 1 -- 172.21.15.117:0/3255107545 wait complete. 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.683+0000 7f0735200700 1 Processor -- start 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.683+0000 7f0735200700 1 -- start start 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f0735200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0730103020 0x7f0730198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f0735200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 0x7f0730199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f0735200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0730199920 con 0x7f07301039d0 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f0735200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073019c540 con 0x7f0730103020 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f072d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 0x7f0730199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f072d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 0x7f0730199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41794/0 (socket says 172.21.15.117:41794) 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f072d400700 1 -- 172.21.15.117:0/637345953 learned_addr learned my addr 172.21.15.117:0/637345953 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f072de00700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0730103020 0x7f0730198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f072d400700 1 -- 172.21.15.117:0/637345953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0730103020 msgr2=0x7f0730198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f072d400700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0730103020 0x7f0730198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.684+0000 7f072d400700 1 -- 172.21.15.117:0/637345953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0095f0 con 0x7f07301039d0 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.685+0000 7f072d400700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 0x7f0730199290 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f071c0092e0 tx=0x7f071c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.685+0000 7f0726a00700 1 -- 172.21.15.117:0/637345953 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f071c01d070 con 0x7f07301039d0 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.685+0000 7f0726a00700 1 -- 172.21.15.117:0/637345953 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f071c00f460 con 0x7f07301039d0 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.685+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073019c7c0 con 0x7f07301039d0 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.685+0000 7f0726a00700 1 -- 172.21.15.117:0/637345953 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f071c017630 con 0x7f07301039d0 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.685+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073019ccb0 con 0x7f07301039d0 2024-04-20T11:37:41.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.686+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073010b420 con 0x7f07301039d0 2024-04-20T11:37:41.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.687+0000 7f0726a00700 1 -- 172.21.15.117:0/637345953 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f071c00f5d0 con 0x7f07301039d0 2024-04-20T11:37:41.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.689+0000 7f0726a00700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f071406ecc0 0x7f0714071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:41.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.689+0000 7f0726a00700 1 -- 172.21.15.117:0/637345953 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f071c090350 con 0x7f07301039d0 2024-04-20T11:37:41.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.689+0000 7f072de00700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f071406ecc0 0x7f0714071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:41.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.690+0000 7f072de00700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f071406ecc0 0x7f0714071180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0720005950 tx=0x7f0720005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:41.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.693+0000 7f0726a00700 1 -- 172.21.15.117:0/637345953 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f071c059090 con 0x7f07301039d0 2024-04-20T11:37:41.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.929+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f073010ffc0 con 0x7f071406ecc0 2024-04-20T11:37:41.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.960+0000 7f0726a00700 1 -- 172.21.15.117:0/637345953 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f073010ffc0 con 0x7f071406ecc0 2024-04-20T11:37:41.961 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:37:41.961 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (26m) 2m ago 40m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:37:41.961 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (40m) 2m ago 40m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:37:41.961 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (39m) 2m ago 39m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (22m) 2m ago 40m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (22m) 2m ago 39m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (25m) 2m ago 39m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (17m) 2m ago 32m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (18m) 2m ago 33m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (2m) 2m ago 32m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (17m) 2m ago 32m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (29m) 2m ago 42m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (27m) 2m ago 38m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (24m) 2m ago 42m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (23m) 2m ago 38m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:37:41.962 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (27m) 2m ago 40m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:37:41.963 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (27m) 2m ago 39m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:37:41.963 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (21m) 2m ago 37m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:37:41.963 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (21m) 2m ago 37m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:37:41.963 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (20m) 2m ago 36m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:37:41.963 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (20m) 2m ago 36m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:37:41.963 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (19m) 2m ago 35m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:37:41.963 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (18m) 2m ago 34m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:37:41.963 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (26m) 2m ago 39m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:37:41.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.963+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f071406ecc0 msgr2=0x7f0714071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.963+0000 7f0735200700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f071406ecc0 0x7f0714071180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0720005950 tx=0x7f0720005cf0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.963+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 msgr2=0x7f0730199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:41.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.963+0000 7f0735200700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 0x7f0730199290 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f071c0092e0 tx=0x7f071c00be20 comp rx=0 tx=0).stop 2024-04-20T11:37:41.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.964+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 shutdown_connections 2024-04-20T11:37:41.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.964+0000 7f0735200700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f071406ecc0 0x7f0714071180 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.964+0000 7f0735200700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0730103020 0x7f0730198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.964+0000 7f0735200700 1 --2- 172.21.15.117:0/637345953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07301039d0 0x7f0730199290 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:41.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.964+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 >> 172.21.15.117:0/637345953 conn(0x7f07300fe890 msgr2=0x7f07300fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:41.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.964+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 shutdown_connections 2024-04-20T11:37:41.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:41.964+0000 7f0735200700 1 -- 172.21.15.117:0/637345953 wait complete. 2024-04-20T11:37:42.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.113+0000 7f7f65a00700 1 -- 172.21.15.117:0/2740631778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60108e10 msgr2=0x7f7f60109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.113+0000 7f7f65a00700 1 --2- 172.21.15.117:0/2740631778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60108e10 0x7f7f60109290 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009910 tx=0x7f7f4c009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.114+0000 7f7f65a00700 1 -- 172.21.15.117:0/2740631778 shutdown_connections 2024-04-20T11:37:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.114+0000 7f7f65a00700 1 --2- 172.21.15.117:0/2740631778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60108e10 0x7f7f60109290 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.114+0000 7f7f65a00700 1 --2- 172.21.15.117:0/2740631778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60107520 0x7f7f60107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.114+0000 7f7f65a00700 1 -- 172.21.15.117:0/2740631778 >> 172.21.15.117:0/2740631778 conn(0x7f7f600768f0 msgr2=0x7f7f60078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.114+0000 7f7f65a00700 1 -- 172.21.15.117:0/2740631778 shutdown_connections 2024-04-20T11:37:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.114+0000 7f7f65a00700 1 -- 172.21.15.117:0/2740631778 wait complete. 2024-04-20T11:37:42.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.115+0000 7f7f65a00700 1 Processor -- start 2024-04-20T11:37:42.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.115+0000 7f7f65a00700 1 -- start start 2024-04-20T11:37:42.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.115+0000 7f7f65a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60107520 0x7f7f6019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f65a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60108e10 0x7f7f6019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f65a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6019ddf0 con 0x7f7f60107520 2024-04-20T11:37:42.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f65a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f601a1a70 con 0x7f7f60108e10 2024-04-20T11:37:42.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f5ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60107520 0x7f7f6019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f5ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60107520 0x7f7f6019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41812/0 (socket says 172.21.15.117:41812) 2024-04-20T11:37:42.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f5ea00700 1 -- 172.21.15.117:0/3287281511 learned_addr learned my addr 172.21.15.117:0/3287281511 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f5ea00700 1 -- 172.21.15.117:0/3287281511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60108e10 msgr2=0x7f7f6019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f5ea00700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60108e10 0x7f7f6019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f5ea00700 1 -- 172.21.15.117:0/3287281511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c0095f0 con 0x7f7f60107520 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.116+0000 7f7f5ea00700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60107520 0x7f7f6019d1d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000c8a0 tx=0x7f7f5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.117+0000 7f7f57400700 1 -- 172.21.15.117:0/3287281511 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f5000cea0 con 0x7f7f60107520 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.117+0000 7f7f57400700 1 -- 172.21.15.117:0/3287281511 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f50004830 con 0x7f7f60107520 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.117+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f601a1d50 con 0x7f7f60107520 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.117+0000 7f7f57400700 1 -- 172.21.15.117:0/3287281511 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f500055b0 con 0x7f7f60107520 2024-04-20T11:37:42.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.117+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f601a2320 con 0x7f7f60107520 2024-04-20T11:37:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.118+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6010f870 con 0x7f7f60107520 2024-04-20T11:37:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.119+0000 7f7f57400700 1 -- 172.21.15.117:0/3287281511 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7f500049a0 con 0x7f7f60107520 2024-04-20T11:37:42.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.120+0000 7f7f57400700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f4406ecd0 0x7f7f44071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.120+0000 7f7f57400700 1 -- 172.21.15.117:0/3287281511 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7f5008ebc0 con 0x7f7f60107520 2024-04-20T11:37:42.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.120+0000 7f7f5e000700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f4406ecd0 0x7f7f44071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.121+0000 7f7f5e000700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f4406ecd0 0x7f7f44071190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006010 tx=0x7f7f4c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.125+0000 7f7f57400700 1 -- 172.21.15.117:0/3287281511 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7f50057900 con 0x7f7f60107520 2024-04-20T11:37:42.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:42 smithi117 ceph-mon[99036]: pgmap v758: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:42.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:42 smithi117 ceph-mon[99036]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:42.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:42 smithi173 ceph-mon[74165]: pgmap v758: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:42.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:42 smithi173 ceph-mon[74165]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:42.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.424+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7f6019e490 con 0x7f7f60107520 2024-04-20T11:37:42.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.425+0000 7f7f57400700 1 -- 172.21.15.117:0/3287281511 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7f50004c50 con 0x7f7f60107520 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:37:42.426 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:37:42.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:37:42.427 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:37:42.427 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:37:42.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:37:42.427 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:37:42.427 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:37:42.427 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:37:42.427 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:37:42.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f4406ecd0 msgr2=0x7f7f44071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f4406ecd0 0x7f7f44071190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006010 tx=0x7f7f4c00b540 comp rx=0 tx=0).stop 2024-04-20T11:37:42.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60107520 msgr2=0x7f7f6019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60107520 0x7f7f6019d1d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000c8a0 tx=0x7f7f5000cc60 comp rx=0 tx=0).stop 2024-04-20T11:37:42.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 shutdown_connections 2024-04-20T11:37:42.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7f4406ecd0 0x7f7f44071190 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f60107520 0x7f7f6019d1d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 --2- 172.21.15.117:0/3287281511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60108e10 0x7f7f6019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 >> 172.21.15.117:0/3287281511 conn(0x7f7f600768f0 msgr2=0x7f7f60077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:42.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 shutdown_connections 2024-04-20T11:37:42.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.428+0000 7f7f65a00700 1 -- 172.21.15.117:0/3287281511 wait complete. 2024-04-20T11:37:42.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.578+0000 7f0f30800700 1 -- 172.21.15.117:0/2313936425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 msgr2=0x7f0f281011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.578+0000 7f0f30800700 1 --2- 172.21.15.117:0/2313936425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 0x7f0f281011f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009910 tx=0x7f0f1c009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:42.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.578+0000 7f0f30800700 1 -- 172.21.15.117:0/2313936425 shutdown_connections 2024-04-20T11:37:42.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.578+0000 7f0f30800700 1 --2- 172.21.15.117:0/2313936425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f281017c0 0x7f0f28105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.578+0000 7f0f30800700 1 --2- 172.21.15.117:0/2313936425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 0x7f0f281011f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.578+0000 7f0f30800700 1 -- 172.21.15.117:0/2313936425 >> 172.21.15.117:0/2313936425 conn(0x7f0f280fc6c0 msgr2=0x7f0f280feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:42.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.578+0000 7f0f30800700 1 -- 172.21.15.117:0/2313936425 shutdown_connections 2024-04-20T11:37:42.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.579+0000 7f0f30800700 1 -- 172.21.15.117:0/2313936425 wait complete. 2024-04-20T11:37:42.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.579+0000 7f0f30800700 1 Processor -- start 2024-04-20T11:37:42.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.580+0000 7f0f30800700 1 -- start start 2024-04-20T11:37:42.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.580+0000 7f0f30800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 0x7f0f28198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.580+0000 7f0f30800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f281017c0 0x7f0f281993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.580+0000 7f0f30800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28199ab0 con 0x7f0f28100e10 2024-04-20T11:37:42.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.580+0000 7f0f30800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2819d780 con 0x7f0f281017c0 2024-04-20T11:37:42.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.580+0000 7f0f2de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 0x7f0f28198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.580+0000 7f0f2de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 0x7f0f28198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41830/0 (socket says 172.21.15.117:41830) 2024-04-20T11:37:42.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.580+0000 7f0f2de00700 1 -- 172.21.15.117:0/2995514459 learned_addr learned my addr 172.21.15.117:0/2995514459 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:42.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f2de00700 1 -- 172.21.15.117:0/2995514459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f281017c0 msgr2=0x7f0f281993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f2de00700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f281017c0 0x7f0f281993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f2de00700 1 -- 172.21.15.117:0/2995514459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c0095f0 con 0x7f0f28100e10 2024-04-20T11:37:42.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f2de00700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 0x7f0f28198e90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0092e0 tx=0x7f0f1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f22a00700 1 -- 172.21.15.117:0/2995514459 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1c01d070 con 0x7f0f28100e10 2024-04-20T11:37:42.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f22a00700 1 -- 172.21.15.117:0/2995514459 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f1c00f460 con 0x7f0f28100e10 2024-04-20T11:37:42.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f22a00700 1 -- 172.21.15.117:0/2995514459 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1c017630 con 0x7f0f28100e10 2024-04-20T11:37:42.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2819da00 con 0x7f0f28100e10 2024-04-20T11:37:42.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.581+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2819def0 con 0x7f0f28100e10 2024-04-20T11:37:42.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.582+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28104f90 con 0x7f0f28100e10 2024-04-20T11:37:42.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.583+0000 7f0f22a00700 1 -- 172.21.15.117:0/2995514459 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0f1c00f5d0 con 0x7f0f28100e10 2024-04-20T11:37:42.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.584+0000 7f0f22a00700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f1006ecc0 0x7f0f10071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.584+0000 7f0f22a00700 1 -- 172.21.15.117:0/2995514459 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0f1c090340 con 0x7f0f28100e10 2024-04-20T11:37:42.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.585+0000 7f0f2d400700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f1006ecc0 0x7f0f10071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.585+0000 7f0f2d400700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f1006ecc0 0x7f0f10071180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0f2819a3f0 tx=0x7f0f18006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.589+0000 7f0f22a00700 1 -- 172.21.15.117:0/2995514459 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0f1c059100 con 0x7f0f28100e10 2024-04-20T11:37:42.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.840+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0f2819a1a0 con 0x7f0f28100e10 2024-04-20T11:37:42.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.841+0000 7f0f22a00700 1 -- 172.21.15.117:0/2995514459 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f0f1c05c7c0 con 0x7f0f28100e10 2024-04-20T11:37:42.842 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:37:42.842 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:37:42.842 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:37:42.843 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:37:42.844 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:37:42.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:37:42.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f1006ecc0 msgr2=0x7f0f10071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f1006ecc0 0x7f0f10071180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0f2819a3f0 tx=0x7f0f18006d20 comp rx=0 tx=0).stop 2024-04-20T11:37:42.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 msgr2=0x7f0f28198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 0x7f0f28198e90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0092e0 tx=0x7f0f1c00be20 comp rx=0 tx=0).stop 2024-04-20T11:37:42.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 shutdown_connections 2024-04-20T11:37:42.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0f1006ecc0 0x7f0f10071180 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f28100e10 0x7f0f28198e90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 --2- 172.21.15.117:0/2995514459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f281017c0 0x7f0f281993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.844+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 >> 172.21.15.117:0/2995514459 conn(0x7f0f280fc6c0 msgr2=0x7f0f280fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:42.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.845+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 shutdown_connections 2024-04-20T11:37:42.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.845+0000 7f0f30800700 1 -- 172.21.15.117:0/2995514459 wait complete. 2024-04-20T11:37:42.847 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:37:42.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.991+0000 7f3dbe400700 1 -- 172.21.15.117:0/176966643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8068570 msgr2=0x7f3db80689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.991+0000 7f3dbe400700 1 --2- 172.21.15.117:0/176966643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8068570 0x7f3db80689d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009910 tx=0x7f3da4009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:42.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.991+0000 7f3dbe400700 1 -- 172.21.15.117:0/176966643 shutdown_connections 2024-04-20T11:37:42.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.991+0000 7f3dbe400700 1 --2- 172.21.15.117:0/176966643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8068570 0x7f3db80689d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.991+0000 7f3dbe400700 1 --2- 172.21.15.117:0/176966643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db81069d0 0x7f3db8106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.991+0000 7f3dbe400700 1 -- 172.21.15.117:0/176966643 >> 172.21.15.117:0/176966643 conn(0x7f3db8075480 msgr2=0x7f3db8075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:42.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.991+0000 7f3dbe400700 1 -- 172.21.15.117:0/176966643 shutdown_connections 2024-04-20T11:37:42.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.991+0000 7f3dbe400700 1 -- 172.21.15.117:0/176966643 wait complete. 2024-04-20T11:37:42.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.993+0000 7f3dbe400700 1 Processor -- start 2024-04-20T11:37:42.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.993+0000 7f3dbe400700 1 -- start start 2024-04-20T11:37:42.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3dbe400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db8068570 0x7f3db819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3dbe400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db81069d0 0x7f3db819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3dbe400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db819d230 con 0x7f3db81069d0 2024-04-20T11:37:42.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3dbe400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81967f0 con 0x7f3db8068570 2024-04-20T11:37:42.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3db7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db81069d0 0x7f3db819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3db7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db81069d0 0x7f3db819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41850/0 (socket says 172.21.15.117:41850) 2024-04-20T11:37:42.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3db7e00700 1 -- 172.21.15.117:0/3845236507 learned_addr learned my addr 172.21.15.117:0/3845236507 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3dbd000700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db8068570 0x7f3db819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3db7e00700 1 -- 172.21.15.117:0/3845236507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db8068570 msgr2=0x7f3db819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3db7e00700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db8068570 0x7f3db819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.994+0000 7f3db7e00700 1 -- 172.21.15.117:0/3845236507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da40095f0 con 0x7f3db81069d0 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.995+0000 7f3db7e00700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db81069d0 0x7f3db819cba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3da400bd70 tx=0x7f3da400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.995+0000 7f3db5600700 1 -- 172.21.15.117:0/3845236507 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da401d070 con 0x7f3db81069d0 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.995+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db8196a70 con 0x7f3db81069d0 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.995+0000 7f3db5600700 1 -- 172.21.15.117:0/3845236507 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3da400f460 con 0x7f3db81069d0 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.995+0000 7f3db5600700 1 -- 172.21.15.117:0/3845236507 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da4017630 con 0x7f3db81069d0 2024-04-20T11:37:42.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.995+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db8196f60 con 0x7f3db81069d0 2024-04-20T11:37:42.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.996+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db8108f60 con 0x7f3db81069d0 2024-04-20T11:37:42.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.997+0000 7f3db5600700 1 -- 172.21.15.117:0/3845236507 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3da400f5d0 con 0x7f3db81069d0 2024-04-20T11:37:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.998+0000 7f3db5600700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3da006ecc0 0x7f3da0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.998+0000 7f3db5600700 1 -- 172.21.15.117:0/3845236507 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3da4090340 con 0x7f3db81069d0 2024-04-20T11:37:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.999+0000 7f3dbd000700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3da006ecc0 0x7f3da0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:42.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:42.999+0000 7f3dbd000700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3da006ecc0 0x7f3da0071180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0060b0 tx=0x7f3dac007b80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.003+0000 7f3db5600700 1 -- 172.21.15.117:0/3845236507 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3da405cc50 con 0x7f3db81069d0 2024-04-20T11:37:43.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:43 smithi117 ceph-mon[99036]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:43 smithi117 ceph-mon[99036]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:43 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3287281511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:37:43.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:43 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2995514459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:37:43.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.240+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3db8197d00 con 0x7f3da006ecc0 2024-04-20T11:37:43.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.246+0000 7f3db5600700 1 -- 172.21.15.117:0/3845236507 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3db8197d00 con 0x7f3da006ecc0 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:37:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:37:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:37:43.248 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:37:43.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.248+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3da006ecc0 msgr2=0x7f3da0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.248+0000 7f3dbe400700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3da006ecc0 0x7f3da0071180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0060b0 tx=0x7f3dac007b80 comp rx=0 tx=0).stop 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.248+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db81069d0 msgr2=0x7f3db819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.248+0000 7f3dbe400700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db81069d0 0x7f3db819cba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3da400bd70 tx=0x7f3da400be50 comp rx=0 tx=0).stop 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.249+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 shutdown_connections 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.249+0000 7f3dbe400700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3da006ecc0 0x7f3da0071180 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.249+0000 7f3dbe400700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db8068570 0x7f3db819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.249+0000 7f3dbe400700 1 --2- 172.21.15.117:0/3845236507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db81069d0 0x7f3db819cba0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.249+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 >> 172.21.15.117:0/3845236507 conn(0x7f3db8075480 msgr2=0x7f3db80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.249+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 shutdown_connections 2024-04-20T11:37:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.249+0000 7f3dbe400700 1 -- 172.21.15.117:0/3845236507 wait complete. 2024-04-20T11:37:43.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.392+0000 7f670a400700 1 -- 172.21.15.117:0/1577713005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67041017c0 msgr2=0x7f6704105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.392+0000 7f670a400700 1 --2- 172.21.15.117:0/1577713005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67041017c0 0x7f6704105810 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009910 tx=0x7f66f0009c20 comp rx=0 tx=0).stop 2024-04-20T11:37:43.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.392+0000 7f670a400700 1 -- 172.21.15.117:0/1577713005 shutdown_connections 2024-04-20T11:37:43.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.392+0000 7f670a400700 1 --2- 172.21.15.117:0/1577713005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67041017c0 0x7f6704105810 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.392+0000 7f670a400700 1 --2- 172.21.15.117:0/1577713005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6704100e10 0x7f67041011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.392+0000 7f670a400700 1 -- 172.21.15.117:0/1577713005 >> 172.21.15.117:0/1577713005 conn(0x7f67040fc6c0 msgr2=0x7f67040feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:43.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.392+0000 7f670a400700 1 -- 172.21.15.117:0/1577713005 shutdown_connections 2024-04-20T11:37:43.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.392+0000 7f670a400700 1 -- 172.21.15.117:0/1577713005 wait complete. 2024-04-20T11:37:43.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.393+0000 7f670a400700 1 Processor -- start 2024-04-20T11:37:43.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.394+0000 7f670a400700 1 -- start start 2024-04-20T11:37:43.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.394+0000 7f670a400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6704100e10 0x7f670419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.394+0000 7f670a400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67041017c0 0x7f670419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.394+0000 7f670a400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670419f5a0 con 0x7f6704100e10 2024-04-20T11:37:43.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.394+0000 7f670a400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6704198b20 con 0x7f67041017c0 2024-04-20T11:37:43.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.394+0000 7f6703400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6704100e10 0x7f670419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.394+0000 7f6703400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6704100e10 0x7f670419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41868/0 (socket says 172.21.15.117:41868) 2024-04-20T11:37:43.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.394+0000 7f6703400700 1 -- 172.21.15.117:0/3531237516 learned_addr learned my addr 172.21.15.117:0/3531237516 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:37:43.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.395+0000 7f6703400700 1 -- 172.21.15.117:0/3531237516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67041017c0 msgr2=0x7f670419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.395+0000 7f6703400700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67041017c0 0x7f670419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.395+0000 7f6703400700 1 -- 172.21.15.117:0/3531237516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f00095f0 con 0x7f6704100e10 2024-04-20T11:37:43.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.395+0000 7f6703400700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6704100e10 0x7f670419e980 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f66f400d8d0 tx=0x7f66f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.395+0000 7f66fbe00700 1 -- 172.21.15.117:0/3531237516 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66f4009940 con 0x7f6704100e10 2024-04-20T11:37:43.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.395+0000 7f66fbe00700 1 -- 172.21.15.117:0/3531237516 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66f4010460 con 0x7f6704100e10 2024-04-20T11:37:43.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.395+0000 7f66fbe00700 1 -- 172.21.15.117:0/3531237516 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66f400f5f0 con 0x7f6704100e10 2024-04-20T11:37:43.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.395+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6704198e00 con 0x7f6704100e10 2024-04-20T11:37:43.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.396+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6704199350 con 0x7f6704100e10 2024-04-20T11:37:43.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.397+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6704103f80 con 0x7f6704100e10 2024-04-20T11:37:43.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.398+0000 7f66fbe00700 1 -- 172.21.15.117:0/3531237516 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f66f40105d0 con 0x7f6704100e10 2024-04-20T11:37:43.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.399+0000 7f66fbe00700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66e806ecc0 0x7f66e8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:37:43.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.399+0000 7f66fbe00700 1 -- 172.21.15.117:0/3531237516 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f66f408ede0 con 0x7f6704100e10 2024-04-20T11:37:43.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.399+0000 7f6702a00700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66e806ecc0 0x7f66e8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:37:43.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.400+0000 7f6702a00700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66e806ecc0 0x7f66e8071180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009fd0 tx=0x7f66f000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:37:43.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.404+0000 7f66fbe00700 1 -- 172.21.15.117:0/3531237516 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f66f405b6d0 con 0x7f6704100e10 2024-04-20T11:37:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:43 smithi173 ceph-mon[74165]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:43 smithi173 ceph-mon[74165]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:43 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3287281511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:37:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:43 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2995514459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:37:43.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.700+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f67041022f0 con 0x7f6704100e10 2024-04-20T11:37:43.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.701+0000 7f66fbe00700 1 -- 172.21.15.117:0/3531237516 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f66f405b260 con 0x7f6704100e10 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:37:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:37:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:37:43.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66e806ecc0 msgr2=0x7f66e8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66e806ecc0 0x7f66e8071180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009fd0 tx=0x7f66f000b540 comp rx=0 tx=0).stop 2024-04-20T11:37:43.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6704100e10 msgr2=0x7f670419e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:37:43.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6704100e10 0x7f670419e980 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f66f400d8d0 tx=0x7f66f400dc90 comp rx=0 tx=0).stop 2024-04-20T11:37:43.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 shutdown_connections 2024-04-20T11:37:43.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f66e806ecc0 0x7f66e8071180 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6704100e10 0x7f670419e980 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 --2- 172.21.15.117:0/3531237516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67041017c0 0x7f670419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:37:43.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.703+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 >> 172.21.15.117:0/3531237516 conn(0x7f67040fc6c0 msgr2=0x7f67040fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:37:43.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.704+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 shutdown_connections 2024-04-20T11:37:43.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:37:43.704+0000 7f670a400700 1 -- 172.21.15.117:0/3531237516 wait complete. 2024-04-20T11:37:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:44 smithi117 ceph-mon[99036]: pgmap v759: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:37:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:44 smithi117 ceph-mon[99036]: from='client.35394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:44 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3531237516' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:37:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:44 smithi173 ceph-mon[74165]: pgmap v759: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:37:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:44 smithi173 ceph-mon[74165]: from='client.35394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:37:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:44 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3531237516' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:37:46.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:46 smithi117 ceph-mon[99036]: pgmap v760: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:46 smithi173 ceph-mon[74165]: pgmap v760: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:48.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:48 smithi117 ceph-mon[99036]: pgmap v761: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:48.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:48 smithi173 ceph-mon[74165]: pgmap v761: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:50.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:50 smithi117 ceph-mon[99036]: pgmap v762: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:50 smithi173 ceph-mon[74165]: pgmap v762: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:52.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:52 smithi117 ceph-mon[99036]: pgmap v763: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:52 smithi173 ceph-mon[74165]: pgmap v763: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:54.362 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:54 smithi117 ceph-mon[99036]: pgmap v764: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:54.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:54 smithi173 ceph-mon[74165]: pgmap v764: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:56 smithi117 ceph-mon[99036]: pgmap v765: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:56.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:56 smithi173 ceph-mon[74165]: pgmap v765: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:58.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:58 smithi117 ceph-mon[99036]: pgmap v766: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:58.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:37:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:37:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:58 smithi173 ceph-mon[74165]: pgmap v766: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:37:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:37:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:38:00.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:00 smithi117 ceph-mon[99036]: pgmap v767: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:00 smithi173 ceph-mon[74165]: pgmap v767: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:02.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:02 smithi117 ceph-mon[99036]: pgmap v768: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:02 smithi173 ceph-mon[74165]: pgmap v768: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:04 smithi117 ceph-mon[99036]: pgmap v769: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:04 smithi173 ceph-mon[74165]: pgmap v769: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:06.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:06 smithi117 ceph-mon[99036]: pgmap v770: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:06.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:06 smithi173 ceph-mon[74165]: pgmap v770: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:08.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:08 smithi117 ceph-mon[99036]: pgmap v771: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:08.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:08 smithi173 ceph-mon[74165]: pgmap v771: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:10.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:10 smithi117 ceph-mon[99036]: pgmap v772: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:10.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:10 smithi173 ceph-mon[74165]: pgmap v772: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:12.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:12 smithi117 ceph-mon[99036]: pgmap v773: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:12.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:12 smithi173 ceph-mon[74165]: pgmap v773: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:13.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:38:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:38:13.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.873+0000 7f8b38c00700 1 -- 172.21.15.117:0/10275987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 msgr2=0x7f8b34104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.873+0000 7f8b38c00700 1 --2- 172.21.15.117:0/10275987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 0x7f8b34104e80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009910 tx=0x7f8b20009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:13.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.874+0000 7f8b38c00700 1 -- 172.21.15.117:0/10275987 shutdown_connections 2024-04-20T11:38:13.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.874+0000 7f8b38c00700 1 --2- 172.21.15.117:0/10275987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 0x7f8b34104e80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.874+0000 7f8b38c00700 1 --2- 172.21.15.117:0/10275987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b34103110 0x7f8b341034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.874+0000 7f8b38c00700 1 -- 172.21.15.117:0/10275987 >> 172.21.15.117:0/10275987 conn(0x7f8b340fe880 msgr2=0x7f8b34100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:13.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.874+0000 7f8b38c00700 1 -- 172.21.15.117:0/10275987 shutdown_connections 2024-04-20T11:38:13.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.874+0000 7f8b38c00700 1 -- 172.21.15.117:0/10275987 wait complete. 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.875+0000 7f8b38c00700 1 Processor -- start 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.875+0000 7f8b38c00700 1 -- start start 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.875+0000 7f8b38c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b34103110 0x7f8b3419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b38c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 0x7f8b3419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b38c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3419f5b0 con 0x7f8b34104a00 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b38c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34198b90 con 0x7f8b34103110 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b31400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 0x7f8b3419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b31400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 0x7f8b3419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36172/0 (socket says 172.21.15.117:36172) 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b31400700 1 -- 172.21.15.117:0/1910256419 learned_addr learned my addr 172.21.15.117:0/1910256419 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b31e00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b34103110 0x7f8b3419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b31400700 1 -- 172.21.15.117:0/1910256419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b34103110 msgr2=0x7f8b3419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b31400700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b34103110 0x7f8b3419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:13.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b31400700 1 -- 172.21.15.117:0/1910256419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b200095f0 con 0x7f8b34104a00 2024-04-20T11:38:13.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.876+0000 7f8b31e00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b34103110 0x7f8b3419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:13.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.877+0000 7f8b31400700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 0x7f8b3419ef90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8b200092e0 tx=0x7f8b2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:13.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.877+0000 7f8b2aa00700 1 -- 172.21.15.117:0/1910256419 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b2001d070 con 0x7f8b34104a00 2024-04-20T11:38:13.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.877+0000 7f8b2aa00700 1 -- 172.21.15.117:0/1910256419 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b2000f460 con 0x7f8b34104a00 2024-04-20T11:38:13.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.877+0000 7f8b2aa00700 1 -- 172.21.15.117:0/1910256419 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b20017630 con 0x7f8b34104a00 2024-04-20T11:38:13.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.877+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b34198e10 con 0x7f8b34104a00 2024-04-20T11:38:13.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.877+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b34199380 con 0x7f8b34104a00 2024-04-20T11:38:13.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.878+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3410b480 con 0x7f8b34104a00 2024-04-20T11:38:13.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.879+0000 7f8b2aa00700 1 -- 172.21.15.117:0/1910256419 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8b2000f5d0 con 0x7f8b34104a00 2024-04-20T11:38:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.880+0000 7f8b2aa00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1806ecd0 0x7f8b18071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.880+0000 7f8b2aa00700 1 -- 172.21.15.117:0/1910256419 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8b20090350 con 0x7f8b34104a00 2024-04-20T11:38:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.881+0000 7f8b31e00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1806ecd0 0x7f8b18071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.881+0000 7f8b31e00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1806ecd0 0x7f8b18071190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8b24005fd0 tx=0x7f8b24005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:13.886+0000 7f8b2aa00700 1 -- 172.21.15.117:0/1910256419 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8b2005cc60 con 0x7f8b34104a00 2024-04-20T11:38:14.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.140+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b3404ec50 con 0x7f8b1806ecd0 2024-04-20T11:38:14.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.143+0000 7f8b2aa00700 1 -- 172.21.15.117:0/1910256419 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8b3404ec50 con 0x7f8b1806ecd0 2024-04-20T11:38:14.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1806ecd0 msgr2=0x7f8b18071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1806ecd0 0x7f8b18071190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8b24005fd0 tx=0x7f8b24005dc0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 msgr2=0x7f8b3419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 0x7f8b3419ef90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8b200092e0 tx=0x7f8b2000be20 comp rx=0 tx=0).stop 2024-04-20T11:38:14.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 shutdown_connections 2024-04-20T11:38:14.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1806ecd0 0x7f8b18071190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b34103110 0x7f8b3419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 --2- 172.21.15.117:0/1910256419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34104a00 0x7f8b3419ef90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.146+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 >> 172.21.15.117:0/1910256419 conn(0x7f8b340fe880 msgr2=0x7f8b340ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:14.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.147+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 shutdown_connections 2024-04-20T11:38:14.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.147+0000 7f8b38c00700 1 -- 172.21.15.117:0/1910256419 wait complete. 2024-04-20T11:38:14.165 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:38:14.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.308+0000 7f8549c00700 1 -- 172.21.15.117:0/4020704919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544108e10 msgr2=0x7f8544109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.308+0000 7f8549c00700 1 --2- 172.21.15.117:0/4020704919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544108e10 0x7f8544109290 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8530009910 tx=0x7f8530009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:14.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.308+0000 7f8549c00700 1 -- 172.21.15.117:0/4020704919 shutdown_connections 2024-04-20T11:38:14.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.308+0000 7f8549c00700 1 --2- 172.21.15.117:0/4020704919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544108e10 0x7f8544109290 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.308+0000 7f8549c00700 1 --2- 172.21.15.117:0/4020704919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544107520 0x7f8544107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.308+0000 7f8549c00700 1 -- 172.21.15.117:0/4020704919 >> 172.21.15.117:0/4020704919 conn(0x7f85440768f0 msgr2=0x7f8544078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:14.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.309+0000 7f8549c00700 1 -- 172.21.15.117:0/4020704919 shutdown_connections 2024-04-20T11:38:14.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.309+0000 7f8549c00700 1 -- 172.21.15.117:0/4020704919 wait complete. 2024-04-20T11:38:14.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.310+0000 7f8549c00700 1 Processor -- start 2024-04-20T11:38:14.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.310+0000 7f8549c00700 1 -- start start 2024-04-20T11:38:14.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.310+0000 7f8549c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544107520 0x7f854419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.310+0000 7f8549c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544108e10 0x7f854419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.310+0000 7f8549c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854419ddf0 con 0x7f8544107520 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.310+0000 7f8549c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85441a1a70 con 0x7f8544108e10 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.310+0000 7f8542a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544107520 0x7f854419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.311+0000 7f8542a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544107520 0x7f854419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36182/0 (socket says 172.21.15.117:36182) 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.311+0000 7f8542a00700 1 -- 172.21.15.117:0/988897817 learned_addr learned my addr 172.21.15.117:0/988897817 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.311+0000 7f8542a00700 1 -- 172.21.15.117:0/988897817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544108e10 msgr2=0x7f854419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.311+0000 7f8542000700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544108e10 0x7f854419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.311+0000 7f8542a00700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544108e10 0x7f854419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.311+0000 7f8542a00700 1 -- 172.21.15.117:0/988897817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85300095f0 con 0x7f8544107520 2024-04-20T11:38:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.311+0000 7f8542000700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544108e10 0x7f854419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.311+0000 7f8542a00700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544107520 0x7f854419d1d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f853400d8d0 tx=0x7f853400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.312+0000 7f853b400700 1 -- 172.21.15.117:0/988897817 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8534009940 con 0x7f8544107520 2024-04-20T11:38:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.312+0000 7f853b400700 1 -- 172.21.15.117:0/988897817 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8534010460 con 0x7f8544107520 2024-04-20T11:38:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.312+0000 7f853b400700 1 -- 172.21.15.117:0/988897817 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f853400f5f0 con 0x7f8544107520 2024-04-20T11:38:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.312+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85441a1d50 con 0x7f8544107520 2024-04-20T11:38:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.312+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85441a22d0 con 0x7f8544107520 2024-04-20T11:38:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.313+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854410f870 con 0x7f8544107520 2024-04-20T11:38:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.314+0000 7f853b400700 1 -- 172.21.15.117:0/988897817 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f85340105d0 con 0x7f8544107520 2024-04-20T11:38:14.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.315+0000 7f853b400700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f852806ecd0 0x7f8528071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.315+0000 7f853b400700 1 -- 172.21.15.117:0/988897817 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f853408eef0 con 0x7f8544107520 2024-04-20T11:38:14.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.315+0000 7f8542000700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f852806ecd0 0x7f8528071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.316+0000 7f8542000700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f852806ecd0 0x7f8528071190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f854419e6e0 tx=0x7f853000b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.320+0000 7f853b400700 1 -- 172.21.15.117:0/988897817 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8534057c30 con 0x7f8544107520 2024-04-20T11:38:14.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:14 smithi117 ceph-mon[99036]: pgmap v774: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:14 smithi173 ceph-mon[74165]: pgmap v774: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:14.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.567+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f854419e500 con 0x7f852806ecd0 2024-04-20T11:38:14.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.570+0000 7f853b400700 1 -- 172.21.15.117:0/988897817 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f854419e500 con 0x7f852806ecd0 2024-04-20T11:38:14.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f852806ecd0 msgr2=0x7f8528071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f852806ecd0 0x7f8528071190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f854419e6e0 tx=0x7f853000b560 comp rx=0 tx=0).stop 2024-04-20T11:38:14.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544107520 msgr2=0x7f854419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544107520 0x7f854419d1d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f853400d8d0 tx=0x7f853400dc90 comp rx=0 tx=0).stop 2024-04-20T11:38:14.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 shutdown_connections 2024-04-20T11:38:14.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f852806ecd0 0x7f8528071190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8544107520 0x7f854419d1d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 --2- 172.21.15.117:0/988897817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544108e10 0x7f854419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 >> 172.21.15.117:0/988897817 conn(0x7f85440768f0 msgr2=0x7f8544077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:14.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 shutdown_connections 2024-04-20T11:38:14.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.574+0000 7f8549c00700 1 -- 172.21.15.117:0/988897817 wait complete. 2024-04-20T11:38:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.733+0000 7f69c6e00700 1 -- 172.21.15.117:0/1042304029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0102710 msgr2=0x7f69c010ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.733+0000 7f69c6e00700 1 --2- 172.21.15.117:0/1042304029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0102710 0x7f69c010ac00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009910 tx=0x7f69ac009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.733+0000 7f69c6e00700 1 -- 172.21.15.117:0/1042304029 shutdown_connections 2024-04-20T11:38:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.733+0000 7f69c6e00700 1 --2- 172.21.15.117:0/1042304029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0102710 0x7f69c010ac00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.733+0000 7f69c6e00700 1 --2- 172.21.15.117:0/1042304029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69c0101df0 0x7f69c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.733+0000 7f69c6e00700 1 -- 172.21.15.117:0/1042304029 >> 172.21.15.117:0/1042304029 conn(0x7f69c00fb5c0 msgr2=0x7f69c00fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.733+0000 7f69c6e00700 1 -- 172.21.15.117:0/1042304029 shutdown_connections 2024-04-20T11:38:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.733+0000 7f69c6e00700 1 -- 172.21.15.117:0/1042304029 wait complete. 2024-04-20T11:38:14.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.734+0000 7f69c6e00700 1 Processor -- start 2024-04-20T11:38:14.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.734+0000 7f69c6e00700 1 -- start start 2024-04-20T11:38:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.735+0000 7f69c6e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0101df0 0x7f69c0196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.735+0000 7f69c6e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69c0102710 0x7f69c0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.735+0000 7f69c6e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c0197a30 con 0x7f69c0101df0 2024-04-20T11:38:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.735+0000 7f69c6e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c019b700 con 0x7f69c0102710 2024-04-20T11:38:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.735+0000 7f69bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0101df0 0x7f69c0196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.735+0000 7f69bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0101df0 0x7f69c0196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36198/0 (socket says 172.21.15.117:36198) 2024-04-20T11:38:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.735+0000 7f69bfe00700 1 -- 172.21.15.117:0/743174754 learned_addr learned my addr 172.21.15.117:0/743174754 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.735+0000 7f69bf400700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69c0102710 0x7f69c0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69bfe00700 1 -- 172.21.15.117:0/743174754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69c0102710 msgr2=0x7f69c0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69bfe00700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69c0102710 0x7f69c0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69bfe00700 1 -- 172.21.15.117:0/743174754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0095f0 con 0x7f69c0101df0 2024-04-20T11:38:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69bfe00700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0101df0 0x7f69c0196e10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f69b000d8d0 tx=0x7f69b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69bcc00700 1 -- 172.21.15.117:0/743174754 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69b0009940 con 0x7f69c0101df0 2024-04-20T11:38:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69bcc00700 1 -- 172.21.15.117:0/743174754 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69b0010460 con 0x7f69c0101df0 2024-04-20T11:38:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69bcc00700 1 -- 172.21.15.117:0/743174754 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69b000f5f0 con 0x7f69c0101df0 2024-04-20T11:38:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c019b9e0 con 0x7f69c0101df0 2024-04-20T11:38:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.736+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c019bf30 con 0x7f69c0101df0 2024-04-20T11:38:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.738+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c0069680 con 0x7f69c0101df0 2024-04-20T11:38:14.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.739+0000 7f69bcc00700 1 -- 172.21.15.117:0/743174754 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f69b00105d0 con 0x7f69c0101df0 2024-04-20T11:38:14.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.740+0000 7f69bcc00700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69a406ecc0 0x7f69a4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:14.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.740+0000 7f69bcc00700 1 -- 172.21.15.117:0/743174754 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f69b008ede0 con 0x7f69c0101df0 2024-04-20T11:38:14.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.740+0000 7f69bf400700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69a406ecc0 0x7f69a4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:14.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.741+0000 7f69bf400700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69a406ecc0 0x7f69a4071180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f69c0198370 tx=0x7f69ac01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:14.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.745+0000 7f69bcc00700 1 -- 172.21.15.117:0/743174754 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f69b0057ba0 con 0x7f69c0101df0 2024-04-20T11:38:14.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:14.984+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69c006a5e0 con 0x7f69a406ecc0 2024-04-20T11:38:15.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.008+0000 7f69bcc00700 1 -- 172.21.15.117:0/743174754 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f69c006a5e0 con 0x7f69a406ecc0 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (26m) 2m ago 40m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (41m) 2m ago 41m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (39m) 2m ago 39m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (23m) 2m ago 41m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (22m) 2m ago 39m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (26m) 2m ago 40m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (18m) 2m ago 33m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (18m) 2m ago 33m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:38:15.009 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (2m) 2m ago 33m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (18m) 2m ago 33m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (29m) 2m ago 42m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (28m) 2m ago 39m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (24m) 2m ago 42m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (24m) 2m ago 39m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (28m) 2m ago 40m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (27m) 2m ago 39m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (22m) 2m ago 38m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (21m) 2m ago 37m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (21m) 2m ago 37m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (20m) 2m ago 36m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:38:15.010 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (20m) 2m ago 35m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:38:15.011 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (19m) 2m ago 35m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:38:15.011 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (27m) 2m ago 40m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:38:15.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69a406ecc0 msgr2=0x7f69a4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69a406ecc0 0x7f69a4071180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f69c0198370 tx=0x7f69ac01a040 comp rx=0 tx=0).stop 2024-04-20T11:38:15.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0101df0 msgr2=0x7f69c0196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0101df0 0x7f69c0196e10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f69b000d8d0 tx=0x7f69b000dc90 comp rx=0 tx=0).stop 2024-04-20T11:38:15.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 shutdown_connections 2024-04-20T11:38:15.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69a406ecc0 0x7f69a4071180 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c0101df0 0x7f69c0196e10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 --2- 172.21.15.117:0/743174754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69c0102710 0x7f69c0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 >> 172.21.15.117:0/743174754 conn(0x7f69c00fb5c0 msgr2=0x7f69c0103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:15.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 shutdown_connections 2024-04-20T11:38:15.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.011+0000 7f69c6e00700 1 -- 172.21.15.117:0/743174754 wait complete. 2024-04-20T11:38:15.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.171+0000 7f01c6600700 1 -- 172.21.15.117:0/182389492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 msgr2=0x7f01c0111520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.171+0000 7f01c6600700 1 --2- 172.21.15.117:0/182389492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 0x7f01c0111520 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009910 tx=0x7f01ac009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:15.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.171+0000 7f01c6600700 1 -- 172.21.15.117:0/182389492 shutdown_connections 2024-04-20T11:38:15.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.171+0000 7f01c6600700 1 --2- 172.21.15.117:0/182389492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 0x7f01c0111520 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.171+0000 7f01c6600700 1 --2- 172.21.15.117:0/182389492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0075410 0x7f01c00757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.171+0000 7f01c6600700 1 -- 172.21.15.117:0/182389492 >> 172.21.15.117:0/182389492 conn(0x7f01c00fe8b0 msgr2=0x7f01c0100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:15.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.171+0000 7f01c6600700 1 -- 172.21.15.117:0/182389492 shutdown_connections 2024-04-20T11:38:15.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.171+0000 7f01c6600700 1 -- 172.21.15.117:0/182389492 wait complete. 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.172+0000 7f01c6600700 1 Processor -- start 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.173+0000 7f01c6600700 1 -- start start 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.173+0000 7f01c6600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0075410 0x7f01c019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.173+0000 7f01c6600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 0x7f01c019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.173+0000 7f01c6600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c019dc90 con 0x7f01c0075d30 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.173+0000 7f01c6600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c01a1960 con 0x7f01c0075410 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.173+0000 7f01bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 0x7f01c019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.173+0000 7f01bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 0x7f01c019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36214/0 (socket says 172.21.15.117:36214) 2024-04-20T11:38:15.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.173+0000 7f01bea00700 1 -- 172.21.15.117:0/620996279 learned_addr learned my addr 172.21.15.117:0/620996279 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:15.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.174+0000 7f01bf400700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0075410 0x7f01c019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.174+0000 7f01bea00700 1 -- 172.21.15.117:0/620996279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0075410 msgr2=0x7f01c019d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.174+0000 7f01bea00700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0075410 0x7f01c019d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.174+0000 7f01bea00700 1 -- 172.21.15.117:0/620996279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0095f0 con 0x7f01c0075d30 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.174+0000 7f01bea00700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 0x7f01c019d5b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f01ac006010 tx=0x7f01ac0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.174+0000 7f01b7e00700 1 -- 172.21.15.117:0/620996279 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01ac01d070 con 0x7f01c0075d30 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.174+0000 7f01b7e00700 1 -- 172.21.15.117:0/620996279 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01ac00bea0 con 0x7f01c0075d30 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.174+0000 7f01b7e00700 1 -- 172.21.15.117:0/620996279 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01ac00fdc0 con 0x7f01c0075d30 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.175+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c01a1c40 con 0x7f01c0075d30 2024-04-20T11:38:15.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.175+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c01a2190 con 0x7f01c0075d30 2024-04-20T11:38:15.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.176+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c010ec90 con 0x7f01c0075d30 2024-04-20T11:38:15.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.177+0000 7f01b7e00700 1 -- 172.21.15.117:0/620996279 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f01ac00f5f0 con 0x7f01c0075d30 2024-04-20T11:38:15.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.178+0000 7f01b7e00700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f01a406ecd0 0x7f01a4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.178+0000 7f01b7e00700 1 -- 172.21.15.117:0/620996279 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f01ac08ff70 con 0x7f01c0075d30 2024-04-20T11:38:15.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.178+0000 7f01bf400700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f01a406ecd0 0x7f01a4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.179+0000 7f01bf400700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f01a406ecd0 0x7f01a4071190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f01b0005fd0 tx=0x7f01b0005f60 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:15.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.182+0000 7f01b7e00700 1 -- 172.21.15.117:0/620996279 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f01ac058cb0 con 0x7f01c0075d30 2024-04-20T11:38:15.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:15 smithi117 ceph-mon[99036]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:15.410 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:15 smithi173 ceph-mon[74165]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:15.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.491+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f01c019e380 con 0x7f01c0075d30 2024-04-20T11:38:15.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.492+0000 7f01b7e00700 1 -- 172.21.15.117:0/620996279 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f01ac05c370 con 0x7f01c0075d30 2024-04-20T11:38:15.492 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:38:15.492 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:38:15.492 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:38:15.493 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:38:15.494 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:38:15.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.494+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f01a406ecd0 msgr2=0x7f01a4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.494+0000 7f01c6600700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f01a406ecd0 0x7f01a4071190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f01b0005fd0 tx=0x7f01b0005f60 comp rx=0 tx=0).stop 2024-04-20T11:38:15.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.494+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 msgr2=0x7f01c019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.494+0000 7f01c6600700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 0x7f01c019d5b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f01ac006010 tx=0x7f01ac0056a0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.495+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 shutdown_connections 2024-04-20T11:38:15.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.495+0000 7f01c6600700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f01a406ecd0 0x7f01a4071190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.495+0000 7f01c6600700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0075410 0x7f01c019d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.495+0000 7f01c6600700 1 --2- 172.21.15.117:0/620996279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01c0075d30 0x7f01c019d5b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.495+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 >> 172.21.15.117:0/620996279 conn(0x7f01c00fe8b0 msgr2=0x7f01c00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:15.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.495+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 shutdown_connections 2024-04-20T11:38:15.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.495+0000 7f01c6600700 1 -- 172.21.15.117:0/620996279 wait complete. 2024-04-20T11:38:15.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.645+0000 7fd465a00700 1 -- 172.21.15.117:0/4036001879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 msgr2=0x7fd46010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.645+0000 7fd465a00700 1 --2- 172.21.15.117:0/4036001879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 0x7fd46010d4a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd450009910 tx=0x7fd450009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:15.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.645+0000 7fd465a00700 1 -- 172.21.15.117:0/4036001879 shutdown_connections 2024-04-20T11:38:15.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.645+0000 7fd465a00700 1 --2- 172.21.15.117:0/4036001879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4601070b0 0x7fd460107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.645+0000 7fd465a00700 1 --2- 172.21.15.117:0/4036001879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 0x7fd46010d4a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.645+0000 7fd465a00700 1 -- 172.21.15.117:0/4036001879 >> 172.21.15.117:0/4036001879 conn(0x7fd460076830 msgr2=0x7fd460078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:15.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.645+0000 7fd465a00700 1 -- 172.21.15.117:0/4036001879 shutdown_connections 2024-04-20T11:38:15.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.645+0000 7fd465a00700 1 -- 172.21.15.117:0/4036001879 wait complete. 2024-04-20T11:38:15.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.646+0000 7fd465a00700 1 Processor -- start 2024-04-20T11:38:15.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.646+0000 7fd465a00700 1 -- start start 2024-04-20T11:38:15.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.646+0000 7fd465a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4601070b0 0x7fd46019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.646+0000 7fd465a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 0x7fd46019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.646+0000 7fd465a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46019db50 con 0x7fd46010d0c0 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.646+0000 7fd465a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4601a1820 con 0x7fd4601070b0 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 0x7fd46019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 0x7fd46019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36228/0 (socket says 172.21.15.117:36228) 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45f400700 1 -- 172.21.15.117:0/3972621341 learned_addr learned my addr 172.21.15.117:0/3972621341 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45fe00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4601070b0 0x7fd46019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45f400700 1 -- 172.21.15.117:0/3972621341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4601070b0 msgr2=0x7fd46019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45f400700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4601070b0 0x7fd46019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45f400700 1 -- 172.21.15.117:0/3972621341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500095f0 con 0x7fd46010d0c0 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45fe00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4601070b0 0x7fd46019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.647+0000 7fd45f400700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 0x7fd46019d470 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00d8d0 tx=0x7fd44c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.648+0000 7fd45cc00700 1 -- 172.21.15.117:0/3972621341 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd44c009940 con 0x7fd46010d0c0 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.648+0000 7fd45cc00700 1 -- 172.21.15.117:0/3972621341 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd44c010460 con 0x7fd46010d0c0 2024-04-20T11:38:15.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.648+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4601a1b00 con 0x7fd46010d0c0 2024-04-20T11:38:15.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.648+0000 7fd45cc00700 1 -- 172.21.15.117:0/3972621341 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd44c00f5f0 con 0x7fd46010d0c0 2024-04-20T11:38:15.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.648+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4601a2050 con 0x7fd46010d0c0 2024-04-20T11:38:15.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.649+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46010f5c0 con 0x7fd46010d0c0 2024-04-20T11:38:15.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.650+0000 7fd45cc00700 1 -- 172.21.15.117:0/3972621341 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd44c0105d0 con 0x7fd46010d0c0 2024-04-20T11:38:15.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.652+0000 7fd45cc00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd44406ecc0 0x7fd444071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:15.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.652+0000 7fd45cc00700 1 -- 172.21.15.117:0/3972621341 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd44c08ee50 con 0x7fd46010d0c0 2024-04-20T11:38:15.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.652+0000 7fd45fe00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd44406ecc0 0x7fd444071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:15.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.653+0000 7fd45fe00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd44406ecc0 0x7fd444071180 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd450006010 tx=0x7fd450005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:15.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.657+0000 7fd45cc00700 1 -- 172.21.15.117:0/3972621341 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd44c057c30 con 0x7fd46010d0c0 2024-04-20T11:38:15.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.908+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd460114220 con 0x7fd46010d0c0 2024-04-20T11:38:15.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.909+0000 7fd45cc00700 1 -- 172.21.15.117:0/3972621341 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fd44c05b2f0 con 0x7fd46010d0c0 2024-04-20T11:38:15.910 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:38:15.910 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:38:15.910 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:38:15.910 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:38:15.910 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:38:15.911 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:38:15.912 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:38:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:38:15.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd44406ecc0 msgr2=0x7fd444071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd44406ecc0 0x7fd444071180 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd450006010 tx=0x7fd450005a90 comp rx=0 tx=0).stop 2024-04-20T11:38:15.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 msgr2=0x7fd46019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:15.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 0x7fd46019d470 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00d8d0 tx=0x7fd44c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:38:15.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 shutdown_connections 2024-04-20T11:38:15.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd44406ecc0 0x7fd444071180 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4601070b0 0x7fd46019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 --2- 172.21.15.117:0/3972621341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd46010d0c0 0x7fd46019d470 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:15.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 >> 172.21.15.117:0/3972621341 conn(0x7fd460076830 msgr2=0x7fd46010c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:15.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 shutdown_connections 2024-04-20T11:38:15.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:15.912+0000 7fd465a00700 1 -- 172.21.15.117:0/3972621341 wait complete. 2024-04-20T11:38:15.915 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:38:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.063+0000 7f0391400700 1 -- 172.21.15.117:0/981929834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c104a00 msgr2=0x7f038c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.063+0000 7f0391400700 1 --2- 172.21.15.117:0/981929834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c104a00 0x7f038c104e80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0378009910 tx=0x7f0378009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.063+0000 7f0391400700 1 -- 172.21.15.117:0/981929834 shutdown_connections 2024-04-20T11:38:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.063+0000 7f0391400700 1 --2- 172.21.15.117:0/981929834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c104a00 0x7f038c104e80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.063+0000 7f0391400700 1 --2- 172.21.15.117:0/981929834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f038c103110 0x7f038c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.063+0000 7f0391400700 1 -- 172.21.15.117:0/981929834 >> 172.21.15.117:0/981929834 conn(0x7f038c0fe880 msgr2=0x7f038c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:16.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.063+0000 7f0391400700 1 -- 172.21.15.117:0/981929834 shutdown_connections 2024-04-20T11:38:16.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.063+0000 7f0391400700 1 -- 172.21.15.117:0/981929834 wait complete. 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.065+0000 7f0391400700 1 Processor -- start 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.065+0000 7f0391400700 1 -- start start 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.065+0000 7f0391400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c103110 0x7f038c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.065+0000 7f0391400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f038c104a00 0x7f038c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.065+0000 7f0391400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c199ac0 con 0x7f038c103110 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.065+0000 7f0391400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c19d740 con 0x7f038c104a00 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f038a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c103110 0x7f038c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f038a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c103110 0x7f038c198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36244/0 (socket says 172.21.15.117:36244) 2024-04-20T11:38:16.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f038a800700 1 -- 172.21.15.117:0/2225721359 learned_addr learned my addr 172.21.15.117:0/2225721359 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:16.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f0389e00700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f038c104a00 0x7f038c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f038a800700 1 -- 172.21.15.117:0/2225721359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f038c104a00 msgr2=0x7f038c1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f038a800700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f038c104a00 0x7f038c1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f038a800700 1 -- 172.21.15.117:0/2225721359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03780095f0 con 0x7f038c103110 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f0389e00700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f038c104a00 0x7f038c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f038a800700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c103110 0x7f038c198ea0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b4b0 tx=0x7f037c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.066+0000 7f0383400700 1 -- 172.21.15.117:0/2225721359 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f037c011800 con 0x7f038c103110 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.067+0000 7f0383400700 1 -- 172.21.15.117:0/2225721359 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f037c011e20 con 0x7f038c103110 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.067+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c19da20 con 0x7f038c103110 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.067+0000 7f0383400700 1 -- 172.21.15.117:0/2225721359 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f037c00f2e0 con 0x7f038c103110 2024-04-20T11:38:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.067+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c19dff0 con 0x7f038c103110 2024-04-20T11:38:16.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.068+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c10b460 con 0x7f038c103110 2024-04-20T11:38:16.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.069+0000 7f0383400700 1 -- 172.21.15.117:0/2225721359 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f037c010380 con 0x7f038c103110 2024-04-20T11:38:16.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.069+0000 7f0383400700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f037006ecd0 0x7f0370071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.070+0000 7f0383400700 1 -- 172.21.15.117:0/2225721359 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f037c08ea70 con 0x7f038c103110 2024-04-20T11:38:16.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.070+0000 7f0389e00700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f037006ecd0 0x7f0370071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.071+0000 7f0389e00700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f037006ecd0 0x7f0370071190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0378006010 tx=0x7f037800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:16.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.074+0000 7f0383400700 1 -- 172.21.15.117:0/2225721359 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f037c0577b0 con 0x7f038c103110 2024-04-20T11:38:16.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.318+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f038c19a160 con 0x7f037006ecd0 2024-04-20T11:38:16.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:16 smithi117 ceph-mon[99036]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:16.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:16 smithi117 ceph-mon[99036]: pgmap v775: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:16.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:16 smithi117 ceph-mon[99036]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:16.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:16 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/620996279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:38:16.319 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:16 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3972621341' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:38:16.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.320+0000 7f0383400700 1 -- 172.21.15.117:0/2225721359 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f038c19a160 con 0x7f037006ecd0 2024-04-20T11:38:16.320 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:38:16.320 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:38:16.320 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:38:16.321 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:38:16.321 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:38:16.321 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:38:16.321 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:38:16.321 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:38:16.321 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:38:16.321 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:38:16.322 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:38:16.322 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:38:16.322 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:38:16.322 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:38:16.322 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:38:16.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f037006ecd0 msgr2=0x7f0370071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f037006ecd0 0x7f0370071190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0378006010 tx=0x7f037800b540 comp rx=0 tx=0).stop 2024-04-20T11:38:16.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c103110 msgr2=0x7f038c198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c103110 0x7f038c198ea0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b4b0 tx=0x7f037c00b870 comp rx=0 tx=0).stop 2024-04-20T11:38:16.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 shutdown_connections 2024-04-20T11:38:16.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f037006ecd0 0x7f0370071190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f038c103110 0x7f038c198ea0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 --2- 172.21.15.117:0/2225721359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f038c104a00 0x7f038c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.322+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 >> 172.21.15.117:0/2225721359 conn(0x7f038c0fe880 msgr2=0x7f038c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:16.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.323+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 shutdown_connections 2024-04-20T11:38:16.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.323+0000 7f0391400700 1 -- 172.21.15.117:0/2225721359 wait complete. 2024-04-20T11:38:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:16 smithi173 ceph-mon[74165]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:16 smithi173 ceph-mon[74165]: pgmap v775: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:16 smithi173 ceph-mon[74165]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:16 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/620996279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:38:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:16 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3972621341' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:38:16.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.477+0000 7fde21600700 1 -- 172.21.15.117:0/3409863427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 msgr2=0x7fde1c111670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.477+0000 7fde21600700 1 --2- 172.21.15.117:0/3409863427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 0x7fde1c111670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fde08009910 tx=0x7fde08009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:16.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.478+0000 7fde21600700 1 -- 172.21.15.117:0/3409863427 shutdown_connections 2024-04-20T11:38:16.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.478+0000 7fde21600700 1 --2- 172.21.15.117:0/3409863427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 0x7fde1c111670 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.478+0000 7fde21600700 1 --2- 172.21.15.117:0/3409863427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c073280 0x7fde1c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.478+0000 7fde21600700 1 -- 172.21.15.117:0/3409863427 >> 172.21.15.117:0/3409863427 conn(0x7fde1c0fc6b0 msgr2=0x7fde1c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:16.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.478+0000 7fde21600700 1 -- 172.21.15.117:0/3409863427 shutdown_connections 2024-04-20T11:38:16.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.478+0000 7fde21600700 1 -- 172.21.15.117:0/3409863427 wait complete. 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde21600700 1 Processor -- start 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde21600700 1 -- start start 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde21600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c073280 0x7fde1c19d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde21600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 0x7fde1c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde21600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c19dd70 con 0x7fde1c073ba0 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde21600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c1a1a40 con 0x7fde1c073280 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde1a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 0x7fde1c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde1a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 0x7fde1c19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36264/0 (socket says 172.21.15.117:36264) 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.479+0000 7fde1a000700 1 -- 172.21.15.117:0/1859212380 learned_addr learned my addr 172.21.15.117:0/1859212380 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:16.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.480+0000 7fde1a000700 1 -- 172.21.15.117:0/1859212380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c073280 msgr2=0x7fde1c19d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.480+0000 7fde1a000700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c073280 0x7fde1c19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.480+0000 7fde1a000700 1 -- 172.21.15.117:0/1859212380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde080095f0 con 0x7fde1c073ba0 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.480+0000 7fde1a000700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 0x7fde1c19d690 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fde080092e0 tx=0x7fde0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.480+0000 7fde13400700 1 -- 172.21.15.117:0/1859212380 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde0801d070 con 0x7fde1c073ba0 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.480+0000 7fde13400700 1 -- 172.21.15.117:0/1859212380 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde0800f460 con 0x7fde1c073ba0 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.480+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c1a1d20 con 0x7fde1c073ba0 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.480+0000 7fde13400700 1 -- 172.21.15.117:0/1859212380 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde08017630 con 0x7fde1c073ba0 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.481+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c1a2270 con 0x7fde1c073ba0 2024-04-20T11:38:16.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.482+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c0749c0 con 0x7fde1c073ba0 2024-04-20T11:38:16.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.483+0000 7fde13400700 1 -- 172.21.15.117:0/1859212380 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fde0800f5d0 con 0x7fde1c073ba0 2024-04-20T11:38:16.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.483+0000 7fde13400700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fde0006ecd0 0x7fde00071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:16.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.484+0000 7fde13400700 1 -- 172.21.15.117:0/1859212380 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fde080903a0 con 0x7fde1c073ba0 2024-04-20T11:38:16.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.484+0000 7fde1aa00700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fde0006ecd0 0x7fde00071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:16.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.484+0000 7fde1aa00700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fde0006ecd0 0x7fde00071190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0097b0 tx=0x7fde0c006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:16.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.488+0000 7fde13400700 1 -- 172.21.15.117:0/1859212380 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fde080590e0 con 0x7fde1c073ba0 2024-04-20T11:38:16.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.782+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fde1c04ebb0 con 0x7fde1c073ba0 2024-04-20T11:38:16.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.783+0000 7fde13400700 1 -- 172.21.15.117:0/1859212380 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fde080601a0 con 0x7fde1c073ba0 2024-04-20T11:38:16.784 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:38:16.784 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:38:16.784 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:38:16.784 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:38:16.784 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:38:16.784 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:38:16.785 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:38:16.786 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:38:16.786 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:38:16.786 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:38:16.786 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:38:16.786 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fde0006ecd0 msgr2=0x7fde00071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fde0006ecd0 0x7fde00071190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0097b0 tx=0x7fde0c006d20 comp rx=0 tx=0).stop 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 msgr2=0x7fde1c19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 0x7fde1c19d690 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fde080092e0 tx=0x7fde0800be20 comp rx=0 tx=0).stop 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 shutdown_connections 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fde0006ecd0 0x7fde00071190 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c073280 0x7fde1c19d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 --2- 172.21.15.117:0/1859212380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde1c073ba0 0x7fde1c19d690 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 >> 172.21.15.117:0/1859212380 conn(0x7fde1c0fc6b0 msgr2=0x7fde1c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:16.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 shutdown_connections 2024-04-20T11:38:16.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:16.786+0000 7fde21600700 1 -- 172.21.15.117:0/1859212380 wait complete. 2024-04-20T11:38:17.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:17 smithi173 ceph-mon[74165]: from='client.35422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:17.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:17 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1859212380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:38:17.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:17 smithi117 ceph-mon[99036]: from='client.35422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:17.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:17 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1859212380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:38:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:18 smithi117 ceph-mon[99036]: pgmap v776: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:18 smithi173 ceph-mon[74165]: pgmap v776: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:20.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:20 smithi117 ceph-mon[99036]: pgmap v777: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:20 smithi173 ceph-mon[74165]: pgmap v777: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:22.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:22 smithi117 ceph-mon[99036]: pgmap v778: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:22 smithi173 ceph-mon[74165]: pgmap v778: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:24 smithi117 ceph-mon[99036]: pgmap v779: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:24 smithi173 ceph-mon[74165]: pgmap v779: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:26.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:26 smithi117 ceph-mon[99036]: pgmap v780: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:26 smithi173 ceph-mon[74165]: pgmap v780: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:28 smithi117 ceph-mon[99036]: pgmap v781: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:38:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:28 smithi173 ceph-mon[74165]: pgmap v781: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:38:30.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:30 smithi117 ceph-mon[99036]: pgmap v782: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:30 smithi173 ceph-mon[74165]: pgmap v782: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:32 smithi117 ceph-mon[99036]: pgmap v783: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:32 smithi173 ceph-mon[74165]: pgmap v783: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:34 smithi117 ceph-mon[99036]: pgmap v784: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:34.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:34 smithi173 ceph-mon[74165]: pgmap v784: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:36 smithi117 ceph-mon[99036]: pgmap v785: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:36 smithi173 ceph-mon[74165]: pgmap v785: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:38.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:38 smithi117 ceph-mon[99036]: pgmap v786: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:38 smithi173 ceph-mon[74165]: pgmap v786: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:40.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:40 smithi117 ceph-mon[99036]: pgmap v787: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:40.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:40 smithi173 ceph-mon[74165]: pgmap v787: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:42.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:42 smithi173 ceph-mon[74165]: pgmap v788: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:42 smithi117 ceph-mon[99036]: pgmap v788: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:38:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:38:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:44 smithi117 ceph-mon[99036]: pgmap v789: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:44 smithi173 ceph-mon[74165]: pgmap v789: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:46 smithi117 ceph-mon[99036]: pgmap v790: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:46 smithi173 ceph-mon[74165]: pgmap v790: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:46.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.958+0000 7f4188600700 1 -- 172.21.15.117:0/1833393683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180108e10 msgr2=0x7f4180109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:46.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.958+0000 7f4188600700 1 --2- 172.21.15.117:0/1833393683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180108e10 0x7f4180109290 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f416c009910 tx=0x7f416c009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:46.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.959+0000 7f4188600700 1 -- 172.21.15.117:0/1833393683 shutdown_connections 2024-04-20T11:38:46.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.959+0000 7f4188600700 1 --2- 172.21.15.117:0/1833393683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180108e10 0x7f4180109290 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:46.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.959+0000 7f4188600700 1 --2- 172.21.15.117:0/1833393683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4180107520 0x7f4180107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:46.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.959+0000 7f4188600700 1 -- 172.21.15.117:0/1833393683 >> 172.21.15.117:0/1833393683 conn(0x7f41800768f0 msgr2=0x7f4180078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:46.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.959+0000 7f4188600700 1 -- 172.21.15.117:0/1833393683 shutdown_connections 2024-04-20T11:38:46.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.959+0000 7f4188600700 1 -- 172.21.15.117:0/1833393683 wait complete. 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.960+0000 7f4188600700 1 Processor -- start 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.960+0000 7f4188600700 1 -- start start 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.960+0000 7f4188600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180107520 0x7f418019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.960+0000 7f4188600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4180108e10 0x7f418019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.960+0000 7f4188600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418019dd60 con 0x7f4180107520 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.960+0000 7f4188600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418019dea0 con 0x7f4180108e10 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.961+0000 7f4185c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180107520 0x7f418019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.961+0000 7f4185c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180107520 0x7f418019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52310/0 (socket says 172.21.15.117:52310) 2024-04-20T11:38:46.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.961+0000 7f4185c00700 1 -- 172.21.15.117:0/1665431564 learned_addr learned my addr 172.21.15.117:0/1665431564 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:46.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.961+0000 7f4185200700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4180108e10 0x7f418019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.961+0000 7f4185c00700 1 -- 172.21.15.117:0/1665431564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4180108e10 msgr2=0x7f418019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.961+0000 7f4185c00700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4180108e10 0x7f418019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.961+0000 7f4185c00700 1 -- 172.21.15.117:0/1665431564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c0095f0 con 0x7f4180107520 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.961+0000 7f4185c00700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180107520 0x7f418019d1d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f417400c930 tx=0x7f417400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.962+0000 7f417e000700 1 -- 172.21.15.117:0/1665431564 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4174007ad0 con 0x7f4180107520 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.962+0000 7f417e000700 1 -- 172.21.15.117:0/1665431564 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4174007c30 con 0x7f4180107520 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.962+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41801a1be0 con 0x7f4180107520 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.962+0000 7f417e000700 1 -- 172.21.15.117:0/1665431564 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4174018610 con 0x7f4180107520 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.962+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41801a21b0 con 0x7f4180107520 2024-04-20T11:38:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.963+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418010f870 con 0x7f4180107520 2024-04-20T11:38:46.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.964+0000 7f417e000700 1 -- 172.21.15.117:0/1665431564 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f417400f450 con 0x7f4180107520 2024-04-20T11:38:46.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.965+0000 7f417e000700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f416806ecd0 0x7f4168071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:46.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.965+0000 7f417e000700 1 -- 172.21.15.117:0/1665431564 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f417408efe0 con 0x7f4180107520 2024-04-20T11:38:46.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.965+0000 7f4185200700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f416806ecd0 0x7f4168071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:46.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.966+0000 7f4185200700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f416806ecd0 0x7f4168071190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f416c005100 tx=0x7f416c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:46.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:46.970+0000 7f417e000700 1 -- 172.21.15.117:0/1665431564 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4174057de0 con 0x7f4180107520 2024-04-20T11:38:47.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.214+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41801144c0 con 0x7f416806ecd0 2024-04-20T11:38:47.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.217+0000 7f417e000700 1 -- 172.21.15.117:0/1665431564 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f41801144c0 con 0x7f416806ecd0 2024-04-20T11:38:47.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f416806ecd0 msgr2=0x7f4168071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:47.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f416806ecd0 0x7f4168071190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f416c005100 tx=0x7f416c01a040 comp rx=0 tx=0).stop 2024-04-20T11:38:47.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180107520 msgr2=0x7f418019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:47.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180107520 0x7f418019d1d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f417400c930 tx=0x7f417400ccf0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 shutdown_connections 2024-04-20T11:38:47.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f416806ecd0 0x7f4168071190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4180107520 0x7f418019d1d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 --2- 172.21.15.117:0/1665431564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4180108e10 0x7f418019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.220+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 >> 172.21.15.117:0/1665431564 conn(0x7f41800768f0 msgr2=0x7f4180077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:47.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.221+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 shutdown_connections 2024-04-20T11:38:47.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.221+0000 7f4188600700 1 -- 172.21.15.117:0/1665431564 wait complete. 2024-04-20T11:38:47.236 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:38:47.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.380+0000 7f3d5e200700 1 -- 172.21.15.117:0/3239350681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 msgr2=0x7f3d581034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:47.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.380+0000 7f3d5e200700 1 --2- 172.21.15.117:0/3239350681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 0x7f3d581034f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009910 tx=0x7f3d48009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:47.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.381+0000 7f3d5e200700 1 -- 172.21.15.117:0/3239350681 shutdown_connections 2024-04-20T11:38:47.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.381+0000 7f3d5e200700 1 --2- 172.21.15.117:0/3239350681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d58104a00 0x7f3d58104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.381+0000 7f3d5e200700 1 --2- 172.21.15.117:0/3239350681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 0x7f3d581034f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.381+0000 7f3d5e200700 1 -- 172.21.15.117:0/3239350681 >> 172.21.15.117:0/3239350681 conn(0x7f3d580fe880 msgr2=0x7f3d58100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:47.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.381+0000 7f3d5e200700 1 -- 172.21.15.117:0/3239350681 shutdown_connections 2024-04-20T11:38:47.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.381+0000 7f3d5e200700 1 -- 172.21.15.117:0/3239350681 wait complete. 2024-04-20T11:38:47.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.382+0000 7f3d5e200700 1 Processor -- start 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.382+0000 7f3d5e200700 1 -- start start 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.382+0000 7f3d5e200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 0x7f3d58198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.382+0000 7f3d5e200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d58104a00 0x7f3d58199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.382+0000 7f3d5e200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58199950 con 0x7f3d58103110 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.382+0000 7f3d5e200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58199a90 con 0x7f3d58104a00 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d57400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 0x7f3d58198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d57400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 0x7f3d58198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52316/0 (socket says 172.21.15.117:52316) 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d57400700 1 -- 172.21.15.117:0/2045352416 learned_addr learned my addr 172.21.15.117:0/2045352416 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d57400700 1 -- 172.21.15.117:0/2045352416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d58104a00 msgr2=0x7f3d58199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d56a00700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d58104a00 0x7f3d58199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d57400700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d58104a00 0x7f3d58199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d57400700 1 -- 172.21.15.117:0/2045352416 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d480095f0 con 0x7f3d58103110 2024-04-20T11:38:47.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d56a00700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d58104a00 0x7f3d58199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:47.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.383+0000 7f3d57400700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 0x7f3d58198dc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3d480092e0 tx=0x7f3d4800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:47.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.384+0000 7f3d4fe00700 1 -- 172.21.15.117:0/2045352416 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d4801d070 con 0x7f3d58103110 2024-04-20T11:38:47.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.384+0000 7f3d4fe00700 1 -- 172.21.15.117:0/2045352416 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d4800f460 con 0x7f3d58103110 2024-04-20T11:38:47.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.384+0000 7f3d4fe00700 1 -- 172.21.15.117:0/2045352416 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d48017630 con 0x7f3d58103110 2024-04-20T11:38:47.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.384+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5819d770 con 0x7f3d58103110 2024-04-20T11:38:47.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.384+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5819dce0 con 0x7f3d58103110 2024-04-20T11:38:47.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.385+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5810b480 con 0x7f3d58103110 2024-04-20T11:38:47.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.386+0000 7f3d4fe00700 1 -- 172.21.15.117:0/2045352416 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3d4800f5d0 con 0x7f3d58103110 2024-04-20T11:38:47.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.387+0000 7f3d4fe00700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3c06ecd0 0x7f3d3c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:47.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.387+0000 7f3d4fe00700 1 -- 172.21.15.117:0/2045352416 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3d48090320 con 0x7f3d58103110 2024-04-20T11:38:47.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.387+0000 7f3d56a00700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3c06ecd0 0x7f3d3c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:47.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.388+0000 7f3d56a00700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3c06ecd0 0x7f3d3c071190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3d44005fd0 tx=0x7f3d44005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:47.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.392+0000 7f3d4fe00700 1 -- 172.21.15.117:0/2045352416 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3d480590e0 con 0x7f3d58103110 2024-04-20T11:38:47.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.639+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d5819a080 con 0x7f3d3c06ecd0 2024-04-20T11:38:47.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.642+0000 7f3d4fe00700 1 -- 172.21.15.117:0/2045352416 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3d5819a080 con 0x7f3d3c06ecd0 2024-04-20T11:38:47.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.644+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3c06ecd0 msgr2=0x7f3d3c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.644+0000 7f3d5e200700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3c06ecd0 0x7f3d3c071190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3d44005fd0 tx=0x7f3d44005e20 comp rx=0 tx=0).stop 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.644+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 msgr2=0x7f3d58198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.644+0000 7f3d5e200700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 0x7f3d58198dc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3d480092e0 tx=0x7f3d4800be20 comp rx=0 tx=0).stop 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.645+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 shutdown_connections 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.645+0000 7f3d5e200700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d3c06ecd0 0x7f3d3c071190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.645+0000 7f3d5e200700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d58103110 0x7f3d58198dc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.645+0000 7f3d5e200700 1 --2- 172.21.15.117:0/2045352416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d58104a00 0x7f3d58199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.645+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 >> 172.21.15.117:0/2045352416 conn(0x7f3d580fe880 msgr2=0x7f3d580ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.645+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 shutdown_connections 2024-04-20T11:38:47.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.645+0000 7f3d5e200700 1 -- 172.21.15.117:0/2045352416 wait complete. 2024-04-20T11:38:47.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.801+0000 7fbc1d000700 1 -- 172.21.15.117:0/1405581923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 msgr2=0x7fbc18107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:47.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.801+0000 7fbc1d000700 1 --2- 172.21.15.117:0/1405581923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 0x7fbc18107a20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009910 tx=0x7fbc04009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:47.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.801+0000 7fbc1d000700 1 -- 172.21.15.117:0/1405581923 shutdown_connections 2024-04-20T11:38:47.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.801+0000 7fbc1d000700 1 --2- 172.21.15.117:0/1405581923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 0x7fbc18107a20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.801+0000 7fbc1d000700 1 --2- 172.21.15.117:0/1405581923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc18103020 0x7fbc18103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.801+0000 7fbc1d000700 1 -- 172.21.15.117:0/1405581923 >> 172.21.15.117:0/1405581923 conn(0x7fbc180fe8b0 msgr2=0x7fbc18100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:47.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.801+0000 7fbc1d000700 1 -- 172.21.15.117:0/1405581923 shutdown_connections 2024-04-20T11:38:47.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.801+0000 7fbc1d000700 1 -- 172.21.15.117:0/1405581923 wait complete. 2024-04-20T11:38:47.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.802+0000 7fbc1d000700 1 Processor -- start 2024-04-20T11:38:47.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.803+0000 7fbc1d000700 1 -- start start 2024-04-20T11:38:47.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.803+0000 7fbc1d000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc18103020 0x7fbc18198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:47.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.803+0000 7fbc1d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 0x7fbc181992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:47.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.803+0000 7fbc1d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181999b0 con 0x7fbc181039d0 2024-04-20T11:38:47.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.803+0000 7fbc1d000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1819d680 con 0x7fbc18103020 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.803+0000 7fbc15400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 0x7fbc181992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.804+0000 7fbc15400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 0x7fbc181992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52320/0 (socket says 172.21.15.117:52320) 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.804+0000 7fbc15400700 1 -- 172.21.15.117:0/475570726 learned_addr learned my addr 172.21.15.117:0/475570726 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.804+0000 7fbc15e00700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc18103020 0x7fbc18198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.804+0000 7fbc15400700 1 -- 172.21.15.117:0/475570726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc18103020 msgr2=0x7fbc18198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.804+0000 7fbc15400700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc18103020 0x7fbc18198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.804+0000 7fbc15400700 1 -- 172.21.15.117:0/475570726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc040095f0 con 0x7fbc181039d0 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.804+0000 7fbc15400700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 0x7fbc181992d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbc04005b40 tx=0x7fbc0400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.804+0000 7fbc0ea00700 1 -- 172.21.15.117:0/475570726 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc0401d070 con 0x7fbc181039d0 2024-04-20T11:38:47.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.805+0000 7fbc0ea00700 1 -- 172.21.15.117:0/475570726 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc0400f460 con 0x7fbc181039d0 2024-04-20T11:38:47.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.805+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1819d900 con 0x7fbc181039d0 2024-04-20T11:38:47.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.805+0000 7fbc0ea00700 1 -- 172.21.15.117:0/475570726 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc04017630 con 0x7fbc181039d0 2024-04-20T11:38:47.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.805+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1819ddf0 con 0x7fbc181039d0 2024-04-20T11:38:47.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.806+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1810b370 con 0x7fbc181039d0 2024-04-20T11:38:47.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.807+0000 7fbc0ea00700 1 -- 172.21.15.117:0/475570726 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbc04003a60 con 0x7fbc181039d0 2024-04-20T11:38:47.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.809+0000 7fbc0ea00700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbfc06ecd0 0x7fbbfc071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:47.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.809+0000 7fbc0ea00700 1 -- 172.21.15.117:0/475570726 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbc04090380 con 0x7fbc181039d0 2024-04-20T11:38:47.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.809+0000 7fbc15e00700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbfc06ecd0 0x7fbbfc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:47.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.810+0000 7fbc15e00700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbfc06ecd0 0x7fbbfc071190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbc08006fd0 tx=0x7fbc08008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:47.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:47.814+0000 7fbc0ea00700 1 -- 172.21.15.117:0/475570726 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbc040590c0 con 0x7fbc181039d0 2024-04-20T11:38:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.045+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc18110040 con 0x7fbbfc06ecd0 2024-04-20T11:38:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.057+0000 7fbc0ea00700 1 -- 172.21.15.117:0/475570726 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbc18110040 con 0x7fbbfc06ecd0 2024-04-20T11:38:48.056 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (27m) 3m ago 41m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (41m) 3m ago 41m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (40m) 3m ago 40m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (23m) 3m ago 41m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (23m) 3m ago 40m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (26m) 3m ago 40m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (19m) 3m ago 34m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (19m) 3m ago 34m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (3m) 3m ago 33m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (18m) 3m ago 34m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (30m) 3m ago 43m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (28m) 3m ago 40m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:38:48.057 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (25m) 3m ago 43m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (24m) 3m ago 39m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (28m) 3m ago 41m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (28m) 3m ago 40m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (22m) 3m ago 38m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (22m) 3m ago 38m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (21m) 3m ago 37m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (21m) 3m ago 37m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (20m) 3m ago 36m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (19m) 3m ago 35m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:38:48.058 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (27m) 3m ago 40m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:38:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbfc06ecd0 msgr2=0x7fbbfc071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbfc06ecd0 0x7fbbfc071190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbc08006fd0 tx=0x7fbc08008040 comp rx=0 tx=0).stop 2024-04-20T11:38:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 msgr2=0x7fbc181992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 0x7fbc181992d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbc04005b40 tx=0x7fbc0400be20 comp rx=0 tx=0).stop 2024-04-20T11:38:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 shutdown_connections 2024-04-20T11:38:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbbfc06ecd0 0x7fbbfc071190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc18103020 0x7fbc18198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 --2- 172.21.15.117:0/475570726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc181039d0 0x7fbc181992d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.059+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 >> 172.21.15.117:0/475570726 conn(0x7fbc180fe8b0 msgr2=0x7fbc180ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:48.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.060+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 shutdown_connections 2024-04-20T11:38:48.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.060+0000 7fbc1d000700 1 -- 172.21.15.117:0/475570726 wait complete. 2024-04-20T11:38:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.207+0000 7f9783000700 1 -- 172.21.15.117:0/3772714968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 msgr2=0x7f977c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.207+0000 7f9783000700 1 --2- 172.21.15.117:0/3772714968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 0x7f977c104e80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9764009910 tx=0x7f9764009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.207+0000 7f9783000700 1 -- 172.21.15.117:0/3772714968 shutdown_connections 2024-04-20T11:38:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.207+0000 7f9783000700 1 --2- 172.21.15.117:0/3772714968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 0x7f977c104e80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.207+0000 7f9783000700 1 --2- 172.21.15.117:0/3772714968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f977c103110 0x7f977c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.207+0000 7f9783000700 1 -- 172.21.15.117:0/3772714968 >> 172.21.15.117:0/3772714968 conn(0x7f977c0fe880 msgr2=0x7f977c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.207+0000 7f9783000700 1 -- 172.21.15.117:0/3772714968 shutdown_connections 2024-04-20T11:38:48.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.207+0000 7f9783000700 1 -- 172.21.15.117:0/3772714968 wait complete. 2024-04-20T11:38:48.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.208+0000 7f9783000700 1 Processor -- start 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.208+0000 7f9783000700 1 -- start start 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.208+0000 7f9783000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f977c103110 0x7f977c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.208+0000 7f9783000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 0x7f977c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f9783000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c19f520 con 0x7f977c104a00 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f9783000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c198b90 con 0x7f977c103110 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f9773e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 0x7f977c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f9773e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 0x7f977c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52340/0 (socket says 172.21.15.117:52340) 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f9773e00700 1 -- 172.21.15.117:0/1457816077 learned_addr learned my addr 172.21.15.117:0/1457816077 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f977be00700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f977c103110 0x7f977c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f9773e00700 1 -- 172.21.15.117:0/1457816077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f977c103110 msgr2=0x7f977c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f9773e00700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f977c103110 0x7f977c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.209+0000 7f9773e00700 1 -- 172.21.15.117:0/1457816077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97640095f0 con 0x7f977c104a00 2024-04-20T11:38:48.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.210+0000 7f9773e00700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 0x7f977c19ef90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f97640092e0 tx=0x7f976400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.210+0000 7f9779600700 1 -- 172.21.15.117:0/1457816077 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f976401d070 con 0x7f977c104a00 2024-04-20T11:38:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.210+0000 7f9779600700 1 -- 172.21.15.117:0/1457816077 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f976400f460 con 0x7f977c104a00 2024-04-20T11:38:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.210+0000 7f9779600700 1 -- 172.21.15.117:0/1457816077 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9764017630 con 0x7f977c104a00 2024-04-20T11:38:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.210+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c198e10 con 0x7f977c104a00 2024-04-20T11:38:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.210+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c199330 con 0x7f977c104a00 2024-04-20T11:38:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.211+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c10b480 con 0x7f977c104a00 2024-04-20T11:38:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.212+0000 7f9779600700 1 -- 172.21.15.117:0/1457816077 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f976400f5d0 con 0x7f977c104a00 2024-04-20T11:38:48.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.213+0000 7f9779600700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f976006ecd0 0x7f9760071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.213+0000 7f9779600700 1 -- 172.21.15.117:0/1457816077 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f97640903e0 con 0x7f977c104a00 2024-04-20T11:38:48.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.213+0000 7f977be00700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f976006ecd0 0x7f9760071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.214+0000 7f977be00700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f976006ecd0 0x7f9760071190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f976c005ea0 tx=0x7f976c005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.218+0000 7f9779600700 1 -- 172.21.15.117:0/1457816077 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f976405cc50 con 0x7f977c104a00 2024-04-20T11:38:48.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:48 smithi117 ceph-mon[99036]: pgmap v791: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:48.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:48 smithi117 ceph-mon[99036]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:48.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.520+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f977c199fe0 con 0x7f977c104a00 2024-04-20T11:38:48.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.521+0000 7f9779600700 1 -- 172.21.15.117:0/1457816077 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f976405c7e0 con 0x7f977c104a00 2024-04-20T11:38:48.521 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:38:48.521 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:38:48.522 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:38:48.523 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f976006ecd0 msgr2=0x7f9760071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f976006ecd0 0x7f9760071190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f976c005ea0 tx=0x7f976c005e30 comp rx=0 tx=0).stop 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 msgr2=0x7f977c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 0x7f977c19ef90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f97640092e0 tx=0x7f976400be20 comp rx=0 tx=0).stop 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 shutdown_connections 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f976006ecd0 0x7f9760071190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f977c103110 0x7f977c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 --2- 172.21.15.117:0/1457816077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f977c104a00 0x7f977c19ef90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 >> 172.21.15.117:0/1457816077 conn(0x7f977c0fe880 msgr2=0x7f977c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 shutdown_connections 2024-04-20T11:38:48.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.524+0000 7f9783000700 1 -- 172.21.15.117:0/1457816077 wait complete. 2024-04-20T11:38:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:48 smithi173 ceph-mon[74165]: pgmap v791: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:48 smithi173 ceph-mon[74165]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:48.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.681+0000 7f7e11600700 1 -- 172.21.15.117:0/1828646700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 msgr2=0x7f7e0c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.681+0000 7f7e11600700 1 --2- 172.21.15.117:0/1828646700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 0x7f7e0c109290 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009910 tx=0x7f7df8009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:48.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.681+0000 7f7e11600700 1 -- 172.21.15.117:0/1828646700 shutdown_connections 2024-04-20T11:38:48.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.681+0000 7f7e11600700 1 --2- 172.21.15.117:0/1828646700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 0x7f7e0c109290 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.682+0000 7f7e11600700 1 --2- 172.21.15.117:0/1828646700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c107520 0x7f7e0c107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.682+0000 7f7e11600700 1 -- 172.21.15.117:0/1828646700 >> 172.21.15.117:0/1828646700 conn(0x7f7e0c0768f0 msgr2=0x7f7e0c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:48.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.682+0000 7f7e11600700 1 -- 172.21.15.117:0/1828646700 shutdown_connections 2024-04-20T11:38:48.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.682+0000 7f7e11600700 1 -- 172.21.15.117:0/1828646700 wait complete. 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.683+0000 7f7e11600700 1 Processor -- start 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.683+0000 7f7e11600700 1 -- start start 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.683+0000 7f7e11600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c107520 0x7f7e0c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.683+0000 7f7e11600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 0x7f7e0c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.683+0000 7f7e11600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c19ddf0 con 0x7f7e0c108e10 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.683+0000 7f7e11600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c1a1a70 con 0x7f7e0c107520 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.683+0000 7f7e0a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 0x7f7e0c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e0a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 0x7f7e0c19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52344/0 (socket says 172.21.15.117:52344) 2024-04-20T11:38:48.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e0a000700 1 -- 172.21.15.117:0/1065893451 learned_addr learned my addr 172.21.15.117:0/1065893451 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e0aa00700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c107520 0x7f7e0c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e0a000700 1 -- 172.21.15.117:0/1065893451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c107520 msgr2=0x7f7e0c19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e0a000700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c107520 0x7f7e0c19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e0a000700 1 -- 172.21.15.117:0/1065893451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df80095f0 con 0x7f7e0c108e10 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e0aa00700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c107520 0x7f7e0c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e0a000700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 0x7f7e0c19d710 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7df8000c00 tx=0x7f7df800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.684+0000 7f7e03400700 1 -- 172.21.15.117:0/1065893451 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df801d070 con 0x7f7e0c108e10 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.685+0000 7f7e03400700 1 -- 172.21.15.117:0/1065893451 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7df80056e0 con 0x7f7e0c108e10 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.685+0000 7f7e03400700 1 -- 172.21.15.117:0/1065893451 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df800fe80 con 0x7f7e0c108e10 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.685+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c1a1cf0 con 0x7f7e0c108e10 2024-04-20T11:38:48.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.685+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c1a2210 con 0x7f7e0c108e10 2024-04-20T11:38:48.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.686+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c10f870 con 0x7f7e0c108e10 2024-04-20T11:38:48.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.687+0000 7f7e03400700 1 -- 172.21.15.117:0/1065893451 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7df800f6f0 con 0x7f7e0c108e10 2024-04-20T11:38:48.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.688+0000 7f7e03400700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7df006ecd0 0x7f7df0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:48.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.688+0000 7f7e03400700 1 -- 172.21.15.117:0/1065893451 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7df80902e0 con 0x7f7e0c108e10 2024-04-20T11:38:48.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.688+0000 7f7e0aa00700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7df006ecd0 0x7f7df0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:48.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.689+0000 7f7e0aa00700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7df006ecd0 0x7f7df0071190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005fd0 tx=0x7f7dfc005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:48.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.693+0000 7f7e03400700 1 -- 172.21.15.117:0/1065893451 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7df8059020 con 0x7f7e0c108e10 2024-04-20T11:38:48.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.955+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7e0c108920 con 0x7f7e0c108e10 2024-04-20T11:38:48.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.956+0000 7f7e03400700 1 -- 172.21.15.117:0/1065893451 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f7df805c6e0 con 0x7f7e0c108e10 2024-04-20T11:38:48.956 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:38:48.957 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:38:48.958 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:38:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:38:48.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7df006ecd0 msgr2=0x7f7df0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7df006ecd0 0x7f7df0071190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005fd0 tx=0x7f7dfc005ee0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 msgr2=0x7f7e0c19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 0x7f7e0c19d710 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7df8000c00 tx=0x7f7df800be20 comp rx=0 tx=0).stop 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 shutdown_connections 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7df006ecd0 0x7f7df0071190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c107520 0x7f7e0c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 --2- 172.21.15.117:0/1065893451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e0c108e10 0x7f7e0c19d710 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 >> 172.21.15.117:0/1065893451 conn(0x7f7e0c0768f0 msgr2=0x7f7e0c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 shutdown_connections 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:48.959+0000 7f7e11600700 1 -- 172.21.15.117:0/1065893451 wait complete. 2024-04-20T11:38:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:38:49.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.110+0000 7f0866800700 1 -- 172.21.15.117:0/3114442832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0860100280 msgr2=0x7f086010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.110+0000 7f0866800700 1 --2- 172.21.15.117:0/3114442832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0860100280 0x7f086010cf40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f084c009910 tx=0x7f084c009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:49.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.110+0000 7f0866800700 1 -- 172.21.15.117:0/3114442832 shutdown_connections 2024-04-20T11:38:49.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.110+0000 7f0866800700 1 --2- 172.21.15.117:0/3114442832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0860100280 0x7f086010cf40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.110+0000 7f0866800700 1 --2- 172.21.15.117:0/3114442832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08600ff8d0 0x7f08600ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.110+0000 7f0866800700 1 -- 172.21.15.117:0/3114442832 >> 172.21.15.117:0/3114442832 conn(0x7f0860076750 msgr2=0x7f08600fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:49.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.111+0000 7f0866800700 1 -- 172.21.15.117:0/3114442832 shutdown_connections 2024-04-20T11:38:49.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.111+0000 7f0866800700 1 -- 172.21.15.117:0/3114442832 wait complete. 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f0866800700 1 Processor -- start 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f0866800700 1 -- start start 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f0866800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08600ff8d0 0x7f086019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f0866800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0860100280 0x7f086019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f0866800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086019f550 con 0x7f08600ff8d0 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f0866800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0860198b30 con 0x7f0860100280 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f085f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08600ff8d0 0x7f086019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f085f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08600ff8d0 0x7f086019e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52358/0 (socket says 172.21.15.117:52358) 2024-04-20T11:38:49.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.112+0000 7f085f400700 1 -- 172.21.15.117:0/1092962773 learned_addr learned my addr 172.21.15.117:0/1092962773 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:49.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.113+0000 7f085f400700 1 -- 172.21.15.117:0/1092962773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0860100280 msgr2=0x7f086019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.113+0000 7f085f400700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0860100280 0x7f086019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.113+0000 7f085f400700 1 -- 172.21.15.117:0/1092962773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c0095f0 con 0x7f08600ff8d0 2024-04-20T11:38:49.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.113+0000 7f085f400700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08600ff8d0 0x7f086019e9f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f085000c8a0 tx=0x7f085000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:49.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.113+0000 7f0857e00700 1 -- 172.21.15.117:0/1092962773 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085000cea0 con 0x7f08600ff8d0 2024-04-20T11:38:49.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.113+0000 7f0857e00700 1 -- 172.21.15.117:0/1092962773 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0850004830 con 0x7f08600ff8d0 2024-04-20T11:38:49.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.113+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0860198e10 con 0x7f08600ff8d0 2024-04-20T11:38:49.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.113+0000 7f0857e00700 1 -- 172.21.15.117:0/1092962773 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08500055b0 con 0x7f08600ff8d0 2024-04-20T11:38:49.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.114+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0860199360 con 0x7f08600ff8d0 2024-04-20T11:38:49.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.115+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08601078c0 con 0x7f08600ff8d0 2024-04-20T11:38:49.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.116+0000 7f0857e00700 1 -- 172.21.15.117:0/1092962773 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f08500049a0 con 0x7f08600ff8d0 2024-04-20T11:38:49.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.117+0000 7f0857e00700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f084406ecc0 0x7f0844071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.117+0000 7f0857e00700 1 -- 172.21.15.117:0/1092962773 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f085008eac0 con 0x7f08600ff8d0 2024-04-20T11:38:49.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.117+0000 7f085ea00700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f084406ecc0 0x7f0844071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.118+0000 7f085ea00700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f084406ecc0 0x7f0844071180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b5c0 tx=0x7f084c005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:49.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.122+0000 7f0857e00700 1 -- 172.21.15.117:0/1092962773 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f085005b3b0 con 0x7f08600ff8d0 2024-04-20T11:38:49.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.360+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f086010ff50 con 0x7f084406ecc0 2024-04-20T11:38:49.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.362+0000 7f0857e00700 1 -- 172.21.15.117:0/1092962773 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f086010ff50 con 0x7f084406ecc0 2024-04-20T11:38:49.362 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:38:49.362 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:38:49.362 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:38:49.362 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:38:49.362 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:38:49.362 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:38:49.363 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:38:49.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.365+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f084406ecc0 msgr2=0x7f0844071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.365+0000 7f0866800700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f084406ecc0 0x7f0844071180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b5c0 tx=0x7f084c005960 comp rx=0 tx=0).stop 2024-04-20T11:38:49.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.365+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08600ff8d0 msgr2=0x7f086019e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.365+0000 7f0866800700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08600ff8d0 0x7f086019e9f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f085000c8a0 tx=0x7f085000cc60 comp rx=0 tx=0).stop 2024-04-20T11:38:49.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.366+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 shutdown_connections 2024-04-20T11:38:49.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.366+0000 7f0866800700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f084406ecc0 0x7f0844071180 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.366+0000 7f0866800700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08600ff8d0 0x7f086019e9f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.366+0000 7f0866800700 1 --2- 172.21.15.117:0/1092962773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0860100280 0x7f086019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.366+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 >> 172.21.15.117:0/1092962773 conn(0x7f0860076750 msgr2=0x7f08600fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:49.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.366+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 shutdown_connections 2024-04-20T11:38:49.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.366+0000 7f0866800700 1 -- 172.21.15.117:0/1092962773 wait complete. 2024-04-20T11:38:49.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.520+0000 7f9774800700 1 -- 172.21.15.117:0/3218829353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c107520 msgr2=0x7f976c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.520+0000 7f9774800700 1 --2- 172.21.15.117:0/3218829353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c107520 0x7f976c107900 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9760009910 tx=0x7f9760009c20 comp rx=0 tx=0).stop 2024-04-20T11:38:49.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.520+0000 7f9774800700 1 -- 172.21.15.117:0/3218829353 shutdown_connections 2024-04-20T11:38:49.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.520+0000 7f9774800700 1 --2- 172.21.15.117:0/3218829353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c108e10 0x7f976c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.520+0000 7f9774800700 1 --2- 172.21.15.117:0/3218829353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c107520 0x7f976c107900 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.520+0000 7f9774800700 1 -- 172.21.15.117:0/3218829353 >> 172.21.15.117:0/3218829353 conn(0x7f976c0768f0 msgr2=0x7f976c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:49.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.520+0000 7f9774800700 1 -- 172.21.15.117:0/3218829353 shutdown_connections 2024-04-20T11:38:49.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.520+0000 7f9774800700 1 -- 172.21.15.117:0/3218829353 wait complete. 2024-04-20T11:38:49.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.521+0000 7f9774800700 1 Processor -- start 2024-04-20T11:38:49.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.521+0000 7f9774800700 1 -- start start 2024-04-20T11:38:49.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9774800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c107520 0x7f976c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9774800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c108e10 0x7f976c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9774800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c19ddd0 con 0x7f976c108e10 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9774800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c1a1aa0 con 0x7f976c107520 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9771400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c108e10 0x7f976c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9771400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c108e10 0x7f976c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52378/0 (socket says 172.21.15.117:52378) 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9771400700 1 -- 172.21.15.117:0/2357033768 learned_addr learned my addr 172.21.15.117:0/2357033768 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9771e00700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c107520 0x7f976c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9771400700 1 -- 172.21.15.117:0/2357033768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c107520 msgr2=0x7f976c19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9771400700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c107520 0x7f976c19d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.522+0000 7f9771400700 1 -- 172.21.15.117:0/2357033768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97600095f0 con 0x7f976c108e10 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.523+0000 7f9771400700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c108e10 0x7f976c19d6f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d8d0 tx=0x7f975c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.523+0000 7f9766a00700 1 -- 172.21.15.117:0/2357033768 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f975c009940 con 0x7f976c108e10 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.523+0000 7f9766a00700 1 -- 172.21.15.117:0/2357033768 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f975c010460 con 0x7f976c108e10 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.523+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c1a1d80 con 0x7f976c108e10 2024-04-20T11:38:49.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.523+0000 7f9766a00700 1 -- 172.21.15.117:0/2357033768 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f975c00f5f0 con 0x7f976c108e10 2024-04-20T11:38:49.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.523+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976c1a2350 con 0x7f976c108e10 2024-04-20T11:38:49.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.524+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976c10f870 con 0x7f976c108e10 2024-04-20T11:38:49.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.526+0000 7f9766a00700 1 -- 172.21.15.117:0/2357033768 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f975c0105d0 con 0x7f976c108e10 2024-04-20T11:38:49.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.526+0000 7f9766a00700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f975406ecd0 0x7f9754071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:38:49.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.527+0000 7f9766a00700 1 -- 172.21.15.117:0/2357033768 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f975c08ee50 con 0x7f976c108e10 2024-04-20T11:38:49.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.527+0000 7f9771e00700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f975406ecd0 0x7f9754071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:38:49.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.528+0000 7f9771e00700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f975406ecd0 0x7f9754071190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9760006010 tx=0x7f976000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:38:49.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.532+0000 7f9766a00700 1 -- 172.21.15.117:0/2357033768 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f975c057c30 con 0x7f976c108e10 2024-04-20T11:38:49.531 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:49 smithi117 ceph-mon[99036]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:49.531 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:49 smithi117 ceph-mon[99036]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:49.531 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:49 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1457816077' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:38:49.532 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:49 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1065893451' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:38:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:49 smithi173 ceph-mon[74165]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:49 smithi173 ceph-mon[74165]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:49 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1457816077' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:38:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:49 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1065893451' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:38:49.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.843+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f976c19e4c0 con 0x7f976c108e10 2024-04-20T11:38:49.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.844+0000 7f9766a00700 1 -- 172.21.15.117:0/2357033768 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f975c05b2f0 con 0x7f976c108e10 2024-04-20T11:38:49.844 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:38:49.845 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:38:49.846 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:38:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.847+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f975406ecd0 msgr2=0x7f9754071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.847+0000 7f9774800700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f975406ecd0 0x7f9754071190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9760006010 tx=0x7f976000b540 comp rx=0 tx=0).stop 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.847+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c108e10 msgr2=0x7f976c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.847+0000 7f9774800700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c108e10 0x7f976c19d6f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d8d0 tx=0x7f975c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.848+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 shutdown_connections 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.848+0000 7f9774800700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f975406ecd0 0x7f9754071190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.848+0000 7f9774800700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c107520 0x7f976c19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.848+0000 7f9774800700 1 --2- 172.21.15.117:0/2357033768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f976c108e10 0x7f976c19d6f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.848+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 >> 172.21.15.117:0/2357033768 conn(0x7f976c0768f0 msgr2=0x7f976c077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.848+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 shutdown_connections 2024-04-20T11:38:49.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:38:49.848+0000 7f9774800700 1 -- 172.21.15.117:0/2357033768 wait complete. 2024-04-20T11:38:50.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:50 smithi117 ceph-mon[99036]: pgmap v792: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:50.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:50 smithi117 ceph-mon[99036]: from='client.35450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:50.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:50 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2357033768' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:38:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:50 smithi173 ceph-mon[74165]: pgmap v792: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:50 smithi173 ceph-mon[74165]: from='client.35450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:38:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:50 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2357033768' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:38:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:52 smithi117 ceph-mon[99036]: pgmap v793: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:52 smithi173 ceph-mon[74165]: pgmap v793: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:54 smithi117 ceph-mon[99036]: pgmap v794: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:54 smithi173 ceph-mon[74165]: pgmap v794: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:56 smithi117 ceph-mon[99036]: pgmap v795: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:56.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:56 smithi173 ceph-mon[74165]: pgmap v795: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:58 smithi117 ceph-mon[99036]: pgmap v796: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:38:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:38:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:58 smithi173 ceph-mon[74165]: pgmap v796: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:38:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:38:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:39:00.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:00 smithi117 ceph-mon[99036]: pgmap v797: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:00 smithi173 ceph-mon[74165]: pgmap v797: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:02.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:02 smithi117 ceph-mon[99036]: pgmap v798: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:02 smithi173 ceph-mon[74165]: pgmap v798: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:04.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:04 smithi117 ceph-mon[99036]: pgmap v799: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:04 smithi173 ceph-mon[74165]: pgmap v799: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:06.592 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:06 smithi117 ceph-mon[99036]: pgmap v800: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:06 smithi173 ceph-mon[74165]: pgmap v800: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:08.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:08 smithi117 ceph-mon[99036]: pgmap v801: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:08 smithi173 ceph-mon[74165]: pgmap v801: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:10 smithi117 ceph-mon[99036]: pgmap v802: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:10 smithi173 ceph-mon[74165]: pgmap v802: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:12.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:12 smithi117 ceph-mon[99036]: pgmap v803: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:12 smithi173 ceph-mon[74165]: pgmap v803: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:39:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:39:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:14 smithi117 ceph-mon[99036]: pgmap v804: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:14 smithi173 ceph-mon[74165]: pgmap v804: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:16 smithi117 ceph-mon[99036]: pgmap v805: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:16 smithi173 ceph-mon[74165]: pgmap v805: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:18 smithi117 ceph-mon[99036]: pgmap v806: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:18 smithi173 ceph-mon[74165]: pgmap v806: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:20.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.016+0000 7fad88800700 1 -- 172.21.15.117:0/1998200056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073280 msgr2=0x7fad80073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.016+0000 7fad88800700 1 --2- 172.21.15.117:0/1998200056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073280 0x7fad80073660 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fad74009910 tx=0x7fad74009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:20.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.016+0000 7fad88800700 1 -- 172.21.15.117:0/1998200056 shutdown_connections 2024-04-20T11:39:20.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.016+0000 7fad88800700 1 --2- 172.21.15.117:0/1998200056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad80073ba0 0x7fad80111680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.016+0000 7fad88800700 1 --2- 172.21.15.117:0/1998200056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073280 0x7fad80073660 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.016+0000 7fad88800700 1 -- 172.21.15.117:0/1998200056 >> 172.21.15.117:0/1998200056 conn(0x7fad800fc6a0 msgr2=0x7fad800feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:20.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.016+0000 7fad88800700 1 -- 172.21.15.117:0/1998200056 shutdown_connections 2024-04-20T11:39:20.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.016+0000 7fad88800700 1 -- 172.21.15.117:0/1998200056 wait complete. 2024-04-20T11:39:20.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.017+0000 7fad88800700 1 Processor -- start 2024-04-20T11:39:20.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.017+0000 7fad88800700 1 -- start start 2024-04-20T11:39:20.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.018+0000 7fad88800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad80073280 0x7fad8019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.018+0000 7fad88800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073ba0 0x7fad8019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.018+0000 7fad88800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8019dc80 con 0x7fad80073ba0 2024-04-20T11:39:20.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.018+0000 7fad88800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801a1950 con 0x7fad80073280 2024-04-20T11:39:20.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.018+0000 7fad85400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073ba0 0x7fad8019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:20.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.018+0000 7fad85400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073ba0 0x7fad8019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53830/0 (socket says 172.21.15.117:53830) 2024-04-20T11:39:20.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.018+0000 7fad85400700 1 -- 172.21.15.117:0/3415318280 learned_addr learned my addr 172.21.15.117:0/3415318280 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:20.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad85e00700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad80073280 0x7fad8019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:20.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad85400700 1 -- 172.21.15.117:0/3415318280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad80073280 msgr2=0x7fad8019d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad85400700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad80073280 0x7fad8019d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad85400700 1 -- 172.21.15.117:0/3415318280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad740095f0 con 0x7fad80073ba0 2024-04-20T11:39:20.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad85e00700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad80073280 0x7fad8019d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:39:20.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad85400700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073ba0 0x7fad8019d5a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d8d0 tx=0x7fad7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:20.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad7aa00700 1 -- 172.21.15.117:0/3415318280 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad70009940 con 0x7fad80073ba0 2024-04-20T11:39:20.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad7aa00700 1 -- 172.21.15.117:0/3415318280 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad70010460 con 0x7fad80073ba0 2024-04-20T11:39:20.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad801a1c30 con 0x7fad80073ba0 2024-04-20T11:39:20.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.019+0000 7fad7aa00700 1 -- 172.21.15.117:0/3415318280 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7000f5f0 con 0x7fad80073ba0 2024-04-20T11:39:20.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.020+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad801a2180 con 0x7fad80073ba0 2024-04-20T11:39:20.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.021+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8010ee00 con 0x7fad80073ba0 2024-04-20T11:39:20.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.022+0000 7fad7aa00700 1 -- 172.21.15.117:0/3415318280 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fad700105d0 con 0x7fad80073ba0 2024-04-20T11:39:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.023+0000 7fad7aa00700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad6806ecd0 0x7fad68071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.023+0000 7fad7aa00700 1 -- 172.21.15.117:0/3415318280 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fad7008edf0 con 0x7fad80073ba0 2024-04-20T11:39:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.023+0000 7fad85e00700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad6806ecd0 0x7fad68071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:20.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.024+0000 7fad85e00700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad6806ecd0 0x7fad68071190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fad74009fd0 tx=0x7fad7400b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:20.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.029+0000 7fad7aa00700 1 -- 172.21.15.117:0/3415318280 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fad70057bb0 con 0x7fad80073ba0 2024-04-20T11:39:20.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.273+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad80114350 con 0x7fad6806ecd0 2024-04-20T11:39:20.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.275+0000 7fad7aa00700 1 -- 172.21.15.117:0/3415318280 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fad80114350 con 0x7fad6806ecd0 2024-04-20T11:39:20.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad6806ecd0 msgr2=0x7fad68071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad6806ecd0 0x7fad68071190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fad74009fd0 tx=0x7fad7400b560 comp rx=0 tx=0).stop 2024-04-20T11:39:20.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073ba0 msgr2=0x7fad8019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073ba0 0x7fad8019d5a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d8d0 tx=0x7fad7000dc90 comp rx=0 tx=0).stop 2024-04-20T11:39:20.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 shutdown_connections 2024-04-20T11:39:20.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad6806ecd0 0x7fad68071190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad80073280 0x7fad8019d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 --2- 172.21.15.117:0/3415318280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad80073ba0 0x7fad8019d5a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 >> 172.21.15.117:0/3415318280 conn(0x7fad800fc6a0 msgr2=0x7fad80104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:20.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 shutdown_connections 2024-04-20T11:39:20.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.278+0000 7fad88800700 1 -- 172.21.15.117:0/3415318280 wait complete. 2024-04-20T11:39:20.294 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:39:20.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.432+0000 7f7922600700 1 -- 172.21.15.117:0/2881201669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 msgr2=0x7f791c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.432+0000 7f7922600700 1 --2- 172.21.15.117:0/2881201669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 0x7f791c109290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7908009910 tx=0x7f7908009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:20.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.432+0000 7f7922600700 1 -- 172.21.15.117:0/2881201669 shutdown_connections 2024-04-20T11:39:20.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.432+0000 7f7922600700 1 --2- 172.21.15.117:0/2881201669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 0x7f791c109290 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.432+0000 7f7922600700 1 --2- 172.21.15.117:0/2881201669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f791c107520 0x7f791c107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.432+0000 7f7922600700 1 -- 172.21.15.117:0/2881201669 >> 172.21.15.117:0/2881201669 conn(0x7f791c0768f0 msgr2=0x7f791c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:20.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.432+0000 7f7922600700 1 -- 172.21.15.117:0/2881201669 shutdown_connections 2024-04-20T11:39:20.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.432+0000 7f7922600700 1 -- 172.21.15.117:0/2881201669 wait complete. 2024-04-20T11:39:20.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.433+0000 7f7922600700 1 Processor -- start 2024-04-20T11:39:20.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.433+0000 7f7922600700 1 -- start start 2024-04-20T11:39:20.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.434+0000 7f7922600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f791c107520 0x7f791c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.434+0000 7f7922600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 0x7f791c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.434+0000 7f7922600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c19dd60 con 0x7f791c108e10 2024-04-20T11:39:20.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.434+0000 7f7922600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c19dea0 con 0x7f791c107520 2024-04-20T11:39:20.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.434+0000 7f791aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 0x7f791c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:20.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.434+0000 7f791aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 0x7f791c19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53848/0 (socket says 172.21.15.117:53848) 2024-04-20T11:39:20.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.434+0000 7f791aa00700 1 -- 172.21.15.117:0/1841594480 learned_addr learned my addr 172.21.15.117:0/1841594480 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:20.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.434+0000 7f791b400700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f791c107520 0x7f791c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:20.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f791aa00700 1 -- 172.21.15.117:0/1841594480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f791c107520 msgr2=0x7f791c19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f791aa00700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f791c107520 0x7f791c19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f791aa00700 1 -- 172.21.15.117:0/1841594480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080095f0 con 0x7f791c108e10 2024-04-20T11:39:20.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f791b400700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f791c107520 0x7f791c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:39:20.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f791aa00700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 0x7f791c19d710 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f79080092e0 tx=0x7f790800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:20.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f7913e00700 1 -- 172.21.15.117:0/1841594480 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f790801d070 con 0x7f791c108e10 2024-04-20T11:39:20.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f7913e00700 1 -- 172.21.15.117:0/1841594480 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f790800f460 con 0x7f791c108e10 2024-04-20T11:39:20.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c1a1b80 con 0x7f791c108e10 2024-04-20T11:39:20.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f7913e00700 1 -- 172.21.15.117:0/1841594480 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7908017630 con 0x7f791c108e10 2024-04-20T11:39:20.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.435+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c1a20f0 con 0x7f791c108e10 2024-04-20T11:39:20.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.436+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c10f870 con 0x7f791c108e10 2024-04-20T11:39:20.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.438+0000 7f7913e00700 1 -- 172.21.15.117:0/1841594480 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f790800f5d0 con 0x7f791c108e10 2024-04-20T11:39:20.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.438+0000 7f7913e00700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f790006ecd0 0x7f7900071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.439+0000 7f7913e00700 1 -- 172.21.15.117:0/1841594480 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7908090320 con 0x7f791c108e10 2024-04-20T11:39:20.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.439+0000 7f791b400700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f790006ecd0 0x7f7900071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:20.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.440+0000 7f791b400700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f790006ecd0 0x7f7900071190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f790c005fd0 tx=0x7f790c005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:20.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.444+0000 7f7913e00700 1 -- 172.21.15.117:0/1841594480 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f79080590e0 con 0x7f791c108e10 2024-04-20T11:39:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:20 smithi117 ceph-mon[99036]: pgmap v807: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:20 smithi173 ceph-mon[74165]: pgmap v807: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:20.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.687+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f791c19e500 con 0x7f790006ecd0 2024-04-20T11:39:20.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.690+0000 7f7913e00700 1 -- 172.21.15.117:0/1841594480 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f791c19e500 con 0x7f790006ecd0 2024-04-20T11:39:20.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.692+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f790006ecd0 msgr2=0x7f7900071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.692+0000 7f7922600700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f790006ecd0 0x7f7900071190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f790c005fd0 tx=0x7f790c005e20 comp rx=0 tx=0).stop 2024-04-20T11:39:20.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.692+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 msgr2=0x7f791c19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.692+0000 7f7922600700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 0x7f791c19d710 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f79080092e0 tx=0x7f790800be20 comp rx=0 tx=0).stop 2024-04-20T11:39:20.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.693+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 shutdown_connections 2024-04-20T11:39:20.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.693+0000 7f7922600700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f790006ecd0 0x7f7900071190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.693+0000 7f7922600700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f791c107520 0x7f791c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.693+0000 7f7922600700 1 --2- 172.21.15.117:0/1841594480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f791c108e10 0x7f791c19d710 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.693+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 >> 172.21.15.117:0/1841594480 conn(0x7f791c0768f0 msgr2=0x7f791c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:20.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.693+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 shutdown_connections 2024-04-20T11:39:20.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.693+0000 7f7922600700 1 -- 172.21.15.117:0/1841594480 wait complete. 2024-04-20T11:39:20.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.838+0000 7fa666c00700 1 -- 172.21.15.117:0/1421341539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 msgr2=0x7fa660103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.838+0000 7fa666c00700 1 --2- 172.21.15.117:0/1421341539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 0x7fa660103240 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa654009910 tx=0x7fa654009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:20.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.838+0000 7fa666c00700 1 -- 172.21.15.117:0/1421341539 shutdown_connections 2024-04-20T11:39:20.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.838+0000 7fa666c00700 1 --2- 172.21.15.117:0/1421341539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa660103810 0x7fa660107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.838+0000 7fa666c00700 1 --2- 172.21.15.117:0/1421341539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 0x7fa660103240 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.838+0000 7fa666c00700 1 -- 172.21.15.117:0/1421341539 >> 172.21.15.117:0/1421341539 conn(0x7fa6600fe710 msgr2=0x7fa660100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:20.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.838+0000 7fa666c00700 1 -- 172.21.15.117:0/1421341539 shutdown_connections 2024-04-20T11:39:20.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.838+0000 7fa666c00700 1 -- 172.21.15.117:0/1421341539 wait complete. 2024-04-20T11:39:20.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.839+0000 7fa666c00700 1 Processor -- start 2024-04-20T11:39:20.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.839+0000 7fa666c00700 1 -- start start 2024-04-20T11:39:20.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa666c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 0x7fa66019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa666c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa660103810 0x7fa66019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa666c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66019f2f0 con 0x7fa660102e60 2024-04-20T11:39:20.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa666c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6601989b0 con 0x7fa660103810 2024-04-20T11:39:20.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa665800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 0x7fa66019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:20.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa665800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 0x7fa66019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53866/0 (socket says 172.21.15.117:53866) 2024-04-20T11:39:20.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa665800700 1 -- 172.21.15.117:0/2702529614 learned_addr learned my addr 172.21.15.117:0/2702529614 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:20.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa665800700 1 -- 172.21.15.117:0/2702529614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa660103810 msgr2=0x7fa66019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:20.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa665800700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa660103810 0x7fa66019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:20.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.840+0000 7fa665800700 1 -- 172.21.15.117:0/2702529614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6540095f0 con 0x7fa660102e60 2024-04-20T11:39:20.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.841+0000 7fa665800700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 0x7fa66019e820 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa6540092e0 tx=0x7fa65400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:20.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.841+0000 7fa65e000700 1 -- 172.21.15.117:0/2702529614 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa65401d070 con 0x7fa660102e60 2024-04-20T11:39:20.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.841+0000 7fa65e000700 1 -- 172.21.15.117:0/2702529614 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa65400f460 con 0x7fa660102e60 2024-04-20T11:39:20.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.841+0000 7fa65e000700 1 -- 172.21.15.117:0/2702529614 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa654017630 con 0x7fa660102e60 2024-04-20T11:39:20.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.841+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa660198c30 con 0x7fa660102e60 2024-04-20T11:39:20.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.841+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa660199120 con 0x7fa660102e60 2024-04-20T11:39:20.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.842+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66010b200 con 0x7fa660102e60 2024-04-20T11:39:20.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.843+0000 7fa65e000700 1 -- 172.21.15.117:0/2702529614 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa65400f5d0 con 0x7fa660102e60 2024-04-20T11:39:20.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.844+0000 7fa65e000700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64806ecd0 0x7fa648071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:20.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.844+0000 7fa65e000700 1 -- 172.21.15.117:0/2702529614 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa654090320 con 0x7fa660102e60 2024-04-20T11:39:20.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.844+0000 7fa664e00700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64806ecd0 0x7fa648071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:20.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.845+0000 7fa664e00700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64806ecd0 0x7fa648071190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005fd0 tx=0x7fa64c005d00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:20.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:20.849+0000 7fa65e000700 1 -- 172.21.15.117:0/2702529614 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa65405cc10 con 0x7fa660102e60 2024-04-20T11:39:21.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.077+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa660199e20 con 0x7fa64806ecd0 2024-04-20T11:39:21.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.109+0000 7fa65e000700 1 -- 172.21.15.117:0/2702529614 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa660199e20 con 0x7fa64806ecd0 2024-04-20T11:39:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:39:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (27m) 3m ago 41m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:39:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (42m) 3m ago 42m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (40m) 3m ago 40m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (24m) 3m ago 42m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (23m) 3m ago 40m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (27m) 3m ago 41m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (19m) 3m ago 34m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (20m) 3m ago 34m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (3m) 3m ago 34m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (19m) 3m ago 34m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (30m) 3m ago 43m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (29m) 3m ago 40m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (25m) 3m ago 43m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (25m) 3m ago 40m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (29m) 3m ago 42m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:39:21.111 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (28m) 3m ago 40m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:39:21.112 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (23m) 3m ago 39m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:39:21.112 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (22m) 3m ago 38m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:39:21.112 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (22m) 3m ago 38m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:39:21.112 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (21m) 3m ago 37m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:39:21.112 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (21m) 3m ago 37m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:39:21.112 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (20m) 3m ago 36m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:39:21.112 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (28m) 3m ago 41m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:39:21.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64806ecd0 msgr2=0x7fa648071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:21.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64806ecd0 0x7fa648071190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005fd0 tx=0x7fa64c005d00 comp rx=0 tx=0).stop 2024-04-20T11:39:21.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 msgr2=0x7fa66019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:21.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 0x7fa66019e820 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa6540092e0 tx=0x7fa65400be20 comp rx=0 tx=0).stop 2024-04-20T11:39:21.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 shutdown_connections 2024-04-20T11:39:21.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64806ecd0 0x7fa648071190 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa660102e60 0x7fa66019e820 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 --2- 172.21.15.117:0/2702529614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa660103810 0x7fa66019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.112+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 >> 172.21.15.117:0/2702529614 conn(0x7fa6600fe710 msgr2=0x7fa6600ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:21.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.113+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 shutdown_connections 2024-04-20T11:39:21.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.113+0000 7fa666c00700 1 -- 172.21.15.117:0/2702529614 wait complete. 2024-04-20T11:39:21.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.261+0000 7f69ef200700 1 -- 172.21.15.117:0/711956882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e81070b0 msgr2=0x7f69e8107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:21.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.261+0000 7f69ef200700 1 --2- 172.21.15.117:0/711956882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e81070b0 0x7f69e8107510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009910 tx=0x7f69d8009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:21.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.261+0000 7f69ef200700 1 -- 172.21.15.117:0/711956882 shutdown_connections 2024-04-20T11:39:21.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.261+0000 7f69ef200700 1 --2- 172.21.15.117:0/711956882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e81070b0 0x7f69e8107510 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.261+0000 7f69ef200700 1 --2- 172.21.15.117:0/711956882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e810d0c0 0x7f69e810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.261+0000 7f69ef200700 1 -- 172.21.15.117:0/711956882 >> 172.21.15.117:0/711956882 conn(0x7f69e8076830 msgr2=0x7f69e8078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:21.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.261+0000 7f69ef200700 1 -- 172.21.15.117:0/711956882 shutdown_connections 2024-04-20T11:39:21.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.261+0000 7f69ef200700 1 -- 172.21.15.117:0/711956882 wait complete. 2024-04-20T11:39:21.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.262+0000 7f69ef200700 1 Processor -- start 2024-04-20T11:39:21.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.263+0000 7f69ef200700 1 -- start start 2024-04-20T11:39:21.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.263+0000 7f69ef200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e81070b0 0x7f69e819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:21.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.263+0000 7f69ef200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e810d0c0 0x7f69e819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:21.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.263+0000 7f69ef200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e819db80 con 0x7f69e810d0c0 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.263+0000 7f69ef200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e81a1850 con 0x7f69e81070b0 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.263+0000 7f69ed400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e810d0c0 0x7f69e819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.263+0000 7f69ed400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e810d0c0 0x7f69e819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53890/0 (socket says 172.21.15.117:53890) 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.263+0000 7f69ed400700 1 -- 172.21.15.117:0/1506181918 learned_addr learned my addr 172.21.15.117:0/1506181918 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69ede00700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e81070b0 0x7f69e819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69ed400700 1 -- 172.21.15.117:0/1506181918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e81070b0 msgr2=0x7f69e819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69ed400700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e81070b0 0x7f69e819cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69ed400700 1 -- 172.21.15.117:0/1506181918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80095f0 con 0x7f69e810d0c0 2024-04-20T11:39:21.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69ed400700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e810d0c0 0x7f69e819d4a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006010 tx=0x7f69d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69e2a00700 1 -- 172.21.15.117:0/1506181918 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d801d070 con 0x7f69e810d0c0 2024-04-20T11:39:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69e2a00700 1 -- 172.21.15.117:0/1506181918 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69d800f460 con 0x7f69e810d0c0 2024-04-20T11:39:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e81a1ad0 con 0x7f69e810d0c0 2024-04-20T11:39:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69e2a00700 1 -- 172.21.15.117:0/1506181918 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d8017630 con 0x7f69e810d0c0 2024-04-20T11:39:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.264+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e81a1fc0 con 0x7f69e810d0c0 2024-04-20T11:39:21.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.266+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e810f5c0 con 0x7f69e810d0c0 2024-04-20T11:39:21.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.267+0000 7f69e2a00700 1 -- 172.21.15.117:0/1506181918 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f69d800f5d0 con 0x7f69e810d0c0 2024-04-20T11:39:21.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.268+0000 7f69e2a00700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69d006ecd0 0x7f69d0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:21.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.268+0000 7f69e2a00700 1 -- 172.21.15.117:0/1506181918 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f69d80903c0 con 0x7f69e810d0c0 2024-04-20T11:39:21.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.268+0000 7f69ede00700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69d006ecd0 0x7f69d0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:21.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.269+0000 7f69ede00700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69d006ecd0 0x7f69d0071190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005950 tx=0x7f69dc005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:21.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.272+0000 7f69e2a00700 1 -- 172.21.15.117:0/1506181918 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f69d80591c0 con 0x7f69e810d0c0 2024-04-20T11:39:21.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.575+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f69e819e2a0 con 0x7f69e810d0c0 2024-04-20T11:39:21.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.576+0000 7f69e2a00700 1 -- 172.21.15.117:0/1506181918 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f69d805c880 con 0x7f69e810d0c0 2024-04-20T11:39:21.576 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:39:21.577 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:39:21.578 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:39:21.578 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:39:21.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69d006ecd0 msgr2=0x7f69d0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:21.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69d006ecd0 0x7f69d0071190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005950 tx=0x7f69dc005cf0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e810d0c0 msgr2=0x7f69e819d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:21.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e810d0c0 0x7f69e819d4a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006010 tx=0x7f69d800be20 comp rx=0 tx=0).stop 2024-04-20T11:39:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 shutdown_connections 2024-04-20T11:39:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69d006ecd0 0x7f69d0071190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e81070b0 0x7f69e819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 --2- 172.21.15.117:0/1506181918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e810d0c0 0x7f69e819d4a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 >> 172.21.15.117:0/1506181918 conn(0x7f69e8076830 msgr2=0x7f69e8078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 shutdown_connections 2024-04-20T11:39:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.579+0000 7f69ef200700 1 -- 172.21.15.117:0/1506181918 wait complete. 2024-04-20T11:39:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:21 smithi173 ceph-mon[74165]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:21.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.737+0000 7f86b6000700 1 -- 172.21.15.117:0/789237742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0100e20 msgr2=0x7f86b0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.737+0000 7f86b6000700 1 --2- 172.21.15.117:0/789237742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0100e20 0x7f86b0101200 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009910 tx=0x7f86a0009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:21.738 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:21 smithi117 ceph-mon[99036]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.737+0000 7f86b6000700 1 -- 172.21.15.117:0/789237742 shutdown_connections 2024-04-20T11:39:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.737+0000 7f86b6000700 1 --2- 172.21.15.117:0/789237742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b01017d0 0x7f86b0105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.737+0000 7f86b6000700 1 --2- 172.21.15.117:0/789237742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b0100e20 0x7f86b0101200 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.737+0000 7f86b6000700 1 -- 172.21.15.117:0/789237742 >> 172.21.15.117:0/789237742 conn(0x7f86b00fc6b0 msgr2=0x7f86b00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.737+0000 7f86b6000700 1 -- 172.21.15.117:0/789237742 shutdown_connections 2024-04-20T11:39:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.737+0000 7f86b6000700 1 -- 172.21.15.117:0/789237742 wait complete. 2024-04-20T11:39:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.738+0000 7f86b6000700 1 Processor -- start 2024-04-20T11:39:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.739+0000 7f86b6000700 1 -- start start 2024-04-20T11:39:21.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.739+0000 7f86b6000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b0100e20 0x7f86b019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:21.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.739+0000 7f86b6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b01017d0 0x7f86b019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:21.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.739+0000 7f86b6000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b019d440 con 0x7f86b01017d0 2024-04-20T11:39:21.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.739+0000 7f86b6000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b0196a20 con 0x7f86b0100e20 2024-04-20T11:39:21.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.739+0000 7f86aea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b01017d0 0x7f86b019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:21.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.740+0000 7f86aea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b01017d0 0x7f86b019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53902/0 (socket says 172.21.15.117:53902) 2024-04-20T11:39:21.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.740+0000 7f86aea00700 1 -- 172.21.15.117:0/3910953979 learned_addr learned my addr 172.21.15.117:0/3910953979 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:21.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.740+0000 7f86af400700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b0100e20 0x7f86b019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:21.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.740+0000 7f86aea00700 1 -- 172.21.15.117:0/3910953979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b0100e20 msgr2=0x7f86b019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:21.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.740+0000 7f86aea00700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b0100e20 0x7f86b019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:21.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.740+0000 7f86aea00700 1 -- 172.21.15.117:0/3910953979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a00095f0 con 0x7f86b01017d0 2024-04-20T11:39:21.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.740+0000 7f86aea00700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b01017d0 0x7f86b019ce20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b480 tx=0x7f869c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:21.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.741+0000 7f86a7e00700 1 -- 172.21.15.117:0/3910953979 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f869c0118c0 con 0x7f86b01017d0 2024-04-20T11:39:21.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.741+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b0196d00 con 0x7f86b01017d0 2024-04-20T11:39:21.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.741+0000 7f86a7e00700 1 -- 172.21.15.117:0/3910953979 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f869c010380 con 0x7f86b01017d0 2024-04-20T11:39:21.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.741+0000 7f86a7e00700 1 -- 172.21.15.117:0/3910953979 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f869c00f4f0 con 0x7f86b01017d0 2024-04-20T11:39:21.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.741+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b0197250 con 0x7f86b01017d0 2024-04-20T11:39:21.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.742+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b0102300 con 0x7f86b01017d0 2024-04-20T11:39:21.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.744+0000 7f86a7e00700 1 -- 172.21.15.117:0/3910953979 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f869c0104f0 con 0x7f86b01017d0 2024-04-20T11:39:21.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.746+0000 7f86a7e00700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f869406ecc0 0x7f8694071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:21.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.746+0000 7f86a7e00700 1 -- 172.21.15.117:0/3910953979 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f869c08eb00 con 0x7f86b01017d0 2024-04-20T11:39:21.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.746+0000 7f86af400700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f869406ecc0 0x7f8694071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:21.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.747+0000 7f86af400700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f869406ecc0 0x7f8694071180 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b5c0 tx=0x7f86a001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:21.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:21.751+0000 7f86a7e00700 1 -- 172.21.15.117:0/3910953979 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f869c05b370 con 0x7f86b01017d0 2024-04-20T11:39:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.024+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f86b0062510 con 0x7f86b01017d0 2024-04-20T11:39:22.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.025+0000 7f86a7e00700 1 -- 172.21.15.117:0/3910953979 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f869c05af00 con 0x7f86b01017d0 2024-04-20T11:39:22.026 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:39:22.027 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:39:22.028 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:39:22.029 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f869406ecc0 msgr2=0x7f8694071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f869406ecc0 0x7f8694071180 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b5c0 tx=0x7f86a001a040 comp rx=0 tx=0).stop 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b01017d0 msgr2=0x7f86b019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b01017d0 0x7f86b019ce20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b480 tx=0x7f869c00b840 comp rx=0 tx=0).stop 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 shutdown_connections 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f869406ecc0 0x7f8694071180 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b0100e20 0x7f86b019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 --2- 172.21.15.117:0/3910953979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86b01017d0 0x7f86b019ce20 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 >> 172.21.15.117:0/3910953979 conn(0x7f86b00fc6b0 msgr2=0x7f86b00fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 shutdown_connections 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.029+0000 7f86b6000700 1 -- 172.21.15.117:0/3910953979 wait complete. 2024-04-20T11:39:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:39:22.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.182+0000 7fb4df800700 1 -- 172.21.15.117:0/594215779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 msgr2=0x7fb4d8105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.182+0000 7fb4df800700 1 --2- 172.21.15.117:0/594215779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 0x7fb4d8105e60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009910 tx=0x7fb4cc009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.182+0000 7fb4df800700 1 -- 172.21.15.117:0/594215779 shutdown_connections 2024-04-20T11:39:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.182+0000 7fb4df800700 1 --2- 172.21.15.117:0/594215779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4d80ffa80 0x7fb4d80fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.182+0000 7fb4df800700 1 --2- 172.21.15.117:0/594215779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 0x7fb4d8105e60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.182+0000 7fb4df800700 1 -- 172.21.15.117:0/594215779 >> 172.21.15.117:0/594215779 conn(0x7fb4d8076750 msgr2=0x7fb4d80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.182+0000 7fb4df800700 1 -- 172.21.15.117:0/594215779 shutdown_connections 2024-04-20T11:39:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.183+0000 7fb4df800700 1 -- 172.21.15.117:0/594215779 wait complete. 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.183+0000 7fb4df800700 1 Processor -- start 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.184+0000 7fb4df800700 1 -- start start 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.184+0000 7fb4df800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4d80ffa80 0x7fb4d819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.184+0000 7fb4df800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 0x7fb4d819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.184+0000 7fb4df800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d819d350 con 0x7fb4d8105a80 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.184+0000 7fb4df800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d8196930 con 0x7fb4d80ffa80 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.184+0000 7fb4d7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 0x7fb4d819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.184+0000 7fb4d7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 0x7fb4d819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53912/0 (socket says 172.21.15.117:53912) 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.184+0000 7fb4d7e00700 1 -- 172.21.15.117:0/3507822242 learned_addr learned my addr 172.21.15.117:0/3507822242 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4dce00700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4d80ffa80 0x7fb4d819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4d7e00700 1 -- 172.21.15.117:0/3507822242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4d80ffa80 msgr2=0x7fb4d819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4d7e00700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4d80ffa80 0x7fb4d819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4d7e00700 1 -- 172.21.15.117:0/3507822242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0095f0 con 0x7fb4d8105a80 2024-04-20T11:39:22.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4dce00700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4d80ffa80 0x7fb4d819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:39:22.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4d7e00700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 0x7fb4d819cd30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400d8d0 tx=0x7fb4c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:22.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4d5600700 1 -- 172.21.15.117:0/3507822242 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4c4009940 con 0x7fb4d8105a80 2024-04-20T11:39:22.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4d5600700 1 -- 172.21.15.117:0/3507822242 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4c4010460 con 0x7fb4d8105a80 2024-04-20T11:39:22.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d8196c10 con 0x7fb4d8105a80 2024-04-20T11:39:22.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4d5600700 1 -- 172.21.15.117:0/3507822242 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4c400f5f0 con 0x7fb4d8105a80 2024-04-20T11:39:22.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.185+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d8197160 con 0x7fb4d8105a80 2024-04-20T11:39:22.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.187+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d81011c0 con 0x7fb4d8105a80 2024-04-20T11:39:22.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.188+0000 7fb4d5600700 1 -- 172.21.15.117:0/3507822242 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb4c40105d0 con 0x7fb4d8105a80 2024-04-20T11:39:22.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.189+0000 7fb4d5600700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb4c006ecc0 0x7fb4c0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:22.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.189+0000 7fb4d5600700 1 -- 172.21.15.117:0/3507822242 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb4c408eef0 con 0x7fb4d8105a80 2024-04-20T11:39:22.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.189+0000 7fb4dce00700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb4c006ecc0 0x7fb4c0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:22.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.190+0000 7fb4dce00700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb4c006ecc0 0x7fb4c0071180 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006010 tx=0x7fb4cc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:22.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.193+0000 7fb4d5600700 1 -- 172.21.15.117:0/3507822242 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb4c405b760 con 0x7fb4d8105a80 2024-04-20T11:39:22.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.424+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4d810cd20 con 0x7fb4c006ecc0 2024-04-20T11:39:22.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.426+0000 7fb4d5600700 1 -- 172.21.15.117:0/3507822242 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb4d810cd20 con 0x7fb4c006ecc0 2024-04-20T11:39:22.427 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:39:22.427 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:39:22.427 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:39:22.427 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:39:22.427 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:39:22.428 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:39:22.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb4c006ecc0 msgr2=0x7fb4c0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb4c006ecc0 0x7fb4c0071180 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006010 tx=0x7fb4cc00b540 comp rx=0 tx=0).stop 2024-04-20T11:39:22.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 msgr2=0x7fb4d819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 0x7fb4d819cd30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400d8d0 tx=0x7fb4c400dc90 comp rx=0 tx=0).stop 2024-04-20T11:39:22.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 shutdown_connections 2024-04-20T11:39:22.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb4c006ecc0 0x7fb4c0071180 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4d80ffa80 0x7fb4d819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 --2- 172.21.15.117:0/3507822242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4d8105a80 0x7fb4d819cd30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 >> 172.21.15.117:0/3507822242 conn(0x7fb4d8076750 msgr2=0x7fb4d80687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:22.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 shutdown_connections 2024-04-20T11:39:22.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.429+0000 7fb4df800700 1 -- 172.21.15.117:0/3507822242 wait complete. 2024-04-20T11:39:22.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.574+0000 7fe8ef200700 1 -- 172.21.15.117:0/4108610581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 msgr2=0x7fe8e81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.574+0000 7fe8ef200700 1 --2- 172.21.15.117:0/4108610581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 0x7fe8e81011f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009910 tx=0x7fe8d8009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:22.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.574+0000 7fe8ef200700 1 -- 172.21.15.117:0/4108610581 shutdown_connections 2024-04-20T11:39:22.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.574+0000 7fe8ef200700 1 --2- 172.21.15.117:0/4108610581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e81017c0 0x7fe8e8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.574+0000 7fe8ef200700 1 --2- 172.21.15.117:0/4108610581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 0x7fe8e81011f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.574+0000 7fe8ef200700 1 -- 172.21.15.117:0/4108610581 >> 172.21.15.117:0/4108610581 conn(0x7fe8e80fc6c0 msgr2=0x7fe8e80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:22.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.574+0000 7fe8ef200700 1 -- 172.21.15.117:0/4108610581 shutdown_connections 2024-04-20T11:39:22.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.574+0000 7fe8ef200700 1 -- 172.21.15.117:0/4108610581 wait complete. 2024-04-20T11:39:22.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.575+0000 7fe8ef200700 1 Processor -- start 2024-04-20T11:39:22.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.575+0000 7fe8ef200700 1 -- start start 2024-04-20T11:39:22.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8ef200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 0x7fe8e8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:22.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8ef200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e81017c0 0x7fe8e8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:22.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8ef200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81998d0 con 0x7fe8e8100e10 2024-04-20T11:39:22.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8ef200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e819c540 con 0x7fe8e81017c0 2024-04-20T11:39:22.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8e7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 0x7fe8e8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:22.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8e7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 0x7fe8e8198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53936/0 (socket says 172.21.15.117:53936) 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8e7e00700 1 -- 172.21.15.117:0/3834514786 learned_addr learned my addr 172.21.15.117:0/3834514786 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8df400700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e81017c0 0x7fe8e8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8e7e00700 1 -- 172.21.15.117:0/3834514786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e81017c0 msgr2=0x7fe8e8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.576+0000 7fe8e7e00700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e81017c0 0x7fe8e8199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.577+0000 7fe8e7e00700 1 -- 172.21.15.117:0/3834514786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80095f0 con 0x7fe8e8100e10 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.577+0000 7fe8df400700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e81017c0 0x7fe8e8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.577+0000 7fe8e7e00700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 0x7fe8e8198d00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8006010 tx=0x7fe8d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.577+0000 7fe8e5600700 1 -- 172.21.15.117:0/3834514786 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8d801d070 con 0x7fe8e8100e10 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.577+0000 7fe8e5600700 1 -- 172.21.15.117:0/3834514786 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8d800f460 con 0x7fe8e8100e10 2024-04-20T11:39:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.577+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e819c7c0 con 0x7fe8e8100e10 2024-04-20T11:39:22.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.577+0000 7fe8e5600700 1 -- 172.21.15.117:0/3834514786 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8d8017630 con 0x7fe8e8100e10 2024-04-20T11:39:22.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.577+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e819ccb0 con 0x7fe8e8100e10 2024-04-20T11:39:22.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.578+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e8103f80 con 0x7fe8e8100e10 2024-04-20T11:39:22.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.579+0000 7fe8e5600700 1 -- 172.21.15.117:0/3834514786 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe8d800fab0 con 0x7fe8e8100e10 2024-04-20T11:39:22.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.580+0000 7fe8e5600700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8cc06ecc0 0x7fe8cc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:22.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.581+0000 7fe8e5600700 1 -- 172.21.15.117:0/3834514786 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe8d80902e0 con 0x7fe8e8100e10 2024-04-20T11:39:22.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.581+0000 7fe8df400700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8cc06ecc0 0x7fe8cc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:22.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.581+0000 7fe8df400700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8cc06ecc0 0x7fe8cc071180 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0005fd0 tx=0x7fe8d0005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:22.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.585+0000 7fe8e5600700 1 -- 172.21.15.117:0/3834514786 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe8d80590c0 con 0x7fe8e8100e10 2024-04-20T11:39:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:22 smithi173 ceph-mon[74165]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:22 smithi173 ceph-mon[74165]: pgmap v808: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:22 smithi173 ceph-mon[74165]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:22 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1506181918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:39:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:22 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3910953979' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:39:22.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.876+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8e819a070 con 0x7fe8e8100e10 2024-04-20T11:39:22.877 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:22 smithi117 ceph-mon[99036]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:22.877 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:22 smithi117 ceph-mon[99036]: pgmap v808: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:22.877 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:22 smithi117 ceph-mon[99036]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:22.877 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:22 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1506181918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:39:22.877 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:22 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3910953979' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:39:22.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.877+0000 7fe8e5600700 1 -- 172.21.15.117:0/3834514786 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fe8d805c780 con 0x7fe8e8100e10 2024-04-20T11:39:22.878 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:39:22.878 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:39:22.878 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:39:22.878 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:39:22.879 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:39:22.880 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:39:22.880 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:39:22.880 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:39:22.880 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:39:22.880 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:39:22.880 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:39:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.880+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8cc06ecc0 msgr2=0x7fe8cc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.880+0000 7fe8ef200700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8cc06ecc0 0x7fe8cc071180 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0005fd0 tx=0x7fe8d0005dc0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.880+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 msgr2=0x7fe8e8198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.880+0000 7fe8ef200700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 0x7fe8e8198d00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8006010 tx=0x7fe8d800be20 comp rx=0 tx=0).stop 2024-04-20T11:39:22.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.881+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 shutdown_connections 2024-04-20T11:39:22.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.881+0000 7fe8ef200700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8cc06ecc0 0x7fe8cc071180 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.881+0000 7fe8ef200700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8100e10 0x7fe8e8198d00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.881+0000 7fe8ef200700 1 --2- 172.21.15.117:0/3834514786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e81017c0 0x7fe8e8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:22.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.881+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 >> 172.21.15.117:0/3834514786 conn(0x7fe8e80fc6c0 msgr2=0x7fe8e80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:22.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.881+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 shutdown_connections 2024-04-20T11:39:22.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:22.881+0000 7fe8ef200700 1 -- 172.21.15.117:0/3834514786 wait complete. 2024-04-20T11:39:23.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:23 smithi117 ceph-mon[99036]: from='client.35478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:23 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3834514786' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:39:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:23 smithi173 ceph-mon[74165]: from='client.35478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:23 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3834514786' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:39:24.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:24 smithi117 ceph-mon[99036]: pgmap v809: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:24 smithi173 ceph-mon[74165]: pgmap v809: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:26.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:26 smithi117 ceph-mon[99036]: pgmap v810: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:26 smithi173 ceph-mon[74165]: pgmap v810: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:28 smithi117 ceph-mon[99036]: pgmap v811: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:39:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:28 smithi173 ceph-mon[74165]: pgmap v811: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:39:30.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:29 smithi117 ceph-mon[99036]: pgmap v812: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:30.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:29 smithi173 ceph-mon[74165]: pgmap v812: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:32.357 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:32 smithi117 ceph-mon[99036]: pgmap v813: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:32.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:32 smithi173 ceph-mon[74165]: pgmap v813: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:34.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:34 smithi117 ceph-mon[99036]: pgmap v814: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:34.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:34 smithi173 ceph-mon[74165]: pgmap v814: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:36.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:36 smithi117 ceph-mon[99036]: pgmap v815: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:36.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:36 smithi173 ceph-mon[74165]: pgmap v815: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:38.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:38 smithi117 ceph-mon[99036]: pgmap v816: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:38.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:38 smithi173 ceph-mon[74165]: pgmap v816: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:40.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:39 smithi117 ceph-mon[99036]: pgmap v817: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:39 smithi173 ceph-mon[74165]: pgmap v817: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:42.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:42 smithi117 ceph-mon[99036]: pgmap v818: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:42.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:42 smithi173 ceph-mon[74165]: pgmap v818: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:43.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:39:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:39:44.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:44 smithi117 ceph-mon[99036]: pgmap v819: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:44 smithi173 ceph-mon[74165]: pgmap v819: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:46.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:46 smithi117 ceph-mon[99036]: pgmap v820: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:46 smithi173 ceph-mon[74165]: pgmap v820: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:48.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:48 smithi117 ceph-mon[99036]: pgmap v821: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:48.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:48 smithi173 ceph-mon[74165]: pgmap v821: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:50 smithi117 ceph-mon[99036]: pgmap v822: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:50 smithi173 ceph-mon[74165]: pgmap v822: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:52.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:52 smithi117 ceph-mon[99036]: pgmap v823: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:52.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:52 smithi173 ceph-mon[74165]: pgmap v823: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:53.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.057+0000 7f601aa00700 1 -- 172.21.15.117:0/1791501377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 msgr2=0x7f6014109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.057+0000 7f601aa00700 1 --2- 172.21.15.117:0/1791501377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 0x7f6014109290 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009910 tx=0x7f5ffc009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:53.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.057+0000 7f601aa00700 1 -- 172.21.15.117:0/1791501377 shutdown_connections 2024-04-20T11:39:53.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.057+0000 7f601aa00700 1 --2- 172.21.15.117:0/1791501377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 0x7f6014109290 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.057+0000 7f601aa00700 1 --2- 172.21.15.117:0/1791501377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6014107520 0x7f6014107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.057+0000 7f601aa00700 1 -- 172.21.15.117:0/1791501377 >> 172.21.15.117:0/1791501377 conn(0x7f60140768f0 msgr2=0x7f6014078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:53.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.057+0000 7f601aa00700 1 -- 172.21.15.117:0/1791501377 shutdown_connections 2024-04-20T11:39:53.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.057+0000 7f601aa00700 1 -- 172.21.15.117:0/1791501377 wait complete. 2024-04-20T11:39:53.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.058+0000 7f601aa00700 1 Processor -- start 2024-04-20T11:39:53.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.058+0000 7f601aa00700 1 -- start start 2024-04-20T11:39:53.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.059+0000 7f601aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6014107520 0x7f601419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.059+0000 7f601aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 0x7f601419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.059+0000 7f601aa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601419ddf0 con 0x7f6014108e10 2024-04-20T11:39:53.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.059+0000 7f601aa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60141a1a70 con 0x7f6014107520 2024-04-20T11:39:53.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.059+0000 7f600b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 0x7f601419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.059+0000 7f600b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 0x7f601419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56702/0 (socket says 172.21.15.117:56702) 2024-04-20T11:39:53.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.059+0000 7f600b400700 1 -- 172.21.15.117:0/1293764801 learned_addr learned my addr 172.21.15.117:0/1293764801 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:53.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.060+0000 7f6013e00700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6014107520 0x7f601419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.060+0000 7f600b400700 1 -- 172.21.15.117:0/1293764801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6014107520 msgr2=0x7f601419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.060+0000 7f600b400700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6014107520 0x7f601419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.060+0000 7f600b400700 1 -- 172.21.15.117:0/1293764801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0095f0 con 0x7f6014108e10 2024-04-20T11:39:53.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.060+0000 7f600b400700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 0x7f601419d710 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0092e0 tx=0x7f5ffc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:53.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.060+0000 7f6011600700 1 -- 172.21.15.117:0/1293764801 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ffc01d070 con 0x7f6014108e10 2024-04-20T11:39:53.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.060+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60141a1cf0 con 0x7f6014108e10 2024-04-20T11:39:53.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.060+0000 7f6011600700 1 -- 172.21.15.117:0/1293764801 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ffc0056e0 con 0x7f6014108e10 2024-04-20T11:39:53.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.061+0000 7f6011600700 1 -- 172.21.15.117:0/1293764801 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ffc00fe80 con 0x7f6014108e10 2024-04-20T11:39:53.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.061+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60141a2260 con 0x7f6014108e10 2024-04-20T11:39:53.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.062+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601410f870 con 0x7f6014108e10 2024-04-20T11:39:53.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.064+0000 7f6011600700 1 -- 172.21.15.117:0/1293764801 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5ffc00f6f0 con 0x7f6014108e10 2024-04-20T11:39:53.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.065+0000 7f6011600700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ff806ecd0 0x7f5ff8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.065+0000 7f6011600700 1 -- 172.21.15.117:0/1293764801 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5ffc090300 con 0x7f6014108e10 2024-04-20T11:39:53.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.065+0000 7f6013e00700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ff806ecd0 0x7f5ff8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.066+0000 7f6013e00700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ff806ecd0 0x7f5ff8071190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6004005950 tx=0x7f60040058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:53.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.071+0000 7f6011600700 1 -- 172.21.15.117:0/1293764801 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5ffc059040 con 0x7f6014108e10 2024-04-20T11:39:53.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.331+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f601419e500 con 0x7f5ff806ecd0 2024-04-20T11:39:53.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.333+0000 7f6011600700 1 -- 172.21.15.117:0/1293764801 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f601419e500 con 0x7f5ff806ecd0 2024-04-20T11:39:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ff806ecd0 msgr2=0x7f5ff8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ff806ecd0 0x7f5ff8071190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6004005950 tx=0x7f60040058e0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 msgr2=0x7f601419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 0x7f601419d710 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0092e0 tx=0x7f5ffc00be20 comp rx=0 tx=0).stop 2024-04-20T11:39:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 shutdown_connections 2024-04-20T11:39:53.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5ff806ecd0 0x7f5ff8071190 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6014107520 0x7f601419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 --2- 172.21.15.117:0/1293764801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6014108e10 0x7f601419d710 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.335+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 >> 172.21.15.117:0/1293764801 conn(0x7f60140768f0 msgr2=0x7f6014077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.336+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 shutdown_connections 2024-04-20T11:39:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.336+0000 7f601aa00700 1 -- 172.21.15.117:0/1293764801 wait complete. 2024-04-20T11:39:53.351 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:39:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.495+0000 7fdc5b000700 1 -- 172.21.15.117:0/1897094816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54102e60 msgr2=0x7fdc54103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.495+0000 7fdc5b000700 1 --2- 172.21.15.117:0/1897094816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54102e60 0x7fdc54103240 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009910 tx=0x7fdc48009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.495+0000 7fdc5b000700 1 -- 172.21.15.117:0/1897094816 shutdown_connections 2024-04-20T11:39:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.495+0000 7fdc5b000700 1 --2- 172.21.15.117:0/1897094816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc54103810 0x7fdc54107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.495+0000 7fdc5b000700 1 --2- 172.21.15.117:0/1897094816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54102e60 0x7fdc54103240 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.495+0000 7fdc5b000700 1 -- 172.21.15.117:0/1897094816 >> 172.21.15.117:0/1897094816 conn(0x7fdc540fe710 msgr2=0x7fdc54100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.495+0000 7fdc5b000700 1 -- 172.21.15.117:0/1897094816 shutdown_connections 2024-04-20T11:39:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.495+0000 7fdc5b000700 1 -- 172.21.15.117:0/1897094816 wait complete. 2024-04-20T11:39:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.496+0000 7fdc5b000700 1 Processor -- start 2024-04-20T11:39:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.496+0000 7fdc5b000700 1 -- start start 2024-04-20T11:39:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc5b000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc54102e60 0x7fdc5419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc5b000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54103810 0x7fdc5419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc5b000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5419f4c0 con 0x7fdc54103810 2024-04-20T11:39:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc5b000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc54198aa0 con 0x7fdc54102e60 2024-04-20T11:39:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc59200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54103810 0x7fdc5419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc59200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54103810 0x7fdc5419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56724/0 (socket says 172.21.15.117:56724) 2024-04-20T11:39:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc59200700 1 -- 172.21.15.117:0/50939166 learned_addr learned my addr 172.21.15.117:0/50939166 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc59c00700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc54102e60 0x7fdc5419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc59200700 1 -- 172.21.15.117:0/50939166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc54102e60 msgr2=0x7fdc5419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc59200700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc54102e60 0x7fdc5419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.497+0000 7fdc59200700 1 -- 172.21.15.117:0/50939166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480095f0 con 0x7fdc54103810 2024-04-20T11:39:53.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.498+0000 7fdc59200700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54103810 0x7fdc5419eea0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000b480 tx=0x7fdc4000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:53.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.498+0000 7fdc52000700 1 -- 172.21.15.117:0/50939166 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc400118c0 con 0x7fdc54103810 2024-04-20T11:39:53.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.498+0000 7fdc52000700 1 -- 172.21.15.117:0/50939166 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc40010380 con 0x7fdc54103810 2024-04-20T11:39:53.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.498+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc54198d80 con 0x7fdc54103810 2024-04-20T11:39:53.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.498+0000 7fdc52000700 1 -- 172.21.15.117:0/50939166 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4000f4f0 con 0x7fdc54103810 2024-04-20T11:39:53.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.498+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc541992d0 con 0x7fdc54103810 2024-04-20T11:39:53.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.499+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5410b200 con 0x7fdc54103810 2024-04-20T11:39:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.500+0000 7fdc52000700 1 -- 172.21.15.117:0/50939166 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdc400104f0 con 0x7fdc54103810 2024-04-20T11:39:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.501+0000 7fdc52000700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecd0 0x7fdc3c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.502+0000 7fdc52000700 1 -- 172.21.15.117:0/50939166 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdc4008ea40 con 0x7fdc54103810 2024-04-20T11:39:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.502+0000 7fdc59c00700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecd0 0x7fdc3c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.502+0000 7fdc59c00700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecd0 0x7fdc3c071190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800b5c0 tx=0x7fdc4801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:53.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.507+0000 7fdc52000700 1 -- 172.21.15.117:0/50939166 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdc4005b330 con 0x7fdc54103810 2024-04-20T11:39:53.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.760+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc5410fea0 con 0x7fdc3c06ecd0 2024-04-20T11:39:53.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.761+0000 7fdc52000700 1 -- 172.21.15.117:0/50939166 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdc5410fea0 con 0x7fdc3c06ecd0 2024-04-20T11:39:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecd0 msgr2=0x7fdc3c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecd0 0x7fdc3c071190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800b5c0 tx=0x7fdc4801a040 comp rx=0 tx=0).stop 2024-04-20T11:39:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54103810 msgr2=0x7fdc5419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54103810 0x7fdc5419eea0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000b480 tx=0x7fdc4000b840 comp rx=0 tx=0).stop 2024-04-20T11:39:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 shutdown_connections 2024-04-20T11:39:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecd0 0x7fdc3c071190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc54102e60 0x7fdc5419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 --2- 172.21.15.117:0/50939166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54103810 0x7fdc5419eea0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 >> 172.21.15.117:0/50939166 conn(0x7fdc540fe710 msgr2=0x7fdc540ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 shutdown_connections 2024-04-20T11:39:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.764+0000 7fdc5b000700 1 -- 172.21.15.117:0/50939166 wait complete. 2024-04-20T11:39:53.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.923+0000 7f494d400700 1 -- 172.21.15.117:0/1826394432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4948105a80 msgr2=0x7f4948105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.923+0000 7f494d400700 1 --2- 172.21.15.117:0/1826394432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4948105a80 0x7f4948105e60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4938009910 tx=0x7f4938009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:53.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.923+0000 7f494d400700 1 -- 172.21.15.117:0/1826394432 shutdown_connections 2024-04-20T11:39:53.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.923+0000 7f494d400700 1 --2- 172.21.15.117:0/1826394432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49480ffa80 0x7f49480fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.923+0000 7f494d400700 1 --2- 172.21.15.117:0/1826394432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4948105a80 0x7f4948105e60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.923+0000 7f494d400700 1 -- 172.21.15.117:0/1826394432 >> 172.21.15.117:0/1826394432 conn(0x7f4948076750 msgr2=0x7f49480fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:53.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.923+0000 7f494d400700 1 -- 172.21.15.117:0/1826394432 shutdown_connections 2024-04-20T11:39:53.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.923+0000 7f494d400700 1 -- 172.21.15.117:0/1826394432 wait complete. 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.924+0000 7f494d400700 1 Processor -- start 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.925+0000 7f494d400700 1 -- start start 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.925+0000 7f494d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49480ffa80 0x7f49480fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.925+0000 7f494d400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4948105a80 0x7f49480fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.925+0000 7f494d400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480fc5c0 con 0x7f49480ffa80 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.925+0000 7f494d400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480fc700 con 0x7f4948105a80 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.925+0000 7f4946800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49480ffa80 0x7f49480fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.925+0000 7f4946800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49480ffa80 0x7f49480fbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43720/0 (socket says 172.21.15.117:43720) 2024-04-20T11:39:53.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.925+0000 7f4946800700 1 -- 172.21.15.117:0/217257373 learned_addr learned my addr 172.21.15.117:0/217257373 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:53.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f4945e00700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4948105a80 0x7f49480fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f4946800700 1 -- 172.21.15.117:0/217257373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4948105a80 msgr2=0x7f49480fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:53.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f4946800700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4948105a80 0x7f49480fc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:53.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f4946800700 1 -- 172.21.15.117:0/217257373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49380095f0 con 0x7f49480ffa80 2024-04-20T11:39:53.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f4945e00700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4948105a80 0x7f49480fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:39:53.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f4946800700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49480ffa80 0x7f49480fbb40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f49380092e0 tx=0x7f493800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:53.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f493f400700 1 -- 172.21.15.117:0/217257373 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f493801d070 con 0x7f49480ffa80 2024-04-20T11:39:53.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f493f400700 1 -- 172.21.15.117:0/217257373 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49380056e0 con 0x7f49480ffa80 2024-04-20T11:39:53.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49480fc980 con 0x7f49480ffa80 2024-04-20T11:39:53.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.926+0000 7f493f400700 1 -- 172.21.15.117:0/217257373 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f493800fe80 con 0x7f49480ffa80 2024-04-20T11:39:53.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.927+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49481a2f30 con 0x7f49480ffa80 2024-04-20T11:39:53.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.928+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49481011c0 con 0x7f49480ffa80 2024-04-20T11:39:53.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.929+0000 7f493f400700 1 -- 172.21.15.117:0/217257373 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f493800f6f0 con 0x7f49480ffa80 2024-04-20T11:39:53.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.930+0000 7f493f400700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f492c06ecc0 0x7f492c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:53.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.930+0000 7f493f400700 1 -- 172.21.15.117:0/217257373 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f4938090230 con 0x7f49480ffa80 2024-04-20T11:39:53.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.930+0000 7f4945e00700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f492c06ecc0 0x7f492c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:53.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.931+0000 7f4945e00700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f492c06ecc0 0x7f492c071180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4934005fd0 tx=0x7f4934005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:53.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:53.935+0000 7f493f400700 1 -- 172.21.15.117:0/217257373 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4938058ff0 con 0x7f49480ffa80 2024-04-20T11:39:54.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.182+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f494810cd40 con 0x7f492c06ecc0 2024-04-20T11:39:54.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.213+0000 7f493f400700 1 -- 172.21.15.117:0/217257373 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f494810cd40 con 0x7f492c06ecc0 2024-04-20T11:39:54.213 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:39:54.213 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (28m) 4m ago 42m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (42m) 4m ago 42m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (41m) 4m ago 41m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (24m) 4m ago 42m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (24m) 4m ago 41m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (27m) 4m ago 42m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (20m) 4m ago 35m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (20m) 4m ago 35m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (4m) 4m ago 35m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (19m) 4m ago 35m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (31m) 4m ago 44m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (30m) 4m ago 41m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (26m) 4m ago 44m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (25m) 4m ago 41m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (29m) 4m ago 42m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (29m) 4m ago 41m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:39:54.214 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (24m) 4m ago 40m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (23m) 4m ago 39m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (22m) 4m ago 38m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (22m) 4m ago 38m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (21m) 4m ago 37m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (21m) 4m ago 37m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (28m) 4m ago 41m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f492c06ecc0 msgr2=0x7f492c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f492c06ecc0 0x7f492c071180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4934005fd0 tx=0x7f4934005ee0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49480ffa80 msgr2=0x7f49480fbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:54.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49480ffa80 0x7f49480fbb40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f49380092e0 tx=0x7f493800be20 comp rx=0 tx=0).stop 2024-04-20T11:39:54.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 shutdown_connections 2024-04-20T11:39:54.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f492c06ecc0 0x7f492c071180 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49480ffa80 0x7f49480fbb40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 --2- 172.21.15.117:0/217257373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4948105a80 0x7f49480fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 >> 172.21.15.117:0/217257373 conn(0x7f4948076750 msgr2=0x7f49480687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:54.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 shutdown_connections 2024-04-20T11:39:54.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.216+0000 7f494d400700 1 -- 172.21.15.117:0/217257373 wait complete. 2024-04-20T11:39:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.367+0000 7f3972800700 1 -- 172.21.15.117:0/2176149664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c107520 msgr2=0x7f396c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.367+0000 7f3972800700 1 --2- 172.21.15.117:0/2176149664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c107520 0x7f396c107900 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f395c009910 tx=0x7f395c009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.367+0000 7f3972800700 1 -- 172.21.15.117:0/2176149664 shutdown_connections 2024-04-20T11:39:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.367+0000 7f3972800700 1 --2- 172.21.15.117:0/2176149664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f396c108e10 0x7f396c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.367+0000 7f3972800700 1 --2- 172.21.15.117:0/2176149664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c107520 0x7f396c107900 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.367+0000 7f3972800700 1 -- 172.21.15.117:0/2176149664 >> 172.21.15.117:0/2176149664 conn(0x7f396c0768f0 msgr2=0x7f396c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.367+0000 7f3972800700 1 -- 172.21.15.117:0/2176149664 shutdown_connections 2024-04-20T11:39:54.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.367+0000 7f3972800700 1 -- 172.21.15.117:0/2176149664 wait complete. 2024-04-20T11:39:54.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.368+0000 7f3972800700 1 Processor -- start 2024-04-20T11:39:54.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.369+0000 7f3972800700 1 -- start start 2024-04-20T11:39:54.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.369+0000 7f3972800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f396c107520 0x7f396c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:54.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.369+0000 7f3972800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c108e10 0x7f396c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:54.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.369+0000 7f3972800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c19de90 con 0x7f396c108e10 2024-04-20T11:39:54.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.369+0000 7f3972800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c1a1b60 con 0x7f396c107520 2024-04-20T11:39:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.369+0000 7f3963e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c108e10 0x7f396c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.369+0000 7f3963e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c108e10 0x7f396c19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43726/0 (socket says 172.21.15.117:43726) 2024-04-20T11:39:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.369+0000 7f3963e00700 1 -- 172.21.15.117:0/3746322460 learned_addr learned my addr 172.21.15.117:0/3746322460 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f396b400700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f396c107520 0x7f396c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f3963e00700 1 -- 172.21.15.117:0/3746322460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f396c107520 msgr2=0x7f396c19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f3963e00700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f396c107520 0x7f396c19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f3963e00700 1 -- 172.21.15.117:0/3746322460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c0095f0 con 0x7f396c108e10 2024-04-20T11:39:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f396b400700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f396c107520 0x7f396c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:39:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f3963e00700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c108e10 0x7f396c19d7b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f395400b4b0 tx=0x7f395400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f3968c00700 1 -- 172.21.15.117:0/3746322460 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3954011800 con 0x7f396c108e10 2024-04-20T11:39:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f3968c00700 1 -- 172.21.15.117:0/3746322460 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3954011e20 con 0x7f396c108e10 2024-04-20T11:39:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c1a1e40 con 0x7f396c108e10 2024-04-20T11:39:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.370+0000 7f3968c00700 1 -- 172.21.15.117:0/3746322460 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f395400f2e0 con 0x7f396c108e10 2024-04-20T11:39:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.371+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c1a2410 con 0x7f396c108e10 2024-04-20T11:39:54.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.371+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c10f870 con 0x7f396c108e10 2024-04-20T11:39:54.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.373+0000 7f3968c00700 1 -- 172.21.15.117:0/3746322460 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3954010380 con 0x7f396c108e10 2024-04-20T11:39:54.372 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:54 smithi117 ceph-mon[99036]: pgmap v824: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:54.372 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:54 smithi117 ceph-mon[99036]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:54.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.373+0000 7f3968c00700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f395006ecd0 0x7f3950071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:54.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.374+0000 7f3968c00700 1 -- 172.21.15.117:0/3746322460 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f395408ea70 con 0x7f396c108e10 2024-04-20T11:39:54.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.374+0000 7f396b400700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f395006ecd0 0x7f3950071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:54.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.375+0000 7f396b400700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f395006ecd0 0x7f3950071190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f395c006010 tx=0x7f395c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:54.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.378+0000 7f3968c00700 1 -- 172.21.15.117:0/3746322460 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f39540577b0 con 0x7f396c108e10 2024-04-20T11:39:54.393 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:54 smithi173 ceph-mon[74165]: pgmap v824: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:54.393 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:54 smithi173 ceph-mon[74165]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:54.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.695+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f396c108920 con 0x7f396c108e10 2024-04-20T11:39:54.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.696+0000 7f3968c00700 1 -- 172.21.15.117:0/3746322460 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f395405ae70 con 0x7f396c108e10 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:39:54.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:39:54.697 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:39:54.697 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:39:54.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:39:54.697 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:39:54.697 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:39:54.697 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:39:54.697 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:39:54.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f395006ecd0 msgr2=0x7f3950071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:54.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f395006ecd0 0x7f3950071190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f395c006010 tx=0x7f395c00b540 comp rx=0 tx=0).stop 2024-04-20T11:39:54.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c108e10 msgr2=0x7f396c19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:54.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c108e10 0x7f396c19d7b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f395400b4b0 tx=0x7f395400b870 comp rx=0 tx=0).stop 2024-04-20T11:39:54.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 shutdown_connections 2024-04-20T11:39:54.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f395006ecd0 0x7f3950071190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f396c107520 0x7f396c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 --2- 172.21.15.117:0/3746322460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f396c108e10 0x7f396c19d7b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.699+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 >> 172.21.15.117:0/3746322460 conn(0x7f396c0768f0 msgr2=0x7f396c078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:54.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.700+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 shutdown_connections 2024-04-20T11:39:54.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.700+0000 7f3972800700 1 -- 172.21.15.117:0/3746322460 wait complete. 2024-04-20T11:39:54.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.850+0000 7f7b86a00700 1 -- 172.21.15.117:0/224532369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80101d40 msgr2=0x7f7b80102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:54.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.850+0000 7f7b86a00700 1 --2- 172.21.15.117:0/224532369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80101d40 0x7f7b80102120 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009910 tx=0x7f7b70009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:54.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.850+0000 7f7b86a00700 1 -- 172.21.15.117:0/224532369 shutdown_connections 2024-04-20T11:39:54.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.851+0000 7f7b86a00700 1 --2- 172.21.15.117:0/224532369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b80102660 0x7f7b8010ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.851+0000 7f7b86a00700 1 --2- 172.21.15.117:0/224532369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80101d40 0x7f7b80102120 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.851+0000 7f7b86a00700 1 -- 172.21.15.117:0/224532369 >> 172.21.15.117:0/224532369 conn(0x7f7b80076750 msgr2=0x7f7b800fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:54.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.851+0000 7f7b86a00700 1 -- 172.21.15.117:0/224532369 shutdown_connections 2024-04-20T11:39:54.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.851+0000 7f7b86a00700 1 -- 172.21.15.117:0/224532369 wait complete. 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.852+0000 7f7b86a00700 1 Processor -- start 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.852+0000 7f7b86a00700 1 -- start start 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.852+0000 7f7b86a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b80101d40 0x7f7b800ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.852+0000 7f7b86a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80102660 0x7f7b800ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.852+0000 7f7b86a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80100420 con 0x7f7b80102660 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.852+0000 7f7b86a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80100560 con 0x7f7b80101d40 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80102660 0x7f7b800ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80102660 0x7f7b800ffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43746/0 (socket says 172.21.15.117:43746) 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7f400700 1 -- 172.21.15.117:0/2718400277 learned_addr learned my addr 172.21.15.117:0/2718400277 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7fe00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b80101d40 0x7f7b800ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7f400700 1 -- 172.21.15.117:0/2718400277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b80101d40 msgr2=0x7f7b800ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:54.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7f400700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b80101d40 0x7f7b800ff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:54.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7f400700 1 -- 172.21.15.117:0/2718400277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b700095f0 con 0x7f7b80102660 2024-04-20T11:39:54.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7fe00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b80101d40 0x7f7b800ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:39:54.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.853+0000 7f7b7f400700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80102660 0x7f7b800ffee0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00c8f0 tx=0x7f7b6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:54.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.854+0000 7f7b7cc00700 1 -- 172.21.15.117:0/2718400277 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6c007ad0 con 0x7f7b80102660 2024-04-20T11:39:54.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.854+0000 7f7b7cc00700 1 -- 172.21.15.117:0/2718400277 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b6c00ce90 con 0x7f7b80102660 2024-04-20T11:39:54.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.854+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b801a2ce0 con 0x7f7b80102660 2024-04-20T11:39:54.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.854+0000 7f7b7cc00700 1 -- 172.21.15.117:0/2718400277 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6c018630 con 0x7f7b80102660 2024-04-20T11:39:54.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.854+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b801a3040 con 0x7f7b80102660 2024-04-20T11:39:54.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.855+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b80068e50 con 0x7f7b80102660 2024-04-20T11:39:54.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.856+0000 7f7b7cc00700 1 -- 172.21.15.117:0/2718400277 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7b6c007c30 con 0x7f7b80102660 2024-04-20T11:39:54.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.857+0000 7f7b7cc00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b6406ecc0 0x7f7b64071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:54.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.857+0000 7f7b7cc00700 1 -- 172.21.15.117:0/2718400277 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7b6c08edf0 con 0x7f7b80102660 2024-04-20T11:39:54.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.857+0000 7f7b7fe00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b6406ecc0 0x7f7b64071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:54.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.858+0000 7f7b7fe00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b6406ecc0 0x7f7b64071180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7b70006010 tx=0x7f7b7000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:54.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:54.862+0000 7f7b7cc00700 1 -- 172.21.15.117:0/2718400277 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7b6c057bb0 con 0x7f7b80102660 2024-04-20T11:39:55.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.113+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7b80100720 con 0x7f7b80102660 2024-04-20T11:39:55.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.114+0000 7f7b7cc00700 1 -- 172.21.15.117:0/2718400277 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f7b6c05b270 con 0x7f7b80102660 2024-04-20T11:39:55.115 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:39:55.116 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:39:55.117 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:39:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:39:55.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.117+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b6406ecc0 msgr2=0x7f7b64071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.117+0000 7f7b86a00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b6406ecc0 0x7f7b64071180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7b70006010 tx=0x7f7b7000b540 comp rx=0 tx=0).stop 2024-04-20T11:39:55.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.117+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80102660 msgr2=0x7f7b800ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.117+0000 7f7b86a00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80102660 0x7f7b800ffee0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00c8f0 tx=0x7f7b6c00ccb0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.118+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 shutdown_connections 2024-04-20T11:39:55.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.118+0000 7f7b86a00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7b6406ecc0 0x7f7b64071180 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.118+0000 7f7b86a00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b80101d40 0x7f7b800ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.118+0000 7f7b86a00700 1 --2- 172.21.15.117:0/2718400277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b80102660 0x7f7b800ffee0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.118+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 >> 172.21.15.117:0/2718400277 conn(0x7f7b80076750 msgr2=0x7f7b80103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:55.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.118+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 shutdown_connections 2024-04-20T11:39:55.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.118+0000 7f7b86a00700 1 -- 172.21.15.117:0/2718400277 wait complete. 2024-04-20T11:39:55.120 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:39:55.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.269+0000 7f5c2c400700 1 -- 172.21.15.117:0/1000659154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24102660 msgr2=0x7f5c2410ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.269+0000 7f5c2c400700 1 --2- 172.21.15.117:0/1000659154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24102660 0x7f5c2410ab50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009910 tx=0x7f5c10009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:55.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.269+0000 7f5c2c400700 1 -- 172.21.15.117:0/1000659154 shutdown_connections 2024-04-20T11:39:55.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.269+0000 7f5c2c400700 1 --2- 172.21.15.117:0/1000659154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24102660 0x7f5c2410ab50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.269+0000 7f5c2c400700 1 --2- 172.21.15.117:0/1000659154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c24101d40 0x7f5c24102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.269+0000 7f5c2c400700 1 -- 172.21.15.117:0/1000659154 >> 172.21.15.117:0/1000659154 conn(0x7f5c24076750 msgr2=0x7f5c240fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:55.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.269+0000 7f5c2c400700 1 -- 172.21.15.117:0/1000659154 shutdown_connections 2024-04-20T11:39:55.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.269+0000 7f5c2c400700 1 -- 172.21.15.117:0/1000659154 wait complete. 2024-04-20T11:39:55.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.270+0000 7f5c2c400700 1 Processor -- start 2024-04-20T11:39:55.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.270+0000 7f5c2c400700 1 -- start start 2024-04-20T11:39:55.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.271+0000 7f5c2c400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24101d40 0x7f5c240ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:55.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.271+0000 7f5c2c400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c24102660 0x7f5c240ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:55.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.271+0000 7f5c2c400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24100420 con 0x7f5c24101d40 2024-04-20T11:39:55.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.271+0000 7f5c2c400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24100560 con 0x7f5c24102660 2024-04-20T11:39:55.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.271+0000 7f5c29a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24101d40 0x7f5c240ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.271+0000 7f5c29a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24101d40 0x7f5c240ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43760/0 (socket says 172.21.15.117:43760) 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.271+0000 7f5c29a00700 1 -- 172.21.15.117:0/3527693259 learned_addr learned my addr 172.21.15.117:0/3527693259 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.271+0000 7f5c29000700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c24102660 0x7f5c240ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c29a00700 1 -- 172.21.15.117:0/3527693259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c24102660 msgr2=0x7f5c240ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c29a00700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c24102660 0x7f5c240ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c29a00700 1 -- 172.21.15.117:0/3527693259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100095f0 con 0x7f5c24101d40 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c29a00700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24101d40 0x7f5c240ff9a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800d8d0 tx=0x7f5c1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c22000700 1 -- 172.21.15.117:0/3527693259 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c18009940 con 0x7f5c24101d40 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c22000700 1 -- 172.21.15.117:0/3527693259 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c18010460 con 0x7f5c24101d40 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c241a2d00 con 0x7f5c24101d40 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c22000700 1 -- 172.21.15.117:0/3527693259 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c1800f5f0 con 0x7f5c24101d40 2024-04-20T11:39:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.272+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c241a3060 con 0x7f5c24101d40 2024-04-20T11:39:55.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.273+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c24068e50 con 0x7f5c24101d40 2024-04-20T11:39:55.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.274+0000 7f5c22000700 1 -- 172.21.15.117:0/3527693259 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5c180105d0 con 0x7f5c24101d40 2024-04-20T11:39:55.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.275+0000 7f5c22000700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c0c06ecc0 0x7f5c0c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:55.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.275+0000 7f5c22000700 1 -- 172.21.15.117:0/3527693259 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5c1808ed10 con 0x7f5c24101d40 2024-04-20T11:39:55.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.275+0000 7f5c29000700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c0c06ecc0 0x7f5c0c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:55.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.276+0000 7f5c29000700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c0c06ecc0 0x7f5c0c071180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5c241010d0 tx=0x7f5c1001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:55.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.280+0000 7f5c22000700 1 -- 172.21.15.117:0/3527693259 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5c18057b70 con 0x7f5c24101d40 2024-04-20T11:39:55.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:55 smithi117 ceph-mon[99036]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:55.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:55 smithi117 ceph-mon[99036]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:55.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:55 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3746322460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:39:55.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:55 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2718400277' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:39:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:55 smithi173 ceph-mon[74165]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:55 smithi173 ceph-mon[74165]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:55 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3746322460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:39:55.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:55 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2718400277' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:39:55.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.515+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c2410dda0 con 0x7f5c0c06ecc0 2024-04-20T11:39:55.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.517+0000 7f5c22000700 1 -- 172.21.15.117:0/3527693259 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5c2410dda0 con 0x7f5c0c06ecc0 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:39:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:39:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:39:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:39:55.519 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:39:55.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.519+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c0c06ecc0 msgr2=0x7f5c0c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.519+0000 7f5c2c400700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c0c06ecc0 0x7f5c0c071180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5c241010d0 tx=0x7f5c1001a040 comp rx=0 tx=0).stop 2024-04-20T11:39:55.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.519+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24101d40 msgr2=0x7f5c240ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.519+0000 7f5c2c400700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24101d40 0x7f5c240ff9a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800d8d0 tx=0x7f5c1800dc90 comp rx=0 tx=0).stop 2024-04-20T11:39:55.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.520+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 shutdown_connections 2024-04-20T11:39:55.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.520+0000 7f5c2c400700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c0c06ecc0 0x7f5c0c071180 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.520+0000 7f5c2c400700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c24101d40 0x7f5c240ff9a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.520+0000 7f5c2c400700 1 --2- 172.21.15.117:0/3527693259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c24102660 0x7f5c240ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.520+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 >> 172.21.15.117:0/3527693259 conn(0x7f5c24076750 msgr2=0x7f5c24103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:55.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.520+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 shutdown_connections 2024-04-20T11:39:55.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.520+0000 7f5c2c400700 1 -- 172.21.15.117:0/3527693259 wait complete. 2024-04-20T11:39:55.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.663+0000 7fec56600700 1 -- 172.21.15.117:0/3712733936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5010d0c0 msgr2=0x7fec5010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.663+0000 7fec56600700 1 --2- 172.21.15.117:0/3712733936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5010d0c0 0x7fec5010d4a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fec44009910 tx=0x7fec44009c20 comp rx=0 tx=0).stop 2024-04-20T11:39:55.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.664+0000 7fec56600700 1 -- 172.21.15.117:0/3712733936 shutdown_connections 2024-04-20T11:39:55.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.664+0000 7fec56600700 1 --2- 172.21.15.117:0/3712733936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec501070b0 0x7fec50107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.664+0000 7fec56600700 1 --2- 172.21.15.117:0/3712733936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5010d0c0 0x7fec5010d4a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.664+0000 7fec56600700 1 -- 172.21.15.117:0/3712733936 >> 172.21.15.117:0/3712733936 conn(0x7fec50076830 msgr2=0x7fec50078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:55.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.664+0000 7fec56600700 1 -- 172.21.15.117:0/3712733936 shutdown_connections 2024-04-20T11:39:55.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.664+0000 7fec56600700 1 -- 172.21.15.117:0/3712733936 wait complete. 2024-04-20T11:39:55.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.665+0000 7fec56600700 1 Processor -- start 2024-04-20T11:39:55.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.665+0000 7fec56600700 1 -- start start 2024-04-20T11:39:55.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.665+0000 7fec56600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec501070b0 0x7fec5019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:55.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.665+0000 7fec56600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec5010d0c0 0x7fec5019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:55.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.665+0000 7fec56600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5019dac0 con 0x7fec501070b0 2024-04-20T11:39:55.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.665+0000 7fec56600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5019dc00 con 0x7fec5010d0c0 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.665+0000 7fec55200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec501070b0 0x7fec5019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec55200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec501070b0 0x7fec5019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43776/0 (socket says 172.21.15.117:43776) 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec55200700 1 -- 172.21.15.117:0/1913131553 learned_addr learned my addr 172.21.15.117:0/1913131553 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec55200700 1 -- 172.21.15.117:0/1913131553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec5010d0c0 msgr2=0x7fec5019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec4fe00700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec5010d0c0 0x7fec5019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec55200700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec5010d0c0 0x7fec5019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec55200700 1 -- 172.21.15.117:0/1913131553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440095f0 con 0x7fec501070b0 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec4fe00700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec5010d0c0 0x7fec5019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec55200700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec501070b0 0x7fec5019cf30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fec440092e0 tx=0x7fec4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.666+0000 7fec4d600700 1 -- 172.21.15.117:0/1913131553 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec4401d070 con 0x7fec501070b0 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.667+0000 7fec4d600700 1 -- 172.21.15.117:0/1913131553 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec4400f460 con 0x7fec501070b0 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.667+0000 7fec4d600700 1 -- 172.21.15.117:0/1913131553 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec44017630 con 0x7fec501070b0 2024-04-20T11:39:55.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.667+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec501a1930 con 0x7fec501070b0 2024-04-20T11:39:55.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.667+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec501a1e20 con 0x7fec501070b0 2024-04-20T11:39:55.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.668+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5010f5e0 con 0x7fec501070b0 2024-04-20T11:39:55.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.669+0000 7fec4d600700 1 -- 172.21.15.117:0/1913131553 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fec4400f5d0 con 0x7fec501070b0 2024-04-20T11:39:55.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.669+0000 7fec4d600700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec3806ecd0 0x7fec38071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:39:55.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.670+0000 7fec4d600700 1 -- 172.21.15.117:0/1913131553 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fec44090340 con 0x7fec501070b0 2024-04-20T11:39:55.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.670+0000 7fec4fe00700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec3806ecd0 0x7fec38071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:39:55.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.670+0000 7fec4fe00700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec3806ecd0 0x7fec38071190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005fd0 tx=0x7fec3c005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:39:55.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.674+0000 7fec4d600700 1 -- 172.21.15.117:0/1913131553 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fec44059100 con 0x7fec501070b0 2024-04-20T11:39:55.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.968+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fec5019e240 con 0x7fec501070b0 2024-04-20T11:39:55.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.969+0000 7fec4d600700 1 -- 172.21.15.117:0/1913131553 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fec4405c7c0 con 0x7fec501070b0 2024-04-20T11:39:55.969 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:39:55.970 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:39:55.971 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:39:55.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec3806ecd0 msgr2=0x7fec38071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec3806ecd0 0x7fec38071190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005fd0 tx=0x7fec3c005e20 comp rx=0 tx=0).stop 2024-04-20T11:39:55.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec501070b0 msgr2=0x7fec5019cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:39:55.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec501070b0 0x7fec5019cf30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fec440092e0 tx=0x7fec4400be20 comp rx=0 tx=0).stop 2024-04-20T11:39:55.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 shutdown_connections 2024-04-20T11:39:55.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fec3806ecd0 0x7fec38071190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec501070b0 0x7fec5019cf30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 --2- 172.21.15.117:0/1913131553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec5010d0c0 0x7fec5019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:39:55.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.972+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 >> 172.21.15.117:0/1913131553 conn(0x7fec50076830 msgr2=0x7fec5010c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:39:55.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.973+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 shutdown_connections 2024-04-20T11:39:55.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:39:55.973+0000 7fec56600700 1 -- 172.21.15.117:0/1913131553 wait complete. 2024-04-20T11:39:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:56 smithi117 ceph-mon[99036]: pgmap v825: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:56.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:56 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1913131553' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:39:56.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:56 smithi173 ceph-mon[74165]: pgmap v825: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:56.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:56 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1913131553' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:39:57.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:57 smithi173 ceph-mon[74165]: from='client.35506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:57 smithi117 ceph-mon[99036]: from='client.35506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:39:58.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:58 smithi117 ceph-mon[99036]: pgmap v826: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:58.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:39:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:39:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:58 smithi173 ceph-mon[74165]: pgmap v826: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:39:58.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:39:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-04-20T11:40:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:40:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:40:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:40:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:40:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:40:00.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:39:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: pgmap v827: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:40:00.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: /usr/bin/podman: timeout after 895 seconds 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: Traceback (most recent call last): 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: return _run_code(code, main_globals, None, 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: exec(code, run_globals) 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:40:00.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: fs cephfs is degraded 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:40:00.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:00 smithi117 ceph-mon[99036]: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:40:00.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: pgmap v827: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:00.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:40:00.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:40:00.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:40:00.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: /usr/bin/podman: timeout after 895 seconds 2024-04-20T11:40:00.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:40:00.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: Traceback (most recent call last): 2024-04-20T11:40:00.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: return _run_code(code, main_globals, None, 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: exec(code, run_globals) 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:40:00.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: fs cephfs is degraded 2024-04-20T11:40:00.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:40:00.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:40:00.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:40:00.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:00 smithi173 ceph-mon[74165]: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:40:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:02 smithi117 ceph-mon[99036]: pgmap v828: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:02 smithi173 ceph-mon[74165]: pgmap v828: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:04 smithi117 ceph-mon[99036]: pgmap v829: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:04 smithi173 ceph-mon[74165]: pgmap v829: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:06.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:06 smithi117 ceph-mon[99036]: pgmap v830: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:06 smithi173 ceph-mon[74165]: pgmap v830: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:08.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:08 smithi117 ceph-mon[99036]: pgmap v831: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:08 smithi173 ceph-mon[74165]: pgmap v831: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:10.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:10 smithi117 ceph-mon[99036]: pgmap v832: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:10 smithi173 ceph-mon[74165]: pgmap v832: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:12.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:12 smithi117 ceph-mon[99036]: pgmap v833: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:12 smithi173 ceph-mon[74165]: pgmap v833: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:13.594 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:14 smithi117 ceph-mon[99036]: pgmap v834: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:14 smithi173 ceph-mon[74165]: pgmap v834: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:16 smithi117 ceph-mon[99036]: pgmap v835: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:16 smithi173 ceph-mon[74165]: pgmap v835: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:18 smithi117 ceph-mon[99036]: pgmap v836: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:18 smithi173 ceph-mon[74165]: pgmap v836: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:20.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:20 smithi117 ceph-mon[99036]: pgmap v837: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:20 smithi173 ceph-mon[74165]: pgmap v837: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:22.384 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:22 smithi173 ceph-mon[74165]: pgmap v838: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:22.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:22 smithi117 ceph-mon[99036]: pgmap v838: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:24 smithi117 ceph-mon[99036]: pgmap v839: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:24 smithi173 ceph-mon[74165]: pgmap v839: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:26.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.142+0000 7f0069800700 1 -- 172.21.15.117:0/800667969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 msgr2=0x7f0064104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.142+0000 7f0069800700 1 --2- 172.21.15.117:0/800667969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 0x7f0064104e80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0050009910 tx=0x7f0050009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:26.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.142+0000 7f0069800700 1 -- 172.21.15.117:0/800667969 shutdown_connections 2024-04-20T11:40:26.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.142+0000 7f0069800700 1 --2- 172.21.15.117:0/800667969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 0x7f0064104e80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.142+0000 7f0069800700 1 --2- 172.21.15.117:0/800667969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0064103110 0x7f00641034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.142+0000 7f0069800700 1 -- 172.21.15.117:0/800667969 >> 172.21.15.117:0/800667969 conn(0x7f00640fe880 msgr2=0x7f0064100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:26.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.142+0000 7f0069800700 1 -- 172.21.15.117:0/800667969 shutdown_connections 2024-04-20T11:40:26.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.142+0000 7f0069800700 1 -- 172.21.15.117:0/800667969 wait complete. 2024-04-20T11:40:26.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.143+0000 7f0069800700 1 Processor -- start 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.144+0000 7f0069800700 1 -- start start 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.144+0000 7f0069800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0064103110 0x7f006419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.144+0000 7f0069800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 0x7f006419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.144+0000 7f0069800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006419f5b0 con 0x7f0064104a00 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.144+0000 7f0069800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0064198b90 con 0x7f0064103110 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.144+0000 7f0062000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 0x7f006419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.144+0000 7f0062000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 0x7f006419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36776/0 (socket says 172.21.15.117:36776) 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.144+0000 7f0062000700 1 -- 172.21.15.117:0/378552474 learned_addr learned my addr 172.21.15.117:0/378552474 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.145+0000 7f0062a00700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0064103110 0x7f006419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.145+0000 7f0062000700 1 -- 172.21.15.117:0/378552474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0064103110 msgr2=0x7f006419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.145+0000 7f0062000700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0064103110 0x7f006419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.145+0000 7f0062000700 1 -- 172.21.15.117:0/378552474 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00500095f0 con 0x7f0064104a00 2024-04-20T11:40:26.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.145+0000 7f0062a00700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0064103110 0x7f006419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:40:26.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.145+0000 7f0062000700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 0x7f006419ef90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0050006010 tx=0x7f005000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:26.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.145+0000 7f005b400700 1 -- 172.21.15.117:0/378552474 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f005001d070 con 0x7f0064104a00 2024-04-20T11:40:26.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.146+0000 7f005b400700 1 -- 172.21.15.117:0/378552474 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f005000f460 con 0x7f0064104a00 2024-04-20T11:40:26.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.146+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0064198e10 con 0x7f0064104a00 2024-04-20T11:40:26.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.146+0000 7f005b400700 1 -- 172.21.15.117:0/378552474 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0050017630 con 0x7f0064104a00 2024-04-20T11:40:26.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.146+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0064199380 con 0x7f0064104a00 2024-04-20T11:40:26.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.147+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006410b480 con 0x7f0064104a00 2024-04-20T11:40:26.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.148+0000 7f005b400700 1 -- 172.21.15.117:0/378552474 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f005000f5d0 con 0x7f0064104a00 2024-04-20T11:40:26.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.149+0000 7f005b400700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f004806ecc0 0x7f0048071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.149+0000 7f005b400700 1 -- 172.21.15.117:0/378552474 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0050090360 con 0x7f0064104a00 2024-04-20T11:40:26.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.149+0000 7f0062a00700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f004806ecc0 0x7f0048071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.150+0000 7f0062a00700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f004806ecc0 0x7f0048071180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0054005fd0 tx=0x7f0054005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:26.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.154+0000 7f005b400700 1 -- 172.21.15.117:0/378552474 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f005005cd10 con 0x7f0064104a00 2024-04-20T11:40:26.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.398+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0064199fe0 con 0x7f004806ecc0 2024-04-20T11:40:26.399 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:26 smithi117 ceph-mon[99036]: pgmap v840: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:26.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.400+0000 7f005b400700 1 -- 172.21.15.117:0/378552474 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0064199fe0 con 0x7f004806ecc0 2024-04-20T11:40:26.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f004806ecc0 msgr2=0x7f0048071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f004806ecc0 0x7f0048071180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0054005fd0 tx=0x7f0054005dc0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 msgr2=0x7f006419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 0x7f006419ef90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0050006010 tx=0x7f005000be20 comp rx=0 tx=0).stop 2024-04-20T11:40:26.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 shutdown_connections 2024-04-20T11:40:26.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f004806ecc0 0x7f0048071180 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0064103110 0x7f006419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 --2- 172.21.15.117:0/378552474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0064104a00 0x7f006419ef90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.403+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 >> 172.21.15.117:0/378552474 conn(0x7f00640fe880 msgr2=0x7f00640ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:26.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.404+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 shutdown_connections 2024-04-20T11:40:26.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.404+0000 7f0069800700 1 -- 172.21.15.117:0/378552474 wait complete. 2024-04-20T11:40:26.419 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:40:26.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.556+0000 7fd86a200700 1 -- 172.21.15.117:0/1474651108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 msgr2=0x7fd8641011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.556+0000 7fd86a200700 1 --2- 172.21.15.117:0/1474651108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 0x7fd8641011f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd854009910 tx=0x7fd854009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:26.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.556+0000 7fd86a200700 1 -- 172.21.15.117:0/1474651108 shutdown_connections 2024-04-20T11:40:26.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.556+0000 7fd86a200700 1 --2- 172.21.15.117:0/1474651108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8641017c0 0x7fd864105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.556+0000 7fd86a200700 1 --2- 172.21.15.117:0/1474651108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 0x7fd8641011f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.556+0000 7fd86a200700 1 -- 172.21.15.117:0/1474651108 >> 172.21.15.117:0/1474651108 conn(0x7fd8640fc6a0 msgr2=0x7fd8640feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:26.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.557+0000 7fd86a200700 1 -- 172.21.15.117:0/1474651108 shutdown_connections 2024-04-20T11:40:26.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.557+0000 7fd86a200700 1 -- 172.21.15.117:0/1474651108 wait complete. 2024-04-20T11:40:26.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.558+0000 7fd86a200700 1 Processor -- start 2024-04-20T11:40:26.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.558+0000 7fd86a200700 1 -- start start 2024-04-20T11:40:26.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.558+0000 7fd86a200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 0x7fd8640731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.558+0000 7fd86a200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8641017c0 0x7fd864073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.558+0000 7fd86a200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd864073c60 con 0x7fd864100e10 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.558+0000 7fd86a200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd864073dd0 con 0x7fd8641017c0 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd863400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 0x7fd8640731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd863400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 0x7fd8640731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36808/0 (socket says 172.21.15.117:36808) 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd863400700 1 -- 172.21.15.117:0/2211865870 learned_addr learned my addr 172.21.15.117:0/2211865870 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd863400700 1 -- 172.21.15.117:0/2211865870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8641017c0 msgr2=0x7fd864073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd85be00700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8641017c0 0x7fd864073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd863400700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8641017c0 0x7fd864073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd863400700 1 -- 172.21.15.117:0/2211865870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540095f0 con 0x7fd864100e10 2024-04-20T11:40:26.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd863400700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 0x7fd8640731e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd85400bd70 tx=0x7fd85400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:26.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.559+0000 7fd860c00700 1 -- 172.21.15.117:0/2211865870 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd85401d070 con 0x7fd864100e10 2024-04-20T11:40:26.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.560+0000 7fd860c00700 1 -- 172.21.15.117:0/2211865870 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8540056e0 con 0x7fd864100e10 2024-04-20T11:40:26.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.560+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd864074050 con 0x7fd864100e10 2024-04-20T11:40:26.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.560+0000 7fd860c00700 1 -- 172.21.15.117:0/2211865870 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd854017400 con 0x7fd864100e10 2024-04-20T11:40:26.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.560+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8641a2f40 con 0x7fd864100e10 2024-04-20T11:40:26.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.561+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8641091b0 con 0x7fd864100e10 2024-04-20T11:40:26.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.562+0000 7fd860c00700 1 -- 172.21.15.117:0/2211865870 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd85400f6f0 con 0x7fd864100e10 2024-04-20T11:40:26.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.563+0000 7fd860c00700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd84806ecd0 0x7fd848071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.563+0000 7fd860c00700 1 -- 172.21.15.117:0/2211865870 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd8540901f0 con 0x7fd864100e10 2024-04-20T11:40:26.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.563+0000 7fd85be00700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd84806ecd0 0x7fd848071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.564+0000 7fd85be00700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd84806ecd0 0x7fd848071190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd84c005950 tx=0x7fd84c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:26.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.567+0000 7fd860c00700 1 -- 172.21.15.117:0/2211865870 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd854058f30 con 0x7fd864100e10 2024-04-20T11:40:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:26 smithi173 ceph-mon[74165]: pgmap v840: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:26.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.807+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd864074050 con 0x7fd84806ecd0 2024-04-20T11:40:26.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.809+0000 7fd860c00700 1 -- 172.21.15.117:0/2211865870 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd864074050 con 0x7fd84806ecd0 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd84806ecd0 msgr2=0x7fd848071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd84806ecd0 0x7fd848071190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd84c005950 tx=0x7fd84c0058e0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 msgr2=0x7fd8640731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 0x7fd8640731e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd85400bd70 tx=0x7fd85400be50 comp rx=0 tx=0).stop 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 shutdown_connections 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd84806ecd0 0x7fd848071190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd864100e10 0x7fd8640731e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 --2- 172.21.15.117:0/2211865870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8641017c0 0x7fd864073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 >> 172.21.15.117:0/2211865870 conn(0x7fd8640fc6a0 msgr2=0x7fd8640fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:26.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.812+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 shutdown_connections 2024-04-20T11:40:26.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.813+0000 7fd86a200700 1 -- 172.21.15.117:0/2211865870 wait complete. 2024-04-20T11:40:26.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.957+0000 7f1617000700 1 -- 172.21.15.117:0/3858085867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 msgr2=0x7f161010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.957+0000 7f1617000700 1 --2- 172.21.15.117:0/3858085867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 0x7f161010cf40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009910 tx=0x7f15f8009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:26.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.957+0000 7f1617000700 1 -- 172.21.15.117:0/3858085867 shutdown_connections 2024-04-20T11:40:26.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.957+0000 7f1617000700 1 --2- 172.21.15.117:0/3858085867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 0x7f161010cf40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.957+0000 7f1617000700 1 --2- 172.21.15.117:0/3858085867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16100ff8d0 0x7f16100ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.957+0000 7f1617000700 1 -- 172.21.15.117:0/3858085867 >> 172.21.15.117:0/3858085867 conn(0x7f1610076750 msgr2=0x7f16100fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:26.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.957+0000 7f1617000700 1 -- 172.21.15.117:0/3858085867 shutdown_connections 2024-04-20T11:40:26.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.957+0000 7f1617000700 1 -- 172.21.15.117:0/3858085867 wait complete. 2024-04-20T11:40:26.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.958+0000 7f1617000700 1 Processor -- start 2024-04-20T11:40:26.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.958+0000 7f1617000700 1 -- start start 2024-04-20T11:40:26.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.959+0000 7f1617000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16100ff8d0 0x7f1610198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.959+0000 7f1617000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 0x7f1610199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.959+0000 7f1617000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16101999f0 con 0x7f1610100280 2024-04-20T11:40:26.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.959+0000 7f1617000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161019d6c0 con 0x7f16100ff8d0 2024-04-20T11:40:26.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.959+0000 7f1607400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 0x7f1610199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.959+0000 7f1607400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 0x7f1610199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36818/0 (socket says 172.21.15.117:36818) 2024-04-20T11:40:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.959+0000 7f1607400700 1 -- 172.21.15.117:0/3271423434 learned_addr learned my addr 172.21.15.117:0/3271423434 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.959+0000 7f160fe00700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16100ff8d0 0x7f1610198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f1607400700 1 -- 172.21.15.117:0/3271423434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16100ff8d0 msgr2=0x7f1610198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f1607400700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16100ff8d0 0x7f1610198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f1607400700 1 -- 172.21.15.117:0/3271423434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f80095f0 con 0x7f1610100280 2024-04-20T11:40:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f1607400700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 0x7f1610199310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b850 tx=0x7f15f80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f160d600700 1 -- 172.21.15.117:0/3271423434 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f801d070 con 0x7f1610100280 2024-04-20T11:40:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f160d600700 1 -- 172.21.15.117:0/3271423434 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15f8003d50 con 0x7f1610100280 2024-04-20T11:40:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f160d600700 1 -- 172.21.15.117:0/3271423434 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f8021410 con 0x7f1610100280 2024-04-20T11:40:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161019d940 con 0x7f1610100280 2024-04-20T11:40:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.960+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161019de90 con 0x7f1610100280 2024-04-20T11:40:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.961+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1610068e50 con 0x7f1610100280 2024-04-20T11:40:26.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.963+0000 7f160d600700 1 -- 172.21.15.117:0/3271423434 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f15f8003890 con 0x7f1610100280 2024-04-20T11:40:26.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.964+0000 7f160d600700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15f406ecc0 0x7f15f4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:26.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.964+0000 7f160d600700 1 -- 172.21.15.117:0/3271423434 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f15f8090170 con 0x7f1610100280 2024-04-20T11:40:26.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.964+0000 7f160fe00700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15f406ecc0 0x7f15f4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:26.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.964+0000 7f160fe00700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15f406ecc0 0x7f15f4071180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1600007dc0 tx=0x7f16000072f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:26.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:26.968+0000 7f160d600700 1 -- 172.21.15.117:0/3271423434 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f15f8058f50 con 0x7f1610100280 2024-04-20T11:40:27.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.201+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f161010ffc0 con 0x7f15f406ecc0 2024-04-20T11:40:27.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.232+0000 7f160d600700 1 -- 172.21.15.117:0/3271423434 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f161010ffc0 con 0x7f15f406ecc0 2024-04-20T11:40:27.232 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:40:27.232 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (28m) 4m ago 42m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:40:27.232 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (43m) 4m ago 43m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (41m) 4m ago 41m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (25m) 4m ago 43m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (25m) 4m ago 41m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (28m) 4m ago 42m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (20m) 4m ago 35m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (21m) 4m ago 35m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (5m) 4m ago 35m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (20m) 4m ago 35m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (32m) 4m ago 44m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (30m) 4m ago 41m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (26m) 4m ago 44m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:40:27.233 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (26m) 4m ago 41m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (30m) 4m ago 43m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (30m) 4m ago 41m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (24m) 4m ago 40m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (24m) 4m ago 39m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (23m) 4m ago 39m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (22m) 4m ago 38m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (22m) 4m ago 38m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (21m) 4m ago 37m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:40:27.234 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (29m) 4m ago 42m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:40:27.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15f406ecc0 msgr2=0x7f15f4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:27.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15f406ecc0 0x7f15f4071180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1600007dc0 tx=0x7f16000072f0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 msgr2=0x7f1610199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:27.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 0x7f1610199310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b850 tx=0x7f15f80056a0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 shutdown_connections 2024-04-20T11:40:27.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f15f406ecc0 0x7f15f4071180 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16100ff8d0 0x7f1610198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 --2- 172.21.15.117:0/3271423434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1610100280 0x7f1610199310 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 >> 172.21.15.117:0/3271423434 conn(0x7f1610076750 msgr2=0x7f16100fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:27.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 shutdown_connections 2024-04-20T11:40:27.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.235+0000 7f1617000700 1 -- 172.21.15.117:0/3271423434 wait complete. 2024-04-20T11:40:27.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.383+0000 7f93ed200700 1 -- 172.21.15.117:0/1521880816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 msgr2=0x7f93e8111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:27.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.383+0000 7f93ed200700 1 --2- 172.21.15.117:0/1521880816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 0x7f93e8111540 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009910 tx=0x7f93d0009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:27.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.383+0000 7f93ed200700 1 -- 172.21.15.117:0/1521880816 shutdown_connections 2024-04-20T11:40:27.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.383+0000 7f93ed200700 1 --2- 172.21.15.117:0/1521880816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 0x7f93e8111540 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.383+0000 7f93ed200700 1 --2- 172.21.15.117:0/1521880816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8075410 0x7f93e80757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.383+0000 7f93ed200700 1 -- 172.21.15.117:0/1521880816 >> 172.21.15.117:0/1521880816 conn(0x7f93e80fe8d0 msgr2=0x7f93e8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:27.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.383+0000 7f93ed200700 1 -- 172.21.15.117:0/1521880816 shutdown_connections 2024-04-20T11:40:27.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.383+0000 7f93ed200700 1 -- 172.21.15.117:0/1521880816 wait complete. 2024-04-20T11:40:27.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.384+0000 7f93ed200700 1 Processor -- start 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.384+0000 7f93ed200700 1 -- start start 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93ed200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8075410 0x7f93e819d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93ed200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 0x7f93e819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93ed200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e819dd10 con 0x7f93e8075d30 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93ed200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81a19e0 con 0x7f93e8075410 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93e5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 0x7f93e819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93e5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 0x7f93e819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36850/0 (socket says 172.21.15.117:36850) 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93e5400700 1 -- 172.21.15.117:0/4256399028 learned_addr learned my addr 172.21.15.117:0/4256399028 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93e5e00700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8075410 0x7f93e819d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93e5400700 1 -- 172.21.15.117:0/4256399028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8075410 msgr2=0x7f93e819d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93e5400700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8075410 0x7f93e819d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.385+0000 7f93e5400700 1 -- 172.21.15.117:0/4256399028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d00095f0 con 0x7f93e8075d30 2024-04-20T11:40:27.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.386+0000 7f93e5400700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 0x7f93e819d630 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f93d000b850 tx=0x7f93d00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:27.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.386+0000 7f93dea00700 1 -- 172.21.15.117:0/4256399028 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d001d070 con 0x7f93e8075d30 2024-04-20T11:40:27.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.386+0000 7f93dea00700 1 -- 172.21.15.117:0/4256399028 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93d0003d50 con 0x7f93e8075d30 2024-04-20T11:40:27.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.386+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e81a1cc0 con 0x7f93e8075d30 2024-04-20T11:40:27.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.386+0000 7f93dea00700 1 -- 172.21.15.117:0/4256399028 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d0021410 con 0x7f93e8075d30 2024-04-20T11:40:27.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.386+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e81a2210 con 0x7f93e8075d30 2024-04-20T11:40:27.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.387+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e8076b50 con 0x7f93e8075d30 2024-04-20T11:40:27.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.388+0000 7f93dea00700 1 -- 172.21.15.117:0/4256399028 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f93d0003890 con 0x7f93e8075d30 2024-04-20T11:40:27.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.389+0000 7f93dea00700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ed20 0x7f93cc0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:27.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.389+0000 7f93dea00700 1 -- 172.21.15.117:0/4256399028 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f93d00900c0 con 0x7f93e8075d30 2024-04-20T11:40:27.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.389+0000 7f93e5e00700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ed20 0x7f93cc0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:27.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.390+0000 7f93e5e00700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ed20 0x7f93cc0711e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f93d8005950 tx=0x7f93d8005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:27.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.394+0000 7f93dea00700 1 -- 172.21.15.117:0/4256399028 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f93d0058f00 con 0x7f93e8075d30 2024-04-20T11:40:27.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:27 smithi117 ceph-mon[99036]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:40:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:27 smithi173 ceph-mon[74165]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:40:27.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.713+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f93e804ebb0 con 0x7f93e8075d30 2024-04-20T11:40:27.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.715+0000 7f93dea00700 1 -- 172.21.15.117:0/4256399028 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f93d005c5c0 con 0x7f93e8075d30 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:40:27.715 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:40:27.716 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:40:27.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:40:27.716 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:40:27.716 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:40:27.716 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:40:27.716 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ed20 msgr2=0x7f93cc0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ed20 0x7f93cc0711e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f93d8005950 tx=0x7f93d8005cf0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 msgr2=0x7f93e819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 0x7f93e819d630 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f93d000b850 tx=0x7f93d00056a0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 shutdown_connections 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ed20 0x7f93cc0711e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8075410 0x7f93e819d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 --2- 172.21.15.117:0/4256399028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8075d30 0x7f93e819d630 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 >> 172.21.15.117:0/4256399028 conn(0x7f93e80fe8d0 msgr2=0x7f93e80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 shutdown_connections 2024-04-20T11:40:27.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.718+0000 7f93ed200700 1 -- 172.21.15.117:0/4256399028 wait complete. 2024-04-20T11:40:27.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.871+0000 7f7881600700 1 -- 172.21.15.117:0/2644894172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c108e10 msgr2=0x7f787c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:27.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.871+0000 7f7881600700 1 --2- 172.21.15.117:0/2644894172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c108e10 0x7f787c109290 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7868009910 tx=0x7f7868009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:27.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.871+0000 7f7881600700 1 -- 172.21.15.117:0/2644894172 shutdown_connections 2024-04-20T11:40:27.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.871+0000 7f7881600700 1 --2- 172.21.15.117:0/2644894172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c108e10 0x7f787c109290 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.871+0000 7f7881600700 1 --2- 172.21.15.117:0/2644894172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f787c107520 0x7f787c107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.871+0000 7f7881600700 1 -- 172.21.15.117:0/2644894172 >> 172.21.15.117:0/2644894172 conn(0x7f787c0768f0 msgr2=0x7f787c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:27.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.871+0000 7f7881600700 1 -- 172.21.15.117:0/2644894172 shutdown_connections 2024-04-20T11:40:27.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.871+0000 7f7881600700 1 -- 172.21.15.117:0/2644894172 wait complete. 2024-04-20T11:40:27.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.872+0000 7f7881600700 1 Processor -- start 2024-04-20T11:40:27.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.873+0000 7f7881600700 1 -- start start 2024-04-20T11:40:27.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.873+0000 7f7881600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c107520 0x7f787c19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:27.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.873+0000 7f7881600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f787c108e10 0x7f787c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:27.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.873+0000 7f7881600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c19ded0 con 0x7f787c107520 2024-04-20T11:40:27.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.873+0000 7f7881600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1a1ba0 con 0x7f787c108e10 2024-04-20T11:40:27.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.873+0000 7f787aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c107520 0x7f787c19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:27.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.873+0000 7f787aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c107520 0x7f787c19d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36868/0 (socket says 172.21.15.117:36868) 2024-04-20T11:40:27.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.873+0000 7f787aa00700 1 -- 172.21.15.117:0/4259328392 learned_addr learned my addr 172.21.15.117:0/4259328392 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:27.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f787aa00700 1 -- 172.21.15.117:0/4259328392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f787c108e10 msgr2=0x7f787c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:27.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f787aa00700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f787c108e10 0x7f787c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:27.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f787aa00700 1 -- 172.21.15.117:0/4259328392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680095f0 con 0x7f787c107520 2024-04-20T11:40:27.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f787aa00700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c107520 0x7f787c19d2b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f786c00b4b0 tx=0x7f786c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:27.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f7873400700 1 -- 172.21.15.117:0/4259328392 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786c011800 con 0x7f787c107520 2024-04-20T11:40:27.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f7873400700 1 -- 172.21.15.117:0/4259328392 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f786c011e20 con 0x7f787c107520 2024-04-20T11:40:27.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c1a1e80 con 0x7f787c107520 2024-04-20T11:40:27.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f7873400700 1 -- 172.21.15.117:0/4259328392 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786c00f2e0 con 0x7f787c107520 2024-04-20T11:40:27.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.874+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c1a2450 con 0x7f787c107520 2024-04-20T11:40:27.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.876+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c10f870 con 0x7f787c107520 2024-04-20T11:40:27.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.876+0000 7f7873400700 1 -- 172.21.15.117:0/4259328392 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f786c010380 con 0x7f787c107520 2024-04-20T11:40:27.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.877+0000 7f7873400700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f786006ecd0 0x7f7860071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:27.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.878+0000 7f7873400700 1 -- 172.21.15.117:0/4259328392 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f786c08ea00 con 0x7f787c107520 2024-04-20T11:40:27.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.878+0000 7f787a000700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f786006ecd0 0x7f7860071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:27.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.878+0000 7f787a000700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f786006ecd0 0x7f7860071190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7868006010 tx=0x7f786800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:27.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:27.882+0000 7f7873400700 1 -- 172.21.15.117:0/4259328392 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f786c057870 con 0x7f787c107520 2024-04-20T11:40:28.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.154+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f787c108920 con 0x7f787c107520 2024-04-20T11:40:28.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.154+0000 7f7873400700 1 -- 172.21.15.117:0/4259328392 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f786c05af30 con 0x7f787c107520 2024-04-20T11:40:28.154 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:40:28.154 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:40:28.154 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:40:28.154 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:40:28.155 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:40:28.156 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:40:28.156 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:40:28.156 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:40:28.156 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:40:28.156 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:40:28.156 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:40:28.156 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:40:28.157 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:40:28.157 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:40:28.157 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:40:28.157 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:40:28.157 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:40:28.157 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:40:28.157 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:40:28.158 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:40:28.158 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:40:28.158 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:40:28.158 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:40:28.158 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:40:28.158 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:40:28.159 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:40:28.159 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:40:28.159 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:40:28.159 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:40:28.159 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:40:28.159 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:40:28.159 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:40:28.160 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:40:28.160 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:40:28.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f786006ecd0 msgr2=0x7f7860071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:28.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f786006ecd0 0x7f7860071190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7868006010 tx=0x7f786800b540 comp rx=0 tx=0).stop 2024-04-20T11:40:28.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c107520 msgr2=0x7f787c19d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:28.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c107520 0x7f787c19d2b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f786c00b4b0 tx=0x7f786c00b870 comp rx=0 tx=0).stop 2024-04-20T11:40:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 shutdown_connections 2024-04-20T11:40:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f786006ecd0 0x7f7860071190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f787c107520 0x7f787c19d2b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 --2- 172.21.15.117:0/4259328392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f787c108e10 0x7f787c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.157+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 >> 172.21.15.117:0/4259328392 conn(0x7f787c0768f0 msgr2=0x7f787c0775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.158+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 shutdown_connections 2024-04-20T11:40:28.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.158+0000 7f7881600700 1 -- 172.21.15.117:0/4259328392 wait complete. 2024-04-20T11:40:28.163 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:40:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.313+0000 7f64f5600700 1 -- 172.21.15.117:0/1685233826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 msgr2=0x7f64f0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.313+0000 7f64f5600700 1 --2- 172.21.15.117:0/1685233826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 0x7f64f0104e80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009910 tx=0x7f64d8009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.313+0000 7f64f5600700 1 -- 172.21.15.117:0/1685233826 shutdown_connections 2024-04-20T11:40:28.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.313+0000 7f64f5600700 1 --2- 172.21.15.117:0/1685233826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 0x7f64f0104e80 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.313+0000 7f64f5600700 1 --2- 172.21.15.117:0/1685233826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64f0103110 0x7f64f01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.313+0000 7f64f5600700 1 -- 172.21.15.117:0/1685233826 >> 172.21.15.117:0/1685233826 conn(0x7f64f00fe880 msgr2=0x7f64f0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.313+0000 7f64f5600700 1 -- 172.21.15.117:0/1685233826 shutdown_connections 2024-04-20T11:40:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.313+0000 7f64f5600700 1 -- 172.21.15.117:0/1685233826 wait complete. 2024-04-20T11:40:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.315+0000 7f64f5600700 1 Processor -- start 2024-04-20T11:40:28.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.315+0000 7f64f5600700 1 -- start start 2024-04-20T11:40:28.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.315+0000 7f64f5600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64f0103110 0x7f64f019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.315+0000 7f64f5600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 0x7f64f019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.315+0000 7f64f5600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f019f5b0 con 0x7f64f0104a00 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.315+0000 7f64f5600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0198b90 con 0x7f64f0103110 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.316+0000 7f64e7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 0x7f64f019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.316+0000 7f64e7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 0x7f64f019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36880/0 (socket says 172.21.15.117:36880) 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.316+0000 7f64e7e00700 1 -- 172.21.15.117:0/1100897493 learned_addr learned my addr 172.21.15.117:0/1100897493 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.316+0000 7f64eea00700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64f0103110 0x7f64f019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.316+0000 7f64e7e00700 1 -- 172.21.15.117:0/1100897493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64f0103110 msgr2=0x7f64f019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.316+0000 7f64e7e00700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64f0103110 0x7f64f019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.316+0000 7f64e7e00700 1 -- 172.21.15.117:0/1100897493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80095f0 con 0x7f64f0104a00 2024-04-20T11:40:28.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.316+0000 7f64e7e00700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 0x7f64f019ef90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f64d80092e0 tx=0x7f64d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.317+0000 7f64e7400700 1 -- 172.21.15.117:0/1100897493 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d801d070 con 0x7f64f0104a00 2024-04-20T11:40:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.317+0000 7f64e7400700 1 -- 172.21.15.117:0/1100897493 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64d800f460 con 0x7f64f0104a00 2024-04-20T11:40:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.317+0000 7f64e7400700 1 -- 172.21.15.117:0/1100897493 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d8017630 con 0x7f64f0104a00 2024-04-20T11:40:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.317+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f0198e10 con 0x7f64f0104a00 2024-04-20T11:40:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.317+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f0199380 con 0x7f64f0104a00 2024-04-20T11:40:28.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.318+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f010b480 con 0x7f64f0104a00 2024-04-20T11:40:28.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.319+0000 7f64e7400700 1 -- 172.21.15.117:0/1100897493 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f64d800f5d0 con 0x7f64f0104a00 2024-04-20T11:40:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.320+0000 7f64e7400700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f64d406ecd0 0x7f64d4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.320+0000 7f64e7400700 1 -- 172.21.15.117:0/1100897493 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f64d8090320 con 0x7f64f0104a00 2024-04-20T11:40:28.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.320+0000 7f64eea00700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f64d406ecd0 0x7f64d4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:28.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.321+0000 7f64eea00700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f64d406ecd0 0x7f64d4071190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005fd0 tx=0x7f64e000a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:28.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.324+0000 7f64e7400700 1 -- 172.21.15.117:0/1100897493 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f64d805cc10 con 0x7f64f0104a00 2024-04-20T11:40:28.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.568+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64f004ec20 con 0x7f64d406ecd0 2024-04-20T11:40:28.568 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:28 smithi117 ceph-mon[99036]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:40:28.568 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:28 smithi117 ceph-mon[99036]: pgmap v841: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:28.568 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:28 smithi117 ceph-mon[99036]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:40:28.569 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:28 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4256399028' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:40:28.569 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:28.569 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:28 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4259328392' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:40:28.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.570+0000 7f64e7400700 1 -- 172.21.15.117:0/1100897493 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f64f004ec20 con 0x7f64d406ecd0 2024-04-20T11:40:28.570 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:40:28.570 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:40:28.570 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:40:28.570 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:40:28.570 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:40:28.570 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:40:28.570 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:40:28.570 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:40:28.571 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:40:28.571 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:40:28.571 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:40:28.571 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:40:28.571 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:40:28.571 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:40:28.571 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:40:28.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f64d406ecd0 msgr2=0x7f64d4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:28.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f64d406ecd0 0x7f64d4071190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005fd0 tx=0x7f64e000a560 comp rx=0 tx=0).stop 2024-04-20T11:40:28.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 msgr2=0x7f64f019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:28.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 0x7f64f019ef90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f64d80092e0 tx=0x7f64d800be20 comp rx=0 tx=0).stop 2024-04-20T11:40:28.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 shutdown_connections 2024-04-20T11:40:28.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f64d406ecd0 0x7f64d4071190 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64f0103110 0x7f64f019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 --2- 172.21.15.117:0/1100897493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f0104a00 0x7f64f019ef90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.573+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 >> 172.21.15.117:0/1100897493 conn(0x7f64f00fe880 msgr2=0x7f64f00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:28.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.574+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 shutdown_connections 2024-04-20T11:40:28.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.574+0000 7f64f5600700 1 -- 172.21.15.117:0/1100897493 wait complete. 2024-04-20T11:40:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:28 smithi173 ceph-mon[74165]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:40:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:28 smithi173 ceph-mon[74165]: pgmap v841: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:28 smithi173 ceph-mon[74165]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:40:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:28 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4256399028' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:40:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:28.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:28 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4259328392' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:40:28.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.719+0000 7f5e3e600700 1 -- 172.21.15.117:0/2087240049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 msgr2=0x7f5e38104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:28.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.719+0000 7f5e3e600700 1 --2- 172.21.15.117:0/2087240049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 0x7f5e38104e80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009910 tx=0x7f5e24009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:28.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.719+0000 7f5e3e600700 1 -- 172.21.15.117:0/2087240049 shutdown_connections 2024-04-20T11:40:28.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.719+0000 7f5e3e600700 1 --2- 172.21.15.117:0/2087240049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 0x7f5e38104e80 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.719+0000 7f5e3e600700 1 --2- 172.21.15.117:0/2087240049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e38103110 0x7f5e381034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.719+0000 7f5e3e600700 1 -- 172.21.15.117:0/2087240049 >> 172.21.15.117:0/2087240049 conn(0x7f5e380fe880 msgr2=0x7f5e38100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:28.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.719+0000 7f5e3e600700 1 -- 172.21.15.117:0/2087240049 shutdown_connections 2024-04-20T11:40:28.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.719+0000 7f5e3e600700 1 -- 172.21.15.117:0/2087240049 wait complete. 2024-04-20T11:40:28.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.720+0000 7f5e3e600700 1 Processor -- start 2024-04-20T11:40:28.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.720+0000 7f5e3e600700 1 -- start start 2024-04-20T11:40:28.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.720+0000 7f5e3e600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e38103110 0x7f5e3819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:28.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.720+0000 7f5e3e600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 0x7f5e3819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:28.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.720+0000 7f5e3e600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3819f5b0 con 0x7f5e38104a00 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.720+0000 7f5e3e600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38198b90 con 0x7f5e38103110 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.721+0000 7f5e36a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 0x7f5e3819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.721+0000 7f5e36a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 0x7f5e3819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36908/0 (socket says 172.21.15.117:36908) 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.721+0000 7f5e36a00700 1 -- 172.21.15.117:0/835246823 learned_addr learned my addr 172.21.15.117:0/835246823 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.721+0000 7f5e36a00700 1 -- 172.21.15.117:0/835246823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e38103110 msgr2=0x7f5e3819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.721+0000 7f5e37400700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e38103110 0x7f5e3819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.721+0000 7f5e36a00700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e38103110 0x7f5e3819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.721+0000 7f5e36a00700 1 -- 172.21.15.117:0/835246823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240095f0 con 0x7f5e38104a00 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.721+0000 7f5e36a00700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 0x7f5e3819ef90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400bd70 tx=0x7f5e2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.722+0000 7f5e2fe00700 1 -- 172.21.15.117:0/835246823 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2401d070 con 0x7f5e38104a00 2024-04-20T11:40:28.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.722+0000 7f5e2fe00700 1 -- 172.21.15.117:0/835246823 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e2400f460 con 0x7f5e38104a00 2024-04-20T11:40:28.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.722+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e38198e10 con 0x7f5e38104a00 2024-04-20T11:40:28.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.722+0000 7f5e2fe00700 1 -- 172.21.15.117:0/835246823 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e24017630 con 0x7f5e38104a00 2024-04-20T11:40:28.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.722+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e38199380 con 0x7f5e38104a00 2024-04-20T11:40:28.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.723+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3810b480 con 0x7f5e38104a00 2024-04-20T11:40:28.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.724+0000 7f5e2fe00700 1 -- 172.21.15.117:0/835246823 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5e2400f5d0 con 0x7f5e38104a00 2024-04-20T11:40:28.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.725+0000 7f5e2fe00700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1c06ecc0 0x7f5e1c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:28.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.725+0000 7f5e2fe00700 1 -- 172.21.15.117:0/835246823 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5e24090340 con 0x7f5e38104a00 2024-04-20T11:40:28.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.725+0000 7f5e37400700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1c06ecc0 0x7f5e1c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:28.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.726+0000 7f5e37400700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1c06ecc0 0x7f5e1c071180 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5e28005950 tx=0x7f5e28005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:28.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:28.730+0000 7f5e2fe00700 1 -- 172.21.15.117:0/835246823 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5e2405cc50 con 0x7f5e38104a00 2024-04-20T11:40:29.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.022+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5e38104510 con 0x7f5e38104a00 2024-04-20T11:40:29.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.023+0000 7f5e2fe00700 1 -- 172.21.15.117:0/835246823 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f5e2405c7e0 con 0x7f5e38104a00 2024-04-20T11:40:29.023 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:40:29.023 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:40:29.023 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:40:29.024 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:40:29.025 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1c06ecc0 msgr2=0x7f5e1c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1c06ecc0 0x7f5e1c071180 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5e28005950 tx=0x7f5e28005cf0 comp rx=0 tx=0).stop 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 msgr2=0x7f5e3819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 0x7f5e3819ef90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400bd70 tx=0x7f5e2400be50 comp rx=0 tx=0).stop 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 shutdown_connections 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5e1c06ecc0 0x7f5e1c071180 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e38103110 0x7f5e3819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 --2- 172.21.15.117:0/835246823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e38104a00 0x7f5e3819ef90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.026+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 >> 172.21.15.117:0/835246823 conn(0x7f5e380fe880 msgr2=0x7f5e380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:29.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.027+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 shutdown_connections 2024-04-20T11:40:29.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:29.027+0000 7f5e3e600700 1 -- 172.21.15.117:0/835246823 wait complete. 2024-04-20T11:40:29.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:29 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/835246823' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:40:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:29 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/835246823' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:40:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:30 smithi117 ceph-mon[99036]: from='client.35534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:40:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:30 smithi117 ceph-mon[99036]: pgmap v842: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:30 smithi173 ceph-mon[74165]: from='client.35534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:40:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:30 smithi173 ceph-mon[74165]: pgmap v842: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:32 smithi117 ceph-mon[99036]: pgmap v843: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:32 smithi173 ceph-mon[74165]: pgmap v843: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:34 smithi117 ceph-mon[99036]: pgmap v844: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:34 smithi173 ceph-mon[74165]: pgmap v844: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:36 smithi117 ceph-mon[99036]: pgmap v845: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:36 smithi173 ceph-mon[74165]: pgmap v845: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:38.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:38 smithi117 ceph-mon[99036]: pgmap v846: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:38 smithi173 ceph-mon[74165]: pgmap v846: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:40.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:40 smithi117 ceph-mon[99036]: pgmap v847: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:40 smithi173 ceph-mon[74165]: pgmap v847: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:42 smithi117 ceph-mon[99036]: pgmap v848: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:42 smithi173 ceph-mon[74165]: pgmap v848: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:44 smithi117 ceph-mon[99036]: pgmap v849: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:44 smithi173 ceph-mon[74165]: pgmap v849: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:46.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:46 smithi117 ceph-mon[99036]: pgmap v850: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:46 smithi173 ceph-mon[74165]: pgmap v850: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:48.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:48 smithi117 ceph-mon[99036]: pgmap v851: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:48 smithi173 ceph-mon[74165]: pgmap v851: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:50.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:50 smithi117 ceph-mon[99036]: pgmap v852: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:50 smithi173 ceph-mon[74165]: pgmap v852: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:52.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:52 smithi117 ceph-mon[99036]: pgmap v853: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:52 smithi173 ceph-mon[74165]: pgmap v853: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:54 smithi117 ceph-mon[99036]: pgmap v854: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:54 smithi173 ceph-mon[74165]: pgmap v854: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:56 smithi117 ceph-mon[99036]: pgmap v855: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:56 smithi173 ceph-mon[74165]: pgmap v855: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:58 smithi117 ceph-mon[99036]: pgmap v856: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:40:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:58 smithi173 ceph-mon[74165]: pgmap v856: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:40:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:40:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:40:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.197+0000 7f1fea600700 1 -- 172.21.15.117:0/1635315235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4108e10 msgr2=0x7f1fe4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.197+0000 7f1fea600700 1 --2- 172.21.15.117:0/1635315235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4108e10 0x7f1fe4109290 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009910 tx=0x7f1fd0009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.197+0000 7f1fea600700 1 -- 172.21.15.117:0/1635315235 shutdown_connections 2024-04-20T11:40:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.197+0000 7f1fea600700 1 --2- 172.21.15.117:0/1635315235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4108e10 0x7f1fe4109290 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.197+0000 7f1fea600700 1 --2- 172.21.15.117:0/1635315235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4107520 0x7f1fe4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.197+0000 7f1fea600700 1 -- 172.21.15.117:0/1635315235 >> 172.21.15.117:0/1635315235 conn(0x7f1fe40768f0 msgr2=0x7f1fe4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.197+0000 7f1fea600700 1 -- 172.21.15.117:0/1635315235 shutdown_connections 2024-04-20T11:40:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.197+0000 7f1fea600700 1 -- 172.21.15.117:0/1635315235 wait complete. 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.198+0000 7f1fea600700 1 Processor -- start 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.198+0000 7f1fea600700 1 -- start start 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.198+0000 7f1fea600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4107520 0x7f1fe419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.198+0000 7f1fea600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4108e10 0x7f1fe419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.198+0000 7f1fea600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419dd60 con 0x7f1fe4107520 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.198+0000 7f1fea600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419dea0 con 0x7f1fe4108e10 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.199+0000 7f1fe3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4107520 0x7f1fe419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.199+0000 7f1fe3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4107520 0x7f1fe419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39952/0 (socket says 172.21.15.117:39952) 2024-04-20T11:40:59.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.199+0000 7f1fe3400700 1 -- 172.21.15.117:0/1119721251 learned_addr learned my addr 172.21.15.117:0/1119721251 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:59.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.199+0000 7f1fe2a00700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4108e10 0x7f1fe419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:59.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.199+0000 7f1fe3400700 1 -- 172.21.15.117:0/1119721251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4108e10 msgr2=0x7f1fe419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:59.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.199+0000 7f1fe3400700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4108e10 0x7f1fe419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.199+0000 7f1fe3400700 1 -- 172.21.15.117:0/1119721251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd00095f0 con 0x7f1fe4107520 2024-04-20T11:40:59.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.200+0000 7f1fe3400700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4107520 0x7f1fe419d1d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400c930 tx=0x7f1fd400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:59.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.200+0000 7f1fdbe00700 1 -- 172.21.15.117:0/1119721251 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd4007ad0 con 0x7f1fe4107520 2024-04-20T11:40:59.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.200+0000 7f1fdbe00700 1 -- 172.21.15.117:0/1119721251 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fd4007c30 con 0x7f1fe4107520 2024-04-20T11:40:59.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.200+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe41a1be0 con 0x7f1fe4107520 2024-04-20T11:40:59.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.200+0000 7f1fdbe00700 1 -- 172.21.15.117:0/1119721251 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd4018610 con 0x7f1fe4107520 2024-04-20T11:40:59.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.200+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe41a2160 con 0x7f1fe4107520 2024-04-20T11:40:59.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.201+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe410f870 con 0x7f1fe4107520 2024-04-20T11:40:59.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.202+0000 7f1fdbe00700 1 -- 172.21.15.117:0/1119721251 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1fd400f450 con 0x7f1fe4107520 2024-04-20T11:40:59.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.203+0000 7f1fdbe00700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fc806ecd0 0x7f1fc8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:59.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.204+0000 7f1fdbe00700 1 -- 172.21.15.117:0/1119721251 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1fd408f0f0 con 0x7f1fe4107520 2024-04-20T11:40:59.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.204+0000 7f1fe2a00700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fc806ecd0 0x7f1fc8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:59.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.205+0000 7f1fe2a00700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fc806ecd0 0x7f1fc8071190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0005f50 tx=0x7f1fd001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:59.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.209+0000 7f1fdbe00700 1 -- 172.21.15.117:0/1119721251 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1fd4057e30 con 0x7f1fe4107520 2024-04-20T11:40:59.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.451+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fe419e530 con 0x7f1fc806ecd0 2024-04-20T11:40:59.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.454+0000 7f1fdbe00700 1 -- 172.21.15.117:0/1119721251 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1fe419e530 con 0x7f1fc806ecd0 2024-04-20T11:40:59.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fc806ecd0 msgr2=0x7f1fc8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:59.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fc806ecd0 0x7f1fc8071190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0005f50 tx=0x7f1fd001a040 comp rx=0 tx=0).stop 2024-04-20T11:40:59.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4107520 msgr2=0x7f1fe419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:59.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4107520 0x7f1fe419d1d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400c930 tx=0x7f1fd400ccf0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 shutdown_connections 2024-04-20T11:40:59.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fc806ecd0 0x7f1fc8071190 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe4107520 0x7f1fe419d1d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 --2- 172.21.15.117:0/1119721251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4108e10 0x7f1fe419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.457+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 >> 172.21.15.117:0/1119721251 conn(0x7f1fe40768f0 msgr2=0x7f1fe4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:59.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.458+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 shutdown_connections 2024-04-20T11:40:59.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.458+0000 7f1fea600700 1 -- 172.21.15.117:0/1119721251 wait complete. 2024-04-20T11:40:59.473 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:40:59.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.602+0000 7f36f2400700 1 -- 172.21.15.117:0/183746612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 msgr2=0x7f36ec103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:59.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.602+0000 7f36f2400700 1 --2- 172.21.15.117:0/183746612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 0x7f36ec103400 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009910 tx=0x7f36dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:40:59.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.602+0000 7f36f2400700 1 -- 172.21.15.117:0/183746612 shutdown_connections 2024-04-20T11:40:59.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.602+0000 7f36f2400700 1 --2- 172.21.15.117:0/183746612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36ec1039d0 0x7f36ec107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.602+0000 7f36f2400700 1 --2- 172.21.15.117:0/183746612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 0x7f36ec103400 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.602+0000 7f36f2400700 1 -- 172.21.15.117:0/183746612 >> 172.21.15.117:0/183746612 conn(0x7f36ec0fe890 msgr2=0x7f36ec100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.602+0000 7f36f2400700 1 -- 172.21.15.117:0/183746612 shutdown_connections 2024-04-20T11:40:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.602+0000 7f36f2400700 1 -- 172.21.15.117:0/183746612 wait complete. 2024-04-20T11:40:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.603+0000 7f36f2400700 1 Processor -- start 2024-04-20T11:40:59.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.604+0000 7f36f2400700 1 -- start start 2024-04-20T11:40:59.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.604+0000 7f36f2400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 0x7f36ec19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:59.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.604+0000 7f36f2400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36ec1039d0 0x7f36ec19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:59.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.604+0000 7f36f2400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec19f5c0 con 0x7f36ec103020 2024-04-20T11:40:59.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.604+0000 7f36f2400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec198bf0 con 0x7f36ec1039d0 2024-04-20T11:40:59.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.604+0000 7f36eb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 0x7f36ec19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:59.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.604+0000 7f36eb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 0x7f36ec19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39960/0 (socket says 172.21.15.117:39960) 2024-04-20T11:40:59.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.604+0000 7f36eb400700 1 -- 172.21.15.117:0/137826810 learned_addr learned my addr 172.21.15.117:0/137826810 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36eb400700 1 -- 172.21.15.117:0/137826810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36ec1039d0 msgr2=0x7f36ec19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36eb400700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36ec1039d0 0x7f36ec19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36eb400700 1 -- 172.21.15.117:0/137826810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc0095f0 con 0x7f36ec103020 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36eb400700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 0x7f36ec19ea60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00bd70 tx=0x7f36dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36e3e00700 1 -- 172.21.15.117:0/137826810 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36dc01d070 con 0x7f36ec103020 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36e3e00700 1 -- 172.21.15.117:0/137826810 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36dc0056e0 con 0x7f36ec103020 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36e3e00700 1 -- 172.21.15.117:0/137826810 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36dc017400 con 0x7f36ec103020 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec198e70 con 0x7f36ec103020 2024-04-20T11:40:59.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.605+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec199310 con 0x7f36ec103020 2024-04-20T11:40:59.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.606+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec10b3c0 con 0x7f36ec103020 2024-04-20T11:40:59.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.607+0000 7f36e3e00700 1 -- 172.21.15.117:0/137826810 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f36dc003a60 con 0x7f36ec103020 2024-04-20T11:40:59.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.608+0000 7f36e3e00700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f36d006ecc0 0x7f36d0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:40:59.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.608+0000 7f36e3e00700 1 -- 172.21.15.117:0/137826810 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f36dc090310 con 0x7f36ec103020 2024-04-20T11:40:59.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.609+0000 7f36eaa00700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f36d006ecc0 0x7f36d0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:40:59.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.609+0000 7f36eaa00700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f36d006ecc0 0x7f36d0071180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005950 tx=0x7f36d80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:40:59.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.613+0000 7f36e3e00700 1 -- 172.21.15.117:0/137826810 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f36dc05cb80 con 0x7f36ec103020 2024-04-20T11:40:59.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.855+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36ec199ff0 con 0x7f36d006ecc0 2024-04-20T11:40:59.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.857+0000 7f36e3e00700 1 -- 172.21.15.117:0/137826810 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f36ec199ff0 con 0x7f36d006ecc0 2024-04-20T11:40:59.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.860+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f36d006ecc0 msgr2=0x7f36d0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:59.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.860+0000 7f36f2400700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f36d006ecc0 0x7f36d0071180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005950 tx=0x7f36d80058e0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.860+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 msgr2=0x7f36ec19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:40:59.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.860+0000 7f36f2400700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 0x7f36ec19ea60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00bd70 tx=0x7f36dc00be50 comp rx=0 tx=0).stop 2024-04-20T11:40:59.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.861+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 shutdown_connections 2024-04-20T11:40:59.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.861+0000 7f36f2400700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f36d006ecc0 0x7f36d0071180 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.861+0000 7f36f2400700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36ec103020 0x7f36ec19ea60 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.861+0000 7f36f2400700 1 --2- 172.21.15.117:0/137826810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36ec1039d0 0x7f36ec19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:40:59.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.861+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 >> 172.21.15.117:0/137826810 conn(0x7f36ec0fe890 msgr2=0x7f36ec0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:40:59.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.861+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 shutdown_connections 2024-04-20T11:40:59.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:40:59.861+0000 7f36f2400700 1 -- 172.21.15.117:0/137826810 wait complete. 2024-04-20T11:41:00.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.009+0000 7fcc93200700 1 -- 172.21.15.117:0/1893103247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 msgr2=0x7fcc94105ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.009+0000 7fcc93200700 1 --2- 172.21.15.117:0/1893103247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 0x7fcc94105ca0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009910 tx=0x7fcc84009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:00.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.009+0000 7fcc93200700 1 -- 172.21.15.117:0/1893103247 shutdown_connections 2024-04-20T11:41:00.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.009+0000 7fcc93200700 1 --2- 172.21.15.117:0/1893103247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc940ff8c0 0x7fcc940ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.009+0000 7fcc93200700 1 --2- 172.21.15.117:0/1893103247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 0x7fcc94105ca0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.009+0000 7fcc93200700 1 -- 172.21.15.117:0/1893103247 >> 172.21.15.117:0/1893103247 conn(0x7fcc940fb3f0 msgr2=0x7fcc940fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:00.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.009+0000 7fcc93200700 1 -- 172.21.15.117:0/1893103247 shutdown_connections 2024-04-20T11:41:00.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.009+0000 7fcc93200700 1 -- 172.21.15.117:0/1893103247 wait complete. 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.010+0000 7fcc93200700 1 Processor -- start 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.011+0000 7fcc93200700 1 -- start start 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.011+0000 7fcc93200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc940ff8c0 0x7fcc940fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.011+0000 7fcc93200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 0x7fcc940fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.011+0000 7fcc93200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940fc650 con 0x7fcc941058c0 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.011+0000 7fcc93200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940fc790 con 0x7fcc940ff8c0 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.011+0000 7fcc91400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 0x7fcc940fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.011+0000 7fcc91400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 0x7fcc940fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39974/0 (socket says 172.21.15.117:39974) 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.011+0000 7fcc91400700 1 -- 172.21.15.117:0/1355239580 learned_addr learned my addr 172.21.15.117:0/1355239580 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:00.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc91e00700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc940ff8c0 0x7fcc940fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:00.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc91400700 1 -- 172.21.15.117:0/1355239580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc940ff8c0 msgr2=0x7fcc940fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc91400700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc940ff8c0 0x7fcc940fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc91400700 1 -- 172.21.15.117:0/1355239580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc840095f0 con 0x7fcc941058c0 2024-04-20T11:41:00.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc91400700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 0x7fcc940fc110 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000d8d0 tx=0x7fcc8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:00.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc8aa00700 1 -- 172.21.15.117:0/1355239580 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc80009940 con 0x7fcc941058c0 2024-04-20T11:41:00.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc8aa00700 1 -- 172.21.15.117:0/1355239580 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc80010460 con 0x7fcc941058c0 2024-04-20T11:41:00.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc941a2ad0 con 0x7fcc941058c0 2024-04-20T11:41:00.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc8aa00700 1 -- 172.21.15.117:0/1355239580 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc8000f5f0 con 0x7fcc941058c0 2024-04-20T11:41:00.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.012+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc941a2e30 con 0x7fcc941058c0 2024-04-20T11:41:00.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.013+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc94108ed0 con 0x7fcc941058c0 2024-04-20T11:41:00.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.015+0000 7fcc8aa00700 1 -- 172.21.15.117:0/1355239580 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcc800105d0 con 0x7fcc941058c0 2024-04-20T11:41:00.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.016+0000 7fcc8aa00700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7806ecc0 0x7fcc78071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.016+0000 7fcc8aa00700 1 -- 172.21.15.117:0/1355239580 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcc8008eef0 con 0x7fcc941058c0 2024-04-20T11:41:00.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.016+0000 7fcc91e00700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7806ecc0 0x7fcc78071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.017+0000 7fcc91e00700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7806ecc0 0x7fcc78071180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006010 tx=0x7fcc8400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.021+0000 7fcc8aa00700 1 -- 172.21.15.117:0/1355239580 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcc80057c30 con 0x7fcc941058c0 2024-04-20T11:41:00.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.250+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc9410cac0 con 0x7fcc7806ecc0 2024-04-20T11:41:00.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.282+0000 7fcc8aa00700 1 -- 172.21.15.117:0/1355239580 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcc9410cac0 con 0x7fcc7806ecc0 2024-04-20T11:41:00.283 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (29m) 5m ago 43m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (43m) 5m ago 43m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (42m) 5m ago 42m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (25m) 5m ago 43m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (25m) 5m ago 42m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (28m) 5m ago 43m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (21m) 5m ago 36m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (21m) 5m ago 36m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (5m) 5m ago 36m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (20m) 5m ago 36m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (32m) 5m ago 45m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:41:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (31m) 5m ago 42m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (27m) 5m ago 45m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (26m) 5m ago 42m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (30m) 5m ago 43m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (30m) 5m ago 42m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (25m) 5m ago 41m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (24m) 5m ago 40m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (24m) 5m ago 39m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (23m) 5m ago 39m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (22m) 5m ago 38m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (22m) 5m ago 38m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:41:00.285 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (30m) 5m ago 42m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:41:00.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7806ecc0 msgr2=0x7fcc78071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7806ecc0 0x7fcc78071180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006010 tx=0x7fcc8400b540 comp rx=0 tx=0).stop 2024-04-20T11:41:00.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 msgr2=0x7fcc940fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 0x7fcc940fc110 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000d8d0 tx=0x7fcc8000dc90 comp rx=0 tx=0).stop 2024-04-20T11:41:00.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 shutdown_connections 2024-04-20T11:41:00.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcc7806ecc0 0x7fcc78071180 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc940ff8c0 0x7fcc940fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 --2- 172.21.15.117:0/1355239580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc941058c0 0x7fcc940fc110 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 >> 172.21.15.117:0/1355239580 conn(0x7fcc940fb3f0 msgr2=0x7fcc94069300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:00.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.285+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 shutdown_connections 2024-04-20T11:41:00.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.286+0000 7fcc93200700 1 -- 172.21.15.117:0/1355239580 wait complete. 2024-04-20T11:41:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.428+0000 7f56c8600700 1 -- 172.21.15.117:0/3389491717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 msgr2=0x7f56c01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.428+0000 7f56c8600700 1 --2- 172.21.15.117:0/3389491717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 0x7f56c01034f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009910 tx=0x7f56b4009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.428+0000 7f56c8600700 1 -- 172.21.15.117:0/3389491717 shutdown_connections 2024-04-20T11:41:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.428+0000 7f56c8600700 1 --2- 172.21.15.117:0/3389491717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56c0104a00 0x7f56c0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.428+0000 7f56c8600700 1 --2- 172.21.15.117:0/3389491717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 0x7f56c01034f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.428+0000 7f56c8600700 1 -- 172.21.15.117:0/3389491717 >> 172.21.15.117:0/3389491717 conn(0x7f56c00fe880 msgr2=0x7f56c0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.428+0000 7f56c8600700 1 -- 172.21.15.117:0/3389491717 shutdown_connections 2024-04-20T11:41:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.428+0000 7f56c8600700 1 -- 172.21.15.117:0/3389491717 wait complete. 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.429+0000 7f56c8600700 1 Processor -- start 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.429+0000 7f56c8600700 1 -- start start 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c8600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 0x7f56c0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c8600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56c0104a00 0x7f56c01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c8600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0199a80 con 0x7f56c0103110 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c8600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019d750 con 0x7f56c0104a00 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c5c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 0x7f56c0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c5c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 0x7f56c0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40004/0 (socket says 172.21.15.117:40004) 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c5c00700 1 -- 172.21.15.117:0/1220091605 learned_addr learned my addr 172.21.15.117:0/1220091605 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c5c00700 1 -- 172.21.15.117:0/1220091605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56c0104a00 msgr2=0x7f56c01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c5c00700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56c0104a00 0x7f56c01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c5c00700 1 -- 172.21.15.117:0/1220091605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40095f0 con 0x7f56c0103110 2024-04-20T11:41:00.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.430+0000 7f56c5c00700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 0x7f56c0198e60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f56b4006010 tx=0x7f56b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:00.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.431+0000 7f56be000700 1 -- 172.21.15.117:0/1220091605 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b401d070 con 0x7f56c0103110 2024-04-20T11:41:00.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.431+0000 7f56be000700 1 -- 172.21.15.117:0/1220091605 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56b400f460 con 0x7f56c0103110 2024-04-20T11:41:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.431+0000 7f56be000700 1 -- 172.21.15.117:0/1220091605 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b4017630 con 0x7f56c0103110 2024-04-20T11:41:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.431+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c019d9d0 con 0x7f56c0103110 2024-04-20T11:41:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.431+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c019def0 con 0x7f56c0103110 2024-04-20T11:41:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.432+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c010b460 con 0x7f56c0103110 2024-04-20T11:41:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.433+0000 7f56be000700 1 -- 172.21.15.117:0/1220091605 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f56b400f5d0 con 0x7f56c0103110 2024-04-20T11:41:00.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.434+0000 7f56be000700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f56a806ecc0 0x7f56a8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.434+0000 7f56be000700 1 -- 172.21.15.117:0/1220091605 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f56b4090370 con 0x7f56c0103110 2024-04-20T11:41:00.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.434+0000 7f56c5200700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f56a806ecc0 0x7f56a8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:00.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.435+0000 7f56c5200700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f56a806ecc0 0x7f56a8071180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f56c019a3c0 tx=0x7f56ac006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:00.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.439+0000 7f56be000700 1 -- 172.21.15.117:0/1220091605 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f56b4059170 con 0x7f56c0103110 2024-04-20T11:41:00.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:00 smithi117 ceph-mon[99036]: pgmap v857: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:00 smithi173 ceph-mon[74165]: pgmap v857: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:00.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.732+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f56c019a170 con 0x7f56c0103110 2024-04-20T11:41:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.733+0000 7f56be000700 1 -- 172.21.15.117:0/1220091605 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f56b405c830 con 0x7f56c0103110 2024-04-20T11:41:00.734 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:41:00.734 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:41:00.734 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:41:00.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:41:00.734 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:41:00.735 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f56a806ecc0 msgr2=0x7f56a8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f56a806ecc0 0x7f56a8071180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f56c019a3c0 tx=0x7f56ac006d20 comp rx=0 tx=0).stop 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 msgr2=0x7f56c0198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 0x7f56c0198e60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f56b4006010 tx=0x7f56b400be20 comp rx=0 tx=0).stop 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 shutdown_connections 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f56a806ecc0 0x7f56a8071180 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56c0103110 0x7f56c0198e60 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 --2- 172.21.15.117:0/1220091605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56c0104a00 0x7f56c01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 >> 172.21.15.117:0/1220091605 conn(0x7f56c00fe880 msgr2=0x7f56c00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 shutdown_connections 2024-04-20T11:41:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.736+0000 7f56c8600700 1 -- 172.21.15.117:0/1220091605 wait complete. 2024-04-20T11:41:00.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.881+0000 7f407e800700 1 -- 172.21.15.117:0/184233242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4078073bd0 msgr2=0x7f407810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.881+0000 7f407e800700 1 --2- 172.21.15.117:0/184233242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4078073bd0 0x7f407810cf10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4064009910 tx=0x7f4064009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.882+0000 7f407e800700 1 -- 172.21.15.117:0/184233242 shutdown_connections 2024-04-20T11:41:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.882+0000 7f407e800700 1 --2- 172.21.15.117:0/184233242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4078073bd0 0x7f407810cf10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.882+0000 7f407e800700 1 --2- 172.21.15.117:0/184233242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40780732b0 0x7f4078073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.882+0000 7f407e800700 1 -- 172.21.15.117:0/184233242 >> 172.21.15.117:0/184233242 conn(0x7f40780fc6a0 msgr2=0x7f40780feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.882+0000 7f407e800700 1 -- 172.21.15.117:0/184233242 shutdown_connections 2024-04-20T11:41:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.882+0000 7f407e800700 1 -- 172.21.15.117:0/184233242 wait complete. 2024-04-20T11:41:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.883+0000 7f407e800700 1 Processor -- start 2024-04-20T11:41:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.883+0000 7f407e800700 1 -- start start 2024-04-20T11:41:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.883+0000 7f407e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40780732b0 0x7f4078198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.883+0000 7f407e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4078073bd0 0x7f4078199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.883+0000 7f407e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40781998d0 con 0x7f40780732b0 2024-04-20T11:41:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.883+0000 7f407e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407819c540 con 0x7f4078073bd0 2024-04-20T11:41:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.884+0000 7f4077400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40780732b0 0x7f4078198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:00.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.884+0000 7f4077400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40780732b0 0x7f4078198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40018/0 (socket says 172.21.15.117:40018) 2024-04-20T11:41:00.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.884+0000 7f4077400700 1 -- 172.21.15.117:0/1115377270 learned_addr learned my addr 172.21.15.117:0/1115377270 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:00.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.884+0000 7f4077400700 1 -- 172.21.15.117:0/1115377270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4078073bd0 msgr2=0x7f4078199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.884+0000 7f4077400700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4078073bd0 0x7f4078199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.884+0000 7f4077400700 1 -- 172.21.15.117:0/1115377270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40640095f0 con 0x7f40780732b0 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.884+0000 7f4077400700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40780732b0 0x7f4078198d00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f406800d8d0 tx=0x7f406800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.885+0000 7f406fe00700 1 -- 172.21.15.117:0/1115377270 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4068009940 con 0x7f40780732b0 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.885+0000 7f406fe00700 1 -- 172.21.15.117:0/1115377270 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4068010460 con 0x7f40780732b0 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.885+0000 7f406fe00700 1 -- 172.21.15.117:0/1115377270 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406800f5f0 con 0x7f40780732b0 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.885+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407819c820 con 0x7f40780732b0 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.885+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407819cd70 con 0x7f40780732b0 2024-04-20T11:41:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.886+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4078074960 con 0x7f40780732b0 2024-04-20T11:41:00.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.887+0000 7f406fe00700 1 -- 172.21.15.117:0/1115377270 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f40680105d0 con 0x7f40780732b0 2024-04-20T11:41:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.888+0000 7f406fe00700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f405c06ecc0 0x7f405c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.888+0000 7f406fe00700 1 -- 172.21.15.117:0/1115377270 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f406808ede0 con 0x7f40780732b0 2024-04-20T11:41:00.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.888+0000 7f4076a00700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f405c06ecc0 0x7f405c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:00.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.889+0000 7f4076a00700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f405c06ecc0 0x7f405c071180 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f406400b5c0 tx=0x7f4064005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:00.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:00.893+0000 7f406fe00700 1 -- 172.21.15.117:0/1115377270 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4068057ba0 con 0x7f40780732b0 2024-04-20T11:41:01.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.163+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4078107840 con 0x7f40780732b0 2024-04-20T11:41:01.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.164+0000 7f406fe00700 1 -- 172.21.15.117:0/1115377270 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f4068016780 con 0x7f40780732b0 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:41:01.166 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:41:01.167 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:41:01.168 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:41:01.169 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:41:01.169 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:41:01.169 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:41:01.169 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:41:01.169 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f405c06ecc0 msgr2=0x7f405c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f405c06ecc0 0x7f405c071180 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f406400b5c0 tx=0x7f4064005960 comp rx=0 tx=0).stop 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40780732b0 msgr2=0x7f4078198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40780732b0 0x7f4078198d00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f406800d8d0 tx=0x7f406800dc90 comp rx=0 tx=0).stop 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 shutdown_connections 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f405c06ecc0 0x7f405c071180 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40780732b0 0x7f4078198d00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 --2- 172.21.15.117:0/1115377270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4078073bd0 0x7f4078199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.167+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 >> 172.21.15.117:0/1115377270 conn(0x7f40780fc6a0 msgr2=0x7f40781053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.168+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 shutdown_connections 2024-04-20T11:41:01.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.168+0000 7f407e800700 1 -- 172.21.15.117:0/1115377270 wait complete. 2024-04-20T11:41:01.171 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:41:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.328+0000 7f033e000700 1 -- 172.21.15.117:0/3138099088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 msgr2=0x7f0338107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.328+0000 7f033e000700 1 --2- 172.21.15.117:0/3138099088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 0x7f0338107a30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0324009910 tx=0x7f0324009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.328+0000 7f033e000700 1 -- 172.21.15.117:0/3138099088 shutdown_connections 2024-04-20T11:41:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.328+0000 7f033e000700 1 --2- 172.21.15.117:0/3138099088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 0x7f0338107a30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.328+0000 7f033e000700 1 --2- 172.21.15.117:0/3138099088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0338103030 0x7f0338103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.328+0000 7f033e000700 1 -- 172.21.15.117:0/3138099088 >> 172.21.15.117:0/3138099088 conn(0x7f03380fe8e0 msgr2=0x7f0338100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.328+0000 7f033e000700 1 -- 172.21.15.117:0/3138099088 shutdown_connections 2024-04-20T11:41:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.328+0000 7f033e000700 1 -- 172.21.15.117:0/3138099088 wait complete. 2024-04-20T11:41:01.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.329+0000 7f033e000700 1 Processor -- start 2024-04-20T11:41:01.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.329+0000 7f033e000700 1 -- start start 2024-04-20T11:41:01.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.329+0000 7f033e000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0338103030 0x7f033819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:01.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f033e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 0x7f033819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:01.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f033e000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033819f520 con 0x7f03381039e0 2024-04-20T11:41:01.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f033e000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338198b90 con 0x7f0338103030 2024-04-20T11:41:01.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f0336a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 0x7f033819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:01.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f0336a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 0x7f033819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40028/0 (socket says 172.21.15.117:40028) 2024-04-20T11:41:01.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f0336a00700 1 -- 172.21.15.117:0/957259391 learned_addr learned my addr 172.21.15.117:0/957259391 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:01.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f0337400700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0338103030 0x7f033819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f0336a00700 1 -- 172.21.15.117:0/957259391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0338103030 msgr2=0x7f033819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f0336a00700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0338103030 0x7f033819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.330+0000 7f0336a00700 1 -- 172.21.15.117:0/957259391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03240095f0 con 0x7f03381039e0 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.331+0000 7f0336a00700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 0x7f033819ef90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f03240092e0 tx=0x7f032400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.331+0000 7f032fe00700 1 -- 172.21.15.117:0/957259391 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f032401d070 con 0x7f03381039e0 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.331+0000 7f032fe00700 1 -- 172.21.15.117:0/957259391 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f032400f460 con 0x7f03381039e0 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.331+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0338198e10 con 0x7f03381039e0 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.331+0000 7f032fe00700 1 -- 172.21.15.117:0/957259391 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0324017610 con 0x7f03381039e0 2024-04-20T11:41:01.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.331+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0338199300 con 0x7f03381039e0 2024-04-20T11:41:01.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.332+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033810b430 con 0x7f03381039e0 2024-04-20T11:41:01.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.333+0000 7f032fe00700 1 -- 172.21.15.117:0/957259391 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f032400f5d0 con 0x7f03381039e0 2024-04-20T11:41:01.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.334+0000 7f032fe00700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f031c06ecd0 0x7f031c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:01.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.334+0000 7f032fe00700 1 -- 172.21.15.117:0/957259391 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f03240903b0 con 0x7f03381039e0 2024-04-20T11:41:01.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.334+0000 7f0337400700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f031c06ecd0 0x7f031c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:01.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.335+0000 7f0337400700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f031c06ecd0 0x7f031c071190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0328005ea0 tx=0x7f0328005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:01.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.339+0000 7f032fe00700 1 -- 172.21.15.117:0/957259391 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f032405cc20 con 0x7f03381039e0 2024-04-20T11:41:01.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.602+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f033819a050 con 0x7f031c06ecd0 2024-04-20T11:41:01.603 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:01 smithi117 ceph-mon[99036]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:01.603 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:01 smithi117 ceph-mon[99036]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:01.603 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:01 smithi117 ceph-mon[99036]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:01.603 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:01 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1220091605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:41:01.603 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:01 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1115377270' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.605+0000 7f032fe00700 1 -- 172.21.15.117:0/957259391 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f033819a050 con 0x7f031c06ecd0 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:41:01.606 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:41:01.607 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:41:01.607 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:41:01.607 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:41:01.607 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:41:01.607 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:41:01.607 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:41:01.607 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:41:01.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f031c06ecd0 msgr2=0x7f031c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:01.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f031c06ecd0 0x7f031c071190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0328005ea0 tx=0x7f0328005e30 comp rx=0 tx=0).stop 2024-04-20T11:41:01.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 msgr2=0x7f033819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:01.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 0x7f033819ef90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f03240092e0 tx=0x7f032400be20 comp rx=0 tx=0).stop 2024-04-20T11:41:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 shutdown_connections 2024-04-20T11:41:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f031c06ecd0 0x7f031c071190 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0338103030 0x7f033819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 --2- 172.21.15.117:0/957259391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03381039e0 0x7f033819ef90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.608+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 >> 172.21.15.117:0/957259391 conn(0x7f03380fe8e0 msgr2=0x7f03380fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.609+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 shutdown_connections 2024-04-20T11:41:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.609+0000 7f033e000700 1 -- 172.21.15.117:0/957259391 wait complete. 2024-04-20T11:41:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:01 smithi173 ceph-mon[74165]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:01 smithi173 ceph-mon[74165]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:01 smithi173 ceph-mon[74165]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:01 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1220091605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:41:01.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:01 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1115377270' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:41:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.762+0000 7efde4200700 1 -- 172.21.15.117:0/3222940690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc101d80 msgr2=0x7efddc102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.762+0000 7efde4200700 1 --2- 172.21.15.117:0/3222940690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc101d80 0x7efddc102160 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009910 tx=0x7efdd0009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.762+0000 7efde4200700 1 -- 172.21.15.117:0/3222940690 shutdown_connections 2024-04-20T11:41:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.762+0000 7efde4200700 1 --2- 172.21.15.117:0/3222940690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efddc1026a0 0x7efddc10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.762+0000 7efde4200700 1 --2- 172.21.15.117:0/3222940690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc101d80 0x7efddc102160 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.762+0000 7efde4200700 1 -- 172.21.15.117:0/3222940690 >> 172.21.15.117:0/3222940690 conn(0x7efddc0fb5d0 msgr2=0x7efddc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.762+0000 7efde4200700 1 -- 172.21.15.117:0/3222940690 shutdown_connections 2024-04-20T11:41:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.762+0000 7efde4200700 1 -- 172.21.15.117:0/3222940690 wait complete. 2024-04-20T11:41:01.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.763+0000 7efde4200700 1 Processor -- start 2024-04-20T11:41:01.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.763+0000 7efde4200700 1 -- start start 2024-04-20T11:41:01.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde4200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efddc101d80 0x7efddc198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:01.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde4200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc1026a0 0x7efddc199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:01.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde4200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc199a70 con 0x7efddc1026a0 2024-04-20T11:41:01.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde4200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc19d740 con 0x7efddc101d80 2024-04-20T11:41:01.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde0e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc1026a0 0x7efddc199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:01.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde0e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc1026a0 0x7efddc199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40036/0 (socket says 172.21.15.117:40036) 2024-04-20T11:41:01.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde0e00700 1 -- 172.21.15.117:0/415957821 learned_addr learned my addr 172.21.15.117:0/415957821 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde0e00700 1 -- 172.21.15.117:0/415957821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efddc101d80 msgr2=0x7efddc198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde1800700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efddc101d80 0x7efddc198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde0e00700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efddc101d80 0x7efddc198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.764+0000 7efde0e00700 1 -- 172.21.15.117:0/415957821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00095f0 con 0x7efddc1026a0 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.765+0000 7efde0e00700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc1026a0 0x7efddc199390 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efdc800b4d0 tx=0x7efdc800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.765+0000 7efdda000700 1 -- 172.21.15.117:0/415957821 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdc8011800 con 0x7efddc1026a0 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.765+0000 7efdda000700 1 -- 172.21.15.117:0/415957821 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdc8011e20 con 0x7efddc1026a0 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.765+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc19da20 con 0x7efddc1026a0 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.765+0000 7efdda000700 1 -- 172.21.15.117:0/415957821 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdc800f2e0 con 0x7efddc1026a0 2024-04-20T11:41:01.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.765+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc19df70 con 0x7efddc1026a0 2024-04-20T11:41:01.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.766+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc10a6f0 con 0x7efddc1026a0 2024-04-20T11:41:01.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.767+0000 7efdda000700 1 -- 172.21.15.117:0/415957821 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efdc8010380 con 0x7efddc1026a0 2024-04-20T11:41:01.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.768+0000 7efdda000700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdc406ecc0 0x7efdc4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:01.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.769+0000 7efdda000700 1 -- 172.21.15.117:0/415957821 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efdc808e970 con 0x7efddc1026a0 2024-04-20T11:41:01.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.769+0000 7efde1800700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdc406ecc0 0x7efdc4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:01.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.769+0000 7efde1800700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdc406ecc0 0x7efdc4071180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efdd000b5c0 tx=0x7efdd0005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:01.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:01.773+0000 7efdda000700 1 -- 172.21.15.117:0/415957821 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efdc8057730 con 0x7efddc1026a0 2024-04-20T11:41:02.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.085+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efddc0696e0 con 0x7efddc1026a0 2024-04-20T11:41:02.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.086+0000 7efdda000700 1 -- 172.21.15.117:0/415957821 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7efdc805adf0 con 0x7efddc1026a0 2024-04-20T11:41:02.087 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:41:02.087 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:41:02.087 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:41:02.087 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:41:02.087 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:41:02.087 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:41:02.087 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:41:02.088 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:41:02.089 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:41:02.089 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:41:02.089 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:41:02.089 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:41:02.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdc406ecc0 msgr2=0x7efdc4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:02.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdc406ecc0 0x7efdc4071180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efdd000b5c0 tx=0x7efdd0005fd0 comp rx=0 tx=0).stop 2024-04-20T11:41:02.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc1026a0 msgr2=0x7efddc199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:02.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc1026a0 0x7efddc199390 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efdc800b4d0 tx=0x7efdc800b890 comp rx=0 tx=0).stop 2024-04-20T11:41:02.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 shutdown_connections 2024-04-20T11:41:02.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efdc406ecc0 0x7efdc4071180 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:02.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efddc101d80 0x7efddc198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:02.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 --2- 172.21.15.117:0/415957821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efddc1026a0 0x7efddc199390 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:02.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 >> 172.21.15.117:0/415957821 conn(0x7efddc0fb5d0 msgr2=0x7efddc101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:02.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.089+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 shutdown_connections 2024-04-20T11:41:02.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:02.090+0000 7efde4200700 1 -- 172.21.15.117:0/415957821 wait complete. 2024-04-20T11:41:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:02 smithi173 ceph-mon[74165]: pgmap v858: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:02 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/415957821' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:41:02.776 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:02 smithi117 ceph-mon[99036]: pgmap v858: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:02.777 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:02 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/415957821' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:41:03.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:03 smithi117 ceph-mon[99036]: from='client.35562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:03.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:03 smithi173 ceph-mon[74165]: from='client.35562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:04.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:04 smithi117 ceph-mon[99036]: pgmap v859: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:04 smithi173 ceph-mon[74165]: pgmap v859: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:06.714 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:06 smithi173 ceph-mon[74165]: pgmap v860: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:06 smithi117 ceph-mon[99036]: pgmap v860: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:08.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:08 smithi117 ceph-mon[99036]: pgmap v861: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:09.670 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:08 smithi173 ceph-mon[74165]: pgmap v861: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:10 smithi117 ceph-mon[99036]: pgmap v862: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:10 smithi173 ceph-mon[74165]: pgmap v862: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:12.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:12 smithi117 ceph-mon[99036]: pgmap v863: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:12 smithi173 ceph-mon[74165]: pgmap v863: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:13.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:41:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:41:14.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:14 smithi117 ceph-mon[99036]: pgmap v864: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:14 smithi173 ceph-mon[74165]: pgmap v864: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:15 smithi117 ceph-mon[99036]: pgmap v865: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:15 smithi173 ceph-mon[74165]: pgmap v865: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:18 smithi117 ceph-mon[99036]: pgmap v866: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:18 smithi173 ceph-mon[74165]: pgmap v866: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:20.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:20 smithi117 ceph-mon[99036]: pgmap v867: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:20 smithi173 ceph-mon[74165]: pgmap v867: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:22.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:22 smithi117 ceph-mon[99036]: pgmap v868: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:22.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:22 smithi173 ceph-mon[74165]: pgmap v868: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:24 smithi173 ceph-mon[74165]: pgmap v869: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:24 smithi117 ceph-mon[99036]: pgmap v869: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:26.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:26 smithi173 ceph-mon[74165]: pgmap v870: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:26.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:26 smithi117 ceph-mon[99036]: pgmap v870: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:28 smithi173 ceph-mon[74165]: pgmap v871: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:41:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:28 smithi117 ceph-mon[99036]: pgmap v871: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:41:30.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:30 smithi117 ceph-mon[99036]: pgmap v872: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:30 smithi173 ceph-mon[74165]: pgmap v872: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:32.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.267+0000 7fcfdf200700 1 -- 172.21.15.117:0/3227944144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 msgr2=0x7fcfd8111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:32.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.267+0000 7fcfdf200700 1 --2- 172.21.15.117:0/3227944144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 0x7fcfd8111360 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009910 tx=0x7fcfc8009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:32.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.267+0000 7fcfdf200700 1 -- 172.21.15.117:0/3227944144 shutdown_connections 2024-04-20T11:41:32.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.267+0000 7fcfdf200700 1 --2- 172.21.15.117:0/3227944144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 0x7fcfd8111360 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.267+0000 7fcfdf200700 1 --2- 172.21.15.117:0/3227944144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfd8075410 0x7fcfd80757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.267+0000 7fcfdf200700 1 -- 172.21.15.117:0/3227944144 >> 172.21.15.117:0/3227944144 conn(0x7fcfd80fe710 msgr2=0x7fcfd8100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:32.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.267+0000 7fcfdf200700 1 -- 172.21.15.117:0/3227944144 shutdown_connections 2024-04-20T11:41:32.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.267+0000 7fcfdf200700 1 -- 172.21.15.117:0/3227944144 wait complete. 2024-04-20T11:41:32.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.268+0000 7fcfdf200700 1 Processor -- start 2024-04-20T11:41:32.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.268+0000 7fcfdf200700 1 -- start start 2024-04-20T11:41:32.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.268+0000 7fcfdf200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfd8075410 0x7fcfd819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:32.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.268+0000 7fcfdf200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 0x7fcfd819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:32.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.268+0000 7fcfdf200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd819dab0 con 0x7fcfd8075d30 2024-04-20T11:41:32.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.268+0000 7fcfdf200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81a1780 con 0x7fcfd8075410 2024-04-20T11:41:32.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.269+0000 7fcfdd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 0x7fcfd819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:32.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.269+0000 7fcfdd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 0x7fcfd819d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43792/0 (socket says 172.21.15.117:43792) 2024-04-20T11:41:32.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.269+0000 7fcfdd400700 1 -- 172.21.15.117:0/1477199812 learned_addr learned my addr 172.21.15.117:0/1477199812 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:32.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.269+0000 7fcfdde00700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfd8075410 0x7fcfd819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:32.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.269+0000 7fcfdd400700 1 -- 172.21.15.117:0/1477199812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfd8075410 msgr2=0x7fcfd819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:32.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.269+0000 7fcfdd400700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfd8075410 0x7fcfd819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.269+0000 7fcfdd400700 1 -- 172.21.15.117:0/1477199812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80095f0 con 0x7fcfd8075d30 2024-04-20T11:41:32.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.269+0000 7fcfdd400700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 0x7fcfd819d3d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80092e0 tx=0x7fcfc80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:32.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.270+0000 7fcfd2a00700 1 -- 172.21.15.117:0/1477199812 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc801d070 con 0x7fcfd8075d30 2024-04-20T11:41:32.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.270+0000 7fcfd2a00700 1 -- 172.21.15.117:0/1477199812 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfc800bed0 con 0x7fcfd8075d30 2024-04-20T11:41:32.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.270+0000 7fcfd2a00700 1 -- 172.21.15.117:0/1477199812 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc8021410 con 0x7fcfd8075d30 2024-04-20T11:41:32.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.270+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd81a1a60 con 0x7fcfd8075d30 2024-04-20T11:41:32.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.270+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd81a1fb0 con 0x7fcfd8075d30 2024-04-20T11:41:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.271+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd810ead0 con 0x7fcfd8075d30 2024-04-20T11:41:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.272+0000 7fcfd2a00700 1 -- 172.21.15.117:0/1477199812 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcfc800f460 con 0x7fcfd8075d30 2024-04-20T11:41:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.273+0000 7fcfd2a00700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcfc006ecd0 0x7fcfc0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.273+0000 7fcfd2a00700 1 -- 172.21.15.117:0/1477199812 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcfc8090090 con 0x7fcfd8075d30 2024-04-20T11:41:32.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.273+0000 7fcfdde00700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcfc006ecd0 0x7fcfc0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:32.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.274+0000 7fcfdde00700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcfc006ecd0 0x7fcfc0071190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc005950 tx=0x7fcfcc005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:32.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.279+0000 7fcfd2a00700 1 -- 172.21.15.117:0/1477199812 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcfc8058e50 con 0x7fcfd8075d30 2024-04-20T11:41:32.440 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:32 smithi173 ceph-mon[74165]: pgmap v873: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:32.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.522+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfd8114160 con 0x7fcfc006ecd0 2024-04-20T11:41:32.522 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:32 smithi117 ceph-mon[99036]: pgmap v873: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:32.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.524+0000 7fcfd2a00700 1 -- 172.21.15.117:0/1477199812 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcfd8114160 con 0x7fcfc006ecd0 2024-04-20T11:41:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.526+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcfc006ecd0 msgr2=0x7fcfc0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.526+0000 7fcfdf200700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcfc006ecd0 0x7fcfc0071190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc005950 tx=0x7fcfcc005cf0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.526+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 msgr2=0x7fcfd819d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.526+0000 7fcfdf200700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 0x7fcfd819d3d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80092e0 tx=0x7fcfc80056a0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.527+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 shutdown_connections 2024-04-20T11:41:32.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.527+0000 7fcfdf200700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcfc006ecd0 0x7fcfc0071190 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.527+0000 7fcfdf200700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfd8075410 0x7fcfd819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.527+0000 7fcfdf200700 1 --2- 172.21.15.117:0/1477199812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfd8075d30 0x7fcfd819d3d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.527+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 >> 172.21.15.117:0/1477199812 conn(0x7fcfd80fe710 msgr2=0x7fcfd80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:32.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.527+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 shutdown_connections 2024-04-20T11:41:32.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.527+0000 7fcfdf200700 1 -- 172.21.15.117:0/1477199812 wait complete. 2024-04-20T11:41:32.542 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:41:32.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.686+0000 7fbc25000700 1 -- 172.21.15.117:0/2814010423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201039c0 msgr2=0x7fbc20107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:32.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.686+0000 7fbc25000700 1 --2- 172.21.15.117:0/2814010423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201039c0 0x7fbc20107a10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009910 tx=0x7fbc0c009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:32.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.686+0000 7fbc25000700 1 -- 172.21.15.117:0/2814010423 shutdown_connections 2024-04-20T11:41:32.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.686+0000 7fbc25000700 1 --2- 172.21.15.117:0/2814010423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201039c0 0x7fbc20107a10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.686+0000 7fbc25000700 1 --2- 172.21.15.117:0/2814010423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc20103010 0x7fbc201033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.686+0000 7fbc25000700 1 -- 172.21.15.117:0/2814010423 >> 172.21.15.117:0/2814010423 conn(0x7fbc200fe8c0 msgr2=0x7fbc20100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:32.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.686+0000 7fbc25000700 1 -- 172.21.15.117:0/2814010423 shutdown_connections 2024-04-20T11:41:32.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.686+0000 7fbc25000700 1 -- 172.21.15.117:0/2814010423 wait complete. 2024-04-20T11:41:32.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.687+0000 7fbc25000700 1 Processor -- start 2024-04-20T11:41:32.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.687+0000 7fbc25000700 1 -- start start 2024-04-20T11:41:32.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc25000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc20103010 0x7fbc20198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:32.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc25000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc201039c0 0x7fbc20199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:32.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc25000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc20199a40 con 0x7fbc20103010 2024-04-20T11:41:32.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc25000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2019d710 con 0x7fbc201039c0 2024-04-20T11:41:32.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc1de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc20103010 0x7fbc20198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:32.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc1de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc20103010 0x7fbc20198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43810/0 (socket says 172.21.15.117:43810) 2024-04-20T11:41:32.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc1de00700 1 -- 172.21.15.117:0/41743222 learned_addr learned my addr 172.21.15.117:0/41743222 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc1de00700 1 -- 172.21.15.117:0/41743222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc201039c0 msgr2=0x7fbc20199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc1de00700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc201039c0 0x7fbc20199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.688+0000 7fbc1de00700 1 -- 172.21.15.117:0/41743222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0095f0 con 0x7fbc20103010 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.689+0000 7fbc1de00700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc20103010 0x7fbc20198e20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000b4b0 tx=0x7fbc1000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.689+0000 7fbc16a00700 1 -- 172.21.15.117:0/41743222 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc10011800 con 0x7fbc20103010 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.689+0000 7fbc16a00700 1 -- 172.21.15.117:0/41743222 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc10011e20 con 0x7fbc20103010 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.689+0000 7fbc16a00700 1 -- 172.21.15.117:0/41743222 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc1000f2e0 con 0x7fbc20103010 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.689+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2019d9f0 con 0x7fbc20103010 2024-04-20T11:41:32.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.689+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2019df40 con 0x7fbc20103010 2024-04-20T11:41:32.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.690+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2010b340 con 0x7fbc20103010 2024-04-20T11:41:32.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.691+0000 7fbc16a00700 1 -- 172.21.15.117:0/41743222 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbc10010380 con 0x7fbc20103010 2024-04-20T11:41:32.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.692+0000 7fbc16a00700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc0406ecc0 0x7fbc04071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:32.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.692+0000 7fbc16a00700 1 -- 172.21.15.117:0/41743222 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbc1008e960 con 0x7fbc20103010 2024-04-20T11:41:32.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.693+0000 7fbc1d400700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc0406ecc0 0x7fbc04071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:32.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.694+0000 7fbc1d400700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc0406ecc0 0x7fbc04071180 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c006010 tx=0x7fbc0c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:32.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.697+0000 7fbc16a00700 1 -- 172.21.15.117:0/41743222 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbc10057870 con 0x7fbc20103010 2024-04-20T11:41:32.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.940+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc20110030 con 0x7fbc0406ecc0 2024-04-20T11:41:32.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.941+0000 7fbc16a00700 1 -- 172.21.15.117:0/41743222 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbc20110030 con 0x7fbc0406ecc0 2024-04-20T11:41:32.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc0406ecc0 msgr2=0x7fbc04071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:32.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc0406ecc0 0x7fbc04071180 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c006010 tx=0x7fbc0c00b540 comp rx=0 tx=0).stop 2024-04-20T11:41:32.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc20103010 msgr2=0x7fbc20198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:32.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc20103010 0x7fbc20198e20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000b4b0 tx=0x7fbc1000b870 comp rx=0 tx=0).stop 2024-04-20T11:41:32.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 shutdown_connections 2024-04-20T11:41:32.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc0406ecc0 0x7fbc04071180 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc20103010 0x7fbc20198e20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 --2- 172.21.15.117:0/41743222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc201039c0 0x7fbc20199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:32.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 >> 172.21.15.117:0/41743222 conn(0x7fbc200fe8c0 msgr2=0x7fbc200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:32.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.944+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 shutdown_connections 2024-04-20T11:41:32.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:32.945+0000 7fbc25000700 1 -- 172.21.15.117:0/41743222 wait complete. 2024-04-20T11:41:33.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.101+0000 7fb239400700 1 -- 172.21.15.117:0/311473708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073310 msgr2=0x7fb2340736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.101+0000 7fb239400700 1 --2- 172.21.15.117:0/311473708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073310 0x7fb2340736f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb224009910 tx=0x7fb224009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:33.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.101+0000 7fb239400700 1 -- 172.21.15.117:0/311473708 shutdown_connections 2024-04-20T11:41:33.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.101+0000 7fb239400700 1 --2- 172.21.15.117:0/311473708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb234073c30 0x7fb23410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.101+0000 7fb239400700 1 --2- 172.21.15.117:0/311473708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073310 0x7fb2340736f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.102+0000 7fb239400700 1 -- 172.21.15.117:0/311473708 >> 172.21.15.117:0/311473708 conn(0x7fb2340fc6c0 msgr2=0x7fb2340feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:33.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.102+0000 7fb239400700 1 -- 172.21.15.117:0/311473708 shutdown_connections 2024-04-20T11:41:33.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.102+0000 7fb239400700 1 -- 172.21.15.117:0/311473708 wait complete. 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb239400700 1 Processor -- start 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb239400700 1 -- start start 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb239400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb234073310 0x7fb234198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb239400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073c30 0x7fb234199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb239400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb234199a20 con 0x7fb234073c30 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb239400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23419d6f0 con 0x7fb234073310 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb231e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073c30 0x7fb234199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb231e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073c30 0x7fb234199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43836/0 (socket says 172.21.15.117:43836) 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.103+0000 7fb231e00700 1 -- 172.21.15.117:0/1954526216 learned_addr learned my addr 172.21.15.117:0/1954526216 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:33.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.104+0000 7fb232800700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb234073310 0x7fb234198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.104+0000 7fb231e00700 1 -- 172.21.15.117:0/1954526216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb234073310 msgr2=0x7fb234198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.104+0000 7fb231e00700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb234073310 0x7fb234198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.104+0000 7fb231e00700 1 -- 172.21.15.117:0/1954526216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240095f0 con 0x7fb234073c30 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.104+0000 7fb232800700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb234073310 0x7fb234198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.104+0000 7fb231e00700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073c30 0x7fb234199340 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb22000b4b0 tx=0x7fb22000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.104+0000 7fb22b400700 1 -- 172.21.15.117:0/1954526216 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb220011800 con 0x7fb234073c30 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.105+0000 7fb22b400700 1 -- 172.21.15.117:0/1954526216 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb220011e20 con 0x7fb234073c30 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.105+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23419d9d0 con 0x7fb234073c30 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.105+0000 7fb22b400700 1 -- 172.21.15.117:0/1954526216 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb22000f2e0 con 0x7fb234073c30 2024-04-20T11:41:33.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.105+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23419df20 con 0x7fb234073c30 2024-04-20T11:41:33.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.106+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2340749c0 con 0x7fb234073c30 2024-04-20T11:41:33.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.107+0000 7fb22b400700 1 -- 172.21.15.117:0/1954526216 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb220010380 con 0x7fb234073c30 2024-04-20T11:41:33.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.108+0000 7fb22b400700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb21806ecc0 0x7fb218071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.108+0000 7fb22b400700 1 -- 172.21.15.117:0/1954526216 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb22008ea70 con 0x7fb234073c30 2024-04-20T11:41:33.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.108+0000 7fb232800700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb21806ecc0 0x7fb218071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.109+0000 7fb232800700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb21806ecc0 0x7fb218071180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb224006010 tx=0x7fb22400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:33.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.113+0000 7fb22b400700 1 -- 172.21.15.117:0/1954526216 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb2200577b0 con 0x7fb234073c30 2024-04-20T11:41:33.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.334+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2341077c0 con 0x7fb21806ecc0 2024-04-20T11:41:33.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.365+0000 7fb22b400700 1 -- 172.21.15.117:0/1954526216 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb2341077c0 con 0x7fb21806ecc0 2024-04-20T11:41:33.365 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:41:33.365 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (30m) 5m ago 44m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:41:33.365 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (44m) 5m ago 44m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:41:33.365 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (43m) 6m ago 43m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (26m) 5m ago 44m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (26m) 6m ago 42m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (29m) 5m ago 43m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (21m) 5m ago 36m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (22m) 5m ago 36m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (6m) 6m ago 36m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (21m) 6m ago 36m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (33m) 5m ago 45m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (31m) 6m ago 42m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (28m) 5m ago 46m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (27m) 6m ago 42m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (31m) 5m ago 44m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (31m) 6m ago 42m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:41:33.366 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (25m) 5m ago 41m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (25m) 5m ago 41m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (24m) 5m ago 40m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (24m) 6m ago 39m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (23m) 6m ago 39m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (22m) 6m ago 38m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (30m) 5m ago 43m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb21806ecc0 msgr2=0x7fb218071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb21806ecc0 0x7fb218071180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb224006010 tx=0x7fb22400b540 comp rx=0 tx=0).stop 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073c30 msgr2=0x7fb234199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073c30 0x7fb234199340 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb22000b4b0 tx=0x7fb22000b870 comp rx=0 tx=0).stop 2024-04-20T11:41:33.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 shutdown_connections 2024-04-20T11:41:33.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb21806ecc0 0x7fb218071180 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb234073310 0x7fb234198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 --2- 172.21.15.117:0/1954526216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb234073c30 0x7fb234199340 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 >> 172.21.15.117:0/1954526216 conn(0x7fb2340fc6c0 msgr2=0x7fb234105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:33.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 shutdown_connections 2024-04-20T11:41:33.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.368+0000 7fb239400700 1 -- 172.21.15.117:0/1954526216 wait complete. 2024-04-20T11:41:33.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.518+0000 7ff399400700 1 -- 172.21.15.117:0/3893981054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff394103020 msgr2=0x7ff394103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.518+0000 7ff399400700 1 --2- 172.21.15.117:0/3893981054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff394103020 0x7ff394103400 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff384009910 tx=0x7ff384009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:33.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.518+0000 7ff399400700 1 -- 172.21.15.117:0/3893981054 shutdown_connections 2024-04-20T11:41:33.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.518+0000 7ff399400700 1 --2- 172.21.15.117:0/3893981054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3941039d0 0x7ff394107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.518+0000 7ff399400700 1 --2- 172.21.15.117:0/3893981054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff394103020 0x7ff394103400 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.518+0000 7ff399400700 1 -- 172.21.15.117:0/3893981054 >> 172.21.15.117:0/3893981054 conn(0x7ff3940fe8b0 msgr2=0x7ff394100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:33.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.518+0000 7ff399400700 1 -- 172.21.15.117:0/3893981054 shutdown_connections 2024-04-20T11:41:33.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.518+0000 7ff399400700 1 -- 172.21.15.117:0/3893981054 wait complete. 2024-04-20T11:41:33.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.519+0000 7ff399400700 1 Processor -- start 2024-04-20T11:41:33.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.519+0000 7ff399400700 1 -- start start 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.519+0000 7ff399400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff394103020 0x7ff39419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.519+0000 7ff399400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3941039d0 0x7ff39419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.519+0000 7ff399400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39419f5c0 con 0x7ff3941039d0 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.519+0000 7ff399400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff394198bf0 con 0x7ff394103020 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.520+0000 7ff391e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3941039d0 0x7ff39419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.520+0000 7ff391e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3941039d0 0x7ff39419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43860/0 (socket says 172.21.15.117:43860) 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.520+0000 7ff391e00700 1 -- 172.21.15.117:0/4046251126 learned_addr learned my addr 172.21.15.117:0/4046251126 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.520+0000 7ff392800700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff394103020 0x7ff39419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.520+0000 7ff391e00700 1 -- 172.21.15.117:0/4046251126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff394103020 msgr2=0x7ff39419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.520+0000 7ff391e00700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff394103020 0x7ff39419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.520+0000 7ff391e00700 1 -- 172.21.15.117:0/4046251126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840095f0 con 0x7ff3941039d0 2024-04-20T11:41:33.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.520+0000 7ff392800700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff394103020 0x7ff39419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:41:33.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.521+0000 7ff391e00700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3941039d0 0x7ff39419efa0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c8f0 tx=0x7ff38000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:33.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.521+0000 7ff38b400700 1 -- 172.21.15.117:0/4046251126 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff380007a50 con 0x7ff3941039d0 2024-04-20T11:41:33.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.521+0000 7ff38b400700 1 -- 172.21.15.117:0/4046251126 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff380007bb0 con 0x7ff3941039d0 2024-04-20T11:41:33.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.521+0000 7ff38b400700 1 -- 172.21.15.117:0/4046251126 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff380018630 con 0x7ff3941039d0 2024-04-20T11:41:33.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.521+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff394198ed0 con 0x7ff3941039d0 2024-04-20T11:41:33.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.521+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff394199420 con 0x7ff3941039d0 2024-04-20T11:41:33.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.522+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39410b3c0 con 0x7ff3941039d0 2024-04-20T11:41:33.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.523+0000 7ff38b400700 1 -- 172.21.15.117:0/4046251126 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff380007d20 con 0x7ff3941039d0 2024-04-20T11:41:33.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.524+0000 7ff38b400700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff37806ecd0 0x7ff378071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.524+0000 7ff38b400700 1 -- 172.21.15.117:0/4046251126 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff38008edd0 con 0x7ff3941039d0 2024-04-20T11:41:33.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.525+0000 7ff392800700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff37806ecd0 0x7ff378071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.525+0000 7ff392800700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff37806ecd0 0x7ff378071190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff384009fd0 tx=0x7ff38400b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:33.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.529+0000 7ff38b400700 1 -- 172.21.15.117:0/4046251126 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff38005b6c0 con 0x7ff3941039d0 2024-04-20T11:41:33.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.824+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff394199ff0 con 0x7ff3941039d0 2024-04-20T11:41:33.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.825+0000 7ff38b400700 1 -- 172.21.15.117:0/4046251126 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff38005b250 con 0x7ff3941039d0 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:41:33.825 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:41:33.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:41:33.826 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:41:33.826 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:41:33.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:41:33.826 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:41:33.826 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:41:33.826 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:41:33.826 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff37806ecd0 msgr2=0x7ff378071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff37806ecd0 0x7ff378071190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff384009fd0 tx=0x7ff38400b560 comp rx=0 tx=0).stop 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3941039d0 msgr2=0x7ff39419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3941039d0 0x7ff39419efa0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c8f0 tx=0x7ff38000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 shutdown_connections 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff37806ecd0 0x7ff378071190 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff394103020 0x7ff39419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 --2- 172.21.15.117:0/4046251126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3941039d0 0x7ff39419efa0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 >> 172.21.15.117:0/4046251126 conn(0x7ff3940fe8b0 msgr2=0x7ff3940fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 shutdown_connections 2024-04-20T11:41:33.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.828+0000 7ff399400700 1 -- 172.21.15.117:0/4046251126 wait complete. 2024-04-20T11:41:33.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.983+0000 7f01ba800700 1 -- 172.21.15.117:0/1352952617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 msgr2=0x7f01b410ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.983+0000 7f01ba800700 1 --2- 172.21.15.117:0/1352952617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 0x7f01b410ceb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009910 tx=0x7f01a0009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:33.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.983+0000 7f01ba800700 1 -- 172.21.15.117:0/1352952617 shutdown_connections 2024-04-20T11:41:33.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.983+0000 7f01ba800700 1 --2- 172.21.15.117:0/1352952617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 0x7f01b410ceb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.983+0000 7f01ba800700 1 --2- 172.21.15.117:0/1352952617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4073310 0x7f01b40736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.983+0000 7f01ba800700 1 -- 172.21.15.117:0/1352952617 >> 172.21.15.117:0/1352952617 conn(0x7f01b40fc6c0 msgr2=0x7f01b40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:33.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.983+0000 7f01ba800700 1 -- 172.21.15.117:0/1352952617 shutdown_connections 2024-04-20T11:41:33.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.984+0000 7f01ba800700 1 -- 172.21.15.117:0/1352952617 wait complete. 2024-04-20T11:41:33.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.984+0000 7f01ba800700 1 Processor -- start 2024-04-20T11:41:33.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.985+0000 7f01ba800700 1 -- start start 2024-04-20T11:41:33.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.985+0000 7f01ba800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4073310 0x7f01b4198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.985+0000 7f01ba800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 0x7f01b41992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.985+0000 7f01ba800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b41999c0 con 0x7f01b4073c30 2024-04-20T11:41:33.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.985+0000 7f01ba800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b419d690 con 0x7f01b4073310 2024-04-20T11:41:33.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.985+0000 7f01b2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 0x7f01b41992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.985+0000 7f01b2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 0x7f01b41992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47386/0 (socket says 172.21.15.117:47386) 2024-04-20T11:41:33.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.985+0000 7f01b2a00700 1 -- 172.21.15.117:0/3230725689 learned_addr learned my addr 172.21.15.117:0/3230725689 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:33.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.986+0000 7f01b3400700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4073310 0x7f01b4198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.986+0000 7f01b2a00700 1 -- 172.21.15.117:0/3230725689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4073310 msgr2=0x7f01b4198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:33.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.986+0000 7f01b2a00700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4073310 0x7f01b4198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:33.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.986+0000 7f01b2a00700 1 -- 172.21.15.117:0/3230725689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a00095f0 con 0x7f01b4073c30 2024-04-20T11:41:33.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.986+0000 7f01b3400700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4073310 0x7f01b4198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:41:33.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.986+0000 7f01b2a00700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 0x7f01b41992e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f01a00092e0 tx=0x7f01a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:33.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.986+0000 7f01abe00700 1 -- 172.21.15.117:0/3230725689 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a001d070 con 0x7f01b4073c30 2024-04-20T11:41:33.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.986+0000 7f01abe00700 1 -- 172.21.15.117:0/3230725689 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01a000f460 con 0x7f01b4073c30 2024-04-20T11:41:33.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.987+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b419d910 con 0x7f01b4073c30 2024-04-20T11:41:33.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.987+0000 7f01abe00700 1 -- 172.21.15.117:0/3230725689 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a0017630 con 0x7f01b4073c30 2024-04-20T11:41:33.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.987+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b419de00 con 0x7f01b4073c30 2024-04-20T11:41:33.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.988+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b40749c0 con 0x7f01b4073c30 2024-04-20T11:41:33.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.989+0000 7f01abe00700 1 -- 172.21.15.117:0/3230725689 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f01a000f5d0 con 0x7f01b4073c30 2024-04-20T11:41:33.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.990+0000 7f01abe00700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f019806ecc0 0x7f0198071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:33.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.990+0000 7f01abe00700 1 -- 172.21.15.117:0/3230725689 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f01a0090340 con 0x7f01b4073c30 2024-04-20T11:41:33.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.990+0000 7f01b3400700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f019806ecc0 0x7f0198071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:33.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.991+0000 7f01b3400700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f019806ecc0 0x7f0198071180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01a4005fd0 tx=0x7f01a4005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:33.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:33.995+0000 7f01abe00700 1 -- 172.21.15.117:0/3230725689 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f01a0059100 con 0x7f01b4073c30 2024-04-20T11:41:34.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.277+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f01b4062510 con 0x7f01b4073c30 2024-04-20T11:41:34.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.278+0000 7f01abe00700 1 -- 172.21.15.117:0/3230725689 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f01a005c7c0 con 0x7f01b4073c30 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:41:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:41:34.280 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:41:34.281 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:41:34.282 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:41:34.282 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f019806ecc0 msgr2=0x7f0198071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f019806ecc0 0x7f0198071180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01a4005fd0 tx=0x7f01a4005dc0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 msgr2=0x7f01b41992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 0x7f01b41992e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f01a00092e0 tx=0x7f01a000be20 comp rx=0 tx=0).stop 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 shutdown_connections 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f019806ecc0 0x7f0198071180 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4073310 0x7f01b4198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 --2- 172.21.15.117:0/3230725689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b4073c30 0x7f01b41992e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.281+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 >> 172.21.15.117:0/3230725689 conn(0x7f01b40fc6c0 msgr2=0x7f01b4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.282+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 shutdown_connections 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.282+0000 7f01ba800700 1 -- 172.21.15.117:0/3230725689 wait complete. 2024-04-20T11:41:34.283 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:41:34.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.437+0000 7f7828c00700 1 -- 172.21.15.117:0/1875501718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7824100e10 msgr2=0x7f78241011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:34.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.437+0000 7f7828c00700 1 --2- 172.21.15.117:0/1875501718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7824100e10 0x7f78241011f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7814009910 tx=0x7f7814009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:34.438 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:34 smithi117 ceph-mon[99036]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:34.438 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:34 smithi117 ceph-mon[99036]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:34.438 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:34 smithi117 ceph-mon[99036]: pgmap v874: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:34.438 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:34 smithi117 ceph-mon[99036]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:34.438 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:34 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4046251126' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:41:34.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.438+0000 7f7828c00700 1 -- 172.21.15.117:0/1875501718 shutdown_connections 2024-04-20T11:41:34.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.438+0000 7f7828c00700 1 --2- 172.21.15.117:0/1875501718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78241017c0 0x7f7824105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.438+0000 7f7828c00700 1 --2- 172.21.15.117:0/1875501718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7824100e10 0x7f78241011f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.438+0000 7f7828c00700 1 -- 172.21.15.117:0/1875501718 >> 172.21.15.117:0/1875501718 conn(0x7f78240fc6c0 msgr2=0x7f78240feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:34.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.438+0000 7f7828c00700 1 -- 172.21.15.117:0/1875501718 shutdown_connections 2024-04-20T11:41:34.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.438+0000 7f7828c00700 1 -- 172.21.15.117:0/1875501718 wait complete. 2024-04-20T11:41:34.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.440+0000 7f7828c00700 1 Processor -- start 2024-04-20T11:41:34.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.440+0000 7f7828c00700 1 -- start start 2024-04-20T11:41:34.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.440+0000 7f7828c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824100e10 0x7f782419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:34.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.440+0000 7f7828c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78241017c0 0x7f782419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.440+0000 7f7828c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782419f5b0 con 0x7f78241017c0 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.440+0000 7f7828c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7824198b30 con 0x7f7824100e10 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.441+0000 7f7821400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78241017c0 0x7f782419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.441+0000 7f7821400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78241017c0 0x7f782419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47402/0 (socket says 172.21.15.117:47402) 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.441+0000 7f7821400700 1 -- 172.21.15.117:0/849367816 learned_addr learned my addr 172.21.15.117:0/849367816 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.441+0000 7f7821e00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824100e10 0x7f782419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.441+0000 7f7821400700 1 -- 172.21.15.117:0/849367816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824100e10 msgr2=0x7f782419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.441+0000 7f7821400700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824100e10 0x7f782419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.441+0000 7f7821400700 1 -- 172.21.15.117:0/849367816 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78140095f0 con 0x7f78241017c0 2024-04-20T11:41:34.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.442+0000 7f7821400700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78241017c0 0x7f782419eed0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f781000d900 tx=0x7f781000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:34.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.442+0000 7f781aa00700 1 -- 172.21.15.117:0/849367816 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7810009900 con 0x7f78241017c0 2024-04-20T11:41:34.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.442+0000 7f781aa00700 1 -- 172.21.15.117:0/849367816 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7810010460 con 0x7f78241017c0 2024-04-20T11:41:34.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.442+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7824198e10 con 0x7f78241017c0 2024-04-20T11:41:34.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.442+0000 7f781aa00700 1 -- 172.21.15.117:0/849367816 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f781000f5f0 con 0x7f78241017c0 2024-04-20T11:41:34.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.442+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7824199360 con 0x7f78241017c0 2024-04-20T11:41:34.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.443+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7824074920 con 0x7f78241017c0 2024-04-20T11:41:34.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.445+0000 7f781aa00700 1 -- 172.21.15.117:0/849367816 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f78100105d0 con 0x7f78241017c0 2024-04-20T11:41:34.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.446+0000 7f781aa00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f780806ecc0 0x7f7808071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:34.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.446+0000 7f781aa00700 1 -- 172.21.15.117:0/849367816 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f781008ed20 con 0x7f78241017c0 2024-04-20T11:41:34.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.446+0000 7f7821e00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f780806ecc0 0x7f7808071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:34.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.447+0000 7f7821e00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f780806ecc0 0x7f7808071180 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7814005b40 tx=0x7f781401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:34.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.452+0000 7f781aa00700 1 -- 172.21.15.117:0/849367816 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f781005b590 con 0x7f78241017c0 2024-04-20T11:41:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:34 smithi173 ceph-mon[74165]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:34 smithi173 ceph-mon[74165]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:34 smithi173 ceph-mon[74165]: pgmap v874: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:34 smithi173 ceph-mon[74165]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:34 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4046251126' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:41:34.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.682+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f782419a040 con 0x7f780806ecc0 2024-04-20T11:41:34.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.684+0000 7f781aa00700 1 -- 172.21.15.117:0/849367816 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f782419a040 con 0x7f780806ecc0 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:41:34.684 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:41:34.685 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:41:34.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f780806ecc0 msgr2=0x7f7808071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f780806ecc0 0x7f7808071180 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7814005b40 tx=0x7f781401a040 comp rx=0 tx=0).stop 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78241017c0 msgr2=0x7f782419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78241017c0 0x7f782419eed0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f781000d900 tx=0x7f781000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 shutdown_connections 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f780806ecc0 0x7f7808071180 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824100e10 0x7f782419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 --2- 172.21.15.117:0/849367816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78241017c0 0x7f782419eed0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.686+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 >> 172.21.15.117:0/849367816 conn(0x7f78240fc6c0 msgr2=0x7f78240fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.687+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 shutdown_connections 2024-04-20T11:41:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.687+0000 7f7828c00700 1 -- 172.21.15.117:0/849367816 wait complete. 2024-04-20T11:41:34.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.842+0000 7fd11c000700 1 -- 172.21.15.117:0/3954844072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 msgr2=0x7fd11410cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:34.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.842+0000 7fd11c000700 1 --2- 172.21.15.117:0/3954844072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 0x7fd11410cf10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd100009910 tx=0x7fd100009c20 comp rx=0 tx=0).stop 2024-04-20T11:41:34.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.842+0000 7fd11c000700 1 -- 172.21.15.117:0/3954844072 shutdown_connections 2024-04-20T11:41:34.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.842+0000 7fd11c000700 1 --2- 172.21.15.117:0/3954844072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 0x7fd11410cf10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.842+0000 7fd11c000700 1 --2- 172.21.15.117:0/3954844072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1140732b0 0x7fd114073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.842+0000 7fd11c000700 1 -- 172.21.15.117:0/3954844072 >> 172.21.15.117:0/3954844072 conn(0x7fd1140fc6a0 msgr2=0x7fd1140feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:34.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.842+0000 7fd11c000700 1 -- 172.21.15.117:0/3954844072 shutdown_connections 2024-04-20T11:41:34.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.842+0000 7fd11c000700 1 -- 172.21.15.117:0/3954844072 wait complete. 2024-04-20T11:41:34.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.844+0000 7fd11c000700 1 Processor -- start 2024-04-20T11:41:34.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.844+0000 7fd11c000700 1 -- start start 2024-04-20T11:41:34.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.844+0000 7fd11c000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1140732b0 0x7fd114198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:34.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.844+0000 7fd11c000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 0x7fd114199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:34.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.844+0000 7fd11c000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1141998d0 con 0x7fd114073bd0 2024-04-20T11:41:34.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.844+0000 7fd11c000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11419c540 con 0x7fd1140732b0 2024-04-20T11:41:34.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.845+0000 7fd118c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 0x7fd114199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:34.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.845+0000 7fd118c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 0x7fd114199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47408/0 (socket says 172.21.15.117:47408) 2024-04-20T11:41:34.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.845+0000 7fd118c00700 1 -- 172.21.15.117:0/2025117120 learned_addr learned my addr 172.21.15.117:0/2025117120 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:41:34.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.845+0000 7fd119600700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1140732b0 0x7fd114198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:34.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.845+0000 7fd118c00700 1 -- 172.21.15.117:0/2025117120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1140732b0 msgr2=0x7fd114198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:34.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.845+0000 7fd118c00700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1140732b0 0x7fd114198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:34.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.845+0000 7fd118c00700 1 -- 172.21.15.117:0/2025117120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000095f0 con 0x7fd114073bd0 2024-04-20T11:41:34.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.846+0000 7fd118c00700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 0x7fd114199240 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd1000092e0 tx=0x7fd10000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:34.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.846+0000 7fd112000700 1 -- 172.21.15.117:0/2025117120 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd10001d070 con 0x7fd114073bd0 2024-04-20T11:41:34.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.846+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11419c7c0 con 0x7fd114073bd0 2024-04-20T11:41:34.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.846+0000 7fd112000700 1 -- 172.21.15.117:0/2025117120 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1000056e0 con 0x7fd114073bd0 2024-04-20T11:41:34.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.846+0000 7fd112000700 1 -- 172.21.15.117:0/2025117120 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd10000fe80 con 0x7fd114073bd0 2024-04-20T11:41:34.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.846+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11419ccb0 con 0x7fd114073bd0 2024-04-20T11:41:34.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.847+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd114074960 con 0x7fd114073bd0 2024-04-20T11:41:34.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.850+0000 7fd112000700 1 -- 172.21.15.117:0/2025117120 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd10000f6f0 con 0x7fd114073bd0 2024-04-20T11:41:34.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.851+0000 7fd112000700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0fc06ecd0 0x7fd0fc071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:41:34.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.851+0000 7fd112000700 1 -- 172.21.15.117:0/2025117120 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd100090230 con 0x7fd114073bd0 2024-04-20T11:41:34.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.851+0000 7fd119600700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0fc06ecd0 0x7fd0fc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:41:34.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.852+0000 7fd119600700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0fc06ecd0 0x7fd0fc071190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd108005950 tx=0x7fd1080058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:41:34.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:34.856+0000 7fd112000700 1 -- 172.21.15.117:0/2025117120 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd100058ff0 con 0x7fd114073bd0 2024-04-20T11:41:35.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.163+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd11404ebb0 con 0x7fd114073bd0 2024-04-20T11:41:35.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.163+0000 7fd112000700 1 -- 172.21.15.117:0/2025117120 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fd10005c6b0 con 0x7fd114073bd0 2024-04-20T11:41:35.163 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:41:35.164 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:41:35.165 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:41:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0fc06ecd0 msgr2=0x7fd0fc071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0fc06ecd0 0x7fd0fc071190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd108005950 tx=0x7fd1080058e0 comp rx=0 tx=0).stop 2024-04-20T11:41:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 msgr2=0x7fd114199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:41:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 0x7fd114199240 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd1000092e0 tx=0x7fd10000be20 comp rx=0 tx=0).stop 2024-04-20T11:41:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 shutdown_connections 2024-04-20T11:41:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd0fc06ecd0 0x7fd0fc071190 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1140732b0 0x7fd114198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 --2- 172.21.15.117:0/2025117120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd114073bd0 0x7fd114199240 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:41:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 >> 172.21.15.117:0/2025117120 conn(0x7fd1140fc6a0 msgr2=0x7fd1141053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:41:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 shutdown_connections 2024-04-20T11:41:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:41:35.166+0000 7fd11c000700 1 -- 172.21.15.117:0/2025117120 wait complete. 2024-04-20T11:41:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:35 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3230725689' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:41:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:35 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2025117120' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:41:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:35 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3230725689' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:41:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:35 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2025117120' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:41:36.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:36 smithi173 ceph-mon[74165]: from='client.35590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:36.563 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:36 smithi173 ceph-mon[74165]: pgmap v875: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:36 smithi117 ceph-mon[99036]: from='client.35590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:41:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:36 smithi117 ceph-mon[99036]: pgmap v875: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:38 smithi117 ceph-mon[99036]: pgmap v876: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:38 smithi173 ceph-mon[74165]: pgmap v876: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:40.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:40 smithi117 ceph-mon[99036]: pgmap v877: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:40.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:40 smithi173 ceph-mon[74165]: pgmap v877: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:42 smithi117 ceph-mon[99036]: pgmap v878: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:42 smithi173 ceph-mon[74165]: pgmap v878: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:41:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:41:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:44 smithi117 ceph-mon[99036]: pgmap v879: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:44 smithi173 ceph-mon[74165]: pgmap v879: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:46.487 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:46 smithi117 ceph-mon[99036]: pgmap v880: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:46 smithi173 ceph-mon[74165]: pgmap v880: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:48.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:48 smithi117 ceph-mon[99036]: pgmap v881: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:48 smithi173 ceph-mon[74165]: pgmap v881: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:50.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:50 smithi117 ceph-mon[99036]: pgmap v882: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:50 smithi173 ceph-mon[74165]: pgmap v882: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:52 smithi117 ceph-mon[99036]: pgmap v883: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:52 smithi173 ceph-mon[74165]: pgmap v883: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:54 smithi117 ceph-mon[99036]: pgmap v884: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:54 smithi173 ceph-mon[74165]: pgmap v884: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:56 smithi117 ceph-mon[99036]: pgmap v885: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:56 smithi173 ceph-mon[74165]: pgmap v885: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:58.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:58 smithi117 ceph-mon[99036]: pgmap v886: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:58.643 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:41:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:58 smithi173 ceph-mon[74165]: pgmap v886: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:41:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:42:00.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:41:59 smithi117 ceph-mon[99036]: pgmap v887: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:00.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:41:59 smithi173 ceph-mon[74165]: pgmap v887: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:02.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:02 smithi117 ceph-mon[99036]: pgmap v888: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:02.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:02 smithi173 ceph-mon[74165]: pgmap v888: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:04.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:04 smithi173 ceph-mon[74165]: pgmap v889: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:04 smithi117 ceph-mon[99036]: pgmap v889: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:05.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.336+0000 7fe490a00700 1 -- 172.21.15.117:0/1718279875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 msgr2=0x7fe48c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:05.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.336+0000 7fe490a00700 1 --2- 172.21.15.117:0/1718279875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 0x7fe48c107900 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009910 tx=0x7fe47c009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:05.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.337+0000 7fe490a00700 1 -- 172.21.15.117:0/1718279875 shutdown_connections 2024-04-20T11:42:05.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.337+0000 7fe490a00700 1 --2- 172.21.15.117:0/1718279875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe48c108e10 0x7fe48c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.337+0000 7fe490a00700 1 --2- 172.21.15.117:0/1718279875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 0x7fe48c107900 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.337+0000 7fe490a00700 1 -- 172.21.15.117:0/1718279875 >> 172.21.15.117:0/1718279875 conn(0x7fe48c0768f0 msgr2=0x7fe48c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:05.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.337+0000 7fe490a00700 1 -- 172.21.15.117:0/1718279875 shutdown_connections 2024-04-20T11:42:05.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.337+0000 7fe490a00700 1 -- 172.21.15.117:0/1718279875 wait complete. 2024-04-20T11:42:05.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.338+0000 7fe490a00700 1 Processor -- start 2024-04-20T11:42:05.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.338+0000 7fe490a00700 1 -- start start 2024-04-20T11:42:05.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.338+0000 7fe490a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 0x7fe48c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:05.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.338+0000 7fe490a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe48c108e10 0x7fe48c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:05.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.338+0000 7fe490a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c19ddd0 con 0x7fe48c107520 2024-04-20T11:42:05.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.338+0000 7fe490a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c1a1aa0 con 0x7fe48c108e10 2024-04-20T11:42:05.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe489e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 0x7fe48c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:05.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe489e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 0x7fe48c19d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43012/0 (socket says 172.21.15.117:43012) 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe489e00700 1 -- 172.21.15.117:0/31274231 learned_addr learned my addr 172.21.15.117:0/31274231 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe483e00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe48c108e10 0x7fe48c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe489e00700 1 -- 172.21.15.117:0/31274231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe48c108e10 msgr2=0x7fe48c19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe489e00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe48c108e10 0x7fe48c19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe489e00700 1 -- 172.21.15.117:0/31274231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c0095f0 con 0x7fe48c107520 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe483e00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe48c108e10 0x7fe48c19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.339+0000 7fe489e00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 0x7fe48c19d1b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe47c006010 tx=0x7fe47c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.340+0000 7fe482a00700 1 -- 172.21.15.117:0/31274231 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe47c01d070 con 0x7fe48c107520 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.340+0000 7fe482a00700 1 -- 172.21.15.117:0/31274231 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe47c00f460 con 0x7fe48c107520 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.340+0000 7fe482a00700 1 -- 172.21.15.117:0/31274231 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe47c017630 con 0x7fe48c107520 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.340+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48c1a1d20 con 0x7fe48c107520 2024-04-20T11:42:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.340+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48c1a2290 con 0x7fe48c107520 2024-04-20T11:42:05.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.341+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48c10f890 con 0x7fe48c107520 2024-04-20T11:42:05.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.342+0000 7fe482a00700 1 -- 172.21.15.117:0/31274231 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe47c00f5d0 con 0x7fe48c107520 2024-04-20T11:42:05.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.343+0000 7fe482a00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe47006ecd0 0x7fe470071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:05.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.343+0000 7fe482a00700 1 -- 172.21.15.117:0/31274231 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe47c090310 con 0x7fe48c107520 2024-04-20T11:42:05.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.343+0000 7fe483e00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe47006ecd0 0x7fe470071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:05.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.344+0000 7fe483e00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe47006ecd0 0x7fe470071190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe474005fd0 tx=0x7fe474005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:05.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.348+0000 7fe482a00700 1 -- 172.21.15.117:0/31274231 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe47c059190 con 0x7fe48c107520 2024-04-20T11:42:05.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.597+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe48c1144c0 con 0x7fe47006ecd0 2024-04-20T11:42:05.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.599+0000 7fe482a00700 1 -- 172.21.15.117:0/31274231 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe48c1144c0 con 0x7fe47006ecd0 2024-04-20T11:42:05.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe47006ecd0 msgr2=0x7fe470071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:05.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe47006ecd0 0x7fe470071190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe474005fd0 tx=0x7fe474005dc0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 msgr2=0x7fe48c19d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:05.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 0x7fe48c19d1b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe47c006010 tx=0x7fe47c00be20 comp rx=0 tx=0).stop 2024-04-20T11:42:05.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 shutdown_connections 2024-04-20T11:42:05.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe47006ecd0 0x7fe470071190 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe48c107520 0x7fe48c19d1b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 --2- 172.21.15.117:0/31274231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe48c108e10 0x7fe48c19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.602+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 >> 172.21.15.117:0/31274231 conn(0x7fe48c0768f0 msgr2=0x7fe48c077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:05.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.603+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 shutdown_connections 2024-04-20T11:42:05.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.603+0000 7fe490a00700 1 -- 172.21.15.117:0/31274231 wait complete. 2024-04-20T11:42:05.618 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:42:05.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.761+0000 7f28bae00700 1 -- 172.21.15.117:0/3382300582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b4100e10 msgr2=0x7f28b41011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:05.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.761+0000 7f28bae00700 1 --2- 172.21.15.117:0/3382300582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b4100e10 0x7f28b41011f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009910 tx=0x7f28a4009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:05.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.761+0000 7f28bae00700 1 -- 172.21.15.117:0/3382300582 shutdown_connections 2024-04-20T11:42:05.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.761+0000 7f28bae00700 1 --2- 172.21.15.117:0/3382300582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b41017c0 0x7f28b4105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.761+0000 7f28bae00700 1 --2- 172.21.15.117:0/3382300582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b4100e10 0x7f28b41011f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.761+0000 7f28bae00700 1 -- 172.21.15.117:0/3382300582 >> 172.21.15.117:0/3382300582 conn(0x7f28b40fc6a0 msgr2=0x7f28b40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:05.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.761+0000 7f28bae00700 1 -- 172.21.15.117:0/3382300582 shutdown_connections 2024-04-20T11:42:05.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.762+0000 7f28bae00700 1 -- 172.21.15.117:0/3382300582 wait complete. 2024-04-20T11:42:05.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.763+0000 7f28bae00700 1 Processor -- start 2024-04-20T11:42:05.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.763+0000 7f28bae00700 1 -- start start 2024-04-20T11:42:05.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.763+0000 7f28bae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4100e10 0x7f28b419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:05.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.763+0000 7f28bae00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b41017c0 0x7f28b419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:05.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.763+0000 7f28bae00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b4198b10 con 0x7f28b41017c0 2024-04-20T11:42:05.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.763+0000 7f28bae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b4198c80 con 0x7f28b4100e10 2024-04-20T11:42:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.763+0000 7f28b3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b41017c0 0x7f28b419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b41017c0 0x7f28b419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43026/0 (socket says 172.21.15.117:43026) 2024-04-20T11:42:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b3400700 1 -- 172.21.15.117:0/1667532401 learned_addr learned my addr 172.21.15.117:0/1667532401 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b3e00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4100e10 0x7f28b419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b3400700 1 -- 172.21.15.117:0/1667532401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4100e10 msgr2=0x7f28b419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b3400700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4100e10 0x7f28b419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b3400700 1 -- 172.21.15.117:0/1667532401 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40095f0 con 0x7f28b41017c0 2024-04-20T11:42:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b3400700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b41017c0 0x7f28b419efa0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d900 tx=0x7f28a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b0c00700 1 -- 172.21.15.117:0/1667532401 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a0009900 con 0x7f28b41017c0 2024-04-20T11:42:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28b0c00700 1 -- 172.21.15.117:0/1667532401 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28a0010460 con 0x7f28b41017c0 2024-04-20T11:42:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.764+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b4198f60 con 0x7f28b41017c0 2024-04-20T11:42:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.765+0000 7f28b0c00700 1 -- 172.21.15.117:0/1667532401 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a000f5f0 con 0x7f28b41017c0 2024-04-20T11:42:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.765+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b41994b0 con 0x7f28b41017c0 2024-04-20T11:42:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.766+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b4104f90 con 0x7f28b41017c0 2024-04-20T11:42:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.767+0000 7f28b0c00700 1 -- 172.21.15.117:0/1667532401 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f28a00105d0 con 0x7f28b41017c0 2024-04-20T11:42:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.768+0000 7f28b0c00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f289806ecd0 0x7f2898071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.768+0000 7f28b0c00700 1 -- 172.21.15.117:0/1667532401 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f28a008ed30 con 0x7f28b41017c0 2024-04-20T11:42:05.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.768+0000 7f28b3e00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f289806ecd0 0x7f2898071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:05.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.769+0000 7f28b3e00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f289806ecd0 0x7f2898071190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f28a4006010 tx=0x7f28a400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:05.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:05.773+0000 7f28b0c00700 1 -- 172.21.15.117:0/1667532401 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f28a0057a70 con 0x7f28b41017c0 2024-04-20T11:42:06.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.020+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28b4199ff0 con 0x7f289806ecd0 2024-04-20T11:42:06.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.021+0000 7f28b0c00700 1 -- 172.21.15.117:0/1667532401 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f28b4199ff0 con 0x7f289806ecd0 2024-04-20T11:42:06.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f289806ecd0 msgr2=0x7f2898071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f289806ecd0 0x7f2898071190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f28a4006010 tx=0x7f28a400b540 comp rx=0 tx=0).stop 2024-04-20T11:42:06.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b41017c0 msgr2=0x7f28b419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b41017c0 0x7f28b419efa0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d900 tx=0x7f28a000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 shutdown_connections 2024-04-20T11:42:06.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f289806ecd0 0x7f2898071190 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4100e10 0x7f28b419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 --2- 172.21.15.117:0/1667532401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28b41017c0 0x7f28b419efa0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 >> 172.21.15.117:0/1667532401 conn(0x7f28b40fc6a0 msgr2=0x7f28b40fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:06.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 shutdown_connections 2024-04-20T11:42:06.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.024+0000 7f28bae00700 1 -- 172.21.15.117:0/1667532401 wait complete. 2024-04-20T11:42:06.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.170+0000 7ff6d9c00700 1 -- 172.21.15.117:0/3570413584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d4101b80 msgr2=0x7ff6d4101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.170+0000 7ff6d9c00700 1 --2- 172.21.15.117:0/3570413584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d4101b80 0x7ff6d4101f60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009910 tx=0x7ff6c4009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:06.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.170+0000 7ff6d9c00700 1 -- 172.21.15.117:0/3570413584 shutdown_connections 2024-04-20T11:42:06.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.170+0000 7ff6d9c00700 1 --2- 172.21.15.117:0/3570413584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6d41024a0 0x7ff6d410a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.170+0000 7ff6d9c00700 1 --2- 172.21.15.117:0/3570413584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d4101b80 0x7ff6d4101f60 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.170+0000 7ff6d9c00700 1 -- 172.21.15.117:0/3570413584 >> 172.21.15.117:0/3570413584 conn(0x7ff6d40fb3f0 msgr2=0x7ff6d40fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:06.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.170+0000 7ff6d9c00700 1 -- 172.21.15.117:0/3570413584 shutdown_connections 2024-04-20T11:42:06.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.171+0000 7ff6d9c00700 1 -- 172.21.15.117:0/3570413584 wait complete. 2024-04-20T11:42:06.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d9c00700 1 Processor -- start 2024-04-20T11:42:06.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d9c00700 1 -- start start 2024-04-20T11:42:06.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d9c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6d4101b80 0x7ff6d419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:06.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d9c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41024a0 0x7ff6d419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:06.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d9c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d419d230 con 0x7ff6d41024a0 2024-04-20T11:42:06.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d9c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41967a0 con 0x7ff6d4101b80 2024-04-20T11:42:06.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41024a0 0x7ff6d419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:06.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41024a0 0x7ff6d419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43040/0 (socket says 172.21.15.117:43040) 2024-04-20T11:42:06.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.172+0000 7ff6d3400700 1 -- 172.21.15.117:0/4239586697 learned_addr learned my addr 172.21.15.117:0/4239586697 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:06.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d3e00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6d4101b80 0x7ff6d419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:06.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d3400700 1 -- 172.21.15.117:0/4239586697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6d4101b80 msgr2=0x7ff6d419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d3400700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6d4101b80 0x7ff6d419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d3400700 1 -- 172.21.15.117:0/4239586697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c40095f0 con 0x7ff6d41024a0 2024-04-20T11:42:06.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d3400700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41024a0 0x7ff6d419cba0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000d6d0 tx=0x7ff6c000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:06.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d0c00700 1 -- 172.21.15.117:0/4239586697 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c000f860 con 0x7ff6d41024a0 2024-04-20T11:42:06.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d0c00700 1 -- 172.21.15.117:0/4239586697 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6c000fe80 con 0x7ff6d41024a0 2024-04-20T11:42:06.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d4196a80 con 0x7ff6d41024a0 2024-04-20T11:42:06.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.173+0000 7ff6d0c00700 1 -- 172.21.15.117:0/4239586697 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c0010ea0 con 0x7ff6d41024a0 2024-04-20T11:42:06.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.174+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d4196fd0 con 0x7ff6d41024a0 2024-04-20T11:42:06.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.175+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d41052a0 con 0x7ff6d41024a0 2024-04-20T11:42:06.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.176+0000 7ff6d0c00700 1 -- 172.21.15.117:0/4239586697 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff6c0010460 con 0x7ff6d41024a0 2024-04-20T11:42:06.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.177+0000 7ff6d0c00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff6b806ecc0 0x7ff6b8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:06.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.177+0000 7ff6d0c00700 1 -- 172.21.15.117:0/4239586697 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff6c008eab0 con 0x7ff6d41024a0 2024-04-20T11:42:06.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.177+0000 7ff6d3e00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff6b806ecc0 0x7ff6b8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:06.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.178+0000 7ff6d3e00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff6b806ecc0 0x7ff6b8071180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4000c00 tx=0x7ff6c400b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:06.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.181+0000 7ff6d0c00700 1 -- 172.21.15.117:0/4239586697 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff6c0090e60 con 0x7ff6d41024a0 2024-04-20T11:42:06.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:06 smithi117 ceph-mon[99036]: pgmap v890: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:06.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.403+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6d410dc00 con 0x7ff6b806ecc0 2024-04-20T11:42:06.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.431+0000 7ff6d0c00700 1 -- 172.21.15.117:0/4239586697 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff6d410dc00 con 0x7ff6b806ecc0 2024-04-20T11:42:06.431 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:42:06.431 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (30m) 6m ago 44m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:42:06.431 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (44m) 6m ago 44m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:42:06.431 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (43m) 6m ago 43m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:42:06.431 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (26m) 6m ago 44m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:42:06.431 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (26m) 6m ago 43m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:42:06.431 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (29m) 6m ago 44m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (22m) 6m ago 37m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (22m) 6m ago 37m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (6m) 6m ago 37m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (21m) 6m ago 37m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (33m) 6m ago 46m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (32m) 6m ago 43m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (28m) 6m ago 46m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (27m) 6m ago 43m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (31m) 6m ago 44m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (31m) 6m ago 43m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (26m) 6m ago 42m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (25m) 6m ago 41m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (25m) 6m ago 40m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (24m) 6m ago 40m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:42:06.432 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (23m) 6m ago 39m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:42:06.433 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (23m) 6m ago 39m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:42:06.433 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (31m) 6m ago 44m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:42:06.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff6b806ecc0 msgr2=0x7ff6b8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff6b806ecc0 0x7ff6b8071180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4000c00 tx=0x7ff6c400b560 comp rx=0 tx=0).stop 2024-04-20T11:42:06.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41024a0 msgr2=0x7ff6d419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41024a0 0x7ff6d419cba0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000d6d0 tx=0x7ff6c000da90 comp rx=0 tx=0).stop 2024-04-20T11:42:06.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 shutdown_connections 2024-04-20T11:42:06.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff6b806ecc0 0x7ff6b8071180 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6d4101b80 0x7ff6d419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 --2- 172.21.15.117:0/4239586697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d41024a0 0x7ff6d419cba0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 >> 172.21.15.117:0/4239586697 conn(0x7ff6d40fb3f0 msgr2=0x7ff6d4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:06.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 shutdown_connections 2024-04-20T11:42:06.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.435+0000 7ff6d9c00700 1 -- 172.21.15.117:0/4239586697 wait complete. 2024-04-20T11:42:06.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.583+0000 7faf56c00700 1 -- 172.21.15.117:0/2952212054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 msgr2=0x7faf50104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.583+0000 7faf56c00700 1 --2- 172.21.15.117:0/2952212054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 0x7faf50104e80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009910 tx=0x7faf3c009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:06.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.584+0000 7faf56c00700 1 -- 172.21.15.117:0/2952212054 shutdown_connections 2024-04-20T11:42:06.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.584+0000 7faf56c00700 1 --2- 172.21.15.117:0/2952212054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 0x7faf50104e80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.584+0000 7faf56c00700 1 --2- 172.21.15.117:0/2952212054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf50103110 0x7faf501034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.584+0000 7faf56c00700 1 -- 172.21.15.117:0/2952212054 >> 172.21.15.117:0/2952212054 conn(0x7faf500fe880 msgr2=0x7faf50100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:06.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.584+0000 7faf56c00700 1 -- 172.21.15.117:0/2952212054 shutdown_connections 2024-04-20T11:42:06.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.584+0000 7faf56c00700 1 -- 172.21.15.117:0/2952212054 wait complete. 2024-04-20T11:42:06.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.585+0000 7faf56c00700 1 Processor -- start 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.585+0000 7faf56c00700 1 -- start start 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.585+0000 7faf56c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf50103110 0x7faf5019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.585+0000 7faf56c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 0x7faf5019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.585+0000 7faf56c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5019f5b0 con 0x7faf50104a00 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.585+0000 7faf56c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50198b90 con 0x7faf50103110 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.585+0000 7faf4f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 0x7faf5019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.586+0000 7faf4f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 0x7faf5019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43060/0 (socket says 172.21.15.117:43060) 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.586+0000 7faf4f400700 1 -- 172.21.15.117:0/2112941099 learned_addr learned my addr 172.21.15.117:0/2112941099 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.586+0000 7faf4fe00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf50103110 0x7faf5019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.586+0000 7faf4f400700 1 -- 172.21.15.117:0/2112941099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf50103110 msgr2=0x7faf5019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.586+0000 7faf4f400700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf50103110 0x7faf5019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.586+0000 7faf4f400700 1 -- 172.21.15.117:0/2112941099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0095f0 con 0x7faf50104a00 2024-04-20T11:42:06.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.586+0000 7faf4f400700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 0x7faf5019ef90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00bd70 tx=0x7faf3c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:06.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.587+0000 7faf4cc00700 1 -- 172.21.15.117:0/2112941099 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3c01d070 con 0x7faf50104a00 2024-04-20T11:42:06.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.587+0000 7faf4cc00700 1 -- 172.21.15.117:0/2112941099 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf3c0056e0 con 0x7faf50104a00 2024-04-20T11:42:06.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.587+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf50198e10 con 0x7faf50104a00 2024-04-20T11:42:06.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.587+0000 7faf4cc00700 1 -- 172.21.15.117:0/2112941099 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3c017400 con 0x7faf50104a00 2024-04-20T11:42:06.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.587+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf50199380 con 0x7faf50104a00 2024-04-20T11:42:06.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.588+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5010b480 con 0x7faf50104a00 2024-04-20T11:42:06.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.589+0000 7faf4cc00700 1 -- 172.21.15.117:0/2112941099 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7faf3c003a60 con 0x7faf50104a00 2024-04-20T11:42:06.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.590+0000 7faf4cc00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faf3406ecc0 0x7faf34071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:06.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.590+0000 7faf4cc00700 1 -- 172.21.15.117:0/2112941099 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7faf3c090310 con 0x7faf50104a00 2024-04-20T11:42:06.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.590+0000 7faf4fe00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faf3406ecc0 0x7faf34071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:06.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.591+0000 7faf4fe00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faf3406ecc0 0x7faf34071180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf40005950 tx=0x7faf400058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:06.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.594+0000 7faf4cc00700 1 -- 172.21.15.117:0/2112941099 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7faf3c05cb80 con 0x7faf50104a00 2024-04-20T11:42:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:06 smithi173 ceph-mon[74165]: pgmap v890: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:06.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.885+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faf50104510 con 0x7faf50104a00 2024-04-20T11:42:06.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.886+0000 7faf4cc00700 1 -- 172.21.15.117:0/2112941099 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7faf3c05c710 con 0x7faf50104a00 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:42:06.887 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:42:06.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:42:06.888 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:42:06.888 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:42:06.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:42:06.888 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:42:06.888 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:42:06.888 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:42:06.888 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:42:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faf3406ecc0 msgr2=0x7faf34071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faf3406ecc0 0x7faf34071180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf40005950 tx=0x7faf400058e0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 msgr2=0x7faf5019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 0x7faf5019ef90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00bd70 tx=0x7faf3c00be50 comp rx=0 tx=0).stop 2024-04-20T11:42:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 shutdown_connections 2024-04-20T11:42:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faf3406ecc0 0x7faf34071180 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf50103110 0x7faf5019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 --2- 172.21.15.117:0/2112941099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf50104a00 0x7faf5019ef90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:06.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 >> 172.21.15.117:0/2112941099 conn(0x7faf500fe880 msgr2=0x7faf500ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:06.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.889+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 shutdown_connections 2024-04-20T11:42:06.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:06.890+0000 7faf56c00700 1 -- 172.21.15.117:0/2112941099 wait complete. 2024-04-20T11:42:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.034+0000 7fe75e600700 1 -- 172.21.15.117:0/145609883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 msgr2=0x7fe758104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.034+0000 7fe75e600700 1 --2- 172.21.15.117:0/145609883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 0x7fe758104e80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe744009910 tx=0x7fe744009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.034+0000 7fe75e600700 1 -- 172.21.15.117:0/145609883 shutdown_connections 2024-04-20T11:42:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.034+0000 7fe75e600700 1 --2- 172.21.15.117:0/145609883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 0x7fe758104e80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.034+0000 7fe75e600700 1 --2- 172.21.15.117:0/145609883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe758103110 0x7fe7581034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.034+0000 7fe75e600700 1 -- 172.21.15.117:0/145609883 >> 172.21.15.117:0/145609883 conn(0x7fe7580fe880 msgr2=0x7fe758100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.034+0000 7fe75e600700 1 -- 172.21.15.117:0/145609883 shutdown_connections 2024-04-20T11:42:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.034+0000 7fe75e600700 1 -- 172.21.15.117:0/145609883 wait complete. 2024-04-20T11:42:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.035+0000 7fe75e600700 1 Processor -- start 2024-04-20T11:42:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.035+0000 7fe75e600700 1 -- start start 2024-04-20T11:42:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.035+0000 7fe75e600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe758103110 0x7fe75819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.035+0000 7fe75e600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 0x7fe75819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe75e600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75819f5b0 con 0x7fe758104a00 2024-04-20T11:42:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe75e600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758198b90 con 0x7fe758103110 2024-04-20T11:42:07.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe756a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 0x7fe75819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe756a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 0x7fe75819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43084/0 (socket says 172.21.15.117:43084) 2024-04-20T11:42:07.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe756a00700 1 -- 172.21.15.117:0/1677119198 learned_addr learned my addr 172.21.15.117:0/1677119198 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe757400700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe758103110 0x7fe75819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe756a00700 1 -- 172.21.15.117:0/1677119198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe758103110 msgr2=0x7fe75819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe756a00700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe758103110 0x7fe75819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.036+0000 7fe756a00700 1 -- 172.21.15.117:0/1677119198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440095f0 con 0x7fe758104a00 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.037+0000 7fe756a00700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 0x7fe75819ef90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe7440092e0 tx=0x7fe74400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.037+0000 7fe74fe00700 1 -- 172.21.15.117:0/1677119198 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe74401d070 con 0x7fe758104a00 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.037+0000 7fe74fe00700 1 -- 172.21.15.117:0/1677119198 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe74400f460 con 0x7fe758104a00 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.037+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe758198e10 con 0x7fe758104a00 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.037+0000 7fe74fe00700 1 -- 172.21.15.117:0/1677119198 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe744017630 con 0x7fe758104a00 2024-04-20T11:42:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.037+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe758199380 con 0x7fe758104a00 2024-04-20T11:42:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.038+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75810b480 con 0x7fe758104a00 2024-04-20T11:42:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.039+0000 7fe74fe00700 1 -- 172.21.15.117:0/1677119198 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe74400f5d0 con 0x7fe758104a00 2024-04-20T11:42:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.040+0000 7fe74fe00700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe73c06ecd0 0x7fe73c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.040+0000 7fe74fe00700 1 -- 172.21.15.117:0/1677119198 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe744090340 con 0x7fe758104a00 2024-04-20T11:42:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.040+0000 7fe757400700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe73c06ecd0 0x7fe73c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.041+0000 7fe757400700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe73c06ecd0 0x7fe73c071190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe7500060b0 tx=0x7fe750008640 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.045+0000 7fe74fe00700 1 -- 172.21.15.117:0/1677119198 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe74405cc30 con 0x7fe758104a00 2024-04-20T11:42:07.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.319+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe758199fe0 con 0x7fe758104a00 2024-04-20T11:42:07.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.320+0000 7fe74fe00700 1 -- 172.21.15.117:0/1677119198 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fe74405c7c0 con 0x7fe758104a00 2024-04-20T11:42:07.320 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:42:07.321 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:42:07.322 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:42:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:42:07.324 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:42:07.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.324+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe73c06ecd0 msgr2=0x7fe73c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.324+0000 7fe75e600700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe73c06ecd0 0x7fe73c071190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe7500060b0 tx=0x7fe750008640 comp rx=0 tx=0).stop 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.324+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 msgr2=0x7fe75819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.324+0000 7fe75e600700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 0x7fe75819ef90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe7440092e0 tx=0x7fe74400be20 comp rx=0 tx=0).stop 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.325+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 shutdown_connections 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.325+0000 7fe75e600700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe73c06ecd0 0x7fe73c071190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.325+0000 7fe75e600700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe758103110 0x7fe75819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.325+0000 7fe75e600700 1 --2- 172.21.15.117:0/1677119198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe758104a00 0x7fe75819ef90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.325+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 >> 172.21.15.117:0/1677119198 conn(0x7fe7580fe880 msgr2=0x7fe7580ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.325+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 shutdown_connections 2024-04-20T11:42:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.325+0000 7fe75e600700 1 -- 172.21.15.117:0/1677119198 wait complete. 2024-04-20T11:42:07.326 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:42:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.471+0000 7f7e4d200700 1 -- 172.21.15.117:0/914959158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48107520 msgr2=0x7f7e48107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.471+0000 7f7e4d200700 1 --2- 172.21.15.117:0/914959158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48107520 0x7f7e48107900 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009910 tx=0x7f7e38009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.471+0000 7f7e4d200700 1 -- 172.21.15.117:0/914959158 shutdown_connections 2024-04-20T11:42:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.471+0000 7f7e4d200700 1 --2- 172.21.15.117:0/914959158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48108e10 0x7f7e48109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.471+0000 7f7e4d200700 1 --2- 172.21.15.117:0/914959158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48107520 0x7f7e48107900 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.471+0000 7f7e4d200700 1 -- 172.21.15.117:0/914959158 >> 172.21.15.117:0/914959158 conn(0x7f7e480768f0 msgr2=0x7f7e48078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:07.471 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:07 smithi117 ceph-mon[99036]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:07.471 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:07 smithi117 ceph-mon[99036]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:07.471 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:07 smithi117 ceph-mon[99036]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:07.472 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:07 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2112941099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:42:07.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.472+0000 7f7e4d200700 1 -- 172.21.15.117:0/914959158 shutdown_connections 2024-04-20T11:42:07.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.472+0000 7f7e4d200700 1 -- 172.21.15.117:0/914959158 wait complete. 2024-04-20T11:42:07.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.473+0000 7f7e4d200700 1 Processor -- start 2024-04-20T11:42:07.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.473+0000 7f7e4d200700 1 -- start start 2024-04-20T11:42:07.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.473+0000 7f7e4d200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48107520 0x7f7e4819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.473+0000 7f7e4d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48108e10 0x7f7e4819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.473+0000 7f7e4d200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4819de90 con 0x7f7e48108e10 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.473+0000 7f7e4d200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e481a1b60 con 0x7f7e48107520 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.474+0000 7f7e45400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48108e10 0x7f7e4819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.474+0000 7f7e45400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48108e10 0x7f7e4819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43104/0 (socket says 172.21.15.117:43104) 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.474+0000 7f7e45400700 1 -- 172.21.15.117:0/1808727515 learned_addr learned my addr 172.21.15.117:0/1808727515 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.474+0000 7f7e45e00700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48107520 0x7f7e4819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.474+0000 7f7e45400700 1 -- 172.21.15.117:0/1808727515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48107520 msgr2=0x7f7e4819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.474+0000 7f7e45400700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48107520 0x7f7e4819d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.474+0000 7f7e45400700 1 -- 172.21.15.117:0/1808727515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380095f0 con 0x7f7e48108e10 2024-04-20T11:42:07.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.474+0000 7f7e45400700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48108e10 0x7f7e4819d7b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400b4b0 tx=0x7f7e3400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.475+0000 7f7e3ea00700 1 -- 172.21.15.117:0/1808727515 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e34011800 con 0x7f7e48108e10 2024-04-20T11:42:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.475+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e481a1e40 con 0x7f7e48108e10 2024-04-20T11:42:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.475+0000 7f7e3ea00700 1 -- 172.21.15.117:0/1808727515 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e34011e20 con 0x7f7e48108e10 2024-04-20T11:42:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.475+0000 7f7e3ea00700 1 -- 172.21.15.117:0/1808727515 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e3400f2e0 con 0x7f7e48108e10 2024-04-20T11:42:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.475+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e481a2410 con 0x7f7e48108e10 2024-04-20T11:42:07.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.476+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4810f870 con 0x7f7e48108e10 2024-04-20T11:42:07.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.478+0000 7f7e3ea00700 1 -- 172.21.15.117:0/1808727515 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7e34010380 con 0x7f7e48108e10 2024-04-20T11:42:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.479+0000 7f7e3ea00700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e2c06ecd0 0x7f7e2c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.479+0000 7f7e3ea00700 1 -- 172.21.15.117:0/1808727515 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7e3408ee00 con 0x7f7e48108e10 2024-04-20T11:42:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.479+0000 7f7e45e00700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e2c06ecd0 0x7f7e2c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.480+0000 7f7e45e00700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e2c06ecd0 0x7f7e2c071190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800b5c0 tx=0x7f7e3801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:07.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.482+0000 7f7e3ea00700 1 -- 172.21.15.117:0/1808727515 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7e34057cf0 con 0x7f7e48108e10 2024-04-20T11:42:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:07 smithi173 ceph-mon[74165]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:07 smithi173 ceph-mon[74165]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:07 smithi173 ceph-mon[74165]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:07 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2112941099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:42:07.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.728+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e4819e5f0 con 0x7f7e2c06ecd0 2024-04-20T11:42:07.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.730+0000 7f7e3ea00700 1 -- 172.21.15.117:0/1808727515 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7e4819e5f0 con 0x7f7e2c06ecd0 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:42:07.730 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:42:07.731 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:42:07.731 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:42:07.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.732+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e2c06ecd0 msgr2=0x7f7e2c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e2c06ecd0 0x7f7e2c071190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800b5c0 tx=0x7f7e3801a040 comp rx=0 tx=0).stop 2024-04-20T11:42:07.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48108e10 msgr2=0x7f7e4819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48108e10 0x7f7e4819d7b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400b4b0 tx=0x7f7e3400b870 comp rx=0 tx=0).stop 2024-04-20T11:42:07.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 shutdown_connections 2024-04-20T11:42:07.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e2c06ecd0 0x7f7e2c071190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48107520 0x7f7e4819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 --2- 172.21.15.117:0/1808727515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e48108e10 0x7f7e4819d7b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 >> 172.21.15.117:0/1808727515 conn(0x7f7e480768f0 msgr2=0x7f7e48078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:07.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 shutdown_connections 2024-04-20T11:42:07.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.733+0000 7f7e4d200700 1 -- 172.21.15.117:0/1808727515 wait complete. 2024-04-20T11:42:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.883+0000 7f130c600700 1 -- 172.21.15.117:0/2560998735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304101d40 msgr2=0x7f1304102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.883+0000 7f130c600700 1 --2- 172.21.15.117:0/2560998735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304101d40 0x7f1304102120 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b0a0 tx=0x7f12f800b3b0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.883+0000 7f130c600700 1 -- 172.21.15.117:0/2560998735 shutdown_connections 2024-04-20T11:42:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.883+0000 7f130c600700 1 --2- 172.21.15.117:0/2560998735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1304102660 0x7f130410ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.883+0000 7f130c600700 1 --2- 172.21.15.117:0/2560998735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304101d40 0x7f1304102120 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.883+0000 7f130c600700 1 -- 172.21.15.117:0/2560998735 >> 172.21.15.117:0/2560998735 conn(0x7f1304076750 msgr2=0x7f13040fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.883+0000 7f130c600700 1 -- 172.21.15.117:0/2560998735 shutdown_connections 2024-04-20T11:42:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.883+0000 7f130c600700 1 -- 172.21.15.117:0/2560998735 wait complete. 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.884+0000 7f130c600700 1 Processor -- start 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.884+0000 7f130c600700 1 -- start start 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.884+0000 7f130c600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1304101d40 0x7f1304198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.884+0000 7f130c600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304102660 0x7f13041992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.884+0000 7f130c600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1304199990 con 0x7f1304102660 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.884+0000 7f130c600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130419d660 con 0x7f1304101d40 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.885+0000 7f1309200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304102660 0x7f13041992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.885+0000 7f1309200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304102660 0x7f13041992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43120/0 (socket says 172.21.15.117:43120) 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.885+0000 7f1309200700 1 -- 172.21.15.117:0/3843001184 learned_addr learned my addr 172.21.15.117:0/3843001184 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.885+0000 7f1309c00700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1304101d40 0x7f1304198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.885+0000 7f1309200700 1 -- 172.21.15.117:0/3843001184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1304101d40 msgr2=0x7f1304198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:07.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.885+0000 7f1309200700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1304101d40 0x7f1304198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:07.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.885+0000 7f1309200700 1 -- 172.21.15.117:0/3843001184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f8009d00 con 0x7f1304102660 2024-04-20T11:42:07.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.886+0000 7f1309200700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304102660 0x7f13041992b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c8f0 tx=0x7f12f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:07.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.886+0000 7f1302000700 1 -- 172.21.15.117:0/3843001184 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f0007ad0 con 0x7f1304102660 2024-04-20T11:42:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.886+0000 7f1302000700 1 -- 172.21.15.117:0/3843001184 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12f000ce90 con 0x7f1304102660 2024-04-20T11:42:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.886+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130419d940 con 0x7f1304102660 2024-04-20T11:42:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.886+0000 7f1302000700 1 -- 172.21.15.117:0/3843001184 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f0018630 con 0x7f1304102660 2024-04-20T11:42:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.886+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130419de90 con 0x7f1304102660 2024-04-20T11:42:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.887+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1304068e50 con 0x7f1304102660 2024-04-20T11:42:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.888+0000 7f1302000700 1 -- 172.21.15.117:0/3843001184 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f12f0007c30 con 0x7f1304102660 2024-04-20T11:42:07.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.889+0000 7f1302000700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12ec06ecc0 0x7f12ec071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:07.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.889+0000 7f1302000700 1 -- 172.21.15.117:0/3843001184 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f12f008ee30 con 0x7f1304102660 2024-04-20T11:42:07.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.889+0000 7f1309c00700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12ec06ecc0 0x7f12ec071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:07.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.890+0000 7f1309c00700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12ec06ecc0 0x7f12ec071180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009850 tx=0x7f12f80097c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:07.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:07.894+0000 7f1302000700 1 -- 172.21.15.117:0/3843001184 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f12f0057bf0 con 0x7f1304102660 2024-04-20T11:42:08.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.204+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f130410de60 con 0x7f1304102660 2024-04-20T11:42:08.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.204+0000 7f1302000700 1 -- 172.21.15.117:0/3843001184 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f12f005b2b0 con 0x7f1304102660 2024-04-20T11:42:08.205 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:42:08.205 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:42:08.205 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:42:08.206 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:42:08.206 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:42:08.206 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:42:08.206 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:42:08.206 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:42:08.206 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:42:08.206 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:42:08.207 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:42:08.207 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:42:08.207 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:42:08.207 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:42:08.207 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:42:08.207 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:42:08.207 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:42:08.207 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:42:08.208 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:42:08.208 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:42:08.208 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:42:08.208 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:42:08.208 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:42:08.208 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:42:08.208 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:42:08.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12ec06ecc0 msgr2=0x7f12ec071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:08.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12ec06ecc0 0x7f12ec071180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009850 tx=0x7f12f80097c0 comp rx=0 tx=0).stop 2024-04-20T11:42:08.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304102660 msgr2=0x7f13041992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:08.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304102660 0x7f13041992b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c8f0 tx=0x7f12f000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:42:08.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 shutdown_connections 2024-04-20T11:42:08.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f12ec06ecc0 0x7f12ec071180 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:08.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1304101d40 0x7f1304198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:08.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 --2- 172.21.15.117:0/3843001184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304102660 0x7f13041992b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:08.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 >> 172.21.15.117:0/3843001184 conn(0x7f1304076750 msgr2=0x7f130410a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:08.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 shutdown_connections 2024-04-20T11:42:08.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:08.207+0000 7f130c600700 1 -- 172.21.15.117:0/3843001184 wait complete. 2024-04-20T11:42:08.548 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:08 smithi173 ceph-mon[74165]: pgmap v891: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:08.549 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:08 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1677119198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:42:08.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:08 smithi117 ceph-mon[99036]: pgmap v891: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:08.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:08 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1677119198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:42:09.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:09 smithi117 ceph-mon[99036]: from='client.35618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:09.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:09 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3843001184' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:42:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:09 smithi173 ceph-mon[74165]: from='client.35618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:09 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3843001184' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:42:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:10 smithi117 ceph-mon[99036]: pgmap v892: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:10 smithi173 ceph-mon[74165]: pgmap v892: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:12.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:12 smithi117 ceph-mon[99036]: pgmap v893: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:12 smithi173 ceph-mon[74165]: pgmap v893: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:42:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:42:14.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:14 smithi117 ceph-mon[99036]: pgmap v894: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:14 smithi173 ceph-mon[74165]: pgmap v894: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:16 smithi117 ceph-mon[99036]: pgmap v895: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:16 smithi173 ceph-mon[74165]: pgmap v895: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:18 smithi117 ceph-mon[99036]: pgmap v896: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:18 smithi173 ceph-mon[74165]: pgmap v896: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:20.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:20 smithi117 ceph-mon[99036]: pgmap v897: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:20 smithi173 ceph-mon[74165]: pgmap v897: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:22.616 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:22 smithi117 ceph-mon[99036]: pgmap v898: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:22 smithi173 ceph-mon[74165]: pgmap v898: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:24.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:24 smithi173 ceph-mon[74165]: pgmap v899: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:24 smithi117 ceph-mon[99036]: pgmap v899: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:26.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:26 smithi117 ceph-mon[99036]: pgmap v900: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:26 smithi173 ceph-mon[74165]: pgmap v900: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:28 smithi117 ceph-mon[99036]: pgmap v901: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:42:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:28 smithi173 ceph-mon[74165]: pgmap v901: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:42:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:30 smithi117 ceph-mon[99036]: pgmap v902: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:30 smithi173 ceph-mon[74165]: pgmap v902: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:32 smithi117 ceph-mon[99036]: pgmap v903: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:32 smithi173 ceph-mon[74165]: pgmap v903: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:34 smithi117 ceph-mon[99036]: pgmap v904: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:34 smithi173 ceph-mon[74165]: pgmap v904: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:36 smithi117 ceph-mon[99036]: pgmap v905: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:36.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:36 smithi173 ceph-mon[74165]: pgmap v905: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:38.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.378+0000 7feb6a000700 1 -- 172.21.15.117:0/1255558406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 msgr2=0x7feb641114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:38.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.378+0000 7feb6a000700 1 --2- 172.21.15.117:0/1255558406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 0x7feb641114a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7feb50009910 tx=0x7feb50009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:38.379 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:38 smithi117 ceph-mon[99036]: pgmap v906: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.378+0000 7feb6a000700 1 -- 172.21.15.117:0/1255558406 shutdown_connections 2024-04-20T11:42:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.378+0000 7feb6a000700 1 --2- 172.21.15.117:0/1255558406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 0x7feb641114a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.378+0000 7feb6a000700 1 --2- 172.21.15.117:0/1255558406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb64073280 0x7feb64073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.378+0000 7feb6a000700 1 -- 172.21.15.117:0/1255558406 >> 172.21.15.117:0/1255558406 conn(0x7feb640fc500 msgr2=0x7feb640fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.379+0000 7feb6a000700 1 -- 172.21.15.117:0/1255558406 shutdown_connections 2024-04-20T11:42:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.379+0000 7feb6a000700 1 -- 172.21.15.117:0/1255558406 wait complete. 2024-04-20T11:42:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb6a000700 1 Processor -- start 2024-04-20T11:42:38.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb6a000700 1 -- start start 2024-04-20T11:42:38.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb6a000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb64073280 0x7feb6419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb6a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 0x7feb6419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb6a000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6419dc20 con 0x7feb64073ba0 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb6a000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb641a18f0 con 0x7feb64073280 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb63e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 0x7feb6419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb63e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 0x7feb6419d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56370/0 (socket says 172.21.15.117:56370) 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.380+0000 7feb63e00700 1 -- 172.21.15.117:0/1264124414 learned_addr learned my addr 172.21.15.117:0/1264124414 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.381+0000 7feb68c00700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb64073280 0x7feb6419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.381+0000 7feb63e00700 1 -- 172.21.15.117:0/1264124414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb64073280 msgr2=0x7feb6419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.381+0000 7feb63e00700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb64073280 0x7feb6419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.381+0000 7feb63e00700 1 -- 172.21.15.117:0/1264124414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500095f0 con 0x7feb64073ba0 2024-04-20T11:42:38.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.381+0000 7feb63e00700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 0x7feb6419d540 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7feb50006010 tx=0x7feb5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.381+0000 7feb61600700 1 -- 172.21.15.117:0/1264124414 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb5001d070 con 0x7feb64073ba0 2024-04-20T11:42:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.382+0000 7feb61600700 1 -- 172.21.15.117:0/1264124414 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb5000f460 con 0x7feb64073ba0 2024-04-20T11:42:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.382+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb641a1bd0 con 0x7feb64073ba0 2024-04-20T11:42:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.382+0000 7feb61600700 1 -- 172.21.15.117:0/1264124414 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb50017630 con 0x7feb64073ba0 2024-04-20T11:42:38.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.382+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb641a2120 con 0x7feb64073ba0 2024-04-20T11:42:38.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.383+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb640749c0 con 0x7feb64073ba0 2024-04-20T11:42:38.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.384+0000 7feb61600700 1 -- 172.21.15.117:0/1264124414 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7feb5000f5d0 con 0x7feb64073ba0 2024-04-20T11:42:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.385+0000 7feb61600700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feb4c06ecc0 0x7feb4c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:38.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.385+0000 7feb61600700 1 -- 172.21.15.117:0/1264124414 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7feb500902d0 con 0x7feb64073ba0 2024-04-20T11:42:38.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.385+0000 7feb68c00700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feb4c06ecc0 0x7feb4c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:38.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.386+0000 7feb68c00700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feb4c06ecc0 0x7feb4c071180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb58009f20 tx=0x7feb580094f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:38.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.390+0000 7feb61600700 1 -- 172.21.15.117:0/1264124414 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7feb50059150 con 0x7feb64073ba0 2024-04-20T11:42:38.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.633+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb64114300 con 0x7feb4c06ecc0 2024-04-20T11:42:38.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.637+0000 7feb61600700 1 -- 172.21.15.117:0/1264124414 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feb64114300 con 0x7feb4c06ecc0 2024-04-20T11:42:38.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feb4c06ecc0 msgr2=0x7feb4c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:38.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feb4c06ecc0 0x7feb4c071180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb58009f20 tx=0x7feb580094f0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 msgr2=0x7feb6419d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:38.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 0x7feb6419d540 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7feb50006010 tx=0x7feb5000be20 comp rx=0 tx=0).stop 2024-04-20T11:42:38.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 shutdown_connections 2024-04-20T11:42:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feb4c06ecc0 0x7feb4c071180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb64073280 0x7feb6419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 --2- 172.21.15.117:0/1264124414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb64073ba0 0x7feb6419d540 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.639+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 >> 172.21.15.117:0/1264124414 conn(0x7feb640fc500 msgr2=0x7feb64100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.640+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 shutdown_connections 2024-04-20T11:42:38.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.640+0000 7feb6a000700 1 -- 172.21.15.117:0/1264124414 wait complete. 2024-04-20T11:42:38.656 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:42:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:38 smithi173 ceph-mon[74165]: pgmap v906: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:38.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.797+0000 7f8e07c00700 1 -- 172.21.15.117:0/3547206023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073ba0 msgr2=0x7f8e00111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:38.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.797+0000 7f8e07c00700 1 --2- 172.21.15.117:0/3547206023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073ba0 0x7f8e00111680 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009910 tx=0x7f8dec009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:38.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.797+0000 7f8e07c00700 1 -- 172.21.15.117:0/3547206023 shutdown_connections 2024-04-20T11:42:38.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.797+0000 7f8e07c00700 1 --2- 172.21.15.117:0/3547206023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073ba0 0x7f8e00111680 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.797+0000 7f8e07c00700 1 --2- 172.21.15.117:0/3547206023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e00073280 0x7f8e00073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.797+0000 7f8e07c00700 1 -- 172.21.15.117:0/3547206023 >> 172.21.15.117:0/3547206023 conn(0x7f8e000fc6c0 msgr2=0x7f8e000feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:38.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.797+0000 7f8e07c00700 1 -- 172.21.15.117:0/3547206023 shutdown_connections 2024-04-20T11:42:38.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.797+0000 7f8e07c00700 1 -- 172.21.15.117:0/3547206023 wait complete. 2024-04-20T11:42:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.798+0000 7f8e07c00700 1 Processor -- start 2024-04-20T11:42:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.798+0000 7f8e07c00700 1 -- start start 2024-04-20T11:42:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.799+0000 7f8e07c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073280 0x7f8e0019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.799+0000 7f8e07c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e00073ba0 0x7f8e0019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.799+0000 7f8e07c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0019dcc0 con 0x7f8e00073280 2024-04-20T11:42:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.799+0000 7f8e07c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001a1990 con 0x7f8e00073ba0 2024-04-20T11:42:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.799+0000 7f8e05200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073280 0x7f8e0019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.799+0000 7f8e05200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073280 0x7f8e0019d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56402/0 (socket says 172.21.15.117:56402) 2024-04-20T11:42:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.799+0000 7f8e05200700 1 -- 172.21.15.117:0/837232591 learned_addr learned my addr 172.21.15.117:0/837232591 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.799+0000 7f8dffe00700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e00073ba0 0x7f8e0019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8e05200700 1 -- 172.21.15.117:0/837232591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e00073ba0 msgr2=0x7f8e0019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8e05200700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e00073ba0 0x7f8e0019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8e05200700 1 -- 172.21.15.117:0/837232591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec0095f0 con 0x7f8e00073280 2024-04-20T11:42:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8e05200700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073280 0x7f8e0019d0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8df400d8d0 tx=0x7f8df400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8dfd600700 1 -- 172.21.15.117:0/837232591 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8df4009940 con 0x7f8e00073280 2024-04-20T11:42:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8dfd600700 1 -- 172.21.15.117:0/837232591 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8df4010460 con 0x7f8e00073280 2024-04-20T11:42:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e001a1c70 con 0x7f8e00073280 2024-04-20T11:42:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8dfd600700 1 -- 172.21.15.117:0/837232591 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8df400f5f0 con 0x7f8e00073280 2024-04-20T11:42:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.800+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e001a21c0 con 0x7f8e00073280 2024-04-20T11:42:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.801+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e000749c0 con 0x7f8e00073280 2024-04-20T11:42:38.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.802+0000 7f8dfd600700 1 -- 172.21.15.117:0/837232591 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8df40105d0 con 0x7f8e00073280 2024-04-20T11:42:38.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.803+0000 7f8dfd600700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8de806ecc0 0x7f8de8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:38.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.803+0000 7f8dfd600700 1 -- 172.21.15.117:0/837232591 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8df408eef0 con 0x7f8e00073280 2024-04-20T11:42:38.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.804+0000 7f8dffe00700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8de806ecc0 0x7f8de8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:38.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.804+0000 7f8dffe00700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8de806ecc0 0x7f8de8071180 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8dec006010 tx=0x7f8dec00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:38.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:38.808+0000 7f8dfd600700 1 -- 172.21.15.117:0/837232591 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8df4057c30 con 0x7f8e00073280 2024-04-20T11:42:39.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.053+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e00114480 con 0x7f8de806ecc0 2024-04-20T11:42:39.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.055+0000 7f8dfd600700 1 -- 172.21.15.117:0/837232591 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8e00114480 con 0x7f8de806ecc0 2024-04-20T11:42:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.057+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8de806ecc0 msgr2=0x7f8de8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.057+0000 7f8e07c00700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8de806ecc0 0x7f8de8071180 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8dec006010 tx=0x7f8dec00b540 comp rx=0 tx=0).stop 2024-04-20T11:42:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.057+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073280 msgr2=0x7f8e0019d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.057+0000 7f8e07c00700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073280 0x7f8e0019d0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8df400d8d0 tx=0x7f8df400dc90 comp rx=0 tx=0).stop 2024-04-20T11:42:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.057+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 shutdown_connections 2024-04-20T11:42:39.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.057+0000 7f8e07c00700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8de806ecc0 0x7f8de8071180 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.057+0000 7f8e07c00700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e00073280 0x7f8e0019d0a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.058+0000 7f8e07c00700 1 --2- 172.21.15.117:0/837232591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e00073ba0 0x7f8e0019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.058+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 >> 172.21.15.117:0/837232591 conn(0x7f8e000fc6c0 msgr2=0x7f8e001008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.058+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 shutdown_connections 2024-04-20T11:42:39.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.058+0000 7f8e07c00700 1 -- 172.21.15.117:0/837232591 wait complete. 2024-04-20T11:42:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.205+0000 7f24bf000700 1 -- 172.21.15.117:0/2631860352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107df0 msgr2=0x7f24b810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.205+0000 7f24bf000700 1 --2- 172.21.15.117:0/2631860352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107df0 0x7f24b810be40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009910 tx=0x7f24a4009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.205+0000 7f24bf000700 1 -- 172.21.15.117:0/2631860352 shutdown_connections 2024-04-20T11:42:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.205+0000 7f24bf000700 1 --2- 172.21.15.117:0/2631860352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107df0 0x7f24b810be40 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.205+0000 7f24bf000700 1 --2- 172.21.15.117:0/2631860352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24b8107440 0x7f24b8107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.205+0000 7f24bf000700 1 -- 172.21.15.117:0/2631860352 >> 172.21.15.117:0/2631860352 conn(0x7f24b8076a10 msgr2=0x7f24b8078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.205+0000 7f24bf000700 1 -- 172.21.15.117:0/2631860352 shutdown_connections 2024-04-20T11:42:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.205+0000 7f24bf000700 1 -- 172.21.15.117:0/2631860352 wait complete. 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.206+0000 7f24bf000700 1 Processor -- start 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.206+0000 7f24bf000700 1 -- start start 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.207+0000 7f24bf000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107440 0x7f24b819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.207+0000 7f24bf000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24b8107df0 0x7f24b819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.207+0000 7f24bf000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b819dd70 con 0x7f24b8107440 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.207+0000 7f24bf000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81a19f0 con 0x7f24b8107df0 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.207+0000 7f24b7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107440 0x7f24b819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.207+0000 7f24b7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107440 0x7f24b819d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56420/0 (socket says 172.21.15.117:56420) 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.207+0000 7f24b7e00700 1 -- 172.21.15.117:0/2101264637 learned_addr learned my addr 172.21.15.117:0/2101264637 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:39.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.207+0000 7f24b7400700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24b8107df0 0x7f24b819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:39.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24b7e00700 1 -- 172.21.15.117:0/2101264637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24b8107df0 msgr2=0x7f24b819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24b7e00700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24b8107df0 0x7f24b819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24b7e00700 1 -- 172.21.15.117:0/2101264637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40095f0 con 0x7f24b8107440 2024-04-20T11:42:39.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24b7e00700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107440 0x7f24b819d150 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f24a800d8d0 tx=0x7f24a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:39.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24b4c00700 1 -- 172.21.15.117:0/2101264637 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24a8009940 con 0x7f24b8107440 2024-04-20T11:42:39.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b81a1cd0 con 0x7f24b8107440 2024-04-20T11:42:39.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24b4c00700 1 -- 172.21.15.117:0/2101264637 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24a8010460 con 0x7f24b8107440 2024-04-20T11:42:39.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b81a2220 con 0x7f24b8107440 2024-04-20T11:42:39.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.208+0000 7f24b4c00700 1 -- 172.21.15.117:0/2101264637 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24a801b620 con 0x7f24b8107440 2024-04-20T11:42:39.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.210+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b810f840 con 0x7f24b8107440 2024-04-20T11:42:39.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.210+0000 7f24b4c00700 1 -- 172.21.15.117:0/2101264637 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f24a801b780 con 0x7f24b8107440 2024-04-20T11:42:39.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.212+0000 7f24b4c00700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f249c06ecc0 0x7f249c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:39.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.212+0000 7f24b4c00700 1 -- 172.21.15.117:0/2101264637 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f24a808f1d0 con 0x7f24b8107440 2024-04-20T11:42:39.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.212+0000 7f24b7400700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f249c06ecc0 0x7f249c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:39.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.213+0000 7f24b7400700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f249c06ecc0 0x7f249c071180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f24b819e660 tx=0x7f24a401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:39.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.217+0000 7f24b4c00700 1 -- 172.21.15.117:0/2101264637 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f24a8057f10 con 0x7f24b8107440 2024-04-20T11:42:39.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.438+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b819e480 con 0x7f249c06ecc0 2024-04-20T11:42:39.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.469+0000 7f24b4c00700 1 -- 172.21.15.117:0/2101264637 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f24b819e480 con 0x7f249c06ecc0 2024-04-20T11:42:39.470 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (31m) 7m ago 45m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (45m) 7m ago 45m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (44m) 7m ago 44m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (27m) 7m ago 45m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (27m) 7m ago 44m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (30m) 7m ago 44m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (22m) 7m ago 37m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (23m) 7m ago 38m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (7m) 7m ago 37m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (22m) 7m ago 37m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (34m) 7m ago 46m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (32m) 7m ago 43m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:42:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (29m) 7m ago 47m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (28m) 7m ago 43m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (32m) 7m ago 45m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (32m) 7m ago 43m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (26m) 7m ago 42m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (26m) 7m ago 42m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (25m) 7m ago 41m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (25m) 7m ago 40m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (24m) 7m ago 40m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (23m) 7m ago 39m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:42:39.472 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (31m) 7m ago 44m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:42:39.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.473+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f249c06ecc0 msgr2=0x7f249c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.473+0000 7f24bf000700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f249c06ecc0 0x7f249c071180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f24b819e660 tx=0x7f24a401a040 comp rx=0 tx=0).stop 2024-04-20T11:42:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.473+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107440 msgr2=0x7f24b819d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.473+0000 7f24bf000700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107440 0x7f24b819d150 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f24a800d8d0 tx=0x7f24a800dc90 comp rx=0 tx=0).stop 2024-04-20T11:42:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.473+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 shutdown_connections 2024-04-20T11:42:39.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.473+0000 7f24bf000700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f249c06ecc0 0x7f249c071180 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.473+0000 7f24bf000700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24b8107440 0x7f24b819d150 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.473+0000 7f24bf000700 1 --2- 172.21.15.117:0/2101264637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24b8107df0 0x7f24b819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.474+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 >> 172.21.15.117:0/2101264637 conn(0x7f24b8076a10 msgr2=0x7f24b8076df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:39.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.474+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 shutdown_connections 2024-04-20T11:42:39.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.474+0000 7f24bf000700 1 -- 172.21.15.117:0/2101264637 wait complete. 2024-04-20T11:42:39.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.629+0000 7fb101200700 1 -- 172.21.15.117:0/3647263286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc1039d0 msgr2=0x7fb0fc107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.629+0000 7fb101200700 1 --2- 172.21.15.117:0/3647263286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc1039d0 0x7fb0fc107a20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009910 tx=0x7fb0e4009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:39.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.629+0000 7fb101200700 1 -- 172.21.15.117:0/3647263286 shutdown_connections 2024-04-20T11:42:39.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.629+0000 7fb101200700 1 --2- 172.21.15.117:0/3647263286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc1039d0 0x7fb0fc107a20 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.629+0000 7fb101200700 1 --2- 172.21.15.117:0/3647263286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0fc103020 0x7fb0fc103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.629+0000 7fb101200700 1 -- 172.21.15.117:0/3647263286 >> 172.21.15.117:0/3647263286 conn(0x7fb0fc0fe8b0 msgr2=0x7fb0fc100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:39.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.629+0000 7fb101200700 1 -- 172.21.15.117:0/3647263286 shutdown_connections 2024-04-20T11:42:39.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.629+0000 7fb101200700 1 -- 172.21.15.117:0/3647263286 wait complete. 2024-04-20T11:42:39.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.630+0000 7fb101200700 1 Processor -- start 2024-04-20T11:42:39.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.631+0000 7fb101200700 1 -- start start 2024-04-20T11:42:39.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.631+0000 7fb101200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc103020 0x7fb0fc198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:39.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.631+0000 7fb101200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0fc1039d0 0x7fb0fc199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:39.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.631+0000 7fb101200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1998e0 con 0x7fb0fc103020 2024-04-20T11:42:39.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.631+0000 7fb101200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19c550 con 0x7fb0fc1039d0 2024-04-20T11:42:39.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.631+0000 7fb0f9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc103020 0x7fb0fc198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:39.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.631+0000 7fb0f9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc103020 0x7fb0fc198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56440/0 (socket says 172.21.15.117:56440) 2024-04-20T11:42:39.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.631+0000 7fb0f9e00700 1 -- 172.21.15.117:0/1015275834 learned_addr learned my addr 172.21.15.117:0/1015275834 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:39.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb0f9e00700 1 -- 172.21.15.117:0/1015275834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0fc1039d0 msgr2=0x7fb0fc199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb0f1400700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0fc1039d0 0x7fb0fc199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:39.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb0f9e00700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0fc1039d0 0x7fb0fc199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb0f9e00700 1 -- 172.21.15.117:0/1015275834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40095f0 con 0x7fb0fc103020 2024-04-20T11:42:39.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb0f9e00700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc103020 0x7fb0fc198d10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0060b0 tx=0x7fb0ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:39.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb0f3400700 1 -- 172.21.15.117:0/1015275834 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0ec00dda0 con 0x7fb0fc103020 2024-04-20T11:42:39.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb0f3400700 1 -- 172.21.15.117:0/1015275834 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0ec00f040 con 0x7fb0fc103020 2024-04-20T11:42:39.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc19c830 con 0x7fb0fc103020 2024-04-20T11:42:39.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.632+0000 7fb0f3400700 1 -- 172.21.15.117:0/1015275834 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0ec014790 con 0x7fb0fc103020 2024-04-20T11:42:39.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.633+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc19cd80 con 0x7fb0fc103020 2024-04-20T11:42:39.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.634+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc10b420 con 0x7fb0fc103020 2024-04-20T11:42:39.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.635+0000 7fb0f3400700 1 -- 172.21.15.117:0/1015275834 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb0ec0148f0 con 0x7fb0fc103020 2024-04-20T11:42:39.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.636+0000 7fb0f3400700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0e006ecc0 0x7fb0e0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:39.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.636+0000 7fb0f3400700 1 -- 172.21.15.117:0/1015275834 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb0ec08e800 con 0x7fb0fc103020 2024-04-20T11:42:39.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.636+0000 7fb0f1400700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0e006ecc0 0x7fb0e0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:39.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.637+0000 7fb0f1400700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0e006ecc0 0x7fb0e0071180 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4006010 tx=0x7fb0e400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:39.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.641+0000 7fb0f3400700 1 -- 172.21.15.117:0/1015275834 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb0ec057710 con 0x7fb0fc103020 2024-04-20T11:42:39.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.949+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb0fc19a080 con 0x7fb0fc103020 2024-04-20T11:42:39.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.950+0000 7fb0f3400700 1 -- 172.21.15.117:0/1015275834 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb0ec08ce60 con 0x7fb0fc103020 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:42:39.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:42:39.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:42:39.952 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:42:39.952 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:42:39.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:42:39.952 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:42:39.952 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:42:39.952 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:42:39.952 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0e006ecc0 msgr2=0x7fb0e0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0e006ecc0 0x7fb0e0071180 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4006010 tx=0x7fb0e400b540 comp rx=0 tx=0).stop 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc103020 msgr2=0x7fb0fc198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc103020 0x7fb0fc198d10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0060b0 tx=0x7fb0ec00d3b0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 shutdown_connections 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0e006ecc0 0x7fb0e0071180 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0fc103020 0x7fb0fc198d10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 --2- 172.21.15.117:0/1015275834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0fc1039d0 0x7fb0fc199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 >> 172.21.15.117:0/1015275834 conn(0x7fb0fc0fe8b0 msgr2=0x7fb0fc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 shutdown_connections 2024-04-20T11:42:39.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:39.953+0000 7fb101200700 1 -- 172.21.15.117:0/1015275834 wait complete. 2024-04-20T11:42:40.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.100+0000 7fe8c0400700 1 -- 172.21.15.117:0/30967595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8108e10 msgr2=0x7fe8b8109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.100+0000 7fe8c0400700 1 --2- 172.21.15.117:0/30967595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8108e10 0x7fe8b8109290 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009910 tx=0x7fe8a4009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:40.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.100+0000 7fe8c0400700 1 -- 172.21.15.117:0/30967595 shutdown_connections 2024-04-20T11:42:40.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.100+0000 7fe8c0400700 1 --2- 172.21.15.117:0/30967595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8108e10 0x7fe8b8109290 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.100+0000 7fe8c0400700 1 --2- 172.21.15.117:0/30967595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8b8107520 0x7fe8b8107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.100+0000 7fe8c0400700 1 -- 172.21.15.117:0/30967595 >> 172.21.15.117:0/30967595 conn(0x7fe8b80768f0 msgr2=0x7fe8b8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:40.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.100+0000 7fe8c0400700 1 -- 172.21.15.117:0/30967595 shutdown_connections 2024-04-20T11:42:40.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.101+0000 7fe8c0400700 1 -- 172.21.15.117:0/30967595 wait complete. 2024-04-20T11:42:40.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.101+0000 7fe8c0400700 1 Processor -- start 2024-04-20T11:42:40.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.102+0000 7fe8c0400700 1 -- start start 2024-04-20T11:42:40.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.102+0000 7fe8c0400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8107520 0x7fe8b819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.102+0000 7fe8c0400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8b8108e10 0x7fe8b819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.102+0000 7fe8c0400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b819ddf0 con 0x7fe8b8107520 2024-04-20T11:42:40.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.102+0000 7fe8c0400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81a1a70 con 0x7fe8b8108e10 2024-04-20T11:42:40.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.102+0000 7fe8bda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8107520 0x7fe8b819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:40.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.102+0000 7fe8bda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8107520 0x7fe8b819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56448/0 (socket says 172.21.15.117:56448) 2024-04-20T11:42:40.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.102+0000 7fe8bda00700 1 -- 172.21.15.117:0/830297843 learned_addr learned my addr 172.21.15.117:0/830297843 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:40.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8bda00700 1 -- 172.21.15.117:0/830297843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8b8108e10 msgr2=0x7fe8b819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8bda00700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8b8108e10 0x7fe8b819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8bda00700 1 -- 172.21.15.117:0/830297843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40095f0 con 0x7fe8b8107520 2024-04-20T11:42:40.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8bda00700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8107520 0x7fe8b819d1d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c8a0 tx=0x7fe8ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:40.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8b6000700 1 -- 172.21.15.117:0/830297843 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8ac00cea0 con 0x7fe8b8107520 2024-04-20T11:42:40.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8b6000700 1 -- 172.21.15.117:0/830297843 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8ac004830 con 0x7fe8b8107520 2024-04-20T11:42:40.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b81a1d50 con 0x7fe8b8107520 2024-04-20T11:42:40.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8b6000700 1 -- 172.21.15.117:0/830297843 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8ac0055b0 con 0x7fe8b8107520 2024-04-20T11:42:40.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.103+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b81a2320 con 0x7fe8b8107520 2024-04-20T11:42:40.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.105+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b810f870 con 0x7fe8b8107520 2024-04-20T11:42:40.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.105+0000 7fe8b6000700 1 -- 172.21.15.117:0/830297843 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe8ac0049a0 con 0x7fe8b8107520 2024-04-20T11:42:40.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.106+0000 7fe8b6000700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8a006ecd0 0x7fe8a0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.106+0000 7fe8b6000700 1 -- 172.21.15.117:0/830297843 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe8ac08eac0 con 0x7fe8b8107520 2024-04-20T11:42:40.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.107+0000 7fe8bd000700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8a006ecd0 0x7fe8a0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:40.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.107+0000 7fe8bd000700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8a006ecd0 0x7fe8a0071190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4006010 tx=0x7fe8a400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:40.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.111+0000 7fe8b6000700 1 -- 172.21.15.117:0/830297843 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe8ac057880 con 0x7fe8b8107520 2024-04-20T11:42:40.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.369+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe8b819e490 con 0x7fe8b8107520 2024-04-20T11:42:40.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.370+0000 7fe8b6000700 1 -- 172.21.15.117:0/830297843 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fe8ac01d070 con 0x7fe8b8107520 2024-04-20T11:42:40.372 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:42:40.372 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:42:40.372 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:42:40.372 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:42:40.372 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:42:40.373 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:42:40.374 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:42:40.374 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:42:40.375 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:42:40.375 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:42:40.375 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:42:40.375 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:42:40.376 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:42:40.377 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:42:40.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8a006ecd0 msgr2=0x7fe8a0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8a006ecd0 0x7fe8a0071190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4006010 tx=0x7fe8a400b540 comp rx=0 tx=0).stop 2024-04-20T11:42:40.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8107520 msgr2=0x7fe8b819d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8107520 0x7fe8b819d1d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c8a0 tx=0x7fe8ac00cc60 comp rx=0 tx=0).stop 2024-04-20T11:42:40.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 shutdown_connections 2024-04-20T11:42:40.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8a006ecd0 0x7fe8a0071190 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b8107520 0x7fe8b819d1d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 --2- 172.21.15.117:0/830297843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8b8108e10 0x7fe8b819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.374+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 >> 172.21.15.117:0/830297843 conn(0x7fe8b80768f0 msgr2=0x7fe8b8077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:40.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.375+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 shutdown_connections 2024-04-20T11:42:40.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.375+0000 7fe8c0400700 1 -- 172.21.15.117:0/830297843 wait complete. 2024-04-20T11:42:40.380 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:42:40.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.540+0000 7fb117a00700 1 -- 172.21.15.117:0/3182873427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110107520 msgr2=0x7fb110107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.540+0000 7fb117a00700 1 --2- 172.21.15.117:0/3182873427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110107520 0x7fb110107900 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb10000b0a0 tx=0x7fb10000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.541+0000 7fb117a00700 1 -- 172.21.15.117:0/3182873427 shutdown_connections 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.541+0000 7fb117a00700 1 --2- 172.21.15.117:0/3182873427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb110108e10 0x7fb110109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.541+0000 7fb117a00700 1 --2- 172.21.15.117:0/3182873427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110107520 0x7fb110107900 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.541+0000 7fb117a00700 1 -- 172.21.15.117:0/3182873427 >> 172.21.15.117:0/3182873427 conn(0x7fb1100768f0 msgr2=0x7fb110078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.541+0000 7fb117a00700 1 -- 172.21.15.117:0/3182873427 shutdown_connections 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.541+0000 7fb117a00700 1 -- 172.21.15.117:0/3182873427 wait complete. 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.542+0000 7fb117a00700 1 Processor -- start 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.542+0000 7fb117a00700 1 -- start start 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.542+0000 7fb117a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb110107520 0x7fb11019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.542+0000 7fb117a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110108e10 0x7fb11019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb117a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11019dd80 con 0x7fb110108e10 2024-04-20T11:42:40.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb117a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11019dec0 con 0x7fb110107520 2024-04-20T11:42:40.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb107e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110108e10 0x7fb11019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:40.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb107e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110108e10 0x7fb11019d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56468/0 (socket says 172.21.15.117:56468) 2024-04-20T11:42:40.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb107e00700 1 -- 172.21.15.117:0/12934300 learned_addr learned my addr 172.21.15.117:0/12934300 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:40.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb115000700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb110107520 0x7fb11019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:40.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb107e00700 1 -- 172.21.15.117:0/12934300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb110107520 msgr2=0x7fb11019d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb107e00700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb110107520 0x7fb11019d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb107e00700 1 -- 172.21.15.117:0/12934300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb100009d00 con 0x7fb110108e10 2024-04-20T11:42:40.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.543+0000 7fb115000700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb110107520 0x7fb11019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:42:40.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.544+0000 7fb107e00700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110108e10 0x7fb11019d730 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c930 tx=0x7fb0fc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:40.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.544+0000 7fb10e000700 1 -- 172.21.15.117:0/12934300 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0fc007ad0 con 0x7fb110108e10 2024-04-20T11:42:40.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.544+0000 7fb10e000700 1 -- 172.21.15.117:0/12934300 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0fc007c30 con 0x7fb110108e10 2024-04-20T11:42:40.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.544+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1101a1c50 con 0x7fb110108e10 2024-04-20T11:42:40.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.544+0000 7fb10e000700 1 -- 172.21.15.117:0/12934300 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0fc018610 con 0x7fb110108e10 2024-04-20T11:42:40.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.544+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1101a2220 con 0x7fb110108e10 2024-04-20T11:42:40.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.545+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11010f870 con 0x7fb110108e10 2024-04-20T11:42:40.548 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:40 smithi117 ceph-mon[99036]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:40.548 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:40 smithi117 ceph-mon[99036]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:40.548 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:40 smithi117 ceph-mon[99036]: pgmap v907: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:40.549 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:40 smithi117 ceph-mon[99036]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:40.549 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:40 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1015275834' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:42:40.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.546+0000 7fb10e000700 1 -- 172.21.15.117:0/12934300 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb0fc00f450 con 0x7fb110108e10 2024-04-20T11:42:40.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.548+0000 7fb10e000700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0f406ecd0 0x7fb0f4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.548+0000 7fb10e000700 1 -- 172.21.15.117:0/12934300 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb0fc08efe0 con 0x7fb110108e10 2024-04-20T11:42:40.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.548+0000 7fb115000700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0f406ecd0 0x7fb0f4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:40.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.549+0000 7fb115000700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0f406ecd0 0x7fb0f4071190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb10000b830 tx=0x7fb100006210 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:40.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.553+0000 7fb10e000700 1 -- 172.21.15.117:0/12934300 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb0fc057de0 con 0x7fb110108e10 2024-04-20T11:42:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:40 smithi173 ceph-mon[74165]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:40 smithi173 ceph-mon[74165]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:40 smithi173 ceph-mon[74165]: pgmap v907: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:40 smithi173 ceph-mon[74165]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:40 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1015275834' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:42:40.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.781+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb11019e570 con 0x7fb0f406ecd0 2024-04-20T11:42:40.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.783+0000 7fb10e000700 1 -- 172.21.15.117:0/12934300 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb11019e570 con 0x7fb0f406ecd0 2024-04-20T11:42:40.784 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:42:40.784 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:42:40.784 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:42:40.785 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:42:40.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.786+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0f406ecd0 msgr2=0x7fb0f4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.786+0000 7fb117a00700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0f406ecd0 0x7fb0f4071190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb10000b830 tx=0x7fb100006210 comp rx=0 tx=0).stop 2024-04-20T11:42:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.786+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110108e10 msgr2=0x7fb11019d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.786+0000 7fb117a00700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110108e10 0x7fb11019d730 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c930 tx=0x7fb0fc00ccf0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.787+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 shutdown_connections 2024-04-20T11:42:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.787+0000 7fb117a00700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb0f406ecd0 0x7fb0f4071190 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.787+0000 7fb117a00700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb110107520 0x7fb11019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.787+0000 7fb117a00700 1 --2- 172.21.15.117:0/12934300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb110108e10 0x7fb11019d730 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.787+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 >> 172.21.15.117:0/12934300 conn(0x7fb1100768f0 msgr2=0x7fb1101064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:40.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.787+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 shutdown_connections 2024-04-20T11:42:40.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.787+0000 7fb117a00700 1 -- 172.21.15.117:0/12934300 wait complete. 2024-04-20T11:42:40.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.931+0000 7fdd8a200700 1 -- 172.21.15.117:0/2140140740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 msgr2=0x7fdd84107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.931+0000 7fdd8a200700 1 --2- 172.21.15.117:0/2140140740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 0x7fdd84107900 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009910 tx=0x7fdd74009c20 comp rx=0 tx=0).stop 2024-04-20T11:42:40.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.931+0000 7fdd8a200700 1 -- 172.21.15.117:0/2140140740 shutdown_connections 2024-04-20T11:42:40.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.931+0000 7fdd8a200700 1 --2- 172.21.15.117:0/2140140740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84108e10 0x7fdd84109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.932+0000 7fdd8a200700 1 --2- 172.21.15.117:0/2140140740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 0x7fdd84107900 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.932+0000 7fdd8a200700 1 -- 172.21.15.117:0/2140140740 >> 172.21.15.117:0/2140140740 conn(0x7fdd840768f0 msgr2=0x7fdd84078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:40.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.932+0000 7fdd8a200700 1 -- 172.21.15.117:0/2140140740 shutdown_connections 2024-04-20T11:42:40.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.932+0000 7fdd8a200700 1 -- 172.21.15.117:0/2140140740 wait complete. 2024-04-20T11:42:40.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.933+0000 7fdd8a200700 1 Processor -- start 2024-04-20T11:42:40.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.933+0000 7fdd8a200700 1 -- start start 2024-04-20T11:42:40.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.933+0000 7fdd8a200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 0x7fdd8419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.933+0000 7fdd8a200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84108e10 0x7fdd8419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.933+0000 7fdd8a200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8419ddd0 con 0x7fdd84107520 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.933+0000 7fdd8a200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd841a1aa0 con 0x7fdd84108e10 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.934+0000 7fdd83400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 0x7fdd8419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.934+0000 7fdd83400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 0x7fdd8419d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56486/0 (socket says 172.21.15.117:56486) 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.934+0000 7fdd83400700 1 -- 172.21.15.117:0/4035564567 learned_addr learned my addr 172.21.15.117:0/4035564567 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.934+0000 7fdd82a00700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84108e10 0x7fdd8419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.934+0000 7fdd83400700 1 -- 172.21.15.117:0/4035564567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84108e10 msgr2=0x7fdd8419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.934+0000 7fdd83400700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84108e10 0x7fdd8419d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:40.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.934+0000 7fdd83400700 1 -- 172.21.15.117:0/4035564567 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740095f0 con 0x7fdd84107520 2024-04-20T11:42:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.934+0000 7fdd83400700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 0x7fdd8419d1b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdd74005f50 tx=0x7fdd7400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.935+0000 7fdd7be00700 1 -- 172.21.15.117:0/4035564567 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd7401d070 con 0x7fdd84107520 2024-04-20T11:42:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.935+0000 7fdd7be00700 1 -- 172.21.15.117:0/4035564567 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd740052a0 con 0x7fdd84107520 2024-04-20T11:42:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.935+0000 7fdd7be00700 1 -- 172.21.15.117:0/4035564567 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd74017400 con 0x7fdd84107520 2024-04-20T11:42:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.935+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd841a1d20 con 0x7fdd84107520 2024-04-20T11:42:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.935+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd841a2290 con 0x7fdd84107520 2024-04-20T11:42:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.936+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8410f890 con 0x7fdd84107520 2024-04-20T11:42:40.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.937+0000 7fdd7be00700 1 -- 172.21.15.117:0/4035564567 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdd7400f7a0 con 0x7fdd84107520 2024-04-20T11:42:40.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.938+0000 7fdd7be00700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd6806ecd0 0x7fdd68071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:42:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.938+0000 7fdd7be00700 1 -- 172.21.15.117:0/4035564567 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdd740902a0 con 0x7fdd84107520 2024-04-20T11:42:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.938+0000 7fdd82a00700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd6806ecd0 0x7fdd68071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:42:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.939+0000 7fdd82a00700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd6806ecd0 0x7fdd68071190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdd8419e710 tx=0x7fdd70009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:42:40.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:40.943+0000 7fdd7be00700 1 -- 172.21.15.117:0/4035564567 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdd74059060 con 0x7fdd84107520 2024-04-20T11:42:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.247+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdd8419e4c0 con 0x7fdd84107520 2024-04-20T11:42:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.248+0000 7fdd7be00700 1 -- 172.21.15.117:0/4035564567 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fdd7405c720 con 0x7fdd84107520 2024-04-20T11:42:41.250 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:42:41.250 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:42:41.250 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:42:41.250 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:42:41.250 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:42:41.250 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:42:41.250 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:42:41.250 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:42:41.251 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:42:41.252 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:42:41.252 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:42:41.252 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:42:41.252 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:42:41.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.252+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd6806ecd0 msgr2=0x7fdd68071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:41.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.252+0000 7fdd8a200700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd6806ecd0 0x7fdd68071190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdd8419e710 tx=0x7fdd70009260 comp rx=0 tx=0).stop 2024-04-20T11:42:41.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.252+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 msgr2=0x7fdd8419d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:42:41.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.252+0000 7fdd8a200700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 0x7fdd8419d1b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdd74005f50 tx=0x7fdd7400b650 comp rx=0 tx=0).stop 2024-04-20T11:42:41.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.252+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 shutdown_connections 2024-04-20T11:42:41.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.252+0000 7fdd8a200700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdd6806ecd0 0x7fdd68071190 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.252+0000 7fdd8a200700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd84107520 0x7fdd8419d1b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.252+0000 7fdd8a200700 1 --2- 172.21.15.117:0/4035564567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84108e10 0x7fdd8419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:42:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.253+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 >> 172.21.15.117:0/4035564567 conn(0x7fdd840768f0 msgr2=0x7fdd84077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:42:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.253+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 shutdown_connections 2024-04-20T11:42:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:42:41.253+0000 7fdd8a200700 1 -- 172.21.15.117:0/4035564567 wait complete. 2024-04-20T11:42:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:41 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/830297843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:42:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:41 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4035564567' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:42:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:41 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/830297843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:42:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:41 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4035564567' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:42:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:42 smithi117 ceph-mon[99036]: from='client.35646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:42 smithi117 ceph-mon[99036]: pgmap v908: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:42 smithi173 ceph-mon[74165]: from='client.35646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:42:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:42 smithi173 ceph-mon[74165]: pgmap v908: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:43.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:42:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:42:44.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:44 smithi117 ceph-mon[99036]: pgmap v909: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:44 smithi173 ceph-mon[74165]: pgmap v909: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:46.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:46 smithi117 ceph-mon[99036]: pgmap v910: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:46 smithi173 ceph-mon[74165]: pgmap v910: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:48.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:48 smithi117 ceph-mon[99036]: pgmap v911: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:48 smithi173 ceph-mon[74165]: pgmap v911: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:50.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:50 smithi117 ceph-mon[99036]: pgmap v912: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:50 smithi173 ceph-mon[74165]: pgmap v912: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:52 smithi117 ceph-mon[99036]: pgmap v913: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:52 smithi173 ceph-mon[74165]: pgmap v913: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:54 smithi117 ceph-mon[99036]: pgmap v914: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:54 smithi173 ceph-mon[74165]: pgmap v914: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:56 smithi117 ceph-mon[99036]: pgmap v915: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:56 smithi173 ceph-mon[74165]: pgmap v915: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:58 smithi117 ceph-mon[99036]: pgmap v916: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:58.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:42:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:42:58.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:58 smithi173 ceph-mon[74165]: pgmap v916: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:42:58.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:42:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:43:00.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:00 smithi173 ceph-mon[74165]: pgmap v917: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:00.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:00 smithi117 ceph-mon[99036]: pgmap v917: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:02 smithi117 ceph-mon[99036]: pgmap v918: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:02 smithi173 ceph-mon[74165]: pgmap v918: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:04 smithi117 ceph-mon[99036]: pgmap v919: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:04 smithi173 ceph-mon[74165]: pgmap v919: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:06 smithi173 ceph-mon[74165]: pgmap v920: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:06.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:06 smithi117 ceph-mon[99036]: pgmap v920: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:08.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:08 smithi117 ceph-mon[99036]: pgmap v921: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:08 smithi173 ceph-mon[74165]: pgmap v921: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:10 smithi173 ceph-mon[74165]: pgmap v922: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:10.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:10 smithi117 ceph-mon[99036]: pgmap v922: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.424+0000 7f69e6600700 1 -- 172.21.15.117:0/2686722904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 msgr2=0x7f69e0105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.424+0000 7f69e6600700 1 --2- 172.21.15.117:0/2686722904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 0x7f69e0105810 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009910 tx=0x7f69c8009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.424+0000 7f69e6600700 1 -- 172.21.15.117:0/2686722904 shutdown_connections 2024-04-20T11:43:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.424+0000 7f69e6600700 1 --2- 172.21.15.117:0/2686722904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 0x7f69e0105810 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.424+0000 7f69e6600700 1 --2- 172.21.15.117:0/2686722904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e0100e10 0x7f69e01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.424+0000 7f69e6600700 1 -- 172.21.15.117:0/2686722904 >> 172.21.15.117:0/2686722904 conn(0x7f69e00fc6a0 msgr2=0x7f69e00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.424+0000 7f69e6600700 1 -- 172.21.15.117:0/2686722904 shutdown_connections 2024-04-20T11:43:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.424+0000 7f69e6600700 1 -- 172.21.15.117:0/2686722904 wait complete. 2024-04-20T11:43:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.426+0000 7f69e6600700 1 Processor -- start 2024-04-20T11:43:11.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.426+0000 7f69e6600700 1 -- start start 2024-04-20T11:43:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.426+0000 7f69e6600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e0100e10 0x7f69e019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.426+0000 7f69e6600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 0x7f69e019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:11.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.426+0000 7f69e6600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e019d360 con 0x7f69e01017c0 2024-04-20T11:43:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.426+0000 7f69e6600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e0196990 con 0x7f69e0100e10 2024-04-20T11:43:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.427+0000 7f69d7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 0x7f69e019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.427+0000 7f69d7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 0x7f69e019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51716/0 (socket says 172.21.15.117:51716) 2024-04-20T11:43:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.427+0000 7f69d7e00700 1 -- 172.21.15.117:0/403040617 learned_addr learned my addr 172.21.15.117:0/403040617 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.427+0000 7f69df400700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e0100e10 0x7f69e019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.427+0000 7f69d7e00700 1 -- 172.21.15.117:0/403040617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e0100e10 msgr2=0x7f69e019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.427+0000 7f69d7e00700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e0100e10 0x7f69e019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.427+0000 7f69d7e00700 1 -- 172.21.15.117:0/403040617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c80095f0 con 0x7f69e01017c0 2024-04-20T11:43:11.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.427+0000 7f69d7e00700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 0x7f69e019cd40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f69c8000c00 tx=0x7f69c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:11.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.428+0000 7f69dcc00700 1 -- 172.21.15.117:0/403040617 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69c801d070 con 0x7f69e01017c0 2024-04-20T11:43:11.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.428+0000 7f69dcc00700 1 -- 172.21.15.117:0/403040617 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69c8005700 con 0x7f69e01017c0 2024-04-20T11:43:11.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.428+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e0196c10 con 0x7f69e01017c0 2024-04-20T11:43:11.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.428+0000 7f69dcc00700 1 -- 172.21.15.117:0/403040617 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69c800fe50 con 0x7f69e01017c0 2024-04-20T11:43:11.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.428+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e0197100 con 0x7f69e01017c0 2024-04-20T11:43:11.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.429+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e0109210 con 0x7f69e01017c0 2024-04-20T11:43:11.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.430+0000 7f69dcc00700 1 -- 172.21.15.117:0/403040617 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f69c800f640 con 0x7f69e01017c0 2024-04-20T11:43:11.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.431+0000 7f69dcc00700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69c406ecd0 0x7f69c4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:11.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.431+0000 7f69dcc00700 1 -- 172.21.15.117:0/403040617 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f69c80902b0 con 0x7f69e01017c0 2024-04-20T11:43:11.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.431+0000 7f69df400700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69c406ecd0 0x7f69c4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:11.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.432+0000 7f69df400700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69c406ecd0 0x7f69c4071190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f69d0005950 tx=0x7f69d00058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:11.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.436+0000 7f69dcc00700 1 -- 172.21.15.117:0/403040617 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f69c805cba0 con 0x7f69e01017c0 2024-04-20T11:43:11.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.681+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69e0197e00 con 0x7f69c406ecd0 2024-04-20T11:43:11.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.683+0000 7f69dcc00700 1 -- 172.21.15.117:0/403040617 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f69e0197e00 con 0x7f69c406ecd0 2024-04-20T11:43:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69c406ecd0 msgr2=0x7f69c4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69c406ecd0 0x7f69c4071190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f69d0005950 tx=0x7f69d00058e0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 msgr2=0x7f69e019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 0x7f69e019cd40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f69c8000c00 tx=0x7f69c800be20 comp rx=0 tx=0).stop 2024-04-20T11:43:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 shutdown_connections 2024-04-20T11:43:11.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f69c406ecd0 0x7f69c4071190 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e0100e10 0x7f69e019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 --2- 172.21.15.117:0/403040617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e01017c0 0x7f69e019cd40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.686+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 >> 172.21.15.117:0/403040617 conn(0x7f69e00fc6a0 msgr2=0x7f69e00fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:11.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.687+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 shutdown_connections 2024-04-20T11:43:11.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.687+0000 7f69e6600700 1 -- 172.21.15.117:0/403040617 wait complete. 2024-04-20T11:43:11.701 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:43:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.844+0000 7f3003200700 1 -- 172.21.15.117:0/3705078579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 msgr2=0x7f2ffc1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.844+0000 7f3003200700 1 --2- 172.21.15.117:0/3705078579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 0x7f2ffc1034f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009910 tx=0x7f2fec009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.844+0000 7f3003200700 1 -- 172.21.15.117:0/3705078579 shutdown_connections 2024-04-20T11:43:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.844+0000 7f3003200700 1 --2- 172.21.15.117:0/3705078579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ffc104a00 0x7f2ffc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.844+0000 7f3003200700 1 --2- 172.21.15.117:0/3705078579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 0x7f2ffc1034f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.844+0000 7f3003200700 1 -- 172.21.15.117:0/3705078579 >> 172.21.15.117:0/3705078579 conn(0x7f2ffc0fe880 msgr2=0x7f2ffc100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.844+0000 7f3003200700 1 -- 172.21.15.117:0/3705078579 shutdown_connections 2024-04-20T11:43:11.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.844+0000 7f3003200700 1 -- 172.21.15.117:0/3705078579 wait complete. 2024-04-20T11:43:11.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.845+0000 7f3003200700 1 Processor -- start 2024-04-20T11:43:11.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.846+0000 7f3003200700 1 -- start start 2024-04-20T11:43:11.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.846+0000 7f3003200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 0x7f2ffc198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:11.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.846+0000 7f3003200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ffc104a00 0x7f2ffc199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:11.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.846+0000 7f3003200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc1999e0 con 0x7f2ffc103110 2024-04-20T11:43:11.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.846+0000 7f3003200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc19d660 con 0x7f2ffc104a00 2024-04-20T11:43:11.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.846+0000 7f2ffbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 0x7f2ffc198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:11.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.846+0000 7f2ffbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 0x7f2ffc198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51734/0 (socket says 172.21.15.117:51734) 2024-04-20T11:43:11.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.846+0000 7f2ffbe00700 1 -- 172.21.15.117:0/2954823694 learned_addr learned my addr 172.21.15.117:0/2954823694 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:11.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.847+0000 7f2ffb400700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ffc104a00 0x7f2ffc199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:11.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.847+0000 7f2ffbe00700 1 -- 172.21.15.117:0/2954823694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ffc104a00 msgr2=0x7f2ffc199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:11.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.847+0000 7f2ffbe00700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ffc104a00 0x7f2ffc199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:11.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.847+0000 7f2ffbe00700 1 -- 172.21.15.117:0/2954823694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec0095f0 con 0x7f2ffc103110 2024-04-20T11:43:11.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.847+0000 7f2ffb400700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ffc104a00 0x7f2ffc199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:43:11.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.847+0000 7f2ffbe00700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 0x7f2ffc198dc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0092e0 tx=0x7f2fec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:11.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.848+0000 7f2ff8c00700 1 -- 172.21.15.117:0/2954823694 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fec01d070 con 0x7f2ffc103110 2024-04-20T11:43:11.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.848+0000 7f2ff8c00700 1 -- 172.21.15.117:0/2954823694 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fec0056e0 con 0x7f2ffc103110 2024-04-20T11:43:11.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.848+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc19d8e0 con 0x7f2ffc103110 2024-04-20T11:43:11.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.848+0000 7f2ff8c00700 1 -- 172.21.15.117:0/2954823694 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fec00fe80 con 0x7f2ffc103110 2024-04-20T11:43:11.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.848+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc19de50 con 0x7f2ffc103110 2024-04-20T11:43:11.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.849+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc10b480 con 0x7f2ffc103110 2024-04-20T11:43:11.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.850+0000 7f2ff8c00700 1 -- 172.21.15.117:0/2954823694 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2fec00f6f0 con 0x7f2ffc103110 2024-04-20T11:43:11.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.851+0000 7f2ff8c00700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2fe006ecd0 0x7f2fe0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:11.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.851+0000 7f2ff8c00700 1 -- 172.21.15.117:0/2954823694 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2fec090350 con 0x7f2ffc103110 2024-04-20T11:43:11.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.851+0000 7f2ffb400700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2fe006ecd0 0x7f2fe0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:11.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.852+0000 7f2ffb400700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2fe006ecd0 0x7f2fe0071190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8005fd0 tx=0x7f2fe8005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:11.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:11.856+0000 7f2ff8c00700 1 -- 172.21.15.117:0/2954823694 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2fec059090 con 0x7f2ffc103110 2024-04-20T11:43:12.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.102+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ffc19a080 con 0x7f2fe006ecd0 2024-04-20T11:43:12.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.104+0000 7f2ff8c00700 1 -- 172.21.15.117:0/2954823694 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2ffc19a080 con 0x7f2fe006ecd0 2024-04-20T11:43:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2fe006ecd0 msgr2=0x7f2fe0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2fe006ecd0 0x7f2fe0071190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8005fd0 tx=0x7f2fe8005ee0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 msgr2=0x7f2ffc198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 0x7f2ffc198dc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0092e0 tx=0x7f2fec00be20 comp rx=0 tx=0).stop 2024-04-20T11:43:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 shutdown_connections 2024-04-20T11:43:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2fe006ecd0 0x7f2fe0071190 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ffc103110 0x7f2ffc198dc0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 --2- 172.21.15.117:0/2954823694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ffc104a00 0x7f2ffc199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.107+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 >> 172.21.15.117:0/2954823694 conn(0x7f2ffc0fe880 msgr2=0x7f2ffc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:12.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.108+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 shutdown_connections 2024-04-20T11:43:12.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.108+0000 7f3003200700 1 -- 172.21.15.117:0/2954823694 wait complete. 2024-04-20T11:43:12.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.257+0000 7faad3e00700 1 -- 172.21.15.117:0/4139824791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 msgr2=0x7faad4100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.257+0000 7faad3e00700 1 --2- 172.21.15.117:0/4139824791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 0x7faad4100ea0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7faabc009910 tx=0x7faabc009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:12.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.257+0000 7faad3e00700 1 -- 172.21.15.117:0/4139824791 shutdown_connections 2024-04-20T11:43:12.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.257+0000 7faad3e00700 1 --2- 172.21.15.117:0/4139824791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 0x7faad4100ea0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.257+0000 7faad3e00700 1 --2- 172.21.15.117:0/4139824791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad4106a20 0x7faad4106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.257+0000 7faad3e00700 1 -- 172.21.15.117:0/4139824791 >> 172.21.15.117:0/4139824791 conn(0x7faad40fc530 msgr2=0x7faad40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:12.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.257+0000 7faad3e00700 1 -- 172.21.15.117:0/4139824791 shutdown_connections 2024-04-20T11:43:12.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.258+0000 7faad3e00700 1 -- 172.21.15.117:0/4139824791 wait complete. 2024-04-20T11:43:12.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.259+0000 7faad3e00700 1 Processor -- start 2024-04-20T11:43:12.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.259+0000 7faad3e00700 1 -- start start 2024-04-20T11:43:12.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.259+0000 7faad3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 0x7faad40731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:12.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.259+0000 7faad3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad4106a20 0x7faad4073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:12.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.259+0000 7faad3e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad4073c60 con 0x7faad4100a20 2024-04-20T11:43:12.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.259+0000 7faad3e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad4073da0 con 0x7faad4106a20 2024-04-20T11:43:12.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 0x7faad40731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:12.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 0x7faad40731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51754/0 (socket says 172.21.15.117:51754) 2024-04-20T11:43:12.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2a00700 1 -- 172.21.15.117:0/3796160809 learned_addr learned my addr 172.21.15.117:0/3796160809 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:12.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2000700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad4106a20 0x7faad4073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:12.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2a00700 1 -- 172.21.15.117:0/3796160809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad4106a20 msgr2=0x7faad4073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2a00700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad4106a20 0x7faad4073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2a00700 1 -- 172.21.15.117:0/3796160809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0095f0 con 0x7faad4100a20 2024-04-20T11:43:12.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2000700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad4106a20 0x7faad4073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:43:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faad2a00700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 0x7faad40731e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faac400b4b0 tx=0x7faac400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.260+0000 7faacb400700 1 -- 172.21.15.117:0/3796160809 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faac4011800 con 0x7faad4100a20 2024-04-20T11:43:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.261+0000 7faacb400700 1 -- 172.21.15.117:0/3796160809 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faac4011e20 con 0x7faad4100a20 2024-04-20T11:43:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.261+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad41a2c40 con 0x7faad4100a20 2024-04-20T11:43:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.261+0000 7faacb400700 1 -- 172.21.15.117:0/3796160809 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faac400f2e0 con 0x7faad4100a20 2024-04-20T11:43:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.261+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad41a2fa0 con 0x7faad4100a20 2024-04-20T11:43:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.262+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad4109040 con 0x7faad4100a20 2024-04-20T11:43:12.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.263+0000 7faacb400700 1 -- 172.21.15.117:0/3796160809 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7faac4010380 con 0x7faad4100a20 2024-04-20T11:43:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.264+0000 7faacb400700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.264+0000 7faacb400700 1 -- 172.21.15.117:0/3796160809 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7faac408e9d0 con 0x7faad4100a20 2024-04-20T11:43:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.264+0000 7faad2000700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.265+0000 7faad2000700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faabc006010 tx=0x7faabc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:12.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.269+0000 7faacb400700 1 -- 172.21.15.117:0/3796160809 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7faac4057790 con 0x7faad4100a20 2024-04-20T11:43:12.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.496+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faad410dca0 con 0x7faab806ecd0 2024-04-20T11:43:12.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.515+0000 7faacb400700 1 -- 172.21.15.117:0/3796160809 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7faad410dca0 con 0x7faab806ecd0 2024-04-20T11:43:12.515 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:43:12.515 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (31m) 7m ago 45m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:43:12.515 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (46m) 7m ago 46m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:43:12.515 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (44m) 7m ago 44m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (27m) 7m ago 45m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (27m) 7m ago 44m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (30m) 7m ago 45m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (23m) 7m ago 38m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (23m) 7m ago 38m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (7m) 7m ago 38m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (23m) 7m ago 38m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (34m) 7m ago 47m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (33m) 7m ago 44m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (29m) 7m ago 47m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (29m) 7m ago 44m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (32m) 7m ago 45m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (32m) 7m ago 44m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:43:12.516 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (27m) 7m ago 43m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:43:12.517 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (26m) 7m ago 42m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:43:12.517 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (26m) 7m ago 42m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:43:12.517 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (25m) 7m ago 41m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:43:12.517 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (25m) 7m ago 40m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:43:12.517 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (24m) 7m ago 40m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:43:12.517 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (32m) 7m ago 45m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 msgr2=0x7faab8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faabc006010 tx=0x7faabc00b540 comp rx=0 tx=0).stop 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 msgr2=0x7faad40731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 0x7faad40731e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faac400b4b0 tx=0x7faac400b870 comp rx=0 tx=0).stop 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 shutdown_connections 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7faab806ecd0 0x7faab8071190 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad4100a20 0x7faad40731e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 --2- 172.21.15.117:0/3796160809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faad4106a20 0x7faad4073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 >> 172.21.15.117:0/3796160809 conn(0x7faad40fc530 msgr2=0x7faad4105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:12.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 shutdown_connections 2024-04-20T11:43:12.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.518+0000 7faad3e00700 1 -- 172.21.15.117:0/3796160809 wait complete. 2024-04-20T11:43:12.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.665+0000 7f92c6400700 1 -- 172.21.15.117:0/3096163681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073200 msgr2=0x7f92c00735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.665+0000 7f92c6400700 1 --2- 172.21.15.117:0/3096163681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073200 0x7f92c00735e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009910 tx=0x7f92b0009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:12.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.665+0000 7f92c6400700 1 -- 172.21.15.117:0/3096163681 shutdown_connections 2024-04-20T11:43:12.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.665+0000 7f92c6400700 1 --2- 172.21.15.117:0/3096163681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c0073b20 0x7f92c010cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.665 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:12 smithi117 ceph-mon[99036]: pgmap v923: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:12.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.665+0000 7f92c6400700 1 --2- 172.21.15.117:0/3096163681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073200 0x7f92c00735e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.665+0000 7f92c6400700 1 -- 172.21.15.117:0/3096163681 >> 172.21.15.117:0/3096163681 conn(0x7f92c00fc6a0 msgr2=0x7f92c00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.665+0000 7f92c6400700 1 -- 172.21.15.117:0/3096163681 shutdown_connections 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.665+0000 7f92c6400700 1 -- 172.21.15.117:0/3096163681 wait complete. 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.666+0000 7f92c6400700 1 Processor -- start 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.666+0000 7f92c6400700 1 -- start start 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.667+0000 7f92c6400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c0073200 0x7f92c019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.667+0000 7f92c6400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073b20 0x7f92c019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.667+0000 7f92c6400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c019f530 con 0x7f92c0073b20 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.667+0000 7f92c6400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c0198b10 con 0x7f92c0073200 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.667+0000 7f92bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073b20 0x7f92c019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.667+0000 7f92bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073b20 0x7f92c019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51772/0 (socket says 172.21.15.117:51772) 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.667+0000 7f92bea00700 1 -- 172.21.15.117:0/1515391907 learned_addr learned my addr 172.21.15.117:0/1515391907 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:12.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.667+0000 7f92bf400700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c0073200 0x7f92c019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:12.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92bea00700 1 -- 172.21.15.117:0/1515391907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c0073200 msgr2=0x7f92c019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92bea00700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c0073200 0x7f92c019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92bea00700 1 -- 172.21.15.117:0/1515391907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b00095f0 con 0x7f92c0073b20 2024-04-20T11:43:12.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92bea00700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073b20 0x7f92c019ef10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c8a0 tx=0x7f92ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:12.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92b7e00700 1 -- 172.21.15.117:0/1515391907 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92ac00cea0 con 0x7f92c0073b20 2024-04-20T11:43:12.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92b7e00700 1 -- 172.21.15.117:0/1515391907 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92ac004830 con 0x7f92c0073b20 2024-04-20T11:43:12.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92b7e00700 1 -- 172.21.15.117:0/1515391907 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92ac0055b0 con 0x7f92c0073b20 2024-04-20T11:43:12.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c0198df0 con 0x7f92c0073b20 2024-04-20T11:43:12.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.668+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c0199340 con 0x7f92c0073b20 2024-04-20T11:43:12.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.670+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c01077d0 con 0x7f92c0073b20 2024-04-20T11:43:12.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.672+0000 7f92b7e00700 1 -- 172.21.15.117:0/1515391907 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f92ac0049a0 con 0x7f92c0073b20 2024-04-20T11:43:12.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.673+0000 7f92b7e00700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92a406ecd0 0x7f92a4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:12.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.673+0000 7f92b7e00700 1 -- 172.21.15.117:0/1515391907 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f92ac08ea20 con 0x7f92c0073b20 2024-04-20T11:43:12.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.673+0000 7f92bf400700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92a406ecd0 0x7f92a4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:12.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.674+0000 7f92bf400700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92a406ecd0 0x7f92a4071190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92b000b5c0 tx=0x7f92b001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:12.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.678+0000 7f92b7e00700 1 -- 172.21.15.117:0/1515391907 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f92ac05b3b0 con 0x7f92c0073b20 2024-04-20T11:43:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:12 smithi173 ceph-mon[74165]: pgmap v923: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:12.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.969+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f92c0074940 con 0x7f92c0073b20 2024-04-20T11:43:12.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.970+0000 7f92b7e00700 1 -- 172.21.15.117:0/1515391907 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f92ac01d070 con 0x7f92c0073b20 2024-04-20T11:43:12.970 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:43:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:43:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:43:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:43:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:43:12.971 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:43:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92a406ecd0 msgr2=0x7f92a4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92a406ecd0 0x7f92a4071190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92b000b5c0 tx=0x7f92b001a040 comp rx=0 tx=0).stop 2024-04-20T11:43:12.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073b20 msgr2=0x7f92c019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:12.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073b20 0x7f92c019ef10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c8a0 tx=0x7f92ac00cc60 comp rx=0 tx=0).stop 2024-04-20T11:43:12.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 shutdown_connections 2024-04-20T11:43:12.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92a406ecd0 0x7f92a4071190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c0073200 0x7f92c019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 --2- 172.21.15.117:0/1515391907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c0073b20 0x7f92c019ef10 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:12.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 >> 172.21.15.117:0/1515391907 conn(0x7f92c00fc6a0 msgr2=0x7f92c0105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:12.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.973+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 shutdown_connections 2024-04-20T11:43:12.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:12.974+0000 7f92c6400700 1 -- 172.21.15.117:0/1515391907 wait complete. 2024-04-20T11:43:13.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.128+0000 7fb521a00700 1 -- 172.21.15.117:0/3987376353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c104a00 msgr2=0x7fb51c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.128+0000 7fb521a00700 1 --2- 172.21.15.117:0/3987376353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c104a00 0x7fb51c104e80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb508009910 tx=0x7fb508009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:13.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.128+0000 7fb521a00700 1 -- 172.21.15.117:0/3987376353 shutdown_connections 2024-04-20T11:43:13.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.128+0000 7fb521a00700 1 --2- 172.21.15.117:0/3987376353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c104a00 0x7fb51c104e80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.128+0000 7fb521a00700 1 --2- 172.21.15.117:0/3987376353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb51c103110 0x7fb51c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.128+0000 7fb521a00700 1 -- 172.21.15.117:0/3987376353 >> 172.21.15.117:0/3987376353 conn(0x7fb51c0fe880 msgr2=0x7fb51c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:13.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.129+0000 7fb521a00700 1 -- 172.21.15.117:0/3987376353 shutdown_connections 2024-04-20T11:43:13.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.129+0000 7fb521a00700 1 -- 172.21.15.117:0/3987376353 wait complete. 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.130+0000 7fb521a00700 1 Processor -- start 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.130+0000 7fb521a00700 1 -- start start 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.130+0000 7fb521a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c103110 0x7fb51c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.130+0000 7fb521a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb51c104a00 0x7fb51c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.130+0000 7fb521a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c19f5b0 con 0x7fb51c103110 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.130+0000 7fb521a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c198b90 con 0x7fb51c104a00 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.130+0000 7fb51aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c103110 0x7fb51c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.131+0000 7fb51aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c103110 0x7fb51c19ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51790/0 (socket says 172.21.15.117:51790) 2024-04-20T11:43:13.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.131+0000 7fb51aa00700 1 -- 172.21.15.117:0/3408790382 learned_addr learned my addr 172.21.15.117:0/3408790382 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:13.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.131+0000 7fb51aa00700 1 -- 172.21.15.117:0/3408790382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb51c104a00 msgr2=0x7fb51c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.131+0000 7fb51a000700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb51c104a00 0x7fb51c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:13.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.131+0000 7fb51aa00700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb51c104a00 0x7fb51c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.131+0000 7fb51aa00700 1 -- 172.21.15.117:0/3408790382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5080095f0 con 0x7fb51c103110 2024-04-20T11:43:13.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.131+0000 7fb51aa00700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c103110 0x7fb51c19ea50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c8a0 tx=0x7fb50c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:13.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.132+0000 7fb513400700 1 -- 172.21.15.117:0/3408790382 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50c00cea0 con 0x7fb51c103110 2024-04-20T11:43:13.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.132+0000 7fb513400700 1 -- 172.21.15.117:0/3408790382 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb50c004830 con 0x7fb51c103110 2024-04-20T11:43:13.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.132+0000 7fb513400700 1 -- 172.21.15.117:0/3408790382 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50c0055b0 con 0x7fb51c103110 2024-04-20T11:43:13.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.132+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c198e70 con 0x7fb51c103110 2024-04-20T11:43:13.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.132+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c199440 con 0x7fb51c103110 2024-04-20T11:43:13.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.133+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c10b480 con 0x7fb51c103110 2024-04-20T11:43:13.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.134+0000 7fb513400700 1 -- 172.21.15.117:0/3408790382 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb50c0049a0 con 0x7fb51c103110 2024-04-20T11:43:13.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.135+0000 7fb513400700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb50006ecd0 0x7fb500071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.135+0000 7fb513400700 1 -- 172.21.15.117:0/3408790382 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb50c08ef10 con 0x7fb51c103110 2024-04-20T11:43:13.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.136+0000 7fb51a000700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb50006ecd0 0x7fb500071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:13.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.136+0000 7fb51a000700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb50006ecd0 0x7fb500071190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb508006010 tx=0x7fb50800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:13.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.139+0000 7fb513400700 1 -- 172.21.15.117:0/3408790382 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb50c05b880 con 0x7fb51c103110 2024-04-20T11:43:13.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.414+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb51c199fe0 con 0x7fb51c103110 2024-04-20T11:43:13.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.415+0000 7fb513400700 1 -- 172.21.15.117:0/3408790382 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fb50c01d070 con 0x7fb51c103110 2024-04-20T11:43:13.415 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:43:13.415 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:43:13.415 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:43:13.416 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:43:13.417 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:43:13.418 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:43:13.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.418+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb50006ecd0 msgr2=0x7fb500071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.418+0000 7fb521a00700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb50006ecd0 0x7fb500071190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb508006010 tx=0x7fb50800b540 comp rx=0 tx=0).stop 2024-04-20T11:43:13.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.418+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c103110 msgr2=0x7fb51c19ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.418+0000 7fb521a00700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c103110 0x7fb51c19ea50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c8a0 tx=0x7fb50c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:43:13.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.419+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 shutdown_connections 2024-04-20T11:43:13.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.419+0000 7fb521a00700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb50006ecd0 0x7fb500071190 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.419+0000 7fb521a00700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb51c103110 0x7fb51c19ea50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.419+0000 7fb521a00700 1 --2- 172.21.15.117:0/3408790382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb51c104a00 0x7fb51c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.419+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 >> 172.21.15.117:0/3408790382 conn(0x7fb51c0fe880 msgr2=0x7fb51c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:13.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.419+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 shutdown_connections 2024-04-20T11:43:13.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.419+0000 7fb521a00700 1 -- 172.21.15.117:0/3408790382 wait complete. 2024-04-20T11:43:13.420 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:43:13.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.565+0000 7f324ee00700 1 -- 172.21.15.117:0/1577342481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 msgr2=0x7f32480736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.565+0000 7f324ee00700 1 --2- 172.21.15.117:0/1577342481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 0x7f32480736f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3238009910 tx=0x7f3238009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:13.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.565+0000 7f324ee00700 1 -- 172.21.15.117:0/1577342481 shutdown_connections 2024-04-20T11:43:13.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.565+0000 7f324ee00700 1 --2- 172.21.15.117:0/1577342481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3248073c30 0x7f324810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.565+0000 7f324ee00700 1 --2- 172.21.15.117:0/1577342481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 0x7f32480736f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.565+0000 7f324ee00700 1 -- 172.21.15.117:0/1577342481 >> 172.21.15.117:0/1577342481 conn(0x7f32480fc6c0 msgr2=0x7f32480feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:13.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.565+0000 7f324ee00700 1 -- 172.21.15.117:0/1577342481 shutdown_connections 2024-04-20T11:43:13.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.565+0000 7f324ee00700 1 -- 172.21.15.117:0/1577342481 wait complete. 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.566+0000 7f324ee00700 1 Processor -- start 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.566+0000 7f324ee00700 1 -- start start 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f324ee00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 0x7f3248198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f324ee00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3248073c30 0x7f3248199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f324ee00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248199a20 con 0x7f3248073310 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f324ee00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819d6f0 con 0x7f3248073c30 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f3247e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 0x7f3248198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f3247e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 0x7f3248198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51810/0 (socket says 172.21.15.117:51810) 2024-04-20T11:43:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f3247e00700 1 -- 172.21.15.117:0/2221112589 learned_addr learned my addr 172.21.15.117:0/2221112589 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:13.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f3247e00700 1 -- 172.21.15.117:0/2221112589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3248073c30 msgr2=0x7f3248199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f3247e00700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3248073c30 0x7f3248199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.567+0000 7f3247e00700 1 -- 172.21.15.117:0/2221112589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380095f0 con 0x7f3248073310 2024-04-20T11:43:13.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.568+0000 7f3247e00700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 0x7f3248198e00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3238006010 tx=0x7f323800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:13.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.568+0000 7f3245600700 1 -- 172.21.15.117:0/2221112589 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f323801d070 con 0x7f3248073310 2024-04-20T11:43:13.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.568+0000 7f3245600700 1 -- 172.21.15.117:0/2221112589 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f323800f460 con 0x7f3248073310 2024-04-20T11:43:13.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.568+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324819d970 con 0x7f3248073310 2024-04-20T11:43:13.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.568+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324819de60 con 0x7f3248073310 2024-04-20T11:43:13.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.568+0000 7f3245600700 1 -- 172.21.15.117:0/2221112589 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3238017630 con 0x7f3248073310 2024-04-20T11:43:13.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.569+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32480749c0 con 0x7f3248073310 2024-04-20T11:43:13.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.570+0000 7f3245600700 1 -- 172.21.15.117:0/2221112589 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f323800f5d0 con 0x7f3248073310 2024-04-20T11:43:13.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.571+0000 7f3245600700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f322c06ecc0 0x7f322c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.571+0000 7f3245600700 1 -- 172.21.15.117:0/2221112589 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3238090370 con 0x7f3248073310 2024-04-20T11:43:13.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.572+0000 7f323f400700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f322c06ecc0 0x7f322c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:13.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.572+0000 7f323f400700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f322c06ecc0 0x7f322c071180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f324819a360 tx=0x7f3230006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:13.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.576+0000 7f3245600700 1 -- 172.21.15.117:0/2221112589 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3238059170 con 0x7f3248073310 2024-04-20T11:43:13.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.806+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32481077c0 con 0x7f322c06ecc0 2024-04-20T11:43:13.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:13 smithi117 ceph-mon[99036]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:13.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:13 smithi117 ceph-mon[99036]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:13.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:13 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1515391907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:43:13.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:43:13.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.809+0000 7f3245600700 1 -- 172.21.15.117:0/2221112589 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f32481077c0 con 0x7f322c06ecc0 2024-04-20T11:43:13.808 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:43:13.808 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:43:13.809 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:43:13.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.811+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f322c06ecc0 msgr2=0x7f322c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.811+0000 7f324ee00700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f322c06ecc0 0x7f322c071180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f324819a360 tx=0x7f3230006d20 comp rx=0 tx=0).stop 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.811+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 msgr2=0x7f3248198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.811+0000 7f324ee00700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 0x7f3248198e00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3238006010 tx=0x7f323800be20 comp rx=0 tx=0).stop 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.812+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 shutdown_connections 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.812+0000 7f324ee00700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f322c06ecc0 0x7f322c071180 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.812+0000 7f324ee00700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3248073310 0x7f3248198e00 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.812+0000 7f324ee00700 1 --2- 172.21.15.117:0/2221112589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3248073c30 0x7f3248199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.812+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 >> 172.21.15.117:0/2221112589 conn(0x7f32480fc6c0 msgr2=0x7f3248105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.812+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 shutdown_connections 2024-04-20T11:43:13.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.812+0000 7f324ee00700 1 -- 172.21.15.117:0/2221112589 wait complete. 2024-04-20T11:43:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:13 smithi173 ceph-mon[74165]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:13 smithi173 ceph-mon[74165]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:13 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1515391907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:43:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:43:13.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.955+0000 7f6640e00700 1 -- 172.21.15.117:0/2675204094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 msgr2=0x7f663c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.955+0000 7f6640e00700 1 --2- 172.21.15.117:0/2675204094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 0x7f663c104e80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6628009910 tx=0x7f6628009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.955+0000 7f6640e00700 1 -- 172.21.15.117:0/2675204094 shutdown_connections 2024-04-20T11:43:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.955+0000 7f6640e00700 1 --2- 172.21.15.117:0/2675204094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 0x7f663c104e80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.955+0000 7f6640e00700 1 --2- 172.21.15.117:0/2675204094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f663c103110 0x7f663c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.955+0000 7f6640e00700 1 -- 172.21.15.117:0/2675204094 >> 172.21.15.117:0/2675204094 conn(0x7f663c0fe880 msgr2=0x7f663c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.955+0000 7f6640e00700 1 -- 172.21.15.117:0/2675204094 shutdown_connections 2024-04-20T11:43:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.955+0000 7f6640e00700 1 -- 172.21.15.117:0/2675204094 wait complete. 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.956+0000 7f6640e00700 1 Processor -- start 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.956+0000 7f6640e00700 1 -- start start 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.956+0000 7f6640e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f663c103110 0x7f663c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.956+0000 7f6640e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 0x7f663c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.956+0000 7f6640e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c199ac0 con 0x7f663c104a00 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.956+0000 7f6640e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c19d740 con 0x7f663c103110 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 0x7f663c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 0x7f663c1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48266/0 (socket says 172.21.15.117:48266) 2024-04-20T11:43:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639400700 1 -- 172.21.15.117:0/2255420749 learned_addr learned my addr 172.21.15.117:0/2255420749 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f663c103110 0x7f663c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639400700 1 -- 172.21.15.117:0/2255420749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f663c103110 msgr2=0x7f663c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639400700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f663c103110 0x7f663c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639400700 1 -- 172.21.15.117:0/2255420749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66280095f0 con 0x7f663c104a00 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f663c103110 0x7f663c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.957+0000 7f6639400700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 0x7f663c1993e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f662800bd70 tx=0x7f662800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.958+0000 7f6632a00700 1 -- 172.21.15.117:0/2255420749 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f662801d070 con 0x7f663c104a00 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.958+0000 7f6632a00700 1 -- 172.21.15.117:0/2255420749 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f662800f460 con 0x7f663c104a00 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.958+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c19d9c0 con 0x7f663c104a00 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.958+0000 7f6632a00700 1 -- 172.21.15.117:0/2255420749 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6628017630 con 0x7f663c104a00 2024-04-20T11:43:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.958+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c19df30 con 0x7f663c104a00 2024-04-20T11:43:13.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.959+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c10b480 con 0x7f663c104a00 2024-04-20T11:43:13.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.960+0000 7f6632a00700 1 -- 172.21.15.117:0/2255420749 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f662800f5d0 con 0x7f663c104a00 2024-04-20T11:43:13.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.961+0000 7f6632a00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f662006ecc0 0x7f6620071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:13.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.961+0000 7f6632a00700 1 -- 172.21.15.117:0/2255420749 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6628090420 con 0x7f663c104a00 2024-04-20T11:43:13.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.961+0000 7f6639e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f662006ecc0 0x7f6620071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:13.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.962+0000 7f6639e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f662006ecc0 0x7f6620071180 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f662c007900 tx=0x7f662c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:13.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:13.966+0000 7f6632a00700 1 -- 172.21.15.117:0/2255420749 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6628059160 con 0x7f663c104a00 2024-04-20T11:43:14.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.272+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f663c104510 con 0x7f663c104a00 2024-04-20T11:43:14.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.272+0000 7f6632a00700 1 -- 172.21.15.117:0/2255420749 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f662805c820 con 0x7f663c104a00 2024-04-20T11:43:14.272 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:43:14.272 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:43:14.272 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:43:14.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:43:14.272 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:43:14.273 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:43:14.274 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:43:14.274 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:43:14.274 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:43:14.274 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:43:14.274 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:43:14.274 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:43:14.274 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f662006ecc0 msgr2=0x7f6620071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f662006ecc0 0x7f6620071180 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f662c007900 tx=0x7f662c008040 comp rx=0 tx=0).stop 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 msgr2=0x7f663c1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 0x7f663c1993e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f662800bd70 tx=0x7f662800be50 comp rx=0 tx=0).stop 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 shutdown_connections 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f662006ecc0 0x7f6620071180 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f663c103110 0x7f663c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 --2- 172.21.15.117:0/2255420749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f663c104a00 0x7f663c1993e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:14.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.275+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 >> 172.21.15.117:0/2255420749 conn(0x7f663c0fe880 msgr2=0x7f663c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:14.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.276+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 shutdown_connections 2024-04-20T11:43:14.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:14.276+0000 7f6640e00700 1 -- 172.21.15.117:0/2255420749 wait complete. 2024-04-20T11:43:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:14 smithi117 ceph-mon[99036]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:14 smithi117 ceph-mon[99036]: pgmap v924: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:14 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3408790382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:43:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:14 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2255420749' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:43:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:14 smithi173 ceph-mon[74165]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:14 smithi173 ceph-mon[74165]: pgmap v924: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:14 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3408790382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:43:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:14 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2255420749' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:43:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:15 smithi117 ceph-mon[99036]: from='client.35674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:15 smithi173 ceph-mon[74165]: from='client.35674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:16 smithi117 ceph-mon[99036]: pgmap v925: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:16 smithi173 ceph-mon[74165]: pgmap v925: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:18.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:18 smithi117 ceph-mon[99036]: pgmap v926: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:18 smithi173 ceph-mon[74165]: pgmap v926: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:20.795 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:20 smithi173 ceph-mon[74165]: pgmap v927: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:20.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:20 smithi117 ceph-mon[99036]: pgmap v927: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:22 smithi117 ceph-mon[99036]: pgmap v928: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:22 smithi173 ceph-mon[74165]: pgmap v928: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:24.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:24 smithi117 ceph-mon[99036]: pgmap v929: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:24 smithi173 ceph-mon[74165]: pgmap v929: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:26.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:26 smithi117 ceph-mon[99036]: pgmap v930: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:26 smithi173 ceph-mon[74165]: pgmap v930: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:28 smithi117 ceph-mon[99036]: pgmap v931: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:43:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:28 smithi173 ceph-mon[74165]: pgmap v931: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:43:30.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:30 smithi117 ceph-mon[99036]: pgmap v932: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:30.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:30 smithi173 ceph-mon[74165]: pgmap v932: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:31.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:31 smithi117 ceph-mon[99036]: pgmap v933: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:31.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:31 smithi173 ceph-mon[74165]: pgmap v933: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:34 smithi117 ceph-mon[99036]: pgmap v934: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:34 smithi173 ceph-mon[74165]: pgmap v934: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:36.463 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:36 smithi173 ceph-mon[74165]: pgmap v935: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:36.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:36 smithi117 ceph-mon[99036]: pgmap v935: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:38 smithi117 ceph-mon[99036]: pgmap v936: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:38 smithi173 ceph-mon[74165]: pgmap v936: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:39 smithi173 ceph-mon[74165]: pgmap v937: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:40.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:39 smithi117 ceph-mon[99036]: pgmap v937: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:42 smithi117 ceph-mon[99036]: pgmap v938: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:42 smithi173 ceph-mon[74165]: pgmap v938: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:43:43.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:43:44.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.450+0000 7f2054a00700 1 -- 172.21.15.117:0/4116820345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 msgr2=0x7f2050109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:44.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.450+0000 7f2054a00700 1 --2- 172.21.15.117:0/4116820345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 0x7f2050109290 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f203c009910 tx=0x7f203c009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:44.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.450+0000 7f2054a00700 1 -- 172.21.15.117:0/4116820345 shutdown_connections 2024-04-20T11:43:44.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.450+0000 7f2054a00700 1 --2- 172.21.15.117:0/4116820345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 0x7f2050109290 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.450+0000 7f2054a00700 1 --2- 172.21.15.117:0/4116820345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050107520 0x7f2050107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.450+0000 7f2054a00700 1 -- 172.21.15.117:0/4116820345 >> 172.21.15.117:0/4116820345 conn(0x7f20500768f0 msgr2=0x7f2050078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:44.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.450+0000 7f2054a00700 1 -- 172.21.15.117:0/4116820345 shutdown_connections 2024-04-20T11:43:44.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.450+0000 7f2054a00700 1 -- 172.21.15.117:0/4116820345 wait complete. 2024-04-20T11:43:44.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.451+0000 7f2054a00700 1 Processor -- start 2024-04-20T11:43:44.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.451+0000 7f2054a00700 1 -- start start 2024-04-20T11:43:44.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.452+0000 7f2054a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050107520 0x7f205019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:44.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.452+0000 7f2054a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 0x7f205019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:44.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.452+0000 7f2054a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205019dd60 con 0x7f2050108e10 2024-04-20T11:43:44.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.452+0000 7f2054a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205019dea0 con 0x7f2050107520 2024-04-20T11:43:44.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.452+0000 7f204d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 0x7f205019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:44.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.452+0000 7f204d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 0x7f205019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53632/0 (socket says 172.21.15.117:53632) 2024-04-20T11:43:44.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.452+0000 7f204d400700 1 -- 172.21.15.117:0/1961845701 learned_addr learned my addr 172.21.15.117:0/1961845701 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:44.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.453+0000 7f204de00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050107520 0x7f205019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:44.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.453+0000 7f204d400700 1 -- 172.21.15.117:0/1961845701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050107520 msgr2=0x7f205019d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:44.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.453+0000 7f204d400700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050107520 0x7f205019d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.453+0000 7f204d400700 1 -- 172.21.15.117:0/1961845701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0095f0 con 0x7f2050108e10 2024-04-20T11:43:44.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.453+0000 7f204d400700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 0x7f205019d710 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f203c0092e0 tx=0x7f203c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:44.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.454+0000 7f2046a00700 1 -- 172.21.15.117:0/1961845701 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203c01d070 con 0x7f2050108e10 2024-04-20T11:43:44.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.454+0000 7f2046a00700 1 -- 172.21.15.117:0/1961845701 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f203c00f460 con 0x7f2050108e10 2024-04-20T11:43:44.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.454+0000 7f2046a00700 1 -- 172.21.15.117:0/1961845701 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203c017630 con 0x7f2050108e10 2024-04-20T11:43:44.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.454+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20501a1b80 con 0x7f2050108e10 2024-04-20T11:43:44.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.454+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501a20f0 con 0x7f2050108e10 2024-04-20T11:43:44.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.455+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205010f870 con 0x7f2050108e10 2024-04-20T11:43:44.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.456+0000 7f2046a00700 1 -- 172.21.15.117:0/1961845701 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f203c00f5d0 con 0x7f2050108e10 2024-04-20T11:43:44.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.457+0000 7f2046a00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f203406ecd0 0x7f2034071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:44.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.457+0000 7f2046a00700 1 -- 172.21.15.117:0/1961845701 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f203c090320 con 0x7f2050108e10 2024-04-20T11:43:44.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.457+0000 7f204de00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f203406ecd0 0x7f2034071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:44.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.458+0000 7f204de00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f203406ecd0 0x7f2034071190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2040005ea0 tx=0x7f2040005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:44.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.462+0000 7f2046a00700 1 -- 172.21.15.117:0/1961845701 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f203c0590e0 con 0x7f2050108e10 2024-04-20T11:43:44.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:44 smithi117 ceph-mon[99036]: pgmap v939: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:44 smithi173 ceph-mon[74165]: pgmap v939: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:44.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.711+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f205019e500 con 0x7f203406ecd0 2024-04-20T11:43:44.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.712+0000 7f2046a00700 1 -- 172.21.15.117:0/1961845701 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f205019e500 con 0x7f203406ecd0 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f203406ecd0 msgr2=0x7f2034071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f203406ecd0 0x7f2034071190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2040005ea0 tx=0x7f2040005e30 comp rx=0 tx=0).stop 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 msgr2=0x7f205019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 0x7f205019d710 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f203c0092e0 tx=0x7f203c00be20 comp rx=0 tx=0).stop 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 shutdown_connections 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f203406ecd0 0x7f2034071190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050107520 0x7f205019d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 --2- 172.21.15.117:0/1961845701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2050108e10 0x7f205019d710 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 >> 172.21.15.117:0/1961845701 conn(0x7f20500768f0 msgr2=0x7f2050077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 shutdown_connections 2024-04-20T11:43:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.715+0000 7f2054a00700 1 -- 172.21.15.117:0/1961845701 wait complete. 2024-04-20T11:43:44.732 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:43:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.878+0000 7f495a400700 1 -- 172.21.15.117:0/1605960218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4954100e10 msgr2=0x7f49541011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.878+0000 7f495a400700 1 --2- 172.21.15.117:0/1605960218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4954100e10 0x7f49541011f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4944009910 tx=0x7f4944009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.878+0000 7f495a400700 1 -- 172.21.15.117:0/1605960218 shutdown_connections 2024-04-20T11:43:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.878+0000 7f495a400700 1 --2- 172.21.15.117:0/1605960218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49541017c0 0x7f4954105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.878+0000 7f495a400700 1 --2- 172.21.15.117:0/1605960218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4954100e10 0x7f49541011f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.878+0000 7f495a400700 1 -- 172.21.15.117:0/1605960218 >> 172.21.15.117:0/1605960218 conn(0x7f49540fc6c0 msgr2=0x7f49540feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.878+0000 7f495a400700 1 -- 172.21.15.117:0/1605960218 shutdown_connections 2024-04-20T11:43:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.878+0000 7f495a400700 1 -- 172.21.15.117:0/1605960218 wait complete. 2024-04-20T11:43:44.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.879+0000 7f495a400700 1 Processor -- start 2024-04-20T11:43:44.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.879+0000 7f495a400700 1 -- start start 2024-04-20T11:43:44.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.880+0000 7f495a400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4954100e10 0x7f4954198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:44.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.880+0000 7f495a400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49541017c0 0x7f4954199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:44.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.880+0000 7f495a400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49541998d0 con 0x7f49541017c0 2024-04-20T11:43:44.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.880+0000 7f495a400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495419c540 con 0x7f4954100e10 2024-04-20T11:43:44.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.880+0000 7f494aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49541017c0 0x7f4954199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:44.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.880+0000 7f494aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49541017c0 0x7f4954199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53652/0 (socket says 172.21.15.117:53652) 2024-04-20T11:43:44.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.880+0000 7f494aa00700 1 -- 172.21.15.117:0/4031396810 learned_addr learned my addr 172.21.15.117:0/4031396810 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f4953400700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4954100e10 0x7f4954198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f494aa00700 1 -- 172.21.15.117:0/4031396810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4954100e10 msgr2=0x7f4954198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f494aa00700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4954100e10 0x7f4954198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f494aa00700 1 -- 172.21.15.117:0/4031396810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440095f0 con 0x7f49541017c0 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f494aa00700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49541017c0 0x7f4954199240 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f493c00d900 tx=0x7f493c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f4950c00700 1 -- 172.21.15.117:0/4031396810 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f493c009900 con 0x7f49541017c0 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f4950c00700 1 -- 172.21.15.117:0/4031396810 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f493c010460 con 0x7f49541017c0 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f4950c00700 1 -- 172.21.15.117:0/4031396810 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f493c00f5f0 con 0x7f49541017c0 2024-04-20T11:43:44.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495419c820 con 0x7f49541017c0 2024-04-20T11:43:44.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.881+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495419cd70 con 0x7f49541017c0 2024-04-20T11:43:44.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.882+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4954103f80 con 0x7f49541017c0 2024-04-20T11:43:44.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.884+0000 7f4950c00700 1 -- 172.21.15.117:0/4031396810 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f493c0105d0 con 0x7f49541017c0 2024-04-20T11:43:44.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.885+0000 7f4950c00700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f493806ecc0 0x7f4938071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:44.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.885+0000 7f4950c00700 1 -- 172.21.15.117:0/4031396810 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f493c08ec70 con 0x7f49541017c0 2024-04-20T11:43:44.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.885+0000 7f4953400700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f493806ecc0 0x7f4938071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:44.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.886+0000 7f4953400700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f493806ecc0 0x7f4938071180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4944006010 tx=0x7f494400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:44.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:44.889+0000 7f4950c00700 1 -- 172.21.15.117:0/4031396810 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f493c095080 con 0x7f49541017c0 2024-04-20T11:43:45.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.136+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4954199fd0 con 0x7f493806ecc0 2024-04-20T11:43:45.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.138+0000 7f4950c00700 1 -- 172.21.15.117:0/4031396810 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4954199fd0 con 0x7f493806ecc0 2024-04-20T11:43:45.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.141+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f493806ecc0 msgr2=0x7f4938071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:45.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.141+0000 7f495a400700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f493806ecc0 0x7f4938071180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4944006010 tx=0x7f494400b540 comp rx=0 tx=0).stop 2024-04-20T11:43:45.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.141+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49541017c0 msgr2=0x7f4954199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:45.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.141+0000 7f495a400700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49541017c0 0x7f4954199240 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f493c00d900 tx=0x7f493c00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.142+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 shutdown_connections 2024-04-20T11:43:45.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.142+0000 7f495a400700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f493806ecc0 0x7f4938071180 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.142+0000 7f495a400700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4954100e10 0x7f4954198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.142+0000 7f495a400700 1 --2- 172.21.15.117:0/4031396810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49541017c0 0x7f4954199240 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.142+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 >> 172.21.15.117:0/4031396810 conn(0x7f49540fc6c0 msgr2=0x7f49540fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:45.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.142+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 shutdown_connections 2024-04-20T11:43:45.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.142+0000 7f495a400700 1 -- 172.21.15.117:0/4031396810 wait complete. 2024-04-20T11:43:45.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.295+0000 7f88a7200700 1 -- 172.21.15.117:0/2507163152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 msgr2=0x7f88a01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:45.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.295+0000 7f88a7200700 1 --2- 172.21.15.117:0/2507163152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 0x7f88a01034f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8890009910 tx=0x7f8890009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:45.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.295+0000 7f88a7200700 1 -- 172.21.15.117:0/2507163152 shutdown_connections 2024-04-20T11:43:45.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.295+0000 7f88a7200700 1 --2- 172.21.15.117:0/2507163152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88a0104a00 0x7f88a0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.295+0000 7f88a7200700 1 --2- 172.21.15.117:0/2507163152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 0x7f88a01034f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.295+0000 7f88a7200700 1 -- 172.21.15.117:0/2507163152 >> 172.21.15.117:0/2507163152 conn(0x7f88a00fe880 msgr2=0x7f88a0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:45.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.295+0000 7f88a7200700 1 -- 172.21.15.117:0/2507163152 shutdown_connections 2024-04-20T11:43:45.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.296+0000 7f88a7200700 1 -- 172.21.15.117:0/2507163152 wait complete. 2024-04-20T11:43:45.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.297+0000 7f88a7200700 1 Processor -- start 2024-04-20T11:43:45.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.297+0000 7f88a7200700 1 -- start start 2024-04-20T11:43:45.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.297+0000 7f88a7200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 0x7f88a0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:45.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.297+0000 7f88a7200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88a0104a00 0x7f88a0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:45.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.297+0000 7f88a7200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a01999e0 con 0x7f88a0103110 2024-04-20T11:43:45.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.297+0000 7f88a7200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a019d660 con 0x7f88a0104a00 2024-04-20T11:43:45.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.298+0000 7f889fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 0x7f88a0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:45.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.298+0000 7f889fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 0x7f88a0198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53666/0 (socket says 172.21.15.117:53666) 2024-04-20T11:43:45.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.298+0000 7f889fe00700 1 -- 172.21.15.117:0/467362209 learned_addr learned my addr 172.21.15.117:0/467362209 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:45.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.298+0000 7f889f400700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88a0104a00 0x7f88a0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.298+0000 7f889fe00700 1 -- 172.21.15.117:0/467362209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88a0104a00 msgr2=0x7f88a0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.298+0000 7f889fe00700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88a0104a00 0x7f88a0199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.298+0000 7f889fe00700 1 -- 172.21.15.117:0/467362209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88900095f0 con 0x7f88a0103110 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.299+0000 7f889fe00700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 0x7f88a0198dc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f88900092e0 tx=0x7f889000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.299+0000 7f889cc00700 1 -- 172.21.15.117:0/467362209 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889001d070 con 0x7f88a0103110 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.299+0000 7f889cc00700 1 -- 172.21.15.117:0/467362209 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f889000f460 con 0x7f88a0103110 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.299+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a019d8e0 con 0x7f88a0103110 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.299+0000 7f889cc00700 1 -- 172.21.15.117:0/467362209 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8890017630 con 0x7f88a0103110 2024-04-20T11:43:45.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.299+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a019de50 con 0x7f88a0103110 2024-04-20T11:43:45.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.301+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a010b480 con 0x7f88a0103110 2024-04-20T11:43:45.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.302+0000 7f889cc00700 1 -- 172.21.15.117:0/467362209 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f889000f5d0 con 0x7f88a0103110 2024-04-20T11:43:45.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.303+0000 7f889cc00700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f888406ecd0 0x7f8884071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:45.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.303+0000 7f889cc00700 1 -- 172.21.15.117:0/467362209 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8890090340 con 0x7f88a0103110 2024-04-20T11:43:45.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.303+0000 7f889f400700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f888406ecd0 0x7f8884071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:45.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.304+0000 7f889f400700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f888406ecd0 0x7f8884071190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f888c005fd0 tx=0x7f888c00a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:45.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.308+0000 7f889cc00700 1 -- 172.21.15.117:0/467362209 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8890059100 con 0x7f88a0103110 2024-04-20T11:43:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.528+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f88a019a080 con 0x7f888406ecd0 2024-04-20T11:43:45.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.559+0000 7f889cc00700 1 -- 172.21.15.117:0/467362209 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f88a019a080 con 0x7f888406ecd0 2024-04-20T11:43:45.560 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:43:45.560 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (32m) 8m ago 46m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (46m) 8m ago 46m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (45m) 8m ago 45m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (28m) 8m ago 46m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (28m) 8m ago 45m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (31m) 8m ago 45m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (24m) 8m ago 38m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (24m) 8m ago 39m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (8m) 8m ago 38m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (23m) 8m ago 39m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (35m) 8m ago 48m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (33m) 8m ago 44m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (30m) 8m ago 48m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:43:45.561 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (29m) 8m ago 44m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (33m) 8m ago 46m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (33m) 8m ago 45m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (27m) 8m ago 43m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (27m) 8m ago 43m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (26m) 8m ago 42m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (26m) 8m ago 42m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (25m) 8m ago 41m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (24m) 8m ago 40m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (32m) 8m ago 45m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:43:45.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.562+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f888406ecd0 msgr2=0x7f8884071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:45.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.562+0000 7f88a7200700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f888406ecd0 0x7f8884071190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f888c005fd0 tx=0x7f888c00a560 comp rx=0 tx=0).stop 2024-04-20T11:43:45.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.562+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 msgr2=0x7f88a0198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:45.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.562+0000 7f88a7200700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 0x7f88a0198dc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f88900092e0 tx=0x7f889000be20 comp rx=0 tx=0).stop 2024-04-20T11:43:45.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.562+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 shutdown_connections 2024-04-20T11:43:45.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.562+0000 7f88a7200700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f888406ecd0 0x7f8884071190 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.562+0000 7f88a7200700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0103110 0x7f88a0198dc0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.563+0000 7f88a7200700 1 --2- 172.21.15.117:0/467362209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88a0104a00 0x7f88a0199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.563+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 >> 172.21.15.117:0/467362209 conn(0x7f88a00fe880 msgr2=0x7f88a00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:45.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.563+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 shutdown_connections 2024-04-20T11:43:45.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.563+0000 7f88a7200700 1 -- 172.21.15.117:0/467362209 wait complete. 2024-04-20T11:43:45.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.716+0000 7fd650a00700 1 -- 172.21.15.117:0/1317121996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107250 msgr2=0x7fd64c107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:45.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.716+0000 7fd650a00700 1 --2- 172.21.15.117:0/1317121996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107250 0x7fd64c107630 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009910 tx=0x7fd63c009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:45.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.716+0000 7fd650a00700 1 -- 172.21.15.117:0/1317121996 shutdown_connections 2024-04-20T11:43:45.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.716+0000 7fd650a00700 1 --2- 172.21.15.117:0/1317121996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd64c107c00 0x7fd64c10bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.716+0000 7fd650a00700 1 --2- 172.21.15.117:0/1317121996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107250 0x7fd64c107630 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.716+0000 7fd650a00700 1 -- 172.21.15.117:0/1317121996 >> 172.21.15.117:0/1317121996 conn(0x7fd64c076810 msgr2=0x7fd64c078c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:45.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.716+0000 7fd650a00700 1 -- 172.21.15.117:0/1317121996 shutdown_connections 2024-04-20T11:43:45.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.716+0000 7fd650a00700 1 -- 172.21.15.117:0/1317121996 wait complete. 2024-04-20T11:43:45.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.717+0000 7fd650a00700 1 Processor -- start 2024-04-20T11:43:45.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.718+0000 7fd650a00700 1 -- start start 2024-04-20T11:43:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.718+0000 7fd650a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd64c107250 0x7fd64c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.718+0000 7fd650a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107c00 0x7fd64c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.718+0000 7fd650a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19dc40 con 0x7fd64c107c00 2024-04-20T11:43:45.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.718+0000 7fd650a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1a1910 con 0x7fd64c107250 2024-04-20T11:43:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.719+0000 7fd649e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107c00 0x7fd64c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.719+0000 7fd649e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107c00 0x7fd64c19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53686/0 (socket says 172.21.15.117:53686) 2024-04-20T11:43:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.719+0000 7fd649e00700 1 -- 172.21.15.117:0/1388422532 learned_addr learned my addr 172.21.15.117:0/1388422532 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.719+0000 7fd64a800700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd64c107250 0x7fd64c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.719+0000 7fd649e00700 1 -- 172.21.15.117:0/1388422532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd64c107250 msgr2=0x7fd64c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.719+0000 7fd649e00700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd64c107250 0x7fd64c19d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.719+0000 7fd649e00700 1 -- 172.21.15.117:0/1388422532 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0095f0 con 0x7fd64c107c00 2024-04-20T11:43:45.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.719+0000 7fd649e00700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107c00 0x7fd64c19d560 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd63800b480 tx=0x7fd63800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.720+0000 7fd643400700 1 -- 172.21.15.117:0/1388422532 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6380118c0 con 0x7fd64c107c00 2024-04-20T11:43:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.720+0000 7fd643400700 1 -- 172.21.15.117:0/1388422532 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd638010380 con 0x7fd64c107c00 2024-04-20T11:43:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.720+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c1a1bf0 con 0x7fd64c107c00 2024-04-20T11:43:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.720+0000 7fd643400700 1 -- 172.21.15.117:0/1388422532 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd63800f4f0 con 0x7fd64c107c00 2024-04-20T11:43:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.720+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c1a2140 con 0x7fd64c107c00 2024-04-20T11:43:45.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.721+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c10f5f0 con 0x7fd64c107c00 2024-04-20T11:43:45.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.723+0000 7fd643400700 1 -- 172.21.15.117:0/1388422532 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd6380104f0 con 0x7fd64c107c00 2024-04-20T11:43:45.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.723+0000 7fd643400700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd63006ecc0 0x7fd630071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:45.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.724+0000 7fd643400700 1 -- 172.21.15.117:0/1388422532 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd63808ea40 con 0x7fd64c107c00 2024-04-20T11:43:45.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.724+0000 7fd64a800700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd63006ecc0 0x7fd630071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:45.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.724+0000 7fd64a800700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd63006ecc0 0x7fd630071180 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd63c006010 tx=0x7fd63c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:45.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:45.728+0000 7fd643400700 1 -- 172.21.15.117:0/1388422532 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd638057800 con 0x7fd64c107c00 2024-04-20T11:43:46.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.045+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd64c108730 con 0x7fd64c107c00 2024-04-20T11:43:46.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.047+0000 7fd643400700 1 -- 172.21.15.117:0/1388422532 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd63805aec0 con 0x7fd64c107c00 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:43:46.049 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.050+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd63006ecc0 msgr2=0x7fd630071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.050+0000 7fd650a00700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd63006ecc0 0x7fd630071180 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd63c006010 tx=0x7fd63c00b540 comp rx=0 tx=0).stop 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.050+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107c00 msgr2=0x7fd64c19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.050+0000 7fd650a00700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107c00 0x7fd64c19d560 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd63800b480 tx=0x7fd63800b840 comp rx=0 tx=0).stop 2024-04-20T11:43:46.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.051+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 shutdown_connections 2024-04-20T11:43:46.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.051+0000 7fd650a00700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd63006ecc0 0x7fd630071180 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.051+0000 7fd650a00700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd64c107250 0x7fd64c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.051+0000 7fd650a00700 1 --2- 172.21.15.117:0/1388422532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd64c107c00 0x7fd64c19d560 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.051+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 >> 172.21.15.117:0/1388422532 conn(0x7fd64c076810 msgr2=0x7fd64c077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:46.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.051+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 shutdown_connections 2024-04-20T11:43:46.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.051+0000 7fd650a00700 1 -- 172.21.15.117:0/1388422532 wait complete. 2024-04-20T11:43:46.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.208+0000 7fb7e8c00700 1 -- 172.21.15.117:0/3827594805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 msgr2=0x7fb7e410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.208+0000 7fb7e8c00700 1 --2- 172.21.15.117:0/3827594805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 0x7fb7e410abf0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009910 tx=0x7fb7d0009c20 comp rx=0 tx=0).stop 2024-04-20T11:43:46.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.208+0000 7fb7e8c00700 1 -- 172.21.15.117:0/3827594805 shutdown_connections 2024-04-20T11:43:46.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.208+0000 7fb7e8c00700 1 --2- 172.21.15.117:0/3827594805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 0x7fb7e410abf0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.208+0000 7fb7e8c00700 1 --2- 172.21.15.117:0/3827594805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e4101de0 0x7fb7e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.208+0000 7fb7e8c00700 1 -- 172.21.15.117:0/3827594805 >> 172.21.15.117:0/3827594805 conn(0x7fb7e4076750 msgr2=0x7fb7e40fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:46.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.208+0000 7fb7e8c00700 1 -- 172.21.15.117:0/3827594805 shutdown_connections 2024-04-20T11:43:46.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.208+0000 7fb7e8c00700 1 -- 172.21.15.117:0/3827594805 wait complete. 2024-04-20T11:43:46.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.209+0000 7fb7e8c00700 1 Processor -- start 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.209+0000 7fb7e8c00700 1 -- start start 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e8c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e4101de0 0x7fb7e4198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e8c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 0x7fb7e41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e8c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41999d0 con 0x7fb7e4102700 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e8c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e419d6a0 con 0x7fb7e4101de0 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 0x7fb7e41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 0x7fb7e41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53704/0 (socket says 172.21.15.117:53704) 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e1400700 1 -- 172.21.15.117:0/4005804711 learned_addr learned my addr 172.21.15.117:0/4005804711 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e1e00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e4101de0 0x7fb7e4198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e1400700 1 -- 172.21.15.117:0/4005804711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e4101de0 msgr2=0x7fb7e4198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e1400700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e4101de0 0x7fb7e4198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.210+0000 7fb7e1400700 1 -- 172.21.15.117:0/4005804711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d00095f0 con 0x7fb7e4102700 2024-04-20T11:43:46.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.211+0000 7fb7e1400700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 0x7fb7e41992f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0005b40 tx=0x7fb7d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:46.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.211+0000 7fb7daa00700 1 -- 172.21.15.117:0/4005804711 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d001d070 con 0x7fb7e4102700 2024-04-20T11:43:46.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.211+0000 7fb7daa00700 1 -- 172.21.15.117:0/4005804711 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7d0005700 con 0x7fb7e4102700 2024-04-20T11:43:46.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.211+0000 7fb7daa00700 1 -- 172.21.15.117:0/4005804711 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d000fe50 con 0x7fb7e4102700 2024-04-20T11:43:46.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.211+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e419d920 con 0x7fb7e4102700 2024-04-20T11:43:46.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.211+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e419de10 con 0x7fb7e4102700 2024-04-20T11:43:46.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.212+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e4108740 con 0x7fb7e4102700 2024-04-20T11:43:46.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.213+0000 7fb7daa00700 1 -- 172.21.15.117:0/4005804711 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb7d000f460 con 0x7fb7e4102700 2024-04-20T11:43:46.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.214+0000 7fb7daa00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7c806ecc0 0x7fb7c8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:46.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.214+0000 7fb7daa00700 1 -- 172.21.15.117:0/4005804711 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb7d0090300 con 0x7fb7e4102700 2024-04-20T11:43:46.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.215+0000 7fb7e1e00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7c806ecc0 0x7fb7c8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:46.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.215+0000 7fb7e1e00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7c806ecc0 0x7fb7c8071180 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4005950 tx=0x7fb7d40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:46.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.219+0000 7fb7daa00700 1 -- 172.21.15.117:0/4005804711 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb7d0059040 con 0x7fb7e4102700 2024-04-20T11:43:46.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:46 smithi117 ceph-mon[99036]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:46 smithi117 ceph-mon[99036]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:46 smithi117 ceph-mon[99036]: pgmap v940: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:46 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1388422532' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:43:46.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.488+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb7e4068db0 con 0x7fb7e4102700 2024-04-20T11:43:46.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.489+0000 7fb7daa00700 1 -- 172.21.15.117:0/4005804711 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fb7d005c700 con 0x7fb7e4102700 2024-04-20T11:43:46.489 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:43:46.489 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:43:46.489 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:43:46.489 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:43:46.489 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:43:46.489 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:43:46.489 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:43:46.490 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:43:46.491 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:43:46.492 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:43:46.492 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:43:46.492 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:43:46.492 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:43:46.492 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:43:46.492 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7c806ecc0 msgr2=0x7fb7c8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7c806ecc0 0x7fb7c8071180 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4005950 tx=0x7fb7d40058e0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 msgr2=0x7fb7e41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 0x7fb7e41992f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0005b40 tx=0x7fb7d000be20 comp rx=0 tx=0).stop 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 shutdown_connections 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb7c806ecc0 0x7fb7c8071180 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7e4101de0 0x7fb7e4198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 --2- 172.21.15.117:0/4005804711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7e4102700 0x7fb7e41992f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 >> 172.21.15.117:0/4005804711 conn(0x7fb7e4076750 msgr2=0x7fb7e410a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 shutdown_connections 2024-04-20T11:43:46.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.492+0000 7fb7e8c00700 1 -- 172.21.15.117:0/4005804711 wait complete. 2024-04-20T11:43:46.494 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:43:46.588 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:46 smithi173 ceph-mon[74165]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:46.589 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:46 smithi173 ceph-mon[74165]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:46.589 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:46 smithi173 ceph-mon[74165]: pgmap v940: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:46.589 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:46 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1388422532' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:43:46.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.644+0000 7fc50c400700 1 -- 172.21.15.117:0/1800576436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc504101da0 msgr2=0x7fc504102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.644+0000 7fc50c400700 1 --2- 172.21.15.117:0/1800576436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc504101da0 0x7fc504102180 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009940 tx=0x7fc4f8009c50 comp rx=0 tx=0).stop 2024-04-20T11:43:46.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.644+0000 7fc50c400700 1 -- 172.21.15.117:0/1800576436 shutdown_connections 2024-04-20T11:43:46.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.644+0000 7fc50c400700 1 --2- 172.21.15.117:0/1800576436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5041026c0 0x7fc50410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.644+0000 7fc50c400700 1 --2- 172.21.15.117:0/1800576436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc504101da0 0x7fc504102180 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.644+0000 7fc50c400700 1 -- 172.21.15.117:0/1800576436 >> 172.21.15.117:0/1800576436 conn(0x7fc504076750 msgr2=0x7fc5040fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:46.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.644+0000 7fc50c400700 1 -- 172.21.15.117:0/1800576436 shutdown_connections 2024-04-20T11:43:46.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.644+0000 7fc50c400700 1 -- 172.21.15.117:0/1800576436 wait complete. 2024-04-20T11:43:46.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.645+0000 7fc50c400700 1 Processor -- start 2024-04-20T11:43:46.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.645+0000 7fc50c400700 1 -- start start 2024-04-20T11:43:46.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.645+0000 7fc50c400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc504101da0 0x7fc50419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:46.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.645+0000 7fc50c400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5041026c0 0x7fc50419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.645+0000 7fc50c400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50419d380 con 0x7fc5041026c0 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.645+0000 7fc50c400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc504196960 con 0x7fc504101da0 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.646+0000 7fc509000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5041026c0 0x7fc50419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.646+0000 7fc509000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5041026c0 0x7fc50419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53732/0 (socket says 172.21.15.117:53732) 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.646+0000 7fc509000700 1 -- 172.21.15.117:0/3803896575 learned_addr learned my addr 172.21.15.117:0/3803896575 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.646+0000 7fc509a00700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc504101da0 0x7fc50419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.646+0000 7fc509000700 1 -- 172.21.15.117:0/3803896575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc504101da0 msgr2=0x7fc50419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.646+0000 7fc509000700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc504101da0 0x7fc50419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.646+0000 7fc509000700 1 -- 172.21.15.117:0/3803896575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f80095f0 con 0x7fc5041026c0 2024-04-20T11:43:46.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.646+0000 7fc509000700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5041026c0 0x7fc50419cd60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000c8f0 tx=0x7fc4f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:46.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.647+0000 7fc502000700 1 -- 172.21.15.117:0/3803896575 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4f0007a50 con 0x7fc5041026c0 2024-04-20T11:43:46.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.647+0000 7fc502000700 1 -- 172.21.15.117:0/3803896575 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4f0007bb0 con 0x7fc5041026c0 2024-04-20T11:43:46.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.647+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc504196c40 con 0x7fc5041026c0 2024-04-20T11:43:46.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.647+0000 7fc502000700 1 -- 172.21.15.117:0/3803896575 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4f0018630 con 0x7fc5041026c0 2024-04-20T11:43:46.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.647+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc504197190 con 0x7fc5041026c0 2024-04-20T11:43:46.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.648+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5041054c0 con 0x7fc5041026c0 2024-04-20T11:43:46.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.649+0000 7fc502000700 1 -- 172.21.15.117:0/3803896575 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc4f0007d20 con 0x7fc5041026c0 2024-04-20T11:43:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.650+0000 7fc502000700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc4ec06ecc0 0x7fc4ec071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:46.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.650+0000 7fc502000700 1 -- 172.21.15.117:0/3803896575 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc4f008ed30 con 0x7fc5041026c0 2024-04-20T11:43:46.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.650+0000 7fc509a00700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc4ec06ecc0 0x7fc4ec071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:46.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.651+0000 7fc509a00700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc4ec06ecc0 0x7fc4ec071180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8006010 tx=0x7fc4f8005ad0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.657+0000 7fc502000700 1 -- 172.21.15.117:0/3803896575 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc4f005b6c0 con 0x7fc5041026c0 2024-04-20T11:43:46.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.886+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc504197e90 con 0x7fc4ec06ecc0 2024-04-20T11:43:46.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.889+0000 7fc502000700 1 -- 172.21.15.117:0/3803896575 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc504197e90 con 0x7fc4ec06ecc0 2024-04-20T11:43:46.889 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:43:46.889 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:43:46.889 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:43:46.889 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:43:46.889 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:43:46.889 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:43:46.889 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:43:46.890 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:43:46.890 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:43:46.890 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:43:46.890 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:43:46.890 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:43:46.890 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:43:46.890 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:43:46.890 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:43:46.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.891+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc4ec06ecc0 msgr2=0x7fc4ec071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.891+0000 7fc50c400700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc4ec06ecc0 0x7fc4ec071180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8006010 tx=0x7fc4f8005ad0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.891+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5041026c0 msgr2=0x7fc50419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.891+0000 7fc50c400700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5041026c0 0x7fc50419cd60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000c8f0 tx=0x7fc4f000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.892+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 shutdown_connections 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.892+0000 7fc50c400700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc4ec06ecc0 0x7fc4ec071180 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.892+0000 7fc50c400700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc504101da0 0x7fc50419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.892+0000 7fc50c400700 1 --2- 172.21.15.117:0/3803896575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5041026c0 0x7fc50419cd60 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.892+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 >> 172.21.15.117:0/3803896575 conn(0x7fc504076750 msgr2=0x7fc504103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.892+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 shutdown_connections 2024-04-20T11:43:46.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:46.892+0000 7fc50c400700 1 -- 172.21.15.117:0/3803896575 wait complete. 2024-04-20T11:43:47.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.045+0000 7ff6b4c00700 1 -- 172.21.15.117:0/1967402950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073310 msgr2=0x7ff6b00736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:47.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.045+0000 7ff6b4c00700 1 --2- 172.21.15.117:0/1967402950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073310 0x7ff6b00736f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b0a0 tx=0x7ff6a000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:43:47.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.045+0000 7ff6b4c00700 1 -- 172.21.15.117:0/1967402950 shutdown_connections 2024-04-20T11:43:47.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.045+0000 7ff6b4c00700 1 --2- 172.21.15.117:0/1967402950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6b0073c30 0x7ff6b010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:47.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.045+0000 7ff6b4c00700 1 --2- 172.21.15.117:0/1967402950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073310 0x7ff6b00736f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:47.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.045+0000 7ff6b4c00700 1 -- 172.21.15.117:0/1967402950 >> 172.21.15.117:0/1967402950 conn(0x7ff6b00fc6c0 msgr2=0x7ff6b00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:47.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.045+0000 7ff6b4c00700 1 -- 172.21.15.117:0/1967402950 shutdown_connections 2024-04-20T11:43:47.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.045+0000 7ff6b4c00700 1 -- 172.21.15.117:0/1967402950 wait complete. 2024-04-20T11:43:47.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.047+0000 7ff6b4c00700 1 Processor -- start 2024-04-20T11:43:47.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.047+0000 7ff6b4c00700 1 -- start start 2024-04-20T11:43:47.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6b4c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6b0073310 0x7ff6b0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:47.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6b4c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073c30 0x7ff6b01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:47.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6b4c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0199980 con 0x7ff6b0073c30 2024-04-20T11:43:47.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6b4c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b019d650 con 0x7ff6b0073310 2024-04-20T11:43:47.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6a5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073c30 0x7ff6b01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6a5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073c30 0x7ff6b01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53740/0 (socket says 172.21.15.117:53740) 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6a5400700 1 -- 172.21.15.117:0/2885647523 learned_addr learned my addr 172.21.15.117:0/2885647523 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6a5400700 1 -- 172.21.15.117:0/2885647523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6b0073310 msgr2=0x7ff6b0198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6a5400700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6b0073310 0x7ff6b0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.048+0000 7ff6a5400700 1 -- 172.21.15.117:0/2885647523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0009d00 con 0x7ff6b0073c30 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.049+0000 7ff6a5400700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073c30 0x7ff6b01992a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c8a0 tx=0x7ff69800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.049+0000 7ff6a7400700 1 -- 172.21.15.117:0/2885647523 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff69800cea0 con 0x7ff6b0073c30 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.049+0000 7ff6a7400700 1 -- 172.21.15.117:0/2885647523 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff698004830 con 0x7ff6b0073c30 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.049+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b019d930 con 0x7ff6b0073c30 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.049+0000 7ff6a7400700 1 -- 172.21.15.117:0/2885647523 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6980055b0 con 0x7ff6b0073c30 2024-04-20T11:43:47.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.049+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b019de80 con 0x7ff6b0073c30 2024-04-20T11:43:47.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.050+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b00749c0 con 0x7ff6b0073c30 2024-04-20T11:43:47.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.051+0000 7ff6a7400700 1 -- 172.21.15.117:0/2885647523 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff6980049a0 con 0x7ff6b0073c30 2024-04-20T11:43:47.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.052+0000 7ff6a7400700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff69406ecc0 0x7ff694071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:43:47.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.053+0000 7ff6a7400700 1 -- 172.21.15.117:0/2885647523 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff69808ea20 con 0x7ff6b0073c30 2024-04-20T11:43:47.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.053+0000 7ff6ade00700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff69406ecc0 0x7ff694071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:43:47.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.054+0000 7ff6ade00700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff69406ecc0 0x7ff694071180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009480 tx=0x7ff6a0006000 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:43:47.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.058+0000 7ff6a7400700 1 -- 172.21.15.117:0/2885647523 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff698057880 con 0x7ff6b0073c30 2024-04-20T11:43:47.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.363+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff6b0107780 con 0x7ff6b0073c30 2024-04-20T11:43:47.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.364+0000 7ff6a7400700 1 -- 172.21.15.117:0/2885647523 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7ff69801d090 con 0x7ff6b0073c30 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:43:47.365 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:43:47.366 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:43:47.367 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:43:47.367 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:43:47.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff69406ecc0 msgr2=0x7ff694071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:47.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff69406ecc0 0x7ff694071180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009480 tx=0x7ff6a0006000 comp rx=0 tx=0).stop 2024-04-20T11:43:47.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073c30 msgr2=0x7ff6b01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:43:47.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073c30 0x7ff6b01992a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c8a0 tx=0x7ff69800cc60 comp rx=0 tx=0).stop 2024-04-20T11:43:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 shutdown_connections 2024-04-20T11:43:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff69406ecc0 0x7ff694071180 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6b0073310 0x7ff6b0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 --2- 172.21.15.117:0/2885647523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b0073c30 0x7ff6b01992a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:43:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.367+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 >> 172.21.15.117:0/2885647523 conn(0x7ff6b00fc6c0 msgr2=0x7ff6b0105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:43:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.368+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 shutdown_connections 2024-04-20T11:43:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:43:47.368+0000 7ff6b4c00700 1 -- 172.21.15.117:0/2885647523 wait complete. 2024-04-20T11:43:47.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:47 smithi117 ceph-mon[99036]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:47.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:47 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4005804711' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:43:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:47 smithi173 ceph-mon[74165]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:47 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4005804711' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:43:48.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:48 smithi117 ceph-mon[99036]: from='client.35702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:48.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:48 smithi117 ceph-mon[99036]: pgmap v941: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:48.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:48 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2885647523' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:43:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:48 smithi173 ceph-mon[74165]: from='client.35702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:43:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:48 smithi173 ceph-mon[74165]: pgmap v941: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:48.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:48 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2885647523' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:43:50.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:50 smithi117 ceph-mon[99036]: pgmap v942: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:50 smithi173 ceph-mon[74165]: pgmap v942: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:52 smithi117 ceph-mon[99036]: pgmap v943: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:52 smithi173 ceph-mon[74165]: pgmap v943: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:54 smithi117 ceph-mon[99036]: pgmap v944: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:54 smithi173 ceph-mon[74165]: pgmap v944: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:56 smithi117 ceph-mon[99036]: pgmap v945: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:56 smithi173 ceph-mon[74165]: pgmap v945: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:58 smithi117 ceph-mon[99036]: pgmap v946: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:43:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:43:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:58 smithi173 ceph-mon[74165]: pgmap v946: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:43:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:43:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:44:00.623 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:00 smithi117 ceph-mon[99036]: pgmap v947: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:00 smithi173 ceph-mon[74165]: pgmap v947: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:02.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:02 smithi173 ceph-mon[74165]: pgmap v948: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:02 smithi117 ceph-mon[99036]: pgmap v948: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:04 smithi117 ceph-mon[99036]: pgmap v949: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:04 smithi173 ceph-mon[74165]: pgmap v949: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:06.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:06 smithi117 ceph-mon[99036]: pgmap v950: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:06 smithi173 ceph-mon[74165]: pgmap v950: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:08.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:08 smithi117 ceph-mon[99036]: pgmap v951: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:08 smithi173 ceph-mon[74165]: pgmap v951: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:10.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:10 smithi117 ceph-mon[99036]: pgmap v952: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:10 smithi173 ceph-mon[74165]: pgmap v952: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:12.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:12 smithi117 ceph-mon[99036]: pgmap v953: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:12 smithi173 ceph-mon[74165]: pgmap v953: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:44:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:44:14.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:14 smithi117 ceph-mon[99036]: pgmap v954: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:14 smithi173 ceph-mon[74165]: pgmap v954: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:16 smithi117 ceph-mon[99036]: pgmap v955: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:16 smithi173 ceph-mon[74165]: pgmap v955: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:17.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.534+0000 7f6a91c00700 1 -- 172.21.15.117:0/282952696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 msgr2=0x7f6a8c10d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.534+0000 7f6a91c00700 1 --2- 172.21.15.117:0/282952696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 0x7f6a8c10d4a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009910 tx=0x7f6a7c009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.534+0000 7f6a91c00700 1 -- 172.21.15.117:0/282952696 shutdown_connections 2024-04-20T11:44:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.534+0000 7f6a91c00700 1 --2- 172.21.15.117:0/282952696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a8c1070b0 0x7f6a8c107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.534+0000 7f6a91c00700 1 --2- 172.21.15.117:0/282952696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 0x7f6a8c10d4a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.534+0000 7f6a91c00700 1 -- 172.21.15.117:0/282952696 >> 172.21.15.117:0/282952696 conn(0x7f6a8c076830 msgr2=0x7f6a8c078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.535+0000 7f6a91c00700 1 -- 172.21.15.117:0/282952696 shutdown_connections 2024-04-20T11:44:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.535+0000 7f6a91c00700 1 -- 172.21.15.117:0/282952696 wait complete. 2024-04-20T11:44:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a91c00700 1 Processor -- start 2024-04-20T11:44:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a91c00700 1 -- start start 2024-04-20T11:44:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a91c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a8c1070b0 0x7f6a8c19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a91c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 0x7f6a8c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a91c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c19dca0 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a91c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1a1920 con 0x7f6a8c1070b0 2024-04-20T11:44:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a8b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 0x7f6a8c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a8b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 0x7f6a8c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39578/0 (socket says 172.21.15.117:39578) 2024-04-20T11:44:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.536+0000 7f6a8b400700 1 -- 172.21.15.117:0/3207055001 learned_addr learned my addr 172.21.15.117:0/3207055001 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.537+0000 7f6a8b400700 1 -- 172.21.15.117:0/3207055001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a8c1070b0 msgr2=0x7f6a8c19d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.537+0000 7f6a8b400700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a8c1070b0 0x7f6a8c19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.537+0000 7f6a8b400700 1 -- 172.21.15.117:0/3207055001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0095f0 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.537+0000 7f6a8b400700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 0x7f6a8c19d5c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800b4b0 tx=0x7f6a7800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.537+0000 7f6a88c00700 1 -- 172.21.15.117:0/3207055001 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a78011800 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.537+0000 7f6a88c00700 1 -- 172.21.15.117:0/3207055001 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a78011e20 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.538+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c1a1c00 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.538+0000 7f6a88c00700 1 -- 172.21.15.117:0/3207055001 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a7800f2e0 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.538+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c1a2150 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.539+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c10f5c0 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.540+0000 7f6a88c00700 1 -- 172.21.15.117:0/3207055001 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6a78010380 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.541+0000 7f6a88c00700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6a7006ecc0 0x7f6a70071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:17.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.541+0000 7f6a88c00700 1 -- 172.21.15.117:0/3207055001 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6a7808ea00 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.542+0000 7f6a8be00700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6a7006ecc0 0x7f6a70071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:17.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.542+0000 7f6a8be00700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6a7006ecc0 0x7f6a70071180 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00b5c0 tx=0x7f6a7c005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:17.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.547+0000 7f6a88c00700 1 -- 172.21.15.117:0/3207055001 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6a78057870 con 0x7f6a8c10d0c0 2024-04-20T11:44:17.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.793+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a8c114290 con 0x7f6a7006ecc0 2024-04-20T11:44:17.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.796+0000 7f6a88c00700 1 -- 172.21.15.117:0/3207055001 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6a8c114290 con 0x7f6a7006ecc0 2024-04-20T11:44:17.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.798+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6a7006ecc0 msgr2=0x7f6a70071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:17.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.798+0000 7f6a91c00700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6a7006ecc0 0x7f6a70071180 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00b5c0 tx=0x7f6a7c005960 comp rx=0 tx=0).stop 2024-04-20T11:44:17.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.798+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 msgr2=0x7f6a8c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:17.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.798+0000 7f6a91c00700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 0x7f6a8c19d5c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800b4b0 tx=0x7f6a7800b870 comp rx=0 tx=0).stop 2024-04-20T11:44:17.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.799+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 shutdown_connections 2024-04-20T11:44:17.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.799+0000 7f6a91c00700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6a7006ecc0 0x7f6a70071180 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.799+0000 7f6a91c00700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a8c1070b0 0x7f6a8c19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.799+0000 7f6a91c00700 1 --2- 172.21.15.117:0/3207055001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c10d0c0 0x7f6a8c19d5c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.799+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 >> 172.21.15.117:0/3207055001 conn(0x7f6a8c076830 msgr2=0x7f6a8c0789e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:17.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.799+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 shutdown_connections 2024-04-20T11:44:17.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.799+0000 7f6a91c00700 1 -- 172.21.15.117:0/3207055001 wait complete. 2024-04-20T11:44:17.815 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:44:17.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.957+0000 7f7e2a200700 1 -- 172.21.15.117:0/2158338881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24104a00 msgr2=0x7f7e24104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.957+0000 7f7e2a200700 1 --2- 172.21.15.117:0/2158338881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24104a00 0x7f7e24104e80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009910 tx=0x7f7e10009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.957+0000 7f7e2a200700 1 -- 172.21.15.117:0/2158338881 shutdown_connections 2024-04-20T11:44:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.957+0000 7f7e2a200700 1 --2- 172.21.15.117:0/2158338881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24104a00 0x7f7e24104e80 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.957+0000 7f7e2a200700 1 --2- 172.21.15.117:0/2158338881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e24103110 0x7f7e241034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.957+0000 7f7e2a200700 1 -- 172.21.15.117:0/2158338881 >> 172.21.15.117:0/2158338881 conn(0x7f7e240fe880 msgr2=0x7f7e24100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.958+0000 7f7e2a200700 1 -- 172.21.15.117:0/2158338881 shutdown_connections 2024-04-20T11:44:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.958+0000 7f7e2a200700 1 -- 172.21.15.117:0/2158338881 wait complete. 2024-04-20T11:44:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.959+0000 7f7e2a200700 1 Processor -- start 2024-04-20T11:44:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.959+0000 7f7e2a200700 1 -- start start 2024-04-20T11:44:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.959+0000 7f7e2a200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24103110 0x7f7e2419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.959+0000 7f7e2a200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e24104a00 0x7f7e2419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.959+0000 7f7e2a200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419f5b0 con 0x7f7e24103110 2024-04-20T11:44:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.959+0000 7f7e2a200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e24198b90 con 0x7f7e24104a00 2024-04-20T11:44:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.960+0000 7f7e23400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24103110 0x7f7e2419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.960+0000 7f7e23400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24103110 0x7f7e2419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39600/0 (socket says 172.21.15.117:39600) 2024-04-20T11:44:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.960+0000 7f7e23400700 1 -- 172.21.15.117:0/108320253 learned_addr learned my addr 172.21.15.117:0/108320253 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.960+0000 7f7e23400700 1 -- 172.21.15.117:0/108320253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e24104a00 msgr2=0x7f7e2419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.960+0000 7f7e23400700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e24104a00 0x7f7e2419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.960+0000 7f7e23400700 1 -- 172.21.15.117:0/108320253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100095f0 con 0x7f7e24103110 2024-04-20T11:44:17.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.960+0000 7f7e23400700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24103110 0x7f7e2419ea50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400c8a0 tx=0x7f7e1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:17.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.961+0000 7f7e1be00700 1 -- 172.21.15.117:0/108320253 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1400cea0 con 0x7f7e24103110 2024-04-20T11:44:17.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.961+0000 7f7e1be00700 1 -- 172.21.15.117:0/108320253 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e14004830 con 0x7f7e24103110 2024-04-20T11:44:17.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.961+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e24198e70 con 0x7f7e24103110 2024-04-20T11:44:17.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.961+0000 7f7e1be00700 1 -- 172.21.15.117:0/108320253 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e140055b0 con 0x7f7e24103110 2024-04-20T11:44:17.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.961+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e24199440 con 0x7f7e24103110 2024-04-20T11:44:17.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.962+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2410b480 con 0x7f7e24103110 2024-04-20T11:44:17.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.963+0000 7f7e1be00700 1 -- 172.21.15.117:0/108320253 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7e140049a0 con 0x7f7e24103110 2024-04-20T11:44:17.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.964+0000 7f7e1be00700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e0806ecc0 0x7f7e08071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:17.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.964+0000 7f7e1be00700 1 -- 172.21.15.117:0/108320253 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7e1408ebc0 con 0x7f7e24103110 2024-04-20T11:44:17.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.964+0000 7f7e22a00700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e0806ecc0 0x7f7e08071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:17.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.965+0000 7f7e22a00700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e0806ecc0 0x7f7e08071180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000b5c0 tx=0x7f7e10005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:17.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:17.969+0000 7f7e1be00700 1 -- 172.21.15.117:0/108320253 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7e1405b430 con 0x7f7e24103110 2024-04-20T11:44:18.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.215+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e2419a080 con 0x7f7e0806ecc0 2024-04-20T11:44:18.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.218+0000 7f7e1be00700 1 -- 172.21.15.117:0/108320253 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7e2419a080 con 0x7f7e0806ecc0 2024-04-20T11:44:18.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e0806ecc0 msgr2=0x7f7e08071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:18.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e0806ecc0 0x7f7e08071180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000b5c0 tx=0x7f7e10005960 comp rx=0 tx=0).stop 2024-04-20T11:44:18.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24103110 msgr2=0x7f7e2419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:18.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24103110 0x7f7e2419ea50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400c8a0 tx=0x7f7e1400cc60 comp rx=0 tx=0).stop 2024-04-20T11:44:18.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 shutdown_connections 2024-04-20T11:44:18.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e0806ecc0 0x7f7e08071180 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e24103110 0x7f7e2419ea50 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 --2- 172.21.15.117:0/108320253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e24104a00 0x7f7e2419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 >> 172.21.15.117:0/108320253 conn(0x7f7e240fe880 msgr2=0x7f7e240ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:18.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 shutdown_connections 2024-04-20T11:44:18.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.221+0000 7f7e2a200700 1 -- 172.21.15.117:0/108320253 wait complete. 2024-04-20T11:44:18.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.379+0000 7f802b000700 1 -- 172.21.15.117:0/3346374462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8024100e10 msgr2=0x7f80241011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:18.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.379+0000 7f802b000700 1 --2- 172.21.15.117:0/3346374462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8024100e10 0x7f80241011f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8014009910 tx=0x7f8014009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:18.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.379+0000 7f802b000700 1 -- 172.21.15.117:0/3346374462 shutdown_connections 2024-04-20T11:44:18.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.379+0000 7f802b000700 1 --2- 172.21.15.117:0/3346374462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80241017c0 0x7f8024105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.379+0000 7f802b000700 1 --2- 172.21.15.117:0/3346374462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8024100e10 0x7f80241011f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.379+0000 7f802b000700 1 -- 172.21.15.117:0/3346374462 >> 172.21.15.117:0/3346374462 conn(0x7f80240fc6c0 msgr2=0x7f80240feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:18.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.379+0000 7f802b000700 1 -- 172.21.15.117:0/3346374462 shutdown_connections 2024-04-20T11:44:18.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.379+0000 7f802b000700 1 -- 172.21.15.117:0/3346374462 wait complete. 2024-04-20T11:44:18.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.380+0000 7f802b000700 1 Processor -- start 2024-04-20T11:44:18.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.380+0000 7f802b000700 1 -- start start 2024-04-20T11:44:18.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f802b000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8024100e10 0x7f802419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:18.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f802b000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80241017c0 0x7f802419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:18.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f802b000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802419d460 con 0x7f80241017c0 2024-04-20T11:44:18.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f802b000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8024196a50 con 0x7f8024100e10 2024-04-20T11:44:18.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f8023400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80241017c0 0x7f802419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:18.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f8023400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80241017c0 0x7f802419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39628/0 (socket says 172.21.15.117:39628) 2024-04-20T11:44:18.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f8023400700 1 -- 172.21.15.117:0/500540713 learned_addr learned my addr 172.21.15.117:0/500540713 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:18.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f8023400700 1 -- 172.21.15.117:0/500540713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8024100e10 msgr2=0x7f802419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:18.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f8023400700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8024100e10 0x7f802419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.381+0000 7f8023400700 1 -- 172.21.15.117:0/500540713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140095f0 con 0x7f80241017c0 2024-04-20T11:44:18.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.382+0000 7f8023400700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80241017c0 0x7f802419cd80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f801000b480 tx=0x7f801000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:18.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.382+0000 7f8020c00700 1 -- 172.21.15.117:0/500540713 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80100118c0 con 0x7f80241017c0 2024-04-20T11:44:18.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.382+0000 7f8020c00700 1 -- 172.21.15.117:0/500540713 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8010010380 con 0x7f80241017c0 2024-04-20T11:44:18.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.382+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8024196d30 con 0x7f80241017c0 2024-04-20T11:44:18.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.382+0000 7f8020c00700 1 -- 172.21.15.117:0/500540713 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f801000f4f0 con 0x7f80241017c0 2024-04-20T11:44:18.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.382+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8024197280 con 0x7f80241017c0 2024-04-20T11:44:18.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.383+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80241091b0 con 0x7f80241017c0 2024-04-20T11:44:18.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.389+0000 7f8020c00700 1 -- 172.21.15.117:0/500540713 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f80100104f0 con 0x7f80241017c0 2024-04-20T11:44:18.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.390+0000 7f8020c00700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f800806ecc0 0x7f8008071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:18.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.390+0000 7f8020c00700 1 -- 172.21.15.117:0/500540713 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f801008ea40 con 0x7f80241017c0 2024-04-20T11:44:18.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.390+0000 7f8023e00700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f800806ecc0 0x7f8008071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:18.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.391+0000 7f8023e00700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f800806ecc0 0x7f8008071180 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f801400b5c0 tx=0x7f8014005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:18.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.398+0000 7f8020c00700 1 -- 172.21.15.117:0/500540713 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f801005b330 con 0x7f80241017c0 2024-04-20T11:44:18.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.632+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f802410de50 con 0x7f800806ecc0 2024-04-20T11:44:18.633 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:18 smithi117 ceph-mon[99036]: pgmap v956: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:18.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.646+0000 7f8020c00700 1 -- 172.21.15.117:0/500540713 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f802410de50 con 0x7f800806ecc0 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (32m) 8m ago 46m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (47m) 8m ago 47m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (45m) 8m ago 45m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (29m) 8m ago 47m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (28m) 8m ago 45m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (32m) 8m ago 46m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (24m) 8m ago 39m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:44:18.647 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (25m) 8m ago 39m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (8m) 8m ago 39m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (24m) 8m ago 39m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (35m) 8m ago 48m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (34m) 8m ago 45m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (30m) 8m ago 48m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (30m) 8m ago 45m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (34m) 8m ago 46m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (33m) 8m ago 45m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (28m) 8m ago 44m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (27m) 8m ago 43m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (27m) 8m ago 43m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (26m) 8m ago 42m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (26m) 8m ago 42m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (25m) 8m ago 41m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:44:18.648 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (33m) 8m ago 46m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:44:18.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.648+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f800806ecc0 msgr2=0x7f8008071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:18.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.648+0000 7f802b000700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f800806ecc0 0x7f8008071180 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f801400b5c0 tx=0x7f8014005960 comp rx=0 tx=0).stop 2024-04-20T11:44:18.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.648+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80241017c0 msgr2=0x7f802419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:18.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.648+0000 7f802b000700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80241017c0 0x7f802419cd80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f801000b480 tx=0x7f801000b840 comp rx=0 tx=0).stop 2024-04-20T11:44:18.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.648+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 shutdown_connections 2024-04-20T11:44:18.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.648+0000 7f802b000700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f800806ecc0 0x7f8008071180 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.648+0000 7f802b000700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8024100e10 0x7f802419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.648+0000 7f802b000700 1 --2- 172.21.15.117:0/500540713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80241017c0 0x7f802419cd80 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.649+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 >> 172.21.15.117:0/500540713 conn(0x7f80240fc6c0 msgr2=0x7f80240fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:18.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.649+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 shutdown_connections 2024-04-20T11:44:18.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.649+0000 7f802b000700 1 -- 172.21.15.117:0/500540713 wait complete. 2024-04-20T11:44:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:18 smithi173 ceph-mon[74165]: pgmap v956: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:18.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.797+0000 7fe291400700 1 -- 172.21.15.117:0/1186702194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c101d40 msgr2=0x7fe28c102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.797+0000 7fe291400700 1 --2- 172.21.15.117:0/1186702194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c101d40 0x7fe28c102120 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b0a0 tx=0x7fe27c00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.797+0000 7fe291400700 1 -- 172.21.15.117:0/1186702194 shutdown_connections 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.797+0000 7fe291400700 1 --2- 172.21.15.117:0/1186702194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c102660 0x7fe28c10ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.797+0000 7fe291400700 1 --2- 172.21.15.117:0/1186702194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c101d40 0x7fe28c102120 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.797+0000 7fe291400700 1 -- 172.21.15.117:0/1186702194 >> 172.21.15.117:0/1186702194 conn(0x7fe28c076750 msgr2=0x7fe28c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.797+0000 7fe291400700 1 -- 172.21.15.117:0/1186702194 shutdown_connections 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.797+0000 7fe291400700 1 -- 172.21.15.117:0/1186702194 wait complete. 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.798+0000 7fe291400700 1 Processor -- start 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.798+0000 7fe291400700 1 -- start start 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.798+0000 7fe291400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c101d40 0x7fe28c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.798+0000 7fe291400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c102660 0x7fe28c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.798+0000 7fe291400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c19d3e0 con 0x7fe28c102660 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.798+0000 7fe291400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c1969c0 con 0x7fe28c101d40 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.799+0000 7fe283e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c102660 0x7fe28c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.799+0000 7fe283e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c102660 0x7fe28c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39648/0 (socket says 172.21.15.117:39648) 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.799+0000 7fe283e00700 1 -- 172.21.15.117:0/719974574 learned_addr learned my addr 172.21.15.117:0/719974574 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.799+0000 7fe283e00700 1 -- 172.21.15.117:0/719974574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c101d40 msgr2=0x7fe28c19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.799+0000 7fe28a800700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c101d40 0x7fe28c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.799+0000 7fe283e00700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c101d40 0x7fe28c19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.799+0000 7fe283e00700 1 -- 172.21.15.117:0/719974574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c009d00 con 0x7fe28c102660 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.800+0000 7fe283e00700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c102660 0x7fe28c19cdc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe27400d6d0 tx=0x7fe27400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.800+0000 7fe283400700 1 -- 172.21.15.117:0/719974574 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe27400f860 con 0x7fe28c102660 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.800+0000 7fe283400700 1 -- 172.21.15.117:0/719974574 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe27400fe80 con 0x7fe28c102660 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.800+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c196ca0 con 0x7fe28c102660 2024-04-20T11:44:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.800+0000 7fe283400700 1 -- 172.21.15.117:0/719974574 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe274010ea0 con 0x7fe28c102660 2024-04-20T11:44:18.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.800+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c1971f0 con 0x7fe28c102660 2024-04-20T11:44:18.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.801+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c1054d0 con 0x7fe28c102660 2024-04-20T11:44:18.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.802+0000 7fe283400700 1 -- 172.21.15.117:0/719974574 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe274010460 con 0x7fe28c102660 2024-04-20T11:44:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.803+0000 7fe283400700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe27006ecc0 0x7fe270071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.803+0000 7fe283400700 1 -- 172.21.15.117:0/719974574 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe27408ea90 con 0x7fe28c102660 2024-04-20T11:44:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.804+0000 7fe28a800700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe27006ecc0 0x7fe270071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.804+0000 7fe28a800700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe27006ecc0 0x7fe270071180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009480 tx=0x7fe27c00b7b0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:18.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:18.808+0000 7fe283400700 1 -- 172.21.15.117:0/719974574 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe27405b300 con 0x7fe28c102660 2024-04-20T11:44:19.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.103+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe28c068e50 con 0x7fe28c102660 2024-04-20T11:44:19.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.105+0000 7fe283400700 1 -- 172.21.15.117:0/719974574 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe27405ae90 con 0x7fe28c102660 2024-04-20T11:44:19.105 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:44:19.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:44:19.107 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:44:19.107 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:44:19.107 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:44:19.107 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:44:19.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe27006ecc0 msgr2=0x7fe270071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe27006ecc0 0x7fe270071180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009480 tx=0x7fe27c00b7b0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c102660 msgr2=0x7fe28c19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c102660 0x7fe28c19cdc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe27400d6d0 tx=0x7fe27400da90 comp rx=0 tx=0).stop 2024-04-20T11:44:19.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 shutdown_connections 2024-04-20T11:44:19.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe27006ecc0 0x7fe270071180 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c101d40 0x7fe28c19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 --2- 172.21.15.117:0/719974574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe28c102660 0x7fe28c19cdc0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.108+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 >> 172.21.15.117:0/719974574 conn(0x7fe28c076750 msgr2=0x7fe28c103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:19.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.109+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 shutdown_connections 2024-04-20T11:44:19.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.109+0000 7fe291400700 1 -- 172.21.15.117:0/719974574 wait complete. 2024-04-20T11:44:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.260+0000 7f8dc0600700 1 -- 172.21.15.117:0/2372643222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 msgr2=0x7f8db8102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.260+0000 7f8dc0600700 1 --2- 172.21.15.117:0/2372643222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 0x7f8db8102120 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009910 tx=0x7f8dac009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.260+0000 7f8dc0600700 1 -- 172.21.15.117:0/2372643222 shutdown_connections 2024-04-20T11:44:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.260+0000 7f8dc0600700 1 --2- 172.21.15.117:0/2372643222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8db8102660 0x7f8db810ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.260+0000 7f8dc0600700 1 --2- 172.21.15.117:0/2372643222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 0x7f8db8102120 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.260+0000 7f8dc0600700 1 -- 172.21.15.117:0/2372643222 >> 172.21.15.117:0/2372643222 conn(0x7f8db8076750 msgr2=0x7f8db80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.260+0000 7f8dc0600700 1 -- 172.21.15.117:0/2372643222 shutdown_connections 2024-04-20T11:44:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.260+0000 7f8dc0600700 1 -- 172.21.15.117:0/2372643222 wait complete. 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.261+0000 7f8dc0600700 1 Processor -- start 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.261+0000 7f8dc0600700 1 -- start start 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dc0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 0x7f8db8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dc0600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8db8102660 0x7f8db81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dc0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db8199ac0 con 0x7f8db8101d40 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dc0600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819d790 con 0x7f8db8102660 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dbdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 0x7f8db8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dbdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 0x7f8db8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39666/0 (socket says 172.21.15.117:39666) 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dbdc00700 1 -- 172.21.15.117:0/43545526 learned_addr learned my addr 172.21.15.117:0/43545526 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dbdc00700 1 -- 172.21.15.117:0/43545526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8db8102660 msgr2=0x7f8db81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dbdc00700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8db8102660 0x7f8db81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.262+0000 7f8dbdc00700 1 -- 172.21.15.117:0/43545526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0095f0 con 0x7f8db8101d40 2024-04-20T11:44:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.263+0000 7f8dbdc00700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 0x7f8db8198ea0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8dac006010 tx=0x7f8dac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.263+0000 7f8db6000700 1 -- 172.21.15.117:0/43545526 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dac01d070 con 0x7f8db8101d40 2024-04-20T11:44:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.263+0000 7f8db6000700 1 -- 172.21.15.117:0/43545526 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dac00f460 con 0x7f8db8101d40 2024-04-20T11:44:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.263+0000 7f8db6000700 1 -- 172.21.15.117:0/43545526 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dac017630 con 0x7f8db8101d40 2024-04-20T11:44:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.263+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db819da10 con 0x7f8db8101d40 2024-04-20T11:44:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.263+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db819deb0 con 0x7f8db8101d40 2024-04-20T11:44:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.264+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db810a6b0 con 0x7f8db8101d40 2024-04-20T11:44:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.265+0000 7f8db6000700 1 -- 172.21.15.117:0/43545526 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8dac00f5d0 con 0x7f8db8101d40 2024-04-20T11:44:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.266+0000 7f8db6000700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8da006ecc0 0x7f8da0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.266+0000 7f8db6000700 1 -- 172.21.15.117:0/43545526 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8dac090370 con 0x7f8db8101d40 2024-04-20T11:44:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.266+0000 7f8dbd200700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8da006ecc0 0x7f8da0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.267+0000 7f8dbd200700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8da006ecc0 0x7f8da0071180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8da4006fd0 tx=0x7f8da4006dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:19.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.271+0000 7f8db6000700 1 -- 172.21.15.117:0/43545526 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8dac0591f0 con 0x7f8db8101d40 2024-04-20T11:44:19.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.542+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8db819a1b0 con 0x7f8db8101d40 2024-04-20T11:44:19.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.542+0000 7f8db6000700 1 -- 172.21.15.117:0/43545526 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f8dac05c8b0 con 0x7f8db8101d40 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:44:19.544 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:44:19.545 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:44:19.546 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:44:19.547 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:44:19.547 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:44:19.547 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:44:19.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8da006ecc0 msgr2=0x7f8da0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8da006ecc0 0x7f8da0071180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8da4006fd0 tx=0x7f8da4006dc0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 msgr2=0x7f8db8198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 0x7f8db8198ea0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8dac006010 tx=0x7f8dac00be20 comp rx=0 tx=0).stop 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 shutdown_connections 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8da006ecc0 0x7f8da0071180 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8db8101d40 0x7f8db8198ea0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 --2- 172.21.15.117:0/43545526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8db8102660 0x7f8db81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 >> 172.21.15.117:0/43545526 conn(0x7f8db8076750 msgr2=0x7f8db8101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.545+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 shutdown_connections 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.546+0000 7f8dc0600700 1 -- 172.21.15.117:0/43545526 wait complete. 2024-04-20T11:44:19.548 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:44:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:19 smithi117 ceph-mon[99036]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:19 smithi117 ceph-mon[99036]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:19 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/719974574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:44:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:19 smithi173 ceph-mon[74165]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:19 smithi173 ceph-mon[74165]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:19 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/719974574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:44:19.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.698+0000 7f4c57200700 1 -- 172.21.15.117:0/2787366015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c501039d0 msgr2=0x7f4c50107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.698+0000 7f4c57200700 1 --2- 172.21.15.117:0/2787366015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c501039d0 0x7f4c50107a20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009910 tx=0x7f4c3c009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:19.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.698+0000 7f4c57200700 1 -- 172.21.15.117:0/2787366015 shutdown_connections 2024-04-20T11:44:19.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.698+0000 7f4c57200700 1 --2- 172.21.15.117:0/2787366015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c501039d0 0x7f4c50107a20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.698+0000 7f4c57200700 1 --2- 172.21.15.117:0/2787366015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c50103020 0x7f4c50103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.698+0000 7f4c57200700 1 -- 172.21.15.117:0/2787366015 >> 172.21.15.117:0/2787366015 conn(0x7f4c500fe890 msgr2=0x7f4c50100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:19.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.698+0000 7f4c57200700 1 -- 172.21.15.117:0/2787366015 shutdown_connections 2024-04-20T11:44:19.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.698+0000 7f4c57200700 1 -- 172.21.15.117:0/2787366015 wait complete. 2024-04-20T11:44:19.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.699+0000 7f4c57200700 1 Processor -- start 2024-04-20T11:44:19.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.699+0000 7f4c57200700 1 -- start start 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.699+0000 7f4c57200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c50103020 0x7f4c50198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.699+0000 7f4c57200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c501039d0 0x7f4c50199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c57200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50199890 con 0x7f4c50103020 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c57200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50199a00 con 0x7f4c501039d0 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c4fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c50103020 0x7f4c50198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c4fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c50103020 0x7f4c50198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39682/0 (socket says 172.21.15.117:39682) 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c4fe00700 1 -- 172.21.15.117:0/1610345733 learned_addr learned my addr 172.21.15.117:0/1610345733 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c4fe00700 1 -- 172.21.15.117:0/1610345733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c501039d0 msgr2=0x7f4c50199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c4fe00700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c501039d0 0x7f4c50199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c4fe00700 1 -- 172.21.15.117:0/1610345733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0095f0 con 0x7f4c50103020 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.700+0000 7f4c4fe00700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c50103020 0x7f4c50198d50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000c930 tx=0x7f4c4000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.701+0000 7f4c4cc00700 1 -- 172.21.15.117:0/1610345733 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c40007ad0 con 0x7f4c50103020 2024-04-20T11:44:19.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.701+0000 7f4c4cc00700 1 -- 172.21.15.117:0/1610345733 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c40007c30 con 0x7f4c50103020 2024-04-20T11:44:19.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.701+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5019c690 con 0x7f4c50103020 2024-04-20T11:44:19.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.701+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5019cbe0 con 0x7f4c50103020 2024-04-20T11:44:19.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.701+0000 7f4c4cc00700 1 -- 172.21.15.117:0/1610345733 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c40018610 con 0x7f4c50103020 2024-04-20T11:44:19.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.702+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c300052f0 con 0x7f4c50103020 2024-04-20T11:44:19.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.703+0000 7f4c4cc00700 1 -- 172.21.15.117:0/1610345733 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4c400188b0 con 0x7f4c50103020 2024-04-20T11:44:19.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.704+0000 7f4c4cc00700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c3406ecd0 0x7f4c34071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:19.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.704+0000 7f4c4cc00700 1 -- 172.21.15.117:0/1610345733 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f4c4008efd0 con 0x7f4c50103020 2024-04-20T11:44:19.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.705+0000 7f4c4f400700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c3406ecd0 0x7f4c34071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:19.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.705+0000 7f4c4f400700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c3406ecd0 0x7f4c34071190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006010 tx=0x7f4c3c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:19.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.709+0000 7f4c4cc00700 1 -- 172.21.15.117:0/1610345733 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4c400c3ff0 con 0x7f4c50103020 2024-04-20T11:44:19.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.936+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c30000bc0 con 0x7f4c3406ecd0 2024-04-20T11:44:19.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.938+0000 7f4c4cc00700 1 -- 172.21.15.117:0/1610345733 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4c30000bc0 con 0x7f4c3406ecd0 2024-04-20T11:44:19.939 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:44:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:44:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:44:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:44:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:44:19.939 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:44:19.940 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:44:19.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c3406ecd0 msgr2=0x7f4c34071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c3406ecd0 0x7f4c34071190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006010 tx=0x7f4c3c00b540 comp rx=0 tx=0).stop 2024-04-20T11:44:19.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c50103020 msgr2=0x7f4c50198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:19.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c50103020 0x7f4c50198d50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000c930 tx=0x7f4c4000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 shutdown_connections 2024-04-20T11:44:19.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c3406ecd0 0x7f4c34071190 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c50103020 0x7f4c50198d50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 --2- 172.21.15.117:0/1610345733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c501039d0 0x7f4c50199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:19.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 >> 172.21.15.117:0/1610345733 conn(0x7f4c500fe890 msgr2=0x7f4c500fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:19.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 shutdown_connections 2024-04-20T11:44:19.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:19.941+0000 7f4c57200700 1 -- 172.21.15.117:0/1610345733 wait complete. 2024-04-20T11:44:20.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.096+0000 7f95e7200700 1 -- 172.21.15.117:0/2225873914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 msgr2=0x7f95e8107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:20.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.096+0000 7f95e7200700 1 --2- 172.21.15.117:0/2225873914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 0x7f95e8107510 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009910 tx=0x7f95d4009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:20.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.097+0000 7f95e7200700 1 -- 172.21.15.117:0/2225873914 shutdown_connections 2024-04-20T11:44:20.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.097+0000 7f95e7200700 1 --2- 172.21.15.117:0/2225873914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 0x7f95e8107510 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:20.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.097+0000 7f95e7200700 1 --2- 172.21.15.117:0/2225873914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e810d0c0 0x7f95e810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:20.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.097+0000 7f95e7200700 1 -- 172.21.15.117:0/2225873914 >> 172.21.15.117:0/2225873914 conn(0x7f95e8076830 msgr2=0x7f95e8078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:20.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.097+0000 7f95e7200700 1 -- 172.21.15.117:0/2225873914 shutdown_connections 2024-04-20T11:44:20.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.097+0000 7f95e7200700 1 -- 172.21.15.117:0/2225873914 wait complete. 2024-04-20T11:44:20.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.098+0000 7f95e7200700 1 Processor -- start 2024-04-20T11:44:20.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.098+0000 7f95e7200700 1 -- start start 2024-04-20T11:44:20.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.099+0000 7f95e7200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 0x7f95e819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:20.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.099+0000 7f95e7200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e810d0c0 0x7f95e819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:20.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.099+0000 7f95e7200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e819db80 con 0x7f95e81070b0 2024-04-20T11:44:20.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.099+0000 7f95e7200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e81a1850 con 0x7f95e810d0c0 2024-04-20T11:44:20.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.099+0000 7f95e5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 0x7f95e819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:20.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.099+0000 7f95e5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 0x7f95e819cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39704/0 (socket says 172.21.15.117:39704) 2024-04-20T11:44:20.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.099+0000 7f95e5e00700 1 -- 172.21.15.117:0/3000971958 learned_addr learned my addr 172.21.15.117:0/3000971958 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:20.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.100+0000 7f95e5400700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e810d0c0 0x7f95e819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:20.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.100+0000 7f95e5e00700 1 -- 172.21.15.117:0/3000971958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e810d0c0 msgr2=0x7f95e819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:20.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.100+0000 7f95e5e00700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e810d0c0 0x7f95e819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:20.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.100+0000 7f95e5e00700 1 -- 172.21.15.117:0/3000971958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40095f0 con 0x7f95e81070b0 2024-04-20T11:44:20.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.100+0000 7f95e5400700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e810d0c0 0x7f95e819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:44:20.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.100+0000 7f95e5e00700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 0x7f95e819cf60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f95d800c8a0 tx=0x7f95d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:20.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.101+0000 7f95dea00700 1 -- 172.21.15.117:0/3000971958 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d800cea0 con 0x7f95e81070b0 2024-04-20T11:44:20.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.101+0000 7f95dea00700 1 -- 172.21.15.117:0/3000971958 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95d8004830 con 0x7f95e81070b0 2024-04-20T11:44:20.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.101+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e81a1b30 con 0x7f95e81070b0 2024-04-20T11:44:20.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.101+0000 7f95dea00700 1 -- 172.21.15.117:0/3000971958 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d80055b0 con 0x7f95e81070b0 2024-04-20T11:44:20.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.101+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e81a2080 con 0x7f95e81070b0 2024-04-20T11:44:20.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.102+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e810f5c0 con 0x7f95e81070b0 2024-04-20T11:44:20.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.103+0000 7f95dea00700 1 -- 172.21.15.117:0/3000971958 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f95d80049a0 con 0x7f95e81070b0 2024-04-20T11:44:20.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.104+0000 7f95dea00700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f95cc06ecd0 0x7f95cc071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.105+0000 7f95dea00700 1 -- 172.21.15.117:0/3000971958 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f95d808eac0 con 0x7f95e81070b0 2024-04-20T11:44:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.105+0000 7f95e5400700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f95cc06ecd0 0x7f95cc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.106+0000 7f95e5400700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f95cc06ecd0 0x7f95cc071190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f95d4006010 tx=0x7f95d400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:20.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.110+0000 7f95dea00700 1 -- 172.21.15.117:0/3000971958 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f95d8057880 con 0x7f95e81070b0 2024-04-20T11:44:20.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.421+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f95e819e270 con 0x7f95e81070b0 2024-04-20T11:44:20.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.422+0000 7f95dea00700 1 -- 172.21.15.117:0/3000971958 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f95d801d090 con 0x7f95e81070b0 2024-04-20T11:44:20.422 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:44:20.423 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:44:20.424 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:44:20.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.424+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f95cc06ecd0 msgr2=0x7f95cc071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:20.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.424+0000 7f95e7200700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f95cc06ecd0 0x7f95cc071190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f95d4006010 tx=0x7f95d400b540 comp rx=0 tx=0).stop 2024-04-20T11:44:20.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 msgr2=0x7f95e819cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:20.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 0x7f95e819cf60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f95d800c8a0 tx=0x7f95d800cc60 comp rx=0 tx=0).stop 2024-04-20T11:44:20.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 shutdown_connections 2024-04-20T11:44:20.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f95cc06ecd0 0x7f95cc071190 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:20.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e81070b0 0x7f95e819cf60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:20.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 --2- 172.21.15.117:0/3000971958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e810d0c0 0x7f95e819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:20.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 >> 172.21.15.117:0/3000971958 conn(0x7f95e8076830 msgr2=0x7f95e8078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:20.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 shutdown_connections 2024-04-20T11:44:20.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:20.425+0000 7f95e7200700 1 -- 172.21.15.117:0/3000971958 wait complete. 2024-04-20T11:44:20.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:20 smithi117 ceph-mon[99036]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:20.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:20 smithi117 ceph-mon[99036]: pgmap v957: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:20.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:20 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/43545526' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:44:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:20 smithi173 ceph-mon[74165]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:20 smithi173 ceph-mon[74165]: pgmap v957: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:20 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/43545526' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:44:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:21 smithi117 ceph-mon[99036]: from='client.35730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:21 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3000971958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:44:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:21 smithi173 ceph-mon[74165]: from='client.35730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:21 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3000971958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:44:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:22 smithi117 ceph-mon[99036]: pgmap v958: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:22 smithi173 ceph-mon[74165]: pgmap v958: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:24 smithi173 ceph-mon[74165]: pgmap v959: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:24.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:24 smithi117 ceph-mon[99036]: pgmap v959: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:26.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:26 smithi173 ceph-mon[74165]: pgmap v960: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:26.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:26 smithi117 ceph-mon[99036]: pgmap v960: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:28.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:28 smithi117 ceph-mon[99036]: pgmap v961: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:28.659 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:44:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:28 smithi173 ceph-mon[74165]: pgmap v961: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:44:30.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:30 smithi117 ceph-mon[99036]: pgmap v962: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:30.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:30 smithi173 ceph-mon[74165]: pgmap v962: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:32 smithi117 ceph-mon[99036]: pgmap v963: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:32 smithi173 ceph-mon[74165]: pgmap v963: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:34.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:34 smithi117 ceph-mon[99036]: pgmap v964: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:34 smithi173 ceph-mon[74165]: pgmap v964: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:36.714 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:36 smithi173 ceph-mon[74165]: pgmap v965: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:36.757 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:36 smithi117 ceph-mon[99036]: pgmap v965: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:38.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:38 smithi117 ceph-mon[99036]: pgmap v966: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:38 smithi173 ceph-mon[74165]: pgmap v966: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:40.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:40 smithi117 ceph-mon[99036]: pgmap v967: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:40.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:40 smithi173 ceph-mon[74165]: pgmap v967: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:42.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:42 smithi117 ceph-mon[99036]: pgmap v968: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:42 smithi173 ceph-mon[74165]: pgmap v968: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:44:43.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:44:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:44 smithi117 ceph-mon[99036]: pgmap v969: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:44 smithi173 ceph-mon[74165]: pgmap v969: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:46.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:46 smithi117 ceph-mon[99036]: pgmap v970: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:46 smithi173 ceph-mon[74165]: pgmap v970: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:48.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:48 smithi117 ceph-mon[99036]: pgmap v971: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:48 smithi173 ceph-mon[74165]: pgmap v971: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:50.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.596+0000 7fc9f5200700 1 -- 172.21.15.117:0/751516277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073c30 msgr2=0x7fc9f010cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:50.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.596+0000 7fc9f5200700 1 --2- 172.21.15.117:0/751516277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073c30 0x7fc9f010cec0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009910 tx=0x7fc9dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:50.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.596+0000 7fc9f5200700 1 -- 172.21.15.117:0/751516277 shutdown_connections 2024-04-20T11:44:50.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.596+0000 7fc9f5200700 1 --2- 172.21.15.117:0/751516277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073c30 0x7fc9f010cec0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:50.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.596+0000 7fc9f5200700 1 --2- 172.21.15.117:0/751516277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f0073310 0x7fc9f00736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:50.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.596+0000 7fc9f5200700 1 -- 172.21.15.117:0/751516277 >> 172.21.15.117:0/751516277 conn(0x7fc9f00fc6f0 msgr2=0x7fc9f00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:50.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.597+0000 7fc9f5200700 1 -- 172.21.15.117:0/751516277 shutdown_connections 2024-04-20T11:44:50.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.597+0000 7fc9f5200700 1 -- 172.21.15.117:0/751516277 wait complete. 2024-04-20T11:44:50.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.598+0000 7fc9f5200700 1 Processor -- start 2024-04-20T11:44:50.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.598+0000 7fc9f5200700 1 -- start start 2024-04-20T11:44:50.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.598+0000 7fc9f5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073310 0x7fc9f0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:50.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.598+0000 7fc9f5200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f0073c30 0x7fc9f0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:50.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.598+0000 7fc9f5200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f01998c0 con 0x7fc9f0073310 2024-04-20T11:44:50.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.598+0000 7fc9f5200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f019c530 con 0x7fc9f0073c30 2024-04-20T11:44:50.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.599+0000 7fc9ede00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073310 0x7fc9f0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:50.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.599+0000 7fc9ede00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073310 0x7fc9f0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33340/0 (socket says 172.21.15.117:33340) 2024-04-20T11:44:50.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.599+0000 7fc9ede00700 1 -- 172.21.15.117:0/704717345 learned_addr learned my addr 172.21.15.117:0/704717345 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:50.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.599+0000 7fc9ed400700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f0073c30 0x7fc9f0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:50.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.599+0000 7fc9ede00700 1 -- 172.21.15.117:0/704717345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f0073c30 msgr2=0x7fc9f0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:50.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.599+0000 7fc9ede00700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f0073c30 0x7fc9f0199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:50.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.599+0000 7fc9ede00700 1 -- 172.21.15.117:0/704717345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc0095f0 con 0x7fc9f0073310 2024-04-20T11:44:50.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.600+0000 7fc9ede00700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073310 0x7fc9f0198cf0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d8d0 tx=0x7fc9e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:50.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.600+0000 7fc9e6a00700 1 -- 172.21.15.117:0/704717345 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e0009940 con 0x7fc9f0073310 2024-04-20T11:44:50.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.600+0000 7fc9e6a00700 1 -- 172.21.15.117:0/704717345 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9e0010460 con 0x7fc9f0073310 2024-04-20T11:44:50.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.600+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f019c810 con 0x7fc9f0073310 2024-04-20T11:44:50.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.600+0000 7fc9e6a00700 1 -- 172.21.15.117:0/704717345 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e000f5f0 con 0x7fc9f0073310 2024-04-20T11:44:50.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.600+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f019cd60 con 0x7fc9f0073310 2024-04-20T11:44:50.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.601+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f00749c0 con 0x7fc9f0073310 2024-04-20T11:44:50.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.602+0000 7fc9e6a00700 1 -- 172.21.15.117:0/704717345 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc9e00105d0 con 0x7fc9f0073310 2024-04-20T11:44:50.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.603+0000 7fc9e6a00700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc9d406ecc0 0x7fc9d4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:50.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.603+0000 7fc9e6a00700 1 -- 172.21.15.117:0/704717345 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc9e008ede0 con 0x7fc9f0073310 2024-04-20T11:44:50.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.604+0000 7fc9ed400700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc9d406ecc0 0x7fc9d4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:50.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.604+0000 7fc9ed400700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc9d406ecc0 0x7fc9d4071180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b5c0 tx=0x7fc9dc0052a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:50.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.609+0000 7fc9e6a00700 1 -- 172.21.15.117:0/704717345 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc9e0057ba0 con 0x7fc9f0073310 2024-04-20T11:44:50.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.850+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9f0107790 con 0x7fc9d406ecc0 2024-04-20T11:44:50.850 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:50 smithi117 ceph-mon[99036]: pgmap v972: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:50.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.853+0000 7fc9e6a00700 1 -- 172.21.15.117:0/704717345 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc9f0107790 con 0x7fc9d406ecc0 2024-04-20T11:44:50.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.856+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc9d406ecc0 msgr2=0x7fc9d4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:50.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.856+0000 7fc9f5200700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc9d406ecc0 0x7fc9d4071180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b5c0 tx=0x7fc9dc0052a0 comp rx=0 tx=0).stop 2024-04-20T11:44:50.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.856+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073310 msgr2=0x7fc9f0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:50.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.856+0000 7fc9f5200700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073310 0x7fc9f0198cf0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d8d0 tx=0x7fc9e000dc90 comp rx=0 tx=0).stop 2024-04-20T11:44:50.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.857+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 shutdown_connections 2024-04-20T11:44:50.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.857+0000 7fc9f5200700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc9d406ecc0 0x7fc9d4071180 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:50.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.857+0000 7fc9f5200700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0073310 0x7fc9f0198cf0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:50.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.857+0000 7fc9f5200700 1 --2- 172.21.15.117:0/704717345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f0073c30 0x7fc9f0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:50.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.857+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 >> 172.21.15.117:0/704717345 conn(0x7fc9f00fc6f0 msgr2=0x7fc9f01053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:50.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.857+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 shutdown_connections 2024-04-20T11:44:50.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:50.857+0000 7fc9f5200700 1 -- 172.21.15.117:0/704717345 wait complete. 2024-04-20T11:44:50.872 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:44:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:50 smithi173 ceph-mon[74165]: pgmap v972: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.012+0000 7fd787600700 1 -- 172.21.15.117:0/3455082044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd780103020 msgr2=0x7fd780103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.012+0000 7fd787600700 1 --2- 172.21.15.117:0/3455082044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd780103020 0x7fd780103400 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd774009910 tx=0x7fd774009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.012+0000 7fd787600700 1 -- 172.21.15.117:0/3455082044 shutdown_connections 2024-04-20T11:44:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.012+0000 7fd787600700 1 --2- 172.21.15.117:0/3455082044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7801039d0 0x7fd780107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.012+0000 7fd787600700 1 --2- 172.21.15.117:0/3455082044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd780103020 0x7fd780103400 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.012+0000 7fd787600700 1 -- 172.21.15.117:0/3455082044 >> 172.21.15.117:0/3455082044 conn(0x7fd7800fe8b0 msgr2=0x7fd780100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.012+0000 7fd787600700 1 -- 172.21.15.117:0/3455082044 shutdown_connections 2024-04-20T11:44:51.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.012+0000 7fd787600700 1 -- 172.21.15.117:0/3455082044 wait complete. 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.013+0000 7fd787600700 1 Processor -- start 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.013+0000 7fd787600700 1 -- start start 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd787600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd780103020 0x7fd780198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd787600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7801039d0 0x7fd7801992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd787600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd780199930 con 0x7fd7801039d0 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd787600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78019c550 con 0x7fd780103020 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd77fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7801039d0 0x7fd7801992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd77fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7801039d0 0x7fd7801992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33358/0 (socket says 172.21.15.117:33358) 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd77fe00700 1 -- 172.21.15.117:0/4294172957 learned_addr learned my addr 172.21.15.117:0/4294172957 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd784c00700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd780103020 0x7fd780198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:51.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd77fe00700 1 -- 172.21.15.117:0/4294172957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd780103020 msgr2=0x7fd780198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd77fe00700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd780103020 0x7fd780198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd77fe00700 1 -- 172.21.15.117:0/4294172957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740095f0 con 0x7fd7801039d0 2024-04-20T11:44:51.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.014+0000 7fd784c00700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd780103020 0x7fd780198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:44:51.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.015+0000 7fd77fe00700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7801039d0 0x7fd7801992a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00d900 tx=0x7fd76c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:51.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.015+0000 7fd77d600700 1 -- 172.21.15.117:0/4294172957 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd76c009900 con 0x7fd7801039d0 2024-04-20T11:44:51.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.015+0000 7fd77d600700 1 -- 172.21.15.117:0/4294172957 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd76c010460 con 0x7fd7801039d0 2024-04-20T11:44:51.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.015+0000 7fd77d600700 1 -- 172.21.15.117:0/4294172957 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd76c00f5f0 con 0x7fd7801039d0 2024-04-20T11:44:51.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.015+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78019c830 con 0x7fd7801039d0 2024-04-20T11:44:51.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.015+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78019cd80 con 0x7fd7801039d0 2024-04-20T11:44:51.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.016+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78010b3c0 con 0x7fd7801039d0 2024-04-20T11:44:51.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.017+0000 7fd77d600700 1 -- 172.21.15.117:0/4294172957 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd76c0105d0 con 0x7fd7801039d0 2024-04-20T11:44:51.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.018+0000 7fd77d600700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd76806ecc0 0x7fd768071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.018+0000 7fd77d600700 1 -- 172.21.15.117:0/4294172957 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd76c08ec70 con 0x7fd7801039d0 2024-04-20T11:44:51.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.019+0000 7fd784c00700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd76806ecc0 0x7fd768071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:51.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.019+0000 7fd784c00700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd76806ecc0 0x7fd768071180 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd774006010 tx=0x7fd77400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:51.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.023+0000 7fd77d600700 1 -- 172.21.15.117:0/4294172957 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd76c095080 con 0x7fd7801039d0 2024-04-20T11:44:51.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.254+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd78019a050 con 0x7fd76806ecc0 2024-04-20T11:44:51.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.257+0000 7fd77d600700 1 -- 172.21.15.117:0/4294172957 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd78019a050 con 0x7fd76806ecc0 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd76806ecc0 msgr2=0x7fd768071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd76806ecc0 0x7fd768071180 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd774006010 tx=0x7fd77400b540 comp rx=0 tx=0).stop 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7801039d0 msgr2=0x7fd7801992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7801039d0 0x7fd7801992a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00d900 tx=0x7fd76c00dcc0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 shutdown_connections 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd76806ecc0 0x7fd768071180 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd780103020 0x7fd780198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 --2- 172.21.15.117:0/4294172957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7801039d0 0x7fd7801992a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 >> 172.21.15.117:0/4294172957 conn(0x7fd7800fe8b0 msgr2=0x7fd7800fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:51.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 shutdown_connections 2024-04-20T11:44:51.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.260+0000 7fd787600700 1 -- 172.21.15.117:0/4294172957 wait complete. 2024-04-20T11:44:51.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.417+0000 7f6b21c00700 1 -- 172.21.15.117:0/2513819508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 msgr2=0x7f6b1c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.417+0000 7f6b21c00700 1 --2- 172.21.15.117:0/2513819508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 0x7f6b1c10abb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009940 tx=0x7f6b08009c50 comp rx=0 tx=0).stop 2024-04-20T11:44:51.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.417+0000 7f6b21c00700 1 -- 172.21.15.117:0/2513819508 shutdown_connections 2024-04-20T11:44:51.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.417+0000 7f6b21c00700 1 --2- 172.21.15.117:0/2513819508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 0x7f6b1c10abb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.417+0000 7f6b21c00700 1 --2- 172.21.15.117:0/2513819508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b1c101da0 0x7f6b1c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.417+0000 7f6b21c00700 1 -- 172.21.15.117:0/2513819508 >> 172.21.15.117:0/2513819508 conn(0x7f6b1c076750 msgr2=0x7f6b1c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:51.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.417+0000 7f6b21c00700 1 -- 172.21.15.117:0/2513819508 shutdown_connections 2024-04-20T11:44:51.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.417+0000 7f6b21c00700 1 -- 172.21.15.117:0/2513819508 wait complete. 2024-04-20T11:44:51.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.418+0000 7f6b21c00700 1 Processor -- start 2024-04-20T11:44:51.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.418+0000 7f6b21c00700 1 -- start start 2024-04-20T11:44:51.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b21c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b1c101da0 0x7f6b1c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b21c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 0x7f6b1c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b21c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c199a50 con 0x7f6b1c1026c0 2024-04-20T11:44:51.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b21c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c19d720 con 0x7f6b1c101da0 2024-04-20T11:44:51.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b1a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 0x7f6b1c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:51.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b1a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 0x7f6b1c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33380/0 (socket says 172.21.15.117:33380) 2024-04-20T11:44:51.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b1a000700 1 -- 172.21.15.117:0/3078149385 learned_addr learned my addr 172.21.15.117:0/3078149385 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b1aa00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b1c101da0 0x7f6b1c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.419+0000 7f6b1a000700 1 -- 172.21.15.117:0/3078149385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b1c101da0 msgr2=0x7f6b1c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b1a000700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b1c101da0 0x7f6b1c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b1a000700 1 -- 172.21.15.117:0/3078149385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080095f0 con 0x7f6b1c1026c0 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b1aa00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b1c101da0 0x7f6b1c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b1a000700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 0x7f6b1c199370 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800b870 tx=0x7f6b080056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b13400700 1 -- 172.21.15.117:0/3078149385 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b0801d070 con 0x7f6b1c1026c0 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b13400700 1 -- 172.21.15.117:0/3078149385 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b0800bed0 con 0x7f6b1c1026c0 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c19d9a0 con 0x7f6b1c1026c0 2024-04-20T11:44:51.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b13400700 1 -- 172.21.15.117:0/3078149385 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b08021410 con 0x7f6b1c1026c0 2024-04-20T11:44:51.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.420+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c19de90 con 0x7f6b1c1026c0 2024-04-20T11:44:51.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.421+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c10a710 con 0x7f6b1c1026c0 2024-04-20T11:44:51.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.423+0000 7f6b13400700 1 -- 172.21.15.117:0/3078149385 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6b0800f810 con 0x7f6b1c1026c0 2024-04-20T11:44:51.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.423+0000 7f6b13400700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b0006ecc0 0x7f6b00071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.424+0000 7f6b13400700 1 -- 172.21.15.117:0/3078149385 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6b0808ffe0 con 0x7f6b1c1026c0 2024-04-20T11:44:51.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.424+0000 7f6b1aa00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b0006ecc0 0x7f6b00071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.425+0000 7f6b1aa00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b0006ecc0 0x7f6b00071180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c007900 tx=0x7f6b0c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:51.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.429+0000 7f6b13400700 1 -- 172.21.15.117:0/3078149385 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6b080c4ff0 con 0x7f6b1c1026c0 2024-04-20T11:44:51.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.664+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b1c10de20 con 0x7f6b0006ecc0 2024-04-20T11:44:51.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.687+0000 7f6b13400700 1 -- 172.21.15.117:0/3078149385 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6b1c10de20 con 0x7f6b0006ecc0 2024-04-20T11:44:51.687 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:44:51.688 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (33m) 9m ago 47m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:44:51.688 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (47m) 9m ago 47m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:44:51.688 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (46m) 9m ago 46m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:44:51.688 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (29m) 9m ago 47m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:44:51.688 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (29m) 9m ago 46m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:44:51.689 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (32m) 9m ago 47m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:44:51.689 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (25m) 9m ago 40m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:44:51.689 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (25m) 9m ago 40m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:44:51.689 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (9m) 9m ago 40m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:44:51.689 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (24m) 9m ago 40m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:44:51.689 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (36m) 9m ago 49m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:44:51.690 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (34m) 9m ago 46m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:44:51.690 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (31m) 9m ago 49m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:44:51.690 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (30m) 9m ago 45m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:44:51.690 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (34m) 9m ago 47m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:44:51.690 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (34m) 9m ago 46m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:44:51.690 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (28m) 9m ago 44m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:44:51.690 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (28m) 9m ago 44m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:44:51.691 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (27m) 9m ago 43m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:44:51.691 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (27m) 9m ago 43m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:44:51.691 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (26m) 9m ago 42m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:44:51.691 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (25m) 9m ago 42m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:44:51.691 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (33m) 9m ago 46m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:44:51.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b0006ecc0 msgr2=0x7f6b00071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b0006ecc0 0x7f6b00071180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c007900 tx=0x7f6b0c008040 comp rx=0 tx=0).stop 2024-04-20T11:44:51.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 msgr2=0x7f6b1c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 0x7f6b1c199370 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800b870 tx=0x7f6b080056a0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 shutdown_connections 2024-04-20T11:44:51.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b0006ecc0 0x7f6b00071180 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b1c101da0 0x7f6b1c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 --2- 172.21.15.117:0/3078149385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1c1026c0 0x7f6b1c199370 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 >> 172.21.15.117:0/3078149385 conn(0x7f6b1c076750 msgr2=0x7f6b1c101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:51.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.690+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 shutdown_connections 2024-04-20T11:44:51.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.691+0000 7f6b21c00700 1 -- 172.21.15.117:0/3078149385 wait complete. 2024-04-20T11:44:51.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.837+0000 7fbb84c00700 1 -- 172.21.15.117:0/2642258810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80104a00 msgr2=0x7fbb80104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.837+0000 7fbb84c00700 1 --2- 172.21.15.117:0/2642258810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80104a00 0x7fbb80104e80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009910 tx=0x7fbb6c009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:51.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.837+0000 7fbb84c00700 1 -- 172.21.15.117:0/2642258810 shutdown_connections 2024-04-20T11:44:51.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.837+0000 7fbb84c00700 1 --2- 172.21.15.117:0/2642258810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80104a00 0x7fbb80104e80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.837+0000 7fbb84c00700 1 --2- 172.21.15.117:0/2642258810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb80103110 0x7fbb801034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.837+0000 7fbb84c00700 1 -- 172.21.15.117:0/2642258810 >> 172.21.15.117:0/2642258810 conn(0x7fbb800fe880 msgr2=0x7fbb80100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:51.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.837+0000 7fbb84c00700 1 -- 172.21.15.117:0/2642258810 shutdown_connections 2024-04-20T11:44:51.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.837+0000 7fbb84c00700 1 -- 172.21.15.117:0/2642258810 wait complete. 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.838+0000 7fbb84c00700 1 Processor -- start 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.838+0000 7fbb84c00700 1 -- start start 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb84c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80103110 0x7fbb80198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb84c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb80104a00 0x7fbb801993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb84c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80199ac0 con 0x7fbb80103110 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb84c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8019d740 con 0x7fbb80104a00 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb7de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80103110 0x7fbb80198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb7de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80103110 0x7fbb80198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33410/0 (socket says 172.21.15.117:33410) 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb7de00700 1 -- 172.21.15.117:0/1925566685 learned_addr learned my addr 172.21.15.117:0/1925566685 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb7de00700 1 -- 172.21.15.117:0/1925566685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb80104a00 msgr2=0x7fbb801993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb7de00700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb80104a00 0x7fbb801993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:51.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.839+0000 7fbb7de00700 1 -- 172.21.15.117:0/1925566685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0095f0 con 0x7fbb80103110 2024-04-20T11:44:51.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.840+0000 7fbb7de00700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80103110 0x7fbb80198ea0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000b4b0 tx=0x7fbb7000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:51.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.840+0000 7fbb76a00700 1 -- 172.21.15.117:0/1925566685 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb70011800 con 0x7fbb80103110 2024-04-20T11:44:51.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.840+0000 7fbb76a00700 1 -- 172.21.15.117:0/1925566685 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb70011e20 con 0x7fbb80103110 2024-04-20T11:44:51.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.840+0000 7fbb76a00700 1 -- 172.21.15.117:0/1925566685 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb7000f2e0 con 0x7fbb80103110 2024-04-20T11:44:51.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.840+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8019da20 con 0x7fbb80103110 2024-04-20T11:44:51.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.840+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8019dff0 con 0x7fbb80103110 2024-04-20T11:44:51.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.841+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8010b460 con 0x7fbb80103110 2024-04-20T11:44:51.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.842+0000 7fbb76a00700 1 -- 172.21.15.117:0/1925566685 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbb70010380 con 0x7fbb80103110 2024-04-20T11:44:51.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.843+0000 7fbb76a00700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6406ecd0 0x7fbb64071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:51.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.843+0000 7fbb76a00700 1 -- 172.21.15.117:0/1925566685 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbb7008ea00 con 0x7fbb80103110 2024-04-20T11:44:51.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.843+0000 7fbb7d400700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6406ecd0 0x7fbb64071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:51.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.844+0000 7fbb7d400700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6406ecd0 0x7fbb64071190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c006010 tx=0x7fbb6c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:51.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:51.848+0000 7fbb76a00700 1 -- 172.21.15.117:0/1925566685 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbb70057870 con 0x7fbb80103110 2024-04-20T11:44:52.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.167+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbb8019a160 con 0x7fbb80103110 2024-04-20T11:44:52.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.168+0000 7fbb76a00700 1 -- 172.21.15.117:0/1925566685 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbb7005af30 con 0x7fbb80103110 2024-04-20T11:44:52.168 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:44:52.168 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:44:52.168 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:44:52.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:44:52.168 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:44:52.168 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:44:52.169 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6406ecd0 msgr2=0x7fbb64071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6406ecd0 0x7fbb64071190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c006010 tx=0x7fbb6c00b540 comp rx=0 tx=0).stop 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80103110 msgr2=0x7fbb80198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80103110 0x7fbb80198ea0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000b4b0 tx=0x7fbb7000b870 comp rx=0 tx=0).stop 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 shutdown_connections 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbb6406ecd0 0x7fbb64071190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb80103110 0x7fbb80198ea0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 --2- 172.21.15.117:0/1925566685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb80104a00 0x7fbb801993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.171+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 >> 172.21.15.117:0/1925566685 conn(0x7fbb800fe880 msgr2=0x7fbb800ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:52.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.172+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 shutdown_connections 2024-04-20T11:44:52.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.172+0000 7fbb84c00700 1 -- 172.21.15.117:0/1925566685 wait complete. 2024-04-20T11:44:52.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.332+0000 7f42c2c00700 1 -- 172.21.15.117:0/894099741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 msgr2=0x7f42bc107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:52.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.332+0000 7f42c2c00700 1 --2- 172.21.15.117:0/894099741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 0x7f42bc107a40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009910 tx=0x7f42a8009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:52.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.332+0000 7f42c2c00700 1 -- 172.21.15.117:0/894099741 shutdown_connections 2024-04-20T11:44:52.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.332+0000 7f42c2c00700 1 --2- 172.21.15.117:0/894099741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 0x7f42bc107a40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.332+0000 7f42c2c00700 1 --2- 172.21.15.117:0/894099741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42bc103040 0x7f42bc103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.332+0000 7f42c2c00700 1 -- 172.21.15.117:0/894099741 >> 172.21.15.117:0/894099741 conn(0x7f42bc0fe8d0 msgr2=0x7f42bc100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:52.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.332+0000 7f42c2c00700 1 -- 172.21.15.117:0/894099741 shutdown_connections 2024-04-20T11:44:52.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.332+0000 7f42c2c00700 1 -- 172.21.15.117:0/894099741 wait complete. 2024-04-20T11:44:52.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.333+0000 7f42c2c00700 1 Processor -- start 2024-04-20T11:44:52.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.333+0000 7f42c2c00700 1 -- start start 2024-04-20T11:44:52.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.334+0000 7f42c2c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42bc103040 0x7f42bc0753f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:52.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.334+0000 7f42c2c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 0x7f42bc075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:52.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.334+0000 7f42c2c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc079470 con 0x7f42bc1039f0 2024-04-20T11:44:52.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.334+0000 7f42c2c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0795e0 con 0x7f42bc103040 2024-04-20T11:44:52.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.334+0000 7f42bb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 0x7f42bc075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:52.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.334+0000 7f42bb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 0x7f42bc075930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33424/0 (socket says 172.21.15.117:33424) 2024-04-20T11:44:52.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.334+0000 7f42bb400700 1 -- 172.21.15.117:0/728359511 learned_addr learned my addr 172.21.15.117:0/728359511 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:52.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.334+0000 7f42bbe00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42bc103040 0x7f42bc0753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:52.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.335+0000 7f42bb400700 1 -- 172.21.15.117:0/728359511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42bc103040 msgr2=0x7f42bc0753f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:52.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.335+0000 7f42bb400700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42bc103040 0x7f42bc0753f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.335+0000 7f42bb400700 1 -- 172.21.15.117:0/728359511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80095f0 con 0x7f42bc1039f0 2024-04-20T11:44:52.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.335+0000 7f42bb400700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 0x7f42bc075930 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f42a8005b40 tx=0x7f42a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:52.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.335+0000 7f42b8c00700 1 -- 172.21.15.117:0/728359511 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42a801d070 con 0x7f42bc1039f0 2024-04-20T11:44:52.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.335+0000 7f42b8c00700 1 -- 172.21.15.117:0/728359511 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42a8005700 con 0x7f42bc1039f0 2024-04-20T11:44:52.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.335+0000 7f42b8c00700 1 -- 172.21.15.117:0/728359511 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42a800fe50 con 0x7f42bc1039f0 2024-04-20T11:44:52.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.335+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc075f30 con 0x7f42bc1039f0 2024-04-20T11:44:52.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.336+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc076250 con 0x7f42bc1039f0 2024-04-20T11:44:52.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.337+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc10b440 con 0x7f42bc1039f0 2024-04-20T11:44:52.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.338+0000 7f42b8c00700 1 -- 172.21.15.117:0/728359511 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f42a800f460 con 0x7f42bc1039f0 2024-04-20T11:44:52.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.339+0000 7f42b8c00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42a006ecc0 0x7f42a0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:52.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.339+0000 7f42b8c00700 1 -- 172.21.15.117:0/728359511 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f42a8090280 con 0x7f42bc1039f0 2024-04-20T11:44:52.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.339+0000 7f42bbe00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42a006ecc0 0x7f42a0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:52.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.340+0000 7f42bbe00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42a006ecc0 0x7f42a0071180 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f42ac005950 tx=0x7f42ac0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:52.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.345+0000 7f42b8c00700 1 -- 172.21.15.117:0/728359511 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f42a8059040 con 0x7f42bc1039f0 2024-04-20T11:44:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.636+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f42bc104520 con 0x7f42bc1039f0 2024-04-20T11:44:52.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.637+0000 7f42b8c00700 1 -- 172.21.15.117:0/728359511 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f42a805c700 con 0x7f42bc1039f0 2024-04-20T11:44:52.638 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:44:52.638 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:44:52.638 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:44:52.638 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:44:52.639 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:44:52.640 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:44:52.641 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:44:52.642 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:44:52.642 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:44:52.642 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:44:52.642 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42a006ecc0 msgr2=0x7f42a0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42a006ecc0 0x7f42a0071180 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f42ac005950 tx=0x7f42ac0058e0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 msgr2=0x7f42bc075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 0x7f42bc075930 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f42a8005b40 tx=0x7f42a800be20 comp rx=0 tx=0).stop 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 shutdown_connections 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f42a006ecc0 0x7f42a0071180 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42bc103040 0x7f42bc0753f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 --2- 172.21.15.117:0/728359511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc1039f0 0x7f42bc075930 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 >> 172.21.15.117:0/728359511 conn(0x7f42bc0fe8d0 msgr2=0x7f42bc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 shutdown_connections 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.640+0000 7f42c2c00700 1 -- 172.21.15.117:0/728359511 wait complete. 2024-04-20T11:44:52.643 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:44:52.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.790+0000 7f926e800700 1 -- 172.21.15.117:0/1169091420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268107520 msgr2=0x7f9268107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:52.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.790+0000 7f926e800700 1 --2- 172.21.15.117:0/1169091420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268107520 0x7f9268107900 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9258009910 tx=0x7f9258009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:52.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.790+0000 7f926e800700 1 -- 172.21.15.117:0/1169091420 shutdown_connections 2024-04-20T11:44:52.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.790+0000 7f926e800700 1 --2- 172.21.15.117:0/1169091420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9268108e10 0x7f9268109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.790+0000 7f926e800700 1 --2- 172.21.15.117:0/1169091420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268107520 0x7f9268107900 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.790+0000 7f926e800700 1 -- 172.21.15.117:0/1169091420 >> 172.21.15.117:0/1169091420 conn(0x7f92680768f0 msgr2=0x7f9268078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:52.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:52 smithi117 ceph-mon[99036]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:52.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:52 smithi117 ceph-mon[99036]: pgmap v973: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:52.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:52 smithi117 ceph-mon[99036]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:52.791 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:52 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1925566685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:44:52.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.791+0000 7f926e800700 1 -- 172.21.15.117:0/1169091420 shutdown_connections 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.791+0000 7f926e800700 1 -- 172.21.15.117:0/1169091420 wait complete. 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.792+0000 7f926e800700 1 Processor -- start 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.792+0000 7f926e800700 1 -- start start 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.792+0000 7f926e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9268107520 0x7f926819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.792+0000 7f926e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268108e10 0x7f926819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.792+0000 7f926e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926819de90 con 0x7f9268108e10 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.792+0000 7f926e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92681a1b60 con 0x7f9268107520 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.793+0000 7f9266a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268108e10 0x7f926819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.793+0000 7f9266a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268108e10 0x7f926819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33452/0 (socket says 172.21.15.117:33452) 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.793+0000 7f9266a00700 1 -- 172.21.15.117:0/3330568855 learned_addr learned my addr 172.21.15.117:0/3330568855 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.793+0000 7f9267400700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9268107520 0x7f926819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.793+0000 7f9266a00700 1 -- 172.21.15.117:0/3330568855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9268107520 msgr2=0x7f926819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:52.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.793+0000 7f9266a00700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9268107520 0x7f926819d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:52.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.793+0000 7f9266a00700 1 -- 172.21.15.117:0/3330568855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580095f0 con 0x7f9268108e10 2024-04-20T11:44:52.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.793+0000 7f9266a00700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268108e10 0x7f926819d7b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f925400b4b0 tx=0x7f925400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:52.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.794+0000 7f925fe00700 1 -- 172.21.15.117:0/3330568855 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9254011800 con 0x7f9268108e10 2024-04-20T11:44:52.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.794+0000 7f925fe00700 1 -- 172.21.15.117:0/3330568855 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9254011e20 con 0x7f9268108e10 2024-04-20T11:44:52.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.794+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92681a1e40 con 0x7f9268108e10 2024-04-20T11:44:52.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.794+0000 7f925fe00700 1 -- 172.21.15.117:0/3330568855 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f925400f2e0 con 0x7f9268108e10 2024-04-20T11:44:52.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.794+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92681a2410 con 0x7f9268108e10 2024-04-20T11:44:52.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.795+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926810f870 con 0x7f9268108e10 2024-04-20T11:44:52.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.797+0000 7f925fe00700 1 -- 172.21.15.117:0/3330568855 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9254010380 con 0x7f9268108e10 2024-04-20T11:44:52.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.797+0000 7f925fe00700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924c06ecd0 0x7f924c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:52.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.798+0000 7f925fe00700 1 -- 172.21.15.117:0/3330568855 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f925408ea00 con 0x7f9268108e10 2024-04-20T11:44:52.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.798+0000 7f9267400700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924c06ecd0 0x7f924c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:52.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.799+0000 7f9267400700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924c06ecd0 0x7f924c071190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f925800b5c0 tx=0x7f925801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:52.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:52.803+0000 7f925fe00700 1 -- 172.21.15.117:0/3330568855 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9254057870 con 0x7f9268108e10 2024-04-20T11:44:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:52 smithi173 ceph-mon[74165]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:52 smithi173 ceph-mon[74165]: pgmap v973: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:52 smithi173 ceph-mon[74165]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:52 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1925566685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:44:53.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.029+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f926819e580 con 0x7f924c06ecd0 2024-04-20T11:44:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.031+0000 7f925fe00700 1 -- 172.21.15.117:0/3330568855 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f926819e580 con 0x7f924c06ecd0 2024-04-20T11:44:53.031 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:44:53.031 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:44:53.031 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:44:53.031 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:44:53.032 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:44:53.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924c06ecd0 msgr2=0x7f924c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:53.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924c06ecd0 0x7f924c071190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f925800b5c0 tx=0x7f925801a040 comp rx=0 tx=0).stop 2024-04-20T11:44:53.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268108e10 msgr2=0x7f926819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:53.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268108e10 0x7f926819d7b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f925400b4b0 tx=0x7f925400b870 comp rx=0 tx=0).stop 2024-04-20T11:44:53.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 shutdown_connections 2024-04-20T11:44:53.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f924c06ecd0 0x7f924c071190 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9268107520 0x7f926819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 --2- 172.21.15.117:0/3330568855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9268108e10 0x7f926819d7b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.034+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 >> 172.21.15.117:0/3330568855 conn(0x7f92680768f0 msgr2=0x7f9268078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:53.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.035+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 shutdown_connections 2024-04-20T11:44:53.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.035+0000 7f926e800700 1 -- 172.21.15.117:0/3330568855 wait complete. 2024-04-20T11:44:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.186+0000 7f3d2b800700 1 -- 172.21.15.117:0/1525389995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24107520 msgr2=0x7f3d24107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.186+0000 7f3d2b800700 1 --2- 172.21.15.117:0/1525389995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24107520 0x7f3d24107900 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009910 tx=0x7f3d18009c20 comp rx=0 tx=0).stop 2024-04-20T11:44:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.186+0000 7f3d2b800700 1 -- 172.21.15.117:0/1525389995 shutdown_connections 2024-04-20T11:44:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.186+0000 7f3d2b800700 1 --2- 172.21.15.117:0/1525389995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d24108e10 0x7f3d24109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.186+0000 7f3d2b800700 1 --2- 172.21.15.117:0/1525389995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24107520 0x7f3d24107900 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.186+0000 7f3d2b800700 1 -- 172.21.15.117:0/1525389995 >> 172.21.15.117:0/1525389995 conn(0x7f3d240768f0 msgr2=0x7f3d24078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.186+0000 7f3d2b800700 1 -- 172.21.15.117:0/1525389995 shutdown_connections 2024-04-20T11:44:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.187+0000 7f3d2b800700 1 -- 172.21.15.117:0/1525389995 wait complete. 2024-04-20T11:44:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.188+0000 7f3d2b800700 1 Processor -- start 2024-04-20T11:44:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.188+0000 7f3d2b800700 1 -- start start 2024-04-20T11:44:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.188+0000 7f3d2b800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d24107520 0x7f3d2419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d2b800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24108e10 0x7f3d2419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d2b800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2419ddd0 con 0x7f3d24108e10 2024-04-20T11:44:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d2b800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241a1aa0 con 0x7f3d24107520 2024-04-20T11:44:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d23e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24108e10 0x7f3d2419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:53.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d23e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24108e10 0x7f3d2419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33468/0 (socket says 172.21.15.117:33468) 2024-04-20T11:44:53.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d23e00700 1 -- 172.21.15.117:0/2959746057 learned_addr learned my addr 172.21.15.117:0/2959746057 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d28e00700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d24107520 0x7f3d2419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d23e00700 1 -- 172.21.15.117:0/2959746057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d24107520 msgr2=0x7f3d2419d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d23e00700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d24107520 0x7f3d2419d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.189+0000 7f3d23e00700 1 -- 172.21.15.117:0/2959746057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180095f0 con 0x7f3d24108e10 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.190+0000 7f3d23e00700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24108e10 0x7f3d2419d6f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000d8d0 tx=0x7f3d1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.190+0000 7f3d21600700 1 -- 172.21.15.117:0/2959746057 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d10009940 con 0x7f3d24108e10 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.190+0000 7f3d21600700 1 -- 172.21.15.117:0/2959746057 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d10010460 con 0x7f3d24108e10 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.190+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d241a1d80 con 0x7f3d24108e10 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.190+0000 7f3d21600700 1 -- 172.21.15.117:0/2959746057 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1000f5f0 con 0x7f3d24108e10 2024-04-20T11:44:53.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.190+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241a2350 con 0x7f3d24108e10 2024-04-20T11:44:53.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.191+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2410f870 con 0x7f3d24108e10 2024-04-20T11:44:53.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.192+0000 7f3d21600700 1 -- 172.21.15.117:0/2959746057 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3d100105d0 con 0x7f3d24108e10 2024-04-20T11:44:53.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.194+0000 7f3d21600700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0c06ecd0 0x7f3d0c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:44:53.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.194+0000 7f3d21600700 1 -- 172.21.15.117:0/2959746057 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3d1008eef0 con 0x7f3d24108e10 2024-04-20T11:44:53.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.194+0000 7f3d28e00700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0c06ecd0 0x7f3d0c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:44:53.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.195+0000 7f3d28e00700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0c06ecd0 0x7f3d0c071190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b5c0 tx=0x7f3d1801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:44:53.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.199+0000 7f3d21600700 1 -- 172.21.15.117:0/2959746057 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3d10057c30 con 0x7f3d24108e10 2024-04-20T11:44:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.511+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3d2419e4c0 con 0x7f3d24108e10 2024-04-20T11:44:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.511+0000 7f3d21600700 1 -- 172.21.15.117:0/2959746057 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f3d1005b2f0 con 0x7f3d24108e10 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:44:53.512 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:44:53.513 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:44:53.514 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:44:53.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.514+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0c06ecd0 msgr2=0x7f3d0c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:53.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.514+0000 7f3d2b800700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0c06ecd0 0x7f3d0c071190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b5c0 tx=0x7f3d1801a040 comp rx=0 tx=0).stop 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.514+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24108e10 msgr2=0x7f3d2419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.514+0000 7f3d2b800700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24108e10 0x7f3d2419d6f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000d8d0 tx=0x7f3d1000dc90 comp rx=0 tx=0).stop 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.515+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 shutdown_connections 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.515+0000 7f3d2b800700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3d0c06ecd0 0x7f3d0c071190 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.515+0000 7f3d2b800700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d24107520 0x7f3d2419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.515+0000 7f3d2b800700 1 --2- 172.21.15.117:0/2959746057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d24108e10 0x7f3d2419d6f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.515+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 >> 172.21.15.117:0/2959746057 conn(0x7f3d240768f0 msgr2=0x7f3d24077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.515+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 shutdown_connections 2024-04-20T11:44:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:44:53.515+0000 7f3d2b800700 1 -- 172.21.15.117:0/2959746057 wait complete. 2024-04-20T11:44:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:53 smithi117 ceph-mon[99036]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:53 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/728359511' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:44:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:53 smithi173 ceph-mon[74165]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:53 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/728359511' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:44:54.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:54 smithi117 ceph-mon[99036]: from='client.35758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:54.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:54 smithi117 ceph-mon[99036]: pgmap v974: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:54.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:54 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2959746057' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:44:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:54 smithi173 ceph-mon[74165]: from='client.35758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:44:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:54 smithi173 ceph-mon[74165]: pgmap v974: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:54 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2959746057' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:44:55.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:55 smithi117 ceph-mon[99036]: pgmap v975: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:55 smithi173 ceph-mon[74165]: pgmap v975: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:58.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:58 smithi117 ceph-mon[99036]: pgmap v976: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:44:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:44:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:58 smithi173 ceph-mon[74165]: pgmap v976: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:44:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:44:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:00.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:00 smithi117 ceph-mon[99036]: pgmap v977: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:00 smithi173 ceph-mon[74165]: pgmap v977: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:02.510 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:02 smithi117 ceph-mon[99036]: pgmap v978: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:02 smithi173 ceph-mon[74165]: pgmap v978: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:04 smithi117 ceph-mon[99036]: pgmap v979: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:04 smithi173 ceph-mon[74165]: pgmap v979: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:06.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:06 smithi117 ceph-mon[99036]: pgmap v980: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:06 smithi173 ceph-mon[74165]: pgmap v980: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:08.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:08 smithi117 ceph-mon[99036]: pgmap v981: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:08 smithi173 ceph-mon[74165]: pgmap v981: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:10 smithi117 ceph-mon[99036]: pgmap v982: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:10.660 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:10 smithi173 ceph-mon[74165]: pgmap v982: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:12.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:12 smithi117 ceph-mon[99036]: pgmap v983: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:12 smithi173 ceph-mon[74165]: pgmap v983: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:14.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:14 smithi117 ceph-mon[99036]: pgmap v984: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:14 smithi173 ceph-mon[74165]: pgmap v984: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:16 smithi117 ceph-mon[99036]: pgmap v985: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:16 smithi173 ceph-mon[74165]: pgmap v985: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:18 smithi117 ceph-mon[99036]: pgmap v986: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:18 smithi173 ceph-mon[74165]: pgmap v986: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:20.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:20 smithi117 ceph-mon[99036]: pgmap v987: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:20 smithi173 ceph-mon[74165]: pgmap v987: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:22.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:22 smithi117 ceph-mon[99036]: pgmap v988: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:22 smithi173 ceph-mon[74165]: pgmap v988: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:23.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.685+0000 7f9073600700 1 -- 172.21.15.117:0/3999154045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c100e00 msgr2=0x7f906c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:23.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.685+0000 7f9073600700 1 --2- 172.21.15.117:0/3999154045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c100e00 0x7f906c1011e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9060009910 tx=0x7f9060009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:23.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.685+0000 7f9073600700 1 -- 172.21.15.117:0/3999154045 shutdown_connections 2024-04-20T11:45:23.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.685+0000 7f9073600700 1 --2- 172.21.15.117:0/3999154045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f906c1017b0 0x7f906c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:23.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.685+0000 7f9073600700 1 --2- 172.21.15.117:0/3999154045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c100e00 0x7f906c1011e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:23.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.685+0000 7f9073600700 1 -- 172.21.15.117:0/3999154045 >> 172.21.15.117:0/3999154045 conn(0x7f906c0fc6b0 msgr2=0x7f906c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:23.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.685+0000 7f9073600700 1 -- 172.21.15.117:0/3999154045 shutdown_connections 2024-04-20T11:45:23.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.685+0000 7f9073600700 1 -- 172.21.15.117:0/3999154045 wait complete. 2024-04-20T11:45:23.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.687+0000 7f9073600700 1 Processor -- start 2024-04-20T11:45:23.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.687+0000 7f9073600700 1 -- start start 2024-04-20T11:45:23.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.687+0000 7f9073600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f906c100e00 0x7f906c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:23.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.687+0000 7f9073600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c1017b0 0x7f906c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:23.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.687+0000 7f9073600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c19f5a0 con 0x7f906c1017b0 2024-04-20T11:45:23.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.687+0000 7f9073600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c198b80 con 0x7f906c100e00 2024-04-20T11:45:23.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f906be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c1017b0 0x7f906c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:23.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f906be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c1017b0 0x7f906c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37746/0 (socket says 172.21.15.117:37746) 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f906be00700 1 -- 172.21.15.117:0/886866614 learned_addr learned my addr 172.21.15.117:0/886866614 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f9070c00700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f906c100e00 0x7f906c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f906be00700 1 -- 172.21.15.117:0/886866614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f906c100e00 msgr2=0x7f906c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f906be00700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f906c100e00 0x7f906c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f906be00700 1 -- 172.21.15.117:0/886866614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90600095f0 con 0x7f906c1017b0 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f9070c00700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f906c100e00 0x7f906c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.688+0000 7f906be00700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c1017b0 0x7f906c19ef80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f905800d900 tx=0x7f905800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.689+0000 7f9069600700 1 -- 172.21.15.117:0/886866614 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9058009900 con 0x7f906c1017b0 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.689+0000 7f9069600700 1 -- 172.21.15.117:0/886866614 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9058010460 con 0x7f906c1017b0 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.689+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c198e60 con 0x7f906c1017b0 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.689+0000 7f9069600700 1 -- 172.21.15.117:0/886866614 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f905800f5f0 con 0x7f906c1017b0 2024-04-20T11:45:23.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.689+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c1993b0 con 0x7f906c1017b0 2024-04-20T11:45:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.690+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c104f80 con 0x7f906c1017b0 2024-04-20T11:45:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.691+0000 7f9069600700 1 -- 172.21.15.117:0/886866614 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f90580105d0 con 0x7f906c1017b0 2024-04-20T11:45:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.692+0000 7f9069600700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f905406ecd0 0x7f9054071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.692+0000 7f9069600700 1 -- 172.21.15.117:0/886866614 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f905808ebd0 con 0x7f906c1017b0 2024-04-20T11:45:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.693+0000 7f9070c00700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f905406ecd0 0x7f9054071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.693+0000 7f9070c00700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f905406ecd0 0x7f9054071190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9060009fd0 tx=0x7f906000b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:23.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.697+0000 7f9069600700 1 -- 172.21.15.117:0/886866614 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f905805b560 con 0x7f906c1017b0 2024-04-20T11:45:23.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.946+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f906c19a040 con 0x7f905406ecd0 2024-04-20T11:45:23.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.949+0000 7f9069600700 1 -- 172.21.15.117:0/886866614 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f906c19a040 con 0x7f905406ecd0 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f905406ecd0 msgr2=0x7f9054071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f905406ecd0 0x7f9054071190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9060009fd0 tx=0x7f906000b560 comp rx=0 tx=0).stop 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c1017b0 msgr2=0x7f906c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c1017b0 0x7f906c19ef80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f905800d900 tx=0x7f905800dcc0 comp rx=0 tx=0).stop 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 shutdown_connections 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f905406ecd0 0x7f9054071190 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f906c100e00 0x7f906c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 --2- 172.21.15.117:0/886866614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f906c1017b0 0x7f906c19ef80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:23.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 >> 172.21.15.117:0/886866614 conn(0x7f906c0fc6b0 msgr2=0x7f906c0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 shutdown_connections 2024-04-20T11:45:23.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:23.952+0000 7f9073600700 1 -- 172.21.15.117:0/886866614 wait complete. 2024-04-20T11:45:23.969 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:45:24.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.112+0000 7fe2c5600700 1 -- 172.21.15.117:0/1388513159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0068570 msgr2=0x7fe2c00689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.112+0000 7fe2c5600700 1 --2- 172.21.15.117:0/1388513159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0068570 0x7fe2c00689d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009910 tx=0x7fe2ac009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:24.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.113+0000 7fe2c5600700 1 -- 172.21.15.117:0/1388513159 shutdown_connections 2024-04-20T11:45:24.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.113+0000 7fe2c5600700 1 --2- 172.21.15.117:0/1388513159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0068570 0x7fe2c00689d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.113+0000 7fe2c5600700 1 --2- 172.21.15.117:0/1388513159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c01069d0 0x7fe2c0106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.113+0000 7fe2c5600700 1 -- 172.21.15.117:0/1388513159 >> 172.21.15.117:0/1388513159 conn(0x7fe2c0075480 msgr2=0x7fe2c0075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:24.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.113+0000 7fe2c5600700 1 -- 172.21.15.117:0/1388513159 shutdown_connections 2024-04-20T11:45:24.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.113+0000 7fe2c5600700 1 -- 172.21.15.117:0/1388513159 wait complete. 2024-04-20T11:45:24.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.114+0000 7fe2c5600700 1 Processor -- start 2024-04-20T11:45:24.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.114+0000 7fe2c5600700 1 -- start start 2024-04-20T11:45:24.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.114+0000 7fe2c5600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0068570 0x7fe2c00ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.114+0000 7fe2c5600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c01069d0 0x7fe2c00fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.114+0000 7fe2c5600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c0100500 con 0x7fe2c01069d0 2024-04-20T11:45:24.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.114+0000 7fe2c5600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c0100640 con 0x7fe2c0068570 2024-04-20T11:45:24.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.115+0000 7fe2bf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c01069d0 0x7fe2c00fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.115+0000 7fe2bfe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0068570 0x7fe2c00ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.115+0000 7fe2bf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c01069d0 0x7fe2c00fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56592/0 (socket says 172.21.15.117:56592) 2024-04-20T11:45:24.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.115+0000 7fe2bf400700 1 -- 172.21.15.117:0/643738052 learned_addr learned my addr 172.21.15.117:0/643738052 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:24.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.116+0000 7fe2bf400700 1 -- 172.21.15.117:0/643738052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0068570 msgr2=0x7fe2c00ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.116+0000 7fe2bf400700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0068570 0x7fe2c00ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.116+0000 7fe2bf400700 1 -- 172.21.15.117:0/643738052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0095f0 con 0x7fe2c01069d0 2024-04-20T11:45:24.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.116+0000 7fe2bf400700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c01069d0 0x7fe2c00fffc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0092e0 tx=0x7fe2ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:24.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.117+0000 7fe2bcc00700 1 -- 172.21.15.117:0/643738052 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2ac01d070 con 0x7fe2c01069d0 2024-04-20T11:45:24.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.117+0000 7fe2bcc00700 1 -- 172.21.15.117:0/643738052 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2ac00f460 con 0x7fe2c01069d0 2024-04-20T11:45:24.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.117+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c01008c0 con 0x7fe2c01069d0 2024-04-20T11:45:24.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.117+0000 7fe2bcc00700 1 -- 172.21.15.117:0/643738052 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2ac017630 con 0x7fe2c01069d0 2024-04-20T11:45:24.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.117+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c01a2ea0 con 0x7fe2c01069d0 2024-04-20T11:45:24.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.118+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c0108f80 con 0x7fe2c01069d0 2024-04-20T11:45:24.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.123+0000 7fe2bcc00700 1 -- 172.21.15.117:0/643738052 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe2ac00f5d0 con 0x7fe2c01069d0 2024-04-20T11:45:24.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.124+0000 7fe2bcc00700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe2a406ecc0 0x7fe2a4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.125+0000 7fe2bcc00700 1 -- 172.21.15.117:0/643738052 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe2ac090230 con 0x7fe2c01069d0 2024-04-20T11:45:24.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.125+0000 7fe2bfe00700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe2a406ecc0 0x7fe2a4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.126+0000 7fe2bfe00700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe2a406ecc0 0x7fe2a4071180 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009a10 tx=0x7fe2b0009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:24.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.131+0000 7fe2bcc00700 1 -- 172.21.15.117:0/643738052 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe2ac058ff0 con 0x7fe2c01069d0 2024-04-20T11:45:24.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.383+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2c010dc30 con 0x7fe2a406ecc0 2024-04-20T11:45:24.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.386+0000 7fe2bcc00700 1 -- 172.21.15.117:0/643738052 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe2c010dc30 con 0x7fe2a406ecc0 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.388+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe2a406ecc0 msgr2=0x7fe2a4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.388+0000 7fe2c5600700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe2a406ecc0 0x7fe2a4071180 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009a10 tx=0x7fe2b0009190 comp rx=0 tx=0).stop 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.388+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c01069d0 msgr2=0x7fe2c00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.388+0000 7fe2c5600700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c01069d0 0x7fe2c00fffc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0092e0 tx=0x7fe2ac00bd70 comp rx=0 tx=0).stop 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.389+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 shutdown_connections 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.389+0000 7fe2c5600700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe2a406ecc0 0x7fe2a4071180 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.389+0000 7fe2c5600700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0068570 0x7fe2c00ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.389+0000 7fe2c5600700 1 --2- 172.21.15.117:0/643738052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c01069d0 0x7fe2c00fffc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.389+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 >> 172.21.15.117:0/643738052 conn(0x7fe2c0075480 msgr2=0x7fe2c00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.389+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 shutdown_connections 2024-04-20T11:45:24.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.389+0000 7fe2c5600700 1 -- 172.21.15.117:0/643738052 wait complete. 2024-04-20T11:45:24.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.544+0000 7f1ffde00700 1 -- 172.21.15.117:0/1783640023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff81026b0 msgr2=0x7f1ff810aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.544+0000 7f1ffde00700 1 --2- 172.21.15.117:0/1783640023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff81026b0 0x7f1ff810aba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009910 tx=0x7f1fe0009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:24.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.544+0000 7f1ffde00700 1 -- 172.21.15.117:0/1783640023 shutdown_connections 2024-04-20T11:45:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.544+0000 7f1ffde00700 1 --2- 172.21.15.117:0/1783640023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff81026b0 0x7f1ff810aba0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.544+0000 7f1ffde00700 1 --2- 172.21.15.117:0/1783640023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff8101d90 0x7f1ff8102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.544+0000 7f1ffde00700 1 -- 172.21.15.117:0/1783640023 >> 172.21.15.117:0/1783640023 conn(0x7f1ff80fb5c0 msgr2=0x7f1ff80fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.544+0000 7f1ffde00700 1 -- 172.21.15.117:0/1783640023 shutdown_connections 2024-04-20T11:45:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.545+0000 7f1ffde00700 1 -- 172.21.15.117:0/1783640023 wait complete. 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.545+0000 7f1ffde00700 1 Processor -- start 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.546+0000 7f1ffde00700 1 -- start start 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.546+0000 7f1ffde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff8101d90 0x7f1ff80ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.546+0000 7f1ffde00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff81026b0 0x7f1ff80fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.546+0000 7f1ffde00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8100470 con 0x7f1ff8101d90 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.546+0000 7f1ffde00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81005b0 con 0x7f1ff81026b0 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.546+0000 7f1ff6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff8101d90 0x7f1ff80ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.546+0000 7f1ff6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff8101d90 0x7f1ff80ff9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56610/0 (socket says 172.21.15.117:56610) 2024-04-20T11:45:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.546+0000 7f1ff6a00700 1 -- 172.21.15.117:0/3704923717 learned_addr learned my addr 172.21.15.117:0/3704923717 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1fefe00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff81026b0 0x7f1ff80fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1ff6a00700 1 -- 172.21.15.117:0/3704923717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff81026b0 msgr2=0x7f1ff80fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1ff6a00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff81026b0 0x7f1ff80fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1ff6a00700 1 -- 172.21.15.117:0/3704923717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe00095f0 con 0x7f1ff8101d90 2024-04-20T11:45:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1ff6a00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff8101d90 0x7f1ff80ff9f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800b4b0 tx=0x7f1fe800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1ffca00700 1 -- 172.21.15.117:0/3704923717 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fe8011800 con 0x7f1ff8101d90 2024-04-20T11:45:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1ffca00700 1 -- 172.21.15.117:0/3704923717 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fe8011e20 con 0x7f1ff8101d90 2024-04-20T11:45:24.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff81a3130 con 0x7f1ff8101d90 2024-04-20T11:45:24.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.547+0000 7f1ffca00700 1 -- 172.21.15.117:0/3704923717 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fe800f2e0 con 0x7f1ff8101d90 2024-04-20T11:45:24.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.548+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff81a3490 con 0x7f1ff8101d90 2024-04-20T11:45:24.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.549+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff80696e0 con 0x7f1ff8101d90 2024-04-20T11:45:24.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.550+0000 7f1ffca00700 1 -- 172.21.15.117:0/3704923717 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1fe8010380 con 0x7f1ff8101d90 2024-04-20T11:45:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.550+0000 7f1ffca00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fdc06ecc0 0x7f1fdc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.551+0000 7f1ffca00700 1 -- 172.21.15.117:0/3704923717 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1fe808ea70 con 0x7f1ff8101d90 2024-04-20T11:45:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.551+0000 7f1fefe00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fdc06ecc0 0x7f1fdc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.552+0000 7f1fefe00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fdc06ecc0 0x7f1fdc071180 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0006010 tx=0x7f1fe000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.555+0000 7f1ffca00700 1 -- 172.21.15.117:0/3704923717 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1fe80577b0 con 0x7f1ff8101d90 2024-04-20T11:45:24.615 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:24 smithi173 ceph-mon[74165]: pgmap v989: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:24 smithi117 ceph-mon[99036]: pgmap v989: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:24.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.791+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ff81054e0 con 0x7f1fdc06ecc0 2024-04-20T11:45:24.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.823+0000 7f1ffca00700 1 -- 172.21.15.117:0/3704923717 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1ff81054e0 con 0x7f1fdc06ecc0 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (33m) 9m ago 47m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (48m) 9m ago 48m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (46m) 9m ago 46m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (30m) 9m ago 48m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (30m) 9m ago 46m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (33m) 9m ago 47m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (25m) 9m ago 40m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:45:24.824 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (26m) 9m ago 40m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (9m) 9m ago 40m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (25m) 9m ago 40m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (37m) 9m ago 49m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (35m) 9m ago 46m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (31m) 9m ago 49m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (31m) 9m ago 46m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (35m) 9m ago 48m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (35m) 9m ago 46m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (29m) 9m ago 45m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (28m) 9m ago 44m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (28m) 9m ago 44m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (27m) 9m ago 43m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (27m) 9m ago 43m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (26m) 9m ago 42m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:45:24.825 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (34m) 9m ago 47m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.825+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fdc06ecc0 msgr2=0x7f1fdc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.825+0000 7f1ffde00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fdc06ecc0 0x7f1fdc071180 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0006010 tx=0x7f1fe000b540 comp rx=0 tx=0).stop 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.825+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff8101d90 msgr2=0x7f1ff80ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.825+0000 7f1ffde00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff8101d90 0x7f1ff80ff9f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800b4b0 tx=0x7f1fe800b870 comp rx=0 tx=0).stop 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.826+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 shutdown_connections 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.826+0000 7f1ffde00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1fdc06ecc0 0x7f1fdc071180 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.826+0000 7f1ffde00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ff8101d90 0x7f1ff80ff9f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.826+0000 7f1ffde00700 1 --2- 172.21.15.117:0/3704923717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff81026b0 0x7f1ff80fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.826+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 >> 172.21.15.117:0/3704923717 conn(0x7f1ff80fb5c0 msgr2=0x7f1ff8103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.826+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 shutdown_connections 2024-04-20T11:45:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.826+0000 7f1ffde00700 1 -- 172.21.15.117:0/3704923717 wait complete. 2024-04-20T11:45:24.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.977+0000 7fbea7800700 1 -- 172.21.15.117:0/4112871380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 msgr2=0x7fbea0105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.977+0000 7fbea7800700 1 --2- 172.21.15.117:0/4112871380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 0x7fbea0105810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009910 tx=0x7fbe8c009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:24.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.977+0000 7fbea7800700 1 -- 172.21.15.117:0/4112871380 shutdown_connections 2024-04-20T11:45:24.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.977+0000 7fbea7800700 1 --2- 172.21.15.117:0/4112871380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 0x7fbea0105810 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.977+0000 7fbea7800700 1 --2- 172.21.15.117:0/4112871380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea0100e10 0x7fbea01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.977+0000 7fbea7800700 1 -- 172.21.15.117:0/4112871380 >> 172.21.15.117:0/4112871380 conn(0x7fbea00fc6c0 msgr2=0x7fbea00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:24.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.977+0000 7fbea7800700 1 -- 172.21.15.117:0/4112871380 shutdown_connections 2024-04-20T11:45:24.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.977+0000 7fbea7800700 1 -- 172.21.15.117:0/4112871380 wait complete. 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.978+0000 7fbea7800700 1 Processor -- start 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.978+0000 7fbea7800700 1 -- start start 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbea7800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea0100e10 0x7fbea019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbea7800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 0x7fbea019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbea7800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea019d4f0 con 0x7fbea01017c0 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbea7800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0196ab0 con 0x7fbea0100e10 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbe9fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 0x7fbea019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbe9fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 0x7fbea019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56630/0 (socket says 172.21.15.117:56630) 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbe9fe00700 1 -- 172.21.15.117:0/4074959984 learned_addr learned my addr 172.21.15.117:0/4074959984 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:24.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbea4e00700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea0100e10 0x7fbea019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbe9fe00700 1 -- 172.21.15.117:0/4074959984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea0100e10 msgr2=0x7fbea019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:24.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbe9fe00700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea0100e10 0x7fbea019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:24.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.979+0000 7fbe9fe00700 1 -- 172.21.15.117:0/4074959984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c0095f0 con 0x7fbea01017c0 2024-04-20T11:45:24.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.980+0000 7fbe9fe00700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 0x7fbea019ce60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0092e0 tx=0x7fbe8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:24.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.980+0000 7fbe9d600700 1 -- 172.21.15.117:0/4074959984 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe8c01d070 con 0x7fbea01017c0 2024-04-20T11:45:24.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.980+0000 7fbe9d600700 1 -- 172.21.15.117:0/4074959984 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe8c00f460 con 0x7fbea01017c0 2024-04-20T11:45:24.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.980+0000 7fbe9d600700 1 -- 172.21.15.117:0/4074959984 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe8c017610 con 0x7fbea01017c0 2024-04-20T11:45:24.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.980+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea0196d30 con 0x7fbea01017c0 2024-04-20T11:45:24.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.980+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea0197220 con 0x7fbea01017c0 2024-04-20T11:45:24.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.981+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea01022f0 con 0x7fbea01017c0 2024-04-20T11:45:24.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.982+0000 7fbe9d600700 1 -- 172.21.15.117:0/4074959984 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbe8c00f5d0 con 0x7fbea01017c0 2024-04-20T11:45:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.983+0000 7fbe9d600700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbe8806ecc0 0x7fbe88071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.983+0000 7fbe9d600700 1 -- 172.21.15.117:0/4074959984 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbe8c0902d0 con 0x7fbea01017c0 2024-04-20T11:45:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.984+0000 7fbea4e00700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbe8806ecc0 0x7fbe88071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:24.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.984+0000 7fbea4e00700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbe8806ecc0 0x7fbe88071180 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbe94006fd0 tx=0x7fbe94008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:24.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:24.988+0000 7fbe9d600700 1 -- 172.21.15.117:0/4074959984 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbe8c05cbc0 con 0x7fbea01017c0 2024-04-20T11:45:25.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.284+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbea010de30 con 0x7fbea01017c0 2024-04-20T11:45:25.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.285+0000 7fbe9d600700 1 -- 172.21.15.117:0/4074959984 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbe8c05c750 con 0x7fbea01017c0 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:45:25.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:45:25.287 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:45:25.287 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:45:25.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:45:25.287 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:45:25.287 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:45:25.287 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:45:25.287 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:45:25.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.287+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbe8806ecc0 msgr2=0x7fbe88071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.287+0000 7fbea7800700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbe8806ecc0 0x7fbe88071180 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbe94006fd0 tx=0x7fbe94008040 comp rx=0 tx=0).stop 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.287+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 msgr2=0x7fbea019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.287+0000 7fbea7800700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 0x7fbea019ce60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0092e0 tx=0x7fbe8c00be20 comp rx=0 tx=0).stop 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.288+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 shutdown_connections 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.288+0000 7fbea7800700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbe8806ecc0 0x7fbe88071180 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.288+0000 7fbea7800700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea0100e10 0x7fbea019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.288+0000 7fbea7800700 1 --2- 172.21.15.117:0/4074959984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea01017c0 0x7fbea019ce60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.288+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 >> 172.21.15.117:0/4074959984 conn(0x7fbea00fc6c0 msgr2=0x7fbea00fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.288+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 shutdown_connections 2024-04-20T11:45:25.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.288+0000 7fbea7800700 1 -- 172.21.15.117:0/4074959984 wait complete. 2024-04-20T11:45:25.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.442+0000 7f1381600700 1 -- 172.21.15.117:0/734682452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 msgr2=0x7f137c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:25.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.442+0000 7f1381600700 1 --2- 172.21.15.117:0/734682452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 0x7f137c1021c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f136c009910 tx=0x7f136c009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.442+0000 7f1381600700 1 -- 172.21.15.117:0/734682452 shutdown_connections 2024-04-20T11:45:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.442+0000 7f1381600700 1 --2- 172.21.15.117:0/734682452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f137c102700 0x7f137c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.442+0000 7f1381600700 1 --2- 172.21.15.117:0/734682452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 0x7f137c1021c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.442+0000 7f1381600700 1 -- 172.21.15.117:0/734682452 >> 172.21.15.117:0/734682452 conn(0x7f137c076750 msgr2=0x7f137c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.442+0000 7f1381600700 1 -- 172.21.15.117:0/734682452 shutdown_connections 2024-04-20T11:45:25.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.443+0000 7f1381600700 1 -- 172.21.15.117:0/734682452 wait complete. 2024-04-20T11:45:25.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.443+0000 7f1381600700 1 Processor -- start 2024-04-20T11:45:25.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.444+0000 7f1381600700 1 -- start start 2024-04-20T11:45:25.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.444+0000 7f1381600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 0x7f137c0ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:25.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.444+0000 7f1381600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f137c102700 0x7f137c0fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:25.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.444+0000 7f1381600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c1004c0 con 0x7f137c101de0 2024-04-20T11:45:25.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.444+0000 7f1381600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c100600 con 0x7f137c102700 2024-04-20T11:45:25.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 0x7f137c0ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:25.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 0x7f137c0ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56640/0 (socket says 172.21.15.117:56640) 2024-04-20T11:45:25.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137aa00700 1 -- 172.21.15.117:0/1165736788 learned_addr learned my addr 172.21.15.117:0/1165736788 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:25.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137aa00700 1 -- 172.21.15.117:0/1165736788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f137c102700 msgr2=0x7f137c0fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:25.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137a000700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f137c102700 0x7f137c0fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:25.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137aa00700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f137c102700 0x7f137c0fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137aa00700 1 -- 172.21.15.117:0/1165736788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0095f0 con 0x7f137c101de0 2024-04-20T11:45:25.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137a000700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f137c102700 0x7f137c0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:45:25.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.445+0000 7f137aa00700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 0x7f137c0ffa40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f136c006010 tx=0x7f136c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:25.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.446+0000 7f1373400700 1 -- 172.21.15.117:0/1165736788 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136c01d070 con 0x7f137c101de0 2024-04-20T11:45:25.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.446+0000 7f1373400700 1 -- 172.21.15.117:0/1165736788 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f136c00f460 con 0x7f137c101de0 2024-04-20T11:45:25.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.446+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c100880 con 0x7f137c101de0 2024-04-20T11:45:25.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.446+0000 7f1373400700 1 -- 172.21.15.117:0/1165736788 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136c017630 con 0x7f137c101de0 2024-04-20T11:45:25.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.446+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c1a2f80 con 0x7f137c101de0 2024-04-20T11:45:25.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.447+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c068db0 con 0x7f137c101de0 2024-04-20T11:45:25.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.448+0000 7f1373400700 1 -- 172.21.15.117:0/1165736788 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f136c00f5d0 con 0x7f137c101de0 2024-04-20T11:45:25.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.449+0000 7f1373400700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f136006ecc0 0x7f1360071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:25.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.449+0000 7f1373400700 1 -- 172.21.15.117:0/1165736788 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f136c0903f0 con 0x7f137c101de0 2024-04-20T11:45:25.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.450+0000 7f137a000700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f136006ecc0 0x7f1360071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:25.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.450+0000 7f137a000700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f136006ecc0 0x7f1360071180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1368005fd0 tx=0x7f1368005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:25.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.454+0000 7f1373400700 1 -- 172.21.15.117:0/1165736788 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f136c0591f0 con 0x7f137c101de0 2024-04-20T11:45:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:25 smithi117 ceph-mon[99036]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:25 smithi117 ceph-mon[99036]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:25 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4074959984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:45:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:25 smithi173 ceph-mon[74165]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:25 smithi173 ceph-mon[74165]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:25 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4074959984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:45:25.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.725+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f137c10dde0 con 0x7f137c101de0 2024-04-20T11:45:25.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.726+0000 7f1373400700 1 -- 172.21.15.117:0/1165736788 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f136c05c8b0 con 0x7f137c101de0 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:45:25.728 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:45:25.729 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:45:25.730 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:45:25.731 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:45:25.731 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f136006ecc0 msgr2=0x7f1360071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f136006ecc0 0x7f1360071180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1368005fd0 tx=0x7f1368005dc0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 msgr2=0x7f137c0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 0x7f137c0ffa40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f136c006010 tx=0x7f136c00be20 comp rx=0 tx=0).stop 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 shutdown_connections 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f136006ecc0 0x7f1360071180 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f137c101de0 0x7f137c0ffa40 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 --2- 172.21.15.117:0/1165736788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f137c102700 0x7f137c0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.729+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 >> 172.21.15.117:0/1165736788 conn(0x7f137c076750 msgr2=0x7f137c1030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.730+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 shutdown_connections 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.730+0000 7f1381600700 1 -- 172.21.15.117:0/1165736788 wait complete. 2024-04-20T11:45:25.732 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:45:25.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.880+0000 7fdff0c00700 1 -- 172.21.15.117:0/1849243382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec108e10 msgr2=0x7fdfec109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:25.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.880+0000 7fdff0c00700 1 --2- 172.21.15.117:0/1849243382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec108e10 0x7fdfec109290 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009910 tx=0x7fdfd8009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:25.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.880+0000 7fdff0c00700 1 -- 172.21.15.117:0/1849243382 shutdown_connections 2024-04-20T11:45:25.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.880+0000 7fdff0c00700 1 --2- 172.21.15.117:0/1849243382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec108e10 0x7fdfec109290 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.880+0000 7fdff0c00700 1 --2- 172.21.15.117:0/1849243382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec107520 0x7fdfec107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.880+0000 7fdff0c00700 1 -- 172.21.15.117:0/1849243382 >> 172.21.15.117:0/1849243382 conn(0x7fdfec0768f0 msgr2=0x7fdfec078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:25.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.880+0000 7fdff0c00700 1 -- 172.21.15.117:0/1849243382 shutdown_connections 2024-04-20T11:45:25.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.880+0000 7fdff0c00700 1 -- 172.21.15.117:0/1849243382 wait complete. 2024-04-20T11:45:25.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.881+0000 7fdff0c00700 1 Processor -- start 2024-04-20T11:45:25.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.881+0000 7fdff0c00700 1 -- start start 2024-04-20T11:45:25.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdff0c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec107520 0x7fdfec19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:25.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdff0c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec108e10 0x7fdfec19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:25.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdff0c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec19ded0 con 0x7fdfec107520 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdff0c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec1a1ba0 con 0x7fdfec108e10 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdfe9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec107520 0x7fdfec19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdfe9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec107520 0x7fdfec19d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56662/0 (socket says 172.21.15.117:56662) 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdfe9e00700 1 -- 172.21.15.117:0/2446119335 learned_addr learned my addr 172.21.15.117:0/2446119335 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdfe9400700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec108e10 0x7fdfec19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.882+0000 7fdfe9e00700 1 -- 172.21.15.117:0/2446119335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec108e10 msgr2=0x7fdfec19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdfe9e00700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec108e10 0x7fdfec19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdfe9e00700 1 -- 172.21.15.117:0/2446119335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd80095f0 con 0x7fdfec107520 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdfe9400700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec108e10 0x7fdfec19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:45:25.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdfe9e00700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec107520 0x7fdfec19d2b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00b4b0 tx=0x7fdfdc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:25.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdfe2a00700 1 -- 172.21.15.117:0/2446119335 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfdc011800 con 0x7fdfec107520 2024-04-20T11:45:25.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdfe2a00700 1 -- 172.21.15.117:0/2446119335 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfdc011e20 con 0x7fdfec107520 2024-04-20T11:45:25.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec1a1e80 con 0x7fdfec107520 2024-04-20T11:45:25.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdfe2a00700 1 -- 172.21.15.117:0/2446119335 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfdc00f2e0 con 0x7fdfec107520 2024-04-20T11:45:25.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.883+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec1a2450 con 0x7fdfec107520 2024-04-20T11:45:25.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.884+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec10f870 con 0x7fdfec107520 2024-04-20T11:45:25.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.885+0000 7fdfe2a00700 1 -- 172.21.15.117:0/2446119335 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdfdc010380 con 0x7fdfec107520 2024-04-20T11:45:25.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.886+0000 7fdfe2a00700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfd006ecd0 0x7fdfd0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:25.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.886+0000 7fdfe2a00700 1 -- 172.21.15.117:0/2446119335 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdfdc08ea00 con 0x7fdfec107520 2024-04-20T11:45:25.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.887+0000 7fdfe9400700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfd006ecd0 0x7fdfd0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:25.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.887+0000 7fdfe9400700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfd006ecd0 0x7fdfd0071190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8006010 tx=0x7fdfd800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:25.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:25.891+0000 7fdfe2a00700 1 -- 172.21.15.117:0/2446119335 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdfdc057870 con 0x7fdfec107520 2024-04-20T11:45:26.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.120+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfec19e5c0 con 0x7fdfd006ecd0 2024-04-20T11:45:26.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.124+0000 7fdfe2a00700 1 -- 172.21.15.117:0/2446119335 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdfec19e5c0 con 0x7fdfd006ecd0 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:45:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:45:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:45:26.126 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:45:26.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfd006ecd0 msgr2=0x7fdfd0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfd006ecd0 0x7fdfd0071190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8006010 tx=0x7fdfd800b540 comp rx=0 tx=0).stop 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec107520 msgr2=0x7fdfec19d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec107520 0x7fdfec19d2b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00b4b0 tx=0x7fdfdc00b870 comp rx=0 tx=0).stop 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 shutdown_connections 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdfd006ecd0 0x7fdfd0071190 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfec107520 0x7fdfec19d2b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 --2- 172.21.15.117:0/2446119335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec108e10 0x7fdfec19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 >> 172.21.15.117:0/2446119335 conn(0x7fdfec0768f0 msgr2=0x7fdfec0775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 shutdown_connections 2024-04-20T11:45:26.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.127+0000 7fdff0c00700 1 -- 172.21.15.117:0/2446119335 wait complete. 2024-04-20T11:45:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.278+0000 7f8716000700 1 -- 172.21.15.117:0/1689428219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 msgr2=0x7f87101034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.278+0000 7f8716000700 1 --2- 172.21.15.117:0/1689428219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 0x7f87101034f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8700009910 tx=0x7f8700009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.278+0000 7f8716000700 1 -- 172.21.15.117:0/1689428219 shutdown_connections 2024-04-20T11:45:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.278+0000 7f8716000700 1 --2- 172.21.15.117:0/1689428219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8710104a00 0x7f8710104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.278+0000 7f8716000700 1 --2- 172.21.15.117:0/1689428219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 0x7f87101034f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.278+0000 7f8716000700 1 -- 172.21.15.117:0/1689428219 >> 172.21.15.117:0/1689428219 conn(0x7f87100fe880 msgr2=0x7f8710100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.278+0000 7f8716000700 1 -- 172.21.15.117:0/1689428219 shutdown_connections 2024-04-20T11:45:26.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.278+0000 7f8716000700 1 -- 172.21.15.117:0/1689428219 wait complete. 2024-04-20T11:45:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.279+0000 7f8716000700 1 Processor -- start 2024-04-20T11:45:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.279+0000 7f8716000700 1 -- start start 2024-04-20T11:45:26.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.279+0000 7f8716000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 0x7f8710198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.279+0000 7f8716000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8710104a00 0x7f8710199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.279+0000 7f8716000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87101999e0 con 0x7f8710103110 2024-04-20T11:45:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.279+0000 7f8716000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871019d660 con 0x7f8710104a00 2024-04-20T11:45:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.280+0000 7f870f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 0x7f8710198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.280+0000 7f870f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 0x7f8710198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56666/0 (socket says 172.21.15.117:56666) 2024-04-20T11:45:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.280+0000 7f870f400700 1 -- 172.21.15.117:0/3169815132 learned_addr learned my addr 172.21.15.117:0/3169815132 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.280+0000 7f870f400700 1 -- 172.21.15.117:0/3169815132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8710104a00 msgr2=0x7f8710199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:26.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.280+0000 7f870f400700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8710104a00 0x7f8710199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.280+0000 7f870f400700 1 -- 172.21.15.117:0/3169815132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87000095f0 con 0x7f8710103110 2024-04-20T11:45:26.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.280+0000 7f870f400700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 0x7f8710198dc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8700000c00 tx=0x7f870000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:26.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.281+0000 7f8707e00700 1 -- 172.21.15.117:0/3169815132 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f870001d070 con 0x7f8710103110 2024-04-20T11:45:26.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.281+0000 7f8707e00700 1 -- 172.21.15.117:0/3169815132 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87000056e0 con 0x7f8710103110 2024-04-20T11:45:26.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.281+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871019d8e0 con 0x7f8710103110 2024-04-20T11:45:26.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.281+0000 7f8707e00700 1 -- 172.21.15.117:0/3169815132 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f870000fe80 con 0x7f8710103110 2024-04-20T11:45:26.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.281+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871019de50 con 0x7f8710103110 2024-04-20T11:45:26.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.282+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871010b480 con 0x7f8710103110 2024-04-20T11:45:26.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.283+0000 7f8707e00700 1 -- 172.21.15.117:0/3169815132 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f870000f6f0 con 0x7f8710103110 2024-04-20T11:45:26.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.284+0000 7f8707e00700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f86f406ecd0 0x7f86f4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:26.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.284+0000 7f8707e00700 1 -- 172.21.15.117:0/3169815132 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8700090230 con 0x7f8710103110 2024-04-20T11:45:26.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.284+0000 7f870ea00700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f86f406ecd0 0x7f86f4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:26.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.285+0000 7f870ea00700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f86f406ecd0 0x7f86f4071190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005950 tx=0x7f86fc0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:26.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.289+0000 7f8707e00700 1 -- 172.21.15.117:0/3169815132 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8700058ff0 con 0x7f8710103110 2024-04-20T11:45:26.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:26 smithi117 ceph-mon[99036]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:26.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:26 smithi117 ceph-mon[99036]: pgmap v990: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:26.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:26 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1165736788' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:45:26.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.594+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f871019a080 con 0x7f8710103110 2024-04-20T11:45:26.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.595+0000 7f8707e00700 1 -- 172.21.15.117:0/3169815132 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f870005c6b0 con 0x7f8710103110 2024-04-20T11:45:26.595 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:45:26.595 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:45:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:45:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:45:26.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f86f406ecd0 msgr2=0x7f86f4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f86f406ecd0 0x7f86f4071190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005950 tx=0x7f86fc0058e0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 msgr2=0x7f8710198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 0x7f8710198dc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8700000c00 tx=0x7f870000be20 comp rx=0 tx=0).stop 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 shutdown_connections 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f86f406ecd0 0x7f86f4071190 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8710103110 0x7f8710198dc0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 --2- 172.21.15.117:0/3169815132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8710104a00 0x7f8710199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 >> 172.21.15.117:0/3169815132 conn(0x7f87100fe880 msgr2=0x7f87100ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 shutdown_connections 2024-04-20T11:45:26.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:26.598+0000 7f8716000700 1 -- 172.21.15.117:0/3169815132 wait complete. 2024-04-20T11:45:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:26 smithi173 ceph-mon[74165]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:26 smithi173 ceph-mon[74165]: pgmap v990: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:26 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1165736788' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:45:27.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:27 smithi117 ceph-mon[99036]: from='client.35786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:27.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:27 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3169815132' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:45:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:27 smithi173 ceph-mon[74165]: from='client.35786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:27 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3169815132' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:45:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:28 smithi117 ceph-mon[99036]: pgmap v991: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:28 smithi173 ceph-mon[74165]: pgmap v991: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:30 smithi117 ceph-mon[99036]: pgmap v992: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:30 smithi173 ceph-mon[74165]: pgmap v992: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:32 smithi117 ceph-mon[99036]: pgmap v993: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:32 smithi173 ceph-mon[74165]: pgmap v993: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:34.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:34 smithi117 ceph-mon[99036]: pgmap v994: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:34 smithi173 ceph-mon[74165]: pgmap v994: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:36 smithi173 ceph-mon[74165]: pgmap v995: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:36.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:36 smithi117 ceph-mon[99036]: pgmap v995: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:38.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:38 smithi117 ceph-mon[99036]: pgmap v996: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:38 smithi173 ceph-mon[74165]: pgmap v996: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:40 smithi173 ceph-mon[74165]: pgmap v997: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:40.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:40 smithi117 ceph-mon[99036]: pgmap v997: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:42.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:42 smithi117 ceph-mon[99036]: pgmap v998: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:42 smithi173 ceph-mon[74165]: pgmap v998: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:43.819 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:44 smithi117 ceph-mon[99036]: pgmap v999: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:44 smithi173 ceph-mon[74165]: pgmap v999: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:46.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:46 smithi117 ceph-mon[99036]: pgmap v1000: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:46 smithi173 ceph-mon[74165]: pgmap v1000: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:48.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:48 smithi117 ceph-mon[99036]: pgmap v1001: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:48 smithi173 ceph-mon[74165]: pgmap v1001: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:50.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:50 smithi173 ceph-mon[74165]: pgmap v1002: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:50.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:50 smithi117 ceph-mon[99036]: pgmap v1002: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:52.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:52 smithi117 ceph-mon[99036]: pgmap v1003: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:52 smithi173 ceph-mon[74165]: pgmap v1003: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:54.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:54 smithi117 ceph-mon[99036]: pgmap v1004: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:54.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:54 smithi173 ceph-mon[74165]: pgmap v1004: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.766+0000 7fc86fa00700 1 -- 172.21.15.117:0/3039025865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 msgr2=0x7fc868109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.766+0000 7fc86fa00700 1 --2- 172.21.15.117:0/3039025865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 0x7fc868109290 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc854009910 tx=0x7fc854009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.766+0000 7fc86fa00700 1 -- 172.21.15.117:0/3039025865 shutdown_connections 2024-04-20T11:45:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.766+0000 7fc86fa00700 1 --2- 172.21.15.117:0/3039025865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 0x7fc868109290 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.766+0000 7fc86fa00700 1 --2- 172.21.15.117:0/3039025865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc868107520 0x7fc868107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.766+0000 7fc86fa00700 1 -- 172.21.15.117:0/3039025865 >> 172.21.15.117:0/3039025865 conn(0x7fc8680768f0 msgr2=0x7fc868078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:56.768 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:56 smithi117 ceph-mon[99036]: pgmap v1005: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.767+0000 7fc86fa00700 1 -- 172.21.15.117:0/3039025865 shutdown_connections 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.767+0000 7fc86fa00700 1 -- 172.21.15.117:0/3039025865 wait complete. 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc86fa00700 1 Processor -- start 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc86fa00700 1 -- start start 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc86fa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc868107520 0x7fc86819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc86fa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 0x7fc86819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc86fa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86819ded0 con 0x7fc868108e10 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc86fa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8681a1ba0 con 0x7fc868107520 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc867e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 0x7fc86819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc867e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 0x7fc86819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33058/0 (socket says 172.21.15.117:33058) 2024-04-20T11:45:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.768+0000 7fc867e00700 1 -- 172.21.15.117:0/1750415399 learned_addr learned my addr 172.21.15.117:0/1750415399 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:56.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.769+0000 7fc867e00700 1 -- 172.21.15.117:0/1750415399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc868107520 msgr2=0x7fc86819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:56.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.769+0000 7fc86d000700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc868107520 0x7fc86819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:56.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.769+0000 7fc867e00700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc868107520 0x7fc86819d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:56.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.769+0000 7fc867e00700 1 -- 172.21.15.117:0/1750415399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540095f0 con 0x7fc868108e10 2024-04-20T11:45:56.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.769+0000 7fc867e00700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 0x7fc86819d7f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc8540092e0 tx=0x7fc85400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:56.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.770+0000 7fc865600700 1 -- 172.21.15.117:0/1750415399 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc85401d070 con 0x7fc868108e10 2024-04-20T11:45:56.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.770+0000 7fc865600700 1 -- 172.21.15.117:0/1750415399 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc85400f460 con 0x7fc868108e10 2024-04-20T11:45:56.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.770+0000 7fc865600700 1 -- 172.21.15.117:0/1750415399 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc854017630 con 0x7fc868108e10 2024-04-20T11:45:56.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.770+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8681a1e20 con 0x7fc868108e10 2024-04-20T11:45:56.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.770+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8681a2390 con 0x7fc868108e10 2024-04-20T11:45:56.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.771+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86810f890 con 0x7fc868108e10 2024-04-20T11:45:56.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.772+0000 7fc865600700 1 -- 172.21.15.117:0/1750415399 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc85400f5d0 con 0x7fc868108e10 2024-04-20T11:45:56.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.773+0000 7fc865600700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc85006ecd0 0x7fc850071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:56.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.773+0000 7fc865600700 1 -- 172.21.15.117:0/1750415399 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc854090320 con 0x7fc868108e10 2024-04-20T11:45:56.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.773+0000 7fc86d000700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc85006ecd0 0x7fc850071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:56.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.774+0000 7fc86d000700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc85006ecd0 0x7fc850071190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc85c006fd0 tx=0x7fc85c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:56.778+0000 7fc865600700 1 -- 172.21.15.117:0/1750415399 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc8540590e0 con 0x7fc868108e10 2024-04-20T11:45:56.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:56 smithi173 ceph-mon[74165]: pgmap v1005: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:57.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.021+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc86819e630 con 0x7fc85006ecd0 2024-04-20T11:45:57.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.023+0000 7fc865600700 1 -- 172.21.15.117:0/1750415399 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc86819e630 con 0x7fc85006ecd0 2024-04-20T11:45:57.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc85006ecd0 msgr2=0x7fc850071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc85006ecd0 0x7fc850071190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc85c006fd0 tx=0x7fc85c008040 comp rx=0 tx=0).stop 2024-04-20T11:45:57.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 msgr2=0x7fc86819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 0x7fc86819d7f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc8540092e0 tx=0x7fc85400be20 comp rx=0 tx=0).stop 2024-04-20T11:45:57.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 shutdown_connections 2024-04-20T11:45:57.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc85006ecd0 0x7fc850071190 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc868107520 0x7fc86819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 --2- 172.21.15.117:0/1750415399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc868108e10 0x7fc86819d7f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.026+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 >> 172.21.15.117:0/1750415399 conn(0x7fc8680768f0 msgr2=0x7fc8680775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:57.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.027+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 shutdown_connections 2024-04-20T11:45:57.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.027+0000 7fc86fa00700 1 -- 172.21.15.117:0/1750415399 wait complete. 2024-04-20T11:45:57.042 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:45:57.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.184+0000 7efea5800700 1 -- 172.21.15.117:0/3698618588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104a00 msgr2=0x7efea0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.184+0000 7efea5800700 1 --2- 172.21.15.117:0/3698618588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104a00 0x7efea0104e80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009910 tx=0x7efe8c009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:57.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.184+0000 7efea5800700 1 -- 172.21.15.117:0/3698618588 shutdown_connections 2024-04-20T11:45:57.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.184+0000 7efea5800700 1 --2- 172.21.15.117:0/3698618588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0104a00 0x7efea0104e80 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.184+0000 7efea5800700 1 --2- 172.21.15.117:0/3698618588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efea0103110 0x7efea01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.184+0000 7efea5800700 1 -- 172.21.15.117:0/3698618588 >> 172.21.15.117:0/3698618588 conn(0x7efea00fe880 msgr2=0x7efea0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:57.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.184+0000 7efea5800700 1 -- 172.21.15.117:0/3698618588 shutdown_connections 2024-04-20T11:45:57.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.184+0000 7efea5800700 1 -- 172.21.15.117:0/3698618588 wait complete. 2024-04-20T11:45:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.185+0000 7efea5800700 1 Processor -- start 2024-04-20T11:45:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.185+0000 7efea5800700 1 -- start start 2024-04-20T11:45:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efea5800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0103110 0x7efea00753f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efea5800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efea0104a00 0x7efea0075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efea5800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea0079470 con 0x7efea0103110 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efea5800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00795e0 con 0x7efea0104a00 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efe9ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0103110 0x7efea00753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efe9ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0103110 0x7efea00753f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33082/0 (socket says 172.21.15.117:33082) 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efe9ea00700 1 -- 172.21.15.117:0/2901396901 learned_addr learned my addr 172.21.15.117:0/2901396901 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efe9ea00700 1 -- 172.21.15.117:0/2901396901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efea0104a00 msgr2=0x7efea0075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efe9ea00700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efea0104a00 0x7efea0075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.186+0000 7efe9ea00700 1 -- 172.21.15.117:0/2901396901 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0095f0 con 0x7efea0103110 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.187+0000 7efe9ea00700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0103110 0x7efea00753f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b4b0 tx=0x7efe9000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.187+0000 7efe97400700 1 -- 172.21.15.117:0/2901396901 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe90011800 con 0x7efea0103110 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.187+0000 7efe97400700 1 -- 172.21.15.117:0/2901396901 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe90011e20 con 0x7efea0103110 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.187+0000 7efe97400700 1 -- 172.21.15.117:0/2901396901 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9000f2e0 con 0x7efea0103110 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.187+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea0075f90 con 0x7efea0103110 2024-04-20T11:45:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.187+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea01a7620 con 0x7efea0103110 2024-04-20T11:45:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.188+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea010b460 con 0x7efea0103110 2024-04-20T11:45:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.189+0000 7efe97400700 1 -- 172.21.15.117:0/2901396901 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efe90010380 con 0x7efea0103110 2024-04-20T11:45:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.190+0000 7efe97400700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe8406ecd0 0x7efe84071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.190+0000 7efe97400700 1 -- 172.21.15.117:0/2901396901 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efe9008ea00 con 0x7efea0103110 2024-04-20T11:45:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.191+0000 7efe9e000700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe8406ecd0 0x7efe84071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.191+0000 7efe9e000700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe8406ecd0 0x7efe84071190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006010 tx=0x7efe8c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.195+0000 7efe97400700 1 -- 172.21.15.117:0/2901396901 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efe90057870 con 0x7efea0103110 2024-04-20T11:45:57.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.440+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efea0076880 con 0x7efe8406ecd0 2024-04-20T11:45:57.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.444+0000 7efe97400700 1 -- 172.21.15.117:0/2901396901 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efea0076880 con 0x7efe8406ecd0 2024-04-20T11:45:57.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe8406ecd0 msgr2=0x7efe84071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe8406ecd0 0x7efe84071190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006010 tx=0x7efe8c00b540 comp rx=0 tx=0).stop 2024-04-20T11:45:57.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0103110 msgr2=0x7efea00753f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0103110 0x7efea00753f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b4b0 tx=0x7efe9000b870 comp rx=0 tx=0).stop 2024-04-20T11:45:57.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 shutdown_connections 2024-04-20T11:45:57.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe8406ecd0 0x7efe84071190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea0103110 0x7efea00753f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 --2- 172.21.15.117:0/2901396901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efea0104a00 0x7efea0075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 >> 172.21.15.117:0/2901396901 conn(0x7efea00fe880 msgr2=0x7efea00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:57.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 shutdown_connections 2024-04-20T11:45:57.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.446+0000 7efea5800700 1 -- 172.21.15.117:0/2901396901 wait complete. 2024-04-20T11:45:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.596+0000 7f4c89c00700 1 -- 172.21.15.117:0/1081072934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c841039f0 msgr2=0x7f4c84107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.596+0000 7f4c89c00700 1 --2- 172.21.15.117:0/1081072934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c841039f0 0x7f4c84107a40 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009910 tx=0x7f4c70009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.596+0000 7f4c89c00700 1 -- 172.21.15.117:0/1081072934 shutdown_connections 2024-04-20T11:45:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.596+0000 7f4c89c00700 1 --2- 172.21.15.117:0/1081072934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c841039f0 0x7f4c84107a40 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.596+0000 7f4c89c00700 1 --2- 172.21.15.117:0/1081072934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c84103040 0x7f4c84103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.596+0000 7f4c89c00700 1 -- 172.21.15.117:0/1081072934 >> 172.21.15.117:0/1081072934 conn(0x7f4c840fe8d0 msgr2=0x7f4c84100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:57.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.596+0000 7f4c89c00700 1 -- 172.21.15.117:0/1081072934 shutdown_connections 2024-04-20T11:45:57.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.596+0000 7f4c89c00700 1 -- 172.21.15.117:0/1081072934 wait complete. 2024-04-20T11:45:57.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.597+0000 7f4c89c00700 1 Processor -- start 2024-04-20T11:45:57.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.597+0000 7f4c89c00700 1 -- start start 2024-04-20T11:45:57.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.597+0000 7f4c89c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c84103040 0x7f4c84198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:57.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c89c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c841039f0 0x7f4c841992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:57.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c89c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c84199940 con 0x7f4c84103040 2024-04-20T11:45:57.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c89c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8419c560 con 0x7f4c841039f0 2024-04-20T11:45:57.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c82a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c84103040 0x7f4c84198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:57.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c82a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c84103040 0x7f4c84198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33086/0 (socket says 172.21.15.117:33086) 2024-04-20T11:45:57.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c82a00700 1 -- 172.21.15.117:0/118856081 learned_addr learned my addr 172.21.15.117:0/118856081 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:57.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c82a00700 1 -- 172.21.15.117:0/118856081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c841039f0 msgr2=0x7f4c841992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c82a00700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c841039f0 0x7f4c841992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.598+0000 7f4c82a00700 1 -- 172.21.15.117:0/118856081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c700095f0 con 0x7f4c84103040 2024-04-20T11:45:57.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.599+0000 7f4c82a00700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c84103040 0x7f4c84198d70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400c8a0 tx=0x7f4c7400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:57.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.599+0000 7f4c7b400700 1 -- 172.21.15.117:0/118856081 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7400cea0 con 0x7f4c84103040 2024-04-20T11:45:57.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.599+0000 7f4c7b400700 1 -- 172.21.15.117:0/118856081 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c74004830 con 0x7f4c84103040 2024-04-20T11:45:57.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.599+0000 7f4c7b400700 1 -- 172.21.15.117:0/118856081 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c740055b0 con 0x7f4c84103040 2024-04-20T11:45:57.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.599+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8419c840 con 0x7f4c84103040 2024-04-20T11:45:57.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.599+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8419cd90 con 0x7f4c84103040 2024-04-20T11:45:57.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.600+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8410b440 con 0x7f4c84103040 2024-04-20T11:45:57.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.601+0000 7f4c7b400700 1 -- 172.21.15.117:0/118856081 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4c740049a0 con 0x7f4c84103040 2024-04-20T11:45:57.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.602+0000 7f4c7b400700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c6806ecc0 0x7f4c68071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:57.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.602+0000 7f4c7b400700 1 -- 172.21.15.117:0/118856081 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f4c7408eac0 con 0x7f4c84103040 2024-04-20T11:45:57.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.602+0000 7f4c82000700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c6806ecc0 0x7f4c68071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:57.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.603+0000 7f4c82000700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c6806ecc0 0x7f4c68071180 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4c70006010 tx=0x7f4c7000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:57.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.607+0000 7f4c7b400700 1 -- 172.21.15.117:0/118856081 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4c74057880 con 0x7f4c84103040 2024-04-20T11:45:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.826+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c8410ffe0 con 0x7f4c6806ecc0 2024-04-20T11:45:57.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.858+0000 7f4c7b400700 1 -- 172.21.15.117:0/118856081 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4c8410ffe0 con 0x7f4c6806ecc0 2024-04-20T11:45:57.859 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (34m) 10m ago 48m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (48m) 10m ago 48m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (47m) 10m ago 47m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (30m) 10m ago 48m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (30m) 10m ago 47m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (33m) 10m ago 48m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (26m) 10m ago 41m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (26m) 10m ago 41m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (10m) 10m ago 41m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (25m) 10m ago 41m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (37m) 10m ago 50m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (36m) 10m ago 47m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:45:57.860 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (32m) 10m ago 50m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (31m) 10m ago 47m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (35m) 10m ago 48m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (35m) 10m ago 47m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (30m) 10m ago 46m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (29m) 10m ago 45m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (29m) 10m ago 44m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (28m) 10m ago 44m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (27m) 10m ago 43m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (27m) 10m ago 43m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:45:57.861 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (35m) 10m ago 47m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:45:57.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c6806ecc0 msgr2=0x7f4c68071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c6806ecc0 0x7f4c68071180 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4c70006010 tx=0x7f4c7000b540 comp rx=0 tx=0).stop 2024-04-20T11:45:57.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c84103040 msgr2=0x7f4c84198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:57.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c84103040 0x7f4c84198d70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400c8a0 tx=0x7f4c7400cc60 comp rx=0 tx=0).stop 2024-04-20T11:45:57.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 shutdown_connections 2024-04-20T11:45:57.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c6806ecc0 0x7f4c68071180 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c84103040 0x7f4c84198d70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 --2- 172.21.15.117:0/118856081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c841039f0 0x7f4c841992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:57.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.861+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 >> 172.21.15.117:0/118856081 conn(0x7f4c840fe8d0 msgr2=0x7f4c840fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:57.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.862+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 shutdown_connections 2024-04-20T11:45:57.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:57.862+0000 7f4c89c00700 1 -- 172.21.15.117:0/118856081 wait complete. 2024-04-20T11:45:58.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.015+0000 7feab4000700 1 -- 172.21.15.117:0/1023551203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 msgr2=0x7feaac1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.015+0000 7feab4000700 1 --2- 172.21.15.117:0/1023551203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 0x7feaac1011f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009910 tx=0x7feaa0009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:58.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.015+0000 7feab4000700 1 -- 172.21.15.117:0/1023551203 shutdown_connections 2024-04-20T11:45:58.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.015+0000 7feab4000700 1 --2- 172.21.15.117:0/1023551203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaac1017c0 0x7feaac105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.015+0000 7feab4000700 1 --2- 172.21.15.117:0/1023551203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 0x7feaac1011f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.015+0000 7feab4000700 1 -- 172.21.15.117:0/1023551203 >> 172.21.15.117:0/1023551203 conn(0x7feaac0fc6a0 msgr2=0x7feaac0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:58.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.015+0000 7feab4000700 1 -- 172.21.15.117:0/1023551203 shutdown_connections 2024-04-20T11:45:58.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.015+0000 7feab4000700 1 -- 172.21.15.117:0/1023551203 wait complete. 2024-04-20T11:45:58.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.016+0000 7feab4000700 1 Processor -- start 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.016+0000 7feab4000700 1 -- start start 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab4000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 0x7feaac19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab4000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaac1017c0 0x7feaac19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab4000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac19d490 con 0x7feaac100e10 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab4000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac196a70 con 0x7feaac1017c0 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab1600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 0x7feaac19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab1600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 0x7feaac19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33104/0 (socket says 172.21.15.117:33104) 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab1600700 1 -- 172.21.15.117:0/31607138 learned_addr learned my addr 172.21.15.117:0/31607138 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab1600700 1 -- 172.21.15.117:0/31607138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaac1017c0 msgr2=0x7feaac19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.017+0000 7feab1600700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaac1017c0 0x7feaac19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.018+0000 7feab1600700 1 -- 172.21.15.117:0/31607138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00095f0 con 0x7feaac100e10 2024-04-20T11:45:58.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.018+0000 7feab1600700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 0x7feaac19c930 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feaa0006010 tx=0x7feaa000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:58.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.018+0000 7feaaa000700 1 -- 172.21.15.117:0/31607138 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa001d070 con 0x7feaac100e10 2024-04-20T11:45:58.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.018+0000 7feaaa000700 1 -- 172.21.15.117:0/31607138 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaa000f460 con 0x7feaac100e10 2024-04-20T11:45:58.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.018+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac196cf0 con 0x7feaac100e10 2024-04-20T11:45:58.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.018+0000 7feaaa000700 1 -- 172.21.15.117:0/31607138 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa0017630 con 0x7feaac100e10 2024-04-20T11:45:58.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.018+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac1971e0 con 0x7feaac100e10 2024-04-20T11:45:58.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.019+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaac1022f0 con 0x7feaac100e10 2024-04-20T11:45:58.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.020+0000 7feaaa000700 1 -- 172.21.15.117:0/31607138 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7feaa000f5d0 con 0x7feaac100e10 2024-04-20T11:45:58.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.021+0000 7feaaa000700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fea9406ecd0 0x7fea94071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.021+0000 7feaaa000700 1 -- 172.21.15.117:0/31607138 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7feaa00903c0 con 0x7feaac100e10 2024-04-20T11:45:58.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.022+0000 7feab0c00700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fea9406ecd0 0x7fea94071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:58.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.022+0000 7feab0c00700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fea9406ecd0 0x7fea94071190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feaac197c30 tx=0x7fea9c006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:58.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.026+0000 7feaaa000700 1 -- 172.21.15.117:0/31607138 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7feaa005ccf0 con 0x7feaac100e10 2024-04-20T11:45:58.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.346+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feaac10def0 con 0x7feaac100e10 2024-04-20T11:45:58.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.348+0000 7feaaa000700 1 -- 172.21.15.117:0/31607138 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7feaa005c880 con 0x7feaac100e10 2024-04-20T11:45:58.349 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:45:58.349 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:45:58.349 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:45:58.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:45:58.349 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:45:58.349 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:45:58.350 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:45:58.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.350+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fea9406ecd0 msgr2=0x7fea94071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.350+0000 7feab4000700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fea9406ecd0 0x7fea94071190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feaac197c30 tx=0x7fea9c006d20 comp rx=0 tx=0).stop 2024-04-20T11:45:58.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.350+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 msgr2=0x7feaac19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.350+0000 7feab4000700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 0x7feaac19c930 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feaa0006010 tx=0x7feaa000be20 comp rx=0 tx=0).stop 2024-04-20T11:45:58.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.350+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 shutdown_connections 2024-04-20T11:45:58.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.350+0000 7feab4000700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fea9406ecd0 0x7fea94071190 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.350+0000 7feab4000700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaac100e10 0x7feaac19c930 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.350+0000 7feab4000700 1 --2- 172.21.15.117:0/31607138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaac1017c0 0x7feaac19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.351+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 >> 172.21.15.117:0/31607138 conn(0x7feaac0fc6a0 msgr2=0x7feaac1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:58.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.351+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 shutdown_connections 2024-04-20T11:45:58.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.351+0000 7feab4000700 1 -- 172.21.15.117:0/31607138 wait complete. 2024-04-20T11:45:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.509+0000 7fe8ef800700 1 -- 172.21.15.117:0/385152282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8108e10 msgr2=0x7fe8e8109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.509+0000 7fe8ef800700 1 --2- 172.21.15.117:0/385152282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8108e10 0x7fe8e8109290 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009910 tx=0x7fe8d4009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.509+0000 7fe8ef800700 1 -- 172.21.15.117:0/385152282 shutdown_connections 2024-04-20T11:45:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.509+0000 7fe8ef800700 1 --2- 172.21.15.117:0/385152282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8108e10 0x7fe8e8109290 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.509+0000 7fe8ef800700 1 --2- 172.21.15.117:0/385152282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e8107520 0x7fe8e8107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.509+0000 7fe8ef800700 1 -- 172.21.15.117:0/385152282 >> 172.21.15.117:0/385152282 conn(0x7fe8e80768f0 msgr2=0x7fe8e8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.510+0000 7fe8ef800700 1 -- 172.21.15.117:0/385152282 shutdown_connections 2024-04-20T11:45:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.510+0000 7fe8ef800700 1 -- 172.21.15.117:0/385152282 wait complete. 2024-04-20T11:45:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ef800700 1 Processor -- start 2024-04-20T11:45:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ef800700 1 -- start start 2024-04-20T11:45:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ef800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8107520 0x7fe8e819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ef800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e8108e10 0x7fe8e819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ef800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e819ded0 con 0x7fe8e8107520 2024-04-20T11:45:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ef800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a1ba0 con 0x7fe8e8108e10 2024-04-20T11:45:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ece00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8107520 0x7fe8e819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:58.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ece00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8107520 0x7fe8e819d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33132/0 (socket says 172.21.15.117:33132) 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.511+0000 7fe8ece00700 1 -- 172.21.15.117:0/1242768280 learned_addr learned my addr 172.21.15.117:0/1242768280 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.512+0000 7fe8ece00700 1 -- 172.21.15.117:0/1242768280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e8108e10 msgr2=0x7fe8e819d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.512+0000 7fe8ece00700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e8108e10 0x7fe8e819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.512+0000 7fe8ece00700 1 -- 172.21.15.117:0/1242768280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40095f0 con 0x7fe8e8107520 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.512+0000 7fe8ece00700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8107520 0x7fe8e819d2b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00b4b0 tx=0x7fe8dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.512+0000 7fe8e5600700 1 -- 172.21.15.117:0/1242768280 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8dc011800 con 0x7fe8e8107520 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.512+0000 7fe8e5600700 1 -- 172.21.15.117:0/1242768280 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8dc011e20 con 0x7fe8e8107520 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.512+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e81a1e80 con 0x7fe8e8107520 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.512+0000 7fe8e5600700 1 -- 172.21.15.117:0/1242768280 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8dc00f2e0 con 0x7fe8e8107520 2024-04-20T11:45:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.513+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e81a2450 con 0x7fe8e8107520 2024-04-20T11:45:58.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.514+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e810f870 con 0x7fe8e8107520 2024-04-20T11:45:58.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.515+0000 7fe8e5600700 1 -- 172.21.15.117:0/1242768280 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe8dc010380 con 0x7fe8e8107520 2024-04-20T11:45:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.516+0000 7fe8e5600700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8d006ecd0 0x7fe8d0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.516+0000 7fe8e5600700 1 -- 172.21.15.117:0/1242768280 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe8dc08ea70 con 0x7fe8e8107520 2024-04-20T11:45:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.516+0000 7fe8e7e00700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8d006ecd0 0x7fe8d0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.517+0000 7fe8e7e00700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8d006ecd0 0x7fe8d0071190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4006010 tx=0x7fe8d400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:58.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.521+0000 7fe8e5600700 1 -- 172.21.15.117:0/1242768280 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe8dc0577b0 con 0x7fe8e8107520 2024-04-20T11:45:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:58 smithi117 ceph-mon[99036]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:58 smithi117 ceph-mon[99036]: pgmap v1006: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:58 smithi117 ceph-mon[99036]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:58 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/31607138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:45:58.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.776+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe8e819e5c0 con 0x7fe8e8107520 2024-04-20T11:45:58.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.777+0000 7fe8e5600700 1 -- 172.21.15.117:0/1242768280 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fe8dc05ae70 con 0x7fe8e8107520 2024-04-20T11:45:58.778 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:45:58.778 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:45:58.778 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:45:58.779 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:45:58.780 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:45:58.781 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:45:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8d006ecd0 msgr2=0x7fe8d0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8d006ecd0 0x7fe8d0071190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4006010 tx=0x7fe8d400b540 comp rx=0 tx=0).stop 2024-04-20T11:45:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8107520 msgr2=0x7fe8e819d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8107520 0x7fe8e819d2b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00b4b0 tx=0x7fe8dc00b870 comp rx=0 tx=0).stop 2024-04-20T11:45:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 shutdown_connections 2024-04-20T11:45:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe8d006ecd0 0x7fe8d0071190 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e8107520 0x7fe8e819d2b0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 --2- 172.21.15.117:0/1242768280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8e8108e10 0x7fe8e819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 >> 172.21.15.117:0/1242768280 conn(0x7fe8e80768f0 msgr2=0x7fe8e80775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 shutdown_connections 2024-04-20T11:45:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.780+0000 7fe8ef800700 1 -- 172.21.15.117:0/1242768280 wait complete. 2024-04-20T11:45:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:45:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:58 smithi173 ceph-mon[74165]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:58 smithi173 ceph-mon[74165]: pgmap v1006: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:45:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:58 smithi173 ceph-mon[74165]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:45:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:58 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/31607138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:45:58.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.938+0000 7fae91a00700 1 -- 172.21.15.117:0/3664708641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c107520 msgr2=0x7fae8c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.938+0000 7fae91a00700 1 --2- 172.21.15.117:0/3664708641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c107520 0x7fae8c107900 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009910 tx=0x7fae7c009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:58.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.938+0000 7fae91a00700 1 -- 172.21.15.117:0/3664708641 shutdown_connections 2024-04-20T11:45:58.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.938+0000 7fae91a00700 1 --2- 172.21.15.117:0/3664708641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae8c108e10 0x7fae8c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.938+0000 7fae91a00700 1 --2- 172.21.15.117:0/3664708641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c107520 0x7fae8c107900 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.938+0000 7fae91a00700 1 -- 172.21.15.117:0/3664708641 >> 172.21.15.117:0/3664708641 conn(0x7fae8c0768f0 msgr2=0x7fae8c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:58.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.938+0000 7fae91a00700 1 -- 172.21.15.117:0/3664708641 shutdown_connections 2024-04-20T11:45:58.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.938+0000 7fae91a00700 1 -- 172.21.15.117:0/3664708641 wait complete. 2024-04-20T11:45:58.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.939+0000 7fae91a00700 1 Processor -- start 2024-04-20T11:45:58.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.940+0000 7fae91a00700 1 -- start start 2024-04-20T11:45:58.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.940+0000 7fae91a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae8c107520 0x7fae8c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.940+0000 7fae91a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c108e10 0x7fae8c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.940+0000 7fae91a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c19ddd0 con 0x7fae8c108e10 2024-04-20T11:45:58.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.940+0000 7fae91a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c1a1aa0 con 0x7fae8c107520 2024-04-20T11:45:58.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.940+0000 7fae8a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c108e10 0x7fae8c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:58.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.940+0000 7fae8a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c108e10 0x7fae8c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33142/0 (socket says 172.21.15.117:33142) 2024-04-20T11:45:58.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.940+0000 7fae8a000700 1 -- 172.21.15.117:0/738825011 learned_addr learned my addr 172.21.15.117:0/738825011 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:58.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.941+0000 7fae8aa00700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae8c107520 0x7fae8c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:58.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.941+0000 7fae8a000700 1 -- 172.21.15.117:0/738825011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae8c107520 msgr2=0x7fae8c19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:58.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.941+0000 7fae8a000700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae8c107520 0x7fae8c19d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:58.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.941+0000 7fae8a000700 1 -- 172.21.15.117:0/738825011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0095f0 con 0x7fae8c108e10 2024-04-20T11:45:58.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.941+0000 7fae8a000700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c108e10 0x7fae8c19d6f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fae7800c8a0 tx=0x7fae7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:58.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.942+0000 7fae83400700 1 -- 172.21.15.117:0/738825011 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae7800cea0 con 0x7fae8c108e10 2024-04-20T11:45:58.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.942+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c1a1d80 con 0x7fae8c108e10 2024-04-20T11:45:58.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.942+0000 7fae83400700 1 -- 172.21.15.117:0/738825011 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae78004830 con 0x7fae8c108e10 2024-04-20T11:45:58.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.942+0000 7fae83400700 1 -- 172.21.15.117:0/738825011 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae780055b0 con 0x7fae8c108e10 2024-04-20T11:45:58.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.942+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c1a2350 con 0x7fae8c108e10 2024-04-20T11:45:58.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.943+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c10f870 con 0x7fae8c108e10 2024-04-20T11:45:58.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.944+0000 7fae83400700 1 -- 172.21.15.117:0/738825011 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fae780049a0 con 0x7fae8c108e10 2024-04-20T11:45:58.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.946+0000 7fae83400700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae7006ecc0 0x7fae70071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:58.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.946+0000 7fae83400700 1 -- 172.21.15.117:0/738825011 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fae7808ebc0 con 0x7fae8c108e10 2024-04-20T11:45:58.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.946+0000 7fae8aa00700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae7006ecc0 0x7fae70071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:58.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.947+0000 7fae8aa00700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae7006ecc0 0x7fae70071180 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b5c0 tx=0x7fae7c005380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:58.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:58.951+0000 7fae83400700 1 -- 172.21.15.117:0/738825011 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fae78057900 con 0x7fae8c108e10 2024-04-20T11:45:59.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.192+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae8c19e510 con 0x7fae7006ecc0 2024-04-20T11:45:59.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.194+0000 7fae83400700 1 -- 172.21.15.117:0/738825011 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fae8c19e510 con 0x7fae7006ecc0 2024-04-20T11:45:59.195 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:45:59.195 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:45:59.195 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:45:59.195 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:45:59.196 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:45:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae7006ecc0 msgr2=0x7fae70071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae7006ecc0 0x7fae70071180 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b5c0 tx=0x7fae7c005380 comp rx=0 tx=0).stop 2024-04-20T11:45:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c108e10 msgr2=0x7fae8c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c108e10 0x7fae8c19d6f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fae7800c8a0 tx=0x7fae7800cc60 comp rx=0 tx=0).stop 2024-04-20T11:45:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 shutdown_connections 2024-04-20T11:45:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fae7006ecc0 0x7fae70071180 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae8c107520 0x7fae8c19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 --2- 172.21.15.117:0/738825011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8c108e10 0x7fae8c19d6f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.197+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 >> 172.21.15.117:0/738825011 conn(0x7fae8c0768f0 msgr2=0x7fae8c077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.198+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 shutdown_connections 2024-04-20T11:45:59.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.198+0000 7fae91a00700 1 -- 172.21.15.117:0/738825011 wait complete. 2024-04-20T11:45:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.348+0000 7f7e63000700 1 -- 172.21.15.117:0/3578938487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 msgr2=0x7f7e5c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.348+0000 7f7e63000700 1 --2- 172.21.15.117:0/3578938487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 0x7f7e5c1011f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009910 tx=0x7f7e4c009c20 comp rx=0 tx=0).stop 2024-04-20T11:45:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.348+0000 7f7e63000700 1 -- 172.21.15.117:0/3578938487 shutdown_connections 2024-04-20T11:45:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.348+0000 7f7e63000700 1 --2- 172.21.15.117:0/3578938487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e5c1017c0 0x7f7e5c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.348+0000 7f7e63000700 1 --2- 172.21.15.117:0/3578938487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 0x7f7e5c1011f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.348+0000 7f7e63000700 1 -- 172.21.15.117:0/3578938487 >> 172.21.15.117:0/3578938487 conn(0x7f7e5c0fc6c0 msgr2=0x7f7e5c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.348+0000 7f7e63000700 1 -- 172.21.15.117:0/3578938487 shutdown_connections 2024-04-20T11:45:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.348+0000 7f7e63000700 1 -- 172.21.15.117:0/3578938487 wait complete. 2024-04-20T11:45:59.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.349+0000 7f7e63000700 1 Processor -- start 2024-04-20T11:45:59.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.349+0000 7f7e63000700 1 -- start start 2024-04-20T11:45:59.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e63000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 0x7f7e5c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:59.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e63000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e5c1017c0 0x7f7e5c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:59.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e63000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c19d360 con 0x7f7e5c100e10 2024-04-20T11:45:59.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e63000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c196940 con 0x7f7e5c1017c0 2024-04-20T11:45:59.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e5be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 0x7f7e5c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:59.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e5be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 0x7f7e5c19c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33162/0 (socket says 172.21.15.117:33162) 2024-04-20T11:45:59.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e5be00700 1 -- 172.21.15.117:0/685098336 learned_addr learned my addr 172.21.15.117:0/685098336 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e5be00700 1 -- 172.21.15.117:0/685098336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e5c1017c0 msgr2=0x7f7e5c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e5be00700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e5c1017c0 0x7f7e5c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.350+0000 7f7e5be00700 1 -- 172.21.15.117:0/685098336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c0095f0 con 0x7f7e5c100e10 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.351+0000 7f7e5be00700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 0x7f7e5c19c800 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006010 tx=0x7f7e4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.351+0000 7f7e58c00700 1 -- 172.21.15.117:0/685098336 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e4c01d070 con 0x7f7e5c100e10 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.351+0000 7f7e58c00700 1 -- 172.21.15.117:0/685098336 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e4c00f460 con 0x7f7e5c100e10 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.351+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c196bc0 con 0x7f7e5c100e10 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.351+0000 7f7e58c00700 1 -- 172.21.15.117:0/685098336 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e4c017630 con 0x7f7e5c100e10 2024-04-20T11:45:59.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.351+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c1970b0 con 0x7f7e5c100e10 2024-04-20T11:45:59.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.352+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c1091b0 con 0x7f7e5c100e10 2024-04-20T11:45:59.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.353+0000 7f7e58c00700 1 -- 172.21.15.117:0/685098336 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7e4c00f5d0 con 0x7f7e5c100e10 2024-04-20T11:45:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.354+0000 7f7e58c00700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e4006ecc0 0x7f7e40071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:45:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.354+0000 7f7e58c00700 1 -- 172.21.15.117:0/685098336 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7e4c0903f0 con 0x7f7e5c100e10 2024-04-20T11:45:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.355+0000 7f7e5b400700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e4006ecc0 0x7f7e40071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:45:59.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.355+0000 7f7e5b400700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e4006ecc0 0x7f7e40071180 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c197b00 tx=0x7f7e48005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:45:59.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.360+0000 7f7e58c00700 1 -- 172.21.15.117:0/685098336 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7e4c05cd20 con 0x7f7e5c100e10 2024-04-20T11:45:59.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.663+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7e5c197e30 con 0x7f7e5c100e10 2024-04-20T11:45:59.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.664+0000 7f7e58c00700 1 -- 172.21.15.117:0/685098336 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f7e4c020370 con 0x7f7e5c100e10 2024-04-20T11:45:59.665 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:45:59.665 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:45:59.666 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:45:59.667 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:45:59.667 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:45:59.667 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:45:59.667 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:45:59.667 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:45:59.667 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:45:59.668 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:45:59.668 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:45:59.668 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:45:59.668 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:45:59.668 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:45:59.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e4006ecc0 msgr2=0x7f7e40071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:59.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e4006ecc0 0x7f7e40071180 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c197b00 tx=0x7f7e48005c80 comp rx=0 tx=0).stop 2024-04-20T11:45:59.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 msgr2=0x7f7e5c19c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:45:59.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 0x7f7e5c19c800 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006010 tx=0x7f7e4c00be20 comp rx=0 tx=0).stop 2024-04-20T11:45:59.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 shutdown_connections 2024-04-20T11:45:59.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7e4006ecc0 0x7f7e40071180 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5c100e10 0x7f7e5c19c800 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 --2- 172.21.15.117:0/685098336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e5c1017c0 0x7f7e5c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:45:59.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 >> 172.21.15.117:0/685098336 conn(0x7f7e5c0fc6c0 msgr2=0x7f7e5c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:45:59.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 shutdown_connections 2024-04-20T11:45:59.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:45:59.667+0000 7f7e63000700 1 -- 172.21.15.117:0/685098336 wait complete. 2024-04-20T11:45:59.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:59 smithi117 ceph-mon[99036]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:59.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:45:59 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1242768280' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:45:59.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:59 smithi173 ceph-mon[74165]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:45:59.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:45:59 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1242768280' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:46:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:00 smithi117 ceph-mon[99036]: from='client.35814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:00 smithi117 ceph-mon[99036]: pgmap v1007: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:00 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/685098336' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:46:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:00 smithi173 ceph-mon[74165]: from='client.35814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:00 smithi173 ceph-mon[74165]: pgmap v1007: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:00 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/685098336' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:46:02.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:02 smithi117 ceph-mon[99036]: pgmap v1008: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:02.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:02 smithi173 ceph-mon[74165]: pgmap v1008: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:04.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:04 smithi117 ceph-mon[99036]: pgmap v1009: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:04 smithi173 ceph-mon[74165]: pgmap v1009: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:06.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:06 smithi117 ceph-mon[99036]: pgmap v1010: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:06 smithi173 ceph-mon[74165]: pgmap v1010: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:08.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:08 smithi117 ceph-mon[99036]: pgmap v1011: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:08 smithi173 ceph-mon[74165]: pgmap v1011: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:10.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:10 smithi117 ceph-mon[99036]: pgmap v1012: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:10 smithi173 ceph-mon[74165]: pgmap v1012: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:12 smithi117 ceph-mon[99036]: pgmap v1013: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:12 smithi173 ceph-mon[74165]: pgmap v1013: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:46:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:13 smithi117 ceph-mon[99036]: pgmap v1014: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:46:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:13 smithi173 ceph-mon[74165]: pgmap v1014: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:16 smithi117 ceph-mon[99036]: pgmap v1015: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:16.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:16 smithi173 ceph-mon[74165]: pgmap v1015: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:18 smithi117 ceph-mon[99036]: pgmap v1016: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:18 smithi173 ceph-mon[74165]: pgmap v1016: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:20.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:20 smithi117 ceph-mon[99036]: pgmap v1017: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:20.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:20 smithi173 ceph-mon[74165]: pgmap v1017: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:22 smithi117 ceph-mon[99036]: pgmap v1018: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:22 smithi173 ceph-mon[74165]: pgmap v1018: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:24 smithi117 ceph-mon[99036]: pgmap v1019: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:24 smithi173 ceph-mon[74165]: pgmap v1019: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:26.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:26 smithi117 ceph-mon[99036]: pgmap v1020: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:26 smithi173 ceph-mon[74165]: pgmap v1020: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:28 smithi117 ceph-mon[99036]: pgmap v1021: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:46:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:28 smithi173 ceph-mon[74165]: pgmap v1021: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:46:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.839+0000 7f1f46c00700 1 -- 172.21.15.117:0/1187573412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40100b50 msgr2=0x7f1f40100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.839+0000 7f1f46c00700 1 --2- 172.21.15.117:0/1187573412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40100b50 0x7f1f40100fd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009910 tx=0x7f1f2c009c20 comp rx=0 tx=0).stop 2024-04-20T11:46:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.839+0000 7f1f46c00700 1 -- 172.21.15.117:0/1187573412 shutdown_connections 2024-04-20T11:46:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.839+0000 7f1f46c00700 1 --2- 172.21.15.117:0/1187573412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40100b50 0x7f1f40100fd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.839+0000 7f1f46c00700 1 --2- 172.21.15.117:0/1187573412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f40106c00 0x7f1f40106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.839+0000 7f1f46c00700 1 -- 172.21.15.117:0/1187573412 >> 172.21.15.117:0/1187573412 conn(0x7f1f400fc640 msgr2=0x7f1f400fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.839+0000 7f1f46c00700 1 -- 172.21.15.117:0/1187573412 shutdown_connections 2024-04-20T11:46:29.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.839+0000 7f1f46c00700 1 -- 172.21.15.117:0/1187573412 wait complete. 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.840+0000 7f1f46c00700 1 Processor -- start 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.840+0000 7f1f46c00700 1 -- start start 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f46c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f40100b50 0x7f1f4019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f46c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40106c00 0x7f1f4019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f46c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4019f460 con 0x7f1f40106c00 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f46c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f40198a90 con 0x7f1f40100b50 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f3f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40106c00 0x7f1f4019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f3f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40106c00 0x7f1f4019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37360/0 (socket says 172.21.15.117:37360) 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f3f400700 1 -- 172.21.15.117:0/2299703750 learned_addr learned my addr 172.21.15.117:0/2299703750 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:46:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f3fe00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f40100b50 0x7f1f4019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:29.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f3f400700 1 -- 172.21.15.117:0/2299703750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f40100b50 msgr2=0x7f1f4019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:29.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f3f400700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f40100b50 0x7f1f4019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:29.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.841+0000 7f1f3f400700 1 -- 172.21.15.117:0/2299703750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0095f0 con 0x7f1f40106c00 2024-04-20T11:46:29.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.842+0000 7f1f3f400700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40106c00 0x7f1f4019ee40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c000c00 tx=0x7f1f2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:29.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.842+0000 7f1f3cc00700 1 -- 172.21.15.117:0/2299703750 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2c01d070 con 0x7f1f40106c00 2024-04-20T11:46:29.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.842+0000 7f1f3cc00700 1 -- 172.21.15.117:0/2299703750 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f2c0056e0 con 0x7f1f40106c00 2024-04-20T11:46:29.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.842+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f40198d10 con 0x7f1f40106c00 2024-04-20T11:46:29.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.842+0000 7f1f3cc00700 1 -- 172.21.15.117:0/2299703750 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2c00fe80 con 0x7f1f40106c00 2024-04-20T11:46:29.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.842+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f40199280 con 0x7f1f40106c00 2024-04-20T11:46:29.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.844+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f40105ea0 con 0x7f1f40106c00 2024-04-20T11:46:29.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.845+0000 7f1f3cc00700 1 -- 172.21.15.117:0/2299703750 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1f2c00f6f0 con 0x7f1f40106c00 2024-04-20T11:46:29.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.846+0000 7f1f3cc00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f2406ecd0 0x7f1f24071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:29.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.846+0000 7f1f3cc00700 1 -- 172.21.15.117:0/2299703750 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1f2c090200 con 0x7f1f40106c00 2024-04-20T11:46:29.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.847+0000 7f1f3fe00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f2406ecd0 0x7f1f24071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:29.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.847+0000 7f1f3fe00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f2406ecd0 0x7f1f24071190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1f30005950 tx=0x7f1f300058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:29.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:29.852+0000 7f1f3cc00700 1 -- 172.21.15.117:0/2299703750 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f1f2c05caf0 con 0x7f1f40106c00 2024-04-20T11:46:30.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.101+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f4010dd40 con 0x7f1f2406ecd0 2024-04-20T11:46:30.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.103+0000 7f1f3cc00700 1 -- 172.21.15.117:0/2299703750 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1f4010dd40 con 0x7f1f2406ecd0 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.105+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f2406ecd0 msgr2=0x7f1f24071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.105+0000 7f1f46c00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f2406ecd0 0x7f1f24071190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1f30005950 tx=0x7f1f300058e0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.105+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40106c00 msgr2=0x7f1f4019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.105+0000 7f1f46c00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40106c00 0x7f1f4019ee40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c000c00 tx=0x7f1f2c00be20 comp rx=0 tx=0).stop 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.106+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 shutdown_connections 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.106+0000 7f1f46c00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f1f2406ecd0 0x7f1f24071190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.106+0000 7f1f46c00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f40100b50 0x7f1f4019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.106+0000 7f1f46c00700 1 --2- 172.21.15.117:0/2299703750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f40106c00 0x7f1f4019ee40 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.106+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 >> 172.21.15.117:0/2299703750 conn(0x7f1f400fc640 msgr2=0x7f1f400fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.106+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 shutdown_connections 2024-04-20T11:46:30.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.106+0000 7f1f46c00700 1 -- 172.21.15.117:0/2299703750 wait complete. 2024-04-20T11:46:30.120 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:46:30.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.268+0000 7fe5ce200700 1 -- 172.21.15.117:0/2992230505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 msgr2=0x7fe5c810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.268+0000 7fe5ce200700 1 --2- 172.21.15.117:0/2992230505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 0x7fe5c810ac20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009910 tx=0x7fe5b4009c20 comp rx=0 tx=0).stop 2024-04-20T11:46:30.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.268+0000 7fe5ce200700 1 -- 172.21.15.117:0/2992230505 shutdown_connections 2024-04-20T11:46:30.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.268+0000 7fe5ce200700 1 --2- 172.21.15.117:0/2992230505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 0x7fe5c810ac20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.268+0000 7fe5ce200700 1 --2- 172.21.15.117:0/2992230505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8101e10 0x7fe5c81021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.268+0000 7fe5ce200700 1 -- 172.21.15.117:0/2992230505 >> 172.21.15.117:0/2992230505 conn(0x7fe5c80fb5c0 msgr2=0x7fe5c80fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:30.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.268+0000 7fe5ce200700 1 -- 172.21.15.117:0/2992230505 shutdown_connections 2024-04-20T11:46:30.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.268+0000 7fe5ce200700 1 -- 172.21.15.117:0/2992230505 wait complete. 2024-04-20T11:46:30.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.269+0000 7fe5ce200700 1 Processor -- start 2024-04-20T11:46:30.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.270+0000 7fe5ce200700 1 -- start start 2024-04-20T11:46:30.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.270+0000 7fe5ce200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8101e10 0x7fe5c8199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.270+0000 7fe5ce200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 0x7fe5c8199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.270+0000 7fe5ce200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c8199c20 con 0x7fe5c8102730 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.270+0000 7fe5ce200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c819d8f0 con 0x7fe5c8101e10 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.270+0000 7fe5c6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 0x7fe5c8199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.270+0000 7fe5c6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 0x7fe5c8199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37380/0 (socket says 172.21.15.117:37380) 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.271+0000 7fe5c6a00700 1 -- 172.21.15.117:0/598503806 learned_addr learned my addr 172.21.15.117:0/598503806 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.271+0000 7fe5c7400700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8101e10 0x7fe5c8199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.271+0000 7fe5c6a00700 1 -- 172.21.15.117:0/598503806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8101e10 msgr2=0x7fe5c8199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.271+0000 7fe5c6a00700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8101e10 0x7fe5c8199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.271+0000 7fe5c6a00700 1 -- 172.21.15.117:0/598503806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b40095f0 con 0x7fe5c8102730 2024-04-20T11:46:30.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.271+0000 7fe5c7400700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8101e10 0x7fe5c8199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:46:30.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.271+0000 7fe5c6a00700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 0x7fe5c8199540 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4000c00 tx=0x7fe5b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:30.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.271+0000 7fe5bfe00700 1 -- 172.21.15.117:0/598503806 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b401d070 con 0x7fe5c8102730 2024-04-20T11:46:30.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.272+0000 7fe5bfe00700 1 -- 172.21.15.117:0/598503806 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5b4005700 con 0x7fe5c8102730 2024-04-20T11:46:30.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.272+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c819db70 con 0x7fe5c8102730 2024-04-20T11:46:30.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.272+0000 7fe5bfe00700 1 -- 172.21.15.117:0/598503806 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b400fe50 con 0x7fe5c8102730 2024-04-20T11:46:30.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.272+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c819e060 con 0x7fe5c8102730 2024-04-20T11:46:30.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.273+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c8069660 con 0x7fe5c8102730 2024-04-20T11:46:30.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.274+0000 7fe5bfe00700 1 -- 172.21.15.117:0/598503806 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe5b400f640 con 0x7fe5c8102730 2024-04-20T11:46:30.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.275+0000 7fe5bfe00700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5ac06ecc0 0x7fe5ac071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:30.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.275+0000 7fe5bfe00700 1 -- 172.21.15.117:0/598503806 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe5b40902d0 con 0x7fe5c8102730 2024-04-20T11:46:30.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.276+0000 7fe5c7400700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5ac06ecc0 0x7fe5ac071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:30.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.276+0000 7fe5c7400700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5ac06ecc0 0x7fe5ac071180 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8005fd0 tx=0x7fe5b8005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:30.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.280+0000 7fe5bfe00700 1 -- 172.21.15.117:0/598503806 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe5b4059010 con 0x7fe5c8102730 2024-04-20T11:46:30.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:30 smithi117 ceph-mon[99036]: pgmap v1022: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:30.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.527+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5c81054c0 con 0x7fe5ac06ecc0 2024-04-20T11:46:30.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.529+0000 7fe5bfe00700 1 -- 172.21.15.117:0/598503806 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe5c81054c0 con 0x7fe5ac06ecc0 2024-04-20T11:46:30.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.532+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5ac06ecc0 msgr2=0x7fe5ac071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.532+0000 7fe5ce200700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5ac06ecc0 0x7fe5ac071180 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8005fd0 tx=0x7fe5b8005ee0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.532+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 msgr2=0x7fe5c8199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.532+0000 7fe5ce200700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 0x7fe5c8199540 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4000c00 tx=0x7fe5b400be20 comp rx=0 tx=0).stop 2024-04-20T11:46:30.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.533+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 shutdown_connections 2024-04-20T11:46:30.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.533+0000 7fe5ce200700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe5ac06ecc0 0x7fe5ac071180 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.533+0000 7fe5ce200700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8101e10 0x7fe5c8199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.533+0000 7fe5ce200700 1 --2- 172.21.15.117:0/598503806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c8102730 0x7fe5c8199540 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.533+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 >> 172.21.15.117:0/598503806 conn(0x7fe5c80fb5c0 msgr2=0x7fe5c81033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:30.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.533+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 shutdown_connections 2024-04-20T11:46:30.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.533+0000 7fe5ce200700 1 -- 172.21.15.117:0/598503806 wait complete. 2024-04-20T11:46:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:30 smithi173 ceph-mon[74165]: pgmap v1022: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:30.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.683+0000 7f666e800700 1 -- 172.21.15.117:0/1867000596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668103110 msgr2=0x7f66681034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.683+0000 7f666e800700 1 --2- 172.21.15.117:0/1867000596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668103110 0x7f66681034f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6658009910 tx=0x7f6658009c20 comp rx=0 tx=0).stop 2024-04-20T11:46:30.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.683+0000 7f666e800700 1 -- 172.21.15.117:0/1867000596 shutdown_connections 2024-04-20T11:46:30.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.683+0000 7f666e800700 1 --2- 172.21.15.117:0/1867000596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6668104a00 0x7f6668104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.683+0000 7f666e800700 1 --2- 172.21.15.117:0/1867000596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668103110 0x7f66681034f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.683+0000 7f666e800700 1 -- 172.21.15.117:0/1867000596 >> 172.21.15.117:0/1867000596 conn(0x7f66680fe880 msgr2=0x7f6668100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:30.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.684+0000 7f666e800700 1 -- 172.21.15.117:0/1867000596 shutdown_connections 2024-04-20T11:46:30.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.684+0000 7f666e800700 1 -- 172.21.15.117:0/1867000596 wait complete. 2024-04-20T11:46:30.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.684+0000 7f666e800700 1 Processor -- start 2024-04-20T11:46:30.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.685+0000 7f666e800700 1 -- start start 2024-04-20T11:46:30.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.685+0000 7f666e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6668103110 0x7f6668198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:30.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.685+0000 7f666e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668104a00 0x7f66681993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:30.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.685+0000 7f666e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668199a80 con 0x7f6668104a00 2024-04-20T11:46:30.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.685+0000 7f666e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666819d750 con 0x7f6668103110 2024-04-20T11:46:30.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.685+0000 7f6666a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668104a00 0x7f66681993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:30.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.685+0000 7f6666a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668104a00 0x7f66681993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37394/0 (socket says 172.21.15.117:37394) 2024-04-20T11:46:30.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.685+0000 7f6666a00700 1 -- 172.21.15.117:0/693306175 learned_addr learned my addr 172.21.15.117:0/693306175 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:46:30.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f6667400700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6668103110 0x7f6668198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:30.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f6666a00700 1 -- 172.21.15.117:0/693306175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6668103110 msgr2=0x7f6668198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f6666a00700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6668103110 0x7f6668198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f6666a00700 1 -- 172.21.15.117:0/693306175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66580095f0 con 0x7f6668104a00 2024-04-20T11:46:30.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f6666a00700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668104a00 0x7f66681993a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f665400b4b0 tx=0x7f665400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:30.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f665fe00700 1 -- 172.21.15.117:0/693306175 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6654011800 con 0x7f6668104a00 2024-04-20T11:46:30.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f665fe00700 1 -- 172.21.15.117:0/693306175 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6654011e20 con 0x7f6668104a00 2024-04-20T11:46:30.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666819da30 con 0x7f6668104a00 2024-04-20T11:46:30.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.686+0000 7f665fe00700 1 -- 172.21.15.117:0/693306175 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665400f2e0 con 0x7f6668104a00 2024-04-20T11:46:30.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.687+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666819e000 con 0x7f6668104a00 2024-04-20T11:46:30.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.688+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666810b460 con 0x7f6668104a00 2024-04-20T11:46:30.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.689+0000 7f665fe00700 1 -- 172.21.15.117:0/693306175 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6654010380 con 0x7f6668104a00 2024-04-20T11:46:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.690+0000 7f665fe00700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f664c06ecd0 0x7f664c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.690+0000 7f665fe00700 1 -- 172.21.15.117:0/693306175 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f665408ea00 con 0x7f6668104a00 2024-04-20T11:46:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.690+0000 7f6667400700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f664c06ecd0 0x7f664c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.691+0000 7f6667400700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f664c06ecd0 0x7f664c071190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f665800b5c0 tx=0x7f665801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:30.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.695+0000 7f665fe00700 1 -- 172.21.15.117:0/693306175 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6654057870 con 0x7f6668104a00 2024-04-20T11:46:30.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.928+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f666804ec50 con 0x7f664c06ecd0 2024-04-20T11:46:30.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.940+0000 7f665fe00700 1 -- 172.21.15.117:0/693306175 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f666804ec50 con 0x7f664c06ecd0 2024-04-20T11:46:30.940 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (34m) 10m ago 49m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (49m) 10m ago 49m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (48m) 10m ago 47m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (31m) 10m ago 49m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (31m) 10m ago 47m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (34m) 10m ago 48m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (26m) 10m ago 41m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (27m) 10m ago 41m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (11m) 10m ago 41m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (26m) 10m ago 41m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (38m) 10m ago 50m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (36m) 10m ago 47m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:46:30.941 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (32m) 10m ago 50m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (32m) 10m ago 47m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (36m) 10m ago 49m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (36m) 10m ago 47m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (30m) 10m ago 46m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (30m) 10m ago 46m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (29m) 10m ago 45m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (29m) 10m ago 44m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (28m) 10m ago 44m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (27m) 10m ago 43m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:46:30.942 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (35m) 10m ago 48m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:46:30.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f664c06ecd0 msgr2=0x7f664c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f664c06ecd0 0x7f664c071190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f665800b5c0 tx=0x7f665801a040 comp rx=0 tx=0).stop 2024-04-20T11:46:30.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668104a00 msgr2=0x7f66681993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:30.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668104a00 0x7f66681993a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f665400b4b0 tx=0x7f665400b870 comp rx=0 tx=0).stop 2024-04-20T11:46:30.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 shutdown_connections 2024-04-20T11:46:30.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f664c06ecd0 0x7f664c071190 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6668103110 0x7f6668198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 --2- 172.21.15.117:0/693306175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6668104a00 0x7f66681993a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.943+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 >> 172.21.15.117:0/693306175 conn(0x7f66680fe880 msgr2=0x7f66680ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.944+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 shutdown_connections 2024-04-20T11:46:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:30.944+0000 7f666e800700 1 -- 172.21.15.117:0/693306175 wait complete. 2024-04-20T11:46:31.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.086+0000 7f3934200700 1 -- 172.21.15.117:0/2388040232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 msgr2=0x7f392c103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.086+0000 7f3934200700 1 --2- 172.21.15.117:0/2388040232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 0x7f392c103400 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3920009910 tx=0x7f3920009c20 comp rx=0 tx=0).stop 2024-04-20T11:46:31.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.086+0000 7f3934200700 1 -- 172.21.15.117:0/2388040232 shutdown_connections 2024-04-20T11:46:31.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.086+0000 7f3934200700 1 --2- 172.21.15.117:0/2388040232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f392c1039d0 0x7f392c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.086+0000 7f3934200700 1 --2- 172.21.15.117:0/2388040232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 0x7f392c103400 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.086+0000 7f3934200700 1 -- 172.21.15.117:0/2388040232 >> 172.21.15.117:0/2388040232 conn(0x7f392c0fe8b0 msgr2=0x7f392c100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:31.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.086+0000 7f3934200700 1 -- 172.21.15.117:0/2388040232 shutdown_connections 2024-04-20T11:46:31.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.086+0000 7f3934200700 1 -- 172.21.15.117:0/2388040232 wait complete. 2024-04-20T11:46:31.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.087+0000 7f3934200700 1 Processor -- start 2024-04-20T11:46:31.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.087+0000 7f3934200700 1 -- start start 2024-04-20T11:46:31.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3934200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 0x7f392c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3934200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f392c1039d0 0x7f392c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3934200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c19f530 con 0x7f392c103020 2024-04-20T11:46:31.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3934200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c198b10 con 0x7f392c1039d0 2024-04-20T11:46:31.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3931800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 0x7f392c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3931800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 0x7f392c19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37412/0 (socket says 172.21.15.117:37412) 2024-04-20T11:46:31.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3931800700 1 -- 172.21.15.117:0/3491431846 learned_addr learned my addr 172.21.15.117:0/3491431846 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:46:31.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3931800700 1 -- 172.21.15.117:0/3491431846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f392c1039d0 msgr2=0x7f392c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.088+0000 7f3930e00700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f392c1039d0 0x7f392c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f3931800700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f392c1039d0 0x7f392c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f3931800700 1 -- 172.21.15.117:0/3491431846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200095f0 con 0x7f392c103020 2024-04-20T11:46:31.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f3930e00700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f392c1039d0 0x7f392c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:46:31.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f3931800700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 0x7f392c19e9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f39200092e0 tx=0x7f392000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:31.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f392a000700 1 -- 172.21.15.117:0/3491431846 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392001d070 con 0x7f392c103020 2024-04-20T11:46:31.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f392a000700 1 -- 172.21.15.117:0/3491431846 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f392000f460 con 0x7f392c103020 2024-04-20T11:46:31.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f392a000700 1 -- 172.21.15.117:0/3491431846 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3920017630 con 0x7f392c103020 2024-04-20T11:46:31.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c198d90 con 0x7f392c103020 2024-04-20T11:46:31.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.089+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c199230 con 0x7f392c103020 2024-04-20T11:46:31.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.090+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c10b3c0 con 0x7f392c103020 2024-04-20T11:46:31.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.091+0000 7f392a000700 1 -- 172.21.15.117:0/3491431846 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f392000f5d0 con 0x7f392c103020 2024-04-20T11:46:31.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.092+0000 7f392a000700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f391406ecc0 0x7f3914071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.092+0000 7f392a000700 1 -- 172.21.15.117:0/3491431846 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f39200903e0 con 0x7f392c103020 2024-04-20T11:46:31.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.092+0000 7f3930e00700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f391406ecc0 0x7f3914071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.093+0000 7f3930e00700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f391406ecc0 0x7f3914071180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3918005fd0 tx=0x7f3918005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:31.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.097+0000 7f392a000700 1 -- 172.21.15.117:0/3491431846 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f392005cc50 con 0x7f392c103020 2024-04-20T11:46:31.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.398+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f392c199fd0 con 0x7f392c103020 2024-04-20T11:46:31.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.399+0000 7f392a000700 1 -- 172.21.15.117:0/3491431846 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f392005c7e0 con 0x7f392c103020 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:46:31.400 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:46:31.401 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:46:31.401 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:46:31.401 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:46:31.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f391406ecc0 msgr2=0x7f3914071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f391406ecc0 0x7f3914071180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3918005fd0 tx=0x7f3918005dc0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 msgr2=0x7f392c19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 0x7f392c19e9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f39200092e0 tx=0x7f392000be20 comp rx=0 tx=0).stop 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 shutdown_connections 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f391406ecc0 0x7f3914071180 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c103020 0x7f392c19e9d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 --2- 172.21.15.117:0/3491431846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f392c1039d0 0x7f392c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 >> 172.21.15.117:0/3491431846 conn(0x7f392c0fe8b0 msgr2=0x7f392c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 shutdown_connections 2024-04-20T11:46:31.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.402+0000 7f3934200700 1 -- 172.21.15.117:0/3491431846 wait complete. 2024-04-20T11:46:31.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.560+0000 7efc1a600700 1 -- 172.21.15.117:0/1645456336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc14100e20 msgr2=0x7efc14101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.560+0000 7efc1a600700 1 --2- 172.21.15.117:0/1645456336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc14100e20 0x7efc14101200 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efc04009910 tx=0x7efc04009c20 comp rx=0 tx=0).stop 2024-04-20T11:46:31.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.560+0000 7efc1a600700 1 -- 172.21.15.117:0/1645456336 shutdown_connections 2024-04-20T11:46:31.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.560+0000 7efc1a600700 1 --2- 172.21.15.117:0/1645456336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc141017d0 0x7efc14105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.560+0000 7efc1a600700 1 --2- 172.21.15.117:0/1645456336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc14100e20 0x7efc14101200 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.560+0000 7efc1a600700 1 -- 172.21.15.117:0/1645456336 >> 172.21.15.117:0/1645456336 conn(0x7efc140fc6b0 msgr2=0x7efc140fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:31.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.560+0000 7efc1a600700 1 -- 172.21.15.117:0/1645456336 shutdown_connections 2024-04-20T11:46:31.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.561+0000 7efc1a600700 1 -- 172.21.15.117:0/1645456336 wait complete. 2024-04-20T11:46:31.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.562+0000 7efc1a600700 1 Processor -- start 2024-04-20T11:46:31.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.562+0000 7efc1a600700 1 -- start start 2024-04-20T11:46:31.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.562+0000 7efc1a600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc14100e20 0x7efc14198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.562+0000 7efc1a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc141017d0 0x7efc141992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.562+0000 7efc1a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc14199990 con 0x7efc141017d0 2024-04-20T11:46:31.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.562+0000 7efc1a600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1419d660 con 0x7efc14100e20 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.563+0000 7efc12a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc141017d0 0x7efc141992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.563+0000 7efc12a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc141017d0 0x7efc141992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37426/0 (socket says 172.21.15.117:37426) 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.563+0000 7efc12a00700 1 -- 172.21.15.117:0/784015429 learned_addr learned my addr 172.21.15.117:0/784015429 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.563+0000 7efc13400700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc14100e20 0x7efc14198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.563+0000 7efc12a00700 1 -- 172.21.15.117:0/784015429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc14100e20 msgr2=0x7efc14198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.563+0000 7efc12a00700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc14100e20 0x7efc14198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.563+0000 7efc12a00700 1 -- 172.21.15.117:0/784015429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc040095f0 con 0x7efc141017d0 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.563+0000 7efc12a00700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc141017d0 0x7efc141992b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efc0000d900 tx=0x7efc0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:31.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.564+0000 7efc0be00700 1 -- 172.21.15.117:0/784015429 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc00009900 con 0x7efc141017d0 2024-04-20T11:46:31.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.564+0000 7efc0be00700 1 -- 172.21.15.117:0/784015429 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc00010460 con 0x7efc141017d0 2024-04-20T11:46:31.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.564+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1419d940 con 0x7efc141017d0 2024-04-20T11:46:31.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.564+0000 7efc0be00700 1 -- 172.21.15.117:0/784015429 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0000f5f0 con 0x7efc141017d0 2024-04-20T11:46:31.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.564+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1419de90 con 0x7efc141017d0 2024-04-20T11:46:31.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.565+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc14104fa0 con 0x7efc141017d0 2024-04-20T11:46:31.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.566+0000 7efc0be00700 1 -- 172.21.15.117:0/784015429 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efc000105d0 con 0x7efc141017d0 2024-04-20T11:46:31.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.567+0000 7efc0be00700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efbf806ed10 0x7efbf80711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.567+0000 7efc0be00700 1 -- 172.21.15.117:0/784015429 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efc0008ec70 con 0x7efc141017d0 2024-04-20T11:46:31.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.568+0000 7efc13400700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efbf806ed10 0x7efbf80711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.568+0000 7efc13400700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efbf806ed10 0x7efbf80711d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efc0400b5c0 tx=0x7efc0401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:31.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.572+0000 7efc0be00700 1 -- 172.21.15.117:0/784015429 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efc00095080 con 0x7efc141017d0 2024-04-20T11:46:31.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:31 smithi117 ceph-mon[99036]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:31 smithi173 ceph-mon[74165]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:31.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.831+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efc14102300 con 0x7efc141017d0 2024-04-20T11:46:31.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.833+0000 7efc0be00700 1 -- 172.21.15.117:0/784015429 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7efc00017780 con 0x7efc141017d0 2024-04-20T11:46:31.833 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:46:31.833 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:46:31.833 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:46:31.833 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:46:31.833 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:46:31.833 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:46:31.833 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:46:31.834 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:46:31.835 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:46:31.836 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:46:31.836 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:46:31.836 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:46:31.836 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:46:31.836 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:46:31.836 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:46:31.836 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efbf806ed10 msgr2=0x7efbf80711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efbf806ed10 0x7efbf80711d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efc0400b5c0 tx=0x7efc0401a040 comp rx=0 tx=0).stop 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc141017d0 msgr2=0x7efc141992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc141017d0 0x7efc141992b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efc0000d900 tx=0x7efc0000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 shutdown_connections 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efbf806ed10 0x7efbf80711d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc14100e20 0x7efc14198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 --2- 172.21.15.117:0/784015429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc141017d0 0x7efc141992b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 >> 172.21.15.117:0/784015429 conn(0x7efc140fc6b0 msgr2=0x7efc140fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.835+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 shutdown_connections 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.836+0000 7efc1a600700 1 -- 172.21.15.117:0/784015429 wait complete. 2024-04-20T11:46:31.837 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:46:31.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.979+0000 7fa66dc00700 1 -- 172.21.15.117:0/834882961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668107520 msgr2=0x7fa668107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.979+0000 7fa66dc00700 1 --2- 172.21.15.117:0/834882961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668107520 0x7fa668107900 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa658009910 tx=0x7fa658009c20 comp rx=0 tx=0).stop 2024-04-20T11:46:31.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.979+0000 7fa66dc00700 1 -- 172.21.15.117:0/834882961 shutdown_connections 2024-04-20T11:46:31.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.979+0000 7fa66dc00700 1 --2- 172.21.15.117:0/834882961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa668108e10 0x7fa668109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.979+0000 7fa66dc00700 1 --2- 172.21.15.117:0/834882961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668107520 0x7fa668107900 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.979+0000 7fa66dc00700 1 -- 172.21.15.117:0/834882961 >> 172.21.15.117:0/834882961 conn(0x7fa6680768f0 msgr2=0x7fa668078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:31.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.980+0000 7fa66dc00700 1 -- 172.21.15.117:0/834882961 shutdown_connections 2024-04-20T11:46:31.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.980+0000 7fa66dc00700 1 -- 172.21.15.117:0/834882961 wait complete. 2024-04-20T11:46:31.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.981+0000 7fa66dc00700 1 Processor -- start 2024-04-20T11:46:31.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.981+0000 7fa66dc00700 1 -- start start 2024-04-20T11:46:31.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.982+0000 7fa66dc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa668107520 0x7fa66819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.982+0000 7fa66dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668108e10 0x7fa66819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.982+0000 7fa66dc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66819de90 con 0x7fa668108e10 2024-04-20T11:46:31.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.982+0000 7fa66dc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6681a1b60 con 0x7fa668107520 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.982+0000 7fa666000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668108e10 0x7fa66819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.982+0000 7fa666000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668108e10 0x7fa66819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37440/0 (socket says 172.21.15.117:37440) 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.982+0000 7fa666000700 1 -- 172.21.15.117:0/1972848500 learned_addr learned my addr 172.21.15.117:0/1972848500 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.982+0000 7fa666a00700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa668107520 0x7fa66819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa666000700 1 -- 172.21.15.117:0/1972848500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa668107520 msgr2=0x7fa66819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa666000700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa668107520 0x7fa66819d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa666000700 1 -- 172.21.15.117:0/1972848500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6580095f0 con 0x7fa668108e10 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa666000700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668108e10 0x7fa66819d7b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa65400b4b0 tx=0x7fa65400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa65f400700 1 -- 172.21.15.117:0/1972848500 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa654011800 con 0x7fa668108e10 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa65f400700 1 -- 172.21.15.117:0/1972848500 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa654011e20 con 0x7fa668108e10 2024-04-20T11:46:31.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6681a1e40 con 0x7fa668108e10 2024-04-20T11:46:31.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa65f400700 1 -- 172.21.15.117:0/1972848500 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa65400f2e0 con 0x7fa668108e10 2024-04-20T11:46:31.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.983+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6681a2410 con 0x7fa668108e10 2024-04-20T11:46:31.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.984+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66810f870 con 0x7fa668108e10 2024-04-20T11:46:31.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.986+0000 7fa65f400700 1 -- 172.21.15.117:0/1972848500 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa654010380 con 0x7fa668108e10 2024-04-20T11:46:31.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.987+0000 7fa65f400700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64c06ecd0 0x7fa64c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:31.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.987+0000 7fa65f400700 1 -- 172.21.15.117:0/1972848500 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa65408ea70 con 0x7fa668108e10 2024-04-20T11:46:31.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.987+0000 7fa666a00700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64c06ecd0 0x7fa64c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:31.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.988+0000 7fa666a00700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64c06ecd0 0x7fa64c071190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa658006010 tx=0x7fa65800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:31.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:31.992+0000 7fa65f400700 1 -- 172.21.15.117:0/1972848500 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa6540577b0 con 0x7fa668108e10 2024-04-20T11:46:32.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.238+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa668114540 con 0x7fa64c06ecd0 2024-04-20T11:46:32.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.240+0000 7fa65f400700 1 -- 172.21.15.117:0/1972848500 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa668114540 con 0x7fa64c06ecd0 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:46:32.240 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:46:32.241 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:46:32.241 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:46:32.241 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:46:32.241 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:46:32.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64c06ecd0 msgr2=0x7fa64c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:32.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64c06ecd0 0x7fa64c071190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa658006010 tx=0x7fa65800b540 comp rx=0 tx=0).stop 2024-04-20T11:46:32.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668108e10 msgr2=0x7fa66819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:32.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668108e10 0x7fa66819d7b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa65400b4b0 tx=0x7fa65400b870 comp rx=0 tx=0).stop 2024-04-20T11:46:32.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 shutdown_connections 2024-04-20T11:46:32.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa64c06ecd0 0x7fa64c071190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa668107520 0x7fa66819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 --2- 172.21.15.117:0/1972848500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa668108e10 0x7fa66819d7b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 >> 172.21.15.117:0/1972848500 conn(0x7fa6680768f0 msgr2=0x7fa668078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:32.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.243+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 shutdown_connections 2024-04-20T11:46:32.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.244+0000 7fa66dc00700 1 -- 172.21.15.117:0/1972848500 wait complete. 2024-04-20T11:46:32.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.399+0000 7fb087c00700 1 -- 172.21.15.117:0/2334773690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080108e10 msgr2=0x7fb080109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:32.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.399+0000 7fb087c00700 1 --2- 172.21.15.117:0/2334773690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080108e10 0x7fb080109290 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009910 tx=0x7fb06c009c20 comp rx=0 tx=0).stop 2024-04-20T11:46:32.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.399+0000 7fb087c00700 1 -- 172.21.15.117:0/2334773690 shutdown_connections 2024-04-20T11:46:32.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.399+0000 7fb087c00700 1 --2- 172.21.15.117:0/2334773690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080108e10 0x7fb080109290 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.399+0000 7fb087c00700 1 --2- 172.21.15.117:0/2334773690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb080107520 0x7fb080107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.399+0000 7fb087c00700 1 -- 172.21.15.117:0/2334773690 >> 172.21.15.117:0/2334773690 conn(0x7fb0800768f0 msgr2=0x7fb080078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:32.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.399+0000 7fb087c00700 1 -- 172.21.15.117:0/2334773690 shutdown_connections 2024-04-20T11:46:32.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.399+0000 7fb087c00700 1 -- 172.21.15.117:0/2334773690 wait complete. 2024-04-20T11:46:32.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.401+0000 7fb087c00700 1 Processor -- start 2024-04-20T11:46:32.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.401+0000 7fb087c00700 1 -- start start 2024-04-20T11:46:32.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.401+0000 7fb087c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080107520 0x7fb08019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.401+0000 7fb087c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb080108e10 0x7fb08019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.401+0000 7fb087c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08019ddf0 con 0x7fb080107520 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.401+0000 7fb087c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0801a1a70 con 0x7fb080108e10 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.402+0000 7fb085200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080107520 0x7fb08019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.402+0000 7fb085200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080107520 0x7fb08019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37448/0 (socket says 172.21.15.117:37448) 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.402+0000 7fb085200700 1 -- 172.21.15.117:0/4285731769 learned_addr learned my addr 172.21.15.117:0/4285731769 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.402+0000 7fb07fe00700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb080108e10 0x7fb08019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.402+0000 7fb085200700 1 -- 172.21.15.117:0/4285731769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb080108e10 msgr2=0x7fb08019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.402+0000 7fb085200700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb080108e10 0x7fb08019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.402+0000 7fb085200700 1 -- 172.21.15.117:0/4285731769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c0095f0 con 0x7fb080107520 2024-04-20T11:46:32.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.403+0000 7fb085200700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080107520 0x7fb08019d1d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb07400d8d0 tx=0x7fb07400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:32.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.403+0000 7fb07d600700 1 -- 172.21.15.117:0/4285731769 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb074009940 con 0x7fb080107520 2024-04-20T11:46:32.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.403+0000 7fb07d600700 1 -- 172.21.15.117:0/4285731769 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb074010460 con 0x7fb080107520 2024-04-20T11:46:32.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.403+0000 7fb07d600700 1 -- 172.21.15.117:0/4285731769 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07400f5f0 con 0x7fb080107520 2024-04-20T11:46:32.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.403+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0801a1d50 con 0x7fb080107520 2024-04-20T11:46:32.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.403+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0801a2320 con 0x7fb080107520 2024-04-20T11:46:32.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.404+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08010f870 con 0x7fb080107520 2024-04-20T11:46:32.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.405+0000 7fb07d600700 1 -- 172.21.15.117:0/4285731769 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb0740105d0 con 0x7fb080107520 2024-04-20T11:46:32.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.406+0000 7fb07d600700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb06806ecc0 0x7fb068071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:46:32.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.406+0000 7fb07d600700 1 -- 172.21.15.117:0/4285731769 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb07408ede0 con 0x7fb080107520 2024-04-20T11:46:32.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.406+0000 7fb07fe00700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb06806ecc0 0x7fb068071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:46:32.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.407+0000 7fb07fe00700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb06806ecc0 0x7fb068071180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005b40 tx=0x7fb06c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:46:32.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.411+0000 7fb07d600700 1 -- 172.21.15.117:0/4285731769 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb074057ba0 con 0x7fb080107520 2024-04-20T11:46:32.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:32 smithi117 ceph-mon[99036]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:32 smithi117 ceph-mon[99036]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:32 smithi117 ceph-mon[99036]: pgmap v1023: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:32 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3491431846' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:46:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:32 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/784015429' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:46:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:32 smithi173 ceph-mon[74165]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:32 smithi173 ceph-mon[74165]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:32 smithi173 ceph-mon[74165]: pgmap v1023: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:32 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3491431846' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:46:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:32 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/784015429' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:46:32.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.726+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb080108920 con 0x7fb080107520 2024-04-20T11:46:32.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.726+0000 7fb07d600700 1 -- 172.21.15.117:0/4285731769 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fb07405b260 con 0x7fb080107520 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:46:32.727 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:46:32.728 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:46:32.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb06806ecc0 msgr2=0x7fb068071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:32.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb06806ecc0 0x7fb068071180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005b40 tx=0x7fb06c01a040 comp rx=0 tx=0).stop 2024-04-20T11:46:32.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080107520 msgr2=0x7fb08019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:46:32.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080107520 0x7fb08019d1d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb07400d8d0 tx=0x7fb07400dc90 comp rx=0 tx=0).stop 2024-04-20T11:46:32.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 shutdown_connections 2024-04-20T11:46:32.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb06806ecc0 0x7fb068071180 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb080107520 0x7fb08019d1d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 --2- 172.21.15.117:0/4285731769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb080108e10 0x7fb08019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:46:32.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.729+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 >> 172.21.15.117:0/4285731769 conn(0x7fb0800768f0 msgr2=0x7fb080077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:46:32.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.730+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 shutdown_connections 2024-04-20T11:46:32.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:46:32.730+0000 7fb087c00700 1 -- 172.21.15.117:0/4285731769 wait complete. 2024-04-20T11:46:33.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:33 smithi117 ceph-mon[99036]: from='client.35842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:33.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:33 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4285731769' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:46:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:33 smithi173 ceph-mon[74165]: from='client.35842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:46:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:33 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4285731769' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:46:34.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:34 smithi117 ceph-mon[99036]: pgmap v1024: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:34 smithi173 ceph-mon[74165]: pgmap v1024: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:36 smithi117 ceph-mon[99036]: pgmap v1025: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:36 smithi173 ceph-mon[74165]: pgmap v1025: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:38 smithi117 ceph-mon[99036]: pgmap v1026: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:38 smithi173 ceph-mon[74165]: pgmap v1026: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:40.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:40 smithi117 ceph-mon[99036]: pgmap v1027: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:40 smithi173 ceph-mon[74165]: pgmap v1027: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:42 smithi117 ceph-mon[99036]: pgmap v1028: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:42 smithi173 ceph-mon[74165]: pgmap v1028: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:46:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:46:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:44 smithi173 ceph-mon[74165]: pgmap v1029: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:44 smithi117 ceph-mon[99036]: pgmap v1029: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:46.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:46 smithi173 ceph-mon[74165]: pgmap v1030: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:46.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:46 smithi117 ceph-mon[99036]: pgmap v1030: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:48.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:48 smithi117 ceph-mon[99036]: pgmap v1031: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:48 smithi173 ceph-mon[74165]: pgmap v1031: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:50.692 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:50 smithi117 ceph-mon[99036]: pgmap v1032: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:50 smithi173 ceph-mon[74165]: pgmap v1032: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:52 smithi173 ceph-mon[74165]: pgmap v1033: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:52.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:52 smithi117 ceph-mon[99036]: pgmap v1033: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:54.901 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:54 smithi117 ceph-mon[99036]: pgmap v1034: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:54 smithi173 ceph-mon[74165]: pgmap v1034: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:56.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:56 smithi117 ceph-mon[99036]: pgmap v1035: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:56.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:56 smithi173 ceph-mon[74165]: pgmap v1035: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:58 smithi117 ceph-mon[99036]: pgmap v1036: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:46:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:46:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:58 smithi173 ceph-mon[74165]: pgmap v1036: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:46:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:46:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:47:00.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:00 smithi117 ceph-mon[99036]: pgmap v1037: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:00 smithi173 ceph-mon[74165]: pgmap v1037: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:02.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:02 smithi117 ceph-mon[99036]: pgmap v1038: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:02.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.899+0000 7fa97f600700 1 -- 172.21.15.117:0/716564408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 msgr2=0x7fa978101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:02.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.899+0000 7fa97f600700 1 --2- 172.21.15.117:0/716564408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 0x7fa978101200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009910 tx=0x7fa96c009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:02.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.899+0000 7fa97f600700 1 -- 172.21.15.117:0/716564408 shutdown_connections 2024-04-20T11:47:02.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.899+0000 7fa97f600700 1 --2- 172.21.15.117:0/716564408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9781017d0 0x7fa978105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:02.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.899+0000 7fa97f600700 1 --2- 172.21.15.117:0/716564408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 0x7fa978101200 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:02.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.900+0000 7fa97f600700 1 -- 172.21.15.117:0/716564408 >> 172.21.15.117:0/716564408 conn(0x7fa9780fc6b0 msgr2=0x7fa9780fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:02.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.900+0000 7fa97f600700 1 -- 172.21.15.117:0/716564408 shutdown_connections 2024-04-20T11:47:02.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.900+0000 7fa97f600700 1 -- 172.21.15.117:0/716564408 wait complete. 2024-04-20T11:47:02.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.901+0000 7fa97f600700 1 Processor -- start 2024-04-20T11:47:02.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.901+0000 7fa97f600700 1 -- start start 2024-04-20T11:47:02.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.901+0000 7fa97f600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 0x7fa97819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:02.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.901+0000 7fa97f600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9781017d0 0x7fa97819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:02.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.901+0000 7fa97f600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97819d390 con 0x7fa978100e20 2024-04-20T11:47:02.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.901+0000 7fa97f600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa978196970 con 0x7fa9781017d0 2024-04-20T11:47:02.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa97cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 0x7fa97819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:02.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa97cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 0x7fa97819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58990/0 (socket says 172.21.15.117:58990) 2024-04-20T11:47:02.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa97cc00700 1 -- 172.21.15.117:0/3489076653 learned_addr learned my addr 172.21.15.117:0/3489076653 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa977e00700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9781017d0 0x7fa97819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa97cc00700 1 -- 172.21.15.117:0/3489076653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9781017d0 msgr2=0x7fa97819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa97cc00700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9781017d0 0x7fa97819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa97cc00700 1 -- 172.21.15.117:0/3489076653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0095f0 con 0x7fa978100e20 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa977e00700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9781017d0 0x7fa97819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.902+0000 7fa97cc00700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 0x7fa97819c830 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0092e0 tx=0x7fa96c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.903+0000 7fa975600700 1 -- 172.21.15.117:0/3489076653 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa96c01d070 con 0x7fa978100e20 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.903+0000 7fa975600700 1 -- 172.21.15.117:0/3489076653 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa96c00f460 con 0x7fa978100e20 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.903+0000 7fa975600700 1 -- 172.21.15.117:0/3489076653 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa96c017630 con 0x7fa978100e20 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.903+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa978196bf0 con 0x7fa978100e20 2024-04-20T11:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.903+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9781970e0 con 0x7fa978100e20 2024-04-20T11:47:02.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.904+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9781091c0 con 0x7fa978100e20 2024-04-20T11:47:02.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.905+0000 7fa975600700 1 -- 172.21.15.117:0/3489076653 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa96c00f5d0 con 0x7fa978100e20 2024-04-20T11:47:02.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.906+0000 7fa975600700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa96006ecc0 0x7fa960071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:02.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.906+0000 7fa975600700 1 -- 172.21.15.117:0/3489076653 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa96c090320 con 0x7fa978100e20 2024-04-20T11:47:02.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.906+0000 7fa977e00700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa96006ecc0 0x7fa960071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:02.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.907+0000 7fa977e00700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa96006ecc0 0x7fa960071180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa968005fd0 tx=0x7fa968005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:02.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:02 smithi173 ceph-mon[74165]: pgmap v1038: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:02.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:02.912+0000 7fa975600700 1 -- 172.21.15.117:0/3489076653 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa96c05cc10 con 0x7fa978100e20 2024-04-20T11:47:03.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.153+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa978197e30 con 0x7fa96006ecc0 2024-04-20T11:47:03.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.156+0000 7fa975600700 1 -- 172.21.15.117:0/3489076653 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa978197e30 con 0x7fa96006ecc0 2024-04-20T11:47:03.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.158+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa96006ecc0 msgr2=0x7fa960071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:03.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.158+0000 7fa97f600700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa96006ecc0 0x7fa960071180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa968005fd0 tx=0x7fa968005dc0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.158+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 msgr2=0x7fa97819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:03.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.158+0000 7fa97f600700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 0x7fa97819c830 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0092e0 tx=0x7fa96c00be20 comp rx=0 tx=0).stop 2024-04-20T11:47:03.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.159+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 shutdown_connections 2024-04-20T11:47:03.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.159+0000 7fa97f600700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa96006ecc0 0x7fa960071180 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.159+0000 7fa97f600700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978100e20 0x7fa97819c830 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.159+0000 7fa97f600700 1 --2- 172.21.15.117:0/3489076653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9781017d0 0x7fa97819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.159+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 >> 172.21.15.117:0/3489076653 conn(0x7fa9780fc6b0 msgr2=0x7fa9780fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:03.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.159+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 shutdown_connections 2024-04-20T11:47:03.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.159+0000 7fa97f600700 1 -- 172.21.15.117:0/3489076653 wait complete. 2024-04-20T11:47:03.175 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:47:03.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.316+0000 7f11a2000700 1 -- 172.21.15.117:0/2504203475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 msgr2=0x7f119c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:03.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.316+0000 7f11a2000700 1 --2- 172.21.15.117:0/2504203475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 0x7f119c0fff00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1184009910 tx=0x7f1184009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:03.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.316+0000 7f11a2000700 1 -- 172.21.15.117:0/2504203475 shutdown_connections 2024-04-20T11:47:03.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.316+0000 7f11a2000700 1 --2- 172.21.15.117:0/2504203475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 0x7f119c0fff00 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.316+0000 7f11a2000700 1 --2- 172.21.15.117:0/2504203475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c105a80 0x7f119c105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.316+0000 7f11a2000700 1 -- 172.21.15.117:0/2504203475 >> 172.21.15.117:0/2504203475 conn(0x7f119c076750 msgr2=0x7f119c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:03.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.316+0000 7f11a2000700 1 -- 172.21.15.117:0/2504203475 shutdown_connections 2024-04-20T11:47:03.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.316+0000 7f11a2000700 1 -- 172.21.15.117:0/2504203475 wait complete. 2024-04-20T11:47:03.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.317+0000 7f11a2000700 1 Processor -- start 2024-04-20T11:47:03.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.318+0000 7f11a2000700 1 -- start start 2024-04-20T11:47:03.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.318+0000 7f11a2000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 0x7f119c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:03.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.318+0000 7f11a2000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c105a80 0x7f119c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:03.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.318+0000 7f11a2000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19d3e0 con 0x7f119c0ffa80 2024-04-20T11:47:03.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.318+0000 7f11a2000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c1969c0 con 0x7f119c105a80 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.318+0000 7f119b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 0x7f119c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.318+0000 7f119b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 0x7f119c19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59010/0 (socket says 172.21.15.117:59010) 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.318+0000 7f119b400700 1 -- 172.21.15.117:0/2272222799 learned_addr learned my addr 172.21.15.117:0/2272222799 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.319+0000 7f1192a00700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c105a80 0x7f119c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.319+0000 7f119b400700 1 -- 172.21.15.117:0/2272222799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c105a80 msgr2=0x7f119c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.319+0000 7f119b400700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c105a80 0x7f119c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.319+0000 7f119b400700 1 -- 172.21.15.117:0/2272222799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840095f0 con 0x7f119c0ffa80 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.319+0000 7f119b400700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 0x7f119c19c880 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c8a0 tx=0x7f118c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:03.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.319+0000 7f1198c00700 1 -- 172.21.15.117:0/2272222799 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118c00cea0 con 0x7f119c0ffa80 2024-04-20T11:47:03.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.319+0000 7f1198c00700 1 -- 172.21.15.117:0/2272222799 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f118c004830 con 0x7f119c0ffa80 2024-04-20T11:47:03.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.319+0000 7f1198c00700 1 -- 172.21.15.117:0/2272222799 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118c0055b0 con 0x7f119c0ffa80 2024-04-20T11:47:03.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.320+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c196ca0 con 0x7f119c0ffa80 2024-04-20T11:47:03.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.320+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c1971f0 con 0x7f119c0ffa80 2024-04-20T11:47:03.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.321+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c1090b0 con 0x7f119c0ffa80 2024-04-20T11:47:03.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.322+0000 7f1198c00700 1 -- 172.21.15.117:0/2272222799 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f118c0049a0 con 0x7f119c0ffa80 2024-04-20T11:47:03.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.323+0000 7f1198c00700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f118006ecc0 0x7f1180071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:03.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.323+0000 7f1198c00700 1 -- 172.21.15.117:0/2272222799 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f118c08ea20 con 0x7f119c0ffa80 2024-04-20T11:47:03.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.323+0000 7f1192a00700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f118006ecc0 0x7f1180071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:03.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.324+0000 7f1192a00700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f118006ecc0 0x7f1180071180 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1184006010 tx=0x7f118400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:03.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.328+0000 7f1198c00700 1 -- 172.21.15.117:0/2272222799 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f118c05b3b0 con 0x7f119c0ffa80 2024-04-20T11:47:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.574+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f119c10cc80 con 0x7f118006ecc0 2024-04-20T11:47:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.577+0000 7f1198c00700 1 -- 172.21.15.117:0/2272222799 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f119c10cc80 con 0x7f118006ecc0 2024-04-20T11:47:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f118006ecc0 msgr2=0x7f1180071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:03.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f118006ecc0 0x7f1180071180 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1184006010 tx=0x7f118400b540 comp rx=0 tx=0).stop 2024-04-20T11:47:03.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 msgr2=0x7f119c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:03.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 0x7f119c19c880 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c8a0 tx=0x7f118c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:47:03.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 shutdown_connections 2024-04-20T11:47:03.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f118006ecc0 0x7f1180071180 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f119c0ffa80 0x7f119c19c880 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 --2- 172.21.15.117:0/2272222799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c105a80 0x7f119c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.580+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 >> 172.21.15.117:0/2272222799 conn(0x7f119c076750 msgr2=0x7f119c068b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:03.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.581+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 shutdown_connections 2024-04-20T11:47:03.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.581+0000 7f11a2000700 1 -- 172.21.15.117:0/2272222799 wait complete. 2024-04-20T11:47:03.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.734+0000 7f8b3e600700 1 -- 172.21.15.117:0/3786041436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 msgr2=0x7f8b38102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:03.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.734+0000 7f8b3e600700 1 --2- 172.21.15.117:0/3786041436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 0x7f8b38102120 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009910 tx=0x7f8b28009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:03.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.734+0000 7f8b3e600700 1 -- 172.21.15.117:0/3786041436 shutdown_connections 2024-04-20T11:47:03.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.734+0000 7f8b3e600700 1 --2- 172.21.15.117:0/3786041436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b38102660 0x7f8b3810ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.734+0000 7f8b3e600700 1 --2- 172.21.15.117:0/3786041436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 0x7f8b38102120 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.734+0000 7f8b3e600700 1 -- 172.21.15.117:0/3786041436 >> 172.21.15.117:0/3786041436 conn(0x7f8b38076750 msgr2=0x7f8b380fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:03.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.734+0000 7f8b3e600700 1 -- 172.21.15.117:0/3786041436 shutdown_connections 2024-04-20T11:47:03.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.735+0000 7f8b3e600700 1 -- 172.21.15.117:0/3786041436 wait complete. 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b3e600700 1 Processor -- start 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b3e600700 1 -- start start 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b3e600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 0x7f8b38198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b3e600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b38102660 0x7f8b381992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b3e600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38199930 con 0x7f8b38101d40 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b3e600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819c550 con 0x7f8b38102660 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b37400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 0x7f8b38198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b37400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 0x7f8b38198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59028/0 (socket says 172.21.15.117:59028) 2024-04-20T11:47:03.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.736+0000 7f8b37400700 1 -- 172.21.15.117:0/413423315 learned_addr learned my addr 172.21.15.117:0/413423315 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.737+0000 7f8b2ea00700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b38102660 0x7f8b381992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.737+0000 7f8b37400700 1 -- 172.21.15.117:0/413423315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b38102660 msgr2=0x7f8b381992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.737+0000 7f8b37400700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b38102660 0x7f8b381992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.737+0000 7f8b37400700 1 -- 172.21.15.117:0/413423315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280095f0 con 0x7f8b38101d40 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.737+0000 7f8b2ea00700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b38102660 0x7f8b381992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.737+0000 7f8b37400700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 0x7f8b38198d60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8b280092e0 tx=0x7f8b2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.737+0000 7f8b34c00700 1 -- 172.21.15.117:0/413423315 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b2801d070 con 0x7f8b38101d40 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.737+0000 7f8b34c00700 1 -- 172.21.15.117:0/413423315 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b280056e0 con 0x7f8b38101d40 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.738+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3819c7d0 con 0x7f8b38101d40 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.738+0000 7f8b34c00700 1 -- 172.21.15.117:0/413423315 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b2800fe80 con 0x7f8b38101d40 2024-04-20T11:47:03.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.738+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3819ccc0 con 0x7f8b38101d40 2024-04-20T11:47:03.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.739+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b381096b0 con 0x7f8b38101d40 2024-04-20T11:47:03.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.740+0000 7f8b34c00700 1 -- 172.21.15.117:0/413423315 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8b2800f6f0 con 0x7f8b38101d40 2024-04-20T11:47:03.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.741+0000 7f8b34c00700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1c06ecc0 0x7f8b1c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:03.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.741+0000 7f8b34c00700 1 -- 172.21.15.117:0/413423315 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8b28090340 con 0x7f8b38101d40 2024-04-20T11:47:03.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.741+0000 7f8b2ea00700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1c06ecc0 0x7f8b1c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:03.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.742+0000 7f8b2ea00700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1c06ecc0 0x7f8b1c071180 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8b3819a230 tx=0x7f8b20009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:03.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.745+0000 7f8b34c00700 1 -- 172.21.15.117:0/413423315 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8b28059080 con 0x7f8b38101d40 2024-04-20T11:47:03.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:03.981+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b3810ddc0 con 0x7f8b1c06ecc0 2024-04-20T11:47:04.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.011+0000 7f8b34c00700 1 -- 172.21.15.117:0/413423315 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8b3810ddc0 con 0x7f8b1c06ecc0 2024-04-20T11:47:04.012 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (35m) 11m ago 49m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (49m) 11m ago 49m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (48m) 11m ago 48m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (31m) 11m ago 49m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (31m) 11m ago 48m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (34m) 11m ago 49m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (27m) 11m ago 42m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (27m) 11m ago 42m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (11m) 11m ago 42m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (26m) 11m ago 42m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (38m) 11m ago 51m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (37m) 11m ago 48m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:47:04.013 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (33m) 11m ago 51m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (32m) 11m ago 48m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (36m) 11m ago 49m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (36m) 11m ago 48m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (31m) 11m ago 47m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (30m) 11m ago 46m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (30m) 11m ago 45m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (29m) 11m ago 45m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (28m) 11m ago 44m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (28m) 11m ago 44m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:47:04.014 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (36m) 11m ago 49m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:47:04.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.014+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1c06ecc0 msgr2=0x7f8b1c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1c06ecc0 0x7f8b1c071180 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8b3819a230 tx=0x7f8b20009040 comp rx=0 tx=0).stop 2024-04-20T11:47:04.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 msgr2=0x7f8b38198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 0x7f8b38198d60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8b280092e0 tx=0x7f8b2800be20 comp rx=0 tx=0).stop 2024-04-20T11:47:04.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 shutdown_connections 2024-04-20T11:47:04.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8b1c06ecc0 0x7f8b1c071180 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b38101d40 0x7f8b38198d60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 --2- 172.21.15.117:0/413423315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b38102660 0x7f8b381992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 >> 172.21.15.117:0/413423315 conn(0x7f8b38076750 msgr2=0x7f8b3810a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:04.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 shutdown_connections 2024-04-20T11:47:04.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.015+0000 7f8b3e600700 1 -- 172.21.15.117:0/413423315 wait complete. 2024-04-20T11:47:04.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.161+0000 7fe7e5600700 1 -- 172.21.15.117:0/707687938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 msgr2=0x7fe7e0107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.161+0000 7fe7e5600700 1 --2- 172.21.15.117:0/707687938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 0x7fe7e0107510 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009910 tx=0x7fe7cc009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:04.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.161+0000 7fe7e5600700 1 -- 172.21.15.117:0/707687938 shutdown_connections 2024-04-20T11:47:04.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.161+0000 7fe7e5600700 1 --2- 172.21.15.117:0/707687938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 0x7fe7e0107510 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.161+0000 7fe7e5600700 1 --2- 172.21.15.117:0/707687938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7e010d0c0 0x7fe7e010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.161+0000 7fe7e5600700 1 -- 172.21.15.117:0/707687938 >> 172.21.15.117:0/707687938 conn(0x7fe7e0076830 msgr2=0x7fe7e0078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:04.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.161+0000 7fe7e5600700 1 -- 172.21.15.117:0/707687938 shutdown_connections 2024-04-20T11:47:04.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.161+0000 7fe7e5600700 1 -- 172.21.15.117:0/707687938 wait complete. 2024-04-20T11:47:04.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.162+0000 7fe7e5600700 1 Processor -- start 2024-04-20T11:47:04.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.162+0000 7fe7e5600700 1 -- start start 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7e5600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 0x7fe7e019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7e5600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7e010d0c0 0x7fe7e019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7e5600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e019dab0 con 0x7fe7e01070b0 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7e5600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e019dbf0 con 0x7fe7e010d0c0 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7dfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 0x7fe7e019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7dfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 0x7fe7e019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43526/0 (socket says 172.21.15.117:43526) 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7dfe00700 1 -- 172.21.15.117:0/1396773575 learned_addr learned my addr 172.21.15.117:0/1396773575 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7dfe00700 1 -- 172.21.15.117:0/1396773575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7e010d0c0 msgr2=0x7fe7e019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7dfe00700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7e010d0c0 0x7fe7e019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7dfe00700 1 -- 172.21.15.117:0/1396773575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc0095f0 con 0x7fe7e01070b0 2024-04-20T11:47:04.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.163+0000 7fe7dfe00700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 0x7fe7e019cf20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c930 tx=0x7fe7d000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:04.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.164+0000 7fe7dcc00700 1 -- 172.21.15.117:0/1396773575 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7d0007ad0 con 0x7fe7e01070b0 2024-04-20T11:47:04.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.164+0000 7fe7dcc00700 1 -- 172.21.15.117:0/1396773575 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7d0007c30 con 0x7fe7e01070b0 2024-04-20T11:47:04.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.164+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e01a1980 con 0x7fe7e01070b0 2024-04-20T11:47:04.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.164+0000 7fe7dcc00700 1 -- 172.21.15.117:0/1396773575 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7d0018610 con 0x7fe7e01070b0 2024-04-20T11:47:04.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.164+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e01a1ed0 con 0x7fe7e01070b0 2024-04-20T11:47:04.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.165+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e010f5c0 con 0x7fe7e01070b0 2024-04-20T11:47:04.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.166+0000 7fe7dcc00700 1 -- 172.21.15.117:0/1396773575 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe7d000f450 con 0x7fe7e01070b0 2024-04-20T11:47:04.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.167+0000 7fe7dcc00700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7c406ed20 0x7fe7c40711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:04.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.167+0000 7fe7dcc00700 1 -- 172.21.15.117:0/1396773575 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe7d008efe0 con 0x7fe7e01070b0 2024-04-20T11:47:04.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.168+0000 7fe7df400700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7c406ed20 0x7fe7c40711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:04.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.168+0000 7fe7df400700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7c406ed20 0x7fe7c40711e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0098e0 tx=0x7fe7cc005c20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:04.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.173+0000 7fe7dcc00700 1 -- 172.21.15.117:0/1396773575 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe7d0057de0 con 0x7fe7e01070b0 2024-04-20T11:47:04.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.482+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe7e0114210 con 0x7fe7e01070b0 2024-04-20T11:47:04.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.482+0000 7fe7dcc00700 1 -- 172.21.15.117:0/1396773575 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe7d005b4a0 con 0x7fe7e01070b0 2024-04-20T11:47:04.483 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:47:04.484 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:47:04.485 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:47:04.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7c406ed20 msgr2=0x7fe7c40711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7c406ed20 0x7fe7c40711e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0098e0 tx=0x7fe7cc005c20 comp rx=0 tx=0).stop 2024-04-20T11:47:04.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 msgr2=0x7fe7e019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 0x7fe7e019cf20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c930 tx=0x7fe7d000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 shutdown_connections 2024-04-20T11:47:04.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe7c406ed20 0x7fe7c40711e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e01070b0 0x7fe7e019cf20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 --2- 172.21.15.117:0/1396773575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7e010d0c0 0x7fe7e019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 >> 172.21.15.117:0/1396773575 conn(0x7fe7e0076830 msgr2=0x7fe7e0078a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:04.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 shutdown_connections 2024-04-20T11:47:04.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.485+0000 7fe7e5600700 1 -- 172.21.15.117:0/1396773575 wait complete. 2024-04-20T11:47:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.629+0000 7fb201800700 1 -- 172.21.15.117:0/3962119577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 msgr2=0x7fb1fc1114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.629+0000 7fb201800700 1 --2- 172.21.15.117:0/3962119577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 0x7fb1fc1114a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009910 tx=0x7fb1e4009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.629+0000 7fb201800700 1 -- 172.21.15.117:0/3962119577 shutdown_connections 2024-04-20T11:47:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.629+0000 7fb201800700 1 --2- 172.21.15.117:0/3962119577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 0x7fb1fc1114a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.629+0000 7fb201800700 1 --2- 172.21.15.117:0/3962119577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1fc073280 0x7fb1fc073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.629+0000 7fb201800700 1 -- 172.21.15.117:0/3962119577 >> 172.21.15.117:0/3962119577 conn(0x7fb1fc0fc500 msgr2=0x7fb1fc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.629+0000 7fb201800700 1 -- 172.21.15.117:0/3962119577 shutdown_connections 2024-04-20T11:47:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.629+0000 7fb201800700 1 -- 172.21.15.117:0/3962119577 wait complete. 2024-04-20T11:47:04.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.630+0000 7fb201800700 1 Processor -- start 2024-04-20T11:47:04.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.631+0000 7fb201800700 1 -- start start 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.631+0000 7fb201800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1fc073280 0x7fb1fc19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.631+0000 7fb201800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 0x7fb1fc19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.631+0000 7fb201800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc19dc20 con 0x7fb1fc073ba0 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.631+0000 7fb201800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc1a18f0 con 0x7fb1fc073280 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.631+0000 7fb1f3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 0x7fb1fc19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.631+0000 7fb1f3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 0x7fb1fc19d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43546/0 (socket says 172.21.15.117:43546) 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.631+0000 7fb1f3400700 1 -- 172.21.15.117:0/472991723 learned_addr learned my addr 172.21.15.117:0/472991723 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.632+0000 7fb1fbe00700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1fc073280 0x7fb1fc19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.632+0000 7fb1f3400700 1 -- 172.21.15.117:0/472991723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1fc073280 msgr2=0x7fb1fc19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.632+0000 7fb1f3400700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1fc073280 0x7fb1fc19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.632+0000 7fb1f3400700 1 -- 172.21.15.117:0/472991723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40095f0 con 0x7fb1fc073ba0 2024-04-20T11:47:04.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.632+0000 7fb1f3400700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 0x7fb1fc19d540 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006010 tx=0x7fb1e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:04.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.633+0000 7fb1f9600700 1 -- 172.21.15.117:0/472991723 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1e401d070 con 0x7fb1fc073ba0 2024-04-20T11:47:04.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.633+0000 7fb1f9600700 1 -- 172.21.15.117:0/472991723 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1e400f460 con 0x7fb1fc073ba0 2024-04-20T11:47:04.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.633+0000 7fb1f9600700 1 -- 172.21.15.117:0/472991723 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1e4017630 con 0x7fb1fc073ba0 2024-04-20T11:47:04.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.633+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc1a1bd0 con 0x7fb1fc073ba0 2024-04-20T11:47:04.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.633+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc1a20d0 con 0x7fb1fc073ba0 2024-04-20T11:47:04.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.634+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc0749c0 con 0x7fb1fc073ba0 2024-04-20T11:47:04.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.635+0000 7fb1f9600700 1 -- 172.21.15.117:0/472991723 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb1e400f5d0 con 0x7fb1fc073ba0 2024-04-20T11:47:04.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.636+0000 7fb1f9600700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb1e006ecc0 0x7fb1e0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:04.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.636+0000 7fb1f9600700 1 -- 172.21.15.117:0/472991723 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb1e4090310 con 0x7fb1fc073ba0 2024-04-20T11:47:04.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.636+0000 7fb1fbe00700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb1e006ecc0 0x7fb1e0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:04.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.637+0000 7fb1fbe00700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb1e006ecc0 0x7fb1e0071180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec006fd0 tx=0x7fb1ec008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:04.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.641+0000 7fb1f9600700 1 -- 172.21.15.117:0/472991723 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb1e4059110 con 0x7fb1fc073ba0 2024-04-20T11:47:04.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:04 smithi117 ceph-mon[99036]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:04.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:04 smithi117 ceph-mon[99036]: pgmap v1039: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:04 smithi173 ceph-mon[74165]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:04 smithi173 ceph-mon[74165]: pgmap v1039: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.913+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb1fc1141f0 con 0x7fb1fc073ba0 2024-04-20T11:47:04.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.915+0000 7fb1f9600700 1 -- 172.21.15.117:0/472991723 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fb1e405c7d0 con 0x7fb1fc073ba0 2024-04-20T11:47:04.916 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:47:04.916 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:47:04.916 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:47:04.916 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:47:04.916 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:47:04.917 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:47:04.918 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:47:04.919 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:47:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.917+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb1e006ecc0 msgr2=0x7fb1e0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.917+0000 7fb201800700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb1e006ecc0 0x7fb1e0071180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec006fd0 tx=0x7fb1ec008040 comp rx=0 tx=0).stop 2024-04-20T11:47:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.917+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 msgr2=0x7fb1fc19d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.917+0000 7fb201800700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 0x7fb1fc19d540 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006010 tx=0x7fb1e400be20 comp rx=0 tx=0).stop 2024-04-20T11:47:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.918+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 shutdown_connections 2024-04-20T11:47:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.918+0000 7fb201800700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb1e006ecc0 0x7fb1e0071180 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.918+0000 7fb201800700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1fc073280 0x7fb1fc19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.918+0000 7fb201800700 1 --2- 172.21.15.117:0/472991723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1fc073ba0 0x7fb1fc19d540 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:04.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.918+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 >> 172.21.15.117:0/472991723 conn(0x7fb1fc0fc500 msgr2=0x7fb1fc100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:04.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.918+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 shutdown_connections 2024-04-20T11:47:04.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:04.918+0000 7fb201800700 1 -- 172.21.15.117:0/472991723 wait complete. 2024-04-20T11:47:04.921 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:47:05.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.070+0000 7fc220200700 1 -- 172.21.15.117:0/4230374234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 msgr2=0x7fc218104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:05.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.070+0000 7fc220200700 1 --2- 172.21.15.117:0/4230374234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 0x7fc218104e80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc204009910 tx=0x7fc204009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:05.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.070+0000 7fc220200700 1 -- 172.21.15.117:0/4230374234 shutdown_connections 2024-04-20T11:47:05.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.070+0000 7fc220200700 1 --2- 172.21.15.117:0/4230374234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 0x7fc218104e80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.070+0000 7fc220200700 1 --2- 172.21.15.117:0/4230374234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc218103110 0x7fc2181034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.070+0000 7fc220200700 1 -- 172.21.15.117:0/4230374234 >> 172.21.15.117:0/4230374234 conn(0x7fc2180fe880 msgr2=0x7fc218100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.070+0000 7fc220200700 1 -- 172.21.15.117:0/4230374234 shutdown_connections 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.070+0000 7fc220200700 1 -- 172.21.15.117:0/4230374234 wait complete. 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.071+0000 7fc220200700 1 Processor -- start 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.072+0000 7fc220200700 1 -- start start 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.072+0000 7fc220200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc218103110 0x7fc218198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.072+0000 7fc220200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 0x7fc2181993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.072+0000 7fc220200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc218199ac0 con 0x7fc218104a00 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.072+0000 7fc220200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21819d740 con 0x7fc218103110 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.072+0000 7fc21ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 0x7fc2181993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.072+0000 7fc21ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 0x7fc2181993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43560/0 (socket says 172.21.15.117:43560) 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.072+0000 7fc21ce00700 1 -- 172.21.15.117:0/2286459346 learned_addr learned my addr 172.21.15.117:0/2286459346 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:05.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc21d800700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc218103110 0x7fc218198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:05.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc21ce00700 1 -- 172.21.15.117:0/2286459346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc218103110 msgr2=0x7fc218198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:05.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc21ce00700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc218103110 0x7fc218198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc21ce00700 1 -- 172.21.15.117:0/2286459346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2040095f0 con 0x7fc218104a00 2024-04-20T11:47:05.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc21d800700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc218103110 0x7fc218198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:47:05.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc21ce00700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 0x7fc2181993e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc2040092e0 tx=0x7fc20400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc216000700 1 -- 172.21.15.117:0/2286459346 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc20401d070 con 0x7fc218104a00 2024-04-20T11:47:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc216000700 1 -- 172.21.15.117:0/2286459346 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc20400f460 con 0x7fc218104a00 2024-04-20T11:47:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21819d9c0 con 0x7fc218104a00 2024-04-20T11:47:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.073+0000 7fc216000700 1 -- 172.21.15.117:0/2286459346 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc204017630 con 0x7fc218104a00 2024-04-20T11:47:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.074+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21819df30 con 0x7fc218104a00 2024-04-20T11:47:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.074+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21810b480 con 0x7fc218104a00 2024-04-20T11:47:05.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.075+0000 7fc216000700 1 -- 172.21.15.117:0/2286459346 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc20400f5d0 con 0x7fc218104a00 2024-04-20T11:47:05.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.076+0000 7fc216000700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc20006ecd0 0x7fc200071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:05.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.076+0000 7fc216000700 1 -- 172.21.15.117:0/2286459346 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc204090340 con 0x7fc218104a00 2024-04-20T11:47:05.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.077+0000 7fc21d800700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc20006ecd0 0x7fc200071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:05.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.077+0000 7fc21d800700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc20006ecd0 0x7fc200071190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc20c007eb0 tx=0x7fc20c0073c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:05.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.081+0000 7fc216000700 1 -- 172.21.15.117:0/2286459346 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc204059100 con 0x7fc218104a00 2024-04-20T11:47:05.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.318+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc21819a160 con 0x7fc20006ecd0 2024-04-20T11:47:05.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.321+0000 7fc216000700 1 -- 172.21.15.117:0/2286459346 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc21819a160 con 0x7fc20006ecd0 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:47:05.322 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:47:05.323 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:47:05.323 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:47:05.323 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:47:05.323 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:47:05.323 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:47:05.323 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc20006ecd0 msgr2=0x7fc200071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc20006ecd0 0x7fc200071190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc20c007eb0 tx=0x7fc20c0073c0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 msgr2=0x7fc2181993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 0x7fc2181993e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc2040092e0 tx=0x7fc20400be20 comp rx=0 tx=0).stop 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 shutdown_connections 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc20006ecd0 0x7fc200071190 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc218103110 0x7fc218198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 --2- 172.21.15.117:0/2286459346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218104a00 0x7fc2181993e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 >> 172.21.15.117:0/2286459346 conn(0x7fc2180fe880 msgr2=0x7fc2180ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:05.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 shutdown_connections 2024-04-20T11:47:05.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.324+0000 7fc220200700 1 -- 172.21.15.117:0/2286459346 wait complete. 2024-04-20T11:47:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.481+0000 7f7411000700 1 -- 172.21.15.117:0/3406769853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 msgr2=0x7f740c10cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.481+0000 7f7411000700 1 --2- 172.21.15.117:0/3406769853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 0x7f740c10cfc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009910 tx=0x7f73f8009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.482+0000 7f7411000700 1 -- 172.21.15.117:0/3406769853 shutdown_connections 2024-04-20T11:47:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.482+0000 7f7411000700 1 --2- 172.21.15.117:0/3406769853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 0x7f740c10cfc0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.482+0000 7f7411000700 1 --2- 172.21.15.117:0/3406769853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740c0ff950 0x7f740c0ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.482+0000 7f7411000700 1 -- 172.21.15.117:0/3406769853 >> 172.21.15.117:0/3406769853 conn(0x7f740c076750 msgr2=0x7f740c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.482+0000 7f7411000700 1 -- 172.21.15.117:0/3406769853 shutdown_connections 2024-04-20T11:47:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.482+0000 7f7411000700 1 -- 172.21.15.117:0/3406769853 wait complete. 2024-04-20T11:47:05.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.483+0000 7f7411000700 1 Processor -- start 2024-04-20T11:47:05.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.483+0000 7f7411000700 1 -- start start 2024-04-20T11:47:05.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7411000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740c0ff950 0x7f740c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7411000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 0x7f740c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7411000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c1998d0 con 0x7f740c100300 2024-04-20T11:47:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7411000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c19c540 con 0x7f740c0ff950 2024-04-20T11:47:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7409400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 0x7f740c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7409400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 0x7f740c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43580/0 (socket says 172.21.15.117:43580) 2024-04-20T11:47:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7409400700 1 -- 172.21.15.117:0/2380984787 learned_addr learned my addr 172.21.15.117:0/2380984787 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:05.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7409e00700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740c0ff950 0x7f740c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:05.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7409400700 1 -- 172.21.15.117:0/2380984787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740c0ff950 msgr2=0x7f740c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:05.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7409400700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740c0ff950 0x7f740c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7409400700 1 -- 172.21.15.117:0/2380984787 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f80095f0 con 0x7f740c100300 2024-04-20T11:47:05.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.484+0000 7f7409e00700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740c0ff950 0x7f740c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:47:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.485+0000 7f7409400700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 0x7f740c199240 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b770 tx=0x7f73f80057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.485+0000 7f7402a00700 1 -- 172.21.15.117:0/2380984787 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f801d070 con 0x7f740c100300 2024-04-20T11:47:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.485+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c19c7c0 con 0x7f740c100300 2024-04-20T11:47:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.485+0000 7f7402a00700 1 -- 172.21.15.117:0/2380984787 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73f80052a0 con 0x7f740c100300 2024-04-20T11:47:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.485+0000 7f7402a00700 1 -- 172.21.15.117:0/2380984787 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f8017400 con 0x7f740c100300 2024-04-20T11:47:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.485+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c19ccb0 con 0x7f740c100300 2024-04-20T11:47:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.486+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c068dd0 con 0x7f740c100300 2024-04-20T11:47:05.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.487+0000 7f7402a00700 1 -- 172.21.15.117:0/2380984787 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f73f800f850 con 0x7f740c100300 2024-04-20T11:47:05.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.488+0000 7f7402a00700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f73f006ecc0 0x7f73f0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:05.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.488+0000 7f7402a00700 1 -- 172.21.15.117:0/2380984787 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f73f80901f0 con 0x7f740c100300 2024-04-20T11:47:05.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.489+0000 7f7409e00700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f73f006ecc0 0x7f73f0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:05.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.489+0000 7f7409e00700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f73f006ecc0 0x7f73f0071180 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009be0 tx=0x7f73fc009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:05.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.493+0000 7f7402a00700 1 -- 172.21.15.117:0/2380984787 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f73f8058f30 con 0x7f740c100300 2024-04-20T11:47:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:05 smithi117 ceph-mon[99036]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:05 smithi117 ceph-mon[99036]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:05 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1396773575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:47:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:05 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/472991723' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:47:05.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.807+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f740c062510 con 0x7f740c100300 2024-04-20T11:47:05.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.807+0000 7f7402a00700 1 -- 172.21.15.117:0/2380984787 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f73f805c5f0 con 0x7f740c100300 2024-04-20T11:47:05.808 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:47:05.808 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:47:05.809 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:47:05.810 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:47:05.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f73f006ecc0 msgr2=0x7f73f0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:05.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f73f006ecc0 0x7f73f0071180 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009be0 tx=0x7f73fc009260 comp rx=0 tx=0).stop 2024-04-20T11:47:05.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 msgr2=0x7f740c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:05.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 0x7f740c199240 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b770 tx=0x7f73f80057d0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 shutdown_connections 2024-04-20T11:47:05.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f73f006ecc0 0x7f73f0071180 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740c0ff950 0x7f740c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 --2- 172.21.15.117:0/2380984787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f740c100300 0x7f740c199240 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:05.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.810+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 >> 172.21.15.117:0/2380984787 conn(0x7f740c076750 msgr2=0x7f740c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:05.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.811+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 shutdown_connections 2024-04-20T11:47:05.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:05.811+0000 7f7411000700 1 -- 172.21.15.117:0/2380984787 wait complete. 2024-04-20T11:47:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:05 smithi173 ceph-mon[74165]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:05 smithi173 ceph-mon[74165]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:05 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1396773575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:47:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:05 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/472991723' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:47:06.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:06 smithi117 ceph-mon[99036]: pgmap v1040: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:06.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:06 smithi117 ceph-mon[99036]: from='client.35870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:06.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:06 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2380984787' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:47:06.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:06 smithi173 ceph-mon[74165]: pgmap v1040: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:06.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:06 smithi173 ceph-mon[74165]: from='client.35870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:06.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:06 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2380984787' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:47:08.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:08 smithi117 ceph-mon[99036]: pgmap v1041: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:08 smithi173 ceph-mon[74165]: pgmap v1041: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:10.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:10 smithi117 ceph-mon[99036]: pgmap v1042: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:10 smithi173 ceph-mon[74165]: pgmap v1042: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:12.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:12 smithi117 ceph-mon[99036]: pgmap v1043: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:12 smithi173 ceph-mon[74165]: pgmap v1043: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:47:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:47:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:14 smithi117 ceph-mon[99036]: pgmap v1044: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:14 smithi173 ceph-mon[74165]: pgmap v1044: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:16 smithi117 ceph-mon[99036]: pgmap v1045: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:16 smithi173 ceph-mon[74165]: pgmap v1045: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:18.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:18 smithi117 ceph-mon[99036]: pgmap v1046: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:18 smithi173 ceph-mon[74165]: pgmap v1046: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:20.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:20 smithi117 ceph-mon[99036]: pgmap v1047: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:20 smithi173 ceph-mon[74165]: pgmap v1047: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:21 smithi117 ceph-mon[99036]: pgmap v1048: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:21 smithi173 ceph-mon[74165]: pgmap v1048: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:24 smithi117 ceph-mon[99036]: pgmap v1049: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:24 smithi173 ceph-mon[74165]: pgmap v1049: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:26.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:26 smithi117 ceph-mon[99036]: pgmap v1050: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:26 smithi173 ceph-mon[74165]: pgmap v1050: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:28 smithi117 ceph-mon[99036]: pgmap v1051: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:47:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:28 smithi173 ceph-mon[74165]: pgmap v1051: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:47:30.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:30 smithi117 ceph-mon[99036]: pgmap v1052: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:30 smithi173 ceph-mon[74165]: pgmap v1052: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:32.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:32 smithi117 ceph-mon[99036]: pgmap v1053: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:32 smithi173 ceph-mon[74165]: pgmap v1053: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:34.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:34 smithi117 ceph-mon[99036]: pgmap v1054: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:34.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:34 smithi173 ceph-mon[74165]: pgmap v1054: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:35.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.980+0000 7f7be0600700 1 -- 172.21.15.117:0/2579979159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8073310 msgr2=0x7f7bd80736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:35.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.980+0000 7f7be0600700 1 --2- 172.21.15.117:0/2579979159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8073310 0x7f7bd80736f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00b0a0 tx=0x7f7bcc00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:47:35.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.980+0000 7f7be0600700 1 -- 172.21.15.117:0/2579979159 shutdown_connections 2024-04-20T11:47:35.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.980+0000 7f7be0600700 1 --2- 172.21.15.117:0/2579979159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd8073c30 0x7f7bd810ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:35.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.980+0000 7f7be0600700 1 --2- 172.21.15.117:0/2579979159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8073310 0x7f7bd80736f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:35.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.980+0000 7f7be0600700 1 -- 172.21.15.117:0/2579979159 >> 172.21.15.117:0/2579979159 conn(0x7f7bd80fc640 msgr2=0x7f7bd80fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:35.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.980+0000 7f7be0600700 1 -- 172.21.15.117:0/2579979159 shutdown_connections 2024-04-20T11:47:35.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.980+0000 7f7be0600700 1 -- 172.21.15.117:0/2579979159 wait complete. 2024-04-20T11:47:35.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.982+0000 7f7be0600700 1 Processor -- start 2024-04-20T11:47:35.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.982+0000 7f7be0600700 1 -- start start 2024-04-20T11:47:35.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.982+0000 7f7be0600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd8073c30 0x7f7bd8199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:35.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.982+0000 7f7be0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8199580 0x7f7bd819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.982+0000 7f7be0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8199ba0 con 0x7f7bd8199580 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.982+0000 7f7be0600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8199d10 con 0x7f7bd8073c30 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bdd200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8199580 0x7f7bd819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bdd200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8199580 0x7f7bd819d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37856/0 (socket says 172.21.15.117:37856) 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bdd200700 1 -- 172.21.15.117:0/2917541277 learned_addr learned my addr 172.21.15.117:0/2917541277 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bddc00700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd8073c30 0x7f7bd8199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bdd200700 1 -- 172.21.15.117:0/2917541277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd8073c30 msgr2=0x7f7bd8199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bdd200700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd8073c30 0x7f7bd8199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bdd200700 1 -- 172.21.15.117:0/2917541277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc009d00 con 0x7f7bd8199580 2024-04-20T11:47:35.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bddc00700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd8073c30 0x7f7bd8199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:47:35.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.983+0000 7f7bdd200700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8199580 0x7f7bd819d930 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400b4b0 tx=0x7f7bc400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:35.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.984+0000 7f7bd6000700 1 -- 172.21.15.117:0/2917541277 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc4011800 con 0x7f7bd8199580 2024-04-20T11:47:35.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.984+0000 7f7bd6000700 1 -- 172.21.15.117:0/2917541277 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bc4011e20 con 0x7f7bd8199580 2024-04-20T11:47:35.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.984+0000 7f7bd6000700 1 -- 172.21.15.117:0/2917541277 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc400f2e0 con 0x7f7bd8199580 2024-04-20T11:47:35.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.984+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd819df30 con 0x7f7bd8199580 2024-04-20T11:47:35.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.984+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd819e480 con 0x7f7bd8199580 2024-04-20T11:47:35.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.985+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd819e750 con 0x7f7bd8199580 2024-04-20T11:47:35.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.986+0000 7f7bd6000700 1 -- 172.21.15.117:0/2917541277 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7bc4010380 con 0x7f7bd8199580 2024-04-20T11:47:35.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.987+0000 7f7bd6000700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7bc006ecc0 0x7f7bc0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:35.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.987+0000 7f7bd6000700 1 -- 172.21.15.117:0/2917541277 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7bc408ea50 con 0x7f7bd8199580 2024-04-20T11:47:35.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.987+0000 7f7bddc00700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7bc006ecc0 0x7f7bc0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:35.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.988+0000 7f7bddc00700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7bc006ecc0 0x7f7bc0071180 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc006070 tx=0x7f7bcc006000 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:35.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:35.992+0000 7f7bd6000700 1 -- 172.21.15.117:0/2917541277 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7bc40578c0 con 0x7f7bd8199580 2024-04-20T11:47:36.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.240+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bd819a350 con 0x7f7bc006ecc0 2024-04-20T11:47:36.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.243+0000 7f7bd6000700 1 -- 172.21.15.117:0/2917541277 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7bd819a350 con 0x7f7bc006ecc0 2024-04-20T11:47:36.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7bc006ecc0 msgr2=0x7f7bc0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:36.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7bc006ecc0 0x7f7bc0071180 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc006070 tx=0x7f7bcc006000 comp rx=0 tx=0).stop 2024-04-20T11:47:36.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8199580 msgr2=0x7f7bd819d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:36.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8199580 0x7f7bd819d930 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400b4b0 tx=0x7f7bc400b870 comp rx=0 tx=0).stop 2024-04-20T11:47:36.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 shutdown_connections 2024-04-20T11:47:36.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7bc006ecc0 0x7f7bc0071180 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd8073c30 0x7f7bd8199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 --2- 172.21.15.117:0/2917541277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd8199580 0x7f7bd819d930 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 >> 172.21.15.117:0/2917541277 conn(0x7f7bd80fc640 msgr2=0x7f7bd8108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:36.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 shutdown_connections 2024-04-20T11:47:36.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.246+0000 7f7be0600700 1 -- 172.21.15.117:0/2917541277 wait complete. 2024-04-20T11:47:36.262 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:47:36.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.407+0000 7f7fc1800700 1 -- 172.21.15.117:0/256229462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc107520 msgr2=0x7f7fbc107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:36.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.407+0000 7f7fc1800700 1 --2- 172.21.15.117:0/256229462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc107520 0x7f7fbc107900 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009910 tx=0x7f7fac009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:36.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.407+0000 7f7fc1800700 1 -- 172.21.15.117:0/256229462 shutdown_connections 2024-04-20T11:47:36.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.407+0000 7f7fc1800700 1 --2- 172.21.15.117:0/256229462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fbc108e10 0x7f7fbc109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.407+0000 7f7fc1800700 1 --2- 172.21.15.117:0/256229462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc107520 0x7f7fbc107900 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.407+0000 7f7fc1800700 1 -- 172.21.15.117:0/256229462 >> 172.21.15.117:0/256229462 conn(0x7f7fbc0768f0 msgr2=0x7f7fbc078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:36.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.407+0000 7f7fc1800700 1 -- 172.21.15.117:0/256229462 shutdown_connections 2024-04-20T11:47:36.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.407+0000 7f7fc1800700 1 -- 172.21.15.117:0/256229462 wait complete. 2024-04-20T11:47:36.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.408+0000 7f7fc1800700 1 Processor -- start 2024-04-20T11:47:36.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.408+0000 7f7fc1800700 1 -- start start 2024-04-20T11:47:36.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.409+0000 7f7fc1800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fbc107520 0x7f7fbc19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:36.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.409+0000 7f7fc1800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc108e10 0x7f7fbc19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:36.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.409+0000 7f7fc1800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc19de90 con 0x7f7fbc108e10 2024-04-20T11:47:36.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.409+0000 7f7fc1800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc1a1b60 con 0x7f7fbc107520 2024-04-20T11:47:36.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.409+0000 7f7fba000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc108e10 0x7f7fbc19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:36.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.409+0000 7f7fba000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc108e10 0x7f7fbc19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37868/0 (socket says 172.21.15.117:37868) 2024-04-20T11:47:36.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.409+0000 7f7fba000700 1 -- 172.21.15.117:0/3605967317 learned_addr learned my addr 172.21.15.117:0/3605967317 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:36.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.409+0000 7f7fbaa00700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fbc107520 0x7f7fbc19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:36.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.410+0000 7f7fba000700 1 -- 172.21.15.117:0/3605967317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fbc107520 msgr2=0x7f7fbc19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:36.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.410+0000 7f7fba000700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fbc107520 0x7f7fbc19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.410+0000 7f7fba000700 1 -- 172.21.15.117:0/3605967317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac0095f0 con 0x7f7fbc108e10 2024-04-20T11:47:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.410+0000 7f7fba000700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc108e10 0x7f7fbc19d7b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800b4b0 tx=0x7f7fa800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.410+0000 7f7fb3400700 1 -- 172.21.15.117:0/3605967317 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa8011800 con 0x7f7fbc108e10 2024-04-20T11:47:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.410+0000 7f7fb3400700 1 -- 172.21.15.117:0/3605967317 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fa8011e20 con 0x7f7fbc108e10 2024-04-20T11:47:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.410+0000 7f7fb3400700 1 -- 172.21.15.117:0/3605967317 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa800f2e0 con 0x7f7fbc108e10 2024-04-20T11:47:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.410+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc1a1e40 con 0x7f7fbc108e10 2024-04-20T11:47:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.411+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc1a2410 con 0x7f7fbc108e10 2024-04-20T11:47:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.412+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fbc10f870 con 0x7f7fbc108e10 2024-04-20T11:47:36.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.413+0000 7f7fb3400700 1 -- 172.21.15.117:0/3605967317 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7fa8010380 con 0x7f7fbc108e10 2024-04-20T11:47:36.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.414+0000 7f7fb3400700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7fa006ecd0 0x7f7fa0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:36.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.415+0000 7f7fb3400700 1 -- 172.21.15.117:0/3605967317 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7fa808ea00 con 0x7f7fbc108e10 2024-04-20T11:47:36.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.415+0000 7f7fbaa00700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7fa006ecd0 0x7f7fa0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:36.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.415+0000 7f7fbaa00700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7fa006ecd0 0x7f7fa0071190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00b5c0 tx=0x7f7fac01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:36.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.420+0000 7f7fb3400700 1 -- 172.21.15.117:0/3605967317 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f7fa8057870 con 0x7f7fbc108e10 2024-04-20T11:47:36.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:36 smithi117 ceph-mon[99036]: pgmap v1055: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:36.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.653+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fbc19e5f0 con 0x7f7fa006ecd0 2024-04-20T11:47:36.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.654+0000 7f7fb3400700 1 -- 172.21.15.117:0/3605967317 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7fbc19e5f0 con 0x7f7fa006ecd0 2024-04-20T11:47:36.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7fa006ecd0 msgr2=0x7f7fa0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:36.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7fa006ecd0 0x7f7fa0071190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00b5c0 tx=0x7f7fac01a040 comp rx=0 tx=0).stop 2024-04-20T11:47:36.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc108e10 msgr2=0x7f7fbc19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:36.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc108e10 0x7f7fbc19d7b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800b4b0 tx=0x7f7fa800b870 comp rx=0 tx=0).stop 2024-04-20T11:47:36.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 shutdown_connections 2024-04-20T11:47:36.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f7fa006ecd0 0x7f7fa0071190 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fbc107520 0x7f7fbc19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 --2- 172.21.15.117:0/3605967317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc108e10 0x7f7fbc19d7b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 >> 172.21.15.117:0/3605967317 conn(0x7f7fbc0768f0 msgr2=0x7f7fbc078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:36.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 shutdown_connections 2024-04-20T11:47:36.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.657+0000 7f7fc1800700 1 -- 172.21.15.117:0/3605967317 wait complete. 2024-04-20T11:47:36.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:36 smithi173 ceph-mon[74165]: pgmap v1055: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.811+0000 7f9e57200700 1 -- 172.21.15.117:0/820421572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 msgr2=0x7f9e50107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.811+0000 7f9e57200700 1 --2- 172.21.15.117:0/820421572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 0x7f9e50107a40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009910 tx=0x7f9e3c009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.811+0000 7f9e57200700 1 -- 172.21.15.117:0/820421572 shutdown_connections 2024-04-20T11:47:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.811+0000 7f9e57200700 1 --2- 172.21.15.117:0/820421572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 0x7f9e50107a40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.811+0000 7f9e57200700 1 --2- 172.21.15.117:0/820421572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e50103040 0x7f9e50103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.811+0000 7f9e57200700 1 -- 172.21.15.117:0/820421572 >> 172.21.15.117:0/820421572 conn(0x7f9e500fe8d0 msgr2=0x7f9e50100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.811+0000 7f9e57200700 1 -- 172.21.15.117:0/820421572 shutdown_connections 2024-04-20T11:47:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.811+0000 7f9e57200700 1 -- 172.21.15.117:0/820421572 wait complete. 2024-04-20T11:47:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.813+0000 7f9e57200700 1 Processor -- start 2024-04-20T11:47:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.813+0000 7f9e57200700 1 -- start start 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.813+0000 7f9e57200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e50103040 0x7f9e50198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.813+0000 7f9e57200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 0x7f9e50199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.813+0000 7f9e57200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50199a70 con 0x7f9e501039f0 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.813+0000 7f9e57200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5019d740 con 0x7f9e50103040 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 0x7f9e50199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 0x7f9e50199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37888/0 (socket says 172.21.15.117:37888) 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4f400700 1 -- 172.21.15.117:0/3727183373 learned_addr learned my addr 172.21.15.117:0/3727183373 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4fe00700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e50103040 0x7f9e50198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4f400700 1 -- 172.21.15.117:0/3727183373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e50103040 msgr2=0x7f9e50198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4f400700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e50103040 0x7f9e50198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:36.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4f400700 1 -- 172.21.15.117:0/3727183373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0095f0 con 0x7f9e501039f0 2024-04-20T11:47:36.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4fe00700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e50103040 0x7f9e50198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:47:36.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.814+0000 7f9e4f400700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 0x7f9e50199390 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005b40 tx=0x7f9e3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:36.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.815+0000 7f9e4cc00700 1 -- 172.21.15.117:0/3727183373 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e3c01d070 con 0x7f9e501039f0 2024-04-20T11:47:36.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.815+0000 7f9e4cc00700 1 -- 172.21.15.117:0/3727183373 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e3c00f460 con 0x7f9e501039f0 2024-04-20T11:47:36.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.815+0000 7f9e4cc00700 1 -- 172.21.15.117:0/3727183373 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e3c017630 con 0x7f9e501039f0 2024-04-20T11:47:36.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.815+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5019d9c0 con 0x7f9e501039f0 2024-04-20T11:47:36.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.815+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5019deb0 con 0x7f9e501039f0 2024-04-20T11:47:36.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.816+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e50104520 con 0x7f9e501039f0 2024-04-20T11:47:36.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.817+0000 7f9e4cc00700 1 -- 172.21.15.117:0/3727183373 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9e3c003a60 con 0x7f9e501039f0 2024-04-20T11:47:36.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.818+0000 7f9e4cc00700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e3406ecc0 0x7f9e34071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:36.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.818+0000 7f9e4cc00700 1 -- 172.21.15.117:0/3727183373 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9e3c0902a0 con 0x7f9e501039f0 2024-04-20T11:47:36.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.819+0000 7f9e4fe00700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e3406ecc0 0x7f9e34071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:36.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.819+0000 7f9e4fe00700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e3406ecc0 0x7f9e34071180 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9e40007900 tx=0x7f9e40008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:36.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:36.823+0000 7f9e4cc00700 1 -- 172.21.15.117:0/3727183373 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9e3c059060 con 0x7f9e501039f0 2024-04-20T11:47:37.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.054+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e50110060 con 0x7f9e3406ecc0 2024-04-20T11:47:37.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.074+0000 7f9e4cc00700 1 -- 172.21.15.117:0/3727183373 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9e50110060 con 0x7f9e3406ecc0 2024-04-20T11:47:37.075 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:47:37.075 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (36m) 12m ago 50m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:47:37.075 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (50m) 12m ago 50m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:47:37.075 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (49m) 12m ago 49m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (32m) 12m ago 50m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (32m) 12m ago 49m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (35m) 12m ago 49m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (27m) 12m ago 42m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (28m) 12m ago 42m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (12m) 12m ago 42m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (27m) 12m ago 42m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (39m) 12m ago 51m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (37m) 12m ago 48m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (34m) 12m ago 52m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (33m) 12m ago 48m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (37m) 12m ago 50m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (37m) 12m ago 48m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:47:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (31m) 12m ago 47m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:47:37.077 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (31m) 12m ago 47m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:47:37.077 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (30m) 12m ago 46m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:47:37.077 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (30m) 12m ago 45m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:47:37.077 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (29m) 12m ago 45m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:47:37.077 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (28m) 12m ago 44m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:47:37.077 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (36m) 12m ago 49m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.076+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e3406ecc0 msgr2=0x7f9e34071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.076+0000 7f9e57200700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e3406ecc0 0x7f9e34071180 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9e40007900 tx=0x7f9e40008040 comp rx=0 tx=0).stop 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.076+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 msgr2=0x7f9e50199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.076+0000 7f9e57200700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 0x7f9e50199390 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005b40 tx=0x7f9e3c00be20 comp rx=0 tx=0).stop 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.077+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 shutdown_connections 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.077+0000 7f9e57200700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9e3406ecc0 0x7f9e34071180 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.077+0000 7f9e57200700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e50103040 0x7f9e50198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.077+0000 7f9e57200700 1 --2- 172.21.15.117:0/3727183373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e501039f0 0x7f9e50199390 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.077+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 >> 172.21.15.117:0/3727183373 conn(0x7f9e500fe8d0 msgr2=0x7f9e500ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.077+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 shutdown_connections 2024-04-20T11:47:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.077+0000 7f9e57200700 1 -- 172.21.15.117:0/3727183373 wait complete. 2024-04-20T11:47:37.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.223+0000 7fdf5f600700 1 -- 172.21.15.117:0/1224120092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58108e10 msgr2=0x7fdf58109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.223+0000 7fdf5f600700 1 --2- 172.21.15.117:0/1224120092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58108e10 0x7fdf58109290 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009910 tx=0x7fdf44009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:37.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.223+0000 7fdf5f600700 1 -- 172.21.15.117:0/1224120092 shutdown_connections 2024-04-20T11:47:37.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.223+0000 7fdf5f600700 1 --2- 172.21.15.117:0/1224120092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58108e10 0x7fdf58109290 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.223+0000 7fdf5f600700 1 --2- 172.21.15.117:0/1224120092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf58107520 0x7fdf58107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.223+0000 7fdf5f600700 1 -- 172.21.15.117:0/1224120092 >> 172.21.15.117:0/1224120092 conn(0x7fdf580768f0 msgr2=0x7fdf58078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:37.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.223+0000 7fdf5f600700 1 -- 172.21.15.117:0/1224120092 shutdown_connections 2024-04-20T11:47:37.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.223+0000 7fdf5f600700 1 -- 172.21.15.117:0/1224120092 wait complete. 2024-04-20T11:47:37.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.224+0000 7fdf5f600700 1 Processor -- start 2024-04-20T11:47:37.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.224+0000 7fdf5f600700 1 -- start start 2024-04-20T11:47:37.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5f600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58107520 0x7fdf5819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:37.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5f600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf58108e10 0x7fdf5819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:37.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5f600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5819ded0 con 0x7fdf58107520 2024-04-20T11:47:37.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5f600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581a1ba0 con 0x7fdf58108e10 2024-04-20T11:47:37.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58107520 0x7fdf5819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:37.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58107520 0x7fdf5819d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37906/0 (socket says 172.21.15.117:37906) 2024-04-20T11:47:37.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5cc00700 1 -- 172.21.15.117:0/2645348658 learned_addr learned my addr 172.21.15.117:0/2645348658 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:37.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5cc00700 1 -- 172.21.15.117:0/2645348658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf58108e10 msgr2=0x7fdf5819d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5cc00700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf58108e10 0x7fdf5819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.225+0000 7fdf5cc00700 1 -- 172.21.15.117:0/2645348658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf440095f0 con 0x7fdf58107520 2024-04-20T11:47:37.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.226+0000 7fdf5cc00700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58107520 0x7fdf5819d2b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00b4b0 tx=0x7fdf4c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:37.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.226+0000 7fdf55600700 1 -- 172.21.15.117:0/2645348658 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4c011800 con 0x7fdf58107520 2024-04-20T11:47:37.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.226+0000 7fdf55600700 1 -- 172.21.15.117:0/2645348658 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf4c011e20 con 0x7fdf58107520 2024-04-20T11:47:37.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.226+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf581a1e80 con 0x7fdf58107520 2024-04-20T11:47:37.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.226+0000 7fdf55600700 1 -- 172.21.15.117:0/2645348658 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4c00f2e0 con 0x7fdf58107520 2024-04-20T11:47:37.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.226+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf581a2450 con 0x7fdf58107520 2024-04-20T11:47:37.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.227+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5810f870 con 0x7fdf58107520 2024-04-20T11:47:37.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.229+0000 7fdf55600700 1 -- 172.21.15.117:0/2645348658 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdf4c010380 con 0x7fdf58107520 2024-04-20T11:47:37.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.230+0000 7fdf55600700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf4006ecc0 0x7fdf40071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:37.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.230+0000 7fdf55600700 1 -- 172.21.15.117:0/2645348658 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdf4c08ea00 con 0x7fdf58107520 2024-04-20T11:47:37.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.230+0000 7fdf57e00700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf4006ecc0 0x7fdf40071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:37.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.231+0000 7fdf57e00700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf4006ecc0 0x7fdf40071180 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b5c0 tx=0x7fdf44005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:37.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.236+0000 7fdf55600700 1 -- 172.21.15.117:0/2645348658 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdf4c057870 con 0x7fdf58107520 2024-04-20T11:47:37.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.552+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf5819e5c0 con 0x7fdf58107520 2024-04-20T11:47:37.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.553+0000 7fdf55600700 1 -- 172.21.15.117:0/2645348658 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fdf4c05af30 con 0x7fdf58107520 2024-04-20T11:47:37.554 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:47:37.554 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:47:37.554 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:47:37.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:47:37.554 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:47:37.555 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:47:37.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf4006ecc0 msgr2=0x7fdf40071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf4006ecc0 0x7fdf40071180 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b5c0 tx=0x7fdf44005960 comp rx=0 tx=0).stop 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58107520 msgr2=0x7fdf5819d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58107520 0x7fdf5819d2b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00b4b0 tx=0x7fdf4c00b870 comp rx=0 tx=0).stop 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 shutdown_connections 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdf4006ecc0 0x7fdf40071180 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf58107520 0x7fdf5819d2b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 --2- 172.21.15.117:0/2645348658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf58108e10 0x7fdf5819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 >> 172.21.15.117:0/2645348658 conn(0x7fdf580768f0 msgr2=0x7fdf580775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 shutdown_connections 2024-04-20T11:47:37.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.557+0000 7fdf5f600700 1 -- 172.21.15.117:0/2645348658 wait complete. 2024-04-20T11:47:37.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:37 smithi117 ceph-mon[99036]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:37 smithi173 ceph-mon[74165]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:37.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.714+0000 7f6052600700 1 -- 172.21.15.117:0/2009319781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 msgr2=0x7f604c0757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.714+0000 7f6052600700 1 --2- 172.21.15.117:0/2009319781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 0x7f604c0757f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f603c009910 tx=0x7f603c009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:37.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.714+0000 7f6052600700 1 -- 172.21.15.117:0/2009319781 shutdown_connections 2024-04-20T11:47:37.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.714+0000 7f6052600700 1 --2- 172.21.15.117:0/2009319781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f604c075d30 0x7f604c111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.714+0000 7f6052600700 1 --2- 172.21.15.117:0/2009319781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 0x7f604c0757f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.714+0000 7f6052600700 1 -- 172.21.15.117:0/2009319781 >> 172.21.15.117:0/2009319781 conn(0x7f604c0fe8d0 msgr2=0x7f604c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:37.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.714+0000 7f6052600700 1 -- 172.21.15.117:0/2009319781 shutdown_connections 2024-04-20T11:47:37.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.714+0000 7f6052600700 1 -- 172.21.15.117:0/2009319781 wait complete. 2024-04-20T11:47:37.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.715+0000 7f6052600700 1 Processor -- start 2024-04-20T11:47:37.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.715+0000 7f6052600700 1 -- start start 2024-04-20T11:47:37.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f6052600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 0x7f604c19d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:37.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f6052600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f604c075d30 0x7f604c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:37.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f6052600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c19dcf0 con 0x7f604c075410 2024-04-20T11:47:37.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f6052600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c19de30 con 0x7f604c075d30 2024-04-20T11:47:37.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f604b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 0x7f604c19d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:37.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f604b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 0x7f604c19d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37926/0 (socket says 172.21.15.117:37926) 2024-04-20T11:47:37.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f604b400700 1 -- 172.21.15.117:0/533741262 learned_addr learned my addr 172.21.15.117:0/533741262 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:37.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f604b400700 1 -- 172.21.15.117:0/533741262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f604c075d30 msgr2=0x7f604c19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f604b400700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f604c075d30 0x7f604c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:37.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.716+0000 7f604b400700 1 -- 172.21.15.117:0/533741262 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c0095f0 con 0x7f604c075410 2024-04-20T11:47:37.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.717+0000 7f604b400700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 0x7f604c19d160 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f603c00b850 tx=0x7f603c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:37.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.717+0000 7f6048c00700 1 -- 172.21.15.117:0/533741262 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f603c01d070 con 0x7f604c075410 2024-04-20T11:47:37.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.717+0000 7f6048c00700 1 -- 172.21.15.117:0/533741262 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f603c003d50 con 0x7f604c075410 2024-04-20T11:47:37.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.717+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c1a1b70 con 0x7f604c075410 2024-04-20T11:47:37.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.717+0000 7f6048c00700 1 -- 172.21.15.117:0/533741262 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f603c021410 con 0x7f604c075410 2024-04-20T11:47:37.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.717+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c1a20c0 con 0x7f604c075410 2024-04-20T11:47:37.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.718+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c076b50 con 0x7f604c075410 2024-04-20T11:47:37.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.719+0000 7f6048c00700 1 -- 172.21.15.117:0/533741262 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f603c003890 con 0x7f604c075410 2024-04-20T11:47:37.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.720+0000 7f6048c00700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f603006ecc0 0x7f6030071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:37.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.720+0000 7f6048c00700 1 -- 172.21.15.117:0/533741262 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f603c090130 con 0x7f604c075410 2024-04-20T11:47:37.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.720+0000 7f6043e00700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f603006ecc0 0x7f6030071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:37.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.721+0000 7f6043e00700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f603006ecc0 0x7f6030071180 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6034005fd0 tx=0x7f6034005d00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:37.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.725+0000 7f6048c00700 1 -- 172.21.15.117:0/533741262 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f603c058ef0 con 0x7f604c075410 2024-04-20T11:47:37.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.993+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f604c114330 con 0x7f604c075410 2024-04-20T11:47:37.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.994+0000 7f6048c00700 1 -- 172.21.15.117:0/533741262 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f603c05c5b0 con 0x7f604c075410 2024-04-20T11:47:37.995 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:47:37.995 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:47:37.995 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:47:37.995 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:47:37.996 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:47:37.997 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:47:37.998 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:47:37.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f603006ecc0 msgr2=0x7f6030071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:37.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f603006ecc0 0x7f6030071180 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6034005fd0 tx=0x7f6034005d00 comp rx=0 tx=0).stop 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 msgr2=0x7f604c19d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 0x7f604c19d160 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f603c00b850 tx=0x7f603c0056a0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 shutdown_connections 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f603006ecc0 0x7f6030071180 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604c075410 0x7f604c19d160 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 --2- 172.21.15.117:0/533741262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f604c075d30 0x7f604c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 >> 172.21.15.117:0/533741262 conn(0x7f604c0fe8d0 msgr2=0x7f604c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.998+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 shutdown_connections 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:37.999+0000 7f6052600700 1 -- 172.21.15.117:0/533741262 wait complete. 2024-04-20T11:47:38.000 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:47:38.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.149+0000 7fd51d000700 1 -- 172.21.15.117:0/2537066926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 msgr2=0x7fd518105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.149+0000 7fd51d000700 1 --2- 172.21.15.117:0/2537066926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 0x7fd518105810 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd504009910 tx=0x7fd504009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:38.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.149+0000 7fd51d000700 1 -- 172.21.15.117:0/2537066926 shutdown_connections 2024-04-20T11:47:38.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.149+0000 7fd51d000700 1 --2- 172.21.15.117:0/2537066926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 0x7fd518105810 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.149+0000 7fd51d000700 1 --2- 172.21.15.117:0/2537066926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd518100e10 0x7fd5181011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.149+0000 7fd51d000700 1 -- 172.21.15.117:0/2537066926 >> 172.21.15.117:0/2537066926 conn(0x7fd5180fc6c0 msgr2=0x7fd5180feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:38.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.149+0000 7fd51d000700 1 -- 172.21.15.117:0/2537066926 shutdown_connections 2024-04-20T11:47:38.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.149+0000 7fd51d000700 1 -- 172.21.15.117:0/2537066926 wait complete. 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.150+0000 7fd51d000700 1 Processor -- start 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.150+0000 7fd51d000700 1 -- start start 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd51d000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd518100e10 0x7fd51819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd51d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 0x7fd51819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd51d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51819f5a0 con 0x7fd5181017c0 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd51d000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd518198b20 con 0x7fd518100e10 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd515400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 0x7fd51819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd515400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 0x7fd51819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37946/0 (socket says 172.21.15.117:37946) 2024-04-20T11:47:38.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd515400700 1 -- 172.21.15.117:0/3080274847 learned_addr learned my addr 172.21.15.117:0/3080274847 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd515e00700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd518100e10 0x7fd51819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd515400700 1 -- 172.21.15.117:0/3080274847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd518100e10 msgr2=0x7fd51819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd515400700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd518100e10 0x7fd51819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.151+0000 7fd515400700 1 -- 172.21.15.117:0/3080274847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040095f0 con 0x7fd5181017c0 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.152+0000 7fd515400700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 0x7fd51819eec0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd504005b40 tx=0x7fd50400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.152+0000 7fd50ea00700 1 -- 172.21.15.117:0/3080274847 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd50401d070 con 0x7fd5181017c0 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.152+0000 7fd50ea00700 1 -- 172.21.15.117:0/3080274847 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd504005700 con 0x7fd5181017c0 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.152+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd518198da0 con 0x7fd5181017c0 2024-04-20T11:47:38.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.152+0000 7fd50ea00700 1 -- 172.21.15.117:0/3080274847 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd50400fe50 con 0x7fd5181017c0 2024-04-20T11:47:38.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.152+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd518199290 con 0x7fd5181017c0 2024-04-20T11:47:38.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.153+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd518103f80 con 0x7fd5181017c0 2024-04-20T11:47:38.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.154+0000 7fd50ea00700 1 -- 172.21.15.117:0/3080274847 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd50400f460 con 0x7fd5181017c0 2024-04-20T11:47:38.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.156+0000 7fd50ea00700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4fc06ecc0 0x7fd4fc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:38.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.156+0000 7fd50ea00700 1 -- 172.21.15.117:0/3080274847 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd5040901f0 con 0x7fd5181017c0 2024-04-20T11:47:38.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.156+0000 7fd515e00700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4fc06ecc0 0x7fd4fc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:38.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.156+0000 7fd515e00700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4fc06ecc0 0x7fd4fc071180 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd508005950 tx=0x7fd5080058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:38.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.160+0000 7fd50ea00700 1 -- 172.21.15.117:0/3080274847 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd50405cb80 con 0x7fd5181017c0 2024-04-20T11:47:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.396+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd51810ddb0 con 0x7fd4fc06ecc0 2024-04-20T11:47:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.397+0000 7fd50ea00700 1 -- 172.21.15.117:0/3080274847 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd51810ddb0 con 0x7fd4fc06ecc0 2024-04-20T11:47:38.398 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:47:38.398 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:47:38.398 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:47:38.398 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:47:38.399 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:47:38.400 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:47:38.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4fc06ecc0 msgr2=0x7fd4fc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4fc06ecc0 0x7fd4fc071180 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd508005950 tx=0x7fd5080058e0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 msgr2=0x7fd51819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 0x7fd51819eec0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd504005b40 tx=0x7fd50400be20 comp rx=0 tx=0).stop 2024-04-20T11:47:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 shutdown_connections 2024-04-20T11:47:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd4fc06ecc0 0x7fd4fc071180 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd518100e10 0x7fd51819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 --2- 172.21.15.117:0/3080274847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5181017c0 0x7fd51819eec0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 >> 172.21.15.117:0/3080274847 conn(0x7fd5180fc6c0 msgr2=0x7fd5180fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 shutdown_connections 2024-04-20T11:47:38.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.400+0000 7fd51d000700 1 -- 172.21.15.117:0/3080274847 wait complete. 2024-04-20T11:47:38.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.555+0000 7f873bc00700 1 -- 172.21.15.117:0/2116449684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734103110 msgr2=0x7f87341034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.555+0000 7f873bc00700 1 --2- 172.21.15.117:0/2116449684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734103110 0x7f87341034f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8728009910 tx=0x7f8728009c20 comp rx=0 tx=0).stop 2024-04-20T11:47:38.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.555+0000 7f873bc00700 1 -- 172.21.15.117:0/2116449684 shutdown_connections 2024-04-20T11:47:38.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.555+0000 7f873bc00700 1 --2- 172.21.15.117:0/2116449684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8734104a00 0x7f8734104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.555+0000 7f873bc00700 1 --2- 172.21.15.117:0/2116449684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734103110 0x7f87341034f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.555+0000 7f873bc00700 1 -- 172.21.15.117:0/2116449684 >> 172.21.15.117:0/2116449684 conn(0x7f87340fe880 msgr2=0x7f8734100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:38.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.555+0000 7f873bc00700 1 -- 172.21.15.117:0/2116449684 shutdown_connections 2024-04-20T11:47:38.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.555+0000 7f873bc00700 1 -- 172.21.15.117:0/2116449684 wait complete. 2024-04-20T11:47:38.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.556+0000 7f873bc00700 1 Processor -- start 2024-04-20T11:47:38.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.556+0000 7f873bc00700 1 -- start start 2024-04-20T11:47:38.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f873bc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734104a00 0x7f8734198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:38.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f873bc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8734199520 0x7f873419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:38.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f873bc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734199b40 con 0x7f8734104a00 2024-04-20T11:47:38.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f873bc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734199cb0 con 0x7f8734199520 2024-04-20T11:47:38.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f8739200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734104a00 0x7f8734198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:38.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f8739200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734104a00 0x7f8734198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37964/0 (socket says 172.21.15.117:37964) 2024-04-20T11:47:38.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f8739200700 1 -- 172.21.15.117:0/1739293727 learned_addr learned my addr 172.21.15.117:0/1739293727 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f8739200700 1 -- 172.21.15.117:0/1739293727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8734199520 msgr2=0x7f873419d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f8739200700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8734199520 0x7f873419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.557+0000 7f8739200700 1 -- 172.21.15.117:0/1739293727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280095f0 con 0x7f8734104a00 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.558+0000 7f8739200700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734104a00 0x7f8734198fe0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8728005b40 tx=0x7f872800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.558+0000 7f8731600700 1 -- 172.21.15.117:0/1739293727 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872801d070 con 0x7f8734104a00 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.558+0000 7f8731600700 1 -- 172.21.15.117:0/1739293727 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8728005700 con 0x7f8734104a00 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.558+0000 7f8731600700 1 -- 172.21.15.117:0/1739293727 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872800fd60 con 0x7f8734104a00 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.558+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873419de70 con 0x7f8734104a00 2024-04-20T11:47:38.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.558+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873419e440 con 0x7f8734104a00 2024-04-20T11:47:38.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.559+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873410b480 con 0x7f8734104a00 2024-04-20T11:47:38.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.560+0000 7f8731600700 1 -- 172.21.15.117:0/1739293727 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f872800f560 con 0x7f8734104a00 2024-04-20T11:47:38.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.561+0000 7f8731600700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f871c06ecd0 0x7f871c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:47:38.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.561+0000 7f8731600700 1 -- 172.21.15.117:0/1739293727 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8728090100 con 0x7f8734104a00 2024-04-20T11:47:38.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.562+0000 7f8733e00700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f871c06ecd0 0x7f871c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:47:38.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.562+0000 7f8733e00700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f871c06ecd0 0x7f871c071190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8724005950 tx=0x7f87240058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:47:38.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.566+0000 7f8731600700 1 -- 172.21.15.117:0/1739293727 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8728058ec0 con 0x7f8734104a00 2024-04-20T11:47:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:38 smithi117 ceph-mon[99036]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:38 smithi117 ceph-mon[99036]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:38 smithi117 ceph-mon[99036]: pgmap v1056: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:38 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2645348658' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:47:38.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:38 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/533741262' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:47:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:38 smithi173 ceph-mon[74165]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:38 smithi173 ceph-mon[74165]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:38 smithi173 ceph-mon[74165]: pgmap v1056: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:38.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:38 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2645348658' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:47:38.662 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:38 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/533741262' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:47:38.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.874+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f87341010b0 con 0x7f8734104a00 2024-04-20T11:47:38.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.875+0000 7f8731600700 1 -- 172.21.15.117:0/1739293727 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f872805c580 con 0x7f8734104a00 2024-04-20T11:47:38.876 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:47:38.877 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:47:38.878 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:47:38.879 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:47:38.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f871c06ecd0 msgr2=0x7f871c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f871c06ecd0 0x7f871c071190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8724005950 tx=0x7f87240058e0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734104a00 msgr2=0x7f8734198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:47:38.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734104a00 0x7f8734198fe0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8728005b40 tx=0x7f872800be20 comp rx=0 tx=0).stop 2024-04-20T11:47:38.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 shutdown_connections 2024-04-20T11:47:38.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f871c06ecd0 0x7f871c071190 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8734104a00 0x7f8734198fe0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 --2- 172.21.15.117:0/1739293727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8734199520 0x7f873419d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:47:38.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 >> 172.21.15.117:0/1739293727 conn(0x7f87340fe880 msgr2=0x7f8734101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:47:38.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.878+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 shutdown_connections 2024-04-20T11:47:38.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:47:38.879+0000 7f873bc00700 1 -- 172.21.15.117:0/1739293727 wait complete. 2024-04-20T11:47:39.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:39 smithi117 ceph-mon[99036]: from='client.35898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:39.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:39 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1739293727' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:47:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:39 smithi173 ceph-mon[74165]: from='client.35898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:47:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:39 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1739293727' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:47:40.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:40 smithi117 ceph-mon[99036]: pgmap v1057: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:40.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:40 smithi173 ceph-mon[74165]: pgmap v1057: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:42.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:42 smithi117 ceph-mon[99036]: pgmap v1058: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:42.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:42 smithi173 ceph-mon[74165]: pgmap v1058: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:47:43.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:47:44.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:44 smithi173 ceph-mon[74165]: pgmap v1059: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:44 smithi117 ceph-mon[99036]: pgmap v1059: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:46.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:46 smithi117 ceph-mon[99036]: pgmap v1060: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:46 smithi173 ceph-mon[74165]: pgmap v1060: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:48.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:48 smithi117 ceph-mon[99036]: pgmap v1061: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:48 smithi173 ceph-mon[74165]: pgmap v1061: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:50.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:49 smithi117 ceph-mon[99036]: pgmap v1062: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:50.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:49 smithi173 ceph-mon[74165]: pgmap v1062: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:52.626 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:52 smithi117 ceph-mon[99036]: pgmap v1063: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:52 smithi173 ceph-mon[74165]: pgmap v1063: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:54.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:54 smithi117 ceph-mon[99036]: pgmap v1064: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:54 smithi173 ceph-mon[74165]: pgmap v1064: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:56.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:56 smithi117 ceph-mon[99036]: pgmap v1065: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:56 smithi173 ceph-mon[74165]: pgmap v1065: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:58 smithi117 ceph-mon[99036]: pgmap v1066: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:47:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:47:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:58 smithi173 ceph-mon[74165]: pgmap v1066: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:47:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:47:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:48:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:00 smithi117 ceph-mon[99036]: pgmap v1067: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:00 smithi173 ceph-mon[74165]: pgmap v1067: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:02.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:02 smithi117 ceph-mon[99036]: pgmap v1068: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:02 smithi173 ceph-mon[74165]: pgmap v1068: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:04.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:04 smithi117 ceph-mon[99036]: pgmap v1069: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:04 smithi173 ceph-mon[74165]: pgmap v1069: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:06.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:06 smithi117 ceph-mon[99036]: pgmap v1070: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:06 smithi173 ceph-mon[74165]: pgmap v1070: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:08.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:08 smithi117 ceph-mon[99036]: pgmap v1071: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:08 smithi173 ceph-mon[74165]: pgmap v1071: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:09.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.050+0000 7f5ce6000700 1 -- 172.21.15.117:0/2226706628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0103110 msgr2=0x7f5ce01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.050+0000 7f5ce6000700 1 --2- 172.21.15.117:0/2226706628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0103110 0x7f5ce01034f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009910 tx=0x7f5cd0009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:09.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.050+0000 7f5ce6000700 1 -- 172.21.15.117:0/2226706628 shutdown_connections 2024-04-20T11:48:09.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.050+0000 7f5ce6000700 1 --2- 172.21.15.117:0/2226706628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0104a00 0x7f5ce0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.050+0000 7f5ce6000700 1 --2- 172.21.15.117:0/2226706628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0103110 0x7f5ce01034f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.050+0000 7f5ce6000700 1 -- 172.21.15.117:0/2226706628 >> 172.21.15.117:0/2226706628 conn(0x7f5ce00fe880 msgr2=0x7f5ce0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:09.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.051+0000 7f5ce6000700 1 -- 172.21.15.117:0/2226706628 shutdown_connections 2024-04-20T11:48:09.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.051+0000 7f5ce6000700 1 -- 172.21.15.117:0/2226706628 wait complete. 2024-04-20T11:48:09.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.052+0000 7f5ce6000700 1 Processor -- start 2024-04-20T11:48:09.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.052+0000 7f5ce6000700 1 -- start start 2024-04-20T11:48:09.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.052+0000 7f5ce6000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0103110 0x7f5ce0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.052+0000 7f5ce6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0104a00 0x7f5ce0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.052+0000 7f5ce6000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce0199950 con 0x7f5ce0104a00 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.052+0000 7f5ce6000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce0199a90 con 0x7f5ce0103110 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.052+0000 7f5cd6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0104a00 0x7f5ce0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.052+0000 7f5cd6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0104a00 0x7f5ce0199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48432/0 (socket says 172.21.15.117:48432) 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.053+0000 7f5cd6a00700 1 -- 172.21.15.117:0/1085083737 learned_addr learned my addr 172.21.15.117:0/1085083737 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.053+0000 7f5cdf400700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0103110 0x7f5ce0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.053+0000 7f5cd6a00700 1 -- 172.21.15.117:0/1085083737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0103110 msgr2=0x7f5ce0198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.053+0000 7f5cd6a00700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0103110 0x7f5ce0198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.053+0000 7f5cd6a00700 1 -- 172.21.15.117:0/1085083737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd00095f0 con 0x7f5ce0104a00 2024-04-20T11:48:09.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.053+0000 7f5cdf400700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0103110 0x7f5ce0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:48:09.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.053+0000 7f5cd6a00700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0104a00 0x7f5ce0199300 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800c930 tx=0x7f5cc800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:09.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.054+0000 7f5cdcc00700 1 -- 172.21.15.117:0/1085083737 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc8007ad0 con 0x7f5ce0104a00 2024-04-20T11:48:09.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.054+0000 7f5cdcc00700 1 -- 172.21.15.117:0/1085083737 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cc8007c30 con 0x7f5ce0104a00 2024-04-20T11:48:09.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.054+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce019d7d0 con 0x7f5ce0104a00 2024-04-20T11:48:09.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.054+0000 7f5cdcc00700 1 -- 172.21.15.117:0/1085083737 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc8018610 con 0x7f5ce0104a00 2024-04-20T11:48:09.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.054+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce019dda0 con 0x7f5ce0104a00 2024-04-20T11:48:09.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.055+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce010b460 con 0x7f5ce0104a00 2024-04-20T11:48:09.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.056+0000 7f5cdcc00700 1 -- 172.21.15.117:0/1085083737 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5cc800f450 con 0x7f5ce0104a00 2024-04-20T11:48:09.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.057+0000 7f5cdcc00700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cc406ecd0 0x7f5cc4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.057+0000 7f5cdcc00700 1 -- 172.21.15.117:0/1085083737 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5cc808f070 con 0x7f5ce0104a00 2024-04-20T11:48:09.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.057+0000 7f5cdf400700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cc406ecd0 0x7f5cc4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.058+0000 7f5cdf400700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cc406ecd0 0x7f5cc4071190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0006010 tx=0x7f5cd000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.062+0000 7f5cdcc00700 1 -- 172.21.15.117:0/1085083737 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5cc8057e50 con 0x7f5ce0104a00 2024-04-20T11:48:09.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.311+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ce019a080 con 0x7f5cc406ecd0 2024-04-20T11:48:09.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.313+0000 7f5cdcc00700 1 -- 172.21.15.117:0/1085083737 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5ce019a080 con 0x7f5cc406ecd0 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cc406ecd0 msgr2=0x7f5cc4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cc406ecd0 0x7f5cc4071190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0006010 tx=0x7f5cd000b540 comp rx=0 tx=0).stop 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0104a00 msgr2=0x7f5ce0199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0104a00 0x7f5ce0199300 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800c930 tx=0x7f5cc800ccf0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 shutdown_connections 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5cc406ecd0 0x7f5cc4071190 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0103110 0x7f5ce0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 --2- 172.21.15.117:0/1085083737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce0104a00 0x7f5ce0199300 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 >> 172.21.15.117:0/1085083737 conn(0x7f5ce00fe880 msgr2=0x7f5ce00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 shutdown_connections 2024-04-20T11:48:09.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.317+0000 7f5ce6000700 1 -- 172.21.15.117:0/1085083737 wait complete. 2024-04-20T11:48:09.333 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:48:09.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.477+0000 7f46d7000700 1 -- 172.21.15.117:0/2270814837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 msgr2=0x7f46d0107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.477+0000 7f46d7000700 1 --2- 172.21.15.117:0/2270814837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 0x7f46d0107820 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009910 tx=0x7f46bc009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:09.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.478+0000 7f46d7000700 1 -- 172.21.15.117:0/2270814837 shutdown_connections 2024-04-20T11:48:09.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.478+0000 7f46d7000700 1 --2- 172.21.15.117:0/2270814837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 0x7f46d0107820 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.478+0000 7f46d7000700 1 --2- 172.21.15.117:0/2270814837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46d0102e20 0x7f46d0103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.478+0000 7f46d7000700 1 -- 172.21.15.117:0/2270814837 >> 172.21.15.117:0/2270814837 conn(0x7f46d00fe6d0 msgr2=0x7f46d0100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:09.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.478+0000 7f46d7000700 1 -- 172.21.15.117:0/2270814837 shutdown_connections 2024-04-20T11:48:09.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.478+0000 7f46d7000700 1 -- 172.21.15.117:0/2270814837 wait complete. 2024-04-20T11:48:09.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.479+0000 7f46d7000700 1 Processor -- start 2024-04-20T11:48:09.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.479+0000 7f46d7000700 1 -- start start 2024-04-20T11:48:09.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.479+0000 7f46d7000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46d0102e20 0x7f46d019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.479+0000 7f46d7000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 0x7f46d019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.479+0000 7f46d7000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d01989c0 con 0x7f46d01037d0 2024-04-20T11:48:09.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.479+0000 7f46d7000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d0198b30 con 0x7f46d0102e20 2024-04-20T11:48:09.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.480+0000 7f46d5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 0x7f46d019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.480+0000 7f46d5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 0x7f46d019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48448/0 (socket says 172.21.15.117:48448) 2024-04-20T11:48:09.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.480+0000 7f46d5200700 1 -- 172.21.15.117:0/2829354537 learned_addr learned my addr 172.21.15.117:0/2829354537 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.480+0000 7f46d5c00700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46d0102e20 0x7f46d019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.480+0000 7f46d5200700 1 -- 172.21.15.117:0/2829354537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46d0102e20 msgr2=0x7f46d019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.480+0000 7f46d5200700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46d0102e20 0x7f46d019e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.480+0000 7f46d5200700 1 -- 172.21.15.117:0/2829354537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc0095f0 con 0x7f46d01037d0 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.480+0000 7f46d5200700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 0x7f46d019ee50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0092e0 tx=0x7f46bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.481+0000 7f46ce000700 1 -- 172.21.15.117:0/2829354537 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc01d070 con 0x7f46d01037d0 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.481+0000 7f46ce000700 1 -- 172.21.15.117:0/2829354537 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46bc005700 con 0x7f46d01037d0 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.481+0000 7f46ce000700 1 -- 172.21.15.117:0/2829354537 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc017400 con 0x7f46d01037d0 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.481+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d0198db0 con 0x7f46d01037d0 2024-04-20T11:48:09.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.481+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d01992a0 con 0x7f46d01037d0 2024-04-20T11:48:09.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.482+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d010b220 con 0x7f46d01037d0 2024-04-20T11:48:09.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.483+0000 7f46ce000700 1 -- 172.21.15.117:0/2829354537 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f46bc003a60 con 0x7f46d01037d0 2024-04-20T11:48:09.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.484+0000 7f46ce000700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f46b806ecc0 0x7f46b8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.484+0000 7f46ce000700 1 -- 172.21.15.117:0/2829354537 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f46bc090170 con 0x7f46d01037d0 2024-04-20T11:48:09.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.484+0000 7f46d5c00700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f46b806ecc0 0x7f46b8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.485+0000 7f46d5c00700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f46b806ecc0 0x7f46b8071180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009c10 tx=0x7f46c4009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:09.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.489+0000 7f46ce000700 1 -- 172.21.15.117:0/2829354537 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f46bc058f30 con 0x7f46d01037d0 2024-04-20T11:48:09.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.735+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46d0199ea0 con 0x7f46b806ecc0 2024-04-20T11:48:09.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.736+0000 7f46ce000700 1 -- 172.21.15.117:0/2829354537 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f46d0199ea0 con 0x7f46b806ecc0 2024-04-20T11:48:09.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.739+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f46b806ecc0 msgr2=0x7f46b8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.739+0000 7f46d7000700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f46b806ecc0 0x7f46b8071180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009c10 tx=0x7f46c4009040 comp rx=0 tx=0).stop 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.739+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 msgr2=0x7f46d019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.739+0000 7f46d7000700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 0x7f46d019ee50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0092e0 tx=0x7f46bc00be20 comp rx=0 tx=0).stop 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.740+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 shutdown_connections 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.740+0000 7f46d7000700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f46b806ecc0 0x7f46b8071180 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.740+0000 7f46d7000700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46d0102e20 0x7f46d019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.740+0000 7f46d7000700 1 --2- 172.21.15.117:0/2829354537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46d01037d0 0x7f46d019ee50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.740+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 >> 172.21.15.117:0/2829354537 conn(0x7f46d00fe6d0 msgr2=0x7f46d00feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.740+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 shutdown_connections 2024-04-20T11:48:09.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.740+0000 7f46d7000700 1 -- 172.21.15.117:0/2829354537 wait complete. 2024-04-20T11:48:09.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.893+0000 7fe7a5600700 1 -- 172.21.15.117:0/3019249309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 msgr2=0x7fe7a0107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.893+0000 7fe7a5600700 1 --2- 172.21.15.117:0/3019249309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 0x7fe7a0107900 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe790009910 tx=0x7fe790009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:09.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.893+0000 7fe7a5600700 1 -- 172.21.15.117:0/3019249309 shutdown_connections 2024-04-20T11:48:09.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.893+0000 7fe7a5600700 1 --2- 172.21.15.117:0/3019249309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7a0108e10 0x7fe7a0109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.893+0000 7fe7a5600700 1 --2- 172.21.15.117:0/3019249309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 0x7fe7a0107900 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.893+0000 7fe7a5600700 1 -- 172.21.15.117:0/3019249309 >> 172.21.15.117:0/3019249309 conn(0x7fe7a00768f0 msgr2=0x7fe7a0078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:09.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.893+0000 7fe7a5600700 1 -- 172.21.15.117:0/3019249309 shutdown_connections 2024-04-20T11:48:09.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.893+0000 7fe7a5600700 1 -- 172.21.15.117:0/3019249309 wait complete. 2024-04-20T11:48:09.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.894+0000 7fe7a5600700 1 Processor -- start 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.894+0000 7fe7a5600700 1 -- start start 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe7a5600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 0x7fe7a019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe7a5600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7a0108e10 0x7fe7a019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe7a5600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019dd40 con 0x7fe7a0107520 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe7a5600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019de80 con 0x7fe7a0108e10 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe79ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 0x7fe7a019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe79ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 0x7fe7a019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48478/0 (socket says 172.21.15.117:48478) 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe79ea00700 1 -- 172.21.15.117:0/3393016866 learned_addr learned my addr 172.21.15.117:0/3393016866 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:09.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe79e000700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7a0108e10 0x7fe7a019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe79ea00700 1 -- 172.21.15.117:0/3393016866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7a0108e10 msgr2=0x7fe7a019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe79ea00700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7a0108e10 0x7fe7a019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.895+0000 7fe79ea00700 1 -- 172.21.15.117:0/3393016866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900095f0 con 0x7fe7a0107520 2024-04-20T11:48:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.896+0000 7fe79ea00700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 0x7fe7a019d1b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe7900092e0 tx=0x7fe79000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.896+0000 7fe797400700 1 -- 172.21.15.117:0/3393016866 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79001d070 con 0x7fe7a0107520 2024-04-20T11:48:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.896+0000 7fe797400700 1 -- 172.21.15.117:0/3393016866 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe79000f460 con 0x7fe7a0107520 2024-04-20T11:48:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.896+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a01a1bb0 con 0x7fe7a0107520 2024-04-20T11:48:09.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.896+0000 7fe797400700 1 -- 172.21.15.117:0/3393016866 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe790017630 con 0x7fe7a0107520 2024-04-20T11:48:09.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.896+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a01a2120 con 0x7fe7a0107520 2024-04-20T11:48:09.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.897+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a010f890 con 0x7fe7a0107520 2024-04-20T11:48:09.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.898+0000 7fe797400700 1 -- 172.21.15.117:0/3393016866 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe79000f5d0 con 0x7fe7a0107520 2024-04-20T11:48:09.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.899+0000 7fe797400700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe78406ecd0 0x7fe784071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:09.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.899+0000 7fe797400700 1 -- 172.21.15.117:0/3393016866 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe7900903e0 con 0x7fe7a0107520 2024-04-20T11:48:09.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.900+0000 7fe79e000700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe78406ecd0 0x7fe784071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:09.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.900+0000 7fe79e000700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe78406ecd0 0x7fe784071190 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe78c005ea0 tx=0x7fe78c005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:09.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:09.904+0000 7fe797400700 1 -- 172.21.15.117:0/3393016866 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe790059120 con 0x7fe7a0107520 2024-04-20T11:48:10.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.137+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7a019e530 con 0x7fe78406ecd0 2024-04-20T11:48:10.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.152+0000 7fe797400700 1 -- 172.21.15.117:0/3393016866 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe7a019e530 con 0x7fe78406ecd0 2024-04-20T11:48:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:48:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (36m) 12m ago 50m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (51m) 12m ago 51m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (49m) 12m ago 49m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (32m) 12m ago 50m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (32m) 12m ago 49m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (35m) 12m ago 50m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (28m) 12m ago 43m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (28m) 12m ago 43m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (12m) 12m ago 43m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (28m) 12m ago 43m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (39m) 12m ago 52m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (38m) 12m ago 49m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (34m) 12m ago 52m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (34m) 12m ago 49m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:48:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (37m) 12m ago 50m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:48:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (37m) 12m ago 49m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:48:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (32m) 12m ago 48m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:48:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (31m) 12m ago 47m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:48:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (31m) 12m ago 47m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:48:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (30m) 12m ago 46m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:48:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (29m) 12m ago 45m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:48:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (29m) 12m ago 45m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:48:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (37m) 12m ago 50m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:48:10.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe78406ecd0 msgr2=0x7fe784071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:10.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe78406ecd0 0x7fe784071190 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe78c005ea0 tx=0x7fe78c005e30 comp rx=0 tx=0).stop 2024-04-20T11:48:10.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 msgr2=0x7fe7a019d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:10.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 0x7fe7a019d1b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe7900092e0 tx=0x7fe79000be20 comp rx=0 tx=0).stop 2024-04-20T11:48:10.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 shutdown_connections 2024-04-20T11:48:10.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe78406ecd0 0x7fe784071190 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a0107520 0x7fe7a019d1b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 --2- 172.21.15.117:0/3393016866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7a0108e10 0x7fe7a019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.154+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 >> 172.21.15.117:0/3393016866 conn(0x7fe7a00768f0 msgr2=0x7fe7a0077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:10.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.155+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 shutdown_connections 2024-04-20T11:48:10.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.155+0000 7fe7a5600700 1 -- 172.21.15.117:0/3393016866 wait complete. 2024-04-20T11:48:10.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.309+0000 7f3a82a00700 1 -- 172.21.15.117:0/3132815274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c107520 msgr2=0x7f3a7c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:10.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.309+0000 7f3a82a00700 1 --2- 172.21.15.117:0/3132815274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c107520 0x7f3a7c107900 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009910 tx=0x7f3a6c009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:10.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.309+0000 7f3a82a00700 1 -- 172.21.15.117:0/3132815274 shutdown_connections 2024-04-20T11:48:10.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.309+0000 7f3a82a00700 1 --2- 172.21.15.117:0/3132815274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a7c108e10 0x7f3a7c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.309+0000 7f3a82a00700 1 --2- 172.21.15.117:0/3132815274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c107520 0x7f3a7c107900 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.309+0000 7f3a82a00700 1 -- 172.21.15.117:0/3132815274 >> 172.21.15.117:0/3132815274 conn(0x7f3a7c0768f0 msgr2=0x7f3a7c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:10.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.309+0000 7f3a82a00700 1 -- 172.21.15.117:0/3132815274 shutdown_connections 2024-04-20T11:48:10.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.309+0000 7f3a82a00700 1 -- 172.21.15.117:0/3132815274 wait complete. 2024-04-20T11:48:10.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.310+0000 7f3a82a00700 1 Processor -- start 2024-04-20T11:48:10.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.310+0000 7f3a82a00700 1 -- start start 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.311+0000 7f3a82a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a7c107520 0x7f3a7c19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.311+0000 7f3a82a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c108e10 0x7f3a7c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.311+0000 7f3a82a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c19ddd0 con 0x7f3a7c108e10 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.311+0000 7f3a82a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a1aa0 con 0x7f3a7c107520 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.311+0000 7f3a7b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c108e10 0x7f3a7c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.311+0000 7f3a7b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c108e10 0x7f3a7c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48494/0 (socket says 172.21.15.117:48494) 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.311+0000 7f3a7b400700 1 -- 172.21.15.117:0/427465314 learned_addr learned my addr 172.21.15.117:0/427465314 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.311+0000 7f3a7b400700 1 -- 172.21.15.117:0/427465314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a7c107520 msgr2=0x7f3a7c19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.312+0000 7f3a7b400700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a7c107520 0x7f3a7c19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.312+0000 7f3a7b400700 1 -- 172.21.15.117:0/427465314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0095f0 con 0x7f3a7c108e10 2024-04-20T11:48:10.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.312+0000 7f3a7b400700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c108e10 0x7f3a7c19d6f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c8a0 tx=0x7f3a6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:10.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.312+0000 7f3a78c00700 1 -- 172.21.15.117:0/427465314 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6800cea0 con 0x7f3a7c108e10 2024-04-20T11:48:10.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.312+0000 7f3a78c00700 1 -- 172.21.15.117:0/427465314 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a68004830 con 0x7f3a7c108e10 2024-04-20T11:48:10.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.312+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c1a1d80 con 0x7f3a7c108e10 2024-04-20T11:48:10.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.312+0000 7f3a78c00700 1 -- 172.21.15.117:0/427465314 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a680055b0 con 0x7f3a7c108e10 2024-04-20T11:48:10.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.312+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c1a2350 con 0x7f3a7c108e10 2024-04-20T11:48:10.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.314+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c10f870 con 0x7f3a7c108e10 2024-04-20T11:48:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.315+0000 7f3a78c00700 1 -- 172.21.15.117:0/427465314 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3a680049a0 con 0x7f3a7c108e10 2024-04-20T11:48:10.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.316+0000 7f3a78c00700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3a6006ecc0 0x7f3a60071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:10.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.316+0000 7f3a78c00700 1 -- 172.21.15.117:0/427465314 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3a6808eac0 con 0x7f3a7c108e10 2024-04-20T11:48:10.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.316+0000 7f3a7be00700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3a6006ecc0 0x7f3a60071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:10.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.317+0000 7f3a7be00700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3a6006ecc0 0x7f3a60071180 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00b5c0 tx=0x7f3a6c005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:10.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.321+0000 7f3a78c00700 1 -- 172.21.15.117:0/427465314 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3a68057880 con 0x7f3a7c108e10 2024-04-20T11:48:10.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.619+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a7c19e4c0 con 0x7f3a7c108e10 2024-04-20T11:48:10.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.621+0000 7f3a78c00700 1 -- 172.21.15.117:0/427465314 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3a6801d070 con 0x7f3a7c108e10 2024-04-20T11:48:10.620 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:48:10.621 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:48:10.622 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:48:10.622 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:48:10.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.623+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3a6006ecc0 msgr2=0x7f3a60071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:10.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.623+0000 7f3a82a00700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3a6006ecc0 0x7f3a60071180 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00b5c0 tx=0x7f3a6c005e20 comp rx=0 tx=0).stop 2024-04-20T11:48:10.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.623+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c108e10 msgr2=0x7f3a7c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:10.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.623+0000 7f3a82a00700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c108e10 0x7f3a7c19d6f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c8a0 tx=0x7f3a6800cc60 comp rx=0 tx=0).stop 2024-04-20T11:48:10.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.623+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 shutdown_connections 2024-04-20T11:48:10.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.623+0000 7f3a82a00700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f3a6006ecc0 0x7f3a60071180 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.624+0000 7f3a82a00700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a7c107520 0x7f3a7c19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.624+0000 7f3a82a00700 1 --2- 172.21.15.117:0/427465314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a7c108e10 0x7f3a7c19d6f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.624+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 >> 172.21.15.117:0/427465314 conn(0x7f3a7c0768f0 msgr2=0x7f3a7c077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:10.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.624+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 shutdown_connections 2024-04-20T11:48:10.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.624+0000 7f3a82a00700 1 -- 172.21.15.117:0/427465314 wait complete. 2024-04-20T11:48:10.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:10 smithi117 ceph-mon[99036]: pgmap v1072: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:10.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:10 smithi117 ceph-mon[99036]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:10 smithi173 ceph-mon[74165]: pgmap v1072: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:10.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:10 smithi173 ceph-mon[74165]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.773+0000 7f5788c00700 1 -- 172.21.15.117:0/2345447517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 msgr2=0x7f5784105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.773+0000 7f5788c00700 1 --2- 172.21.15.117:0/2345447517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 0x7f5784105820 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5770009910 tx=0x7f5770009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.773+0000 7f5788c00700 1 -- 172.21.15.117:0/2345447517 shutdown_connections 2024-04-20T11:48:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.773+0000 7f5788c00700 1 --2- 172.21.15.117:0/2345447517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 0x7f5784105820 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.773+0000 7f5788c00700 1 --2- 172.21.15.117:0/2345447517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5784100e20 0x7f5784101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.773+0000 7f5788c00700 1 -- 172.21.15.117:0/2345447517 >> 172.21.15.117:0/2345447517 conn(0x7f57840fc6b0 msgr2=0x7f57840fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.773+0000 7f5788c00700 1 -- 172.21.15.117:0/2345447517 shutdown_connections 2024-04-20T11:48:10.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.773+0000 7f5788c00700 1 -- 172.21.15.117:0/2345447517 wait complete. 2024-04-20T11:48:10.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.774+0000 7f5788c00700 1 Processor -- start 2024-04-20T11:48:10.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.774+0000 7f5788c00700 1 -- start start 2024-04-20T11:48:10.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5788c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5784100e20 0x7f57840731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:10.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5788c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 0x7f5784073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:10.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5788c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5784073c60 con 0x7f57841017d0 2024-04-20T11:48:10.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5788c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5784073da0 con 0x7f5784100e20 2024-04-20T11:48:10.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5781400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 0x7f5784073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:10.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5781400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 0x7f5784073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48512/0 (socket says 172.21.15.117:48512) 2024-04-20T11:48:10.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5781400700 1 -- 172.21.15.117:0/3092904399 learned_addr learned my addr 172.21.15.117:0/3092904399 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5781e00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5784100e20 0x7f57840731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5781400700 1 -- 172.21.15.117:0/3092904399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5784100e20 msgr2=0x7f57840731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5781400700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5784100e20 0x7f57840731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5781400700 1 -- 172.21.15.117:0/3092904399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700095f0 con 0x7f57841017d0 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.775+0000 7f5781e00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5784100e20 0x7f57840731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.776+0000 7f5781400700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 0x7f5784073720 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f57700092e0 tx=0x7f577000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.776+0000 7f577aa00700 1 -- 172.21.15.117:0/3092904399 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f577001d070 con 0x7f57841017d0 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.776+0000 7f577aa00700 1 -- 172.21.15.117:0/3092904399 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f577000f460 con 0x7f57841017d0 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.776+0000 7f577aa00700 1 -- 172.21.15.117:0/3092904399 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5770017630 con 0x7f57841017d0 2024-04-20T11:48:10.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.776+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5784074020 con 0x7f57841017d0 2024-04-20T11:48:10.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.776+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57841a3100 con 0x7f57841017d0 2024-04-20T11:48:10.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.777+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5784102300 con 0x7f57841017d0 2024-04-20T11:48:10.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.778+0000 7f577aa00700 1 -- 172.21.15.117:0/3092904399 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f577000f5d0 con 0x7f57841017d0 2024-04-20T11:48:10.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.779+0000 7f577aa00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f576806ecc0 0x7f5768071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:10.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.779+0000 7f577aa00700 1 -- 172.21.15.117:0/3092904399 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5770090260 con 0x7f57841017d0 2024-04-20T11:48:10.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.780+0000 7f5781e00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f576806ecc0 0x7f5768071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:10.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.780+0000 7f5781e00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f576806ecc0 0x7f5768071180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5774007900 tx=0x7f5774008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:10.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:10.784+0000 7f577aa00700 1 -- 172.21.15.117:0/3092904399 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5770059020 con 0x7f57841017d0 2024-04-20T11:48:11.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.058+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f578410de40 con 0x7f57841017d0 2024-04-20T11:48:11.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.059+0000 7f577aa00700 1 -- 172.21.15.117:0/3092904399 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f5770005740 con 0x7f57841017d0 2024-04-20T11:48:11.059 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:48:11.059 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:48:11.059 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:48:11.059 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:48:11.059 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:48:11.059 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:48:11.059 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:48:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:48:11.061 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:48:11.062 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:48:11.062 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:48:11.062 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:48:11.062 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:48:11.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f576806ecc0 msgr2=0x7f5768071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f576806ecc0 0x7f5768071180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5774007900 tx=0x7f5774008040 comp rx=0 tx=0).stop 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 msgr2=0x7f5784073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 0x7f5784073720 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f57700092e0 tx=0x7f577000be20 comp rx=0 tx=0).stop 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 shutdown_connections 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f576806ecc0 0x7f5768071180 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5784100e20 0x7f57840731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 --2- 172.21.15.117:0/3092904399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57841017d0 0x7f5784073720 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 >> 172.21.15.117:0/3092904399 conn(0x7f57840fc6b0 msgr2=0x7f57840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 shutdown_connections 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.062+0000 7f5788c00700 1 -- 172.21.15.117:0/3092904399 wait complete. 2024-04-20T11:48:11.063 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:48:11.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.213+0000 7f298e000700 1 -- 172.21.15.117:0/2231870848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 msgr2=0x7f2988105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.213+0000 7f298e000700 1 --2- 172.21.15.117:0/2231870848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 0x7f2988105810 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2974009910 tx=0x7f2974009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:11.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.213+0000 7f298e000700 1 -- 172.21.15.117:0/2231870848 shutdown_connections 2024-04-20T11:48:11.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.213+0000 7f298e000700 1 --2- 172.21.15.117:0/2231870848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 0x7f2988105810 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.213+0000 7f298e000700 1 --2- 172.21.15.117:0/2231870848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2988100e10 0x7f29881011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.213+0000 7f298e000700 1 -- 172.21.15.117:0/2231870848 >> 172.21.15.117:0/2231870848 conn(0x7f29880fc6a0 msgr2=0x7f29880feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:11.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.213+0000 7f298e000700 1 -- 172.21.15.117:0/2231870848 shutdown_connections 2024-04-20T11:48:11.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.213+0000 7f298e000700 1 -- 172.21.15.117:0/2231870848 wait complete. 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.214+0000 7f298e000700 1 Processor -- start 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.214+0000 7f298e000700 1 -- start start 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.215+0000 7f298e000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2988100e10 0x7f298819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.215+0000 7f298e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 0x7f298819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.215+0000 7f298e000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298819d360 con 0x7f29881017c0 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.215+0000 7f298e000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2988196990 con 0x7f2988100e10 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.215+0000 7f2986a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 0x7f298819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.215+0000 7f2986a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 0x7f298819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48528/0 (socket says 172.21.15.117:48528) 2024-04-20T11:48:11.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.215+0000 7f2986a00700 1 -- 172.21.15.117:0/2434125831 learned_addr learned my addr 172.21.15.117:0/2434125831 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:11.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f2986a00700 1 -- 172.21.15.117:0/2434125831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2988100e10 msgr2=0x7f298819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f2986a00700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2988100e10 0x7f298819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f2986a00700 1 -- 172.21.15.117:0/2434125831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740095f0 con 0x7f29881017c0 2024-04-20T11:48:11.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f2986a00700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 0x7f298819cd40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2974005b40 tx=0x7f297400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:11.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f297fe00700 1 -- 172.21.15.117:0/2434125831 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297401d070 con 0x7f29881017c0 2024-04-20T11:48:11.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f297fe00700 1 -- 172.21.15.117:0/2434125831 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2974005700 con 0x7f29881017c0 2024-04-20T11:48:11.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2988196c10 con 0x7f29881017c0 2024-04-20T11:48:11.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f297fe00700 1 -- 172.21.15.117:0/2434125831 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297400fe50 con 0x7f29881017c0 2024-04-20T11:48:11.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.216+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2988197100 con 0x7f29881017c0 2024-04-20T11:48:11.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.217+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2988109210 con 0x7f29881017c0 2024-04-20T11:48:11.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.219+0000 7f297fe00700 1 -- 172.21.15.117:0/2434125831 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f297400f460 con 0x7f29881017c0 2024-04-20T11:48:11.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.219+0000 7f297fe00700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f296c06ecd0 0x7f296c071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:11.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.220+0000 7f297fe00700 1 -- 172.21.15.117:0/2434125831 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f29740902e0 con 0x7f29881017c0 2024-04-20T11:48:11.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.220+0000 7f2987400700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f296c06ecd0 0x7f296c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:11.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.220+0000 7f2987400700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f296c06ecd0 0x7f296c071190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2978005950 tx=0x7f29780058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:11.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.224+0000 7f297fe00700 1 -- 172.21.15.117:0/2434125831 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f297405cb50 con 0x7f29881017c0 2024-04-20T11:48:11.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.456+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f298810ddb0 con 0x7f296c06ecd0 2024-04-20T11:48:11.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.459+0000 7f297fe00700 1 -- 172.21.15.117:0/2434125831 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f298810ddb0 con 0x7f296c06ecd0 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:48:11.459 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:48:11.460 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:48:11.460 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:48:11.460 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:48:11.460 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:48:11.460 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:48:11.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f296c06ecd0 msgr2=0x7f296c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f296c06ecd0 0x7f296c071190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2978005950 tx=0x7f29780058e0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 msgr2=0x7f298819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 0x7f298819cd40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2974005b40 tx=0x7f297400be20 comp rx=0 tx=0).stop 2024-04-20T11:48:11.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 shutdown_connections 2024-04-20T11:48:11.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f296c06ecd0 0x7f296c071190 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2988100e10 0x7f298819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 --2- 172.21.15.117:0/2434125831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29881017c0 0x7f298819cd40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.462+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 >> 172.21.15.117:0/2434125831 conn(0x7f29880fc6a0 msgr2=0x7f29880fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:11.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.463+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 shutdown_connections 2024-04-20T11:48:11.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.463+0000 7f298e000700 1 -- 172.21.15.117:0/2434125831 wait complete. 2024-04-20T11:48:11.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.606+0000 7f4097000700 1 -- 172.21.15.117:0/1411858724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 msgr2=0x7f4090107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.606+0000 7f4097000700 1 --2- 172.21.15.117:0/1411858724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 0x7f4090107a40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f407c009910 tx=0x7f407c009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.606+0000 7f4097000700 1 -- 172.21.15.117:0/1411858724 shutdown_connections 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.606+0000 7f4097000700 1 --2- 172.21.15.117:0/1411858724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 0x7f4090107a40 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.606+0000 7f4097000700 1 --2- 172.21.15.117:0/1411858724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4090103040 0x7f4090103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.606+0000 7f4097000700 1 -- 172.21.15.117:0/1411858724 >> 172.21.15.117:0/1411858724 conn(0x7f40900fe8d0 msgr2=0x7f4090100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.606+0000 7f4097000700 1 -- 172.21.15.117:0/1411858724 shutdown_connections 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.606+0000 7f4097000700 1 -- 172.21.15.117:0/1411858724 wait complete. 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.607+0000 7f4097000700 1 Processor -- start 2024-04-20T11:48:11.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.607+0000 7f4097000700 1 -- start start 2024-04-20T11:48:11.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f4097000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4090103040 0x7f4090198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:11.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f4097000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 0x7f40901992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:11.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f4097000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4090199940 con 0x7f40901039f0 2024-04-20T11:48:11.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f4097000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409019c560 con 0x7f4090103040 2024-04-20T11:48:11.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f408f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 0x7f40901992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:11.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f408f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 0x7f40901992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48550/0 (socket says 172.21.15.117:48550) 2024-04-20T11:48:11.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f408f400700 1 -- 172.21.15.117:0/3301512153 learned_addr learned my addr 172.21.15.117:0/3301512153 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:11.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f408f400700 1 -- 172.21.15.117:0/3301512153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4090103040 msgr2=0x7f4090198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f408f400700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4090103040 0x7f4090198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.608+0000 7f408f400700 1 -- 172.21.15.117:0/3301512153 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0095f0 con 0x7f40901039f0 2024-04-20T11:48:11.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.609+0000 7f408f400700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 0x7f40901992b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f407c005b40 tx=0x7f407c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:11.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.609+0000 7f408cc00700 1 -- 172.21.15.117:0/3301512153 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407c01d070 con 0x7f40901039f0 2024-04-20T11:48:11.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.609+0000 7f408cc00700 1 -- 172.21.15.117:0/3301512153 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f407c00f460 con 0x7f40901039f0 2024-04-20T11:48:11.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.609+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409019c7e0 con 0x7f40901039f0 2024-04-20T11:48:11.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.609+0000 7f408cc00700 1 -- 172.21.15.117:0/3301512153 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407c017630 con 0x7f40901039f0 2024-04-20T11:48:11.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.609+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409019ccd0 con 0x7f40901039f0 2024-04-20T11:48:11.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.610+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409010b440 con 0x7f40901039f0 2024-04-20T11:48:11.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.612+0000 7f408cc00700 1 -- 172.21.15.117:0/3301512153 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f407c003a60 con 0x7f40901039f0 2024-04-20T11:48:11.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.613+0000 7f408cc00700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f407406ecc0 0x7f4074071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:11.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.613+0000 7f408cc00700 1 -- 172.21.15.117:0/3301512153 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f407c0902e0 con 0x7f40901039f0 2024-04-20T11:48:11.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.613+0000 7f408fe00700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f407406ecc0 0x7f4074071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:11.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.614+0000 7f408fe00700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f407406ecc0 0x7f4074071180 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f408000a990 tx=0x7f4080005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:11.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.617+0000 7f408cc00700 1 -- 172.21.15.117:0/3301512153 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f407c0590c0 con 0x7f40901039f0 2024-04-20T11:48:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:11 smithi173 ceph-mon[74165]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:11 smithi173 ceph-mon[74165]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:11 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/427465314' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:48:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:11 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3092904399' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:48:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:11 smithi117 ceph-mon[99036]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:11 smithi117 ceph-mon[99036]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:11 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/427465314' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:48:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:11 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3092904399' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:48:11.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.917+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4090199ff0 con 0x7f40901039f0 2024-04-20T11:48:11.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.918+0000 7f408cc00700 1 -- 172.21.15.117:0/3301512153 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f407c05c780 con 0x7f40901039f0 2024-04-20T11:48:11.918 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:48:11.918 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:48:11.919 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:48:11.920 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:48:11.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f407406ecc0 msgr2=0x7f4074071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f407406ecc0 0x7f4074071180 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f408000a990 tx=0x7f4080005c80 comp rx=0 tx=0).stop 2024-04-20T11:48:11.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 msgr2=0x7f40901992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:11.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 0x7f40901992b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f407c005b40 tx=0x7f407c00be20 comp rx=0 tx=0).stop 2024-04-20T11:48:11.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 shutdown_connections 2024-04-20T11:48:11.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f407406ecc0 0x7f4074071180 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4090103040 0x7f4090198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 --2- 172.21.15.117:0/3301512153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40901039f0 0x7f40901992b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:11.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 >> 172.21.15.117:0/3301512153 conn(0x7f40900fe8d0 msgr2=0x7f40900fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:11.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 shutdown_connections 2024-04-20T11:48:11.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:11.921+0000 7f4097000700 1 -- 172.21.15.117:0/3301512153 wait complete. 2024-04-20T11:48:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:12 smithi173 ceph-mon[74165]: pgmap v1073: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:12 smithi173 ceph-mon[74165]: from='client.35926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:12 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3301512153' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:48:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:12 smithi117 ceph-mon[99036]: pgmap v1073: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:12 smithi117 ceph-mon[99036]: from='client.35926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:12 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3301512153' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:48:13.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:48:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:48:14.888 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:14 smithi173 ceph-mon[74165]: pgmap v1074: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:14.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:14 smithi117 ceph-mon[99036]: pgmap v1074: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:16 smithi117 ceph-mon[99036]: pgmap v1075: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:16 smithi173 ceph-mon[74165]: pgmap v1075: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:18.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:18 smithi117 ceph-mon[99036]: pgmap v1076: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:18 smithi173 ceph-mon[74165]: pgmap v1076: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:20.694 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:20 smithi117 ceph-mon[99036]: pgmap v1077: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:20 smithi173 ceph-mon[74165]: pgmap v1077: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:22 smithi117 ceph-mon[99036]: pgmap v1078: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:22 smithi173 ceph-mon[74165]: pgmap v1078: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:24.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:24 smithi117 ceph-mon[99036]: pgmap v1079: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:24 smithi173 ceph-mon[74165]: pgmap v1079: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:26.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:26 smithi117 ceph-mon[99036]: pgmap v1080: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:26 smithi173 ceph-mon[74165]: pgmap v1080: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:28.773 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:28 smithi117 ceph-mon[99036]: pgmap v1081: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:28.773 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:48:28.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:28 smithi173 ceph-mon[74165]: pgmap v1081: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:28.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:48:30.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:30 smithi173 ceph-mon[74165]: pgmap v1082: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:30.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:30 smithi117 ceph-mon[99036]: pgmap v1082: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:32.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:32 smithi117 ceph-mon[99036]: pgmap v1083: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:32 smithi173 ceph-mon[74165]: pgmap v1083: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:34.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:34 smithi117 ceph-mon[99036]: pgmap v1084: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:34 smithi173 ceph-mon[74165]: pgmap v1084: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:36.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:36 smithi173 ceph-mon[74165]: pgmap v1085: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:36.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:36 smithi117 ceph-mon[99036]: pgmap v1085: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:38.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:38 smithi117 ceph-mon[99036]: pgmap v1086: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:38 smithi173 ceph-mon[74165]: pgmap v1086: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:40.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:40 smithi117 ceph-mon[99036]: pgmap v1087: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:40.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:40 smithi173 ceph-mon[74165]: pgmap v1087: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:42.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.089+0000 7fdb48800700 1 -- 172.21.15.117:0/557342366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40103110 msgr2=0x7fdb401034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.089+0000 7fdb48800700 1 --2- 172.21.15.117:0/557342366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40103110 0x7fdb401034f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009910 tx=0x7fdb34009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:42.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.089+0000 7fdb48800700 1 -- 172.21.15.117:0/557342366 shutdown_connections 2024-04-20T11:48:42.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.089+0000 7fdb48800700 1 --2- 172.21.15.117:0/557342366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb40104a00 0x7fdb40104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.089+0000 7fdb48800700 1 --2- 172.21.15.117:0/557342366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40103110 0x7fdb401034f0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.089+0000 7fdb48800700 1 -- 172.21.15.117:0/557342366 >> 172.21.15.117:0/557342366 conn(0x7fdb400fe880 msgr2=0x7fdb40100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:42.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.089+0000 7fdb48800700 1 -- 172.21.15.117:0/557342366 shutdown_connections 2024-04-20T11:48:42.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.089+0000 7fdb48800700 1 -- 172.21.15.117:0/557342366 wait complete. 2024-04-20T11:48:42.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.090+0000 7fdb48800700 1 Processor -- start 2024-04-20T11:48:42.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.091+0000 7fdb48800700 1 -- start start 2024-04-20T11:48:42.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.091+0000 7fdb48800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb40103110 0x7fdb40198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.091+0000 7fdb48800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40104a00 0x7fdb40199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.091+0000 7fdb48800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb401999e0 con 0x7fdb40104a00 2024-04-20T11:48:42.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.091+0000 7fdb48800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4019d660 con 0x7fdb40103110 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.091+0000 7fdb45400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40104a00 0x7fdb40199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.091+0000 7fdb45400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40104a00 0x7fdb40199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33340/0 (socket says 172.21.15.117:33340) 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.091+0000 7fdb45400700 1 -- 172.21.15.117:0/502348477 learned_addr learned my addr 172.21.15.117:0/502348477 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.092+0000 7fdb45e00700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb40103110 0x7fdb40198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.092+0000 7fdb45400700 1 -- 172.21.15.117:0/502348477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb40103110 msgr2=0x7fdb40198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.092+0000 7fdb45400700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb40103110 0x7fdb40198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.092+0000 7fdb45400700 1 -- 172.21.15.117:0/502348477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb340095f0 con 0x7fdb40104a00 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.092+0000 7fdb45e00700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb40103110 0x7fdb40198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:48:42.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.092+0000 7fdb45400700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40104a00 0x7fdb40199300 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000d8d0 tx=0x7fdb3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:42.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.093+0000 7fdb3aa00700 1 -- 172.21.15.117:0/502348477 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb30009940 con 0x7fdb40104a00 2024-04-20T11:48:42.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.093+0000 7fdb3aa00700 1 -- 172.21.15.117:0/502348477 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb30010460 con 0x7fdb40104a00 2024-04-20T11:48:42.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.093+0000 7fdb3aa00700 1 -- 172.21.15.117:0/502348477 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3000f5f0 con 0x7fdb40104a00 2024-04-20T11:48:42.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.093+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4019d940 con 0x7fdb40104a00 2024-04-20T11:48:42.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.093+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4019df10 con 0x7fdb40104a00 2024-04-20T11:48:42.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.094+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4010b460 con 0x7fdb40104a00 2024-04-20T11:48:42.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.096+0000 7fdb3aa00700 1 -- 172.21.15.117:0/502348477 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdb300105d0 con 0x7fdb40104a00 2024-04-20T11:48:42.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.097+0000 7fdb3aa00700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdb2806ecd0 0x7fdb28071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.097+0000 7fdb3aa00700 1 -- 172.21.15.117:0/502348477 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdb3008edf0 con 0x7fdb40104a00 2024-04-20T11:48:42.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.097+0000 7fdb45e00700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdb2806ecd0 0x7fdb28071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:42.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.098+0000 7fdb45e00700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdb2806ecd0 0x7fdb28071190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdb34005b40 tx=0x7fdb3401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.103+0000 7fdb3aa00700 1 -- 172.21.15.117:0/502348477 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdb30057bb0 con 0x7fdb40104a00 2024-04-20T11:48:42.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.344+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb4019a080 con 0x7fdb2806ecd0 2024-04-20T11:48:42.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.346+0000 7fdb3aa00700 1 -- 172.21.15.117:0/502348477 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdb4019a080 con 0x7fdb2806ecd0 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdb2806ecd0 msgr2=0x7fdb28071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdb2806ecd0 0x7fdb28071190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdb34005b40 tx=0x7fdb3401a040 comp rx=0 tx=0).stop 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40104a00 msgr2=0x7fdb40199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40104a00 0x7fdb40199300 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000d8d0 tx=0x7fdb3000dc90 comp rx=0 tx=0).stop 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 shutdown_connections 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdb2806ecd0 0x7fdb28071190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb40103110 0x7fdb40198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 --2- 172.21.15.117:0/502348477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb40104a00 0x7fdb40199300 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 >> 172.21.15.117:0/502348477 conn(0x7fdb400fe880 msgr2=0x7fdb400ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:42.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 shutdown_connections 2024-04-20T11:48:42.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.349+0000 7fdb48800700 1 -- 172.21.15.117:0/502348477 wait complete. 2024-04-20T11:48:42.365 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:48:42.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.507+0000 7f5160200700 1 -- 172.21.15.117:0/606795253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158103110 msgr2=0x7f51581034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.507+0000 7f5160200700 1 --2- 172.21.15.117:0/606795253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158103110 0x7f51581034f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f514c009910 tx=0x7f514c009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:42.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.507+0000 7f5160200700 1 -- 172.21.15.117:0/606795253 shutdown_connections 2024-04-20T11:48:42.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.507+0000 7f5160200700 1 --2- 172.21.15.117:0/606795253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5158104a00 0x7f5158104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.507+0000 7f5160200700 1 --2- 172.21.15.117:0/606795253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158103110 0x7f51581034f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.507+0000 7f5160200700 1 -- 172.21.15.117:0/606795253 >> 172.21.15.117:0/606795253 conn(0x7f51580fe880 msgr2=0x7f5158100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:42.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.508+0000 7f5160200700 1 -- 172.21.15.117:0/606795253 shutdown_connections 2024-04-20T11:48:42.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.508+0000 7f5160200700 1 -- 172.21.15.117:0/606795253 wait complete. 2024-04-20T11:48:42.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.509+0000 7f5160200700 1 Processor -- start 2024-04-20T11:48:42.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.509+0000 7f5160200700 1 -- start start 2024-04-20T11:48:42.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f5160200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5158103110 0x7f5158198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f5160200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158104a00 0x7f5158199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f5160200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51581999e0 con 0x7f5158104a00 2024-04-20T11:48:42.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f5160200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515819d660 con 0x7f5158103110 2024-04-20T11:48:42.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f515ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158104a00 0x7f5158199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f515ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158104a00 0x7f5158199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33354/0 (socket says 172.21.15.117:33354) 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f515ce00700 1 -- 172.21.15.117:0/4067068572 learned_addr learned my addr 172.21.15.117:0/4067068572 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f515d800700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5158103110 0x7f5158198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f515ce00700 1 -- 172.21.15.117:0/4067068572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5158103110 msgr2=0x7f5158198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f515ce00700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5158103110 0x7f5158198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.510+0000 7f515ce00700 1 -- 172.21.15.117:0/4067068572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0095f0 con 0x7f5158104a00 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.511+0000 7f515ce00700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158104a00 0x7f5158199300 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f514400d8d0 tx=0x7f514400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.511+0000 7f5156000700 1 -- 172.21.15.117:0/4067068572 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5144009940 con 0x7f5158104a00 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.511+0000 7f5156000700 1 -- 172.21.15.117:0/4067068572 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5144010460 con 0x7f5158104a00 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.511+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515819d940 con 0x7f5158104a00 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.511+0000 7f5156000700 1 -- 172.21.15.117:0/4067068572 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f514400f5f0 con 0x7f5158104a00 2024-04-20T11:48:42.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.511+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515819df10 con 0x7f5158104a00 2024-04-20T11:48:42.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.512+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515810b460 con 0x7f5158104a00 2024-04-20T11:48:42.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.513+0000 7f5156000700 1 -- 172.21.15.117:0/4067068572 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f51440105d0 con 0x7f5158104a00 2024-04-20T11:48:42.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.514+0000 7f5156000700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f514006ecd0 0x7f5140071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.515+0000 7f5156000700 1 -- 172.21.15.117:0/4067068572 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f514408eef0 con 0x7f5158104a00 2024-04-20T11:48:42.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.515+0000 7f515d800700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f514006ecd0 0x7f5140071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:42.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.515+0000 7f515d800700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f514006ecd0 0x7f5140071190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f514c00b5c0 tx=0x7f514c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:42.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.519+0000 7f5156000700 1 -- 172.21.15.117:0/4067068572 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5144057c30 con 0x7f5158104a00 2024-04-20T11:48:42.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:42 smithi117 ceph-mon[99036]: pgmap v1088: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:42.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.760+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f515819a080 con 0x7f514006ecd0 2024-04-20T11:48:42.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.763+0000 7f5156000700 1 -- 172.21.15.117:0/4067068572 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f515819a080 con 0x7f514006ecd0 2024-04-20T11:48:42.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f514006ecd0 msgr2=0x7f5140071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f514006ecd0 0x7f5140071190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f514c00b5c0 tx=0x7f514c01a040 comp rx=0 tx=0).stop 2024-04-20T11:48:42.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158104a00 msgr2=0x7f5158199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158104a00 0x7f5158199300 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f514400d8d0 tx=0x7f514400dc90 comp rx=0 tx=0).stop 2024-04-20T11:48:42.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 shutdown_connections 2024-04-20T11:48:42.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f514006ecd0 0x7f5140071190 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5158103110 0x7f5158198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 --2- 172.21.15.117:0/4067068572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5158104a00 0x7f5158199300 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.765+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 >> 172.21.15.117:0/4067068572 conn(0x7f51580fe880 msgr2=0x7f51580ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:42.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.766+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 shutdown_connections 2024-04-20T11:48:42.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.766+0000 7f5160200700 1 -- 172.21.15.117:0/4067068572 wait complete. 2024-04-20T11:48:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:42 smithi173 ceph-mon[74165]: pgmap v1088: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:42.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.913+0000 7f5775400700 1 -- 172.21.15.117:0/616592022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770104a00 msgr2=0x7f5770104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.913+0000 7f5775400700 1 --2- 172.21.15.117:0/616592022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770104a00 0x7f5770104e80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f575c009910 tx=0x7f575c009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:42.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.913+0000 7f5775400700 1 -- 172.21.15.117:0/616592022 shutdown_connections 2024-04-20T11:48:42.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.913+0000 7f5775400700 1 --2- 172.21.15.117:0/616592022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770104a00 0x7f5770104e80 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.913+0000 7f5775400700 1 --2- 172.21.15.117:0/616592022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5770103110 0x7f57701034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.913+0000 7f5775400700 1 -- 172.21.15.117:0/616592022 >> 172.21.15.117:0/616592022 conn(0x7f57700fe880 msgr2=0x7f5770100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:42.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.913+0000 7f5775400700 1 -- 172.21.15.117:0/616592022 shutdown_connections 2024-04-20T11:48:42.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.913+0000 7f5775400700 1 -- 172.21.15.117:0/616592022 wait complete. 2024-04-20T11:48:42.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.914+0000 7f5775400700 1 Processor -- start 2024-04-20T11:48:42.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.914+0000 7f5775400700 1 -- start start 2024-04-20T11:48:42.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f5775400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770103110 0x7f577019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f5775400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5770104a00 0x7f577019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f5775400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577019f5b0 con 0x7f5770103110 2024-04-20T11:48:42.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f5775400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770198b90 con 0x7f5770104a00 2024-04-20T11:48:42.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f576e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770103110 0x7f577019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:42.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f576e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770103110 0x7f577019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33368/0 (socket says 172.21.15.117:33368) 2024-04-20T11:48:42.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f576e800700 1 -- 172.21.15.117:0/1386735672 learned_addr learned my addr 172.21.15.117:0/1386735672 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:42.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f576e800700 1 -- 172.21.15.117:0/1386735672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5770104a00 msgr2=0x7f577019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:42.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f576e800700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5770104a00 0x7f577019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:42.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.915+0000 7f576e800700 1 -- 172.21.15.117:0/1386735672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c0095f0 con 0x7f5770103110 2024-04-20T11:48:42.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.916+0000 7f576e800700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770103110 0x7f577019ea50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f576000d8d0 tx=0x7f576000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:42.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.916+0000 7f5767400700 1 -- 172.21.15.117:0/1386735672 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5760009940 con 0x7f5770103110 2024-04-20T11:48:42.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.916+0000 7f5767400700 1 -- 172.21.15.117:0/1386735672 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5760010460 con 0x7f5770103110 2024-04-20T11:48:42.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.916+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5770198e70 con 0x7f5770103110 2024-04-20T11:48:42.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.916+0000 7f5767400700 1 -- 172.21.15.117:0/1386735672 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f576000f5f0 con 0x7f5770103110 2024-04-20T11:48:42.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.916+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5770199440 con 0x7f5770103110 2024-04-20T11:48:42.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.917+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577010b480 con 0x7f5770103110 2024-04-20T11:48:42.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.918+0000 7f5767400700 1 -- 172.21.15.117:0/1386735672 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f57600105d0 con 0x7f5770103110 2024-04-20T11:48:42.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.919+0000 7f5767400700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f575406ecd0 0x7f5754071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:42.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.919+0000 7f5767400700 1 -- 172.21.15.117:0/1386735672 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f576008eef0 con 0x7f5770103110 2024-04-20T11:48:42.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.919+0000 7f576de00700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f575406ecd0 0x7f5754071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:42.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.920+0000 7f576de00700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f575406ecd0 0x7f5754071190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f575c006010 tx=0x7f575c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:42.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:42.924+0000 7f5767400700 1 -- 172.21.15.117:0/1386735672 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f576005b760 con 0x7f5770103110 2024-04-20T11:48:43.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.145+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f577019a080 con 0x7f575406ecd0 2024-04-20T11:48:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.173+0000 7f5767400700 1 -- 172.21.15.117:0/1386735672 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f577019a080 con 0x7f575406ecd0 2024-04-20T11:48:43.174 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (37m) 13m ago 51m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (51m) 13m ago 51m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (50m) 13m ago 50m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (33m) 13m ago 51m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (33m) 13m ago 50m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (36m) 13m ago 50m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (28m) 13m ago 43m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (29m) 13m ago 44m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (13m) 13m ago 43m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (28m) 13m ago 44m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (40m) 13m ago 53m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (38m) 13m ago 49m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (35m) 13m ago 53m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:48:43.175 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (34m) 13m ago 49m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (38m) 13m ago 51m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (38m) 13m ago 50m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (32m) 13m ago 48m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (32m) 13m ago 48m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (31m) 13m ago 47m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (31m) 13m ago 47m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (30m) 13m ago 46m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (29m) 13m ago 45m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:48:43.176 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (37m) 13m ago 50m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f575406ecd0 msgr2=0x7f5754071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f575406ecd0 0x7f5754071190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f575c006010 tx=0x7f575c00b540 comp rx=0 tx=0).stop 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770103110 msgr2=0x7f577019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770103110 0x7f577019ea50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f576000d8d0 tx=0x7f576000dc90 comp rx=0 tx=0).stop 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 shutdown_connections 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f575406ecd0 0x7f5754071190 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5770103110 0x7f577019ea50 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 --2- 172.21.15.117:0/1386735672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5770104a00 0x7f577019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.176+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 >> 172.21.15.117:0/1386735672 conn(0x7f57700fe880 msgr2=0x7f57700ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:43.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.177+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 shutdown_connections 2024-04-20T11:48:43.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.177+0000 7f5775400700 1 -- 172.21.15.117:0/1386735672 wait complete. 2024-04-20T11:48:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.323+0000 7fe706a00700 1 -- 172.21.15.117:0/1962679324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7001017c0 msgr2=0x7fe700105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.323+0000 7fe706a00700 1 --2- 172.21.15.117:0/1962679324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7001017c0 0x7fe700105810 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009910 tx=0x7fe6ec009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.323+0000 7fe706a00700 1 -- 172.21.15.117:0/1962679324 shutdown_connections 2024-04-20T11:48:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.323+0000 7fe706a00700 1 --2- 172.21.15.117:0/1962679324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7001017c0 0x7fe700105810 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.323+0000 7fe706a00700 1 --2- 172.21.15.117:0/1962679324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe700100e10 0x7fe7001011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.323+0000 7fe706a00700 1 -- 172.21.15.117:0/1962679324 >> 172.21.15.117:0/1962679324 conn(0x7fe7000fc6a0 msgr2=0x7fe7000feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:43.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.323+0000 7fe706a00700 1 -- 172.21.15.117:0/1962679324 shutdown_connections 2024-04-20T11:48:43.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.323+0000 7fe706a00700 1 -- 172.21.15.117:0/1962679324 wait complete. 2024-04-20T11:48:43.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.324+0000 7fe706a00700 1 Processor -- start 2024-04-20T11:48:43.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe706a00700 1 -- start start 2024-04-20T11:48:43.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe706a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe700100e10 0x7fe7000731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:43.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe706a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7001017c0 0x7fe700073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:43.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe706a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700073c60 con 0x7fe700100e10 2024-04-20T11:48:43.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe706a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700073da0 con 0x7fe7001017c0 2024-04-20T11:48:43.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe6ffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe700100e10 0x7fe7000731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:43.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe6ffe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe700100e10 0x7fe7000731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33384/0 (socket says 172.21.15.117:33384) 2024-04-20T11:48:43.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe6ffe00700 1 -- 172.21.15.117:0/2245819990 learned_addr learned my addr 172.21.15.117:0/2245819990 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:43.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe6ffe00700 1 -- 172.21.15.117:0/2245819990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7001017c0 msgr2=0x7fe700073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:43.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.325+0000 7fe6ffe00700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7001017c0 0x7fe700073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.326+0000 7fe6ffe00700 1 -- 172.21.15.117:0/2245819990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec0095f0 con 0x7fe700100e10 2024-04-20T11:48:43.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.326+0000 7fe6ffe00700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe700100e10 0x7fe7000731e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000d8d0 tx=0x7fe6f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:43.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.326+0000 7fe6fcc00700 1 -- 172.21.15.117:0/2245819990 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f0009940 con 0x7fe700100e10 2024-04-20T11:48:43.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.326+0000 7fe6fcc00700 1 -- 172.21.15.117:0/2245819990 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6f0010460 con 0x7fe700100e10 2024-04-20T11:48:43.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.326+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7001a2d10 con 0x7fe700100e10 2024-04-20T11:48:43.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.326+0000 7fe6fcc00700 1 -- 172.21.15.117:0/2245819990 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f000f5f0 con 0x7fe700100e10 2024-04-20T11:48:43.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.326+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7001a3070 con 0x7fe700100e10 2024-04-20T11:48:43.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.327+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe700109210 con 0x7fe700100e10 2024-04-20T11:48:43.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.328+0000 7fe6fcc00700 1 -- 172.21.15.117:0/2245819990 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe6f00105d0 con 0x7fe700100e10 2024-04-20T11:48:43.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.330+0000 7fe6fcc00700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6e406ecd0 0x7fe6e4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:43.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.330+0000 7fe6fcc00700 1 -- 172.21.15.117:0/2245819990 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe6f008eee0 con 0x7fe700100e10 2024-04-20T11:48:43.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.330+0000 7fe6ff400700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6e406ecd0 0x7fe6e4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:43.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.331+0000 7fe6ff400700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6e406ecd0 0x7fe6e4071190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec006010 tx=0x7fe6ec00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:43.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.335+0000 7fe6fcc00700 1 -- 172.21.15.117:0/2245819990 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe6f0057c20 con 0x7fe700100e10 2024-04-20T11:48:43.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.639+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe70010ddb0 con 0x7fe700100e10 2024-04-20T11:48:43.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.640+0000 7fe6fcc00700 1 -- 172.21.15.117:0/2245819990 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe6f005b2e0 con 0x7fe700100e10 2024-04-20T11:48:43.642 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:48:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:48:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:48:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:48:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:48:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:48:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:48:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:48:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:48:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:48:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:48:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:48:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:48:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:48:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:48:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:48:43.644 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:48:43.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.643+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6e406ecd0 msgr2=0x7fe6e4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:43.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.643+0000 7fe706a00700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6e406ecd0 0x7fe6e4071190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec006010 tx=0x7fe6ec00b540 comp rx=0 tx=0).stop 2024-04-20T11:48:43.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.643+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe700100e10 msgr2=0x7fe7000731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:43.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.643+0000 7fe706a00700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe700100e10 0x7fe7000731e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000d8d0 tx=0x7fe6f000dc90 comp rx=0 tx=0).stop 2024-04-20T11:48:43.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.644+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 shutdown_connections 2024-04-20T11:48:43.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.644+0000 7fe706a00700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe6e406ecd0 0x7fe6e4071190 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.644+0000 7fe706a00700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe700100e10 0x7fe7000731e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.644+0000 7fe706a00700 1 --2- 172.21.15.117:0/2245819990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7001017c0 0x7fe700073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.644+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 >> 172.21.15.117:0/2245819990 conn(0x7fe7000fc6a0 msgr2=0x7fe7000fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:43.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.644+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 shutdown_connections 2024-04-20T11:48:43.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.644+0000 7fe706a00700 1 -- 172.21.15.117:0/2245819990 wait complete. 2024-04-20T11:48:43.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.796+0000 7fbf15e00700 1 -- 172.21.15.117:0/1280832106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 msgr2=0x7fbf10107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:43.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.796+0000 7fbf15e00700 1 --2- 172.21.15.117:0/1280832106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 0x7fbf10107900 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009910 tx=0x7fbf00009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:43.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.796+0000 7fbf15e00700 1 -- 172.21.15.117:0/1280832106 shutdown_connections 2024-04-20T11:48:43.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.796+0000 7fbf15e00700 1 --2- 172.21.15.117:0/1280832106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf10108e10 0x7fbf10109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.796+0000 7fbf15e00700 1 --2- 172.21.15.117:0/1280832106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 0x7fbf10107900 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.796+0000 7fbf15e00700 1 -- 172.21.15.117:0/1280832106 >> 172.21.15.117:0/1280832106 conn(0x7fbf100768f0 msgr2=0x7fbf10078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:43.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:43 smithi117 ceph-mon[99036]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:43.797 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:48:43.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.796+0000 7fbf15e00700 1 -- 172.21.15.117:0/1280832106 shutdown_connections 2024-04-20T11:48:43.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.796+0000 7fbf15e00700 1 -- 172.21.15.117:0/1280832106 wait complete. 2024-04-20T11:48:43.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.797+0000 7fbf15e00700 1 Processor -- start 2024-04-20T11:48:43.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.797+0000 7fbf15e00700 1 -- start start 2024-04-20T11:48:43.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.797+0000 7fbf15e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 0x7fbf1019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:43.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.797+0000 7fbf15e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf10108e10 0x7fbf1019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:43.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf15e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1019ddd0 con 0x7fbf10107520 2024-04-20T11:48:43.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf15e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf101a1aa0 con 0x7fbf10108e10 2024-04-20T11:48:43.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf0ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 0x7fbf1019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:43.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf0ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 0x7fbf1019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33402/0 (socket says 172.21.15.117:33402) 2024-04-20T11:48:43.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf0ea00700 1 -- 172.21.15.117:0/3908880688 learned_addr learned my addr 172.21.15.117:0/3908880688 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:43.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf0e000700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf10108e10 0x7fbf1019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:43.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf0ea00700 1 -- 172.21.15.117:0/3908880688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf10108e10 msgr2=0x7fbf1019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:43.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf0ea00700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf10108e10 0x7fbf1019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:43.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf0ea00700 1 -- 172.21.15.117:0/3908880688 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000095f0 con 0x7fbf10107520 2024-04-20T11:48:43.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.798+0000 7fbf0e000700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf10108e10 0x7fbf1019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:48:43.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.799+0000 7fbf0ea00700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 0x7fbf1019d1b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fbf000092e0 tx=0x7fbf0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:43.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.799+0000 7fbf14a00700 1 -- 172.21.15.117:0/3908880688 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf0001d070 con 0x7fbf10107520 2024-04-20T11:48:43.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.799+0000 7fbf14a00700 1 -- 172.21.15.117:0/3908880688 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf0000f460 con 0x7fbf10107520 2024-04-20T11:48:43.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.799+0000 7fbf14a00700 1 -- 172.21.15.117:0/3908880688 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf00017630 con 0x7fbf10107520 2024-04-20T11:48:43.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.799+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf101a1d20 con 0x7fbf10107520 2024-04-20T11:48:43.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.799+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf101a2290 con 0x7fbf10107520 2024-04-20T11:48:43.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.800+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1010f890 con 0x7fbf10107520 2024-04-20T11:48:43.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.802+0000 7fbf14a00700 1 -- 172.21.15.117:0/3908880688 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbf0000f5d0 con 0x7fbf10107520 2024-04-20T11:48:43.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.802+0000 7fbf14a00700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbef406ecd0 0x7fbef4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:43.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.803+0000 7fbf14a00700 1 -- 172.21.15.117:0/3908880688 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbf00090d00 con 0x7fbf10107520 2024-04-20T11:48:43.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.803+0000 7fbf0e000700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbef406ecd0 0x7fbef4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:43.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.803+0000 7fbf0e000700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbef406ecd0 0x7fbef4071190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbefc005fd0 tx=0x7fbefc005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:43.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:43.806+0000 7fbf14a00700 1 -- 172.21.15.117:0/3908880688 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbf00059be0 con 0x7fbf10107520 2024-04-20T11:48:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:43 smithi173 ceph-mon[74165]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:48:44.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.080+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf1019e4c0 con 0x7fbf10107520 2024-04-20T11:48:44.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.081+0000 7fbf14a00700 1 -- 172.21.15.117:0/3908880688 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fbf0005d2a0 con 0x7fbf10107520 2024-04-20T11:48:44.083 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:48:44.083 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:48:44.083 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:48:44.083 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:48:44.083 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:48:44.083 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:48:44.084 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:48:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:48:44.086 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:48:44.086 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:48:44.086 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:48:44.086 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:48:44.086 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:48:44.086 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:48:44.086 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:48:44.086 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbef406ecd0 msgr2=0x7fbef4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbef406ecd0 0x7fbef4071190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbefc005fd0 tx=0x7fbefc005dc0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 msgr2=0x7fbf1019d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 0x7fbf1019d1b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fbf000092e0 tx=0x7fbf0000be20 comp rx=0 tx=0).stop 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 shutdown_connections 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbef406ecd0 0x7fbef4071190 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf10107520 0x7fbf1019d1b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 --2- 172.21.15.117:0/3908880688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf10108e10 0x7fbf1019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 >> 172.21.15.117:0/3908880688 conn(0x7fbf100768f0 msgr2=0x7fbf10077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:44.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 shutdown_connections 2024-04-20T11:48:44.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.086+0000 7fbf15e00700 1 -- 172.21.15.117:0/3908880688 wait complete. 2024-04-20T11:48:44.088 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:48:44.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.244+0000 7f93eca00700 1 -- 172.21.15.117:0/3283752129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 msgr2=0x7f93e8107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.244+0000 7f93eca00700 1 --2- 172.21.15.117:0/3283752129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 0x7f93e8107900 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009910 tx=0x7f93d8009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:44.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.244+0000 7f93eca00700 1 -- 172.21.15.117:0/3283752129 shutdown_connections 2024-04-20T11:48:44.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.244+0000 7f93eca00700 1 --2- 172.21.15.117:0/3283752129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8108e10 0x7f93e8109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.244+0000 7f93eca00700 1 --2- 172.21.15.117:0/3283752129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 0x7f93e8107900 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.244+0000 7f93eca00700 1 -- 172.21.15.117:0/3283752129 >> 172.21.15.117:0/3283752129 conn(0x7f93e80768f0 msgr2=0x7f93e8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:44.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.244+0000 7f93eca00700 1 -- 172.21.15.117:0/3283752129 shutdown_connections 2024-04-20T11:48:44.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.245+0000 7f93eca00700 1 -- 172.21.15.117:0/3283752129 wait complete. 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.245+0000 7f93eca00700 1 Processor -- start 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.246+0000 7f93eca00700 1 -- start start 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.246+0000 7f93eca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 0x7f93e819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.246+0000 7f93eca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8108e10 0x7f93e819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.246+0000 7f93eca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e819ddd0 con 0x7f93e8107520 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.246+0000 7f93eca00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81a1aa0 con 0x7f93e8108e10 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.246+0000 7f93e5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 0x7f93e819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.246+0000 7f93e5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 0x7f93e819d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33024/0 (socket says 172.21.15.117:33024) 2024-04-20T11:48:44.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.246+0000 7f93e5e00700 1 -- 172.21.15.117:0/375499011 learned_addr learned my addr 172.21.15.117:0/375499011 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:44.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93e5400700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8108e10 0x7f93e819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:44.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93e5e00700 1 -- 172.21.15.117:0/375499011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8108e10 msgr2=0x7f93e819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93e5e00700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8108e10 0x7f93e819d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93e5e00700 1 -- 172.21.15.117:0/375499011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80095f0 con 0x7f93e8107520 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93e5400700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8108e10 0x7f93e819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93e5e00700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 0x7f93e819d1b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f93d800bd70 tx=0x7f93d800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93dea00700 1 -- 172.21.15.117:0/375499011 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d801d070 con 0x7f93e8107520 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93dea00700 1 -- 172.21.15.117:0/375499011 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93d800f460 con 0x7f93e8107520 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e81a1d20 con 0x7f93e8107520 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.247+0000 7f93dea00700 1 -- 172.21.15.117:0/375499011 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d8017630 con 0x7f93e8107520 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.248+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e81a2290 con 0x7f93e8107520 2024-04-20T11:48:44.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.249+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e810f890 con 0x7f93e8107520 2024-04-20T11:48:44.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.249+0000 7f93dea00700 1 -- 172.21.15.117:0/375499011 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f93d800f5d0 con 0x7f93e8107520 2024-04-20T11:48:44.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.250+0000 7f93dea00700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ecc0 0x7f93cc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:44.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.251+0000 7f93dea00700 1 -- 172.21.15.117:0/375499011 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f93d8090300 con 0x7f93e8107520 2024-04-20T11:48:44.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.251+0000 7f93e5400700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ecc0 0x7f93cc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:44.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.251+0000 7f93e5400700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ecc0 0x7f93cc071180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f93d4005fd0 tx=0x7f93d4005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:44.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.255+0000 7f93dea00700 1 -- 172.21.15.117:0/375499011 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f93d80590c0 con 0x7f93e8107520 2024-04-20T11:48:44.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.525+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93e819e530 con 0x7f93cc06ecc0 2024-04-20T11:48:44.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.528+0000 7f93dea00700 1 -- 172.21.15.117:0/375499011 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f93e819e530 con 0x7f93cc06ecc0 2024-04-20T11:48:44.528 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:48:44.528 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:48:44.529 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:48:44.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ecc0 msgr2=0x7f93cc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ecc0 0x7f93cc071180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f93d4005fd0 tx=0x7f93d4005dc0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 msgr2=0x7f93e819d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 0x7f93e819d1b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f93d800bd70 tx=0x7f93d800be50 comp rx=0 tx=0).stop 2024-04-20T11:48:44.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 shutdown_connections 2024-04-20T11:48:44.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f93cc06ecc0 0x7f93cc071180 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93e8107520 0x7f93e819d1b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 --2- 172.21.15.117:0/375499011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8108e10 0x7f93e819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 >> 172.21.15.117:0/375499011 conn(0x7f93e80768f0 msgr2=0x7f93e8077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:44.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 shutdown_connections 2024-04-20T11:48:44.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.531+0000 7f93eca00700 1 -- 172.21.15.117:0/375499011 wait complete. 2024-04-20T11:48:44.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.683+0000 7f2169c00700 1 -- 172.21.15.117:0/3247416878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 msgr2=0x7f21641034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.683+0000 7f2169c00700 1 --2- 172.21.15.117:0/3247416878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 0x7f21641034f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2154009910 tx=0x7f2154009c20 comp rx=0 tx=0).stop 2024-04-20T11:48:44.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.683+0000 7f2169c00700 1 -- 172.21.15.117:0/3247416878 shutdown_connections 2024-04-20T11:48:44.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.683+0000 7f2169c00700 1 --2- 172.21.15.117:0/3247416878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2164104a00 0x7f2164104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.683+0000 7f2169c00700 1 --2- 172.21.15.117:0/3247416878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 0x7f21641034f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.683+0000 7f2169c00700 1 -- 172.21.15.117:0/3247416878 >> 172.21.15.117:0/3247416878 conn(0x7f21640fe880 msgr2=0x7f2164100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:44.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.683+0000 7f2169c00700 1 -- 172.21.15.117:0/3247416878 shutdown_connections 2024-04-20T11:48:44.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.683+0000 7f2169c00700 1 -- 172.21.15.117:0/3247416878 wait complete. 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.684+0000 7f2169c00700 1 Processor -- start 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.685+0000 7f2169c00700 1 -- start start 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.685+0000 7f2169c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 0x7f2164198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.685+0000 7f2169c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2164104a00 0x7f2164199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.685+0000 7f2169c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21641999e0 con 0x7f2164103110 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.685+0000 7f2169c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216419d660 con 0x7f2164104a00 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.685+0000 7f2162a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 0x7f2164198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.685+0000 7f2162a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 0x7f2164198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33038/0 (socket says 172.21.15.117:33038) 2024-04-20T11:48:44.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.685+0000 7f2162a00700 1 -- 172.21.15.117:0/2295013802 learned_addr learned my addr 172.21.15.117:0/2295013802 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:48:44.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.686+0000 7f2162000700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2164104a00 0x7f2164199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:44.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.686+0000 7f2162a00700 1 -- 172.21.15.117:0/2295013802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2164104a00 msgr2=0x7f2164199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.686+0000 7f2162a00700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2164104a00 0x7f2164199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.686+0000 7f2162a00700 1 -- 172.21.15.117:0/2295013802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21540095f0 con 0x7f2164103110 2024-04-20T11:48:44.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.686+0000 7f2162a00700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 0x7f2164198dc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f21540092e0 tx=0x7f215400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:44.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.686+0000 7f215b400700 1 -- 172.21.15.117:0/2295013802 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215401d070 con 0x7f2164103110 2024-04-20T11:48:44.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.686+0000 7f215b400700 1 -- 172.21.15.117:0/2295013802 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21540056e0 con 0x7f2164103110 2024-04-20T11:48:44.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.687+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216419d8e0 con 0x7f2164103110 2024-04-20T11:48:44.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.687+0000 7f215b400700 1 -- 172.21.15.117:0/2295013802 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215400fe80 con 0x7f2164103110 2024-04-20T11:48:44.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.687+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216419de50 con 0x7f2164103110 2024-04-20T11:48:44.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.688+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216410b480 con 0x7f2164103110 2024-04-20T11:48:44.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.689+0000 7f215b400700 1 -- 172.21.15.117:0/2295013802 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f215400f6f0 con 0x7f2164103110 2024-04-20T11:48:44.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.690+0000 7f215b400700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f214806ecd0 0x7f2148071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:48:44.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.690+0000 7f215b400700 1 -- 172.21.15.117:0/2295013802 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2154090340 con 0x7f2164103110 2024-04-20T11:48:44.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.690+0000 7f2162000700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f214806ecd0 0x7f2148071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:48:44.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.690+0000 7f2162000700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f214806ecd0 0x7f2148071190 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2150005950 tx=0x7f21500058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:48:44.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.694+0000 7f215b400700 1 -- 172.21.15.117:0/2295013802 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2154059080 con 0x7f2164103110 2024-04-20T11:48:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:44 smithi117 ceph-mon[99036]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:44 smithi117 ceph-mon[99036]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:44 smithi117 ceph-mon[99036]: pgmap v1089: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:44 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2245819990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:48:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:44 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3908880688' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:48:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:44 smithi173 ceph-mon[74165]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:44 smithi173 ceph-mon[74165]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:44 smithi173 ceph-mon[74165]: pgmap v1089: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:44 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2245819990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:48:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:44 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3908880688' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:48:44.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.988+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f216419a080 con 0x7f2164103110 2024-04-20T11:48:44.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.988+0000 7f215b400700 1 -- 172.21.15.117:0/2295013802 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f215405c740 con 0x7f2164103110 2024-04-20T11:48:44.989 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:48:44.989 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:48:44.990 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:48:44.991 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:48:44.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f214806ecd0 msgr2=0x7f2148071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f214806ecd0 0x7f2148071190 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2150005950 tx=0x7f21500058e0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 msgr2=0x7f2164198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:48:44.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 0x7f2164198dc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f21540092e0 tx=0x7f215400be20 comp rx=0 tx=0).stop 2024-04-20T11:48:44.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 shutdown_connections 2024-04-20T11:48:44.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f214806ecd0 0x7f2148071190 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2164103110 0x7f2164198dc0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 --2- 172.21.15.117:0/2295013802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2164104a00 0x7f2164199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:48:44.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.991+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 >> 172.21.15.117:0/2295013802 conn(0x7f21640fe880 msgr2=0x7f21640ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:48:44.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.992+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 shutdown_connections 2024-04-20T11:48:44.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:48:44.992+0000 7f2169c00700 1 -- 172.21.15.117:0/2295013802 wait complete. 2024-04-20T11:48:45.855 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:45 smithi173 ceph-mon[74165]: from='client.35954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:45.855 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:45 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2295013802' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:48:45.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:45 smithi117 ceph-mon[99036]: from='client.35954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:48:45.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:45 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2295013802' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:48:46.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:46 smithi117 ceph-mon[99036]: pgmap v1090: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:46 smithi173 ceph-mon[74165]: pgmap v1090: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:48.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:48 smithi117 ceph-mon[99036]: pgmap v1091: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:48 smithi173 ceph-mon[74165]: pgmap v1091: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:49.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:49 smithi117 ceph-mon[99036]: pgmap v1092: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:49 smithi173 ceph-mon[74165]: pgmap v1092: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:52.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:52 smithi117 ceph-mon[99036]: pgmap v1093: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:52.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:52 smithi173 ceph-mon[74165]: pgmap v1093: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:54.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:54 smithi117 ceph-mon[99036]: pgmap v1094: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:54.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:54 smithi173 ceph-mon[74165]: pgmap v1094: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:56.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:56 smithi117 ceph-mon[99036]: pgmap v1095: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:56.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:56 smithi173 ceph-mon[74165]: pgmap v1095: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:58 smithi117 ceph-mon[99036]: pgmap v1096: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:48:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:48:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:58 smithi173 ceph-mon[74165]: pgmap v1096: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:48:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:48:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:49:00.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:00 smithi117 ceph-mon[99036]: pgmap v1097: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:00 smithi173 ceph-mon[74165]: pgmap v1097: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:02 smithi173 ceph-mon[74165]: pgmap v1098: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:02.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:02 smithi117 ceph-mon[99036]: pgmap v1098: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:04.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:04 smithi173 ceph-mon[74165]: pgmap v1099: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:04.873 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:04 smithi117 ceph-mon[99036]: pgmap v1099: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:06.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:06 smithi173 ceph-mon[74165]: pgmap v1100: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:06 smithi117 ceph-mon[99036]: pgmap v1100: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:08.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:08 smithi117 ceph-mon[99036]: pgmap v1101: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:08 smithi173 ceph-mon[74165]: pgmap v1101: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:10.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:10 smithi117 ceph-mon[99036]: pgmap v1102: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:10.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:10 smithi173 ceph-mon[74165]: pgmap v1102: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:12 smithi117 ceph-mon[99036]: pgmap v1103: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:12 smithi173 ceph-mon[74165]: pgmap v1103: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:49:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:49:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:14 smithi117 ceph-mon[99036]: pgmap v1104: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:14 smithi173 ceph-mon[74165]: pgmap v1104: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:15.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.161+0000 7f8ec0c00700 1 -- 172.21.15.117:0/1022585059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc104a00 msgr2=0x7f8ebc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.161+0000 7f8ec0c00700 1 --2- 172.21.15.117:0/1022585059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc104a00 0x7f8ebc104e80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009910 tx=0x7f8ea4009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:15.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.162+0000 7f8ec0c00700 1 -- 172.21.15.117:0/1022585059 shutdown_connections 2024-04-20T11:49:15.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.162+0000 7f8ec0c00700 1 --2- 172.21.15.117:0/1022585059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc104a00 0x7f8ebc104e80 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.162+0000 7f8ec0c00700 1 --2- 172.21.15.117:0/1022585059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ebc103110 0x7f8ebc1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.162+0000 7f8ec0c00700 1 -- 172.21.15.117:0/1022585059 >> 172.21.15.117:0/1022585059 conn(0x7f8ebc0fe880 msgr2=0x7f8ebc100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:15.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.162+0000 7f8ec0c00700 1 -- 172.21.15.117:0/1022585059 shutdown_connections 2024-04-20T11:49:15.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.162+0000 7f8ec0c00700 1 -- 172.21.15.117:0/1022585059 wait complete. 2024-04-20T11:49:15.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.163+0000 7f8ec0c00700 1 Processor -- start 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.163+0000 7f8ec0c00700 1 -- start start 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.163+0000 7f8ec0c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc103110 0x7f8ebc19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.163+0000 7f8ec0c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ebc104a00 0x7f8ebc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.163+0000 7f8ec0c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19f5b0 con 0x7f8ebc103110 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.163+0000 7f8ec0c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc198b90 con 0x7f8ebc104a00 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.164+0000 7f8eb9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc103110 0x7f8ebc19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.164+0000 7f8eb9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc103110 0x7f8ebc19ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44896/0 (socket says 172.21.15.117:44896) 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.164+0000 7f8eb9e00700 1 -- 172.21.15.117:0/2624046221 learned_addr learned my addr 172.21.15.117:0/2624046221 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.164+0000 7f8eb1400700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ebc104a00 0x7f8ebc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.164+0000 7f8eb9e00700 1 -- 172.21.15.117:0/2624046221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ebc104a00 msgr2=0x7f8ebc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.164+0000 7f8eb9e00700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ebc104a00 0x7f8ebc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.164+0000 7f8eb9e00700 1 -- 172.21.15.117:0/2624046221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea40095f0 con 0x7f8ebc103110 2024-04-20T11:49:15.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.165+0000 7f8eb9e00700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc103110 0x7f8ebc19ea50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ada0 tx=0x7f8eac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:15.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.165+0000 7f8eb3400700 1 -- 172.21.15.117:0/2624046221 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eac00f070 con 0x7f8ebc103110 2024-04-20T11:49:15.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.165+0000 7f8eb3400700 1 -- 172.21.15.117:0/2624046221 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8eac007bf0 con 0x7f8ebc103110 2024-04-20T11:49:15.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.165+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc198e70 con 0x7f8ebc103110 2024-04-20T11:49:15.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.165+0000 7f8eb3400700 1 -- 172.21.15.117:0/2624046221 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eac005690 con 0x7f8ebc103110 2024-04-20T11:49:15.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.165+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc199440 con 0x7f8ebc103110 2024-04-20T11:49:15.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.166+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc10b480 con 0x7f8ebc103110 2024-04-20T11:49:15.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.167+0000 7f8eb3400700 1 -- 172.21.15.117:0/2624046221 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8eac005900 con 0x7f8ebc103110 2024-04-20T11:49:15.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.168+0000 7f8eb3400700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ea006ecc0 0x7f8ea0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:15.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.168+0000 7f8eb3400700 1 -- 172.21.15.117:0/2624046221 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8eac08e910 con 0x7f8ebc103110 2024-04-20T11:49:15.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.168+0000 7f8eb1400700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ea006ecc0 0x7f8ea0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:15.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.169+0000 7f8eb1400700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ea006ecc0 0x7f8ea0071180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4005f50 tx=0x7f8ea401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:15.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.173+0000 7f8eb3400700 1 -- 172.21.15.117:0/2624046221 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8eac05b180 con 0x7f8ebc103110 2024-04-20T11:49:15.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.425+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ebc199fe0 con 0x7f8ea006ecc0 2024-04-20T11:49:15.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.427+0000 7f8eb3400700 1 -- 172.21.15.117:0/2624046221 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8ebc199fe0 con 0x7f8ea006ecc0 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ea006ecc0 msgr2=0x7f8ea0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ea006ecc0 0x7f8ea0071180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4005f50 tx=0x7f8ea401a040 comp rx=0 tx=0).stop 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc103110 msgr2=0x7f8ebc19ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc103110 0x7f8ebc19ea50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ada0 tx=0x7f8eac00c330 comp rx=0 tx=0).stop 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 shutdown_connections 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8ea006ecc0 0x7f8ea0071180 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ebc103110 0x7f8ebc19ea50 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 --2- 172.21.15.117:0/2624046221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ebc104a00 0x7f8ebc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 >> 172.21.15.117:0/2624046221 conn(0x7f8ebc0fe880 msgr2=0x7f8ebc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 shutdown_connections 2024-04-20T11:49:15.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.431+0000 7f8ec0c00700 1 -- 172.21.15.117:0/2624046221 wait complete. 2024-04-20T11:49:15.448 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:49:15.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.591+0000 7fcde0200700 1 -- 172.21.15.117:0/715895738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 msgr2=0x7fcdd8103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.591+0000 7fcde0200700 1 --2- 172.21.15.117:0/715895738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 0x7fcdd8103420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009910 tx=0x7fcdcc009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:15.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.592+0000 7fcde0200700 1 -- 172.21.15.117:0/715895738 shutdown_connections 2024-04-20T11:49:15.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.592+0000 7fcde0200700 1 --2- 172.21.15.117:0/715895738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdd81039f0 0x7fcdd8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.592+0000 7fcde0200700 1 --2- 172.21.15.117:0/715895738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 0x7fcdd8103420 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.592+0000 7fcde0200700 1 -- 172.21.15.117:0/715895738 >> 172.21.15.117:0/715895738 conn(0x7fcdd80fe8b0 msgr2=0x7fcdd8100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:15.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.592+0000 7fcde0200700 1 -- 172.21.15.117:0/715895738 shutdown_connections 2024-04-20T11:49:15.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.592+0000 7fcde0200700 1 -- 172.21.15.117:0/715895738 wait complete. 2024-04-20T11:49:15.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.593+0000 7fcde0200700 1 Processor -- start 2024-04-20T11:49:15.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.593+0000 7fcde0200700 1 -- start start 2024-04-20T11:49:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.593+0000 7fcde0200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 0x7fcdd8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.593+0000 7fcde0200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdd81039f0 0x7fcdd8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.593+0000 7fcde0200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd81998d0 con 0x7fcdd8103040 2024-04-20T11:49:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.593+0000 7fcde0200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd819c540 con 0x7fcdd81039f0 2024-04-20T11:49:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.594+0000 7fcddd800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 0x7fcdd8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.594+0000 7fcddd800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 0x7fcdd8198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44918/0 (socket says 172.21.15.117:44918) 2024-04-20T11:49:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.594+0000 7fcddd800700 1 -- 172.21.15.117:0/1031444644 learned_addr learned my addr 172.21.15.117:0/1031444644 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.594+0000 7fcddce00700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdd81039f0 0x7fcdd8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.594+0000 7fcddd800700 1 -- 172.21.15.117:0/1031444644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdd81039f0 msgr2=0x7fcdd8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.594+0000 7fcddd800700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdd81039f0 0x7fcdd8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.594+0000 7fcddd800700 1 -- 172.21.15.117:0/1031444644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc0095f0 con 0x7fcdd8103040 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.594+0000 7fcddd800700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 0x7fcdd8198d00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc006010 tx=0x7fcdcc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.595+0000 7fcdd6000700 1 -- 172.21.15.117:0/1031444644 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdcc01d070 con 0x7fcdd8103040 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.595+0000 7fcdd6000700 1 -- 172.21.15.117:0/1031444644 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdcc00f460 con 0x7fcdd8103040 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.595+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd819c7c0 con 0x7fcdd8103040 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.595+0000 7fcdd6000700 1 -- 172.21.15.117:0/1031444644 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdcc017630 con 0x7fcdd8103040 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.595+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd819ccb0 con 0x7fcdd8103040 2024-04-20T11:49:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.596+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd810b3e0 con 0x7fcdd8103040 2024-04-20T11:49:15.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.597+0000 7fcdd6000700 1 -- 172.21.15.117:0/1031444644 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcdcc00f5d0 con 0x7fcdd8103040 2024-04-20T11:49:15.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.598+0000 7fcdd6000700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdc006ecc0 0x7fcdc0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:15.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.599+0000 7fcdd6000700 1 -- 172.21.15.117:0/1031444644 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcdcc0903b0 con 0x7fcdd8103040 2024-04-20T11:49:15.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.599+0000 7fcddce00700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdc006ecc0 0x7fcdc0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:15.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.599+0000 7fcddce00700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdc006ecc0 0x7fcdc0071180 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcdd819a220 tx=0x7fcdc4005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:15.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.603+0000 7fcdd6000700 1 -- 172.21.15.117:0/1031444644 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcdcc0591b0 con 0x7fcdd8103040 2024-04-20T11:49:15.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.844+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdd8199fd0 con 0x7fcdc006ecc0 2024-04-20T11:49:15.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.847+0000 7fcdd6000700 1 -- 172.21.15.117:0/1031444644 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcdd8199fd0 con 0x7fcdc006ecc0 2024-04-20T11:49:15.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.849+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdc006ecc0 msgr2=0x7fcdc0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.849+0000 7fcde0200700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdc006ecc0 0x7fcdc0071180 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcdd819a220 tx=0x7fcdc4005cf0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.849+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 msgr2=0x7fcdd8198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.849+0000 7fcde0200700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 0x7fcdd8198d00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc006010 tx=0x7fcdcc00be20 comp rx=0 tx=0).stop 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.849+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 shutdown_connections 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.850+0000 7fcde0200700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcdc006ecc0 0x7fcdc0071180 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.850+0000 7fcde0200700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd8103040 0x7fcdd8198d00 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.850+0000 7fcde0200700 1 --2- 172.21.15.117:0/1031444644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdd81039f0 0x7fcdd8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.850+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 >> 172.21.15.117:0/1031444644 conn(0x7fcdd80fe8b0 msgr2=0x7fcdd80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.850+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 shutdown_connections 2024-04-20T11:49:15.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.850+0000 7fcde0200700 1 -- 172.21.15.117:0/1031444644 wait complete. 2024-04-20T11:49:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.994+0000 7fa11ae00700 1 -- 172.21.15.117:0/31137054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114103110 msgr2=0x7fa1141034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.994+0000 7fa11ae00700 1 --2- 172.21.15.117:0/31137054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114103110 0x7fa1141034f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa104009910 tx=0x7fa104009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.995+0000 7fa11ae00700 1 -- 172.21.15.117:0/31137054 shutdown_connections 2024-04-20T11:49:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.995+0000 7fa11ae00700 1 --2- 172.21.15.117:0/31137054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa114104a00 0x7fa114104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.995+0000 7fa11ae00700 1 --2- 172.21.15.117:0/31137054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114103110 0x7fa1141034f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.995+0000 7fa11ae00700 1 -- 172.21.15.117:0/31137054 >> 172.21.15.117:0/31137054 conn(0x7fa1140fe880 msgr2=0x7fa114100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.995+0000 7fa11ae00700 1 -- 172.21.15.117:0/31137054 shutdown_connections 2024-04-20T11:49:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.995+0000 7fa11ae00700 1 -- 172.21.15.117:0/31137054 wait complete. 2024-04-20T11:49:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.996+0000 7fa11ae00700 1 Processor -- start 2024-04-20T11:49:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.996+0000 7fa11ae00700 1 -- start start 2024-04-20T11:49:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.996+0000 7fa11ae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa114104a00 0x7fa114198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.996+0000 7fa11ae00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114199520 0x7fa11419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.996+0000 7fa11ae00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa114199ab0 con 0x7fa114199520 2024-04-20T11:49:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.996+0000 7fa11ae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa114199c20 con 0x7fa114104a00 2024-04-20T11:49:15.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.997+0000 7fa113400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114199520 0x7fa11419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:15.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.997+0000 7fa113400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114199520 0x7fa11419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44938/0 (socket says 172.21.15.117:44938) 2024-04-20T11:49:15.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.997+0000 7fa113400700 1 -- 172.21.15.117:0/2547435769 learned_addr learned my addr 172.21.15.117:0/2547435769 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.997+0000 7fa113e00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa114104a00 0x7fa114198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.997+0000 7fa113400700 1 -- 172.21.15.117:0/2547435769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa114104a00 msgr2=0x7fa114198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.997+0000 7fa113400700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa114104a00 0x7fa114198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.997+0000 7fa113400700 1 -- 172.21.15.117:0/2547435769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1040095f0 con 0x7fa114199520 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.997+0000 7fa113400700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114199520 0x7fa11419d8d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c930 tx=0x7fa10000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.998+0000 7fa110c00700 1 -- 172.21.15.117:0/2547435769 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa100007ad0 con 0x7fa114199520 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.998+0000 7fa110c00700 1 -- 172.21.15.117:0/2547435769 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa100007c30 con 0x7fa114199520 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.998+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11419de70 con 0x7fa114199520 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.998+0000 7fa110c00700 1 -- 172.21.15.117:0/2547435769 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa100018610 con 0x7fa114199520 2024-04-20T11:49:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.998+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11419e440 con 0x7fa114199520 2024-04-20T11:49:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:15.999+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11419e720 con 0x7fa114199520 2024-04-20T11:49:16.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.000+0000 7fa110c00700 1 -- 172.21.15.117:0/2547435769 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa10000f450 con 0x7fa114199520 2024-04-20T11:49:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.001+0000 7fa110c00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa0f806ecd0 0x7fa0f8071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.001+0000 7fa110c00700 1 -- 172.21.15.117:0/2547435769 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa10008f080 con 0x7fa114199520 2024-04-20T11:49:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.002+0000 7fa113e00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa0f806ecd0 0x7fa0f8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:16.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.002+0000 7fa113e00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa0f806ecd0 0x7fa0f8071190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa104006010 tx=0x7fa10400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:16.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.006+0000 7fa110c00700 1 -- 172.21.15.117:0/2547435769 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa100057e40 con 0x7fa114199520 2024-04-20T11:49:16.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.230+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa11404ebb0 con 0x7fa0f806ecd0 2024-04-20T11:49:16.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.247+0000 7fa110c00700 1 -- 172.21.15.117:0/2547435769 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa11404ebb0 con 0x7fa0f806ecd0 2024-04-20T11:49:16.247 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (37m) 13m ago 51m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (52m) 13m ago 52m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (50m) 13m ago 50m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (34m) 13m ago 52m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (33m) 13m ago 50m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (37m) 13m ago 51m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (29m) 13m ago 44m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (29m) 13m ago 44m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (13m) 13m ago 44m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (29m) 13m ago 44m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (40m) 13m ago 53m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (39m) 13m ago 50m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (35m) 13m ago 53m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:49:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (35m) 13m ago 50m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (39m) 13m ago 51m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (38m) 13m ago 50m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (33m) 13m ago 49m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (32m) 13m ago 48m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (32m) 13m ago 48m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (31m) 13m ago 47m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (31m) 13m ago 47m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (30m) 13m ago 46m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:49:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (38m) 13m ago 51m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa0f806ecd0 msgr2=0x7fa0f8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa0f806ecd0 0x7fa0f8071190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa104006010 tx=0x7fa10400b540 comp rx=0 tx=0).stop 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114199520 msgr2=0x7fa11419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114199520 0x7fa11419d8d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c930 tx=0x7fa10000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 shutdown_connections 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa0f806ecd0 0x7fa0f8071190 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa114104a00 0x7fa114198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 --2- 172.21.15.117:0/2547435769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa114199520 0x7fa11419d8d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 >> 172.21.15.117:0/2547435769 conn(0x7fa1140fe880 msgr2=0x7fa114101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:16.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 shutdown_connections 2024-04-20T11:49:16.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.250+0000 7fa11ae00700 1 -- 172.21.15.117:0/2547435769 wait complete. 2024-04-20T11:49:16.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.393+0000 7faca9400700 1 -- 172.21.15.117:0/3076644934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4101d70 msgr2=0x7faca4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:16.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.393+0000 7faca9400700 1 --2- 172.21.15.117:0/3076644934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4101d70 0x7faca4102150 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fac94009910 tx=0x7fac94009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:16.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.393+0000 7faca9400700 1 -- 172.21.15.117:0/3076644934 shutdown_connections 2024-04-20T11:49:16.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.393+0000 7faca9400700 1 --2- 172.21.15.117:0/3076644934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca4102690 0x7faca410ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.393+0000 7faca9400700 1 --2- 172.21.15.117:0/3076644934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4101d70 0x7faca4102150 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.393+0000 7faca9400700 1 -- 172.21.15.117:0/3076644934 >> 172.21.15.117:0/3076644934 conn(0x7faca40fb5c0 msgr2=0x7faca40fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:16.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.393+0000 7faca9400700 1 -- 172.21.15.117:0/3076644934 shutdown_connections 2024-04-20T11:49:16.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.393+0000 7faca9400700 1 -- 172.21.15.117:0/3076644934 wait complete. 2024-04-20T11:49:16.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.394+0000 7faca9400700 1 Processor -- start 2024-04-20T11:49:16.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.394+0000 7faca9400700 1 -- start start 2024-04-20T11:49:16.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.395+0000 7faca9400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca4101d70 0x7faca419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:16.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.395+0000 7faca9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4102690 0x7faca419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:16.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.395+0000 7faca9400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca419d420 con 0x7faca4102690 2024-04-20T11:49:16.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.395+0000 7faca9400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4196a00 con 0x7faca4101d70 2024-04-20T11:49:16.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.395+0000 7faca1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4102690 0x7faca419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:16.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.395+0000 7faca1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4102690 0x7faca419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44956/0 (socket says 172.21.15.117:44956) 2024-04-20T11:49:16.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.395+0000 7faca1e00700 1 -- 172.21.15.117:0/3462528443 learned_addr learned my addr 172.21.15.117:0/3462528443 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:16.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.396+0000 7faca2800700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca4101d70 0x7faca419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:16.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.396+0000 7faca1e00700 1 -- 172.21.15.117:0/3462528443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca4101d70 msgr2=0x7faca419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:16.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.396+0000 7faca1e00700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca4101d70 0x7faca419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.396+0000 7faca1e00700 1 -- 172.21.15.117:0/3462528443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac940095f0 con 0x7faca4102690 2024-04-20T11:49:16.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.396+0000 7faca1e00700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4102690 0x7faca419ce00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c8f0 tx=0x7fac9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:16.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.396+0000 7fac9b400700 1 -- 172.21.15.117:0/3462528443 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac90007ad0 con 0x7faca4102690 2024-04-20T11:49:16.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.396+0000 7fac9b400700 1 -- 172.21.15.117:0/3462528443 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac9000ce90 con 0x7faca4102690 2024-04-20T11:49:16.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.396+0000 7fac9b400700 1 -- 172.21.15.117:0/3462528443 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac90018630 con 0x7faca4102690 2024-04-20T11:49:16.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.397+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca4196ce0 con 0x7faca4102690 2024-04-20T11:49:16.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.397+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca4197230 con 0x7faca4102690 2024-04-20T11:49:16.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.398+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4105490 con 0x7faca4102690 2024-04-20T11:49:16.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.399+0000 7fac9b400700 1 -- 172.21.15.117:0/3462528443 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fac90007c30 con 0x7faca4102690 2024-04-20T11:49:16.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.400+0000 7fac9b400700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac8806ecc0 0x7fac88071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:16.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.400+0000 7fac9b400700 1 -- 172.21.15.117:0/3462528443 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fac9008ed50 con 0x7faca4102690 2024-04-20T11:49:16.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.400+0000 7faca2800700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac8806ecc0 0x7fac88071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:16.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.401+0000 7faca2800700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac8806ecc0 0x7fac88071180 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fac94005f50 tx=0x7fac9401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:16.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.405+0000 7fac9b400700 1 -- 172.21.15.117:0/3462528443 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fac9005b640 con 0x7faca4102690 2024-04-20T11:49:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:16 smithi117 ceph-mon[99036]: pgmap v1105: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:16.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:16 smithi117 ceph-mon[99036]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:16.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.705+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faca4197ec0 con 0x7faca4102690 2024-04-20T11:49:16.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.707+0000 7fac9b400700 1 -- 172.21.15.117:0/3462528443 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fac9005b1d0 con 0x7faca4102690 2024-04-20T11:49:16.706 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:49:16.707 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:49:16.708 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:49:16.708 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac8806ecc0 msgr2=0x7fac88071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac8806ecc0 0x7fac88071180 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fac94005f50 tx=0x7fac9401a040 comp rx=0 tx=0).stop 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4102690 msgr2=0x7faca419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4102690 0x7faca419ce00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c8f0 tx=0x7fac9000ccb0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 shutdown_connections 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fac8806ecc0 0x7fac88071180 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca4101d70 0x7faca419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 --2- 172.21.15.117:0/3462528443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca4102690 0x7faca419ce00 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 >> 172.21.15.117:0/3462528443 conn(0x7faca40fb5c0 msgr2=0x7faca4103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:16.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 shutdown_connections 2024-04-20T11:49:16.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.710+0000 7faca9400700 1 -- 172.21.15.117:0/3462528443 wait complete. 2024-04-20T11:49:16.817 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:16 smithi173 ceph-mon[74165]: pgmap v1105: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:16.817 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:16 smithi173 ceph-mon[74165]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:16.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.854+0000 7f4895200700 1 -- 172.21.15.117:0/796263099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 msgr2=0x7f489010abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:16.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.854+0000 7f4895200700 1 --2- 172.21.15.117:0/796263099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 0x7f489010abd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f487c009940 tx=0x7f487c009c50 comp rx=0 tx=0).stop 2024-04-20T11:49:16.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.854+0000 7f4895200700 1 -- 172.21.15.117:0/796263099 shutdown_connections 2024-04-20T11:49:16.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.854+0000 7f4895200700 1 --2- 172.21.15.117:0/796263099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 0x7f489010abd0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.854+0000 7f4895200700 1 --2- 172.21.15.117:0/796263099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4890101dc0 0x7f48901021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.854+0000 7f4895200700 1 -- 172.21.15.117:0/796263099 >> 172.21.15.117:0/796263099 conn(0x7f4890076750 msgr2=0x7f48900fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:16.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.854+0000 7f4895200700 1 -- 172.21.15.117:0/796263099 shutdown_connections 2024-04-20T11:49:16.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.854+0000 7f4895200700 1 -- 172.21.15.117:0/796263099 wait complete. 2024-04-20T11:49:16.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.855+0000 7f4895200700 1 Processor -- start 2024-04-20T11:49:16.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.855+0000 7f4895200700 1 -- start start 2024-04-20T11:49:16.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f4895200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4890101dc0 0x7f489019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:16.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f4895200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 0x7f489019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:16.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f4895200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019d3c0 con 0x7f48901026e0 2024-04-20T11:49:16.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f4895200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48901969a0 con 0x7f4890101dc0 2024-04-20T11:49:16.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f488d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 0x7f489019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:16.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f488d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 0x7f489019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44972/0 (socket says 172.21.15.117:44972) 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f488d400700 1 -- 172.21.15.117:0/1966788843 learned_addr learned my addr 172.21.15.117:0/1966788843 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f488de00700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4890101dc0 0x7f489019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f488d400700 1 -- 172.21.15.117:0/1966788843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4890101dc0 msgr2=0x7f489019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f488d400700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4890101dc0 0x7f489019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f488d400700 1 -- 172.21.15.117:0/1966788843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0095f0 con 0x7f48901026e0 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.856+0000 7f488de00700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4890101dc0 0x7f489019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.857+0000 7f488d400700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 0x7f489019cda0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f487c00b870 tx=0x7f487c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.857+0000 7f4886a00700 1 -- 172.21.15.117:0/1966788843 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487c01d070 con 0x7f48901026e0 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.857+0000 7f4886a00700 1 -- 172.21.15.117:0/1966788843 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f487c00bed0 con 0x7f48901026e0 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.857+0000 7f4886a00700 1 -- 172.21.15.117:0/1966788843 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487c021410 con 0x7f48901026e0 2024-04-20T11:49:16.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.857+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4890196c20 con 0x7f48901026e0 2024-04-20T11:49:16.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.857+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4890197110 con 0x7f48901026e0 2024-04-20T11:49:16.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.858+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48901054e0 con 0x7f48901026e0 2024-04-20T11:49:16.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.859+0000 7f4886a00700 1 -- 172.21.15.117:0/1966788843 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f487c00f810 con 0x7f48901026e0 2024-04-20T11:49:16.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.860+0000 7f4886a00700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f487406ecc0 0x7f4874071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:16.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.860+0000 7f4886a00700 1 -- 172.21.15.117:0/1966788843 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f487c08ffe0 con 0x7f48901026e0 2024-04-20T11:49:16.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.860+0000 7f488de00700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f487406ecc0 0x7f4874071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:16.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.861+0000 7f488de00700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f487406ecc0 0x7f4874071180 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4880005fd0 tx=0x7f4880005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:16.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:16.865+0000 7f4886a00700 1 -- 172.21.15.117:0/1966788843 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f487c0c4ff0 con 0x7f48901026e0 2024-04-20T11:49:17.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.117+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4890068dd0 con 0x7f48901026e0 2024-04-20T11:49:17.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.118+0000 7f4886a00700 1 -- 172.21.15.117:0/1966788843 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f487c05c640 con 0x7f48901026e0 2024-04-20T11:49:17.119 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:49:17.119 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:49:17.119 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:49:17.119 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:49:17.120 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:49:17.121 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:49:17.122 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:49:17.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.120+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f487406ecc0 msgr2=0x7f4874071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.120+0000 7f4895200700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f487406ecc0 0x7f4874071180 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4880005fd0 tx=0x7f4880005dc0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.120+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 msgr2=0x7f489019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.120+0000 7f4895200700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 0x7f489019cda0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f487c00b870 tx=0x7f487c0056a0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.121+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 shutdown_connections 2024-04-20T11:49:17.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.121+0000 7f4895200700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f487406ecc0 0x7f4874071180 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.121+0000 7f4895200700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4890101dc0 0x7f489019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.121+0000 7f4895200700 1 --2- 172.21.15.117:0/1966788843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48901026e0 0x7f489019cda0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.121+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 >> 172.21.15.117:0/1966788843 conn(0x7f4890076750 msgr2=0x7f48901030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:17.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.121+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 shutdown_connections 2024-04-20T11:49:17.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.121+0000 7f4895200700 1 -- 172.21.15.117:0/1966788843 wait complete. 2024-04-20T11:49:17.124 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:49:17.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.277+0000 7f8404000700 1 -- 172.21.15.117:0/948089112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 msgr2=0x7f83fc0757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.277+0000 7f8404000700 1 --2- 172.21.15.117:0/948089112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 0x7f83fc0757f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009910 tx=0x7f83f0009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:17.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.277+0000 7f8404000700 1 -- 172.21.15.117:0/948089112 shutdown_connections 2024-04-20T11:49:17.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.277+0000 7f8404000700 1 --2- 172.21.15.117:0/948089112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83fc075d30 0x7f83fc111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.277+0000 7f8404000700 1 --2- 172.21.15.117:0/948089112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 0x7f83fc0757f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.277+0000 7f8404000700 1 -- 172.21.15.117:0/948089112 >> 172.21.15.117:0/948089112 conn(0x7f83fc0fe8d0 msgr2=0x7f83fc100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:17.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.277+0000 7f8404000700 1 -- 172.21.15.117:0/948089112 shutdown_connections 2024-04-20T11:49:17.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.277+0000 7f8404000700 1 -- 172.21.15.117:0/948089112 wait complete. 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.278+0000 7f8404000700 1 Processor -- start 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.279+0000 7f8404000700 1 -- start start 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.279+0000 7f8404000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 0x7f83fc19d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.279+0000 7f8404000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83fc075d30 0x7f83fc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.279+0000 7f8404000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc19dd80 con 0x7f83fc075410 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.279+0000 7f8404000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc1a1a00 con 0x7f83fc075d30 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.279+0000 7f8401600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 0x7f83fc19d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.279+0000 7f8401600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 0x7f83fc19d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44984/0 (socket says 172.21.15.117:44984) 2024-04-20T11:49:17.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.279+0000 7f8401600700 1 -- 172.21.15.117:0/1450726092 learned_addr learned my addr 172.21.15.117:0/1450726092 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:17.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f8400c00700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83fc075d30 0x7f83fc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:17.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f8401600700 1 -- 172.21.15.117:0/1450726092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83fc075d30 msgr2=0x7f83fc19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f8401600700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83fc075d30 0x7f83fc19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f8401600700 1 -- 172.21.15.117:0/1450726092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00095f0 con 0x7f83fc075410 2024-04-20T11:49:17.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f8401600700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 0x7f83fc19d160 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f83f00092e0 tx=0x7f83f00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:17.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f83fa000700 1 -- 172.21.15.117:0/1450726092 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83f001d070 con 0x7f83fc075410 2024-04-20T11:49:17.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f83fa000700 1 -- 172.21.15.117:0/1450726092 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83f000bed0 con 0x7f83fc075410 2024-04-20T11:49:17.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc1a1ce0 con 0x7f83fc075410 2024-04-20T11:49:17.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.280+0000 7f83fa000700 1 -- 172.21.15.117:0/1450726092 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83f0021410 con 0x7f83fc075410 2024-04-20T11:49:17.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.281+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc1a2230 con 0x7f83fc075410 2024-04-20T11:49:17.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.282+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc076b50 con 0x7f83fc075410 2024-04-20T11:49:17.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.283+0000 7f83fa000700 1 -- 172.21.15.117:0/1450726092 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f83f000f460 con 0x7f83fc075410 2024-04-20T11:49:17.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.284+0000 7f83fa000700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83e406ecd0 0x7f83e4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:17.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.284+0000 7f83fa000700 1 -- 172.21.15.117:0/1450726092 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f83f0090120 con 0x7f83fc075410 2024-04-20T11:49:17.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.284+0000 7f8400c00700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83e406ecd0 0x7f83e4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:17.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.285+0000 7f8400c00700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83e406ecd0 0x7f83e4071190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f83e8005fd0 tx=0x7f83e800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:17.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.289+0000 7f83fa000700 1 -- 172.21.15.117:0/1450726092 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f83f0058e60 con 0x7f83fc075410 2024-04-20T11:49:17.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.531+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f83fc114330 con 0x7f83e406ecd0 2024-04-20T11:49:17.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.533+0000 7f83fa000700 1 -- 172.21.15.117:0/1450726092 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f83fc114330 con 0x7f83e406ecd0 2024-04-20T11:49:17.533 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:49:17.533 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:49:17.533 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:49:17.533 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:49:17.534 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:49:17.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83e406ecd0 msgr2=0x7f83e4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83e406ecd0 0x7f83e4071190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f83e8005fd0 tx=0x7f83e800a560 comp rx=0 tx=0).stop 2024-04-20T11:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 msgr2=0x7f83fc19d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 0x7f83fc19d160 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f83f00092e0 tx=0x7f83f00056a0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 shutdown_connections 2024-04-20T11:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83e406ecd0 0x7f83e4071190 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83fc075410 0x7f83fc19d160 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 --2- 172.21.15.117:0/1450726092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83fc075d30 0x7f83fc19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 >> 172.21.15.117:0/1450726092 conn(0x7f83fc0fe8d0 msgr2=0x7f83fc0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:17.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 shutdown_connections 2024-04-20T11:49:17.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.536+0000 7f8404000700 1 -- 172.21.15.117:0/1450726092 wait complete. 2024-04-20T11:49:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.685+0000 7f8efdc00700 1 -- 172.21.15.117:0/2342741603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef8100300 msgr2=0x7f8ef810cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.685+0000 7f8efdc00700 1 --2- 172.21.15.117:0/2342741603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef8100300 0x7f8ef810cfc0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009910 tx=0x7f8ee4009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.685+0000 7f8efdc00700 1 -- 172.21.15.117:0/2342741603 shutdown_connections 2024-04-20T11:49:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.685+0000 7f8efdc00700 1 --2- 172.21.15.117:0/2342741603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef8100300 0x7f8ef810cfc0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.685+0000 7f8efdc00700 1 --2- 172.21.15.117:0/2342741603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef80ff950 0x7f8ef80ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.685+0000 7f8efdc00700 1 -- 172.21.15.117:0/2342741603 >> 172.21.15.117:0/2342741603 conn(0x7f8ef8076750 msgr2=0x7f8ef80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.685+0000 7f8efdc00700 1 -- 172.21.15.117:0/2342741603 shutdown_connections 2024-04-20T11:49:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.685+0000 7f8efdc00700 1 -- 172.21.15.117:0/2342741603 wait complete. 2024-04-20T11:49:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.686+0000 7f8efdc00700 1 Processor -- start 2024-04-20T11:49:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.687+0000 7f8efdc00700 1 -- start start 2024-04-20T11:49:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.687+0000 7f8efdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef80ff950 0x7f8ef8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.687+0000 7f8efdc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef8100300 0x7f8ef8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.687+0000 7f8efdc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8199840 con 0x7f8ef80ff950 2024-04-20T11:49:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.687+0000 7f8efdc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef81999b0 con 0x7f8ef8100300 2024-04-20T11:49:17.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.687+0000 7f8ef6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef80ff950 0x7f8ef8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.687+0000 7f8ef6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef80ff950 0x7f8ef8198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45004/0 (socket says 172.21.15.117:45004) 2024-04-20T11:49:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.687+0000 7f8ef6a00700 1 -- 172.21.15.117:0/426211939 learned_addr learned my addr 172.21.15.117:0/426211939 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8ef6000700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef8100300 0x7f8ef8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8ef6a00700 1 -- 172.21.15.117:0/426211939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef8100300 msgr2=0x7f8ef8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8ef6a00700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef8100300 0x7f8ef8199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8ef6a00700 1 -- 172.21.15.117:0/426211939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee40095f0 con 0x7f8ef80ff950 2024-04-20T11:49:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8ef6000700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef8100300 0x7f8ef8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:49:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8ef6a00700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef80ff950 0x7f8ef8198d00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c930 tx=0x7f8ee800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8eef400700 1 -- 172.21.15.117:0/426211939 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ee8007ad0 con 0x7f8ef80ff950 2024-04-20T11:49:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8eef400700 1 -- 172.21.15.117:0/426211939 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ee8007c30 con 0x7f8ef80ff950 2024-04-20T11:49:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef819c690 con 0x7f8ef80ff950 2024-04-20T11:49:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8eef400700 1 -- 172.21.15.117:0/426211939 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ee8018610 con 0x7f8ef80ff950 2024-04-20T11:49:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.688+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef819cbe0 con 0x7f8ef80ff950 2024-04-20T11:49:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.689+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef8068dd0 con 0x7f8ef80ff950 2024-04-20T11:49:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.690+0000 7f8eef400700 1 -- 172.21.15.117:0/426211939 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8ee800f450 con 0x7f8ef80ff950 2024-04-20T11:49:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.691+0000 7f8eef400700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8edc06ecc0 0x7f8edc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.692+0000 7f8eef400700 1 -- 172.21.15.117:0/426211939 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8ee808f0f0 con 0x7f8ef80ff950 2024-04-20T11:49:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.692+0000 7f8ef6000700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8edc06ecc0 0x7f8edc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:17.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.692+0000 7f8ef6000700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8edc06ecc0 0x7f8edc071180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b850 tx=0x7f8ee400b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:17.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.696+0000 7f8eef400700 1 -- 172.21.15.117:0/426211939 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8ee8057e30 con 0x7f8ef80ff950 2024-04-20T11:49:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:17 smithi117 ceph-mon[99036]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:17 smithi117 ceph-mon[99036]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:17 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3462528443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:49:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:17 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1966788843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:49:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:17 smithi173 ceph-mon[74165]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:17 smithi173 ceph-mon[74165]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:17 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3462528443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:49:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:17 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1966788843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:49:17.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.988+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8ef810ff50 con 0x7f8ef80ff950 2024-04-20T11:49:17.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.989+0000 7f8eef400700 1 -- 172.21.15.117:0/426211939 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f8ee805b4f0 con 0x7f8ef80ff950 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:49:17.990 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:49:17.991 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:49:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:49:17.992 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:49:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:49:17.992 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.991+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8edc06ecc0 msgr2=0x7f8edc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.991+0000 7f8efdc00700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8edc06ecc0 0x7f8edc071180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b850 tx=0x7f8ee400b560 comp rx=0 tx=0).stop 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.991+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef80ff950 msgr2=0x7f8ef8198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.991+0000 7f8efdc00700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef80ff950 0x7f8ef8198d00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c930 tx=0x7f8ee800ccf0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.992+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 shutdown_connections 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.992+0000 7f8efdc00700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8edc06ecc0 0x7f8edc071180 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.992+0000 7f8efdc00700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef80ff950 0x7f8ef8198d00 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.992+0000 7f8efdc00700 1 --2- 172.21.15.117:0/426211939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef8100300 0x7f8ef8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.992+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 >> 172.21.15.117:0/426211939 conn(0x7f8ef8076750 msgr2=0x7f8ef80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.992+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 shutdown_connections 2024-04-20T11:49:17.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:17.992+0000 7f8efdc00700 1 -- 172.21.15.117:0/426211939 wait complete. 2024-04-20T11:49:18.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:18 smithi117 ceph-mon[99036]: pgmap v1106: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:18.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:18 smithi117 ceph-mon[99036]: from='client.35982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:18.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:18 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/426211939' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:49:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:18 smithi173 ceph-mon[74165]: pgmap v1106: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:18 smithi173 ceph-mon[74165]: from='client.35982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:18 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/426211939' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:49:20.324 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:20 smithi117 ceph-mon[99036]: pgmap v1107: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:20.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:20 smithi173 ceph-mon[74165]: pgmap v1107: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:22.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:22 smithi117 ceph-mon[99036]: pgmap v1108: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:22.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:22 smithi173 ceph-mon[74165]: pgmap v1108: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:24.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:24 smithi117 ceph-mon[99036]: pgmap v1109: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:24.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:24 smithi173 ceph-mon[74165]: pgmap v1109: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:26.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:26 smithi173 ceph-mon[74165]: pgmap v1110: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:26.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:26 smithi117 ceph-mon[99036]: pgmap v1110: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:28.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:28 smithi117 ceph-mon[99036]: pgmap v1111: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:28.658 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:49:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:28 smithi173 ceph-mon[74165]: pgmap v1111: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:49:30.663 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:30 smithi173 ceph-mon[74165]: pgmap v1112: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:30.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:30 smithi117 ceph-mon[99036]: pgmap v1112: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:32.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:32 smithi173 ceph-mon[74165]: pgmap v1113: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:32.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:32 smithi117 ceph-mon[99036]: pgmap v1113: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:34.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:34 smithi117 ceph-mon[99036]: pgmap v1114: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:34.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:34 smithi173 ceph-mon[74165]: pgmap v1114: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:36.713 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:36 smithi173 ceph-mon[74165]: pgmap v1115: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:36.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:36 smithi117 ceph-mon[99036]: pgmap v1115: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:38.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:38 smithi117 ceph-mon[99036]: pgmap v1116: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:38.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:38 smithi173 ceph-mon[74165]: pgmap v1116: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:40.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:40 smithi117 ceph-mon[99036]: pgmap v1117: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:40.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:40 smithi173 ceph-mon[74165]: pgmap v1117: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:42.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:42 smithi117 ceph-mon[99036]: pgmap v1118: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:42.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:42 smithi173 ceph-mon[74165]: pgmap v1118: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:49:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:49:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:44 smithi117 ceph-mon[99036]: pgmap v1119: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:44 smithi173 ceph-mon[74165]: pgmap v1119: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:46.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:46 smithi117 ceph-mon[99036]: pgmap v1120: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:46.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:46 smithi173 ceph-mon[74165]: pgmap v1120: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:48.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.163+0000 7fd6a8e00700 1 -- 172.21.15.117:0/3714130535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4101c20 msgr2=0x7fd6a4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.163+0000 7fd6a8e00700 1 --2- 172.21.15.117:0/3714130535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4101c20 0x7fd6a4102000 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd694009910 tx=0x7fd694009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:48.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.163+0000 7fd6a8e00700 1 -- 172.21.15.117:0/3714130535 shutdown_connections 2024-04-20T11:49:48.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.163+0000 7fd6a8e00700 1 --2- 172.21.15.117:0/3714130535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a4102540 0x7fd6a410aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.163+0000 7fd6a8e00700 1 --2- 172.21.15.117:0/3714130535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4101c20 0x7fd6a4102000 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.163+0000 7fd6a8e00700 1 -- 172.21.15.117:0/3714130535 >> 172.21.15.117:0/3714130535 conn(0x7fd6a40fb3f0 msgr2=0x7fd6a40fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:48.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.163+0000 7fd6a8e00700 1 -- 172.21.15.117:0/3714130535 shutdown_connections 2024-04-20T11:49:48.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.163+0000 7fd6a8e00700 1 -- 172.21.15.117:0/3714130535 wait complete. 2024-04-20T11:49:48.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.164+0000 7fd6a8e00700 1 Processor -- start 2024-04-20T11:49:48.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.165+0000 7fd6a8e00700 1 -- start start 2024-04-20T11:49:48.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.165+0000 7fd6a8e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a4101c20 0x7fd6a419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.165+0000 7fd6a8e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4102540 0x7fd6a419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.165+0000 7fd6a8e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a41989b0 con 0x7fd6a4102540 2024-04-20T11:49:48.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.165+0000 7fd6a8e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a4198b20 con 0x7fd6a4101c20 2024-04-20T11:49:48.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.165+0000 7fd6a2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4102540 0x7fd6a419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:48.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.165+0000 7fd6a2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4102540 0x7fd6a419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44134/0 (socket says 172.21.15.117:44134) 2024-04-20T11:49:48.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.165+0000 7fd6a2a00700 1 -- 172.21.15.117:0/4127099301 learned_addr learned my addr 172.21.15.117:0/4127099301 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:48.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd6a3400700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a4101c20 0x7fd6a419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:48.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd6a2a00700 1 -- 172.21.15.117:0/4127099301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a4101c20 msgr2=0x7fd6a419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd6a2a00700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a4101c20 0x7fd6a419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd6a2a00700 1 -- 172.21.15.117:0/4127099301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940095f0 con 0x7fd6a4102540 2024-04-20T11:49:48.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd6a2a00700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4102540 0x7fd6a419edf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd69000d900 tx=0x7fd69000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:48.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd69be00700 1 -- 172.21.15.117:0/4127099301 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd690009900 con 0x7fd6a4102540 2024-04-20T11:49:48.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd69be00700 1 -- 172.21.15.117:0/4127099301 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd690010460 con 0x7fd6a4102540 2024-04-20T11:49:48.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd69be00700 1 -- 172.21.15.117:0/4127099301 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd69000f5f0 con 0x7fd6a4102540 2024-04-20T11:49:48.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.166+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a4198e00 con 0x7fd6a4102540 2024-04-20T11:49:48.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.167+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a4199350 con 0x7fd6a4102540 2024-04-20T11:49:48.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.167+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a4069640 con 0x7fd6a4102540 2024-04-20T11:49:48.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.169+0000 7fd69be00700 1 -- 172.21.15.117:0/4127099301 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd690009c70 con 0x7fd6a4102540 2024-04-20T11:49:48.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.170+0000 7fd69be00700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd68806ecc0 0x7fd688071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.170+0000 7fd69be00700 1 -- 172.21.15.117:0/4127099301 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd69008ed00 con 0x7fd6a4102540 2024-04-20T11:49:48.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.171+0000 7fd6a3400700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd68806ecc0 0x7fd688071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:48.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.171+0000 7fd6a3400700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd68806ecc0 0x7fd688071180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd69400b5c0 tx=0x7fd694005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:48.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.174+0000 7fd69be00700 1 -- 172.21.15.117:0/4127099301 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd690057b40 con 0x7fd6a4102540 2024-04-20T11:49:48.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.414+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6a4199630 con 0x7fd68806ecc0 2024-04-20T11:49:48.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.417+0000 7fd69be00700 1 -- 172.21.15.117:0/4127099301 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd6a4199630 con 0x7fd68806ecc0 2024-04-20T11:49:48.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd68806ecc0 msgr2=0x7fd688071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd68806ecc0 0x7fd688071180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd69400b5c0 tx=0x7fd694005960 comp rx=0 tx=0).stop 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4102540 msgr2=0x7fd6a419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4102540 0x7fd6a419edf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd69000d900 tx=0x7fd69000dcc0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 shutdown_connections 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd68806ecc0 0x7fd688071180 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a4101c20 0x7fd6a419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 --2- 172.21.15.117:0/4127099301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6a4102540 0x7fd6a419edf0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 >> 172.21.15.117:0/4127099301 conn(0x7fd6a40fb3f0 msgr2=0x7fd6a410a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 shutdown_connections 2024-04-20T11:49:48.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.420+0000 7fd6a8e00700 1 -- 172.21.15.117:0/4127099301 wait complete. 2024-04-20T11:49:48.434 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:49:48.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.566+0000 7fa78b000700 1 -- 172.21.15.117:0/795084571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7841017c0 msgr2=0x7fa784105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.566+0000 7fa78b000700 1 --2- 172.21.15.117:0/795084571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7841017c0 0x7fa784105810 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa770009910 tx=0x7fa770009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:48.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.567+0000 7fa78b000700 1 -- 172.21.15.117:0/795084571 shutdown_connections 2024-04-20T11:49:48.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.567+0000 7fa78b000700 1 --2- 172.21.15.117:0/795084571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7841017c0 0x7fa784105810 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.567+0000 7fa78b000700 1 --2- 172.21.15.117:0/795084571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa784100e10 0x7fa7841011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.567+0000 7fa78b000700 1 -- 172.21.15.117:0/795084571 >> 172.21.15.117:0/795084571 conn(0x7fa7840fc6a0 msgr2=0x7fa7840feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:48.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.567+0000 7fa78b000700 1 -- 172.21.15.117:0/795084571 shutdown_connections 2024-04-20T11:49:48.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.567+0000 7fa78b000700 1 -- 172.21.15.117:0/795084571 wait complete. 2024-04-20T11:49:48.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa78b000700 1 Processor -- start 2024-04-20T11:49:48.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa78b000700 1 -- start start 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa78b000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa784100e10 0x7fa78419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa78b000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7841017c0 0x7fa78419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa78b000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78419d360 con 0x7fa784100e10 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa78b000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa784196990 con 0x7fa7841017c0 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa783e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa784100e10 0x7fa78419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa783e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa784100e10 0x7fa78419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44154/0 (socket says 172.21.15.117:44154) 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.568+0000 7fa783e00700 1 -- 172.21.15.117:0/1379343515 learned_addr learned my addr 172.21.15.117:0/1379343515 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.569+0000 7fa783e00700 1 -- 172.21.15.117:0/1379343515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7841017c0 msgr2=0x7fa78419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.569+0000 7fa783e00700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7841017c0 0x7fa78419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.569+0000 7fa783e00700 1 -- 172.21.15.117:0/1379343515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700095f0 con 0x7fa784100e10 2024-04-20T11:49:48.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.569+0000 7fa783e00700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa784100e10 0x7fa78419c800 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa77400d8d0 tx=0x7fa77400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:48.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.569+0000 7fa780c00700 1 -- 172.21.15.117:0/1379343515 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa774009940 con 0x7fa784100e10 2024-04-20T11:49:48.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.569+0000 7fa780c00700 1 -- 172.21.15.117:0/1379343515 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa774010460 con 0x7fa784100e10 2024-04-20T11:49:48.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.569+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa784196c70 con 0x7fa784100e10 2024-04-20T11:49:48.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.569+0000 7fa780c00700 1 -- 172.21.15.117:0/1379343515 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa77400f5f0 con 0x7fa784100e10 2024-04-20T11:49:48.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.570+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7841971c0 con 0x7fa784100e10 2024-04-20T11:49:48.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.571+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa784109210 con 0x7fa784100e10 2024-04-20T11:49:48.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.571+0000 7fa780c00700 1 -- 172.21.15.117:0/1379343515 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa7740105d0 con 0x7fa784100e10 2024-04-20T11:49:48.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.572+0000 7fa780c00700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa76806ecd0 0x7fa768071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.572+0000 7fa780c00700 1 -- 172.21.15.117:0/1379343515 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa77408ee50 con 0x7fa784100e10 2024-04-20T11:49:48.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.573+0000 7fa783400700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa76806ecd0 0x7fa768071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:48.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.573+0000 7fa783400700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa76806ecd0 0x7fa768071190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa770006010 tx=0x7fa77000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:48.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.577+0000 7fa780c00700 1 -- 172.21.15.117:0/1379343515 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa77405b760 con 0x7fa784100e10 2024-04-20T11:49:48.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.824+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa784197ea0 con 0x7fa76806ecd0 2024-04-20T11:49:48.824 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:48 smithi117 ceph-mon[99036]: pgmap v1121: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:48.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.827+0000 7fa780c00700 1 -- 172.21.15.117:0/1379343515 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa784197ea0 con 0x7fa76806ecd0 2024-04-20T11:49:48.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.829+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa76806ecd0 msgr2=0x7fa768071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.829+0000 7fa78b000700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa76806ecd0 0x7fa768071190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa770006010 tx=0x7fa77000b540 comp rx=0 tx=0).stop 2024-04-20T11:49:48.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.829+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa784100e10 msgr2=0x7fa78419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.829+0000 7fa78b000700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa784100e10 0x7fa78419c800 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa77400d8d0 tx=0x7fa77400dc90 comp rx=0 tx=0).stop 2024-04-20T11:49:48.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.830+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 shutdown_connections 2024-04-20T11:49:48.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.830+0000 7fa78b000700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa76806ecd0 0x7fa768071190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.830+0000 7fa78b000700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa784100e10 0x7fa78419c800 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.830+0000 7fa78b000700 1 --2- 172.21.15.117:0/1379343515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7841017c0 0x7fa78419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.830+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 >> 172.21.15.117:0/1379343515 conn(0x7fa7840fc6a0 msgr2=0x7fa7840fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:48.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.830+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 shutdown_connections 2024-04-20T11:49:48.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.830+0000 7fa78b000700 1 -- 172.21.15.117:0/1379343515 wait complete. 2024-04-20T11:49:48.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:48 smithi173 ceph-mon[74165]: pgmap v1121: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:48.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.986+0000 7f79f1e00700 1 -- 172.21.15.117:0/1465183694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec107520 msgr2=0x7f79ec107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.986+0000 7f79f1e00700 1 --2- 172.21.15.117:0/1465183694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec107520 0x7f79ec107900 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009910 tx=0x7f79dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:48.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.986+0000 7f79f1e00700 1 -- 172.21.15.117:0/1465183694 shutdown_connections 2024-04-20T11:49:48.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.986+0000 7f79f1e00700 1 --2- 172.21.15.117:0/1465183694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79ec108e10 0x7f79ec109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.986+0000 7f79f1e00700 1 --2- 172.21.15.117:0/1465183694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec107520 0x7f79ec107900 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.986+0000 7f79f1e00700 1 -- 172.21.15.117:0/1465183694 >> 172.21.15.117:0/1465183694 conn(0x7f79ec0768f0 msgr2=0x7f79ec078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:48.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.986+0000 7f79f1e00700 1 -- 172.21.15.117:0/1465183694 shutdown_connections 2024-04-20T11:49:48.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.986+0000 7f79f1e00700 1 -- 172.21.15.117:0/1465183694 wait complete. 2024-04-20T11:49:48.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.987+0000 7f79f1e00700 1 Processor -- start 2024-04-20T11:49:48.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.987+0000 7f79f1e00700 1 -- start start 2024-04-20T11:49:48.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79f1e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79ec107520 0x7f79ec19d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79f1e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec108e10 0x7f79ec19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79f1e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec19ddd0 con 0x7f79ec108e10 2024-04-20T11:49:48.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79f1e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1a1aa0 con 0x7f79ec107520 2024-04-20T11:49:48.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79e3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec108e10 0x7f79ec19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:48.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79e3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec108e10 0x7f79ec19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44180/0 (socket says 172.21.15.117:44180) 2024-04-20T11:49:48.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79e3e00700 1 -- 172.21.15.117:0/4011758085 learned_addr learned my addr 172.21.15.117:0/4011758085 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:48.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79eaa00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79ec107520 0x7f79ec19d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:48.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79e3e00700 1 -- 172.21.15.117:0/4011758085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79ec107520 msgr2=0x7f79ec19d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:48.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79e3e00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79ec107520 0x7f79ec19d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:48.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.988+0000 7f79e3e00700 1 -- 172.21.15.117:0/4011758085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0095f0 con 0x7f79ec108e10 2024-04-20T11:49:48.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.989+0000 7f79e3e00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec108e10 0x7f79ec19d6f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f79d400d8d0 tx=0x7f79d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:48.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.989+0000 7f79f0a00700 1 -- 172.21.15.117:0/4011758085 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d4009940 con 0x7f79ec108e10 2024-04-20T11:49:48.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.989+0000 7f79f0a00700 1 -- 172.21.15.117:0/4011758085 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79d4010460 con 0x7f79ec108e10 2024-04-20T11:49:48.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.989+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec1a1d80 con 0x7f79ec108e10 2024-04-20T11:49:48.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.989+0000 7f79f0a00700 1 -- 172.21.15.117:0/4011758085 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d400f5f0 con 0x7f79ec108e10 2024-04-20T11:49:48.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.989+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec1a2350 con 0x7f79ec108e10 2024-04-20T11:49:48.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.990+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec10f870 con 0x7f79ec108e10 2024-04-20T11:49:48.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.991+0000 7f79f0a00700 1 -- 172.21.15.117:0/4011758085 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f79d40105d0 con 0x7f79ec108e10 2024-04-20T11:49:48.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.992+0000 7f79f0a00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f79d006ecc0 0x7f79d0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:48.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.992+0000 7f79f0a00700 1 -- 172.21.15.117:0/4011758085 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f79d408ede0 con 0x7f79ec108e10 2024-04-20T11:49:48.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.992+0000 7f79eaa00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f79d006ecc0 0x7f79d0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:48.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.993+0000 7f79eaa00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f79d006ecc0 0x7f79d0071180 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00b5c0 tx=0x7f79dc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:48.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:48.997+0000 7f79f0a00700 1 -- 172.21.15.117:0/4011758085 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f79d4057ba0 con 0x7f79ec108e10 2024-04-20T11:49:49.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.232+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f79ec19e530 con 0x7f79d006ecc0 2024-04-20T11:49:49.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.264+0000 7f79f0a00700 1 -- 172.21.15.117:0/4011758085 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f79ec19e530 con 0x7f79d006ecc0 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (38m) 14m ago 52m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (52m) 14m ago 52m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (51m) 14m ago 51m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (34m) 14m ago 52m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (34m) 14m ago 51m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (37m) 14m ago 51m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (30m) 14m ago 45m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (30m) 14m ago 45m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (14m) 14m ago 44m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (29m) 14m ago 45m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (41m) 14m ago 54m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:49:49.265 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (39m) 14m ago 51m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (36m) 14m ago 54m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (35m) 14m ago 50m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (39m) 14m ago 52m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (39m) 14m ago 51m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (33m) 14m ago 49m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (33m) 14m ago 49m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (32m) 14m ago 48m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (32m) 14m ago 48m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (31m) 14m ago 47m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (30m) 14m ago 46m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:49:49.266 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (38m) 14m ago 51m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:49:49.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f79d006ecc0 msgr2=0x7f79d0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:49.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f79d006ecc0 0x7f79d0071180 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00b5c0 tx=0x7f79dc01a040 comp rx=0 tx=0).stop 2024-04-20T11:49:49.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec108e10 msgr2=0x7f79ec19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:49.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec108e10 0x7f79ec19d6f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f79d400d8d0 tx=0x7f79d400dc90 comp rx=0 tx=0).stop 2024-04-20T11:49:49.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 shutdown_connections 2024-04-20T11:49:49.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f79d006ecc0 0x7f79d0071180 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79ec107520 0x7f79ec19d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 --2- 172.21.15.117:0/4011758085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ec108e10 0x7f79ec19d6f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.267+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 >> 172.21.15.117:0/4011758085 conn(0x7f79ec0768f0 msgr2=0x7f79ec077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:49.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.268+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 shutdown_connections 2024-04-20T11:49:49.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.268+0000 7f79f1e00700 1 -- 172.21.15.117:0/4011758085 wait complete. 2024-04-20T11:49:49.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.434+0000 7fa279c00700 1 -- 172.21.15.117:0/640836387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 msgr2=0x7fa274108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:49.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.434+0000 7fa279c00700 1 --2- 172.21.15.117:0/640836387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 0x7fa274108520 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa264009910 tx=0x7fa264009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:49.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.434+0000 7fa279c00700 1 -- 172.21.15.117:0/640836387 shutdown_connections 2024-04-20T11:49:49.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.434+0000 7fa279c00700 1 --2- 172.21.15.117:0/640836387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa274108af0 0x7fa27410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.434+0000 7fa279c00700 1 --2- 172.21.15.117:0/640836387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 0x7fa274108520 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.434+0000 7fa279c00700 1 -- 172.21.15.117:0/640836387 >> 172.21.15.117:0/640836387 conn(0x7fa27406bcd0 msgr2=0x7fa27406c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:49.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.434+0000 7fa279c00700 1 -- 172.21.15.117:0/640836387 shutdown_connections 2024-04-20T11:49:49.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.434+0000 7fa279c00700 1 -- 172.21.15.117:0/640836387 wait complete. 2024-04-20T11:49:49.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.435+0000 7fa279c00700 1 Processor -- start 2024-04-20T11:49:49.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.435+0000 7fa279c00700 1 -- start start 2024-04-20T11:49:49.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.435+0000 7fa279c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 0x7fa2741a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:49.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.435+0000 7fa279c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa274108af0 0x7fa2741a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:49.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa279c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741a2240 con 0x7fa274108140 2024-04-20T11:49:49.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa279c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741a5f10 con 0x7fa274108af0 2024-04-20T11:49:49.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa272a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 0x7fa2741a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:49.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa272a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 0x7fa2741a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44188/0 (socket says 172.21.15.117:44188) 2024-04-20T11:49:49.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa272a00700 1 -- 172.21.15.117:0/2399561260 learned_addr learned my addr 172.21.15.117:0/2399561260 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:49.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa272000700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa274108af0 0x7fa2741a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:49.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa272a00700 1 -- 172.21.15.117:0/2399561260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa274108af0 msgr2=0x7fa2741a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:49.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa272a00700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa274108af0 0x7fa2741a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.436+0000 7fa272a00700 1 -- 172.21.15.117:0/2399561260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640095f0 con 0x7fa274108140 2024-04-20T11:49:49.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.437+0000 7fa272a00700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 0x7fa2741a1620 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa264006010 tx=0x7fa26400bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:49.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.437+0000 7fa26b400700 1 -- 172.21.15.117:0/2399561260 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26401d070 con 0x7fa274108140 2024-04-20T11:49:49.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.437+0000 7fa26b400700 1 -- 172.21.15.117:0/2399561260 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa26400be90 con 0x7fa274108140 2024-04-20T11:49:49.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.437+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2741a6190 con 0x7fa274108140 2024-04-20T11:49:49.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.437+0000 7fa26b400700 1 -- 172.21.15.117:0/2399561260 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa264021410 con 0x7fa274108140 2024-04-20T11:49:49.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.437+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2741a6680 con 0x7fa274108140 2024-04-20T11:49:49.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.438+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2740729f0 con 0x7fa274108140 2024-04-20T11:49:49.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.439+0000 7fa26b400700 1 -- 172.21.15.117:0/2399561260 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa26400f460 con 0x7fa274108140 2024-04-20T11:49:49.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.440+0000 7fa26b400700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa25806ecd0 0x7fa258071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:49.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.440+0000 7fa26b400700 1 -- 172.21.15.117:0/2399561260 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa26408ffe0 con 0x7fa274108140 2024-04-20T11:49:49.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.441+0000 7fa272000700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa25806ecd0 0x7fa258071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:49.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.441+0000 7fa272000700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa25806ecd0 0x7fa258071190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa260006fd0 tx=0x7fa260008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:49.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.446+0000 7fa26b400700 1 -- 172.21.15.117:0/2399561260 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa264058dc0 con 0x7fa274108140 2024-04-20T11:49:49.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:49 smithi117 ceph-mon[99036]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:49.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.759+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa2741166b0 con 0x7fa274108140 2024-04-20T11:49:49.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.761+0000 7fa26b400700 1 -- 172.21.15.117:0/2399561260 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa26400f780 con 0x7fa274108140 2024-04-20T11:49:49.760 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:49:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:49:49.762 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:49:49.762 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:49:49.762 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:49:49.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa25806ecd0 msgr2=0x7fa258071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:49.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa25806ecd0 0x7fa258071190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa260006fd0 tx=0x7fa260008040 comp rx=0 tx=0).stop 2024-04-20T11:49:49.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 msgr2=0x7fa2741a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:49.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 0x7fa2741a1620 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa264006010 tx=0x7fa26400bbf0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 shutdown_connections 2024-04-20T11:49:49.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa25806ecd0 0x7fa258071190 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa274108140 0x7fa2741a1620 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 --2- 172.21.15.117:0/2399561260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa274108af0 0x7fa2741a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.763+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 >> 172.21.15.117:0/2399561260 conn(0x7fa27406bcd0 msgr2=0x7fa274113930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:49.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.764+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 shutdown_connections 2024-04-20T11:49:49.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.764+0000 7fa279c00700 1 -- 172.21.15.117:0/2399561260 wait complete. 2024-04-20T11:49:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.905+0000 7f18e5e00700 1 -- 172.21.15.117:0/2821735168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 msgr2=0x7f18e0105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.905+0000 7f18e5e00700 1 --2- 172.21.15.117:0/2821735168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 0x7f18e0105840 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009910 tx=0x7f18cc009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.905+0000 7f18e5e00700 1 -- 172.21.15.117:0/2821735168 shutdown_connections 2024-04-20T11:49:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.905+0000 7f18e5e00700 1 --2- 172.21.15.117:0/2821735168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 0x7f18e0105840 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.905+0000 7f18e5e00700 1 --2- 172.21.15.117:0/2821735168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18e0100e40 0x7f18e0101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.905+0000 7f18e5e00700 1 -- 172.21.15.117:0/2821735168 >> 172.21.15.117:0/2821735168 conn(0x7f18e00fc710 msgr2=0x7f18e00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.906+0000 7f18e5e00700 1 -- 172.21.15.117:0/2821735168 shutdown_connections 2024-04-20T11:49:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.906+0000 7f18e5e00700 1 -- 172.21.15.117:0/2821735168 wait complete. 2024-04-20T11:49:49.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.907+0000 7f18e5e00700 1 Processor -- start 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.907+0000 7f18e5e00700 1 -- start start 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.907+0000 7f18e5e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18e0100e40 0x7f18e019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.907+0000 7f18e5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 0x7f18e019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.907+0000 7f18e5e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e019d4e0 con 0x7f18e01017f0 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.907+0000 7f18e5e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e0196ac0 con 0x7f18e0100e40 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.907+0000 7f18de000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 0x7f18e019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.908+0000 7f18de000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 0x7f18e019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44196/0 (socket says 172.21.15.117:44196) 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.908+0000 7f18de000700 1 -- 172.21.15.117:0/2693987256 learned_addr learned my addr 172.21.15.117:0/2693987256 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.908+0000 7f18de000700 1 -- 172.21.15.117:0/2693987256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18e0100e40 msgr2=0x7f18e019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.908+0000 7f18dea00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18e0100e40 0x7f18e019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.908+0000 7f18de000700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18e0100e40 0x7f18e019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.908+0000 7f18de000700 1 -- 172.21.15.117:0/2693987256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc0095f0 con 0x7f18e01017f0 2024-04-20T11:49:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.908+0000 7f18de000700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 0x7f18e019cec0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005b40 tx=0x7f18cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:49.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.909+0000 7f18e4a00700 1 -- 172.21.15.117:0/2693987256 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18cc01d070 con 0x7f18e01017f0 2024-04-20T11:49:49.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.909+0000 7f18e4a00700 1 -- 172.21.15.117:0/2693987256 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18cc00f460 con 0x7f18e01017f0 2024-04-20T11:49:49.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.909+0000 7f18e4a00700 1 -- 172.21.15.117:0/2693987256 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18cc017630 con 0x7f18e01017f0 2024-04-20T11:49:49.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.909+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e0196d40 con 0x7f18e01017f0 2024-04-20T11:49:49.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.909+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e0197230 con 0x7f18e01017f0 2024-04-20T11:49:49.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.910+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e0102320 con 0x7f18e01017f0 2024-04-20T11:49:49.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.911+0000 7f18e4a00700 1 -- 172.21.15.117:0/2693987256 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f18cc003a60 con 0x7f18e01017f0 2024-04-20T11:49:49.910 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:49 smithi173 ceph-mon[74165]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:49.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.912+0000 7f18e4a00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f18c406ecd0 0x7f18c4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:49.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.912+0000 7f18e4a00700 1 -- 172.21.15.117:0/2693987256 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f18cc090380 con 0x7f18e01017f0 2024-04-20T11:49:49.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.912+0000 7f18dea00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f18c406ecd0 0x7f18c4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:49.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.913+0000 7f18dea00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f18c406ecd0 0x7f18c4071190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f18d0006fd0 tx=0x7f18d0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:49.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:49.917+0000 7f18e4a00700 1 -- 172.21.15.117:0/2693987256 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f18cc05cbf0 con 0x7f18e01017f0 2024-04-20T11:49:50.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.186+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f18e010de60 con 0x7f18e01017f0 2024-04-20T11:49:50.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.187+0000 7f18e4a00700 1 -- 172.21.15.117:0/2693987256 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f18cc05c780 con 0x7f18e01017f0 2024-04-20T11:49:50.187 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:49:50.187 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:49:50.187 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:49:50.187 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:49:50.187 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:49:50.187 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:49:50.187 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:49:50.188 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:49:50.189 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:49:50.190 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:49:50.190 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:49:50.190 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:49:50.190 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:49:50.190 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:49:50.190 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:49:50.190 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f18c406ecd0 msgr2=0x7f18c4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f18c406ecd0 0x7f18c4071190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f18d0006fd0 tx=0x7f18d0008040 comp rx=0 tx=0).stop 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 msgr2=0x7f18e019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 0x7f18e019cec0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005b40 tx=0x7f18cc00be20 comp rx=0 tx=0).stop 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 shutdown_connections 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f18c406ecd0 0x7f18c4071190 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18e0100e40 0x7f18e019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 --2- 172.21.15.117:0/2693987256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18e01017f0 0x7f18e019cec0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 >> 172.21.15.117:0/2693987256 conn(0x7f18e00fc710 msgr2=0x7f18e00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:50.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 shutdown_connections 2024-04-20T11:49:50.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.192+0000 7f18e5e00700 1 -- 172.21.15.117:0/2693987256 wait complete. 2024-04-20T11:49:50.193 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:49:50.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.344+0000 7f2f24e00700 1 -- 172.21.15.117:0/1168252742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20108e10 msgr2=0x7f2f20109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:50.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.344+0000 7f2f24e00700 1 --2- 172.21.15.117:0/1168252742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20108e10 0x7f2f20109290 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009910 tx=0x7f2f0c009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:50.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.344+0000 7f2f24e00700 1 -- 172.21.15.117:0/1168252742 shutdown_connections 2024-04-20T11:49:50.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.344+0000 7f2f24e00700 1 --2- 172.21.15.117:0/1168252742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20108e10 0x7f2f20109290 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.344+0000 7f2f24e00700 1 --2- 172.21.15.117:0/1168252742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f20107520 0x7f2f20107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.344+0000 7f2f24e00700 1 -- 172.21.15.117:0/1168252742 >> 172.21.15.117:0/1168252742 conn(0x7f2f200768f0 msgr2=0x7f2f20078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:50.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.344+0000 7f2f24e00700 1 -- 172.21.15.117:0/1168252742 shutdown_connections 2024-04-20T11:49:50.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.344+0000 7f2f24e00700 1 -- 172.21.15.117:0/1168252742 wait complete. 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.345+0000 7f2f24e00700 1 Processor -- start 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.345+0000 7f2f24e00700 1 -- start start 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.346+0000 7f2f24e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20107520 0x7f2f2019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.346+0000 7f2f24e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f20108e10 0x7f2f2019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.346+0000 7f2f24e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2019ddf0 con 0x7f2f20107520 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.346+0000 7f2f24e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f201a1a70 con 0x7f2f20108e10 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.346+0000 7f2f1de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20107520 0x7f2f2019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.346+0000 7f2f1de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20107520 0x7f2f2019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44210/0 (socket says 172.21.15.117:44210) 2024-04-20T11:49:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.346+0000 7f2f1de00700 1 -- 172.21.15.117:0/504452020 learned_addr learned my addr 172.21.15.117:0/504452020 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:50.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.346+0000 7f2f1d400700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f20108e10 0x7f2f2019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:50.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f1de00700 1 -- 172.21.15.117:0/504452020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f20108e10 msgr2=0x7f2f2019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:50.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f1de00700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f20108e10 0x7f2f2019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f1de00700 1 -- 172.21.15.117:0/504452020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c0095f0 con 0x7f2f20107520 2024-04-20T11:49:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f1de00700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20107520 0x7f2f2019d1d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000c8a0 tx=0x7f2f1000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f16a00700 1 -- 172.21.15.117:0/504452020 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f1000cea0 con 0x7f2f20107520 2024-04-20T11:49:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f16a00700 1 -- 172.21.15.117:0/504452020 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f10004830 con 0x7f2f20107520 2024-04-20T11:49:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f201a1d50 con 0x7f2f20107520 2024-04-20T11:49:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f16a00700 1 -- 172.21.15.117:0/504452020 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f100055b0 con 0x7f2f20107520 2024-04-20T11:49:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.347+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f201a2320 con 0x7f2f20107520 2024-04-20T11:49:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.348+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2010f870 con 0x7f2f20107520 2024-04-20T11:49:50.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.349+0000 7f2f16a00700 1 -- 172.21.15.117:0/504452020 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2f100049a0 con 0x7f2f20107520 2024-04-20T11:49:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.350+0000 7f2f16a00700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2f0406ecd0 0x7f2f04071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.351+0000 7f2f16a00700 1 -- 172.21.15.117:0/504452020 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2f1008ebc0 con 0x7f2f20107520 2024-04-20T11:49:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.351+0000 7f2f1d400700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2f0406ecd0 0x7f2f04071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:50.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.351+0000 7f2f1d400700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2f0406ecd0 0x7f2f04071190 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c006010 tx=0x7f2f0c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:50.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.355+0000 7f2f16a00700 1 -- 172.21.15.117:0/504452020 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2f10057900 con 0x7f2f20107520 2024-04-20T11:49:50.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:50 smithi117 ceph-mon[99036]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:50.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:50 smithi117 ceph-mon[99036]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:50.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:50 smithi117 ceph-mon[99036]: pgmap v1122: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:50.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:50 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2399561260' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:49:50.486 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:50 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2693987256' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:49:50.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.588+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f201144c0 con 0x7f2f0406ecd0 2024-04-20T11:49:50.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.590+0000 7f2f16a00700 1 -- 172.21.15.117:0/504452020 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2f201144c0 con 0x7f2f0406ecd0 2024-04-20T11:49:50.590 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:49:50.590 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:49:50.590 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:49:50.590 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:49:50.591 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:49:50.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.593+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2f0406ecd0 msgr2=0x7f2f04071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:50.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.593+0000 7f2f24e00700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2f0406ecd0 0x7f2f04071190 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c006010 tx=0x7f2f0c00b540 comp rx=0 tx=0).stop 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.593+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20107520 msgr2=0x7f2f2019d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.593+0000 7f2f24e00700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20107520 0x7f2f2019d1d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000c8a0 tx=0x7f2f1000cc60 comp rx=0 tx=0).stop 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.594+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 shutdown_connections 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.594+0000 7f2f24e00700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2f0406ecd0 0x7f2f04071190 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.594+0000 7f2f24e00700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f20107520 0x7f2f2019d1d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.594+0000 7f2f24e00700 1 --2- 172.21.15.117:0/504452020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f20108e10 0x7f2f2019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.594+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 >> 172.21.15.117:0/504452020 conn(0x7f2f200768f0 msgr2=0x7f2f20077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.594+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 shutdown_connections 2024-04-20T11:49:50.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.594+0000 7f2f24e00700 1 -- 172.21.15.117:0/504452020 wait complete. 2024-04-20T11:49:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.748+0000 7f1729000700 1 -- 172.21.15.117:0/687899767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107df0 msgr2=0x7f172410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.748+0000 7f1729000700 1 --2- 172.21.15.117:0/687899767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107df0 0x7f172410be40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1710009910 tx=0x7f1710009c20 comp rx=0 tx=0).stop 2024-04-20T11:49:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.748+0000 7f1729000700 1 -- 172.21.15.117:0/687899767 shutdown_connections 2024-04-20T11:49:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.748+0000 7f1729000700 1 --2- 172.21.15.117:0/687899767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107df0 0x7f172410be40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.748+0000 7f1729000700 1 --2- 172.21.15.117:0/687899767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724107440 0x7f1724107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.748+0000 7f1729000700 1 -- 172.21.15.117:0/687899767 >> 172.21.15.117:0/687899767 conn(0x7f1724076a10 msgr2=0x7f1724078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.748+0000 7f1729000700 1 -- 172.21.15.117:0/687899767 shutdown_connections 2024-04-20T11:49:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.748+0000 7f1729000700 1 -- 172.21.15.117:0/687899767 wait complete. 2024-04-20T11:49:50.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1729000700 1 Processor -- start 2024-04-20T11:49:50.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1729000700 1 -- start start 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1729000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107440 0x7f172419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1729000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724107df0 0x7f172419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1729000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172419dd10 con 0x7f1724107440 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1729000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17241a19e0 con 0x7f1724107df0 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1721e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107440 0x7f172419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1721e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107440 0x7f172419d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44234/0 (socket says 172.21.15.117:44234) 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.750+0000 7f1721e00700 1 -- 172.21.15.117:0/2751947446 learned_addr learned my addr 172.21.15.117:0/2751947446 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.751+0000 7f1721400700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724107df0 0x7f172419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.751+0000 7f1721e00700 1 -- 172.21.15.117:0/2751947446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724107df0 msgr2=0x7f172419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.751+0000 7f1721e00700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724107df0 0x7f172419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.751+0000 7f1721e00700 1 -- 172.21.15.117:0/2751947446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17100095f0 con 0x7f1724107440 2024-04-20T11:49:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.751+0000 7f1721e00700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107440 0x7f172419d0f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f171400c8a0 tx=0x7f171400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.752+0000 7f171aa00700 1 -- 172.21.15.117:0/2751947446 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f171400cea0 con 0x7f1724107440 2024-04-20T11:49:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.752+0000 7f171aa00700 1 -- 172.21.15.117:0/2751947446 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1714004830 con 0x7f1724107440 2024-04-20T11:49:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.752+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17241a1cc0 con 0x7f1724107440 2024-04-20T11:49:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.752+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17241a2210 con 0x7f1724107440 2024-04-20T11:49:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.752+0000 7f171aa00700 1 -- 172.21.15.117:0/2751947446 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17140055b0 con 0x7f1724107440 2024-04-20T11:49:50.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.753+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172410f840 con 0x7f1724107440 2024-04-20T11:49:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.754+0000 7f171aa00700 1 -- 172.21.15.117:0/2751947446 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f17140049a0 con 0x7f1724107440 2024-04-20T11:49:50.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.755+0000 7f171aa00700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f170806ecd0 0x7f1708071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:49:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.755+0000 7f171aa00700 1 -- 172.21.15.117:0/2751947446 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f171408eba0 con 0x7f1724107440 2024-04-20T11:49:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.756+0000 7f1721400700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f170806ecd0 0x7f1708071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:49:50.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.756+0000 7f1721400700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f170806ecd0 0x7f1708071190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f172419e650 tx=0x7f171001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:49:50.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:50.760+0000 7f171aa00700 1 -- 172.21.15.117:0/2751947446 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f17140578e0 con 0x7f1724107440 2024-04-20T11:49:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:50 smithi173 ceph-mon[74165]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:50 smithi173 ceph-mon[74165]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:50 smithi173 ceph-mon[74165]: pgmap v1122: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:50 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2399561260' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:49:50.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:50 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2693987256' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:49:51.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.069+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f172419e400 con 0x7f1724107440 2024-04-20T11:49:51.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.070+0000 7f171aa00700 1 -- 172.21.15.117:0/2751947446 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f1714004c50 con 0x7f1724107440 2024-04-20T11:49:51.070 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:49:51.070 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:49:51.070 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:49:51.070 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:49:51.070 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:49:51.070 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:49:51.070 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:49:51.070 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:49:51.071 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:49:51.072 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:49:51.072 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:49:51.072 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:49:51.072 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f170806ecd0 msgr2=0x7f1708071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f170806ecd0 0x7f1708071190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f172419e650 tx=0x7f171001a040 comp rx=0 tx=0).stop 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107440 msgr2=0x7f172419d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107440 0x7f172419d0f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f171400c8a0 tx=0x7f171400cc60 comp rx=0 tx=0).stop 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 shutdown_connections 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f170806ecd0 0x7f1708071190 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724107440 0x7f172419d0f0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 --2- 172.21.15.117:0/2751947446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724107df0 0x7f172419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 >> 172.21.15.117:0/2751947446 conn(0x7f1724076a10 msgr2=0x7f1724076df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 shutdown_connections 2024-04-20T11:49:51.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:49:51.073+0000 7f1729000700 1 -- 172.21.15.117:0/2751947446 wait complete. 2024-04-20T11:49:51.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:51 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2751947446' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:49:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:51 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2751947446' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:49:52.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:52 smithi117 ceph-mon[99036]: from='client.36010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:52.792 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:52 smithi117 ceph-mon[99036]: pgmap v1123: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:52 smithi173 ceph-mon[74165]: from='client.36010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:49:52.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:52 smithi173 ceph-mon[74165]: pgmap v1123: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:54.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:54 smithi117 ceph-mon[99036]: pgmap v1124: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:54 smithi173 ceph-mon[74165]: pgmap v1124: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:56.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:56 smithi117 ceph-mon[99036]: pgmap v1125: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:56.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:56 smithi173 ceph-mon[74165]: pgmap v1125: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:58 smithi117 ceph-mon[99036]: pgmap v1126: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:49:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:49:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:58 smithi173 ceph-mon[74165]: pgmap v1126: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:49:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:49:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:50:00.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:00 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:49:59.999+0000 7fe6aa000700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:50:00.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:00 smithi117 ceph-mon[99036]: pgmap v1127: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:00.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:00 smithi117 ceph-mon[99036]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:50:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:00 smithi173 ceph-mon[74165]: pgmap v1127: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:00.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:00 smithi173 ceph-mon[74165]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:50:02.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:02 smithi117 ceph-mon[99036]: pgmap v1128: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:02.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:02 smithi173 ceph-mon[74165]: pgmap v1128: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:04.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:04 smithi117 ceph-mon[99036]: pgmap v1129: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:04.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:04 smithi173 ceph-mon[74165]: pgmap v1129: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:06.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:06 smithi117 ceph-mon[99036]: pgmap v1130: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:06 smithi173 ceph-mon[74165]: pgmap v1130: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:08.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:08 smithi173 ceph-mon[74165]: pgmap v1131: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:08.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:08 smithi117 ceph-mon[99036]: pgmap v1131: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:10.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:10 smithi117 ceph-mon[99036]: pgmap v1132: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:10 smithi173 ceph-mon[74165]: pgmap v1132: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:12.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:12 smithi117 ceph-mon[99036]: pgmap v1133: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:12.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:12 smithi173 ceph-mon[74165]: pgmap v1133: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:13.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:50:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:50:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:14 smithi117 ceph-mon[99036]: pgmap v1134: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:14 smithi173 ceph-mon[74165]: pgmap v1134: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:15 smithi117 ceph-mon[99036]: pgmap v1135: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:15 smithi173 ceph-mon[74165]: pgmap v1135: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:18.657 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:18 smithi117 ceph-mon[99036]: pgmap v1136: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:18.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:18 smithi173 ceph-mon[74165]: pgmap v1136: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:20.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:20 smithi117 ceph-mon[99036]: pgmap v1137: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:20 smithi173 ceph-mon[74165]: pgmap v1137: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:21.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.241+0000 7ff0f6400700 1 -- 172.21.15.117:0/1481718501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f01017d0 msgr2=0x7ff0f0105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:21.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.241+0000 7ff0f6400700 1 --2- 172.21.15.117:0/1481718501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f01017d0 0x7ff0f0105820 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009910 tx=0x7ff0dc009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:21.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.241+0000 7ff0f6400700 1 -- 172.21.15.117:0/1481718501 shutdown_connections 2024-04-20T11:50:21.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.241+0000 7ff0f6400700 1 --2- 172.21.15.117:0/1481718501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f01017d0 0x7ff0f0105820 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.241+0000 7ff0f6400700 1 --2- 172.21.15.117:0/1481718501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f0100e20 0x7ff0f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.241+0000 7ff0f6400700 1 -- 172.21.15.117:0/1481718501 >> 172.21.15.117:0/1481718501 conn(0x7ff0f00fc6b0 msgr2=0x7ff0f00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:21.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.242+0000 7ff0f6400700 1 -- 172.21.15.117:0/1481718501 shutdown_connections 2024-04-20T11:50:21.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.242+0000 7ff0f6400700 1 -- 172.21.15.117:0/1481718501 wait complete. 2024-04-20T11:50:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.243+0000 7ff0f6400700 1 Processor -- start 2024-04-20T11:50:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.243+0000 7ff0f6400700 1 -- start start 2024-04-20T11:50:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.243+0000 7ff0f6400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f0100e20 0x7ff0f00731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.243+0000 7ff0f6400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f01017d0 0x7ff0f0073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.243+0000 7ff0f6400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0075160 con 0x7ff0f0100e20 2024-04-20T11:50:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.243+0000 7ff0f6400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0073c60 con 0x7ff0f01017d0 2024-04-20T11:50:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0ef400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f0100e20 0x7ff0f00731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0ef400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f0100e20 0x7ff0f00731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48894/0 (socket says 172.21.15.117:48894) 2024-04-20T11:50:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0ef400700 1 -- 172.21.15.117:0/3949729276 learned_addr learned my addr 172.21.15.117:0/3949729276 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0eea00700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f01017d0 0x7ff0f0073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0ef400700 1 -- 172.21.15.117:0/3949729276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f01017d0 msgr2=0x7ff0f0073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0ef400700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f01017d0 0x7ff0f0073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0ef400700 1 -- 172.21.15.117:0/3949729276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc0095f0 con 0x7ff0f0100e20 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0eea00700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f01017d0 0x7ff0f0073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.244+0000 7ff0ef400700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f0100e20 0x7ff0f00731e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c930 tx=0x7ff0e000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.245+0000 7ff0e7e00700 1 -- 172.21.15.117:0/3949729276 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0e0007ad0 con 0x7ff0f0100e20 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.245+0000 7ff0e7e00700 1 -- 172.21.15.117:0/3949729276 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0e0007c30 con 0x7ff0f0100e20 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.245+0000 7ff0e7e00700 1 -- 172.21.15.117:0/3949729276 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0e0018610 con 0x7ff0f0100e20 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.245+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f0073f40 con 0x7ff0f0100e20 2024-04-20T11:50:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.245+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f01a2fc0 con 0x7ff0f0100e20 2024-04-20T11:50:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.246+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f0109220 con 0x7ff0f0100e20 2024-04-20T11:50:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.247+0000 7ff0e7e00700 1 -- 172.21.15.117:0/3949729276 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff0e000f450 con 0x7ff0f0100e20 2024-04-20T11:50:21.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.248+0000 7ff0e7e00700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0d406ecc0 0x7ff0d4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.248+0000 7ff0e7e00700 1 -- 172.21.15.117:0/3949729276 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff0e008efd0 con 0x7ff0f0100e20 2024-04-20T11:50:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.248+0000 7ff0eea00700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0d406ecc0 0x7ff0d4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.249+0000 7ff0eea00700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0d406ecc0 0x7ff0d4071180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc006010 tx=0x7ff0dc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:21.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.254+0000 7ff0e7e00700 1 -- 172.21.15.117:0/3949729276 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff0e00c3ff0 con 0x7ff0f0100e20 2024-04-20T11:50:21.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.502+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0f010ddc0 con 0x7ff0d406ecc0 2024-04-20T11:50:21.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.504+0000 7ff0e7e00700 1 -- 172.21.15.117:0/3949729276 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff0f010ddc0 con 0x7ff0d406ecc0 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.507+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0d406ecc0 msgr2=0x7ff0d4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.507+0000 7ff0f6400700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0d406ecc0 0x7ff0d4071180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc006010 tx=0x7ff0dc00b540 comp rx=0 tx=0).stop 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.507+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f0100e20 msgr2=0x7ff0f00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.507+0000 7ff0f6400700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f0100e20 0x7ff0f00731e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c930 tx=0x7ff0e000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.508+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 shutdown_connections 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.508+0000 7ff0f6400700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff0d406ecc0 0x7ff0d4071180 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.508+0000 7ff0f6400700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f0100e20 0x7ff0f00731e0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.508+0000 7ff0f6400700 1 --2- 172.21.15.117:0/3949729276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0f01017d0 0x7ff0f0073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.508+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 >> 172.21.15.117:0/3949729276 conn(0x7ff0f00fc6b0 msgr2=0x7ff0f00fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.508+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 shutdown_connections 2024-04-20T11:50:21.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.508+0000 7ff0f6400700 1 -- 172.21.15.117:0/3949729276 wait complete. 2024-04-20T11:50:21.525 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:50:21.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.669+0000 7f34a9600700 1 -- 172.21.15.117:0/664871648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 msgr2=0x7f34a410cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:21.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.669+0000 7f34a9600700 1 --2- 172.21.15.117:0/664871648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 0x7f34a410cf30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3490009910 tx=0x7f3490009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:21.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.670+0000 7f34a9600700 1 -- 172.21.15.117:0/664871648 shutdown_connections 2024-04-20T11:50:21.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.670+0000 7f34a9600700 1 --2- 172.21.15.117:0/664871648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 0x7f34a410cf30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.670+0000 7f34a9600700 1 --2- 172.21.15.117:0/664871648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34a4073200 0x7f34a40735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.670+0000 7f34a9600700 1 -- 172.21.15.117:0/664871648 >> 172.21.15.117:0/664871648 conn(0x7f34a40fc6a0 msgr2=0x7f34a40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:21.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.670+0000 7f34a9600700 1 -- 172.21.15.117:0/664871648 shutdown_connections 2024-04-20T11:50:21.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.670+0000 7f34a9600700 1 -- 172.21.15.117:0/664871648 wait complete. 2024-04-20T11:50:21.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.671+0000 7f34a9600700 1 Processor -- start 2024-04-20T11:50:21.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.671+0000 7f34a9600700 1 -- start start 2024-04-20T11:50:21.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.672+0000 7f34a9600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34a4073200 0x7f34a4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:21.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.672+0000 7f34a9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 0x7f34a4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:21.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.672+0000 7f34a9600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4199830 con 0x7f34a4073b20 2024-04-20T11:50:21.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.672+0000 7f34a9600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a41999a0 con 0x7f34a4073200 2024-04-20T11:50:21.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.672+0000 7f34a2000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 0x7f34a4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:21.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.672+0000 7f34a2000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 0x7f34a4199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48920/0 (socket says 172.21.15.117:48920) 2024-04-20T11:50:21.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.672+0000 7f34a2000700 1 -- 172.21.15.117:0/315954211 learned_addr learned my addr 172.21.15.117:0/315954211 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:21.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.672+0000 7f34a2a00700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34a4073200 0x7f34a4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:21.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.673+0000 7f34a2000700 1 -- 172.21.15.117:0/315954211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34a4073200 msgr2=0x7f34a4198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:21.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.673+0000 7f34a2000700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34a4073200 0x7f34a4198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.673+0000 7f34a2000700 1 -- 172.21.15.117:0/315954211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34900095f0 con 0x7f34a4073b20 2024-04-20T11:50:21.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.673+0000 7f34a2000700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 0x7f34a4199230 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f34900092e0 tx=0x7f349000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:21.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.673+0000 7f349b400700 1 -- 172.21.15.117:0/315954211 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f349001d070 con 0x7f34a4073b20 2024-04-20T11:50:21.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.673+0000 7f349b400700 1 -- 172.21.15.117:0/315954211 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f349000f460 con 0x7f34a4073b20 2024-04-20T11:50:21.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.673+0000 7f349b400700 1 -- 172.21.15.117:0/315954211 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3490017630 con 0x7f34a4073b20 2024-04-20T11:50:21.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.673+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a419c620 con 0x7f34a4073b20 2024-04-20T11:50:21.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.674+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a419cb10 con 0x7f34a4073b20 2024-04-20T11:50:21.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.675+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a4074940 con 0x7f34a4073b20 2024-04-20T11:50:21.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.676+0000 7f349b400700 1 -- 172.21.15.117:0/315954211 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f349000f5d0 con 0x7f34a4073b20 2024-04-20T11:50:21.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.677+0000 7f349b400700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f348806ecd0 0x7f3488071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:21.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.677+0000 7f349b400700 1 -- 172.21.15.117:0/315954211 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3490090320 con 0x7f34a4073b20 2024-04-20T11:50:21.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.677+0000 7f34a2a00700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f348806ecd0 0x7f3488071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:21.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.678+0000 7f34a2a00700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f348806ecd0 0x7f3488071190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3494005ea0 tx=0x7f3494005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:21.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.682+0000 7f349b400700 1 -- 172.21.15.117:0/315954211 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f34900590e0 con 0x7f34a4073b20 2024-04-20T11:50:21.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.930+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34a41077d0 con 0x7f348806ecd0 2024-04-20T11:50:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.932+0000 7f349b400700 1 -- 172.21.15.117:0/315954211 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f34a41077d0 con 0x7f348806ecd0 2024-04-20T11:50:21.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.935+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f348806ecd0 msgr2=0x7f3488071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:21.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.935+0000 7f34a9600700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f348806ecd0 0x7f3488071190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3494005ea0 tx=0x7f3494005e30 comp rx=0 tx=0).stop 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.935+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 msgr2=0x7f34a4199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.935+0000 7f34a9600700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 0x7f34a4199230 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f34900092e0 tx=0x7f349000be20 comp rx=0 tx=0).stop 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.935+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 shutdown_connections 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.936+0000 7f34a9600700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f348806ecd0 0x7f3488071190 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.936+0000 7f34a9600700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34a4073200 0x7f34a4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.936+0000 7f34a9600700 1 --2- 172.21.15.117:0/315954211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34a4073b20 0x7f34a4199230 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.936+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 >> 172.21.15.117:0/315954211 conn(0x7f34a40fc6a0 msgr2=0x7f34a4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.936+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 shutdown_connections 2024-04-20T11:50:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:21.936+0000 7f34a9600700 1 -- 172.21.15.117:0/315954211 wait complete. 2024-04-20T11:50:22.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.094+0000 7f4c76200700 1 -- 172.21.15.117:0/550518245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c701017c0 msgr2=0x7f4c70105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:22.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.094+0000 7f4c76200700 1 --2- 172.21.15.117:0/550518245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c701017c0 0x7f4c70105810 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009910 tx=0x7f4c58009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:22.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.094+0000 7f4c76200700 1 -- 172.21.15.117:0/550518245 shutdown_connections 2024-04-20T11:50:22.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.094+0000 7f4c76200700 1 --2- 172.21.15.117:0/550518245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c701017c0 0x7f4c70105810 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.094+0000 7f4c76200700 1 --2- 172.21.15.117:0/550518245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c70100e10 0x7f4c701011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.094+0000 7f4c76200700 1 -- 172.21.15.117:0/550518245 >> 172.21.15.117:0/550518245 conn(0x7f4c700fc6a0 msgr2=0x7f4c700feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:22.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.094+0000 7f4c76200700 1 -- 172.21.15.117:0/550518245 shutdown_connections 2024-04-20T11:50:22.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.095+0000 7f4c76200700 1 -- 172.21.15.117:0/550518245 wait complete. 2024-04-20T11:50:22.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.096+0000 7f4c76200700 1 Processor -- start 2024-04-20T11:50:22.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.096+0000 7f4c76200700 1 -- start start 2024-04-20T11:50:22.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.096+0000 7f4c76200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c70100e10 0x7f4c700731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:22.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.096+0000 7f4c76200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c701017c0 0x7f4c70073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:22.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.096+0000 7f4c76200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70075160 con 0x7f4c70100e10 2024-04-20T11:50:22.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.096+0000 7f4c76200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70073c60 con 0x7f4c701017c0 2024-04-20T11:50:22.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.097+0000 7f4c6f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c70100e10 0x7f4c700731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:22.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.097+0000 7f4c6f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c70100e10 0x7f4c700731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48940/0 (socket says 172.21.15.117:48940) 2024-04-20T11:50:22.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.097+0000 7f4c6f400700 1 -- 172.21.15.117:0/2228243365 learned_addr learned my addr 172.21.15.117:0/2228243365 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:22.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.097+0000 7f4c6f400700 1 -- 172.21.15.117:0/2228243365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c701017c0 msgr2=0x7f4c70073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:22.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.097+0000 7f4c6f400700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c701017c0 0x7f4c70073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.097+0000 7f4c6f400700 1 -- 172.21.15.117:0/2228243365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580095f0 con 0x7f4c70100e10 2024-04-20T11:50:22.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.097+0000 7f4c6f400700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c70100e10 0x7f4c700731e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c930 tx=0x7f4c6000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:22.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.098+0000 7f4c6cc00700 1 -- 172.21.15.117:0/2228243365 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c60007ad0 con 0x7f4c70100e10 2024-04-20T11:50:22.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.098+0000 7f4c6cc00700 1 -- 172.21.15.117:0/2228243365 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c60007c30 con 0x7f4c70100e10 2024-04-20T11:50:22.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.098+0000 7f4c6cc00700 1 -- 172.21.15.117:0/2228243365 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c60018610 con 0x7f4c70100e10 2024-04-20T11:50:22.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.098+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c70073f40 con 0x7f4c70100e10 2024-04-20T11:50:22.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.098+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c701a3030 con 0x7f4c70100e10 2024-04-20T11:50:22.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.099+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c70109210 con 0x7f4c70100e10 2024-04-20T11:50:22.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.100+0000 7f4c6cc00700 1 -- 172.21.15.117:0/2228243365 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4c6000f450 con 0x7f4c70100e10 2024-04-20T11:50:22.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.101+0000 7f4c6cc00700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c5406ecd0 0x7f4c54071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:22.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.101+0000 7f4c6cc00700 1 -- 172.21.15.117:0/2228243365 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f4c6008efe0 con 0x7f4c70100e10 2024-04-20T11:50:22.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.101+0000 7f4c67e00700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c5406ecd0 0x7f4c54071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:22.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.102+0000 7f4c67e00700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c5406ecd0 0x7f4c54071190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4c58006010 tx=0x7f4c5800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:22.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.106+0000 7f4c6cc00700 1 -- 172.21.15.117:0/2228243365 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4c60057de0 con 0x7f4c70100e10 2024-04-20T11:50:22.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.419+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c7010ddb0 con 0x7f4c5406ecd0 2024-04-20T11:50:22.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.455+0000 7f4c6cc00700 1 -- 172.21.15.117:0/2228243365 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4c7010ddb0 con 0x7f4c5406ecd0 2024-04-20T11:50:22.456 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:50:22.456 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (38m) 14m ago 52m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (53m) 14m ago 53m 10.3M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 3c55ce272165 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (51m) 14m ago 51m 10.1M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ed636c361e8 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (35m) 14m ago 53m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (35m) 14m ago 51m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (38m) 14m ago 52m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (30m) 14m ago 45m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (31m) 14m ago 45m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (14m) 14m ago 45m 11.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (30m) 14m ago 45m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (41m) 14m ago 54m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (40m) 14m ago 51m 459M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (36m) 14m ago 54m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (36m) 14m ago 51m 48.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (40m) 14m ago 53m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:50:22.457 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (39m) 14m ago 51m 8056k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:50:22.458 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (34m) 14m ago 50m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:50:22.458 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (33m) 14m ago 49m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:50:22.458 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (33m) 14m ago 49m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:50:22.458 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (32m) 14m ago 48m 93.7M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:50:22.458 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (32m) 14m ago 48m 86.9M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:50:22.458 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (31m) 14m ago 47m 81.1M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:50:22.458 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (39m) 14m ago 52m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:50:22.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c5406ecd0 msgr2=0x7f4c54071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:22.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c5406ecd0 0x7f4c54071190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4c58006010 tx=0x7f4c5800b540 comp rx=0 tx=0).stop 2024-04-20T11:50:22.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c70100e10 msgr2=0x7f4c700731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c70100e10 0x7f4c700731e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c930 tx=0x7f4c6000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 shutdown_connections 2024-04-20T11:50:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c5406ecd0 0x7f4c54071190 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c70100e10 0x7f4c700731e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 --2- 172.21.15.117:0/2228243365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c701017c0 0x7f4c70073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 >> 172.21.15.117:0/2228243365 conn(0x7f4c700fc6a0 msgr2=0x7f4c700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 shutdown_connections 2024-04-20T11:50:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.459+0000 7f4c76200700 1 -- 172.21.15.117:0/2228243365 wait complete. 2024-04-20T11:50:22.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.613+0000 7f1320000700 1 -- 172.21.15.117:0/2185645847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102660 msgr2=0x7f131810ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:22.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.613+0000 7f1320000700 1 --2- 172.21.15.117:0/2185645847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102660 0x7f131810ab50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f1304009910 tx=0x7f1304009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:22.614 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:22 smithi117 ceph-mon[99036]: pgmap v1138: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:22.614 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:22 smithi117 ceph-mon[99036]: from='client.36018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:22.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.613+0000 7f1320000700 1 -- 172.21.15.117:0/2185645847 shutdown_connections 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.613+0000 7f1320000700 1 --2- 172.21.15.117:0/2185645847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318102660 0x7f131810ab50 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.613+0000 7f1320000700 1 --2- 172.21.15.117:0/2185645847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318101d40 0x7f1318102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.613+0000 7f1320000700 1 -- 172.21.15.117:0/2185645847 >> 172.21.15.117:0/2185645847 conn(0x7f1318076750 msgr2=0x7f13180fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.613+0000 7f1320000700 1 -- 172.21.15.117:0/2185645847 shutdown_connections 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.613+0000 7f1320000700 1 -- 172.21.15.117:0/2185645847 wait complete. 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.614+0000 7f1320000700 1 Processor -- start 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.615+0000 7f1320000700 1 -- start start 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.615+0000 7f1320000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318101d40 0x7f131819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.615+0000 7f1320000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318102660 0x7f131819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.615+0000 7f1320000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131819d3a0 con 0x7f1318101d40 2024-04-20T11:50:22.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.615+0000 7f1320000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318196980 con 0x7f1318102660 2024-04-20T11:50:22.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.615+0000 7f131d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318101d40 0x7f131819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.615+0000 7f131d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318101d40 0x7f131819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48952/0 (socket says 172.21.15.117:48952) 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.615+0000 7f131d600700 1 -- 172.21.15.117:0/198838660 learned_addr learned my addr 172.21.15.117:0/198838660 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.616+0000 7f131cc00700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318102660 0x7f131819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.616+0000 7f131d600700 1 -- 172.21.15.117:0/198838660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318102660 msgr2=0x7f131819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.616+0000 7f131d600700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318102660 0x7f131819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.616+0000 7f131d600700 1 -- 172.21.15.117:0/198838660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13040095f0 con 0x7f1318101d40 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.616+0000 7f131d600700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318101d40 0x7f131819c840 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d8d0 tx=0x7f130c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.616+0000 7f1316000700 1 -- 172.21.15.117:0/198838660 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c009940 con 0x7f1318101d40 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.616+0000 7f1316000700 1 -- 172.21.15.117:0/198838660 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f130c010460 con 0x7f1318101d40 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.617+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1318196c60 con 0x7f1318101d40 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.617+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13181971b0 con 0x7f1318101d40 2024-04-20T11:50:22.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.617+0000 7f1316000700 1 -- 172.21.15.117:0/198838660 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c00f5f0 con 0x7f1318101d40 2024-04-20T11:50:22.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.618+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13181054d0 con 0x7f1318101d40 2024-04-20T11:50:22.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.619+0000 7f1316000700 1 -- 172.21.15.117:0/198838660 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f130c0105d0 con 0x7f1318101d40 2024-04-20T11:50:22.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.620+0000 7f1316000700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f130006ecc0 0x7f1300071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:22.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.620+0000 7f1316000700 1 -- 172.21.15.117:0/198838660 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f130c08eee0 con 0x7f1318101d40 2024-04-20T11:50:22.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.620+0000 7f131cc00700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f130006ecc0 0x7f1300071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:22.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.621+0000 7f131cc00700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f130006ecc0 0x7f1300071180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1304006010 tx=0x7f130400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:22.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.625+0000 7f1316000700 1 -- 172.21.15.117:0/198838660 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f130c05b750 con 0x7f1318101d40 2024-04-20T11:50:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:22 smithi173 ceph-mon[74165]: pgmap v1138: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:22.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:22 smithi173 ceph-mon[74165]: from='client.36018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:22.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.941+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1318197e40 con 0x7f1318101d40 2024-04-20T11:50:22.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.942+0000 7f1316000700 1 -- 172.21.15.117:0/198838660 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f130c05b2e0 con 0x7f1318101d40 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:50:22.943 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:50:22.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:50:22.944 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:50:22.944 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:50:22.944 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:50:22.944 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:50:22.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.944+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f130006ecc0 msgr2=0x7f1300071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:22.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.944+0000 7f1320000700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f130006ecc0 0x7f1300071180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1304006010 tx=0x7f130400b540 comp rx=0 tx=0).stop 2024-04-20T11:50:22.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.944+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318101d40 msgr2=0x7f131819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:22.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.944+0000 7f1320000700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318101d40 0x7f131819c840 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d8d0 tx=0x7f130c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:50:22.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.945+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 shutdown_connections 2024-04-20T11:50:22.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.945+0000 7f1320000700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f130006ecc0 0x7f1300071180 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.945+0000 7f1320000700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1318101d40 0x7f131819c840 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.945+0000 7f1320000700 1 --2- 172.21.15.117:0/198838660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318102660 0x7f131819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:22.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.945+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 >> 172.21.15.117:0/198838660 conn(0x7f1318076750 msgr2=0x7f1318103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:22.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.945+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 shutdown_connections 2024-04-20T11:50:22.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:22.945+0000 7f1320000700 1 -- 172.21.15.117:0/198838660 wait complete. 2024-04-20T11:50:23.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.103+0000 7f83ce800700 1 -- 172.21.15.117:0/3993842754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 msgr2=0x7f83c8109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.103+0000 7f83ce800700 1 --2- 172.21.15.117:0/3993842754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 0x7f83c8109290 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009910 tx=0x7f83b0009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:23.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.103+0000 7f83ce800700 1 -- 172.21.15.117:0/3993842754 shutdown_connections 2024-04-20T11:50:23.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.103+0000 7f83ce800700 1 --2- 172.21.15.117:0/3993842754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 0x7f83c8109290 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.103+0000 7f83ce800700 1 --2- 172.21.15.117:0/3993842754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c8107520 0x7f83c8107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.103+0000 7f83ce800700 1 -- 172.21.15.117:0/3993842754 >> 172.21.15.117:0/3993842754 conn(0x7f83c80768f0 msgr2=0x7f83c8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:23.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.103+0000 7f83ce800700 1 -- 172.21.15.117:0/3993842754 shutdown_connections 2024-04-20T11:50:23.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.103+0000 7f83ce800700 1 -- 172.21.15.117:0/3993842754 wait complete. 2024-04-20T11:50:23.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.104+0000 7f83ce800700 1 Processor -- start 2024-04-20T11:50:23.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.104+0000 7f83ce800700 1 -- start start 2024-04-20T11:50:23.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83ce800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c8107520 0x7f83c819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 0x7f83c819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83ce800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c819ded0 con 0x7f83c8108e10 2024-04-20T11:50:23.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83ce800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81a1ba0 con 0x7f83c8107520 2024-04-20T11:50:23.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 0x7f83c819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 0x7f83c819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48970/0 (socket says 172.21.15.117:48970) 2024-04-20T11:50:23.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83bfe00700 1 -- 172.21.15.117:0/1536913666 learned_addr learned my addr 172.21.15.117:0/1536913666 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83c7400700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c8107520 0x7f83c819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83bfe00700 1 -- 172.21.15.117:0/1536913666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c8107520 msgr2=0x7f83c819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83bfe00700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c8107520 0x7f83c819d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.105+0000 7f83bfe00700 1 -- 172.21.15.117:0/1536913666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00095f0 con 0x7f83c8108e10 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.106+0000 7f83c7400700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c8107520 0x7f83c819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.106+0000 7f83bfe00700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 0x7f83c819d7f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f83b00092e0 tx=0x7f83b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.106+0000 7f83c4c00700 1 -- 172.21.15.117:0/1536913666 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b001d070 con 0x7f83c8108e10 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.106+0000 7f83c4c00700 1 -- 172.21.15.117:0/1536913666 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83b000f460 con 0x7f83c8108e10 2024-04-20T11:50:23.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.106+0000 7f83c4c00700 1 -- 172.21.15.117:0/1536913666 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b0017630 con 0x7f83c8108e10 2024-04-20T11:50:23.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.106+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c81a1e20 con 0x7f83c8108e10 2024-04-20T11:50:23.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.106+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c81a2390 con 0x7f83c8108e10 2024-04-20T11:50:23.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.107+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c810f890 con 0x7f83c8108e10 2024-04-20T11:50:23.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.108+0000 7f83c4c00700 1 -- 172.21.15.117:0/1536913666 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f83b000f5d0 con 0x7f83c8108e10 2024-04-20T11:50:23.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.109+0000 7f83c4c00700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83ac06ecd0 0x7f83ac071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.109+0000 7f83c4c00700 1 -- 172.21.15.117:0/1536913666 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f83b00903e0 con 0x7f83c8108e10 2024-04-20T11:50:23.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.110+0000 7f83c7400700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83ac06ecd0 0x7f83ac071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.110+0000 7f83c7400700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83ac06ecd0 0x7f83ac071190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f83b8007900 tx=0x7f83b8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:23.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.114+0000 7f83c4c00700 1 -- 172.21.15.117:0/1536913666 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f83b0059120 con 0x7f83c8108e10 2024-04-20T11:50:23.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.390+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f83c8114540 con 0x7f83c8108e10 2024-04-20T11:50:23.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.391+0000 7f83c4c00700 1 -- 172.21.15.117:0/1536913666 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f83b005c7e0 con 0x7f83c8108e10 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:e22 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 13 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:19:07.882610+0000 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:50:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:up {} 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:failed 0,1 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:50:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 0 members: 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:50:23.394 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:50:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83ac06ecd0 msgr2=0x7f83ac071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83ac06ecd0 0x7f83ac071190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f83b8007900 tx=0x7f83b8008040 comp rx=0 tx=0).stop 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 msgr2=0x7f83c819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 0x7f83c819d7f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f83b00092e0 tx=0x7f83b000be20 comp rx=0 tx=0).stop 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 shutdown_connections 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f83ac06ecd0 0x7f83ac071190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c8107520 0x7f83c819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 --2- 172.21.15.117:0/1536913666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8108e10 0x7f83c819d7f0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 >> 172.21.15.117:0/1536913666 conn(0x7f83c80768f0 msgr2=0x7f83c80775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 shutdown_connections 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.394+0000 7f83ce800700 1 -- 172.21.15.117:0/1536913666 wait complete. 2024-04-20T11:50:23.396 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:50:23.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.541+0000 7f7178a00700 1 -- 172.21.15.117:0/1686169544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71741070b0 msgr2=0x7f7174107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.541+0000 7f7178a00700 1 --2- 172.21.15.117:0/1686169544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71741070b0 0x7f7174107510 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7160009910 tx=0x7f7160009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:23.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.541+0000 7f7178a00700 1 -- 172.21.15.117:0/1686169544 shutdown_connections 2024-04-20T11:50:23.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.541+0000 7f7178a00700 1 --2- 172.21.15.117:0/1686169544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71741070b0 0x7f7174107510 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.541+0000 7f7178a00700 1 --2- 172.21.15.117:0/1686169544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f717410d0c0 0x7f717410d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.541+0000 7f7178a00700 1 -- 172.21.15.117:0/1686169544 >> 172.21.15.117:0/1686169544 conn(0x7f7174076830 msgr2=0x7f7174078c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:23.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.541+0000 7f7178a00700 1 -- 172.21.15.117:0/1686169544 shutdown_connections 2024-04-20T11:50:23.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.541+0000 7f7178a00700 1 -- 172.21.15.117:0/1686169544 wait complete. 2024-04-20T11:50:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.542+0000 7f7178a00700 1 Processor -- start 2024-04-20T11:50:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.542+0000 7f7178a00700 1 -- start start 2024-04-20T11:50:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7178a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71741070b0 0x7f717419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7178a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717410d0c0 0x7f717419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7178a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717419db80 con 0x7f717410d0c0 2024-04-20T11:50:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7178a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71741a1850 con 0x7f71741070b0 2024-04-20T11:50:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7172a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717410d0c0 0x7f717419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7172a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717410d0c0 0x7f717419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48982/0 (socket says 172.21.15.117:48982) 2024-04-20T11:50:23.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7172a00700 1 -- 172.21.15.117:0/2128366202 learned_addr learned my addr 172.21.15.117:0/2128366202 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:23.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7173400700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71741070b0 0x7f717419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7172a00700 1 -- 172.21.15.117:0/2128366202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71741070b0 msgr2=0x7f717419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.543+0000 7f7172a00700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71741070b0 0x7f717419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.544+0000 7f7172a00700 1 -- 172.21.15.117:0/2128366202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71600095f0 con 0x7f717410d0c0 2024-04-20T11:50:23.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.544+0000 7f7173400700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71741070b0 0x7f717419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:50:23.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.544+0000 7f7172a00700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717410d0c0 0x7f717419d4a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f716000bd70 tx=0x7f716000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:23.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.544+0000 7f716be00700 1 -- 172.21.15.117:0/2128366202 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f716001d070 con 0x7f717410d0c0 2024-04-20T11:50:23.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.544+0000 7f716be00700 1 -- 172.21.15.117:0/2128366202 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f716000f460 con 0x7f717410d0c0 2024-04-20T11:50:23.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.544+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71741a1ad0 con 0x7f717410d0c0 2024-04-20T11:50:23.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.544+0000 7f716be00700 1 -- 172.21.15.117:0/2128366202 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7160017630 con 0x7f717410d0c0 2024-04-20T11:50:23.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.544+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71741a1fc0 con 0x7f717410d0c0 2024-04-20T11:50:23.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.545+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717410f5c0 con 0x7f717410d0c0 2024-04-20T11:50:23.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.546+0000 7f716be00700 1 -- 172.21.15.117:0/2128366202 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f716000f5d0 con 0x7f717410d0c0 2024-04-20T11:50:23.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.547+0000 7f716be00700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f715806ecc0 0x7f7158071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.547+0000 7f716be00700 1 -- 172.21.15.117:0/2128366202 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7160090330 con 0x7f717410d0c0 2024-04-20T11:50:23.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.548+0000 7f7173400700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f715806ecc0 0x7f7158071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.548+0000 7f7173400700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f715806ecc0 0x7f7158071180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7164005fd0 tx=0x7f7164005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:23.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.552+0000 7f716be00700 1 -- 172.21.15.117:0/2128366202 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f71600590f0 con 0x7f717410d0c0 2024-04-20T11:50:23.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.795+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f717419e270 con 0x7f715806ecc0 2024-04-20T11:50:23.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:23 smithi117 ceph-mon[99036]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:23.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:23 smithi117 ceph-mon[99036]: from='client.36026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:23.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:23 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/198838660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:23.796 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:23 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1536913666' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.797+0000 7f716be00700 1 -- 172.21.15.117:0/2128366202 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f717419e270 con 0x7f715806ecc0 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:50:23.798 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:50:23.799 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:50:23.799 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:50:23.799 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:50:23.799 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "16/23 daemons upgraded", 2024-04-20T11:50:23.799 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading mds daemons", 2024-04-20T11:50:23.799 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:50:23.799 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:50:23.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f715806ecc0 msgr2=0x7f7158071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f715806ecc0 0x7f7158071180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7164005fd0 tx=0x7f7164005dc0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717410d0c0 msgr2=0x7f717419d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717410d0c0 0x7f717419d4a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f716000bd70 tx=0x7f716000be50 comp rx=0 tx=0).stop 2024-04-20T11:50:23.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 shutdown_connections 2024-04-20T11:50:23.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f715806ecc0 0x7f7158071180 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71741070b0 0x7f717419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 --2- 172.21.15.117:0/2128366202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717410d0c0 0x7f717419d4a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.800+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 >> 172.21.15.117:0/2128366202 conn(0x7f7174076830 msgr2=0x7f7174078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:23.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.801+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 shutdown_connections 2024-04-20T11:50:23.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.801+0000 7f7178a00700 1 -- 172.21.15.117:0/2128366202 wait complete. 2024-04-20T11:50:23.842 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:23 smithi173 ceph-mon[74165]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:23.843 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:23 smithi173 ceph-mon[74165]: from='client.36026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:23.843 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:23 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/198838660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:23.843 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:23 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1536913666' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:50:23.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.958+0000 7feed6600700 1 -- 172.21.15.117:0/2165933245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0103110 msgr2=0x7feed01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.958+0000 7feed6600700 1 --2- 172.21.15.117:0/2165933245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0103110 0x7feed01034f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7feec0009910 tx=0x7feec0009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:23.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.958+0000 7feed6600700 1 -- 172.21.15.117:0/2165933245 shutdown_connections 2024-04-20T11:50:23.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.958+0000 7feed6600700 1 --2- 172.21.15.117:0/2165933245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed0104a00 0x7feed0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.958+0000 7feed6600700 1 --2- 172.21.15.117:0/2165933245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0103110 0x7feed01034f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.958+0000 7feed6600700 1 -- 172.21.15.117:0/2165933245 >> 172.21.15.117:0/2165933245 conn(0x7feed00fe880 msgr2=0x7feed0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:23.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.958+0000 7feed6600700 1 -- 172.21.15.117:0/2165933245 shutdown_connections 2024-04-20T11:50:23.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.958+0000 7feed6600700 1 -- 172.21.15.117:0/2165933245 wait complete. 2024-04-20T11:50:23.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.959+0000 7feed6600700 1 Processor -- start 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.959+0000 7feed6600700 1 -- start start 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feed6600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed0103110 0x7feed0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feed6600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0104a00 0x7feed0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feed6600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01999e0 con 0x7feed0104a00 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feed6600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed019d660 con 0x7feed0103110 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feec7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0104a00 0x7feed0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feec7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0104a00 0x7feed0199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35834/0 (socket says 172.21.15.117:35834) 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feec7e00700 1 -- 172.21.15.117:0/786658742 learned_addr learned my addr 172.21.15.117:0/786658742 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:23.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feecf400700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed0103110 0x7feed0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feec7e00700 1 -- 172.21.15.117:0/786658742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed0103110 msgr2=0x7feed0198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:23.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feec7e00700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed0103110 0x7feed0198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:23.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feec7e00700 1 -- 172.21.15.117:0/786658742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00095f0 con 0x7feed0104a00 2024-04-20T11:50:23.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.960+0000 7feecf400700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed0103110 0x7feed0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:50:23.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.961+0000 7feec7e00700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0104a00 0x7feed0199300 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7feeb800d8d0 tx=0x7feeb800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:23.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.961+0000 7feeccc00700 1 -- 172.21.15.117:0/786658742 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeb8009940 con 0x7feed0104a00 2024-04-20T11:50:23.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.961+0000 7feeccc00700 1 -- 172.21.15.117:0/786658742 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeb8010460 con 0x7feed0104a00 2024-04-20T11:50:23.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.961+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed019d940 con 0x7feed0104a00 2024-04-20T11:50:23.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.961+0000 7feeccc00700 1 -- 172.21.15.117:0/786658742 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeb800f5f0 con 0x7feed0104a00 2024-04-20T11:50:23.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.961+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed019df10 con 0x7feed0104a00 2024-04-20T11:50:23.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.962+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed010b460 con 0x7feed0104a00 2024-04-20T11:50:23.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.963+0000 7feeccc00700 1 -- 172.21.15.117:0/786658742 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7feeb80105d0 con 0x7feed0104a00 2024-04-20T11:50:23.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.964+0000 7feeccc00700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feeb406ecd0 0x7feeb4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:23.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.964+0000 7feeccc00700 1 -- 172.21.15.117:0/786658742 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7feeb808ee50 con 0x7feed0104a00 2024-04-20T11:50:23.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.965+0000 7feecf400700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feeb406ecd0 0x7feeb4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:23.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.965+0000 7feecf400700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feeb406ecd0 0x7feeb4071190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feec0006010 tx=0x7feec000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:23.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:23.969+0000 7feeccc00700 1 -- 172.21.15.117:0/786658742 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7feeb8057c30 con 0x7feed0104a00 2024-04-20T11:50:24.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.310+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feed019a080 con 0x7feed0104a00 2024-04-20T11:50:24.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.310+0000 7feeccc00700 1 -- 172.21.15.117:0/786658742 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7feeb805b2f0 con 0x7feed0104a00 2024-04-20T11:50:24.312 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-20T11:50:24.312 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:50:24.312 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:50:24.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:50:24.312 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:50:24.312 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:50:24.312 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:50:24.312 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp1pubojyl:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is degraded 2024-04-20T11:50:24.313 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:50:24.314 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:50:24.314 INFO:teuthology.orchestra.run.smithi117.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-20T11:50:24.314 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-20T11:50:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.313+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feeb406ecd0 msgr2=0x7feeb4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.313+0000 7feed6600700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feeb406ecd0 0x7feeb4071190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feec0006010 tx=0x7feec000b540 comp rx=0 tx=0).stop 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.313+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0104a00 msgr2=0x7feed0199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.313+0000 7feed6600700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0104a00 0x7feed0199300 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7feeb800d8d0 tx=0x7feeb800dc90 comp rx=0 tx=0).stop 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.313+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 shutdown_connections 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.313+0000 7feed6600700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7feeb406ecd0 0x7feeb4071190 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.313+0000 7feed6600700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed0103110 0x7feed0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.313+0000 7feed6600700 1 --2- 172.21.15.117:0/786658742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feed0104a00 0x7feed0199300 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.314+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 >> 172.21.15.117:0/786658742 conn(0x7feed00fe880 msgr2=0x7feed00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.314+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 shutdown_connections 2024-04-20T11:50:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:24.314+0000 7feed6600700 1 -- 172.21.15.117:0/786658742 wait complete. 2024-04-20T11:50:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:24 smithi117 ceph-mon[99036]: pgmap v1139: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:24.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:24 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/786658742' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:50:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:24 smithi173 ceph-mon[74165]: pgmap v1139: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:24 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/786658742' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:50:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:25 smithi117 ceph-mon[99036]: from='client.36038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:25 smithi173 ceph-mon[74165]: from='client.36038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:26.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:26 smithi117 ceph-mon[99036]: pgmap v1140: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:26.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:26 smithi173 ceph-mon[74165]: pgmap v1140: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:28 smithi117 ceph-mon[99036]: pgmap v1141: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:50:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:28 smithi173 ceph-mon[74165]: pgmap v1141: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:50:30.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:30 smithi117 ceph-mon[99036]: pgmap v1142: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:30.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:30 smithi173 ceph-mon[74165]: pgmap v1142: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:31.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.miaplu"}]': finished 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.umapup"}]': finished 2024-04-20T11:50:31.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:50:31.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi173.pfrgat"}]': finished 2024-04-20T11:50:31.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:50:31.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi173.pybzht"}]': finished 2024-04-20T11:50:31.892 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:31 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-20T11:50:31.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.miaplu"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.miaplu"}]': finished 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.umapup"}]: dispatch 2024-04-20T11:50:31.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi117.umapup"}]': finished 2024-04-20T11:50:31.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi173.pfrgat"}]: dispatch 2024-04-20T11:50:31.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi173.pfrgat"}]': finished 2024-04-20T11:50:31.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi173.pybzht"}]: dispatch 2024-04-20T11:50:31.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi173.pybzht"}]': finished 2024-04-20T11:50:31.913 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:31 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-20T11:50:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: pgmap v1143: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: pgmap v1144: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all mds 2024-04-20T11:50:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: Upgrade: Setting filesystem cephfs Joinable 2024-04-20T11:50:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-20T11:50:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: Standby daemon mds.cephfs.smithi117.umapup assigned to filesystem cephfs as rank 0 2024-04-20T11:50:32.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: Standby daemon mds.cephfs.smithi117.miaplu assigned to filesystem cephfs as rank 1 2024-04-20T11:50:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T11:50:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:50:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:replay,1=cephfs.smithi117.miaplu=up:replay} 2 up:standby 2024-04-20T11:50:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:32.891 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:32 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: pgmap v1143: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: pgmap v1144: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:32.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all mds 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: Upgrade: Setting filesystem cephfs Joinable 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: Standby daemon mds.cephfs.smithi117.umapup assigned to filesystem cephfs as rank 0 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: Standby daemon mds.cephfs.smithi117.miaplu assigned to filesystem cephfs as rank 1 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:replay,1=cephfs.smithi117.miaplu=up:replay} 2 up:standby 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:32.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:32 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:33 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all rgw 2024-04-20T11:50:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:33 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all rbd-mirror 2024-04-20T11:50:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:33 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:resolve 2024-04-20T11:50:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:33 smithi117 ceph-mon[99036]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:replay,1=cephfs.smithi117.miaplu=up:resolve} 2 up:standby 2024-04-20T11:50:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:33 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all rgw 2024-04-20T11:50:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:33 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all rbd-mirror 2024-04-20T11:50:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:33 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:resolve 2024-04-20T11:50:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:33 smithi173 ceph-mon[74165]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:replay,1=cephfs.smithi117.miaplu=up:resolve} 2 up:standby 2024-04-20T11:50:34.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:34 smithi117 ceph-mon[99036]: pgmap v1145: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:34.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:34 smithi173 ceph-mon[74165]: pgmap v1145: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:50:35.884 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:35 smithi117 ceph-mon[99036]: Upgrade: Updating ceph-exporter.smithi117 (1/2) 2024-04-20T11:50:35.884 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:35.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:50:35.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:35 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:50:35.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:35 smithi117 ceph-mon[99036]: Deploying daemon ceph-exporter.smithi117 on smithi117 2024-04-20T11:50:35.885 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:35 smithi117 ceph-mon[99036]: pgmap v1146: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 863 KiB/s rd, 0 op/s 2024-04-20T11:50:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:35 smithi173 ceph-mon[74165]: Upgrade: Updating ceph-exporter.smithi117 (1/2) 2024-04-20T11:50:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:36.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi117", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:50:36.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:35 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:50:36.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:35 smithi173 ceph-mon[74165]: Deploying daemon ceph-exporter.smithi117 on smithi117 2024-04-20T11:50:36.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:35 smithi173 ceph-mon[74165]: pgmap v1146: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 863 KiB/s rd, 0 op/s 2024-04-20T11:50:38.030 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:37 smithi117 ceph-mon[99036]: pgmap v1147: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 9.4 MiB/s rd, 3 op/s 2024-04-20T11:50:38.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:37 smithi173 ceph-mon[74165]: pgmap v1147: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 9.4 MiB/s rd, 3 op/s 2024-04-20T11:50:40.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:39 smithi173 ceph-mon[74165]: pgmap v1148: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 6 op/s 2024-04-20T11:50:40.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:39 smithi117 ceph-mon[99036]: pgmap v1148: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 6 op/s 2024-04-20T11:50:41.258 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:41.258 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:41 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:41.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:41.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:41 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:42.361 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:42 smithi173 ceph-mon[74165]: pgmap v1149: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-04-20T11:50:42.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:42 smithi117 ceph-mon[99036]: pgmap v1149: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-04-20T11:50:43.361 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:43 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:resolve 2024-04-20T11:50:43.361 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:43 smithi173 ceph-mon[74165]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:resolve,1=cephfs.smithi117.miaplu=up:resolve} 2 up:standby 2024-04-20T11:50:43.361 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:43 smithi173 ceph-mon[74165]: Upgrade: Updating ceph-exporter.smithi173 (2/2) 2024-04-20T11:50:43.362 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:43.362 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:50:43.362 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:50:43.362 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:43 smithi173 ceph-mon[74165]: Deploying daemon ceph-exporter.smithi173 on smithi173 2024-04-20T11:50:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:43 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:resolve 2024-04-20T11:50:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:43 smithi117 ceph-mon[99036]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:resolve,1=cephfs.smithi117.miaplu=up:resolve} 2 up:standby 2024-04-20T11:50:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:43 smithi117 ceph-mon[99036]: Upgrade: Updating ceph-exporter.smithi173 (2/2) 2024-04-20T11:50:43.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:43.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi173", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-20T11:50:43.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:50:43.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:43 smithi117 ceph-mon[99036]: Deploying daemon ceph-exporter.smithi173 on smithi173 2024-04-20T11:50:44.375 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: pgmap v1150: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: reconnect by client.24295 192.168.0.1:0/272380713 after 0.00199996 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: reconnect by client.24295 192.168.0.1:0/272380713 after 0.00399992 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: reconnect by client.14524 192.168.0.1:0/846389270 after 0.00399992 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: reconnect by client.14524 192.168.0.1:0/846389270 after 0.0049999 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:reconnect 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:reconnect 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:reconnect,1=cephfs.smithi117.miaplu=up:reconnect} 2 up:standby 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:rejoin 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:rejoin 2024-04-20T11:50:44.376 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:44 smithi173 ceph-mon[74165]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:rejoin,1=cephfs.smithi117.miaplu=up:rejoin} 2 up:standby 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: pgmap v1150: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: reconnect by client.24295 192.168.0.1:0/272380713 after 0.00199996 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: reconnect by client.24295 192.168.0.1:0/272380713 after 0.00399992 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: reconnect by client.14524 192.168.0.1:0/846389270 after 0.00399992 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: reconnect by client.14524 192.168.0.1:0/846389270 after 0.0049999 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:reconnect 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:reconnect 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:reconnect,1=cephfs.smithi117.miaplu=up:reconnect} 2 up:standby 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:rejoin 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:rejoin 2024-04-20T11:50:44.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:44 smithi117 ceph-mon[99036]: fsmap cephfs:2/2 {0=cephfs.smithi117.umapup=up:rejoin,1=cephfs.smithi117.miaplu=up:rejoin} 2 up:standby 2024-04-20T11:50:45.309 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:45 smithi117 ceph-mon[99036]: daemon mds.cephfs.smithi117.miaplu is now active in filesystem cephfs as rank 1 2024-04-20T11:50:45.309 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:45 smithi117 ceph-mon[99036]: daemon mds.cephfs.smithi117.umapup is now active in filesystem cephfs as rank 0 2024-04-20T11:50:45.334 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:45 smithi173 ceph-mon[74165]: daemon mds.cephfs.smithi117.miaplu is now active in filesystem cephfs as rank 1 2024-04-20T11:50:45.334 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:45 smithi173 ceph-mon[74165]: daemon mds.cephfs.smithi117.umapup is now active in filesystem cephfs as rank 0 2024-04-20T11:50:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:46 smithi117 ceph-mon[99036]: pgmap v1151: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-04-20T11:50:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:46 smithi117 ceph-mon[99036]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T11:50:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:46 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:active 2024-04-20T11:50:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:46 smithi117 ceph-mon[99036]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:active 2024-04-20T11:50:46.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:46 smithi117 ceph-mon[99036]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi117.miaplu=up:active} 2 up:standby 2024-04-20T11:50:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:46 smithi173 ceph-mon[74165]: pgmap v1151: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-04-20T11:50:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:46 smithi173 ceph-mon[74165]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T11:50:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:46 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] up:active 2024-04-20T11:50:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:46 smithi173 ceph-mon[74165]: mds.? [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] up:active 2024-04-20T11:50:46.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:46 smithi173 ceph-mon[74165]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi117.miaplu=up:active} 2 up:standby 2024-04-20T11:50:48.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:48 smithi117 ceph-mon[99036]: pgmap v1152: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 255 B/s wr, 8 op/s 2024-04-20T11:50:48.404 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:48 smithi173 ceph-mon[74165]: pgmap v1152: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 255 B/s wr, 8 op/s 2024-04-20T11:50:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:50:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:49 smithi117 ceph-mon[99036]: pgmap v1153: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 341 B/s wr, 8 op/s 2024-04-20T11:50:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:49 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:50:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:49 smithi173 ceph-mon[74165]: pgmap v1153: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 341 B/s wr, 8 op/s 2024-04-20T11:50:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:49.912 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:49 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:52.121 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:51 smithi173 ceph-mon[74165]: pgmap v1154: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:50:52.220 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:51 smithi117 ceph-mon[99036]: pgmap v1154: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:50:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.480+0000 7f6d77400700 1 -- 172.21.15.117:0/1656780998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 msgr2=0x7f6d701011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.480+0000 7f6d77400700 1 --2- 172.21.15.117:0/1656780998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 0x7f6d701011f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009910 tx=0x7f6d64009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.480+0000 7f6d77400700 1 -- 172.21.15.117:0/1656780998 shutdown_connections 2024-04-20T11:50:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.480+0000 7f6d77400700 1 --2- 172.21.15.117:0/1656780998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d701017c0 0x7f6d70105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.480+0000 7f6d77400700 1 --2- 172.21.15.117:0/1656780998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 0x7f6d701011f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.480+0000 7f6d77400700 1 -- 172.21.15.117:0/1656780998 >> 172.21.15.117:0/1656780998 conn(0x7f6d700fc6c0 msgr2=0x7f6d700feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.481+0000 7f6d77400700 1 -- 172.21.15.117:0/1656780998 shutdown_connections 2024-04-20T11:50:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.481+0000 7f6d77400700 1 -- 172.21.15.117:0/1656780998 wait complete. 2024-04-20T11:50:54.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:54 smithi117 ceph-mon[99036]: pgmap v1155: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 3.7 MiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-04-20T11:50:54.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:54 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:54.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:54 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:54.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.482+0000 7f6d77400700 1 Processor -- start 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.482+0000 7f6d77400700 1 -- start start 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.483+0000 7f6d77400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 0x7f6d7019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.483+0000 7f6d77400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d701017c0 0x7f6d7019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.483+0000 7f6d77400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7019d360 con 0x7f6d70100e10 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.483+0000 7f6d77400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70196940 con 0x7f6d701017c0 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.483+0000 7f6d74a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 0x7f6d7019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.483+0000 7f6d74a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 0x7f6d7019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49806/0 (socket says 172.21.15.117:49806) 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.483+0000 7f6d74a00700 1 -- 172.21.15.117:0/1049561508 learned_addr learned my addr 172.21.15.117:0/1049561508 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.483+0000 7f6d6fe00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d701017c0 0x7f6d7019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d74a00700 1 -- 172.21.15.117:0/1049561508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d701017c0 msgr2=0x7f6d7019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d74a00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d701017c0 0x7f6d7019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d74a00700 1 -- 172.21.15.117:0/1049561508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640095f0 con 0x7f6d70100e10 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d74a00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 0x7f6d7019c800 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400bd70 tx=0x7f6d6400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d6d600700 1 -- 172.21.15.117:0/1049561508 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6401d070 con 0x7f6d70100e10 2024-04-20T11:50:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d6d600700 1 -- 172.21.15.117:0/1049561508 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d640056e0 con 0x7f6d70100e10 2024-04-20T11:50:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d77400700 1 -- 172.21.15.117:0/1049561508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d70196bc0 con 0x7f6d70100e10 2024-04-20T11:50:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d6d600700 1 -- 172.21.15.117:0/1049561508 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d64017400 con 0x7f6d70100e10 2024-04-20T11:50:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.484+0000 7f6d77400700 1 -- 172.21.15.117:0/1049561508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d701970b0 con 0x7f6d70100e10 2024-04-20T11:50:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.486+0000 7f6d62a00700 1 -- 172.21.15.117:0/1049561508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c0052f0 con 0x7f6d70100e10 2024-04-20T11:50:54.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.487+0000 7f6d6d600700 1 -- 172.21.15.117:0/1049561508 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6d64003a60 con 0x7f6d70100e10 2024-04-20T11:50:54.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.488+0000 7f6d6d600700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d5806ecc0 0x7f6d58071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:54.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.488+0000 7f6d6d600700 1 -- 172.21.15.117:0/1049561508 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6d64090310 con 0x7f6d70100e10 2024-04-20T11:50:54.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.488+0000 7f6d6fe00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d5806ecc0 0x7f6d58071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:54.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.489+0000 7f6d6fe00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d5806ecc0 0x7f6d58071180 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c005950 tx=0x7f6d5c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:54.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.494+0000 7f6d6d600700 1 -- 172.21.15.117:0/1049561508 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6d6405cb80 con 0x7f6d70100e10 2024-04-20T11:50:54.559 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:54 smithi173 ceph-mon[74165]: pgmap v1155: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 3.7 MiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-04-20T11:50:54.559 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:54 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:54.559 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:54 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:54.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.758+0000 7f6d62a00700 1 -- 172.21.15.117:0/1049561508 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d4c000bc0 con 0x7f6d5806ecc0 2024-04-20T11:50:54.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.760+0000 7f6d6d600700 1 -- 172.21.15.117:0/1049561508 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f6d4c000bc0 con 0x7f6d5806ecc0 2024-04-20T11:50:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 -- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d5806ecc0 msgr2=0x7f6d58071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d5806ecc0 0x7f6d58071180 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c005950 tx=0x7f6d5c0058e0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 -- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 msgr2=0x7f6d7019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 0x7f6d7019c800 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400bd70 tx=0x7f6d6400be50 comp rx=0 tx=0).stop 2024-04-20T11:50:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 -- 172.21.15.117:0/1049561508 shutdown_connections 2024-04-20T11:50:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d5806ecc0 0x7f6d58071180 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d70100e10 0x7f6d7019c800 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 --2- 172.21.15.117:0/1049561508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d701017c0 0x7f6d7019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.762+0000 7f6d62a00700 1 -- 172.21.15.117:0/1049561508 >> 172.21.15.117:0/1049561508 conn(0x7f6d700fc6c0 msgr2=0x7f6d700fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.763+0000 7f6d62a00700 1 -- 172.21.15.117:0/1049561508 shutdown_connections 2024-04-20T11:50:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.763+0000 7f6d62a00700 1 -- 172.21.15.117:0/1049561508 wait complete. 2024-04-20T11:50:54.777 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:50:54.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.914+0000 7fccda800700 1 -- 172.21.15.117:0/1791435496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd41017c0 msgr2=0x7fccd4105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.914+0000 7fccda800700 1 --2- 172.21.15.117:0/1791435496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd41017c0 0x7fccd4105810 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009910 tx=0x7fccbc009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.915+0000 7fccda800700 1 -- 172.21.15.117:0/1791435496 shutdown_connections 2024-04-20T11:50:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.915+0000 7fccda800700 1 --2- 172.21.15.117:0/1791435496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd41017c0 0x7fccd4105810 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.915+0000 7fccda800700 1 --2- 172.21.15.117:0/1791435496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd4100e10 0x7fccd41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.915+0000 7fccda800700 1 -- 172.21.15.117:0/1791435496 >> 172.21.15.117:0/1791435496 conn(0x7fccd40fc6c0 msgr2=0x7fccd40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.915+0000 7fccda800700 1 -- 172.21.15.117:0/1791435496 shutdown_connections 2024-04-20T11:50:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.915+0000 7fccda800700 1 -- 172.21.15.117:0/1791435496 wait complete. 2024-04-20T11:50:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccda800700 1 Processor -- start 2024-04-20T11:50:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccda800700 1 -- start start 2024-04-20T11:50:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccda800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd4100e10 0x7fccd40731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccda800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd41017c0 0x7fccd4073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccda800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd4073c60 con 0x7fccd4100e10 2024-04-20T11:50:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccda800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd4073da0 con 0x7fccd41017c0 2024-04-20T11:50:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccd3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd4100e10 0x7fccd40731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccd3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd4100e10 0x7fccd40731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49820/0 (socket says 172.21.15.117:49820) 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.916+0000 7fccd3400700 1 -- 172.21.15.117:0/1889534768 learned_addr learned my addr 172.21.15.117:0/1889534768 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fcccaa00700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd41017c0 0x7fccd4073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fccd3400700 1 -- 172.21.15.117:0/1889534768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd41017c0 msgr2=0x7fccd4073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fccd3400700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd41017c0 0x7fccd4073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fccd3400700 1 -- 172.21.15.117:0/1889534768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc0095f0 con 0x7fccd4100e10 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fccd3400700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd4100e10 0x7fccd40731e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fccc400d8d0 tx=0x7fccc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fccd0c00700 1 -- 172.21.15.117:0/1889534768 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccc4009940 con 0x7fccd4100e10 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fccd0c00700 1 -- 172.21.15.117:0/1889534768 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccc4010460 con 0x7fccd4100e10 2024-04-20T11:50:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fccd0c00700 1 -- 172.21.15.117:0/1889534768 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccc400f5f0 con 0x7fccd4100e10 2024-04-20T11:50:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.917+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd41a2cf0 con 0x7fccd4100e10 2024-04-20T11:50:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.918+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd41a3050 con 0x7fccd4100e10 2024-04-20T11:50:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.919+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd4109210 con 0x7fccd4100e10 2024-04-20T11:50:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.920+0000 7fccd0c00700 1 -- 172.21.15.117:0/1889534768 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fccc40105d0 con 0x7fccd4100e10 2024-04-20T11:50:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.920+0000 7fccd0c00700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccb806ecc0 0x7fccb8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.921+0000 7fccd0c00700 1 -- 172.21.15.117:0/1889534768 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fccc408eef0 con 0x7fccd4100e10 2024-04-20T11:50:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.921+0000 7fcccaa00700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccb806ecc0 0x7fccb8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.922+0000 7fcccaa00700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccb806ecc0 0x7fccb8071180 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fccd40748c0 tx=0x7fccbc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:54.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:54.925+0000 7fccd0c00700 1 -- 172.21.15.117:0/1889534768 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fccc4057c30 con 0x7fccd4100e10 2024-04-20T11:50:55.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.176+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fccd4074710 con 0x7fccb806ecc0 2024-04-20T11:50:55.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.178+0000 7fccd0c00700 1 -- 172.21.15.117:0/1889534768 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fccd4074710 con 0x7fccb806ecc0 2024-04-20T11:50:55.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccb806ecc0 msgr2=0x7fccb8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccb806ecc0 0x7fccb8071180 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fccd40748c0 tx=0x7fccbc01a040 comp rx=0 tx=0).stop 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd4100e10 msgr2=0x7fccd40731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd4100e10 0x7fccd40731e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fccc400d8d0 tx=0x7fccc400dc90 comp rx=0 tx=0).stop 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 shutdown_connections 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fccb806ecc0 0x7fccb8071180 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd4100e10 0x7fccd40731e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 --2- 172.21.15.117:0/1889534768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd41017c0 0x7fccd4073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 >> 172.21.15.117:0/1889534768 conn(0x7fccd40fc6c0 msgr2=0x7fccd40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 shutdown_connections 2024-04-20T11:50:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.181+0000 7fccda800700 1 -- 172.21.15.117:0/1889534768 wait complete. 2024-04-20T11:50:55.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.339+0000 7f1513200700 1 -- 172.21.15.117:0/3839976498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 msgr2=0x7f1514103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:55.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.339+0000 7f1513200700 1 --2- 172.21.15.117:0/3839976498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 0x7f1514103220 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1504009910 tx=0x7f1504009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:55.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.339+0000 7f1513200700 1 -- 172.21.15.117:0/3839976498 shutdown_connections 2024-04-20T11:50:55.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.339+0000 7f1513200700 1 --2- 172.21.15.117:0/3839976498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15141037f0 0x7f1514107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.339+0000 7f1513200700 1 --2- 172.21.15.117:0/3839976498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 0x7f1514103220 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.339+0000 7f1513200700 1 -- 172.21.15.117:0/3839976498 >> 172.21.15.117:0/3839976498 conn(0x7f15140fe6f0 msgr2=0x7f1514100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:55.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.340+0000 7f1513200700 1 -- 172.21.15.117:0/3839976498 shutdown_connections 2024-04-20T11:50:55.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.340+0000 7f1513200700 1 -- 172.21.15.117:0/3839976498 wait complete. 2024-04-20T11:50:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.341+0000 7f1513200700 1 Processor -- start 2024-04-20T11:50:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.341+0000 7f1513200700 1 -- start start 2024-04-20T11:50:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.341+0000 7f1513200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 0x7f151419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.341+0000 7f1513200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15141037f0 0x7f151419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.341+0000 7f1513200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151419f470 con 0x7f1514102e40 2024-04-20T11:50:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.341+0000 7f1513200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514198a50 con 0x7f15141037f0 2024-04-20T11:50:55.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.342+0000 7f1511e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 0x7f151419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:55.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.342+0000 7f1511e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 0x7f151419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49836/0 (socket says 172.21.15.117:49836) 2024-04-20T11:50:55.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.342+0000 7f1511e00700 1 -- 172.21.15.117:0/1987241641 learned_addr learned my addr 172.21.15.117:0/1987241641 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.342+0000 7f1511400700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15141037f0 0x7f151419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.342+0000 7f1511e00700 1 -- 172.21.15.117:0/1987241641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15141037f0 msgr2=0x7f151419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.342+0000 7f1511e00700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15141037f0 0x7f151419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.342+0000 7f1511e00700 1 -- 172.21.15.117:0/1987241641 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15040095f0 con 0x7f1514102e40 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.342+0000 7f1511e00700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 0x7f151419e910 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f1504006010 tx=0x7f150400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.343+0000 7f150aa00700 1 -- 172.21.15.117:0/1987241641 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f150401d070 con 0x7f1514102e40 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.343+0000 7f150aa00700 1 -- 172.21.15.117:0/1987241641 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f150400f460 con 0x7f1514102e40 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.343+0000 7f150aa00700 1 -- 172.21.15.117:0/1987241641 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1504017630 con 0x7f1514102e40 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.343+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1514198cd0 con 0x7f1514102e40 2024-04-20T11:50:55.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.343+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1514199120 con 0x7f1514102e40 2024-04-20T11:50:55.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.344+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151410b1e0 con 0x7f1514102e40 2024-04-20T11:50:55.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.345+0000 7f150aa00700 1 -- 172.21.15.117:0/1987241641 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f150400f5d0 con 0x7f1514102e40 2024-04-20T11:50:55.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.346+0000 7f150aa00700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f14f806ecc0 0x7f14f8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:55.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.346+0000 7f150aa00700 1 -- 172.21.15.117:0/1987241641 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1504090530 con 0x7f1514102e40 2024-04-20T11:50:55.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.347+0000 7f1511400700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f14f806ecc0 0x7f14f8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:55.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.347+0000 7f1511400700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f14f806ecc0 0x7f14f8071180 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1514199c10 tx=0x7f1500005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:55.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.350+0000 7f150aa00700 1 -- 172.21.15.117:0/1987241641 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f150405cf60 con 0x7f1514102e40 2024-04-20T11:50:55.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.614+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1514199f10 con 0x7f14f806ecc0 2024-04-20T11:50:55.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.633+0000 7f150aa00700 1 -- 172.21.15.117:0/1987241641 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1514199f10 con 0x7f14f806ecc0 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (39m) 15m ago 53m 15.1M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 starting - - - - 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (7s) 2s ago 52m 7302k - 19.0.0-3133-ga8c84f59 76bc6b91b726 1ed89bb400a5 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (35m) 15m ago 53m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (35m) 2s ago 52m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (38m) 15m ago 53m 73.7M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (31m) 15m ago 46m 18.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (31m) 15m ago 46m 18.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:50:55.633 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (15m) 2s ago 46m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (30m) 2s ago 46m 23.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (42m) 15m ago 55m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (41m) 2s ago 52m 460M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (37m) 15m ago 55m 73.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (36m) 2s ago 52m 55.1M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (40m) 15m ago 53m 8841k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (40m) 2s ago 52m 8380k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (35m) 15m ago 51m 88.4M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (34m) 15m ago 50m 90.0M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (33m) 15m ago 49m 87.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (33m) 2s ago 49m 103M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (32m) 2s ago 48m 92.3M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (32m) 2s ago 48m 89.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:50:55.634 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (40m) 15m ago 52m 81.3M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:50:55.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f14f806ecc0 msgr2=0x7f14f8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:55.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f14f806ecc0 0x7f14f8071180 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1514199c10 tx=0x7f1500005cf0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 msgr2=0x7f151419e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:55.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 0x7f151419e910 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f1504006010 tx=0x7f150400be20 comp rx=0 tx=0).stop 2024-04-20T11:50:55.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 shutdown_connections 2024-04-20T11:50:55.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f14f806ecc0 0x7f14f8071180 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1514102e40 0x7f151419e910 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 --2- 172.21.15.117:0/1987241641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15141037f0 0x7f151419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 >> 172.21.15.117:0/1987241641 conn(0x7f15140fe6f0 msgr2=0x7f15140ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:55.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.636+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 shutdown_connections 2024-04-20T11:50:55.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.637+0000 7f1513200700 1 -- 172.21.15.117:0/1987241641 wait complete. 2024-04-20T11:50:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.802+0000 7f5025800700 1 -- 172.21.15.117:0/3109055165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50201017c0 msgr2=0x7f5020105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.802+0000 7f5025800700 1 --2- 172.21.15.117:0/3109055165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50201017c0 0x7f5020105810 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f500c009910 tx=0x7f500c009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.802+0000 7f5025800700 1 -- 172.21.15.117:0/3109055165 shutdown_connections 2024-04-20T11:50:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.802+0000 7f5025800700 1 --2- 172.21.15.117:0/3109055165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50201017c0 0x7f5020105810 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.802+0000 7f5025800700 1 --2- 172.21.15.117:0/3109055165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5020100e10 0x7f50201011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.802+0000 7f5025800700 1 -- 172.21.15.117:0/3109055165 >> 172.21.15.117:0/3109055165 conn(0x7f50200fc6c0 msgr2=0x7f50200feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.802+0000 7f5025800700 1 -- 172.21.15.117:0/3109055165 shutdown_connections 2024-04-20T11:50:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.802+0000 7f5025800700 1 -- 172.21.15.117:0/3109055165 wait complete. 2024-04-20T11:50:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.803+0000 7f5025800700 1 Processor -- start 2024-04-20T11:50:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.804+0000 7f5025800700 1 -- start start 2024-04-20T11:50:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.804+0000 7f5025800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5020100e10 0x7f5020198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:55.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.804+0000 7f5025800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50201017c0 0x7f50201993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:55.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.804+0000 7f5025800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5020199ad0 con 0x7f5020100e10 2024-04-20T11:50:55.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.804+0000 7f5025800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502019d6e0 con 0x7f50201017c0 2024-04-20T11:50:55.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.805+0000 7f501ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5020100e10 0x7f5020198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:55.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.805+0000 7f501ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5020100e10 0x7f5020198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49856/0 (socket says 172.21.15.117:49856) 2024-04-20T11:50:55.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.805+0000 7f501ea00700 1 -- 172.21.15.117:0/3397903329 learned_addr learned my addr 172.21.15.117:0/3397903329 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:55.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.805+0000 7f501e000700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50201017c0 0x7f50201993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:55.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.805+0000 7f501ea00700 1 -- 172.21.15.117:0/3397903329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50201017c0 msgr2=0x7f50201993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:55.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.805+0000 7f501ea00700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50201017c0 0x7f50201993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:55.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.805+0000 7f501ea00700 1 -- 172.21.15.117:0/3397903329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c0095f0 con 0x7f5020100e10 2024-04-20T11:50:55.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.805+0000 7f501ea00700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5020100e10 0x7f5020198eb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f501000b4b0 tx=0x7f501000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:55.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.806+0000 7f5017400700 1 -- 172.21.15.117:0/3397903329 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5010011800 con 0x7f5020100e10 2024-04-20T11:50:55.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.806+0000 7f5017400700 1 -- 172.21.15.117:0/3397903329 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5010011e20 con 0x7f5020100e10 2024-04-20T11:50:55.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.806+0000 7f5017400700 1 -- 172.21.15.117:0/3397903329 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f501000f2e0 con 0x7f5020100e10 2024-04-20T11:50:55.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.806+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502019d9c0 con 0x7f5020100e10 2024-04-20T11:50:55.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.806+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502019dee0 con 0x7f5020100e10 2024-04-20T11:50:55.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.807+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50201022f0 con 0x7f5020100e10 2024-04-20T11:50:55.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.808+0000 7f5017400700 1 -- 172.21.15.117:0/3397903329 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5010010380 con 0x7f5020100e10 2024-04-20T11:50:55.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.810+0000 7f5017400700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f500406ecc0 0x7f5004071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:55.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.810+0000 7f5017400700 1 -- 172.21.15.117:0/3397903329 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f501008ea00 con 0x7f5020100e10 2024-04-20T11:50:55.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.810+0000 7f501e000700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f500406ecc0 0x7f5004071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:55.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.811+0000 7f501e000700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f500406ecc0 0x7f5004071180 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f502019a350 tx=0x7f500c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:55.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:55.815+0000 7f5017400700 1 -- 172.21.15.117:0/3397903329 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5010057870 con 0x7f5020100e10 2024-04-20T11:50:56.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.150+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f502010de30 con 0x7f5020100e10 2024-04-20T11:50:56.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.151+0000 7f5017400700 1 -- 172.21.15.117:0/3397903329 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f501005af30 con 0x7f5020100e10 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:50:56.151 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:50:56.152 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:50:56.152 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:50:56.152 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:50:56.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f500406ecc0 msgr2=0x7f5004071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f500406ecc0 0x7f5004071180 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f502019a350 tx=0x7f500c01a040 comp rx=0 tx=0).stop 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5020100e10 msgr2=0x7f5020198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5020100e10 0x7f5020198eb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f501000b4b0 tx=0x7f501000b870 comp rx=0 tx=0).stop 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 shutdown_connections 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f500406ecc0 0x7f5004071180 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5020100e10 0x7f5020198eb0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 --2- 172.21.15.117:0/3397903329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50201017c0 0x7f50201993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.154+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 >> 172.21.15.117:0/3397903329 conn(0x7f50200fc6c0 msgr2=0x7f5020074740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.155+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 shutdown_connections 2024-04-20T11:50:56.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.155+0000 7f5025800700 1 -- 172.21.15.117:0/3397903329 wait complete. 2024-04-20T11:50:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.312+0000 7f8e26800700 1 -- 172.21.15.117:0/2802706301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073ba0 msgr2=0x7f8e20111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.313+0000 7f8e26800700 1 --2- 172.21.15.117:0/2802706301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073ba0 0x7f8e20111690 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009910 tx=0x7f8e0c009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.313+0000 7f8e26800700 1 -- 172.21.15.117:0/2802706301 shutdown_connections 2024-04-20T11:50:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.313+0000 7f8e26800700 1 --2- 172.21.15.117:0/2802706301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073ba0 0x7f8e20111690 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.313+0000 7f8e26800700 1 --2- 172.21.15.117:0/2802706301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e20073280 0x7f8e20073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.313+0000 7f8e26800700 1 -- 172.21.15.117:0/2802706301 >> 172.21.15.117:0/2802706301 conn(0x7f8e200fc6f0 msgr2=0x7f8e200feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.313+0000 7f8e26800700 1 -- 172.21.15.117:0/2802706301 shutdown_connections 2024-04-20T11:50:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.313+0000 7f8e26800700 1 -- 172.21.15.117:0/2802706301 wait complete. 2024-04-20T11:50:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.314+0000 7f8e26800700 1 Processor -- start 2024-04-20T11:50:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.314+0000 7f8e26800700 1 -- start start 2024-04-20T11:50:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.314+0000 7f8e26800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073280 0x7f8e2019d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.314+0000 7f8e26800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e20073ba0 0x7f8e2019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e26800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2019dc40 con 0x7f8e20073280 2024-04-20T11:50:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e26800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2019dd80 con 0x7f8e20073ba0 2024-04-20T11:50:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e1f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073280 0x7f8e2019d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e1f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073280 0x7f8e2019d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49888/0 (socket says 172.21.15.117:49888) 2024-04-20T11:50:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e1f400700 1 -- 172.21.15.117:0/2161829423 learned_addr learned my addr 172.21.15.117:0/2161829423 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e1ea00700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e20073ba0 0x7f8e2019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e1f400700 1 -- 172.21.15.117:0/2161829423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e20073ba0 msgr2=0x7f8e2019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e1f400700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e20073ba0 0x7f8e2019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.315+0000 7f8e1f400700 1 -- 172.21.15.117:0/2161829423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c0095f0 con 0x7f8e20073280 2024-04-20T11:50:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.316+0000 7f8e1f400700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073280 0x7f8e2019d0b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c930 tx=0x7f8e1000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.316+0000 7f8e17e00700 1 -- 172.21.15.117:0/2161829423 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e10007ad0 con 0x7f8e20073280 2024-04-20T11:50:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.316+0000 7f8e17e00700 1 -- 172.21.15.117:0/2161829423 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e10007c30 con 0x7f8e20073280 2024-04-20T11:50:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.316+0000 7f8e17e00700 1 -- 172.21.15.117:0/2161829423 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e10018610 con 0x7f8e20073280 2024-04-20T11:50:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.316+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e201a1b10 con 0x7f8e20073280 2024-04-20T11:50:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.316+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e201a2060 con 0x7f8e20073280 2024-04-20T11:50:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.317+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e200749c0 con 0x7f8e20073280 2024-04-20T11:50:56.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.319+0000 7f8e17e00700 1 -- 172.21.15.117:0/2161829423 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8e1000f450 con 0x7f8e20073280 2024-04-20T11:50:56.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.319+0000 7f8e17e00700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e0406ecc0 0x7f8e04071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:56.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.320+0000 7f8e17e00700 1 -- 172.21.15.117:0/2161829423 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8e1008ef80 con 0x7f8e20073280 2024-04-20T11:50:56.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.320+0000 7f8e1ea00700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e0406ecc0 0x7f8e04071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:56.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.321+0000 7f8e1ea00700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e0406ecc0 0x7f8e04071180 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e2019e610 tx=0x7f8e0c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:56.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.325+0000 7f8e17e00700 1 -- 172.21.15.117:0/2161829423 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f8e10057de0 con 0x7f8e20073280 2024-04-20T11:50:56.506 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:56 smithi173 ceph-mon[74165]: from='client.36050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:56.507 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:56 smithi173 ceph-mon[74165]: pgmap v1156: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 350 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-20T11:50:56.507 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:56 smithi173 ceph-mon[74165]: from='client.36054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:56.507 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:56 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:56.507 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:56 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:56.507 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:56 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:56.507 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:56 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:56.507 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:56 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3397903329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:56.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.597+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8e201144e0 con 0x7f8e20073280 2024-04-20T11:50:56.597 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:56 smithi117 ceph-mon[99036]: from='client.36050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:56.597 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:56 smithi117 ceph-mon[99036]: pgmap v1156: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 350 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-20T11:50:56.597 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:56 smithi117 ceph-mon[99036]: from='client.36054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:56.597 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:56 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:56.597 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:56 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:56.597 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:56 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:56.598 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:56 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:56.598 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:56 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3397903329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:50:56.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.598+0000 7f8e17e00700 1 -- 172.21.15.117:0/2161829423 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1973 (secure 0 0 0) 0x7f8e1005b4a0 con 0x7f8e20073280 2024-04-20T11:50:56.598 INFO:teuthology.orchestra.run.smithi117.stdout:e29 2024-04-20T11:50:56.598 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:50:56.598 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:50:56.598 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 29 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:50:45.111226+0000 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:50:56.599 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=34436,1=34468} 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:50:56.600 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 34436 members: 34436,34468 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:34436} state up:active seq 475 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{1:34468} state up:active seq 468 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:50:56.601 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:50:56.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.601+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e0406ecc0 msgr2=0x7f8e04071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:56.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.601+0000 7f8e26800700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e0406ecc0 0x7f8e04071180 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e2019e610 tx=0x7f8e0c01a040 comp rx=0 tx=0).stop 2024-04-20T11:50:56.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.601+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073280 msgr2=0x7f8e2019d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:56.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.601+0000 7f8e26800700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073280 0x7f8e2019d0b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c930 tx=0x7f8e1000ccf0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.602+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 shutdown_connections 2024-04-20T11:50:56.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.602+0000 7f8e26800700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f8e0406ecc0 0x7f8e04071180 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.602+0000 7f8e26800700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e20073280 0x7f8e2019d0b0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.602+0000 7f8e26800700 1 --2- 172.21.15.117:0/2161829423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e20073ba0 0x7f8e2019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.602+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 >> 172.21.15.117:0/2161829423 conn(0x7f8e200fc6f0 msgr2=0x7f8e20100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:56.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.602+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 shutdown_connections 2024-04-20T11:50:56.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.602+0000 7f8e26800700 1 -- 172.21.15.117:0/2161829423 wait complete. 2024-04-20T11:50:56.603 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-04-20T11:50:56.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.758+0000 7f3039800700 1 -- 172.21.15.117:0/734366174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 msgr2=0x7f3034107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:56.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.758+0000 7f3039800700 1 --2- 172.21.15.117:0/734366174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 0x7f3034107a40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3020009910 tx=0x7f3020009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:56.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.758+0000 7f3039800700 1 -- 172.21.15.117:0/734366174 shutdown_connections 2024-04-20T11:50:56.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.758+0000 7f3039800700 1 --2- 172.21.15.117:0/734366174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 0x7f3034107a40 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.758+0000 7f3039800700 1 --2- 172.21.15.117:0/734366174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3034103040 0x7f3034103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.758+0000 7f3039800700 1 -- 172.21.15.117:0/734366174 >> 172.21.15.117:0/734366174 conn(0x7f30340fe8d0 msgr2=0x7f3034100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:56.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.758+0000 7f3039800700 1 -- 172.21.15.117:0/734366174 shutdown_connections 2024-04-20T11:50:56.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.758+0000 7f3039800700 1 -- 172.21.15.117:0/734366174 wait complete. 2024-04-20T11:50:56.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.759+0000 7f3039800700 1 Processor -- start 2024-04-20T11:50:56.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.759+0000 7f3039800700 1 -- start start 2024-04-20T11:50:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3039800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3034103040 0x7f3034198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3039800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 0x7f3034199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3039800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034199a70 con 0x7f30341039f0 2024-04-20T11:50:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3039800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303419d740 con 0x7f3034103040 2024-04-20T11:50:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3032000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 0x7f3034199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3032000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 0x7f3034199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49904/0 (socket says 172.21.15.117:49904) 2024-04-20T11:50:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3032000700 1 -- 172.21.15.117:0/1514813274 learned_addr learned my addr 172.21.15.117:0/1514813274 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3032a00700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3034103040 0x7f3034198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3032000700 1 -- 172.21.15.117:0/1514813274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3034103040 msgr2=0x7f3034198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3032000700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3034103040 0x7f3034198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.760+0000 7f3032000700 1 -- 172.21.15.117:0/1514813274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30200095f0 con 0x7f30341039f0 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.761+0000 7f3032000700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 0x7f3034199390 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3020005b40 tx=0x7f302000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.761+0000 7f302b400700 1 -- 172.21.15.117:0/1514813274 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302001d070 con 0x7f30341039f0 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.761+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303419d9c0 con 0x7f30341039f0 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.761+0000 7f302b400700 1 -- 172.21.15.117:0/1514813274 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f302000f460 con 0x7f30341039f0 2024-04-20T11:50:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.761+0000 7f302b400700 1 -- 172.21.15.117:0/1514813274 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3020017630 con 0x7f30341039f0 2024-04-20T11:50:56.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.761+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303419deb0 con 0x7f30341039f0 2024-04-20T11:50:56.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.763+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3034104520 con 0x7f30341039f0 2024-04-20T11:50:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.765+0000 7f302b400700 1 -- 172.21.15.117:0/1514813274 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3020003a60 con 0x7f30341039f0 2024-04-20T11:50:56.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.766+0000 7f302b400700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301806ecc0 0x7f3018071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:56.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.766+0000 7f302b400700 1 -- 172.21.15.117:0/1514813274 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3020090570 con 0x7f30341039f0 2024-04-20T11:50:56.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.766+0000 7f3032a00700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301806ecc0 0x7f3018071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.767+0000 7f3032a00700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301806ecc0 0x7f3018071180 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3024006fd0 tx=0x7f3024008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:56.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:56.772+0000 7f302b400700 1 -- 172.21.15.117:0/1514813274 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f3020059330 con 0x7f30341039f0 2024-04-20T11:50:57.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.029+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3034110060 con 0x7f301806ecc0 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.032+0000 7f302b400700 1 -- 172.21.15.117:0/1514813274 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f3034110060 con 0x7f301806ecc0 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1", 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": true, 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [ 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr", 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-04-20T11:50:57.032 INFO:teuthology.orchestra.run.smithi117.stdout: "mon", 2024-04-20T11:50:57.033 INFO:teuthology.orchestra.run.smithi117.stdout: "osd", 2024-04-20T11:50:57.033 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph-exporter", 2024-04-20T11:50:57.033 INFO:teuthology.orchestra.run.smithi117.stdout: "mds" 2024-04-20T11:50:57.033 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-04-20T11:50:57.033 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": "18/23 daemons upgraded", 2024-04-20T11:50:57.033 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-04-20T11:50:57.033 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:50:57.033 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:50:57.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301806ecc0 msgr2=0x7f3018071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:57.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301806ecc0 0x7f3018071180 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3024006fd0 tx=0x7f3024008040 comp rx=0 tx=0).stop 2024-04-20T11:50:57.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 msgr2=0x7f3034199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:57.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 0x7f3034199390 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3020005b40 tx=0x7f302000be20 comp rx=0 tx=0).stop 2024-04-20T11:50:57.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 shutdown_connections 2024-04-20T11:50:57.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f301806ecc0 0x7f3018071180 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3034103040 0x7f3034198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 --2- 172.21.15.117:0/1514813274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30341039f0 0x7f3034199390 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 >> 172.21.15.117:0/1514813274 conn(0x7f30340fe8d0 msgr2=0x7f30340ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:57.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 shutdown_connections 2024-04-20T11:50:57.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.034+0000 7f3039800700 1 -- 172.21.15.117:0/1514813274 wait complete. 2024-04-20T11:50:57.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.186+0000 7f6b44a00700 1 -- 172.21.15.117:0/1724506916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 msgr2=0x7f6b40103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.186+0000 7f6b44a00700 1 --2- 172.21.15.117:0/1724506916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 0x7f6b40103420 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009910 tx=0x7f6b30009c20 comp rx=0 tx=0).stop 2024-04-20T11:50:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.186+0000 7f6b44a00700 1 -- 172.21.15.117:0/1724506916 shutdown_connections 2024-04-20T11:50:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.186+0000 7f6b44a00700 1 --2- 172.21.15.117:0/1724506916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b401039f0 0x7f6b40107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.186+0000 7f6b44a00700 1 --2- 172.21.15.117:0/1724506916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 0x7f6b40103420 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.186+0000 7f6b44a00700 1 -- 172.21.15.117:0/1724506916 >> 172.21.15.117:0/1724506916 conn(0x7f6b400fe8d0 msgr2=0x7f6b40100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.187+0000 7f6b44a00700 1 -- 172.21.15.117:0/1724506916 shutdown_connections 2024-04-20T11:50:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.187+0000 7f6b44a00700 1 -- 172.21.15.117:0/1724506916 wait complete. 2024-04-20T11:50:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.188+0000 7f6b44a00700 1 Processor -- start 2024-04-20T11:50:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.188+0000 7f6b44a00700 1 -- start start 2024-04-20T11:50:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.188+0000 7f6b44a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 0x7f6b40198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.188+0000 7f6b44a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b401039f0 0x7f6b401992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.188+0000 7f6b44a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b401999a0 con 0x7f6b40103040 2024-04-20T11:50:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.188+0000 7f6b44a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4019d670 con 0x7f6b401039f0 2024-04-20T11:50:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.189+0000 7f6b3de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 0x7f6b40198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.189+0000 7f6b3de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 0x7f6b40198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49924/0 (socket says 172.21.15.117:49924) 2024-04-20T11:50:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.189+0000 7f6b3de00700 1 -- 172.21.15.117:0/292353660 learned_addr learned my addr 172.21.15.117:0/292353660 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:50:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.189+0000 7f6b3d400700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b401039f0 0x7f6b401992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.189+0000 7f6b3de00700 1 -- 172.21.15.117:0/292353660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b401039f0 msgr2=0x7f6b401992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.189+0000 7f6b3de00700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b401039f0 0x7f6b401992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.189+0000 7f6b3de00700 1 -- 172.21.15.117:0/292353660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300095f0 con 0x7f6b40103040 2024-04-20T11:50:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.190+0000 7f6b3de00700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 0x7f6b40198d80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6b300092e0 tx=0x7f6b3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.190+0000 7f6b36a00700 1 -- 172.21.15.117:0/292353660 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b3001d070 con 0x7f6b40103040 2024-04-20T11:50:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.190+0000 7f6b36a00700 1 -- 172.21.15.117:0/292353660 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b3000f460 con 0x7f6b40103040 2024-04-20T11:50:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.190+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4019d8f0 con 0x7f6b40103040 2024-04-20T11:50:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.190+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4019dde0 con 0x7f6b40103040 2024-04-20T11:50:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.190+0000 7f6b36a00700 1 -- 172.21.15.117:0/292353660 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b30017630 con 0x7f6b40103040 2024-04-20T11:50:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.191+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4010b3e0 con 0x7f6b40103040 2024-04-20T11:50:57.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.193+0000 7f6b36a00700 1 -- 172.21.15.117:0/292353660 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6b3000f5d0 con 0x7f6b40103040 2024-04-20T11:50:57.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.194+0000 7f6b36a00700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b2406ecc0 0x7f6b24071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:50:57.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.194+0000 7f6b36a00700 1 -- 172.21.15.117:0/292353660 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6b30090570 con 0x7f6b40103040 2024-04-20T11:50:57.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.194+0000 7f6b3d400700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b2406ecc0 0x7f6b24071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:50:57.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.195+0000 7f6b3d400700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b2406ecc0 0x7f6b24071180 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6b4019a2e0 tx=0x7f6b2c005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:50:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.202+0000 7f6b36a00700 1 -- 172.21.15.117:0/292353660 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6b300593b0 con 0x7f6b40103040 2024-04-20T11:50:57.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:57 smithi173 ceph-mon[74165]: from='client.36058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:57.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:57 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2161829423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:50:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.531+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6b4019a130 con 0x7f6b40103040 2024-04-20T11:50:57.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.532+0000 7f6b36a00700 1 -- 172.21.15.117:0/292353660 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4271 (secure 0 0 0) 0x7f6b3005ca70 con 0x7f6b40103040 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem with deprecated feature inline_data 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout: host smithi173 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi173/config 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpsbbdwnn7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout:Traceback (most recent call last): 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout: return _run_code(code, main_globals, None, 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout: exec(code, run_globals) 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5618, in 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 5606, in main 2024-04-20T11:50:57.532 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 408, in _infer_config 2024-04-20T11:50:57.533 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 323, in _infer_fsid 2024-04-20T11:50:57.533 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 436, in _infer_image 2024-04-20T11:50:57.533 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 310, in _validate_fsid 2024-04-20T11:50:57.533 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/__main__.py", line 3359, in command_ceph_volume 2024-04-20T11:50:57.533 INFO:teuthology.orchestra.run.smithi117.stdout: File "/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/cephadm.632956097c576d0ec9244cad5b14f642b02543f0b519abdb4fc441decfe6d478/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-20T11:50:57.533 INFO:teuthology.orchestra.run.smithi117.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 -e NODE_NAME=smithi173 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/run/ceph:z -v /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpsbbdwnn7:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 inventory --format=json-pretty --filter-for-batch 2024-04-20T11:50:57.533 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:50:57.533 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:50:57.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b2406ecc0 msgr2=0x7f6b24071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:57.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b2406ecc0 0x7f6b24071180 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6b4019a2e0 tx=0x7f6b2c005cf0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 msgr2=0x7f6b40198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:50:57.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 0x7f6b40198d80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6b300092e0 tx=0x7f6b3000be20 comp rx=0 tx=0).stop 2024-04-20T11:50:57.536 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:57 smithi117 ceph-mon[99036]: from='client.36058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:57.536 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:57 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2161829423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:50:57.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 shutdown_connections 2024-04-20T11:50:57.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6b2406ecc0 0x7f6b24071180 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b40103040 0x7f6b40198d80 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 --2- 172.21.15.117:0/292353660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b401039f0 0x7f6b401992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:50:57.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.536+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 >> 172.21.15.117:0/292353660 conn(0x7f6b400fe8d0 msgr2=0x7f6b400fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:50:57.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.537+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 shutdown_connections 2024-04-20T11:50:57.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:50:57.537+0000 7f6b44a00700 1 -- 172.21.15.117:0/292353660 wait complete. 2024-04-20T11:50:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:58 smithi117 ceph-mon[99036]: pgmap v1157: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 8.9 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-20T11:50:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:58 smithi117 ceph-mon[99036]: from='client.36070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:58 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/292353660' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:50:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:50:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:50:58.652 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:58 smithi173 ceph-mon[74165]: pgmap v1157: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 8.9 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-20T11:50:58.652 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:58 smithi173 ceph-mon[74165]: from='client.36070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:50:58.652 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:58 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/292353660' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:50:58.652 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:58.652 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:58.652 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:50:58.652 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:50:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:51:00.527 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:00 smithi117 ceph-mon[99036]: pgmap v1158: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-20T11:51:00.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:00 smithi173 ceph-mon[74165]: pgmap v1158: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-20T11:51:02.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:02.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:01 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:02.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:01 smithi117 ceph-mon[99036]: pgmap v1159: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-04-20T11:51:02.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:02.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:01 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:02.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:01 smithi173 ceph-mon[74165]: pgmap v1159: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-04-20T11:51:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:03 smithi173 ceph-mon[74165]: pgmap v1160: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s wr, 1 op/s 2024-04-20T11:51:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:04.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:51:04.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:51:04.162 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:03 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:03 smithi117 ceph-mon[99036]: pgmap v1160: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s wr, 1 op/s 2024-04-20T11:51:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:04.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:04.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:51:04.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:51:04.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:03 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: pgmap v1161: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:51:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: Upgrade: Setting filesystem cephfs Joinable 2024-04-20T11:51:05.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-20T11:51:05.391 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:04 smithi117 ceph-mon[99036]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-20T11:51:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: pgmap v1161: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:51:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: Upgrade: Setting filesystem cephfs Joinable 2024-04-20T11:51:05.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-20T11:51:05.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:04 smithi173 ceph-mon[74165]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi117.miaplu=up:active} 2 up:standby 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all ceph-exporter 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi117"}]: dispatch 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi117"}]': finished 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi173"}]: dispatch 2024-04-20T11:51:06.392 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi173"}]': finished 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all iscsi 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all nfs 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: Upgrade: Setting container_image for all nvmeof 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.393 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: Upgrade: Finalizing container_image settings 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-20T11:51:06.394 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-20T11:51:06.395 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: Upgrade: Complete! 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-20T11:51:06.396 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:51:06.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:51:06.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:51:06.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:51:06.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:51:06.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:51:06.397 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:05 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: fsmap cephfs:2 {0=cephfs.smithi117.umapup=up:active,1=cephfs.smithi117.miaplu=up:active} 2 up:standby 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all ceph-exporter 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi117"}]: dispatch 2024-04-20T11:51:06.412 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi117"}]': finished 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi173"}]: dispatch 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi173"}]': finished 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all iscsi 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all nfs 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: Upgrade: Setting container_image for all nvmeof 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.413 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:06.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: Upgrade: Finalizing container_image settings 2024-04-20T11:51:06.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-20T11:51:06.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-20T11:51:06.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-20T11:51:06.415 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: Upgrade: Complete! 2024-04-20T11:51:06.416 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:51:06.417 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:05 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:07.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:07 smithi117 ceph-mon[99036]: pgmap v1162: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:07 smithi173 ceph-mon[74165]: pgmap v1162: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:09.606 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:09 smithi173 ceph-mon[74165]: pgmap v1163: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:09.606 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:09.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:09 smithi117 ceph-mon[99036]: pgmap v1163: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:09.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:11 smithi117 ceph-mon[99036]: pgmap v1164: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:11 smithi173 ceph-mon[74165]: pgmap v1164: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:13 smithi117 ceph-mon[99036]: pgmap v1165: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:51:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:13 smithi173 ceph-mon[74165]: pgmap v1165: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:51:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:51:15.806 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:15 smithi117 ceph-mon[99036]: pgmap v1166: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:15 smithi173 ceph-mon[74165]: pgmap v1166: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:17 smithi117 ceph-mon[99036]: pgmap v1167: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:17 smithi173 ceph-mon[74165]: pgmap v1167: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:19.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:19 smithi117 ceph-mon[99036]: pgmap v1168: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:19 smithi173 ceph-mon[74165]: pgmap v1168: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:21.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:21 smithi117 ceph-mon[99036]: pgmap v1169: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:21 smithi173 ceph-mon[74165]: pgmap v1169: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:23 smithi117 ceph-mon[99036]: pgmap v1170: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:23 smithi173 ceph-mon[74165]: pgmap v1170: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:25 smithi117 ceph-mon[99036]: pgmap v1171: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:25 smithi173 ceph-mon[74165]: pgmap v1171: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:27.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.722+0000 7fdc5c800700 1 -- 172.21.15.117:0/1405725061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 msgr2=0x7fdc541011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:27.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.722+0000 7fdc5c800700 1 --2- 172.21.15.117:0/1405725061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 0x7fdc541011f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009910 tx=0x7fdc48009c20 comp rx=0 tx=0).stop 2024-04-20T11:51:27.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.722+0000 7fdc5c800700 1 -- 172.21.15.117:0/1405725061 shutdown_connections 2024-04-20T11:51:27.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.722+0000 7fdc5c800700 1 --2- 172.21.15.117:0/1405725061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc541017c0 0x7fdc54105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:27.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.722+0000 7fdc5c800700 1 --2- 172.21.15.117:0/1405725061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 0x7fdc541011f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:27.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.722+0000 7fdc5c800700 1 -- 172.21.15.117:0/1405725061 >> 172.21.15.117:0/1405725061 conn(0x7fdc540fc6a0 msgr2=0x7fdc540feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:27.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.722+0000 7fdc5c800700 1 -- 172.21.15.117:0/1405725061 shutdown_connections 2024-04-20T11:51:27.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.722+0000 7fdc5c800700 1 -- 172.21.15.117:0/1405725061 wait complete. 2024-04-20T11:51:27.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.723+0000 7fdc5c800700 1 Processor -- start 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.724+0000 7fdc5c800700 1 -- start start 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.724+0000 7fdc5c800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 0x7fdc540731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.724+0000 7fdc5c800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc541017c0 0x7fdc54073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.724+0000 7fdc5c800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc54073c60 con 0x7fdc54100e10 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.724+0000 7fdc5c800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc54073da0 con 0x7fdc541017c0 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.724+0000 7fdc59e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 0x7fdc540731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.724+0000 7fdc59e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 0x7fdc540731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52814/0 (socket says 172.21.15.117:52814) 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.724+0000 7fdc59e00700 1 -- 172.21.15.117:0/264842921 learned_addr learned my addr 172.21.15.117:0/264842921 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.725+0000 7fdc59400700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc541017c0 0x7fdc54073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.725+0000 7fdc59e00700 1 -- 172.21.15.117:0/264842921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc541017c0 msgr2=0x7fdc54073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.725+0000 7fdc59e00700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc541017c0 0x7fdc54073720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:27.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.725+0000 7fdc59e00700 1 -- 172.21.15.117:0/264842921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480095f0 con 0x7fdc54100e10 2024-04-20T11:51:27.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.725+0000 7fdc59e00700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 0x7fdc540731e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006010 tx=0x7fdc4800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:27.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.725+0000 7fdc4ea00700 1 -- 172.21.15.117:0/264842921 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4801d070 con 0x7fdc54100e10 2024-04-20T11:51:27.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.725+0000 7fdc4ea00700 1 -- 172.21.15.117:0/264842921 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc480056e0 con 0x7fdc54100e10 2024-04-20T11:51:27.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.726+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc54074020 con 0x7fdc54100e10 2024-04-20T11:51:27.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.726+0000 7fdc4ea00700 1 -- 172.21.15.117:0/264842921 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc48017400 con 0x7fdc54100e10 2024-04-20T11:51:27.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.726+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc541a2f70 con 0x7fdc54100e10 2024-04-20T11:51:27.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.727+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc541091b0 con 0x7fdc54100e10 2024-04-20T11:51:27.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.728+0000 7fdc4ea00700 1 -- 172.21.15.117:0/264842921 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdc4800f6f0 con 0x7fdc54100e10 2024-04-20T11:51:27.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.729+0000 7fdc4ea00700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecc0 0x7fdc3c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:27.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.729+0000 7fdc4ea00700 1 -- 172.21.15.117:0/264842921 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdc48090280 con 0x7fdc54100e10 2024-04-20T11:51:27.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.729+0000 7fdc59400700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecc0 0x7fdc3c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:27.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.730+0000 7fdc59400700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecc0 0x7fdc3c071180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005950 tx=0x7fdc440058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:27.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.734+0000 7fdc4ea00700 1 -- 172.21.15.117:0/264842921 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fdc48058fc0 con 0x7fdc54100e10 2024-04-20T11:51:27.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:27 smithi117 ceph-mon[99036]: pgmap v1172: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:27.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:27 smithi173 ceph-mon[74165]: pgmap v1172: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:27.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.979+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc54074750 con 0x7fdc3c06ecc0 2024-04-20T11:51:27.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.981+0000 7fdc4ea00700 1 -- 172.21.15.117:0/264842921 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fdc54074750 con 0x7fdc3c06ecc0 2024-04-20T11:51:27.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecc0 msgr2=0x7fdc3c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:27.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecc0 0x7fdc3c071180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005950 tx=0x7fdc440058e0 comp rx=0 tx=0).stop 2024-04-20T11:51:27.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 msgr2=0x7fdc540731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:27.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 0x7fdc540731e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006010 tx=0x7fdc4800be20 comp rx=0 tx=0).stop 2024-04-20T11:51:27.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 shutdown_connections 2024-04-20T11:51:27.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fdc3c06ecc0 0x7fdc3c071180 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:27.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc54100e10 0x7fdc540731e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:27.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 --2- 172.21.15.117:0/264842921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc541017c0 0x7fdc54073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:27.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 >> 172.21.15.117:0/264842921 conn(0x7fdc540fc6a0 msgr2=0x7fdc540fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:27.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 shutdown_connections 2024-04-20T11:51:27.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:27.984+0000 7fdc5c800700 1 -- 172.21.15.117:0/264842921 wait complete. 2024-04-20T11:51:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:51:28.801 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph orch ps' 2024-04-20T11:51:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:51:29.215 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:51:29.621 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:29 smithi117 ceph-mon[99036]: pgmap v1173: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:29.621 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:29 smithi117 ceph-mon[99036]: from='client.36078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:51:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:29 smithi173 ceph-mon[74165]: pgmap v1173: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:29 smithi173 ceph-mon[74165]: from='client.36078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:51:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.533+0000 7fc358200700 1 -- 172.21.15.117:0/1471295818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350103110 msgr2=0x7fc3501034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.533+0000 7fc358200700 1 --2- 172.21.15.117:0/1471295818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350103110 0x7fc3501034f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc344009910 tx=0x7fc344009c20 comp rx=0 tx=0).stop 2024-04-20T11:51:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.533+0000 7fc358200700 1 -- 172.21.15.117:0/1471295818 shutdown_connections 2024-04-20T11:51:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.533+0000 7fc358200700 1 --2- 172.21.15.117:0/1471295818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc350104a00 0x7fc350104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.533+0000 7fc358200700 1 --2- 172.21.15.117:0/1471295818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350103110 0x7fc3501034f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.533+0000 7fc358200700 1 -- 172.21.15.117:0/1471295818 >> 172.21.15.117:0/1471295818 conn(0x7fc3500fe880 msgr2=0x7fc350100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.534+0000 7fc358200700 1 -- 172.21.15.117:0/1471295818 shutdown_connections 2024-04-20T11:51:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.534+0000 7fc358200700 1 -- 172.21.15.117:0/1471295818 wait complete. 2024-04-20T11:51:31.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.535+0000 7fc358200700 1 Processor -- start 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.535+0000 7fc358200700 1 -- start start 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.535+0000 7fc358200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc350103110 0x7fc350198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.535+0000 7fc358200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350104a00 0x7fc350199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.535+0000 7fc358200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3501999e0 con 0x7fc350104a00 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.535+0000 7fc358200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35019d660 con 0x7fc350103110 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.536+0000 7fc354e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350104a00 0x7fc350199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.536+0000 7fc354e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350104a00 0x7fc350199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52834/0 (socket says 172.21.15.117:52834) 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.536+0000 7fc354e00700 1 -- 172.21.15.117:0/724301148 learned_addr learned my addr 172.21.15.117:0/724301148 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.536+0000 7fc355800700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc350103110 0x7fc350198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.536+0000 7fc354e00700 1 -- 172.21.15.117:0/724301148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc350103110 msgr2=0x7fc350198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.536+0000 7fc354e00700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc350103110 0x7fc350198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.536+0000 7fc354e00700 1 -- 172.21.15.117:0/724301148 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3440095f0 con 0x7fc350104a00 2024-04-20T11:51:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.536+0000 7fc354e00700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350104a00 0x7fc350199300 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c8a0 tx=0x7fc34000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.537+0000 7fc34e000700 1 -- 172.21.15.117:0/724301148 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc34000cea0 con 0x7fc350104a00 2024-04-20T11:51:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.537+0000 7fc34e000700 1 -- 172.21.15.117:0/724301148 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc340004830 con 0x7fc350104a00 2024-04-20T11:51:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.537+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35019d940 con 0x7fc350104a00 2024-04-20T11:51:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.537+0000 7fc34e000700 1 -- 172.21.15.117:0/724301148 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3400055b0 con 0x7fc350104a00 2024-04-20T11:51:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.537+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35019df10 con 0x7fc350104a00 2024-04-20T11:51:31.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.538+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35010b460 con 0x7fc350104a00 2024-04-20T11:51:31.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.539+0000 7fc34e000700 1 -- 172.21.15.117:0/724301148 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc3400049a0 con 0x7fc350104a00 2024-04-20T11:51:31.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.540+0000 7fc34e000700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc33806ecc0 0x7fc338071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:31.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.540+0000 7fc34e000700 1 -- 172.21.15.117:0/724301148 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc34008eac0 con 0x7fc350104a00 2024-04-20T11:51:31.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.540+0000 7fc355800700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc33806ecc0 0x7fc338071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:31.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.541+0000 7fc355800700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc33806ecc0 0x7fc338071180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc344005f50 tx=0x7fc34401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:31.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.545+0000 7fc34e000700 1 -- 172.21.15.117:0/724301148 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc340057880 con 0x7fc350104a00 2024-04-20T11:51:31.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.781+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc35019a080 con 0x7fc33806ecc0 2024-04-20T11:51:31.781 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:31 smithi117 ceph-mon[99036]: pgmap v1174: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.799+0000 7fc34e000700 1 -- 172.21.15.117:0/724301148 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc35019a080 con 0x7fc33806ecc0 2024-04-20T11:51:31.800 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:51:31.800 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (39m) 35s ago 54m 15.3M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:51:31.800 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (52s) 35s ago 54m 7968k - 19.0.0-3133-ga8c84f59 76bc6b91b726 a6ba8f975041 2024-04-20T11:51:31.800 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (43s) 38s ago 53m 7302k - 19.0.0-3133-ga8c84f59 76bc6b91b726 1ed89bb400a5 2024-04-20T11:51:31.800 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (36m) 35s ago 54m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:51:31.800 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (36m) 38s ago 52m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (39m) 35s ago 53m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (31m) 35s ago 46m 24.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (32m) 35s ago 46m 152M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (16m) 38s ago 46m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (31m) 38s ago 46m 23.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (43m) 35s ago 55m 518M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (41m) 38s ago 52m 460M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (37m) 35s ago 55m 88.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (37m) 38s ago 52m 55.1M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (41m) 35s ago 54m 8330k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (41m) 38s ago 52m 8380k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (35m) 35s ago 51m 94.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (35m) 35s ago 51m 100M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (34m) 35s ago 50m 95.9M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:51:31.801 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (34m) 38s ago 49m 103M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:51:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (33m) 38s ago 49m 92.3M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:51:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (32m) 38s ago 48m 89.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:51:31.802 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (40m) 35s ago 53m 89.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:51:31.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc33806ecc0 msgr2=0x7fc338071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc33806ecc0 0x7fc338071180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc344005f50 tx=0x7fc34401a040 comp rx=0 tx=0).stop 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350104a00 msgr2=0x7fc350199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350104a00 0x7fc350199300 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c8a0 tx=0x7fc34000cc60 comp rx=0 tx=0).stop 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 shutdown_connections 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc33806ecc0 0x7fc338071180 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc350103110 0x7fc350198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 --2- 172.21.15.117:0/724301148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc350104a00 0x7fc350199300 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.802+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 >> 172.21.15.117:0/724301148 conn(0x7fc3500fe880 msgr2=0x7fc3500ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:31.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.803+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 shutdown_connections 2024-04-20T11:51:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:31.803+0000 7fc358200700 1 -- 172.21.15.117:0/724301148 wait complete. 2024-04-20T11:51:31.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:31 smithi173 ceph-mon[74165]: pgmap v1174: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:32.661 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph orch upgrade status' 2024-04-20T11:51:33.065 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:51:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:33 smithi117 ceph-mon[99036]: pgmap v1175: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:33 smithi117 ceph-mon[99036]: from='client.36082 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:51:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:33 smithi173 ceph-mon[74165]: pgmap v1175: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:33 smithi173 ceph-mon[74165]: from='client.36082 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:51:35.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.338+0000 7f233e400700 1 -- 172.21.15.117:0/350658664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 msgr2=0x7f2338107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.338+0000 7f233e400700 1 --2- 172.21.15.117:0/350658664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 0x7f2338107900 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2328009910 tx=0x7f2328009c20 comp rx=0 tx=0).stop 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.339+0000 7f233e400700 1 -- 172.21.15.117:0/350658664 shutdown_connections 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.339+0000 7f233e400700 1 --2- 172.21.15.117:0/350658664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2338108e10 0x7f2338109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.339+0000 7f233e400700 1 --2- 172.21.15.117:0/350658664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 0x7f2338107900 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.339+0000 7f233e400700 1 -- 172.21.15.117:0/350658664 >> 172.21.15.117:0/350658664 conn(0x7f23380768f0 msgr2=0x7f2338078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.339+0000 7f233e400700 1 -- 172.21.15.117:0/350658664 shutdown_connections 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.339+0000 7f233e400700 1 -- 172.21.15.117:0/350658664 wait complete. 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.340+0000 7f233e400700 1 Processor -- start 2024-04-20T11:51:35.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.340+0000 7f233e400700 1 -- start start 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.340+0000 7f233e400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 0x7f233819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.340+0000 7f233e400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2338108e10 0x7f233819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.340+0000 7f233e400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233819ddd0 con 0x7f2338107520 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.340+0000 7f233e400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23381a1aa0 con 0x7f2338108e10 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.341+0000 7f2337400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 0x7f233819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.341+0000 7f2337400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 0x7f233819d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53706/0 (socket says 172.21.15.117:53706) 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.341+0000 7f2337400700 1 -- 172.21.15.117:0/4277254851 learned_addr learned my addr 172.21.15.117:0/4277254851 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.341+0000 7f232fe00700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2338108e10 0x7f233819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.341+0000 7f2337400700 1 -- 172.21.15.117:0/4277254851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2338108e10 msgr2=0x7f233819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.341+0000 7f2337400700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2338108e10 0x7f233819d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:35.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.341+0000 7f2337400700 1 -- 172.21.15.117:0/4277254851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280095f0 con 0x7f2338107520 2024-04-20T11:51:35.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.341+0000 7f2337400700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 0x7f233819d1b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2328006010 tx=0x7f232800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:35.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.342+0000 7f2334c00700 1 -- 172.21.15.117:0/4277254851 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232801d070 con 0x7f2338107520 2024-04-20T11:51:35.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.342+0000 7f2334c00700 1 -- 172.21.15.117:0/4277254851 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f232800f460 con 0x7f2338107520 2024-04-20T11:51:35.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.342+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23381a1d20 con 0x7f2338107520 2024-04-20T11:51:35.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.342+0000 7f2334c00700 1 -- 172.21.15.117:0/4277254851 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2328017630 con 0x7f2338107520 2024-04-20T11:51:35.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.342+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23381a2290 con 0x7f2338107520 2024-04-20T11:51:35.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.343+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233810f890 con 0x7f2338107520 2024-04-20T11:51:35.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.344+0000 7f2334c00700 1 -- 172.21.15.117:0/4277254851 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f232800f5d0 con 0x7f2338107520 2024-04-20T11:51:35.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.345+0000 7f2334c00700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f231c06ecc0 0x7f231c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:35.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.345+0000 7f2334c00700 1 -- 172.21.15.117:0/4277254851 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2328090360 con 0x7f2338107520 2024-04-20T11:51:35.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.345+0000 7f232fe00700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f231c06ecc0 0x7f231c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:35.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.346+0000 7f232fe00700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f231c06ecc0 0x7f231c071180 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f233819e710 tx=0x7f2320005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:35.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.350+0000 7f2334c00700 1 -- 172.21.15.117:0/4277254851 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f23280591e0 con 0x7f2338107520 2024-04-20T11:51:35.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.597+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f233819e510 con 0x7f231c06ecc0 2024-04-20T11:51:35.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.599+0000 7f2334c00700 1 -- 172.21.15.117:0/4277254851 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f233819e510 con 0x7f231c06ecc0 2024-04-20T11:51:35.599 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:51:35.599 INFO:teuthology.orchestra.run.smithi117.stdout: "target_image": null, 2024-04-20T11:51:35.599 INFO:teuthology.orchestra.run.smithi117.stdout: "in_progress": false, 2024-04-20T11:51:35.599 INFO:teuthology.orchestra.run.smithi117.stdout: "which": "", 2024-04-20T11:51:35.599 INFO:teuthology.orchestra.run.smithi117.stdout: "services_complete": [], 2024-04-20T11:51:35.600 INFO:teuthology.orchestra.run.smithi117.stdout: "progress": null, 2024-04-20T11:51:35.600 INFO:teuthology.orchestra.run.smithi117.stdout: "message": "", 2024-04-20T11:51:35.600 INFO:teuthology.orchestra.run.smithi117.stdout: "is_paused": false 2024-04-20T11:51:35.600 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:51:35.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.602+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f231c06ecc0 msgr2=0x7f231c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:35.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.602+0000 7f233e400700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f231c06ecc0 0x7f231c071180 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f233819e710 tx=0x7f2320005cf0 comp rx=0 tx=0).stop 2024-04-20T11:51:35.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.602+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 msgr2=0x7f233819d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:35.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.602+0000 7f233e400700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 0x7f233819d1b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2328006010 tx=0x7f232800be20 comp rx=0 tx=0).stop 2024-04-20T11:51:35.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.602+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 shutdown_connections 2024-04-20T11:51:35.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.602+0000 7f233e400700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f231c06ecc0 0x7f231c071180 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:35.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.602+0000 7f233e400700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2338107520 0x7f233819d1b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:35.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.602+0000 7f233e400700 1 --2- 172.21.15.117:0/4277254851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2338108e10 0x7f233819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:35.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.603+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 >> 172.21.15.117:0/4277254851 conn(0x7f23380768f0 msgr2=0x7f2338077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:35.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.603+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 shutdown_connections 2024-04-20T11:51:35.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:35.603+0000 7f233e400700 1 -- 172.21.15.117:0/4277254851 wait complete. 2024-04-20T11:51:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:35 smithi117 ceph-mon[99036]: pgmap v1176: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:35 smithi173 ceph-mon[74165]: pgmap v1176: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:36.484 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph health detail' 2024-04-20T11:51:36.892 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:51:37.581 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:37 smithi117 ceph-mon[99036]: from='client.36086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:51:37.582 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:37 smithi117 ceph-mon[99036]: pgmap v1177: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:37.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:37 smithi173 ceph-mon[74165]: from='client.36086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:51:37.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:37 smithi173 ceph-mon[74165]: pgmap v1177: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:39.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.236+0000 7fd700600700 1 -- 172.21.15.117:0/1196602815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 msgr2=0x7fd6f80736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:39.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.236+0000 7fd700600700 1 --2- 172.21.15.117:0/1196602815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 0x7fd6f80736f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00b0a0 tx=0x7fd6ec00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:51:39.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.237+0000 7fd700600700 1 -- 172.21.15.117:0/1196602815 shutdown_connections 2024-04-20T11:51:39.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.237+0000 7fd700600700 1 --2- 172.21.15.117:0/1196602815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6f8073c30 0x7fd6f810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:39.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.237+0000 7fd700600700 1 --2- 172.21.15.117:0/1196602815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 0x7fd6f80736f0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:39.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.237+0000 7fd700600700 1 -- 172.21.15.117:0/1196602815 >> 172.21.15.117:0/1196602815 conn(0x7fd6f80fc6c0 msgr2=0x7fd6f80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:39.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.237+0000 7fd700600700 1 -- 172.21.15.117:0/1196602815 shutdown_connections 2024-04-20T11:51:39.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.237+0000 7fd700600700 1 -- 172.21.15.117:0/1196602815 wait complete. 2024-04-20T11:51:39.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.238+0000 7fd700600700 1 Processor -- start 2024-04-20T11:51:39.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.238+0000 7fd700600700 1 -- start start 2024-04-20T11:51:39.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd700600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 0x7fd6f8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:39.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd700600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6f8073c30 0x7fd6f8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:39.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd700600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8199840 con 0x7fd6f8073310 2024-04-20T11:51:39.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd700600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81999b0 con 0x7fd6f8073c30 2024-04-20T11:51:39.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd6fdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 0x7fd6f8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:39.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd6fdc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 0x7fd6f8198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53728/0 (socket says 172.21.15.117:53728) 2024-04-20T11:51:39.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd6fdc00700 1 -- 172.21.15.117:0/1647357987 learned_addr learned my addr 172.21.15.117:0/1647357987 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:51:39.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd6fd200700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6f8073c30 0x7fd6f8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:39.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd6fdc00700 1 -- 172.21.15.117:0/1647357987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6f8073c30 msgr2=0x7fd6f8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:39.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.239+0000 7fd6fdc00700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6f8073c30 0x7fd6f8199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:39.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.240+0000 7fd6fdc00700 1 -- 172.21.15.117:0/1647357987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec009d00 con 0x7fd6f8073310 2024-04-20T11:51:39.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.240+0000 7fd6fdc00700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 0x7fd6f8198d00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec015040 tx=0x7fd6ec003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:39.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.240+0000 7fd6f6000700 1 -- 172.21.15.117:0/1647357987 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6ec00e040 con 0x7fd6f8073310 2024-04-20T11:51:39.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.240+0000 7fd6f6000700 1 -- 172.21.15.117:0/1647357987 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6ec007cd0 con 0x7fd6f8073310 2024-04-20T11:51:39.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.240+0000 7fd6f6000700 1 -- 172.21.15.117:0/1647357987 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6ec01d6b0 con 0x7fd6f8073310 2024-04-20T11:51:39.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.240+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f819c630 con 0x7fd6f8073310 2024-04-20T11:51:39.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.240+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f819cb20 con 0x7fd6f8073310 2024-04-20T11:51:39.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.242+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f80749c0 con 0x7fd6f8073310 2024-04-20T11:51:39.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.243+0000 7fd6f6000700 1 -- 172.21.15.117:0/1647357987 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd6ec004040 con 0x7fd6f8073310 2024-04-20T11:51:39.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.244+0000 7fd6f6000700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd6e006ecc0 0x7fd6e0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:39.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.244+0000 7fd6f6000700 1 -- 172.21.15.117:0/1647357987 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd6ec0905e0 con 0x7fd6f8073310 2024-04-20T11:51:39.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.244+0000 7fd6fd200700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd6e006ecc0 0x7fd6e0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:39.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.245+0000 7fd6fd200700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd6e006ecc0 0x7fd6e0071180 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4005ea0 tx=0x7fd6e4005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:39.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.249+0000 7fd6f6000700 1 -- 172.21.15.117:0/1647357987 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd6ec0593c0 con 0x7fd6f8073310 2024-04-20T11:51:39.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.566+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd6f8110070 con 0x7fd6f8073310 2024-04-20T11:51:39.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.567+0000 7fd6f6000700 1 -- 172.21.15.117:0/1647357987 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd6ec05ca80 con 0x7fd6f8073310 2024-04-20T11:51:39.567 INFO:teuthology.orchestra.run.smithi117.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-20T11:51:39.567 INFO:teuthology.orchestra.run.smithi117.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-20T11:51:39.567 INFO:teuthology.orchestra.run.smithi117.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-20T11:51:39.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd6e006ecc0 msgr2=0x7fd6e0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:39.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd6e006ecc0 0x7fd6e0071180 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4005ea0 tx=0x7fd6e4005e30 comp rx=0 tx=0).stop 2024-04-20T11:51:39.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 msgr2=0x7fd6f8198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:39.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 0x7fd6f8198d00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec015040 tx=0x7fd6ec003ce0 comp rx=0 tx=0).stop 2024-04-20T11:51:39.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 shutdown_connections 2024-04-20T11:51:39.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd6e006ecc0 0x7fd6e0071180 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:39.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f8073310 0x7fd6f8198d00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:39.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 --2- 172.21.15.117:0/1647357987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6f8073c30 0x7fd6f8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:39.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 >> 172.21.15.117:0/1647357987 conn(0x7fd6f80fc6c0 msgr2=0x7fd6f8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:39.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 shutdown_connections 2024-04-20T11:51:39.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:39.570+0000 7fd700600700 1 -- 172.21.15.117:0/1647357987 wait complete. 2024-04-20T11:51:39.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:39 smithi117 ceph-mon[99036]: pgmap v1178: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:39.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:39 smithi173 ceph-mon[74165]: pgmap v1178: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:40.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph versions' 2024-04-20T11:51:40.716 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:51:40.827 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:40 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1647357987' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:51:40.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:40 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1647357987' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-20T11:51:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:41 smithi117 ceph-mon[99036]: pgmap v1179: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:41 smithi173 ceph-mon[74165]: pgmap v1179: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.901+0000 7efc50400700 1 -- 172.21.15.117:0/2118383366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc48073bd0 msgr2=0x7efc4810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.901+0000 7efc50400700 1 --2- 172.21.15.117:0/2118383366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc48073bd0 0x7efc4810cf10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7efc34009910 tx=0x7efc34009c20 comp rx=0 tx=0).stop 2024-04-20T11:51:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.901+0000 7efc50400700 1 -- 172.21.15.117:0/2118383366 shutdown_connections 2024-04-20T11:51:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.901+0000 7efc50400700 1 --2- 172.21.15.117:0/2118383366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc48073bd0 0x7efc4810cf10 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.901+0000 7efc50400700 1 --2- 172.21.15.117:0/2118383366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc480732b0 0x7efc48073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.901+0000 7efc50400700 1 -- 172.21.15.117:0/2118383366 >> 172.21.15.117:0/2118383366 conn(0x7efc480fc6a0 msgr2=0x7efc480feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.901+0000 7efc50400700 1 -- 172.21.15.117:0/2118383366 shutdown_connections 2024-04-20T11:51:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.901+0000 7efc50400700 1 -- 172.21.15.117:0/2118383366 wait complete. 2024-04-20T11:51:42.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.902+0000 7efc50400700 1 Processor -- start 2024-04-20T11:51:42.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.903+0000 7efc50400700 1 -- start start 2024-04-20T11:51:42.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.903+0000 7efc50400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc480732b0 0x7efc48198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:42.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.903+0000 7efc50400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc48073bd0 0x7efc48199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:42.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.903+0000 7efc50400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc481998d0 con 0x7efc480732b0 2024-04-20T11:51:42.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.903+0000 7efc50400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4819c540 con 0x7efc48073bd0 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.903+0000 7efc4da00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc480732b0 0x7efc48198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.903+0000 7efc4da00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc480732b0 0x7efc48198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53748/0 (socket says 172.21.15.117:53748) 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.903+0000 7efc4da00700 1 -- 172.21.15.117:0/998470735 learned_addr learned my addr 172.21.15.117:0/998470735 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.904+0000 7efc4d000700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc48073bd0 0x7efc48199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.904+0000 7efc4da00700 1 -- 172.21.15.117:0/998470735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc48073bd0 msgr2=0x7efc48199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.904+0000 7efc4da00700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc48073bd0 0x7efc48199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.904+0000 7efc4da00700 1 -- 172.21.15.117:0/998470735 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc340095f0 con 0x7efc480732b0 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.904+0000 7efc4d000700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc48073bd0 0x7efc48199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:51:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.904+0000 7efc4da00700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc480732b0 0x7efc48198d00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00c8a0 tx=0x7efc3c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.905+0000 7efc46000700 1 -- 172.21.15.117:0/998470735 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3c00cea0 con 0x7efc480732b0 2024-04-20T11:51:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.905+0000 7efc46000700 1 -- 172.21.15.117:0/998470735 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc3c004830 con 0x7efc480732b0 2024-04-20T11:51:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.905+0000 7efc46000700 1 -- 172.21.15.117:0/998470735 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3c0055b0 con 0x7efc480732b0 2024-04-20T11:51:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.905+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4819c820 con 0x7efc480732b0 2024-04-20T11:51:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.905+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4819cd70 con 0x7efc480732b0 2024-04-20T11:51:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.906+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc48074960 con 0x7efc480732b0 2024-04-20T11:51:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.907+0000 7efc46000700 1 -- 172.21.15.117:0/998470735 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efc3c0049a0 con 0x7efc480732b0 2024-04-20T11:51:42.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.909+0000 7efc46000700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efc3006ecd0 0x7efc30071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:51:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.909+0000 7efc46000700 1 -- 172.21.15.117:0/998470735 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efc3c08eac0 con 0x7efc480732b0 2024-04-20T11:51:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.909+0000 7efc4d000700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efc3006ecd0 0x7efc30071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:51:42.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.910+0000 7efc4d000700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efc3006ecd0 0x7efc30071190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efc34006010 tx=0x7efc3400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:51:42.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:42.914+0000 7efc46000700 1 -- 172.21.15.117:0/998470735 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efc3c057880 con 0x7efc480732b0 2024-04-20T11:51:43.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.244+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efc481077e0 con 0x7efc480732b0 2024-04-20T11:51:43.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.246+0000 7efc46000700 1 -- 172.21.15.117:0/998470735 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efc3c01d070 con 0x7efc480732b0 2024-04-20T11:51:43.246 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:51:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:51:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:51:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:51:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:51:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:51:43.247 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:51:43.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.248+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efc3006ecd0 msgr2=0x7efc30071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:43.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.248+0000 7efc50400700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efc3006ecd0 0x7efc30071190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efc34006010 tx=0x7efc3400b540 comp rx=0 tx=0).stop 2024-04-20T11:51:43.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.248+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc480732b0 msgr2=0x7efc48198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:51:43.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.248+0000 7efc50400700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc480732b0 0x7efc48198d00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00c8a0 tx=0x7efc3c00cc60 comp rx=0 tx=0).stop 2024-04-20T11:51:43.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.249+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 shutdown_connections 2024-04-20T11:51:43.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.249+0000 7efc50400700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efc3006ecd0 0x7efc30071190 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:43.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.249+0000 7efc50400700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc480732b0 0x7efc48198d00 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:43.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.249+0000 7efc50400700 1 --2- 172.21.15.117:0/998470735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc48073bd0 0x7efc48199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:51:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.249+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 >> 172.21.15.117:0/998470735 conn(0x7efc480fc6a0 msgr2=0x7efc481053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:51:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.249+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 shutdown_connections 2024-04-20T11:51:43.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:51:43.249+0000 7efc50400700 1 -- 172.21.15.117:0/998470735 wait complete. 2024-04-20T11:51:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:43 smithi117 ceph-mon[99036]: pgmap v1180: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:51:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:43 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/998470735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:43 smithi173 ceph-mon[74165]: pgmap v1180: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:51:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:43 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/998470735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:51:44.083 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-20T11:51:44.492 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:51:45.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:45 smithi117 ceph-mon[99036]: pgmap v1181: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:45 smithi173 ceph-mon[74165]: pgmap v1181: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:46.692 INFO:teuthology.orchestra.run.smithi117.stdout:wait for servicemap items w/ changing names to refresh 2024-04-20T11:51:47.424 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'sleep 60' 2024-04-20T11:51:47.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:47 smithi117 ceph-mon[99036]: pgmap v1182: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:47.840 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:51:47.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:47 smithi173 ceph-mon[74165]: pgmap v1182: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:49.872 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:49 smithi117 ceph-mon[99036]: pgmap v1183: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:49 smithi173 ceph-mon[74165]: pgmap v1183: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:51.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:51 smithi117 ceph-mon[99036]: pgmap v1184: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:51 smithi173 ceph-mon[74165]: pgmap v1184: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:53.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:53 smithi117 ceph-mon[99036]: pgmap v1185: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:53.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:53 smithi173 ceph-mon[74165]: pgmap v1185: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:54.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:54 smithi117 ceph-mon[99036]: pgmap v1186: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:54.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:54 smithi173 ceph-mon[74165]: pgmap v1186: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:57.100 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:56 smithi117 ceph-mon[99036]: pgmap v1187: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:56 smithi173 ceph-mon[74165]: pgmap v1187: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:59.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:58 smithi117 ceph-mon[99036]: pgmap v1188: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:59.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:51:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:51:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:58 smithi173 ceph-mon[74165]: pgmap v1188: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:51:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:51:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:01.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:00 smithi117 ceph-mon[99036]: pgmap v1189: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:01.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:00 smithi173 ceph-mon[74165]: pgmap v1189: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:03.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:02 smithi117 ceph-mon[99036]: pgmap v1190: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:03.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:02 smithi173 ceph-mon[74165]: pgmap v1190: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:05.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:04 smithi117 ceph-mon[99036]: pgmap v1191: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:05.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:04 smithi173 ceph-mon[74165]: pgmap v1191: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:07.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:06 smithi117 ceph-mon[99036]: pgmap v1192: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:07.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:06 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:52:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:06 smithi173 ceph-mon[74165]: pgmap v1192: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:06 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:52:08.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:52:08.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:52:08.063 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:07 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:52:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:52:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:52:08.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:07 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:52:09.620 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:09 smithi173 ceph-mon[74165]: pgmap v1193: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:09.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:09 smithi117 ceph-mon[99036]: pgmap v1193: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:11.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:11 smithi117 ceph-mon[99036]: pgmap v1194: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:11 smithi173 ceph-mon[74165]: pgmap v1194: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:13 smithi117 ceph-mon[99036]: pgmap v1195: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:13 smithi173 ceph-mon[74165]: pgmap v1195: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:15 smithi117 ceph-mon[99036]: pgmap v1196: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:15 smithi173 ceph-mon[74165]: pgmap v1196: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:17.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:17 smithi117 ceph-mon[99036]: pgmap v1197: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:17 smithi173 ceph-mon[74165]: pgmap v1197: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:19 smithi173 ceph-mon[74165]: pgmap v1198: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:19 smithi117 ceph-mon[99036]: pgmap v1198: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:21.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:21 smithi117 ceph-mon[99036]: pgmap v1199: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:21 smithi173 ceph-mon[74165]: pgmap v1199: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:23.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:23 smithi117 ceph-mon[99036]: pgmap v1200: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:23 smithi173 ceph-mon[74165]: pgmap v1200: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:25 smithi117 ceph-mon[99036]: pgmap v1201: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:25 smithi173 ceph-mon[74165]: pgmap v1201: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:27.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:27 smithi117 ceph-mon[99036]: pgmap v1202: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:27 smithi173 ceph-mon[74165]: pgmap v1202: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:28.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:29 smithi117 ceph-mon[99036]: pgmap v1203: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:29 smithi173 ceph-mon[74165]: pgmap v1203: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:31.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:31 smithi117 ceph-mon[99036]: pgmap v1204: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:31 smithi173 ceph-mon[74165]: pgmap v1204: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:33.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:33 smithi117 ceph-mon[99036]: pgmap v1205: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:33 smithi173 ceph-mon[74165]: pgmap v1205: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:35 smithi117 ceph-mon[99036]: pgmap v1206: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:35 smithi173 ceph-mon[74165]: pgmap v1206: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:37.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:37 smithi117 ceph-mon[99036]: pgmap v1207: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:37 smithi173 ceph-mon[74165]: pgmap v1207: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:39.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:39 smithi117 ceph-mon[99036]: pgmap v1208: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:39 smithi173 ceph-mon[74165]: pgmap v1208: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:41 smithi117 ceph-mon[99036]: pgmap v1209: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:41 smithi173 ceph-mon[74165]: pgmap v1209: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:43 smithi117 ceph-mon[99036]: pgmap v1210: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:43 smithi173 ceph-mon[74165]: pgmap v1210: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:45.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:45 smithi117 ceph-mon[99036]: pgmap v1211: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:45 smithi173 ceph-mon[74165]: pgmap v1211: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:47.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:47 smithi117 ceph-mon[99036]: pgmap v1212: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:47 smithi173 ceph-mon[74165]: pgmap v1212: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:49.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:49 smithi117 ceph-mon[99036]: pgmap v1213: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:49 smithi173 ceph-mon[74165]: pgmap v1213: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:50.803 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph orch ps' 2024-04-20T11:52:51.217 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:52:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:51 smithi117 ceph-mon[99036]: pgmap v1214: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:51 smithi173 ceph-mon[74165]: pgmap v1214: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.507+0000 7f53eda00700 1 -- 172.21.15.117:0/1411781590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e81039f0 msgr2=0x7f53e8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:52:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.507+0000 7f53eda00700 1 --2- 172.21.15.117:0/1411781590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e81039f0 0x7f53e8107a40 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009910 tx=0x7f53d4009c20 comp rx=0 tx=0).stop 2024-04-20T11:52:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.507+0000 7f53eda00700 1 -- 172.21.15.117:0/1411781590 shutdown_connections 2024-04-20T11:52:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.507+0000 7f53eda00700 1 --2- 172.21.15.117:0/1411781590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e81039f0 0x7f53e8107a40 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.507+0000 7f53eda00700 1 --2- 172.21.15.117:0/1411781590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e8103040 0x7f53e8103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.507+0000 7f53eda00700 1 -- 172.21.15.117:0/1411781590 >> 172.21.15.117:0/1411781590 conn(0x7f53e80fe8d0 msgr2=0x7f53e8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:52:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.507+0000 7f53eda00700 1 -- 172.21.15.117:0/1411781590 shutdown_connections 2024-04-20T11:52:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.508+0000 7f53eda00700 1 -- 172.21.15.117:0/1411781590 wait complete. 2024-04-20T11:52:53.510 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:53 smithi117 ceph-mon[99036]: pgmap v1215: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.509+0000 7f53eda00700 1 Processor -- start 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.509+0000 7f53eda00700 1 -- start start 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.509+0000 7f53eda00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e8103040 0x7f53e8198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.509+0000 7f53eda00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e81039f0 0x7f53e81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.509+0000 7f53eda00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e8199940 con 0x7f53e8103040 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.509+0000 7f53eda00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e819c560 con 0x7f53e81039f0 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.510+0000 7f53e6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e8103040 0x7f53e8198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.510+0000 7f53e6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e8103040 0x7f53e8198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56608/0 (socket says 172.21.15.117:56608) 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.510+0000 7f53e6a00700 1 -- 172.21.15.117:0/1341990630 learned_addr learned my addr 172.21.15.117:0/1341990630 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.510+0000 7f53e6000700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e81039f0 0x7f53e81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:52:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.510+0000 7f53e6a00700 1 -- 172.21.15.117:0/1341990630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e81039f0 msgr2=0x7f53e81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:52:53.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.510+0000 7f53e6a00700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e81039f0 0x7f53e81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:53.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.510+0000 7f53e6a00700 1 -- 172.21.15.117:0/1341990630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40095f0 con 0x7f53e8103040 2024-04-20T11:52:53.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.511+0000 7f53e6a00700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e8103040 0x7f53e8198d70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f53d800d8d0 tx=0x7f53d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:52:53.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.511+0000 7f53df400700 1 -- 172.21.15.117:0/1341990630 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d8009940 con 0x7f53e8103040 2024-04-20T11:52:53.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.511+0000 7f53df400700 1 -- 172.21.15.117:0/1341990630 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53d8010460 con 0x7f53e8103040 2024-04-20T11:52:53.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.511+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e819c840 con 0x7f53e8103040 2024-04-20T11:52:53.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.511+0000 7f53df400700 1 -- 172.21.15.117:0/1341990630 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d800f5f0 con 0x7f53e8103040 2024-04-20T11:52:53.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.511+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e819cd60 con 0x7f53e8103040 2024-04-20T11:52:53.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.512+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e810b440 con 0x7f53e8103040 2024-04-20T11:52:53.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.514+0000 7f53df400700 1 -- 172.21.15.117:0/1341990630 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f53d80105d0 con 0x7f53e8103040 2024-04-20T11:52:53.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.515+0000 7f53df400700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f53cc06ecc0 0x7f53cc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:52:53.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.515+0000 7f53df400700 1 -- 172.21.15.117:0/1341990630 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f53d808ee10 con 0x7f53e8103040 2024-04-20T11:52:53.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.516+0000 7f53e6000700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f53cc06ecc0 0x7f53cc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:52:53.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.516+0000 7f53e6000700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f53cc06ecc0 0x7f53cc071180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f53e819a2e0 tx=0x7f53d401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:52:53.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.520+0000 7f53df400700 1 -- 172.21.15.117:0/1341990630 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f53d8057bd0 con 0x7f53e8103040 2024-04-20T11:52:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:53 smithi173 ceph-mon[74165]: pgmap v1215: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:53.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.761+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 --> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f53e819a060 con 0x7f53cc06ecc0 2024-04-20T11:52:53.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.774+0000 7f53df400700 1 -- 172.21.15.117:0/1341990630 <== mgr.34100 v2:172.21.15.117:6800/2604153943 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f53e819a060 con 0x7f53cc06ecc0 2024-04-20T11:52:53.775 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:52:53.775 INFO:teuthology.orchestra.run.smithi117.stdout:alertmanager.smithi117 smithi117 *:9093,9094 running (41m) 117s ago 55m 15.3M - 0.25.0 c8568f914cd2 b2e137c66677 2024-04-20T11:52:53.775 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi117 smithi117 running (2m) 117s ago 55m 7968k - 19.0.0-3133-ga8c84f59 76bc6b91b726 a6ba8f975041 2024-04-20T11:52:53.775 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-exporter.smithi173 smithi173 running (2m) 2m ago 54m 7302k - 19.0.0-3133-ga8c84f59 76bc6b91b726 1ed89bb400a5 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi117 smithi117 running (37m) 117s ago 55m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 623887b87564 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:crash.smithi173 smithi173 running (37m) 2m ago 54m 6660k - 19.0.0-3133-ga8c84f59 76bc6b91b726 d39737d8918d 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:grafana.smithi117 smithi117 *:3000 running (40m) 117s ago 55m 73.9M - 9.4.12 2bacad6d85d8 00c216cd853b 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.miaplu smithi117 running (33m) 117s ago 48m 24.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 64696c9854da 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi117.umapup smithi117 running (33m) 117s ago 48m 152M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fee7675ecf58 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pfrgat smithi173 running (17m) 2m ago 48m 18.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3a4857c38e80 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:mds.cephfs.smithi173.pybzht smithi173 running (32m) 2m ago 48m 23.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 02909b6d3a9f 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi117.ivnklj smithi117 *:8443,9283,8765 running (44m) 117s ago 57m 518M - 19.0.0-3133-ga8c84f59 76bc6b91b726 82ee1a674ae4 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.smithi173.gnrjof smithi173 *:8443,9283,8765 running (43m) 2m ago 54m 460M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2a4cd6fc7632 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi117 smithi117 running (39m) 117s ago 57m 88.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 ec2a5c1b3288 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:mon.smithi173 smithi173 running (38m) 2m ago 53m 55.1M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 32e7ccffcc30 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi117 smithi117 *:9100 running (42m) 117s ago 55m 8330k - 1.5.0 0da6a335fe13 0f33f81c7a2f 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:node-exporter.smithi173 smithi173 *:9100 running (42m) 2m ago 54m 8380k - 1.5.0 0da6a335fe13 83567ac68de2 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (37m) 117s ago 53m 94.6M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 8eceb629d7d2 2024-04-20T11:52:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi117 running (36m) 117s ago 52m 100M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 fbba94ccd1b4 2024-04-20T11:52:53.777 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi117 running (35m) 117s ago 51m 95.9M 1216M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b78740c3941 2024-04-20T11:52:53.777 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi173 running (35m) 2m ago 51m 103M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 266ace61d6c6 2024-04-20T11:52:53.777 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi173 running (34m) 2m ago 50m 92.3M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 91012d4ad129 2024-04-20T11:52:53.777 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi173 running (34m) 2m ago 50m 89.0M 2240M 19.0.0-3133-ga8c84f59 76bc6b91b726 b9aa7862f8f5 2024-04-20T11:52:53.777 INFO:teuthology.orchestra.run.smithi117.stdout:prometheus.smithi117 smithi117 *:9095 running (41m) 117s ago 54m 89.4M - 2.43.0 a07b618ecd1d d9635df18397 2024-04-20T11:52:53.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f53cc06ecc0 msgr2=0x7f53cc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:52:53.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f53cc06ecc0 0x7f53cc071180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f53e819a2e0 tx=0x7f53d401a040 comp rx=0 tx=0).stop 2024-04-20T11:52:53.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e8103040 msgr2=0x7f53e8198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:52:53.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e8103040 0x7f53e8198d70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f53d800d8d0 tx=0x7f53d800dc90 comp rx=0 tx=0).stop 2024-04-20T11:52:53.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 shutdown_connections 2024-04-20T11:52:53.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f53cc06ecc0 0x7f53cc071180 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:53.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e8103040 0x7f53e8198d70 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:53.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 --2- 172.21.15.117:0/1341990630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e81039f0 0x7f53e81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:53.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 >> 172.21.15.117:0/1341990630 conn(0x7f53e80fe8d0 msgr2=0x7f53e80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:52:53.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 shutdown_connections 2024-04-20T11:52:53.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:53.777+0000 7f53eda00700 1 -- 172.21.15.117:0/1341990630 wait complete. 2024-04-20T11:52:54.597 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph versions' 2024-04-20T11:52:55.077 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:52:55.482 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:55 smithi117 ceph-mon[99036]: pgmap v1216: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:55.483 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:55 smithi117 ceph-mon[99036]: from='client.36098 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:52:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:55 smithi173 ceph-mon[74165]: pgmap v1216: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:55 smithi173 ceph-mon[74165]: from='client.36098 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:52:57.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.388+0000 7f24c8600700 1 -- 172.21.15.117:0/1397236165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c01017c0 msgr2=0x7f24c0105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:52:57.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.388+0000 7f24c8600700 1 --2- 172.21.15.117:0/1397236165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c01017c0 0x7f24c0105810 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009910 tx=0x7f24ac009c20 comp rx=0 tx=0).stop 2024-04-20T11:52:57.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.388+0000 7f24c8600700 1 -- 172.21.15.117:0/1397236165 shutdown_connections 2024-04-20T11:52:57.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.388+0000 7f24c8600700 1 --2- 172.21.15.117:0/1397236165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c01017c0 0x7f24c0105810 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:57.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.388+0000 7f24c8600700 1 --2- 172.21.15.117:0/1397236165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24c0100e10 0x7f24c01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:57.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.388+0000 7f24c8600700 1 -- 172.21.15.117:0/1397236165 >> 172.21.15.117:0/1397236165 conn(0x7f24c00fc6c0 msgr2=0x7f24c00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:52:57.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.388+0000 7f24c8600700 1 -- 172.21.15.117:0/1397236165 shutdown_connections 2024-04-20T11:52:57.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.388+0000 7f24c8600700 1 -- 172.21.15.117:0/1397236165 wait complete. 2024-04-20T11:52:57.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.389+0000 7f24c8600700 1 Processor -- start 2024-04-20T11:52:57.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.390+0000 7f24c8600700 1 -- start start 2024-04-20T11:52:57.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.390+0000 7f24c8600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c0100e10 0x7f24c0198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:52:57.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.390+0000 7f24c8600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24c01017c0 0x7f24c01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:52:57.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.390+0000 7f24c8600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c0199ad0 con 0x7f24c0100e10 2024-04-20T11:52:57.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.390+0000 7f24c8600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c019d6e0 con 0x7f24c01017c0 2024-04-20T11:52:57.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.390+0000 7f24c5c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c0100e10 0x7f24c0198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:52:57.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.390+0000 7f24c5c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c0100e10 0x7f24c0198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60054/0 (socket says 172.21.15.117:60054) 2024-04-20T11:52:57.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.390+0000 7f24c5c00700 1 -- 172.21.15.117:0/126493452 learned_addr learned my addr 172.21.15.117:0/126493452 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:52:57.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24c5200700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24c01017c0 0x7f24c01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:52:57.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24c5c00700 1 -- 172.21.15.117:0/126493452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24c01017c0 msgr2=0x7f24c01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:52:57.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24c5c00700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24c01017c0 0x7f24c01993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:57.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24c5c00700 1 -- 172.21.15.117:0/126493452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac0095f0 con 0x7f24c0100e10 2024-04-20T11:52:57.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24c5c00700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c0100e10 0x7f24c0198eb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b4b0 tx=0x7f24b400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:52:57.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24be000700 1 -- 172.21.15.117:0/126493452 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24b4011800 con 0x7f24c0100e10 2024-04-20T11:52:57.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24be000700 1 -- 172.21.15.117:0/126493452 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24b4011e20 con 0x7f24c0100e10 2024-04-20T11:52:57.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24be000700 1 -- 172.21.15.117:0/126493452 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24b400f2e0 con 0x7f24c0100e10 2024-04-20T11:52:57.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.391+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c019d9c0 con 0x7f24c0100e10 2024-04-20T11:52:57.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.392+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c019df10 con 0x7f24c0100e10 2024-04-20T11:52:57.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.393+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c01022f0 con 0x7f24c0100e10 2024-04-20T11:52:57.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.394+0000 7f24be000700 1 -- 172.21.15.117:0/126493452 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f24b4010380 con 0x7f24c0100e10 2024-04-20T11:52:57.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.395+0000 7f24be000700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f24a806ecc0 0x7f24a8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:52:57.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.395+0000 7f24be000700 1 -- 172.21.15.117:0/126493452 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f24b408ea00 con 0x7f24c0100e10 2024-04-20T11:52:57.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.395+0000 7f24c5200700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f24a806ecc0 0x7f24a8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:52:57.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.396+0000 7f24c5200700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f24a806ecc0 0x7f24a8071180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f24ac006010 tx=0x7f24ac00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:52:57.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.400+0000 7f24be000700 1 -- 172.21.15.117:0/126493452 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f24b4057870 con 0x7f24c0100e10 2024-04-20T11:52:57.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:57 smithi117 ceph-mon[99036]: pgmap v1217: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:57.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:57 smithi173 ceph-mon[74165]: pgmap v1217: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.716+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f24c010de30 con 0x7f24c0100e10 2024-04-20T11:52:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.717+0000 7f24be000700 1 -- 172.21.15.117:0/126493452 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f24b405af30 con 0x7f24c0100e10 2024-04-20T11:52:57.718 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-20T11:52:57.718 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": { 2024-04-20T11:52:57.718 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:52:57.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:52:57.718 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": { 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 2 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": { 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 6 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 4 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: "overall": { 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)": 14 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-20T11:52:57.719 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f24a806ecc0 msgr2=0x7f24a8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f24a806ecc0 0x7f24a8071180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f24ac006010 tx=0x7f24ac00b540 comp rx=0 tx=0).stop 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c0100e10 msgr2=0x7f24c0198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c0100e10 0x7f24c0198eb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b4b0 tx=0x7f24b400b870 comp rx=0 tx=0).stop 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 shutdown_connections 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f24a806ecc0 0x7f24a8071180 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c0100e10 0x7f24c0198eb0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 --2- 172.21.15.117:0/126493452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24c01017c0 0x7f24c01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:52:57.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.720+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 >> 172.21.15.117:0/126493452 conn(0x7f24c00fc6c0 msgr2=0x7f24c0074740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:52:57.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.721+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 shutdown_connections 2024-04-20T11:52:57.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:52:57.721+0000 7f24c8600700 1 -- 172.21.15.117:0/126493452 wait complete. 2024-04-20T11:52:58.502 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:58 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/126493452' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:52:58.502 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:58.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-20T11:52:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:58 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/126493452' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:52:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:52:58.983 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:52:59.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:52:59 smithi117 ceph-mon[99036]: pgmap v1218: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:52:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:52:59 smithi173 ceph-mon[74165]: pgmap v1218: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.274+0000 7f6d54a00700 1 -- 172.21.15.117:0/2024415888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50107520 msgr2=0x7f6d50107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.274+0000 7f6d54a00700 1 --2- 172.21.15.117:0/2024415888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50107520 0x7f6d50107900 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009910 tx=0x7f6d40009c20 comp rx=0 tx=0).stop 2024-04-20T11:53:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.275+0000 7f6d54a00700 1 -- 172.21.15.117:0/2024415888 shutdown_connections 2024-04-20T11:53:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.275+0000 7f6d54a00700 1 --2- 172.21.15.117:0/2024415888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d50108e10 0x7f6d50109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.275+0000 7f6d54a00700 1 --2- 172.21.15.117:0/2024415888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50107520 0x7f6d50107900 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.275+0000 7f6d54a00700 1 -- 172.21.15.117:0/2024415888 >> 172.21.15.117:0/2024415888 conn(0x7f6d500768f0 msgr2=0x7f6d50078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.275+0000 7f6d54a00700 1 -- 172.21.15.117:0/2024415888 shutdown_connections 2024-04-20T11:53:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.275+0000 7f6d54a00700 1 -- 172.21.15.117:0/2024415888 wait complete. 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.276+0000 7f6d54a00700 1 Processor -- start 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.276+0000 7f6d54a00700 1 -- start start 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.276+0000 7f6d54a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d50107520 0x7f6d5019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.276+0000 7f6d54a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50108e10 0x7f6d5019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.276+0000 7f6d54a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5019ddd0 con 0x7f6d50108e10 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.276+0000 7f6d54a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d501a1aa0 con 0x7f6d50107520 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.277+0000 7f6d4d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50108e10 0x7f6d5019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.277+0000 7f6d4d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50108e10 0x7f6d5019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60082/0 (socket says 172.21.15.117:60082) 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.277+0000 7f6d4d400700 1 -- 172.21.15.117:0/42406457 learned_addr learned my addr 172.21.15.117:0/42406457 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.277+0000 7f6d4de00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d50107520 0x7f6d5019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.277+0000 7f6d4d400700 1 -- 172.21.15.117:0/42406457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d50107520 msgr2=0x7f6d5019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.277+0000 7f6d4d400700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d50107520 0x7f6d5019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:01.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.277+0000 7f6d4d400700 1 -- 172.21.15.117:0/42406457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d400095f0 con 0x7f6d50108e10 2024-04-20T11:53:01.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.278+0000 7f6d4d400700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50108e10 0x7f6d5019d6f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d8d0 tx=0x7f6d3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:01.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.278+0000 7f6d46a00700 1 -- 172.21.15.117:0/42406457 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d3c009940 con 0x7f6d50108e10 2024-04-20T11:53:01.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.278+0000 7f6d46a00700 1 -- 172.21.15.117:0/42406457 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d3c010460 con 0x7f6d50108e10 2024-04-20T11:53:01.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.278+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d501a1d80 con 0x7f6d50108e10 2024-04-20T11:53:01.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.278+0000 7f6d46a00700 1 -- 172.21.15.117:0/42406457 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d3c00f5f0 con 0x7f6d50108e10 2024-04-20T11:53:01.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.278+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d501a2350 con 0x7f6d50108e10 2024-04-20T11:53:01.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.279+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5010f870 con 0x7f6d50108e10 2024-04-20T11:53:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.281+0000 7f6d46a00700 1 -- 172.21.15.117:0/42406457 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6d3c0105d0 con 0x7f6d50108e10 2024-04-20T11:53:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.282+0000 7f6d46a00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d3406ecd0 0x7f6d34071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.282+0000 7f6d46a00700 1 -- 172.21.15.117:0/42406457 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6d3c08eef0 con 0x7f6d50108e10 2024-04-20T11:53:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.282+0000 7f6d4de00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d3406ecd0 0x7f6d34071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.283+0000 7f6d4de00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d3406ecd0 0x7f6d34071190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d4001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:01.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.287+0000 7f6d46a00700 1 -- 172.21.15.117:0/42406457 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6d3c057c30 con 0x7f6d50108e10 2024-04-20T11:53:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.608+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d5019e4c0 con 0x7f6d50108e10 2024-04-20T11:53:01.609 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:01 smithi117 ceph-mon[99036]: pgmap v1219: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:01.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.609+0000 7f6d46a00700 1 -- 172.21.15.117:0/42406457 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6d3c05b2f0 con 0x7f6d50108e10 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d3406ecd0 msgr2=0x7f6d34071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d3406ecd0 0x7f6d34071190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d4001a040 comp rx=0 tx=0).stop 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50108e10 msgr2=0x7f6d5019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50108e10 0x7f6d5019d6f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d8d0 tx=0x7f6d3c00dc90 comp rx=0 tx=0).stop 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 shutdown_connections 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6d3406ecd0 0x7f6d34071190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d50107520 0x7f6d5019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 --2- 172.21.15.117:0/42406457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d50108e10 0x7f6d5019d6f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 >> 172.21.15.117:0/42406457 conn(0x7f6d500768f0 msgr2=0x7f6d50077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 shutdown_connections 2024-04-20T11:53:01.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:01.612+0000 7f6d54a00700 1 -- 172.21.15.117:0/42406457 wait complete. 2024-04-20T11:53:01.628 INFO:teuthology.orchestra.run.smithi117.stdout:true 2024-04-20T11:53:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:01 smithi173 ceph-mon[74165]: pgmap v1219: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:02.641 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -e sha1=a8c84f5997950fd7d008204839253291d52e6eda -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-20T11:53:02.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:02 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/42406457' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:53:02.718 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:02 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/42406457' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:53:03.068 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:53:03.478 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:03 smithi117 ceph-mon[99036]: pgmap v1220: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:03 smithi173 ceph-mon[74165]: pgmap v1220: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:05.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.435+0000 7efe8da00700 1 -- 172.21.15.117:0/632656600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe880ffa80 msgr2=0x7efe880fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:05.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.435+0000 7efe8da00700 1 --2- 172.21.15.117:0/632656600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe880ffa80 0x7efe880fff00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7efe74009910 tx=0x7efe74009c20 comp rx=0 tx=0).stop 2024-04-20T11:53:05.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.435+0000 7efe8da00700 1 -- 172.21.15.117:0/632656600 shutdown_connections 2024-04-20T11:53:05.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.435+0000 7efe8da00700 1 --2- 172.21.15.117:0/632656600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe880ffa80 0x7efe880fff00 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:05.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.435+0000 7efe8da00700 1 --2- 172.21.15.117:0/632656600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe88105a80 0x7efe88105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:05.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.435+0000 7efe8da00700 1 -- 172.21.15.117:0/632656600 >> 172.21.15.117:0/632656600 conn(0x7efe88076750 msgr2=0x7efe880fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:05.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.435+0000 7efe8da00700 1 -- 172.21.15.117:0/632656600 shutdown_connections 2024-04-20T11:53:05.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.435+0000 7efe8da00700 1 -- 172.21.15.117:0/632656600 wait complete. 2024-04-20T11:53:05.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.436+0000 7efe8da00700 1 Processor -- start 2024-04-20T11:53:05.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.437+0000 7efe8da00700 1 -- start start 2024-04-20T11:53:05.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.437+0000 7efe8da00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe880ffa80 0x7efe88198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:05.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.437+0000 7efe8da00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88105a80 0x7efe88199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:05.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.437+0000 7efe8da00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe881999f0 con 0x7efe88105a80 2024-04-20T11:53:05.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.437+0000 7efe8da00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8819d6c0 con 0x7efe880ffa80 2024-04-20T11:53:05.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88105a80 0x7efe88199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:05.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88105a80 0x7efe88199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48188/0 (socket says 172.21.15.117:48188) 2024-04-20T11:53:05.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86000700 1 -- 172.21.15.117:0/1177212586 learned_addr learned my addr 172.21.15.117:0/1177212586 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:05.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86a00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe880ffa80 0x7efe88198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86000700 1 -- 172.21.15.117:0/1177212586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe880ffa80 msgr2=0x7efe88198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86000700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe880ffa80 0x7efe88198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86000700 1 -- 172.21.15.117:0/1177212586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740095f0 con 0x7efe88105a80 2024-04-20T11:53:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86a00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe880ffa80 0x7efe88198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:53:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.438+0000 7efe86000700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88105a80 0x7efe88199310 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7efe740092e0 tx=0x7efe7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.439+0000 7efe7f400700 1 -- 172.21.15.117:0/1177212586 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7401d070 con 0x7efe88105a80 2024-04-20T11:53:05.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.439+0000 7efe7f400700 1 -- 172.21.15.117:0/1177212586 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe7400f460 con 0x7efe88105a80 2024-04-20T11:53:05.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.439+0000 7efe7f400700 1 -- 172.21.15.117:0/1177212586 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe74017630 con 0x7efe88105a80 2024-04-20T11:53:05.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.439+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8819d940 con 0x7efe88105a80 2024-04-20T11:53:05.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.439+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8819de30 con 0x7efe88105a80 2024-04-20T11:53:05.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.440+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe881011c0 con 0x7efe88105a80 2024-04-20T11:53:05.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.441+0000 7efe7f400700 1 -- 172.21.15.117:0/1177212586 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efe7400f5d0 con 0x7efe88105a80 2024-04-20T11:53:05.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.442+0000 7efe7f400700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe6c06ecc0 0x7efe6c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:05.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.442+0000 7efe7f400700 1 -- 172.21.15.117:0/1177212586 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efe74090340 con 0x7efe88105a80 2024-04-20T11:53:05.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.442+0000 7efe86a00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe6c06ecc0 0x7efe6c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:05.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.443+0000 7efe86a00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe6c06ecc0 0x7efe6c071180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efe780099d0 tx=0x7efe78008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:05.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.447+0000 7efe7f400700 1 -- 172.21.15.117:0/1177212586 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efe74059100 con 0x7efe88105a80 2024-04-20T11:53:05.641 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:05 smithi117 ceph-mon[99036]: pgmap v1221: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.764+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efe8810cda0 con 0x7efe88105a80 2024-04-20T11:53:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.765+0000 7efe7f400700 1 -- 172.21.15.117:0/1177212586 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efe7405c7c0 con 0x7efe88105a80 2024-04-20T11:53:05.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe6c06ecc0 msgr2=0x7efe6c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:05.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe6c06ecc0 0x7efe6c071180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efe780099d0 tx=0x7efe78008040 comp rx=0 tx=0).stop 2024-04-20T11:53:05.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88105a80 msgr2=0x7efe88199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:05.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88105a80 0x7efe88199310 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7efe740092e0 tx=0x7efe7400be20 comp rx=0 tx=0).stop 2024-04-20T11:53:05.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 shutdown_connections 2024-04-20T11:53:05.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe6c06ecc0 0x7efe6c071180 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:05.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe880ffa80 0x7efe88198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:05.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 --2- 172.21.15.117:0/1177212586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88105a80 0x7efe88199310 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:05.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.768+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 >> 172.21.15.117:0/1177212586 conn(0x7efe88076750 msgr2=0x7efe88068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:05.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.769+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 shutdown_connections 2024-04-20T11:53:05.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:05.769+0000 7efe8da00700 1 -- 172.21.15.117:0/1177212586 wait complete. 2024-04-20T11:53:05.786 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)" 2024-04-20T11:53:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:05 smithi173 ceph-mon[74165]: pgmap v1221: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:06.529 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:06 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1177212586' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:53:06.595 DEBUG:teuthology.parallel:result is None 2024-04-20T11:53:06.596 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:53:06.607 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T11:53:06.607 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-20T11:53:06.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:06 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1177212586' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-20T11:53:07.022 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/mon.smithi117/config 2024-04-20T11:53:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:07 smithi173 ceph-mon[74165]: pgmap v1222: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:07.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:07 smithi117 ceph-mon[99036]: pgmap v1222: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:08.756 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:08 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:53:08.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:08 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:53:09.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.272+0000 7fbedce00700 1 -- 172.21.15.117:0/1800341732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 msgr2=0x7fbed81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:09.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.272+0000 7fbedce00700 1 --2- 172.21.15.117:0/1800341732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 0x7fbed81011f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b0a0 tx=0x7fbec800b3b0 comp rx=0 tx=0).stop 2024-04-20T11:53:09.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.272+0000 7fbedce00700 1 -- 172.21.15.117:0/1800341732 shutdown_connections 2024-04-20T11:53:09.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.272+0000 7fbedce00700 1 --2- 172.21.15.117:0/1800341732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed81017c0 0x7fbed8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:09.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.272+0000 7fbedce00700 1 --2- 172.21.15.117:0/1800341732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 0x7fbed81011f0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:09.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.272+0000 7fbedce00700 1 -- 172.21.15.117:0/1800341732 >> 172.21.15.117:0/1800341732 conn(0x7fbed80fc6c0 msgr2=0x7fbed80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.272+0000 7fbedce00700 1 -- 172.21.15.117:0/1800341732 shutdown_connections 2024-04-20T11:53:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.272+0000 7fbedce00700 1 -- 172.21.15.117:0/1800341732 wait complete. 2024-04-20T11:53:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.273+0000 7fbedce00700 1 Processor -- start 2024-04-20T11:53:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.273+0000 7fbedce00700 1 -- start start 2024-04-20T11:53:09.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbedce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 0x7fbed819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbedce00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed81017c0 0x7fbed819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbedce00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed819d4c0 con 0x7fbed8100e10 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbedce00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8196af0 con 0x7fbed81017c0 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbed5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 0x7fbed819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbed5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 0x7fbed819c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48208/0 (socket says 172.21.15.117:48208) 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbed5e00700 1 -- 172.21.15.117:0/4204372516 learned_addr learned my addr 172.21.15.117:0/4204372516 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbed5e00700 1 -- 172.21.15.117:0/4204372516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed81017c0 msgr2=0x7fbed819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbed5e00700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed81017c0 0x7fbed819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.274+0000 7fbed5e00700 1 -- 172.21.15.117:0/4204372516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009d00 con 0x7fbed8100e10 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.275+0000 7fbed5e00700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 0x7fbed819c960 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b070 tx=0x7fbec8007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.275+0000 7fbecf400700 1 -- 172.21.15.117:0/4204372516 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbec800e040 con 0x7fbed8100e10 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.275+0000 7fbecf400700 1 -- 172.21.15.117:0/4204372516 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbec8003de0 con 0x7fbed8100e10 2024-04-20T11:53:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.275+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed8196d70 con 0x7fbed8100e10 2024-04-20T11:53:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.275+0000 7fbecf400700 1 -- 172.21.15.117:0/4204372516 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbec801d6b0 con 0x7fbed8100e10 2024-04-20T11:53:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.275+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed8197260 con 0x7fbed8100e10 2024-04-20T11:53:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.276+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed81022f0 con 0x7fbed8100e10 2024-04-20T11:53:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.278+0000 7fbecf400700 1 -- 172.21.15.117:0/4204372516 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbec8004040 con 0x7fbed8100e10 2024-04-20T11:53:09.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.278+0000 7fbecf400700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbebc06ecc0 0x7fbebc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:09.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.279+0000 7fbecf400700 1 -- 172.21.15.117:0/4204372516 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbec8090670 con 0x7fbed8100e10 2024-04-20T11:53:09.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.279+0000 7fbecd400700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbebc06ecc0 0x7fbebc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:09.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.279+0000 7fbecd400700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbebc06ecc0 0x7fbebc071180 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbed8197c60 tx=0x7fbec0006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:09.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.284+0000 7fbecf400700 1 -- 172.21.15.117:0/4204372516 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbec805cf60 con 0x7fbed8100e10 2024-04-20T11:53:09.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.564+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbed810de50 con 0x7fbed8100e10 2024-04-20T11:53:09.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.565+0000 7fbecf400700 1 -- 172.21.15.117:0/4204372516 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1973 (secure 0 0 0) 0x7fbec80177d0 con 0x7fbed8100e10 2024-04-20T11:53:09.566 INFO:teuthology.orchestra.run.smithi117.stdout:e30 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 30 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-04-20T11:04:33.734620+0000 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-04-20T11:51:04.022338+0000 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-04-20T11:53:09.567 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 60 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 82 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:snap_rstat 1 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 2 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=34436,1=34468} 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-04-20T11:53:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data enabled 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 1 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 34436 members: 34468,34436 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.umapup{0:34436} state up:active seq 475 join_fscid=1 addr [v2:172.21.15.117:6826/2747056266,v1:172.21.15.117:6827/2747056266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi117.miaplu{1:34468} state up:active seq 468 join_fscid=1 addr [v2:172.21.15.117:6828/2314031055,v1:172.21.15.117:6829/2314031055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:09.569 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pybzht{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6824/1527553112,v1:172.21.15.173:6825/1527553112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:53:09.570 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.cephfs.smithi173.pfrgat{-1:44477} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.173:6826/1210494947,v1:172.21.15.173:6827/1210494947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:53:09.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.568+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbebc06ecc0 msgr2=0x7fbebc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:09.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.568+0000 7fbedce00700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbebc06ecc0 0x7fbebc071180 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbed8197c60 tx=0x7fbec0006d20 comp rx=0 tx=0).stop 2024-04-20T11:53:09.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.568+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 msgr2=0x7fbed819c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.568+0000 7fbedce00700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 0x7fbed819c960 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b070 tx=0x7fbec8007bd0 comp rx=0 tx=0).stop 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.569+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 shutdown_connections 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.569+0000 7fbedce00700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbebc06ecc0 0x7fbebc071180 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.569+0000 7fbedce00700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbed8100e10 0x7fbed819c960 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.569+0000 7fbedce00700 1 --2- 172.21.15.117:0/4204372516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed81017c0 0x7fbed819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.569+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 >> 172.21.15.117:0/4204372516 conn(0x7fbed80fc6c0 msgr2=0x7fbed80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.569+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 shutdown_connections 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:09.569+0000 7fbedce00700 1 -- 172.21.15.117:0/4204372516 wait complete. 2024-04-20T11:53:09.571 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-04-20T11:53:09.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:09 smithi173 ceph-mon[74165]: pgmap v1223: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:09.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:53:09.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:53:09.813 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:53:09.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:09 smithi117 ceph-mon[99036]: pgmap v1223: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:09.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:53:09.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:53:09.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:53:10.439 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-20T11:53:10.450 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:53:10.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:10 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4204372516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:53:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.662+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4063728916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 msgr2=0x7fe1dc1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.662+0000 7fe1e3c00640 1 --2- 172.21.15.117:0/4063728916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 0x7fe1dc1028b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009880 tx=0x7fe1d002bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.662+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4063728916 shutdown_connections 2024-04-20T11:53:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.662+0000 7fe1e3c00640 1 --2- 172.21.15.117:0/4063728916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1dc102e80 0x7fe1dc106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.662+0000 7fe1e3c00640 1 --2- 172.21.15.117:0/4063728916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 0x7fe1dc1028b0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.662+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4063728916 >> 172.21.15.117:0/4063728916 conn(0x7fe1dc0fdd10 msgr2=0x7fe1dc100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:10.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.662+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4063728916 shutdown_connections 2024-04-20T11:53:10.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.662+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4063728916 wait complete. 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.663+0000 7fe1e3c00640 1 Processor -- start 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.663+0000 7fe1e3c00640 1 -- start start 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 0x7fe1dc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e3c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1dc1983e0 0x7fe1dc19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc1989e0 con 0x7fe1dc1024d0 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e3c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc198b50 con 0x7fe1dc1983e0 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 0x7fe1dc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 0x7fe1dc197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48220/0 (socket says 172.21.15.117:48220) 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e1200640 1 -- 172.21.15.117:0/4005125359 learned_addr learned my addr 172.21.15.117:0/4005125359 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e1200640 1 -- 172.21.15.117:0/4005125359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1dc1983e0 msgr2=0x7fe1dc19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:10.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e1200640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1dc1983e0 0x7fe1dc19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:10.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.664+0000 7fe1e1200640 1 -- 172.21.15.117:0/4005125359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0009560 con 0x7fe1dc1024d0 2024-04-20T11:53:10.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.665+0000 7fe1e1200640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 0x7fe1dc197ea0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009430 tx=0x7fe1d0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:10.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.665+0000 7fe1d9600640 1 -- 172.21.15.117:0/4005125359 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d00035f0 con 0x7fe1dc1024d0 2024-04-20T11:53:10.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.665+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc19cde0 con 0x7fe1dc1024d0 2024-04-20T11:53:10.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.665+0000 7fe1d9600640 1 -- 172.21.15.117:0/4005125359 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1d0003c10 con 0x7fe1dc1024d0 2024-04-20T11:53:10.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.665+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc19d340 con 0x7fe1dc1024d0 2024-04-20T11:53:10.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.665+0000 7fe1d9600640 1 -- 172.21.15.117:0/4005125359 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d0005630 con 0x7fe1dc1024d0 2024-04-20T11:53:10.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.667+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc04e040 con 0x7fe1dc1024d0 2024-04-20T11:53:10.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.667+0000 7fe1d9600640 1 -- 172.21.15.117:0/4005125359 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe1d0004dd0 con 0x7fe1dc1024d0 2024-04-20T11:53:10.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.668+0000 7fe1d9600640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe1c406ecf0 0x7fe1c40711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:10.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.668+0000 7fe1d9600640 1 -- 172.21.15.117:0/4005125359 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe1d00b1570 con 0x7fe1dc1024d0 2024-04-20T11:53:10.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.668+0000 7fe1dbe00640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe1c406ecf0 0x7fe1c40711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:10.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.669+0000 7fe1dbe00640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe1c406ecf0 0x7fe1c40711b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc005fd0 tx=0x7fe1cc005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:10.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.673+0000 7fe1d9600640 1 -- 172.21.15.117:0/4005125359 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fe1d007a350 con 0x7fe1dc1024d0 2024-04-20T11:53:10.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.875+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1dc19d5b0 con 0x7fe1dc1024d0 2024-04-20T11:53:10.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.878+0000 7fe1d9600640 1 -- 172.21.15.117:0/4005125359 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16635 (secure 0 0 0) 0x7fe1d007da10 con 0x7fe1dc1024d0 2024-04-20T11:53:10.878 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:10.878 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":86,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","created":"2024-04-20T10:55:31.778050+0000","modified":"2024-04-20T11:35:20.567312+0000","last_up_change":"2024-04-20T11:19:01.116321+0000","last_in_change":"2024-04-20T11:02:33.545131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:01:14.283466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-20T11:04:32.294012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-20T11:04:33.229067+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6803","nonce":664509927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6805","nonce":664509927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6809","nonce":664509927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6807","nonce":664509927}]},"public_addr":"172.21.15.117:6803/664509927","cluster_addr":"172.21.15.117:6805/664509927","heartbeat_back_addr":"172.21.15.117:6809/664509927","heartbeat_front_addr":"172.21.15.117:6807/664509927","state":["exists","up"]},{"osd":1,"uuid":"75ea37ae-266a-459b-a32b-e1d91927cbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6811","nonce":2323295100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6813","nonce":2323295100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6817","nonce":2323295100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6815","nonce":2323295100}]},"public_addr":"172.21.15.117:6811/2323295100","cluster_addr":"172.21.15.117:6813/2323295100","heartbeat_back_addr":"172.21.15.117:6817/2323295100","heartbeat_front_addr":"172.21.15.117:6815/2323295100","state":["exists","up"]},{"osd":2,"uuid":"2323b6ba-7d04-4744-b61f-4936e4f419bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6819","nonce":155800786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6821","nonce":155800786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6825","nonce":155800786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6823","nonce":155800786}]},"public_addr":"172.21.15.117:6819/155800786","cluster_addr":"172.21.15.117:6821/155800786","heartbeat_back_addr":"172.21.15.117:6825/155800786","heartbeat_front_addr":"172.21.15.117:6823/155800786","state":["exists","up"]},{"osd":3,"uuid":"b87d3df6-0d27-4e2d-9e40-55895e0a891d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6801","nonce":151782600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6803","nonce":151782600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6807","nonce":151782600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6805","nonce":151782600}]},"public_addr":"172.21.15.173:6801/151782600","cluster_addr":"172.21.15.173:6803/151782600","heartbeat_back_addr":"172.21.15.173:6807/151782600","heartbeat_front_addr":"172.21.15.173:6805/151782600","state":["exists","up"]},{"osd":4,"uuid":"c5ceed42-a923-47e4-919d-98caab4e06ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":36,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6809","nonce":3480065294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6811","nonce":3480065294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6815","nonce":3480065294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6813","nonce":3480065294}]},"public_addr":"172.21.15.173:6809/3480065294","cluster_addr":"172.21.15.173:6811/3480065294","heartbeat_back_addr":"172.21.15.173:6815/3480065294","heartbeat_front_addr":"172.21.15.173:6813/3480065294","state":["exists","up"]},{"osd":5,"uuid":"47b99706-a27f-4f30-9e6d-619ee6a35ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":41,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6817","nonce":3281347854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6819","nonce":3281347854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6823","nonce":3281347854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6821","nonce":3281347854}]},"public_addr":"172.21.15.173:6817/3281347854","cluster_addr":"172.21.15.173:6819/3281347854","heartbeat_back_addr":"172.21.15.173:6823/3281347854","heartbeat_front_addr":"172.21.15.173:6821/3281347854","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:59:57.050973+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:00:33.939553+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:11.618080+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:46.287328+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:20.465163+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:56.550872+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.173:6825/3563933435":"2024-04-21T11:19:07.882600+0000","172.21.15.173:6824/3563933435":"2024-04-21T11:19:07.882600+0000","172.21.15.173:0/1016803704":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/3713452282":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/519807094":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/2304890710":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/4182681741":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/571056469":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/1807203325":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1384022397":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2961758989":"2024-04-21T11:07:41.749629+0000","172.21.15.117:6801/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6800/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.173:0/997010605":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6801/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2230602257":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1375611439":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2096654470":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1089633358":"2024-04-21T10:58:13.447117+0000","172.21.15.173:6829/2992581883":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6800/2612478075":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/638889142":"2024-04-21T11:13:42.885571+0000","172.21.15.173:0/3679484323":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/4088938050":"2024-04-21T10:56:04.789705+0000","172.21.15.173:0/3475739843":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/894992429":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6800/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6826/361271135":"2024-04-21T11:19:07.882570+0000","172.21.15.117:6800/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.117:6801/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.173:6828/2992581883":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6801/2612478075":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/1003331392":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6801/3003776987":"2024-04-21T11:13:42.885571+0000","172.21.15.117:0/3651098074":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6800/3003776987":"2024-04-21T11:13:42.885571+0000","172.21.15.173:0/2377885188":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6827/361271135":"2024-04-21T11:19:07.882570+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:53:10.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe1c406ecf0 msgr2=0x7fe1c40711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:10.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe1c406ecf0 0x7fe1c40711b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc005fd0 tx=0x7fe1cc005850 comp rx=0 tx=0).stop 2024-04-20T11:53:10.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 msgr2=0x7fe1dc197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:10.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 0x7fe1dc197ea0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009430 tx=0x7fe1d0004040 comp rx=0 tx=0).stop 2024-04-20T11:53:10.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 shutdown_connections 2024-04-20T11:53:10.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fe1c406ecf0 0x7fe1c40711b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:10.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1dc1983e0 0x7fe1dc19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:10.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 --2- 172.21.15.117:0/4005125359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1dc1024d0 0x7fe1dc197ea0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:10.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.881+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 >> 172.21.15.117:0/4005125359 conn(0x7fe1dc0fdd10 msgr2=0x7fe1dc100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:10.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.882+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 shutdown_connections 2024-04-20T11:53:10.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:10.882+0000 7fe1e3c00640 1 -- 172.21.15.117:0/4005125359 wait complete. 2024-04-20T11:53:10.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:01:14.283466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-20T11:04:32.294012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-20T11:04:33.229067+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:53:10.901 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:53:10.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:10 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4204372516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:53:11.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.100+0000 7f30cf200640 1 -- 172.21.15.117:0/2601082224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c8100a70 msgr2=0x7f30c8104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.100+0000 7f30cf200640 1 --2- 172.21.15.117:0/2601082224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c8100a70 0x7f30c8104ab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009880 tx=0x7f30b802bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.100+0000 7f30cf200640 1 -- 172.21.15.117:0/2601082224 shutdown_connections 2024-04-20T11:53:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.100+0000 7f30cf200640 1 --2- 172.21.15.117:0/2601082224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c8100a70 0x7f30c8104ab0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.101+0000 7f30cf200640 1 --2- 172.21.15.117:0/2601082224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c81000c0 0x7f30c81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.101+0000 7f30cf200640 1 -- 172.21.15.117:0/2601082224 >> 172.21.15.117:0/2601082224 conn(0x7f30c80fb920 msgr2=0x7f30c80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.101+0000 7f30cf200640 1 -- 172.21.15.117:0/2601082224 shutdown_connections 2024-04-20T11:53:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.101+0000 7f30cf200640 1 -- 172.21.15.117:0/2601082224 wait complete. 2024-04-20T11:53:11.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cf200640 1 Processor -- start 2024-04-20T11:53:11.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cf200640 1 -- start start 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cf200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c81000c0 0x7f30c8197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c81982c0 0x7f30c819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cf200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c81988c0 con 0x7f30c81982c0 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cf200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8198a30 con 0x7f30c81000c0 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cd400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c81982c0 0x7f30c819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cd400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c81982c0 0x7f30c819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48234/0 (socket says 172.21.15.117:48234) 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.102+0000 7f30cd400640 1 -- 172.21.15.117:0/3504814290 learned_addr learned my addr 172.21.15.117:0/3504814290 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.103+0000 7f30cd400640 1 -- 172.21.15.117:0/3504814290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c81000c0 msgr2=0x7f30c8197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.103+0000 7f30cd400640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c81000c0 0x7f30c8197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.103+0000 7f30cd400640 1 -- 172.21.15.117:0/3504814290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8009560 con 0x7f30c81982c0 2024-04-20T11:53:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.103+0000 7f30cd400640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c81982c0 0x7f30c819c720 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036040 tx=0x7f30b802c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:11.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.104+0000 7f30c2a00640 1 -- 172.21.15.117:0/3504814290 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30b8004520 con 0x7f30c81982c0 2024-04-20T11:53:11.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.104+0000 7f30c2a00640 1 -- 172.21.15.117:0/3504814290 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30b8003ab0 con 0x7f30c81982c0 2024-04-20T11:53:11.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.104+0000 7f30c2a00640 1 -- 172.21.15.117:0/3504814290 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30b8034440 con 0x7f30c81982c0 2024-04-20T11:53:11.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.104+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c819ccc0 con 0x7f30c81982c0 2024-04-20T11:53:11.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.104+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c819d1d0 con 0x7f30c81982c0 2024-04-20T11:53:11.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.105+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c804e040 con 0x7f30c81982c0 2024-04-20T11:53:11.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.106+0000 7f30c2a00640 1 -- 172.21.15.117:0/3504814290 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f30b8003c20 con 0x7f30c81982c0 2024-04-20T11:53:11.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.107+0000 7f30c2a00640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f30b006ec30 0x7f30b00710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.107+0000 7f30c2a00640 1 -- 172.21.15.117:0/3504814290 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f30b80b12e0 con 0x7f30c81982c0 2024-04-20T11:53:11.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.107+0000 7f30cde00640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f30b006ec30 0x7f30b00710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:11.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.108+0000 7f30cde00640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f30b006ec30 0x7f30b00710f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f30bc005fd0 tx=0x7f30bc005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:11.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.112+0000 7f30c2a00640 1 -- 172.21.15.117:0/3504814290 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f30b807a020 con 0x7f30c81982c0 2024-04-20T11:53:11.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.315+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30c819d4a0 con 0x7f30c81982c0 2024-04-20T11:53:11.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.315+0000 7f30c2a00640 1 -- 172.21.15.117:0/3504814290 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30b807d6e0 con 0x7f30c81982c0 2024-04-20T11:53:11.315 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-04-20T11:53:11.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f30b006ec30 msgr2=0x7f30b00710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f30b006ec30 0x7f30b00710f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f30bc005fd0 tx=0x7f30bc005850 comp rx=0 tx=0).stop 2024-04-20T11:53:11.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c81982c0 msgr2=0x7f30c819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c81982c0 0x7f30c819c720 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036040 tx=0x7f30b802c7a0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 shutdown_connections 2024-04-20T11:53:11.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f30b006ec30 0x7f30b00710f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c81982c0 0x7f30c819c720 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 --2- 172.21.15.117:0/3504814290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c81000c0 0x7f30c8197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.318+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 >> 172.21.15.117:0/3504814290 conn(0x7f30c80fb920 msgr2=0x7f30c80fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:11.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.319+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 shutdown_connections 2024-04-20T11:53:11.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.319+0000 7f30cf200640 1 -- 172.21.15.117:0/3504814290 wait complete. 2024-04-20T11:53:11.337 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-20T11:53:11.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.548+0000 7f40a9400640 1 -- 172.21.15.117:0/1282037395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 msgr2=0x7f40a4109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.548+0000 7f40a9400640 1 --2- 172.21.15.117:0/1282037395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 0x7f40a4109fd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f4090009880 tx=0x7f409002bd70 comp rx=0 tx=0).stop 2024-04-20T11:53:11.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.548+0000 7f40a9400640 1 -- 172.21.15.117:0/1282037395 shutdown_connections 2024-04-20T11:53:11.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.548+0000 7f40a9400640 1 --2- 172.21.15.117:0/1282037395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 0x7f40a4109fd0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.548+0000 7f40a9400640 1 --2- 172.21.15.117:0/1282037395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a41011d0 0x7f40a41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.548+0000 7f40a9400640 1 -- 172.21.15.117:0/1282037395 >> 172.21.15.117:0/1282037395 conn(0x7f40a40fa9f0 msgr2=0x7f40a40fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:11.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.548+0000 7f40a9400640 1 -- 172.21.15.117:0/1282037395 shutdown_connections 2024-04-20T11:53:11.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.548+0000 7f40a9400640 1 -- 172.21.15.117:0/1282037395 wait complete. 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.549+0000 7f40a9400640 1 Processor -- start 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.549+0000 7f40a9400640 1 -- start start 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.549+0000 7f40a9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a41011d0 0x7f40a419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.549+0000 7f40a9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 0x7f40a419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.549+0000 7f40a9400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4195b40 con 0x7f40a4101af0 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.549+0000 7f40a9400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4195cb0 con 0x7f40a41011d0 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.550+0000 7f40a1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 0x7f40a419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.550+0000 7f40a1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 0x7f40a419c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48254/0 (socket says 172.21.15.117:48254) 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.550+0000 7f40a1e00640 1 -- 172.21.15.117:0/2966585772 learned_addr learned my addr 172.21.15.117:0/2966585772 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.550+0000 7f40a2800640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a41011d0 0x7f40a419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.550+0000 7f40a1e00640 1 -- 172.21.15.117:0/2966585772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a41011d0 msgr2=0x7f40a419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.550+0000 7f40a1e00640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a41011d0 0x7f40a419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.550+0000 7f40a1e00640 1 -- 172.21.15.117:0/2966585772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4090009560 con 0x7f40a4101af0 2024-04-20T11:53:11.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.551+0000 7f40a1e00640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 0x7f40a419c010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f4090036040 tx=0x7f409002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:11.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.551+0000 7f409b400640 1 -- 172.21.15.117:0/2966585772 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40900035f0 con 0x7f40a4101af0 2024-04-20T11:53:11.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.551+0000 7f409b400640 1 -- 172.21.15.117:0/2966585772 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4090003c10 con 0x7f40a4101af0 2024-04-20T11:53:11.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.551+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a4195f30 con 0x7f40a4101af0 2024-04-20T11:53:11.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.551+0000 7f409b400640 1 -- 172.21.15.117:0/2966585772 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4090005600 con 0x7f40a4101af0 2024-04-20T11:53:11.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.551+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a4196490 con 0x7f40a4101af0 2024-04-20T11:53:11.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.553+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a404e040 con 0x7f40a4101af0 2024-04-20T11:53:11.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.554+0000 7f409b400640 1 -- 172.21.15.117:0/2966585772 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4090004dd0 con 0x7f40a4101af0 2024-04-20T11:53:11.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.555+0000 7f409b400640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ec30 0x7f40880710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.555+0000 7f409b400640 1 -- 172.21.15.117:0/2966585772 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f40900b18e0 con 0x7f40a4101af0 2024-04-20T11:53:11.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.555+0000 7f40a2800640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ec30 0x7f40880710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:11.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.556+0000 7f40a2800640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ec30 0x7f40880710f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4094005fd0 tx=0x7f4094005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:11.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.560+0000 7f409b400640 1 -- 172.21.15.117:0/2966585772 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f409007a740 con 0x7f40a4101af0 2024-04-20T11:53:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:11 smithi117 ceph-mon[99036]: pgmap v1224: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:11 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4005125359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:53:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:11 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3504814290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:53:11.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.765+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f40a4107640 con 0x7f40a4101af0 2024-04-20T11:53:11.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.766+0000 7f409b400640 1 -- 172.21.15.117:0/2966585772 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4090043020 con 0x7f40a4101af0 2024-04-20T11:53:11.765 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-04-20T11:53:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.768+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ec30 msgr2=0x7f40880710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ec30 0x7f40880710f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4094005fd0 tx=0x7f4094005850 comp rx=0 tx=0).stop 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 msgr2=0x7f40a419c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 0x7f40a419c010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f4090036040 tx=0x7f409002c7e0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 shutdown_connections 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f408806ec30 0x7f40880710f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4101af0 0x7f40a419c010 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 --2- 172.21.15.117:0/2966585772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a41011d0 0x7f40a419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 >> 172.21.15.117:0/2966585772 conn(0x7f40a40fa9f0 msgr2=0x7f40a4104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 shutdown_connections 2024-04-20T11:53:11.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.769+0000 7f40a9400640 1 -- 172.21.15.117:0/2966585772 wait complete. 2024-04-20T11:53:11.787 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-20T11:53:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:11 smithi173 ceph-mon[74165]: pgmap v1224: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:11 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4005125359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:53:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:11 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3504814290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:53:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.990+0000 7f42a1400640 1 -- 172.21.15.117:0/1652307410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c1022e0 msgr2=0x7f429c1026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.990+0000 7f42a1400640 1 --2- 172.21.15.117:0/1652307410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c1022e0 0x7f429c1026c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f428c009880 tx=0x7f428c02bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.990+0000 7f42a1400640 1 -- 172.21.15.117:0/1652307410 shutdown_connections 2024-04-20T11:53:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.990+0000 7f42a1400640 1 --2- 172.21.15.117:0/1652307410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429c102c90 0x7f429c106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.990+0000 7f42a1400640 1 --2- 172.21.15.117:0/1652307410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c1022e0 0x7f429c1026c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.990+0000 7f42a1400640 1 -- 172.21.15.117:0/1652307410 >> 172.21.15.117:0/1652307410 conn(0x7f429c0fdb20 msgr2=0x7f429c0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.990+0000 7f42a1400640 1 -- 172.21.15.117:0/1652307410 shutdown_connections 2024-04-20T11:53:11.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.990+0000 7f42a1400640 1 -- 172.21.15.117:0/1652307410 wait complete. 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.991+0000 7f42a1400640 1 Processor -- start 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.991+0000 7f42a1400640 1 -- start start 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.991+0000 7f42a1400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429c1022e0 0x7f429c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.991+0000 7f42a1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c198310 0x7f429c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.991+0000 7f42a1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c198910 con 0x7f429c198310 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.991+0000 7f42a1400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c198a80 con 0x7f429c1022e0 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.992+0000 7f429b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c198310 0x7f429c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.992+0000 7f429b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c198310 0x7f429c19c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48274/0 (socket says 172.21.15.117:48274) 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.992+0000 7f429b400640 1 -- 172.21.15.117:0/734949756 learned_addr learned my addr 172.21.15.117:0/734949756 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.992+0000 7f429be00640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429c1022e0 0x7f429c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.992+0000 7f429b400640 1 -- 172.21.15.117:0/734949756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429c1022e0 msgr2=0x7f429c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.992+0000 7f429b400640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429c1022e0 0x7f429c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.992+0000 7f429b400640 1 -- 172.21.15.117:0/734949756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c009560 con 0x7f429c198310 2024-04-20T11:53:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.993+0000 7f429b400640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c198310 0x7f429c19c770 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f428800c920 tx=0x7f428800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.993+0000 7f4298c00640 1 -- 172.21.15.117:0/734949756 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4288007b80 con 0x7f429c198310 2024-04-20T11:53:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.993+0000 7f4298c00640 1 -- 172.21.15.117:0/734949756 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42880054e0 con 0x7f429c198310 2024-04-20T11:53:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.993+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429c19cd70 con 0x7f429c198310 2024-04-20T11:53:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.993+0000 7f4298c00640 1 -- 172.21.15.117:0/734949756 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428800f450 con 0x7f429c198310 2024-04-20T11:53:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.993+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c19d2d0 con 0x7f429c198310 2024-04-20T11:53:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.994+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c19d5b0 con 0x7f429c198310 2024-04-20T11:53:11.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.996+0000 7f4298c00640 1 -- 172.21.15.117:0/734949756 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f42880046c0 con 0x7f429c198310 2024-04-20T11:53:11.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.996+0000 7f4298c00640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f428006ec30 0x7f42800710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:11.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.996+0000 7f4298c00640 1 -- 172.21.15.117:0/734949756 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f428808ed30 con 0x7f429c198310 2024-04-20T11:53:11.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.997+0000 7f429be00640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f428006ec30 0x7f42800710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:11.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:11.997+0000 7f429be00640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f428006ec30 0x7f42800710f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f428c000c00 tx=0x7f428c02cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:12.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.001+0000 7f4298c00640 1 -- 172.21.15.117:0/734949756 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4288057a70 con 0x7f429c198310 2024-04-20T11:53:12.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.205+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f429c19cf00 con 0x7f429c198310 2024-04-20T11:53:12.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.206+0000 7f4298c00640 1 -- 172.21.15.117:0/734949756 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f428801d020 con 0x7f429c198310 2024-04-20T11:53:12.205 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-04-20T11:53:12.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.208+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f428006ec30 msgr2=0x7f42800710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:12.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f428006ec30 0x7f42800710f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f428c000c00 tx=0x7f428c02cf90 comp rx=0 tx=0).stop 2024-04-20T11:53:12.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c198310 msgr2=0x7f429c19c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:12.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c198310 0x7f429c19c770 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f428800c920 tx=0x7f428800cde0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 shutdown_connections 2024-04-20T11:53:12.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f428006ec30 0x7f42800710f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c198310 0x7f429c19c770 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 --2- 172.21.15.117:0/734949756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429c1022e0 0x7f429c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 >> 172.21.15.117:0/734949756 conn(0x7f429c0fdb20 msgr2=0x7f429c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:12.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 shutdown_connections 2024-04-20T11:53:12.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.209+0000 7f42a1400640 1 -- 172.21.15.117:0/734949756 wait complete. 2024-04-20T11:53:12.228 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:53:12.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.433+0000 7f413f200640 1 -- 172.21.15.117:0/1819739851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 msgr2=0x7f41381004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.433+0000 7f413f200640 1 --2- 172.21.15.117:0/1819739851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 0x7f41381004b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f412c009880 tx=0x7f412c02bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.434+0000 7f413f200640 1 -- 172.21.15.117:0/1819739851 shutdown_connections 2024-04-20T11:53:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.434+0000 7f413f200640 1 --2- 172.21.15.117:0/1819739851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4138100a80 0x7f4138104ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.434+0000 7f413f200640 1 --2- 172.21.15.117:0/1819739851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 0x7f41381004b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.434+0000 7f413f200640 1 -- 172.21.15.117:0/1819739851 >> 172.21.15.117:0/1819739851 conn(0x7f41380fb930 msgr2=0x7f41380fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.434+0000 7f413f200640 1 -- 172.21.15.117:0/1819739851 shutdown_connections 2024-04-20T11:53:12.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.434+0000 7f413f200640 1 -- 172.21.15.117:0/1819739851 wait complete. 2024-04-20T11:53:12.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413f200640 1 Processor -- start 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413f200640 1 -- start start 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 0x7f4138197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4138198290 0x7f413819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138198890 con 0x7f41381000d0 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138198a00 con 0x7f4138198290 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 0x7f4138197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 0x7f4138197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48294/0 (socket says 172.21.15.117:48294) 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.435+0000 7f413de00640 1 -- 172.21.15.117:0/4111971882 learned_addr learned my addr 172.21.15.117:0/4111971882 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.436+0000 7f413d400640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4138198290 0x7f413819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.436+0000 7f413de00640 1 -- 172.21.15.117:0/4111971882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4138198290 msgr2=0x7f413819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.436+0000 7f413de00640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4138198290 0x7f413819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.436+0000 7f413de00640 1 -- 172.21.15.117:0/4111971882 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c009560 con 0x7f41381000d0 2024-04-20T11:53:12.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.436+0000 7f413d400640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4138198290 0x7f413819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:53:12.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.436+0000 7f413de00640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 0x7f4138197d50 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f412c009430 tx=0x7f412c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:12.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.436+0000 7f4132a00640 1 -- 172.21.15.117:0/4111971882 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f412c0035f0 con 0x7f41381000d0 2024-04-20T11:53:12.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.437+0000 7f4132a00640 1 -- 172.21.15.117:0/4111971882 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f412c003c10 con 0x7f41381000d0 2024-04-20T11:53:12.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.437+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413819cc90 con 0x7f41381000d0 2024-04-20T11:53:12.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.437+0000 7f4132a00640 1 -- 172.21.15.117:0/4111971882 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f412c005670 con 0x7f41381000d0 2024-04-20T11:53:12.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.437+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413819d1f0 con 0x7f41381000d0 2024-04-20T11:53:12.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.438+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413804e890 con 0x7f41381000d0 2024-04-20T11:53:12.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.439+0000 7f4132a00640 1 -- 172.21.15.117:0/4111971882 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f412c004dd0 con 0x7f41381000d0 2024-04-20T11:53:12.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.440+0000 7f4132a00640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f412006ec90 0x7f4120071150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:12.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.440+0000 7f4132a00640 1 -- 172.21.15.117:0/4111971882 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f412c0b1250 con 0x7f41381000d0 2024-04-20T11:53:12.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.440+0000 7f413d400640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f412006ec90 0x7f4120071150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:12.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.441+0000 7f413d400640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f412006ec90 0x7f4120071150 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4138069420 tx=0x7f4128005e50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:12.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.445+0000 7f4132a00640 1 -- 172.21.15.117:0/4111971882 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f412c07a030 con 0x7f41381000d0 2024-04-20T11:53:12.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:12 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2966585772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-20T11:53:12.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:12 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/734949756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-20T11:53:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:12 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2966585772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-20T11:53:12.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:12 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/734949756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-20T11:53:12.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.688+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f413819ce20 con 0x7f41381000d0 2024-04-20T11:53:12.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.690+0000 7f4132a00640 1 -- 172.21.15.117:0/4111971882 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5262 (secure 0 0 0) 0x7f413819ce20 con 0x7f41381000d0 2024-04-20T11:53:12.690 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:12.690 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:51:04.022338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi117.umapup","rank":0,"incarnation":24,"state":"up:active","state_seq":475,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":1,"incarnation":24,"state":"up:active","state_seq":468,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34436,"qdb_cluster":[34468,34436]},"id":1}]} 2024-04-20T11:53:12.690 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-04-20T11:53:12.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.692+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f412006ec90 msgr2=0x7f4120071150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.692+0000 7f413f200640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f412006ec90 0x7f4120071150 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4138069420 tx=0x7f4128005e50 comp rx=0 tx=0).stop 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.692+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 msgr2=0x7f4138197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.692+0000 7f413f200640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 0x7f4138197d50 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f412c009430 tx=0x7f412c004040 comp rx=0 tx=0).stop 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.692+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 shutdown_connections 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.692+0000 7f413f200640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f412006ec90 0x7f4120071150 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.693+0000 7f413f200640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4138198290 0x7f413819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.693+0000 7f413f200640 1 --2- 172.21.15.117:0/4111971882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41381000d0 0x7f4138197d50 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.693+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 >> 172.21.15.117:0/4111971882 conn(0x7f41380fb930 msgr2=0x7f41380fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.693+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 shutdown_connections 2024-04-20T11:53:12.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.693+0000 7f413f200640 1 -- 172.21.15.117:0/4111971882 wait complete. 2024-04-20T11:53:12.711 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 12, 'max_mds': 2, 'flags': 18} 2024-04-20T11:53:12.711 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 13 2024-04-20T11:53:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.915+0000 7fa6f8a00640 1 -- 172.21.15.117:0/1266393204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f4102e70 msgr2=0x7fa6f4106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.915+0000 7fa6f8a00640 1 --2- 172.21.15.117:0/1266393204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f4102e70 0x7fa6f4106eb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009880 tx=0x7fa6dc02bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.915+0000 7fa6f8a00640 1 -- 172.21.15.117:0/1266393204 shutdown_connections 2024-04-20T11:53:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.915+0000 7fa6f8a00640 1 --2- 172.21.15.117:0/1266393204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f4102e70 0x7fa6f4106eb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.915+0000 7fa6f8a00640 1 --2- 172.21.15.117:0/1266393204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f41024c0 0x7fa6f41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.915+0000 7fa6f8a00640 1 -- 172.21.15.117:0/1266393204 >> 172.21.15.117:0/1266393204 conn(0x7fa6f40fdd40 msgr2=0x7fa6f4100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.915+0000 7fa6f8a00640 1 -- 172.21.15.117:0/1266393204 shutdown_connections 2024-04-20T11:53:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.915+0000 7fa6f8a00640 1 -- 172.21.15.117:0/1266393204 wait complete. 2024-04-20T11:53:12.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.916+0000 7fa6f8a00640 1 Processor -- start 2024-04-20T11:53:12.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.916+0000 7fa6f8a00640 1 -- start start 2024-04-20T11:53:12.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.916+0000 7fa6f8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f41024c0 0x7fa6f4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:12.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.916+0000 7fa6f8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f41984b0 0x7fa6f419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.916+0000 7fa6f8a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4198ab0 con 0x7fa6f41024c0 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.916+0000 7fa6f8a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4198c20 con 0x7fa6f41984b0 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.917+0000 7fa6f1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f41024c0 0x7fa6f4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.917+0000 7fa6f1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f41024c0 0x7fa6f4197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48302/0 (socket says 172.21.15.117:48302) 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.917+0000 7fa6f1e00640 1 -- 172.21.15.117:0/92889376 learned_addr learned my addr 172.21.15.117:0/92889376 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.917+0000 7fa6f1e00640 1 -- 172.21.15.117:0/92889376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f41984b0 msgr2=0x7fa6f419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.917+0000 7fa6f1e00640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f41984b0 0x7fa6f419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.917+0000 7fa6f1e00640 1 -- 172.21.15.117:0/92889376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc009560 con 0x7fa6f41024c0 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.917+0000 7fa6f1e00640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f41024c0 0x7fa6f4197f70 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c920 tx=0x7fa6e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.918+0000 7fa6eaa00640 1 -- 172.21.15.117:0/92889376 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6e4007b80 con 0x7fa6f41024c0 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.918+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f419cf10 con 0x7fa6f41024c0 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.918+0000 7fa6eaa00640 1 -- 172.21.15.117:0/92889376 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6e40054e0 con 0x7fa6f41024c0 2024-04-20T11:53:12.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.918+0000 7fa6eaa00640 1 -- 172.21.15.117:0/92889376 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6e400f450 con 0x7fa6f41024c0 2024-04-20T11:53:12.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.918+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f419d470 con 0x7fa6f41024c0 2024-04-20T11:53:12.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.919+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f419d750 con 0x7fa6f41024c0 2024-04-20T11:53:12.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.920+0000 7fa6eaa00640 1 -- 172.21.15.117:0/92889376 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa6e4004f60 con 0x7fa6f41024c0 2024-04-20T11:53:12.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.922+0000 7fa6eaa00640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa6d806ec30 0x7fa6d80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:12.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.922+0000 7fa6eaa00640 1 -- 172.21.15.117:0/92889376 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa6e408ed80 con 0x7fa6f41024c0 2024-04-20T11:53:12.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.922+0000 7fa6f1400640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa6d806ec30 0x7fa6d80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:12.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.923+0000 7fa6f1400640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa6d806ec30 0x7fa6d80710f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc036040 tx=0x7fa6dc000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:12.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:12.927+0000 7fa6eaa00640 1 -- 172.21.15.117:0/92889376 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa6e4057ac0 con 0x7fa6f41024c0 2024-04-20T11:53:13.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.184+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 13, "format": "json"} v 0) v1 -- 0x7fa6f4199390 con 0x7fa6f41024c0 2024-04-20T11:53:13.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.185+0000 7fa6eaa00640 1 -- 172.21.15.117:0/92889376 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 13, "format": "json"}]=0 dumped fsmap epoch 13 v30) v1 ==== 107+0+3311 (secure 0 0 0) 0x7fa6e401dd00 con 0x7fa6f41024c0 2024-04-20T11:53:13.185 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:13.185 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6829/1089389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1089389303},{"type":"v1","addr":"172.21.15.117:6829","nonce":1089389303}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:13.186 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 13 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa6d806ec30 msgr2=0x7fa6d80710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa6d806ec30 0x7fa6d80710f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc036040 tx=0x7fa6dc000bc0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f41024c0 msgr2=0x7fa6f4197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f41024c0 0x7fa6f4197f70 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c920 tx=0x7fa6e400cde0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 shutdown_connections 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa6d806ec30 0x7fa6d80710f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f41984b0 0x7fa6f419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 --2- 172.21.15.117:0/92889376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f41024c0 0x7fa6f4197f70 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 >> 172.21.15.117:0/92889376 conn(0x7fa6f40fdd40 msgr2=0x7fa6f40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 shutdown_connections 2024-04-20T11:53:13.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.188+0000 7fa6f8a00640 1 -- 172.21.15.117:0/92889376 wait complete. 2024-04-20T11:53:13.206 DEBUG:tasks.fs:max_mds not reduced in epoch 13 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:13.206 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-04-20T11:53:13.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.406+0000 7ff77ee00640 1 -- 172.21.15.117:0/691551345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7781024c0 msgr2=0x7ff7781028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:13.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.406+0000 7ff77ee00640 1 --2- 172.21.15.117:0/691551345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7781024c0 0x7ff7781028a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff768009880 tx=0x7ff76802bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.406+0000 7ff77ee00640 1 -- 172.21.15.117:0/691551345 shutdown_connections 2024-04-20T11:53:13.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.406+0000 7ff77ee00640 1 --2- 172.21.15.117:0/691551345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff778102e70 0x7ff778106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.406+0000 7ff77ee00640 1 --2- 172.21.15.117:0/691551345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7781024c0 0x7ff7781028a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.406+0000 7ff77ee00640 1 -- 172.21.15.117:0/691551345 >> 172.21.15.117:0/691551345 conn(0x7ff7780fdd40 msgr2=0x7ff778100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:13.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.406+0000 7ff77ee00640 1 -- 172.21.15.117:0/691551345 shutdown_connections 2024-04-20T11:53:13.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.406+0000 7ff77ee00640 1 -- 172.21.15.117:0/691551345 wait complete. 2024-04-20T11:53:13.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.407+0000 7ff77ee00640 1 Processor -- start 2024-04-20T11:53:13.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.407+0000 7ff77ee00640 1 -- start start 2024-04-20T11:53:13.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.407+0000 7ff77ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7781024c0 0x7ff7781980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:13.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.407+0000 7ff77ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff778198600 0x7ff77819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.407+0000 7ff77ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff778198c00 con 0x7ff778198600 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.407+0000 7ff77ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff778198d70 con 0x7ff7781024c0 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.408+0000 7ff777400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff778198600 0x7ff77819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.408+0000 7ff777400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff778198600 0x7ff77819ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48320/0 (socket says 172.21.15.117:48320) 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.408+0000 7ff777400640 1 -- 172.21.15.117:0/686124565 learned_addr learned my addr 172.21.15.117:0/686124565 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.408+0000 7ff777e00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7781024c0 0x7ff7781980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.408+0000 7ff777400640 1 -- 172.21.15.117:0/686124565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7781024c0 msgr2=0x7ff7781980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.408+0000 7ff777400640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7781024c0 0x7ff7781980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.408+0000 7ff777400640 1 -- 172.21.15.117:0/686124565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768009560 con 0x7ff778198600 2024-04-20T11:53:13.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.408+0000 7ff777400640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff778198600 0x7ff77819ca60 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff76000b400 tx=0x7ff76000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:13.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.409+0000 7ff774c00640 1 -- 172.21.15.117:0/686124565 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff760011800 con 0x7ff778198600 2024-04-20T11:53:13.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.409+0000 7ff774c00640 1 -- 172.21.15.117:0/686124565 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff760011e20 con 0x7ff778198600 2024-04-20T11:53:13.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.409+0000 7ff774c00640 1 -- 172.21.15.117:0/686124565 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff76000f2e0 con 0x7ff778198600 2024-04-20T11:53:13.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.409+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77819d060 con 0x7ff778198600 2024-04-20T11:53:13.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.409+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77819d5c0 con 0x7ff778198600 2024-04-20T11:53:13.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.410+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77819d890 con 0x7ff778198600 2024-04-20T11:53:13.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.411+0000 7ff774c00640 1 -- 172.21.15.117:0/686124565 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff760010380 con 0x7ff778198600 2024-04-20T11:53:13.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.413+0000 7ff774c00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff75c06ec30 0x7ff75c0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:13.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.413+0000 7ff774c00640 1 -- 172.21.15.117:0/686124565 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff76008ea80 con 0x7ff778198600 2024-04-20T11:53:13.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.413+0000 7ff777e00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff75c06ec30 0x7ff75c0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:13.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.414+0000 7ff777e00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff75c06ec30 0x7ff75c0710f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff768036040 tx=0x7ff76802cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:13.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.418+0000 7ff774c00640 1 -- 172.21.15.117:0/686124565 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7ff7600578e0 con 0x7ff778198600 2024-04-20T11:53:13.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.657+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7ff7781994b0 con 0x7ff778198600 2024-04-20T11:53:13.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.658+0000 7ff774c00640 1 -- 172.21.15.117:0/686124565 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v30) v1 ==== 107+0+4872 (secure 0 0 0) 0x7ff76005afa0 con 0x7ff778198600 2024-04-20T11:53:13.658 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:13.658 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6829/1089389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1089389303},{"type":"v1","addr":"172.21.15.117:6829","nonce":1089389303}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34432,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/175884674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":175884674},{"type":"v1","addr":"172.21.15.117:6827","nonce":175884674}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":44241,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/2867925301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2867925301},{"type":"v1","addr":"172.21.15.173:6825","nonce":2867925301}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:13.658 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 14 2024-04-20T11:53:13.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff75c06ec30 msgr2=0x7ff75c0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:13.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff75c06ec30 0x7ff75c0710f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff768036040 tx=0x7ff76802cf90 comp rx=0 tx=0).stop 2024-04-20T11:53:13.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff778198600 msgr2=0x7ff77819ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:13.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff778198600 0x7ff77819ca60 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff76000b400 tx=0x7ff76000b8c0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 shutdown_connections 2024-04-20T11:53:13.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7ff75c06ec30 0x7ff75c0710f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff778198600 0x7ff77819ca60 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 --2- 172.21.15.117:0/686124565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7781024c0 0x7ff7781980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.660+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 >> 172.21.15.117:0/686124565 conn(0x7ff7780fdd40 msgr2=0x7ff7780ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:13.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.661+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 shutdown_connections 2024-04-20T11:53:13.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.661+0000 7ff77ee00640 1 -- 172.21.15.117:0/686124565 wait complete. 2024-04-20T11:53:13.678 DEBUG:tasks.fs:max_mds not reduced in epoch 14 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:13.678 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-04-20T11:53:13.747 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:13 smithi117 ceph-mon[99036]: pgmap v1225: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:13.747 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:13 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4111971882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:53:13.747 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:53:13.747 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:13 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/92889376' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-04-20T11:53:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.881+0000 7fc095800640 1 -- 172.21.15.117:0/112692148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901024a0 msgr2=0x7fc090102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.881+0000 7fc095800640 1 --2- 172.21.15.117:0/112692148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901024a0 0x7fc090102880 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc080009c20 tx=0x7fc08002c1b0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.881+0000 7fc095800640 1 -- 172.21.15.117:0/112692148 shutdown_connections 2024-04-20T11:53:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.881+0000 7fc095800640 1 --2- 172.21.15.117:0/112692148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc090102e50 0x7fc090106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.881+0000 7fc095800640 1 --2- 172.21.15.117:0/112692148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901024a0 0x7fc090102880 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.881+0000 7fc095800640 1 -- 172.21.15.117:0/112692148 >> 172.21.15.117:0/112692148 conn(0x7fc0900fdd00 msgr2=0x7fc090100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:13.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.881+0000 7fc095800640 1 -- 172.21.15.117:0/112692148 shutdown_connections 2024-04-20T11:53:13.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.881+0000 7fc095800640 1 -- 172.21.15.117:0/112692148 wait complete. 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.882+0000 7fc095800640 1 Processor -- start 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.882+0000 7fc095800640 1 -- start start 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc095800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0901024a0 0x7fc090197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc095800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901983c0 0x7fc09019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc095800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901989c0 con 0x7fc0901983c0 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc095800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc090198b30 con 0x7fc0901024a0 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc08e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901983c0 0x7fc09019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc08e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901983c0 0x7fc09019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51362/0 (socket says 172.21.15.117:51362) 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc08e000640 1 -- 172.21.15.117:0/2716048069 learned_addr learned my addr 172.21.15.117:0/2716048069 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc08e000640 1 -- 172.21.15.117:0/2716048069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0901024a0 msgr2=0x7fc090197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc08e000640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0901024a0 0x7fc090197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.883+0000 7fc08e000640 1 -- 172.21.15.117:0/2716048069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009900 con 0x7fc0901983c0 2024-04-20T11:53:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.884+0000 7fc08e000640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901983c0 0x7fc09019c820 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c920 tx=0x7fc07800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:13.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.884+0000 7fc087400640 1 -- 172.21.15.117:0/2716048069 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc078007b80 con 0x7fc0901983c0 2024-04-20T11:53:13.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.884+0000 7fc087400640 1 -- 172.21.15.117:0/2716048069 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc078007ce0 con 0x7fc0901983c0 2024-04-20T11:53:13.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.884+0000 7fc087400640 1 -- 172.21.15.117:0/2716048069 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0780187d0 con 0x7fc0901983c0 2024-04-20T11:53:13.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.884+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09019ce20 con 0x7fc0901983c0 2024-04-20T11:53:13.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.884+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09019d330 con 0x7fc0901983c0 2024-04-20T11:53:13.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.885+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09019d660 con 0x7fc0901983c0 2024-04-20T11:53:13.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.886+0000 7fc087400640 1 -- 172.21.15.117:0/2716048069 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc07800f450 con 0x7fc0901983c0 2024-04-20T11:53:13.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.887+0000 7fc087400640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc07406ecf0 0x7fc0740711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:13.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.888+0000 7fc087400640 1 -- 172.21.15.117:0/2716048069 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc07808f0a0 con 0x7fc0901983c0 2024-04-20T11:53:13.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.888+0000 7fc08ea00640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc07406ecf0 0x7fc0740711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:13.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.888+0000 7fc08ea00640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc07406ecf0 0x7fc0740711b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc080009430 tx=0x7fc080009360 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:13.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:13.892+0000 7fc087400640 1 -- 172.21.15.117:0/2716048069 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fc078057de0 con 0x7fc0901983c0 2024-04-20T11:53:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:13 smithi173 ceph-mon[74165]: pgmap v1225: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:13 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4111971882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:53:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:53:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:13 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/92889376' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-04-20T11:53:14.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.140+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7fc09004e040 con 0x7fc0901983c0 2024-04-20T11:53:14.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.141+0000 7fc087400640 1 -- 172.21.15.117:0/2716048069 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v30) v1 ==== 107+0+4093 (secure 0 0 0) 0x7fc07805b4a0 con 0x7fc0901983c0 2024-04-20T11:53:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:14.141 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6829/1089389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1089389303},{"type":"v1","addr":"172.21.15.117:6829","nonce":1089389303}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44241,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/2867925301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2867925301},{"type":"v1","addr":"172.21.15.173:6825","nonce":2867925301}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:14.141 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 15 2024-04-20T11:53:14.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc07406ecf0 msgr2=0x7fc0740711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:14.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc07406ecf0 0x7fc0740711b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc080009430 tx=0x7fc080009360 comp rx=0 tx=0).stop 2024-04-20T11:53:14.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901983c0 msgr2=0x7fc09019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:14.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901983c0 0x7fc09019c820 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c920 tx=0x7fc07800cde0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 shutdown_connections 2024-04-20T11:53:14.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fc07406ecf0 0x7fc0740711b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0901983c0 0x7fc09019c820 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 --2- 172.21.15.117:0/2716048069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0901024a0 0x7fc090197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.144+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 >> 172.21.15.117:0/2716048069 conn(0x7fc0900fdd00 msgr2=0x7fc0900ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:14.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.145+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 shutdown_connections 2024-04-20T11:53:14.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.145+0000 7fc095800640 1 -- 172.21.15.117:0/2716048069 wait complete. 2024-04-20T11:53:14.163 DEBUG:tasks.fs:max_mds not reduced in epoch 15 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:14.163 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-20T11:53:14.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.352+0000 7effd1000640 1 -- 172.21.15.117:0/3959296693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072640 msgr2=0x7effcc072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:14.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.352+0000 7effd1000640 1 --2- 172.21.15.117:0/3959296693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072640 0x7effcc072a20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7effbc009880 tx=0x7effbc02bd70 comp rx=0 tx=0).stop 2024-04-20T11:53:14.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.352+0000 7effd1000640 1 -- 172.21.15.117:0/3959296693 shutdown_connections 2024-04-20T11:53:14.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.352+0000 7effd1000640 1 --2- 172.21.15.117:0/3959296693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc072f60 0x7effcc110920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.352+0000 7effd1000640 1 --2- 172.21.15.117:0/3959296693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072640 0x7effcc072a20 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.352+0000 7effd1000640 1 -- 172.21.15.117:0/3959296693 >> 172.21.15.117:0/3959296693 conn(0x7effcc0fb920 msgr2=0x7effcc0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:14.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.353+0000 7effd1000640 1 -- 172.21.15.117:0/3959296693 shutdown_connections 2024-04-20T11:53:14.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.353+0000 7effd1000640 1 -- 172.21.15.117:0/3959296693 wait complete. 2024-04-20T11:53:14.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.353+0000 7effd1000640 1 Processor -- start 2024-04-20T11:53:14.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.354+0000 7effd1000640 1 -- start start 2024-04-20T11:53:14.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.354+0000 7effd1000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc072640 0x7effcc19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:14.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.354+0000 7effd1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072f60 0x7effcc19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:14.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.354+0000 7effd1000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc19ccb0 con 0x7effcc072f60 2024-04-20T11:53:14.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.354+0000 7effd1000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc1a0a80 con 0x7effcc072640 2024-04-20T11:53:14.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.354+0000 7effcaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072f60 0x7effcc19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:14.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.354+0000 7effcaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072f60 0x7effcc19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51376/0 (socket says 172.21.15.117:51376) 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.354+0000 7effcaa00640 1 -- 172.21.15.117:0/719733399 learned_addr learned my addr 172.21.15.117:0/719733399 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effcb400640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc072640 0x7effcc19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effcaa00640 1 -- 172.21.15.117:0/719733399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc072640 msgr2=0x7effcc19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effcaa00640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc072640 0x7effcc19c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effcaa00640 1 -- 172.21.15.117:0/719733399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc009560 con 0x7effcc072f60 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effcaa00640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072f60 0x7effcc19c620 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7effb400d6f0 tx=0x7effb400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effc3e00640 1 -- 172.21.15.117:0/719733399 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effb4009a30 con 0x7effcc072f60 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc1a0d60 con 0x7effcc072f60 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effc3e00640 1 -- 172.21.15.117:0/719733399 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effb40054e0 con 0x7effcc072f60 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.355+0000 7effc3e00640 1 -- 172.21.15.117:0/719733399 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effb4010460 con 0x7effcc072f60 2024-04-20T11:53:14.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.356+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc1a12c0 con 0x7effcc072f60 2024-04-20T11:53:14.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.357+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effcc04e040 con 0x7effcc072f60 2024-04-20T11:53:14.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.358+0000 7effc3e00640 1 -- 172.21.15.117:0/719733399 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7effb4009b90 con 0x7effcc072f60 2024-04-20T11:53:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.359+0000 7effc3e00640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ec30 0x7effb00710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.359+0000 7effc3e00640 1 -- 172.21.15.117:0/719733399 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7effb408ea80 con 0x7effcc072f60 2024-04-20T11:53:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.359+0000 7effcb400640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ec30 0x7effb00710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.360+0000 7effcb400640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ec30 0x7effb00710f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc005a20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:14.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.364+0000 7effc3e00640 1 -- 172.21.15.117:0/719733399 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7effb40577c0 con 0x7effcc072f60 2024-04-20T11:53:14.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.614+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7effcc10e010 con 0x7effcc072f60 2024-04-20T11:53:14.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.616+0000 7effc3e00640 1 -- 172.21.15.117:0/719733399 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v30) v1 ==== 107+0+4943 (secure 0 0 0) 0x7effb4017090 con 0x7effcc072f60 2024-04-20T11:53:14.616 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:14.616 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.117:6829/1089389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1089389303},{"type":"v1","addr":"172.21.15.117:6829","nonce":1089389303}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34436,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":44241,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/2867925301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2867925301},{"type":"v1","addr":"172.21.15.173:6825","nonce":2867925301}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:14.616 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 16 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.618+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ec30 msgr2=0x7effb00710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.618+0000 7effd1000640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ec30 0x7effb00710f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc005a20 comp rx=0 tx=0).stop 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072f60 msgr2=0x7effcc19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072f60 0x7effcc19c620 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7effb400d6f0 tx=0x7effb400dbb0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 shutdown_connections 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7effb006ec30 0x7effb00710f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effcc072f60 0x7effcc19c620 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 --2- 172.21.15.117:0/719733399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effcc072640 0x7effcc19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 >> 172.21.15.117:0/719733399 conn(0x7effcc0fb920 msgr2=0x7effcc102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:14.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 shutdown_connections 2024-04-20T11:53:14.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.619+0000 7effd1000640 1 -- 172.21.15.117:0/719733399 wait complete. 2024-04-20T11:53:14.638 DEBUG:tasks.fs:max_mds not reduced in epoch 16 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:14.638 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-20T11:53:14.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.829+0000 7f92f2200640 1 -- 172.21.15.117:0/3471638978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec1022c0 msgr2=0x7f92ec1026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:14.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.829+0000 7f92f2200640 1 --2- 172.21.15.117:0/3471638978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec1022c0 0x7f92ec1026a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009880 tx=0x7f92dc02bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.829+0000 7f92f2200640 1 -- 172.21.15.117:0/3471638978 shutdown_connections 2024-04-20T11:53:14.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.829+0000 7f92f2200640 1 --2- 172.21.15.117:0/3471638978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92ec102c70 0x7f92ec106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.829+0000 7f92f2200640 1 --2- 172.21.15.117:0/3471638978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec1022c0 0x7f92ec1026a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.829+0000 7f92f2200640 1 -- 172.21.15.117:0/3471638978 >> 172.21.15.117:0/3471638978 conn(0x7f92ec0fdb20 msgr2=0x7f92ec0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:14.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.829+0000 7f92f2200640 1 -- 172.21.15.117:0/3471638978 shutdown_connections 2024-04-20T11:53:14.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.829+0000 7f92f2200640 1 -- 172.21.15.117:0/3471638978 wait complete. 2024-04-20T11:53:14.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.830+0000 7f92f2200640 1 Processor -- start 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.830+0000 7f92f2200640 1 -- start start 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.830+0000 7f92f2200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92ec1022c0 0x7f92ec197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92f2200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec198310 0x7f92ec19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92f2200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec198910 con 0x7f92ec198310 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92f2200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec198a80 con 0x7f92ec1022c0 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec198310 0x7f92ec19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec198310 0x7f92ec19c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51396/0 (socket says 172.21.15.117:51396) 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92ebe00640 1 -- 172.21.15.117:0/2110748658 learned_addr learned my addr 172.21.15.117:0/2110748658 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92f0e00640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92ec1022c0 0x7f92ec197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92ebe00640 1 -- 172.21.15.117:0/2110748658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92ec1022c0 msgr2=0x7f92ec197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:14.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92ebe00640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92ec1022c0 0x7f92ec197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:14.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.831+0000 7f92ebe00640 1 -- 172.21.15.117:0/2110748658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc009560 con 0x7f92ec198310 2024-04-20T11:53:14.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.832+0000 7f92ebe00640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec198310 0x7f92ec19c770 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f92d800d6f0 tx=0x7f92d800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:14.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.832+0000 7f92e9600640 1 -- 172.21.15.117:0/2110748658 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d80051f0 con 0x7f92ec198310 2024-04-20T11:53:14.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.832+0000 7f92e9600640 1 -- 172.21.15.117:0/2110748658 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92d8009ce0 con 0x7f92ec198310 2024-04-20T11:53:14.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.832+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec19cd70 con 0x7f92ec198310 2024-04-20T11:53:14.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.832+0000 7f92e9600640 1 -- 172.21.15.117:0/2110748658 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d8004e80 con 0x7f92ec198310 2024-04-20T11:53:14.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.832+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec19d2d0 con 0x7f92ec198310 2024-04-20T11:53:14.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.833+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec04e890 con 0x7f92ec198310 2024-04-20T11:53:14.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.835+0000 7f92e9600640 1 -- 172.21.15.117:0/2110748658 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f92d80046c0 con 0x7f92ec198310 2024-04-20T11:53:14.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.835+0000 7f92e9600640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ec80 0x7f92d0071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:14.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.836+0000 7f92e9600640 1 -- 172.21.15.117:0/2110748658 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f92d808e8d0 con 0x7f92ec198310 2024-04-20T11:53:14.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.836+0000 7f92f0e00640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ec80 0x7f92d0071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:14.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.837+0000 7f92f0e00640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ec80 0x7f92d0071140 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f92dc000c00 tx=0x7f92dc02cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:14.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:14.841+0000 7f92e9600640 1 -- 172.21.15.117:0/2110748658 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f92d8057690 con 0x7f92ec198310 2024-04-20T11:53:14.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:14 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/686124565' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-20T11:53:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:14 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2716048069' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-20T11:53:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:14 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/686124565' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-20T11:53:14.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:14 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2716048069' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-20T11:53:15.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.081+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f92ec199190 con 0x7f92ec198310 2024-04-20T11:53:15.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.082+0000 7f92e9600640 1 -- 172.21.15.117:0/2110748658 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v30) v1 ==== 107+0+4161 (secure 0 0 0) 0x7f92d805ad50 con 0x7f92ec198310 2024-04-20T11:53:15.136 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:15.136 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34436,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":44241,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/2867925301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2867925301},{"type":"v1","addr":"172.21.15.173:6825","nonce":2867925301}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 17 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ec80 msgr2=0x7f92d0071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ec80 0x7f92d0071140 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f92dc000c00 tx=0x7f92dc02cf90 comp rx=0 tx=0).stop 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec198310 msgr2=0x7f92ec19c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec198310 0x7f92ec19c770 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f92d800d6f0 tx=0x7f92d800dbb0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 shutdown_connections 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f92d006ec80 0x7f92d0071140 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ec198310 0x7f92ec19c770 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 --2- 172.21.15.117:0/2110748658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92ec1022c0 0x7f92ec197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 >> 172.21.15.117:0/2110748658 conn(0x7f92ec0fdb20 msgr2=0x7f92ec0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:15.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 shutdown_connections 2024-04-20T11:53:15.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.085+0000 7f92f2200640 1 -- 172.21.15.117:0/2110748658 wait complete. 2024-04-20T11:53:15.139 DEBUG:tasks.fs:max_mds not reduced in epoch 17 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:15.139 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-20T11:53:15.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.345+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3083401315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 msgr2=0x7f5c8810ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.345+0000 7f5c8ee00640 1 --2- 172.21.15.117:0/3083401315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 0x7f5c8810ca80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009880 tx=0x7f5c7802bd70 comp rx=0 tx=0).stop 2024-04-20T11:53:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.346+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3083401315 shutdown_connections 2024-04-20T11:53:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.346+0000 7f5c8ee00640 1 --2- 172.21.15.117:0/3083401315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c881066a0 0x7f5c88106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.346+0000 7f5c8ee00640 1 --2- 172.21.15.117:0/3083401315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 0x7f5c8810ca80 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.346+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3083401315 >> 172.21.15.117:0/3083401315 conn(0x7f5c88075e50 msgr2=0x7f5c88078270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.346+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3083401315 shutdown_connections 2024-04-20T11:53:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.346+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3083401315 wait complete. 2024-04-20T11:53:15.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.347+0000 7f5c8ee00640 1 Processor -- start 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.347+0000 7f5c8ee00640 1 -- start start 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.347+0000 7f5c8ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c881066a0 0x7f5c8819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.347+0000 7f5c8ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 0x7f5c8819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.347+0000 7f5c8ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8819cdf0 con 0x7f5c8810c6a0 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.347+0000 7f5c8ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881a0bc0 con 0x7f5c881066a0 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.348+0000 7f5c7f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 0x7f5c8819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.348+0000 7f5c7f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 0x7f5c8819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51414/0 (socket says 172.21.15.117:51414) 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.348+0000 7f5c7f400640 1 -- 172.21.15.117:0/3839543753 learned_addr learned my addr 172.21.15.117:0/3839543753 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.348+0000 7f5c87e00640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c881066a0 0x7f5c8819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.348+0000 7f5c7f400640 1 -- 172.21.15.117:0/3839543753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c881066a0 msgr2=0x7f5c8819c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.348+0000 7f5c7f400640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c881066a0 0x7f5c8819c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.348+0000 7f5c7f400640 1 -- 172.21.15.117:0/3839543753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78009560 con 0x7f5c8810c6a0 2024-04-20T11:53:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.348+0000 7f5c7f400640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 0x7f5c8819c760 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000d8d0 tx=0x7f5c7000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.349+0000 7f5c85600640 1 -- 172.21.15.117:0/3839543753 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7000bea0 con 0x7f5c8810c6a0 2024-04-20T11:53:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.349+0000 7f5c85600640 1 -- 172.21.15.117:0/3839543753 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c70009c00 con 0x7f5c8810c6a0 2024-04-20T11:53:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.349+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c881a0ea0 con 0x7f5c8810c6a0 2024-04-20T11:53:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.349+0000 7f5c85600640 1 -- 172.21.15.117:0/3839543753 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7000f710 con 0x7f5c8810c6a0 2024-04-20T11:53:15.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.349+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c881a1400 con 0x7f5c8810c6a0 2024-04-20T11:53:15.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.350+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8804e040 con 0x7f5c8810c6a0 2024-04-20T11:53:15.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.351+0000 7f5c85600640 1 -- 172.21.15.117:0/3839543753 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5c70010460 con 0x7f5c8810c6a0 2024-04-20T11:53:15.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.352+0000 7f5c85600640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c6c06ed40 0x7f5c6c071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:15.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.352+0000 7f5c85600640 1 -- 172.21.15.117:0/3839543753 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5c7008eb50 con 0x7f5c8810c6a0 2024-04-20T11:53:15.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.352+0000 7f5c87e00640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c6c06ed40 0x7f5c6c071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:15.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.353+0000 7f5c87e00640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c6c06ed40 0x7f5c6c071200 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5c78036040 tx=0x7f5c78005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:15.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.357+0000 7f5c85600640 1 -- 172.21.15.117:0/3839543753 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5c70057910 con 0x7f5c8810c6a0 2024-04-20T11:53:15.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.601+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f5c8819d570 con 0x7f5c8810c6a0 2024-04-20T11:53:15.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.603+0000 7f5c85600640 1 -- 172.21.15.117:0/3839543753 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v30) v1 ==== 107+0+5011 (secure 0 0 0) 0x7f5c7005afd0 con 0x7f5c8810c6a0 2024-04-20T11:53:15.603 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:15.603 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34436,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44241,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/2867925301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2867925301},{"type":"v1","addr":"172.21.15.173:6825","nonce":2867925301}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:15.603 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 18 2024-04-20T11:53:15.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.605+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c6c06ed40 msgr2=0x7f5c6c071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.605+0000 7f5c8ee00640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c6c06ed40 0x7f5c6c071200 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5c78036040 tx=0x7f5c78005fd0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.605+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 msgr2=0x7f5c8819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.605+0000 7f5c8ee00640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 0x7f5c8819c760 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000d8d0 tx=0x7f5c7000dd90 comp rx=0 tx=0).stop 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.606+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 shutdown_connections 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.606+0000 7f5c8ee00640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5c6c06ed40 0x7f5c6c071200 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.606+0000 7f5c8ee00640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c8810c6a0 0x7f5c8819c760 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.606+0000 7f5c8ee00640 1 --2- 172.21.15.117:0/3839543753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c881066a0 0x7f5c8819c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.606+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 >> 172.21.15.117:0/3839543753 conn(0x7f5c88075e50 msgr2=0x7f5c88077820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.606+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 shutdown_connections 2024-04-20T11:53:15.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.606+0000 7f5c8ee00640 1 -- 172.21.15.117:0/3839543753 wait complete. 2024-04-20T11:53:15.624 DEBUG:tasks.fs:max_mds not reduced in epoch 18 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:15.624 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-20T11:53:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.820+0000 7fcd76e00640 1 -- 172.21.15.117:0/1858130344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 msgr2=0x7fcd70102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.820+0000 7fcd76e00640 1 --2- 172.21.15.117:0/1858130344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 0x7fcd70102680 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009880 tx=0x7fcd6402bcf0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.820+0000 7fcd76e00640 1 -- 172.21.15.117:0/1858130344 shutdown_connections 2024-04-20T11:53:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.820+0000 7fcd76e00640 1 --2- 172.21.15.117:0/1858130344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd70102c50 0x7fcd70106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.820+0000 7fcd76e00640 1 --2- 172.21.15.117:0/1858130344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 0x7fcd70102680 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.820+0000 7fcd76e00640 1 -- 172.21.15.117:0/1858130344 >> 172.21.15.117:0/1858130344 conn(0x7fcd700fdb20 msgr2=0x7fcd700fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.820+0000 7fcd76e00640 1 -- 172.21.15.117:0/1858130344 shutdown_connections 2024-04-20T11:53:15.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.820+0000 7fcd76e00640 1 -- 172.21.15.117:0/1858130344 wait complete. 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.821+0000 7fcd76e00640 1 Processor -- start 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.821+0000 7fcd76e00640 1 -- start start 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd76e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 0x7fcd70197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd76e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd701982d0 0x7fcd7019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd76e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701988d0 con 0x7fcd701022a0 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd76e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70198a40 con 0x7fcd701982d0 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd75a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 0x7fcd70197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd75a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 0x7fcd70197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51424/0 (socket says 172.21.15.117:51424) 2024-04-20T11:53:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd75a00640 1 -- 172.21.15.117:0/1508467041 learned_addr learned my addr 172.21.15.117:0/1508467041 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:15.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd75a00640 1 -- 172.21.15.117:0/1508467041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd701982d0 msgr2=0x7fcd7019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:15.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd75a00640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd701982d0 0x7fcd7019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:15.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.822+0000 7fcd75a00640 1 -- 172.21.15.117:0/1508467041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64009560 con 0x7fcd701022a0 2024-04-20T11:53:15.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.823+0000 7fcd75a00640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 0x7fcd70197d90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009430 tx=0x7fcd64004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:15.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.823+0000 7fcd6e000640 1 -- 172.21.15.117:0/1508467041 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd640035f0 con 0x7fcd701022a0 2024-04-20T11:53:15.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.823+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7019cd30 con 0x7fcd701022a0 2024-04-20T11:53:15.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.823+0000 7fcd6e000640 1 -- 172.21.15.117:0/1508467041 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd64003c10 con 0x7fcd701022a0 2024-04-20T11:53:15.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.823+0000 7fcd6e000640 1 -- 172.21.15.117:0/1508467041 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd64034440 con 0x7fcd701022a0 2024-04-20T11:53:15.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.823+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7019d290 con 0x7fcd701022a0 2024-04-20T11:53:15.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.824+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7004e890 con 0x7fcd701022a0 2024-04-20T11:53:15.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.825+0000 7fcd6e000640 1 -- 172.21.15.117:0/1508467041 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fcd64005060 con 0x7fcd701022a0 2024-04-20T11:53:15.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.826+0000 7fcd6e000640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd5806ec80 0x7fcd58071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:15.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.826+0000 7fcd6e000640 1 -- 172.21.15.117:0/1508467041 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fcd640b1520 con 0x7fcd701022a0 2024-04-20T11:53:15.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.827+0000 7fcd75000640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd5806ec80 0x7fcd58071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:15.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.827+0000 7fcd75000640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd5806ec80 0x7fcd58071140 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcd70074430 tx=0x7fcd5c005bf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:15.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:15.830+0000 7fcd6e000640 1 -- 172.21.15.117:0/1508467041 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fcd6407a420 con 0x7fcd701022a0 2024-04-20T11:53:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:15 smithi117 ceph-mon[99036]: pgmap v1226: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:15 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/719733399' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-20T11:53:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:15 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2110748658' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-20T11:53:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:15 smithi173 ceph-mon[74165]: pgmap v1226: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:15 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/719733399' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-20T11:53:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:15 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2110748658' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-20T11:53:16.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.060+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fcd70199140 con 0x7fcd701022a0 2024-04-20T11:53:16.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.061+0000 7fcd6e000640 1 -- 172.21.15.117:0/1508467041 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v30) v1 ==== 107+0+4229 (secure 0 0 0) 0x7fcd6407dae0 con 0x7fcd701022a0 2024-04-20T11:53:16.061 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:16.061 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34436,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:16.061 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 19 2024-04-20T11:53:16.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.063+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd5806ec80 msgr2=0x7fcd58071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.063+0000 7fcd76e00640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd5806ec80 0x7fcd58071140 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcd70074430 tx=0x7fcd5c005bf0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.063+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 msgr2=0x7fcd70197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.063+0000 7fcd76e00640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 0x7fcd70197d90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009430 tx=0x7fcd64004040 comp rx=0 tx=0).stop 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.064+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 shutdown_connections 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.064+0000 7fcd76e00640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fcd5806ec80 0x7fcd58071140 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.064+0000 7fcd76e00640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd701982d0 0x7fcd7019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.064+0000 7fcd76e00640 1 --2- 172.21.15.117:0/1508467041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd701022a0 0x7fcd70197d90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.064+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 >> 172.21.15.117:0/1508467041 conn(0x7fcd700fdb20 msgr2=0x7fcd700ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.064+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 shutdown_connections 2024-04-20T11:53:16.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.064+0000 7fcd76e00640 1 -- 172.21.15.117:0/1508467041 wait complete. 2024-04-20T11:53:16.080 DEBUG:tasks.fs:max_mds not reduced in epoch 19 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:16.081 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-20T11:53:16.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.273+0000 7f0ddfe00640 1 -- 172.21.15.117:0/3847987653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 msgr2=0x7f0de0109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.273+0000 7f0ddfe00640 1 --2- 172.21.15.117:0/3847987653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 0x7f0de0109e30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009880 tx=0x7f0dcc02bd80 comp rx=0 tx=0).stop 2024-04-20T11:53:16.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.273+0000 7f0ddfe00640 1 -- 172.21.15.117:0/3847987653 shutdown_connections 2024-04-20T11:53:16.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.273+0000 7f0ddfe00640 1 --2- 172.21.15.117:0/3847987653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 0x7f0de0109e30 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.273+0000 7f0ddfe00640 1 --2- 172.21.15.117:0/3847987653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de0101030 0x7f0de0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.273+0000 7f0ddfe00640 1 -- 172.21.15.117:0/3847987653 >> 172.21.15.117:0/3847987653 conn(0x7f0de00fa850 msgr2=0x7f0de00fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:16.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.273+0000 7f0ddfe00640 1 -- 172.21.15.117:0/3847987653 shutdown_connections 2024-04-20T11:53:16.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.274+0000 7f0ddfe00640 1 -- 172.21.15.117:0/3847987653 wait complete. 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.274+0000 7f0ddfe00640 1 Processor -- start 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.275+0000 7f0ddfe00640 1 -- start start 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.275+0000 7f0ddfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de0101030 0x7f0de0197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.275+0000 7f0ddfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 0x7f0de01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.275+0000 7f0ddfe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de0198950 con 0x7f0de0101950 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.275+0000 7f0ddfe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de019c720 con 0x7f0de0101030 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.275+0000 7f0dde000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 0x7f0de01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.275+0000 7f0dde000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 0x7f0de01982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51438/0 (socket says 172.21.15.117:51438) 2024-04-20T11:53:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.275+0000 7f0dde000640 1 -- 172.21.15.117:0/2845745318 learned_addr learned my addr 172.21.15.117:0/2845745318 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0ddea00640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de0101030 0x7f0de0197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0dde000640 1 -- 172.21.15.117:0/2845745318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de0101030 msgr2=0x7f0de0197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0dde000640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de0101030 0x7f0de0197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0dde000640 1 -- 172.21.15.117:0/2845745318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc009560 con 0x7f0de0101950 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0dde000640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 0x7f0de01982c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc036040 tx=0x7f0dcc02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0dd7400640 1 -- 172.21.15.117:0/2845745318 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dcc0035f0 con 0x7f0de0101950 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0dd7400640 1 -- 172.21.15.117:0/2845745318 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dcc003c10 con 0x7f0de0101950 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de019c9a0 con 0x7f0de0101950 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.276+0000 7f0dd7400640 1 -- 172.21.15.117:0/2845745318 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dcc005690 con 0x7f0de0101950 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.277+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de019cf00 con 0x7f0de0101950 2024-04-20T11:53:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.278+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de004e040 con 0x7f0de0101950 2024-04-20T11:53:16.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.279+0000 7f0dd7400640 1 -- 172.21.15.117:0/2845745318 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0dcc004ed0 con 0x7f0de0101950 2024-04-20T11:53:16.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.280+0000 7f0dd7400640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dc406ebd0 0x7f0dc4071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:16.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.280+0000 7f0dd7400640 1 -- 172.21.15.117:0/2845745318 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0dcc0b1b30 con 0x7f0de0101950 2024-04-20T11:53:16.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.281+0000 7f0ddea00640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dc406ebd0 0x7f0dc4071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:16.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.281+0000 7f0ddea00640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dc406ebd0 0x7f0dc4071090 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00058c0 tx=0x7f0dd0005c60 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:16.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.285+0000 7f0dd7400640 1 -- 172.21.15.117:0/2845745318 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f0dcc07a870 con 0x7f0de0101950 2024-04-20T11:53:16.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.533+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f0de0109900 con 0x7f0de0101950 2024-04-20T11:53:16.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.534+0000 7f0dd7400640 1 -- 172.21.15.117:0/2845745318 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v30) v1 ==== 107+0+5079 (secure 0 0 0) 0x7f0dcc07df30 con 0x7f0de0101950 2024-04-20T11:53:16.534 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:16.535 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6827/46112047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":46112047},{"type":"v1","addr":"172.21.15.173:6827","nonce":46112047}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34436,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:16.535 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 20 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dc406ebd0 msgr2=0x7f0dc4071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dc406ebd0 0x7f0dc4071090 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00058c0 tx=0x7f0dd0005c60 comp rx=0 tx=0).stop 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 msgr2=0x7f0de01982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 0x7f0de01982c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc036040 tx=0x7f0dcc02c8d0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 shutdown_connections 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f0dc406ebd0 0x7f0dc4071090 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0de0101950 0x7f0de01982c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 --2- 172.21.15.117:0/2845745318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de0101030 0x7f0de0197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 >> 172.21.15.117:0/2845745318 conn(0x7f0de00fa850 msgr2=0x7f0de00ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 shutdown_connections 2024-04-20T11:53:16.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.537+0000 7f0ddfe00640 1 -- 172.21.15.117:0/2845745318 wait complete. 2024-04-20T11:53:16.554 DEBUG:tasks.fs:max_mds not reduced in epoch 20 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:16.555 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-20T11:53:16.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.752+0000 7f6bc3200640 1 -- 172.21.15.117:0/2309296195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc41022c0 msgr2=0x7f6bc41026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.752+0000 7f6bc3200640 1 --2- 172.21.15.117:0/2309296195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc41022c0 0x7f6bc41026a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009880 tx=0x7f6bb402bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.752+0000 7f6bc3200640 1 -- 172.21.15.117:0/2309296195 shutdown_connections 2024-04-20T11:53:16.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.752+0000 7f6bc3200640 1 --2- 172.21.15.117:0/2309296195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4102c70 0x7f6bc4106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.752+0000 7f6bc3200640 1 --2- 172.21.15.117:0/2309296195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc41022c0 0x7f6bc41026a0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.752+0000 7f6bc3200640 1 -- 172.21.15.117:0/2309296195 >> 172.21.15.117:0/2309296195 conn(0x7f6bc40fdb20 msgr2=0x7f6bc40fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:16.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.752+0000 7f6bc3200640 1 -- 172.21.15.117:0/2309296195 shutdown_connections 2024-04-20T11:53:16.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.752+0000 7f6bc3200640 1 -- 172.21.15.117:0/2309296195 wait complete. 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.753+0000 7f6bc3200640 1 Processor -- start 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.753+0000 7f6bc3200640 1 -- start start 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.754+0000 7f6bc3200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc41022c0 0x7f6bc4197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.754+0000 7f6bc3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc4198310 0x7f6bc419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.754+0000 7f6bc3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4198910 con 0x7f6bc4198310 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.754+0000 7f6bc3200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4198a80 con 0x7f6bc41022c0 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.754+0000 7f6bc1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc4198310 0x7f6bc419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.754+0000 7f6bc1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc4198310 0x7f6bc419c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51446/0 (socket says 172.21.15.117:51446) 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.754+0000 7f6bc1400640 1 -- 172.21.15.117:0/3250600637 learned_addr learned my addr 172.21.15.117:0/3250600637 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:16.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.754+0000 7f6bc1e00640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc41022c0 0x7f6bc4197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bc1400640 1 -- 172.21.15.117:0/3250600637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc41022c0 msgr2=0x7f6bc4197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bc1400640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc41022c0 0x7f6bc4197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bc1400640 1 -- 172.21.15.117:0/3250600637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4009560 con 0x7f6bc4198310 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bc1e00640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc41022c0 0x7f6bc4197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bc1400640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc4198310 0x7f6bc419c770 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00d6f0 tx=0x7f6bac00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bbaa00640 1 -- 172.21.15.117:0/3250600637 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bac0051f0 con 0x7f6bc4198310 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bbaa00640 1 -- 172.21.15.117:0/3250600637 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bac009ce0 con 0x7f6bc4198310 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc419cd70 con 0x7f6bc4198310 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bbaa00640 1 -- 172.21.15.117:0/3250600637 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bac004e80 con 0x7f6bc4198310 2024-04-20T11:53:16.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.755+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc419d2d0 con 0x7f6bc4198310 2024-04-20T11:53:16.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.756+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc419d5a0 con 0x7f6bc4198310 2024-04-20T11:53:16.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.758+0000 7f6bbaa00640 1 -- 172.21.15.117:0/3250600637 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6bac0046c0 con 0x7f6bc4198310 2024-04-20T11:53:16.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.759+0000 7f6bbaa00640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6ba806ec40 0x7f6ba8071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:16.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.759+0000 7f6bbaa00640 1 -- 172.21.15.117:0/3250600637 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6bac08e970 con 0x7f6bc4198310 2024-04-20T11:53:16.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.759+0000 7f6bc1e00640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6ba806ec40 0x7f6ba8071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:16.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.760+0000 7f6bc1e00640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6ba806ec40 0x7f6ba8071100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4036040 tx=0x7f6bb402cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:16.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.764+0000 7f6bbaa00640 1 -- 172.21.15.117:0/3250600637 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6bac0576b0 con 0x7f6bc4198310 2024-04-20T11:53:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:16 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3839543753' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-20T11:53:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:16 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1508467041' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-20T11:53:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:16 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3839543753' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-20T11:53:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:16 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1508467041' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-20T11:53:16.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.993+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f6bc4199120 con 0x7f6bc4198310 2024-04-20T11:53:16.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.994+0000 7f6bbaa00640 1 -- 172.21.15.117:0/3250600637 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v30) v1 ==== 107+0+4303 (secure 0 0 0) 0x7f6bac05ad70 con 0x7f6bc4198310 2024-04-20T11:53:16.994 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:16.994 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:16.994 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 21 2024-04-20T11:53:16.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6ba806ec40 msgr2=0x7f6ba8071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6ba806ec40 0x7f6ba8071100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4036040 tx=0x7f6bb402cf90 comp rx=0 tx=0).stop 2024-04-20T11:53:16.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc4198310 msgr2=0x7f6bc419c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:16.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc4198310 0x7f6bc419c770 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00d6f0 tx=0x7f6bac00dbb0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 shutdown_connections 2024-04-20T11:53:16.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f6ba806ec40 0x7f6ba8071100 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc4198310 0x7f6bc419c770 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 --2- 172.21.15.117:0/3250600637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc41022c0 0x7f6bc4197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:16.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 >> 172.21.15.117:0/3250600637 conn(0x7f6bc40fdb20 msgr2=0x7f6bc40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:16.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.997+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 shutdown_connections 2024-04-20T11:53:16.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:16.998+0000 7f6bc3200640 1 -- 172.21.15.117:0/3250600637 wait complete. 2024-04-20T11:53:17.015 DEBUG:tasks.fs:max_mds not reduced in epoch 21 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:17.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-20T11:53:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.204+0000 7efeb2a00640 1 -- 172.21.15.117:0/292983743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100c20 msgr2=0x7efeac104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.204+0000 7efeb2a00640 1 --2- 172.21.15.117:0/292983743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100c20 0x7efeac104c60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7efe94009880 tx=0x7efe9402bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.204+0000 7efeb2a00640 1 -- 172.21.15.117:0/292983743 shutdown_connections 2024-04-20T11:53:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.204+0000 7efeb2a00640 1 --2- 172.21.15.117:0/292983743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100c20 0x7efeac104c60 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.204+0000 7efeb2a00640 1 --2- 172.21.15.117:0/292983743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeac100270 0x7efeac100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.204+0000 7efeb2a00640 1 -- 172.21.15.117:0/292983743 >> 172.21.15.117:0/292983743 conn(0x7efeac0fbad0 msgr2=0x7efeac0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.204+0000 7efeb2a00640 1 -- 172.21.15.117:0/292983743 shutdown_connections 2024-04-20T11:53:17.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.204+0000 7efeb2a00640 1 -- 172.21.15.117:0/292983743 wait complete. 2024-04-20T11:53:17.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.205+0000 7efeb2a00640 1 Processor -- start 2024-04-20T11:53:17.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.205+0000 7efeb2a00640 1 -- start start 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.205+0000 7efeb2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100270 0x7efeac19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.205+0000 7efeb2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeac19c020 0x7efeac195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.205+0000 7efeb2a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac19c5f0 con 0x7efeac100270 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.205+0000 7efeb2a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac196090 con 0x7efeac19c020 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efeabe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100270 0x7efeac19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efeabe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100270 0x7efeac19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51468/0 (socket says 172.21.15.117:51468) 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efeabe00640 1 -- 172.21.15.117:0/4233481810 learned_addr learned my addr 172.21.15.117:0/4233481810 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efea3400640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeac19c020 0x7efeac195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efeabe00640 1 -- 172.21.15.117:0/4233481810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeac19c020 msgr2=0x7efeac195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efeabe00640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeac19c020 0x7efeac195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efeabe00640 1 -- 172.21.15.117:0/4233481810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94009560 con 0x7efeac100270 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efea3400640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeac19c020 0x7efeac195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.206+0000 7efeabe00640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100270 0x7efeac19bae0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00c8f0 tx=0x7efe9c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:17.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.207+0000 7efea9600640 1 -- 172.21.15.117:0/4233481810 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9c007b80 con 0x7efeac100270 2024-04-20T11:53:17.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.207+0000 7efea9600640 1 -- 172.21.15.117:0/4233481810 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe9c0054e0 con 0x7efeac100270 2024-04-20T11:53:17.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.207+0000 7efea9600640 1 -- 172.21.15.117:0/4233481810 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9c00f450 con 0x7efeac100270 2024-04-20T11:53:17.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.207+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac196370 con 0x7efeac100270 2024-04-20T11:53:17.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.207+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac1968d0 con 0x7efeac100270 2024-04-20T11:53:17.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.208+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac04e040 con 0x7efeac100270 2024-04-20T11:53:17.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.209+0000 7efea9600640 1 -- 172.21.15.117:0/4233481810 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efe9c004f60 con 0x7efeac100270 2024-04-20T11:53:17.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.210+0000 7efea9600640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe9006ecf0 0x7efe900711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:17.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.210+0000 7efea9600640 1 -- 172.21.15.117:0/4233481810 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7efe9c08ec60 con 0x7efeac100270 2024-04-20T11:53:17.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.210+0000 7efea3400640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe9006ecf0 0x7efe900711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:17.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.211+0000 7efea3400640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe9006ecf0 0x7efe900711b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe94005ec0 tx=0x7efe94005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:17.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.215+0000 7efea9600640 1 -- 172.21.15.117:0/4233481810 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7efe9c057a20 con 0x7efeac100270 2024-04-20T11:53:17.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.438+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7efeac196ff0 con 0x7efeac100270 2024-04-20T11:53:17.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.440+0000 7efea9600640 1 -- 172.21.15.117:0/4233481810 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v30) v1 ==== 107+0+5153 (secure 0 0 0) 0x7efe9c01d090 con 0x7efeac100270 2024-04-20T11:53:17.439 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:17.439 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:19:07.882610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:17.440 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 22 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe9006ecf0 msgr2=0x7efe900711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe9006ecf0 0x7efe900711b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe94005ec0 tx=0x7efe94005fd0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100270 msgr2=0x7efeac19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100270 0x7efeac19bae0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00c8f0 tx=0x7efe9c00cdb0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 shutdown_connections 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7efe9006ecf0 0x7efe900711b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeac19c020 0x7efeac195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 --2- 172.21.15.117:0/4233481810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeac100270 0x7efeac19bae0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 >> 172.21.15.117:0/4233481810 conn(0x7efeac0fbad0 msgr2=0x7efeac0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:17.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 shutdown_connections 2024-04-20T11:53:17.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.442+0000 7efeb2a00640 1 -- 172.21.15.117:0/4233481810 wait complete. 2024-04-20T11:53:17.460 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-20T11:53:17.460 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-20T11:53:17.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.656+0000 7f750bc00640 1 -- 172.21.15.117:0/1648013968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 msgr2=0x7f750410ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:17.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.656+0000 7f750bc00640 1 --2- 172.21.15.117:0/1648013968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 0x7f750410ca40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009880 tx=0x7f74f802bcf0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.656+0000 7f750bc00640 1 -- 172.21.15.117:0/1648013968 shutdown_connections 2024-04-20T11:53:17.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.656+0000 7f750bc00640 1 --2- 172.21.15.117:0/1648013968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75041066a0 0x7f7504106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.656+0000 7f750bc00640 1 --2- 172.21.15.117:0/1648013968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 0x7f750410ca40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.656+0000 7f750bc00640 1 -- 172.21.15.117:0/1648013968 >> 172.21.15.117:0/1648013968 conn(0x7f7504075e80 msgr2=0x7f75040782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:17.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.656+0000 7f750bc00640 1 -- 172.21.15.117:0/1648013968 shutdown_connections 2024-04-20T11:53:17.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.656+0000 7f750bc00640 1 -- 172.21.15.117:0/1648013968 wait complete. 2024-04-20T11:53:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.657+0000 7f750bc00640 1 Processor -- start 2024-04-20T11:53:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f750bc00640 1 -- start start 2024-04-20T11:53:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f750bc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75041066a0 0x7f750419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f750bc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 0x7f750419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f750bc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750419d090 con 0x7f750410c660 2024-04-20T11:53:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f750bc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75041a0e60 con 0x7f75041066a0 2024-04-20T11:53:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f7503e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 0x7f750419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:17.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f7503e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 0x7f750419ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51480/0 (socket says 172.21.15.117:51480) 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f7503e00640 1 -- 172.21.15.117:0/1586825592 learned_addr learned my addr 172.21.15.117:0/1586825592 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.658+0000 7f7509200640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75041066a0 0x7f750419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f7503e00640 1 -- 172.21.15.117:0/1586825592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75041066a0 msgr2=0x7f750419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f7503e00640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75041066a0 0x7f750419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f7503e00640 1 -- 172.21.15.117:0/1586825592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f8009560 con 0x7f750410c660 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f7503e00640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 0x7f750419ca00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f74f000c8f0 tx=0x7f74f000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f7501600640 1 -- 172.21.15.117:0/1586825592 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74f0007b80 con 0x7f750410c660 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f7501600640 1 -- 172.21.15.117:0/1586825592 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74f00054e0 con 0x7f750410c660 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75041a1140 con 0x7f750410c660 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f7501600640 1 -- 172.21.15.117:0/1586825592 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74f000f450 con 0x7f750410c660 2024-04-20T11:53:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.659+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75041a16a0 con 0x7f750410c660 2024-04-20T11:53:17.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.661+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750404e040 con 0x7f750410c660 2024-04-20T11:53:17.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.662+0000 7f7501600640 1 -- 172.21.15.117:0/1586825592 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f74f0004f60 con 0x7f750410c660 2024-04-20T11:53:17.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.663+0000 7f7501600640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74ec06ec30 0x7f74ec0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:17.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.663+0000 7f7501600640 1 -- 172.21.15.117:0/1586825592 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f74f008ecf0 con 0x7f750410c660 2024-04-20T11:53:17.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.663+0000 7f7509200640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74ec06ec30 0x7f74ec0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:17.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.664+0000 7f7509200640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74ec06ec30 0x7f74ec0710f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f74f8006010 tx=0x7f74f8000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:17.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.667+0000 7f7501600640 1 -- 172.21.15.117:0/1586825592 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f74f0057a70 con 0x7f750410c660 2024-04-20T11:53:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:17 smithi117 ceph-mon[99036]: pgmap v1227: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:17 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2845745318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-20T11:53:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:17 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3250600637' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-20T11:53:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:17 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4233481810' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-20T11:53:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:17 smithi173 ceph-mon[74165]: pgmap v1227: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:17 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2845745318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-20T11:53:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:17 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3250600637' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-20T11:53:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:17 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4233481810' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-20T11:53:17.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.911+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f750410ebc0 con 0x7f750410c660 2024-04-20T11:53:17.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.913+0000 7f7501600640 1 -- 172.21.15.117:0/1586825592 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v30) v1 ==== 107+0+5152 (secure 0 0 0) 0x7f74f005b130 con 0x7f750410c660 2024-04-20T11:53:17.913 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:17.913 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi117.umapup","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16},{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:50:31.296778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:17.913 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 23 2024-04-20T11:53:17.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.915+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74ec06ec30 msgr2=0x7f74ec0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:17.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.915+0000 7f750bc00640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74ec06ec30 0x7f74ec0710f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f74f8006010 tx=0x7f74f8000bc0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.915+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 msgr2=0x7f750419ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:17.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.915+0000 7f750bc00640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 0x7f750419ca00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f74f000c8f0 tx=0x7f74f000cdb0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.915+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 shutdown_connections 2024-04-20T11:53:17.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.916+0000 7f750bc00640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f74ec06ec30 0x7f74ec0710f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.916+0000 7f750bc00640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750410c660 0x7f750419ca00 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.916+0000 7f750bc00640 1 --2- 172.21.15.117:0/1586825592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75041066a0 0x7f750419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:17.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.916+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 >> 172.21.15.117:0/1586825592 conn(0x7f7504075e80 msgr2=0x7f75040781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:17.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.916+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 shutdown_connections 2024-04-20T11:53:17.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:17.916+0000 7f750bc00640 1 -- 172.21.15.117:0/1586825592 wait complete. 2024-04-20T11:53:17.934 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-20T11:53:18.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.130+0000 7f5d48e00640 1 -- 172.21.15.117:0/363694132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d440ff950 msgr2=0x7f5d44103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:18.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.130+0000 7f5d48e00640 1 --2- 172.21.15.117:0/363694132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d440ff950 0x7f5d44103990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009880 tx=0x7f5d3002bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.130+0000 7f5d48e00640 1 -- 172.21.15.117:0/363694132 shutdown_connections 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.130+0000 7f5d48e00640 1 --2- 172.21.15.117:0/363694132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d440ff950 0x7f5d44103990 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.130+0000 7f5d48e00640 1 --2- 172.21.15.117:0/363694132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d440fefa0 0x7f5d440ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.130+0000 7f5d48e00640 1 -- 172.21.15.117:0/363694132 >> 172.21.15.117:0/363694132 conn(0x7f5d440fa7e0 msgr2=0x7f5d440fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.130+0000 7f5d48e00640 1 -- 172.21.15.117:0/363694132 shutdown_connections 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.131+0000 7f5d48e00640 1 -- 172.21.15.117:0/363694132 wait complete. 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.131+0000 7f5d48e00640 1 Processor -- start 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.132+0000 7f5d48e00640 1 -- start start 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.132+0000 7f5d48e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d440fefa0 0x7f5d4419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.132+0000 7f5d48e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4419bf50 0x7f5d44195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.132+0000 7f5d48e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4419c520 con 0x7f5d4419bf50 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.132+0000 7f5d48e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44195fc0 con 0x7f5d440fefa0 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.132+0000 7f5d42a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4419bf50 0x7f5d44195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:18.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.132+0000 7f5d42a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4419bf50 0x7f5d44195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51494/0 (socket says 172.21.15.117:51494) 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.132+0000 7f5d42a00640 1 -- 172.21.15.117:0/4225031820 learned_addr learned my addr 172.21.15.117:0/4225031820 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d42a00640 1 -- 172.21.15.117:0/4225031820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d440fefa0 msgr2=0x7f5d4419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d42a00640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d440fefa0 0x7f5d4419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d42a00640 1 -- 172.21.15.117:0/4225031820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d30009560 con 0x7f5d4419bf50 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d42a00640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4419bf50 0x7f5d44195a80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009230 tx=0x7f5d30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d3be00640 1 -- 172.21.15.117:0/4225031820 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d300037c0 con 0x7f5d4419bf50 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d3be00640 1 -- 172.21.15.117:0/4225031820 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d30003de0 con 0x7f5d4419bf50 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d44196240 con 0x7f5d4419bf50 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d3be00640 1 -- 172.21.15.117:0/4225031820 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d30034440 con 0x7f5d4419bf50 2024-04-20T11:53:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.133+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d44196740 con 0x7f5d4419bf50 2024-04-20T11:53:18.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.135+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4404e040 con 0x7f5d4419bf50 2024-04-20T11:53:18.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.136+0000 7f5d3be00640 1 -- 172.21.15.117:0/4225031820 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5d30004fb0 con 0x7f5d4419bf50 2024-04-20T11:53:18.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.136+0000 7f5d3be00640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d2806ec30 0x7f5d280710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:18.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.137+0000 7f5d3be00640 1 -- 172.21.15.117:0/4225031820 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5d300b13c0 con 0x7f5d4419bf50 2024-04-20T11:53:18.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.137+0000 7f5d43400640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d2806ec30 0x7f5d280710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:18.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.137+0000 7f5d43400640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d2806ec30 0x7f5d280710f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009710 tx=0x7f5d34006c90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:18.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.141+0000 7f5d3be00640 1 -- 172.21.15.117:0/4225031820 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f5d3007a100 con 0x7f5d4419bf50 2024-04-20T11:53:18.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.370+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f5d44196f90 con 0x7f5d4419bf50 2024-04-20T11:53:18.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.372+0000 7f5d3be00640 1 -- 172.21.15.117:0/4225031820 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v30) v1 ==== 107+0+5243 (secure 0 0 0) 0x7f5d3007d7c0 con 0x7f5d4419bf50 2024-04-20T11:53:18.372 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:18.372 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:50:32.296716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi117.umapup","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":1,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:18.372 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 24 2024-04-20T11:53:18.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.374+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d2806ec30 msgr2=0x7f5d280710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.374+0000 7f5d48e00640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d2806ec30 0x7f5d280710f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009710 tx=0x7f5d34006c90 comp rx=0 tx=0).stop 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.374+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4419bf50 msgr2=0x7f5d44195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.374+0000 7f5d48e00640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4419bf50 0x7f5d44195a80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009230 tx=0x7f5d30004270 comp rx=0 tx=0).stop 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.375+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 shutdown_connections 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.375+0000 7f5d48e00640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f5d2806ec30 0x7f5d280710f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.375+0000 7f5d48e00640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d4419bf50 0x7f5d44195a80 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.375+0000 7f5d48e00640 1 --2- 172.21.15.117:0/4225031820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d440fefa0 0x7f5d4419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.375+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 >> 172.21.15.117:0/4225031820 conn(0x7f5d440fa7e0 msgr2=0x7f5d44103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.375+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 shutdown_connections 2024-04-20T11:53:18.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.375+0000 7f5d48e00640 1 -- 172.21.15.117:0/4225031820 wait complete. 2024-04-20T11:53:18.393 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-20T11:53:18.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.588+0000 7fbf27400640 1 -- 172.21.15.117:0/2128367514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf20102e30 msgr2=0x7fbf20106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:18.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.588+0000 7fbf27400640 1 --2- 172.21.15.117:0/2128367514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf20102e30 0x7fbf20106e70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009880 tx=0x7fbf0802bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.588+0000 7fbf27400640 1 -- 172.21.15.117:0/2128367514 shutdown_connections 2024-04-20T11:53:18.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.588+0000 7fbf27400640 1 --2- 172.21.15.117:0/2128367514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf20102e30 0x7fbf20106e70 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.588+0000 7fbf27400640 1 --2- 172.21.15.117:0/2128367514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf20102480 0x7fbf20102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.588+0000 7fbf27400640 1 -- 172.21.15.117:0/2128367514 >> 172.21.15.117:0/2128367514 conn(0x7fbf200fdd00 msgr2=0x7fbf20100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:18.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.588+0000 7fbf27400640 1 -- 172.21.15.117:0/2128367514 shutdown_connections 2024-04-20T11:53:18.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.589+0000 7fbf27400640 1 -- 172.21.15.117:0/2128367514 wait complete. 2024-04-20T11:53:18.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf27400640 1 Processor -- start 2024-04-20T11:53:18.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf27400640 1 -- start start 2024-04-20T11:53:18.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf27400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf20102480 0x7fbf20197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:18.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf27400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf201983c0 0x7fbf2019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:18.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf27400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201989c0 con 0x7fbf201983c0 2024-04-20T11:53:18.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf27400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20198b30 con 0x7fbf20102480 2024-04-20T11:53:18.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf17e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf201983c0 0x7fbf2019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:18.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf17e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf201983c0 0x7fbf2019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51510/0 (socket says 172.21.15.117:51510) 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.590+0000 7fbf17e00640 1 -- 172.21.15.117:0/4029054356 learned_addr learned my addr 172.21.15.117:0/4029054356 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf17e00640 1 -- 172.21.15.117:0/4029054356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf20102480 msgr2=0x7fbf20197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf17e00640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf20102480 0x7fbf20197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf17e00640 1 -- 172.21.15.117:0/4029054356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf08009560 con 0x7fbf201983c0 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf17e00640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf201983c0 0x7fbf2019c820 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbf08036040 tx=0x7fbf0802c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf1e000640 1 -- 172.21.15.117:0/4029054356 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf08004520 con 0x7fbf201983c0 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf1e000640 1 -- 172.21.15.117:0/4029054356 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf08003ab0 con 0x7fbf201983c0 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2019cdc0 con 0x7fbf201983c0 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf1e000640 1 -- 172.21.15.117:0/4029054356 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf08034440 con 0x7fbf201983c0 2024-04-20T11:53:18.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.591+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2019d320 con 0x7fbf201983c0 2024-04-20T11:53:18.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.593+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2004e040 con 0x7fbf201983c0 2024-04-20T11:53:18.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.593+0000 7fbf1e000640 1 -- 172.21.15.117:0/4029054356 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbf08003c20 con 0x7fbf201983c0 2024-04-20T11:53:18.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.594+0000 7fbf1e000640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf0406ecf0 0x7fbf040711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:18.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.595+0000 7fbf1e000640 1 -- 172.21.15.117:0/4029054356 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbf080b1200 con 0x7fbf201983c0 2024-04-20T11:53:18.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.595+0000 7fbf24a00640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf0406ecf0 0x7fbf040711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:18.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.595+0000 7fbf24a00640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf0406ecf0 0x7fbf040711b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005fd0 tx=0x7fbf10005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:18.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.599+0000 7fbf1e000640 1 -- 172.21.15.117:0/4029054356 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbf08079f40 con 0x7fbf201983c0 2024-04-20T11:53:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:18 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1586825592' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-20T11:53:18.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:18 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4225031820' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-20T11:53:18.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.842+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fbf20102e30 con 0x7fbf201983c0 2024-04-20T11:53:18.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.843+0000 7fbf1e000640 1 -- 172.21.15.117:0/4029054356 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v30) v1 ==== 107+0+5246 (secure 0 0 0) 0x7fbf0807d600 con 0x7fbf201983c0 2024-04-20T11:53:18.843 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:18.843 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:50:32.323715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi117.umapup","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":1,"incarnation":24,"state":"up:resolve","state_seq":463,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:18.843 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 25 2024-04-20T11:53:18.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.846+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf0406ecf0 msgr2=0x7fbf040711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:18.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.846+0000 7fbf27400640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf0406ecf0 0x7fbf040711b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005fd0 tx=0x7fbf10005850 comp rx=0 tx=0).stop 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.846+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf201983c0 msgr2=0x7fbf2019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.846+0000 7fbf27400640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf201983c0 0x7fbf2019c820 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbf08036040 tx=0x7fbf0802c7a0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.847+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 shutdown_connections 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.847+0000 7fbf27400640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbf0406ecf0 0x7fbf040711b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.847+0000 7fbf27400640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf201983c0 0x7fbf2019c820 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.847+0000 7fbf27400640 1 --2- 172.21.15.117:0/4029054356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf20102480 0x7fbf20197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.847+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 >> 172.21.15.117:0/4029054356 conn(0x7fbf200fdd00 msgr2=0x7fbf201000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.847+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 shutdown_connections 2024-04-20T11:53:18.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:18.847+0000 7fbf27400640 1 -- 172.21.15.117:0/4029054356 wait complete. 2024-04-20T11:53:18.864 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-20T11:53:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:18 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1586825592' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-20T11:53:18.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:18 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4225031820' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-20T11:53:19.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.064+0000 7f2d2fe00640 1 -- 172.21.15.117:0/3713785633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d30106490 msgr2=0x7f2d30106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:19.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.064+0000 7f2d2fe00640 1 --2- 172.21.15.117:0/3713785633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d30106490 0x7f2d30106910 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009880 tx=0x7f2d1c02bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.064+0000 7f2d2fe00640 1 -- 172.21.15.117:0/3713785633 shutdown_connections 2024-04-20T11:53:19.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.064+0000 7f2d2fe00640 1 --2- 172.21.15.117:0/3713785633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d30106490 0x7f2d30106910 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.064+0000 7f2d2fe00640 1 --2- 172.21.15.117:0/3713785633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d3010c550 0x7f2d3010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.064+0000 7f2d2fe00640 1 -- 172.21.15.117:0/3713785633 >> 172.21.15.117:0/3713785633 conn(0x7f2d30075c60 msgr2=0x7f2d30078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:19.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.064+0000 7f2d2fe00640 1 -- 172.21.15.117:0/3713785633 shutdown_connections 2024-04-20T11:53:19.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.064+0000 7f2d2fe00640 1 -- 172.21.15.117:0/3713785633 wait complete. 2024-04-20T11:53:19.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.065+0000 7f2d2fe00640 1 Processor -- start 2024-04-20T11:53:19.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.065+0000 7f2d2fe00640 1 -- start start 2024-04-20T11:53:19.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.065+0000 7f2d2fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d30106490 0x7f2d3019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:19.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.065+0000 7f2d2fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d3010c550 0x7f2d3019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:19.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.065+0000 7f2d2fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3019cc90 con 0x7f2d3010c550 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.065+0000 7f2d2fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d301a0a60 con 0x7f2d30106490 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.066+0000 7f2d2e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d3010c550 0x7f2d3019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.066+0000 7f2d2e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d3010c550 0x7f2d3019c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51530/0 (socket says 172.21.15.117:51530) 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.066+0000 7f2d2e000640 1 -- 172.21.15.117:0/2689977725 learned_addr learned my addr 172.21.15.117:0/2689977725 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.066+0000 7f2d2ea00640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d30106490 0x7f2d3019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.066+0000 7f2d2e000640 1 -- 172.21.15.117:0/2689977725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d30106490 msgr2=0x7f2d3019c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.066+0000 7f2d2e000640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d30106490 0x7f2d3019c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.066+0000 7f2d2e000640 1 -- 172.21.15.117:0/2689977725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c009560 con 0x7f2d3010c550 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.066+0000 7f2d2e000640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d3010c550 0x7f2d3019c600 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c006010 tx=0x7f2d1c02c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.067+0000 7f2d27400640 1 -- 172.21.15.117:0/2689977725 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d1c004520 con 0x7f2d3010c550 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.067+0000 7f2d27400640 1 -- 172.21.15.117:0/2689977725 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d1c003ab0 con 0x7f2d3010c550 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.067+0000 7f2d27400640 1 -- 172.21.15.117:0/2689977725 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d1c005690 con 0x7f2d3010c550 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.067+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d301a0ce0 con 0x7f2d3010c550 2024-04-20T11:53:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.067+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d301a11e0 con 0x7f2d3010c550 2024-04-20T11:53:19.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.068+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3004e890 con 0x7f2d3010c550 2024-04-20T11:53:19.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.069+0000 7f2d27400640 1 -- 172.21.15.117:0/2689977725 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2d1c004ed0 con 0x7f2d3010c550 2024-04-20T11:53:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.070+0000 7f2d27400640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2d1406ec40 0x7f2d14071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.070+0000 7f2d27400640 1 -- 172.21.15.117:0/2689977725 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2d1c0b1200 con 0x7f2d3010c550 2024-04-20T11:53:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.070+0000 7f2d2ea00640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2d1406ec40 0x7f2d14071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.071+0000 7f2d2ea00640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2d1406ec40 0x7f2d14071100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005fd0 tx=0x7f2d20005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:19.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.076+0000 7f2d27400640 1 -- 172.21.15.117:0/2689977725 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f2d1c079fe0 con 0x7f2d3010c550 2024-04-20T11:53:19.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.327+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f2d30113950 con 0x7f2d3010c550 2024-04-20T11:53:19.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.328+0000 7f2d27400640 1 -- 172.21.15.117:0/2689977725 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v30) v1 ==== 107+0+5249 (secure 0 0 0) 0x7f2d1c07d6a0 con 0x7f2d3010c550 2024-04-20T11:53:19.328 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:19.328 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:50:41.687341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi117.umapup","rank":0,"incarnation":24,"state":"up:resolve","state_seq":472,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":1,"incarnation":24,"state":"up:resolve","state_seq":463,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:19.328 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 26 2024-04-20T11:53:19.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2d1406ec40 msgr2=0x7f2d14071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:19.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2d1406ec40 0x7f2d14071100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005fd0 tx=0x7f2d20005850 comp rx=0 tx=0).stop 2024-04-20T11:53:19.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d3010c550 msgr2=0x7f2d3019c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:19.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d3010c550 0x7f2d3019c600 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c006010 tx=0x7f2d1c02c830 comp rx=0 tx=0).stop 2024-04-20T11:53:19.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 shutdown_connections 2024-04-20T11:53:19.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f2d1406ec40 0x7f2d14071100 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d3010c550 0x7f2d3019c600 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 --2- 172.21.15.117:0/2689977725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d30106490 0x7f2d3019c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 >> 172.21.15.117:0/2689977725 conn(0x7f2d30075c60 msgr2=0x7f2d300774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:19.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 shutdown_connections 2024-04-20T11:53:19.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.331+0000 7f2d2fe00640 1 -- 172.21.15.117:0/2689977725 wait complete. 2024-04-20T11:53:19.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-20T11:53:19.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.560+0000 7f4081c00640 1 -- 172.21.15.117:0/679972453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 msgr2=0x7f407c072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:19.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.560+0000 7f4081c00640 1 --2- 172.21.15.117:0/679972453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 0x7f407c072a20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f406c009880 tx=0x7f406c02bd70 comp rx=0 tx=0).stop 2024-04-20T11:53:19.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.561+0000 7f4081c00640 1 -- 172.21.15.117:0/679972453 shutdown_connections 2024-04-20T11:53:19.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.561+0000 7f4081c00640 1 --2- 172.21.15.117:0/679972453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f407c072f60 0x7f407c110920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.561+0000 7f4081c00640 1 --2- 172.21.15.117:0/679972453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 0x7f407c072a20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.561+0000 7f4081c00640 1 -- 172.21.15.117:0/679972453 >> 172.21.15.117:0/679972453 conn(0x7f407c0fb920 msgr2=0x7f407c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:19.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.561+0000 7f4081c00640 1 -- 172.21.15.117:0/679972453 shutdown_connections 2024-04-20T11:53:19.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.561+0000 7f4081c00640 1 -- 172.21.15.117:0/679972453 wait complete. 2024-04-20T11:53:19.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.562+0000 7f4081c00640 1 Processor -- start 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.562+0000 7f4081c00640 1 -- start start 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.562+0000 7f4081c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 0x7f407c19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.562+0000 7f4081c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f407c072f60 0x7f407c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.562+0000 7f4081c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c19cd40 con 0x7f407c072640 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.562+0000 7f4081c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c1a0b10 con 0x7f407c072f60 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.562+0000 7f407be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 0x7f407c19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.563+0000 7f407be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 0x7f407c19c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51546/0 (socket says 172.21.15.117:51546) 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.563+0000 7f407be00640 1 -- 172.21.15.117:0/3505475410 learned_addr learned my addr 172.21.15.117:0/3505475410 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.563+0000 7f4073400640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f407c072f60 0x7f407c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.563+0000 7f407be00640 1 -- 172.21.15.117:0/3505475410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f407c072f60 msgr2=0x7f407c19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:19.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.563+0000 7f407be00640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f407c072f60 0x7f407c19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.563+0000 7f407be00640 1 -- 172.21.15.117:0/3505475410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c009560 con 0x7f407c072640 2024-04-20T11:53:19.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.563+0000 7f407be00640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 0x7f407c19c170 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f406c036040 tx=0x7f406c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:19.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.564+0000 7f4079600640 1 -- 172.21.15.117:0/3505475410 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406c0035f0 con 0x7f407c072640 2024-04-20T11:53:19.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.564+0000 7f4079600640 1 -- 172.21.15.117:0/3505475410 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f406c003c10 con 0x7f407c072640 2024-04-20T11:53:19.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.564+0000 7f4079600640 1 -- 172.21.15.117:0/3505475410 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406c005690 con 0x7f407c072640 2024-04-20T11:53:19.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.564+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c1a0d90 con 0x7f407c072640 2024-04-20T11:53:19.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.564+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c1a12f0 con 0x7f407c072640 2024-04-20T11:53:19.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.565+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c04e040 con 0x7f407c072640 2024-04-20T11:53:19.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.566+0000 7f4079600640 1 -- 172.21.15.117:0/3505475410 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f406c004ed0 con 0x7f407c072640 2024-04-20T11:53:19.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.567+0000 7f4079600640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f406006ec30 0x7f40600710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:19.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.567+0000 7f4079600640 1 -- 172.21.15.117:0/3505475410 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f406c0b18f0 con 0x7f407c072640 2024-04-20T11:53:19.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.567+0000 7f4073400640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f406006ec30 0x7f40600710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:19.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.568+0000 7f4073400640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f406006ec30 0x7f40600710f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4064005fd0 tx=0x7f4064005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:19.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.572+0000 7f4079600640 1 -- 172.21.15.117:0/3505475410 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f406c07a750 con 0x7f407c072640 2024-04-20T11:53:19.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:19 smithi117 ceph-mon[99036]: pgmap v1228: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:19 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4029054356' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-20T11:53:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:19 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2689977725' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-20T11:53:19.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.819+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f407c10e010 con 0x7f407c072640 2024-04-20T11:53:19.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.820+0000 7f4079600640 1 -- 172.21.15.117:0/3505475410 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v30) v1 ==== 107+0+5253 (secure 0 0 0) 0x7f406c07de10 con 0x7f407c072640 2024-04-20T11:53:19.820 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:19.821 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:50:42.187753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi117.umapup","rank":0,"incarnation":24,"state":"up:reconnect","state_seq":473,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":1,"incarnation":24,"state":"up:reconnect","state_seq":466,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:19.821 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 27 2024-04-20T11:53:19.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f406006ec30 msgr2=0x7f40600710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:19.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f406006ec30 0x7f40600710f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4064005fd0 tx=0x7f4064005850 comp rx=0 tx=0).stop 2024-04-20T11:53:19.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 msgr2=0x7f407c19c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:19.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 0x7f407c19c170 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f406c036040 tx=0x7f406c004040 comp rx=0 tx=0).stop 2024-04-20T11:53:19.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 shutdown_connections 2024-04-20T11:53:19.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f406006ec30 0x7f40600710f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f407c072f60 0x7f407c19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 --2- 172.21.15.117:0/3505475410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f407c072640 0x7f407c19c170 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:19.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.823+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 >> 172.21.15.117:0/3505475410 conn(0x7f407c0fb920 msgr2=0x7f407c102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:19.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.824+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 shutdown_connections 2024-04-20T11:53:19.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:19.824+0000 7f4081c00640 1 -- 172.21.15.117:0/3505475410 wait complete. 2024-04-20T11:53:19.845 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-20T11:53:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:19 smithi173 ceph-mon[74165]: pgmap v1228: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:19 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4029054356' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-20T11:53:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:19 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2689977725' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-20T11:53:20.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.036+0000 7fa28fe00640 1 -- 172.21.15.117:0/2640338964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 msgr2=0x7fa2901026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:20.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.036+0000 7fa28fe00640 1 --2- 172.21.15.117:0/2640338964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 0x7fa2901026a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa280009c20 tx=0x7fa28002c1b0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.037+0000 7fa28fe00640 1 -- 172.21.15.117:0/2640338964 shutdown_connections 2024-04-20T11:53:20.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.037+0000 7fa28fe00640 1 --2- 172.21.15.117:0/2640338964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa290102c70 0x7fa290106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.037+0000 7fa28fe00640 1 --2- 172.21.15.117:0/2640338964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 0x7fa2901026a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.037+0000 7fa28fe00640 1 -- 172.21.15.117:0/2640338964 >> 172.21.15.117:0/2640338964 conn(0x7fa2900fdb20 msgr2=0x7fa2900fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:20.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.037+0000 7fa28fe00640 1 -- 172.21.15.117:0/2640338964 shutdown_connections 2024-04-20T11:53:20.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.037+0000 7fa28fe00640 1 -- 172.21.15.117:0/2640338964 wait complete. 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.038+0000 7fa28fe00640 1 Processor -- start 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.038+0000 7fa28fe00640 1 -- start start 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.038+0000 7fa28fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 0x7fa290197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.038+0000 7fa28fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa290198310 0x7fa29019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.038+0000 7fa28fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa290198910 con 0x7fa2901022c0 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.038+0000 7fa28fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa290198a80 con 0x7fa290198310 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.039+0000 7fa28ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 0x7fa290197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.039+0000 7fa28ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 0x7fa290197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51564/0 (socket says 172.21.15.117:51564) 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.039+0000 7fa28ea00640 1 -- 172.21.15.117:0/2045732144 learned_addr learned my addr 172.21.15.117:0/2045732144 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.039+0000 7fa286000640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa290198310 0x7fa29019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.039+0000 7fa28ea00640 1 -- 172.21.15.117:0/2045732144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa290198310 msgr2=0x7fa29019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.039+0000 7fa28ea00640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa290198310 0x7fa29019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.039+0000 7fa28ea00640 1 -- 172.21.15.117:0/2045732144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa280009900 con 0x7fa2901022c0 2024-04-20T11:53:20.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.039+0000 7fa28ea00640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 0x7fa290197dd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa280000f80 tx=0x7fa280032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:20.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.040+0000 7fa287e00640 1 -- 172.21.15.117:0/2045732144 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa280032ed0 con 0x7fa2901022c0 2024-04-20T11:53:20.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.040+0000 7fa287e00640 1 -- 172.21.15.117:0/2045732144 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa280003e30 con 0x7fa2901022c0 2024-04-20T11:53:20.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.040+0000 7fa287e00640 1 -- 172.21.15.117:0/2045732144 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa28003f440 con 0x7fa2901022c0 2024-04-20T11:53:20.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.040+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29019cd10 con 0x7fa2901022c0 2024-04-20T11:53:20.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.040+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29019d270 con 0x7fa2901022c0 2024-04-20T11:53:20.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.041+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29004e890 con 0x7fa2901022c0 2024-04-20T11:53:20.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.042+0000 7fa287e00640 1 -- 172.21.15.117:0/2045732144 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa28003b040 con 0x7fa2901022c0 2024-04-20T11:53:20.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.043+0000 7fa287e00640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa27406ec80 0x7fa274071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:20.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.043+0000 7fa287e00640 1 -- 172.21.15.117:0/2045732144 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa2800b1aa0 con 0x7fa2901022c0 2024-04-20T11:53:20.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.043+0000 7fa286000640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa27406ec80 0x7fa274071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:20.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.044+0000 7fa286000640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa27406ec80 0x7fa274071140 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa278005fd0 tx=0x7fa278005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:20.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.048+0000 7fa287e00640 1 -- 172.21.15.117:0/2045732144 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fa28007a860 con 0x7fa2901022c0 2024-04-20T11:53:20.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.295+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fa29010a5c0 con 0x7fa2901022c0 2024-04-20T11:53:20.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.297+0000 7fa287e00640 1 -- 172.21.15.117:0/2045732144 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v30) v1 ==== 107+0+5247 (secure 0 0 0) 0x7fa28007df20 con 0x7fa2901022c0 2024-04-20T11:53:20.296 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:20.296 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:50:43.097166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi117.umapup","rank":0,"incarnation":24,"state":"up:rejoin","state_seq":474,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":1,"incarnation":24,"state":"up:rejoin","state_seq":467,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T11:53:20.297 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 28 2024-04-20T11:53:20.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa27406ec80 msgr2=0x7fa274071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa27406ec80 0x7fa274071140 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa278005fd0 tx=0x7fa278005850 comp rx=0 tx=0).stop 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 msgr2=0x7fa290197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 0x7fa290197dd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa280000f80 tx=0x7fa280032b60 comp rx=0 tx=0).stop 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 shutdown_connections 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fa27406ec80 0x7fa274071140 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa290198310 0x7fa29019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 --2- 172.21.15.117:0/2045732144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2901022c0 0x7fa290197dd0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 >> 172.21.15.117:0/2045732144 conn(0x7fa2900fdb20 msgr2=0x7fa2900ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 shutdown_connections 2024-04-20T11:53:20.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.300+0000 7fa28fe00640 1 -- 172.21.15.117:0/2045732144 wait complete. 2024-04-20T11:53:20.317 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.512+0000 7fbca7200640 1 -- 172.21.15.117:0/1986928916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca8102c70 msgr2=0x7fbca8106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.512+0000 7fbca7200640 1 --2- 172.21.15.117:0/1986928916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca8102c70 0x7fbca8106cb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009880 tx=0x7fbc9402bcf0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.512+0000 7fbca7200640 1 -- 172.21.15.117:0/1986928916 shutdown_connections 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.512+0000 7fbca7200640 1 --2- 172.21.15.117:0/1986928916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca8102c70 0x7fbca8106cb0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.512+0000 7fbca7200640 1 --2- 172.21.15.117:0/1986928916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca81022c0 0x7fbca81026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.512+0000 7fbca7200640 1 -- 172.21.15.117:0/1986928916 >> 172.21.15.117:0/1986928916 conn(0x7fbca80fdb20 msgr2=0x7fbca80fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.512+0000 7fbca7200640 1 -- 172.21.15.117:0/1986928916 shutdown_connections 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.512+0000 7fbca7200640 1 -- 172.21.15.117:0/1986928916 wait complete. 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.513+0000 7fbca7200640 1 Processor -- start 2024-04-20T11:53:20.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.513+0000 7fbca7200640 1 -- start start 2024-04-20T11:53:20.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca81022c0 0x7fbca8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:20.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca8198300 0x7fbca819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:20.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8198900 con 0x7fbca81022c0 2024-04-20T11:53:20.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8198a70 con 0x7fbca8198300 2024-04-20T11:53:20.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca81022c0 0x7fbca8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:20.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca81022c0 0x7fbca8197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51578/0 (socket says 172.21.15.117:51578) 2024-04-20T11:53:20.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca5e00640 1 -- 172.21.15.117:0/4192661633 learned_addr learned my addr 172.21.15.117:0/4192661633 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:20.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca5400640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca8198300 0x7fbca819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca5e00640 1 -- 172.21.15.117:0/4192661633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca8198300 msgr2=0x7fbca819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca5e00640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca8198300 0x7fbca819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.514+0000 7fbca5e00640 1 -- 172.21.15.117:0/4192661633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc94009560 con 0x7fbca81022c0 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.515+0000 7fbca5e00640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca81022c0 0x7fbca8197dc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800d900 tx=0x7fbc9800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.515+0000 7fbc9ea00640 1 -- 172.21.15.117:0/4192661633 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc98009c00 con 0x7fbca81022c0 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.515+0000 7fbc9ea00640 1 -- 172.21.15.117:0/4192661633 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc98009d60 con 0x7fbca81022c0 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.515+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca819cd60 con 0x7fbca81022c0 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.515+0000 7fbc9ea00640 1 -- 172.21.15.117:0/4192661633 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9800f710 con 0x7fbca81022c0 2024-04-20T11:53:20.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.515+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca819d2c0 con 0x7fbca81022c0 2024-04-20T11:53:20.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.516+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca804e040 con 0x7fbca81022c0 2024-04-20T11:53:20.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.517+0000 7fbc9ea00640 1 -- 172.21.15.117:0/4192661633 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fbc98010460 con 0x7fbca81022c0 2024-04-20T11:53:20.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.518+0000 7fbc9ea00640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc8c06ec30 0x7fbc8c0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:20.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.518+0000 7fbc9ea00640 1 -- 172.21.15.117:0/4192661633 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbc9808ece0 con 0x7fbca81022c0 2024-04-20T11:53:20.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.519+0000 7fbca5400640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc8c06ec30 0x7fbc8c0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:20.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.519+0000 7fbca5400640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc8c06ec30 0x7fbc8c0710f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbc9402c8b0 tx=0x7fbc94005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:20.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.523+0000 7fbc9ea00640 1 -- 172.21.15.117:0/4192661633 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fbc98057a20 con 0x7fbca81022c0 2024-04-20T11:53:20.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:20 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3505475410' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-20T11:53:20.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:20 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2045732144' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-20T11:53:20.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.762+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fbca8199110 con 0x7fbca81022c0 2024-04-20T11:53:20.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.763+0000 7fbc9ea00640 1 -- 172.21.15.117:0/4192661633 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v30) v1 ==== 107+0+5262 (secure 0 0 0) 0x7fbc9805b0e0 con 0x7fbca81022c0 2024-04-20T11:53:20.763 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:20.763 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44277,"name":"cephfs.smithi173.pybzht","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6825/1527553112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1527553112},{"type":"v1","addr":"172.21.15.173:6825","nonce":1527553112}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44477,"name":"cephfs.smithi173.pfrgat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6827/1210494947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1210494947},{"type":"v1","addr":"172.21.15.173:6827","nonce":1210494947}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T11:04:33.734620+0000","modified":"2024-04-20T11:50:45.111226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi117.umapup","rank":0,"incarnation":24,"state":"up:active","state_seq":475,"addr":"172.21.15.117:6827/2747056266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2747056266},{"type":"v1","addr":"172.21.15.117:6827","nonce":2747056266}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi117.miaplu","rank":1,"incarnation":24,"state":"up:active","state_seq":468,"addr":"172.21.15.117:6829/2314031055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2314031055},{"type":"v1","addr":"172.21.15.117:6829","nonce":2314031055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34436,"qdb_cluster":[34436,34468]},"id":1}]} 2024-04-20T11:53:20.763 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-04-20T11:53:20.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc8c06ec30 msgr2=0x7fbc8c0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:20.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc8c06ec30 0x7fbc8c0710f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbc9402c8b0 tx=0x7fbc94005fd0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca81022c0 msgr2=0x7fbca8197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:20.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca81022c0 0x7fbca8197dc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800d900 tx=0x7fbc9800ddc0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 shutdown_connections 2024-04-20T11:53:20.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fbc8c06ec30 0x7fbc8c0710f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca8198300 0x7fbca819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 --2- 172.21.15.117:0/4192661633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca81022c0 0x7fbca8197dc0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:20.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 >> 172.21.15.117:0/4192661633 conn(0x7fbca80fdb20 msgr2=0x7fbca80ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:20.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 shutdown_connections 2024-04-20T11:53:20.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:20.765+0000 7fbca7200640 1 -- 172.21.15.117:0/4192661633 wait complete. 2024-04-20T11:53:20.783 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-20T11:53:20.795 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-20T11:53:20.796 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T11:53:20.796 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:53:20.841 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T11:53:20.841 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:53:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:20 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3505475410' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-20T11:53:20.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:20 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2045732144' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-20T11:53:20.936 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:53:21.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.132+0000 7fd765e00640 1 -- 172.21.15.117:0/1901878528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd760102e30 msgr2=0x7fd760106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:21.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.132+0000 7fd765e00640 1 --2- 172.21.15.117:0/1901878528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd760102e30 0x7fd760106e70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd748009880 tx=0x7fd74802bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.132+0000 7fd765e00640 1 -- 172.21.15.117:0/1901878528 shutdown_connections 2024-04-20T11:53:21.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.132+0000 7fd765e00640 1 --2- 172.21.15.117:0/1901878528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd760102e30 0x7fd760106e70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.132+0000 7fd765e00640 1 --2- 172.21.15.117:0/1901878528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd760102480 0x7fd760102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.132+0000 7fd765e00640 1 -- 172.21.15.117:0/1901878528 >> 172.21.15.117:0/1901878528 conn(0x7fd7600fdd00 msgr2=0x7fd760100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:21.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.132+0000 7fd765e00640 1 -- 172.21.15.117:0/1901878528 shutdown_connections 2024-04-20T11:53:21.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.132+0000 7fd765e00640 1 -- 172.21.15.117:0/1901878528 wait complete. 2024-04-20T11:53:21.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.133+0000 7fd765e00640 1 Processor -- start 2024-04-20T11:53:21.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.133+0000 7fd765e00640 1 -- start start 2024-04-20T11:53:21.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.133+0000 7fd765e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd760102480 0x7fd760197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:21.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.133+0000 7fd765e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7601983c0 0x7fd76019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:21.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.133+0000 7fd765e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7601989c0 con 0x7fd7601983c0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.133+0000 7fd765e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd760198b30 con 0x7fd760102480 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.134+0000 7fd756000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7601983c0 0x7fd76019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.134+0000 7fd756000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7601983c0 0x7fd76019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51602/0 (socket says 172.21.15.117:51602) 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.134+0000 7fd756000640 1 -- 172.21.15.117:0/1269540129 learned_addr learned my addr 172.21.15.117:0/1269540129 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.134+0000 7fd75ea00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd760102480 0x7fd760197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.134+0000 7fd756000640 1 -- 172.21.15.117:0/1269540129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd760102480 msgr2=0x7fd760197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.134+0000 7fd756000640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd760102480 0x7fd760197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.134+0000 7fd756000640 1 -- 172.21.15.117:0/1269540129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd748009560 con 0x7fd7601983c0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.134+0000 7fd756000640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7601983c0 0x7fd76019c820 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd748036040 tx=0x7fd74802c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.135+0000 7fd764a00640 1 -- 172.21.15.117:0/1269540129 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd748004520 con 0x7fd7601983c0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.135+0000 7fd764a00640 1 -- 172.21.15.117:0/1269540129 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd748003ab0 con 0x7fd7601983c0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.135+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76019cdc0 con 0x7fd7601983c0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.135+0000 7fd764a00640 1 -- 172.21.15.117:0/1269540129 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd748034440 con 0x7fd7601983c0 2024-04-20T11:53:21.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.135+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76019d320 con 0x7fd7601983c0 2024-04-20T11:53:21.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.136+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76004e040 con 0x7fd7601983c0 2024-04-20T11:53:21.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.137+0000 7fd764a00640 1 -- 172.21.15.117:0/1269540129 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd748003c20 con 0x7fd7601983c0 2024-04-20T11:53:21.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.138+0000 7fd764a00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd74406ecf0 0x7fd7440711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:21.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.138+0000 7fd764a00640 1 -- 172.21.15.117:0/1269540129 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd7480b1240 con 0x7fd7601983c0 2024-04-20T11:53:21.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.138+0000 7fd75ea00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd74406ecf0 0x7fd7440711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:21.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.139+0000 7fd75ea00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd74406ecf0 0x7fd7440711b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd750005c40 tx=0x7fd750005bd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:21.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.144+0000 7fd764a00640 1 -- 172.21.15.117:0/1269540129 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd74807a020 con 0x7fd7601983c0 2024-04-20T11:53:21.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.348+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd760102e30 con 0x7fd7601983c0 2024-04-20T11:53:21.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.351+0000 7fd764a00640 1 -- 172.21.15.117:0/1269540129 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16635 (secure 0 0 0) 0x7fd74807d6e0 con 0x7fd7601983c0 2024-04-20T11:53:21.351 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:21.351 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":86,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","created":"2024-04-20T10:55:31.778050+0000","modified":"2024-04-20T11:35:20.567312+0000","last_up_change":"2024-04-20T11:19:01.116321+0000","last_in_change":"2024-04-20T11:02:33.545131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:01:14.283466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-20T11:04:32.294012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-20T11:04:33.229067+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6803","nonce":664509927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6805","nonce":664509927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6809","nonce":664509927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6807","nonce":664509927}]},"public_addr":"172.21.15.117:6803/664509927","cluster_addr":"172.21.15.117:6805/664509927","heartbeat_back_addr":"172.21.15.117:6809/664509927","heartbeat_front_addr":"172.21.15.117:6807/664509927","state":["exists","up"]},{"osd":1,"uuid":"75ea37ae-266a-459b-a32b-e1d91927cbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6811","nonce":2323295100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6813","nonce":2323295100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6817","nonce":2323295100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6815","nonce":2323295100}]},"public_addr":"172.21.15.117:6811/2323295100","cluster_addr":"172.21.15.117:6813/2323295100","heartbeat_back_addr":"172.21.15.117:6817/2323295100","heartbeat_front_addr":"172.21.15.117:6815/2323295100","state":["exists","up"]},{"osd":2,"uuid":"2323b6ba-7d04-4744-b61f-4936e4f419bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6819","nonce":155800786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6821","nonce":155800786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6825","nonce":155800786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6823","nonce":155800786}]},"public_addr":"172.21.15.117:6819/155800786","cluster_addr":"172.21.15.117:6821/155800786","heartbeat_back_addr":"172.21.15.117:6825/155800786","heartbeat_front_addr":"172.21.15.117:6823/155800786","state":["exists","up"]},{"osd":3,"uuid":"b87d3df6-0d27-4e2d-9e40-55895e0a891d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6801","nonce":151782600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6803","nonce":151782600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6807","nonce":151782600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6805","nonce":151782600}]},"public_addr":"172.21.15.173:6801/151782600","cluster_addr":"172.21.15.173:6803/151782600","heartbeat_back_addr":"172.21.15.173:6807/151782600","heartbeat_front_addr":"172.21.15.173:6805/151782600","state":["exists","up"]},{"osd":4,"uuid":"c5ceed42-a923-47e4-919d-98caab4e06ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":36,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6809","nonce":3480065294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6811","nonce":3480065294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6815","nonce":3480065294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6813","nonce":3480065294}]},"public_addr":"172.21.15.173:6809/3480065294","cluster_addr":"172.21.15.173:6811/3480065294","heartbeat_back_addr":"172.21.15.173:6815/3480065294","heartbeat_front_addr":"172.21.15.173:6813/3480065294","state":["exists","up"]},{"osd":5,"uuid":"47b99706-a27f-4f30-9e6d-619ee6a35ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":41,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6817","nonce":3281347854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6819","nonce":3281347854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6823","nonce":3281347854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6821","nonce":3281347854}]},"public_addr":"172.21.15.173:6817/3281347854","cluster_addr":"172.21.15.173:6819/3281347854","heartbeat_back_addr":"172.21.15.173:6823/3281347854","heartbeat_front_addr":"172.21.15.173:6821/3281347854","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:59:57.050973+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:00:33.939553+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:11.618080+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:46.287328+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:20.465163+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:56.550872+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.173:6825/3563933435":"2024-04-21T11:19:07.882600+0000","172.21.15.173:6824/3563933435":"2024-04-21T11:19:07.882600+0000","172.21.15.173:0/1016803704":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/3713452282":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/519807094":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/2304890710":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/4182681741":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/571056469":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/1807203325":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1384022397":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2961758989":"2024-04-21T11:07:41.749629+0000","172.21.15.117:6801/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6800/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.173:0/997010605":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6801/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2230602257":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1375611439":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2096654470":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1089633358":"2024-04-21T10:58:13.447117+0000","172.21.15.173:6829/2992581883":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6800/2612478075":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/638889142":"2024-04-21T11:13:42.885571+0000","172.21.15.173:0/3679484323":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/4088938050":"2024-04-21T10:56:04.789705+0000","172.21.15.173:0/3475739843":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/894992429":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6800/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6826/361271135":"2024-04-21T11:19:07.882570+0000","172.21.15.117:6800/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.117:6801/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.173:6828/2992581883":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6801/2612478075":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/1003331392":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6801/3003776987":"2024-04-21T11:13:42.885571+0000","172.21.15.117:0/3651098074":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6800/3003776987":"2024-04-21T11:13:42.885571+0000","172.21.15.173:0/2377885188":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6827/361271135":"2024-04-21T11:19:07.882570+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:53:21.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd74406ecf0 msgr2=0x7fd7440711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:21.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd74406ecf0 0x7fd7440711b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd750005c40 tx=0x7fd750005bd0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7601983c0 msgr2=0x7fd76019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:21.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7601983c0 0x7fd76019c820 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd748036040 tx=0x7fd74802c7a0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 shutdown_connections 2024-04-20T11:53:21.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd74406ecf0 0x7fd7440711b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7601983c0 0x7fd76019c820 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 --2- 172.21.15.117:0/1269540129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd760102480 0x7fd760197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.354+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 >> 172.21.15.117:0/1269540129 conn(0x7fd7600fdd00 msgr2=0x7fd7601000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:21.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.355+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 shutdown_connections 2024-04-20T11:53:21.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.355+0000 7fd765e00640 1 -- 172.21.15.117:0/1269540129 wait complete. 2024-04-20T11:53:21.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:01:14.283466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-20T11:04:32.294012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-20T11:04:33.229067+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:53:21.373 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:53:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.581+0000 7fd473200640 1 -- 172.21.15.117:0/4093313136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd474100a70 msgr2=0x7fd474104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.581+0000 7fd473200640 1 --2- 172.21.15.117:0/4093313136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd474100a70 0x7fd474104ab0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009880 tx=0x7fd45c02bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.581+0000 7fd473200640 1 -- 172.21.15.117:0/4093313136 shutdown_connections 2024-04-20T11:53:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.581+0000 7fd473200640 1 --2- 172.21.15.117:0/4093313136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd474100a70 0x7fd474104ab0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.581+0000 7fd473200640 1 --2- 172.21.15.117:0/4093313136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4741000c0 0x7fd4741004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.581+0000 7fd473200640 1 -- 172.21.15.117:0/4093313136 >> 172.21.15.117:0/4093313136 conn(0x7fd4740fb920 msgr2=0x7fd4740fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.581+0000 7fd473200640 1 -- 172.21.15.117:0/4093313136 shutdown_connections 2024-04-20T11:53:21.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.581+0000 7fd473200640 1 -- 172.21.15.117:0/4093313136 wait complete. 2024-04-20T11:53:21.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.582+0000 7fd473200640 1 Processor -- start 2024-04-20T11:53:21.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.583+0000 7fd473200640 1 -- start start 2024-04-20T11:53:21.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.583+0000 7fd473200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4741000c0 0x7fd474197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:21.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.583+0000 7fd473200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4741982c0 0x7fd47419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:21.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.583+0000 7fd473200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741988c0 con 0x7fd4741982c0 2024-04-20T11:53:21.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.583+0000 7fd473200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd474198a30 con 0x7fd4741000c0 2024-04-20T11:53:21.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.583+0000 7fd46be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4741982c0 0x7fd47419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:21.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.583+0000 7fd46be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4741982c0 0x7fd47419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51622/0 (socket says 172.21.15.117:51622) 2024-04-20T11:53:21.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.583+0000 7fd46be00640 1 -- 172.21.15.117:0/4187637300 learned_addr learned my addr 172.21.15.117:0/4187637300 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:21.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.584+0000 7fd471e00640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4741000c0 0x7fd474197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:21.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.584+0000 7fd46be00640 1 -- 172.21.15.117:0/4187637300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4741000c0 msgr2=0x7fd474197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:21.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.584+0000 7fd46be00640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4741000c0 0x7fd474197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.584+0000 7fd46be00640 1 -- 172.21.15.117:0/4187637300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c009560 con 0x7fd4741982c0 2024-04-20T11:53:21.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.584+0000 7fd46be00640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4741982c0 0x7fd47419c720 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd45c036040 tx=0x7fd45c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:21.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.584+0000 7fd46aa00640 1 -- 172.21.15.117:0/4187637300 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45c0035f0 con 0x7fd4741982c0 2024-04-20T11:53:21.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.585+0000 7fd46aa00640 1 -- 172.21.15.117:0/4187637300 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd45c003c10 con 0x7fd4741982c0 2024-04-20T11:53:21.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.585+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47419ccc0 con 0x7fd4741982c0 2024-04-20T11:53:21.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.585+0000 7fd46aa00640 1 -- 172.21.15.117:0/4187637300 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45c034440 con 0x7fd4741982c0 2024-04-20T11:53:21.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.585+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47419d220 con 0x7fd4741982c0 2024-04-20T11:53:21.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.586+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47404e040 con 0x7fd4741982c0 2024-04-20T11:53:21.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.587+0000 7fd46aa00640 1 -- 172.21.15.117:0/4187637300 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd45c005060 con 0x7fd4741982c0 2024-04-20T11:53:21.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.588+0000 7fd46aa00640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd45406ec30 0x7fd4540710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:21.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.588+0000 7fd46aa00640 1 -- 172.21.15.117:0/4187637300 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd45c0b1280 con 0x7fd4741982c0 2024-04-20T11:53:21.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.588+0000 7fd471e00640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd45406ec30 0x7fd4540710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:21.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.589+0000 7fd471e00640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd45406ec30 0x7fd4540710f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd4640058c0 tx=0x7fd464005c60 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:21.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.593+0000 7fd46aa00640 1 -- 172.21.15.117:0/4187637300 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd45c07a040 con 0x7fd4741982c0 2024-04-20T11:53:21.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:21 smithi117 ceph-mon[99036]: pgmap v1229: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:21 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4192661633' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-20T11:53:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:21 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1269540129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:53:21.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.799+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4741990d0 con 0x7fd4741982c0 2024-04-20T11:53:21.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.800+0000 7fd46aa00640 1 -- 172.21.15.117:0/4187637300 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd45c07d700 con 0x7fd4741982c0 2024-04-20T11:53:21.800 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-04-20T11:53:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd45406ec30 msgr2=0x7fd4540710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd45406ec30 0x7fd4540710f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd4640058c0 tx=0x7fd464005c60 comp rx=0 tx=0).stop 2024-04-20T11:53:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4741982c0 msgr2=0x7fd47419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4741982c0 0x7fd47419c720 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd45c036040 tx=0x7fd45c004040 comp rx=0 tx=0).stop 2024-04-20T11:53:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 shutdown_connections 2024-04-20T11:53:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd45406ec30 0x7fd4540710f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4741982c0 0x7fd47419c720 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 --2- 172.21.15.117:0/4187637300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4741000c0 0x7fd474197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:21.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 >> 172.21.15.117:0/4187637300 conn(0x7fd4740fb920 msgr2=0x7fd4740fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:21.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 shutdown_connections 2024-04-20T11:53:21.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:21.803+0000 7fd473200640 1 -- 172.21.15.117:0/4187637300 wait complete. 2024-04-20T11:53:21.821 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-20T11:53:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:21 smithi173 ceph-mon[74165]: pgmap v1229: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:21 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4192661633' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-20T11:53:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:21 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1269540129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:53:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.025+0000 7f4cb7200640 1 -- 172.21.15.117:0/2199565798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100a40 msgr2=0x7f4cb8104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.025+0000 7f4cb7200640 1 --2- 172.21.15.117:0/2199565798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100a40 0x7f4cb8104a80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009880 tx=0x7f4ca002bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.025+0000 7f4cb7200640 1 -- 172.21.15.117:0/2199565798 shutdown_connections 2024-04-20T11:53:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.025+0000 7f4cb7200640 1 --2- 172.21.15.117:0/2199565798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100a40 0x7f4cb8104a80 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.025+0000 7f4cb7200640 1 --2- 172.21.15.117:0/2199565798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cb8100090 0x7f4cb8100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.025+0000 7f4cb7200640 1 -- 172.21.15.117:0/2199565798 >> 172.21.15.117:0/2199565798 conn(0x7f4cb80fb8f0 msgr2=0x7f4cb80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.025+0000 7f4cb7200640 1 -- 172.21.15.117:0/2199565798 shutdown_connections 2024-04-20T11:53:22.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.025+0000 7f4cb7200640 1 -- 172.21.15.117:0/2199565798 wait complete. 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.026+0000 7f4cb7200640 1 Processor -- start 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.026+0000 7f4cb7200640 1 -- start start 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.026+0000 7f4cb7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100090 0x7f4cb819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.026+0000 7f4cb7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cb819bec0 0x7f4cb81959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.026+0000 7f4cb7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb819c490 con 0x7f4cb8100090 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.026+0000 7f4cb7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb8195f30 con 0x7f4cb819bec0 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.027+0000 7f4cb5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100090 0x7f4cb819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.027+0000 7f4cb5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100090 0x7f4cb819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51636/0 (socket says 172.21.15.117:51636) 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.027+0000 7f4cb5e00640 1 -- 172.21.15.117:0/3243022481 learned_addr learned my addr 172.21.15.117:0/3243022481 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.027+0000 7f4cb5e00640 1 -- 172.21.15.117:0/3243022481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cb819bec0 msgr2=0x7f4cb81959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.027+0000 7f4cad400640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cb819bec0 0x7f4cb81959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.027+0000 7f4cb5e00640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cb819bec0 0x7f4cb81959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.027+0000 7f4cb5e00640 1 -- 172.21.15.117:0/3243022481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca0009560 con 0x7f4cb8100090 2024-04-20T11:53:22.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.027+0000 7f4cb5e00640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100090 0x7f4cb819b980 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800d8d0 tx=0x7f4ca800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:22.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.028+0000 7f4caf400640 1 -- 172.21.15.117:0/3243022481 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ca800bea0 con 0x7f4cb8100090 2024-04-20T11:53:22.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.028+0000 7f4caf400640 1 -- 172.21.15.117:0/3243022481 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ca8009c00 con 0x7f4cb8100090 2024-04-20T11:53:22.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.028+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb8196210 con 0x7f4cb8100090 2024-04-20T11:53:22.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.028+0000 7f4caf400640 1 -- 172.21.15.117:0/3243022481 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ca800f710 con 0x7f4cb8100090 2024-04-20T11:53:22.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.028+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb8196770 con 0x7f4cb8100090 2024-04-20T11:53:22.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.029+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb804e040 con 0x7f4cb8100090 2024-04-20T11:53:22.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.030+0000 7f4caf400640 1 -- 172.21.15.117:0/3243022481 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4ca800f990 con 0x7f4cb8100090 2024-04-20T11:53:22.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.031+0000 7f4caf400640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c9c06ec30 0x7f4c9c0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.031+0000 7f4caf400640 1 -- 172.21.15.117:0/3243022481 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f4ca808ec20 con 0x7f4cb8100090 2024-04-20T11:53:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.031+0000 7f4cad400640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c9c06ec30 0x7f4c9c0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.032+0000 7f4cad400640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c9c06ec30 0x7f4c9c0710f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0036040 tx=0x7f4ca0005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:22.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.036+0000 7f4caf400640 1 -- 172.21.15.117:0/3243022481 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f4ca8057960 con 0x7f4cb8100090 2024-04-20T11:53:22.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.228+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f4cb8108390 con 0x7f4cb8100090 2024-04-20T11:53:22.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.229+0000 7f4caf400640 1 -- 172.21.15.117:0/3243022481 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4ca805b020 con 0x7f4cb8100090 2024-04-20T11:53:22.229 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-04-20T11:53:22.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.231+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c9c06ec30 msgr2=0x7f4c9c0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c9c06ec30 0x7f4c9c0710f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0036040 tx=0x7f4ca0005fd0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100090 msgr2=0x7f4cb819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100090 0x7f4cb819b980 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800d8d0 tx=0x7f4ca800dd90 comp rx=0 tx=0).stop 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 shutdown_connections 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f4c9c06ec30 0x7f4c9c0710f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cb819bec0 0x7f4cb81959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 --2- 172.21.15.117:0/3243022481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cb8100090 0x7f4cb819b980 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 >> 172.21.15.117:0/3243022481 conn(0x7f4cb80fb8f0 msgr2=0x7f4cb80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 shutdown_connections 2024-04-20T11:53:22.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.232+0000 7f4cb7200640 1 -- 172.21.15.117:0/3243022481 wait complete. 2024-04-20T11:53:22.251 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-20T11:53:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.460+0000 7fb158400640 1 -- 172.21.15.117:0/636631436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501002b0 msgr2=0x7fb150100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.460+0000 7fb158400640 1 --2- 172.21.15.117:0/636631436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501002b0 0x7fb150100690 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb144009880 tx=0x7fb14402bce0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.460+0000 7fb158400640 1 -- 172.21.15.117:0/636631436 shutdown_connections 2024-04-20T11:53:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.460+0000 7fb158400640 1 --2- 172.21.15.117:0/636631436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb150100c60 0x7fb150104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.460+0000 7fb158400640 1 --2- 172.21.15.117:0/636631436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501002b0 0x7fb150100690 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.460+0000 7fb158400640 1 -- 172.21.15.117:0/636631436 >> 172.21.15.117:0/636631436 conn(0x7fb1500fbb50 msgr2=0x7fb1500fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.460+0000 7fb158400640 1 -- 172.21.15.117:0/636631436 shutdown_connections 2024-04-20T11:53:22.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.460+0000 7fb158400640 1 -- 172.21.15.117:0/636631436 wait complete. 2024-04-20T11:53:22.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.461+0000 7fb158400640 1 Processor -- start 2024-04-20T11:53:22.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.461+0000 7fb158400640 1 -- start start 2024-04-20T11:53:22.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.461+0000 7fb158400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1501002b0 0x7fb150195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.461+0000 7fb158400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501963a0 0x7fb15019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.461+0000 7fb158400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501969a0 con 0x7fb1501963a0 2024-04-20T11:53:22.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.461+0000 7fb158400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb150196b10 con 0x7fb1501002b0 2024-04-20T11:53:22.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.461+0000 7fb155000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501963a0 0x7fb15019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.462+0000 7fb155000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501963a0 0x7fb15019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51646/0 (socket says 172.21.15.117:51646) 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.462+0000 7fb155000640 1 -- 172.21.15.117:0/1684990280 learned_addr learned my addr 172.21.15.117:0/1684990280 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.462+0000 7fb155a00640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1501002b0 0x7fb150195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.462+0000 7fb155000640 1 -- 172.21.15.117:0/1684990280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1501002b0 msgr2=0x7fb150195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.462+0000 7fb155000640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1501002b0 0x7fb150195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.462+0000 7fb155000640 1 -- 172.21.15.117:0/1684990280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb144009560 con 0x7fb1501963a0 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.462+0000 7fb155000640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501963a0 0x7fb15019a800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00d6f0 tx=0x7fb13c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.462+0000 7fb14e000640 1 -- 172.21.15.117:0/1684990280 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb13c0051f0 con 0x7fb1501963a0 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.463+0000 7fb14e000640 1 -- 172.21.15.117:0/1684990280 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb13c009ce0 con 0x7fb1501963a0 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.463+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15019ae00 con 0x7fb1501963a0 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.463+0000 7fb14e000640 1 -- 172.21.15.117:0/1684990280 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb13c004e80 con 0x7fb1501963a0 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.463+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15019b360 con 0x7fb1501963a0 2024-04-20T11:53:22.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.464+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15019b630 con 0x7fb1501963a0 2024-04-20T11:53:22.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.465+0000 7fb14e000640 1 -- 172.21.15.117:0/1684990280 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb13c0046c0 con 0x7fb1501963a0 2024-04-20T11:53:22.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.466+0000 7fb14e000640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb13806ebd0 0x7fb138071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.466+0000 7fb14e000640 1 -- 172.21.15.117:0/1684990280 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb13c08e950 con 0x7fb1501963a0 2024-04-20T11:53:22.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.466+0000 7fb155a00640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb13806ebd0 0x7fb138071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.467+0000 7fb155a00640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb13806ebd0 0x7fb138071090 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb144036040 tx=0x7fb14402cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:22.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.471+0000 7fb14e000640 1 -- 172.21.15.117:0/1684990280 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb13c0576d0 con 0x7fb1501963a0 2024-04-20T11:53:22.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.672+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb1501971e0 con 0x7fb1501963a0 2024-04-20T11:53:22.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.672+0000 7fb14e000640 1 -- 172.21.15.117:0/1684990280 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb13c05ad90 con 0x7fb1501963a0 2024-04-20T11:53:22.672 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-04-20T11:53:22.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb13806ebd0 msgr2=0x7fb138071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb13806ebd0 0x7fb138071090 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb144036040 tx=0x7fb14402cf90 comp rx=0 tx=0).stop 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501963a0 msgr2=0x7fb15019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501963a0 0x7fb15019a800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00d6f0 tx=0x7fb13c00dbb0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 shutdown_connections 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb13806ebd0 0x7fb138071090 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501963a0 0x7fb15019a800 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 --2- 172.21.15.117:0/1684990280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1501002b0 0x7fb150195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 >> 172.21.15.117:0/1684990280 conn(0x7fb1500fbb50 msgr2=0x7fb1500fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 shutdown_connections 2024-04-20T11:53:22.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.675+0000 7fb158400640 1 -- 172.21.15.117:0/1684990280 wait complete. 2024-04-20T11:53:22.694 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:53:22.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.889+0000 7f907ea00640 1 -- 172.21.15.117:0/4136305863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9078072ff0 msgr2=0x7f907810c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.889+0000 7f907ea00640 1 --2- 172.21.15.117:0/4136305863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9078072ff0 0x7f907810c170 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9064009880 tx=0x7f906402bd90 comp rx=0 tx=0).stop 2024-04-20T11:53:22.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.889+0000 7f907ea00640 1 -- 172.21.15.117:0/4136305863 shutdown_connections 2024-04-20T11:53:22.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.889+0000 7f907ea00640 1 --2- 172.21.15.117:0/4136305863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9078072ff0 0x7f907810c170 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.889+0000 7f907ea00640 1 --2- 172.21.15.117:0/4136305863 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90780726d0 0x7f9078072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.889+0000 7f907ea00640 1 -- 172.21.15.117:0/4136305863 >> 172.21.15.117:0/4136305863 conn(0x7f90780fb960 msgr2=0x7f90780fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:22.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.889+0000 7f907ea00640 1 -- 172.21.15.117:0/4136305863 shutdown_connections 2024-04-20T11:53:22.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.889+0000 7f907ea00640 1 -- 172.21.15.117:0/4136305863 wait complete. 2024-04-20T11:53:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:22 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4187637300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:53:22.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:22 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3243022481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-20T11:53:22.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.890+0000 7f907ea00640 1 Processor -- start 2024-04-20T11:53:22.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.890+0000 7f907ea00640 1 -- start start 2024-04-20T11:53:22.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.891+0000 7f907ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780726d0 0x7f907819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.891+0000 7f907ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9078072ff0 0x7f907819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.891+0000 7f907ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9078197b50 con 0x7f90780726d0 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.891+0000 7f907ea00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9078197cc0 con 0x7f9078072ff0 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.891+0000 7f907d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780726d0 0x7f907819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.891+0000 7f907d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780726d0 0x7f907819dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51660/0 (socket says 172.21.15.117:51660) 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.891+0000 7f907d600640 1 -- 172.21.15.117:0/2238760299 learned_addr learned my addr 172.21.15.117:0/2238760299 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.891+0000 7f907cc00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9078072ff0 0x7f907819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.892+0000 7f907d600640 1 -- 172.21.15.117:0/2238760299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9078072ff0 msgr2=0x7f907819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.892+0000 7f907d600640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9078072ff0 0x7f907819e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.892+0000 7f907d600640 1 -- 172.21.15.117:0/2238760299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9064009560 con 0x7f90780726d0 2024-04-20T11:53:22.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.892+0000 7f907cc00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9078072ff0 0x7f907819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:53:22.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.892+0000 7f907d600640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780726d0 0x7f907819dae0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c810 tx=0x7f906c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:22.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.893+0000 7f9076000640 1 -- 172.21.15.117:0/2238760299 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f906c0041f0 con 0x7f90780726d0 2024-04-20T11:53:22.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.893+0000 7f9076000640 1 -- 172.21.15.117:0/2238760299 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f906c007e70 con 0x7f90780726d0 2024-04-20T11:53:22.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.893+0000 7f9076000640 1 -- 172.21.15.117:0/2238760299 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f906c0049d0 con 0x7f90780726d0 2024-04-20T11:53:22.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.893+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9078197fa0 con 0x7f90780726d0 2024-04-20T11:53:22.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.893+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9078198500 con 0x7f90780726d0 2024-04-20T11:53:22.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.894+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907804e890 con 0x7f90780726d0 2024-04-20T11:53:22.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.895+0000 7f9076000640 1 -- 172.21.15.117:0/2238760299 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f906c0079b0 con 0x7f90780726d0 2024-04-20T11:53:22.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.896+0000 7f9076000640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f906006ec30 0x7f90600710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:22.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.896+0000 7f9076000640 1 -- 172.21.15.117:0/2238760299 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f906c08eb70 con 0x7f90780726d0 2024-04-20T11:53:22.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.896+0000 7f907cc00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f906006ec30 0x7f90600710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:22.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.897+0000 7f907cc00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f906006ec30 0x7f90600710f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f90780fcf80 tx=0x7f9064005a40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:22.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:22.903+0000 7f9076000640 1 -- 172.21.15.117:0/2238760299 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f906c0579d0 con 0x7f90780726d0 2024-04-20T11:53:22.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:22 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4187637300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:53:22.914 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:22 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3243022481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-20T11:53:23.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.119+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9078198ff0 con 0x7f90780726d0 2024-04-20T11:53:23.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.120+0000 7f9076000640 1 -- 172.21.15.117:0/2238760299 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 38 entries v86) v1 ==== 81+0+2283 (secure 0 0 0) 0x7f906c01d080 con 0x7f90780726d0 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:6825/3563933435 2024-04-21T11:19:07.882600+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:6824/3563933435 2024-04-21T11:19:07.882600+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/1016803704 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/3713452282 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/519807094 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2304890710 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/4182681741 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/571056469 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1807203325 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1384022397 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2961758989 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/1996106837 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/1996106837 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/997010605 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2597372910 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2230602257 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1375611439 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2096654470 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1089633358 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:6829/2992581883 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2612478075 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/638889142 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/3679484323 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/4088938050 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/3475739843 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/894992429 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2597372910 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6826/361271135 2024-04-21T11:19:07.882570+0000 2024-04-20T11:53:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/1432319225 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/1432319225 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:6828/2992581883 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2612478075 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1003331392 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/3003776987 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/3651098074 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/3003776987 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/2377885188 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6827/361271135 2024-04-21T11:19:07.882570+0000 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:listed 38 entries 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.122+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f906006ec30 msgr2=0x7f90600710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.122+0000 7f907ea00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f906006ec30 0x7f90600710f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f90780fcf80 tx=0x7f9064005a40 comp rx=0 tx=0).stop 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.122+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780726d0 msgr2=0x7f907819dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.122+0000 7f907ea00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780726d0 0x7f907819dae0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c810 tx=0x7f906c00ccd0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.123+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 shutdown_connections 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.123+0000 7f907ea00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f906006ec30 0x7f90600710f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.123+0000 7f907ea00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9078072ff0 0x7f907819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.123+0000 7f907ea00640 1 --2- 172.21.15.117:0/2238760299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780726d0 0x7f907819dae0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.123+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 >> 172.21.15.117:0/2238760299 conn(0x7f90780fb960 msgr2=0x7f90781069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:23.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.123+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 shutdown_connections 2024-04-20T11:53:23.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.123+0000 7f907ea00640 1 -- 172.21.15.117:0/2238760299 wait complete. 2024-04-20T11:53:23.141 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-20T11:53:23.141 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:53:23.189 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:53:23.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.433+0000 7f64ab200640 1 -- 172.21.15.117:0/2661282219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067350 msgr2=0x7f64ac067730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:23.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.433+0000 7f64ab200640 1 --2- 172.21.15.117:0/2661282219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067350 0x7f64ac067730 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f649c009880 tx=0x7f649c02bd30 comp rx=0 tx=0).stop 2024-04-20T11:53:23.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.433+0000 7f64ab200640 1 -- 172.21.15.117:0/2661282219 shutdown_connections 2024-04-20T11:53:23.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.433+0000 7f64ab200640 1 --2- 172.21.15.117:0/2661282219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac067c70 0x7f64ac10c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.433+0000 7f64ab200640 1 --2- 172.21.15.117:0/2661282219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067350 0x7f64ac067730 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.433+0000 7f64ab200640 1 -- 172.21.15.117:0/2661282219 >> 172.21.15.117:0/2661282219 conn(0x7f64ac0748e0 msgr2=0x7f64ac074cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:23.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.433+0000 7f64ab200640 1 -- 172.21.15.117:0/2661282219 shutdown_connections 2024-04-20T11:53:23.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.433+0000 7f64ab200640 1 -- 172.21.15.117:0/2661282219 wait complete. 2024-04-20T11:53:23.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.434+0000 7f64ab200640 1 Processor -- start 2024-04-20T11:53:23.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.434+0000 7f64ab200640 1 -- start start 2024-04-20T11:53:23.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64ab200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac067350 0x7f64ac19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:23.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64ab200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067c70 0x7f64ac19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:23.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64ab200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac197ae0 con 0x7f64ac067c70 2024-04-20T11:53:23.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64ab200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac197c50 con 0x7f64ac067350 2024-04-20T11:53:23.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64a9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067c70 0x7f64ac19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:23.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64a9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067c70 0x7f64ac19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51686/0 (socket says 172.21.15.117:51686) 2024-04-20T11:53:23.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64a9400640 1 -- 172.21.15.117:0/537237538 learned_addr learned my addr 172.21.15.117:0/537237538 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:23.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64a9e00640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac067350 0x7f64ac19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:23.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64a9400640 1 -- 172.21.15.117:0/537237538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac067350 msgr2=0x7f64ac19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:23.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.435+0000 7f64a9400640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac067350 0x7f64ac19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.436+0000 7f64a9400640 1 -- 172.21.15.117:0/537237538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c009560 con 0x7f64ac067c70 2024-04-20T11:53:23.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.436+0000 7f64a9400640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067c70 0x7f64ac19dfb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f649800c8f0 tx=0x7f649800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:23.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.436+0000 7f64a2a00640 1 -- 172.21.15.117:0/537237538 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6498007b80 con 0x7f64ac067c70 2024-04-20T11:53:23.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.436+0000 7f64a2a00640 1 -- 172.21.15.117:0/537237538 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64980054e0 con 0x7f64ac067c70 2024-04-20T11:53:23.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.436+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac197f30 con 0x7f64ac067c70 2024-04-20T11:53:23.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.436+0000 7f64a2a00640 1 -- 172.21.15.117:0/537237538 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649800f450 con 0x7f64ac067c70 2024-04-20T11:53:23.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.436+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac198490 con 0x7f64ac067c70 2024-04-20T11:53:23.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.437+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac04e890 con 0x7f64ac067c70 2024-04-20T11:53:23.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.438+0000 7f64a2a00640 1 -- 172.21.15.117:0/537237538 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6498004f60 con 0x7f64ac067c70 2024-04-20T11:53:23.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.439+0000 7f64a2a00640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f649006ec80 0x7f6490071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:23.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.439+0000 7f64a2a00640 1 -- 172.21.15.117:0/537237538 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f649808ed30 con 0x7f64ac067c70 2024-04-20T11:53:23.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.440+0000 7f64a9e00640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f649006ec80 0x7f6490071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:23.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.440+0000 7f64a9e00640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f649006ec80 0x7f6490071140 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f649c036040 tx=0x7f649c0058d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:23.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.444+0000 7f64a2a00640 1 -- 172.21.15.117:0/537237538 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f6498057a70 con 0x7f64ac067c70 2024-04-20T11:53:23.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.653+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64ac109b50 con 0x7f64ac067c70 2024-04-20T11:53:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.655+0000 7f64a2a00640 1 -- 172.21.15.117:0/537237538 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16635 (secure 0 0 0) 0x7f649805b130 con 0x7f64ac067c70 2024-04-20T11:53:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-20T11:53:23.656 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":86,"fsid":"59ecfd22-ff04-11ee-bc92-c7b262605968","created":"2024-04-20T10:55:31.778050+0000","modified":"2024-04-20T11:35:20.567312+0000","last_up_change":"2024-04-20T11:19:01.116321+0000","last_in_change":"2024-04-20T11:02:33.545131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:01:14.283466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-20T11:04:32.294012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-20T11:04:33.229067+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee5f9645-4fd0-48f7-8eb7-7b38ea0bae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6803","nonce":664509927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6805","nonce":664509927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6809","nonce":664509927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":664509927},{"type":"v1","addr":"172.21.15.117:6807","nonce":664509927}]},"public_addr":"172.21.15.117:6803/664509927","cluster_addr":"172.21.15.117:6805/664509927","heartbeat_back_addr":"172.21.15.117:6809/664509927","heartbeat_front_addr":"172.21.15.117:6807/664509927","state":["exists","up"]},{"osd":1,"uuid":"75ea37ae-266a-459b-a32b-e1d91927cbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6811","nonce":2323295100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6813","nonce":2323295100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6817","nonce":2323295100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2323295100},{"type":"v1","addr":"172.21.15.117:6815","nonce":2323295100}]},"public_addr":"172.21.15.117:6811/2323295100","cluster_addr":"172.21.15.117:6813/2323295100","heartbeat_back_addr":"172.21.15.117:6817/2323295100","heartbeat_front_addr":"172.21.15.117:6815/2323295100","state":["exists","up"]},{"osd":2,"uuid":"2323b6ba-7d04-4744-b61f-4936e4f419bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6819","nonce":155800786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6821","nonce":155800786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6825","nonce":155800786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":155800786},{"type":"v1","addr":"172.21.15.117:6823","nonce":155800786}]},"public_addr":"172.21.15.117:6819/155800786","cluster_addr":"172.21.15.117:6821/155800786","heartbeat_back_addr":"172.21.15.117:6825/155800786","heartbeat_front_addr":"172.21.15.117:6823/155800786","state":["exists","up"]},{"osd":3,"uuid":"b87d3df6-0d27-4e2d-9e40-55895e0a891d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6801","nonce":151782600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6803","nonce":151782600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6807","nonce":151782600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":151782600},{"type":"v1","addr":"172.21.15.173:6805","nonce":151782600}]},"public_addr":"172.21.15.173:6801/151782600","cluster_addr":"172.21.15.173:6803/151782600","heartbeat_back_addr":"172.21.15.173:6807/151782600","heartbeat_front_addr":"172.21.15.173:6805/151782600","state":["exists","up"]},{"osd":4,"uuid":"c5ceed42-a923-47e4-919d-98caab4e06ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":36,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6809","nonce":3480065294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6811","nonce":3480065294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6815","nonce":3480065294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3480065294},{"type":"v1","addr":"172.21.15.173:6813","nonce":3480065294}]},"public_addr":"172.21.15.173:6809/3480065294","cluster_addr":"172.21.15.173:6811/3480065294","heartbeat_back_addr":"172.21.15.173:6815/3480065294","heartbeat_front_addr":"172.21.15.173:6813/3480065294","state":["exists","up"]},{"osd":5,"uuid":"47b99706-a27f-4f30-9e6d-619ee6a35ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":41,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6817","nonce":3281347854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6819","nonce":3281347854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6823","nonce":3281347854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3281347854},{"type":"v1","addr":"172.21.15.173:6821","nonce":3281347854}]},"public_addr":"172.21.15.173:6817/3281347854","cluster_addr":"172.21.15.173:6819/3281347854","heartbeat_back_addr":"172.21.15.173:6823/3281347854","heartbeat_front_addr":"172.21.15.173:6821/3281347854","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:59:57.050973+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:00:33.939553+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:11.618080+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:01:46.287328+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:20.465163+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:02:56.550872+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.173:6825/3563933435":"2024-04-21T11:19:07.882600+0000","172.21.15.173:6824/3563933435":"2024-04-21T11:19:07.882600+0000","172.21.15.173:0/1016803704":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/3713452282":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/519807094":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/2304890710":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/4182681741":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/571056469":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/1807203325":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1384022397":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2961758989":"2024-04-21T11:07:41.749629+0000","172.21.15.117:6801/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.117:6800/1996106837":"2024-04-21T10:58:13.447117+0000","172.21.15.173:0/997010605":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6801/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/2230602257":"2024-04-21T10:56:46.887330+0000","172.21.15.117:0/1375611439":"2024-04-21T10:56:04.789705+0000","172.21.15.117:0/2096654470":"2024-04-21T10:58:13.447117+0000","172.21.15.117:0/1089633358":"2024-04-21T10:58:13.447117+0000","172.21.15.173:6829/2992581883":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6800/2612478075":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/638889142":"2024-04-21T11:13:42.885571+0000","172.21.15.173:0/3679484323":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/4088938050":"2024-04-21T10:56:04.789705+0000","172.21.15.173:0/3475739843":"2024-04-21T11:08:45.884891+0000","172.21.15.117:0/894992429":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6800/2597372910":"2024-04-21T10:56:46.887330+0000","172.21.15.117:6826/361271135":"2024-04-21T11:19:07.882570+0000","172.21.15.117:6800/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.117:6801/1432319225":"2024-04-21T10:56:04.789705+0000","172.21.15.173:6828/2992581883":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6801/2612478075":"2024-04-21T11:07:41.749629+0000","172.21.15.117:0/1003331392":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6801/3003776987":"2024-04-21T11:13:42.885571+0000","172.21.15.117:0/3651098074":"2024-04-21T11:13:42.885571+0000","172.21.15.117:6800/3003776987":"2024-04-21T11:13:42.885571+0000","172.21.15.173:0/2377885188":"2024-04-21T11:08:45.884891+0000","172.21.15.117:6827/361271135":"2024-04-21T11:19:07.882570+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f649006ec80 msgr2=0x7f6490071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f649006ec80 0x7f6490071140 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f649c036040 tx=0x7f649c0058d0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067c70 msgr2=0x7f64ac19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067c70 0x7f64ac19dfb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f649800c8f0 tx=0x7f649800cdb0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 shutdown_connections 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f649006ec80 0x7f6490071140 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac067c70 0x7f64ac19dfb0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 --2- 172.21.15.117:0/537237538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac067350 0x7f64ac19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 >> 172.21.15.117:0/537237538 conn(0x7f64ac0748e0 msgr2=0x7f64ac0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 shutdown_connections 2024-04-20T11:53:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.658+0000 7f64ab200640 1 -- 172.21.15.117:0/537237538 wait complete. 2024-04-20T11:53:23.674 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:23 smithi117 ceph-mon[99036]: pgmap v1230: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:23.674 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:23 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/1684990280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-20T11:53:23.674 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:23 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/2238760299' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T11:53:23.678 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:01:14.283466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-20T11:04:32.294012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-20T11:04:33.229067+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:53:23.678 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:53:23.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.877+0000 7fb62fe00640 1 -- 172.21.15.117:0/3737869745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072640 msgr2=0x7fb630072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:23.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.877+0000 7fb62fe00640 1 --2- 172.21.15.117:0/3737869745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072640 0x7fb630072a20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb620009880 tx=0x7fb62002bd70 comp rx=0 tx=0).stop 2024-04-20T11:53:23.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.877+0000 7fb62fe00640 1 -- 172.21.15.117:0/3737869745 shutdown_connections 2024-04-20T11:53:23.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.877+0000 7fb62fe00640 1 --2- 172.21.15.117:0/3737869745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb630072f60 0x7fb630110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.877+0000 7fb62fe00640 1 --2- 172.21.15.117:0/3737869745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072640 0x7fb630072a20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.877+0000 7fb62fe00640 1 -- 172.21.15.117:0/3737869745 >> 172.21.15.117:0/3737869745 conn(0x7fb6300fb970 msgr2=0x7fb6300fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:23.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.877+0000 7fb62fe00640 1 -- 172.21.15.117:0/3737869745 shutdown_connections 2024-04-20T11:53:23.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.877+0000 7fb62fe00640 1 -- 172.21.15.117:0/3737869745 wait complete. 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.878+0000 7fb62fe00640 1 Processor -- start 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.878+0000 7fb62fe00640 1 -- start start 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb630072640 0x7fb63019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072f60 0x7fb63019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63019cdc0 con 0x7fb630072f60 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301a0b90 con 0x7fb630072640 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072f60 0x7fb63019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072f60 0x7fb63019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41058/0 (socket says 172.21.15.117:41058) 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62e000640 1 -- 172.21.15.117:0/4233485700 learned_addr learned my addr 172.21.15.117:0/4233485700 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62ea00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb630072640 0x7fb63019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:23.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62e000640 1 -- 172.21.15.117:0/4233485700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb630072640 msgr2=0x7fb63019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62e000640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb630072640 0x7fb63019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62e000640 1 -- 172.21.15.117:0/4233485700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb620009560 con 0x7fb630072f60 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.879+0000 7fb62ea00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb630072640 0x7fb63019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.880+0000 7fb62e000640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072f60 0x7fb63019c730 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b420 tx=0x7fb61c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.880+0000 7fb634a00640 1 -- 172.21.15.117:0/4233485700 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61c011800 con 0x7fb630072f60 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.880+0000 7fb634a00640 1 -- 172.21.15.117:0/4233485700 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb61c011e20 con 0x7fb630072f60 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.880+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6301a0e70 con 0x7fb630072f60 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.880+0000 7fb634a00640 1 -- 172.21.15.117:0/4233485700 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61c00f2e0 con 0x7fb630072f60 2024-04-20T11:53:23.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.880+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6301a13d0 con 0x7fb630072f60 2024-04-20T11:53:23.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.881+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63004e040 con 0x7fb630072f60 2024-04-20T11:53:23.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.882+0000 7fb634a00640 1 -- 172.21.15.117:0/4233485700 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb61c010380 con 0x7fb630072f60 2024-04-20T11:53:23.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.884+0000 7fb634a00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb61406ec30 0x7fb6140710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:23.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.884+0000 7fb634a00640 1 -- 172.21.15.117:0/4233485700 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb61c08e850 con 0x7fb630072f60 2024-04-20T11:53:23.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.884+0000 7fb62ea00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb61406ec30 0x7fb6140710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:23.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.885+0000 7fb62ea00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb61406ec30 0x7fb6140710f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb62002c930 tx=0x7fb62002cc50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:23.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:23.890+0000 7fb634a00640 1 -- 172.21.15.117:0/4233485700 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fb61c057590 con 0x7fb630072f60 2024-04-20T11:53:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:23 smithi173 ceph-mon[74165]: pgmap v1230: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:23 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/1684990280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-20T11:53:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:23 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/2238760299' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T11:53:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.093+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb630113720 con 0x7fb630072f60 2024-04-20T11:53:24.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.094+0000 7fb634a00640 1 -- 172.21.15.117:0/4233485700 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb61c055e60 con 0x7fb630072f60 2024-04-20T11:53:24.094 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb61406ec30 msgr2=0x7fb6140710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb61406ec30 0x7fb6140710f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb62002c930 tx=0x7fb62002cc50 comp rx=0 tx=0).stop 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072f60 msgr2=0x7fb63019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072f60 0x7fb63019c730 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b420 tx=0x7fb61c00b8e0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 shutdown_connections 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fb61406ec30 0x7fb6140710f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb630072f60 0x7fb63019c730 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 --2- 172.21.15.117:0/4233485700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb630072640 0x7fb63019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 >> 172.21.15.117:0/4233485700 conn(0x7fb6300fb970 msgr2=0x7fb630102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 shutdown_connections 2024-04-20T11:53:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.096+0000 7fb62fe00640 1 -- 172.21.15.117:0/4233485700 wait complete. 2024-04-20T11:53:24.114 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-20T11:53:24.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.312+0000 7f9a47200640 1 -- 172.21.15.117:0/3310543061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 msgr2=0x7f9a401028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.312+0000 7f9a47200640 1 --2- 172.21.15.117:0/3310543061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 0x7f9a401028a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009880 tx=0x7f9a3002bcf0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.312+0000 7f9a47200640 1 -- 172.21.15.117:0/3310543061 shutdown_connections 2024-04-20T11:53:24.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.312+0000 7f9a47200640 1 --2- 172.21.15.117:0/3310543061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a40102e70 0x7f9a40106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.312+0000 7f9a47200640 1 --2- 172.21.15.117:0/3310543061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 0x7f9a401028a0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.312+0000 7f9a47200640 1 -- 172.21.15.117:0/3310543061 >> 172.21.15.117:0/3310543061 conn(0x7f9a400fdd40 msgr2=0x7f9a40100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:24.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.312+0000 7f9a47200640 1 -- 172.21.15.117:0/3310543061 shutdown_connections 2024-04-20T11:53:24.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.312+0000 7f9a47200640 1 -- 172.21.15.117:0/3310543061 wait complete. 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.313+0000 7f9a47200640 1 Processor -- start 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a47200640 1 -- start start 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a47200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 0x7f9a40198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a47200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a40198670 0x7f9a4019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a47200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a40198c70 con 0x7f9a401024c0 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a47200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a40198de0 con 0x7f9a40198670 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 0x7f9a40198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 0x7f9a40198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41086/0 (socket says 172.21.15.117:41086) 2024-04-20T11:53:24.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a3fe00640 1 -- 172.21.15.117:0/4190788247 learned_addr learned my addr 172.21.15.117:0/4190788247 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a3fe00640 1 -- 172.21.15.117:0/4190788247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a40198670 msgr2=0x7f9a4019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a3fe00640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a40198670 0x7f9a4019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.314+0000 7f9a3fe00640 1 -- 172.21.15.117:0/4190788247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30009560 con 0x7f9a401024c0 2024-04-20T11:53:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.315+0000 7f9a3fe00640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 0x7f9a40198130 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9a30004270 tx=0x7f9a300042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.315+0000 7f9a3cc00640 1 -- 172.21.15.117:0/4190788247 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a300035f0 con 0x7f9a401024c0 2024-04-20T11:53:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.315+0000 7f9a3cc00640 1 -- 172.21.15.117:0/4190788247 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a30003c10 con 0x7f9a401024c0 2024-04-20T11:53:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.315+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4019d0d0 con 0x7f9a401024c0 2024-04-20T11:53:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.315+0000 7f9a3cc00640 1 -- 172.21.15.117:0/4190788247 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a30034440 con 0x7f9a401024c0 2024-04-20T11:53:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.315+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4019d630 con 0x7f9a401024c0 2024-04-20T11:53:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.317+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4004e040 con 0x7f9a401024c0 2024-04-20T11:53:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.317+0000 7f9a3cc00640 1 -- 172.21.15.117:0/4190788247 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9a30005060 con 0x7f9a401024c0 2024-04-20T11:53:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.318+0000 7f9a3cc00640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a2406ec30 0x7f9a240710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.318+0000 7f9a3cc00640 1 -- 172.21.15.117:0/4190788247 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9a300b1350 con 0x7f9a401024c0 2024-04-20T11:53:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.319+0000 7f9a3f400640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a2406ec30 0x7f9a240710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:24.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.319+0000 7f9a3f400640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a2406ec30 0x7f9a240710f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9a40069560 tx=0x7f9a2c006c90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:24.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.324+0000 7f9a3cc00640 1 -- 172.21.15.117:0/4190788247 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7f9a3007a090 con 0x7f9a401024c0 2024-04-20T11:53:24.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.526+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f9a4010a7c0 con 0x7f9a401024c0 2024-04-20T11:53:24.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.527+0000 7f9a3cc00640 1 -- 172.21.15.117:0/4190788247 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9a3007d750 con 0x7f9a401024c0 2024-04-20T11:53:24.526 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-04-20T11:53:24.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a2406ec30 msgr2=0x7f9a240710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a2406ec30 0x7f9a240710f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9a40069560 tx=0x7f9a2c006c90 comp rx=0 tx=0).stop 2024-04-20T11:53:24.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 msgr2=0x7f9a40198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 0x7f9a40198130 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9a30004270 tx=0x7f9a300042a0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 shutdown_connections 2024-04-20T11:53:24.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7f9a2406ec30 0x7f9a240710f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a40198670 0x7f9a4019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 --2- 172.21.15.117:0/4190788247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a401024c0 0x7f9a40198130 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.529+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 >> 172.21.15.117:0/4190788247 conn(0x7f9a400fdd40 msgr2=0x7f9a400ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:24.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.530+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 shutdown_connections 2024-04-20T11:53:24.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.530+0000 7f9a47200640 1 -- 172.21.15.117:0/4190788247 wait complete. 2024-04-20T11:53:24.546 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-20T11:53:24.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.740+0000 7fd061600640 1 -- 172.21.15.117:0/1879684436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c102c90 msgr2=0x7fd05c106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.740+0000 7fd061600640 1 --2- 172.21.15.117:0/1879684436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c102c90 0x7fd05c106cd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009880 tx=0x7fd04c02bcf0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.740+0000 7fd061600640 1 -- 172.21.15.117:0/1879684436 shutdown_connections 2024-04-20T11:53:24.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.740+0000 7fd061600640 1 --2- 172.21.15.117:0/1879684436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c102c90 0x7fd05c106cd0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.740+0000 7fd061600640 1 --2- 172.21.15.117:0/1879684436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd05c1022e0 0x7fd05c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.740+0000 7fd061600640 1 -- 172.21.15.117:0/1879684436 >> 172.21.15.117:0/1879684436 conn(0x7fd05c0fdb60 msgr2=0x7fd05c0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:24.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.740+0000 7fd061600640 1 -- 172.21.15.117:0/1879684436 shutdown_connections 2024-04-20T11:53:24.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.740+0000 7fd061600640 1 -- 172.21.15.117:0/1879684436 wait complete. 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.741+0000 7fd061600640 1 Processor -- start 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.741+0000 7fd061600640 1 -- start start 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.741+0000 7fd061600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd05c1022e0 0x7fd05c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.741+0000 7fd061600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c198480 0x7fd05c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.741+0000 7fd061600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c198a80 con 0x7fd05c198480 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.741+0000 7fd061600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c198bf0 con 0x7fd05c1022e0 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.742+0000 7fd053400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c198480 0x7fd05c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.742+0000 7fd053400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c198480 0x7fd05c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41100/0 (socket says 172.21.15.117:41100) 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.742+0000 7fd053400640 1 -- 172.21.15.117:0/3434968941 learned_addr learned my addr 172.21.15.117:0/3434968941 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:24.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.742+0000 7fd05be00640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd05c1022e0 0x7fd05c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:24.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.742+0000 7fd053400640 1 -- 172.21.15.117:0/3434968941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd05c1022e0 msgr2=0x7fd05c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.742+0000 7fd053400640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd05c1022e0 0x7fd05c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.742+0000 7fd053400640 1 -- 172.21.15.117:0/3434968941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c009560 con 0x7fd05c198480 2024-04-20T11:53:24.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.742+0000 7fd05be00640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd05c1022e0 0x7fd05c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:53:24.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.743+0000 7fd053400640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c198480 0x7fd05c19c8e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd04c036040 tx=0x7fd04c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:24.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.743+0000 7fd059600640 1 -- 172.21.15.117:0/3434968941 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd04c0037c0 con 0x7fd05c198480 2024-04-20T11:53:24.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.743+0000 7fd059600640 1 -- 172.21.15.117:0/3434968941 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd04c003de0 con 0x7fd05c198480 2024-04-20T11:53:24.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.743+0000 7fd059600640 1 -- 172.21.15.117:0/3434968941 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd04c03e460 con 0x7fd05c198480 2024-04-20T11:53:24.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.743+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05c19cee0 con 0x7fd05c198480 2024-04-20T11:53:24.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.743+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c19d440 con 0x7fd05c198480 2024-04-20T11:53:24.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.744+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c04e040 con 0x7fd05c198480 2024-04-20T11:53:24.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.745+0000 7fd059600640 1 -- 172.21.15.117:0/3434968941 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd04c004fd0 con 0x7fd05c198480 2024-04-20T11:53:24.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.747+0000 7fd059600640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd04006ec30 0x7fd0400710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:24.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.747+0000 7fd059600640 1 -- 172.21.15.117:0/3434968941 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd04c0b16c0 con 0x7fd05c198480 2024-04-20T11:53:24.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.747+0000 7fd05be00640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd04006ec30 0x7fd0400710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:24.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.748+0000 7fd05be00640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd04006ec30 0x7fd0400710f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd044006fd0 tx=0x7fd044008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:24.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.752+0000 7fd059600640 1 -- 172.21.15.117:0/3434968941 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fd04c07a520 con 0x7fd05c198480 2024-04-20T11:53:24.865 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:24 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/537237538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:53:24.865 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:24 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4233485700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:53:24.865 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:24 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/4190788247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-20T11:53:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:24 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/537237538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:53:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:24 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4233485700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:53:24.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:24 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/4190788247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-20T11:53:24.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.964+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd05c19d740 con 0x7fd05c198480 2024-04-20T11:53:24.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.965+0000 7fd059600640 1 -- 172.21.15.117:0/3434968941 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd04c07dbe0 con 0x7fd05c198480 2024-04-20T11:53:24.965 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd04006ec30 msgr2=0x7fd0400710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd04006ec30 0x7fd0400710f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd044006fd0 tx=0x7fd044008040 comp rx=0 tx=0).stop 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c198480 msgr2=0x7fd05c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c198480 0x7fd05c19c8e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd04c036040 tx=0x7fd04c004270 comp rx=0 tx=0).stop 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 shutdown_connections 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fd04006ec30 0x7fd0400710f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd05c198480 0x7fd05c19c8e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 --2- 172.21.15.117:0/3434968941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd05c1022e0 0x7fd05c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 >> 172.21.15.117:0/3434968941 conn(0x7fd05c0fdb60 msgr2=0x7fd05c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 shutdown_connections 2024-04-20T11:53:24.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:24.969+0000 7fd061600640 1 -- 172.21.15.117:0/3434968941 wait complete. 2024-04-20T11:53:24.990 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:53:25.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.167+0000 7fadba600640 1 -- 172.21.15.117:0/2548498023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4100c20 msgr2=0x7fadb4104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:25.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.167+0000 7fadba600640 1 --2- 172.21.15.117:0/2548498023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4100c20 0x7fadb4104c60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fada0009880 tx=0x7fada002bcf0 comp rx=0 tx=0).stop 2024-04-20T11:53:25.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.168+0000 7fadba600640 1 -- 172.21.15.117:0/2548498023 shutdown_connections 2024-04-20T11:53:25.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.168+0000 7fadba600640 1 --2- 172.21.15.117:0/2548498023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4100c20 0x7fadb4104c60 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:25.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.168+0000 7fadba600640 1 --2- 172.21.15.117:0/2548498023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadb4100270 0x7fadb4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:25.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.168+0000 7fadba600640 1 -- 172.21.15.117:0/2548498023 >> 172.21.15.117:0/2548498023 conn(0x7fadb40fbad0 msgr2=0x7fadb40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:25.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.168+0000 7fadba600640 1 -- 172.21.15.117:0/2548498023 shutdown_connections 2024-04-20T11:53:25.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.168+0000 7fadba600640 1 -- 172.21.15.117:0/2548498023 wait complete. 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadba600640 1 Processor -- start 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadba600640 1 -- start start 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadba600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadb4100270 0x7fadb4195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadba600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4196370 0x7fadb419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadba600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb4196970 con 0x7fadb4196370 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadba600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb4196ae0 con 0x7fadb4100270 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadb2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4196370 0x7fadb419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadb2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4196370 0x7fadb419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41122/0 (socket says 172.21.15.117:41122) 2024-04-20T11:53:25.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.169+0000 7fadb2a00640 1 -- 172.21.15.117:0/585353229 learned_addr learned my addr 172.21.15.117:0/585353229 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadb2a00640 1 -- 172.21.15.117:0/585353229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadb4100270 msgr2=0x7fadb4195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadb2a00640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadb4100270 0x7fadb4195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadb2a00640 1 -- 172.21.15.117:0/585353229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada0009560 con 0x7fadb4196370 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadb2a00640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4196370 0x7fadb419a7d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fada0036040 tx=0x7fada0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadabe00640 1 -- 172.21.15.117:0/585353229 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fada00037c0 con 0x7fadb4196370 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb419add0 con 0x7fadb4196370 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadabe00640 1 -- 172.21.15.117:0/585353229 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fada0003de0 con 0x7fadb4196370 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadabe00640 1 -- 172.21.15.117:0/585353229 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fada003e460 con 0x7fadb4196370 2024-04-20T11:53:25.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.170+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb419b330 con 0x7fadb4196370 2024-04-20T11:53:25.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.171+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb404e040 con 0x7fadb4196370 2024-04-20T11:53:25.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.172+0000 7fadabe00640 1 -- 172.21.15.117:0/585353229 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 40) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fada0004fd0 con 0x7fadb4196370 2024-04-20T11:53:25.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.173+0000 7fadabe00640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad9806ecf0 0x7fad980711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:53:25.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.173+0000 7fadabe00640 1 -- 172.21.15.117:0/585353229 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fada00b1830 con 0x7fadb4196370 2024-04-20T11:53:25.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.173+0000 7fadb3400640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad9806ecf0 0x7fad980711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:53:25.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.174+0000 7fadb3400640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad9806ecf0 0x7fad980711b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fada4009710 tx=0x7fada4006c90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:53:25.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.178+0000 7fadabe00640 1 -- 172.21.15.117:0/585353229 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188381 (secure 0 0 0) 0x7fada007a570 con 0x7fadb4196370 2024-04-20T11:53:25.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.382+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fadb419b4c0 con 0x7fadb4196370 2024-04-20T11:53:25.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.383+0000 7fadabe00640 1 -- 172.21.15.117:0/585353229 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 38 entries v86) v1 ==== 81+0+2283 (secure 0 0 0) 0x7fada007dc30 con 0x7fadb4196370 2024-04-20T11:53:25.383 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:6825/3563933435 2024-04-21T11:19:07.882600+0000 2024-04-20T11:53:25.383 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:6824/3563933435 2024-04-21T11:19:07.882600+0000 2024-04-20T11:53:25.383 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/1016803704 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:25.383 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/3713452282 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:25.383 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/519807094 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2304890710 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/4182681741 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/571056469 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1807203325 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1384022397 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2961758989 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/1996106837 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/1996106837 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/997010605 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2597372910 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2230602257 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1375611439 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2096654470 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1089633358 2024-04-21T10:58:13.447117+0000 2024-04-20T11:53:25.384 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:6829/2992581883 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2612478075 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/638889142 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/3679484323 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/4088938050 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/3475739843 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/894992429 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2597372910 2024-04-21T10:56:46.887330+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6826/361271135 2024-04-21T11:19:07.882570+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/1432319225 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/1432319225 2024-04-21T10:56:04.789705+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:6828/2992581883 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2612478075 2024-04-21T11:07:41.749629+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1003331392 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/3003776987 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/3651098074 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:25.385 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/3003776987 2024-04-21T11:13:42.885571+0000 2024-04-20T11:53:25.386 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.173:0/2377885188 2024-04-21T11:08:45.884891+0000 2024-04-20T11:53:25.386 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6827/361271135 2024-04-21T11:19:07.882570+0000 2024-04-20T11:53:25.386 INFO:teuthology.orchestra.run.smithi117.stderr:listed 38 entries 2024-04-20T11:53:25.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad9806ecf0 msgr2=0x7fad980711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad9806ecf0 0x7fad980711b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fada4009710 tx=0x7fada4006c90 comp rx=0 tx=0).stop 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4196370 msgr2=0x7fadb419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4196370 0x7fadb419a7d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fada0036040 tx=0x7fada0004270 comp rx=0 tx=0).stop 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 shutdown_connections 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:6800/2604153943,v1:172.21.15.117:6801/2604153943] conn(0x7fad9806ecf0 0x7fad980711b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb4196370 0x7fadb419a7d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 --2- 172.21.15.117:0/585353229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadb4100270 0x7fadb4195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 >> 172.21.15.117:0/585353229 conn(0x7fadb40fbad0 msgr2=0x7fadb40fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.386+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 shutdown_connections 2024-04-20T11:53:25.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-20T11:53:25.387+0000 7fadba600640 1 -- 172.21.15.117:0/585353229 wait complete. 2024-04-20T11:53:25.403 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi117.front.sepia.ceph.com... 2024-04-20T11:53:25.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:53:25.403 DEBUG:teuthology.orchestra.run.smithi117:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T11:53:25.484 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:53:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:25 smithi117 ceph-mon[99036]: pgmap v1231: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:25 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/3434968941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-20T11:53:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:25 smithi117 ceph-mon[99036]: from='client.? 172.21.15.117:0/585353229' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T11:53:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:25 smithi173 ceph-mon[74165]: pgmap v1231: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:25 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/3434968941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-20T11:53:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:25 smithi173 ceph-mon[74165]: from='client.? 172.21.15.117:0/585353229' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T11:53:27.856 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:27 smithi173 ceph-mon[74165]: pgmap v1232: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:27.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:27 smithi117 ceph-mon[99036]: pgmap v1232: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:53:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:53:29.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:29 smithi117 ceph-mon[99036]: pgmap v1233: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:29 smithi173 ceph-mon[74165]: pgmap v1233: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:31.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:31 smithi117 ceph-mon[99036]: pgmap v1234: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:31.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:31 smithi173 ceph-mon[74165]: pgmap v1234: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:33 smithi117 ceph-mon[99036]: pgmap v1235: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:33 smithi173 ceph-mon[74165]: pgmap v1235: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:35 smithi117 ceph-mon[99036]: pgmap v1236: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:35 smithi173 ceph-mon[74165]: pgmap v1236: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:37.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:37 smithi117 ceph-mon[99036]: pgmap v1237: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:37.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:37 smithi173 ceph-mon[74165]: pgmap v1237: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:39.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:39 smithi117 ceph-mon[99036]: pgmap v1238: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:39.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:39 smithi173 ceph-mon[74165]: pgmap v1238: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:41.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:41 smithi117 ceph-mon[99036]: pgmap v1239: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:41 smithi173 ceph-mon[74165]: pgmap v1239: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:43 smithi117 ceph-mon[99036]: pgmap v1240: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:53:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:43 smithi173 ceph-mon[74165]: pgmap v1240: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:53:44.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:44 smithi117 ceph-mon[99036]: pgmap v1241: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:44.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:44 smithi173 ceph-mon[74165]: pgmap v1241: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:47.139 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:46 smithi117 ceph-mon[99036]: pgmap v1242: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:47.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:46 smithi173 ceph-mon[74165]: pgmap v1242: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:49.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:48 smithi117 ceph-mon[99036]: pgmap v1243: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:49.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:48 smithi173 ceph-mon[74165]: pgmap v1243: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:50.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:50 smithi117 ceph-mon[99036]: pgmap v1244: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:51.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:50 smithi173 ceph-mon[74165]: pgmap v1244: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:53.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:52 smithi117 ceph-mon[99036]: pgmap v1245: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:53.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:52 smithi173 ceph-mon[74165]: pgmap v1245: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:55.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:54 smithi117 ceph-mon[99036]: pgmap v1246: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:55.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:54 smithi173 ceph-mon[74165]: pgmap v1246: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:57.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:56 smithi117 ceph-mon[99036]: pgmap v1247: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:56 smithi173 ceph-mon[74165]: pgmap v1247: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:59.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:58 smithi117 ceph-mon[99036]: pgmap v1248: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:59.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:53:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:53:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:58 smithi173 ceph-mon[74165]: pgmap v1248: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:53:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:53:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:54:01.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:00 smithi117 ceph-mon[99036]: pgmap v1249: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:01.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:00 smithi173 ceph-mon[74165]: pgmap v1249: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:03.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:02 smithi117 ceph-mon[99036]: pgmap v1250: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:03.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:02 smithi173 ceph-mon[74165]: pgmap v1250: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:05.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:04 smithi117 ceph-mon[99036]: pgmap v1251: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:05.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:04 smithi173 ceph-mon[74165]: pgmap v1251: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:07.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:06 smithi117 ceph-mon[99036]: pgmap v1252: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:07.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:06 smithi173 ceph-mon[74165]: pgmap v1252: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:09.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:08 smithi117 ceph-mon[99036]: pgmap v1253: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:09.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:08 smithi173 ceph-mon[74165]: pgmap v1253: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:10.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:09 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:54:10.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:09 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:54:11.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:10 smithi117 ceph-mon[99036]: pgmap v1254: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:11.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:54:11.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:54:11.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:10 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:54:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:10 smithi173 ceph-mon[74165]: pgmap v1254: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:54:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:54:11.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:10 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:54:13.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:12 smithi173 ceph-mon[74165]: pgmap v1255: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:12 smithi117 ceph-mon[99036]: pgmap v1255: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:14.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:54:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:54:15.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:14 smithi173 ceph-mon[74165]: pgmap v1256: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:15.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:14 smithi117 ceph-mon[99036]: pgmap v1256: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:17.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:16 smithi117 ceph-mon[99036]: pgmap v1257: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:17.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:16 smithi173 ceph-mon[74165]: pgmap v1257: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:19.346 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:18 smithi173 ceph-mon[74165]: pgmap v1258: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:18 smithi117 ceph-mon[99036]: pgmap v1258: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:20 smithi117 ceph-mon[99036]: pgmap v1259: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:21.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:20 smithi173 ceph-mon[74165]: pgmap v1259: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:23.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:22 smithi117 ceph-mon[99036]: pgmap v1260: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:23.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:22 smithi173 ceph-mon[74165]: pgmap v1260: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:25.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:24 smithi117 ceph-mon[99036]: pgmap v1261: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:24 smithi173 ceph-mon[74165]: pgmap v1261: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:27.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:26 smithi117 ceph-mon[99036]: pgmap v1262: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:27.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:26 smithi173 ceph-mon[74165]: pgmap v1262: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:28 smithi117 ceph-mon[99036]: pgmap v1263: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:29.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:54:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:28 smithi173 ceph-mon[74165]: pgmap v1263: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:29.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:54:31.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:30 smithi117 ceph-mon[99036]: pgmap v1264: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:31.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:30 smithi173 ceph-mon[74165]: pgmap v1264: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:33.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:32 smithi117 ceph-mon[99036]: pgmap v1265: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:33.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:32 smithi173 ceph-mon[74165]: pgmap v1265: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:35.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:34 smithi173 ceph-mon[74165]: pgmap v1266: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:35.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:34 smithi117 ceph-mon[99036]: pgmap v1266: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:37.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:36 smithi117 ceph-mon[99036]: pgmap v1267: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:37.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:36 smithi173 ceph-mon[74165]: pgmap v1267: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:39.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:38 smithi117 ceph-mon[99036]: pgmap v1268: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:39.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:38 smithi173 ceph-mon[74165]: pgmap v1268: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:41.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:41 smithi117 ceph-mon[99036]: pgmap v1269: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:41.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:41 smithi173 ceph-mon[74165]: pgmap v1269: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:43.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:43 smithi117 ceph-mon[99036]: pgmap v1270: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:43.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:43 smithi173 ceph-mon[74165]: pgmap v1270: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:44.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:44 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:54:44.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:44 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:54:45.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:45 smithi117 ceph-mon[99036]: pgmap v1271: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:45.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:45 smithi173 ceph-mon[74165]: pgmap v1271: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:47.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:47 smithi117 ceph-mon[99036]: pgmap v1272: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:47.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:47 smithi173 ceph-mon[74165]: pgmap v1272: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:49.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:49 smithi117 ceph-mon[99036]: pgmap v1273: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:49.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:49 smithi173 ceph-mon[74165]: pgmap v1273: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:51.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:51 smithi117 ceph-mon[99036]: pgmap v1274: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:51.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:51 smithi173 ceph-mon[74165]: pgmap v1274: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:53.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:53 smithi117 ceph-mon[99036]: pgmap v1275: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:53.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:53 smithi173 ceph-mon[74165]: pgmap v1275: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:55.379 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:55 smithi173 ceph-mon[74165]: pgmap v1276: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:55.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:55 smithi117 ceph-mon[99036]: pgmap v1276: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:57.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:57 smithi117 ceph-mon[99036]: pgmap v1277: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:57.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:57 smithi173 ceph-mon[74165]: pgmap v1277: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:59 smithi117 ceph-mon[99036]: pgmap v1278: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:54:59 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:54:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:59 smithi173 ceph-mon[74165]: pgmap v1278: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:54:59.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:54:59 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:01 smithi117 ceph-mon[99036]: pgmap v1279: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:01.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:01 smithi173 ceph-mon[74165]: pgmap v1279: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:03 smithi117 ceph-mon[99036]: pgmap v1280: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:03 smithi173 ceph-mon[74165]: pgmap v1280: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:05 smithi117 ceph-mon[99036]: pgmap v1281: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:05 smithi173 ceph-mon[74165]: pgmap v1281: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:07.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:07 smithi117 ceph-mon[99036]: pgmap v1282: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:07.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:07 smithi173 ceph-mon[74165]: pgmap v1282: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:09.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:09 smithi117 ceph-mon[99036]: pgmap v1283: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:09.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:09 smithi173 ceph-mon[74165]: pgmap v1283: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:11.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:11 smithi117 ceph-mon[99036]: pgmap v1284: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:11.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:11 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:55:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:11 smithi173 ceph-mon[74165]: pgmap v1284: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:11 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:55:13.131 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:13 smithi117 ceph-mon[99036]: pgmap v1285: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:13.131 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:55:13.131 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:55:13.131 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:55:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:13 smithi173 ceph-mon[74165]: pgmap v1285: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:55:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:55:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:55:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:15.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:15 smithi117 ceph-mon[99036]: pgmap v1286: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:15.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:15 smithi173 ceph-mon[74165]: pgmap v1286: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:17.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:17 smithi117 ceph-mon[99036]: pgmap v1287: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:17.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:17 smithi173 ceph-mon[74165]: pgmap v1287: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:19.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:19 smithi117 ceph-mon[99036]: pgmap v1288: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:19.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:19 smithi173 ceph-mon[74165]: pgmap v1288: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:21.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:21 smithi117 ceph-mon[99036]: pgmap v1289: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:21.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:21 smithi173 ceph-mon[74165]: pgmap v1289: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:23.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:23 smithi117 ceph-mon[99036]: pgmap v1290: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:23.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:23 smithi173 ceph-mon[74165]: pgmap v1290: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:25.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:25 smithi173 ceph-mon[74165]: pgmap v1291: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:25 smithi117 ceph-mon[99036]: pgmap v1291: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:27.414 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:27 smithi173 ceph-mon[74165]: pgmap v1292: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:27.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:27 smithi117 ceph-mon[99036]: pgmap v1292: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:28.639 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:29.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:29 smithi117 ceph-mon[99036]: pgmap v1293: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:29 smithi173 ceph-mon[74165]: pgmap v1293: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:31.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:31 smithi117 ceph-mon[99036]: pgmap v1294: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:31 smithi173 ceph-mon[74165]: pgmap v1294: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:33.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:33 smithi117 ceph-mon[99036]: pgmap v1295: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:33 smithi173 ceph-mon[74165]: pgmap v1295: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:35.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:35 smithi117 ceph-mon[99036]: pgmap v1296: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:35 smithi173 ceph-mon[74165]: pgmap v1296: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:37.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:37 smithi117 ceph-mon[99036]: pgmap v1297: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:37 smithi173 ceph-mon[74165]: pgmap v1297: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:39.464 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:39 smithi173 ceph-mon[74165]: pgmap v1298: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:39.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:39 smithi117 ceph-mon[99036]: pgmap v1298: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:41.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:41 smithi117 ceph-mon[99036]: pgmap v1299: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:41.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:41 smithi173 ceph-mon[74165]: pgmap v1299: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:43 smithi117 ceph-mon[99036]: pgmap v1300: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:43.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:43 smithi173 ceph-mon[74165]: pgmap v1300: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:43.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:45.642 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:45 smithi117 ceph-mon[99036]: pgmap v1301: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:45.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:45 smithi173 ceph-mon[74165]: pgmap v1301: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:47.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:47 smithi117 ceph-mon[99036]: pgmap v1302: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:47.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:47 smithi173 ceph-mon[74165]: pgmap v1302: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:49.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:49 smithi117 ceph-mon[99036]: pgmap v1303: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:49.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:49 smithi173 ceph-mon[74165]: pgmap v1303: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:51.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:51 smithi117 ceph-mon[99036]: pgmap v1304: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:51.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:51 smithi173 ceph-mon[74165]: pgmap v1304: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:53.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:53 smithi117 ceph-mon[99036]: pgmap v1305: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:53.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:53 smithi173 ceph-mon[74165]: pgmap v1305: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:55.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:55 smithi117 ceph-mon[99036]: pgmap v1306: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:55.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:55 smithi173 ceph-mon[74165]: pgmap v1306: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:57.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:57 smithi173 ceph-mon[74165]: pgmap v1307: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:57.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:57 smithi117 ceph-mon[99036]: pgmap v1307: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:58.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:58.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:55:59.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:55:59 smithi117 ceph-mon[99036]: pgmap v1308: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:55:59.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:55:59 smithi173 ceph-mon[74165]: pgmap v1308: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:01.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:01 smithi117 ceph-mon[99036]: pgmap v1309: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:01.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:01 smithi173 ceph-mon[74165]: pgmap v1309: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:03.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:03 smithi117 ceph-mon[99036]: pgmap v1310: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:03.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:03 smithi173 ceph-mon[74165]: pgmap v1310: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:05.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:05 smithi117 ceph-mon[99036]: pgmap v1311: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:05.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:05 smithi173 ceph-mon[74165]: pgmap v1311: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:07.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:07 smithi117 ceph-mon[99036]: pgmap v1312: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:07.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:07 smithi173 ceph-mon[74165]: pgmap v1312: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:09.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:09 smithi117 ceph-mon[99036]: pgmap v1313: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:09.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:09 smithi173 ceph-mon[74165]: pgmap v1313: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:11.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:11 smithi117 ceph-mon[99036]: pgmap v1314: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:11.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:11 smithi173 ceph-mon[74165]: pgmap v1314: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:13 smithi117 ceph-mon[99036]: pgmap v1315: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:56:13.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:56:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:13 smithi173 ceph-mon[74165]: pgmap v1315: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:56:13.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:56:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:56:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:56:14.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:56:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:56:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:56:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:56:15.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:15 smithi117 ceph-mon[99036]: pgmap v1316: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:15.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:15 smithi173 ceph-mon[74165]: pgmap v1316: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:17.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:17 smithi117 ceph-mon[99036]: pgmap v1317: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:17.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:17 smithi173 ceph-mon[74165]: pgmap v1317: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:19 smithi117 ceph-mon[99036]: pgmap v1318: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:19 smithi173 ceph-mon[74165]: pgmap v1318: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:21 smithi117 ceph-mon[99036]: pgmap v1319: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:21 smithi173 ceph-mon[74165]: pgmap v1319: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:23.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:23 smithi117 ceph-mon[99036]: pgmap v1320: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:23 smithi173 ceph-mon[74165]: pgmap v1320: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:25 smithi117 ceph-mon[99036]: pgmap v1321: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:25 smithi173 ceph-mon[74165]: pgmap v1321: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:27.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:27 smithi117 ceph-mon[99036]: pgmap v1322: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:27 smithi173 ceph-mon[74165]: pgmap v1322: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:28.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:56:28.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:56:29.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:29 smithi117 ceph-mon[99036]: pgmap v1323: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:29.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:29 smithi173 ceph-mon[74165]: pgmap v1323: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:31.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:31 smithi117 ceph-mon[99036]: pgmap v1324: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:31.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:31 smithi173 ceph-mon[74165]: pgmap v1324: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:33.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:33 smithi117 ceph-mon[99036]: pgmap v1325: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:33.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:33 smithi173 ceph-mon[74165]: pgmap v1325: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:35.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:35 smithi173 ceph-mon[74165]: pgmap v1326: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:35 smithi117 ceph-mon[99036]: pgmap v1326: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:37.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:37 smithi173 ceph-mon[74165]: pgmap v1327: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:37.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:37 smithi117 ceph-mon[99036]: pgmap v1327: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:39.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:39 smithi173 ceph-mon[74165]: pgmap v1328: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:39.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:39 smithi117 ceph-mon[99036]: pgmap v1328: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:41.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:41 smithi117 ceph-mon[99036]: pgmap v1329: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:41 smithi173 ceph-mon[74165]: pgmap v1329: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:43 smithi117 ceph-mon[99036]: pgmap v1330: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:56:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:43 smithi173 ceph-mon[74165]: pgmap v1330: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:56:45.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:45 smithi117 ceph-mon[99036]: pgmap v1331: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:45 smithi173 ceph-mon[74165]: pgmap v1331: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:47.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:47 smithi117 ceph-mon[99036]: pgmap v1332: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:47.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:47 smithi173 ceph-mon[74165]: pgmap v1332: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:49.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:49 smithi117 ceph-mon[99036]: pgmap v1333: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:49 smithi173 ceph-mon[74165]: pgmap v1333: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:51.745 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:51 smithi117 ceph-mon[99036]: pgmap v1334: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:51 smithi173 ceph-mon[74165]: pgmap v1334: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:53.775 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:53 smithi173 ceph-mon[74165]: pgmap v1335: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:53.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:53 smithi117 ceph-mon[99036]: pgmap v1335: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:55.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:55 smithi117 ceph-mon[99036]: pgmap v1336: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:55 smithi173 ceph-mon[74165]: pgmap v1336: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:57.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:57 smithi117 ceph-mon[99036]: pgmap v1337: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:57.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:57 smithi173 ceph-mon[74165]: pgmap v1337: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:58.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:56:58.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:56:59.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:56:59 smithi117 ceph-mon[99036]: pgmap v1338: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:56:59.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:56:59 smithi173 ceph-mon[74165]: pgmap v1338: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:01.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:01 smithi117 ceph-mon[99036]: pgmap v1339: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:01.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:01 smithi173 ceph-mon[74165]: pgmap v1339: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:03.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:03 smithi117 ceph-mon[99036]: pgmap v1340: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:03.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:03 smithi173 ceph-mon[74165]: pgmap v1340: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:05.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:05 smithi117 ceph-mon[99036]: pgmap v1341: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:05.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:05 smithi173 ceph-mon[74165]: pgmap v1341: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:07.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:07 smithi117 ceph-mon[99036]: pgmap v1342: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:07.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:07 smithi173 ceph-mon[74165]: pgmap v1342: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:09.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:09 smithi117 ceph-mon[99036]: pgmap v1343: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:09.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:09 smithi173 ceph-mon[74165]: pgmap v1343: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:11.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:11 smithi117 ceph-mon[99036]: pgmap v1344: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:11.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:11 smithi173 ceph-mon[74165]: pgmap v1344: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:13 smithi117 ceph-mon[99036]: pgmap v1345: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:13.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:57:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:13 smithi173 ceph-mon[74165]: pgmap v1345: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:13.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:57:14.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:14 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:57:14.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:14 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:57:15.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:15 smithi117 ceph-mon[99036]: pgmap v1346: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:15.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:15 smithi173 ceph-mon[74165]: pgmap v1346: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:57:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:57:16.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:57:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:57:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:57:16.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:57:17.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:17 smithi117 ceph-mon[99036]: pgmap v1347: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:17.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:17 smithi173 ceph-mon[74165]: pgmap v1347: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:19.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:19 smithi117 ceph-mon[99036]: pgmap v1348: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:19.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:19 smithi173 ceph-mon[74165]: pgmap v1348: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:21.889 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:21 smithi117 ceph-mon[99036]: pgmap v1349: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:21.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:21 smithi173 ceph-mon[74165]: pgmap v1349: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:23.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:23 smithi117 ceph-mon[99036]: pgmap v1350: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:23.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:23 smithi173 ceph-mon[74165]: pgmap v1350: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:25.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:25 smithi117 ceph-mon[99036]: pgmap v1351: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:25.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:25 smithi173 ceph-mon[74165]: pgmap v1351: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:27.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:27 smithi117 ceph-mon[99036]: pgmap v1352: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:27.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:27 smithi173 ceph-mon[74165]: pgmap v1352: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:28.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:28 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:57:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:28 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:57:29.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:29 smithi117 ceph-mon[99036]: pgmap v1353: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:29.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:29 smithi173 ceph-mon[74165]: pgmap v1353: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:31.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:31 smithi117 ceph-mon[99036]: pgmap v1354: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:31.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:31 smithi173 ceph-mon[74165]: pgmap v1354: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:33.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:33 smithi117 ceph-mon[99036]: pgmap v1355: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:33.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:33 smithi173 ceph-mon[74165]: pgmap v1355: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:35.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:35 smithi117 ceph-mon[99036]: pgmap v1356: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:35.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:35 smithi173 ceph-mon[74165]: pgmap v1356: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:37.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:37 smithi117 ceph-mon[99036]: pgmap v1357: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:37.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:37 smithi173 ceph-mon[74165]: pgmap v1357: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:39.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:39 smithi117 ceph-mon[99036]: pgmap v1358: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:39.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:39 smithi173 ceph-mon[74165]: pgmap v1358: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:41.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:41 smithi117 ceph-mon[99036]: pgmap v1359: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:41.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:41 smithi173 ceph-mon[74165]: pgmap v1359: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:43 smithi117 ceph-mon[99036]: pgmap v1360: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:43.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:43 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:57:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:43 smithi173 ceph-mon[74165]: pgmap v1360: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:43.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:43 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:57:45.890 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:45 smithi117 ceph-mon[99036]: pgmap v1361: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:45.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:45 smithi173 ceph-mon[74165]: pgmap v1361: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:47.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:47 smithi173 ceph-mon[74165]: pgmap v1362: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:48.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:47 smithi117 ceph-mon[99036]: pgmap v1362: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:49.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:49 smithi173 ceph-mon[74165]: pgmap v1363: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:50.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:49 smithi117 ceph-mon[99036]: pgmap v1363: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:51.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:51 smithi173 ceph-mon[74165]: pgmap v1364: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:52.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:51 smithi117 ceph-mon[99036]: pgmap v1364: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:53.942 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:53 smithi117 ceph-mon[99036]: pgmap v1365: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:54.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:53 smithi173 ceph-mon[74165]: pgmap v1365: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:55.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:55 smithi173 ceph-mon[74165]: pgmap v1366: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:56.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:55 smithi117 ceph-mon[99036]: pgmap v1366: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:57.140 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:56 smithi117 ceph-mon[99036]: pgmap v1367: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:57.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:56 smithi173 ceph-mon[74165]: pgmap v1367: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:58 smithi173 ceph-mon[74165]: pgmap v1368: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:59.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:57:58 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:57:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:58 smithi117 ceph-mon[99036]: pgmap v1368: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:57:59.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:57:58 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:58:01.161 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:00 smithi173 ceph-mon[74165]: pgmap v1369: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:01.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:00 smithi117 ceph-mon[99036]: pgmap v1369: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:03.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:02 smithi117 ceph-mon[99036]: pgmap v1370: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:03.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:02 smithi173 ceph-mon[74165]: pgmap v1370: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:05.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:04 smithi117 ceph-mon[99036]: pgmap v1371: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:05.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:04 smithi173 ceph-mon[74165]: pgmap v1371: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:07.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:06 smithi117 ceph-mon[99036]: pgmap v1372: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:07.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:06 smithi173 ceph-mon[74165]: pgmap v1372: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:09.214 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:08 smithi173 ceph-mon[74165]: pgmap v1373: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:09.355 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:08 smithi117 ceph-mon[99036]: pgmap v1373: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:11.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:10 smithi117 ceph-mon[99036]: pgmap v1374: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:11.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:10 smithi173 ceph-mon[74165]: pgmap v1374: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:13.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:12 smithi117 ceph-mon[99036]: pgmap v1375: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:13.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:12 smithi173 ceph-mon[74165]: pgmap v1375: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:14.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:13 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:58:14.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:13 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-20T11:58:15.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:14 smithi117 ceph-mon[99036]: pgmap v1376: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:15.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:14 smithi173 ceph-mon[74165]: pgmap v1376: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:16.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:16 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:58:16.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:16 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:58:17.389 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:17 smithi117 ceph-mon[99036]: pgmap v1377: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:17.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:17 smithi173 ceph-mon[74165]: pgmap v1377: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:18 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:58:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:18 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:58:18.390 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:18 smithi117 ceph-mon[99036]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:58:18.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:18 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:58:18.452 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:18 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:58:18.452 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:18 smithi173 ceph-mon[74165]: from='mgr.34100 172.21.15.117:0/3966707786' entity='mgr.smithi117.ivnklj' 2024-04-20T11:58:19.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:19 smithi117 ceph-mon[99036]: pgmap v1378: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:19.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:19 smithi173 ceph-mon[74165]: pgmap v1378: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:21.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:21 smithi117 ceph-mon[99036]: pgmap v1379: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:21.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:21 smithi173 ceph-mon[74165]: pgmap v1379: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:23.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:23 smithi117 ceph-mon[99036]: pgmap v1380: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:23.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:23 smithi173 ceph-mon[74165]: pgmap v1380: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:24.553 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-20T11:58:24.554 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-20T11:58:24.556 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-20T11:58:24.567 INFO:tasks.cephadm:Teardown begin 2024-04-20T11:58:24.567 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/cephadm.py", line 2246, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-20T11:58:24.569 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:58:24.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:58:24.759 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-20T11:58:24.759 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T11:58:24.816 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T11:58:24.873 INFO:tasks.cephadm:Stopping all daemons... 2024-04-20T11:58:24.873 INFO:tasks.cephadm.mon.smithi117:Stopping mon.smithi117... 2024-04-20T11:58:24.873 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117 2024-04-20T11:58:25.294 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:24 smithi117 systemd[1]: Stopping Ceph mon.smithi117 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:58:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:25 smithi117 ceph-mon[99036]: pgmap v1381: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:25 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:58:25.493+0000 7fe6adc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi117 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:58:25.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:25 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117[99032]: 2024-04-20T11:58:25.493+0000 7fe6adc00700 -1 mon.smithi117@0(leader) e3 *** Got Signal Terminated *** 2024-04-20T11:58:25.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:25 smithi173 ceph-mon[74165]: pgmap v1381: 65 pgs: 65 active+clean; 260 MiB data, 1014 MiB used, 535 GiB / 536 GiB avail 2024-04-20T11:58:26.136 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:25 smithi117 podman[154886]: 2024-04-20 11:58:25.792317046 +0000 UTC m=+0.441075528 container died ec2a5c1b3288fdfd9027bd7a90dadf1878982e5c340a46661df012b1660c7540 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-20T11:58:26.386 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:26 smithi117 podman[154886]: 2024-04-20 11:58:26.352923851 +0000 UTC m=+1.001682326 container cleanup ec2a5c1b3288fdfd9027bd7a90dadf1878982e5c340a46661df012b1660c7540 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T11:58:26.387 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:26 smithi117 bash[154886]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117 2024-04-20T11:58:26.640 INFO:journalctl@ceph.mon.smithi117.smithi117.stdout:Apr 20 11:58:26 smithi117 podman[154910]: 2024-04-20 11:58:26.546353834 +0000 UTC m=+0.744801685 container remove ec2a5c1b3288fdfd9027bd7a90dadf1878982e5c340a46661df012b1660c7540 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi117, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:58:27.007 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi117.service' 2024-04-20T11:58:27.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:58:27.132 INFO:tasks.cephadm.mon.smithi117:Stopped mon.smithi117 2024-04-20T11:58:27.132 INFO:tasks.cephadm.mon.smithi173:Stopping mon.smithi173... 2024-04-20T11:58:27.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi173 2024-04-20T11:58:27.661 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:27 smithi173 systemd[1]: Stopping Ceph mon.smithi173 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:58:28.132 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:27 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173[74161]: 2024-04-20T11:58:27.690+0000 7f6b74c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi173 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:58:28.132 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:27 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173[74161]: 2024-04-20T11:58:27.690+0000 7f6b74c00700 -1 mon.smithi173@1(peon) e3 *** Got Signal Terminated *** 2024-04-20T11:58:28.132 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:27 smithi173 podman[110445]: 2024-04-20 11:58:27.783721454 +0000 UTC m=+0.222797079 container died 32e7ccffcc30e223f211ce00e99726691fa9b1ae90206ff7ae3d46f5414bb4fe (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-20T11:58:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:28 smithi173 podman[110445]: 2024-04-20 11:58:28.343432181 +0000 UTC m=+0.782507799 container cleanup 32e7ccffcc30e223f211ce00e99726691fa9b1ae90206ff7ae3d46f5414bb4fe (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:58:28.411 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:28 smithi173 bash[110445]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173 2024-04-20T11:58:28.911 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Apr 20 11:58:28 smithi173 podman[110459]: 2024-04-20 11:58:28.603754863 +0000 UTC m=+0.805626090 container remove 32e7ccffcc30e223f211ce00e99726691fa9b1ae90206ff7ae3d46f5414bb4fe (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-mon-smithi173, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-20T11:58:29.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@mon.smithi173.service' 2024-04-20T11:58:29.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:58:29.241 INFO:tasks.cephadm.mon.smithi173:Stopped mon.smithi173 2024-04-20T11:58:29.241 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-20T11:58:29.242 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.0 2024-04-20T11:58:29.639 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:29 smithi117 systemd[1]: Stopping Ceph osd.0 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:58:30.140 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:29 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[110451]: 2024-04-20T11:58:29.770+0000 7f5c13000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:58:30.140 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:29 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[110451]: 2024-04-20T11:58:29.770+0000 7f5c13000700 -1 osd.0 86 *** Got signal Terminated *** 2024-04-20T11:58:30.141 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:29 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0[110451]: 2024-04-20T11:58:29.770+0000 7f5c13000700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:58:35.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:35 smithi117 podman[155025]: 2024-04-20 11:58:35.231929042 +0000 UTC m=+5.606066525 container died 8eceb629d7d297ad6b5f25559466c6e783dc7babbe296e309a98290518deca11 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD) 2024-04-20T11:58:35.890 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:35 smithi117 podman[155025]: 2024-04-20 11:58:35.674136367 +0000 UTC m=+6.048273871 container cleanup 8eceb629d7d297ad6b5f25559466c6e783dc7babbe296e309a98290518deca11 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:58:35.890 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:35 smithi117 bash[155025]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0 2024-04-20T11:58:36.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:36 smithi117 podman[155044]: 2024-04-20 11:58:36.000417071 +0000 UTC m=+0.754569865 container remove 8eceb629d7d297ad6b5f25559466c6e783dc7babbe296e309a98290518deca11 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-20T11:58:37.125 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:36 smithi117 podman[155135]: 2024-04-20 11:58:36.69625863 +0000 UTC m=+0.032191854 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:58:37.125 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:36 smithi117 podman[155135]: 2024-04-20 11:58:36.80972028 +0000 UTC m=+0.145653524 container create 93198cff42890aa8062996798d1bf4924534c16d1a904ebd5e2d64c558aaa0d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:58:37.125 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:37 smithi117 podman[155135]: 2024-04-20 11:58:37.122661698 +0000 UTC m=+0.458594978 container init 93198cff42890aa8062996798d1bf4924534c16d1a904ebd5e2d64c558aaa0d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:58:37.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:37 smithi117 podman[155135]: 2024-04-20 11:58:37.135735205 +0000 UTC m=+0.471668451 container start 93198cff42890aa8062996798d1bf4924534c16d1a904ebd5e2d64c558aaa0d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:58:37.390 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:37 smithi117 podman[155135]: 2024-04-20 11:58:37.242784735 +0000 UTC m=+0.578717998 container attach 93198cff42890aa8062996798d1bf4924534c16d1a904ebd5e2d64c558aaa0d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-20T11:58:37.890 INFO:journalctl@ceph.osd.0.smithi117.stdout:Apr 20 11:58:37 smithi117 podman[155135]: 2024-04-20 11:58:37.593233069 +0000 UTC m=+0.929166359 container died 93198cff42890aa8062996798d1bf4924534c16d1a904ebd5e2d64c558aaa0d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:58:38.380 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.0.service' 2024-04-20T11:58:38.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:58:38.499 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-20T11:58:38.499 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-20T11:58:38.499 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.1 2024-04-20T11:58:38.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:38 smithi117 systemd[1]: Stopping Ceph osd.1 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:58:39.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:39 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[116266]: 2024-04-20T11:58:39.068+0000 7f8904400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:58:39.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:39 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[116266]: 2024-04-20T11:58:39.068+0000 7f8904400700 -1 osd.1 86 *** Got signal Terminated *** 2024-04-20T11:58:39.390 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:39 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1[116266]: 2024-04-20T11:58:39.068+0000 7f8904400700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:58:44.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:44 smithi117 podman[155243]: 2024-04-20 11:58:44.581677378 +0000 UTC m=+5.666990195 container died fbba94ccd1b4079fddb27ef691b52a265d8fa7901889476fcf54fc118166cca5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T11:58:45.464 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:45 smithi117 podman[155243]: 2024-04-20 11:58:45.193865864 +0000 UTC m=+6.279178729 container cleanup fbba94ccd1b4079fddb27ef691b52a265d8fa7901889476fcf54fc118166cca5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-20T11:58:45.464 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:45 smithi117 bash[155243]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1 2024-04-20T11:58:45.890 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:45 smithi117 podman[155264]: 2024-04-20 11:58:45.464095208 +0000 UTC m=+0.866231740 container remove fbba94ccd1b4079fddb27ef691b52a265d8fa7901889476fcf54fc118166cca5 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-20T11:58:46.640 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:46 smithi117 podman[155353]: 2024-04-20 11:58:46.170777506 +0000 UTC m=+0.032640899 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:58:46.641 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:46 smithi117 podman[155353]: 2024-04-20 11:58:46.304648321 +0000 UTC m=+0.166511750 container create 7f6538ae7114a93c24212bb23f132b02a6bb1be11473a968d1f02eb4f7306fe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-20T11:58:47.110 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:46 smithi117 podman[155353]: 2024-04-20 11:58:46.643222156 +0000 UTC m=+0.505085619 container init 7f6538ae7114a93c24212bb23f132b02a6bb1be11473a968d1f02eb4f7306fe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T11:58:47.111 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:46 smithi117 podman[155353]: 2024-04-20 11:58:46.655816329 +0000 UTC m=+0.517679727 container start 7f6538ae7114a93c24212bb23f132b02a6bb1be11473a968d1f02eb4f7306fe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:58:47.111 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:46 smithi117 podman[155353]: 2024-04-20 11:58:46.731901718 +0000 UTC m=+0.593765184 container attach 7f6538ae7114a93c24212bb23f132b02a6bb1be11473a968d1f02eb4f7306fe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-20T11:58:47.389 INFO:journalctl@ceph.osd.1.smithi117.stdout:Apr 20 11:58:47 smithi117 podman[155353]: 2024-04-20 11:58:47.111612866 +0000 UTC m=+0.973476315 container died 7f6538ae7114a93c24212bb23f132b02a6bb1be11473a968d1f02eb4f7306fe8 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:58:47.934 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.1.service' 2024-04-20T11:58:48.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:58:48.043 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-20T11:58:48.043 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-20T11:58:48.043 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.2 2024-04-20T11:58:48.595 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:48 smithi117 systemd[1]: Stopping Ceph osd.2 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:58:48.890 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:48 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[121658]: 2024-04-20T11:58:48.593+0000 7fb01c400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:58:48.890 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:48 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[121658]: 2024-04-20T11:58:48.593+0000 7fb01c400700 -1 osd.2 86 *** Got signal Terminated *** 2024-04-20T11:58:48.891 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:48 smithi117 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2[121658]: 2024-04-20T11:58:48.593+0000 7fb01c400700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:58:54.454 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:54 smithi117 podman[155460]: 2024-04-20 11:58:54.141913985 +0000 UTC m=+5.691160813 container died 3b78740c3941bdcaeeac8d3981c8a032dac36561d90fb443c6f4716f7cdad656 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T11:58:54.840 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:54 smithi117 podman[155460]: 2024-04-20 11:58:54.563166352 +0000 UTC m=+6.112413221 container cleanup 3b78740c3941bdcaeeac8d3981c8a032dac36561d90fb443c6f4716f7cdad656 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-20T11:58:54.840 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:54 smithi117 bash[155460]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2 2024-04-20T11:58:55.141 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:54 smithi117 podman[155481]: 2024-04-20 11:58:54.838994863 +0000 UTC m=+0.685799921 container remove 3b78740c3941bdcaeeac8d3981c8a032dac36561d90fb443c6f4716f7cdad656 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-20T11:58:55.873 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:55 smithi117 podman[155572]: 2024-04-20 11:58:55.587707822 +0000 UTC m=+0.032866860 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:58:55.873 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:55 smithi117 podman[155572]: 2024-04-20 11:58:55.706790337 +0000 UTC m=+0.151949438 container create fc754522d9f80795fcdf31797f26bba7c1eafb796664d7d8d5113098479fcad2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-20T11:58:56.140 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:55 smithi117 podman[155572]: 2024-04-20 11:58:55.986554952 +0000 UTC m=+0.431714077 container init fc754522d9f80795fcdf31797f26bba7c1eafb796664d7d8d5113098479fcad2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-20T11:58:56.140 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:56 smithi117 podman[155572]: 2024-04-20 11:58:56.000062975 +0000 UTC m=+0.445222110 container start fc754522d9f80795fcdf31797f26bba7c1eafb796664d7d8d5113098479fcad2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:58:56.140 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:56 smithi117 podman[155572]: 2024-04-20 11:58:56.100668445 +0000 UTC m=+0.545827566 container attach fc754522d9f80795fcdf31797f26bba7c1eafb796664d7d8d5113098479fcad2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:58:56.795 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 20 11:58:56 smithi117 podman[155572]: 2024-04-20 11:58:56.448727654 +0000 UTC m=+0.893886692 container died fc754522d9f80795fcdf31797f26bba7c1eafb796664d7d8d5113098479fcad2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:58:57.184 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.2.service' 2024-04-20T11:58:57.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:58:57.273 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-20T11:58:57.274 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-20T11:58:57.274 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.3 2024-04-20T11:58:57.661 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:58:57 smithi173 systemd[1]: Stopping Ceph osd.3 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:58:58.161 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:58:57 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:58:57.903+0000 7f22f1a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:58:58.161 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:58:57 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:58:57.903+0000 7f22f1a00700 -1 osd.3 86 *** Got signal Terminated *** 2024-04-20T11:58:58.162 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:58:57 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:58:57.903+0000 7f22f1a00700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:58:58.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:58:58 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:58:58.764+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:58:58.912 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:58:58 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:58:58.654+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:58:59.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:58:59 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:58:59.755+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:58:59.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:58:59 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:58:59.608+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:00.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:58:59 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:58:59.920+0000 7f22e8200700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.566764+0000 front 2024-04-20T11:58:32.566860+0000 (oldest deadline 2024-04-20T11:58:59.566886+0000) 2024-04-20T11:59:00.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:00 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:00.792+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:00.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:00 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:00.643+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:01.411 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:00 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:59:00.949+0000 7f22e8200700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.566764+0000 front 2024-04-20T11:58:32.566860+0000 (oldest deadline 2024-04-20T11:58:59.566886+0000) 2024-04-20T11:59:01.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:01 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:01.778+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:01.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:01 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:01.611+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:02.298 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:01 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3[83710]: 2024-04-20T11:59:01.991+0000 7f22e8200700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.566764+0000 front 2024-04-20T11:58:32.566860+0000 (oldest deadline 2024-04-20T11:58:59.566886+0000) 2024-04-20T11:59:02.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:02 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:02.765+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:02.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:02 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:02.588+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:03.783 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:03 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:03.593+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:03.784 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:03 smithi173 podman[110582]: 2024-04-20 11:59:03.484339103 +0000 UTC m=+5.734511472 container died 266ace61d6c6c17a1ec028b58759ce36a1c10a43c0a836295147b1d920abdf7b (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:59:04.161 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:03 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:03.781+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:04.161 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:03 smithi173 podman[110582]: 2024-04-20 11:59:03.92262092 +0000 UTC m=+6.172793313 container cleanup 266ace61d6c6c17a1ec028b58759ce36a1c10a43c0a836295147b1d920abdf7b (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:59:04.161 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:03 smithi173 bash[110582]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3 2024-04-20T11:59:04.614 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:04 smithi173 podman[110602]: 2024-04-20 11:59:04.216081026 +0000 UTC m=+0.713097638 container remove 266ace61d6c6c17a1ec028b58759ce36a1c10a43c0a836295147b1d920abdf7b (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-20T11:59:04.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:04 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:04.734+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:04.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:04 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:04.612+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:05.278 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:04 smithi173 podman[110689]: 2024-04-20 11:59:04.868348955 +0000 UTC m=+0.037202536 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:59:05.278 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:04 smithi173 podman[110689]: 2024-04-20 11:59:04.994916759 +0000 UTC m=+0.163770392 container create b7935e4273c4922900f2dc19b4dc5180a6a0ee17892651116c143742f81c3c32 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:59:05.531 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:05 smithi173 podman[110689]: 2024-04-20 11:59:05.278848593 +0000 UTC m=+0.447702216 container init b7935e4273c4922900f2dc19b4dc5180a6a0ee17892651116c143742f81c3c32 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-20T11:59:05.532 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:05 smithi173 podman[110689]: 2024-04-20 11:59:05.28938056 +0000 UTC m=+0.458234146 container start b7935e4273c4922900f2dc19b4dc5180a6a0ee17892651116c143742f81c3c32 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T11:59:05.532 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:05 smithi173 podman[110689]: 2024-04-20 11:59:05.411070155 +0000 UTC m=+0.579923747 container attach b7935e4273c4922900f2dc19b4dc5180a6a0ee17892651116c143742f81c3c32 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-20T11:59:05.843 INFO:journalctl@ceph.osd.3.smithi173.stdout:Apr 20 11:59:05 smithi173 podman[110689]: 2024-04-20 11:59:05.743031996 +0000 UTC m=+0.911885575 container died b7935e4273c4922900f2dc19b4dc5180a6a0ee17892651116c143742f81c3c32 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:59:05.844 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:05 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:05.661+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:05.844 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:05 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:05.661+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:05.844 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:05 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:05.692+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:06.689 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.3.service' 2024-04-20T11:59:06.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:59:06.783 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-20T11:59:06.784 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-20T11:59:06.784 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.4 2024-04-20T11:59:06.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:06 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:06.688+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:06.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:06 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:06.688+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:41.149246+0000 front 2024-04-20T11:58:41.149315+0000 (oldest deadline 2024-04-20T11:59:06.449059+0000) 2024-04-20T11:59:06.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:06 smithi173 systemd[1]: Stopping Ceph osd.4 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:59:06.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:06 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:06.658+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:06.912 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:06 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:06.658+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:07.634 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:07 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:07.332+0000 7f1483000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:59:07.634 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:07 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:07.332+0000 7f1483000700 -1 osd.4 86 *** Got signal Terminated *** 2024-04-20T11:59:07.634 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:07 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:07.332+0000 7f1483000700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:59:07.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:07 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:07.696+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:07.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:07 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:07.696+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:41.149246+0000 front 2024-04-20T11:58:41.149315+0000 (oldest deadline 2024-04-20T11:59:06.449059+0000) 2024-04-20T11:59:07.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:07 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:07.633+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:07.912 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:07 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:07.633+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:08.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:08 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:08.657+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:08.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:08 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:08.657+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:41.149246+0000 front 2024-04-20T11:58:41.149315+0000 (oldest deadline 2024-04-20T11:59:06.449059+0000) 2024-04-20T11:59:08.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:08 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:08.670+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:08.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:08 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:08.670+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:09.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:09.612+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:09.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:09.612+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:41.149246+0000 front 2024-04-20T11:58:41.149315+0000 (oldest deadline 2024-04-20T11:59:06.449059+0000) 2024-04-20T11:59:09.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:09.678+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:09.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:09 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:09.678+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:10.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:10 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:10.597+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:10.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:10 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:10.597+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:41.149246+0000 front 2024-04-20T11:58:41.149315+0000 (oldest deadline 2024-04-20T11:59:06.449059+0000) 2024-04-20T11:59:10.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:10 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:10.721+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:10.912 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:10 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:10.721+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:11.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:11 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:11.600+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:34.747615+0000 front 2024-04-20T11:58:34.747771+0000 (oldest deadline 2024-04-20T11:58:58.247409+0000) 2024-04-20T11:59:11.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:11 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4[88606]: 2024-04-20T11:59:11.600+0000 7f147e800700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:41.149246+0000 front 2024-04-20T11:58:41.149315+0000 (oldest deadline 2024-04-20T11:59:06.449059+0000) 2024-04-20T11:59:11.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:11 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:11.739+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:11.912 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:11 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:11.739+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:12.831 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:12 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:12.699+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:12.832 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:12 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:12.699+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:12.832 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:12 smithi173 podman[110806]: 2024-04-20 11:59:12.74616682 +0000 UTC m=+5.540805079 container died 91012d4ad129120ee028046a91008e94d73a7e38e17a3855ff7bd84439dd103f (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T11:59:13.658 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:13 smithi173 podman[110806]: 2024-04-20 11:59:13.185350149 +0000 UTC m=+5.979988424 container cleanup 91012d4ad129120ee028046a91008e94d73a7e38e17a3855ff7bd84439dd103f (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-20T11:59:13.658 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:13 smithi173 bash[110806]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4 2024-04-20T11:59:13.658 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:13 smithi173 podman[110826]: 2024-04-20 11:59:13.403232817 +0000 UTC m=+0.643617934 container remove 91012d4ad129120ee028046a91008e94d73a7e38e17a3855ff7bd84439dd103f (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-20T11:59:13.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:13 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:13.657+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:13.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:13 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:13.657+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:14.475 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:14 smithi173 podman[110916]: 2024-04-20 11:59:14.080675996 +0000 UTC m=+0.037399039 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:59:14.475 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:14 smithi173 podman[110916]: 2024-04-20 11:59:14.195796078 +0000 UTC m=+0.152519148 container create 0c1bb62e8c49dd81031ffd1e129ef7c38bac2b4b67c476bd9d7835423834cb8e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-20T11:59:14.475 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:14 smithi173 podman[110916]: 2024-04-20 11:59:14.474744967 +0000 UTC m=+0.431468077 container init 0c1bb62e8c49dd81031ffd1e129ef7c38bac2b4b67c476bd9d7835423834cb8e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T11:59:14.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:14 smithi173 podman[110916]: 2024-04-20 11:59:14.485860051 +0000 UTC m=+0.442583128 container start 0c1bb62e8c49dd81031ffd1e129ef7c38bac2b4b67c476bd9d7835423834cb8e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-20T11:59:14.911 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:14 smithi173 podman[110916]: 2024-04-20 11:59:14.62370893 +0000 UTC m=+0.580432050 container attach 0c1bb62e8c49dd81031ffd1e129ef7c38bac2b4b67c476bd9d7835423834cb8e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-20T11:59:14.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:14 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:14.639+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:14.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:14 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:14.639+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:15.283 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 20 11:59:14 smithi173 podman[110916]: 2024-04-20 11:59:14.941470888 +0000 UTC m=+0.898193962 container died 0c1bb62e8c49dd81031ffd1e129ef7c38bac2b4b67c476bd9d7835423834cb8e (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-20T11:59:15.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.4.service' 2024-04-20T11:59:15.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:59:15.804 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-20T11:59:15.805 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-20T11:59:15.805 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.5 2024-04-20T11:59:15.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:15 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:15.615+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:15.912 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:15 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:15.615+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:16.379 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:15 smithi173 systemd[1]: Stopping Ceph osd.5 for 59ecfd22-ff04-11ee-bc92-c7b262605968... 2024-04-20T11:59:16.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:16 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:16.378+0000 7f23fa200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:59:16.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:16 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:16.378+0000 7f23fa200700 -1 osd.5 86 *** Got signal Terminated *** 2024-04-20T11:59:16.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:16 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:16.378+0000 7f23fa200700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:59:16.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:16 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:16.659+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:16.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:16 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:16.659+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:17.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:17 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:17.622+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:17.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:17 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:17.622+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:18.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:18 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:18.575+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:18.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:18 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:18.575+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:18.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:18 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:18.575+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-04-20T11:58:51.505119+0000 front 2024-04-20T11:58:51.505014+0000 (oldest deadline 2024-04-20T11:59:17.905189+0000) 2024-04-20T11:59:19.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:19 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:19.584+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:19.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:19 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:19.584+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:19.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:19 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:19.584+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-04-20T11:58:51.505119+0000 front 2024-04-20T11:58:51.505014+0000 (oldest deadline 2024-04-20T11:59:17.905189+0000) 2024-04-20T11:59:20.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:20 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:20.626+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-04-20T11:58:32.102498+0000 front 2024-04-20T11:58:32.102233+0000 (oldest deadline 2024-04-20T11:58:58.001889+0000) 2024-04-20T11:59:20.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:20 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:20.626+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-04-20T11:58:43.303953+0000 front 2024-04-20T11:58:43.303751+0000 (oldest deadline 2024-04-20T11:59:05.603295+0000) 2024-04-20T11:59:20.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:20 smithi173 ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5[93918]: 2024-04-20T11:59:20.626+0000 7f23f0a00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-04-20T11:58:51.505119+0000 front 2024-04-20T11:58:51.505014+0000 (oldest deadline 2024-04-20T11:59:17.905189+0000) 2024-04-20T11:59:22.060 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:21 smithi173 podman[111022]: 2024-04-20 11:59:21.746687207 +0000 UTC m=+5.501731718 container died b9aa7862f8f5625ad7dd79be23c15a90335ca97f4ec539ba5a2b94b5b38feca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-20T11:59:22.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:22 smithi173 podman[111022]: 2024-04-20 11:59:22.18004182 +0000 UTC m=+5.935086372 container cleanup b9aa7862f8f5625ad7dd79be23c15a90335ca97f4ec539ba5a2b94b5b38feca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-20T11:59:22.412 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:22 smithi173 bash[111022]: ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5 2024-04-20T11:59:22.897 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:22 smithi173 podman[111041]: 2024-04-20 11:59:22.472996006 +0000 UTC m=+0.708023227 container remove b9aa7862f8f5625ad7dd79be23c15a90335ca97f4ec539ba5a2b94b5b38feca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:59:23.570 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:23 smithi173 podman[111131]: 2024-04-20 11:59:23.172789287 +0000 UTC m=+0.034191846 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:59:23.570 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:23 smithi173 podman[111131]: 2024-04-20 11:59:23.298942367 +0000 UTC m=+0.160344947 container create 163ca6d23f6892419ad141efc76bbd1566622b7ba6545aaa82a8e7123531c81c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-20T11:59:23.570 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:23 smithi173 podman[111131]: 2024-04-20 11:59:23.569643811 +0000 UTC m=+0.431046401 container init 163ca6d23f6892419ad141efc76bbd1566622b7ba6545aaa82a8e7123531c81c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-20T11:59:23.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:23 smithi173 podman[111131]: 2024-04-20 11:59:23.582691179 +0000 UTC m=+0.444093729 container start 163ca6d23f6892419ad141efc76bbd1566622b7ba6545aaa82a8e7123531c81c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:59:23.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:23 smithi173 podman[111131]: 2024-04-20 11:59:23.660126062 +0000 UTC m=+0.521528689 container attach 163ca6d23f6892419ad141efc76bbd1566622b7ba6545aaa82a8e7123531c81c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:59:24.392 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 20 11:59:24 smithi173 podman[111157]: 2024-04-20 11:59:24.091008993 +0000 UTC m=+0.033404750 container died 163ca6d23f6892419ad141efc76bbd1566622b7ba6545aaa82a8e7123531c81c (image=quay.ceph.io/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-59ecfd22-ff04-11ee-bc92-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T11:59:24.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59ecfd22-ff04-11ee-bc92-c7b262605968@osd.5.service' 2024-04-20T11:59:24.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:59:24.935 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-20T11:59:24.935 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 --force --keep-logs 2024-04-20T11:59:25.206 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T11:59:32.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[63413]: fuse finished with error 0 and tester_r 0 2024-04-20T11:59:59.701 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 --force --keep-logs 2024-04-20T11:59:59.975 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T12:00:20.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T12:00:20.395 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T12:00:20.485 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-20T12:00:20.488 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491/remote/smithi117/crash 2024-04-20T12:00:20.488 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash -- . 2024-04-20T12:00:20.573 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T12:00:20.573 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-04-20T12:00:20.578 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491/remote/smithi173/crash 2024-04-20T12:00:20.579 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash -- . 2024-04-20T12:00:20.663 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T12:00:20.664 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2024-04-20T12:00:20.667 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-20T12:00:20.667 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-20T12:00:20.772 INFO:teuthology.orchestra.run.smithi117.stdout:2024-04-20T11:20:00.000322+0000 mon.smithi117 (mon.0) 631 : cluster [ERR] fs cephfs is degraded 2024-04-20T12:00:20.773 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-20T12:00:20.773 INFO:tasks.cephadm:Compressing logs... 2024-04-20T12:00:20.773 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T12:00:20.777 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T12:00:20.837 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-20T12:00:20.838 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-20T12:00:20.839 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.log 2024-04-20T12:00:20.840 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.2.log 2024-04-20T12:00:20.840 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.audit.log 2024-04-20T12:00:20.842 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.1.log 2024-04-20T12:00:20.844 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.0.log 2024-04-20T12:00:20.845 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi117.umapup.log 2024-04-20T12:00:20.846 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.0.log: 89.5% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.log.gz 2024-04-20T12:00:20.846 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi117.miaplu.log 2024-04-20T12:00:20.848 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi117.umapup.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.cephadm.log 2024-04-20T12:00:20.851 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi117.miaplu.log: /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-client.ceph-exporter.smithi117.log 2024-04-20T12:00:20.851 INFO:teuthology.orchestra.run.smithi117.stderr: 91.4% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.audit.log.gz 2024-04-20T12:00:20.853 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mgr.smithi117.ivnklj.log 2024-04-20T12:00:20.853 INFO:teuthology.orchestra.run.smithi117.stderr: 85.7% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.cephadm.log.gz 2024-04-20T12:00:20.857 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-client.ceph-exporter.smithi117.log: /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mgr.smithi117.ivnklj.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-volume.log 2024-04-20T12:00:20.858 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/ceph-client.1.log 2024-04-20T12:00:20.859 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-04-20T12:00:20.859 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mon.smithi117.log 2024-04-20T12:00:20.860 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.log 2024-04-20T12:00:20.861 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.audit.log 2024-04-20T12:00:20.861 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mon.smithi117.log: 94.5% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-client.ceph-exporter.smithi117.log.gz 2024-04-20T12:00:20.862 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.5.log 2024-04-20T12:00:20.863 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.3.log 2024-04-20T12:00:20.864 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.4.log 2024-04-20T12:00:20.865 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T12:00:20.866 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-client.ceph-exporter.smithi173.log 2024-04-20T12:00:20.867 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi173.pybzht.log 2024-04-20T12:00:20.869 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-client.ceph-exporter.smithi173.log: 91.4% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.audit.log.gz 2024-04-20T12:00:20.870 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi173.pfrgat.log 2024-04-20T12:00:20.870 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi173.pybzht.log: 89.6% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.log.gz 2024-04-20T12:00:20.871 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.cephadm.log 2024-04-20T12:00:20.874 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi173.pfrgat.log: /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mon.smithi173.log 2024-04-20T12:00:20.875 INFO:teuthology.orchestra.run.smithi173.stderr: 94.6% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-client.ceph-exporter.smithi173.log.gz 2024-04-20T12:00:20.875 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mgr.smithi173.gnrjof.log 2024-04-20T12:00:20.876 INFO:teuthology.orchestra.run.smithi173.stderr: 85.7% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph.cephadm.log.gz 2024-04-20T12:00:20.877 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-volume.log 2024-04-20T12:00:20.884 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mon.smithi173.log: /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mgr.smithi173.gnrjof.log: /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T12:00:20.890 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/cephadm.log: 94.2% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi117.miaplu.log.gz 2024-04-20T12:00:20.891 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi173.pfrgat.log.gz 2024-04-20T12:00:20.898 INFO:teuthology.orchestra.run.smithi173.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T12:00:20.914 INFO:teuthology.orchestra.run.smithi173.stderr: 90.9% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-volume.log.gz 2024-04-20T12:00:20.926 INFO:teuthology.orchestra.run.smithi117.stderr: 91.0% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-volume.log.gz 2024-04-20T12:00:20.934 INFO:teuthology.orchestra.run.smithi117.stderr: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T12:00:20.937 INFO:teuthology.orchestra.run.smithi173.stderr: 93.1% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi173.pybzht.log.gz 2024-04-20T12:00:20.954 INFO:teuthology.orchestra.run.smithi173.stderr: 89.9% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mgr.smithi173.gnrjof.log.gz 2024-04-20T12:00:21.529 INFO:teuthology.orchestra.run.smithi117.stderr: 89.4% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mgr.smithi117.ivnklj.log.gz 2024-04-20T12:00:21.537 INFO:teuthology.orchestra.run.smithi173.stderr: 92.5% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mon.smithi173.log.gz 2024-04-20T12:00:22.875 INFO:teuthology.orchestra.run.smithi117.stderr: 89.8% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mon.smithi117.log.gz 2024-04-20T12:00:25.253 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.4.log.gz 2024-04-20T12:00:25.771 INFO:teuthology.orchestra.run.smithi117.stderr: 93.6% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.0.log.gz 2024-04-20T12:00:26.048 INFO:teuthology.orchestra.run.smithi117.stderr: 93.7% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.2.log.gz 2024-04-20T12:00:26.451 INFO:teuthology.orchestra.run.smithi173.stderr: 93.8% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.5.log.gz 2024-04-20T12:00:26.933 INFO:teuthology.orchestra.run.smithi117.stderr: 93.7% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.1.log.gz 2024-04-20T12:00:27.086 INFO:teuthology.orchestra.run.smithi173.stderr: 93.7% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-osd.3.log.gz 2024-04-20T12:00:35.005 INFO:teuthology.orchestra.run.smithi173.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-20T12:00:35.006 INFO:teuthology.orchestra.run.smithi173.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-20T12:00:35.010 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-04-20T12:00:35.011 INFO:teuthology.orchestra.run.smithi173.stderr:real 0m14.181s 2024-04-20T12:00:35.011 INFO:teuthology.orchestra.run.smithi173.stderr:user 0m29.355s 2024-04-20T12:00:35.011 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m1.931s 2024-04-20T12:00:37.472 INFO:teuthology.orchestra.run.smithi117.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-20T12:01:56.727 INFO:teuthology.orchestra.run.smithi117.stderr: 92.9% -- replaced with /var/log/ceph/59ecfd22-ff04-11ee-bc92-c7b262605968/ceph-mds.cephfs.smithi117.umapup.log.gz 2024-04-20T12:01:56.730 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-04-20T12:01:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:real 1m35.919s 2024-04-20T12:01:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:user 2m2.214s 2024-04-20T12:01:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m9.286s 2024-04-20T12:01:56.732 INFO:tasks.cephadm:Archiving logs... 2024-04-20T12:01:56.733 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491/remote/smithi117/log 2024-04-20T12:01:56.734 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T12:02:15.786 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491/remote/smithi173/log 2024-04-20T12:02:15.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T12:02:20.303 INFO:tasks.cephadm:Removing cluster... 2024-04-20T12:02:20.303 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 --force 2024-04-20T12:02:20.576 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T12:02:21.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59ecfd22-ff04-11ee-bc92-c7b262605968 --force 2024-04-20T12:02:21.895 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 59ecfd22-ff04-11ee-bc92-c7b262605968 2024-04-20T12:02:22.564 INFO:tasks.cephadm:Removing cephadm ... 2024-04-20T12:02:22.564 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T12:02:22.621 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T12:02:22.676 INFO:tasks.cephadm:Teardown complete 2024-04-20T12:02:22.676 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T12:02:22.687 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_fuse.py", line 183, in task File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-20T12:02:22.688 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T12:02:22.688 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T12:02:22.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T12:02:22.823 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T12:02:23.186 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T12:02:23.186 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T12:02:23.194 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T12:02:23.242 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-04-20T12:02:23.251 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-20T12:02:23.252 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T12:02:23.252 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-04-20T12:02:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 209 +810us[ +798us] +/- 72ms 2024-04-20T12:02:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv02.front.sepia.ceph.com 3 8 377 209 -163us[ -175us] +/- 69ms 2024-04-20T12:02:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 81 -686us[ -686us] +/- 63ms 2024-04-20T12:02:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T12:02:23.261 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T12:02:23.261 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-20T12:02:23.262 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 15 +886us[ +886us] +/- 75ms 2024-04-20T12:02:23.262 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv02.front.sepia.ceph.com 3 7 377 16 -96us[ -138us] +/- 52ms 2024-04-20T12:02:23.262 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -647us[ -647us] +/- 64ms 2024-04-20T12:02:23.262 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T12:02:23.263 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T12:02:23.284 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T12:02:23.285 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T12:02:23.327 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T12:02:23.506 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T12:02:23.685 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T12:02:23.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:02:23.779 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T12:02:23.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:02:23.873 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T12:02:23.884 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T12:02:23.915 INFO:teuthology.task.internal:Duration was 4366.559259 seconds 2024-04-20T12:02:23.916 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T12:02:23.937 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T12:02:23.937 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T12:02:23.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T12:02:24.066 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T12:02:24.080 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T12:02:24.475 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T12:02:24.475 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-04-20T12:02:24.475 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T12:02:24.537 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-04-20T12:02:24.537 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T12:02:24.615 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T12:02:24.615 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T12:02:24.623 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T12:02:24.703 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T12:02:24.703 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T12:02:24.708 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T12:02:26.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T12:02:26.030 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T12:02:26.030 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T12:02:26.103 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T12:02:26.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T12:02:26.206 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T12:02:26.213 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T12:02:26.304 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-04-20T12:02:26.315 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-04-20T12:02:26.373 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T12:02:26.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:02:26.426 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T12:02:26.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:02:26.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T12:02:26.494 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T12:02:26.495 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491/remote/smithi117 2024-04-20T12:02:26.495 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T12:02:26.613 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491/remote/smithi173 2024-04-20T12:02:26.613 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T12:02:26.714 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T12:02:26.714 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T12:02:26.722 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T12:02:26.777 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T12:02:26.790 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T12:02:26.790 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T12:02:26.862 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T12:02:26.862 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T12:02:26.871 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T12:02:26.921 INFO:teuthology.orchestra.run.smithi117.stdout: 393251 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 20 12:02 /home/ubuntu/cephtest 2024-04-20T12:02:26.921 INFO:teuthology.orchestra.run.smithi117.stdout: 682083 4 d--------- 2 ubuntu ubuntu 4096 Apr 20 11:05 /home/ubuntu/cephtest/mnt.0 2024-04-20T12:02:26.921 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-20T12:02:26.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:02:26.925 INFO:teuthology.orchestra.run.smithi117.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-20T12:02:26.925 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_fuse.py", line 183, in task File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi117 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-20T12:02:26.926 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T12:02:26.939 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T12:02:27.053 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-20T12:02:27.054 INFO:teuthology.nuke:Checking targets against current locks 2024-04-20T12:02:27.081 DEBUG:teuthology.nuke:shortname: smithi117 2024-04-20T12:02:27.081 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T12:02:27.106 DEBUG:teuthology.nuke:shortname: smithi173 2024-04-20T12:02:27.106 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T12:02:27.109 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:40:49.785588', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKu78gkLLaiWRbDPuhWO5MIBRTaiOWdP6u8pIqGkprNrWsI7pLi5+Ar9l6+i+KOpT++HOFfEgvqUoeM27EqOjtY='} 2024-04-20T12:02:27.131 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664491', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:40:49.783977', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPr/8/+o9OmJU2qLSKgC/92YW26KhRc4916aPpRcO7v6MBEl2glsf18mn/vfW5Si6BXo6XAbf/0pzAEZU5W2HFo='} 2024-04-20T12:02:27.160 INFO:teuthology.orchestra.console.smithi117:Power off 2024-04-20T12:02:27.160 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T12:02:27.181 DEBUG:teuthology.orchestra.console.smithi117:power off output: Chassis Power Control: Down/Off 2024-04-20T12:02:27.181 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T12:02:27.198 INFO:teuthology.orchestra.console.smithi173:Power off 2024-04-20T12:02:27.198 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T12:02:27.206 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-04-20T12:02:27.219 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2024-04-20T12:02:27.219 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T12:02:27.239 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-04-20T12:02:31.207 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T12:02:31.240 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T12:02:31.318 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-04-20T12:02:31.350 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-04-20T12:02:35.322 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T12:02:35.351 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T12:02:35.432 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-04-20T12:02:35.461 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-04-20T12:02:39.434 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T12:02:39.462 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T12:02:39.545 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off 2024-04-20T12:02:39.572 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-04-20T12:02:39.646 INFO:teuthology.orchestra.console.smithi117:Power off completed 2024-04-20T12:02:39.673 INFO:teuthology.orchestra.console.smithi173:Power off completed 2024-04-20T12:02:39.778 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 4366.559258699417 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 host.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology status: fail success: false 2024-04-20T12:02:39.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T12:02:39.883 INFO:teuthology.run:FAIL